Developing Databricks Apps Locally
I want to be able to work on a Databricks App locally for the following reasons:
- IDE of choice
- AI plugin of choice
- push changes & deploy in one command
Here are the options I explored for different use cases.
Level 1: git credentials only
This option is slower to use, but only requires git credentials for your Databricks workspace. It doesn’t meet requirement 3 because it uses the UI instead of the databricks CLI
Steps:
- git commit/push
- (browser tab) databricks git folder: click Pull
- (browser tab) databricks App page: click Deploy
Useful in a situation like: working from your company laptop, you have access to a client git repo, but no PAT access to the client Databricks space.
Level 2: git AND Databricks credentials, collaborating
Configure your databricks profiles in the cli and you’ll only need to hit the browser to refresh the App tab.
Steps:
- git commit/push
databricks repos update <DATABRICKS_APP_CODE_URL> --branch <YOUR_BRANCH>databricks apps deploy <APP_NAME>
A little clunky for one developer, but using version control will make sure you don’t step on a teammate’s changes. If you’re developing solo you can go faster…
Level 3: git AND Databricks credentials, solo dev
Leave databricks sync --watch on to sync your changes to the Workspace as you make them. Deploy from cli. Commit when you make appreciable progress.
Steps:
databricks sync --watch . <DATABRICKS_APP_CODE_URL>(Leave this running in a terminal tab or background task)databricks apps deploy <APP_NAME>gitwhen you feel like it
This approach is fastest but is best suited to solo development. Two people syncing to the same directory would overwrite the other constantly!