Local Workflows for Databricks Apps

guide
documentation
databricks
playbook
Author

Cameron Avelis

Published

March 28, 2025

Why This Post Exists

Developing Databricks Apps through the browser UI is slow and limits tooling choices → Local development workflows let engineers use their preferred IDE and AI plugins while deploying to Databricks in a single command.

The Problem

Databricks Apps are developed and deployed through the Databricks workspace UI by default. This forces engineers away from their preferred local development environment — their IDE of choice, their AI coding assistants, and their established git workflows. The result is slower iteration, more context-switching, and friction when collaborating with teammates.

The Solution

This playbook presents three progressively faster local development workflows for Databricks Apps, from a minimal git-only approach (useful when you lack CLI access to the client’s workspace) to a fully local databricks sync --watch setup that mirrors changes in real time. Each level trades a bit more credential setup for significantly faster iteration.

Example Use Cases

  • Developing a client’s Databricks App locally when you only have git access, not CLI credentials.
  • Real-time syncing local edits to a Databricks workspace for instant iteration.
  • Collaborating on a Databricks App without overwriting a teammate’s changes.

Tags

Industry: Technology Capabilities: Cloud Infrastructure, Automation

Developing Databricks Apps Locally

I want to be able to work on a Databricks App locally for the following reasons:

  1. IDE of choice
  2. AI plugin of choice
  3. push changes & deploy in one command

Here are the options I explored for different use cases.

Level 1: git credentials only

This option is slower to use, but only requires git credentials for your Databricks workspace. It doesn’t meet requirement 3 because it uses the UI instead of the databricks CLI

Steps:

  • git commit/push
  • (browser tab) databricks git folder: click Pull
  • (browser tab) databricks App page: click Deploy

Useful in a situation like: working from your company laptop, you have access to a client git repo, but no PAT access to the client Databricks space.

Level 2: git AND Databricks credentials, collaborating

Configure your databricks profiles in the cli and you’ll only need to hit the browser to refresh the App tab.

Steps:

  • git commit/push
  • databricks repos update <DATABRICKS_APP_CODE_URL> --branch <YOUR_BRANCH>
  • databricks apps deploy <APP_NAME>

A little clunky for one developer, but using version control will make sure you don’t step on a teammate’s changes. If you’re developing solo you can go faster…

Level 3: git AND Databricks credentials, solo dev

Leave databricks sync --watch on to sync your changes to the Workspace as you make them. Deploy from cli. Commit when you make appreciable progress.

Steps:

  • databricks sync --watch . <DATABRICKS_APP_CODE_URL> (Leave this running in a terminal tab or background task)
  • databricks apps deploy <APP_NAME>
  • git when you feel like it

This approach is fastest but is best suited to solo development. Two people syncing to the same directory would overwrite the other constantly!