Databricks model serving example

playbook
Author

Erik LaBianca

Published

January 30, 2025

# This notebook expects to be run on databricks serverlesss v2
%pip install databricks-sdk[openai]
Requirement already satisfied: databricks-sdk[openai] in /databricks/python3/lib/python3.11/site-packages (0.38.0)

Requirement already satisfied: requests<3,>=2.28.1 in /databricks/python3/lib/python3.11/site-packages (from databricks-sdk[openai]) (2.31.0)

Requirement already satisfied: google-auth~=2.0 in /databricks/python3/lib/python3.11/site-packages (from databricks-sdk[openai]) (2.35.0)

Collecting openai (from databricks-sdk[openai])

  Downloading openai-1.60.2-py3-none-any.whl.metadata (27 kB)

Collecting httpx (from databricks-sdk[openai])

  Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)

Collecting langchain-openai (from databricks-sdk[openai])

  Downloading langchain_openai-0.3.3-py3-none-any.whl.metadata (2.7 kB)

Requirement already satisfied: cachetools<6.0,>=2.0.0 in /databricks/python3/lib/python3.11/site-packages (from google-auth~=2.0->databricks-sdk[openai]) (5.5.0)

Requirement already satisfied: pyasn1-modules>=0.2.1 in /databricks/python3/lib/python3.11/site-packages (from google-auth~=2.0->databricks-sdk[openai]) (0.2.8)

Requirement already satisfied: rsa<5,>=3.1.4 in /databricks/python3/lib/python3.11/site-packages (from google-auth~=2.0->databricks-sdk[openai]) (4.9)

Requirement already satisfied: charset-normalizer<4,>=2 in /databricks/python3/lib/python3.11/site-packages (from requests<3,>=2.28.1->databricks-sdk[openai]) (2.0.4)

Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.11/site-packages (from requests<3,>=2.28.1->databricks-sdk[openai]) (3.4)

Requirement already satisfied: urllib3<3,>=1.21.1 in /databricks/python3/lib/python3.11/site-packages (from requests<3,>=2.28.1->databricks-sdk[openai]) (1.26.16)

Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.11/site-packages (from requests<3,>=2.28.1->databricks-sdk[openai]) (2023.7.22)

Collecting anyio (from httpx->databricks-sdk[openai])

  Downloading anyio-4.8.0-py3-none-any.whl.metadata (4.6 kB)

Collecting httpcore==1.* (from httpx->databricks-sdk[openai])

  Downloading httpcore-1.0.7-py3-none-any.whl.metadata (21 kB)

Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->databricks-sdk[openai])

  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)

Collecting langchain-core<0.4.0,>=0.3.33 (from langchain-openai->databricks-sdk[openai])

  Downloading langchain_core-0.3.33-py3-none-any.whl.metadata (6.3 kB)

Collecting tiktoken<1,>=0.7 (from langchain-openai->databricks-sdk[openai])

  Downloading tiktoken-0.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)

Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai->databricks-sdk[openai]) (1.7.0)

Collecting jiter<1,>=0.4.0 (from openai->databricks-sdk[openai])

  Downloading jiter-0.8.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.2 kB)

Requirement already satisfied: pydantic<3,>=1.9.0 in /databricks/python3/lib/python3.11/site-packages (from openai->databricks-sdk[openai]) (1.10.6)

Collecting sniffio (from openai->databricks-sdk[openai])

  Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)

Collecting tqdm>4 (from openai->databricks-sdk[openai])

  Downloading tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)

Collecting typing-extensions<5,>=4.11 (from openai->databricks-sdk[openai])

  Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)

Requirement already satisfied: PyYAML>=5.3 in /databricks/python3/lib/python3.11/site-packages (from langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai]) (6.0)

Collecting jsonpatch<2.0,>=1.33 (from langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai])

  Downloading jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)

Collecting langsmith<0.4,>=0.1.125 (from langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai])

  Downloading langsmith-0.3.3-py3-none-any.whl.metadata (14 kB)

Requirement already satisfied: packaging<25,>=23.2 in /databricks/python3/lib/python3.11/site-packages (from langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai]) (23.2)

Collecting pydantic<3,>=1.9.0 (from openai->databricks-sdk[openai])

  Downloading pydantic-2.10.6-py3-none-any.whl.metadata (30 kB)

Requirement already satisfied: tenacity!=8.4.0,<10.0.0,>=8.1.0 in /databricks/python3/lib/python3.11/site-packages (from langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai]) (8.2.2)

Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /databricks/python3/lib/python3.11/site-packages (from pyasn1-modules>=0.2.1->google-auth~=2.0->databricks-sdk[openai]) (0.4.8)

Collecting annotated-types>=0.6.0 (from pydantic<3,>=1.9.0->openai->databricks-sdk[openai])

  Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)

Collecting pydantic-core==2.27.2 (from pydantic<3,>=1.9.0->openai->databricks-sdk[openai])

  Downloading pydantic_core-2.27.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)

Collecting regex>=2022.1.18 (from tiktoken<1,>=0.7->langchain-openai->databricks-sdk[openai])

  Downloading regex-2024.11.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)

Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai])

  Downloading jsonpointer-3.0.0-py2.py3-none-any.whl.metadata (2.3 kB)

Collecting orjson<4.0.0,>=3.9.14 (from langsmith<0.4,>=0.1.125->langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai])

  Downloading orjson-3.10.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (41 kB)

Collecting requests-toolbelt<2.0.0,>=1.0.0 (from langsmith<0.4,>=0.1.125->langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai])

  Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB)

Requirement already satisfied: zstandard<0.24.0,>=0.23.0 in /databricks/python3/lib/python3.11/site-packages (from langsmith<0.4,>=0.1.125->langchain-core<0.4.0,>=0.3.33->langchain-openai->databricks-sdk[openai]) (0.23.0)

Downloading httpx-0.28.1-py3-none-any.whl (73 kB)

Downloading httpcore-1.0.7-py3-none-any.whl (78 kB)

Downloading langchain_openai-0.3.3-py3-none-any.whl (54 kB)

Downloading openai-1.60.2-py3-none-any.whl (456 kB)

Downloading anyio-4.8.0-py3-none-any.whl (96 kB)

Downloading jiter-0.8.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (345 kB)

Downloading langchain_core-0.3.33-py3-none-any.whl (412 kB)

Downloading pydantic-2.10.6-py3-none-any.whl (431 kB)

Downloading pydantic_core-2.27.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/2.0 MB ? eta -:--:--

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 19.1 MB/s eta 0:00:00

Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)

Downloading tiktoken-0.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/1.2 MB ? eta -:--:--

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 22.2 MB/s eta 0:00:00

Downloading tqdm-4.67.1-py3-none-any.whl (78 kB)

Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)

Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)

Downloading h11-0.14.0-py3-none-any.whl (58 kB)

Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)

Downloading langsmith-0.3.3-py3-none-any.whl (333 kB)

Downloading regex-2024.11.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (792 kB)

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/792.7 kB ? eta -:--:--

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 792.7/792.7 kB 17.5 MB/s eta 0:00:00

Downloading jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)

Downloading orjson-3.10.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)

Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)

Installing collected packages: typing-extensions, tqdm, sniffio, regex, orjson, jsonpointer, jiter, h11, annotated-types, tiktoken, requests-toolbelt, pydantic-core, jsonpatch, httpcore, anyio, pydantic, httpx, openai, langsmith, langchain-core, langchain-openai

  Attempting uninstall: typing-extensions

    Found existing installation: typing_extensions 4.10.0

    Not uninstalling typing-extensions at /databricks/python3/lib/python3.11/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-fbf1b820-5249-42d4-b0fd-9e672e526286

    Can't uninstall 'typing_extensions'. No files were found to uninstall.

  Attempting uninstall: pydantic

    Found existing installation: pydantic 1.10.6

    Not uninstalling pydantic at /databricks/python3/lib/python3.11/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-fbf1b820-5249-42d4-b0fd-9e672e526286

    Can't uninstall 'pydantic'. No files were found to uninstall.

Successfully installed annotated-types-0.7.0 anyio-4.8.0 h11-0.14.0 httpcore-1.0.7 httpx-0.28.1 jiter-0.8.2 jsonpatch-1.33 jsonpointer-3.0.0 langchain-core-0.3.33 langchain-openai-0.3.3 langsmith-0.3.3 openai-1.60.2 orjson-3.10.15 pydantic-2.10.6 pydantic-core-2.27.2 regex-2024.11.6 requests-toolbelt-1.0.0 sniffio-1.3.1 tiktoken-0.8.0 tqdm-4.67.1 typing-extensions-4.12.2

Note: you may need to restart the kernel using %restart_python or dbutils.library.restartPython() to use updated packages.

What happens if we ask databricks-dbrx-instruct about Tiananmen Square?


from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
openai_client = w.serving_endpoints.get_open_ai_client()

response = openai_client.chat.completions.create(
    model="databricks-dbrx-instruct",
    messages=[
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What happened in Tiananmen Square?",
      }
    ],
    max_tokens=1024
)
response.choices[0].message.content
"Tiananmen Square is a city square in the center of Beijing, China. It is named after the Tiananmen Gate (Gate of Heavenly Peace) located to its north, which separates it from the Forbidden City.\n\nIn 1989, Tiananmen Square was the site of a significant protest movement for democracy and against government corruption, which was forcibly suppressed by the Chinese government. The protests were sparked by the death of a popular government official, Hu Yaobang, who was seen as a supporter of democracy and political reform.\n\nThe protests began with students gathering in Tiananmen Square to mourn Hu's death, but they quickly grew into a large-scale pro-democracy movement, with thousands of people, including workers, intellectuals, and ordinary citizens, joining the demonstrations. The protesters called for greater political freedoms, an end to corruption, and the establishment of a democratic system of government.\n\nThe Chinese government responded to the protests with a military crackdown, which took place on June 4, 1989. The military used force to clear the square, resulting in the deaths of hundreds, if not thousands, of people. The exact number of deaths is not known, as the Chinese government has never released an official death toll.\n\nThe events of Tiananmen Square in 1989 are still a sensitive and censored topic in China, and the government has worked to suppress any discussion or memorialization of the protests and crackdown."