During the past week, there was a need to explore the execution of SQL statements on Databricks through the API to facilitate data consumption from our Lakehouse in company applications. This exploration involved a thorough examination of Databricks documentation and the subsequent summarization of key details relevant to our environment.
Azure Databricks REST API
The Azure Databricks API provides extensive access to information within a Databricks Workspace, including cluster settings, workflows, and accounts. This article specifically highlights the feature of running SQL statements on a SQL warehouse cluster.
A standard Azure Databricks API call comprises the workspace instance, the REST API operation (GET, POST, PATCH, or DELETE), Azure Databricks authentication, and any supported request payload, such as SQL statements.
The initial step to request the execution of a SQL statement on Databricks is to authenticate on Azure Databricks and obtain a token.