LogoLogo
SaaSBook a demo
  • Immuta Documentation - SaaS
  • Configuration
    • Connect Data Platforms
      • Data Platforms Overview
      • Amazon S3 Integration
      • AWS Lake Formation
        • Getting Started with AWS Lake Formation
        • Register an AWS Lake Formation Connection
        • Reference Guides
          • AWS Lake Formation
          • Security and Compliance
          • Protecting Data
          • Accessing Data
      • Azure Synapse Analytics
        • Getting Started with Azure Synapse Analytics
        • Configure Azure Synapse Analytics Integration
        • Reference Guides
          • Azure Synapse Analytics Overview
          • Azure Synapse Analytics Pre-Configuration Details
      • Databricks
        • Databricks Spark
          • Getting Started with Databricks Spark
          • How-to Guides
            • Configure a Databricks Spark Integration
            • Manually Update Your Databricks Cluster
            • Install a Trusted Library
            • Project UDFs Cache Settings
            • Run R and Scala spark-submit Jobs on Databricks
            • DBFS Access
            • Troubleshooting
          • Reference Guides
            • Databricks Spark Integration Configuration
              • Installation and Compliance
              • Customizing the Integration
              • Setting Up Users
              • Spark Environment Variables
              • Ephemeral Overrides
            • Security and Compliance
            • Registering and Protecting Data
            • Accessing Data
              • Delta Lake API
        • Databricks Unity Catalog
          • Getting Started with Databricks Unity Catalog
          • How-to Guides
            • Register a Databricks Unity Catalog Connection
            • Configure a Databricks Unity Catalog Integration
            • Migrating to Unity Catalog
          • Databricks Unity Catalog Integration Reference Guide
      • Google BigQuery Integration
      • Redshift
        • Getting Started with Redshift
        • How-to Guides
          • Configure Redshift Integration
          • Configure Redshift Spectrum
        • Reference Guides
          • Redshift Overview
          • Redshift Pre-Configuration Details
      • Snowflake
        • Getting Started with Snowflake
        • How-to Guides
          • Register a Snowflake Connection
          • Configure a Snowflake Integration
          • Edit or Remove Your Snowflake Integration
          • Integration Settings
            • Snowflake Table Grants Private Preview Migration
            • Enable Snowflake Table Grants
            • Using Snowflake Data Sharing with Immuta
            • Enable Snowflake Low Row Access Policy Mode
              • Upgrade Snowflake Low Row Access Policy Mode
            • Configure Snowflake Lineage Tag Propagation
        • Reference Guides
          • Snowflake Integration
          • Snowflake Table Grants
          • Snowflake Data Sharing with Immuta
          • Snowflake Low Row Access Policy Mode
          • Snowflake Lineage Tag Propagation
          • Warehouse Sizing Recommendations
        • Explanatory Guides
          • Phased Snowflake Onboarding
      • Starburst (Trino)
        • Getting Started with Starburst (Trino)
        • How-to Guides
          • Configure Starburst (Trino) Integration
          • Customize Read and Write Access Policies for Starburst (Trino)
        • Starburst (Trino) Integration Reference Guide
      • Queries Immuta Runs in Your Data Platform
      • Connect Your Data
        • Registering a Connection
          • How-to Guides
            • Run Object Sync
            • Manage Connection Settings
            • Use the Connection Upgrade Manager
              • Troubleshooting
          • Reference Guides
            • Connections
            • Upgrading to Connections
              • Before You Begin
              • API Changes
              • FAQ
        • Registering Metadata
          • Data Sources in Immuta
          • Register Data Sources
            • Amazon S3 Data Source
            • Azure Synapse Analytics Data Source
            • Databricks Data Source
            • Google BigQuery Data Source
            • Redshift Data Source
            • Snowflake Data Source
              • Bulk Create Snowflake Data Sources
            • Create a Starburst (Trino) Data Source
          • Data Source Settings
            • How-to Guides
              • Manage Data Source Settings
              • Manage Data Source Members
              • Manage Access Requests and Tasks
              • Manage Data Dictionary Descriptions
              • Disable Immuta from Sampling Raw Data
            • Data Source Health Checks Reference Guide
          • Schema Monitoring
            • How-to Guides
              • Manage Schema Monitoring
              • Run Schema Monitoring and Column Detection Jobs
            • Reference Guides
              • Schema Monitoring
              • Schema Projects
            • Why Use Schema Monitoring Concept Guide
    • Manage Data Metadata
      • Connect External Catalogs
        • Configure an External Catalog
        • Reference Guides
          • External Catalog Introduction
          • Custom REST Catalog Interface Introduction
          • Custom REST Catalog Interface Endpoints
      • Data Discovery
        • Introduction
        • Getting Started with Data Discovery
        • How-to Guides
          • Use Identifiers in Domains
          • Use Sensitive Data Discovery (SDD)
          • Manage Identification Frameworks
          • Manage Identifiers
          • Run and Manage Sensitive Data Discovery on Data Sources
        • Reference Guides
          • Identifiers in Domains
          • Built-in Identifier Reference
          • Improved Pack: Built-in Identifier Reference
          • Built-in Discovered Tags Reference
          • How Competitive Pattern Analysis Works
      • Data Classification
        • How-to Guides
          • Activate Classification Frameworks
          • Adjust Identification and Classification Framework Tags
          • How to Use a Classification Framework with Your Own Tags
        • Reference Guide
          • Classification Frameworks
      • Manage Tags
        • How-to Guides
          • Create and Manage Tags
          • Add Tags to Data Sources and Projects
        • Tags Reference Guide
    • Manage Users
      • Getting Started with Users
      • Identity Managers (IAMs)
        • How-to Guides
          • Okta LDAP Interface
          • OpenID Connect
            • OpenID Connect Protocol
            • Okta and OpenID Connect
            • OneLogin with OpenID Connect
          • SAML
            • SAML Protocol
            • Microsoft Entra ID
            • Okta SAML SCIM
        • Reference Guides
          • Identity Managers
          • SAML Protocol Configuration Options
          • SAML Single Logout
      • Immuta Users
        • How-to Guides
          • Managing Personas and Permissions
          • User Impersonation
          • Manage Attributes and Groups
          • External User ID Mapping
          • External User Info Endpoint
        • Reference Guides
          • Permissions and Personas
          • Attributes and Groups in Immuta
    • Organize Data into Domains
      • Getting Started with Domains
      • Domains Reference Guide
    • Application Settings
      • How-to Guides
        • App Settings
        • Private Networking Support
          • Data Connection Private Networking
            • AWS PrivateLink for Redshift
            • AWS PrivateLink for API Gateway
            • Databricks Private Connectivity
              • AWS PrivateLink for Databricks
              • Azure Private Link for Databricks
            • Snowflake Private Connectivity
              • AWS PrivateLink for Snowflake
              • Azure Private Link for Snowflake
            • Starburst (Trino) Private Connectivity
              • AWS PrivateLink for Starburst (Trino)
              • Azure Private Link for Starburst (Trino)
          • Immuta SaaS Private Networking
            • Immuta SaaS Private Networking Over AWS PrivateLink
        • BI Tools
          • BI Tool Configuration Recommendations
          • Power BI Configuration Example
          • Tableau Configuration Example
        • IP Filtering
        • System Status Bundle
      • Reference Guides
        • Deployment Options
        • Data Processing
        • Encryption and Masking Practices
  • Marketplace
    • Introduction
      • User Types
      • Walkthrough
    • Share Data Products
      • How-to Guides
        • Manage Data Products
        • View and Respond to Access Requests
        • Manage Request Forms
        • Customize the Marketplace Branding
      • Reference Guides
        • Marketplace App Requirements
        • Data Products
        • Marketplace Permissions Matrix
        • Understanding Access Provisioning and Underlying Policies in Immuta
          • S3 Provisioning Best Practices
        • Integrating with Existing Catalogs
        • Setting Up Domains for Marketplace
    • Access Data Products
      • How-to Guides
        • Logging into Marketplace
        • Requesting Access to a Data Product
      • Reference Guide
        • Data Source Access Status
    • Short-Term Limitations
  • Governance
    • Introduction
      • Automate Data Access Control Decisions
        • The Two Paths
        • Managing User Metadata
        • Managing Data Metadata
        • Author Policy
        • Test and Deploy Policy
      • Compliantly Open More Sensitive Data for ML and Analytics
        • Managing User Metadata
        • Managing Data Metadata
        • Author Policy
    • Author Policies for Data Access Control
      • Introduction
        • Scalability and Evolvability
        • Understandability
        • Distributed Stewardship
        • Consistency
        • Availability of Data
      • Policies
        • Authoring Policies at Scale
        • Data Engineering with Limited Policy Downtime
        • Subscription Policies
          • Overview
          • How-to Guides
            • Author a Subscription Policy
            • Author an ABAC Subscription Policy
            • Subscription Policies Advanced DSL Guide
            • Author a Restricted Subscription Policy
            • Clone, Activate, or Stage a Global Policy
          • Reference Guides
            • Subscription Policy Access Types
            • Advanced Use of Special Functions
        • Data Policies
          • Overview
          • How-to Guides
            • Author a Masking Data Policy
            • Author a Minimization Policy
            • Author a Purpose-Based Restriction Policy
            • Author a Restricted Data Policy
            • Author a Row-Level Policy
            • Author a Time-Based Restriction Policy
            • Policy Certifications and Diffs
          • Reference Guides
            • Data Policy Types
            • Masking Policies
            • Row-Level Policies
            • Custom WHERE Clause Functions
            • Data Policy Conflicts and Fallback
            • Custom Data Policy Certifications
            • Orchestrated Masking Policies
      • Projects and Purpose-Based Access Control
        • Projects and Purpose Controls
          • Getting Started
          • How-to Guides
            • Create a Project
            • Create and Manage Purposes
            • Project Management
              • Manage Projects and Project Settings
              • Manage Project Data Sources
              • Manage Project Members
          • Reference Guides
            • Projects and Purposes
          • Concept Guide
            • Why Use Purposes?
        • Equalized Access
          • Manage Project Equalization How-to Guide
          • Equalized Access Reference Guide
          • Why Use Project Equalization?
        • Masked Joins
          • Enable Masked Joins How-to Guide
          • Why Use Masked Joins?
        • Writing to Projects
          • How-to Guides
            • Create and Manage Snowflake Project Workspaces
            • Create and Manage Databricks Spark Project Workspaces
            • Write Data to the Workspace
          • Reference Guides
            • Writing to Projects
            • Project UDFs (Databricks)
      • Data Consumers
        • Subscribe to a Data Source
        • Query Data
          • Querying Snowflake Data
          • Querying Databricks Data
          • Querying Starburst (Trino) Data
          • Querying Databricks SQL Data
          • Querying Redshift Data
          • Querying Azure Synapse Analytics Data
        • Subscribe to Projects
    • Observe Access and Activity
      • Introduction
      • Audit
        • How-to Guides
          • Export Audit Logs to S3
          • Export Audit Logs to ADLS
          • Use Immuta Audit
          • Run Governance Reports
        • Reference Guides
          • Universal Audit Model (UAM)
            • UAM Schema Reference Guide
          • Query Audit Logs
            • Snowflake Query Audit Logs
            • Databricks Unity Catalog Query Audit Logs
            • Databricks Spark Query Audit Logs
            • Starburst (Trino) Query Audit Logs
          • Audit Export GraphQL Reference Guide
          • Unknown Users in Audit Logs
          • Governance Report Types
      • Dashboards
        • Use the Audit Dashboards How-To Guide
        • Audit Dashboards Reference Guide
      • Monitors
        • Manage Monitors and Observations
        • Monitors Reference Guide
  • Releases
    • Deployment Notes
      • 2024
      • 2023
      • 2022
    • Scheduled Maintenance Windows
    • Immuta Support Matrix Overview
    • Immuta CLI Release Notes
    • Preview Features
      • Features in Preview
    • Deprecations
  • Developer Guides
    • The Immuta CLI
      • Install and Configure the Immuta CLI
      • Manage Your Immuta Tenant
      • Manage Data Sources
      • Manage Sensitive Data Discovery
        • Manage Sensitive Data Discovery Rules
        • Manage Identification Frameworks
        • Run Sensitive Data Discovery on Data Sources
      • Manage Policies
      • Manage Projects
      • Manage Purposes
      • Manage Audit Export
    • The Immuta API
      • Integrations API
        • Getting Started
        • How-to Guides
          • Configure an Amazon S3 Integration
          • Configure an Azure Synapse Analytics Integration
          • Configure a Databricks Unity Catalog Integration
          • Configure a Google BigQuery Integration
          • Configure a Redshift Integration
          • Configure a Snowflake Integration
          • Configure a Starburst (Trino) Integration
        • Reference Guides
          • Integrations API Endpoints
          • Integration Configuration Payload
          • Response Schema
          • HTTP Status Codes and Error Messages
      • Connections API
        • How-to Guides
          • Register a Connection
            • Register a Snowflake Connection
            • Register a Databricks Unity Catalog Connection
            • Register an AWS Lake Formation Connection
          • Manage a Connection
          • Deregister a Connection
        • Connection Registration Payloads Reference Guide
      • Marketplace API
        • Marketplace API Endpoints
        • Source Controlling Data Products
      • Immuta V2 API
        • Data Source Payload Attribute Details
          • Data Source Request Payload Examples
        • Create Policies API Examples
        • Create Projects API Examples
        • Create Purposes API Examples
      • Immuta V1 API
        • Authenticate with the API
        • Configure Your Instance of Immuta
          • Get Job Status
          • Manage Frameworks
          • Manage IAMs
          • Manage Licenses
          • Manage Notifications
          • Manage Identifiers in Domains
            • API Changes - Identification Frameworks to Identifiers in Domains
          • Manage Sensitive Data Discovery (SDD)
          • Manage Tags
          • Manage Webhooks
          • Search Filters
        • Connect Your Data
          • Create and Manage an Amazon S3 Data Source
          • Create an Azure Synapse Analytics Data Source
          • Create a Databricks Data Source
          • Create a Redshift Data Source
          • Create a Snowflake Data Source
          • Create a Starburst (Trino) Data Source
          • Manage the Data Dictionary
        • Use Domains
        • Manage Data Access
          • Manage Access Requests
          • Manage Data and Subscription Policies
          • Manage Write Policies
            • Write Policies Payloads and Response Schema Reference Guide
          • Policy Handler Objects
          • Search Connection Strings
          • Search for Organizations
          • Search Schemas
        • Subscribe to and Manage Data Sources
        • Manage Projects and Purposes
          • Manage Projects
          • Manage Purposes
        • Generate Governance Reports
Powered by GitBook

Self-managed versions

  • 2025.1
  • 2024.3
  • 2024.2

Resources

  • Immuta Changelog

Copyright © 2014-2025 Immuta Inc. All rights reserved.

On this page
  • Requirements
  • Databricks workflow
  • Create a data source
  • Request example
  • Response example
  • Get information about a data source
  • Request example
  • Response example
  • Manage data sources
  • Update a specific data source
  • Update multiple data sources
  • Recalculate the high cardinality column for a data source

Was this helpful?

Export as PDF
  1. Developer Guides
  2. The Immuta API
  3. Immuta V1 API
  4. Connect Your Data

Create a Databricks Data Source

Databricks data source API reference guide

PreviousCreate an Azure Synapse Analytics Data SourceNextCreate a Redshift Data Source

Last updated 6 days ago

Was this helpful?

Deprecation notice

Support for registering Databricks Unity Catalog data sources using this legacy workflow has been deprecated. Instead, register your data using .

The databricks endpoint allows you to connect and manage Databricks data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Requirements

Databricks Spark integration

When exposing a table or view from an Immuta-enabled Databricks cluster, be sure that at least one of these traits is true:

  • The user exposing the tables has READ_METADATA and SELECT permissions on the target views/tables (specifically if Table ACLs are enabled).

  • The user exposing the tables is listed in the immuta.spark.acl.allowlist configuration on the target cluster.

  • The user exposing the tables is a Databricks workspace administrator.

Databricks Unity Catalog integration

When registering Databricks Unity Catalog securables in Immuta, use and ensure it has the privileges listed below. Immuta uses this service principal continuously to orchestrate Unity Catalog policies and maintain state between Immuta and Databricks.

  • USE CATALOG and MANAGE on all catalogs containing securables registered as Immuta data sources.

  • USE SCHEMA on all schemas containing securables registered as Immuta data sources.

  • MODIFY and SELECT on all securables you want registered as Immuta data sources.

MANAGE and MODIFY are required so that the service principal can apply row filters and column masks on the securable; to do so, the service principal must also have SELECT on the securable as well as USE CATALOG on its parent catalog and USE SCHEMA on its parent schema. Since privileges are inherited, you can grant the service principal the MODIFY and SELECT privilege on all catalogs or schemas containing Immuta data sources, which automatically grants the service principal the MODIFY and SELECT privilege on all current and future securables in the catalog or schema. The service principal also inherits MANAGE from the parent catalog for the purpose of applying row filters and column masks, but that privilege must be set directly on the parent catalog in order for grants to be fully applied.

Azure Databricks Unity Catalog limitation

Databricks workflow

Create a data source

POST /databricks/handler

Save the provided connection information as a data source.

Required Immuta permission: CREATE_DATA_SOURCE

Payload parameters

Attribute
Description
Required

private

boolean When false, the data source will be publicly available in the Immuta UI.

Yes

blobHandler

array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.

Yes

blobHandlerType

string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).

Yes

recordFormat

string The data format of blobs in the data source, such as json, xml, html, or jpeg.

Yes

type

string The type of data source: queryable (metadata is dynamically queried).

Yes

name

string The name of the data source. It must be unique within the Immuta tenant.

Yes

sqlTableName

string A string that represents this data source's table in Immuta.

Yes

organization

string The organization that owns the data source.

Yes

category

string The category of the data source.

No

description

string The description of the data source.

No

hasExamples

boolean When true, the data source contains examples.

No

Response parameters

Attribute
Description

id

integer The handler ID.

dataSourceId

integer The ID of the data source.

warnings

string This message describes issues with the created data source, such as the data source being unhealthy.

connectionString

string The connection string used to connect the data source to Immuta.

Request example

This request creates two Databricks data sources.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/handler

Payload example

{
  "handler": [{
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "port": 443,
      "hostname": "your-hostname.cloud.databricks.com",
      "database": "default",
      "httpPath": "sql/your/http/0/path",
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "default",
      "bodataTableName": "applicant_data",
      "dataSourceName": "Default Applicant Data",
      "table": "applicant_data",
      "schema": "default"
    }
  }, {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "port": 443,
      "hostname": "your-hostname.cloud.databricks.com",
      "database": "default",
      "httpPath": "sql/your/http/0/path",
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "default",
      "bodataTableName": "cities",
      "dataSourceName": "Default Cities",
      "table": "cities",
      "schema": "default"
    }
  }],
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Databricks",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString": "your-hostname.cloud.databricks.com:443/default"
}

Get information about a data source

GET /databricks/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute
Description
Required

handlerId

integer The ID of the handler.

Yes

skipCache

boolean When true, will skip the handler cache when retrieving metadata.

No

Response parameters

Attribute
Description

body

array[object] Metadata about the data source, including the data source ID, schema, database, and connection string.

Request example

This request returns metadata for the handler with the ID 48.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/databricks/handler/48

Response example

{
  "dataSourceId": 49,
  "metadata": {
    "ssl": true,
    "port": 443,
    "paths": ["/user/hive/warehouse/cities"],
    "query": null,
    "table": "cities",
    "schema": "default",
    "scheme": "dbfs",
    "database": "default",
    "hostname": "your-hostname.cloud.databricks.com",
    "httpPath": "sql/your/http/0/path",
    "pathUris": ["dbfs:/user/hive/warehouse/cities"],
    "ephemeral": true,
    "eventTime": null,
    "userFiles": [],
    "clusterName": null,
    "dataSourceName": "Default Cities",
    "bodataTableName": "cities",
    "metastoreTables": ["default.cities"],
    "bodataSchemaName": "default",
    "columnsNormalized": false,
    "schemaProjectName": "Default",
    "staleDataTolerance": 86400,
    "authenticationMethod": "Access Token"
  },
  "type": "queryable",
  "connectionString": "your-hostname.cloud.databricks.com:443/default",
  "id": 48,
  "createdAt": "2021-10-06T17:53:09.640Z",
  "updatedAt": "2021-10-06T17:53:09.882Z",
  "dbms": {
    "name": "databricks"
  }
}

Manage data sources

Method
Path
Purpose

PUT

/databricks/handler/{handlerId}

PUT

/databricks/bulk

PUT

/databricks/handler/{handlerId}/triggerHighCardinalityJob

Update a specific data source

PUT /databricks/handler/{handlerId}

Update the data source metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute
Description
Required

handlerId

integer The ID of the handler.

Yes

skipCache

boolean When true, will skip the handler cache when retrieving metadata.

No

Payload parameters

Attribute
Description
Required

handler

metadata Includes metadata about the handler, such as ssl, port, database, hostname, username, and password.

Yes

connectionString

string The connection string used to connect to the data source.

Yes

Response parameters

Attribute
Description

id

integer The ID of the handler.

ca

string The certificate authority.

columns

array[object] This is a Data Dictionary object, which provides metadata about the columns in the data source, including the name and data type of the column.

Request example

This request updates the metadata for the data source with the handler ID 48.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/handler/48

Payload example

The payload below updates the dataSourceName to Cities.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 49,
    "metadata": {
      "ssl": true,
      "port": 443,
      "paths": ["/user/hive/warehouse/cities"],
      "table": "cities",
      "schema": "default",
      "scheme": "dbfs",
      "database": "default",
      "hostname": "your-hostname.cloud.databricks.com",
      "httpPath": "sql/your/http/0/path",
      "pathUris": ["dbfs:/user/hive/warehouse/cities"],
      "ephemeral": true,
      "eventTime": null,
      "userFiles": [],
      "clusterName": null,
      "dataSourceName": "Cities",
      "bodataTableName": "cities",
      "metastoreTables": ["default.cities"],
      "bodataSchemaName": "default",
      "columnsNormalized": false,
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "authenticationMethod": "Access Token",
      "columns": [{
        "name": "OBJECTID",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "URBID",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "LIGHTDCW",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "ES90POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ES95POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ES00POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "PCOUNT",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "SCHNM",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "NAME",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "SQKM_FINAL",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ISO3",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "ISOURBID",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "REMOVED_PO",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "ADDED_POIN",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "YEAR_V1_01",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "POP_V1_01",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "Unsdcode",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "Countryeng",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "Continent",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "geometry",
        "dataType": "struct",
        "remoteType": "struct<__geom__:bigint,_is_empty:boolean,_ndim:bigint>",
        "nullable": true,
        "children": [{
          "name": "__geom__",
          "dataType": "bigint"
        }, {
          "name": "_is_empty",
          "dataType": "boolean"
        }, {
          "name": "_ndim",
          "dataType": "bigint"
        }]
      }, {
        "name": "wkt",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }],
      "password": "your-password"
    },
    "type": "queryable",
    "connectionString": "dbc-d3fe40ca-b4fb.cloud.databricks.com:443/default",
    "id": 48,
    "createdAt": "2021-10-06T17:53:09.640Z",
    "updatedAt": "2021-10-06T17:53:09.882Z",
    "dbms": {
      "name": "databricks"
    }
  }
}

Response example

{
  "id": 48,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "OBJECTID",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "URBID",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "LIGHTDCW",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "ES90POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ES95POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ES00POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "PCOUNT",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "SCHNM",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "NAME",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "SQKM_FINAL",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ISO3",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "ISOURBID",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "REMOVED_PO",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "ADDED_POIN",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "YEAR_V1_01",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "POP_V1_01",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "Unsdcode",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "Countryeng",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "Continent",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "geometry",
      "dataType": "struct",
      "remoteType": "struct<__geom__:bigint,_is_empty:boolean,_ndim:bigint>",
      "nullable": true,
      "children": [{
        "name": "__geom__",
        "dataType": "bigint"
      }, {
        "name": "_is_empty",
        "dataType": "boolean"
      }, {
        "name": "_ndim",
        "dataType": "bigint"
      }]
    }, {
      "name": "wkt",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }]
  }
}

Update multiple data sources

PUT /databricks/bulk

Update the data source metadata associated with the provided connection string.

Required: The global GOVERNANCE permission or be the data source owner

Payload parameters

Attribute
Description
Required

handler

metadata Includes metadata about the handler, such as ssl, port, database, hostname, username, and password.

Yes

connectionString

string The connection string used to connect to the data sources.

Yes

Response parameters

Attribute
Description

bulkId

string The ID of the bulk data source update.

connectionString

string The connection string shared by the data sources bulk updated.

jobsCreated

integer The number of jobs that ran to update the data sources; this number corresponds to the number of data sources updated.

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/bulk

Payload example

The payload below adds a certificate (certificate.json) to connect to the data sources with the provided connection.

{
  "handler": {
    "metadata": {
      "ssl": true,
      "port": 443,
      "database": "default",
      "hostname": "your-hostname.cloud.databricks.com",
      "userFiles": [{
        "keyName": "test",
        "filename": "6dc06a3310b9ba33c543e483d1e745b3ac9bc648.json",
        "userFilename": "certificate.json"
      }],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "httpPath": "sql/your/http/0/path"
    }
  },
  "connectionString": "your-hostname.cloud.databricks.com:443/default/default"
}

Response example

{
  "bulkId": "bulk_ds_update_9ae5bfd85a3a47a8b454c618043e2aa3",
  "connectionString": "your-hostname.cloud.databricks.com:443/default",
  "jobsCreated": 2
}

Recalculate the high cardinality column for a data source

PUT /databricks/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute
Description
Required

handlerId

integer The ID of the handler.

Yes

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 47.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/databricks/handler/47/triggerHighCardinalityJob

Response example

f6ac1ad0-26d0-11ec-8078-d36bbf5b90fb

Set all table-level ownership on your Unity Catalog data sources to an individual user or service principal instead of a Databricks group before proceeding. Otherwise, Immuta cannot apply data policies to the table in Unity Catalog. See the for details.

.

.

.

. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary.

.

.

Create a data source
Get information about a data source
Manage data sources
Update the data source metadata associated with the provided handler ID
Update the data source metadata associated with the provided connection string
Recalculate the high cardinality column for the specified data source
connections
the service principal from the integration configuration
Azure Databricks Unity Catalog limitation