Databricks Unity Catalog
Immuta’s integration with Unity Catalog allows you to manage multiple Databricks workspaces through Unity Catalog while protecting your data with Immuta policies. Instead of manually creating UDFs or granting access to each table in Databricks, you can author your policies in Immuta and have Immuta manage and enforce Unity Catalog access-control policies on your data in Databricks clusters or SQL warehouses.
Use the /integrations
endpoint to
Requirements
APPLICATION_ADMIN
Immuta permissionA system account user with the following privileges is required for Immuta to manage permissions on objects in the Unity Catalog metastore:
CREATE CATALOG
on the Unity Catalog metastore to create an Immuta-owned catalog and tablesOWNERSHIP
on the Immuta catalog you configureUSE CATALOG
andUSE SCHEMA
on all catalogs and schemas with tables managed by ImmutaSELECT
andMODIFY
on all tables in the metastore managed my ImmutaNative query audit:
USE CATALOG
on thesystem
catalog for native query auditUSE SCHEMA
on thesystem.access
schema for native query auditSELECT
on the following system tables for native query audit:system.access.audit
system.access.table_lineage
system.access.column_lineage
Generate a personal access token for the system account user that Immuta will use to manage policies in Unity Catalog.
Prerequisite
Enable Databricks Unity Catalog on the Immuta app settings page:
Click the App Settings icon in the left sidebar.
Scroll to the Global Integrations Settings section and check the Enable Databricks Unity Catalog support in Immuta checkbox. Leave the additional settings in this section with their default values. They are only relevant to the Databricks Spark with Unity Catalog integration and will not have any effect on the Unity Catalog integration.
Configure the integration
You have two options for configuring your Databricks Unity Catalog integration:
Automatic setup: When performing an automatic setup, the Databricks personal access token you configure below must be attached to an account with these Databricks permissions for the metastore associated with the specified Databricks workspace. Immuta creates the catalogs, schemas, tables, and functions using the integration's configured personal access token.
Manual setup: Run the Immuta script in Databricks yourself to create the catalog. You can also modify the script to customize your storage location for tables, schemas, or catalogs. The user running the script needs to have the
CREATE CATALOG
permission on the workspace metastore. The Databricks personal access token you configure must be attached to an account with the Databricks permissions listed in the requirements section.
Automatic setup
Copy the request example, and replace the values with your own as directed to configure the integration settings. The example provided uses JSON format, but the request also accepts YAML.
See the config object description for parameter definitions, value types, and additional configuration options.
Replace the Immuta URL and API key with your own.
Change the config values to your own, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Response
The response returns the status of the Databricks Unity Catalog integration configuration connection. See the response schema reference for details about the response schema.
A successful response includes the validation tests statuses.
Manual setup
To manually configure the integration, complete the following steps:
Generate the script
Copy the request example, and replace the values with your own as directed to configure the integration settings. The example provided uses JSON format, but the request also accepts YAML.
See the config object description for parameter definitions, value types, and additional configuration options.
Replace the Immuta URL and API key with your own.
Change the config values to your own, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Run the script returned in the response in your Databricks environment.
Response
The response returns the script for you to run in your environment.
Configure the integration in Immuta
Copy the request example, and replace the values with your own as directed to configure the integration settings. The example provided uses JSON format, but the request also accepts YAML. The payload you provide must match the payload sent when generating the script.
See the config object description for parameter definitions, value types, and additional configuration options.
Replace the Immuta URL and API key with your own.
Pass the same payload you sent when generating the script, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Response
The response returns the status of the Databricks Unity Catalog integration configuration connection. See the response schema reference for details about the response schema.
A successful response includes the validation tests statuses.
Get an integration
Copy the request example.
Replace the Immuta URL and API key with your own.
Replace the {id} request parameter with the unique identifier of the integration you want to get. Alternatively, you can get a list of all integrations and their IDs with the
GET /integrations
endpoint.
Response
The response returns a Databricks Unity Catalog integration configuration. See the response schema reference for details about the response schema. An unsuccessful request returns the status code and an error message. See the HTTP status codes and error messages for a list of statuses, error messages, and troubleshooting guidance.
Get all integrations
Copy the request example.
Replace the Immuta URL and API key with your own.
Response
The response returns the configuration for all integrations. See the response schema reference for details about the response schema. An unsuccessful request returns the status code and an error message. See the HTTP status codes and error messages for a list of statuses, error messages, and troubleshooting guidance.
Update an integration configuration
You have two options for updating your integration. Follow the steps that match your initial configuration of autoBootstrap:
automatic update (autoBootstrap is
true
)manual update (autoBootstrap is
false
)
Automatic update
Copy the request example, and replace the values with your own as directed to configure the integration settings. The examples provided use JSON format, but the request also accepts YAML.
See the config object description for parameter definitions, value types, and additional configuration options.
This example updates the access token.
Replace the Immuta URL and API key with your own.
Replace the {id} request parameter with the unique identifier of the integration you want to update.
Change the config values to your own, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Response
The response returns the status of the Databricks Unity Catalog integration configuration connection. See the response schema reference for details about the response schema.
A successful response includes the validation tests statuses.
Manual update
To manually update the integration, complete the following steps:
Generate the updated script
Copy the request example, and replace the values with your own as directed to generate the script. The example provided uses JSON format, but the request also accepts YAML.
See the config object description for parameter definitions, value types, and additional configuration options.
Replace the Immuta URL and API key with your own.
Replace the {id} request parameter with the unique identifier of the integration you want to update.
Change the config values to your own, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Run the script returned in the response in your Databricks environment.
Response
The response returns the script for you to run in your Databricks environment.
Update the integration in Immuta
Copy the request example, and replace the values with your own as directed to update the integration settings. The example provided uses JSON format, but the request also accepts YAML. The payload you provide must match the payload sent when generating the script.
See the config object description for parameter definitions, value types, and additional configuration options.
Replace the Immuta URL and API key with your own.
Replace the {id} request parameter with the unique identifier of the integration you want to update.
Pass the same payload you sent when updating the script, where
workspaceUrl is your Databricks workspace URL.
httpPath is the HTTP path of your Databricks cluster or SQL warehouse.
token is the Databricks personal access token. This is the access token for the Immuta system account user.
catalog is the name of the Databricks catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.
Response
The response returns the status of the Databricks Unity Catalog integration configuration connection. See the response schema reference for details about the response schema.
A successful response includes the validation tests statuses.
Delete an integration
Copy the request example.
Replace the Immuta URL and API key with your own.
Replace the {id} request parameter with the unique identifier of the integration you want to delete.
Response
The response returns the status of the Databricks Unity Catalog integration configuration that has been deleted. See the response schema reference for details about the response schema. An unsuccessful request returns the status code and an error message. See the HTTP status codes and error messages for a list of statuses, error messages, and troubleshooting guidance.
Last updated