Register a Databricks Unity Catalog Connection

Public preview

This feature is public preview and available to select accounts. Reach out to your Immuta support professional to enable it on your tenant.

Requirements

The following permissions and personas are used in the registration process:

• Immuta permission: CREATE_DATA_SOURCE

• Databricks privileges for the user registering the connection and running the script:

  • Account or workspace admin

  • CREATE CATALOG privilege on the Unity Catalog metastore to create an Immuta-owned catalog and tables

• Databricks privileges for the service principal you create:

  • OWNER privilege on the Immuta catalog you configure.

  • OWNER privilege on catalogs with schemas and tables registered as Immuta data sources so that Immuta can administer Unity Catalog row-level and column-level security controls. This privilege can be applied by granting OWNER on a catalog to a Databricks group that includes the Immuta service principal to allow for multiple owners. If the OWNER privilege cannot be applied at the catalog- or schema-level, each table registered as an Immuta data source must individually have the OWNER privilege granted to the Immuta service principal.

  • USE CATALOG and USE SCHEMA on parent catalogs and schemas of tables registered as Immuta data sources so that the Immuta service principal can interact with those tables.

  • SELECT and MODIFY on all tables registered as Immuta data sources so that the Immuta service principal can grant and revoke access to tables and apply Unity Catalog row- and column-level security controls.

  • USE CATALOG on the system catalog for native query audit.

  • USE SCHEMA on the system.access schema for native query audit.

  • SELECT on the following system tables for native query audit:

    • system.access.audit

    • system.access.table_lineage

    • system.access.column_lineage

Prerequisites

• Unity Catalog metastore created and attached to a Databricks workspace. See the Databricks Unity Catalog reference guide for information on workspaces and catalog isolation support with Immuta.

• Unity Catalog enabled on your Databricks cluster or SQL warehouse. All SQL warehouses have Unity Catalog enabled if your workspace is attached to a Unity Catalog metastore. Immuta recommends linking a SQL warehouse to your Immuta tenant rather than a cluster for both performance and availability reasons.

Register a connection

1. Click Data and select the Infrastructure tab in the navigation menu.

2. Click the + Add Host button.

3. Select the Databricks data platform tile.

4. Enter the connection information:

   • Host: The hostname of your Databricks workspace.

   • Port: Your Databricks port.

   • HTTP Path: The HTTP path of your Databricks cluster or SQL warehouse.

   • Immuta Catalog: The name of the catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.

   • Connection Key: The connection key represents the unique name of your connection and will be used as prefix in the name for all data objects associated with this connection. It will also appear as the display name in the UI and will be used in all API calls made to update or delete the connection.

5. Click Next.

6. Select Access Token authentication method from the dropdown menu.

7. Enter the Access Token in the Immuta System Account Credentials section. This is the access token for the Immuta service principal. This service principal must have the metastore privileges listed in the requirements section at the top of this page for the metastore associated with the Databricks workspace. If this token is configured to expire, update this field regularly for the integration to continue to function. This authentication information will be included in the script populated later on the page.

8. Copy the provided script and run it in Databricks as a user with the CREATE CATALOG privilege on the Unity Catalog metastore.

9. Click Validate Connection.

10. If the connection is successful, click Next. If there are any errors, check the connection details and credentials to ensure they are correct and try again.

11. Ensure all the details are correct in the summary and click Complete Setup.