LogoLogo
2024.2
  • Immuta Documentation - 2024.2
  • What is Immuta?
  • Self-Managed Deployment
    • Getting Started
    • Deployment Requirements
    • Install
      • Managed Public Cloud
      • Red Hat OpenShift
      • Generic Installation
      • Immuta in an Air-Gapped Environment
      • Deploy Immuta without Elasticsearch
    • Configure
      • Ingress Configuration
      • Cosign Verification
      • TLS Configuration
      • Immuta in Production
      • External Cache Configuration
      • Rotating Credentials
      • Enabling Legacy Query Engine and Fingerprint
    • Upgrade
      • Upgrade Immuta
      • Upgrade to Immuta 2024.2 LTS
    • Disaster Recovery
    • Troubleshooting
    • Conventions
    • Release Notes
  • Data and Integrations
    • Immuta Integrations
    • Snowflake
      • Getting Started
      • How-to Guides
        • Configure a Snowflake Integration
        • Snowflake Table Grants Migration
        • Edit or Remove Your Snowflake Integration
        • Integration Settings
          • Enable Snowflake Table Grants
          • Use Snowflake Data Sharing with Immuta
          • Configure Snowflake Lineage Tag Propagation
          • Enable Snowflake Low Row Access Policy Mode
            • Upgrade Snowflake Low Row Access Policy Mode
      • Reference Guides
        • Snowflake Integration
        • Snowflake Data Sharing
        • Snowflake Lineage Tag Propagation
        • Snowflake Low Row Access Policy Mode
        • Snowflake Table Grants
        • Warehouse Sizing Recommendations
      • Phased Snowflake Onboarding Concept Guide
    • Databricks Unity Catalog
      • Getting Started
      • How-to Guides
        • Configure a Databricks Unity Catalog Integration
        • Migrate to Unity Catalog
      • Databricks Unity Catalog Integration Reference Guide
    • Databricks Spark
      • How-to Guides
        • Configuration
          • Simplified Databricks Configuration
          • Manual Databricks Configuration
          • Manually Update Your Databricks Cluster
          • Install a Trusted Library
        • DBFS Access
        • Limited Enforcement in Databricks
        • Hide the Immuta Database in Databricks
        • Run spark-submit Jobs on Databricks
        • Configure Project UDFs Cache Settings
        • External Metastores
      • Reference Guides
        • Databricks Spark Integration
        • Databricks Spark Pre-Configuration Details
        • Configuration Settings
          • Cluster Policies
            • Python & SQL
            • Python & SQL & R
            • Python & SQL & R with Library Support
            • Scala
            • Sparklyr
          • Environment Variables
          • Ephemeral Overrides
          • Py4j Security Error
          • Scala Cluster Security Details
          • Databricks Security Configuration for Performance
        • Databricks Change Data Feed
        • Databricks Libraries Introduction
        • Delta Lake API
        • Spark Direct File Reads
        • Databricks Metastore Magic
    • Starburst (Trino)
      • Getting Started
      • How-to Guides
        • Configure Starburst (Trino) Integration
        • Customize Read and Write Access Policies for Starburst (Trino)
      • Starburst (Trino) Integration Reference Guide
    • Redshift
      • Getting Started
      • How-to Guides
        • Configure Redshift Integration
        • Configure Redshift Spectrum
      • Reference Guides
        • Redshift Integration
        • Redshift Pre-Configuration Details
    • Azure Synapse Analytics
      • Getting Started
      • Configure Azure Synapse Analytics Integration
      • Reference Guides
        • Azure Synapse Analytics Integration
        • Azure Synapse Analytics Pre-Configuration Details
    • Amazon S3
    • Google BigQuery
    • Legacy Integrations
      • Securing Hive and Impala Without Sentry
      • Enabling ImmutaGroupsMapping
    • Registering Metadata
      • Data Sources in Immuta
      • Register Data Sources
        • Create a Data Source
        • Create an Amazon S3 Data Source
        • Create a Google BigQuery Data Source
        • Bulk Create Snowflake Data Sources
      • Data Source Settings
        • How-to Guides
          • Manage Data Sources and Data Source Settings
          • Manage Data Source Members
          • Manage Access Requests and Tasks
          • Manage Data Dictionary Descriptions
          • Disable Immuta from Sampling Raw Data
        • Data Source Health Checks Reference Guide
      • Schema Monitoring
        • How-to Guides
          • Run Schema Monitoring and Column Detection Jobs
          • Manage Schema Monitoring
        • Reference Guides
          • Schema Monitoring
          • Schema Projects
        • Why Use Schema Monitoring?
    • Catalogs
      • Getting Started with External Catalogs
      • Configure an External Catalog
      • Reference Guides
        • External Catalogs
        • Custom REST Catalogs
          • Custom REST Catalog Interface Endpoints
    • Tags
      • How-to Guides
        • Create and Manage Tags
        • Add Tags to Data Sources and Projects
      • Tags Reference Guide
  • People
    • Getting Started
    • Identity Managers (IAMs)
      • How-to Guides
        • Microsoft Entra ID
        • Okta LDAP Interface
        • Okta and OpenID Connect
        • Integrate Okta SAML SCIM with Immuta
        • OneLogin with OpenID
        • Configure SAML IAM Protocol
      • Reference Guides
        • Identity Managers
        • SAML Single Logout
        • SAML Protocol Configuration Options
    • Immuta Users
      • How-to Guides
        • Managing Personas and Permissions
        • Manage Attributes and Groups
        • User Impersonation
        • External User ID Mapping
        • External User Info Endpoint
      • Reference Guides
        • Attributes and Groups in Immuta
        • Permissions and Personas
  • Discover Your Data
    • Getting Started
    • Introduction
    • Architecture
    • Data Discovery
      • How-to Guides
        • Enable Sensitive Data Discovery (SDD)
        • Manage Identification Frameworks
        • Manage Patterns
        • Manage Rules
        • Manage SDD on Data Sources
        • Manage Global SDD Settings
        • Migrate From Legacy to Native SDD
      • Reference Guides
        • How Competitive Pattern Analysis Works
        • Built-in Pattern Reference
        • Built-in Discovered Tags Reference
    • Data Classification
      • How-to Guides
        • Activate Classification Frameworks
        • Adjust Identification and Classification Framework Tags
        • How to Use a Built-In Classification Framework with Your Own Tags
      • Built-in Classification Frameworks Reference Guide
  • Detect Your Activity
    • Getting Started
      • Monitor and Secure Sensitive Data Platform Query Activity
        • User Identity Best Practices
        • Integration Architecture
        • Snowflake Roles Best Practices
        • Register Data Sources
        • Automate Entity and Sensitivity Discovery
        • Detect with Discover: Onboarding Guide
        • Using Immuta Detect
      • General Immuta Configuration
        • User Identity Best Practices
        • Integration Architecture
        • Databricks Roles Best Practices
        • Register Data Sources
    • Introduction
    • Audit
      • How-to Guides
        • Export Audit Logs to S3
        • Export Audit Logs to ADLS
        • Run Governance Reports
      • Reference Guides
        • Universal Audit Model (UAM)
        • Snowflake Query Audit Logs
        • Databricks Unity Catalog Audit Logs
        • Databricks Query Audit Logs
        • Starburst (Trino) Query Audit Logs
        • UAM Schema
        • Audit Export CLI
        • Governance Report Types
      • Deprecated Audit Guides
        • Legacy to UAM Migration
        • Download Audit Logs
        • System Audit Logs
    • Detection
      • Use the Detect Dashboards
      • Reference Guides
        • Detect
        • Detect Dashboards
        • Unknown Users in Audit Logs
    • Monitors
      • Manage Monitors and Observations
      • Detect Monitors Reference Guide
  • Secure Your Data
    • Getting Started with Secure
      • Automate Data Access Control Decisions
        • The Two Paths: Orchestrated RBAC and ABAC
        • Managing User Metadata
        • Managing Data Metadata
        • Author Policy
        • Test and Deploy Policy
      • Compliantly Open More Sensitive Data for ML and Analytics
        • Managing User Metadata
        • Managing Data Metadata
        • Author Policy
      • Federated Governance for Data Mesh and Self-Serve Data Access
        • Defining Domains
        • Managing Data Products
        • Managing Data Metadata
        • Apply Federated Governance
        • Discover and Subscribe to Data Products
    • Introduction
      • Scalability and Evolvability
      • Understandability
      • Distributed Stewardship
      • Consistency
      • Availability of Data
    • Authoring Policies in Secure
      • Authoring Policies at Scale
      • Data Engineering with Limited Policy Downtime
      • Subscription Policies
        • How-to Guides
          • Author a Subscription Policy
          • Author an ABAC Subscription Policy
          • Subscription Policies Advanced DSL Guide
          • Author a Restricted Subscription Policy
          • Clone, Activate, or Stage a Global Policy
        • Reference Guides
          • Subscription Policies
          • Subscription Policy Access Types
          • Advanced Use of Special Functions
      • Data Policies
        • Overview
        • How-to Guides
          • Author a Masking Data Policy
          • Author a Minimization Policy
          • Author a Purpose-Based Restriction Policy
          • Author a Restricted Data Policy
          • Author a Row-Level Policy
          • Author a Time-Based Restriction Policy
          • Certifications Exemptions and Diffs
          • External Masking Interface
        • Reference Guides
          • Data Policy Types
          • Masking Policies
          • Row-Level Policies
          • Custom WHERE Clause Functions
          • Data Policy Conflicts and Fallback
          • Custom Data Policy Certifications
          • Orchestrated Masking Policies
    • Domains
      • Getting Started with Domains
      • Domains Reference Guide
    • Projects and Purpose-Based Access Control
      • Projects and Purpose Controls
        • Getting Started
        • How-to Guides
          • Create a Project
          • Create and Manage Purposes
          • Adjust a Policy
          • Project Management
            • Manage Projects and Project Settings
            • Manage Project Data Sources
            • Manage Project Members
        • Reference Guides
          • Projects and Purposes
          • Policy Adjustments
        • Why Use Purposes?
      • Equalized Access
        • Manage Project Equalization
        • Project Equalization Reference Guide
        • Why Use Project Equalization?
      • Masked Joins
        • Enable Masked Joins
        • Why Use Masked Joins?
      • Writing to Projects
        • How-to Guides
          • Create and Manage Snowflake Project Workspaces
          • Create and Manage Databricks Project Workspaces
          • Write Data to the Workspace
        • Reference Guides
          • Project Workspaces
          • Project UDFs (Databricks)
    • Data Consumers
      • Subscribe to a Data Source
      • Query Data
        • Querying Snowflake Data
        • Querying Databricks Data
        • Querying Databricks SQL Data
        • Querying Starburst (Trino) Data
        • Querying Redshift Data
        • Querying Azure Synapse Analytics Data
      • Subscribe to Projects
  • Application Settings
    • How-to Guides
      • App Settings
      • BI Tools
        • BI Tool Configuration Recommendations
        • Power BI Configuration Example
        • Tableau Configuration Example
      • Add a License Key
      • Add ODBC Drivers
      • Manage Encryption Keys
      • System Status Bundle
    • Reference Guides
      • Data Processing, Encryption, and Masking Practices
      • Metadata Ingestion
  • Releases
    • Immuta v2024.2 Release Notes
    • Immuta Release Lifecycle
    • Immuta LTS Changelog
    • Immuta Support Matrix Overview
    • Immuta CLI Release Notes
    • Immuta Image Digests
    • Preview Features
      • Features in Preview
    • Deprecations
  • Developer Guides
    • The Immuta CLI
      • Install and Configure the Immuta CLI
      • Manage Your Immuta Tenant
      • Manage Data Sources
      • Manage Sensitive Data Discovery
        • Manage Sensitive Data Discovery Rules
        • Manage Identification Frameworks
        • Run Sensitive Data Discovery on Data Sources
      • Manage Policies
      • Manage Projects
      • Manage Purposes
    • The Immuta API
      • Integrations API
        • Getting Started
        • How-to Guides
          • Configure an Amazon S3 Integration
          • Configure an Azure Synapse Analytics Integration
          • Configure a Databricks Unity Catalog Integration
          • Configure a Google BigQuery Integration
          • Configure a Redshift Integration
          • Configure a Snowflake Integration
          • Configure a Starburst (Trino) Integration
        • Reference Guides
          • Integrations API Endpoints
          • Integration Configuration Payload
          • Response Schema
          • HTTP Status Codes and Error Messages
      • Immuta V2 API
        • Data Source Payload Attribute Details
        • Data Source Request Payload Examples
        • Create Policies API Examples
        • Create Projects API Examples
        • Create Purposes API Examples
      • Immuta V1 API
        • Authenticate with the API
        • Configure Your Instance of Immuta
          • Get Fingerprint Status
          • Get Job Status
          • Manage Frameworks
          • Manage IAMs
          • Manage Licenses
          • Manage Notifications
          • Manage Sensitive Data Discovery (SDD)
          • Manage Tags
          • Manage Webhooks
          • Search Filters
        • Connect Your Data
          • Create and Manage an Amazon S3 Data Source
          • Create an Azure Synapse Analytics Data Source
          • Create an Azure Blob Storage Data Source
          • Create a Databricks Data Source
          • Create a Presto Data Source
          • Create a Redshift Data Source
          • Create a Snowflake Data Source
          • Create a Starburst (Trino) Data Source
          • Manage the Data Dictionary
        • Manage Data Access
          • Manage Access Requests
          • Manage Data and Subscription Policies
          • Manage Domains
          • Manage Write Policies
            • Write Policies Payloads and Response Schema Reference Guide
          • Policy Handler Objects
          • Search Audit Logs
          • Search Connection Strings
          • Search for Organizations
          • Search Schemas
        • Subscribe to and Manage Data Sources
        • Manage Projects and Purposes
          • Manage Projects
          • Manage Purposes
        • Generate Governance Reports
Powered by GitBook

Other versions

  • SaaS
  • 2024.3

Copyright © 2014-2024 Immuta Inc. All rights reserved.

On this page
  • Permissions
  • Requirements
  • Migrate data to Unity Catalog
  • Configure the Databricks Unity Catalog integration
  • Automatic setup
  • Manual setup
  • Enable query audit for Unity Catalog
  • Map Databricks users to Immuta
  • Register data

Was this helpful?

Export as PDF
  1. Data and Integrations
  2. Databricks Unity Catalog
  3. How-to Guides

Configure a Databricks Unity Catalog Integration

PreviousHow-to GuidesNextMigrate to Unity Catalog

Last updated 16 days ago

Was this helpful?

allows you to manage and access data in your Databricks account across all of your workspaces. With Immuta’s Databricks Unity Catalog integration, you can write your policies in Immuta and have them enforced automatically by Databricks across data in your Unity Catalog metastore.

Permissions

The following permissions and personas are used in the registration process.

  • Immuta user: An Immuta user with the APPLICATION_ADMIN Immuta permission must configure the Databricks Unity Catalog integration.

  • Databricks user: The Databricks user must have the following privileges.

    • Account admin

    • CREATE CATALOG privilege on the Unity Catalog metastore to create an Immuta-owned catalog and tables

    • (only required if enabling query audit)

  • :

    • USE CATALOG and MANAGE on all catalogs containing securables registered as Immuta data sources and USE SCHEMA on all schemas containing securables registered as Immuta data sources.

    • MODIFY and SELECT on all securables registered as Immuta data sources. MANAGE and MODIFY are required so that the service principal can apply row filters and column masks on the securable; to do so, the service principal must also have SELECT on the securable as well as USE CATALOG on its parent catalog and USE SCHEMA on its parent schema. Since privileges are inherited, you can grant the service principal the MODIFY and SELECT privilege on all catalogs or schemas containing Immuta data sources, which automatically grants the service principal the MODIFY and SELECT privilege on all current and future securables in the catalog or schema. The service principal also inherits MANAGE from the parent catalog for the purpose of applying row filters and column masks, but that privilege must be set directly on the parent catalog in order for grants to be fully applied.

    • Optionally, to include audit, the service principal needs the following additional privileges:

      • USE CATALOG on system catalog

        • USE SCHEMA on system.access schema

        • SELECT on system.access.audit table

        • SELECT on system.access.table_lineage table

        • SELECT on system.access.column_lineage table

      Access to system tables is governed by Unity Catalog. No user has access to these system schemas by default. To grant access, a user that is both a metastore admin and an account admin must grant USE and SELECT permissions on the system schemas to the service principal. See . The system.access schema must also be on the metastore before it can be used.

Requirements

Before you configure the Databricks Unity Catalog integration, ensure that you have fulfilled the following requirements:

  • Unity Catalog enabled on your Databricks cluster or SQL warehouse. All SQL warehouses have Unity Catalog enabled if your workspace is attached to a Unity Catalog metastore. Immuta recommends linking a SQL warehouse to your Immuta tenant rather than a cluster for both performance and availability reasons.

  • If you select single user access mode for your cluster, you must

    • enable serverless compute for your workspace.

Unity Catalog best practices

Ensure your integration with Unity Catalog goes smoothly by following these guidelines:

  • Use a Databricks SQL warehouse to configure the integration. Databricks SQL warehouses are faster to start than traditional clusters, require less management, and can run all the SQL that Immuta requires for policy administration. A serverless warehouse provides nearly instant startup time and is the preferred option for connecting to Immuta.

  • Move all data into Unity Catalog before configuring Immuta with Unity Catalog. The default catalog used once Unity Catalog support is enabled in Immuta is the hive_metastore, which is not supported by the Unity Catalog integration. Data sources in the Hive Metastore must be managed by the Databricks Spark integration. Existing data sources will need to be re-created after they are moved to Unity Catalog and the Unity Catalog integration is configured.

Migrate data to Unity Catalog

  1. Ensure that all Databricks clusters that have Immuta installed are stopped and the Immuta configuration is removed from the cluster. Immuta-specific cluster configuration is no longer needed with the Databricks Unity Catalog integration.

Configure the Databricks Unity Catalog integration

You have two options for configuring your Databricks Unity Catalog integration:

Automatic setup

  1. Click the App Settings icon in the left sidebar.

  2. Scroll to the Global Integrations Settings section and check the Enable Databricks Unity Catalog support in Immuta checkbox.

  3. Click the Integrations tab.

  4. Click + Add Integration and select Databricks Unity Catalog from the dropdown menu.

  5. Complete the following fields:

    • Server Hostname is the hostname of your Databricks workspace.

    • HTTP Path is the HTTP path of your Databricks cluster or SQL warehouse.

    • Immuta Catalog is the name of the catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.

  6. If using a proxy server with Databricks Unity Catalog, click the Enable Proxy Support checkbox and complete the Proxy Host and Proxy Port fields. The username and password fields are optional.

  7. Opt to fill out the Exemption Group field with the name of a group in Databricks that will be excluded from having data policies applied and must not be changed from the default value. Create this account-level group for privileged users and service accounts that require an unmasked view of data before configuring the integration in Immuta.

    1. Opt to scope the query audit ingestion by entering in Unity Catalog Workspace IDs. Enter a comma-separated list of the workspace IDs that you want Immuta to ingest audit records for. If left empty, Immuta will audit all tables and users in Unity Catalog.

    2. Enter how often, in hours, you want Immuta to ingest audit events from Unity Catalog as an integer between 1 and 24.

    3. Continue with your integration configuration.

  8. Select your authentication method from the dropdown:

    • OAuth machine-to-machine (M2M):

      • AWS Databricks:

        • Fill out the Token Endpoint with the full URL of the identity provider. This is where the generated token is sent. The default value is https://<your workspace name>.cloud.databricks.com/oidc/v1/token.

        • Enter the Client Secret you created above. Immuta uses this secret to authenticate with the authorization server when it requests a token.

      • Azure Databricks:

        • Within Immuta, fill out the Token Endpoint with the full URL of the identity provider. This is where the generated token is sent. The default value is https://<your workspace name>.azuredatabricks.net/oidc/v1/token.

        • Enter the Client Secret you created above. Immuta uses this secret to authenticate with the authorization server when it requests a token.

  9. Click Save.

Manual setup

  1. Click the App Settings icon in the left sidebar.

  2. Scroll to the Global Integrations Settings section and check the Enable Databricks Unity Catalog support in Immuta checkbox.

  3. Click the Integrations tab.

  4. Click + Add Integration and select Databricks Unity Catalog from the dropdown menu.

  5. Complete the following fields:

    • Server Hostname is the hostname of your Databricks workspace.

    • HTTP Path is the HTTP path of your Databricks cluster or SQL warehouse.

    • Immuta Catalog is the name of the catalog Immuta will create to store internal entitlements and other user data specific to Immuta. This catalog will only be readable for the Immuta service principal and should not be granted to other users. The catalog name may only contain letters, numbers, and underscores and cannot start with a number.

  6. If using a proxy server with Databricks Unity Catalog, click the Enable Proxy Support checkbox and complete the Proxy Host and Proxy Port fields. The username and password fields are optional.

  7. Opt to fill out the Exemption Group field with the name of a group in Databricks that will be excluded from having data policies applied and must not be changed from the default value. Create this account-level group for privileged users and service accounts that require an unmasked view of data before configuring the integration in Immuta.

    1. Opt to scope the query audit ingestion by entering in Unity Catalog Workspace IDs. Enter a comma-separated list of the workspace IDs that you want Immuta to ingest audit records for. If left empty, Immuta will audit all tables and users in Unity Catalog.

    2. Enter how often, in hours, you want Immuta to ingest audit events from Unity Catalog as an integer between 1 and 24.

    3. Continue with your integration configuration.

  8. Select your authentication method from the dropdown:

    • OAuth machine-to-machine (M2M):

      • AWS Databricks:

        • Fill out the Token Endpoint with the full URL of the identity provider. This is where the generated token is sent. The default value is https://<your workspace name>.cloud.databricks.com/oidc/v1/token.

        • Enter the Client Secret you created above. Immuta uses this secret to authenticate with the authorization server when it requests a token.

      • Azure Databricks:

        • Within Immuta, fill out the Token Endpoint with the full URL of the identity provider. This is where the generated token is sent. The default value is https://<your workspace name>.azuredatabricks.net/oidc/v1/token.

        • Enter the Client Secret you created above. Immuta uses this secret to authenticate with the authorization server when it requests a token.

  9. Select the Manual toggle and copy or download the script. You can modify the script to customize your storage location for tables, schemas, or catalogs.

  10. Run the script in Databricks.

  11. Click Save.

Enable query audit for Unity Catalog

To enable query audit for Unity Catalog, complete the following steps before configuring the integration:

    • USE CATALOG on the system catalog

    • USE SCHEMA on the system.access schema

    • SELECT on the following system tables:

      • system.access.audit

      • system.access.table_lineage

      • system.access.column_lineage

  1. Use the Databricks Personal Access Token in the configuration above for the account you just granted system table access. This account will be the Immuta service principal.

Map Databricks users to Immuta

If the usernames in Immuta do not match usernames in Databricks, map each Databricks username to each Immuta user account to ensure Immuta properly enforces policies using one of the methods linked below:

Register data

See the for more details about Unity Catalog privileges and securable objects.

Unity Catalog and attached to a Databricks workspace. Immuta supports configuring a single metastore for each configured integration, and that metastore may be attached to multiple Databricks workspaces.

with the outlined above has been created for Immuta to use to manage policies in Unity Catalog.

for query audit.

use Databricks Runtime 15.4 LTS and above. Unity Catalog row- and column-level security controls are unsupported for single user access mode on Databricks Runtime 15.3 and below. See the for details.

Move all data into Unity Catalog before configuring Immuta with Unity Catalog. Existing data sources will need to be re-created after they are moved to Unity Catalog and the Unity Catalog integration is configured. If you don't move all data before configuring the integration, will protect your existing data sources throughout the migration process.

Existing data source migration: If you have existing Databricks data sources, complete these steps before proceeding.

: Immuta creates the catalogs, schemas, tables, and functions using the integration's configured personal access token.

: Run the Immuta script in Databricks yourself to create the catalog. You can also modify the script to customize your storage location for tables, schemas, or catalogs.

Required permissions: When performing an automatic setup, the credentials provided must have the .

is enabled by default. .

Configure the by scrolling to Integrations Settings and find the Unity Catalog Audit Sync Schedule section.

Access Token: Enter a Databricks Personal Access Token. This is the access token for the Immuta service principal. This service principal must have the for the metastore associated with the Databricks workspace. If this token is configured to expire, update this field regularly for the integration to continue to function.

Follow for the Immuta service principal and assign this service principal the for the metastore associated with the Databricks workspace.

Fill out the Client ID. This is a combination of letters, numbers, or symbols, used as a public identifier and is the .

Enter the Scope (string). The scope limits the operations and roles allowed in Databricks by the access token. See the for details about scopes.

Follow to create a service principal within Azure and then populate to your Databricks account and workspace.

Assign this service principal the for the metastore associated with the Databricks workspace.

Within Databricks, . This completes your Databricks-based service principal setup.

Fill out the Client ID. This is a combination of letters, numbers, or symbols, used as a public identifier and is the (note that Azure Databricks uses the Azure SP Client ID; it will be identical).

Enter the Scope (string). The scope limits the operations and roles allowed in Databricks by the access token. See the for details about scopes.

Required permissions: When performing a manual setup, the Databricks user running the script must have the .

is enabled by default. .

Configure the by scrolling to Integrations Settings and find the Unity Catalog Audit Sync Schedule section.

Access Token: Enter a Databricks Personal Access Token. This is the access token for the Immuta service principal. This service principal must have the for the metastore associated with the Databricks workspace. If this token is configured to expire, update this field regularly for the integration to continue to function.

Follow for the Immuta service principal and assign this service principal the for the metastore associated with the Databricks workspace.

Fill out the Client ID. This is a combination of letters, numbers, or symbols, used as a public identifier and is the .

Enter the Scope (string). The scope limits the operations and roles allowed in Databricks by the access token. See the for details about scopes.

Follow to create a service principal within Azure and then populate to your Databricks account and workspace.

Assign this service principal the for the metastore associated with the Databricks workspace.

Within Databricks, . This completes your Databricks-based service principal setup.

Fill out the Client ID. This is a combination of letters, numbers, or symbols, used as a public identifier and is the (note that Azure Databricks uses the Azure SP Client ID; it will be identical).

Enter the Scope (string). The scope limits the operations and roles allowed in Databricks by the access token. See the for details about scopes.

.

. For Databricks Unity Catalog audit to work, Immuta must have, at minimum, the following access.

.

If the Databricks user doesn't exist in Databricks when you configure the integration, after they are created in Databricks. Otherwise, policies will not be enforced correctly for them in Databricks. Databricks user identities for Immuta users are automatically marked as invalid when the user is not found during policy application, preventing them from being affected by Databricks policy until their Immuta user identity is manually mapped to their Databricks identity.

.

Databricks Unity Catalog
Manage privileges in Unity Catalog
enabled
Databricks documentation
metastore created
Databricks documentation
metastore magic
migration
client ID displayed in Databricks when creating the client secret for the service principal
OAuth 2.0 documentation
Databricks documentation
create an OAuth client secret for the service principal
client ID displayed in Databricks when creating the client secret for the service principal
OAuth 2.0 documentation
client ID displayed in Databricks when creating the client secret for the service principal
OAuth 2.0 documentation
Databricks documentation
create an OAuth client secret for the service principal
client ID displayed in Databricks when creating the client secret for the service principal
OAuth 2.0 documentation
Enable a system schema where the <SCHEMA_NAME> is access
Grant the Immuta service principal access to the Databricks Unity Catalog system tables
Enable verbose audit logs in Unity Catalog
Register Databricks securables in Immuta
A service principal
Databricks permissions
Unity Catalog system tables enabled
Automatic setup
Manual setup
permissions listed above
Unity Catalog query audit
Ensure you have enabled system tables in Unity Catalog and provided the required access to the Immuta service principal
metastore privileges listed above
Databricks documentation to create a client secret
privileges listed above
privileges listed above
permissions listed above
Unity Catalog query audit
Ensure you have enabled system tables in Unity Catalog and provided the required access to the Immuta service principal
metastore privileges listed above
Databricks documentation to create a client secret
privileges listed above
privileges listed above
Map the external IDs from an external identity manager
Manually map the external IDs on the user's profile page
manually link their Immuta username to Databricks
audit frequency
audit frequency