> For the complete documentation index, see [llms.txt](https://documentation.immuta.com/saas/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.immuta.com/saas/configuration/integrations/databricks/databricks-unity-catalog/reference-guides/databricks-unity-catalog-integration-reference-guide.md).

# Databricks Unity Catalog Integration Reference Guide

{% hint style="info" %}
**Preview documentation for upcoming feature**

This page currently reflects the UI that users will see in future updates.
{% endhint %}

Using the Databricks Unity Catalog connection, you can register a Databricks Unity Catalog integration to register data sources, apply policies to securables, audit user queries, and use other supported Immuta features.

The sequence diagram below outlines the events that occur when an Immuta user wants to query a Databricks Unity Catalog table that has been registered as an Immuta data source.

<figure><img src="/files/CHxHuwXItImqx9dEZu0H" alt=""><figcaption></figcaption></figure>

Immuta’s integration with Unity Catalog allows you to enforce fine-grained access controls on Unity Catalog securable objects with Immuta policies. Instead of manually creating UDFs or granting access to each table in Databricks, you can author your policies in Immuta and have Immuta manage and orchestrate Unity Catalog access-control policies on your data in Databricks clusters or SQL warehouses.

## What does Immuta do in my Databricks environment?

### Registering a connection

Immuta utilizes [connections](/saas/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview/new-connections-reference-guide.md) to register and manage data from your entire Databricks Unity Catalog environment all at once. This approach simplifies data registration and allows Immuta to automatically monitor your Databricks Unity Catalog platform for changes. Data sources are then added or removed to reflect the current state of your data platform.

When a [connection is first registered](/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog.md), Immuta will ask your application admin to provide the following:

* The name of an **Immuta system catalog** in Databricks Unity Catalog that Immuta will control and create schemas in to contain policy definitions and user entitlements.
* A **set of Databricks credentials to a** [**service principal**](#user-content-fn-1)[^1] with the [required privileges](/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog.md#requirements) to manage all data protected by Immuta. These credentials will be used to initially register all your [Databricks objects](#supported-object-types) as data objects. Immuta will also continue to use those credentials for scheduled [object syncs](/saas/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview/new-connections-reference-guide.md#object-sync), [identification](/saas/configuration/tags/data-discovery.md), and to orchestrate policies in Databricks Unity Catalog.

The application admin will then run the setup script in Databricks Unity Catalog, which will create the following:

* The **new Immuta system catalog** to contain the `immuta_system` schema with internal Immuta data and the `immuta_policies_n` schema with the Immuta policy UDFs

After the setup and registration is complete, Immuta ingests and stores connection metadata in the Immuta metadata database. In the example below, the Immuta application administrator connects the database that contains `marketing-data`, `research-data`, and `cs-data` tables. Immuta registers these tables as data objects and stores the view metadata in the Immuta metadata database. Creating the connection does not impact any user's existing access in Databricks Unity Catalog until policy sync is enabled for those tables.

<figure><img src="/files/WodrVNKVbPkDscWqOw60" alt=""><figcaption></figcaption></figure>

Beyond making the registration of your data more intuitive, connections provide more control. Instead of performing operations on individual schemas or tables, you can perform operations (such as object sync) at the connection level.

See the [Connections reference guide](/saas/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview/new-connections-reference-guide.md) for details about connections and how to manage them. To configure your Databricks Unity Catalog integration and register data, see the [Register a Databricks Unity Catalog connection guide](/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog.md).

### Applying policies

When policies require changes to be pushed to Unity Catalog, Immuta updates the internal tables in the `immuta_system` schema with the updated policy information. If necessary, new UDFs are pushed to replace any out-of-date policies in the `immuta_policies_n` schemas and any row filters or column masks are updated to point at the new policies. Many of these operations require compute on the configured Databricks cluster or SQL warehouse, so compute must be available for these policies to succeed.

<figure><img src="/files/Qfn7WlD8FcMBsE5NeMz4" alt=""><figcaption></figcaption></figure>

#### Subscription policies

Immuta enforces read and write [subscription policies](/saas/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-policies.md) on Databricks Unity Catalog data sources by granting access to the appropriate securables in Databricks.

Once the connection is registered, this is what happens when a user applies a subscription policy to a Databricks Unity Catalog data source:

1. Immuta determines which users should have access based on the subscription policy.
2. If an appropriate Immuta-managed Databricks group does not already exist, Immuta creates one at the Databricks account level.
3. Immuta grants the required Unity Catalog privileges on the catalog, schema, or table to that Immuta-managed group using the Unity Catalog API.
4. Immuta adds the entitled users to the group.
5. Users who are no longer entitled to access the data source are removed from the group, causing them to lose access to the securable object.

Immuta-managed groups are created at the account level and not assigned to any particular workspace.

#### Group naming convention

When Immuta creates Databricks Unity Catalog groups to enforce access controls, the group naming convention differs slightly based on whether access was granted by an automatic subscription policy or the [group was manually subscribed](#user-content-fn-2)[^2] to the data source.

* **Automatic subscription of a user or Immuta group**: The group name comprises the Immuta external ID, the `connectionKey`, and the policy hash.
  * Naming convention: `IMMUTA_<Immuta external ID>_<connectionKey>_<policy hash>`
  * Example: `IMMUTA_123456789_UnityCatalogConnection_ce86d2f5f86471f66fcd08741d0eb0f4447a6e73a005094c9da07753a8d630f7`
* **Manual subscription of an Immuta group**: The group name comprises the Immuta external ID, the `connectionkey`, and the Immuta group name.
  * Naming convention: `IMMUTA_<Immuta external ID>_<connectionKey>_MANUAL_<group name>`
  * Example: `IMMUTA_123456789_UnityCatalogConnection_MANUAL_Research`

#### Data policies

You can author [data policies](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md) in Immuta to enforce fine-grained access controls on Databricks Unity Catalog data sources. For more details abour how enforcement works, see the [Protecting data page](/saas/configuration/integrations/databricks/databricks-unity-catalog/reference-guides/databricks-unity-catalog-integration-reference-guide/protecting-data.md).

When a data policy is applied to a Databricks Unity Catalog data source in Immuta,

1. A data owner, data governor, or administrator [creates or changes a policy](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/how-to-guides/data-policy-tutorial.md), or a [user's attributes change](/saas/configuration/people/users-index/how-to-guides/managing-attribute-and-group.md) in Immuta.
2. Immuta generates and applies SQL user-defined functions (UDFs) in Databricks to enforce the configured policy.
   * **Row-level security** is enforced by applying SQL UDFs that filter rows returned to the querying user.
   * **Column-level security** is enforced by applying column-mask SQL UDFs to protected columns.
3. A Databricks user who is subscribed to the data source in Immuta queries the corresponding table directly in Databricks and the SQL UDFs are evaluated during query execution, ensuring that only authorized rows and unmasked column values are returned.

See the integration support matrix on the [Data policis reference guide](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md#data-policy-support-matrix) for a list of supported data policy types for Databricks Unity Catalog.

#### Databricks Unity Catalog privileges granted by Immuta

See the [Subscription policy access types page](/saas/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-access-types.md#granting-databricks-unity-catalog-privileges) for details about the Databricks Unity Catalog privileges granted to users when they are subscribed to a data source protected by a subscription policy.

### Required Databricks Unity Catalog privileges

The privileges the Databricks Unity Catalog integration requires align to the least privilege security principle. The table below describes each privilege required in Databricks Unity Catalog for the [setup user](#user-content-fn-3)[^3] and the [Immuta service principal](#user-content-fn-4)[^4].

| Databricks Unity Catalog privilege                                                                                                                                                                                                                                                                                                                                                                                                                        | User requiring the privilege | Explanation                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Account admin                                                                                                                                                                                                                                                                                                                                                                                                                                             | Setup user                   | This privilege allows the setup user to grant the Immuta service principal the necessary permissions to orchestrate Unity Catalog access controls and maintain state between Immuta and Databricks Unity Catalog.                                                                                                                                                                                                                                                     |
| `CREATE CATALOG` on the Unity Catalog metastore                                                                                                                                                                                                                                                                                                                                                                                                           | Setup user                   | This privilege allows the setup user to create an Immuta-owned catalog and tables.                                                                                                                                                                                                                                                                                                                                                                                    |
| Metastore admin                                                                                                                                                                                                                                                                                                                                                                                                                                           | Setup user                   | This privilege is required only if enabling query audit, which requires granting access to system tables to the Immuta service principal. To grant access, a user that is both a metastore admin and an account admin must grant `USE` and `SELECT` permissions on the system schemas to the service principal. See [Manage privileges in Unity Catalog](https://docs.databricks.com/en/data-governance/unity-catalog/manage-privileges/index.html) for more details. |
| Workspace admin                                                                                                                                                                                                                                                                                                                                                                                                                                           | Immuta service principal     | This privilege allows Immuta to create and manage Databricks groups to enforce access controls. Immuta-managed groups are created at the account level and not assigned to any particular workspace.                                                                                                                                                                                                                                                                  |
| <ul><li><code>USE CATALOG</code> and <code>MANAGE</code> on all catalogs containing securables registered as Immuta data sources</li><li><code>USE SCHEMA</code> on all schemas containing securables registered as Immuta data sources</li></ul>                                                                                                                                                                                                         | Immuta service principal     | These privileges allow the service principal to apply row filters and column masks on the securable.                                                                                                                                                                                                                                                                                                                                                                  |
| `MODIFY` and `SELECT` on all securables registered as Immuta data sources                                                                                                                                                                                                                                                                                                                                                                                 | Immuta service principal     | These privileges allow the service principal to apply row filters and column masks on the securable. Additionally, they are required for [identification](/saas/configuration/tags/data-discovery.md) to run on the securable.                                                                                                                                                                                                                                        |
| `OWNER` on the Immuta catalog                                                                                                                                                                                                                                                                                                                                                                                                                             | Immuta service principal     | The Immuta service principal must own the catalog Immuta creates during setup that stores the Immuta policy information. The Immuta setup script grants ownership of this catalog to the Immuta service principal when you configure the integration.                                                                                                                                                                                                                 |
| <ul><li><code>USE CATALOG</code> on the <code>system</code> catalog</li><li><code>USE SCHEMA</code> on the <code>system.access</code> and <code>system.query</code> schemas</li><li><p><code>SELECT</code> on the following system tables:</p><ul><li><code>system.access.table\_lineage</code></li><li><code>system.access.column\_lineage</code></li><li><code>system.access.audit</code></li><li><code>system.query.history</code></li></ul></li></ul> | Immuta service principal     | These privileges allow Immuta to audit user queries in Databricks Unity Catalog.                                                                                                                                                                                                                                                                                                                                                                                      |
| <ul><li><code>USE CATALOG</code> on the <code>system</code> catalog</li><li><code>USE SCHEMA</code> on the <code>system.access</code> schema</li><li><p><code>SELECT</code> on the following system table:</p><ul><li><code>system.access.audit</code></li></ul></li></ul>                                                                                                                                                                                | Immuta service principal     | These privileges allow Immuta to ingest and apply Databricks Unity Catalog tags to Immuta data sources.                                                                                                                                                                                                                                                                                                                                                               |

### Maintaining state with Databricks Unity Catalog

The following user actions spur various processes in the Databricks Unity Catalog integration so that Immuta data remains synchronous with data in Databricks Unity Catalog:

* **Data source created**: Immuta registers data source metadata and stores that metadata in the Immuta metadata database.
* **Data source deleted**: Immuta deletes the data source metadata from the metadata database and removes subscription policies from that table.
* **User account is mapped to Immuta**: When a user account is mapped to Immuta, their metadata is stored in the metadata database.
* **Subscription policy created, updated, or deleted:** Immuta provisions access in these scenarios by creating and managing Databricks groups and granting or revoking users from those groups.
  * **User subscribed to a data source by an automatic subscription policy**: Immuta creates a group for the affected users (if a group for them does not already exist) and grants Databricks privileges on the data object to that group.
  * **Automatic subscription policy applied to or updated on a data source**: Immuta calculates the users and data sources affected by the policy change and grants or revokes privileges on the Databricks Unity Catalog object by adding or removing users from the Immuta-created group in Databricks.
  * **Subscription policy deleted**: Immuta calculates the users and Databricks groups affected by the policy and revokes privileges on the Databricks Unity Catalog object by removing users from the Immuta-created group in Databricks. If there are no longer any members of an Immuta-created group in Databricks, that group is deleted by Immuta.
* **User manually added to or removed from a data source:** Immuta provisions access in these scenarios by granting or revoking Databricks access to the securable object directly from the user.
  * **User added to a data source by a data owner**: Immuta grants Databricks privileges on the data object directly to that user.
  * **User removed from a data source**: If the user was previously subscribed to the data source through an automatic subscription policy, Immuta removes the user from the Immuta-created group in Databricks. If the user was previously manually added to the data source by a data owner, Immuta revokes the Databricks privileges from the user.

### Supported object types

| Object type                    | Subscription policy support | Data policy support                              | Request app support  |
| ------------------------------ | --------------------------- | ------------------------------------------------ | -------------------- |
| Table                          | :white\_check\_mark:        | :white\_check\_mark:                             | :white\_check\_mark: |
| View                           | :white\_check\_mark:        | :x:                                              | :white\_check\_mark: |
| Materialized view              | :white\_check\_mark:        | :white\_check\_mark:                             | :white\_check\_mark: |
| Metric views                   | :white\_check\_mark:        | :x:                                              | :white\_check\_mark: |
| Streaming table                | :white\_check\_mark:        | :white\_check\_mark:                             | :white\_check\_mark: |
| External table                 | :white\_check\_mark:        | :white\_check\_mark:                             | :white\_check\_mark: |
| Foreign table                  | :white\_check\_mark:        | :white\_check\_mark:                             | :white\_check\_mark: |
| Volumes (external and managed) | :white\_check\_mark:        | :x:                                              | :white\_check\_mark: |
| Models                         | :white\_check\_mark:        | :x:                                              | :white\_check\_mark: |
| Functions                      | :white\_check\_mark:        | :x:                                              | :white\_check\_mark: |
| Delta Shares                   | :white\_check\_mark:        | [Supported with caveats](#user-content-fn-5)[^5] | :white\_check\_mark: |

#### Policy support with `hive_metastore`

When enabling Unity Catalog support in Immuta, the catalog for all Databricks data sources will be updated to point at the default `hive_metastore` catalog. Internally, Databricks exposes this catalog as a proxy to the workspace-level Hive metastore that schemas and tables were kept in before Unity Catalog. Since this catalog is not a real Unity Catalog catalog, it does not support any Unity Catalog policies. Therefore, Immuta will ignore any data sources in the `hive_metastore` in any Databricks Unity Catalog integration, and policies will not be applied to tables there.

However, with [Databricks metastore magic](/saas/configuration/integrations/databricks.md#metastore-magic) you can use `hive_metastore` and enforce subscription and data policies with the [Databricks Spark integration](/saas/configuration/integrations/databricks/databricks-spark/how-to-guides/simplified.md).

## Supported policies

The Unity Catalog integration supports the following policy types:

* [Subscription policies](/saas/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-policies.md)
* [Select masking policies](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md#masking-policies)
  * Conditional masking
  * Constant
  * Custom masking
  * Hashing
  * Null (including on ARRAY, MAP, and STRUCT type columns)
  * Regex: You must use the global regex flag (`g`) when creating a regex masking policy in this integration. You cannot use the case insensitive regex flag (`i`) when creating a regex masking policy in this integration. See the [limitations section](#limitations-and-known-issues) for examples.
  * Rounding (date and numeric rounding)
* [Row-level policies](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md#row-level-security-policies)
  * Matching (only show rows where)
    * Custom WHERE
    * Never
    * Where user
    * Where value in column
  * Minimization
  * Time-based restrictions

## Security and compliance

### Authentication methods

The Databricks Unity Catalog integration supports the following authentication methods to configure the integration and create data sources:

* **Personal access token (PAT)**: This is the access token for the Immuta service principal. This service principal must have the metastore privileges listed in the [requirements](/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog.md#requirements) section for the metastore associated with the Databricks workspace. If this token is configured to expire, update this field regularly for the integration to continue to function.
* **OAuth machine-to-machine (M2M)**: Immuta uses the [Client Credentials Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/client-credentials-flow) to integrate with [Databricks OAuth machine-to-machine authentication](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html), which allows Immuta to authenticate with Databricks using a client secret. Once Databricks verifies the Immuta service principal’s identity using the client secret, Immuta is granted a temporary OAuth token to perform token-based authentication in subsequent requests. When that token expires (after one hour), Immuta requests a new temporary token. See the [Databricks OAuth machine-to-machine (M2M) authentication page](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) for more details.

#### User registration and ID mapping <a href="#user-registration-and-id-mapping" id="user-registration-and-id-mapping"></a>

The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead. Each of the [supported IAM protocols](https://documentation.immuta.com/saas/~/changes/1034/releases/support-matrix#external-iams) includes a set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta.

For policies to impact the right users, the user account in Immuta must be mapped to the user account in Databricks Unity Catalog. You can ensure these accounts are mapped correctly in the following ways:

* [Automatically](/saas/configuration/people/users-index/how-to-guides/external-user-mapping.md#configure-external-user-id-mapping-on-app-settings-page): If usernames in Databricks Unity Catalog align with usernames in the external IAM and those accounts align with an IAM attribute, you can enter that IAM attribute on the app settings page to automatically map user IDs in Immuta to Databricks Unity Catalog.
* [Manually](/saas/configuration/people/users-index/how-to-guides/external-user-mapping.md#manually-configure-external-user-id-mapping-on-a-users-page): You can manually map user IDs for individual users.

For guidance on connecting your IAM to Immuta, see the [how-to guide for your protocol](/saas/configuration/people/section-contents.md#how-to-guides).

### Policy exemption group

The Databricks group configured as the policy exemption group in Immuta will be exempt from Immuta data policy enforcement. This account-level group is created and managed in Databricks, not in Immuta. This group does not need to be assigned to a Databricks workspace.

If you have service or system accounts that need to be exempt from masking and row-level policy enforcement, add them to an account-level group in Databricks and include this group name in the Databricks Unity Catalog configuration in Immuta. Then, group members will be excluded from having data policies applied to them when they query Immuta-protected tables in Databricks.

Typically, service or system accounts that perform the following actions are added to an exemption group in Databricks:

* Automated queries
* ETL
* Report generation

If you have multiple groups that must be exempt from data policies, add each group to a single group in Databricks that you then set as the policy exemption group in Immuta.

The service principal used to register data sources in Immuta will be automatically added to the exemption group for the Databricks securables it registers. Consequently, accounts added to the exemption group and used to register data sources in Immuta should be limited to service accounts.

### Tag ingestion

{% hint style="info" %}
**Private preview**: This feature is only available to select accounts. Contact your Immuta representative to enable this feature.
{% endhint %}

You can enable tag ingestion to allow Immuta to ingest Databricks Unity Catalog table and column tags so that you can use them in Immuta policies to enforce access controls. When you enable this feature, Immuta uses the credentials and connection information from the Databricks Unity Catalog integration to pull tags from Databricks and apply them to data sources as they are registered in Immuta. If Databricks data sources preexist the Databricks Unity Catalog tag ingestion enablement, those data sources will automatically sync to the catalog and tags will apply.

Immuta checks for changes to tags in Databricks and syncs Immuta data sources to those changes every hour by default. Immuta's tag ingestion process has a delta logic in order to establish all resources that have had a tag or description change inside Databricks Unity Catalog within a given timeframe to reduce excessive processing time and reduce compute cost.

{% hint style="info" %}
**Access requirements for Databricks Unity Catalog tag ingestion (delta logic)**

Since the delta logic leverages the `system.access.audit` table in Databricks, Immuta must have, at minimum, the following access:

* `USE CATALOG` on the `system` catalog
* `USE SCHEMA` on the `system.access` schema
* `SELECT` on the following system table:
  * `system.access.audit`

Note that without these permissions, Immuta will not be able to process any tag changes post the initial onboarding of data sources.
{% endhint %}

Once external tags are applied to Databricks data sources, those tags can be used to create [subscription](/saas/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-policies.md) and [data policies](/saas/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md).

To enable Databricks Unity Catalog tag ingestion, see the [Register a Databricks Unity Catalog connection page](/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog.md#opt-to-enable-databricks-unity-catalog-tag-ingestion).

#### Syncing tag changes

After making changes to tags in Databricks, you can [manually sync the catalog](/saas/configuration/tags/catalogs/configure.md#manually-sync-external-catalog-tags) so that the changes immediately apply to the data sources in Immuta. Otherwise, tag changes will automatically sync within a one hour timeframe. Please note that you may see this timeframe being exceeded in cases where Immuta has to process a lot of tag changes.

When syncing data sources to Databricks Unity Catalog tags, Immuta pulls the following information:

* **Table tags**: These tags apply to the table and appear on the data source details tab. Databricks tags' key and value pairs are reflected in Immuta as a hierarchy with each level separated by a `.` delimiter. For example, the Databricks Unity Catalog tag `Location: US` would be represented as `Location.US` in Immuta.
* **Column tags**: These tags are applied to data source columns and appear on the columns listed for the data source. Databricks tags' key and value pairs are reflected in Immuta as a hierarchy with each level separated by a `.` delimiter. For example, the Databricks Unity Catalog tag `Location: US` would be represented as `Location.US` in Immuta.
* **Table comments field**: This content appears as the data source description on the data source details tab.
* **Column comments field**: This content appears as column descriptions on the columns tab.

#### Limitations

* Only tags that apply to Databricks data sources in Immuta are available to build policies in Immuta. Immuta will not pull tags in from Databricks Unity Catalog unless those tags apply to registered data sources.
* Cost implications: Tag ingestion in Databricks Unity Catalog requires compute resources. Therefore, having many Databricks data sources or frequently manually syncing data sources to Databricks Unity Catalog may incur additional costs.
* Databricks Unity Catalog tag ingestion only supports tenants with fewer than 10,000 data sources registered.

## Limitations and known issues

* Direct group memberships: A principal can be a member of up to 1,500 groups.
* Unity Catalog privileges per object: Databricks limits privileges to 4,000 privileges for parent objects and 1,000 privileges for non-parent objects.
* Groups per account: Databricks has a limit of 250,000 groups for Databricks customers using Account SCIM 2.1. All other Databricks environments are subject to a 5,000 group limit.
* Row access policies with more than 1023 columns are unsupported. This is an underlying limitation of UDFs in Databricks. Immuta will only create row access policies with the minimum number of referenced columns. This limit will therefore apply to the number of columns referenced in the policy and not the total number in the table.
* If you disable table grants, Immuta revokes the grants. Therefore, if users had access to a table before enabling Immuta, they’ll lose access.
* If multiple Immuta tenants are connected to your Databricks environment, you must create a separate Immuta catalog for each of those tenants during configuration. Having multiple Immuta tenants use the same Immuta catalog causes failures in policy enforcement.
* You must use the global regex flag (`g`) when creating a regex masking policy in this integration, and you cannot use the case insensitive regex flag (`i`) when creating a regex masking policy in this integration. See the examples below for guidance:
  * regex with a global flag (supported): `/^ssn|social ?security$/g`
  * regex without a global flag (unsupported): `/^ssn|social ?security$/`
  * regex with a case insensitive flag (unsupported): `/^ssn|social ?security$/gi`
  * regex without a case insensitive flag (supported): `/^ssn|social ?security$/g`

### Azure Databricks Unity Catalog limitation

If a registered data source is owned by a Databricks group at the table level, then the Unity Catalog integration cannot apply data masking policies to that table in Unity Catalog.

Therefore, set all table-level ownership on your Unity Catalog data sources to an individual user or service principal instead of a Databricks group. Catalogs and schemas can still be owned by a Databricks group, as ownership at that level doesn't interfere with the integration.

### Feature limitations

The following features are currently unsupported:

* Immuta project workspaces
* Multiple IAMs on a single cluster
* Row filters and column masking policies on the following object types:
  * Functions
  * Models
  * Views
  * Volumes
* Mixing masking policies on the same column
* R and Scala cluster support
* Scratch paths
* User impersonation
* Policy enforcement on raw Spark reads
* Python UDFs for advanced masking functions
* Direct file-to-SQL reads
* Data policies (except for masking with NULL) on ARRAY, MAP, or STRUCT type columns
* Shallow clones

[^1]: Immuta only manages grants for securables and users who are registered in Immuta. Therefore, to avoid having this service principal's access to data revoked in Databricks, do not register this service principal as an Immuta user.

[^2]: If a user is manually subscribed to a data source instead of a group, Immuta grants Databricks privileges on the data object directly to that user instead of creating a group.

[^3]: The user configuring the Databricks Unity Catalog integration.

[^4]: This service principal is used by Immuta to manage the Databricks Unity Catalog integration once it has been set up.

[^5]: Immuta supports data policies for shares that contain external, managed, and foreign tables. Data policies are not supported for shares that contain streaming tables or materialized views, which is a Unity Catalog limitation. See the [Databricks documentation](https://docs.databricks.com/aws/en/delta-sharing/create-share) for more details.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://documentation.immuta.com/saas/configuration/integrations/databricks/databricks-unity-catalog/reference-guides/databricks-unity-catalog-integration-reference-guide.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
