Immuta integrates with your data platforms so you can register your data and effectively manage access controls on that data.

This section includes guidance for connecting your data platform and keeping it synced with Immuta.

Integrations overview

This reference guide outlines the features, policies, and audit capabilities of each data platform Immuta supports.

Integrations

The guides in these sections include information about how to connect your data platform to Immuta:

• Amazon Redshift

• Amazon S3

• AWS Lake Formation

• Azure Synapse Analytics

• Databricks: This section includes guides for Databricks Lakebase, Databricks Spark, and Databricks Unity Catalog integrations.

• Google BigQuery

• MariaDB

• Oracle

• PostgreSQL

• Snowflake

• SQL Server

• Starburst (Trino)

• Teradata

Queries Immuta runs in remote platforms

This reference guide outlines the actions and features that trigger Immuta queries in your remote platform that may incur cost.

Connect your data

Immuta integrates with your data platforms so you can register your data and effectively manage access controls on that data. This section includes concept, reference, and how-to guides for registering and managing data sources.

Immuta gives everyone fast, governed access to data with the built-in controls, collaboration workflows, automated provisioning, and continuous monitoring you need to keep risk low and compliance high.

Configure Immuta

Explore Immuta

Immuta offers two integrations for Amazon Redshift:

• : In this integration, Immuta uses to configure the integration and register data objects in a single step. Once data is registered, Immuta can enforce subscription policies on that data.

• : In this integration, Immuta generates policy-enforced views in your configured Redshift schema for tables registered as Immuta data sources. The integration is configured separately from data source registration.

In this integration, Immuta generates policy-enforced views in your configured Redshift schema for tables registered as Immuta data sources.

This guide outlines how to integrate Amazon Redshift with Immuta.

How-to guides

Reference guides

• : This guide describes the design of the integration and policy enforcement.

• : This guide describes the requirements and supported features for the integration.

The Amazon Redshift viewless integration allows you to configure your integration and register data from Amazon Redshift in Immuta in a single step. Once data is registered, Immuta can enforce subscription policies on that data.

Getting started with the Amazon Redshift viewless integration

This getting started guide outlines how to connect Amazon Redshift to Immuta.

How-to guide

Register an Amazon Redshift connection

Reference guides

• Amazon Redshift viewless integration reference guide: This guide describes the design and components of the integration.

• Security and compliance: This guide provides an overview of the Immuta features that provide security for your users and that allow you to prove compliance and monitor for anomalies.

• Protecting data: This guide provides an overview of how to protect securables with Immuta policies.

• Accessing data: This guide provides an overview of how Amazon Redshift users access data registered in Immuta.

In the Lake Formation integration, Immuta orchestrates Lake Formation access controls on data registered in the Glue Data Catalog. Then, Immuta users who have been granted access to the Glue Data Catalog table or view can query it using one of these analytic engines:

• Amazon Athena

• Amazon EMR Spark

• Amazon Redshift Spectrum

Getting started with AWS Lake Formation

This getting started guide outlines how to integrate AWS Lake Formation with Immuta.

How-to guide

Register an AWS Lake Formation connection

Reference guides

• AWS Lake Formation: This guide describes the design and components of the integration.

• Security and compliance: This guide provides an overview of the Immuta features that provide security for your users and that allow you to prove compliance and monitor for anomalies.

• Protecting data: This guide provides an overview of how to protect AWS securables with Immuta policies.

• Accessing data: This guide provides an overview of how AWS users access data registered in Immuta.

This integration allows you to manage and access data in your Databricks account across all of your workspaces. With Immuta’s Databricks Unity Catalog integration, you can write your policies in Immuta and have them enforced automatically by Databricks across data in your Unity Catalog metastore.

Getting started

This getting started guide outlines how to integrate Databricks Unity Catalog with Immuta.

How-to guides

• Databricks Unity Catalog configuration: Configure the Databricks Unity Catalog integration.

• Migrate to Databricks Unity Catalog: Migrate from the legacy Databricks Spark integrations to the Databricks Unity Catalog integration.

Reference guide

Databricks Unity Catalog integration reference guide: This guide describes the design and components of the integration.

In this integration, Immuta generates policy-enforced views in a schema in your configured Azure Synapse Analytics Dedicated SQL pool for tables registered as Immuta data sources.

Getting started

This guide outlines how to integrate Azure Synapse Analytics with Immuta.

How-to guide

Azure Synapse Analytics configuration: Configure the integration in Immuta.

Reference guide

Azure Synapse Analytics integration reference guide: This guide describes the design and components of the integration.

Prerequisites

This upgrade step is necessary if you meet both of the following criteria:

• You have the Snowflake low row access policy mode enabled in private preview.

• You have user impersonation enabled.

If you do not meet this criteria, follow the instructions on the configuration guide.

Upgrade to Snowflake low row access policy mode

To upgrade to the generally available version of the feature, disable your Snowflake integration on the app settings page and then re-enable it.

To migrate from the private preview version of table grants (available before September 2022) to the GA version, complete the steps below.

1. Navigate to the App Settings page.

2. Scroll to the Global Integrations Settings section.

3. Uncheck the Snowflake Table Grants checkbox to disable the feature.

4. Click Save. Wait for about 1 minute per 1000 users. This gives time for Immuta to drop all the previously created user roles.

5. Use the Enable Snowflake table grants tutorial to re-enable the feature.

The MariaDB integration allows you to register data from MariaDB in Immuta.

How-to guide

Register a MariaDB connection

Reference guide

MariaDB integration reference guide: This guide describes the design and components of the integration.

The Oracle integration allows you to register data from Oracle in Immuta.

How-to guide

Register an Oracle connection

Reference guide

Oracle integration reference guide: This guide describes the design and components of the integration.

This page outlines the configuration for setting up project UDFs, which allow users to set their current project in Immuta through Spark. For details about the specific functions available and how to use them, see the Use Project UDFs (Databricks) page.

1. Lower the web service cache timeout in Immuta:

   1. Click the App Settings icon and scroll to the HDFS Cache Settings section.

   2. Lower the Cache TTL of HDFS user names (ms) to 0.

   3. Click Save.

2. Raise the cache timeout on your Databricks cluster: In the Spark environment variables section, set the IMMUTA_CURRENT_PROJECT_CACHE_TIMEOUT_SECONDS and IMMUTA_PROJECT_CACHE_TIMEOUT_SECONDS to high values (like 10000).

   Note: These caches will be invalidated on cluster when a user calls immuta.set_current_project, so they can effectively be cached permanently on cluster to avoid periodically reaching out to the web service.

If a Databricks cluster needs to be manually updated to reflect changes in the Immuta init script or cluster policies, you can remove and set up your integration again to get the updated policies and init script.

1. Log in to Immuta as an Application Admin.

2. Click the App Settings icon in the navigation menu and scroll to the Integration Settings section.

3. Your existing Databricks Spark integration should be listed here; expand it and note the configuration values. Now select Remove to remove your integration.

4. Click Add Integration and select Databricks Integration to add a new integration.

5. Enter your Databricks Spark integration settings again as configured previously.

6. Click Add Integration to add the integration, and then select Configure Cluster Policies to set up the updated cluster policies and init script.

7. Select the cluster policies you wish to use for your Immuta-enabled Databricks clusters.

8. Automatically push cluster policies and the init script (recommended) or manually update your cluster policies.

   • Automatically push cluster policies

     1. Select Automatically Push Cluster Policies and enter your privileged Databricks access token. This token must have privileges to write to cluster policies.

     2. Select Apply Policies to push the cluster policies and init script again.

     3. Click Save and Confirm to deploy your changes.

   • Manually update cluster policies

     1. Download the init script and the new cluster policies to your local computer.

     2. Click Save and Confirm to save your changes in Immuta.

     3. Log in to your Databricks workspace with your administrator account to set up cluster policies.

     4. Get the path you will upload the init script (immuta_cluster_init_script_proxy.sh) to by opening one of the cluster policy .json files and looking for the defaultValue of the field init_scripts.0.dbfs.destination. This should be a DBFS path in the form of dbfs:/immuta-plugin/hostname/immuta_cluster_init_script_proxy.sh.

     5. Click Data in the left pane to upload your init script to DBFS to the path you found above.

     6. To find your existing cluster policies you need to update, click Compute in the left pane and select the Cluster policies tab.

     7. Edit each of these cluster policies that were configured before and overwrite the contents of the JSON with the new cluster policy JSON you downloaded.

9. Restart any Databricks clusters using these updated policies for the changes to take effect.

This page provides guidelines for troubleshooting issues with the Databricks Spark integration and resolving Py4J security and Databricks trusted library errors.

Debugging the integration

For easier debugging of the Databricks Spark integration, follow the recommendations below.

• Enable cluster init script logging:

  • In the cluster page in Databricks for the target cluster, navigate to Advanced Options -> Logging.

  • Change the Destination from NONE to DBFS and change the path to the desired output location. Note: The unique cluster ID will be added onto the end of the provided path.

• View the Spark UI on your target Databricks cluster: On the cluster page, click the Spark UI tab, which shows the Spark application UI for the cluster. If you encounter issues creating Databricks data sources in Immuta, you can also view the JDBC/ODBC Server portion of the Spark UI to see the result of queries that have been sent from Immuta to Databricks.

Using the validation and debugging notebook

The validation and debugging notebook is designed to be used by or under the guidance of an Immuta support professional. Reach out to your Immuta representative for assistance.

1. Import the notebook into a Databricks workspace by navigating to Home in your Databricks instance.

2. Click the arrow next to your name and select Import.

3. Once you have executed commands in the notebook and populated it with debugging information, export the notebook and its contents by opening the File menu, selecting Export, and then selecting DBC Archive.

Py4J security error

• Error Message: py4j.security.Py4JSecurityException: Constructor <> is not allowlisted

• Explanation: This error indicates you are being blocked by Py4J security rather than the Immuta Security Manager. Py4J security is strict and generally ends up blocking many ML libraries.

• Solution: Turn off Py4J security on the offending cluster by setting IMMUTA_SPARK_DATABRICKS_PY4J_STRICT_ENABLED=false in the environment variables section. Additionally, because there are limitations to the security mechanisms Immuta employs on-cluster when Py4J security is disabled, ensure that all users on the cluster have the same level of access to data, as users could theoretically see (policy-enforced) data that other users have queried.

Databricks trusted library errors

Check the driver logs for details. Some possible causes of failure include

• One of the Immuta-configured trusted library URIs does not point to a Databricks library. Check that you have configured the correct URI for the Databricks library.

• For trusted Maven artifacts, the URI must follow this format: maven:/group.id:artifact-id:version.

• Databricks failed to install a library. Any Databricks library installation errors will appear in the Databricks UI under the Libraries tab.

Once data is registered through the PostgreSQL connection, you will access your data through your PostgreSQL client as you normally would. If you are subscribed to the data source, Immuta grants you access to the data in PostgreSQL.

When you submit a query, the PostgreSQL client submits the SQL query to the PostgreSQL server, which then processes the query and determines what data your role is allowed to see. Then, the PostgreSQL server queries the database and returns the query results to the PostgreSQL client, which then returns policy-enforced data to you.

The diagram below illustrates how Immuta, the PostgreSQL server, and PostgreSQL client interact to access data.

Once data is registered through the Amazon Redshift connection, you will access your data through Amazon Redshift as you normally would. If you are subscribed to the data source, Immuta grants you access to the data in Amazon Redshift.

When you submit a query, the SQL client submits the query to Amazon Redshift, which then processes the query and determines what data your role is allowed to see. Then, Amazon Redshift queries the database and returns the query results to the SQL client, which then returns policy-enforced data to you.

The diagram below illustrates how Immuta, Amazon Redshift, and the SQL client interact when a user queries data registered in Immuta.

Querying data

Because subscription policies are managed through roles, you must be acting under the role Immuta creates for you (immuta_<username>) to get access to the data sources you are subscribed to.

The how-to guides linked on this page illustrate how to use AWS Lake Formation with Immuta. See the for information about the AWS Lake Formation integration.

Immuta offers several features to provide security for your users and to prove compliance and monitor for anomalies.

Security

Data processing and encryption

See the and the guides for information about transmission of policy decision data, encryption of data in transit and at rest, and encryption key management.

Authentication

Registering the connection

The Amazon Redshift connection supports username and password authentication to register a connection. The credentials provided must be for an account with the permissions listed in the .

Identity providers for user authentication

The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead.

Each of the supported identity providers includes a specific set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta.

See the for a list of supported providers and details.

See the for details about mapping user accounts to Immuta.

Auditing and compliance

Immuta provides governance reports so that data owners and governors can monitor users' access to data and detect anomalies in behavior.

Immuta governance reports allow users with the GOVERNANCE Immuta permission to use a natural language builder to instantly create reports that delineate user activity across Immuta. These reports can be based on various entity types, including users, groups, projects, data sources, purposes, policy types, or connection types.

See the page for a list of report types and guidance.

The Databricks Lakebase integration registers data from Databricks Lakebase in Immuta and enforces subscription policies on that data.

This getting started guide outlines how to connect Databricks Lakebase to Immuta.

How-to guide

Reference guides

• : This guide describes the design and components of the integration.

• : This guide provides an overview of the Immuta features that provide security for your users and that allow you to prove compliance and monitor for anomalies.

• : This guide provides an overview of how to protect securables with Immuta policies.

• : This guide provides an overview of how Databricks Lakebase users access data registered in Immuta.

This integration enforces policies on Databricks securables registered in the legacy Hive metastore. Once these securables are registered as Immuta data sources, users can query policy-enforced data on Databricks clusters.

The guides in this section outline how to integrate Databricks Spark with Immuta.

This getting started guide outlines how to integrate Databricks with Immuta.

How-to guides

• : Configure the Databricks Spark integration.

• : Manually update your cluster to reflect changes in the Immuta init script or cluster policies.

• : Register a Databricks library with Immuta as a trusted library to avoid Immuta security manager errors when using third-party libraries.

• : Raise the caching on-cluster and lower the cache timeouts for the Immuta web service to allow use of project UDFs in Spark jobs.

• : Run R and Scala spark-submit jobs on your Databricks cluster.

• : Access DBFS in Databricks for non-sensitive data.

• : Resolve errors in the Databricks Spark configuration.

Reference guides

• : This guide describes the design and components of the integration.

• : This guide provides an overview of the Immuta features that provide security for your users and Databricks clusters and that allow you to prove compliance and monitor for anomalies.

• : This guide provides an overview of registering Databricks securables and protecting them with Immuta policies.

• : This guide provides an overview of how Databricks users access data registered in Immuta.

In the Databricks Lakebase integration, Immuta administers PostgreSQL privileges on data registered in Immuta. Then, Immuta users who have been granted access to the tables can query them with policies enforced.

The sequence diagram below outlines the events that occur when an Immuta user who is subscribed to a data source queries it in PostgreSQL.

Registering a connection

Databricks Lakebase is configured and data is registered through , an Immuta feature that allows administrators to register data objects in a technology through a single connection to make data registration more scalable for your organization.

Once the Databricks Lakebase connection is registered, you can author subscription policies in Immuta to enforce access controls.

See the for more details about registering a connection.

Protecting data

After tables are registered in Immuta, you can author subscription policies in Immuta to enforce access controls.

When a policy is applied to a data source, users who meet the conditions of the policy will be automatically subscribed to the data source. Then, Immuta issues a SQL statement in PostgreSQL that grants the SELECT privilege to users on those tables.

Consider the following example that illustrates how Immuta enforces a subscription policy that only allows users in the analysts group to access to yellow-table. When this policy is authored and applied to the data source, Immuta issues a SQL statement in PostgreSQL that grants the SELECT privilege on yellow-table to users (registered in Immuta) that are part of the analysts group.

In the image above, the user in the analysts group accesses yellow-table , while the user who is a part of the research group is denied access. See the for guidance on applying a subscription policy to a data source. See the page details about the subscription policy types supported and PostgreSQL privileges Immuta grants on tables registered as Immuta data sources.

In the Amazon Redshift viewless integration, Immuta administers Amazon Redshift privileges on data registered in Immuta. Then, Immuta users who have been granted access to the data sources can query them.

The sequence diagram below outlines the events that occur when an Immuta user who is subscribed to a data source queries it in Amazon Redshift.

Registering a connection

The Amazon Redshift viewless integration is configured and data is registered through , an Immuta feature that allows administrators to register data objects in a technology through a single connection to make data registration more scalable for your organization.

Once the Amazon Redshift connection is registered, you can author subscription policies in Immuta to enforce access controls.

See the for more details about registering a connection.

Protecting data

After data is registered in Immuta, you can author subscription policies in Immuta to enforce access controls.

When a policy is applied to a data source, users who meet the conditions of the policy will be automatically subscribed to the data source. Immuta creates roles for those users (if an Immuta-generated role for them does not already exist) and grants Amazon Redshift privileges to that role.

Consider the following example that illustrates how Immuta enforces a subscription policy that only allows users in the analysts group to access the yellow-table. When this policy is authored and applied to the data source, Immuta issues a SQL statement in Amazon Redshift that grants the SELECT privilege on yellow-table to users registered in Immuta that are part of the analysts group.

In the image above, the user in the analysts group accesses yellow-table , while the user who is a part of the research group is denied access. See the for guidance on applying a subscription policy to a data source. See the page for details about the subscription policy access types supported and Amazon Redshift privileges Immuta grants on views registered as Immuta data sources.

Immuta offers several features to provide security for your users and to prove compliance and monitor for anomalies.

Security

Data processing and encryption

See the and the guides for more information about transmission of policy decision data, encryption of data in transit and at rest, and encryption key management.

Authentication

Registering the connection

The Lake Formation integration supports the following authentication methods to register a connection:

• Access using AWS IAM role (recommended): Immuta will assume this role when interacting with the AWS API. This option allows you to provide Immuta with an IAM role from your AWS account that is granted a trust relationship with Immuta's IAM role. Immuta will assume this IAM role from Immuta's AWS account in order to perform any operations in your AWS account.

• Access using access key and secret access key: These credentials are used temporarily by Immuta to register the connection.

Identity providers for user authentication

The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead.

Each of the supported identity providers includes a specific set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta.

See the for a list of supported providers and details.

See the for details about user provisioning and mapping AWS user accounts to Immuta.

Auditing and compliance

Immuta provides governance reports so that data owners and governors can monitor users' access to data and detect anomalies in behavior.

Immuta governance reports allow users with the GOVERNANCE Immuta permission to use a natural language builder to instantly create reports that delineate user activity across Immuta. These reports can be based on various entity types, including users, groups, projects, data sources, purposes, policy types, or connection types.

See the page for a list of report types and guidance.

In the PostgreSQL integration, Immuta registers data from PostgreSQL and enforces subscription policies on that data.

This getting started guide outlines how to integrate PostgreSQL with Immuta.

How-to guide

Reference guides

• : This guide describes the design and components of the integration.

• : This guide provides an overview of the Immuta features that provide security for your users and that allow you to prove compliance and monitor for anomalies.

• : This guide provides an overview of how to protect securables with Immuta policies.

• : This guide provides an overview of how PostgreSQL users access data registered in Immuta.

The Databricks Spark integration is one of two integrations Immuta offers for Databricks.

In this integration, Immuta installs an Immuta-maintained Spark plugin on your Databricks cluster. When a user queries data that has been registered in Immuta as a data source, the plugin injects policy logic into the plan Spark builds so that the results returned to the user only include data that specific user should see.

The reference guides in this section are written for Databricks administrators who are responsible for setting up the integration, securing Databricks clusters, and setting up users:

• : This guide includes information about what Immuta creates in your Databricks environment and securing your Databricks clusters.

• : Consult this guide for information about customizing the Databricks Spark integration settings.

• : Consult this guide for information about connecting data users and setting up user impersonation.

• : This guide provides a list of Spark environment variables used to configure the integration.

• : This guide describes ephemeral overrides and how to configure them to reduce the risk that a user has overrides set to a cluster (or multiple clusters) that aren't currently up.

Immuta offers several features to provide security for your users and to prove compliance and monitor for anomalies.

Security

Data processing and encryption

See the and the guides for information about transmission of policy decision data, encryption of data in transit and at rest, and encryption key management.

Authentication

Registering the connection

The Databricks Lakebase connection supports OAuth machine-to-machine (M2M) authentication to register a connection.

The Databricks Lakebase connection authenticates as a Databricks identity and generates an OAuth token. Immuta then uses that token as a password when connecting to PostgreSQL. To enable secure, automated machine-to machine access to the database instance, the connection must obtain an OAuth token using a Databricks service principal. See the for more details.

Identity providers for user authentication

The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead.

Each of the supported identity providers includes a specific set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta.

See the for a list of supported providers and details.

See the for details about user user provisioning and mapping user accounts to Immuta.

Auditing and compliance

Immuta provides governance reports so that data owners and governors can monitor users' access to data and detect anomalies in behavior.

Immuta governance reports allow users with the GOVERNANCE Immuta permission to use a natural language builder to instantly create reports that delineate user activity across Immuta. These reports can be based on various entity types, including users, groups, projects, data sources, purposes, policy types, or connection types.

See the page for a list of report types and guidance.

1. Navigate to the App Settings page.

2. Scroll to the Global Integrations Settings section.

3. Ensure the Snowflake Table Grants checkbox is checked. It is enabled by default.

4. Opt to change the Role Prefix. Snowflake table grants creates a new Snowflake role for each Immuta user. To ensure these Snowflake role names do not collide with existing Snowflake roles, each Snowflake role created for Snowflake table grants requires a common prefix. When using multiple Immuta accounts within a single Snowflake account, the Snowflake table grants role prefix should be unique for each Immuta account. The prefix must adhere to and be less than 50 characters. Once the configuration is saved, the prefix cannot be modified; however, the Snowflake table grants feature can be disabled and re-enabled to change the prefix.

5. Finish configuring your integration by following one of these guidelines:

   • New Snowflake integration: Set up a new Snowflake integration by following the .

   • Existing Snowflake integration (automatic setup): You will be prompted to enter connection information for a Snowflake user. Immuta will execute the migration to Snowflake table grants using a connection established with this Snowflake user. The Snowflake user you provide here must have Snowflake privileges to run these .

   • Existing Snowflake integration (manual setup): Immuta will display a link to a migration script you must run in Snowflake and a link to a rollback script for use in the event of a failed migration. Important: Execute the migration script in Snowflake before clicking Save on the app settings page.

Immuta is compatible with . Using both Immuta and Snowflake, organizations can share the policy-protected data of their Snowflake database with other Snowflake accounts with Immuta policies enforced in real time.

Prerequisites:

Create Immuta Policies to Protect the Data

Required Permission: Immuta: GOVERNANCE

to fit your organization's compliance requirements.

It's important to understand that subscription policies are not relevant to Snowflake data shares, because the act of sharing the data is the subscription policy. Data policies can be enforced on the consuming account from the producer account on a share following these instructions.

Register the Snowflake Data Consumer with Immuta

Required Permission: Immuta: USER_ADMIN

To register the Snowflake data consumer in Immuta,

1. .

2. to match the account ID for the data consumer. This value is the output on the data consumer side when SELECT CURRENT_ACCOUNT() is run in Snowflake.

3. for your organization's policies.

4. .

Create the Snowflake Data Share

Required Permission: Snowflake ACCOUNTADMIN

To share the policy-protected data source,

1. of the Snowflake table that has been registered in Immuta.

2. Grant reference usage on the Immuta database to the share you created:

   Replace the content in angle brackets above with the name of your Immuta database and Snowflake data share.

Once data is registered through the Databricks Lakebase connection, you will access your data as you normally would. If you are subscribed to the data source, Immuta grants you access to the data through a PostgreSQL role.

When you submit a query (through PostgreSQL or through the Databricks Lakebase instance), the PostgreSQL client submits the SQL query to the PostgreSQL server, which then processes the query and determines what data your role is allowed to see. Then, the PostgreSQL server queries the database and returns the query results to the PostgreSQL client, which then returns policy-enforced data to you.

The diagram below illustrates how Immuta, the PostgreSQL server, and PostgreSQL client interact to access data.

Allow Immuta to create secure views of your external tables through one of these methods:

• Configure the integration with an existing database that contains the external tables: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift

• Configure the integration by creating a new immuta database and re-create all of your external tables in that database.

For an overview of the integration, see the Redshift overview documentation.

Requirements

• A Redshift cluster with an AWS row-level security patch applied. Contact Immuta for guidance.

• An AWS IAM role for Redshift that is associated with your Redshift cluster.

• The enable_case_sensitive_identifier parameter must be set to false (default setting) for your Redshift cluster.

• The Redshift role used to run the Immuta bootstrap script must have the following privileges when configuring the integration to

  • Use an existing database:

    • ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.

    • CREATE USER

    • GRANT TEMP ON DATABASE

  • Create a new database:

    • CREATE DATABASE

    • CREATE USER

    • GRANT TEMP ON DATABASE

    • REVOKE ALL PRIVILEGES ON DATABASE

• A Redshift database that contains an external schema and external tables.

Use an existing database

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab.

3. Click the +Add Integration button and select Redshift from the dropdown menu.

4. Complete the Host and Port fields.

5. Enter the name of the database you created the external schema in as the Immuta Database. This database will store all secure schemas and Immuta-created views.

6. Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.

7. Select Manual and download both of the bootstrap scripts from the Setup section. The specified role used to run the bootstrap needs to have the following privileges:

   • ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.

   • CREATE USER

   • GRANT TEMP ON DATABASE

8. Run the bootstrap script (Immuta database) in the Redshift database that contains the external schema.

9. Choose your authentication method, and enter the credentials from the bootstrap script for the Immuta_System_Account.

10. Click Save.

Register data

Register Redshift data in Immuta.

Create a new Immuta database

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab.

3. Click the +Add Integration button and select Redshift from the dropdown menu.

4. Complete the Host and Port fields.

5. Enter an Immuta Database. This is a new database where all secure schemas and Immuta created views will be stored.

6. Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.

7. Select Manual and download both of the bootstrap scripts from the Setup section. The specified role used to run the bootstrap needs to have the following privileges:

   • ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.

   • CREATE DATABASE

   • CREATE USER

   • GRANT TEMP ON DATABASE

8. Run the bootstrap script (initial database) in the Redshift initial database.

9. Run the bootstrap script (Immuta database) in the new Immuta Database in Redshift.

10. Choose your authentication method, and enter the credentials from the bootstrap script for the Immuta_System_Account.

11. Click Save.

Then, add your external tables to the Immuta database.

Register data

Register Redshift data in Immuta.

The how-to guides linked on this page illustrate how to integrate Azure Synapse Analytics with Immuta. See the reference guide for information about the Azure Synapse Analytics integration.

Requirement: A running Dedicated SQL pool

The how-to guides linked on this page illustrate how to integrate Amazon Redshift with Immuta. See the reference guide for information about the Amazon Redshift view-based integration.

Requirement: Redshift cluster with an RA3 node is required for the multi-database integration. For other instance types, you may configure a single-database integration using one of the Redshift Spectrum options.

The how-to guides linked on this page illustrate how to use Amazon Redshift with Immuta. See the reference guide for information about the Amazon Redshift viewless integration.

When you enable Unity Catalog, Immuta automatically migrates your existing Databricks data sources in Immuta to reference the legacy hive_metastore catalog to account for Unity Catalog's three-level hierarchy. New data sources will reference the Unity Catalog metastore you create and attach to your Databricks workspace.

Because the hive_metastore catalog is not managed by Unity Catalog, existing data sources in the hive_metastore cannot have Unity Catalog access controls applied to them. Data sources in the Hive Metastore must be managed by the Databricks Spark integration.

To allow Immuta to administer Unity Catalog access controls on that data, move the data to Unity Catalog and re-register those tables in Immuta by completing the steps below. If you don't move all data before configuring the integration, metastore magic will protect your existing data sources throughout the migration process.

1. Ensure that all Databricks clusters that have Immuta installed are stopped and the Immuta configuration is removed from the cluster. Immuta-specific cluster configuration is no longer needed with the Databricks Unity Catalog integration.

2. Move all data into Unity Catalog before configuring Immuta with Unity Catalog. Existing data sources will need to be re-created after they are moved to Unity Catalog and the Unity Catalog integration is configured.

3. Enable Unity Catalog.

In the context of the Databricks Spark integration, Immuta uses the term ephemeral to describe data sources where the associated compute resources can vary over time. This means that the compute bound to these data sources is not fixed and can change. All Databricks data sources in Immuta are ephemeral.

Ephemeral overrides are specific to each data source and user. They effectively bind cluster compute resources to a data source for a given user. Immuta uses these overrides to determine which cluster compute to use when connecting to Databricks for various maintenance operations.

The operations that use the ephemeral overrides include

• Visibility checks on the data source for a particular user. These checks assess how to apply row-level policies for specific users.

• Stats collection triggered by a specific user.

• Validating a custom WHERE clause policy against a data source. When owners or governors create custom WHERE clause policies, Immuta uses compute resources to validate the SQL in the policy. In this case, the ephemeral overrides for the user writing the policy are used to contact a cluster for SQL validation.

• High cardinality column detection. Certain advanced policy types (e.g., minimization) in Immuta require a high cardinality column, and that column is computed on data source creation. It can be recomputed on demand and, if so, will use the ephemeral overrides for the user requesting computation.

Triggering an ephemeral override request

An ephemeral override request can be triggered when a user queries the securable corresponding to a data source in a Databricks cluster with the Spark plug-in configured. The actual triggering of this request depends on the configuration settings.

Ephemeral overrides can also be set for a data source in the Immuta UI by navigating to a data source page, clicking on the data source actions button, and selecting Ephemeral overrides from the dropdown menu.

Ephemeral override requests made from a cluster for data sources and users where ephemeral overrides were set in the UI will not be successful.

If ephemeral overrides are never set (either through the user interface or the cluster configuration), the system will continue to use the connection details directly associated with the data source, which are set during data source registration.

Configuring overrides in Immuta-enabled clusters

Ephemeral overrides can be problematic in environments that have a dedicated cluster to handle maintenance activities, since ephemeral overrides can cause these operations to execute on a different cluster than the dedicated one.

To reduce the risk that a user has overrides set to a cluster (or multiple clusters) that aren't currently up, complete one of the following actions:

• Direct all clusters' HTTP paths for overrides to a cluster dedicated for metadata queries using the IMMUTA_EPHEMERAL_HOST_OVERRIDE_HTTPPATH Spark environment variable.

• Disable ephemeral overrides completely by setting the IMMTUA_EPHEMERAL_HOST_OVERRIDE Spark environment variable to false.

This page outlines how to enable access to DBFS in Databricks for non-sensitive data. Databricks administrators should place the desired configuration in the Spark environment variables.

DBFS FUSE mount

This Databricks feature mounts DBFS to the local cluster filesystem at /dbfs. Although disabled when using process isolation, this feature can safely be enabled if raw, unfiltered data is not stored in DBFS and all users on the cluster are authorized to see each other’s files. When enabled, the entirety of DBFS essentially becomes a scratch path where users can read and write files in /dfbs/path/to/my/file as though they were local files.

For example,

%sh echo "I'm creating a new file in DBFS" > /dbfs/my/newfile.txt

In Python,

%python
with open("/dbfs/my/newfile.txt", "w") as f:
  f.write("I'm creating a new file in DBFS")

Note: This solution also works in R and Scala.

Enable DBFS FUSE mount

To enable the DBFS FUSE mount, set this configuration in the Spark environment variables: IMMUTA_SPARK_DATABRICKS_DBFS_MOUNT_ENABLED=true.

Scala DBUtils (and %fs magic) with scratch paths

Scratch paths will work when performing arbitrary remote filesystem operations with fs magic or Scala dbutils.fs functions. For example,

%fs put -f s3://my-bucket/my/scratch/path/mynewfile.txt "I'm creating a new file in S3"
%scala dbutils.fs.put("s3://my-bucket/my/scratch/path/mynewfile.txt", "I'm creating a new file in S3")

Configure Scala DBUtils (and %fs magic) with scratch paths

To support %fs magic and Scala DBUtils with scratch paths, configure

       <property>
           <name>immuta.spark.databricks.scratch.paths</name>
           <value>s3://my-bucket/my/scratch/path</value>
       </property>

Configure DBUtils in Python

To use dbutils in Python, set this configuration: immuta.spark.databricks.py4j.strict.enabled=false.

Example workflow

This section illustrates the workflow for getting a file from a remote scratch path, editing it locally with Python, and writing it back to a remote scratch path.

%python
import os
import shutil

s3ScratchFile = "s3://some-bucket/path/to/scratch/file"
localScratchDir = os.environ.get("IMMUTA_LOCAL_SCRATCH_DIR")
localScratchFile = "{}/myfile.txt".format(localScratchDir)
localScratchFileCopy = "{}/myfile_copy.txt".format(localScratchDir)

1. Get the file from remote storage:

   Copy

   dbutils.fs.cp(s3ScratchFile, "file://{}".format(localScratchFile))

2. Make a copy if you want to explicitly edit localScratchFile, as it will be read-only and owned by root:

   Copy

   shutil.copy(localScratchFile, localScratchFileCopy)
   with open(localScratchFileCopy, "a") as f:
       f.write("Some appended file content")

3. Write the new file back to remote storage:

   Copy

   dbutils.fs.cp("file://{}".format(localScratchFileCopy), s3ScratchFile)

The how-to guides linked on this page illustrate how to integrate Snowflake with Immuta. See the reference guide for information about the Snowflake integration.

Requirements

• Snowflake enterprise edition

• Access to a Snowflake account that can create a Snowflake user

Immuta manages access to Snowflake tables by administering Snowflake row access policies and column masking policies on those tables, allowing users to query tables directly in Snowflake while dynamic policies are enforced.

Getting started

This getting started guide outlines how to integrate your Snowflake account with Immuta.

How-to guides

• Configure a Snowflake integration: Configure the Snowflake integration.

• Edit or remove an existing integration: Manage integration settings or delete your existing Snowflake integration.

• Integration settings:

  • Enable Snowflake table grants: Enable Snowflake table grants and configure the Snowflake role prefix.

  • Use Snowflake data sharing with Immuta: Use Snowflake data sharing with table grants or project workspaces.

  • Snowflake low row access policy mode: Enable Snowflake low row access policy mode.

  • Snowflake lineage tag propagation: Configure your Snowflake integration to automatically apply tags added to a Snowflake table to its descendant data source columns in Immuta.

Reference guides

• Snowflake integration reference guide: This reference guide describes the design and features of the Snowflake integration.

• Snowflake table grants: Snowflake table grants simplifies the management of privileges in Snowflake when using Immuta. Instead of manually granting users access to tables registered in Immuta, you allow Immuta to manage privileges on your Snowflake tables and views according to subscription policies. This guide describes the components of Snowflake table grants and how they are used in Immuta's Snowflake integration.

• Snowflake data sharing with Immuta: Organizations can share the policy-protected data of their Snowflake database with other Snowflake accounts with Immuta policies enforced in real time. This guide describes the components of using Immuta with Snowflake data shares.

• Snowflake low row access policy mode: The Snowflake low row access policy mode improves query performance in Immuta's Snowflake integration. To do so, this mode decreases the number of Snowflake row access policies Immuta creates and uses table grants to manage user access. This guide describes the design and requirements of this mode.

• Snowflake lineage tag propagation: Snowflake column lineage specifies how data flows from source tables or columns to the target tables in write operations. When Snowflake lineage tag propagation is enabled in Immuta, Immuta automatically applies tags added to a Snowflake table to its descendant data source columns in Immuta so you can build policies using those tags to restrict access to sensitive data.

• Warehouse sizing recommendations: Adjust the size and scale of clusters for your warehouse to manage workloads so that you can use Snowflake compute resources the most cost effectively.

Explanatory guide

Phased Snowflake onboarding: A phased onboarding approach to configuring the Snowflake integration ensures that your users will not be immediately affected by changes as you add data sources and policies. This guide describes the settings and requirements for implementing this phased approach.

The how-to guides linked on this page illustrate how to use Databricks Lakebase with Immuta. See the reference guide for information about the Databricks Lakebase integration.

Immuta offers several features to provide security for your users and to prove compliance and monitor for anomalies.

Security

Data processing and encryption

See the Data processing and the Encryption and masking practices guides for information about transmission of policy decision data, encryption of data in transit and at rest, and encryption key management.

Authentication

Registering the connection

The PostgreSQL integration supports the following authentication methods to register a connection:

• Amazon Aurora and Amazon RDS deployments

  • Access using AWS IAM role (recommended): Immuta will assume this IAM role from Immuta's AWS account when interacting with the AWS API to perform any operations in your AWS account. This option allows you to provide Immuta with an IAM role from your AWS account that is granted a trust relationship with Immuta's IAM role.

  • Access using access key and secret access key: These credentials are used temporarily by Immuta to register the connection. The access key ID and secret access key provided must be for an AWS account with the permissions listed in the Register a PostgreSQL connection guide.

• Neon and PostgreSQL deployments

  • Username and password: These credentials are used temporarily by Immuta to register the connection. The credentials provided must be for an account with the permissions listed in the Register a PostgreSQL connection guide.

Identity providers for user authentication

The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead.

Each of the supported identity providers includes a specific set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta.

See the Identity managers guide for a list of supported providers and details.

See the PostgreSQL integration reference guide for details about user provisioning and mapping user accounts to Immuta.

Auditing and compliance

Immuta provides governance reports so that data owners and governors can monitor users' access to data and detect anomalies in behavior.

Immuta governance reports allow users with the GOVERNANCE Immuta permission to use a natural language builder to instantly create reports that delineate user activity across Immuta. These reports can be based on various entity types, including users, groups, projects, data sources, purposes, policy types, or connection types.

See the Governance report types page for a list of report types and guidance.

When using Delta Lake, the API does not go through the normal Spark execution path. This means that Immuta's Spark extensions do not provide protection for the API. To solve this issue and ensure that Immuta has control over what a user can access, the Delta Lake API is blocked.

Spark SQL can be used instead to give the same functionality with all of Immuta's data protections.

Requests

Below is a table of the Delta Lake API with the Spark SQL that may be used instead.

See here for a complete list of the Delta SQL Commands.

Merging tables in workspaces

When a table is created in a project workspace, you can merge a different Immuta data source from that workspace into that table you created.

1. Create a table in the project workspace.

2. Create a temporary view of the Immuta data source you want to merge into that table.

3. Use that temporary view as the data source you add to the project workspace.

4. Run the following command:

   Copy

   MERGE INTO delta_native.target_native as target
   USING immuta_temp_view_data_source as source
   ON target.dr_number = source.dr_number
   WHEN MATCHED THEN
   UPDATE SET target.date_reported = source.date_reported

Immuta offers three integrations for Databricks:

• Databricks Unity Catalog integration: This integration supports working with database objects registered in Unity Catalog.

• Databricks Lakebase integration: This connection supports working with Lakebase Postgres database objects within Databricks Lakebase.

• Databricks Spark integration: This integration supports working with database objects registered in the legacy Hive metastore.

Which integration should you use?

To determine which integration you should use, consider which metastore you use:

• Legacy Hive metastore: Databricks recommends that you migrate all data from the legacy Hive metastore to Unity Catalog. However, when this migration is not possible, use the Databricks Spark integration to protect securables registered in the Hive metastore.

• Unity Catalog: To protect securables registered in the Unity Catalog metastore, use the Databricks Unity Catalog integration.

  • Databricks Lakebase: To register and protect fully managed PostgreSQL-compatible data objects, use the Databricks Lakebase integration.

• Legacy Hive metastore and Unity Catalog: If you need to work with database objects registered in both the legacy Hive metastore and in Unity Catalog, metastore magic allows you to use both integrations.

Metastore magic

Databricks metastore magic allows you to migrate your data from the Databricks legacy Hive metastore to the Unity Catalog metastore while protecting data and maintaining your current processes in a single Immuta instance.

Databricks metastore magic is for organizations who intend to use the Databricks Unity Catalog integration, but must still protect tables in the Hive metastore until they can migrate all of their data to Unity Catalog.

Requirement

Unity Catalog support is enabled in Immuta.

Databricks metastores and Immuta policy enforcement

Databricks has two built-in metastores that contain metadata about your tables, views, and storage credentials:

• Legacy Hive metastore: Created at the workspace level. This metastore contains metadata of the registered securables in that workspace available to query.

• Unity Catalog metastore: Created at the account level and is attached to one or more Databricks workspaces. This metastore contains metadata of the registered securables available to query. All clusters on that workspace use the configured metastore and all workspaces that are configured to use a single metastore share those securables.

Databricks allows you to use the legacy Hive metastore and the Unity Catalog metastore simultaneously. However, Unity Catalog does not support controls on the Hive metastore, so you must attach a Unity Catalog metastore to your workspace and move existing databases and tables to the attached Unity Catalog metastore to use the governance capabilities of Unity Catalog.

Immuta's Databricks Spark integration and Unity Catalog integration enforce access controls on the Hive and Unity Catalog metastores, respectively. However, because these metastores have two distinct security models, users were discouraged from using both in a single Immuta instance before metastore magic; the Databricks Spark integration and Unity Catalog integration were unaware of each other, so using both concurrently caused undefined behavior.

Databricks metastore magic solution

Metastore magic reconciles the distinct security models of the legacy Hive metastore and the Unity Catalog metastore, allowing you to use multiple metastores (specifically, the Hive metastore or AWS Glue Data Catalog alongside Unity Catalog metastores) within a Databricks workspace and single Immuta instance and keep policies enforced on all your tables as you migrate them. The diagram below shows Immuta enforcing policies on registered tables across workspaces.

In clusters A and D, Immuta enforces policies on data sources in each workspace's Hive metastore and in the Unity Catalog metastore shared by those workspaces. In clusters B, C, and E (which don't have Unity Catalog enabled in Databricks), Immuta enforces policies on data sources in the Hive metastores for each workspace.

Enforce policies as you migrate

With metastore magic, the Databricks Spark integration enforces policies only on data in the Hive metastore, while the Unity Catalog integration enforces policies on tables in the Unity Catalog metastore. The table below illustrates this policy enforcement.

To enforce plugin-based policies on Hive metastore tables and Unity Catalog native controls on Unity Catalog metastore tables, enable the Databricks Spark integration and the Databricks Unity Catalog integration. Note that some Immuta policies are not supported in the Databricks Unity Catalog integration. See the Databricks Unity Catalog integration reference guide for details.

Enforcing policies on Databricks SQL

Databricks SQL cannot run the Databricks Spark plugin to protect tables, so Hive metastore data sources will not be policy enforced in Databricks SQL.

To enforce policies on data sources in Databricks SQL, use Hive metastore table access controls to manually lock down Hive metastore data sources and the Databricks Unity Catalog integration to protect tables in the Unity Catalog metastore. Table access control is enabled by default on SQL warehouses, and any Databricks cluster without the Immuta plugin must have table access control enabled.

Once data is registered through the AWS Lake Formation connection, you will access your data in one of these AWS analytic engines as you normally would:

• Amazon Athena

• Amazon EMR Spark

• Amazon Redshift Spectrum

If you are subscribed to the data source, Immuta either directly grants you access to the resource through Lake Formation or generates and assigns a Lake Formation tag to that resource to grant you access. See the Protecting data page for details about how policies are enforced.

When you submit a query, the analytic engine requests metadata from Glue Data Catalog, which then queries Lake Formation to determine what data you are allowed to see. Then, the analytic engine requests temporary access from Lake Formation, retrieves the data from S3, and filters the data to returns policy-enforced data to you.

The diagram below illustrates how the analytic engine interacts with Glue Data Catalog and Lake Formation to access data.

This page describes the Azure Synapse Analytics integration, through which Immuta applies policies directly in Azure Synapse Analytics. For a tutorial on configuring Azure Synapse Analytics see the Azure Synapse Integration page.

Overview

The Azure Synapse Analytics is a policy push integration that allows Immuta to apply policies directly in Azure Synapse Analytics Dedicated SQL pools without the need for users to go through a proxy. Instead, users can work within their existing Synapse Studio and have per-user policies dynamically applied at query time.

Architecture

This integration works on a per-Dedicated-SQL-pool basis: all of Immuta's policy definitions and user entitlements data need to be in the same pool as the target data sources because Dedicated SQL pools do not support cross-database joins. Immuta creates schemas inside the configured Dedicated SQL pool that contain policy-enforced views that users query.

When the integration is configured, the Application Admin specifies the

• Immuta Database: This is the pre-existing database Immuta uses. Immuta will create views from the tables contained in this database, and all schemas and views created by Immuta will exist in this database, such as the schemas immuta_system, immuta_functions, and the immuta_procedures that contain the tables, views, UDFs, and stored procedures that support the integration.

• Immuta Schema: The schema that Immuta manages. All views generated by Immuta for tables registered as data sources will be created in this schema.

• User Profile Delimiters: Since Azure Synapse Analytics dedicated SQL pools do not support array or hash objects, certain user access information is stored as delimited strings; the Application Admin can modify those delimiters to ensure they do not conflict with possible characters in strings.

For a tutorial on configuring the integration see the Azure Synapse Integration page.

Data source naming convention

Synapse data sources are represented as views and are under one schema instead of a database, so their view names are a combination of their schema and table name, separated by an underscore.

For example, with a configuration that uses IMMUTA as the schema in the database dedicated_pool, the view name for the data source dedicated_pool.tpc.case would be dedicated_pool.IMMUTA.tpc_case.

You can see the view information on the data source details page under Connection Information.

Policy enforcement

This integration uses webhooks to keep views up-to-date with the corresponding Immuta data sources. When a data source or policy is created, updated, or disabled, a webhook is called that creates, modifies, or deletes the dynamic view in the Immuta schema. Note that only standard views are available because Azure Synapse Analytics Dedicated SQL pools do not support secure views.

Integration health status

The status of the integration is visible on the integrations tab of the Immuta application settings page. If errors occur in the integration, a banner will appear in the Immuta UI with guidance for remediating the error.

The definitions for each status and the state of configured data platform integrations is available in the response schema of the integrations API. However, the UI consolidates these error statuses and provides detail in the error messages.

Data flow

1. An Immuta Application Administrator configures the Synapse integration, registering their initial Synapse Dedicated SQL pool with Immuta.

2. Immuta creates Immuta schemas inside the configured Synapse Dedicated SQL pool.

3. A Data Owner registers Synapse tables in Immuta as data sources. A Data Owner, Data Governor, or Administrator creates or changes a policy or user in Immuta.

4. Data source metadata, tags, user metadata, and policy definitions are stored in Immuta's Metadata Database.

5. The Immuta Web Service calls a stored procedure that modifies the user entitlements or policies and updates data source view definitions as necessary.

6. A Synapse user who is subscribed to the data source in Immuta queries the corresponding data source view in Synapse and sees policy-enforced data.

In the AWS Lake Formation integration, Immuta orchestrates Lake Formation access controls on data registered in the Glue Data Catalog. Then, Immuta users who have been granted access to the Glue Data Catalog table or view can query it using one of these analytic engines:

• Amazon Athena

• Amazon EMR Spark

• Amazon Redshift Spectrum

The sequence diagram below outlines the events that occur when an Immuta user who is subscribed to a data source submits a query in their AWS analytic engine.

See the AWS Lake Formation documentation for more details about Lake Formation access controls.

Registering a connection

AWS Lake Formation is configured and data is registered through connections, an Immuta feature that allows administrators to register data objects in a technology through a single connection to make data registration more scalable for your organization.

Once the Lake Formation connection is registered, you can author policies in Immuta to orchestrate Lake Formation access controls.

See the AWS Lake Formation reference guide for more details about registering a connection.

Protecting data

After Glue Data Catalog views and tables are registered in Immuta, you can author subscription policies in Immuta to orchestrate Lake Formation access controls. Once a subscription policy is applied, users can be subscribed to data sources in the following ways:

• Manually subscribed: If a data owner manually adds a user to the data source, Immuta issues a grant directly to the data object in AWS.

• Automatically subscribed through policy logic: When a policy is applied to a data source, users who meet the conditions of the policy will be automatically subscribed to the data source. Then, Immuta generates a Lake Formation tag and applies it to the corresponding data object in AWS and grants subscribers access to that tag, which in turn grants them access to the data. See the AWS Lake Formation reference guide for details about this process.

Consider the following example that illustrates how Immuta enforces a subscription policy that only allows users in the analysts group to access the yellow-table. When this policy is authored and applied to the data source, Immuta generates a Lake Formation (LF) tag that is applied to the Glue Data Catalog yellow-table and permissions on that tag are granted to all AWS users (registered in Immuta) that are part of the analysts group.

In the image above, the user in the analysts group accesses yellow-table , while the user who is a part of the research group is denied access.

See the Author a subscription policy page for guidance on applying a subscription policy to a data source. See the Subscription policy access types page for details about the subscription policy types supported and permissions Immuta grants on securables registered as Immuta data sources.

1. In the Databricks Clusters UI, install your third-party library .jar or Maven artifact with Library Source Upload, DBFS, DBFS/S3, or Maven. Alternatively, use the Databricks libraries API.

2. In the Databricks Clusters UI, add the IMMUTA_SPARK_DATABRICKS_TRUSTED_LIB_URIS property as a Spark environment variable and set it to your artifact's URI:

IMMUTA_SPARK_DATABRICKS_TRUSTED_LIB_URIS=maven:/com.github.immuta.hadoop.immuta-spark-third-party-maven-lib-test:2020-11-17-144644

IMMUTA_SPARK_DATABRICKS_TRUSTED_LIB_URIS=dbfs:/immuta/bstabile/jars/immuta-spark-third-party-lib-test.jar

1. Once you've finished making your changes, restart the cluster.

2. Once the cluster is up, execute a command in a notebook. If the trusted library installation is successful, you should see driver log messages like this:

   Copy

   TrustedLibraryUtils: Successfully found all configured Immuta configured trusted libraries in Databricks.
   TrustedLibraryUtils: Wrote trusted libs file to [/databricks/immuta/immutaTrustedLibs.json]: true.
   TrustedLibraryUtils: Added trusted libs file with 1 entries to spark context.
   TrustedLibraryUtils: Trusted library installation complete.

The how-to guides linked on this page illustrate how to integrate Databricks Unity Catalog with Immuta. See the for information about the Databricks Unity Catalog integration.

Requirements:

• Unity Catalog and attached to a Databricks workspace. Immuta supports configuring a single metastore for each configured integration, and that metastore may be attached to multiple Databricks workspaces.

• Unity Catalog enabled on your Databricks cluster or SQL warehouse. All SQL warehouses have Unity Catalog enabled if your workspace is attached to a Unity Catalog metastore.

Permissions

The user registering the connection must have the permissions below.

• APPLICATION_ADMIN Immuta permission

• The Amazon Redshift user registering the connection must be a superuser or have the following Amazon Redshift privileges:

  • CREATEDB

  • CREATE USER

  • sys:secadmin role

  • USAGE on all databases and schemas that contain data you want to register

  • The following privileges WITH GRANT OPTION on objects registered in Immuta:

    • DELETE

    • INSERT

    • SELECT

    • TRUNCATE

    • UPDATE

  For descriptions and explanations of privileges Immuta needs to enforce policies and maintain state in Amazon Redshift, see the .

Create the database user

1. . Immuta will use this system account continuously to crawl the connection.

2. :

   • USAGE on all databases and schemas that contain data you want to register

   • CREATE ROLE

   • sys:secadmin role

   • The following privileges WITH GRANT OPTION on objects registered in Immuta:

     • DELETE

     • INSERT

     • SELECT

     • TRUNCATE

     • UPDATE

Register the connection

1. In your Amazon Redshift environment, create an Immuta database that Immuta can use to connect to your Amazon Redshift instance to register the connection and maintain state with Amazon Redshift.

   Having this separate database for Immuta prevents custom ETL processes or jobs deleting the database you use to register the connection, which would break the connection.

2. In Immuta, click Data and select Connections in the navigation menu.

3. Click the + Add Connection button.

4. Select the Amazon Redshift tile.

5. Enter the host connection information:

   1. Display Name: This is the name of your new connection. This name will be used in the API (connectionKey), in data source names from the host, and on the connections page.

   2. Hostname: URL of your Amazon Redshift instance.

   3. Port: Port configured for Amazon Redshift.

   4. Database: The Redshift database you created for Immuta. All databases in the host will be registered.

6. Enter the username and password of the .

7. Click Save connection.

Map users

Requirement: USER_ADMIN Immuta permission

Map Amazon Redshift usernames to each Immuta user account to ensure Immuta properly enforces policies.

The instructions below illustrate how to do this for individual users, but you can also configure user mapping in your .

1. Click People and select Users in the navigation menu.

2. Click the user's name to navigate to their page and scroll to the External User Mapping section.

3. Click Edit in the Redshift User row.

4. Enter the user's Redshift username.

5. Click Save.

Requirements

Permissions

The user registering the connection must have the permissions below.

• APPLICATION_ADMIN Immuta permission

• The account credentials you provide to register the connection should be a and it must have these Databricks Lakebase privileges:

  • databricks_superuser

  • CREATEROLE

  For descriptions and explanations of privileges Immuta needs to enforce policies and maintain state in Databricks Lakebase, see the .

Register a Databricks Lakebase connection

1. In Immuta, click Data and select Connections in the navigation menu.

2. Click the + Add Connection button.

3. Select the Databricks Lakebase tile.

4. Enter the host connection information:

   1. Display Name: This is the name of your new connection. This name will be used in the API (connectionKey), in data source names from the host, and on the connections page.

   2. Hostname

   3. Port

   4. Database: This should be the PostgreSQL dbname in the Databricks Lakebase connection details.

5. Enter privileged credentials to register the connection using OAuth M2M:

   1. Follow for the Immuta service principal and assign this service principal the for the Databricks Lakebase.

   2. Fill out the Workspace URL (e.g., https://<your workspace name>.cloud.databricks.com).

   3. Fill out the Client ID. This is a combination of letters, numbers, or symbols, used as a public identifier and is the .

   4. Enter the Client Secret you created above. Immuta uses this secret to authenticate with the authorization server when it requests a token.

6. Click Save Connection.

Map users

Requirement: USER_ADMIN Immuta permission

Map PostgreSQL usernames to each Immuta user account to ensure Immuta properly enforces policies when the user queries the Databricks Lakebase objects in PostgreSQL.

The instructions below illustrate how to do this for individual users, but you can also configure user mapping in your .

1. Click People and select Users in the navigation menu.

2. Click the user's name to navigate to their page and scroll to the External User Mapping section.

3. Click Edit in the PostgreSQL row.

4. Select one of the following options from the dropdown:

   1. Select PostgreSQL Username to map the PostgreSQL username to the Immuta user and enter the PostgreSQL username in the field. Username mapping is case insensitive.

   2. Select Unset (fallback to Immuta username) to use the Immuta username as the assumed PostgreSQL username. Use this option if the user's PostgreSQL username exactly matches the user's Immuta username. Username mapping is case insensitive.

   3. Select None (user does not exist in PostgreSQL) if this is an Immuta-only user. This option will improve performance for Immuta users who do not have a mapping to PostgreSQL users and will be automatically selected by Immuta if an Immuta user is not found in PostgreSQL. To ensure your PostgreSQL users have policies correctly applied, manually map their usernames using the first option above.

5. Click Save.

In the PostgreSQL integration, Immuta administers PostgreSQL privileges on data registered in Immuta. Then, Immuta users who have been granted access to the tables can query them with policies enforced.

The sequence diagram below outlines the events that occur when an Immuta user who is subscribed to a data source queries it in PostgreSQL.

Registering a connection

PostgreSQL is configured and data is registered through , an Immuta feature that allows administrators to register data objects in a technology through a single connection to make data registration more scalable for your organization.

Once the PostgreSQL connection is registered, you can author subscription policies in Immuta to enforce access controls.

See the for more details about registering a connection.

Protecting data

After tables are registered in Immuta, you can author subscription policies in Immuta to enforce access controls.

When a policy is applied to a data source, users who meet the conditions of the policy will be automatically subscribed to the data source. Then, Immuta issues a SQL statement in PostgreSQL that grants the SELECT privilege to users on those tables.

Consider the following example that illustrates how Immuta enforces a subscription policy that only allows users in the analysts group to access the yellow-table. When this policy is authored and applied to the data source, Immuta issues a SQL statement in PostgreSQL that grants the SELECT privilege on yellow-table to users (registered in Immuta) that are part of the analysts group.

In the image above, the user in the analysts group accesses yellow-table , while the user who is a part of the research group is denied access. See the for guidance on applying a subscription policy to a data source. See the page for details about the subscription policy types supported and PostgreSQL privileges Immuta grants on tables registered as Immuta data sources.

The how-to guides linked on this page illustrate how to use PostgreSQL with Immuta. See the for information about the PostgreSQL integration.

When the Databricks Spark plugin is running on a Databricks cluster, all Databricks users running jobs or queries are either a privileged user or a non-privileged user:

• Privileged users: Privileged users can effectively read from and write to any table or view in the cluster Metastore, or any file path accessible by the cluster, without restriction. Privileged users are either or users specified in . Any user writing queries or jobs impersonating another user is a non-privileged user, even if they are impersonating a privileged user.\

  Privileged users have effective authority to read from and write to any securable in the cluster metastore or file path, because in almost all cases Databricks clusters running with the Immuta Spark plug-in installed have disabled . However, if Hive metastore table access control is enabled on the cluster, privileged users will have the authority granted to them that is specified by table access control.

• Non-privileged users: Non-privileged users are any users who are not privileged users, and all authorization for non-privileged users is determined by Immuta policies.

Whether a user is a privileged user or a non-privileged user, for a given query or job, is cached once first determined, based on . This caching can be disabled entirely by setting the value of that environment variable to 0.

Mapping Databricks users to Immuta

Usernames in Databricks must match the usernames in the connected Immuta tenant. By default, the Immuta Spark plugin checks the Databricks username against the username within Immuta's internal IAM to determine access. However, you can integrate your existing IAM with Immuta and use that instead of the default internal IAM. Ideally, you should use the same identity manager for Immuta that you use for Databricks. See the for a list of supported identity providers and protocols.

It is possible within Immuta to have multiple users share the same username if they exist within different IAMs. In this case, the cluster can be configured to look up users from a specified IAM. To do this, the value of the must be updated to be the targeted IAM ID configured within the Immuta tenant. The targeted IAM ID can be found on the . Each Databricks cluster can only be mapped to one IAM.

User impersonation

Databricks user impersonation allows a Databricks user to impersonate an Immuta user. With this feature,

• the Immuta user who is being impersonated does not have to have a Databricks account, but they must have an Immuta account.

• the Databricks user who is impersonating an Immuta user does not have to be associated with Immuta. For example, this could be a service account.

When acting under impersonation, the Databricks user loses their privileged access, so they can only access the tables the Immuta user has access to and only perform DDL commands when that user is acting under an allowed circumstance (such as workspaces, scratch paths, or non-Immuta reads/writes).

Use the to enable user impersonation.

1. Click the App Settings icon in the navigation menu and scroll to the Global Integration Settings section.

2. Click the Enable Snowflake Low Row Access Policy Mode checkbox to enable the feature.

3. Confirm to allow Immuta to automatically disable impersonation for the Snowflake integration. If you do not confirm, you will not be able to enable Snowflake low row access policy mode.

4. Click Save.

Configure your Snowflake integration

If you already have a configured, you don't need to reconfigure your integration. Your Snowflake policies automatically refresh when you enable Snowflake low row access policy mode.

1. . Note that you will not be able to enable project workspaces or user impersonation with Snowflake low row access policy mode enabled.

2. Click Save and Confirm your changes.

Once a Databricks securable is registered in Immut as a data source and you are subscribed to that data source, you must access that data through SQL:

df = spark.sql("select * from immuta.table")

import org.apache.spark.sql.SparkSession

val spark = SparkSession
  .builder()
  .appName("Spark SQL basic example")
  .config("spark.some.config.option", "some-value")
  .getOrCreate()
val sqlDF = spark.sql("SELECT * FROM immuta.table")

%sql
select * from immuta.table

library(SparkR)
df <- SparkR::sql("SELECT * from immuta.table")

See the sections below for more guidance on accessing data using Delta Lake, direct file reads in Spark for file paths, and user impersonation.

Delta Lake

When using Delta Lake, the API does not go through the normal Spark execution path. This means that Immuta's Spark extensions do not provide protection for the API. To solve this issue and ensure that Immuta has control over what a user can access, the Delta Lake API is blocked.

Spark SQL can be used instead to give the same functionality with all of Immuta's data protections. See the Delta API reference guide for a list of corresponding Spark SQL calls to use.

Spark direct file reads

In addition to supporting direct file reads through workspace and scratch paths, Immuta allows direct file reads in Spark for file paths. As a result, users who prefer to interact with their data using file paths or who have existing workflows revolving around file paths can continue to use these workflows without rewriting those queries for Immuta.

When reading from a path in Spark, the Immuta Databricks Spark plugin queries the Immuta Web Service to find Databricks data sources for the current user that are backed by data from the specified path. If found, the query plan maps to the Immuta data source and follows existing code paths for policy enforcement.

Users can read data from individual parquet files in a sub-directory and partitioned data from a sub-directory (or by using a where predicate). Expand the blocks below to view examples of reading data using these methods.

spark.read.format("parquet").load("s3:/my_bucket/path/to/my_parquet_table/partition_column=01/my_file.parquet")

spark.read.format("parquet").load("s3:/my_bucket/path/to/my_parquet_table/partition_column=01")

spark.read.format("parquet").load("s3:/my_bucket/path/to/my_parquet_table").where("partition_column=01")Read partitioned data from a sub-directory

Limitations

• Direct file reads for Immuta data sources only apply to data sources created from tables, not data sources created from views or queries.

• If more than one data source has been created for a path, Immuta will use the first valid data source it finds. It is therefore not recommended to use this integration when more than one data source has been created for a path.

• In Databricks, multiple input paths are supported as long as they belong to the same data source.

• CSV-backed tables are not currently supported.

• Loading a delta partition from a sub-directory is not recommended by Spark and is not supported in Immuta. Instead, use a where predicate:

  Copy

  # Not recommended by Spark and not supported in Immuta
  spark.read.format("delta").load("s3:/my_bucket/path/to/my_delta_table/partition_column=01")
  
  # Recommended by Spark and supported in Immuta.
  spark.read.format("delta").load("s3:/my_bucket/path/to/my_delta_table").where("partition_column=01")
  

User impersonation

User impersonation allows Databricks users to query data as another Immuta user. To impersonate another user, see the User impersonation page.

To edit or remove a Snowflake integration, you have two options:

• Automatic: Grant Immuta one-time use of credentials with the following privileges to automatically edit or remove the integration:

  • CREATE DATABASE ON ACCOUNT WITH GRANT OPTION

  • CREATE ROLE ON ACCOUNT WITH GRANT OPTION

  • CREATE USER ON ACCOUNT WITH GRANT OPTION

  • MANAGE GRANTS ON ACCOUNT WITH GRANT OPTION

• Manual: Run the Immuta script in your Snowflake environment as a user with the following privileges to edit or remove the integration:

  • CREATE DATABASE ON ACCOUNT WITH GRANT OPTION

  • CREATE ROLE ON ACCOUNT WITH GRANT OPTION

  • CREATE USER ON ACCOUNT WITH GRANT OPTION

  • MANAGE GRANTS ON ACCOUNT WITH GRANT OPTION

  • APPLY MASKING POLICY ON ACCOUNT WITH GRANT OPTION

  • APPLY ROW ACCESS POLICY ON ACCOUNT WITH GRANT OPTION

Edit a Snowflake integration

Select one of the following options for editing your integration:

• Automatic: Grant Immuta one-time use of credentials to automatically edit the integration.

• Manual: Run the Immuta script in your Snowflake environment yourself to edit the integration.

Automatic edit

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab and click the down arrow next to the Snowflake integration.

3. Edit the field you want to change or check a checkbox of a feature you would like to enable. Note any field shadowed is not editable, and the integration must be disabled and re-installed to change it.

4. From the Select Authentication Method Dropdown, select either Username and Password or Key Pair Authentication:

   • Username and Password option: Complete the Username, Password, and Role fields.

   • Key Pair Authentication option:

     1. Complete the Username field.

     2. Click Key Pair (Required), and upload a Snowflake key pair file.

     3. Complete the Role field.

5. Click Save.

Manual edit

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab and click the down arrow next to the Snowflake integration.

3. Edit the field you want to change or check a checkbox of a feature you would like to enable. Note any field shadowed is not editable, and the integration must be disabled and re-installed to change it.

4. Click edit script to download the script, and then run it in Snowflake.

5. Click Save.

Remove a Snowflake integration

Select one of the following options for deleting your integration:

• Automatic: Grant Immuta one-time use of credentials to automatically remove the integration and Immuta-managed resources from your Snowflake environment.

• Manual: Run the Immuta script in your Snowflake environment yourself to remove Immuta-managed resources and policies from Snowflake.

Automatic removal

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab and click the down arrow next to the Snowflake integration.

3. Click the checkbox to disable the integration.

4. Enter the Username, Password, and Role that was entered when the integration was configured.

5. Click Save.

Manual removal

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab and click the down arrow next to the Snowflake integration.

3. Click the checkbox to disable the integration.

4. Click cleanup script to download the script.

5. Click Save.

6. Run the cleanup script in Snowflake.

This page describes the Amazon Redshift view-based integration, configuration options, and features. For a tutorial to enable this integration, see the installation guide.

Feature Availability

Prerequisites

• For automated installations, the credentials provided must be a superuser or have the ability to create databases and users and modify grants.

Supported Features

• Redshift datashares

• Redshift Serverless

• Redshift Spectrum For configuration and data source registration instructions, see the configuration page.

Authentication Methods

The Amazon Redshift view-based integration supports the following authentication methods to configure the integration and create data sources:

• Username and Password: Users can authenticate with their Redshift username and password.

• AWS Access Key: Users can authenticate with an AWS access key.

Tag Ingestion

Immuta cannot ingest tags from Redshift, but you can connect any of these supported external catalogs to work with your integration.

User Impersonation

Impersonation allows users to query data as another Immuta user in Redshift. To enable user impersonation, see the User Impersonation page.

Multiple Integrations

Users can enable multiple Amazon Redshift view-based integrations with a single Immuta tenant.

Redshift Limitations

• The host of the data source must match the host of the connection for the view to be created.

• When using multiple Amazon Redshift view-based integrations, a user has to have the same user account across all hosts.

• Case sensitivity of database, table, and column identifiers is not supported. The enable_case_sensitive_identifier parameter must be set to false (default setting) for your Redshift cluster to configure the integration and register data sources.

This page describes the Azure Synapse integration, configuration options, and features. See the Azure Synapse integration page for a tutorial on enabling the integration and these features through the app settings page.

Feature availability

Prerequisite

• A running dedicated SQL pool

Authentication methods

The Azure Synapse Analytics integration supports the following authentication methods to configure the integration and create data sources:

• Username and password: Immuta supports SQL authentication with username and password for Azure Synapse Analytics. See the SQL Authentication in Azure Synapse Analytics documentation for details.

• OAuth authentication with Microsoft Entra ID: You can use this authentication method to register data sources or configure the Azure Synapse Analytics integration using the manual setup method. To use this authentication method, OAuth must be set up via Microsoft Entra ID app registration with a client secret. See the Microsoft Entra documentation for details about using OAuth authentication with Microsoft Entra ID.

Tag ingestion

Immuta cannot ingest tags from Synapse, but you can connect any of these supported external catalogs to work with your integration.

User impersonation

Impersonation allows users to query data as another Immuta user in Synapse. To enable user impersonation, see the User Impersonation page.

Multiple integrations

A user can configure multiple integrations of Synapse to a single Immuta tenant.

Limitations

• Immuta does not support the following masking types in this integration because of limitations with dedicated SQL pools (linked below). Any column assigned one of these masking types will be masked to NULL:

  • Reversible Masking: Synapse UDFs currently only support SQL, but Immuta needs to execute code (such as JavaScript or Python) to support this masking feature. See the Synapse Documentation for details.

  • Format Preserving Masking: Synapse UDFs currently only support SQL, but Immuta needs to execute code (such as JavaScript or Python) to support this masking feature. See the Synapse Documentation for details.

  • Regex: The built in string replace function does not support full regex. See the Synapse Documentation for details.

• The delimiters configured when enabling the integration cannot be changed once they are set. To change the delimiters, the integration has to be disabled and re-enabled.

• If the generated view name is more than 128 characters, then the view name is shortened to 128 characters. This could cause collisions between view names if the shortened version is the same for two different data sources.

• For proper updates, the dedicated SQL pools have to be running when changes are made to users or data sources in Immuta.

This page provides an overview of the Amazon Redshift view-based integration in Immuta. For a tutorial detailing how to enable this integration, see the installation guide.

The Amazon Redshift view-based integration is a policy push integration that allows Immuta to apply policies directly in Redshift. This allows data analysts to query Redshift views directly instead of going through a proxy and have per-user policies dynamically applied at query time.

Architecture

The Amazon Redshift view-based integration will create views from the tables within the database specified when configured. Then, the user can choose the name for the schema where all the Immuta generated views will reside. Immuta will also create the schemas immuta_system, immuta_functions, and immuta_procedures to contain the tables, views, UDFs, and stored procedures that support the integration. Immuta then creates a system role and gives that system account the following privileges:

• ALL PRIVILEGES ON DATABASE IMMUTA_DB

• ALL PRIVILEGES ON ALL SCHEMAS IN DATABASE IMMUTA_DB

• USAGE ON FUTURE PROCEDURES IN SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES

• USAGE ON LANGUAGE PLPYTHONU

Additionally the PUBLIC role will be granted the following privileges:

• USAGE ON DATABASE IMMUTA_DB

• TEMP ON DATABASE IMMUTA_DB

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS

• USAGE ON FUTURE FUNCTIONS IN SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_SYSTEM

• SELECT ON TABLES TO public

Integration type

Immuta supports the Amazon Redshift view-based integration as both multi-database and single-database integrations. In either integration type, Immuta supports a single integration with secure views in a single database per cluster.

Multi-database integration

If using a multi-database integration, you must use a Redshift cluster with an RA3 node because Immuta requires cross-database views.

Single-database integration

If using a single-database integration, all Redshift cluster types are supported. However, because cross-database queries are not supported in any types other than RA3, Immuta's views must exist in the same database as the raw tables. Consequently, the steps for configuring the integration for Redshift clusters with external tables differ slightly from those that don't have external tables. Allow Immuta to create secure views of your external tables through one of these methods:

• configure the integration with an existing database that contains the external tables: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift.

• configure the integration by creating a new immuta database and re-create all of your external tables in that database.

Policy enforcement

SQL statements are used to create all views, including a join to the secure view: immuta_system.user_profile. This secure view is a select from the immuta_system.profile table (which contains all Immuta users and their current groups, attributes, projects, and a list of valid tables they have access to) with a constraint immuta__userid = current_user() to ensure it only contains the profile row for the current user. The immuta_system.user_profile view is readable by all users, but will only display the data that corresponds to the user executing the query.

The Amazon Redshift view-based integration uses webhooks to keep views up-to-date with Immuta data sources. When a data source or policy is created, updated, or disabled, a webhook will be called that will create, modify, or delete the dynamic view. The immuta_system.profile table is updated through webhooks when a user's groups or attributes change, they switch projects, they acknowledge a purpose, or when their data source access is approved or revoked. The profile table can only be read and updated by the Immuta system account.

Integration health status

The status of the integration is visible on the integrations tab of the Immuta application settings page. If errors occur in the integration, a banner will appear in the Immuta UI with guidance for remediating the error.

The definitions for each status and the state of configured data platform integrations is available in the response schema of the integrations API. However, the UI consolidates these error statuses and provides detail in the error messages.

Data flow

1. An Immuta Application Administrator configures the Redshift integration and registers Redshift warehouse and databases with Immuta.

2. Immuta creates a database inside the configured Redshift ecosystem that contains Immuta policy definitions and user entitlements.

3. A Data Owner registers Redshift tables in Immuta as data sources.

4. A Data Owner, Data Governor, or Administrator creates or changes a policy or user in Immuta.

5. Data source metadata, tags, user metadata, and policy definitions are stored in Immuta's Metadata Database.

6. The Immuta Web Service calls a stored procedure that modifies the user entitlements or policies.

7. A Redshift user who is subscribed to the data source in Immuta queries the corresponding table directly in Redshift through the immuta database and sees policy-enforced data.

Redshift Spectrum

Redshift Spectrum (Redshift external tables) allows Redshift users to query external data directly from files on Amazon S3. Because cross-database queries are not supported in Redshift Spectrum, Immuta's views must exist in the same database as the raw tables. Consequently, the steps for configuring the integration for Redshift clusters with external tables differ slightly from those that don't have external tables. Allow Immuta to create secure views of your external tables through one of these methods:

• configure the integration with an existing database that contains the external tables: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift

• configure the integration by creating a new immuta database and re-create all of your external tables in that database.

Once the integration is configured, Data Owners must register Redshift Spectrum data sources using the Immuta CLI or V2 API.

This page illustrates how to configure the on the Immuta app settings page. To configure this integration via the Immuta API, see the .

For instructions on configuring Redshift Spectrum, see the guide.

Requirements

• A Redshift cluster with an RA3 node is required for the multi-database integration. You must use a Redshift RA3 instance type because Immuta requires cross-database views, which are only supported in Redshift RA3 instance types. For other instance types, you may configure a single-database integration using one of the .

• For automated installations, the credentials provided must be a Superuser or have the ability to create databases and users and modify grants.

• The must be set to false (default setting) for your Redshift cluster.

Add a Redshift integration

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab.

3. Click the +Add Integration button and select Redshift from the dropdown menu.

4. Complete the Host and Port fields.

5. Enter an Immuta Database. This is a new database where all secure schemas and Immuta created views will be stored.

6. Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.

Select your configuration method

You have two options for configuring your Redshift environment:

• : Grant Immuta one-time use of credentials to automatically configure your Redshift environment and the integration.

• : Run the Immuta script in your Redshift environment yourself to configure your environment and the integration.

Automatic setup

1. Select Automatic.

2. Enter an Initial Database from your Redshift integration for Immuta to use to connect.

3. Use the dropdown menu to select your Authentication Method.

   1. Username and Password: Enter the Username and Password of the privileged user.

   2. AWS Access Key: Enter the Database User, Access Key ID, and Secret Key. Opt to enter in the Session Token.

Manual setup

1. Select Manual and download both of the bootstrap scripts.

2. Run the bootstrap script (initial database) in the Redshift initial database.

3. Run the bootstrap script (Immuta database) in the new Immuta Database in Redshift.

4. Choose your authentication method, and enter the information of the newly created account.

Save the configuration

Click Save.

Register data

.

Edit a Redshift integration

1. Click the App Settings icon in the navigation menu.

2. Navigate to the Integrations tab and click the down arrow next to the Redshift Integration.

3. Edit the field you want to change. Note any field shadowed is not editable, and the integration must be disabled and re-installed to change it.

4. Enter Username and Password.

5. Click Save.

Remove a Redshift integration

1. Click the App Settings icon in the navigation menu.

2. Navigate to the Integrations tab and click the down arrow next to the Redshift Integration.

3. Click the checkbox to disable the integration.

4. Enter the username and password that were used to initially configure the integration.

5. Click Save.

Requirement

• Amazon RDS for MariaDB

Permissions

The user registering the connection must have the permissions below.

• APPLICATION_ADMIN Immuta permission

• The MariaDB user setting up the connection must be the root user or have the GRANT OPTION MariaDB privilege.

Create a database user account

1. Create a new database user in MariaDB to serve as the Immuta system account. Immuta will use this system account continuously to crawl the database you register. How you create this user depends on your . Follow the instructions linked below to create this user:

   1. : Follow the to create the database user in and assign that user a password.

   2. :

      2. .

      3. .

2. . A sample command that provides all these privileges to all databases and views is provided below:

   1. SHOW DATABASES on all databases in the server

   2. SELECT on all databases, tables, and views in the server

   3. SHOW VIEW on all views in the server

Register a MariaDB connection

1. In Immuta, click Data and select Connections in the navigation menu.

2. Click the + Add Connection button.

3. Select the MariaDB tile.

4. Select RDS as the deployment method.

5. Enter the host connection information:

   1. Display Name: This is the name of your new connection. This name will be used in the API (connectionKey), in data source names from the host, and on the connections page.

   2. Hostname: URL of your MariaDB instance.

   3. Port: Port configured with MariaDB.

   4. Region: The region of the AWS account with your MariaDB instance.

6. Select an authentication method from the dropdown menu.

   1. AWS Access Key: Provide the access key ID and secret access key for the .

   2. AWS Assumed Role (recommended): Immuta will assume this IAM role from Immuta's AWS account to request temporary credentials that it can use to perform operations in the registered MariaDB database. Before proceeding, contact your Immuta representative and provide your service principal's IAM role. Immuta will allowlist the service principal so that Immuta can successfully assume that role. Your Immuta representative will provide the account to add to your trust relationship. Then, complete the steps below.

      1. Enter the Role ARN of the .

      2. Set the external ID provided in a condition on the trust relationship for the role specified above. See the for guidance.

   3. Username and Password: Enter the credentials for the you created above.

7. Click Save connection.

Requirements

• APPLICATION_ADMIN Immuta permission

• The Snowflake user registering the connection and running the script must have the following privileges:

  • CREATE DATABASE ON ACCOUNT WITH GRANT OPTION

  • CREATE ROLE ON ACCOUNT WITH GRANT OPTION

  • CREATE USER ON ACCOUNT WITH GRANT OPTION

  • MANAGE GRANTS ON ACCOUNT WITH GRANT OPTION

  • APPLY MASKING POLICY ON ACCOUNT WITH GRANT OPTION

  • APPLY ROW ACCESS POLICY ON ACCOUNT WITH GRANT OPTION

Prerequisites

No Snowflake integration configured in Immuta. If your Snowflake integration is already configured on the app settings page, follow the guide.

Set up the Immuta system account

Complete the following actions in Snowflake:

1. . Immuta will use this system account continuously to orchestrate Snowflake policies and maintain state between Immuta and Snowflake.

2. with a minimum of the following privileges:

   • USAGE on all databases and schemas with registered data sources.

   • REFERENCES on all tables and views registered in Immuta.

   • SELECT on all tables and views registered in Immuta.

3. to the system account you just created.

Register a connection

To register a Snowflake connection, follow the instructions below.

1. Click Data and select the Connections tab in the navigation menu.

2. Click the + Add Connection button.

3. Select the Snowflake data platform tile.

4. Enter the connection information:

   • Host: The URL of your Snowflake account.

   • Port: Your Snowflake port.

   • Warehouse: The warehouse the Immuta system account user will use to run queries and perform Snowflake operations.

   • Immuta Database: The new, empty database for Immuta to manage. This is where system views, user entitlements, row access policies, column-level policies, procedures, and functions managed by Immuta will be created and stored.

   • Display Name: The display name represents the unique name of your connection and will be used as prefix in the name for all data objects associated with this connection. It will also appear as the display name in the UI and will be used in all API calls made to update or delete the connection.

5. Click Next.

6. Select an authentication method from the dropdown menu and enter the authentication information for the . Enter the Role with the , then continue to enter the authentication information:

   1. Username and password (): Choose one of the following options.

      1. Select Immuta Generated to have Immuta populate the system account name and password.

      2. Select User Provided to enter your own name and password for the Immuta system account.

   2. Snowflake External OAuth:

      1. Fill out the Token Endpoint, which is where the generated token is sent. It is also known as aud (audience) and iss (issuer).

      2. Fill out the Client ID, which is the subject of the generated token. It is also known as sub (subject).

      3. Opt to fill out the Resource field with a URI of the resource where the requested token will be used.

      4. Enter the x509 Certificate Thumbprint. This identifies the corresponding key to the token and is often abbreviated as x5t or is called kid (key identifier).

      5. Upload the PEM Certificate, which is the client certificate that is used to sign the authorization request.

   3. :

      1. Complete the Username field. This user must be .

      2. If using an encrypted private key, enter the Private Key Password.

      3. Click Select a File, and upload the Snowflake private key pair file.

7. Copy the provided script and run it in Snowflake as a user with the privileges listed in the requirements section. Running this script grants the following privileges to the Immuta system account:

   1. CREATE ROLE ON ACCOUNT WITH GRANT OPTION

   2. APPLY MASKING POLICY ON ACCOUNT WITH GRANT OPTION

   3. APPLY ROW ACCESS POLICY ON ACCOUNT WITH GRANT OPTION

   4. MANAGE GRANTS ON ACCOUNT WITH GRANT OPTION

   Alternatively, you can grant the Immuta system account OWNERSHIP on the objects that Immuta will secure, instead of granting MANAGE GRANTS ON ACCOUNT. The current role that has OWNERSHIP on the securables will need to be granted to the Immuta system role. However, if granting OWNERSHIP instead of MANAGE GRANTS ON ACCOUNT, Immuta will not be able to manage the role that is granted to the account, so it is recommended to run the script as-is, without changes.

8. Click Test Connection.

9. If the connection is successful, click Next. If there are any errors, check the connection details and credentials to ensure they are correct and try again.

10. Ensure all the details are correct in the summary and click Complete Setup.

The how-to guides linked on this page illustrate how to integrate Databricks Spark with Immuta.

Requirements

• If Databricks Unity Catalog is enabled in a Databricks workspace, you must use an when you set up the Databricks Spark integration to create an Immuta-enabled cluster.

• If Databricks Unity Catalog is not enabled in your Databricks workspace, you must disable Unity Catalog in your Immuta tenant before proceeding with your configuration of Databricks Spark:

  1. Navigate to the App Settings page and click Integration Settings.

  2. Uncheck the Enable Unity Catalog checkbox.

  3. Click Save.

This page provides a tutorial for enabling the Azure Synapse Analytics integration on the Immuta app settings page. To configure this integration via the Immuta API, see the .

For an overview of the integration, see the documentation.

Requirement

• A running dedicated SQL pool is required.

Prerequisites

If you are using the OAuth authentication method,

• Ensure that Microsoft Entra ID is on the same account as the Azure Synapse Analytics workspace and dedicated SQL pool.

• .

• Select Accounts in this organizational directory only as the account type.

Add an Azure Synapse Analytics integration

1. Click the App Settings icon in the navigation menu.

2. Click the Integrations tab.

3. Click the +Add Integration button and select Azure Synapse Analytics from the dropdown menu.

4. Complete the Host, Port, Immuta Database, and Immuta Schema fields.

5. Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.

6. Opt to update the User Profile Delimiters. This will be necessary if any of the provided symbols are used in user profile information.

Select your configuration method

You have two options for configuring your Azure Synapse Analytic environment:

• : Grant Immuta one-time use of credentials to automatically configure your environment and the integration.

• : Run the Immuta script in your Azure Synapse Analytics environment yourself to configure the integration.

Automatic setup

Enter the username and password in the Privileged User Credentials section.

Manual setup

1. Select Manual.

2. Download, fill out the appropriate fields, and run the bootstrap master script and bootstrap script linked in the Setup section. Note: The master script is not required if you're using the OAuth authentication method.

3. Select the authentication method:

   1. Username and Password: Enter the username and password in the Immuta System Account Credentials section. The username and password provided must be the credentials that were set in the bootstrap master script when you created the user.

   2. Entra ID OAuth Client Secret: The values below can be found on the overview page of the application you created in Microsoft Entra ID. Before you enter this information, ensure you have completed the .

      1. Display Name: This must match the name of the OAuth application you registered.

      2. Tenant Id

      3. Client Id

      4. Client Secret: Enter the Value of the secret, not the secret ID.

Save the configuration

Click Save.

Register data

.

Edit an Azure Synapse Analytics integration

1. Click the App Settings icon in the navigation menu.

2. Navigate to the Integrations tab and click the down arrow next to the Azure Synapse Analytics Integration.

3. Edit the field you want to change. Note any field shadowed is not editable, and the integration must be disabled and re-installed to change it.

4. Use the authentication method and credentials you provided when initially configuring the integration.

5. Click Save.

Remove an Azure Synapse Analytics integration

1. Click the App Settings icon in the navigation menu.

2. Navigate to the Integrations tab and click the down arrow next to the Azure Synapse Analytics Integration.

3. Click the checkbox to disable the integration.

4. Enter the credentials that were used to initially configure the integration.

5. Click Save.

Requirement

Permissions

The user registering the connection must have the permissions below.

• APPLICATION_ADMIN Immuta permission

• Either of the following Oracle system privileges:

  • GRANT ANY ROLE

  • GRANT ANY PRIVILEGE

Create the database user

1. . Immuta will use this system account continuously to crawl the connection.

2. on the system views listed below:

   • V$DATABASE

   • CDB_PDBS

   • SYS.DBA_USERS

   • SYS.DBA_TABLES

   • SYS.DBA_VIEWS

   • SYS.DBA_MVIEWS

   • SYS.DBA_TAB_COLUMNS

   • SYS.DBA_OBJECTS

   • SYS.DBA_CONSTRAINTS

   • SYS.DBA_CONS_COLUMNS

Register an Oracle connection

1. In Immuta, click Data and select Connections in the navigation menu.

2. Click the + Add Connection button.

3. Select the Oracle tile.

4. Select RDS as the deployment method.

5. Enter the host connection information:

   1. Display Name: This is the name of your new connection. This name will be used in the API (connectionKey), in data source names from the host, and on the connections page.

   2. Hostname: URL of your Oracle instance.

   3. Port: Port configured for Oracle.

   4. Database: The Oracle database you want to connect to. All databases in the host will be registered.

   5. Region: The region of the AWS account with your Oracle instance.

6. Enter the username and password of the .

7. Click Save connection.