Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
The how-to guides linked on this page illustrate how to integrate Redshift with Immuta.
Requirement: Redshift cluster with an RA3 node is required for the multi-database integration. For other instance types, you may configure a single-database integration using one of the Redshift Spectrum options.
These guides provide information on the recommended feature to enable with Redshift.
Select None as your default subscription policy.
These guides provide instructions for organizing your Redshift data to align with your governance structure.
Private preview: Native SDD for Redshift is currently in private preview and available to all accounts.
These guides provide instructions for discovering, classifying, and tagging your data.
Register a subset of your tables to configure and validate SDD.
Configure SDD to discover entities of interest for your policy needs.
Register your remaining tables at the schema level with schema monitoring turned on.
These guides provide instructions for configuring and securing your data with governance policies, or see the Secure use cases for a comprehensive guide on creating policies to fit your organization's use case.
Validate the policies. You do not have to validate every policy you create in Immuta; instead, examine a few to validate the behavior you expect to see.
Once all Immuta policies are in place, remove or alter old permissions and revoke access to the ungoverned tables.
In this integration, Immuta generates policy-enforced views in your configured Redshift schema for tables registered as Immuta data sources.
This guide outlines how to integrate Redshift with Immuta.
: Configure the integration in Immuta.
: Configure Redshift Spectrum in Immuta.
: This guide describes the design and components of the integration.
This page provides an overview of the Redshift integration in Immuta. For a tutorial detailing how to enable this integration, see the .
Redshift is a policy push integration that allows Immuta to apply policies directly in Redshift. This allows data analysts to query Redshift views directly instead of going through a proxy and have per-user policies dynamically applied at query time.
The Redshift integration will create views from the tables within the database specified when configured. Then, the user can choose the name for the schema where all the Immuta generated views will reside. Immuta will also create the schemas immuta_system, immuta_functions, and immuta_procedures to contain the tables, views, UDFs, and stored procedures that support the integration. Immuta then creates a system role and gives that system account the following privileges:
ALL PRIVILEGES ON DATABASE IMMUTA_DB
ALL PRIVILEGES ON ALL SCHEMAS IN DATABASE IMMUTA_DB
USAGE ON FUTURE PROCEDURES IN SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES
USAGE ON LANGUAGE PLPYTHONU
Additionally the PUBLIC role will be granted the following privileges:
USAGE ON DATABASE IMMUTA_DB
TEMP ON DATABASE IMMUTA_DB
USAGE ON SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES
USAGE ON SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS
USAGE ON FUTURE FUNCTIONS IN SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS
USAGE ON SCHEMA IMMUTA_DB.IMMUTA_SYSTEM
SELECT ON TABLES TO public
Immuta supports the Redshift integration as both multi-database and single-database integrations. In either integration type, Immuta supports a single native integration with secure views in a single database per cluster.
If using a multi-database integration, you must use a Redshift cluster with an RA3 node because Immuta requires cross-database views.
If using a single-database integration, all Redshift cluster types are supported. However, because cross-database queries are not supported in any types other than RA3, Immuta's views must exist in the same database as the raw tables. Consequently, the steps for configuring the integration for Redshift clusters with external tables differ slightly from those that don't have external tables. Allow Immuta to create secure views of your external tables through one of these methods:
SQL statements are used to create all views, including a join to the secure view: immuta_system.user_profile. This secure view is a select from the immuta_system.profile table (which contains all Immuta users and their current groups, attributes, projects, and a list of valid tables they have access to) with a constraint immuta__userid = current_user() to ensure it only contains the profile row for the current user. The immuta_system.user_profile view is readable by all users, but will only display the data that corresponds to the user executing the query.
The Redshift integration uses webhooks to keep views up-to-date with Immuta data sources. When a data source or policy is created, updated, or disabled, a webhook will be called that will create, modify, or delete the dynamic view. The immuta_system.profile table is updated through webhooks when a user's groups or attributes change, they switch projects, they acknowledge a purpose, or when their data source access is approved or revoked. The profile table can only be read and updated by the Immuta system account.
Immuta creates a database inside the configured Redshift ecosystem that contains Immuta policy definitions and user entitlements.
Data source metadata, tags, user metadata, and policy definitions are stored in Immuta's Metadata Database.
The Immuta Web Service calls a stored procedure that modifies the user entitlements or policies.
: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift.
and re-create all of your external tables in that database.
An Immuta Application Administrator and registers Redshift warehouse and databases with Immuta.
A Data Owner registers Redshift tables in Immuta as .
A Data Owner, Data Governor, or Administrator or user in Immuta.
A Redshift user who is subscribed to the data source in Immuta directly in Redshift through the immuta database and sees policy-enforced data.
Redshift Spectrum () allows Redshift users to query external data directly from files on Amazon S3. Because cross-database queries are not supported in Redshift Spectrum, Immuta's views must exist in the same database as the raw tables. Consequently, the steps for configuring the integration for Redshift clusters with external tables differ slightly from those that don't have external tables. Allow Immuta to create secure views of your external tables through one of these methods:
: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift
and re-create all of your external tables in that database.
Once the integration is configured, Data Owners must .
This page describes the Redshift integration, configuration options, and features. For a tutorial to enable this integration, see the installation guide.
Project Workspaces
Query Audit
For automated installations, the credentials provided must be a Superuser or have the ability to create databases and users and modify grants.
Redshift Serverless.
Redshift Spectrum For configuration and data source registration instructions, see the configuration page.
The Redshift integration supports the following authentication methods to configure the integration and create data sources:
Username and Password: Users can authenticate with their Redshift username and password.
AWS Access Key: Users can authenticate with an AWS access key.
Okta: Users can authenticate with their Okta credentials when installing the integration with the manual configuration.
Deprecation notice
Support for Okta authentication has been deprecated.
Immuta cannot ingest tags from Redshift, but you can connect any of these supported external catalogs to work with your integration.
Required Redshift privileges
Setup User:
OWNERSHIP ON GROUP IMMUTA_IMPERSONATOR_ROLE
CREATE GROUP
Immuta System Account:
GRANT EXECUTE ON PROCEDURE grant_impersonation
GRANT EXECUTE ON PROCEDURE revoke_impersonation
Impersonation allows users to query data as another Immuta user in Redshift. To enable user impersonation, see the User Impersonation page.
Users can enable multiple Redshift integrations with a single Immuta tenant.
The host of the data source must match the host of the native connection for the native view to be created.
When using multiple Redshift integrations, a user has to have the same user account across all hosts.
Registering Redshift datashares as Immuta data sources is unsupported.
Case sensitivity of database, table, and column identifiers is not supported. The enable_case_sensitive_identifier parameter must be set to false (default setting) for your Redshift cluster to configure the integration and register data sources.
For most policy types in Redshift, Immuta uses SQL clauses to implement enforcement logic; however Immuta uses Python UDFs in the Redshift integration to implement the following masking policies:
Masking using a regular expression
Reversible masking
Format-preserving masking
Randomized response
The number of Python UDFs that can run concurrently per Redshift cluster is limited to one-fourth of the total concurrency level for the cluster. For example, if the Redshift cluster is configured with a concurrency of 15, a maximum of three Python UDFs can run concurrently. After the limit is reached, Python UDFs are queued for execution within workload management queues.
The SVL_QUERY_QUEUE_INFO view in Redshift, which is visible to a Redshift superuser, summarizes details for queries that spent time in a workload management (WLM) query queue. Queries must be completed in order to appear as results in the SVL_QUERY_QUEUE_INFO view.
If you find that queries on Immuta-built views are spending time in the workload management (WLM) query queue, you should either edit your Redshift cluster configuration to increase concurrency, or use fewer of the masking policies which leverage Python UDFs. For more information on increasing concurrency, see the Redshift docs on implementing workload management.
The status of the integration is visible on the integrations tab of the Immuta application settings page. If errors occur in the integration, a banner will appear in the Immuta UI with guidance for remediating the error .
The definitions for each status and the state of configured data platform integrations is available in the response schema of the integrations API. However, the UI consolidates these error statuses and provides detail in the error messages.
Allow Immuta to create secure views of your external tables through one of these methods:
Configure the integration with an existing database that contains the external tables: Instead of creating an immuta database that manages all schemas and views created when Redshift data is registered in Immuta, the integration adds the Immuta-managed schemas and views to an existing database in Redshift
Configure the integration by creating a new immuta database and re-create all of your external tables in that database.
For an overview of the integration, see the Redshift overview documentation.
A Redshift cluster with an AWS row-level security patch applied. Contact Immuta for guidance.
The enable_case_sensitive_identifier parameter must be set to false (default setting) for your Redshift cluster.
The Redshift role used to run the Immuta bootstrap script must have the following privileges when configuring the integration to
Use an existing database:
ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.
CREATE USER
GRANT TEMP ON DATABASE
Create a new database:
CREATE DATABASE
CREATE USER
GRANT TEMP ON DATABASE
REVOKE ALL PRIVILEGES ON DATABASE
Click the App Settings icon in the left sidebar.
Click the Integrations tab.
Click the +Add Native Integration button and select Redshift from the dropdown menu.
Complete the Host and Port fields.
Enter the name of the database you created the external schema in as the Immuta Database. This database will store all secure schemas and Immuta-created views.
Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.
Select Manual and download both of the bootstrap scripts from the Setup section. The specified role used to run the bootstrap needs to have the following privileges:
ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.
CREATE USER
GRANT TEMP ON DATABASE
Run the bootstrap script (Immuta database) in the Redshift database that contains the external schema.
Choose your authentication method, and enter the credentials from the bootstrap script for the Immuta_System_Account.
Click Save.
Register Redshift data in Immuta.
Click the App Settings icon in the left sidebar.
Click the Integrations tab.
Click the +Add Native Integration button and select Redshift from the dropdown menu.
Complete the Host and Port fields.
Enter an Immuta Database. This is a new database where all secure schemas and Immuta created views will be stored.
Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.
Select Manual and download both of the bootstrap scripts from the Setup section. The specified role used to run the bootstrap needs to have the following privileges:
ALL PRIVILEGES ON DATABASE for the database you configure the integration with, as you must manage grants on that database.
CREATE DATABASE
CREATE USER
GRANT TEMP ON DATABASE
Run the bootstrap script (initial database) in the Redshift initial database.
Run the bootstrap script (Immuta database) in the new Immuta Database in Redshift.
Choose your authentication method, and enter the credentials from the bootstrap script for the Immuta_System_Account.
Click Save.
Then, add your external tables to the Immuta database.
This page illustrates how to configure the on the Immuta app settings page. To configure this integration via the Immuta API, see the .
For instructions on configuring Redshift Spectrum, see the guide.
A Redshift cluster with an RA3 node is required for the multi-database integration. You must use a Redshift RA3 instance type because Immuta requires cross-database views, which are only supported in Redshift RA3 instance types. For other instance types, you may configure a single-database integration using one of the .
For automated installations, the credentials provided must be a Superuser or have the ability to create databases and users and modify grants.
The must be set to false (default setting) for your Redshift cluster.
Click the App Settings icon in the left sidebar.
Click the Integrations tab.
Click the +Add Native Integration button and select Redshift from the dropdown menu.
Complete the Host and Port fields.
Enter an Immuta Database. This is a new database where all secure schemas and Immuta created views will be stored.
Opt to check the Enable Impersonation box and customize the Impersonation Role name as needed. This will allow users to natively impersonate another user.
You have two options for configuring your Redshift environment:
Immuta requires temporary, one-time use of credentials with specific privileges
When performing an automated installation, Immuta requires temporary, one-time use of credentials with the following privileges:
CREATE DATABASE
CREATE USER
REVOKE ALL PRIVILEGES ON DATABASE
GRANT TEMP ON DATABASE
MANAGE GRANTS ON ACCOUNT
These privileges will be used to create and configure a new IMMUTA database within the specified Redshift instance. The credentials are not stored or saved by Immuta, and Immuta doesn’t retain access to them after initial setup is complete.
You can create a new account for Immuta to use that has these privileges, or you can grant temporary use of a pre-existing account. By default, the pre-existing account with appropriate privileges is a Superuser. If you create a new account, it can be deleted after initial setup is complete.
Alternatively, you can create the IMMUTA database within the specified Redshift instance without giving Immuta user credentials for a Superuser using the manual setup option.
Select Automatic.
Enter an Initial Database from your Redshift integration for Immuta to use to connect.
Use the dropdown menu to select your Authentication Method.
Username and Password: Enter the Username and Password of the privileged user.
AWS Access Key: Enter the Database User, Access Key ID, and Secret Key. Opt to enter in the Session Token.
Required privileges
The specified role used to run the bootstrap needs to have the following privileges:
CREATE DATABASE
CREATE USER
REVOKE ALL PRIVILEGES ON DATABASE
GRANT TEMP ON DATABASE
MANAGE GRANTS ON ACCOUNT
Select Manual and download both of the bootstrap scripts.
Run the bootstrap script (initial database) in the Redshift initial database.
Run the bootstrap script (Immuta database) in the new Immuta Database in Redshift.
Choose your authentication method, and enter the information of the newly created account.
Click Save.
Click the App Settings icon in the left sidebar.
Navigate to the Integrations tab and click the down arrow next to the Redshift Integration.
Edit the field you want to change. Note any field shadowed is not editable, and the integration must be disabled and re-installed to change it.
Enter Username and Password.
Click Save.
Required privileges
When performing edits to an integration, Immuta requires temporary, one-time use of credentials of a Superuser or a user with the following permissions:
Create Databases
Create users
Modify grants
Alternatively, you can download the Edit Script and run it in Redshift.
Disabling Redshift Spectrum
Click the App Settings icon in the left sidebar.
Navigate to the Integrations tab and click the down arrow next to the Redshift Integration.
Click the checkbox to disable the integration.
Enter the username and password that were used to initially configure the integration.
Click Save.
: Grant Immuta one-time use of credentials to automatically configure your Redshift environment and the integration.
: Run the Immuta script in your Redshift environment yourself to configure your environment and the integration.
.
Disabling the Redshift integration is not supported when you set the fields nativeWorkspaceName, nativeViewName, and nativeSchemaName to . Disabling the integration when these fields are used in metadata ingestion causes undefined behavior.