Amazon Redshift Spectrum Integration Overview

This page provides an overview of the Amazon Redshift Spectrum integration in Immuta. For a tutorial detailing how to enable this integration, see the installation guide.

How the integration works

The Amazon Redshift Spectrum integration is a policy push integration that allows Immuta to apply policies directly on Immuta-created views in Redshift. This allows data analysts to query Redshift views directly instead of going through a proxy and have per-user policies dynamically applied at query time.

The Amazon Redshift Spectrum integration creates views from the tables within the database specified when configured. Then, the user can choose the name for the schema where all the Immuta-generated views will reside. Immuta will also create the schemas immuta_system, immuta_functions, and immuta_procedures to contain the tables, views, UDFs, and stored procedures that support the integration. Immuta then creates a system role and gives that system account the following privileges:

• ALL PRIVILEGES ON DATABASE IMMUTA_DB

• ALL PRIVILEGES ON ALL SCHEMAS IN DATABASE IMMUTA_DB

• USAGE ON FUTURE PROCEDURES IN SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES

• USAGE ON LANGUAGE PLPYTHONU

Additionally the PUBLIC role will be granted the following privileges:

• USAGE ON DATABASE IMMUTA_DB

• TEMP ON DATABASE IMMUTA_DB

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_PROCEDURES

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS

• USAGE ON FUTURE FUNCTIONS IN SCHEMA IMMUTA_DB.IMMUTA_FUNCTIONS

• USAGE ON SCHEMA IMMUTA_DB.IMMUTA_SYSTEM

• SELECT ON TABLES TO public

Once the integration is configured, data owners must register Redshift Spectrum data sources using the Immuta CLI or V2 API.

Data flow

1. An Immuta application administrator, creates an immuta database in Amazon Redshift (that will contain Immuta policy definitions and user entitlements), configures the Redshift Spectrum integration, and registers Redshift warehouse and databases with Immuta.

2. A data owner registers Redshift tables in Immuta as data sources.

3. A data owner, data governor, or administrator creates or changes a policy or user in Immuta.

4. Data source metadata, tags, user metadata, and policy definitions are stored in Immuta's Metadata Database.

5. The Immuta Web Service calls a stored procedure that modifies the user entitlements or policies.

6. A Redshift user who is subscribed to the data source in Immuta queries the corresponding view directly in Redshift through the immuta database and sees policy-enforced data.

Policy enforcement

SQL statements are used to create all views, including a join to the secure view: immuta_system.user_profile. This secure view is a select from the immuta_system.profile table (which contains all Immuta users and their current groups, attributes, projects, and a list of valid tables they have access to) with a constraint immuta__userid = current_user() to ensure it only contains the profile row for the current user. The immuta_system.user_profile view is readable by all users, but will only display the data that corresponds to the user executing the query.

The Amazon Redshift Spectrum integration uses webhooks to keep views up-to-date with Immuta data sources. When a data source or policy is created, updated, or disabled, a webhook will be called that will create, modify, or delete the dynamic view. The immuta_system.profile table is updated through webhooks when a user's groups or attributes change, they switch projects, they acknowledge a purpose, or when their data source access is approved or revoked. The profile table can only be read and updated by the Immuta system account.

Integration health status

The status of the integration is visible on the integrations tab of the Immuta application settings page. If errors occur in the integration, a banner will appear in the Immuta UI with guidance for remediating the error.

The definitions for each status and the state of configured data platform integrations is available in the response schema of the integrations API. However, the UI consolidates these error statuses and provides detail in the error messages.

Supported cluster types

All Redshift cluster types are supported for the Amazon Redshift Spectrum integration, and Immuta's views must exist in the same database as the raw tables. See the Configure an Amazon Redshift Spectrum guide for details about setting up this database for Immuta-managed resources.

Immuta supports a single integration with secure views in a single database per cluster.

Authentication method

The Amazon Redshift Spectrum integration supports username and password authentication to configure the integration and create data sources.

Tag ingestion

Immuta cannot ingest tags from Amazon Redshift Spectrum, but you can connect any of these supported external catalogs to work with your integration.

User impersonation

Required Redshift privileges

Setup user

• OWNERSHIP ON GROUP IMMUTA_IMPERSONATOR_ROLE

• CREATE GROUP

Immuta system account

• GRANT EXECUTE ON PROCEDURE grant_impersonation

• GRANT EXECUTE ON PROCEDURE revoke_impersonation

Impersonation allows users to query data as another Immuta user in Amazon Redshift. To enable user impersonation, see the User impersonation page.

Multiple integrations

Users can enable multiple Amazon Redshift Spectrum integrations with a single Immuta tenant.

Limitations

• The host of the data source must match the host of the integration for the view to be created.

• When using multiple Amazon Redshift Spectrum integrations, a user has to have the same user account across all hosts.

• Case sensitivity of database, table, and column identifiers is not supported. The enable_case_sensitive_identifier parameter must be set to false (default setting) for your Redshift cluster to configure the integration and register data sources.