Register an AWS Lake Formation Connection

Public preview: This connection is available to all accounts. Some permissions and requirements listed for the setup of this connection refer to Immuta governing data or orchestrating AWS Lake Formation policies. While enforcing policies is not currently supported for this connection, that support will be available in a subsequent release. Therefore, configure your AWS Lake Formation permissions and connection as outlined below to prepare for that policy support.

Requirements

• Data lake is set up in AWS Lake Formation. The account in which this is set up is referred to as the admin account. This is the account that you will use to initially configure IAM and AWS Lake Formation permissions to give the Immuta service principal access to perform operations. The user in this account must be able to manage IAM permissions and Lake Formation permissions for all data in the Glue Data Catalog.

• No AWS Lake Formation connections configured in the same Immuta instance for the same Glue Data Catalog.

• The databases and tables you want Immuta to govern must be configured in AWS to respect the AWS Lake Formation permissions. Immuta cannot govern resources that use IAM access control or hybrid access mode. To ensure Immuta can govern your resources, verify that the default Data Catalog settings in AWS are unchecked. See the screenshot below and AWS documentation for instructions on changing these settings:

  

  AWS Glue Data Catalog settings must be unchecked for Immuta to govern access.
• : IDC is the best approach for user provisioning because it treats users as users, not users as roles. Consequently, access controls are enforced for the querying user, nothing more. This approach eliminates over-provisioning and permits granular access control. Furthermore, IDC uses trusted identity propagation, meaning AWS propagates a user's identity wherever that user may operate within the AWS ecosystem. As a result, a user's identity always remains known and consistent as they navigate across AWS services, which is a key requirement for organizations to properly govern that user. Enabling IDC does not impact any existing access controls; it is additive. See the map users section for instructions on mapping users from AWS IDC to user accounts in Immuta.

Permissions

These are permissions that the user registering the connection must have in order to successfully complete setup.

• APPLICATION_ADMIN Immuta permission to register the connection

• DESCRIBE AWS permission. You must have the DESCRIBE permission on the required resources in AWS:

  • All databases that should be registered in the connection

  • All tables that should be registered in the connection

  • Any LF-Tags you are using on the resources that should be registered in the connection

• The AWS account credentials or you provide for the Immuta service principal must have permissions to perform the following actions to register data and apply policies:

  • Glue Data Catalog actions

    • glue:GetDatabase

    • glue:GetTables

    • glue:GetDatabases

    • glue:GetTable

  • Lake Formation actions

    • lakeformation:ListPermissions

    • lakeformation:BatchGrantPermissions

    • lakeformation:BatchRevokePermissions

    • lakeformation:CreateLFTag

    • lakeformation:UpdateLFTag

    • lakeformation:DeleteLFTag

    • lakeformation:AddLFTagsToResource

    • lakeformation:RemoveLFTagsFromResource

    • lakeformation:GetResourceLFTags

    • lakeformation:ListLFTags

    • lakeformation:GetLFTag

    • lakeformation:SearchTablesByLFTags

    • lakeformation:SearchDatabasesByLFTags

Set up the Immuta service principal

The Immuta service principal is the to perform operations in your AWS account. This role must have all the necessary permissions in AWS Glue and AWS Lake Formation to allow Immuta to register data sources.

1. Create an AWS Account IAM role (select AWS account as the trusted entity type) that can be used by Immuta to set up the connection. Immuta will assume this IAM role from Immuta's AWS account in order to perform any operations in your AWS account. Before proceeding, contact your Immuta representative for the AWS account to add to your trust policy. Then, complete the steps below.
2. Add the following IAM permissions to the service principal from the admin account. These permissions will allow the service principal to register data sources.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "glue:GetDatabase",
           "glue:GetTables",
           "glue:GetDatabases",
           "glue:GetTable",
           "lakeformation:ListPermissions",
           "lakeformation:BatchGrantPermissions",
           "lakeformation:BatchRevokePermissions",
           "lakeformation:CreateLFTag",
           "lakeformation:UpdateLFTag",
           "lakeformation:DeleteLFTag",
           "lakeformation:AddLFTagsToResource",
           "lakeformation:RemoveLFTagsFromResource",
           "lakeformation:GetResourceLFTags",
           "lakeformation:ListLFTags",
           "lakeformation:GetLFTag",
           "lakeformation:SearchTablesByLFTags",
           "lakeformation:SearchDatabasesByLFTags"
         ],
         "Resource": "*"
       }
     ]
   }

3. Add the service principal as an LF-Tag Creator.

   1. In the Lake Formation console, navigate to Permissions.

   2. Select LF-Tags and permissions.

   3. Select LF-Tag creators, and then Add LF-Tag creators.

   4. Enter your service principal, and grant it the Create LF-Tag permission and grantable permission.

   5. Click Add to save your changes.

4. Grant the service principal permissions on any tables that will be registered in Immuta. There are two ways to give the service principal these permissions: either make a new LF-Tag that gives the appropriate permissions and apply it to , or make the role a superuser in Lake Formation.

LF-Tag permissions (recommended)

This method follows the principle of least privilege and is the most flexible way of granting permissions to the service principal. LF-Tags cascade down from databases to tables, while allowing for exceptions. This means that when you apply this tag to a database, it will automatically apply to all tables within that database and allow you to remove it from any tables if those should be out of the scope of Immuta’s governance.

1. Create a new LF-Tag, giving yourself permissions to grant that tag to a user, which will ultimately be your service principal.

   1. In the Lake Formation console, navigate to LF-Tags and permissions and click Add LF-Tag. You will need the Create LF-Tag permission to do this.

   2. Create a tag key and value.

   3. On the LF-Tag key-value pair, grant the ASSOCIATE LF-Tag permission to your own IAM principal.

2. Grant this tag to the Immuta service principal.

   1. In the Lake Formation console, navigate to Data permissions and click Grant.

   2. Enter the service principal’s IAM role.

   3. Add the key-value pair of the tag you created in step 1.

   4. Under Table Permissions, select the following grantable permissions: SELECT, DESCRIBE, INSERT, DELETE .

   5. Click Grant.

3. Apply this tag to the resources you would like Immuta to govern. The Immuta service principal will now have the minimum required permissions on these resources.

Superuser

This option enables all Lake Formation operations on all data in the Glue Data Catalog. This is highly privileged and runs the risk of managing permissions on data you did not intend to.

This method will grant all necessary permissions to the service principal, but grants more than the service principal needs without being as flexible, since it does not allow for exceptions like the LF-Tag method. You can make the service principal a superuser on the entire catalog or specify individual resources.

1. In the Lake Formation console, navigate to Data permissions and click Grant.

2. Enter your service principal’s IAM role.

3. Select Named Data Catalog resources, and input the Glue Data Catalog ID and any databases or tables you wish to specify.

4. Under Grantable permissions, select Super and click Grant.

Follow the AWS documentation to grant ALL permissions to the DataLakePrincipalIdentifier for the Immuta service principal ARN.

Register an AWS Lake Formation connection

1. Click Data and select Connections in the navigation menu.

2. Click the + Add Connection button.

3. Select the AWS Lake Formation tile.

4. Enter the host connection information:

   1. Display Name: This is the name of your new connection. This name will be used in the API (connectionKey), in data source names from the host, and on the connections page.

   2. AWS Account ID: The ID of the AWS account associated with the Glue Data Catalog.

   3. AWS Region: The region of the AWS account associated with the Glue Data Catalog.

5. Click Next.

6. Select an authentication method from the dropdown menu.

   1. AWS Access Key and Secret Access Key: Provide the access key ID and secret access key for an AWS account with the AWS permissions listed in the set up the Immuta service principal section.
   2. AWS IAM Role (recommended): Immuta will assume this IAM role from Immuta's AWS account in order to perform any operations in your AWS account. Contact your Immuta representative before proceeding, and Immuta will

      1. Provide the AWS account to add to your trust policy.

      2. Update the Immuta AWS configuration to allow Immuta to assume the role of your service principal.

      Then, complete the steps below.

      1. Enter the role ARN in the AWS IAM Role field. Immuta will assume this role when interacting with AWS.

      2. Set the external ID provided in a condition on the trust relationship for the cross-account IAM specified above. See the AWS documentation for guidance.
7. Ensure that you have the correct permissions and click Validate Connection.

8. If the connection is successful, click Next. If there are any errors, check the connection details and credentials to ensure they are correct and try again.

9. Ensure all the details are correct in the summary and click Complete Setup.