AWS PrivateLink for Databricks

Getting started with Databricks PrivateLink

AWS PrivateLink in Databricks provides a secure way for data communication between the Immuta SaaS platform and Databricks instances. AWS PrivateLink provides private connectivity between AWS VPCs as well as AWS services without exposing the traffic to the public network.

This front-end PrivateLink connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over a VPC interface endpoint. For details about AWS PrivateLink in Databricks and the network flow in a typical implementation, explore the Databricks documentation.

Configuration requirements

Databricks

Ensure that your accounts meet the following requirements:

• Your Databricks account is on the E2 version of the platform.
• Your Databricks account is on the Enterprise pricing tier.
• You have your Databricks account ID from the account console.
• You have an Immuta SaaS tenant.
• AWS PrivateLink for Databricks has been enabled.

Databricks workspace

Ensure that your workspace meets the following requirements:

• Your workspace must be in an AWS region that supports the E2 version of the platform. The us-west-1 region does not support PrivateLink even for workspaces on the E2 version of the platform.
• Your Databricks workspace must use Customer-managed VPC to add any PrivateLink connection.
• Your workspaces must be configured with private_access_settings objects.

Enablement

Contact your Databricks representative to enable AWS PrivateLink on your account.

Configure Databricks PrivateLink

1. Contact your Immuta representative, who will provide the PrivateLink endpoint IDs to register with your accounts. You will need to provide the following information:

   • AWS region
   • Databricks hostname
   • Private access level (either ACCOUNT or ENDPOINT)

2. Register the PrivateLink endpoint IDs.

   • If the private_access_level on your private_access_settings object is set to ACCOUNT, you should not need to do anything else beyond the registration.

   • If the private_access_level on your private_access_settings object is set to ENDPOINT, you will need to add it to the allowed_vpc_endpoint_ids list inside your private_access_settings object in Databricks. For example,

     "private_access_settings_name": "immuta-access",
     "region": "us-east-1",
     "public_access_enabled": false,
     "private_access_level": "ENDPOINT",
     "allowed_vpc_endpoint_ids": [
          "vpce-0fe5b17a0707d6fa5"
     ]