# AWS PrivateLink for Databricks

[AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html) provides private connectivity from the Immuta SaaS platform to Databricks accounts hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

This front-end PrivateLink connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over a VPC interface endpoint. For details about AWS PrivateLink in Databricks and the network flow in a typical implementation, explore the [Databricks documentation](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html).

This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

<figure><img src="https://1751699907-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FlWBda5Pt4s8apEhzXGl7%2Fuploads%2Fgit-blob-e813f8f9a8dc709f524f4a38ebada511070e5b4c%2Fdbx-pl-on-aws.png?alt=media" alt=""><figcaption></figcaption></figure>

## Requirements

### Databricks

Ensure that your accounts meet the following requirements:

* Your Databricks account is on the E2 version of the platform.
* Your Databricks account is on the [Enterprise pricing tier](https://www.databricks.com/product/aws-pricing).
* You have your Databricks account ID from the [account console](https://docs.databricks.com/administration-guide/account-settings/index.html#account-console).
* You have an Immuta SaaS tenant.
* [AWS PrivateLink for Databricks](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html) has been enabled.

### Databricks workspace

Ensure that your workspace meets the following requirements:

* Your workspace must be in an [AWS region that supports the E2 version of the platform](https://docs.databricks.com/resources/supported-regions.html).
* Your Databricks workspace must use a [customer-managed VPC](https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html) to add any PrivateLink connection.
* Your workspaces must be [<mark style="color:blue;">configured with</mark> <mark style="color:blue;">`private_access_settings`</mark> <mark style="color:blue;">objects</mark>](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html#updates-of-existing-privatelink-configuration-objects).

{% hint style="warning" %}
**You cannot configure a connection to your workspace over the public internet if PrivateLink is enabled.**

If you have PrivateLink configured on your workspace, Databricks will update the DNS records for that workspace URL to resolve to `<region>.privatelink.cloud.databricks.com`. Immuta SaaS uses these publicly-resolvable records to direct traffic to a PrivateLink endpoint on our network.

This means that **if you have PrivateLink enabled on your workspace, you must follow these instructions to configure your integration.** Even if your workspace is also publicly-routable, Databricks's DNS resolution forces the traffic over PrivateLink.

The two supported configurations are

* A workspace with no PrivateLink configuration, which resolves to public IP addresses.
* A workspace with PrivateLink configuration, which allows access from the Immuta SaaS regional endpoint (listed below).
  {% endhint %}

### Enablement

Contact your Databricks representative to enable AWS PrivateLink on your account.

## Configure Databricks with AWS PrivateLink

1. [Register the Immuta VPC endpoint ](https://docs.databricks.com/en/security/network/classic/vpc-endpoints.html)for the applicable AWS region with your Databricks workspaces. The Immuta VPC endpoint IDs are listed in the table below.

| AWS region                                                                      | VPC endpoint ID          |
| ------------------------------------------------------------------------------- | ------------------------ |
| <p><strong><code>ap-northeast-1</code></strong><br>Asia Pacific (Tokyo)</p>     | `vpce-08cadda15f0f70462` |
| <p><strong><code>ap-northeast-2</code></strong><br>Asia Pacific (Seoul)</p>     | `vpce-0e45ce26a7f8d00af` |
| <p><strong><code>ap-south-1</code></strong><br>Asia Pacific (Mumbai)</p>        | `vpce-0efef886a4fbd9532` |
| <p><strong><code>ap-southeast-1</code></strong><br>Asia Pacific (Singapore)</p> | `vpce-07e9890053f5084b2` |
| <p><strong><code>ap-southeast-2</code></strong><br>Asia Pacific (Sydney)</p>    | `vpce-0d363d9ea82658bec` |
| <p><strong><code>ca-central-1</code></strong><br>Canada (Central)</p>           | `vpce-01933bcf30ac4ed19` |
| <p><strong><code>eu-central-1</code></strong><br>Europe (Frankfurt)</p>         | `vpce-0048e36edfb27d0aa` |
| <p><strong><code>eu-west-1</code></strong><br>Europe (Ireland)</p>              | `vpce-0783d9412b046df1f` |
| <p><strong><code>eu-west-2</code></strong><br>Europe (London)</p>               | `vpce-0f546cc413bf70baa` |
| <p><strong><code>us-east-1</code></strong><br>US East (Virginia)</p>            | `vpce-0c6e8f337e0753aa9` |
| <p><strong><code>us-east-2</code></strong><br>US East (Ohio)</p>                | `vpce-00ba42c4e2be20721` |
| <p><strong><code>us-west-2</code></strong><br>US West (Oregon)</p>              | `vpce-029306c6a510f7b79` |

2. Identify your [private access level](https://docs.databricks.com/en/security/network/classic/private-access-settings.html) (either `ACCOUNT` or `ENDPOINT`) and configure your Databricks workspace accordingly.
   1. If the `private_access_level` on your `private_access_settings` object is set to `ACCOUNT`, no additional configuration is required.
   2. If the `private_access_level` on your `private_access_settings` object is set to `ENDPOINT`, using the table above, you will need to add it to the `allowed_vpc_endpoint_ids` list inside your `private_access_settings` object in Databricks. For example,

```json
"private_access_settings_name": "immuta-access",
"region": "us-east-1",
"public_access_enabled": false,
"private_access_level": "ENDPOINT",
"allowed_vpc_endpoint_ids": [
        "vpce-0fe5b17a0707d6fa5"
]
```

3. Configure Databricks depending on your integration type:
   1. [Configure the Databricks Unity Catalog integration](https://documentation.immuta.com/saas/configuration/integrations/databricks/databricks-unity-catalog/how-to-guides/connect-unity-catalog) using your standard `cloud.databricks.com` workspace URL as the **Host**.
   2. Configure the [Databricks Spark integration](https://documentation.immuta.com/saas/configuration/integrations/databricks/databricks-spark/how-to-guides/simplified) using your standard `cloud.databricks.com` URL. And [register your tables as Immuta data sources](https://documentation.immuta.com/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/databricks-tutorial) using the `cloud.databricks.com` as the **Server** when registering data sources.