All pages
Powered by GitBook
1 of 17

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Data Connection Private Networking

Learn about the private connectivity options for data platforms and APIs

These sections contain information about private connectivity options from Immuta to various data platforms and APIs.

Data platform connectivity guides

  • AWS PrivateLink for API gateway

  • AWS PrivateLink for Redshift

  • AWS PrivateLink for PostgreSQL

  • AWS PrivateLink for S3

  • Databricks private connectivity

  • GCP BigQuery Private Service Connect

  • Snowflake private connectivity

  • Starburst (Trino) private connectivity

  • Azure Private Link for Azure Synapse dedicated SQL pools

AWS PrivateLink for S3

Understand how Immuta uses S3 Gateway VPC endpoints to establish private connectivity for all S3 services in AWS

The Immuta SaaS platform uses AWS PrivateLink for S3 to provide private connectivity for all S3 services in AWS. Immuta is utilizing S3 Gateway VPC endpoints for private connectivity from all SaaS networks, so connections to S3 connect over PrivateLink by default without any extra configuration. This feature is supported in all regions across Immuta's global segments (NA, EU, and AP).

If you have bucket policies limiting access to specific VPCs or have any questions about availability, contact your Immuta representative for more information about configuration (e.g., VPC ID) to be able to configure your bucket policy.

Databricks Private Connectivity

Understand the private connectivity options for Databricks integrations

This section contains information about private connectivity options for Databricks integrations.

Overview

The Immuta SaaS platform supports private connectivity to Databricks on AWS and the Azure Databricks Service. This allows organizations to meet security and compliance controls by ensuring that traffic to data sources from Immuta SaaS only traverses private networks, never the public internet.

  • Support for AWS PrivateLink is available in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

  • Support for Azure Private Link is available in all Databricks-supported Azure regions.

Configuration guides

  • AWS PrivateLink

  • Azure Private Link

GCP BigQuery Private Service Connect

Understand how Immuta uses Private Service Connect to establish private connectivity to Google APIs

The Immuta SaaS platform uses Private Service Connect for all BigQuery connections through the Immuta Private Cloud Exchange. This means that all connectivity to Google APIs, including BigQuery, are over a private network connection. This functionality is enabled by default and no customer configuration is required for the private connectivity (unless you have VPC Service Controls enabled). This feature is supported in all regions across Immuta's global segments (NA, EU, and AP).

VPC Service Controls

If you have VPC Service Controls (VPC-SCs) enabled for BigQuery, contact your Immuta representative for the VPC information required to configure resource access restrictions. If you have VPC-SCs configured without Immuta's VPC information, Immuta SaaS will be unable to connect to your Google BigQuery instances.

AWS PrivateLink for API Gateway

Set up private connectivity between Immuta SaaS and AWS API Gateway endpoints

Private preview: This feature is available to select accounts. Contact your Immuta representative for details.

AWS PrivateLink provides private connectivity from the Immuta SaaS platform to API Gateway endpoints hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

This feature is supported in all regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

Requirements

  • You have an Immuta SaaS tenant.

  • You have an Amazon API Gateway private API.

  • Your private API must exist in one of the regions in our global segments.

Configuring API Gateway with AWS PrivateLink

  1. Update your API Gateway resource policy to allow for access from the Immuta VPC endpoint in the applicable AWS region. The Immuta VPC endpoint IDs are listed in the table below.

AWS region
VPC endpoint ID

ap-northeast-1 Asia Pacific (Tokyo)

vpce-09b3a20743b64ecc9

ap-south-1 Asia Pacific (Mumbai)

vpce-00620d5f59239fa03

ap-southeast-1 Asia Pacific (Singapore)

vpce-0b470f0df2b0e03f3

Here is an example resource policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "execute-api:Invoke",
            "Resource": [
                "execute-api:/*"
            ]
        },
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "execute-api:Invoke",
            "Resource": [
                "execute-api:/*"
            ],
            "Condition" : {
                "StringNotEquals": {
                    "aws:SourceVpce": [
                        "vpce-1a2b3c4d5e6f7g8h9", # customer internal VPC Endpoint
                        "vpce-0421446f7bf694e56"  # Immuta VPC Endpoint added to list
                    ]
                }
            }
        }
    ]

Once you have made changes to your resource policy, you must for the updates to take effect.

  1. You should now be able to connect to your private API from your Immuta SaaS tenant using your API endpoint, i.e. <api-gateway-id>.execute-api.<region>.amazonaws.com/<stage>/<endpoint>.

Troubleshooting

Issue: I received a permissions error when trying to invoke my private API from Immuta

If you get an error similar to the following:

{"Message":"User: anonymous is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:us-east-1:****************/foo/GET/bar with an explicit deny"}

Check to make sure that the following is true:

  • You have authorized the correct VPC endpoint for the region you are targeting in your resource policy.

  • Your resource policy allows for execute-api:Invoke privileges on the endpoint you are making requests to from Immuta.

  • You have deployed your API after making changes to your resource policy.

AWS PrivateLink for Redshift

Set up private connectivity between Immuta SaaS and Redshift clusters hosted on AWS

AWS PrivateLink provides private connectivity from the Immuta SaaS platform to Redshift clusters hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

Requirements

  • You have an Immuta SaaS tenant.

  • You have set up an AWS PrivateLink Service for your Redshift Cluster endpoints.

    • When creating the service, make sure that the Require Acceptance option is checked (this does not allow anyone to connect, all connections will be blocked until the Immuta service principal is added).

Configure Redshift with AWS PrivateLink

  1. Open a support ticket with Immuta Support with the following information:

    • AWS region

    • AWS subnet availability zones IDs (e.g. use1-az3; these are not the account-specific identifiers like us-east-1a or eu-west-2c)

    • VPC endpoint service ID (e.g., vpce-0a02f54c1d339e98a)

    • Ports used

  2. provided by your representative so that Immuta can complete the VPC endpoint configuration.

  3. .

  4. .

AWS PrivateLink for PostgreSQL

Set up private connectivity between Immuta SaaS and PostgreSQL database engines on Amazon Aurora and Amazon RDS

AWS PrivateLink provides private connectivity from the Immuta SaaS platform to PostgreSQL database engines hosted on Amazon Aurora and Amazon RDS. It ensures that all traffic to the configured endpoints only traverses private networks.

This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

Requirements

  • You have an Immuta SaaS tenant.

  • You have set up an AWS PrivateLink Service for your Amazon Aurora/Amazon RDS endpoints.

    • When creating the service, make sure that the Require Acceptance option is checked (this does not allow anyone to connect, all connections will be blocked until the Immuta service principal is added).

Like all data connection private networking integrations that Immuta offers for AWS, the Amazon Aurora/Amazon RDS PrivateLink integration relies on a Network Load Balancer (NLB) that targets a private IP address - in this case, the private IP address of the Aurora/RDS instance. In a Multi-AZ configuration, the primary instance's private IP address changes during a failover event.

The NLB will not automatically detect this new IP address. Consequently, after an RDS failover, Immuta will be unable to connect to the database until the NLB's target group is updated with the new primary instance's private IP address.

To avoid the need for manual updates to your NLB configuration, you must implement an automated solution. A common approach is to use an AWS Lambda function that listens for RDS failover events and automatically updates the NLB target group with the new IP address. For a detailed guide on this automation, refer to the AWS blog post:

Configure PostgreSQL with AWS PrivateLink

  1. Open a support ticket with Immuta Support with the following information:

    • AWS region

    • AWS subnet availability zones IDs (e.g. use1-az3; these are not the account-specific identifiers like us-east-1a or eu-west-2c)

    • VPC endpoint service ID (e.g., vpce-0a02f54c1d339e98a)

    • Ports used

  2. provided by your representative so that Immuta can complete the VPC endpoint configuration.

  3. .

AWS PrivateLink for Databricks

Set up private connectivity between Immuta SaaS and Databricks accounts hosted on AWS

AWS PrivateLink provides private connectivity from the Immuta SaaS platform to Databricks accounts hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

This front-end PrivateLink connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over a VPC interface endpoint. For details about AWS PrivateLink in Databricks and the network flow in a typical implementation, explore the Databricks documentation.

This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

Requirements

Databricks

Ensure that your accounts meet the following requirements:

  • Your Databricks account is on the E2 version of the platform.

  • Your Databricks account is on the .

  • You have your Databricks account ID from the .

Databricks workspace

Ensure that your workspace meets the following requirements:

  • Your workspace must be in an .

  • Your Databricks workspace must use a to add any PrivateLink connection.

  • Your workspaces must be .

Enablement

Contact your Databricks representative to enable AWS PrivateLink on your account.

Configure Databricks with AWS PrivateLink

  1. for the applicable AWS region with your Databricks workspaces. The Immuta VPC endpoint IDs are listed in the table below.

AWS region
VPC endpoint ID
  1. Identify your (either ACCOUNT or ENDPOINT) and configure your Databricks workspace accordingly.

    1. If the private_access_level on your private_access_settings object is set to ACCOUNT, no additional configuration is required.

  1. Configure Databricks depending on your integration type:

    1. using your standard cloud.databricks.com workspace URL as the Host.

    2. Configure the using your standard cloud.databricks.com URL. And

GCP Private Service Connect for Databricks

Set up private connectivity between Immuta SaaS and Databricks accounts hosted on Google Cloud Platform

Private preview: This feature is available to select accounts. Contact your Immuta representative for details.

GCP Private Service Connect provides private connectivity from the Immuta SaaS platform to Databricks accounts hosted on Google Cloud Platform (GCP). It ensures that all traffic to the configured endpoints only traverses private networks over the Immuta private cloud exchange. This front-end Private Service Connect connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over a VPC endpoint.

Requirements

Ensure that your accounts meet the following requirements:

  • You have an Immuta SaaS tenant.

  • Your Databricks workspace is hosted on Google Cloud Platform (GCP) that has been created with private access configured.

  • You have your Databricks Enterprise account ID.

This process will require configuring a service account in GCP and administrative access to the Databricks account in GCP. For details about Databricks authentication with Google Identity, see the Databricks documentation on Google ID authentication.

Configure Databricks with GCP Private Service Connect

Follow these steps to establish private connectivity between Immuta and your Databricks environment:

  1. Create a service account in GCP. Ensure that a principal (either a user or a different service account) has the roles/iam.serviceAccountTokenCreator role attached for this newly created service account. For more information, refer to the GCP documentation on service account impersonation.

  2. Add the newly created service account email to your Databricks account with admin rights to be able to add network endpoints. For guidance, see the Databricks documentation on adding user accounts.

  3. Open an Immuta support ticket and provide the following information:

    • Service account email

    • Databricks account ID (See )

    • GCP region(s) and workspace URLs in each region

  4. Immuta will create the Private Service Connect (PSC) endpoints in the different regions that contain your workspaces and attach a role to the provided service account that allows it to view the created VPC endpoints. Immuta will then provide you the following details:

    • VPC endpoint ID and region

    • Immuta project ID

  5. Run the script (or manually make the ) to connect the Immuta-created PSC endpoints to your Databricks account using all of the information provided by the Immuta support team.

    To run the script, you will need to have gcloud, curl, and jq installed and be logged in with a principal that can impersonate the service account that was provided to Immuta.

#!/bin/bash

# Script to connect a GCP PSC endpoint to a Databricks account by
# impersonating a service account and retrieving tokens.
#
# Usage:
#   ./accept-databricks-psc.sh -s SERVICE_ACCOUNT_EMAIL \
#       -d DATABRICKS_ACCOUNT_ID \
#       -e ENDPOINT_NAME \
#       -r ENDPOINT_REGION \
#       -p PROJECT_ID \
#       [OPTIONS]
#
# Example:
#   ./accept-databricks-psc.sh -s [email protected] \
#       -d 12345678-90ab-cdef-1234-567890abcdef \
#       -e dbx-company-project-region \
#       -r us-east4 \
#       -p my-gcp-project \
#       -j # (with JSON logging)

set -euo pipefail

# Default values
LOG_FORMAT="text"

# Function to display usage
usage() {
    cat << EOF
Usage: $0 -s SERVICE_ACCOUNT_EMAIL [OPTIONS]

Verify gcloud setup and service account impersonation, retrieving access and ID tokens.

Required:
    -s SERVICE_ACCOUNT        Service account email to impersonate
    -d DATABRICKS_ACCOUNT_ID  Databricks account ID (UUID format)
    -e ENDPOINT_NAME          Name of the VPC endpoint to attach (e.g., "dbx-company-project-region")
    -r ENDPOINT_REGION        Region where the VPC endpoint is located (e.g., "us-east4")
    -p PROJECT_ID             GCP project ID where the VPC endpoint is located (Immuta's SaaS project ID)
Options:
    -h                              Show this help message
    -j                              Enable JSON logging format
    -D DATABRICKS_ENDPOINT_NAME     Databricks Endpoint Name (defaults to "immuta-<region>-psc-endpoint")

Example:
    # Basic usage
    $0 -s [email protected] \
       -d 12345678-90ab-cdef-1234-567890abcdef \
       -e dbx-company-project-region \
       -r us-east4 \
       -p immuta-gcp-project

EOF
}

DATABRICKS_ENDPOINT_NAME=""

# Parse command line arguments
while getopts "s:d:e:r:p:D:jh" opt; do
  case ${opt} in
    s)
      SERVICE_ACCOUNT="${OPTARG}"
      ;;
    d)
      DATABRICKS_ACCOUNT_ID="${OPTARG}"
      ;;
    e)
      ENDPOINT_NAME="${OPTARG}"
      ;;
    r)
      ENDPOINT_REGION="${OPTARG}"
      ;;
    p)
      PROJECT_ID="${OPTARG}"
      ;;
    D)
      DATABRICKS_ENDPOINT_NAME="${OPTARG}"
      ;;
    j)
      LOG_FORMAT="json"
      ;;
    h)
      usage
      exit 0
      ;;
    \?)
      echo "Error: Invalid option -${OPTARG}" >&2
      echo "Run '$0 -h' for usage information"
      exit 1
      ;;
    :)
      echo "Error: Option -${OPTARG} requires an argument" >&2
      exit 1
      ;;
  esac
done

# Check if service account was provided
if [[ -z "${SERVICE_ACCOUNT:-}" ]]; then
  echo "Error: Service account email is required"
  usage
  exit 1
fi

# Check if Databricks account ID was provided
if [[ -z "${DATABRICKS_ACCOUNT_ID:-}" ]]; then
  echo "Error: Databricks account ID is required"
  usage
  exit 1
fi

# Check if VPC endpoint name was provided
if [[ -z "${ENDPOINT_NAME:-}" ]]; then
  echo "Error: VPC endpoint name is required"
  usage
  exit 1
fi

# Check if VPC endpoint region was provided
if [[ -z "${ENDPOINT_REGION:-}" ]]; then
  echo "Error: VPC endpoint region is required"
  usage
  exit 1
fi

# Check if GCP project ID was provided
if [[ -z "${PROJECT_ID:-}" ]]; then
  echo "Error: GCP project ID is required"
  usage
  exit 1
fi

# Check if Databricks endpoint name was provided, if not set default name
if [[ -z "${DATABRICKS_ENDPOINT_NAME:-}" ]]; then
  DATABRICKS_ENDPOINT_NAME="immuta-${ENDPOINT_REGION}-psc-endpoint"
fi

# Function to log with simplified or JSON format
log() {
  local level="${1:-INFO}"
  # Shift the first argument (log level) so that $* does not include the level
  shift || true
  local message="$*"
  local timestamp
  # Use portable date format (macOS date doesn't support %N)
  if date --version &>/dev/null; then
    # GNU date
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%S.%3NZ")
  else
    # BSD date (macOS)
    timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
  fi

  if [[ "${LOG_FORMAT}" == "json" ]]; then
    # JSON format: timestamp, level, and message (properly escaped)
    jq -nc --arg ts "${timestamp}" --arg lvl "${level}" --arg msg "${message}" \
      '{timestamp: $ts, level: $lvl, message: $msg}'
  else
    # Color codes for different log levels
    local color_reset='\033[0m'
    local color_level=""

    case "${level}" in
      INFO)
        color_level='\033[0;36m'  # Cyan
        ;;
      WARN)
        color_level='\033[0;33m'  # Yellow
        ;;
      ERROR)
        color_level='\033[0;31m'  # Red
        ;;
      DEBUG)
        color_level='\033[0;90m'  # Gray
        ;;
      *)
        color_level='\033[0m'     # No color
        ;;
    esac

    # Simple format with color: [timestamp] [level] message
    printf "[%s] [${color_level}%s${color_reset}] %s\n" "${timestamp}" "${level}" "${message}"
  fi
}

if [[ "${LOG_FORMAT}" == "json" ]]; then
  log INFO "Starting Immuta Databricks GCP PSC Acceptance Script"
else
  log INFO "================================================"
  log INFO "Immuta Databricks GCP PSC Acceptance Script"
  log INFO "================================================"
fi
log INFO "Service Account: ${SERVICE_ACCOUNT}"
log INFO "Databricks Account ID: ${DATABRICKS_ACCOUNT_ID}"
log INFO "VPC Endpoint Name: ${ENDPOINT_NAME}"
log INFO "VPC Endpoint Region: ${ENDPOINT_REGION}"
log INFO "Databricks Endpoint Name: ${DATABRICKS_ENDPOINT_NAME}"

# Step 1: Check if gcloud, curl, and jq are installed
log INFO "[1/6] Checking if dependencies are installed..."
if ! command -v gcloud &> /dev/null; then
  log ERROR "❌ ERROR: gcloud CLI is not installed"
  log ERROR "   Please install it from: https://cloud.google.com/sdk/docs/install"
  exit 1
fi
if ! command -v curl &> /dev/null; then
  log ERROR "❌ ERROR: curl is not installed"
  log ERROR "   Please install it using your package manager (e.g., apt, yum, brew)"
  exit 1
fi
if ! command -v jq &> /dev/null; then
  log ERROR "❌ ERROR: jq is not installed"
  log ERROR "   Please install it using your package manager (e.g., apt, yum, brew)"
  exit 1
fi
GCLOUD_VERSION=$(gcloud version --format="value(core)" 2>/dev/null || echo "unknown")
log INFO "✅ gcloud is installed (version: ${GCLOUD_VERSION})"
if [[ $LOG_FORMAT != "json" ]]; then log INFO ""; fi

# Step 2: Check if user is logged in
log INFO "[2/6] Checking if you are logged in to gcloud..."
CURRENT_ACCOUNT=$(gcloud config get-value account 2>/dev/null || echo "")
if [[ -z "${CURRENT_ACCOUNT}" ]]; then
  log ERROR "❌ ERROR: Not logged in to gcloud"
  log ERROR "   Run: gcloud auth login"
  exit 1
fi
log INFO "✅ Logged in as: ${CURRENT_ACCOUNT}"
if [[ $LOG_FORMAT != "json" ]]; then log INFO ""; fi

# Step 3: Verify impersonation by getting an access token
log INFO "[3/6] Verifying service account impersonation (access token)..."
set +e  # Temporarily disable exit on error
GCLOUD_OUTPUT=$(gcloud auth print-access-token \
  --impersonate-service-account="${SERVICE_ACCOUNT}" \
  --verbosity=error 2>&1)
GCLOUD_EXIT=$?
set -e  # Re-enable exit on error

if [[ ${GCLOUD_EXIT} -ne 0 ]]; then
  log ERROR "Failed to impersonate service account for access token"
  log ERROR "${GCLOUD_OUTPUT}"
  # while IFS= read -r line; do
  #   [[ -n "${line}" ]] && log ERROR "${line}"
  # done <<< "${GCLOUD_OUTPUT}"
  exit 1
fi
ACCESS_TOKEN="${GCLOUD_OUTPUT}"
log INFO "✅ Successfully obtained access token"
if [[ $LOG_FORMAT != "json" ]]; then log INFO ""; fi

# Step 4: Get an ID token
log INFO "[4/6] Getting ID token for service account..."
set +e  # Temporarily disable exit on error
GCLOUD_OUTPUT=$(gcloud auth print-identity-token \
  --impersonate-service-account="${SERVICE_ACCOUNT}" \
  --include-email \
  --verbosity=error \
  --audiences="https://accounts.gcp.databricks.com" 2>&1)
GCLOUD_EXIT=$?
set -e  # Re-enable exit on error

if [[ ${GCLOUD_EXIT} -ne 0 ]]; then
  log ERROR "Failed to get ID token"
  while IFS= read -r line; do
    [[ -n "${line}" ]] && log ERROR "${line}"
  done <<< "${GCLOUD_OUTPUT}"
  exit 1
fi
ID_TOKEN="${GCLOUD_OUTPUT}"
log INFO "✅ Successfully obtained ID token"
if [[ $LOG_FORMAT != "json" ]]; then log INFO ""; fi

# Step 5: List existing VPC endpoints
log INFO "[5/6] Listing existing VPC endpoints..."
if ! RESULT=$(curl -s -XGET \
  --header "Authorization: Bearer ${ID_TOKEN}" \
  "https://accounts.gcp.databricks.com/api/2.0/accounts/${DATABRICKS_ACCOUNT_ID}/vpc-endpoints"); then
  log ERROR "❌ ERROR: Failed to call Databricks API to list VPC endpoints"
  log ERROR "   ${RESULT}"
  exit 1
fi
echo "${RESULT}" | jq -er '.[] | select(.gcp_vpc_endpoint_info.psc_endpoint_name == "'"${ENDPOINT_NAME}"'")' > /dev/null && {
  log ERROR "❌ Existing VPC endpoint found with name '${ENDPOINT_NAME}'"
  log ERROR "    If this is unexpected, please delete the existing endpoint in the Databricks console and try again"
  exit 1
}
log INFO "✅ No existing VPC endpoint with name '${ENDPOINT_NAME}' found"
if [[ $LOG_FORMAT != "json" ]]; then log INFO ""; fi

log INFO "[6/6] Creating VPC endpoint attachment..."
REQUEST=$(cat <<EOF
{
  "gcp_vpc_endpoint_info": {
    "endpoint_region": "${ENDPOINT_REGION}",
    "project_id": "${PROJECT_ID}",
    "psc_endpoint_name": "${ENDPOINT_NAME}"
  },
  "vpc_endpoint_name": "${DATABRICKS_ENDPOINT_NAME}"
}
EOF
)
RESULT=$(curl -s -XPOST \
  -d "${REQUEST}" \
  --header "Content-Type: application/json" \
  --header "Authorization: Bearer ${ID_TOKEN}" \
  --header "X-Databricks-GCP-SA-Access-Token: ${ACCESS_TOKEN}" \
  "https://accounts.gcp.databricks.com/api/2.0/accounts/${DATABRICKS_ACCOUNT_ID}/vpc-endpoints")
if [[ "$(echo "${RESULT}" | jq -r '.error_code // empty')" != "" ]]; then
  log ERROR "❌ ERROR: Failed to create VPC endpoint attachment"
  log ERROR "   ${RESULT}"
  exit 1
fi


# Output based on mode
if [[ "${LOG_FORMAT}" == "json" ]]; then
  log INFO "✅ VPC Endpoint Attachment Created"
else
  # Display token information
  log INFO "================================================"
  log INFO "✅ SUCCESS - VPC Endpoint Attachment Created"
  log INFO "================================================"
  log INFO "The Endpoint has been attached to your account in Databricks."
fi
  1. Validate that any private access settings attached to your workspaces that need connectivity have the newly created endpoints added (either accepting the Account level or adding the specific endpoints to the private access setting).

After these steps, you should be able to connect your Immuta tenant to Databricks workspaces in GCP under the connected account.

Authorize the service principal
Configure the Redshift integration
Register your tables as Immuta data sources
Locate your Databricks account ID
necessary API calls to Databricks

ap-southeast-2 Asia Pacific (Sydney)

vpce-0afc6a24f0959847c

ca-central-1 Canada (Central)

vpce-07dfc91c761a8f2f9

eu-central-1 Europe (Frankfurt)

vpce-04bc9a3cd6020a865

eu-west-1 Europe (Ireland)

vpce-079feae086b944dad

eu-west-2 Europe (London)

vpce-091d282f539081cf5

us-east-1 US East (Virginia)

vpce-0421446f7bf694e56

us-east-2 US East (Ohio)

vpce-071ef6403fa277210

us-west-2 US West (Oregon)

vpce-01f8edfbf6da1095d

deploy your API
.
Authorize the service principal
Register the PostgreSQL connection
Access Amazon RDS across VPCs using AWS PrivateLink and Network Load Balancer
You have an Immuta SaaS tenant.
  • AWS PrivateLink for Databricks has been enabled.

  • vpce-07e9890053f5084b2

    ap-southeast-2 Asia Pacific (Sydney)

    vpce-0d363d9ea82658bec

    ca-central-1 Canada (Central)

    vpce-01933bcf30ac4ed19

    eu-central-1 Europe (Frankfurt)

    vpce-0048e36edfb27d0aa

    eu-west-1 Europe (Ireland)

    vpce-0783d9412b046df1f

    eu-west-2 Europe (London)

    vpce-0f546cc413bf70baa

    us-east-1 US East (Virginia)

    vpce-0c6e8f337e0753aa9

    us-east-2 US East (Ohio)

    vpce-00ba42c4e2be20721

    us-west-2 US West (Oregon)

    vpce-029306c6a510f7b79

    If the private_access_level on your private_access_settings object is set to ENDPOINT, using the table above, you will need to add it to the allowed_vpc_endpoint_ids list inside your private_access_settings object in Databricks. For example,

    using the
    cloud.databricks.com
    as the
    Server
    when registering data sources.

    ap-northeast-1 Asia Pacific (Tokyo)

    vpce-08cadda15f0f70462

    ap-northeast-2 Asia Pacific (Seoul)

    vpce-0e45ce26a7f8d00af

    ap-south-1 Asia Pacific (Mumbai)

    vpce-0efef886a4fbd9532

    You cannot configure a connection to your workspace over the public internet if PrivateLink is enabled.

    If you have PrivateLink configured on your workspace, Databricks will update the DNS records for that workspace URL to resolve to <region>.privatelink.cloud.databricks.com. Immuta SaaS uses these publicly-resolvable records to direct traffic to a PrivateLink endpoint on our network.

    This means that if you have PrivateLink enabled on your workspace, you must follow these instructions to configure your integration. Even if your workspace is also publicly-routable, Databricks's DNS resolution forces the traffic over PrivateLink.

    The two supported configurations are

    • A workspace with no PrivateLink configuration, which resolves to public IP addresses.

    • A workspace with PrivateLink configuration, which allows access from the Immuta SaaS regional endpoint (listed below).

    Enterprise pricing tier
    account console
    AWS region that supports the E2 version of the platform
    customer-managed VPC
    configured with private_access_settings objects
    Register the Immuta VPC endpoint
    private access level
    Configure the Databricks Unity Catalog integration
    Databricks Spark integration
    register your tables as Immuta data sources

    ap-southeast-1 Asia Pacific (Singapore)

    "private_access_settings_name": "immuta-access",
    "region": "us-east-1",
    "public_access_enabled": false,
    "private_access_level": "ENDPOINT",
    "allowed_vpc_endpoint_ids": [
            "vpce-0fe5b17a0707d6fa5"
    ]

    Snowflake Private Connectivity

    Understand the private connectivity options for Snowflake integrations

    This section contains information about private connectivity options for Snowflake integrations.

    Overview

    The Immuta SaaS platform supports private connectivity to Snowflake accounts hosted in both AWS and Azure. This allows organizations to meet security and compliance controls by ensuring that traffic to data sources from Immuta SaaS only traverses private networks, never the public internet.

    • Support for AWS PrivateLink is available in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

    • Support for Azure Private Link is available in .

    Configuration guides

    Starburst (Trino) Private Connectivity

    Understand the private connectivity options for Starburst (Trino) integrations

    This section contains information about private connectivity options for Starburst (Trino) integrations.

    Overview

    The Immuta SaaS platform supports private connectivity to Starburst (Trino) clusters hosted in both AWS and Azure. This allows organizations to meet security and compliance controls by ensuring that traffic to data sources from Immuta SaaS only traverses private networks, never the public internet.

    • Support for AWS PrivateLink is available in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

    • Support for Azure Private Link is available in .

    Configuration guides

    Azure Private Link for Starburst (Trino)

    Set up private connectivity between Immuta SaaS and Starburst (Trino) clusters on Azure

    Azure Private Link provides private connectivity from the Immuta SaaS platform, hosted on AWS, to Starburst (Trino) clusters on Azure. It ensures that all traffic to the configured endpoints only traverses private networks over the Immuta Private Cloud Exchange.

    Support for Azure Private Link is available in all Azure regions.

    Prerequisites

    • You have an Immuta SaaS tenant.

    • Your Starburst (Trino) cluster is hosted on Azure.

    • You have set up an for your Starburst cluster.

      • The Private Link Service's should be set to Restricted by Subscription.

    Configure Starburst (Trino) with Azure Private Link

    1. Open a support ticket with with the following information:

      • Azure region

      • Azure Private Link service resource ID or alias

    Azure Private Link for Azure Synapse Dedicated SQL Pools

    Set up private connectivity between Immuta SaaS and Azure Synapse workspaces

    Public preview: This feature is available to all accounts that request to enable it for their tenant. Contact your Immuta representative to enable it.

    Azure Private Link provides private connectivity from the Immuta SaaS platform, hosted on AWS, to Azure Synapse workspaces. It ensures that all traffic to the configured endpoints only traverses private networks over the Immuta private cloud exchange.

    This uses the Sql Private Link connection to allow users to connect to the Azure Synapse dedicated SQL pools over an Azure Private Endpoint. For details about Azure Private Link for Azure Synapse and the network flow in a typical implementation, see the Azure Synapse documentation.

    Support for Azure Private Link is available in all Synapse-supported Azure regions.

    Requirements

    Ensure that your accounts meet the following requirements:

    • You have an Immuta SaaS tenant.

    • You have an Azure Synapse workspace with a dedicated SQL pool.

    Configure Synapse workspace with Azure Private Link

    1. Contact your Immuta representative and provide the following information for each Azure Synapse workspace you want to connect to:

      • Azure region

      • Azure Synapse workspace SQL

    AWS PrivateLink for Snowflake

    Set up private connectivity between Immuta SaaS and Snowflake accounts hosted on AWS

    provides private connectivity from the Immuta SaaS platform to Snowflake accounts hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

    This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta representative if you have questions about availability.

    Requirements

    AWS PrivateLink for Starburst (Trino)

    Set up private connectivity between Immuta SaaS and Starburst (Trino) clusters hosted on AWS

    AWS PrivateLink provides private connectivity from the Immuta SaaS platform to Starburst (Trino) Clusters hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

    This feature is supported in most regions across Immuta's global segments (NA, EU, and AP); contact your Immuta account manager if you have questions about availability.

    Requirements

    all Snowflake-supported Azure regions
    AWS PrivateLink
    Azure Private Link
    all Azure regions
    AWS PrivateLink
    Azure Private Link
    DNS hostname
  • Your Immuta representative will provide you with the Immuta subscription ID that needs to be authorized to consume the service.

  • Once the Immuta Azure subscription is authorized, inform your representative so that Immuta can complete Private Link endpoint configuration.

  • Your representative will inform you when the two Azure Private Link connections have been made available. Accept them in the Private Link Center of your Azure Portal.

  • Configure the Starburst (Trino) integration.

  • Register your tables as Immuta data sources.

  • Azure Private Link Service
    Access Security
    Immuta Support
    Azure Synapse workspace resource ID or alias (can be found in the console by going to the Synapse workspace -> Settings -> Properties and getting the Resource ID)
  • Your representative will inform you when the two Azure Private Link connections have been made available. Accept them in your Azure Synapse workspace configuration.

  • Configure the Azure Synapse data source using your standard sql.azuresynapse.net URL for the dedicated SQL pool.

  • hostname

    You have an Immuta SaaS tenant.

  • Your Snowflake account is hosted on AWS.

  • Your Snowflake account is on the Business Critical Edition.

  • You have ACCOUNTADMIN role on your Snowflake account to configure the Private Link connection.

  • You have enabled AWS PrivateLink for Snowflake.

  • Using Snowflake network policies with AWS PrivateLink

    Snowflake network policies allow you to limit access to your Snowflake service endpoints. Network rules can be used with those network policies to define the specific IP CIDR blocks or AWS VPC endpoints that are allowed. Immuta supports both, but we highly recommend that you configure your network rules to reference our VPC endpoints and not our CIDR block.

    VPC endpoint network rule

    With a network rule type of AWSVPCEID, you can use the following table of Immuta's VPC endpoints by AWS region to configure access from Immuta SaaS to your Snowflake service:

    AWS region
    VPC endpoint ID

    ap-northeast-1 Asia Pacific (Tokyo)

    vpce-0c738d241aa0bfde7

    ap-northeast-2 Asia Pacific (Seoul)

    vpce-00daddfa7477666eb

    ap-south-1 Asia Pacific (Mumbai)

    vpce-08a6d075ddd92df58

    IPv4 network rule

    With a network rule type of IPV4, you must configure an IP address block of 10.0.0.0/8.

    This size of block is required because traffic could come from anywhere in Immuta's network. Immuta has globally distributed compute and does not assign static IP addresses to any workloads. This is why you should use VPC endpoint network rules instead.

    Configure Snowflake with AWS PrivateLink

    1. In your Snowflake environment, run the following SQL query, which will return a JSON object with the connection information you will need to include in your support ticket:

      select SYSTEM$GET_PRIVATELINK_CONFIG()
    2. Copy the returned JSON object into a support ticket with Immuta Support to request for the feature to be enabled on your Immuta SaaS tenant.

    3. Configure the Snowflake integration using the privatelink-account-url from the JSON object in step one as the Host.

    AWS PrivateLink

    You have an Immuta SaaS tenant.

  • Your Starburst (Trino) Cluster is hosted on AWS.

  • You have set up an AWS PrivateLink Service for your Starburst Cluster endpoints.

    • When creating the service, make sure that the Require Acceptance option is checked (this does not allow anyone to connect; all connections will be blocked until the Immuta Service Principal is added).

    • Only TCP connections over IPv4 are supported.

  • Your Starburst (Trino) cluster endpoints are secured with TLS certificates for encryption in-transit.

    • The presented certificate must have the Fully-Qualified Domain Name (FQDN) of your cluster hostname as a Subject Alternative Name (SAN). This can either be a wildcard (e.g. *.example.com) or specific to the host (e.g. starburst.example.com)

    • The presented certificate must be issued from a publicly-trusted certificate authority (CA). Private CAs and/or internal top-level domains (TLDs) are not supported.

  • Why are private top-level domains (TLDs) not supported?

    Private TLDs are not allowed for two reasons:

    1. A private TLD cannot have a publicly-trusted certificate authority (CA), which means that Immuta SaaS systems could not verify the certificate and would refuse to send traffic to the PrivateLink endpoint.

    2. Immuta SaaS cannot accept private CAs for private TLDs because there are no means of verifying domain ownership and preventing overlap between an organization's private DNS. For example, if Company A uses a .localTLD, Immuta cannot verify that they own it because it's impossible to have exclusive ownership over a private TLD. If Company B also uses a .local TLD, it becomes impossible to determine what the canonical trust chain for either company's private CA is.

      Immuta also cannot prevent DNS overlap between different tenants. If both companies use starburst-prod.local only one endpoint can be routable from that DNS hostname.

    Configure Starburst (Trino) with AWS PrivateLink

    1. Open a support ticket with Immuta Support with the following information:

      • AWS region

      • AWS subnet availability zones IDs (e.g. use1-az3; these are not the account-specific identifiers like us-east-1a or eu-west-2c)

      • VPC endpoint service ID (e.g., com.amazonaws.vpce.us-east-1.vpce-svc-0471015b106ad1d47)

      • DNS hostname

        • This is the hostname of the cluster that will be used to ingest data sources. It must match the Common Name (CN) or a Subject Alternative Name (SAN) on your cluster's TLS certificate.

      • Ports used

    2. provided by your representative so that Immuta can complete the VPC endpoint configuration.

    3. .

    4. .

    ap-southeast-1 Asia Pacific (Singapore)

    vpce-030933ffc228d94ac

    ap-southeast-2 Asia Pacific (Sydney)

    vpce-0803dc2285d0d695f

    ca-central-1 Canada (Central)

    vpce-0ebff3192617126c9

    eu-central-1 Europe (Frankfurt)

    vpce-07f633ac50bc430c2

    eu-north-1 Europe (Stockholm)

    vpce-05c586fedca0a4112

    eu-west-1 Europe (Ireland)

    vpce-0ac01be5c06a919b0

    eu-west-2 Europe (London)

    vpce-0dd3c340c3dd64a5b

    us-east-1 US East (Virginia)

    vpce-03b3bf4334aa34d88

    us-east-2 US East (Ohio)

    vpce-04fdafe0ed07caace

    us-west-2 US West (Oregon)

    vpce-06624165eaa569250

    Authorize the service principal
    Configure the Starburst (Trino) integration
    Register your tables as Immuta data sources

    Azure Private Link for Databricks

    Set up private connectivity between Immuta SaaS and Azure Databricks accounts

    Azure Private Link provides private connectivity from the Immuta SaaS platform, hosted on AWS, to Azure Databricks accounts. It ensures that all traffic to the configured endpoints only traverses private networks over the Immuta Private Cloud Exchange.

    This front-end Private Link connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over an Azure Private Endpoint. For details about Azure Private Link for Databricks and the network flow in a typical implementation, explore the Databricks documentation.

    Support for Azure Private Link is available in all Databricks-supported Azure regions.

    Requirements

    Ensure that your accounts meet the following requirements:

    • You have an Immuta SaaS tenant.

    • Your Azure Databricks workspace must be on the .

    • has been configured and enabled.

    • You have your Databricks account ID from the

    Configure Databricks with Azure Private Link

    1. Contact your Immuta representative, and provide the following information for each Azure Databricks Workspace you wish to connect to:

      • Azure region

      • Azure Databricks hostname

    .
    Azure Databricks resource ID or alias
  • Your representative will inform you when the two Azure Private Link connections have been made available. Accept them in your Azure Databricks workspace configuration.

  • Configure Databricks depending on your integration type:

    1. Configure the Databricks Unity Catalog integration using your standard cloud.databricks.com workspace URL as the Host.

    2. Configure the Databricks Spark integration using your standard azuredatabricks.net URL. And register your tables as Immuta data sources using the privatelink-account-url from the JSON object in step one as the Server when registering data sources.

  • Premium or Enterprise pricing tier
    Azure Private Link for Databricks
    account console

    Azure Private Link for Snowflake

    Set up private connectivity between Immuta SaaS and Snowflake accounts on Azure

    Azure Private Link provides private connectivity from the Immuta SaaS platform, hosted on AWS, to Snowflake Accounts on Azure. It ensures that all traffic to the configured endpoints only traverses private networks over the Immuta Private Cloud Exchange.

    Support for Azure Private Link is available in all Snowflake-supported Azure regions.

    Requirements

    • You have an Immuta SaaS tenant.

    • Your Snowflake account is hosted on Azure.

    • Your Snowflake account is on the .

    • You have ACCOUNTADMIN role on your Snowflake account to configure the Private Link connection.

    Configure Snowflake with Azure Private Link

    1. In your Snowflake environment, run the following SQL query, which will return a JSON object with the connection information you will need to include in your support ticket:

    2. Copy the returned JSON object into a support ticket with to request for the feature to be enabled on your Immuta SaaS tenant.

    3. Your Immuta representative will work with you to schedule a time in which to accept the connection in your Snowflake account. They will provide you with a SQL query to run using the ACCOUNTADMIN role. The SQL query will be in this format:

    The query should return the following response: Private link access authorized.
  • using the privatelink-account-url from the JSON object in step one as the Host.

  • select SYSTEM$GET_PRIVATELINK_CONFIG()
    SELECT SYSTEM$AUTHORIZE_PRIVATELINK (
    '/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Network/privateEndpoints/abc12345.east-us-2.azure.snowflakecomputing.com-eus2',
      '<ACCESS_TOKEN>'
    );

    Snowflake that an Azure temporary access token be used when configuring the Azure Private Link connection. Due to the constraint imposed by the 1-hour token expiration, your Immuta representative will ask for a time window in which you can accept the connection in your Snowflake account. During this window, the token will be generated by Immuta and provided to you when you're ready to run the following SQL query.

    Business Critical Edition
    Immuta Support
    Configure the Snowflake integration
    requires