# Configure a Google BigQuery Integration

{% hint style="info" %}
**Private preview**: This integration is available to all accounts.
{% endhint %}

{% @mermaid/diagram content="graph LR
A(Authenticate) --> B(Configure integration) --> C(Map users and create policies) --> D(Register metadata);
style A fill:#fff
style B fill:#2F6BA7,color:#fff
style C fill:#fff
style D fill:#fff" %}

The Google BigQuery resource allows you to create, configure, and manage your Google BigQuery integration. In this integration, Immuta generates policy-enforced views in your configured Google BigQuery dataset for tables registered as Immuta data sources.

Use the `/integrations` endpoint to

* [configure a Google BigQuery integration](#configure-the-integration)
* [get a Google BigQuery integration](#get-an-integration)
* [get all integrations](#get-all-integrations)
* [update a Google BigQuery integration](#update-an-integration-configuration)
* [delete a Google BigQuery integration](#delete-an-integration)

## Requirements

* `APPLICATION_ADMIN` Immuta permission
* Google BigQuery integration enabled in Immuta (work with your Immuta representative to enable this integration)
* To execute the Immuta script from your command line to [create a Google Cloud service account and role](#prerequisite), you must be authenticated to the gcloud CLI utility as a user with all of the following roles:
  * `roles/iam.roleAdmin`
  * `roles/iam.serviceAccountAdmin`
  * `roles/serviceusage.serviceUsageAdmin`

## Prerequisite

[Create a Google Cloud service account and role](https://documentation.immuta.com/saas/configuration/integrations/google-bigquery#google-cloud-service-account-and-role-used-by-immuta-to-connect-to-google-bigquery) by either using the Google Cloud console or the provided Immuta script.

## Configure the integration

```bash
curl -X 'POST' \
    'https://www.organization.immuta.com/integrations' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: 846e9e43c86a4ct1be14290d95127d13f' \
    -d '{
    "type": "Google BigQuery",
    "autoBootstrap": false,
    "config": {
      "role": "immuta",
      "datasetSuffix": "_secureView",
      "dataset": "immuta",
      "location": "us-east-1",
      "credential": "{\"type\":\"service_account\",\"project_id\":\"innate-conquest-123456\",\"private_key_id\":\"9163c12345690924f5dd218ff39\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\nXXXXXXXro0s\n/yQlPQijowkccmrmWJyr93kdLnwJzBvLHCto/+W\ncvF2ygX9oM/dyUK//z//4nptMp+Ck//Yw3D4rIBwGu4DWiR1qRnf\nDoGyXfThPTQ==\n-----END PRIVATE KEY-----\n\",\"client_email\":\"service-account-id@innate-conquest-123456.iam.gserviceaccount.com\",\"client_id\":\"1166290***432952487857\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",\"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/service-account-id%40innate-conquest-123456.iam.gserviceaccount.com\",\"universe_domain\":\"googleapis.com\"}"
    }
    }'
```

1. Copy the request example. The example uses JSON format, but the request also accepts YAML.
2. Replace the **Immuta URL** and [**API key**](https://documentation.immuta.com/saas/developer-guides/api-intro/getting-started#authenticate-with-the-api) with your own.
3. Change the **config** values to your own, where
   * **role** is the Google Cloud role used to connect to Google BigQuery.
   * **datasetSuffix** is the suffix to postfix to the name of each dataset created to store secure views. This string must start with an underscore.
   * **dataset** is the name of the BigQuery dataset to provision inside of the project for Immuta metadata storage.
   * **location** is the dataset's location, which can be any valid GCP location (such as `us-east-1`).
   * **credential** is the Google BigQuery service account JSON keyfile credential content. See the [Google documentation](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) for guidance on generating and downloading this keyfile.

See the [**config** object description](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/integration-configuration-payload#google-bigquery-configuration-object) for parameter definitions, value types, and additional configuration options.

### Response

The response returns the status of the Google BigQuery integration configuration connection. See the [response schema reference](https://documentation.immuta.com/saas/developer-guides/api-intro/integrations-api/reference-guides/response-schema) for details about the response schema.

{% tabs %}
{% tab title="200 response" %}
A successful response includes the validation tests statuses.

```json
{
  "id": "123456789",
  "status": "creating",
  "validationResults": {
    "status": "passed",
    "validationTests": [
    {
      "name": "Basic Validation: Verify service account not being used for data source connection credentials",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Immuta Service Account postfix",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Non-matching service account in key file",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Connection can be made to BigQuery",
      "status": "passed"
    },
    {
      "name": "Initialize Validation: [Dataset - exists] innate-conquest-123456.immuta",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - insert] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "warning",
      "message": "Billing has not been enabled for this project. Enable billing at https://console.cloud.google.com/billing. DML queries are not allowed in the free tier. Set up a billing account to remove this restriction. Confirm that the bootstrap script was executed without any modifications. This warning will not prevent the connection but continue at your own risk as some features may not function."
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After insert)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - update] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After update)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    }
    ]
  }
}
```

{% endtab %}

{% tab title="Unsuccessful request" %}
An unsuccessful request returns the status code and an error message. See the [HTTP status codes and error messages](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/status-codes#post-errors) for a list of statuses, error messages, and troubleshooting guidance.

```json
{
  "statusCode": 409,
  "error": "Conflict",
  "message": "Google BigQuery integration already exists on host organization.cloud.google.com (id = 123456789)"
}
```

{% endtab %}
{% endtabs %}

## Get an integration

```bash
curl -X 'GET' \
    'https://www.organization.immuta.com/integrations/{id}' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: 846e9e43c86a4ct1be14290d95127d13f'
```

1. Copy the request example.
2. Replace the **Immuta URL** and [**API key**](https://documentation.immuta.com/saas/developer-guides/api-intro/getting-started#authenticate-with-the-api) with your own.
3. Replace the **{id}** request parameter with the unique identifier of the integration you want to get. Alternatively, you can get a list of all integrations and their IDs with the [<mark style="color:blue;">`GET /integrations`</mark> <mark style="color:blue;">endpoint</mark>](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/integrations-api#get-integrations).

### Response

The response returns a Google BigQuery integration configuration. See the [response schema reference](https://documentation.immuta.com/saas/developer-guides/api-intro/integrations-api/reference-guides/response-schema) for details about the response schema. An unsuccessful response returns the status code and an error message. See the [HTTP status codes and error messages](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/status-codes#get-error) for a list of statuses, error messages, and troubleshooting guidance.

```json
{
  "id": "123456789",
  "status": "enabled",
  "validationResults": {
    "status": "passed",
    "validationTests": [
    {
      "name": "Basic Validation: Verify service account not being used for data source connection credentials",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Immuta Service Account postfix",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Non-matching service account in key file",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Connection can be made to BigQuery",
      "status": "passed"
    },
    {
      "name": "Initialize Validation: [Dataset - exists] innate-conquest-123456.immuta",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - insert] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "warning",
      "message": "Billing has not been enabled for this project. Enable billing at https://console.cloud.google.com/billing. DML queries are not allowed in the free tier. Set up a billing account to remove this restriction. Confirm that the bootstrap script was executed without any modifications. This warning will not prevent the connection but continue at your own risk as some features may not function."
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After insert)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - update] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After update)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    }
    ]
  },
  "type": "Google BigQuery",
  "autoBootstrap": false,
  "config": {
    "port": 443,
    "role": "immuta",
    "datasetSuffix": "_secureView",
    "dataset": "immuta",
    "location": "us-east-1",
    "credential": {
      "type": "service_account",
      "project_id": "innate-conquest-123456",
      "private_key_id": "9163c12345690924f5dd218ff39",
      "private_key": "-----BEGIN PRIVATE KEY-----\nXXXXXXXro0s\n/yQlPQijowkccmrmWJyr93kdLnwJzBvLHCto/+W\ncvF2ygX9oM/dyUK//z\//4nptMp+Ck//Yw3D4rIBwGu4DWiR1qRnf\nDoGyXfThPTQ==\n-----END PRIVATE KEY-----\n",
      "client_email": "service-account-id@innate-conquest-123456.iam.gserviceaccount.com",
      "client_id": "1166290***432952487857",
      "auth_uri": "https://accounts.google.com/o/oauth2/auth",
      "token_uri": "https://oauth2.googleapis.com/token",
      "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
      "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-id%40innate-conquest-123456.iam.gserviceaccount.com",
      "universe_domain": "googleapis.com"
    }
  }
  }
```

## Get all integrations

```bash
curl -X 'GET' \
    'https://www.organization.immuta.com/integrations' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: 846e9e43c86a4ct1be14290d95127d13f'
```

1. Copy the request example.
2. Replace the **Immuta URL** and [**API key**](https://documentation.immuta.com/saas/developer-guides/api-intro/getting-started#authenticate-with-the-api) with your own.

### Response

The response returns the configuration for all integrations. See the [response schema reference](https://documentation.immuta.com/saas/developer-guides/api-intro/integrations-api/reference-guides/response-schema) for details about the response schema. An unsuccessful request returns the status code and an error message. See the [HTTP status codes and error messages](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/status-codes#get-error) for a list of statuses, error messages, and troubleshooting guidance.

```json
[
  {
    "id": "1",
    "status": "enabled",
    "validationResults": {
      "status": "passed",
      "validationTests": [
        {
          "name": "Initial Validation: Basic Connection Test",
          "status": "passed"
        },
        {
          "name": "Initial Validation: Default Warehouse Access Test",
          "status": "passed",
          "result": []
        },
        {
          "name": "Initial Validation: Validate access to Privileged Role",
          "status": "passed",
          "result": []
        },
        {
          "name": "Validate Automatic: Database Does Not Exist",
          "status": "passed"
        },
        {
          "name": "Validate Automatic: Impersonation Role Does Not Exist",
          "status": "skipped"
        },
        {
          "name": "Validate Automatic Bootstrap User Grants",
          "status": "passed"
        }
      ]
    },
    "type": "Snowflake",
    "autoBootstrap": true,
    "config": {
      "host": "organization.us-east-1.snowflakecomputing.com",
      "warehouse": "SAMPLE_WAREHOUSE",
      "database": "SNOWFLAKE_SAMPLE_DATA",
      "port": 443,
      "audit": {
        "enabled": false
      },
      "workspaces": {
        "enabled": false
      },
      "impersonation": {
        "enabled": false
      },
      "lineage": {
        "enabled": false
      },
      "authenticationType": "userPassword",
      "username": "<REDACTED>",
      "password": "<REDACTED>",
      "role": "ACCOUNTADMIN"
    }
  },
  {
    "id": "2",
    "status": "enabled",
    "type": "Databricks",
    "validationResults": {
      "status": "passed",
      "validationTests": [
        {
          "name": "Metastore validation",
          "status": "passed"
        },
        {
          "name": "Basic Connection Test",
          "result": [
            {
              "1": 1
            }
          ],
          "status": "passed"
        }
      ]
    },
    "autoBootstrap": true,
    "config": {
      "workspaceUrl": "www.example-workspace.cloud.databricks.com",
      "httpPath": "sql/protocolv1/o/0/0000-00000-abc123",
      "token": "REDACTED",
      "audit": {
        "enabled": false
      },
      "catalog": "immuta"
    }
  }
]
```

## Update an integration configuration

```bash
curl -X 'PUT' \
    'https://www.organization.immuta.com/integrations/{id}' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: 846e9e43c86a4ct1be14290d95127d13f' \
    -d '{
    "type": "Google BigQuery",
    "autoBootstrap": false,
    "config": {
      "role": "immuta",
      "datasetSuffix": "_secureView",
      "dataset": "immuta",
      "location": "us-east-1",
      "credential": "{\"type\":\"service_account\",\"project_id\":\"innate-conquest-123456\",\"private_key_id\":\"9163c12345690924f5dd218ff39\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\nXXXXXXXro0s\n/yQlPQijowkccmrmWJyr93kdLnwJzBvLHCto/+W\ncvF2ygX9oM/dyUK//z//4nptMp+Ck//Yw3D4rIBwGu4DWiR1qRnf\nDoGyXfThPTQ==\n-----END PRIVATE KEY-----\n\",\"client_email\":\"service-account-id@innate-conquest-123456.iam.gserviceaccount.com\",\"client_id\":\"1166290***432952487857\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",\"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/service-account-id%40innate-conquest-123456.iam.gserviceaccount.com\",\"universe_domain\":\"googleapis.com\"}"
    }
    }'
```

1. Copy the request example, which updates the private key. The example uses JSON format, but the request also accepts YAML.
2. Replace the **Immuta URL** and [**API key**](https://documentation.immuta.com/saas/developer-guides/api-intro/getting-started#authenticate-with-the-api) with your own.
3. Replace the **{id}** request parameter with the unique identifier of the integration you want to update.
4. Change the **config** values to your own, where
   * **role** is the Google Cloud role used to connect to Google BigQuery.
   * **datasetSuffix** is the suffix to postfix to the name of each dataset created to store secure views. This string must start with an underscore.
   * **dataset** is the name of the BigQuery dataset to provision inside of the project for Immuta metadata storage.
   * **location** is the dataset's location, which can be any valid GCP location (such as `us-east-1`).
   * **credential** is the Google BigQuery service account JSON keyfile credential content. See the [Google documentation](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) for guidance on generating and downloading this keyfile.

See the [**config** object description](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/integration-configuration-payload#google-bigquery-configuration-object) for parameter definitions, value types, and additional configuration options.

### Response

The response returns the status of the Google BigQuery integration configuration connection. See the [response schema reference](https://documentation.immuta.com/saas/developer-guides/api-intro/integrations-api/reference-guides/response-schema) for details about the response schema.

{% tabs %}
{% tab title="200 response" %}
A successful response includes the validation tests statuses.

```json
{
  "id": "123456789",
  "status": "editing",
  "validationResults": {
    "status": "passed",
    "validationTests": [
    {
      "name": "Basic Validation: Verify service account not being used for data source connection credentials",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Immuta Service Account postfix",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Non-matching service account in key file",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Connection can be made to BigQuery",
      "status": "passed"
    },
    {
      "name": "Initialize Validation: [Dataset - exists] innate-conquest-123456.immuta",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - create] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Table - insert] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "warning",
      "message": "Billing has not been enabled for this project. Enable billing at https://console.cloud.google.com/billing. DML queries are not allowed in the free tier. Set up a billing account to remove this restriction. Confirm that the bootstrap script was executed without any modifications. This warning will not prevent the connection but continue at your own risk as some features may not function."
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After insert)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - update] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - get] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE (After update)",
      "status": "skipped"
    },
    {
      "name": "Validate Manual: [Table - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144.IMMUTA_VERIFY_TABLE",
      "status": "passed"
    },
    {
      "name": "Validate Manual: [Dataset - delete] innate-conquest-123456.IMMUTA_VERIFY_DATASET_144",
      "status": "passed"
    }
    ]
  }
}
```

{% endtab %}

{% tab title="Unsuccessful request" %}
An unsuccessful request returns the status code and an error message. See the [HTTP status codes and error messages](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/status-codes#put-errors) for a list of statuses, error messages, and troubleshooting guidance.

```json
{
  "statusCode": 409,
  "error": "Conflict",
  "message": "Unable to edit integration with ID 123456789 in current state editing."
}
```

{% endtab %}
{% endtabs %}

## Delete an integration

```bash
curl -X 'DELETE' \
    'https://www.organization.immuta.com/integrations/{id}' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: 846e9e43c86a4ct1be14290d95127d13f'
```

1. Copy the request example.
2. Replace the **Immuta URL** and [**API key**](https://documentation.immuta.com/saas/developer-guides/api-intro/getting-started#authenticate-with-the-api) with your own.
3. Replace the **{id}** request parameter with the unique identifier of the integration you want to delete.

### Response

The response returns the status of the Google BigQuery integration configuration that has been deleted. See the [response schema reference](https://documentation.immuta.com/saas/developer-guides/api-intro/integrations-api/reference-guides/response-schema) for details about the response schema. An unsuccessful request returns the status code and an error message. See the [HTTP status codes and error messages](https://documentation.immuta.com/saas/developer-guides/api-intro/reference-guides/status-codes#delete-errors) for a list of statuses, error messages, and troubleshooting guidance.

```json
{
  "id": "123456789",
  "status": "deleting",
  "validationResults": {
    "status": "passed",
    "validationTests": [
    {
      "name": "Basic Validation: Verify service account not being used for data source connection credentials",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Immuta Service Account postfix",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Non-matching service account in key file",
      "status": "passed"
    },
    {
      "name": "Basic Validation: Connection can be made to BigQuery",
      "status": "passed"
    }
    ]
  }
}
```