> For the complete documentation index, see [llms.txt](https://documentation.immuta.com/SaaS/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.immuta.com/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration.md). # Google BigQuery Viewless Integration Reference Guide Learn about how the Google BigQuery viewless integration works and what Immuta does in your environment to register your data and keep it in sync {% hint style="info" %} **Public preview**: This integration is available to all accounts that request to enable it for their tenant. Contact your Immuta representative to enable it. {% endhint %} The Google BigQuery viewless integration allows you to register data from Google BigQuery in Immuta and enforce subscription policies and row-level data policies on that data. The sequence diagram below outlines the events that occur when an Immuta user queries a Google BigQuery table that has been registered as an Immuta data source.
## What does Immuta do in my environment? ### Registering a connection Google BigQuery is configured and data is registered through [connections](/SaaS/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview.md), an Immuta feature that allows you to register your data objects through a single connection to make data registration more scalable for your organization. Instead of registering schema and databases individually, you can register them all at once and allow Immuta to monitor your data platform for changes so that data sources are added and removed automatically to reflect the state of data in your data platform. When the [connection is registered](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/register-a-google-bigquery-connection.md), Immuta ingests and stores connection metadata in the Immuta metadata database. In the example below, the Immuta application administrator connects the database that contains `marketing-data`, `research-data`, and `cs-data` tables. Immuta registers[^1] these tables as data sources and stores the table metadata in the Immuta metadata database.
Immuta presents a hierarchical view of your data that reflects the hierarchy of objects in Google BigQuery after registration is complete: * Project * Dataset * Data object Beyond making the registration of your data more intuitive, connections provides more control. Instead of performing operations on individual schemas or tables, you can perform operations (such as object sync) at the connection level. See the [Connections reference guide](/SaaS/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview.md) for details about connections and how to manage them. To configure your Google BigQuery viewless integration and register data, see the [Register a Google BigQuery connection guide](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/register-a-google-bigquery-connection.md). ### Applying policies After objects are registered in Immuta, you can author subscription policies or row-level data policies in Immuta to enforce access controls. The Google BigQuery viewless integration is additive: Immuta does not update or override existing access controls on securables unless an Immuta policy explicitly dictates an access change. #### Subscription policies Immuta enforces read and write subscription policies on Google BigQuery objects by updating [allow policies](https://docs.cloud.google.com/iam/docs/overview#allow-policies) on Google BigQuery resources that grant and revoke access to objects according to the allow policy. When a subscription policy is activated on an Immuta data source, 1. Immuta creates a [custom role](https://docs.cloud.google.com/iam/docs/roles-overview) in the Google BigQuery project with the necessary [permissions](/SaaS/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-access-types.md#granting-google-bigquery-viewless-privileges) on the data object. 2. Immuta calculates the users affected by the subscription policy and creates a [Google group](https://docs.cloud.google.com/iam/docs/groups-in-cloud-console) for those users. 3. Immuta adds the appropriate Google groups to a Google BigQuery allow policy and binds them to the custom role. This binding grants the custom role to those Google groups, which then permits users in those groups to access the data object. The [Protecting data in Google BigQuery page](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration/protecting-data.md#subscription-policies) provides an example of this policy enforcement. See the [Subscription policy access types page](/SaaS/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-access-types.md#granting-google-bigquery-viewless-privileges) for details about the Google BigQuery privileges granted to users when they are subscribed to a data source protected by a subscription policy. #### Data policies Immuta applies [Google BigQuery row-level security controls](https://docs.cloud.google.com/bigquery/docs/row-level-security-intro) that are enforced natively within Google BigQuery. When a row-level data policy is applied to a table registered in Immuta, Immuta creates a [Google BigQuery row-level access policy](https://docs.cloud.google.com/bigquery/docs/managing-row-level-security#create-policy) that includes the target table and a `filter_expression` that compares the querying user's entitlements (saved in the `immuta_profiles` table) against those specified in the data policy. Then, when a user queries that data source, the rows returned are filtered according to what the policy dictates that they're allowed to see. The [Protecting data in Google BigQuery page](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration/protecting-data.md#data-policies) provides an example of this policy enforcement. ### Required Google BigQuery privileges The privileges that the Google BigQuery integration requires align to the least privilege security principle. The table below describes each privilege required by the [setup user](#user-content-fn-2)[^2] and the [service account](#user-content-fn-3)[^3]. {% hint style="info" %} **Google BigQuery users with access to Immuta-governed Google BigQuery data sources will be able to read the `immuta_profiles` table** Google BigQuery states that grantees must have the `bigquery.tables.getData` permission on the target and referenced tables in policies. Because Immuta policy predicates read `immuta_profiles` for every entitlement check, Immuta grants every user querying a protected table read access to `immuta_profiles`. {% endhint %} | Google BigQuery privilege | User requiring the privilege | Explanation | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Setup user | This privilege is required so that the setup user can create the service account and custom role for Immuta. | | | Service account | These privileges allow the Immuta service account to get metadata for datasets and their objects. | | | Service account | These privileges allow the Immuta service account to create Google groups to enforce read and write subscription policies and row-level data policies on data objects registered as Immuta data sources. Immuta does not manage or have access to your existing Google groups managed outside Immuta. | ### Maintaining state with Google BigQuery The following user actions spur various processes in the Google BigQuery viewless integration so that Immuta data remains synchronous with data in Google BigQuery: * **Data source created or updated**: Immuta registers data source metadata and stores that metadata in the Immuta metadata database. * **Data source deleted**: Immuta deletes the data source metadata from the metadata database. * [**User account is mapped to Immuta**](#user-registration-and-id-mapping): When a user account is mapped to Immuta, their metadata is stored in the metadata database. * **User subscribed to a data source**: When users are added to a data source by a data owner or through a subscription policy, Immuta calculates the users affected by the subscription policy and creates a [Google group](https://docs.cloud.google.com/iam/docs/groups-in-cloud-console) for those users. Then Immuta grants Google BigQuery privileges to their group through an [allow policy](https://docs.cloud.google.com/iam/docs/overview#allow-policies). * **Automatic subscription policy applied to or updated on a data source**: Immuta calculates the users and data sources affected by the policy change and grants or revokes users' privileges on the data object by adding them to or removing them from the Immuta-managed Google group that has access to the securable. See the [Protecting data in Google BigQuery page](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration/protecting-data.md#protecting-data) for details about this process. * **Subscription policy deleted**: Immuta calculates the users and Google groups affected by the policy and revokes privileges on the Google BigQuery object by removing users from the Immuta-created Google group. If there are no longer any members of an Immuta-created Google group, that group is deleted by Immuta. * **User removed from a data source**: Immuta removes the user from the Immuta-managed Google group. #### Google BigQuery destructive operations that remove row access policies [Google BigQuery silently removes row access policies](https://docs.cloud.google.com/bigquery/docs/row-level-security-intro#implicit_deletion_of_row-level_access_policies) on tables that have any of the following Google BigQuery operations run against them: * `CREATE OR REPLACE TABLE` * `bq load --replace` * `WRITE_TRUNCATE` jobs * `bq cp -f` If you run any of these operations in Google BigQuery against objects protected by Immuta, you must [manually trigger Immuta policies to resync](/SaaS/configuration/integrations/data-and-integrations/registering-metadata/data-source-settings.md#resync-policies). See the [Google Cloud documentation](https://docs.cloud.google.com/bigquery/docs/row-level-security-intro#implicit_deletion_of_row-level_access_policies) for details about this behavior. ## Supported object types | Object type | Subscription policy support | Data policy support | Request app support | | ------------------ | --------------------------- | -------------------- | -------------------- | | Tables | :white\_check\_mark: | :white\_check\_mark: | :white\_check\_mark: | | Views | :white\_check\_mark: | :x: | :white\_check\_mark: | | Materialized views | :white\_check\_mark: | :x: | :white\_check\_mark: | ## Supported policies The Google BigQuery integration allows users to author subscription and row-level data policies to enforce access controls. See the corresponding pages for details about specific types of policies supported: * [Subscription policy access types](/SaaS/govern/secure-your-data/authoring-policies-in-secure/section-contents/reference-guides/subscription-access-types.md) * [Data policies reference guide](/SaaS/govern/secure-your-data/authoring-policies-in-secure/data-policies/reference-guides/data-policies.md#data-policy-support-matrix) See the [applying policies section](#applying-policies) for details about policy enforcement. ## Security and compliance See the [Security and compliance in Google BigQuery guide](/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration/security-and-compliance.md) for details about security, supported authentication methods, and audit. ## User registration and ID mapping The built-in Immuta IAM can be used as a complete solution for authentication and user entitlement. However, you can connect your existing identity management provider to Immuta to use that system for authentication and user entitlement instead. Each of the [supported IAM protocols](/SaaS/releases/support-matrix.md#iam-protocols) includes a set of configuration options that enable Immuta to communicate with the IAM system and map the users, permissions, groups, and attributes into Immuta. For policies to impact the right users, the user account in Immuta must be mapped to the user account in Google BigQuery. You can ensure these accounts are mapped correctly in the following ways: * [Automatically](/SaaS/configuration/people/users-index/how-to-guides/external-user-mapping.md#configure-external-user-id-mapping-on-app-settings-page): If usernames in Google BigQuery align with usernames in the external IAM and those accounts align with an IAM attribute, you can enter that IAM attribute on the app settings page to automatically map user IDs in Immuta to Google BigQuery. * [Manually](/SaaS/configuration/people/users-index/how-to-guides/external-user-mapping.md#manually-configure-external-user-id-mapping-on-a-users-page): You can manually map user IDs for individual users. For guidance on connecting your IAM to Immuta, see the [how-to guide for your protocol](/SaaS/configuration/people/section-contents.md#how-to-guides). ## Limitations and known issues The following Immuta features are unsupported: * Identification * Select data policies * Limit to purpose * Masking * Minimization * Time-based restrictions * Tag ingestion * Query audit Google BigQuery has the following limitations: * [Limit of one allow policy per resource](https://docs.cloud.google.com/bigquery/docs/control-access-to-resources-iam) * [Limit of 250 Google groups in an allow policy](https://docs.cloud.google.com/iam/docs/allow-policies) * [Limit of 1500 groups Immuta can own](https://knowledge.workspace.google.com/admin/groups/understand-groups-policies-and-limits) * [Limit of 400 row-access policies per table](https://docs.cloud.google.com/bigquery/quotas#row-level_security) * **Destructive operations that remove row access policies**: Google BigQuery silently removes row access policies on tables that have any of the following Google BigQuery operations run against them: * `CREATE OR REPLACE TABLE` * `bq load --replace` * `WRITE_TRUNCATE` jobs * `bq cp -f` If you run any of these operations in Google BigQuery against objects protected by Immuta, you must [manually trigger Immuta policies to resync](/SaaS/configuration/integrations/data-and-integrations/registering-metadata/data-source-settings.md#resync-policies). See the [Google Cloud documentation](https://docs.cloud.google.com/bigquery/docs/row-level-security-intro#implicit_deletion_of_row-level_access_policies) for details about this behavior. [^1]: See the [Connections reference guide](/SaaS/configuration/integrations/data-and-integrations/registering-a-connection/reference-guides/connections-overview.md#object-sync) for more details about how data objects are synced with Immuta so that the objects stay synchronous with the registered objects in Immuta. [^2]: The Google BigQuery user registering the connection. [^3]: This account is used by Immuta to manage the integration once it has been set up. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://documentation.immuta.com/SaaS/configuration/integrations/google-bigquery/google-bigquery-viewless-integration/google-bigquery-viewless-integration.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.