All pages
Powered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

Export Audit Logs to ADLS

Public preview: This feature is public preview and available to all accounts.

Requirements:

  • Immuta permission AUDIT

  • If you will use the Immuta CLI instead of GraphQL API, . Must be CLI v1.4.0 or newer.

Create an ADLS shared access signature token for the export

Before Immuta can export audit events to your Azure Data Lake Storage (ADLS) Gen2 storage account, you need to create a shared access signature (SAS) token that allows the Immuta audit service to add audit logs to your specified ADLS storage account and file system.

Follow the Azure documentation to create the following in Azure:

  1. with the following settings required for audit export:

    • Enable hierarchical namespace

    • Standard performance is adequate, but premium may be used

  2. .

Save the SAS token to use in the next steps. Do not navigate away from the SAS page unless you have saved the token.

Configure the audit export to ADLS

Configure the audit export to ADLS using the Immuta CLI or GraphQL API with the following fields:

  • interval: The interval at which audit logs will be exported to your ADLS storage. They can be sent at 2-, 4-, 6-, 12-, or 24-hour intervals.

  • storage account: The name of the storage account you created that your audit logs will be sent to.

  • file system: The name of the file system (or container) you created that your audit logs will be written to.

  • path: The name of the path in the file system. This will be a new folder or directory in the container where Immuta will send your audit logs for storage.

Run the following command with the above fields in a JSON file:

Example ./your-exportConfig.json file

For additional CLI commands, see the .

Run the following mutation to this URL, https://your-immuta.com/api/audit/graphql, with the above fields passed directly:

Example response

For additional GraphQL API commands, see the .

Troubleshooting

Error: Unknown error occurred making request to http://your-immuta.com/api/audit/graphql

You are not running the required CLI version, which must be CLI v1.4.0 or newer. Update your CLI and reconfigure the CLI with your Immuta tenant.

list returns 0

You do not have any exports configured. Start again from a "Configure the audit export to S3" step and ensure the authentication is correct.

How-to Guides

A shared access signature (SAS) for your dedicated container with at least the following permissions at the storage account or container level:

  • Create

  • Write

  • SAS token: The previously-generated SAS token.

  • install and configure the Immuta CLI
    An ADLS Gen2 storage account
    A dedicated container
    audit CLI reference guide
    GraphQL API reference guide
    immuta audit exportConfig create:adls:sasToken ./your-exportConfig.json
    {
    "interval": "EVERY_12_HOURS",
    "storageAccount": "your-adls-storage-account",
    "fileSystem": "audit-output",
    "path": "immuta-audit",
    "sasToken": "your-sas-token"
    }
    mutation {
      createAdlsSasTokenEndpointConfiguration(
        data: {
          interval: "EVERY_12_HOURS",
          storageAccount: "your-adls-storage-account",
          fileSystem: "audit-output",
          path: "immuta-audit",
          sasToken: "your-sas-token"
        }
      )
        {
            id
            interval
            enabled
            connectionStatus
            endpointConfiguration {
                ... on AdlsSasTokenEndpointConfiguration {
                    storageAccount
                    fileSystem
                    path
                }
            }
        }
    }
    {
        "data": {
            "createAdlsSasTokenEndpointConfiguration": {
                "id": "259fc41c-b502-418a-a8ff-d875335dbe9b",
                "interval": "EVERY_12_HOURS",
                "enabled": true,
                "connectionStatus": "SUCCESS",
                "endpointConfiguration": {
                    "storageAccount": "your-adls-storage-account",
                    "fileSystem": "audit-output",
                    "path": "immuta-audit"
                }
            }
        }
    }

    Run Governance Reports

    Build and export Immuta reports

    Immuta reports allow data governors to use a natural language builder to instantly create reports that detail user activity across Immuta.

    1. Click select entity and choose the option you would like the report based on from the dropdown menu. Your options include User, Group, Project, Data Source, Purpose, Policy Type, Connection, or Tag.

    2. After making your selection, type your entity name in the enter name field.

    3. Select the name from the dropdown menu that appears. Once the entity name has been selected, a number of reports will populate the center window.

    4. Click a tile with the description of the report to run that report. You may only see up to 100 rows of output in the UI when you run a report. To see the full results of your report, follow the step below to export to CSV.

    5. Once you've run the report, you can click the Export to CSV button in the top right of the page to download the report.

    If you would like to switch reports from this page, you can make changes by clicking the dropdown menu and then Refresh to run a new report. Otherwise, click Back to Report Builder in the top right of the page to return to the full report builder.

    Default 9-minute timeout

    Governance report queries will timeout after 9 minutes to avoid overconsumption of resources. If your governance report was not generated because of this timeout, to change the default setting.

    submit a support ticket

    Export Audit Logs to S3

    Public preview: This feature is public preview and available to all accounts.

    Requirements:

    • Immuta permission AUDIT

    • If you will use the Immuta CLI instead of GraphQL API, . Must be CLI v1.4.0 or newer.

    Use the following how-to to configure a periodical export of your Immuta audit logs to an S3 bucket. This export configuration requires access to your S3 bucket to add objects using one of the following authentication methods:

    How to export using an access key

    Configure your Immuta audit logs to export to your S3 bucket and allow Immuta to authenticate using your AWS access key ID and secret access key.

    Create an S3 bucket policy for the export

    Before Immuta can export audit events to your S3 bucket, you need to create a bucket policy that allows the Immuta audit service to add objects to your specified S3 bucket. The following Amazon S3 action will be granted to the audit service in the bucket policy:

    • : Adds an object to a bucket.

    To create the policy for the bucket, you must be the bucket owner.

    1. Follow for adding a bucket policy in the Amazon S3 console. To create the policy for the bucket, you must be the bucket owner.

    2. Edit the JSON in the Policy section to include a bucket policy like the example below. In this example, the policy allows immuta-audit-service (the ) to add objects to customer-bucket-name (and the contents within that bucket).

      Note: If you use this example, replace the content in angle brackets with your and bucket name.

    3. Save your changes.

    Configure the audit export to S3

    Configure the audit export to S3 using the Immuta CLI or GraphQL API with the following fields:

    • interval: The interval at which audit logs will be exported to your S3 bucket. They can be sent at 2-, 4-, 6-, 12-, or 24-hour intervals.

    • bucket name: Name of the bucket your audit logs will be sent to that your added the policy to above.

    • bucket path: The name of the folder within the bucket to put the audit logs in. This field is optional.

    • region: AWS region (such as "us-east-1").

    Run the following command with the above fields in a JSON file:

    Example ./exportConfig.json file

    For additional CLI commands, see the .

    Run the following mutation to this URL, https://your-immuta.com/api/audit/graphql, with the above fields passed directly:

    Example response

    For additional GraphQL API commands, see the.

    How to export using an AWS IAM role

    Configure your Immuta audit logs to export to your S3 bucket and allow Immuta to authenticate using an .

    Immuta requires a role with the following allowed action to the S3 bucket you want the audit logs exported to:

    • which allows to the role to add an object to a bucket.

    1 - Create an AWS policy for the role

    1. Follow to create a new role for Immuta to assume and add objects to your S3 bucket.

    2. Follow for creating IAM policies in the Amazon S3 console for the new role. Use the example JSON below to allow the provided role to add objects to the specified buckets. Ensure the buckets provided here are the ones used when configuring the export.

      Note: If you use this example, replace the content in angle brackets with your bucket name.

    2 - Configure the audit export to S3

    Response error

    When creating the export configuration, this step will return an error. Take the returned export configuration ID and continue with step 3 and 4 to create a trust relationship and verify the connection between Immuta and S3.

    Configure the audit export to S3 using the Immuta CLI or GraphQL API with the following fields:

    • interval: The interval at which audit logs will be exported to your S3 bucket. They can be sent at 2-, 4-, 6-, 12-, or 24-hour intervals.

    • bucket name: Name of the bucket your audit logs will be sent to.

    • bucket path: The name of the folder within the bucket to put the audit logs in. This field is optional.

    • region: AWS region (such as "us-east-1").

    Run the following command with the above fields in a JSON file:

    Example ./exportConfig.json file

    Example response:

    For additional CLI commands, see the

    Run the following mutation to this URL, https://your-immuta.com/api/audit/graphql, with the above fields passed directly:

    Example response

    For additional GraphQL API commands, see the .

    3 - Create a trust relationship

    Follow for creating IAM policies in the Amazon S3 console. Use the example JSON below to create a trust policy that sets the export configuration external ID as the requirement for assuming the role.

    Fill in the content in angle brackets with the following:

    • AWS Account ID: Your AWS account ID.

    • Export Configuration ID: Insert the ID from step 2's response.

    4 - Verify the export configuration connection

    Now that the configuration and the trust relationship have been created, test the connection from Immuta to S3 to ensure your audit logs are exported to your S3 bucket.

    If connectionStatus returns SUCCESS, your export configuration has been successfully set up.

    Run the following command

    Run the following mutation to this URL, https://your-immuta.com/api/audit/graphql:

    Troubleshooting

    Error: Unknown error occurred making request to http://your-immuta.com/api/audit/graphql

    You are not running the required CLI version, which must be CLI v1.4.0 or newer. Update your CLI and reconfigure the CLI with your Immuta tenant.

    list returns 0

    You do not have any exports configured. Start again from a "Configure the audit export to S3" step and ensure the authentication is correct.

  • accessKeyId: AWS access key ID for authentication. See the AWS documentation for information about using an access key ID and secret access key.

  • secretAccessKey: AWS secret access key for authentication.

  • roleArn: AWS role ARN for authentication that you added the policies to above. Immuta will assume this role when exporting audit logs to S3.

  • install and configure the Immuta CLI
    Access and secret key
    AWS IAM role
    s3:PutObject
    AWS documentation
    Principal
    Amazon Resource Name (ARN)
    audit CLI reference guide
    GraphQL API reference guide
    AWS role
    s3:PutObject
    AWS documentation
    AWS documentation
    audit CLI reference guide.
    GraphQL API reference guide
    AWS documentation
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::<Your AWS account ID>:user/<Your IAM User>"
                },
                "Action": [
                    "s3:PutObject"
                ],
                "Resource": [
                    "arn:aws:s3:::<your-s3-bucket>",
                    "arn:aws:s3:::<your-s3-bucket>/*"
                ]
            }
        ]
    }
    immuta audit exportConfig create:s3:accessKey ./exportConfig.json
    {
    "interval": "EVERY_12_HOURS",
    "bucket": "your-s3-bucket",
    "path": "your-optional-s3-bucket-path",
    "region": "your-aws-region",
    "accessKeyId": "YOURACCESSKEYID",
    "secretAccessKey": "YOUR/SECRET/ACCESSKEY"
    }
    mutation {
      createS3AccessKeyExportConfiguration(
        data: {
          interval: EVERY_12_HOURS
          bucket: "your-s3-bucket"
          path: "your-optional-s3-bucket-path"
          region: "your-aws-region"
          accessKeyId: "YOURACCESSKEYID"
          secretAccessKey: "YOUR/SECRET/ACCESSKEY"
        }
      )
        {
            id
            interval
            enabled
            connectionStatus
            endpointConfiguration {
                ... on S3AccessKeyEndpointConfiguration {
                    bucket
                    path
                    region
                }
            }
        }
    }
    {
        "data": {
            "createS3AccessKeyExportConfiguration": {
                "id": "259fc41c-b502-418a-a8ff-d875335dbe9b",
                "interval": "EVERY_12_HOURS",
                "enabled": true,
                "connectionStatus": "SUCCESS",
                "endpointConfiguration": {
                    "bucket": "your-s3-bucket",
                    "path": "your-optional-s3-bucket-path",
                    "region": "your-region",
                }
            }
        }
    }
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:PutObject"
                ],
                "Resource": [
                    "arn:aws:s3:::<your-s3-bucket>",
                    "arn:aws:s3:::<your-s3-bucket>/*"
                ]
            }
        ]
    }
    immuta audit exportConfig create:s3:assumedRole ./exportConfig.json
    {
    "interval": "EVERY_12_HOURS",
    "bucket": "your-s3-bucket",
    "path": "your-optional-s3-bucket-path",
    "region": "your-region",
    "roleArn": "arn:aws:iam::<Your AWS Account ID>:role/<the prepared role>"
    }
    {
        "data": {
            "createS3AssumedRoleExportConfiguration": {
                "id": "259fc41c-b502-418a-a8ff-d875335dbe9b",
                "interval": "EVERY_12_HOURS",
                "enabled": true,
                "connectionStatus": "Error testing access to S3 using AssumedRole <Your Role> - User: <Immuta Account> is not authorized to perform: sts:AssumeRole on resource: <Your Role>",
                "endpointConfiguration": {
                    "__typename": "S3AssumedRoleEndpointConfiguration",
                    "bucket": "your-s3-bucket",
                    "path": "your-optional-s3-bucket-path",
                    "region": "your-region",
                }
            }
        }
    }
    mutation {
      createS3AssumedRoleExportConfiguration(
        data: {
          interval: EVERY_12_HOURS
          bucket: "your-s3-bucket"
          path: "your-optional-s3-bucket-path"
          region: "your-region"
          roleArn: "arn:aws:iam::<Your AWS Account ID>:role/<the prepared role>"
        }
      )
        {
            id
            interval
            enabled
            connectionStatus
            endpointConfiguration {
                ... on S3AssumedRoleEndpointConfiguration {
                    bucket
                    path
                    region
                }
            }
        }
    }
    {
        "data": {
            "createS3AssumedRoleExportConfiguration": {
                "id": "259fc41c-b502-418a-a8ff-d875335dbe9b",
                "interval": "EVERY_12_HOURS",
                "enabled": true,
                "connectionStatus": "Error testing access to S3 using AssumedRole <Your Role> - User: <Immuta Account> is not authorized to perform: sts:AssumeRole on resource: <Your Role>",
                "endpointConfiguration": {
                    "bucket": "your-s3-bucket",
                    "path": "your-optional-s3-bucket-path",
                    "region": "your-region",
                }
            }
        }
    }
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::<AWS Account ID>:root"
                },
                "Action": "sts:AssumeRole",
                "Condition": {"StringEquals": {"sts:ExternalId": "<Export Configuration ID>"}}
            }
        ]
    }
    immuta audit exportConfig list
    query {
        getAllExportConfigurations{
            id
            connectionStatus
        }
    }