Upgrading to Immuta 2024.2 LTS

Architectural Prerequisites

• External metadata database (e.g. RDS/Azure PostgreSQL)

• Opensearch/Elasticsearch endpoint with username/password that can create an index

Demonstration - 2022.5 LTS to 2024.2 LTS

Infrastructure deployment is not covered in this article. For details on deploying the required infrastructure, consult the articles below Azure Immuta Deployment

AWS Immuta Deployment

Deployment of 2022.5

helm upgrade --install --create-namespace -n immuta-upgrade immuta immuta/immuta -f immuta-2022.5.yaml

immuta-2022.5.yaml

---
immutaVersion: 2022.5.13
tagVersion: 2022.5.13
externalHostname: lts-upgrade.immuta.us
global:
  imageRegistry: 231431240278.dkr.ecr.us-east-1.amazonaws.com
web:
  ingress:
    ingressClassName: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/ssl-redirect: '443'
      alb.ingress.kubernetes.io/backend-protocol: HTTPS
database:
  enabled: true
  persistence:
    enabled: true
    volumeClaimSpec:
      resources:
        requests:
          storage: 20Gi
queryEngine:
  rehydration:
    enabled: false
  replicas: 1
  persistence:
    enabled: true
    volumeClaimSpec:
      resources:
        requests:
          storage: 20Gi
nginxIngress:
  enabled: false
backup:
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::231431240278:role/eksctl-immuta-lts-addon-iamserviceaccount-imm-Role1-7OXoZbcVdvLk
  enabled: true
  type: s3
  s3:
    bucket: lts-upgrade
  restore:
    enabled: false

Validate Deployment

Add a license and a policy, for example

2022.5 Immuta Instance

Backup Deployment

Take an adhoc backup of this deployment. If OIDC/IRSA ensure the immuta-backup SA can write to S3

eksctl create iamserviceaccount --cluster immuta-lts \
                                --name immuta-backup \
                                --namespace immuta-upgrade  \
                                --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess \
                                --override-existing-serviceaccounts \
                                --approve

kubectl -n immuta-upgrade create job adhoc --from cronjob/immuta-backup

Uninstall 2022.5

helm uninstall -n immuta-upgrade immuta
kubectl -n immuta-upgrade delete pvc pg-data-immuta-database-0
kubectl -n immuta-upgrade delete pvc pg-data-immuta-query-engine-0

Restore to 2024.1 with external database

If using IRSA ensure the following trust relationship is set on the role with S3 Access

Each of the mentioned service accounts needs to have access to backup and restore

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::231431240278:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/B029A8A32AFEA1B88A63F4207DBF9964"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.us-east-1.amazonaws.com/id/B029A8A32AFEA1B88A63F4207DBF9964:aud": "sts.amazonaws.com",
                    "oidc.eks.us-east-1.amazonaws.com/id/B029A8A32AFEA1B88A63F4207DBF9964:sub": [
                        "system:serviceaccount:immuta-upgrade:immuta-database-initialize-hook",
                        "system:serviceaccount:immuta-upgrade:immuta-query-engine",
                        "system:serviceaccount:immuta-upgrade:immuta-backup"
                    ]
                }
            }
        }
    ]
}

helm upgrade --install --create-namespace -n immuta-upgrade immuta immuta/immuta -f immuta-2022.5.yaml

immuta-2024.1.yaml

immutaVersion: 2024.1.8
externalHostname: lts-upgrade.immuta.us
global:
  imageRegistry: 231431240278.dkr.ecr.us-east-1.amazonaws.com
hooks:
  databaseInitialize:
    serviceAccountAnnotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::231431240278:role/eksctl-immuta-lts-addon-iamserviceaccount-imm-Role1-7OXoZbcVdvLk
web:
  ingress:
    ingressClassName: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/ssl-redirect: '443'
      alb.ingress.kubernetes.io/backend-protocol: HTTPS
database:
  enabled: false
externalDatabase:
  enabled: true
  host: immuta-lts.cfzynskvahpp.us-east-1.rds.amazonaws.com
  dbname: bometadata
  username: bometa
  password: secret
queryEngine:
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::231431240278:role/eksctl-immuta-lts-addon-iamserviceaccount-imm-Role1-7OXoZbcVdvLk
  rehydration:
    enabled: false
  replicas: 1
  persistence:
    enabled: true
    volumeClaimSpec:
      resources:
        requests:
          storage: 20Gi
nginxIngress:
  enabled: false
backup:
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::231431240278:role/eksctl-immuta-lts-addon-iamserviceaccount-imm-Role1-7OXoZbcVdvLk
  enabled: false
  type: s3
  s3:
    bucket: lts-upgrade
  restore:
    enabled: true

Upgrade to 2024.2

helm uninstall -n immuta-upgrade immuta
helm upgrade --install --create-namespace -n immuta-upgrade immuta ./immuta-enterprise-2024.2.0-rc.3.tgz -f immuta-2024.2.yaml

immuta-2024.2.yaml

global:
  tenantId: immuta-lts-upgrade
  imageRegistry: 231431240278.dkr.ecr.us-east-1.amazonaws.com
  imageTag: v2024.2.0_20240430
audit:
  config:
    databaseConnectionString: postgres://bometa:[e-mail]:5432/bometadata
    elasticsearchEndpoint: https://vpc-immuta-lts-zepgp3w7ceu27on5ybgsoneule.us-east-1.es.amazonaws.com
    elasticsearchUsername: immuta-audit-service
    elasticsearchPassword: RandomPassword123!
secure:
  extraEnvVars:
    - name: FeatureFlag_AuditService
      value: "true"
    - name: FeatureFlag_detect
      value: "true"
    - name: FeatureFlag_auditLegacyViewHide
      value: "true"
  ingress:
    hostname: lts-upgrade.immuta.us
    ingressClassName: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/ssl-redirect: '443'
      alb.ingress.kubernetes.io/backend-protocol: HTTP
    tls: true
  postgresql:
    host: immuta-lts.cfzynskvahpp.us-east-1.rds.amazonaws.com
    port: 5432
    database: bometadata
    username: bometa
    password: secret
    ssl: true

Validate via web

2024.2 LTS upgraded from 2022.5