Skip to content

You are viewing documentation for Immuta version 2022.3.

For the latest version, view our documentation for Immuta SaaS or the latest self-hosted version.

Importing Backups from an Existing Immuta Instance

Audience: System Administrators

Content Summary: You can import backups from your existing Immuta instance into Kubernetes. Deployment types that can be imported include Kubernetes and Single Node Docker. This page outlines how to import backups from an Immuta instance into Kubernetes.

Backing up your Existing Single Node Docker Instance

Backup the database roles for your existing Immuta instance separately. If you have a combined database instance both of these commands will be run from that database instance.

  1. Exec into the immuta-db container:

    docker exec -it immuta-db /bin/bash
    
  2. Backup the Immuta role:

    backup-immuta.sh \
      -r immuta \
      -d /var/lib/pgsql/11/backups \
      -f immuta-00000000000000.tar.gz
    
  3. Backup the Metadata role:

    backup-immuta.sh \
      -r bometadata \
      -d /var/lib/pgsql/11/backups \
      -f metadata-00000000000000.tar.gz
    
  4. Copy these output files from the existing instance to a location that has kubectl access to the new cluster. The remainder of this guide will assume that it is being run from a directory containing these two backup files.

Create PersistentVolumeClaims

StorageClassName

Depending on the cloud provider, the storageClassName will change in the below file. Please make sure you test which ones are available to you by using kubectl get storageclass and selecting one that is recommended.

It's generally safe to use aws-efs (if using AWS) or azurefile-csi (if using AKS).

Create a PersistentVolumeClaim using the appropriate storage class for the cloud provider.

  1. Create a file, immuta-backup-pvc.yaml, with the following contents.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: immuta-backups
      labels:
        app: immuta
        volume-use: backup
    spec:
      accessModes:
        - ReadWriteMany
      storageClassName: foo
      resources:
        requests:
          storage: 100Gi
    
  2. Create the PersistentVolumeClaim by running the following command:

    kubectl create -f immuta-backup-pvc.yaml
    
  3. Verify that the claim is successfully bound:

    kubectl get persistentvolumeclaims -l app=immuta,volume-use=backup
    

The volume claim should be present and report status "Bound".

Populating PersistentVolumeClaims with Backups

The easiest way to populate existing backups into the PersistentVolumeClaims is to create a temporary deployment that mounts the PersistentVolumeClaims, and copy the backups in using kubectl cp.

  1. Create a file, immuta-backup-importer-deployment.yaml, with the following contents.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: immuta-backup-importer
      labels:
        app: immuta
        component: backup-importer
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: immuta
          component: backup-importer
      template:
        metadata:
          labels:
            app: immuta
            component: backup-importer
        spec:
          containers:
          - name: importer
            image: nginx
            tty: true
            stdin: true
            command: ["/bin/bash"]
            volumeMounts:
            - mountPath: "/var/run/immuta/backup/database"
              name: immuta-backup
              subPath: database
            - mountPath: "/var/run/immuta/backup/query-engine"
              name: immuta-backup
              subPath: query-engine
          volumes:
            - name: immuta-backup
              persistentVolumeClaim:
                claimName: immuta-backups
    
  2. Create the deployment:

    kubectl create -f immuta-backup-importer-deployment.yaml
    

Once the immuta-backup-importer Pod has successfully started running, you can copy the backups into the appropriate locations.

  1. First, get the pod name, and save it into a variable. Next copy the backups you took above into the volume paths specified in the Deployment.

    backup_importer_pod_name=$(kubectl get pod \
      -l app=immuta,component=backup-importer \
      -o go-template='{{(index .items 0).metadata.name}}')
    
    kubectl cp \
      immuta-00000000000000.tar.gz \
      ${backup_importer_pod_name}:/var/run/immuta/backup/query-engine/
    
    kubectl cp \
      metadata-00000000000000.tar.gz \
      ${backup_importer_pod_name}:/var/run/immuta/backup/database/immuta-00000000000000.tar.gz
    

Once this has completed successfully, you can safely delete the deployment. The PersistentVolumeClaims will be re-used for the Immuta deployment.

kubectl delete -f immuta-backup-importer-deployment.yaml

Finish the Restore

Data Persistence

If you are persisting data from a previous deployment, it is best to destroy the existing PVCs for database and query-engine; otherwise, the backups will not be restored properly. If this is the case, uninstall the Helm release, delete the PVCs, and then install again using the restore method.

Make sure the following is set in your immuta-values.yaml in order to ensure backups are pulled from the PVC.

backup:
  enabled: true
  type: volume
  volume:
    claimName: immuta-backups
  restore:
    enabled: true
    queryEngineFile: immuta-00000000000000.tar.gz # This can be whatever filename you are using for your backup
    databaseFile: immuta-00000000000000.tar.gz # This can be whatever filename you are using for your backup

You may now install or upgrade Immuta using Helm:

helm install <release name> immuta/immuta --values <path to immuta-values.yaml> -n <immuta namespace>

Importing Backups from AWS and Azure

For further detail on importing backups from bucket or blob storage, refer to the backup sections of the below documentation.

AWS

Azure

Helm Installation