> For the complete documentation index, see [llms.txt](https://documentation.immuta.com/latest/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.immuta.com/latest/configuration/self-managed-deployment/troubleshooting.md). # Troubleshooting ## Upgrade failure 1. List the upgrade jobs and find the one that failed. ```sh kubectl get jobs ``` 2. Read the job's log. ```sh kubectl logs job/ ``` 3. Use the [upgrade failure messages table](#upgrade-failure-messages) to determine remediation steps. #### Upgrade failure messages | Message | Description | Fix | | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `secure.ingress was removed in 2026.2` | The chart refuses to upgrade while your values file contains a `secure.ingress` block because applying it would delete your existing [ingress](/latest/configuration/self-managed-deployment/upgrade-immuta.md#ingress-migration) and its cloud load balancer. No changes were applied to your deployment. | Follow the ingress migration steps in the [upgrade guide](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta), and then run the upgrade again. | | `onboarding.postgres.database is required` | 2026.2 introduces the [Onboarding service](/latest/configuration/self-managed-deployment/upgrade-immuta.md#onboarding-service), which requires `onboarding.postgresql.database` to be set in your values file to the same database name Secure uses. No changes were applied to your deployment. | Follow the [upgrade guide](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta) to add the Onboarding service, and then run the upgrade again. | | `onboarding.postgresql.password is required` | Onboarding could not find a database password. This happens when your database password is stored in an existing Kubernetes secret (`secure.existingSecret`), because Onboarding does not read that secret. No changes were applied to your deployment. | Add `ONBOARDING_DATABASE_URL` to the secret and set `onboarding.existingSecret`, as described in [Existing Kubernetes secrets](/latest/configuration/self-managed-deployment/upgrade-immuta.md#existing-kubernetes-secrets), and then run the upgrade again. | | `couldn't find key ONBOARDING_DATABASE_URL in Secret` on a secure pod | Your existing Kubernetes secret does not contain the `ONBOARDING_DATABASE_URL` key, which Secure reads in 2026.2. | Add the key to the secret as described in [Existing Kubernetes secrets](/latest/configuration/self-managed-deployment/upgrade-immuta.md#existing-kubernetes-secrets). The pods start once the key exists. | | `global.tenantId must be a valid UUID` | The [tenant ID](/latest/configuration/self-managed-deployment/upgrade-immuta.md#tenant-id) is not set to a UUID. No changes were applied to your deployment. | Set the tenant ID to a UUID and run the [upgrade again](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta). | | `TENANT_ID must be a valid UUID and must not be the all-zeros UUID` (in the `audit-init` job) | There is no `global.tenantId` set in the immuta-values.yaml file. | Set the tenant ID to a UUID and run the [upgrade again](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta). | | `Missing required environment variables for self-managed use: SEARCH_INDEX_NAME, SEARCH_INDEX_ALIAS_NAME` (in the `audit-init` job) | Index names were not set. | Set these environment variables and run the [upgrade again](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta). | | `UPGRADE FAILED: ... timed out` while the `secure-database-migrate` job is still running | The Helm timeout was too short. | Wait for the job to complete (`kubectl get jobs`), and then run the [upgrade again](/latest/configuration/self-managed-deployment/upgrade-immuta.md#upgrade-immuta) with a longer `--timeout`. | | `ImagePullBackOff` on a job pod | Registry credentials have expired. | Recreate the registry secret from the [installation guide](/latest/configuration/self-managed-deployment/install/managed-public-cloud.md#create-registry-secret) and delete the stuck pod; the job will retry. | ## PostgreSQL ### How do I determine if the database is accepting connections? 1. Create a pod named `debug-postgres` and spawn an interactive shell. ```bash kubectl run debug-postgres --stdin --tty --rm --image docker.io/bitnami/postgresql:latest -- sh ``` 2. Validate that the database is listening. ```bash pg_isready --host --port 5432 ``` ## Redis ### How can a TCP connection be established without using Redis CLI? 1. Create a pod named `debug-redis` and spawn an interactive shell. ```bash kubectl run debug-redis --stdin --tty --rm --image docker.io/rockylinux/rockylinux:9 -- sh ``` 2. Send a raw TCP message to the database using Netcat. ```bash nc -zv 6379 ``` ### How do I establish a TCP connection? 1. Create a pod named `debug-redis` and spawn an interactive shell. ```bash kubectl run debug-redis --stdin --tty --rm --image docker.io/bitnami/redis:latest -- sh ``` 2. Establish a connection to the database using the Redis client. If a connection can be established with Netcat and the `redis-cli` command does not return, then Redis could be expecting a TLS connection. Pass option `--tls`. ```sh redis-cli -h -p 6379 ``` ## Elasticsearch ### How do I query the API using cURL? 1. Create a pod named `debug-elasticsearch` and spawn an interactive shell. ```bash kubectl run debug-elasticsearch --stdin --tty --rm --image docker.io/rockylinux/rockylinux:9 -- sh ``` 2. Install package `curl`. ```bash dnf install curl ``` 3. Check the [cluster health](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html). ```sh curl --fail --request GET "http://:9200/_cluster/health?pretty" ``` {% hint style="info" %} **Basic authentication** Depending on the cluster's configuration it might be necessary to use [basic auth](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication). Pass option `--header "Authorization: Basic $token"` where token equals `$(printf '%s:%s' "" "" | base64)` {% endhint %} ## Helm ### When installing the helm chart from ocir.immuta.com, I get error `scheme "oci" not supported`. What's going on? The Immuta Enterprise Helm chart (IEHC) is distributed as an OCI artifact, and your current Helm version might not support it. Refer to the [Helm documentation](https://helm.sh/docs/topics/registries/#oci-support-prior-to-v380) for further assistance. 1. Determine your Helm version. ```bash helm version ``` 2. If older than 3.8.0 you'll need to upgrade. Before this version OCI support wasn't enabled by default. ## Kubernetes ### `kubectl wait` timed out and some pods never became ready. How do I find out why? 1. Identify the pods that are not ready. ```bash kubectl get pods ``` 2. Inspect the events for a failing pod. Look for image pull errors (check your registry secret) and failed probes. ```bash kubectl describe pod ``` 3. Read the pod's logs. Failed connections to PostgreSQL or Elasticsearch are the most common cause; verify that each [FQDN is resolvable](#how-can-i-ensure-the-fully-qualified-domain-name-fqdn-is-resolvable-from-within-the-kubernetes-cluster) and accepting connections from within the cluster. ```bash kubectl logs --previous ``` ## Frequently asked questions ### How can I ensure the fully qualified domain name (FQDN) is resolvable from within the Kubernetes cluster? 1. Create a pod named `debug-dns` and spawn an interactive shell. ```bash kubectl run debug-dns --stdin --tty --rm --image docker.io/rockylinux/rockylinux:9 -- sh ``` 2. Install package `bind-utils`. ```bash dnf install bind-utils ``` 3. Perform DNS lookups on a given FQDN. ```bash dig ``` ### I'm unsure which Kubernetes namespace or Helm release is associated with my Immuta installation. How can I find this out? ```bash helm list --all-namespaces --output json | jq '.[]|select(.chart | startswith("immuta"))' ``` ### I no longer have my `immuta-values.yaml` Helm values file. How do I recover this file? ```bash helm get values > immuta-values.yaml ``` ### I don't want to keep passing option `--namespace` every time I run a Helm command. How do I set a default? ```bash kubectl config set-context --current --namespace= ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://documentation.immuta.com/latest/configuration/self-managed-deployment/troubleshooting.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.