# Helm-based Kubernetes Deployment - Upgrade/Rollback

> Source: https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/updating_tdk/update_helm
> For the complete documentation index, see [llms.txt](https://docs.synthesized.io/llms.txt).

## Prerequisites

Before performing the upgrade, ensure the following steps are completed: \* **Helm** is installed and configured to interact with your Kubernetes cluster. \* The **Governor Helm chart archive** (e.g., `governor-helm-chart-{new-version}.tgz`) is downloaded from the appropriate repository or file storage.

- If you have a network-limited environment (no internet access), transfer the archive manually to the target server using a USB drive, SCP, or another method.

## Upgrade Process

1. **Backup Database** Always back up both the data and schema together before performing any upgrades to ensure a consistent state for rollback.
  
  - **Backup Data and Schema**: Create a full backup of the database (both data and schema) using the `pg_dump` command:
    
    ```bash
    pg_dump -U apiuser -h localhost -F c -b -v -f /path/to/backup/db-backup.full.dump governor
    ```
2. **Download the New Version of Governor Helm Chart** Download the new version of the Governor Helm chart. This typically contains the updated `values.yaml` configuration file and other necessary Kubernetes setup files.
  
  - Download the new **Governor Helm chart** archive from the repository:
    
    - Example: `governor-helm-chart-{new-version}.tgz`
3. **Upgrade the Helm Release** Use Helm to upgrade the existing release with the new version of the chart:
  
  ```bash
  helm upgrade <release-name> /path/to/governor-helm-chart-{new-version}.tgz --values /path/to/values.yaml
  ```
  
  Ensure that the necessary environment variables such as database credentials and `JWT_SECRET` are correctly set in the `values.yaml`. In case you have no access to existing values.yaml file, you can get actual values with 'helm get values <release-name>' command and save them to a new file.
4. **Verify the Update** Ensure the update is successful by checking the status of the release and Kubernetes pods:
  
  ```bash
  kubectl get pods
  kubectl describe deployment <release-name>
  kubectl logs <pod-name>
  ```
  
  Access the application in the browser and confirm that the upgrade has been applied correctly.

## Rollback Process (Without Dropping Schema)

If the upgrade does not go as planned and you need to roll back, follow these steps to restore the previous state:

1. **Restore the Database from Backup** If data corruption or issues occurred during the upgrade, restore the database from the backup you created earlier.
  
  - **Restore Full Backup**: Use the `pg_restore` command to restore both data and schema to their previous state:
    
    ```bash
    pg_restore -U apiuser -h db_host -d governor /path/to/backup/db-backup.full.dump
    ```
2. **Rollback the Helm Release** Rollback the Helm release with the previous version.
  
  ```bash
  helm rollback <release-name>
  ```
3. **Verify the Rollback** Check the Kubernetes logs to ensure that the rollback was successful.
  
  ```bash
  kubectl logs <pod-name>
  ```
  
  Access the application in the browser and confirm that the system is functioning as expected with the rolled-back data and schema.

## Upgrading from `agent` to `worker` naming

Starting with chart version 1.153.0, the `agent` component has been renamed to `worker`. The upgrade is handled automatically, but depending on your setup you may need to take action.

### Default behavior (no action needed for most users)

On upgrade, the chart renames Kubernetes resources from `governor-agent` to `governor-worker`. Helm’s own three-way merge deletes the old Deployment, ConfigMap, and Secret automatically. The new `governor-worker` Deployment starts in their place.

Legacy `agent:` and `tdkAgents:` values in your `values.yaml` still work — the chart merges them on top of the new `worker:` / `tdkWorkers:` defaults. You will see deprecation warnings in the Helm output until you migrate your values file.

### Preserving the old resource name

If you have monitoring, NetworkPolicies, ServiceMonitors, or scripts that reference `governor-agent` by name, add this to your `values.yaml` **before** upgrading:

```yaml
agent:
  name: governor-agent
```

This keeps the existing Kubernetes resource names unchanged. No PVC annotation or cleanup is needed in this case.

### Worker persistence (PVC)

If you had `agent.persistence.enabled: true`, the pre-upgrade hook automatically annotates `agent-pvc` with `helm.sh/resource-policy: keep` so that Helm does not delete it. Your data is preserved.

However, the renamed `governor-worker` Deployment will **not** mount the legacy PVC unless you explicitly opt in:

```yaml
worker:
  persistence:
    enabled: true
    claimName: agent-pvc
    name: worker-pv
    mountPath: /app/rocksdb
```

Alternatively, keep `agent.name: governor-agent` along with your existing `agent.persistence` block — the old names stay, and everything mounts as before.

If the hook cannot annotate the PVC (e.g. insufficient RBAC or an admission webhook), **the upgrade will abort** to prevent data loss. The Job logs will show the exact error and remediation steps.

If you do **not** need the existing worker data, set `migrationCleanup.deleteLegacyPvc: true` in your values. The hook will then skip the annotation and Helm will delete the PVC during the upgrade.

### Migrating your values file

When you are ready to adopt the new naming:

1. Rename `agent:` to `worker:` (the structure is identical).
2. Rename `tdkAgents:` to `tdkWorkers:` if present.
3. Do not keep both `agent:` and `worker:` blocks — `agent:` overrides take precedence on conflict, which can be confusing.

## Important Notes

- **Testing Rollback**: It’s always a good practice to test the rollback process in a staging environment before performing it in production. Ensure that the data and schema are restored correctly, and there are no issues with the application after rollback.
- **Database Migrations**: If you applied schema migrations during the upgrade, this process will restore the schema as it was at the time of the backup. If you prefer not to revert migrations, you will need to manually undo any schema changes.
- **Backup Frequency**: Ensure you take regular backups of both your data and schema, especially before any significant update. You can automate this process to prevent human error.

## Troubleshooting

If you encounter issues during the upgrade or rollback process, refer to the Kubernetes pod logs for detailed error messages.

```bash
kubectl logs <pod-name>
```