Helm-based Kubernetes Deployment - Upgrade/Rollback

Prerequisites

Before performing the upgrade, ensure the following steps are completed: * Helm is installed and configured to interact with your Kubernetes cluster. * The Governor Helm chart archive (e.g., governor-helm-chart-{new-version}.tgz) is downloaded from the appropriate repository or file storage.

If you have a network-limited environment (no internet access), transfer the archive manually to the target server using a USB drive, SCP, or another method.

Upgrade Process

Backup Database Always back up both the data and schema together before performing any upgrades to ensure a consistent state for rollback.
- Backup Data and Schema: Create a full backup of the database (both data and schema) using the pg_dump command:
  pg_dump -U apiuser -h localhost -F c -b -v -f /path/to/backup/db-backup.full.dump governor
Download the New Version of Governor Helm Chart Download the new version of the Governor Helm chart. This typically contains the updated values.yaml configuration file and other necessary Kubernetes setup files.
- Download the new Governor Helm chart archive from the repository:
  - Example: governor-helm-chart-{new-version}.tgz
Upgrade the Helm Release Use Helm to upgrade the existing release with the new version of the chart:
```
helm upgrade <release-name> /path/to/governor-helm-chart-{new-version}.tgz --values /path/to/values.yaml
```
Ensure that the necessary environment variables such as database credentials and JWT_SECRET are correctly set in the values.yaml. In case you have no access to existing values.yaml file, you can get actual values with 'helm get values <release-name>' command and save them to a new file.
Verify the Update Ensure the update is successful by checking the status of the release and Kubernetes pods:
```
kubectl get pods
kubectl describe deployment <release-name>
kubectl logs <pod-name>
```
Access the application in the browser and confirm that the upgrade has been applied correctly.

Rollback Process (Without Dropping Schema)

If the upgrade does not go as planned and you need to roll back, follow these steps to restore the previous state:

Restore the Database from Backup If data corruption or issues occurred during the upgrade, restore the database from the backup you created earlier.
- Restore Full Backup: Use the pg_restore command to restore both data and schema to their previous state:
  pg_restore -U apiuser -h db_host -d governor /path/to/backup/db-backup.full.dump
Rollback the Helm Release Rollback the Helm release with the previous version.
```
helm rollback <release-name>
```
Verify the Rollback Check the Kubernetes logs to ensure that the rollback was successful.
```
kubectl logs <pod-name>
```
Access the application in the browser and confirm that the system is functioning as expected with the rolled-back data and schema.

Important Notes

Testing Rollback: It’s always a good practice to test the rollback process in a staging environment before performing it in production. Ensure that the data and schema are restored correctly, and there are no issues with the application after rollback.
Database Migrations: If you applied schema migrations during the upgrade, this process will restore the schema as it was at the time of the backup. If you prefer not to revert migrations, you will need to manually undo any schema changes.
Backup Frequency: Ensure you take regular backups of both your data and schema, especially before any significant update. You can automate this process to prevent human error.

Troubleshooting

If you encounter issues during the upgrade or rollback process, refer to the Kubernetes pod logs for detailed error messages.

kubectl logs <pod-name>