Offline Installation - Upgrade/Rollback

Prerequisites

Before performing the upgrade, ensure the following steps are completed: * PostgreSQL server (version 13 or later) is installed and running. * Docker and Docker Compose are available on the production server. * The Governor Docker Compose setup archive (e.g., governor-compose-prod_{new-version}.zip) is downloaded from the appropriate repository or file storage.

If you have a network-limited environment (no internet access), transfer the archive manually to the target server using a USB drive, SCP, or another method.
If needed, follow the demo setup instructions for initial setup: demo setup.

Upgrade Process

Backup Database Always back up both the data and schema together before performing any upgrades to ensure a consistent state for rollback.
- Backup Data and Schema: Create a full backup of the database (both data and schema) using the pg_dump command:
  pg_dump -U apiuser -h localhost -F c -b -v -f /path/to/backup/db-backup.full.dump governor
Download and Extract the New Version of Governor Download the new version of Governor and unzip the archive containing the updated Docker Compose configuration and other necessary files.
- Download the new governor-compose-prod.zip from the repository.
- Extract the archive:
  unzip governor-compose-prod_{new-version}.zip -d /path/to/extract/
Update the docker-compose.yml Configuration Replace the old docker-compose.yml file with the newly extracted version. Then, modify the following environment variables to match your environment:
- Update the SPRING_DATASOURCE_URL to reflect your PostgreSQL connection details (e.g., change the host if needed).
- Update the SPRING_DATASOURCE_USERNAME and SPRING_DATASOURCE_PASSWORD with your PostgreSQL credentials.
  Example:
  environment: - SPRING_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/governor - SPRING_DATASOURCE_USERNAME=apiuser - SPRING_DATASOURCE_PASSWORD=apipassword
- Ensure other environment variables (such as JWT_SECRET) are also updated according to your setup.
Pull and load the Latest Docker Images In the case of offline environments, you would need to load Docker images manually. Ensure you have already downloaded the necessary .tar files for the required images.
- Pull Docker images from:
  docker-compose pull

Save the images to the separate files using the following commands:

docker save -o /path/to/images/synthesized-api_v<new-version>_linux_amd64.tar synthesized-api:<new-version>
docker save -o /path/to/images/synthesized-front_v<new-version>_linux_amd64.tar synthesized-front:<new-version>
docker save -o /path/to/images/synthesized-agent_v<new-version>_linux_amd64.tar synthesized-agent:<new-version>

Transfer files and images to the target machine using a USB drive, SCP, or another file transfer method.
Restart Docker Containers Restart your Docker containers to apply the new version of Governor.
```
docker-compose down
docker-compose up -d
```
Verify the Update Ensure the update is successful by checking the Docker container logs and testing the application.
```
docker-compose logs
```
Access the application in the browser and confirm that the upgrade has been applied correctly.

Rollback Process (Without Dropping Schema)

If the upgrade does not go as planned and you need to roll back, follow these steps to restore the previous state:

Stop the Docker Containers First, stop the running Docker containers to prevent any further issues.
```
docker-compose down
```
Revert Docker Compose Configuration Replace the updated docker-compose.yml file with the previous version of the Docker Compose configuration, if changes were made during the upgrade. Ensure that the database connection settings (e.g., SPRING_DATASOURCE_URL, SPRING_DATASOURCE_USERNAME, and SPRING_DATASOURCE_PASSWORD) are reverted back to their original values.
Restore the Database from Backup If data corruption or issues occurred during the upgrade, restore the database from the backup you created earlier.
- Restore Full Backup: Use the pg_restore command to restore both data and schema to their previous state:
  pg_restore -U apiuser -h localhost -d governor /path/to/backup/db-backup.full.dump
Load the Previous Docker Images If the Docker images were updated during the upgrade, you may need to load the previous versions of the Docker images.
- Load Docker images from the backup:
  docker load -i /path/to/images/synthesized-api_v<previous-version>_linux_amd64.tar docker load -i /path/to/images/synthesized-front_v<previous-version>_linux_amd64.tar
Rebuild and Restart Docker Containers After restoring the data and schema, and loading the previous Docker images, bring the Docker containers back up to restore the application.
```
docker-compose up -d --build
```
Verify the Rollback Check the Docker container logs to ensure that the rollback was successful.
```
docker-compose logs
```
Access the application in the browser and confirm that the system is functioning as expected with the rolled-back data and schema.

Important Notes

Testing Rollback: It’s always a good practice to test the rollback process in a staging environment before performing it in production. Ensure that the data and schema are restored correctly, and there are no issues with the application after rollback.
Database Migrations: If you applied schema migrations during the upgrade, this process will restore the schema as it was at the time of the backup. If you prefer not to revert migrations, you will need to manually undo any schema changes.
Backup Frequency: Ensure you take regular backups of both your data and schema, especially before any significant update. You can automate this process to prevent human error.

Troubleshooting

If you encounter issues during the upgrade or rollback process, refer to the Docker container logs for detailed error messages.

docker-compose logs

You can also access individual service logs using:

docker-compose logs <service-name>