Veeam Kasten GSB backups fail with the error "mkdir /tmp/kopia-log: read-only file system"
Challenge
The backup for the workloads that uses Generic Storage Backup (GSB) fails with the error:
mkdir /tmp/kopia-log: read-only file system\r\nUnable to create logs directory: mkdir /tmp/kopia-log: read-only file system
Cause
Veeam Kasten for Kubernetes's datamover needs write access to the /tmp directory for storing the config files and log files. If readOnlyRootFilesystem is set to true in the securityContext of the kanister-sidecar container, any writes to /tmp are restricted, causing the backup in Kasten to fail with the above error message.
Starting from Veeam Kasten for Kubernetes version 7.0.8, the automatic sidecar injector for GSB merges the securityContext of all the other containers in the workload and selects the most restrictive securityContext for the kanister-sidecar container.
Due to this change, there may be instances where customers need to manually adjust the kanister-sidecar to ensure the GSB functions properly. For example, when the usage of readOnlyRootFilesystem securityContext in any application containers in which the sidecar is injected.
Solution
This issue can be mitigated by updating the kanister-sidecar container in the customers' workload by adding an emptyDir volumeMount for the /tmp so that the datamover can write its config/log files to that directory.
Below is an example command to patch the kanister-sidecar container in a deployment with the emptyDir volume.
More Information
By default, data operations done by Kasten requires the Linux Capabilities 'CHOWN', 'DAC_OVERRIDE', and `FOWNER' to be allowed to run in the container.
Starting in Veeam Kasten for Kubernetes v7.0.8, any workload that is injected with a sidecar will automatically incorporate the necessary capabilities. However, existing workloads that have already been injected with sidecars will require a manual update to include these capabilities.
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.