Challenge
This KB documents the process for forcibly terminating all jobs for a given Veeam Backup & Replication installation. The steps outlined in this KB will terminate all jobs.Note: Some jobs may take some time to stop, please allow up to 60 minutes for a jobs to stop on their own before forcibly terminating them.
Solution
The problem may occur because a process or task that the job is waiting for has not been completed. Such a task can be snapshot removal. Therefore, as the first step it is recommended to check the vSphere Client to see if there is a snapshot removal process pending/working on the vSphere side.The following steps will forcibly terminate ALL jobs/restores.
- Close the Veeam Backup & Replication console.
- Stop all services that begin with Veeam.
- Open the Task Manager on the Veeam Server and kill all VeeamAgent.exe processes.
Note: Some VeeamAgent.exe process will be located on Source Proxies and Windows Repositories that are not the Veeam Server. - Wait 5-10 minutes for the tasks to timeout and fail.
- Remove snapshots from VM(s) that are part of the stuck jobs. (If they have not been already removed).
- Start the services that were stopped in step 2.
Note. If you use the Virtual Appliance (HOTADD) mode, before removing the snapshots make sure there are no stuck disks on the Veeam Backup server or one of the backup proxies. Otherwise, the snapshots can be orphaned. https://www.veeam.com/kb1775
─────────────────────────
If the jobs are still listed as “Stopping” after performing the steps outlined above please proceed through the following steps.
For information on how to apply SQL scripts please review https://www.veeam.com/kb1443.
- Back up the Veeam database. (http://www.veeam.com/kb1471)
- Run the following query against the VeeamBackup database:
SET QUOTED_IDENTIFIER ON
UPDATE [Backup.Model.JobSessions]
SET [state] = '-1'
WHERE [state] != '-1'
If running this under SQL Server Management Studio, typically the statement SET QUOTED_IDENTIFIER ON is not needed. However, if using SQLCMD, it is almost always required, else the query will fail.
Once run, you should see a specified number of rows reported affected by the query. Allow up to 60 second for the UI to update to reflect this. Typically, the update happens quickly, but it is not real-time.
UPDATE [Backup.Model.JobSessions]
SET [state] = '-1'
WHERE [state] != '-1'
If running this under SQL Server Management Studio, typically the statement SET QUOTED_IDENTIFIER ON is not needed. However, if using SQLCMD, it is almost always required, else the query will fail.
Once run, you should see a specified number of rows reported affected by the query. Allow up to 60 second for the UI to update to reflect this. Typically, the update happens quickly, but it is not real-time.