Veeam Agent backup job fails with "Failed to call RPC function 'TestCompatible': Error code: 0x80070008. Cannot initialize COM runtime" error

Challenge

A Veeam Agent backup job managed by the backup server fails with the following error:

Processing Error: Failed to call RPC function 'TestCompatible': Error code: 0x80070008. Cannot initialize COM runtime.

Cause

When a backup job starts, Veeam Backup & Replication connects to the Veeam Agent machine to initiate the backup process. The connection is based on Windows API and uses RPC and WMI queries. If Microsoft Windows is low on desktop heap memory for services running under LocalSystem account, Microsoft Windows can reject the incoming queries. As a result, the following message is displayed in System Event Log of the Veeam Agent machine:
 

Log Name:      System
Source:        Win32k
Event ID:      243
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:     
Description:
A desktop heap allocation failed.

DCOM errors (Event ID 10010) can be found in System Event Log of the Veeam Agent machine too.
 

Solution

To resolve this issue, update the registry key to increase the desktop heap memory size. Mind that this change affects the desktop heap of all services, so do not make the value larger than necessary. To update the registry key, do as follows:

1. Run regedit.exe.
2. Locate the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems key.
3. Locate the Windows string.

4. Edit the third "SharedSection" value in the Value data field as shown below:

       Before:

        After:

5. Click OK.
6. Reboot the Veeam Agent machine.

More Information

To learn more about desktop heap memory, see:

• Microsoft Support article
• Microsoft Developer Network article

You can check the desktop heap memory usage on your own in the following way:

1. Download Windows Software Development Kit (SDK).
2. Run Sdksetup.exe and install Debugging Tools for Windows.
3. Download LiveKd.
4. Copy livekd.exe to C:\Program Files (x86)\Windows Kits\10\Debuggers\x64.
5. Run Command Prompt.
6. Type the following commands:

livekd -y srv*http://msdl.microsoft.com/download/symbols
!dskheap

Command Prompt returns the list of sessions. If the desktop heap is running out of memory, the Used Rate(%) value for the session is close to 100%.

0: kd> !dskheap

  Winstation\Desktop            Heap Size(KB)   Used Rate(%)
------------------------------------------------------------
  WinSta0\Default                  20480                 0%
  WinSta0\Disconnect                  96                 4%
  WinSta0\Winlogon                   192                 2%
  Service-0x0-3e7$\Default           768                 4%
  Service-0x0-3e4$\Default           768                 1%
  Service-0x0-3e5$\Default           768                 0%
  Service-0x0-17eac$\Default         768                 2%
  Service-0x0-afb53$\Default         768                 0%

7. Take the last part of the session name and convert the HEX value into Decimal:
   1. Open PowerShell.
   2. Paste the last part of the session name (after the last dash).
   3. Add 0x before the last part of the session name.

8. Return to Command Prompt.
9. Type the following WMI query to retrieve all objects related to the session and the name of the account:

Get-WmiObject -Query "ASSOCIATORS OF {Win32_logonsession.LogonId=999}"

Command Prompt returns the list of processes running under that account and the name of the account:

Caption                    Name                       SID
-------                    ----                       ---
SUP-A2025\SYSTEM           SYSTEM                     S-1-5-18
LsaIso.exe                 LsaIso.exe
lsass.exe                  lsass.exe
svchost.exe                svchost.exe
…

If one of the processes running under Local System Account consumes too much memory, we recommend to kill non-Microsoft processes one by one and re-run the !dskheap command (Step 6) to find the leaking application.