Quantcast
Channel: Veeam Support Knowledge Base
Viewing all articles
Browse latest Browse all 4362

Backup fails with: "The system cannot find the file specified." or "The device is not ready."

$
0
0

Backup fails with: "The system cannot find the file specified." or "The device is not ready."

KB ID: 4573
Product: Veeam Backup & Replication
Veeam Agent for Microsoft Windows
Published: 2024-04-18
Last Modified: 2024-04-18
Version Applicability
This article does not list specific versions of Veeam products because the underlying issue stems from a problem with the Microsoft Volume Shadow Copy Service and is independent of any Veeam product versions. As such, the issue can affect environments regardless of the Veeam software version in use.

Challenge

A backup job for a Windows machine being processed using Veeam Agent for Microsoft Windows fails during the disk read operation with either of the following errors:

  • The system cannot find the file specified.
    Asynchronous read operation failed
    Failed to upload disk.
    Agent failed to process method {DataTransfer.SyncDisk}.
    
  • The device is not ready.
    Failed to schedule asynchronous read operation
    Failed to upload disk.
    Agent failed to process method {DataTransfer.SyncDisk}.
    


Example job report:

Agent Backup job: Agent _Backup_KB4573 - XMPLSRV
 
Error
1 of 1 VMs processed
Sunday, January 01, 2024 00:00:00
Success 0 Start time 00:00:00 Total size 32.7 TB Backup size 1.3 TB  
Warning 0 End time 12:00:00 Data read 16.5 TB Dedupe 1.0x
Error 1 Duration 12:00:00 Transferred 1.2 TB Compression 2.9x
Details
Name Status Start time End time Size Read Transferred Duration Details
XMPLSRV Error 00:00:00 12:00:00 32.7 TB 16.5 TB 1.2 TB 12:00:00 New Volume (X:) (16.1 TB)
Error: The system cannot find the file specified. Asynchronous read operation failed Failed to upload disk. Skipped arguments: [EmulatedDiskSpec>]; Agent failed to process method {DataTransfer.SyncDisk}. Exception from server: The system cannot find the file specified. Unable to retrieve next block transmission command. Number of already processed blocks: [8675309]. Failed to download disk 'b4cf8db1-9710-4043-8150-a9ba11d70844'
Processing finished with errors at 01/01/2024 12:00:00

Cause

These errors occur either because:

  • The shadow copy that Veeam Agent for Microsoft Windows was reading from has been unexpectedly removed.

    or
  • The shadow copy that Veeam Agent for Microsoft Windows requested the VSS subsystem create was unable to be created.
     

Veeam Agent for Microsoft Windows issues requests to the Microsoft Software Shadow Copy Provider to create a volume shadow copy during a backup. If the shadow copy cannot be created or is unexpectedly removed during the backup operation, the backup job will fail.

Solution

Since these issues stem from the Microsoft Volume Shadow Copy Service, it is essential to examine the System Event logs for 'volsnap' events to pinpoint the root cause within your environment. Typically, the most effective resolution for such issues involves redirecting shadow copy storage to a volume that has ample free space and a high number of available IOPS.
Example Error
Example volsnap Error
Event Filter
Event Filter Example

General Troubleshooting Advice

The 'volsnap' error will provide details about which drive is impacted and the underlying issue.

For example:

  • The most common Volsnap error reported by customers is:

    Event ID: 25
    Description: The shadow copies of volume X: were deleted because the shadow copy storage could not grow in time.  Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.
    

    As the error indicates, the shadow copy was deleted when the shadow copy could not grow. Since Veeam Agent for Microsoft Windows was reading from that shadow copy, the backup immediately fails with, "The system cannot find the file specified. Asynchronous read  operation failed." The underlying cause, as explained in the error, is that the disk's IO load was too high. The solution is to reduce the disk IO load, implement faster storage, or redirect shadow copies to a volume with less IO load. Customers most commonly select redirection of shadow copies, as it requires no infrastructure changes.

  • The next most common Volsnap error reported by customers is:

    Event ID: 36
    Description: The shadow copies of volume X: were aborted because the shadow copy storage could not grow due to a user imposed limit.
    

    Like Event ID:25 above, the shadow copy could not grow and instead was deleted. This again causes the Veeam Agent for Microsoft Windows backup job to fail with the error, "The system cannot find the file specified. Asynchronous read operation failed." In this case, the resolution is to review the shadow copy storage limitations or redirect shadow copies to a volume with more storage space.

Additional examples of Volsnap errors can be found in the More Information section of this article.

Manage Shadow Copy Settings

The most common solution for almost all issues is redirecting shadow copy storage to a volume with sufficient free space and plenty of available IOPS.

Click a section below to expand and view instructions.

Configure Shadow Copies Using the Shadow Copies Utility (Server OS Only)
Shadow Copies Utility (Server OS Only)
  1. Right-click any volume and choose Configure Shadow Copies.
    Note: On older server operating systems, this may instead be a tab in Properties.
  2. Select the volume mentioned in the volsnap error.
  3. Click Settings…
  4. Review and update the Storage area settings.
    • If the volsnap error that occurred implies that there is an issue with the storage, try assigning the shadow copy storage to a drive that is expected to have less IO load.
    • If the volsnap error mentions a "user defined limit" or "user imposed limit", update the Maximum size setting to large value or set no limit.
servershadows
Volume Shadow Copy Service supports a volume size of up to 64 TB, larger volumes cannot be selected as storage area. Manual changes may cause software malfunction.
Configure Shadow Copies Using the Command Line (Desktop and Server OS)

Note: All commands below must be run from an Administrative Command Prompt.

View Existing Shadow Storage Associations

Run the following command to view existing shadow storage associations.
If no shadow storage exists for any volume, the command response will be “No items found that satisfy the query.”

vssadmin list shadowstorage
Add Shadow Storage Associations

If no association is listed for the volume, use the following command to change the location of the shadow storage area:

vssadmin add shadowstorage /for=X: /on=Z: /maxsize=##GB

For example, to redirect shadows from D: to E:

vssadmin add shadowstorage /for=D: /on=E: /maxsize=50GB
Resize Shadow Storage Association

If a shadow storage association already exists, use the following command to change the size shadow storage area:

vssadmin resize shadowstorage /for=X: /on=Z: /maxsize=###GB

For example, to resize the shadow storage in place for a redirection from from D: to E:

vssadmin resize shadowstorage /for=D: /on=E: /maxsize=200GB

If no value is specified for /maxsize, there is no limit placed on the amount of storage space that can be used.
Otherwise, a value of 15-20% of the source volume’s size is generally recommended for busy servers.

More Information

Common Volsnap Errors Reported by Customers

  • Event ID: 5 — Description: The shadow copy of volume E: could not be created due to insufficient non-paged memory pool for a bitmap structure.
  • Event ID: 8 — Description: The flush and hold writes operation on volume D: timed out while waiting for a release writes command.
  • Event ID: 9 — Description: The flush and hold writes operation on volume E: timed out while waiting for file system cleanup.
  • Event ID: 14 — Description: The shadow copies of volume E: were aborted because of an IO failure on volume R:.

    In this example, the shadow copies for E: were redirected to R:, but the underlying storage for R: was failing.
  • Event ID: 16 — Description: The shadow copies of volume R: were aborted because volume , which contains shadow copy storage for this shadow copy, was force dismounted.

    In this example, the volume where the shadow copies were being stored for R: was dismounted during the backup operation (i.e., drive failure).
  • Event ID: 20 — Description: The shadow copies of volume S: were aborted because of a failed free space computation.
  • Event ID: 24 — Description: There was insufficient disk space on volume C: to grow the shadow copy storage for shadow copies of C:. As a result of this failure all shadow copies of volume C: are at risk of being deleted.
  • Event ID: 25 — Description: The shadow copies of volume T: were deleted because the shadow copy storage could not grow in time.  Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.

    In this example, the T: drive was where the exchange transaction logs were stored and the IO load on the volume was too high to grow the shadow copy, resulting in the need to redirect shadow copies to a different volume.
  • Event ID: 27 — Description: The shadow copies of volume L: were aborted during detection because a critical control file could not be opened.
  • Event ID: 28 — Description: The shadow copy of volume C: could not be created due to a failure in creating the necessary on disk structures.
  • Event ID: 32 — Description: The shadow copies of volume H: were aborted because the shadow copy storage volume was not present.
  • Event ID: 33 — Description: The oldest shadow copy of volume C: was deleted to keep disk space usage for shadow copies of volume C: below the user defined limit.
  • Event ID: 35 — Description: The shadow copies of volume C: were aborted because the shadow copy storage failed to grow.
  • Event ID: 36 — Description: The shadow copies of volume F: were aborted because the shadow copy storage could not grow due to a user imposed limit.
  • Event ID: 39 — Description: When preparing a new volume shadow copy for volume C:, the shadow copy storage on volume C: could not be located in non-critical space. Consider using a shadow copy storage volume that does not have any shadow copies.
  • Event ID: 41 — Description: When preparing a new volume shadow copy for volume C:, the shadow copy storage on volume C: did not have sufficiently large contiguous blocks. Consider deleting unnecessary files on the shadow copy storage volume or use a different shadow copy storage volume.
To submit feedback regarding this article, please click this link: Send Article Feedback
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

Viewing all articles
Browse latest Browse all 4362

Trending Articles