Challenge
When accessing backup files on a CIFS (SMB) repository, Veeam Backup and Replication reports either of the following errors:
The specified network name is no longer available. (code 59, 0x8007003B)
An unexpected network error occurred. (code 64, 0x80070040)
Veeam Backup & Replication uses SMB connections for backup repositories, guest processing, and when first adding a Windows server to the Backup Infrastructure. This KB only addresses error messages that occur in the context of a backup file path, such as:
The specified network name is no longer available. Failed to read data from the file [\\10.0.0.81\backups\SQL VM New\SQL VM New2015-09-23T200441.vbk].
Note: If the failure is occurring in a tape job and no UNC path is specified in the error message, please contact Veeam Support.
Cause
These error messages are passed directly from Microsoft Windows, and indicate that some unknown problem led to a sudden failure of the connection between the repository gateway (acting as Server Message Block client) and the SMB server (typically a NAS appliance).
There are many possible reasons the SMB connection might fail:
- Network issues are a common reason, constituting a potentially unlimited variety of hardware failure or network hardware configuration problems; such issues may or may not involve packet loss.
- Simple firewalls may block a connection consistently, while advanced firewalls might destroy an existing connection seemingly at random or under very specific conditions.
- Either client or server might close the connection due to one of a number of timeouts or retry limits either at the SMB protocol layer or the underlying TCP layer; such limits are often encountered due to network congestion or excessive load on the storage.
- The NAS may stop responding entirely due to hardware or software issues; it may in some cases be misconfigured. Appliances using post-process deduplication may stop responding if the process starts prematurely.
- In specific circumstances, various GPO security policies might limit the connection, for instance - Microsoft network client: Digitally sign communications. Make sure that source and target hosts are able to communicate using this policy, otherwise, in case of any network failures, consider this to disable. More QoS policies can be enabled on SMB client or target, creating unreasonably low bandwidth.
Solution
Common Workarounds
- If the repository share is located on a Windows server, try creating a Windows repository on that server instead of a CIFS (SMB) repository.
- Reboot both the SMB client machine (repository gateway) and the SMB server (NAS).
- If other storage is available, try writing the backup to the other storage as both verification of the problem and as a temporary workaround.
- Consider reconfiguring the NAS as iSCSI or an NFS mount on a separate Linux repository server, depending on what is supported by the NAS. This change may involve data loss. iSCSI usually offers the best performance.
- Writing backup files to SMB shares over a slow or unreliable network, such as a WAN connection, is not recommended. Deploy a gateway server with a fast (Ethernet 100Mbps+) connection to the share.
- To rule out interference from a firewall, temporarily disable any software firewalls (including certain antivirus products) on the SMB client or server, as well as any hardware firewalls between them.
Registry Settings
These registry values must be created on the the SMB gateway, which is specified in the repository settings as “Gateway server”. If the gateway is set to “automatic”, add these registry values to all Windows servers managed by Veeam Backup & Replication.
- NetUseShareAccess
For Veeam B&R 8.0.0.917 or later, the following registry value allows Data Movers to open backup files by a resilient method similar to the Windows net use command.
Key: HKLM\SOFTWARE\Veeam\Veeam Backup and Replication\
Note: If credentials are required, specify Domain\User or Host\User in the repository settings. .\User credentials are not supported with NetUseShareAccess. For example, specify “NAS-01\Admin” instead of “.\Admin”.
DWORD: NetUseShareAccess
Value: 1
- SessTimeout
This increases the amount of time the Windows SMB client will wait for a response from an SMB server before it aborts the connection. The default timeout is one minute.
Note: Reboot Required
Key: HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\
Note: This is a value in seconds. Try a value of 600 decimal (10 minutes).
DWORD: SessTimeout
Value: 600
- TcpMaxDataRetransmissions
This increases the number of times the Windows TCP implementation will retransmit a data segment before it aborts the connection.
Note: Reboot Required
Key: HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
DWORD: TcpMaxDataRetransmissions
Value: 10 (Default: 5)
Additional Troubleshooting
- Open the repository settings and enable Limit combined data rate. If you can measure the existing data rate, start with a limit between 50% and 70% of that value to verify that reducing throughput prevents the error. If the network is still congested after limiting the repository data rate, and the source of the congestion is due to other Veeam processes, enable throttling for the relevant connections.
- Change the gateway server in the repository settings to test whether particular routes to the share perform better than others. For example, if connections from any Windows server on an ESXi host tend to fail, but connections from a physical Windows machine do not fail, the root cause probably involves that ESXi host or its network connection.
- If the SMB connection consistently fails after 10 hours, it may be due to expiration of a Kerberos service ticket. The most reliable workaround is to create non-domain credentials on the NAS, then specify those credentials in the repository settings. For more information and other possible workarounds, see this SAMBA mailing list discussion and the Authentication Expiration Timer heading in this MSDN blog post.