Challenge
Data transfer is failing across a connection that is protected by an advanced firewall. The firewall uses signature-based detection. Relevant features may have names like antivirus, anti-spyware, intrusion prevention, or application control.This may affect any make or model of firewall with similar features, but support cases have been opened for Dell SonicWALL, Check Point, Cisco, Palo Alto, and Fortinet appliances.
Any type of data transfer may be affected, including backup, replication, and backup copy jobs. This problem may manifest in a wide variety of error messages and failure patterns. It may appear random, or completely consistent. One common failure pattern is for transfer of specific VM disks to consistently fail at or around the same percentage of completion.
Common error messages include:
An existing connection was forcibly closed by the remote host.
Unstable connection: unable to transmit data.
Depending on the job type and the version of Veeam Backup & Replication, connection failures may cause the job to fail immediately, or the connection may retry several times. While the connection is retried, the job may appear to be frozen because it is unable to transmit data. For more information, see:
Resume on Disconnect
Backup Copy: Automatic Job Retries
WAN Acceleration: Data Transport on WAN Disconnect
Cause
Disk images contain a potentially unlimited variety of data blocks. Because the traffic is compressed (and in most cases, encrypted), data blocks analyzed by a firewall will be different from data as it exists on the virtual disk. Over the long term, this approximates feeding random data into the signature-based threat detector: false positives are inevitable.Data transfer is not actually random: a particular data block will always have the same signature after compression and encryption. If the source data does not change, the same blocks will be re-sent on every reconnect attempt and every retry of the job. In this case, the firewall will close the network connection every time Veeam Backup & Replication tries to transfer that data block, because it incorrectly detects a pattern of data within that block as matching the signature of a known threat.
Solution
Create exclusions for Veeam data traffic. In most cases, the relevant traffic will be between proxies or repositories over a default port range of TCP 2500-5000. This range can be configured for each managed server in the backup infrastructure settings.Cloud Connect Service Providers should create exclusions for data traffic sent to Cloud Gateways on port 6180 (TCP and UDP).
For more information on port ranges, find the relevant product or component user guide in the Help Center and consult the Used Ports page.
For information on how to configure exclusions on a specific firewall appliance, please contact your vendor.
More Information
To isolate whether or not an issue is caused by a firewall, temporarily disable all signature-based features in the firewall’s configuration. For best results, do this while data transfer appears to be frozen – traffic should resume in no more than a few minutes. In some cases, the firewall may allow you to selectively disable specific sites or zones; this can be a useful as a solution, but it is not a good isolation step because such features are easily misconfigured.Reset packets generated by firewall appliances can usually be distinguished from normal traffic by their IP time to live. For example, if most packets in a TCP stream have a TTL of 128, but the reset packet that closes the stream has a TTL of 64, the connection was closed by a firewall.
Firewall features that block encrypted key exchange will block the majority of WAN connections used by Veeam Backup & Replication.