Backup jobs fail intermittently with Error observed by underlying BIO:

Challenge

Intermittent network failures when communicating to the VMware host. The errors can vary but all have some variation of: 
Guest processing credentials test will also fail over VIX with this error, and you may get a pop up error bellow when editing the Vsphere host: 



But on a second try it will continue.

Cause

Unsupported network configuration in VMware.  A vm kernel nic with Management is set on a port group no longer becomes eligible to have vms on it.  If a vmkernel port with management option is added to a network with vms attached you will see the following warning in VMware. 



In Veeam we intermittently get the error “Error observed by underlying BIO: No such file or directoryDetail: 'SSL connect failed in tcp_connect()', endpoint:”,  however sometimes the jobs will work if retried enough times. 

Solution

Remove the Management vmkernel port from the vm network, or move the vms to a network without a management vmkernel port.