Challenge
Web plugin is a new feature, which has little in common with other features and it may seem to be difficult to troubleshoot. It is suggested to follow this guide rather than trying to analyze error message, as it is not very informative and almost always misleading.Solution
Known issues and solutionsUnable to connect Enterprise Manager
Make sure that Enterprise Manager and RestAPI services are running. Please disable proxy in Internet Explorer on vCenter and EM servers, disable firewalls.
If it doesn't help, then review plugin log on vCenter, if you see something like "Certificate error: Remote servers certificate thumbprint does not match the thumbprint stored in the application settings" then
1. Perform plugin uninstall with cleanup(see section below)
2. Reinstall Enterprise Manager and select required certificate during installation
Invalid SSO Token
Steps to resolve:
1. Make sure, that EM and RestAPI services are running under domain account with local administrator priveleges.
2. Add user(that is used to log on to vSphere web client) in EM Roles explicitly.
If it doesn't help, then analyze SSO token.
Server error: null
Plugin files are not completely installed, corrupted or 2 versions of plugin are installed. If you have vShpere 5.5 then plugin might have old version.
Steps to resolve:
1. Verify plugin version installed in EM(Configuration - vCenter Servers) and EM version(EM - configuration - about - Veeam Backup Enterprise Manager Service). For vSphere 5.5 it should be not lower than 7.0.0.764. If needed, apply the latest patch once again. If Enterprise Manager and Veeam B&R are installed on different servers, make sure that the latest patch is installed on both servers.
2. Perform Plugin uninstall with cleanup
3. Install plugin
Installation fails with message "Check if there is an existing one already installed" but the plugin is not already installed
If Svc.VeeamBES.log contains “NoPermissionFault”: Make sure the user account that is accessing the vCenter is listed in the vCenter permissions at the root level (or is part of a group at that level), and is assigned to a role with Extension privileges.
If Svc.VeeamBES.log contains “null reference exception”: In IIS Manager on the Enterprise Manager server, make sure that an SSL certificate is assigned to the HTTPS binding for the VeeamBackup site.
Generic troubleshooting guide
If the issue is not listed in known issues section or it didn't help, then follow this guide before escalating a ticket.
1. Disable proxy in IE and firewall both on EM and VC
2. Verify that EM version is current (EM - configuration - about - Veeam Backup Enterprise Manager Service) and install latest patch if needed. In case of upgrade, uninstall plugin from EM, restart vSphere web client service and install it again.
3. Add user(used to log on to VC) to EM roles explicitly
4. Make sure that RestAPI and EM services are running under domain account with local admin priveleges
Check if RestAPI service is available and responding:
1. Login to EM server and open in browser https://<em-server>:9398/web. You should receive an XML. Username and password may need to be entered. You can safely ignore untrusted certificate warning.
2. If this page doesn't open, check if RestApi service is started
3. If service is started, run netstat -abn from command line to check if port 9398 is listened by Veeam.Backup.EnterpriseService.exe
4. Try to open "http://<em-server>:9399/web", if this page opens while "https://<em-server>:9398/web" doesn't - you need to re-install SSL certificate for RestAPI (see corresponding chapter below - RestAPI certificate change)
If nothing, listed above helps:
1. Perform plugin uninstall with cleanup(see section below)
2. Reinstall EM
3. Install plugin
If the above steps do not resolve the issue, please contact Veeam Support.
Thoubleshooting steps
Plugin uninstall with cleanup
1. Uninstall plugin from EM
2. Restart vCenter Web Client Service. On windows server from services.msc console, on VC appliance from Vcenter Management web page
3. Remove all pieces of web plugin, that were not removed:
3a. Open SSH session to VC appliance and remove all enties that are returned by find / -name '*veeam*'.
3b. On windows server remove all files and folders containing veeam in %programdata%\vmware and %programfiles%\Vmware
SSO token analyze
1. Open Svc.VeeamRestAPI.log
2. SSO token is Base64 encoded. Copy encoded text between "Failed to authorize user. SSO token: [" and closing "]" into notepad and remove all occurrences of "\n" (replace to empty string)
3. Decode resulting Base64 string. You can use http://www.base64decode.org/ for that.
4. In the resulting xml look for tag "saml2:Issuer". There is a URL to the VC SSO service. Verify if the URL points to the VC. Try opening it from Veeam server. Response file in XML format should be opened or downloaded, depending on browser settings. Certificate not trusted error can be safely ignored. If the URL is wrong, then customer should reconfigure SSO.
RestAPI certificate change
To view all SSL certificates assigned on different ports, run command prompt on EM server "netsh http show sslcert".
Locate 2 records:
0.0.0.0:9443 (stands for Enterprise Manager)
0.0.0.0:9398 (RestAPI service)
1. Copy "Application ID" value from the 0.0.0.0:9398 section
2. Copy "Certificate Hash" value from 0.0.0.0:9443 section
3. Execute "netsh http delete sslcert ipport=0.0.0.0:9398" to delete existing cert
4. Execute "netsh http add sslcert ipport=0.0.0.0:9398 certhash=<hash> appid=<id>"(where <id> and <hash> are values, gathered before) to set new certificate
More Information
Required logs1. (From EM) Svc.VeeamBES.log
2. (From EM) Svc.VeeamRestAPI.log
3. (From VC) VMWare web client log(logs location on VC - http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2004090
4. (From VC) For vSphere 5.1 or older, web plugin logs from VC. Should be located in the same folder, where VMWare Web Client logs are. For vSphere 5.5 there is no dedicated Veeam Plugin log, info is written in Web Client log instead
5. (From VC) For vSphere 6 please see - https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2110014