Challenge

Product Information: 

Product Family: IBM Midrange Tape Library 
Status: Veeam Ready - Tape 
Classification Description: Tape device where available hardware features have been tested to work with Veeam.

Solution

Product Details: 

Model number: TS3500 and TS4300 
Library firmware version: F030 
Drive firmware version: F980 
Driver for tape drive: 6.2.5.8 
Driver for media changer: 6.2.5.8 
Media Type: LTO-7
General product family overview: IBM® TS3500 Tape Library is designed to provide a highly-scalable, automated tape library for mainframe and open-systems backup and archive that can scale from midsized to large enterprise environments. 

Veeam Details: 

Veeam Build Number: 9.0.0.1715

More Information

Company Information: 

Company name: IBM 
Company overview: IBM is the worldwide tape market share leader offering a full range of tape storage solutions including drives, autoloaders, libraries, virtual tape systems and Spectrum Archive software which can make tape as easy to use as disk. 

Challenge

When attempting to add a managed Linux server, you may receive one of the errors below.

Failed to negotiate key exchange algorithm
Client encryption algorithm not found
Server HMAC algorithm not found

Cause

When Veeam connects to a Linux target, we require Diffie-Helman key exchange capabilities for successful secure connections and to reduce the possibility that a password will not be intercepted when authenticating to the storage. In some Linux distributions, /etc/ssh/sshd_conf is missing the KexAlgorithms and Cipher fields to describe which methods are supported by the SSH daemon.

Solution

Review the configuration of your /etc/ssh/sshd_config file and verify at least one of the Ciphers, KexAlgorithms, and MACs listed below are present

Supported ciphers:

3des-cbc, aes128-cbc, aes192-cbc, aes256-cbc, aes128-ctr, aes192-ctr, aes256-ctr, blowfish-cbc, cast128-cbc, arcfour, twofish

Supported Key Exchange algorithms:

diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1, diffie-hellman-group14-sha1, diffie-hellman-group1-sha1

Supported HMACs

hmac-md5, hmac-md5-96, hmac-sha1, hmac-sha1-96, hmac-sha2-256, hmac-sha2-256-96, hmac-sha2-512, hmac-sha2-512-96, hmac-ripemd160, hmac-ripemd160@openssh.com

See the man page for your sshd_config file and/or query for the supported ciphers, key exchange algorithms and keyed-hash message authentication codes using the following command:

sshd -T

If needed, modify the sshd_config file. Then, to generate the newly added keys, run

ssh-keygen -A 

and restart the sshd service on the machine (reboot works fine, too).

More Information

Some systems offer an option in the GUI to disable or re-enable SSH logon, but these do not always actually restart the daemon. Typically, it is best to restart the service using the command

service ssh restart

or your distribution’s equivalent.

Challenge

Product Information:

Product Family: HPE StoreOnce w/Catalyst
Status: Veeam Ready – Integrated; Veeam Ready - Repository 
Classification Description: Integrated storage where joint development activities between the manufacturer and Veeam have occurred to create advanced backup and restore functionalities. This device can support all Veeam backup and restore features.   

Solution

Product Details:

Model number: All StoreOnce Models
Firmware version: 3.13.1-1546.2 or later (3.15.1-1636.1 or later required for vPower features)
General product family overview: From ROBO to the entire enterprise, HPE StoreOnce System with HPE StoreOnce Catalyst is a single and agile backup and recovery solution. Reduce costs and keep pace with rapid data growth, confident that your SLAs are securely met.

Veeam Details:

Veeam Build Number: 9.0.0.902
Veeam Settings:

• Repository Type: Deduplicating Storage Appliance
• Deduplication: OFF
• Compression: Optimal
• Storage Optimization: Local target (16TB+ backup files)
• Per-VM Backup Files: ON
• Decompress before storing: ON
• Align backup file blocks: OFF

More Information

Company Information:

Company name: Hewlett Packard Enterprise
Company overview: HPE delivers high-quality, high-value products, consulting, and support services in a single package. That’s one of their principal differentiators. They have industry-leading positions in servers, storage, wired and wireless networking, converged systems, software, services and cloud.

Challenge

Cause

The VMware API call to remove the VMDK was not received or did not complete properly.

Another possible cause for VMDKs to not being removed successfully is a backup or replication job creating a snapshot against the backup proxy while in use. See solutions for this event in next section.
 

Solution

Through vSphere, you should be able to right-click the Veeam Proxy VM and select edit settings. Look for the Hard disks attached to the VM. Normally the stuck disks will be listed at the bottom. See below: 



You should also be able to identify the disks by the name and the datastore they are in under the disk file box (top right).
 
To remove them, select the disk by clicking on it and select remove at the top right. The below will be shown:



Under removal options choose “remove from virtual machine” DO NOT choose the option that deletes it from disk (the second option).
 
We add the disk as Independent Nonpersistent, so we do not need to worry about making changes in Windows or removing it from disk management.
 
After successfully removing the disk, you will need to remove the snapshot the VM being backed up is running on.
 
To do this, right-click the VM in vSphere and select snapshot then snapshot manager. The below box will appear.



Select the snapshot (named TEMP in this example) and select “delete” and confirm the removal. This will consolidate all the changes back into the base VMDK of the VM.

If your backup and/or replication jobs from the previous section are found to be the root cause please remove the backup proxy from the job(s) and/or add exclusions to prevent them from being proceed.

More Information

Challenge

To ensure sufficient use of resources and faster jobs performance, it is important to plan your backup, replication and copying jobs in a proper way.

Solution

For information regarding Job Layout, Object Selection, Job Scheduling,  and Best Practices for any Backup Job Settings, please refer to this page:
 https://bp.veeam.expert/job_configuration/backup_job.html
 
 For information regarding the Backup Methods and the differences here and Pros and Cons for each, please refer to this page:
 https://bp.veeam.expert/job_configuration/backup_methods.html
 
For information regarding Encryption and Recommendations on When to Utilize it, please refer to this page:
 https://bp.veeam.expert/job_configuration/encryption.html
 
For information regarding Deduplication and Compression settings for any backup jobs, please refer to this page:
 https://bp.veeam.expert/job_configuration/deduplication_and_compression.html
 
For information regarding Backup Copy Jobs and Suggestions for Scheduling, Job Layout, and Object Selection, please refer to this page:
 https://bp.veeam.expert/job_configuration/backup_copy_job.html
 
For information regarding Replication Jobs and Proper Setup and Configuration, please refer to this page:
 https://bp.veeam.expert/job_configuration/replication_job.html
 
For information regarding Application-Aware Image Processing and When to Use and How it Works, please refer to this page:
 https://bp.veeam.expert/job_configuration/application_aware_image_processing.html

More Information

Challenge

Cause

• Report generated by an external SQL Server Reporting Services (SSRS) server and exported as Adobe PDF file: all links are disabled.
• Report generated by an external SQL Server Reporting Services (SSRS) server and exported as MS Excel file: the links refer to drill-down reports on the SSRS server and available for viewing through a web browser. 
• If a report is generated by the default Veeam ONE report viewer and exported both as Adobe PDF file and MS Excel file, all links are also disabled.

Solution

More Information

Challenge

Solution

Microsoft has provided us with options for relocation of databases and their associated transaction logs, as well as tools to perform these operations with ease and reliability in both small and large computing environments. Regardless if you have five servers or a complex infrastructure with hundreds of servers, this process will allow for the seamless movement of databases from any compatible operating system.

Moving a database by using detach-and-attach operations involves the following stages:

1) Detaching the database.
Detaching a database removes it from the instance of the Microsoft SQL Server Database Engine, but leaves the database intact with its data files and transaction log files. To detach the database using SQL Management Studio, you will expand the Instance, right-click on the Database you wish to Detach, click Tasks, and then click Detach. Be sure you have selected the correct database performing the Detach action.

2) Moving the database files to the other server or disk.

The database is traditionally stored in the Data location of the SQL Server program folder on the local disk. Each instance contains a “.#” suffix following the MSSQL folder, the default instance contains a 1, followed by each other instance which increases by 1 respectively. In the data folder, you will see the.mdf (database) and .ldf (transactional logs) files.
Both of these files will need to be relocated to their new server They do not necessarily have to be placed in a similar location - they can be placed anywhere as long as the SQL Server has direct access to the storage. It is actually Microsoft's best practice to place these files on different storage volumes as the transaction log file grows indefinitely, and can become significantly larger than the database file.

3) Attaching the database by specifying the new location of the moved file or files.

Once you have successfully logged into the SQL Server instance where you would like the database to be attached to, you will need to expand the instance, right-click on the Database node, and click Properties. Here, you will find the Attach option. Next you can browse to the new location where our original database files now reside. Only the .mdf files will be displayed in this view. You will have the option to verify the location for the .ldf files after click OK.

Detaching the database




Locating the Database Files



Attaching the database

More Information

If the Management Studio is not available, we can perform these same steps using the SQLCMD from the Command Prompt. We will need to use the –d parameters to define the database which we would like to log into locally.



We can then execute this statement to detach the database:

Once it has been successfully detached, we can then manually move the files into their new location.



Then we can perform this command to login to the remote server:

Now, you can perform this task programmatically or without the help of Microsoft SQL Management Studio.

Challenge

Solution

More Information

Challenge

Solution

• Ensure none of the cluster nodes are in maintenance mode, have the cluster service stopped, are powered off, or otherwise not accessible.
• Ensure all nodes within the cluster can communicate with each other.  This can be tested by pinging between the nodes and using wbemtest (see http://www.veeam.com/kb1926) to connect between nodes via WMI.  For wbemtest, the two namespaces you want to ensure can be connected to are root\cimv2 and root\virtualization (root\virtualization\v2 for Windows Server 2012+).
• Make sure the hosts are not added to the Veeam console as standalone hosts if they belong to a cluster.  This prevents changed block tracking data from being replicated to all nodes.  The cluster must be added to the Veeam console.

• Ensure the SMB server is added to the Backup & Replication console.

• Ensure the shares are visible in the Backup & Replication console.

• Select the option to disable changed block tracking on all Hyper-V hosts in the cluster.  To do this, right-click on each host in the cluster, select Manage Volumes and then check the box to disable changed block tracking.

 

• Run sc stop VeeamFCT, in an elevated command prompt, on each Hyper-V host in the cluster to ensure the CBT service is stopped.
• Find, export and then delete the following registry key on all Hyper-V hosts in the cluster:  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VeeamFCT\Parameters\CTDB
• Go back to the Manage Volumes for each host and uncheck the disable changed block tracking option.  Try the job after this.  Allow a couple of job runs before ensuring CBT is working.
• If the issue persists, reboot each Hyper-V host in the cluster.  This will typically involve migrating VMs from one host to another and rebooting the host once it does not own any VMs.  Repeat these steps for all Hyper-V hosts in the cluster.

Challenge

• Restoring single lost DC in a multi-DC environment
• Restoring entire AD infrastructure (AKA “all DC’s are lost”)
• Restoring from Active Directory corruption

Solution

More Information

Challenge

Cause

Solution

More Information

Please reference the Microsoft Knowledge Base Article below for further details:

http://technet.microsoft.com/en-us/library/cc816897(WS.10).aspx

For a complete guide of how to restore a Domain Controller from a Veeam Backup please see this KB:

https://www.veeam.com/kb2119

Challenge

Cause

Solution

Challenge

Event ID = 5125, Failover Clustering:

Cluster Shared Volume 'VolumeX' ('VolumeX') has identified one or more active filter drivers on this device stack that could interfere with CSV operations. I/O access will be redirected to the storage device over the network through another Cluster node. This may result in degraded performance. Please contact the filter driver vendor to verify interoperability with Cluster Shared Volumes.

Active filter drivers found:
WIMMount (HSM)

Cause

Solution

1. Reboot the Hyper-V host after the Recovery Media creation to disable the WIMMount driver.
2. Stop WIMMount manually.

SERVICE_NAME: wimmount
        TYPE               : 2  FILE_SYSTEM_DRIVER
        STATE              : 1  STOPPED
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

Direct IO will resume after a while.

More Information

If you have further questions, open a support case as follows:

1. Right-click the Veeam Endpoint Backup / Veeam Agent for Windows icon in the system tray and select Control Panel.
2. Click the Support link at the top of the window.
3. Click Technical Support to submit a support case to the Veeam Support Team directly from the product.

Challenge

Cause

Solution

Challenge

Solution

More Information

Challenge

Product Information: 

Product Family: ETERNUS DX 
Status: Veeam Ready - Archive 
Classification Description: Veeam Ready – Archive: Verified disk archive storage that can be used as a Backup Copy target.  Synthetic full backups, granular restores, and vPower features may not provide sufficient performance or be supported. 

Solution

Product Details: 

Model number: DX200-S4 
Number of Drives: 16 
Drive type: 300GB SAS (RAID-10) 
Firmware version: V10L71-0000 
General product family overview: The scalable and unified Fujitsu Storage ETERNUS DX200 S4 delivers enterprise-class functionality to small and medium-sized companies and subsidiaries with an excellent price/performance ratio. It is the perfect solution when consolidating data for server virtualization, e-mail, databases and business applications as well as centralized file services http://www.fujitsu.com/global/products/computing/storage/disk/eternus-dx/dx200-s4/  

Veeam Details: 

Veeam Build Number: 9.5 
Veeam Settings:  

• Repository Type: Shared Folder (CIFS) 
• Deduplication: No 
• Compression: Optimal 
• Storage Optimization: LAN Target 
• Per-VM Backup Files: Yes 
• Decompress before storing: No 
• Align backup file blocks: No 

More Information

Company Information: 

Company name: Fujitsu 
Company overview: Fujitsu is the leading Japanese information and communication technology (ICT) company, offering a full range of technology products, solutions and services. Approximately 156,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE: 6702) reported consolidated revenues of 4.7 trillion yen (US$40 billion) for the fiscal year ended March 31, 2016. For more information, please see http://www.fujitsu.com. 

Challenge

Product Information: 

Product Family: Lenovo DS 
Status: Veeam Ready – Repository  
Classification Description: Verified backup storage that supports all Veeam backup and restore features

Solution

Product Details: 

Model number: 4599HC2 
Number of Drives: 12x 2TB 7.2K, 2x 400GB 
Drive type: SAS, SSD 
Firmware version: GT250R006 
Additional support: Any hardware configuration with an equal or greater hardware configuration 
General product family overview: The ThinkSystem DS Series SAN products are most suited to Small-to-medium businesses (SMB) and to Enterprise customers, and are ideal for databases, backup & archive, media streaming and departmental high-performance computing (HPC) and OpenStack.   The DS Series comes in 3 performance levels, the DS2200, DS4200, and DS6200 and is ideal for large enterprises, remote/branch office, and small/medium business. 
Each system features Rapid Technology which includes: 

Veeam Details: 

Veeam Build Number: 9.5.0.1038 
Veeam Settings:  

• Repository Type: Windows 
• Deduplication: No 
• Compression: Optimal 
• Storage Optimization: Local Target 
• Per-VM Backup Files: Yes 
• Decompress before storing: No 
• Align backup file blocks: Yes 

More Information

Company Information: 

Company name: Lenovo 
Company overview: Lenovo is a $46 billion global Fortune 500 company and a leader in providing innovative consumer, commercial, and enterprise technology. Lenovo enterprise systems deliver industry-leading performance, reliability, and security in virtualized and cloud environments for analytics, database, virtual desktop, infrastructure, and web workloads. Lenovo also offers simplified and extensible systems management tools so you can manage your infrastructure on your own terms. Consistently ranked #1 in reliability and customer satisfaction, the Lenovo enterprise server, storage, and networking portfolio provides the hardware for businesses that never stand still. 

Challenge

Product Information:

Product Family: Quantum DXi
Status: Veeam Ready - Repository
Classification Description: Verified disk archive storage that supports all Veeam backup and restore features.

Solution

Product Details:

Model number: DXi 4700
Number of Drives: 12
Drive type: 4 TB
Firmware version: 3..4.0 (13857-59970 Build21)
Additional support: All models and configurations of DXi with equal or above the one outlined above
General product family overview: Powered by DXi® Software, Quantum’s new portfolio of deduplication solutions provides patented variable-length deduplication that not only reduces disk usage and capital expenditures, but also enables efficient data movement across the WAN to other sites and to the cloud.

Veeam Details:

Veeam Build Number: 9.5 Update 2
Veeam Settings:

• Repository Type: Linux
• Deduplication: OFF
• Compression: Optimal
• Storage Optimization: Local Target
• Per-VM Backup Files: ON
• Decompress before storing: ON
• Align backup file blocks: OFF

More Information

Company Information:

Company name: Quantum
Company overview: Quantum is a leading expert in scale-out storage, archive and data protection, providing intelligent solutions for capturing, sharing and preserving digital assets over the entire data lifecycle. We help customers maximize the value of these assets to achieve their goals, whether it’s top movie studios looking to create the next blockbuster, researchers working to accelerate scientific discovery, or small businesses trying to streamline their operations. With a comprehensive portfolio of best-in-class disk, tape and software solutions for physical, virtual and cloud environments, we enable customers to address their most demanding workflow challenges and opportunities.

Challenge

Cause

Solution

• remove the existing Veeam Agent for Microsoft Windows backup job from jobs (right lick > Delete)
• remove the backup files from configuration: right click Disk (backups) > Endpoint Backups > Remove from configuration. 

2. Place the backup chain into the required folder.

2.1 Example for a non-cloud repository

Backup files reside in C:\VeeamBackup\Backup Job 1 and need to be moved to a Veeam Backup & Replication repository into the following folder: C:\Backup.

Under C:\Backup you will need to create a folder named after the account that is used to access the repository. In this example, the account is DOMAIN1\user1, so the folder name would be DOMAIN1_user1.

The complete path to the backup files would be C:\Backup\DOMAIN1_user1\Backup Job 1

2.2 Example for a cloud repository

The backup files reside in C:\VeeamBackup\Backup Job 1 and need to be moved to a cloud Veeam repository into following folder: C:\Backup.

Under C:\Backup you will need to create a folder named after the tenant account that is used to access the repository. If the account name is Tenant2, the complete path to the backup files would be C:\Backup\Tenant2\Backup Job 1

If a subtenant is used with the account name Subtenant3, the path will be C:\Backup\Tenant2\Subtenant3\Backup Job 1

3. Rescan the Veeam Backup & Replication repository. Here is a brief guide on repository rescan. (Warning: Skip this step if you are using a Cloud Connect repository)

4. The Veeam Agent for Microsoft Windows backup should appear under Disk > Imported.

5. Point the Veeam Agent for Microsoft Windows job to the required Veeam Backup & Replication repository. Do not run it immediately, just close the wizard after making the changes.

6. On the Veeam Backup & Replication machine, the backup should be moved from the Imported section to Backup > Disk. Make sure that backups are located in Backups > Disks (not Disk (imported)).

7. Run the Veeam Agent for Microsoft Windows backup job. It should continue the imported backup chain.

Note: The described solution works for Veeam Backup & Replication repository and Veeam Cloud Connect repository.

More Information

Challenge

You receive the following error within moments of a job starting:
 
Cannot connect to the host's administrative share. Host:  [VM hostname or IP]. Account: [domain\user].
Win32 error: The network path was not found. Code: 53

NOTE: This applies to both VMware and Hyper-V hypervisors.

Cause

This error means that the Veeam Backup & Replication server cannot connect to the Windows guest’s OS feature to deploy guest processing components. This path will be the ADMIN$ share (\\hostname\admin$) of the guest itself.
 
These components are deployed directly from the Veeam Backup & Replication server. If networking is inherently impossible to the guest from the B&R server (different networks, firewall, etc), processing protocol may require inversion (VMware only).

Solution

• Make sure the Windows time on the Veeam Backup server is the same as in the guest OS.
• Make sure you do not have a firewall in the way (you can try disabling it for testing).
• Make sure you can connect to the admin share (for example, \\Server Name\admin$) with the same credentials as provided to Veeam Backup & Replication for VSS.
• Make sure the VSS writers are set to automatic in the guest OS.
• Make sure you provide Local Administrator (MACHINE\Administrator) or Domain Administrator (DOMAIN\Administrator) account privileges.
• Make sure the guest OS is in the same network as the Veeam Backup server.
• Make sure File and Printer Sharing is enabled in the guest OS.
  • ​Once File and Printer Sharing is Enabled on the guest OS, ensure to provide permissions through Windows Firewall if applicable
• If the guest OS is Vista/2008 or later:
  • In the Network and Sharing Center, verify that it is not set for a public domain.
  • Verify that the Remote Registry Service is started.

More Information