Quantcast
Channel: Veeam Support Knowledge Base
Viewing all 4470 articles
Browse latest View live

Procedure to split Veeam Backup & Replication Server and Veeam Cloud Connect Server

$
0
0

Challenge

In the situation where the same Veeam Server is used to serve both Backup and Replication services, and Veeam Cloud Connect, Veeam is requiring service providers to split these two services into two distinct Veeam servers.

Cause

As Veeam Cloud Connect services, especially Replication, involves more components and even more external connections from tenants, the following procedure is designed to keep the existing VBR (Veeam Backup & Replication) server as the Veeam Cloud Connect management platform, and to create a second server to become responsible for the other services.  This also applies to WAN accelerators, Backup Repositories, and Backup Proxies that are managed by the respective Veeam Backup Server. 

Note: This does not apply if you are only using Veeam Cloud Connect for Veeam Managed Backup Portal.

Solution

1.    Prepare the new backup server according to the system requirements. The new backup server must run the same product version as the existing server does, in regards of both major version and updates.

Note: The previous backup server and the new backup server must not be registered under the same Enterprise Manager. Consider using separate ones, instead.  If backup encryption is enabled, export keyset from the existing Enterprise Manager to the new Enterprise Manager:

https://helpcenter.veeam.com/backup/em/em_export_import_keys.html

2.    Declare maintenance windows for backup, replication, restore and failover operations and Veeam Cloud Connect service.

Note: During maintenance window tenants might lose access to different services.
 
3.    Stop and disable all jobs on the existing backup server.

4.    Block access to both the existing and the new backup servers from the internet.

5.    Create encrypted configuration backup of the existing backup server: https://helpcenter.veeam.com/backup/vsphere/config_backup_encrypted.html

Note: Encryption is mandatory! Without encryption enabled for configuration backup,  Cloud Connect tenant credentials as well as managed server credentials will be lost.

6.    Close backup console and stop all Veeam specific services (“Veeam*” can be used for convenience) on the existing backup server.

7.    Transfer the configuration backup to the new backup server.

8.    Disconnect the new backup server from the network.

9.    Restore the configuration backup to the new backup server using Migration Mode: https://helpcenter.veeam.com/backup/vsphere/vbr_config_migrate.html

Once the restore is completed, logon to the new server to verify it functions correctly and all jobs have been restored correctly.

Note: If on the existing backup server there are backup jobs that have guest indexing enabled, and you want to preserve guest indexes, follow the procedure described below:

•    On both existing and new backup servers stop Veeam Guest Catalog Service
•    Copy VBRCatalog folder from the existing backup server to the new backup server. Check the corresponding path in registry:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup Catalog\CatalogPath

Default file path: C:\VBRCatalog

10.    On the new backup server remove:

•    all cloud gateways
•    Cloud Connect tenants that are subscribed only to backup resources
•    WAN Accelerators that are not used by local backup and replication jobs

Note: Cloud Connect tenants subscribed only to replication resources or to both backup and replication resources cannot be removed.

WAN Accelerators that have been used in Cloud Connect tenants’ replication jobs cannot be removed. If needed, WAN Accelerator components can be removed manually via Windows Add/Remove programs.

11.    On the new backup server install new non Cloud Connect license.

Note: Once a new license is installed, the Cloud Connect node should automatically disappear along with unremoved Cloud Connect components, if any is left after previous step.

12.    Connect the new backup server to network.

13.    If you have any backups stored on the existing backup server locally, move them to the new backup server: https://www.veeam.com/kb1729

Note: Move only backups created by you; Cloud Connect tenants’ backups should stay intact.

14.    Start all Veeam specific services (“Veeam*” can be used for convenience) on the existing backup server.

15.    Re-enable internet access on the existing backup server.

16.    Enable and test all jobs on the new backup server.

17.    Delete all jobs from the existing backup server.

18.    Perform “Remove from configuration” operation on local backups and replicas on the existing backup server.

Note: Do not use “Delete from Disk” option.

Note: If the existing backup server has been used as a target by one or several Veeam Endpoint Backups, update backup server name on them to reflect IP address or DNS name of the new backup server.

19.    On the existing backup server install new Cloud Connect only license.

How to Move Veeam Backup & Replication Backup Files

$
0
0

Challenge

This article provides a method to change the repository where backup files are stored.

Solution

To manually move Veeam Backup and Replication backup files and continue using the same job to create a new backup chain, follow these steps:

1. In Backup Infrastructure > Backup Repositories, create a new repository for the new location where backup files will be located.

2. Manually move the backup files to the new repository path. Keep in mind that there will be folders created inside the directory/path for job names. If your repository is declared as E:\Backups, Job A will go to E:\Backups\Job A.

3.Rescan the new repository.
User-added image
4. If your backup files are encrypted, you will need to specify the password under Backups > Encrypted on the Backup and Replication tab. If your backup files are not use encrypted, skip this step.  

5. Edit the backup job. Go to the Storage tab, and from the drop down menu, select the new repository.
User-added image
 
 
 

To map the existing backup files to a new or different job, follow these steps:

1. Under Backups > Disk, right click the job you are changing repository for, and do either Remove From Backups (v8) or Remove from Configuration (v9).
User-added image

2. In Backup Infrastructure > Backup Repositories, create a new repository for the new location where backup files are located if you have moved the backup files. If you have not moved backup files to a new location, skip this step.
 
3. Rescan the repository where the backup files are located.
User-added image
4. If your backup files are encrypted, you will need to specify the password under Backups > Encrypted on the Backup and Replication tab. If your backup files are not use encrypted, skip this step.
 
5. Edit the backup job. Go to the Storage tab, and from the drop down menu, select the repository if it has changed in Step 2.
User-added image
 
6. Click Map Backup and select the existing backup chain
User-added image

 
 

More Information

If you are attempting to manually seed a backup copy job, please see the following KB: https://www.veeam.com/kb1856
 

How to configure antivirus exclusions to prevent interaction with Veeam Backup & Replication

$
0
0

Challenge

This article documents antivirus exclusions that have been found to help prevent antivirus software from interrupting Veeam Backup & Replication communications with components such as proxies and repositories or with the guest OS of VMs that are being backed up or replicated.

Cause

In some rare instances antivirus can prevent certain executables, processes, or agents from being correctly deployed or leveraged from the Veeam server, proxy, repository, or even the guest OS.

Solution

Antivirus Exclusions:

Folders on the Veeam Server:
C:\Program Files\Veeam
C:\Program Files (x86)\Veeam
C:\Program Files\Common Files\Veeam
C:\Program Files (x86)\Common Files\Veeam
VBRCatalog ( [HKLM\SOFTWARE\Veeam\Veeam Backup Catalog] CatalogPath value)
NFS (Configured in each repository, stored in [HKLM\SOFTWARE\Wow6432Node\Veeam\Veeam NFS] RootFolder value)
C:\VeeamFLR\*
C:\Windows\Veeam

 
Folder on Guest OS for VSS:
C:\Windows\VeeamVssSupport
C:\Windows\VeeamLogShipper

Folders on VMware Backup Proxies and CIFS Repository Gateway:
C:\Program Files(x86)\Veeam
C:\Windows\Veeam


Folders on Windows Repositories:
C:\Program Files(x86)\Veeam
C:\Windows\Veeam
*Also include the path the backup files*
 
Folders on WAN Accelerator:
C:\Program Files(x86)\Veeam
C:\Windows\Veeam
*Path of WAN cache on WAN accelerators*


Files:
VeeamAgent.exe
VeeamAgent64.exe
*.vmdk
 
Due to the complex nature of antivirus software some additional exclusions may be needed.
If your antivirus has a logging or history system please review this to see actions it may have taken that may affect Veeam Backup & Replication operations.

 

Veeam Backup & Replication - SQL Backup Guide: Using Microsoft SQL Server Agent With Application - Aware Image Processing

$
0
0

Challenge

This guide was created to assist you with successfully backing up, and restoring, Microsoft SQL databases.

Cause

Some highlights from this guide:

• Veeam Backup v7 most commonly used restore options for SQL
• Veeam Backup Job Configuration when backing up SQL
• Log Truncation, when and when not to use it
• Excluding the .bak and .trn Files
• Many more technical considerations included

Solution

Troubleshooting Signature-Based Firewalls

$
0
0

Challenge

Data transfer is failing across a connection that is protected by an advanced firewall. The firewall uses signature-based detection. Relevant features may have names like antivirus, anti-spyware, intrusion prevention, or application control.
 
This may affect any make or model of firewall with similar features, but support cases have been opened for Dell SonicWALL, Check Point, Cisco, Palo Alto, and Fortinet appliances.
 
Any type of data transfer may be affected, including backup, replication, and backup copy jobs. This problem may manifest in a wide variety of error messages and failure patterns. It may appear random, or completely consistent. One common failure pattern is for transfer of specific VM disks to consistently fail at or around the same percentage of completion.
 
Common error messages include:
 

An existing connection was forcibly closed by the remote host.
 
Unstable connection: unable to transmit data.
 

Depending on the job type and the version of Veeam Backup & Replication, connection failures may cause the job to fail immediately, or the connection may retry several times. While the connection is retried, the job may appear to be frozen because it is unable to transmit data. For more information, see:
Resume on Disconnect
Backup Copy: Automatic Job Retries
WAN Acceleration: Data Transport on WAN Disconnect

 

Cause

Disk images contain a potentially unlimited variety of data blocks. Because the traffic is compressed (and in most cases, encrypted), data blocks analyzed by a firewall will be different from data as it exists on the virtual disk. Over the long term, this approximates feeding random data into the signature-based threat detector: false positives are inevitable.
 
Data transfer is not actually random: a particular data block will always have the same signature after compression and encryption. If the source data does not change, the same blocks will be re-sent on every reconnect attempt and every retry of the job. In this case, the firewall will close the network connection every time Veeam Backup & Replication tries to transfer that data block, because it incorrectly detects a pattern of data within that block as matching the signature of a known threat.

 

Solution

Create exclusions for Veeam data traffic. In most cases, the relevant traffic will be between proxies or repositories over a default port range of TCP 2500-5000. This range can be configured for each managed server in the backup infrastructure settings.
 
Cloud Connect Service Providers should create exclusions for data traffic sent to Cloud Gateways on port 6180 (TCP and UDP).
 
For more information on port ranges, find the relevant product or component user guide in the Help Center and consult the Used Ports page.
 
For information on how to configure exclusions on a specific firewall appliance, please contact your vendor.

 

More Information

To isolate whether or not an issue is caused by a firewall, temporarily disable all signature-based features in the firewall’s configuration. For best results, do this while data transfer appears to be frozen – traffic should resume in no more than a few minutes. In some cases, the firewall may allow you to selectively disable specific sites or zones; this can be a useful as a solution, but it is not a good isolation step because such features are easily misconfigured.
 
Reset packets generated by firewall appliances can usually be distinguished from normal traffic by their IP time to live. For example, if most packets in a TCP stream have a TTL of 128, but the reset packet that closes the stream has a TTL of 64, the connection was closed by a firewall.
 
Firewall features that block encrypted key exchange will block the majority of WAN connections used by Veeam Backup & Replication.

“File does not exist” error after rename failure on deduplication appliance

$
0
0

Challenge

A backup or backup copy job writing to a Quantum DXi deduplication appliance is failing with:
 
Error: File does not exist. File: [\\dxi01\veeam\backup_job_1\backup_job_12016-06-13T210000.vbk]. Agent failed to process method {Stg.OpenRead}.
 
The initial error may or may not be visible in the statistics or the HTML report, but it is present in the debug logs:
 
error: boost::filesystem::rename: The specified network name is no longer available

Cause

By default, Windows SMB (CIFS) sessions time out after 60 seconds. The Quantum DXi may not respond to rename requests within 60 seconds if the file to be renamed is being processed by a deduplication job. When Windows closes the SMB connection, it reports the error message “The specified network name is no longer available.”
 
Veeam Backup & Replication assumes that when a rename request fails, the file is not renamed. However, the Quantum DXi does rename the file.
 

Solution

To resolve the immediate problem, manually rename the *.vbk file to match the file name in the error message. If there are multiple *.vbk files, be careful to rename the correct one; if it is unclear, please contact Veeam Support for assistance.
 
To prevent the error from re-occurring, either change the backup method to one that does not rename files, or extend the timeout on the gateway server(s).
 
Backup Methods
 
For backup jobs, use forward incremental with active full backups.
 
Forward incremental with transform to rollbacks, forward forever incremental, and reverse incremental backups all rename *.vbk files after merging new data into the full backup. Forward incremental with synthetic full backups will not rename *.vbk files, but performance is usually much better with active full backups.
 
For backup copy jobs, enable the Read the entire restore point from source instead of synthesizing it from increments option. If this option is not enabled, *.vbk files will be renamed.
 
More info is available in User Guide:
Backup Methods
Retention Policy
Backup Copy Methods
 
Timeout
 
To increase the SMB session timeout, create this registry value on the gateway server:
 
Location: HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\
Value name: SessTimeout
Value type: DWord
 
Decimal values as low as 300 (5 minutes) have been sufficient for some customers; in other cases, extending the timeout to 3600 (1 hour) was required. A reboot is required.
 
If using automatic gateway selection, you may need to make this change on all Windows servers in the backup infrastructure.
 

More Information

Relevant debug logs will be located on the repository gateway server and the backup server, at %programdata%\Veeam\Backup\<job name>\. The rename error may be found in Job.*.log, Task.*.log, or Agent.*.Transform.Target.log.
 
An active full backup will also work around the immediate problem, but this will cause retention policy to fail for older files. That is, these files must be deleted manually when they are no longer needed.
 
Deduplication Appliance Best Practices

Standalone Tape Fails All Operations

$
0
0

Challenge

Tape fails inventory, erase (short or long), or any backups.

Cause

Standalone tape drive reports Empty MTF, or unable to read header on tapes that are brand new, or even when they are erased using native vendor tools.

Solution

The block size being used for the tape drive needs to be set to 65536 within Veeam:
 
  1. Go to Tape Infrastructure within the Veeam GUI.
  2. Expand the tree to display the tape drive(s).
  3. Right-click the drive, click Properties.
  4. In the drop-down box for Block size, choose 65536.
 
Test tape operations after to confirm resolved.

SQL VSS Writer is missing: databases will be backed up in crash-consistent state and transaction log processing will be skipped

$
0
0

Challenge

Job reports warning "SQL VSS Writer is missing: databases will be backed up in crash-consistent state and transaction log processing will be skipped"
Running vssadmin list writers shows that SqlServerWriter in not in the list.

Cause

There are different causes of the issue:
  • the SQL instance have databases with names ending in a space character. 
  • the account under which SQL VSS Writer service is running doesn't have sysadmin role on a SQL server
  • SQL VSS Writer service stuck in an invalid state

Solution

Depending on a particular cause:
  • Please rename the database to a new name (without a space in it)
  • Grant the SQL VSS Writer service user a sysadmin role (Instructions in KB here here: https://www.veeam.com/kb1978)
  • Restart SQL VSS Writer service (Instructions in KB here herehttps://www.veeam.com/kb2041)
  • In the case of SBS machines that are also Domain Controllers, ensure that the SQL Writer is running as a domain administrator and not local system
  • Allow the SQL Writer service account access to the Volume Shadow Copy service via the registry:
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\VssAccessControl
    If the DWORD value “NT SERVICE\SQLWriter” is present in this key, it must be set to 1.
    If the Volume Shadow Copy service is running, stop it after changing this registry value. Do not disable it.
     

More Information

Microsoft documentation states that the SQL Writer service must run as Local System.
  • In SQL Server 2008R2 and earlier, this means that the writer service account appears to SQL Server as “NT AUTHORITY\System”.
  • In SQL Server 2012 and later, the writer service account appears to SQL Server as “NT Service\SQLWriter”.

Console Error - Failed to connect to Veeam Backup & Replication Server

$
0
0

Challenge

When attempting to connect to the Veeam Backup & Replication server using the console the connection fails with the error:

Failed to connect to the Veeam Backup & Replication server:
No connection could be made because the target machine actively refused it <IP>:9392

User-added image



 

Cause

There are multiple causes for this failure to connect, the following is a list of the most common causes:
 
1. The server where the console is installed cannot reach the Veeam Backup & Replication server due to network limitation.
2. The server where the console is installed cannot resolve the hostname or FQDN of the Veeam Backup & Replication server entered.
3. A firewall is preventing communication between the server where the console is being run and the Veeam Backup & Replication server.
4. On the Veeam Backup & Replication server the Veeam Backup Service is not running.
 
There may be other causes, please contact Veeam Technical Support if you find a cause not listed.

Solution

Connectivity/DNS
From the server where the console is installed attempt to ping the IP/Hostname/FQDN of the Veeam Backup & Replication server specified when using the Veeam Backup & Replication console. If unable to ping the Veeam Backup & Replication server, please resolve the network level issue.
 
Firewall
Please check all firewalls that exist between the server where the Veeam Backup & Replication console installed and the Veeam Backup & Replication server.
 
A tool such as PortQry can be used to test if port 9392 can be reached on the Veeam Backup & Replication server from the console server. https://www.microsoft.com/en-us/download/confirmation.aspx?id=17148
 
Veeam Backup Service
Connect to the Veeam Backup & Replication server and make sure that the Veeam Backup Service is running. If it is stopped, start it. Check that the SQL Service for the instance Veeam Backup & Replication is using is running and you are able to connect to it.
 

More Information

If none of the above methods resolve the issue, please open a support case for further investigation. If the service is not running, this qualifies as a Severity 1 issue provided you do not have any other Veeam consoles and meet the criteria for a severity 1 issue as defined here: https://www.veeam.com/veeam_software_support_policy_pg.pdf
 

VDDK error: 13 - Troubleshooting

$
0
0

Challenge

Backup/Replication jobs fail with:
 
VDDK error: 13.You do not have access rights to this file

Solution

Below is a list of possible solutions to this issue sorted by what transport mode was being using when this error occurred.
Note: There are many causes for VDDK 13, this list is not intended to be all encompassing.

Using Transport Mode: Network Mode
  • Solution 1:
    In some instances, you may receive this error when permissions are not correctly set. To confirm, please follow the steps outlined in the following KB article to test ports, permissions, and DNS resolution
    http://www.veeam.com/kb1198
Using Transport Mode: Hotadd Mode
  • Solution 1:
    https://www.veeam.com/kb1775
    Stuck Hotadded Disks – Review all virtual proxies for disks attached that do not belong to the VM in question, remove the disks not owned by this VM without deleting from the datastore. Once deleted, consolidate snapshots on the affected VMs.

     
  • Solution 2:
    kb.vmware.com/kb/2003638
    Orphaned Snapshots – Consolidate snapshots on affected VMs

     
  • Solution 3:
    http://www.veeam.com/kb1875
    Permissions - Please configure the account leveraged to access the vCenter as an Administrator within vCenter

     
  • Solution 4:
    http://www.veeam.com/kb1882
    Automount – In some cases, it may be necessary to perform an automount scrub to clear any existing mount points that are no longer in the system.

    Run the following via CMD Prompt on the proxy server:
    diskpart
    automount disable
    automount scrub
    exit

     
  • Solution 5:
    http://www.veeam.com/patches.html
    Patching – Please ensure that all VMware and Veeam components are fully up to date.

     
  • Solution 6:
    In some rare instances, performing a storage vMotion has been known to resolve the issue due to a locked VM file on the datastore where the VM files reside.

Using Transport Mode: Direct SAN
Solution 1:
When receiving VDDK Error 13 when leveraging DirectSAN access mode, please confirm that the LUNs are presented to the proxy correctly and have the proper permissions. For assistance deploying & troubleshooting DirectSAN please follow the below KB articles:
http://www.veeam.com/kb1895

 

More Information

Additional Troubleshooting(All versions):
  • Manual Hotadd - http://www.veeam.com/kb1184
  • Hotadd Requirements - http://www.veeam.com/kb1054
  • Deploy an additional standalone proxy to isolate failures to a single specified proxy
  • To isolate a specific transport mode as the point of failure, you may also test the other transport modes to ensure that connectivity is correctly established and backup jobs work as intended via the other transport mode.
Additional Troubleshooting (vSphere 5.1 and below):
http://www.veeam.com/kb1777
Backup via Standalone host – Configure backup job to backup via Standalone host rather than through vCenter; this will isolate vCenter as the point of failure. If backup via standalone host is successful, you can attempt restarting the vCenter as well as the management agents on the host(s) where the proxies/VM(s) reside.

Information: Hotadd will only work when the proxy is on the same host as the object being backed up

Additional Troubleshooting (vSphere 5.5):
When attempting to hotadd leveraging a standalone host configuration (that is managed by a vCenter object) with vSphere 5.5 you will receive VDDK Error 13, as well as the error listed below:

Error message: [00680 error 'transport'] Reconfigure attempt failed for VM "2" with Vmomi::MethodFault::Exception ("vim.fault.HostAccessRestrictedToManagementServer")\n

To resolve this particular issue, please leverage Network Mode or Direct SAN transport mode as this configuration is not supported due to limitations of VMware

Information: For more information regarding ‘HostAccessRestrictedToManagementServer’ please see below VMware documentation.

http://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.fault.HostAccessRestrictedToManagementServer.html

 

Deduplication Appliance Best Practices

$
0
0

Challenge

Performance or deduplication ratio is low when backing up to a deduplication enabled storage appliance from Veeam Backup & Replication. 
Please note that following article does not pertain to ExaGrid Storage or EMC DataDomain. If you use either of these please see the corresponding articles:
If you are using ExaGrid Storage, please see https://www.veeam.com/kb2056
If you are using EMC DataDomain, please see https://www.veeam.com/kb1956

Cause

The default options within Veeam Backup & Replication are intended for non-deduplication storage.

Solution

It is advised to configure the job as following when backing up to a deduplication appliance:

  • Use “Incremental”
  • Uncheck “Enable synthetic fulls” 
  • Check “Perform active full backups periodically”
  • Uncheck “Enable inline  data deduplication”
  • Change the Compression Level to “Dedupe-friendly”
  • Change Optimize for to “Local target (16TB+ backup files)”
Veeam Backup & Replication v8

User-added image
User-added image



 

More Information

  Forever Forward Incremental, Synthetic Full, and Reverse Incremental operations will perform poorly with deduplication devices because the backup files that have been written to the storage previously must be interacted with during these operations. The act of reading existing data on deduplication  device is slow because each block requested must be rehydrated and uncompressed to be read. With DataDomain deduplication appliances, however, it is possible to leverage DataDomain's own mechanism of updating file blocks' metadata internally, and consequently achieve greater performance with Synthetic Full. New full backup files can then be created without physically moving data into the file, but rather by synthesizing them from the existing data.
 
The process of rehydration and uncompressing during read operations from the deduplication storage will also impact the performance of all restore operations.
 

Contact your storage vendor to inquire about further specific details to optimize Veeam Backup & Replication to work with their storage solution.
 
 

Leap Second Compatibility

$
0
0

Challenge

A leap second is a one-second adjustment that is occasionally applied to Coordinated Universal Time (UTC) in order to keep its time of day close to the mean solar time. Some software experiences problems when leap seconds are inserted (e.g. inserting it at an earlier date).
 

Solution

The previous leap second insertion, which occurred on June 30, 2015, revealed no issues with any Veeam products, and it is expected that this will be the case for the next leap second insertion.

However, this may not be true for any other infrastructure components and/or software, and it is recommended to check with third-party vendors which your company's business relies upon.

Cannot connect to Virtual Proxy during VMware backup

$
0
0

Challenge

Attempting to connect via RDP to a Veeam Proxy VM in VMware during a Virtual Appliance (hot-add) backup fails.

Cause

This occurs due to hung backup job utilizing that specific proxy, and that proxy is not configured to have automount disabled.

Solution

In an Administrator Command Prompt run the following commands:

User-added image

C:\Windows\system32>diskpart
Microsoft DiskPart version 6.3.9600
Copyright (C) 1999-2013 Microsoft Corporation.
On computer:VBR7
DISKPART> automount disable
Automatic mounting of new volumes disabled.
DISKPART> automount scrub
DiskPart successfully scrubbed the mount point settings in the system.
Automatic mounting of new volumes disabled.

More Information

Please note, if you are using a Domain Controller as a proxy which isn’t advised, you should not perform the above as it will cause problems with backing up the server with VSS: http://www.veeam.com/kb1697

 

Failed to truncate transaction logs for SQL instances on Shared VHDX

$
0
0

Challenge

When attempting to run a backup job for a SQL Server that is in either a SQL Server Failover Cluster or AlwaysOn Failover Cluster with a shared VHDX the backup job reports the warning:

“Failed to truncate transaction logs for SQL instances: <>. Possible reasons: lack of permissions, or transaction log corruption” or SQL log backup job fails with “Regular database <> was not backed up in Oib”

On the guest, within the VeeamGuestHelper.log found in %programdata%\Veeam\Backup the following line can be found:

OLEDB Error: 'BACKUP LOG cannot be performed because there is no current database backup.'

Cause

This is behavior by design. Backup of VMs with shared VHDX disks can only be performed in crash consistent mode, with Application-Aware Procesing disabled.
 
Because the VSS subsystem is not involved in the backup process, the SQL Server is not aware of a backup having occurred and does not update date of last database backup. Due to this lack of an update to the last database backup entry, when the SQL Instances receives the command to truncate logs (“backup log”), it cannot find a corresponding database backup event and returns the error above.

Solution

SQL logs truncation or backup is not possible in this configuration. Please change to using a AlwaysOn Availability Groups SQL installation with individual VHD(X) for each VM.

More Information

Please see the Release Notes for details:
https://www.veeam.com/documentation-guides-datasheets.html

Error Backup Repository ‘Repository ID #’ was not found

$
0
0

Challenge

Running a Veeam Endpoint Backup to a repository in Veeam Backup & Replication and the following error message occurs:
Error Backup repository ‘ID Number’ was not found (System.Exception)

Cause

Veeam Backup & Replication is providing an incorrect repository ID due to Endpoint backups being imported in.

Solution

Check the Veeam Backup & Replication GUI under Backup & Replication view, and verify if the backup job shows under disk imported. If it does, remove the backups from configuration. The Endpoint backup should run again afterwards.

More Information

If the problem continues, go to cp.veeam.com to open a Veeam Support case.

Unable to allocate processing resources

$
0
0

Challenge

VMware backup fails with the error message “Unable to allocate processing resources. Error: No backup proxy is able to backup this VM. Check processing mode settings on proxies”

Cause

This is generally caused when there are no backup proxies available to process the job. Reasons for this include:
  1. Too many jobs are running at a time for the number of concurrent tasks set across all proxies available to the job that failed.
  2. Proxies may be unavailable (Shutdown, not connected to the network, etc.).
  3. The existing set of backup proxies does not have enough resources to process the backup jobs in a timely manner.
  4. The virtual machine might be being processed by another job

Solution

Finding the best solution can be different for each environment and backup infrastructure. Choose what fits your situation the best.
 
  1. Stagger your backup jobs and replication jobs at different time slots so that a job isn’t waiting on available proxies when it starts. (Ex. Instead of scheduling your jobs to all start at 8:00PM, start one job at 8:00, another at 8:30, and another at 9:00.)
  2. Increase the number of concurrent tasks allowed on your proxies. However, keep in mind the system requirements.  See How to set max concurrent tasks. If your backup proxy is a virtual machine, you should increase the amount of CPU and RAM available to the proxy.
  • VERSION 7.X through to VERSION 9.X with default job settings requires 1 CPU core and 200 MB RAM per task that you wish the proxy to run at a time, plus 2 GB RAM for the operating system and Veeam services. Although only 200 MB per task is required, a 2GB per CPU core ratio is recommended for best performance and reliability. (2 Tasks at once = 2 CPU cores and 2.4 GB RAM required, 4GB recommended)
    • If using high compression, 2 cores per concurrent task are required.
    • Memory usage may be greater than the minimum system requirements. If memory usage on the proxy frequently exceeds 80%, consider increasing the available memory.
    • VERSION 7.X through to 9.X runs tasks in parallel, but can run sequentially with the parallel processing option disabled. Processing multiple disks on one VM will reserve multiple concurrent tasks. See Enabling Parallel Processing.
  • Remember to not under-allocate your Veeam Backup server, especially if it is your backup proxy.  If the configuration database is installed on a local SQL instance (the default installation), then SQL Express should be allocated 1 CPU core and 1-1.5 GB RAM when planning your resource usage. A full version of SQL may use significantly more resources in a large environment. 500 MB RAM per concurrent task is recommended for the Veeam Backup services and the SQL server combined; 4 GB minimum.
  1. Deploy additional backup proxies from within “Backup Infrastructure->Backup Proxies”.
    • This can be a physical server, an existing VM, or a newly-deployed VM. If adding existing virtual machines, bear in mind that Changed Block tracking will be disabled when backing up these VMs, which can substantially increase processing time.
  2. Make sure that your backup job or replication job is selecting the correct proxies. Choose “Automatic selection” to have Veeam try to choose the best proxy from your available pool of proxy machines, or choose a specific set of proxies. The latter can be useful if you have set up a specific proxy with a large amount of resources to be used on a large backup or replication job.
  • However, be careful as a backup job set to use a single proxy can fail with the above error if that proxy is unavailable. It might be beneficial to choose more than one if manually specifying proxies.
  • Also note that automatic proxy selection will detect available transport modes for each proxy, and then wait for a proxy with the best available transport mode. The modes are ranked as follows: Direct SAN (SAN) > Virtual Appliance (HOTADD) > Network (NBD). For example, if you have a virtual machine proxy on each of three hosts, and all of your VMs are on local storage, a job set to automatic will wait for the proxy that has access to the local storage (HOTADD) rather than use an available proxy that can only read the disk over the network (NBD). For more information, see Transport Modes.
  1. Investigate backup job performance. If specific jobs are taking longer to process than normal, check for warnings, compare the bottleneck statistics to previous jobs sessions, and try to isolate the problem to a specific proxy, repository, host, or datastore.

More Information

Similar errors can occur when the number of concurrent tasks exceeds the limit set on a repository. See Repository Settings: Limit maximum concurrent tasks

Best Practices for Deployment & Configuration (VMware)


 

The requested operation could not be completed due to a file system limitation

$
0
0

Challenge

Transform operations of a large file located on Windows 2012&2012R2 deduplicated volume are failing with “Error: Client error The requested operation could not be completed due to a file system limitation Failed to flush file buffers. File: Filename.vbk

Cause

Veeam is not able to write any data to the affected file because of a NTFS limitation.
Manual un-deduplication (Expand-DedupFile) of the this file also fails with the same error.

Solution

Backup repository must be located on a volume formatted with the large FRS using the command “Format <Drive:> /FS:NTFS /L”

Workaround:
1. Disable Windows deduplication
2. Un-dedupe backup files: copy backup files to a temporary folder, delete original backup files and move copied files back to the original location.
3. Disable NTFS compression on backup repository volume.

More Information

For Windows Server 2008(R2) please refer to Microsoft KB for a detailed description of the issue:http://support.microsoft.com/kb/967351

This issue is resolved in Windows Server 2012(R2), however formatting with the large FRS using the command “Format <Drive:> /FS:NTFS /L is still needed.

Important: the command Format <Drive:> /FS:NTFS /L erases all data from the drive.


Additional information regarding suggested configuration of Server 2012 dedupe for backup storage can be found in this article: http://technet.microsoft.com/en-us/library/dn891438.aspx
 
Depending on the total size of the volume to be formated you may have to set the cluster size during the format process. Please reference this article to determine which cluster size is needed.
 
 
For more information related to this issue please see this forum post:
 
http://forums.veeam.com/veeam-backup-replication-f2/error-related-to-file-system-limitation-t21448.html

Please note: reformating the volume with /L flag usually helps, but you can still get the same error message later, you can still hit limitation and 4096 Bytes Per FileRecord Segment will not be enough

Release Notes for Veeam Backup & Replication 9.0 Update 2

$
0
0

Challenge

Release Notes for Veeam Backup & Replication 9.0 Update 2

Cause

Please confirm you are running version 9.0.0.902 or 9.0.0.1491 prior to installing this update. You can check this under Help | About in Veeam Backup & Replication console. If you are using partner preview build 9.0.0.773, you must upgrade to GA build 9.0.0.902 first by installing Day 0 Update > KB2084

After upgrading, your build will be version 9.0.0.1715

Prior to installing this update please reboot the Veeam server to clear any locks on the Veeam services and when the reboot is done, please stop all the Veeam jobs and services before applying the update.

After installing the update, during the first start of the Veeam Backup Service, required modifications will be made to the configuration database automatically to optimize its performance. These modifications may take up to 10 minutes to complete. Please do not reboot the Veeam server, or attempt to stop the service during this operation. If there is concern regarding the time that the Veeam Backup Service takes to start after upgrade, please contact Veeam Customer Support.

Once Veeam Backup Service starts, please open the console and allow Veeam Backup & Replication to update its remote components.

To obtain this update, please click here.

Solution

As a result of on-going R&D effort and in response to customer feedback, Update 2 includes over 300 enhancements and bug fixes, the most significant of which are listed below:

New platform support
  • Cisco HyperFlex HX-Series support for Direct NFS backup mode.
  • EMC Unity support for Backup from Storage Snapshots and Veeam Explorer for Storage Snapshots functionality.
  • EMC Data Domain DD OS 5.7 support for DD Boost integration.
  • ExaGrid 4.8.0.351.P28 is now the minimal ExaGrid firmware version supported.
  • NetApp Data ONTAP 8.3.2 support.
  • VMware vCloud Director 8.10 support.
  • VMware VSAN 6.2 support.

Engine
  • Backported a number of isolated Enterprise Scalability enhancements from 9.5 code branch to improve transaction log backup, tape backup and user interface performance.
  • Updated OpenSSH client to version 7.2 to enable out of the box support for modern Linux distributions.
  • Improved iSCSI target performance (iSCSI target is used to mount backup remotely in certain file-level and item-level recovery scenarios).
  • iSCSI mount operations are now retried automatically to workaround occasion “The device is not ready” errors which happen when mount operation takes too long. By default, the mount is retried 6 times every 10 seconds. To change the number of retries, create IscsiMountFsCheckRetriesCount (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the backup server.

Backup Copy
  • To reduce backup server load, Backup Copy jobs targeting shared folder or deduplicating appliance backed repositories with the gateway server setting set to Automatic selection will now start data moves on the mount server associated with the backup repository (as opposed to the backup server). In cases when the mount server is unavailable, the data mover will be started on the backup server as before.
  • Backup Copy performance should now be more consistent due to preserving backup files cache when the job is switching to idle mode.
  • Minor reliability improvements in GFS full backup creation algorithm.

Microsoft Hyper-V
  • Find-VBRHvEntity cmdlet performance has been improved significantly when used against Hyper-V cluster.
  • Backup infrastructure resource scheduler should schedule Guest Interaction Proxy resource dramatically faster in large infrastructures (for example, 5 seconds instead of 15 minutes).

Microsoft SQL Server
  • Improved performance and reduced resource consumption of Microsoft SQL Server transaction log backups.

Microsoft Exchange
  • Added ability to force CAS server for Veeam Explorer for Microsoft Exchange (instead of automatically detecting one) via DefaultCASServer (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the backup server.
  • Added ability to change the order of Exchange autodiscovery policies for Veeam Explorer for Exchange (support only setting).

Oracle
  • Added ability to restore Oracle databases while preserving certain parameters which are critical to RMAN in scenarios such as database name change via new <Parameter Name="diagnostic_dest" StartupPath="true"/> parameter in PFileParameters.xml file of Veeam Explorer for Oracle.

Cloud Connect Replication
  • Support for Planned Failover functionality with cloud replicas. You can now perform planned failover to achieve zero data loss, for example when a natural disaster can be predicted in advance.

Veeam Cloud & Service Provider Partners
  • Update 2 introduces important changes and fixes around rental licensing, including a pilot functionality of usage reporting directly from the user interface. For additional information, as well as the list of other service provider specific enhancements and bug fixes included in this update, please refer to the issue tracking topic in the private VCSP forum. If you are VCSP but don't have access, please apply to Cloud & Service Providers group using Veeam forum's User Control Panel.

More Information

To obtain this update, please click here.

Release Notes for Veeam Backup & Replication 9.0 Update 1

$
0
0

Challenge

Release Notes for Veeam Backup & Replication 9.0 Update 1

Cause

Please confirm you are running version 9.0.0.902 prior to installing this update. You can check this under Help | About in Veeam Backup & Replication console. If you are using partner preview build 9.0.0.773, you must upgrade to GA build 9.0.0.902 first by installing Day 0 Update > KB2084

After upgrading, your build will be version 9.0.0.1491

Prior to installing this update please reboot the Veeam server to clear any locks on the Veeam services and when the reboot is done, please stop all the Veeam jobs and services before applying the update.

After installing the update, during the first start of the Veeam Backup Service, required modifications will be made to the configuration database automatically to optimize its performance. These modifications may take up to 10 minutes to complete. Please do not reboot the Veeam server, or attempt to stop the service during this operation. If there is concern regarding the time that the Veeam Backup Service takes to start after upgrade, please contact Veeam Customer Support.

Once Veeam Backup Service starts, please open the console and allow Veeam Backup & Replication to update its remote components.

Note: Veeam Backup & Replication 9.0 Update 2 is available. Click here to obtain it.

To obtain Veeam ONE v9 Update 1, please follow this link

Solution

As a result of on-going R&D effort and in response to customer feedback, Update 1 includes over 300 enhancements and bug fixes, the most significant of which are listed below:

General
  • Enhanced extent selection logic for scale-out backup repositories with and without per-VM backup file chains enabled.
  • Added reversed incremental backup mode support for storage-level corruption guard and backup file compact and defrag functionality.
  • Guest Interaction Proxy selection algorithm has been improved to increase proxy selection performance.
  • Added support for BitLooker folder exclusion functionality on Japanese and Chinese OS. This requires that the backup proxy used by the job is installed on the OS with the same locale as the processed VM.

Cloud Connect Replication
  • Support for replica seeding.
  • Support for replication from backup (except from backup files stored in cloud repositories).
  • Improved failover issues reporting.

VMware
  • Storage fingerprint check can now be disabled using SshFingerprintCheck (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key.
  • Datastore based jobs can now be set to not include VMs with mounted ISO residing on the selected datastore. To enable this behavior, create StrictDatastoreScope (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key.

Hyper-V
  • Replica disk exclusion and Linux file-level recovery should now work correctly on Windows Server 2012.

Application-aware processing
  • To increase the chance of Microsoft VSS processing succeeding, the default VSS snapshot timeout has been doubled to 1200 sec. Additionally, VSSGuestSnapshotTimeout (DWORD) registry value in seconds under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key has been added to allow you to control this value.

Oracle
  • Added support for Oracle ASM on GPT disks.
  • Redo log backup for non-ASM deployments is now performed directly from logs directory, without first copying the logs into a temp folder.
  • Added ability to skip NOARCHIVELOG mode databases without warnings. To enable this behavior, create OracleSkipNoArchiveLogDB (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the processed VM.
  • Added ability to disable Oracle processing without having to disable application-aware processing for the rest of applications running on the same VM. To enables this behavior, create DisableOracleProcessing (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the processed VM.
  • Improved reliability of abandoned log backup process termination in cases such as loss of network connectivity or backup repository reboot.

Veeam Explorers
  • Improved Veeam Explorer for SQL Server startup and restore initialization time on large backup sets.
  • Added support for exporting very large PST files with Veeam Explorer for Microsoft Exchange.

Tape
  • Improved reliability of detecting, erasing, ejecting and exporting tapes.
  • Improved GFS candidate restore point selection logic for certain scenarios.

SureBackup
  • SureBackup jobs will no longer remove VM snapshot before deleting the temporary VM itself, thus speeding up job completion time. Instead, an additional functionality has been added to automatically clean up VM leftovers, which occasionally may be left on the datastore used for I/O redirection due to a VMware bug caused by deleting a VM with the existing snapshot.
  • SureBackup role descriptions and application test scripts will now be obtained from the backup server, as opposed to the console.

User interface
  • Improved user interface responsiveness and reduced configuration database SQL Server load in the environments with large amount of backups and replicas.
  • Deduplicating storage appliance users will now receive a warning if their backup job or storage appliance settings do not match the recommended settings.

Enterprise Manager web UI
  • Addressed cross site scripting web vulnerability has been discovered in Veeam Backup Enterprise Manager web interface. Veeam thanks Davide 'Peru' Peruzzi [GoSecure!] for vulnerability discovery.

Upgrade
  • Console will now auto-update itself when connecting to backup server of higher version by downloading the update package (or updated files) directly from the backup server. This functionality is supported for both updates and hot fixes.
  • Network extensions appliances are now included into the existing remote components auto update process (Upgrade wizard), thus removing the requirement to re-deploy them manually.


Veeam Cloud & Service Provider Partners

Starting from Update 1, when automatic license update is enabled, backup servers with per-VM hosting licenses will send protected VM counts for the past week (high watermark) to the Veeam licensing server. The collected data will allow our backend system to automatically approve monthly usage reports as long as they do not deviate significantly from any high watermark for the reporting period. It's the first of several steps Veeam is taking to streamline usage reporting and reduce processing costs. Automatic usage logging is currently optional and can be turned off by disabling automatic license update.

For the information on Veeam Cloud Connect infrastructure specific enhancements and bug fixes included in this update, please refer to the issue tracking topic in the private VCSP forum. If you are VCSP but don't have access, please apply to Cloud & Service Providers group using Veeam forum's User Control Panel.

More Information

Note: Veeam Backup & Replication 9.0 Update 2 is available. Click here to obtain it.

To obtain Veeam ONE v9 Update 1, please follow this link

Veeam Cloud Connect Service Provider Licensing

$
0
0

Challenge

Starting with Veeam Backup & Replication 9.0 Update 2, there have been several changes to Cloud Connect licensing for service providers.

Solution

Usage reporting

Veeam Backup & Replication 9.0 Update 2 (or Enterprise Manager Server, if present) introduces usage reporting by the backup server. This is considered a pilot and does not yet replace existing usage reporting processes. If using service provider rental licenses, monthly usage should be reported as usual.

We encourage service providers to also submit usage reports from their backup servers and provide feedback on the experience. Usage reporting by the backup server is enabled by selecting the "Update license key automatically" check box in the License Information window. More information about automatic license updating is available in the
User Guide.


Enforcement of VM counts

Veeam Backup & Replication 9.0 Update 2 also enforces VM license counts. Previous 9.0 releases issued warnings but did not restrict usage.

To accommodate customer growth, Veeam Backup & Replication and Veeam Cloud Connect allows service providers to exceed their license count for a period of time. So as you bring on new customers or your existing customers expand their use of your Veeam-powered services, you may be able to process additional VMs without having to get new license keys. (You are still responsible for reporting and paying for additional usage.)

•    Veeam Backup & Replication: You can exceed your license count by 20 percent or 20 VMs (whichever is greater). If you use service provider rental licenses, your license count is based on reported usage, and you can exceed your license count for up to 2 months. If you use service provider perpetual licenses, your license count is based on the number of VM licenses purchased, and you can exceed your license count for up to 30 days.

•    Veeam Cloud Connect: You can exceed your license count by any number of VMs for up to 2 months.

Please note that extra capacity is available exclusively with service provider licenses. Veeam licenses for end customer use do not include this feature.


If you anticipate customer growth exceeding the extra capacity built-in to your service provider license keys, please contact your Veeam representative. Be sure to plan ahead and allow at least 2 business days for a larger license key to be approved and issued.
Viewing all 4470 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>