When it comes to choosing a backup mode in Veeam Backup & Replication,there is a constant trade-off between space efficiency and I/O efficiency.
Many users select the Forward Mode simply because is the default one. It has a great I/O profile since it requires only 1write I/O per saved block, but on the other end in order to guarantee the consistency of the weekly incremental chain, the oldest full backup needs to stay in place a week after its expiration, thus consuming more disk space then planned by simply counting the retention period.
When they realize the space consumption is too much for their backup storage, they start to evaluate different backup modes, especially the Reversed mode. This mode is great for space efficiency, since no matter how long is the chain there is always only one full backup file and a chain of small increments. However, the trade-off in I/O consumption is pretty heavy: for each saved block, 3 I/O operations are required. If the backup storage is not able to guarantee this level of performances, the final result is a long and slow backup operation.
In order to overcome both limits, and offer an even better solution, Veeam Backup & Replication v8 will have a new backup mode, and it will also be the default one. Derived from the Backup Copy job, it will be probably called “Forward Incremental-Forever Backup” and it will allow as the name implies forever incremental backups like the Reversed mode, but with a different I/O profile that results in faster backups.
The way it works is pretty simple to explain: it starts just like a forward incremental as long as the retention period is not reached. Once the last increment has been created (14 days by default), on the next execution here is what will happen:
– the new increment is written as the newest restore point of the chain (using 1 write I/O as forward incremental)
– VBR realizes there is 1 more restore point than configured
– the oldest file is the full backup, created at the beginning of the chain
– the oldest incremental file is the second oldest file, just 1 day (or 1 execution) newer than the full backup
– the incremental is injected into the full backup, and the superseeded blocks are removed from the full backup. This requires 2 I/O: 1 to read the block from the incremental, and 1 to write it into the full backup
– the incremental file is removed from the chain, and the full backup file is moved in its position
As days go by, this operation is executed every time after a new increment is created.
Thanks to this method, the synthetic operation to create a new full is only limited to the size of the incremental file, instead of the complete size of a full backup file as it would happen in a “forward mode with synthetic fulls”. The overall consumed I/O is the same as the Reversed incremental, but is important “how” the I/O is consumed. During the duration of the backup activity only 1 write I/O is used and so the snapshot of the VM is open for less time than the Reversed incremental; the remaining 2 I/O are used to update the full backup file and most of all after the proper backup operation is already finished; and these 2 I/O are faster anyway to be completed than a Reversed mode.
Last week I presented at the vBrownbag booth at VeeamON conference, and I explained on a whiteboard this new mode. Here is the video, let me know what do you think of the new mode!
With this backup method there is the first write of the incremental blocks since the last backup, which is a single write to the backup storage, then the oldest incremental is written into the full backup, which requires a read and a write for each block.
This totals three operations, two writes and one read. There may not be as many reads and writes if the oldest incremental was very small, or it may be larger, that’s almost impossible to predict.
I am confused as to how this new backup method is actually more storage efficient than a reversed incremental backup if there are still the same number of operations on the storage (or potentially more if the latest incremental is only small but the oldest incremental, which has to be written into the full backup, was a large incremental).
Or does the read of the block in the oldest incremental and subsequent write into the full backup equal one I/O and not two?
It’s not more efficient in regards to IO. Still a total of 3 IO. It is better in that the production VM is running off of snapshot for a shorter duration. I’m assuming that it snapshots the vm, does its incremental write, removes the snapshot and then does the process to write the old incremental into the full. If it doesn’t do it like that then I don’t see how it would be better at all.
Hi all,
thanks for the comments. The original post was more focused on the transform operation and I realized the initial creation was poorly explained. Post is now updated.
As for the questions, no this method is not “more” efficient than the previous two. But is at the same time as efficient as them at the same time both for I/O and space; actually is more I/O efficient than Reversed since the transform happens after the proper backup is created and the VM snapshot is already committed.
WOW! this is a great feature thank you!!
this is really interesting in my situation, where i am facing a slow backup appliance and an overloaded main storage.
I will test if for sure!
How would this effect tape backups. We have a FSC requirement to ensure that we can restore a file for ever, so I back up to disk on a monthly cycle using forward with synthetic full, and, due to our large backup disk storage, have a long retention period. just before the last backup of the retention period I run a tape backup job to copy the files off to tape, so hopefully all incremental and full backups are on tapes which contain all files, even those deleted many months ago. Would this new method potentially mean that I could miss deleted files because the incremental backup had been incorporated into the full, the old changed blocks deleted, and then the old incremental file deleted, or even that the tape backups could be larger as the full backup is constantly changed, so would have to be written to tape more often?
The situation will be the same as is with backup copy job – specifically for backup copy and backup jobs with new incremental mode we will allow users to create the so called synthesized backup.
In order for a backup to tape job to create a synthesized full backup, the following requirements should be met:
1) There must be a increment that hasn’t been copied by a backup to tape job already.
2) A backup to tape must run on the very same day that is scheduled for a synthesized full backup.
As to missing restore points, a customer will have to make sure that the old restore points won’t get purged, until they are copied to tapes.
With the current Forward Incremental you can schedule a periodic “active full” from the primary data versus doing a weekly synthetic. How will doing occasional active fulls work with the new method? I just noticed that on the Backup-Copy jobs you can schedule a “compaction” but no active full.
You will still have the possibility to request active full in the middle of the chain if so desired.
Now that v8 is out, where would we find the option for this new backup mode? I went to edit an existing backup job and under Storage/Advanced Settings/Backup I only have the options for Incremental or Reverse Incremental. Am I looking in the wrong place, or is the new mode not available for existing jobs?
No, you are right, new jobs have the forever forward configured as default, but existing jobs are not changed, since we do not want to change the user experience and affect jobs execution. But you can edit them after the upgrade, and to obtain a forever incremental you just need to remove any configuration regarding active or synthetic fulls, there will remain only the forward incremental, without any periodic full creation.
Thanks for the reply. Just to be sure I’m clear, if a backup job is set for “Incremental” backup mode, and the options for Synthetic Full and Active Full backups are not selected, then the job will create a single full backup, and automatically roll each incremental backup into the full as the retention period passes, is that correct?
Is this functionality completely disabled when Active full backups are turned on? For example, if I have a retention period of 7 days, but on the last day of the month I want an active full backup, would the “forever incremental” functionality still work once the Active full becomes 7 days old?
Lastly, since veeam creates a new .vbk for synthetic fulls and then deletes the old .vbk when it’s finished, we currently have to keep enough free disk space available in our repository for an additional full backup. Assuming no active or synthetic full backups, does the forever incremental mode still require that amount of free space be kept available, or does the forever incremental inject the incremental directly into the existing .vbk file?
Hi Steve:
– yes, once you switch to forever forward, the oldest full will be used and at reaching retention all following incrementals will be injected in it.
– yes, enabling active full or synthetic full completely disable the forever forward
– no, incremental .vib file is directly injected into the .vbk without any temporary additional VBK file. However, I still suggest to keep at least the size of a full as free space, for non planned activities like recreating a new chain if the existing becomes corrupted for any reason, or to restore an old vbk from a remote location or tape…
Thanks.
While a very good explanation for this ‘NEW’ backup mode, the responses and answers create misinformation.
“– yes, once you switch to forever forward, the oldest full will be used and at reaching retention all following incrementals will be injected in it.
– yes, enabling active full or synthetic full completely disable the forever forward”
Actually one cannot DisAble the Synthetic Full and be able to EnAble the Incremental transform. (I wish I could cut & past that menu item to show & tell.)
Although one can DisAble the Create Active Fulls periodically.