Full Backups no independent entity

Christian, welcome to these public User Forums.

I have just learned in a support call that even full backups files (tibx) are not independent from their respective backup chain.

This means, that even if they had been validated successfully, any backup-file copy at some other location can't be used anymore after the original file got removed by eg the ATI 2020 cleanup.

I am not sure that the above statements are correct!

The key to all new backup files using .tibx file format in ATI 2020 is to try to keep files together, but most importantly to keep the metadata file with any files that may be copied to another location.

If the orginal backup task that created these files is still shown in the ATI GUI, then you should not attempt to add the files created by that task from a different destination location unless you have first removed the backup task settings from the GUI.

If there is no matching task in the GUI, then you can use the option to 'Add existing backup' for files stored in another location, just the same as you could for files created using the Acronis Rescue Media offline from Windows.

The following screen images were captured for making a series of full backup images to my external backup drive and show how the very first file of the backup sequence becomes the metadata file when the automatic cleanup rules start cleaning up the oldest backup files.

In the top image, the first file was the same size as the next three files, but then my cleanup rule of storing no more than 3 versions was applied and the top file becomes on 12kb to store the metadata for the chain.  If you click on that 12kb file, it will show you the other 3 files as if they were embedded in the file and you can navigate these files.

You should be able to copy these files to another backup location for security protection but need to copy again that first 12kb file after each change, i.e. when file -0004 is added and file -0001 is deleted.

The fact that Acronis denies or isn't able any more to work with a full backup file - no matter where it comes from - but considers everything outside of the respective backup chain being corrupt puts a huge question mark about the reasonability of ATI 2020 into my mind - and rather represents a show stopper to me, as it is really contradictive to the concept of full backups which is an essential aspect.

Sorry but not true. ATI continues to work fine with full backup files provided you observe the new rules of engagement.  The moved backup files should not be regarded as being corrupt provided you maintain the integrity of the chain (as it is created on the initial backup drive).

This can be achieved by using a simple Robocopy command to mirror the backups to a second location.   robocopy d:\ f:\ /mir  (where d: is the backup location, f: the target for the copy).

Is there any reason that is against sticking with ATI 2019 (Win 10 x64, 1903) until this is hopefully remediated?

Entirely your choice if you want to go back to ATI 2019 provided you understand that there will be no further updates or support etc.

If there is no matching task in the GUI, then you can use the option to 'Add existing backup' for files stored in another location, just the same as you could for files created using the Acronis Rescue Media offline from Windows. 

Tried that with the Acronis Technician today - but wasn't successful.  

You should be able to copy these files to another backup location for security protection but need to copy again that first 12kb file after each change, i.e. when file -0004 is added and file -0001 is deleted.

This seems to be the key aspect. 

If you do not have this additional 12kb file you can also bin the rest - no matter if this contains a valid backup or not.

I assume that you would always keep the correct 12kb file together with your full backup (the one where the respective full backup file was the last one in the chain?).

As said to me that is contradictive - there should at least be a way to rebuild this file from a single full backup file.

Being dependent on another file but the full backup file doesn't sound reasonable to me.

See the following KB documents published by Acronis with regards to .tibx files.

KB 63518: Acronis True Image 2020: do not delete first tibx file

KB 63227: Acronis True Image: Do not delete .TIB or .TIBX files outside of Acronis True Image

KB 63498: Acronis True Image 2020: new tibx backup format FAQ

KB 63425: Acronis True Image 2020: Limitations of tibx backups

KB 63516: Acronis True Image 2020: Incremental backups do not create separate files when using new backup format

KB 63445: Acronis True Image 2020: how to view and manage backup versions in new backup format

KB 63444: Acronis True Image 2020: tibx backups in local destinations

Christian, as a test, I just did the following steps:

1. Took the Delete option in the ATI 2020 GUI for the Full backup task, then just deleted the task settings leaving the .tibx files in situ.
2. Turned off Acronis Active Protection.
3. Moved all the .tibx files (4 files as shown in the last screen image above) to a new folder.
4. Turned on Acronis Active Protection.
5. Used the option in the ATI 2020 GUI to 'Add existing backup' by selecting only the initial 12kb metadata file.
6. Validated the added task successfully.

Steve, In your example, does this mean to recover Test-0001.tibx and Test-0002.tibx, all I need to have access is the file being recovered

If I want to recover Test-0003.tibx, and all future backup files I also need access to Test.tibx. If this is correct I have to change my off-site backup file procedure.
 

I would recommend keeping all of the files in the chain together where possible, but a minimum of the first and another file because of the metadata information in the first file.

Please note that I have not tested this directly at present but will try to do so later just to see what dependencies there are with full backup files!

Steve Smith wrote:

I would recommend keeping all of the files in the chain together where possible, but a minimum of the first and another file because of the metadata information in the first file.

Just tested the above after moving files -0001 & -0002 to another folder but all recovery and validation attempts fail due to missing files!

So the bottom line is that there are dependencies even with Full backup files and all the files in the chain need to be kept together!

08/11/2019 18:06:32 :449  -----
08/11/2019 18:06:32 :449  ATI Demon started. Version: 24.4.1.21400.
08/11/2019 18:06:32 :487  Operation Backup validation started manually.
08/11/2019 18:06:33 :437  Operation: Validation
08/11/2019 18:06:33 :438  Priority changed to Low.
08/11/2019 18:06:35 :942  Error 0x40011: The specified file does not exist.
08/11/2019 18:06:35 :948  Error 0x40011: The specified file does not exist.
08/11/2019 18:06:36 :016  Error 0x13c0005: Operation has completed with errors.

08/11/2019 18:05:50 :154  -----
08/11/2019 18:05:50 :156  ATI Demon started. Version: 24.4.1.21400.
08/11/2019 18:05:50 :282  Operation Full Test started manually.
08/11/2019 18:05:50 :671  Operation: Recovery
08/11/2019 18:05:50 :672  Priority changed to Low.
08/11/2019 18:05:51 :569  Error 0x1e50023: The specified file does not exist: S:\New folder\Full Test-0001.tibx.
08/11/2019 18:05:51 :622  Error 0x13c0005: Operation has completed with errors.

Moving the missing -0001 & -0002 files back to the new folder and retrying the Recovery has allowed it to start correctly and it is proceding with this.

Steve, thanks a lot for your feedback and input - highly appreciated!

While I have known some of the linked KB articles I didn't know all - I will look at them by chance but might take a bit.

Maybe I didn't manage to express clear enough though - let me try to explain with an example.

Following the conventions from you: 

1.) please simply copy the Full-Test-0001.tibx from the location where True Image stores all the backup files to some other location for having it safe (without the 12kb File).

2.) Delete Full-Test-0001.tibx from within True Image

3.) Try to access the content of Full-Test-0001.tibx or try to add this file as a backup from the ATI GUI.

Based on my experience and what I got from the support today this won't be feasible - and it seems there is no way to overcome this hurdle - meaning this full backup rendered useless.

Thinking of your example with robocopy I could imagine such a scenario happens quickly - simply by letting Robocopy overwrite the destination files but not delete anything - I f I got it right you might quickly end up with quite a number of useless full backup files...

So based on your input this is probably expected behaviour - made by design.

If so please allow me to question this design then.

I am still convinced that a full backup should contain everything for recovery - not being dependent on some maybe catalogue file.

But IF such a catalogue file with such high relevance is required - there should be at least means to recover this catalogue file somehow - else I am sorry but would consider this being a severe design gap...

Reading now from your insights with the dependency on ALL other full backups in a backup chain I really wonder why anybody would care about doing full or differential backups anymore - as the risk is probably equally high like doing incremental backups... - to me this is simply incredible...

Thanks a lot again though for helping to think things through! :)

Recovery was successful with all files present even though stored in a different folder location than where created.

08/11/2019 18:12:01 :315  -----
08/11/2019 18:12:01 :315  ATI Demon started. Version: 24.4.1.21400.
08/11/2019 18:12:01 :372  Operation Full Test started manually.
08/11/2019 18:12:02 :891  Operation: Recovery
08/11/2019 18:12:02 :892  Priority changed to Low.
08/11/2019 18:26:22 :702  Operation has succeeded.

Recovery was successful with all files present even though stored in a different folder location than where created.

Steve, can we agree on the fact that the rationale / purpose of doing full backups is to NOT requiring multiple files for a recovery? 

Else I could do incremental backups as well... 

Does anybody know if Acronis Backup 12.5 shows the same behaviour?

Steve, can we agree on the fact that the rationale / purpose of doing full backups is to NOT requiring multiple files for a recovery? 

Else I could do incremental backups as well... 

Christian, fully agree that any full backup should stand completely independently but unfortunately that is not the case at present with ATI 2020!

However, one way around that is to only create backup chains of a single full backup file, then make a new task for the next backup etc...

Now if ATI 2020 allowed for the new Replication feature to go to local or network drives instead of just to the Acronis Cloud, that would offer an alternative protection method.

Christian,

I have been following this thread since you started it.  It seems that you are perplexed at the logic behind the new .tibx format used for backups as that relates to Full disk/volume backups.

You ask in your last post if anyone knows if Acronis Backup 12.5 shows the same behavior that is discussed here in this thread.  I can confirm that it does in fact have the same behavior.  The .tibx format was introduced with Acronis Backup 12.5 and provided methods of increased usability, reliability, security, functionality and performance.

True Image 2020 introduces this new format as well however, it lacks some of the form and function found in Acronis Backup 12.5.  Still, it is intended to improve usability, reliability, security, functionality and performance.  The downside however is that there is a definite leaning curve for the user with this new format.

That learning curve as in your case means that you probably need to change the way in which you view backup to fit how the .tibx handles your data.  Without going into great detail the new behavior concept of backup is that a backup Task in TI 2020 is the same as a backup Plan in Backup 12.5.  All files of a task/plan are housed in a single file container and have dependency on each other as you have discovered.  For an understanding of this concept please have a look at the link below which references Backup 12.5 behavior.

Acronis Backup 12.5 Retention rules

I practice the Full backup as do you preferring that over incremental or differential.  My method to backup is as follows:

• Create full disk backup to local disk in dedicated folder using a Do not schedule, Custom, Full task/plan
• After that backup is created it is moved/copied to a number of other locations, ie. another local disk, NAS devices, removable portable drive.
• When it is desired to create a new Full backup the task plan is run again.
• After that backup is created it is renamed then moved/copied to the destinations as described above.
• Renaming is simply to add the date of backup to the file name before move/copy.  (10_8_19)

The above practice avoids the situation where you have multiple full backup versions in a single backup task/plan and facilitates the advice that Steve gives to Add a backup back into the GUI when needed.  Having said that this is not optimal and a bit cumbersome as well. 

Where Acronis Backup 12.5 has the advantage in this is that that product has Backup Consolidation functionality where is True Image 2020 had this same functionality your method of backup would work fine.   The downside to Consolidation is that it increases backup time substantially and so is disabled as default.  See the link below for more on this topic.

Acronis Backup 12.5 Consolidation

I hope this helps you come to terms with the behavior of the new file format.

Thanks guys!

Trying to wrap up to ensure I understood things properly:

1) Running some automated backup results in a backup chain, usually with n files (.tibx).

• All those n files are dependent on each other and are required to be available in a consistent state for recovering data from this backup chain.
   
• If a single file out of those is lost no data from this backup chain can be restored anymore at all.
   
• It should be expected that any manual file operation on any of those n files breaks the consistency of the backup chain - so no no data from this backup chain could be restored anymore at all.

   

• Consequence: the more files exist in a backup chain - eg. to have a longer history - the higher the risk to lose all data by losing a single one out of those n files 
   
• If someone isn't using  the cloud - eg. due to bandwidth reasons - backing up all files from within the backup chain by other means - right after an ATI operation (eg. backup, cleanup) has changed any of those n files - would probably be recommended) 

   

2) A possible work around for having independent backup files is the following manual process:

• creating an unplanned full-backup job without any automated cleanup
• after a backup has been written and validated deactivate Acronis Active Protection
• rename that single file (eg by adding the date) 
• copy/move the renamed file to some secure place for the time when needed
• activate Acronis Active Protection
   
• for writing a new backup the full-backup job would simply have to be restarted
   
• recovery of such files would have to happen by adding one as an existing backup

   

3) Acronis Backup 12.5 is working according to the same logic.

Is this a reasonable wrap up?

If so I would be wondering if you would have some script to support the above post backup tasks.

If you could share the same that would be of great help for not having to reinvent the wheel from scratch. 

Especially for deactivating AAP I wouldn't have any idea on how to manage that in a script...

Thanks a lot!

Christian, your points under 1) are all correct understanding.

For points under 2) then your bullet point:

>> for writing a new backup the full-backup job would simply have to be restarted

is not correct unless you elect to use 'Single version' for the backup scheme type, so that only one backup file is being created, but even then, with the new .tibx rules, I would expect there to be 2 files going forward, a 12kb metadata file plus a new numbered file -000?.tibx - this is because ATI will not delete anything until a new backup file is successfully created.

This also means that renaming files is likely to cause problems because the metadata file will hold the original, un-renamed file name.

The simpler solution here, rather than renaming any files, would be to copy or move the files to a suitably named folder that uses the name you want, including a date as needed.  This way all files keep their created names and avoid any issues that renaming may cause.

For point 3) I bow to Bob's knowledge of AB 12.5 as I have never used it and it cannot be installed alongside ATI 2020 for testing purposes.

With regards to the scripting of such actions, I can probably help with that provided you would be happy to run Powershell scripts on your system.  I have already got some scripts where I have been testing deactivating / reactivating AAP though this became more difficult with ATI 2020.

The scripts may take a few days before I can focus on modifying some I already have as will be away for a time over the next 2 weeks.

Hi Steve,

Thanks a lot for your additional clarification!

Would be great if you could share indeed your scripts and experience in this regards - but please no stress and don't put too much efforts into it - currently there is no urgency and I still have to make up my mind around using powershell scripts.

Thanks a lot and have a good time!

Cristian, Steve,

I concur point 1 as correct.

Point 2 is also correct.  Where Steve points out that you would need to set the task/plan as a Single version Scheme, I have found that using Custom Scheme - Non Scheduled achieves the objective.  The reason for file renaming is simple.  If you do not rename then you end up with multiple same named files as you are not recreating the task/plan.

My method uses the original backup file destination as simply a Temp folder.  To go a step further, I have multiple M.2 PCIe drives installed in most of my systems.  I use one as a dedicated scratch disk.  I create my backups on that scratch disk.  So as an example, one of my systems Windows OS disk has a bit over 77GB of data.  I can backup that drive to the scratch disk in around 5 minutes with a resulting file size of around 40GB.  After that completes I turn off AAP, rename the file using File Explorer adding the date, then move that file to my external USB 3.0 disk.  My NAS devices, of which I have two, are then destinations for this file/copy.  When finished I have 3 copies of the file.  So all told doing it all probably takes less than a half hour however, I continue to work with the PC while the copy operations run so 5 minutes for the backup, then a bit longer to move the file and start a copy task and I do what ever while that completes and then run the copy a second time.

I also have data disks in my systems and for those I have incremental tasks defined with cleanup rules.  I find that to cover my needs.  I also have a couple of old internal SATA HDD's that I use for archive purposes for some of the backups.  I copy created backups to these drives about once every 4 months.  When I do that I delete the previous archives before doing so.

For point 3, I am not a script writer so I do not use them.  I have dabbled a bit but I do not run them in any meaningful way.  My method would benefit from scripts but I do not mind the manual aspect of the method so I do not bother.

I have found that using Custom Scheme - Non Scheduled achieves the objective.  The reason for file renaming is simple.  If you do not rename then you end up with multiple same named files as you are not recreating the task/plan.

Bob, if you are just renaming the .tibx file on the scratch M.2 SSD, do you not also get an error when you next run the task to complain that it can't find the renamed backup, or have you set the Advanced > Error handling options so that the task is in 'quiet mode' to prevent such error messages from interupting the task?

No, no error because I Move that file so it no longer exists in the folder.  I do apologize however as I meant to write "Custom Full - Non Scheduled" rather than just Custom.  I believe that is the key to making it work.

I just made some changes in folder structure for this so recreated things a bit.  The screenshot below shows this method run this morning in Activity tab view for your convenience.

Bob, thanks for the information - I was able to setup a test full backup task using a Custom > Full backup scheme with no schedule or cleanup actions, then use a Powershell script to turn off AAP, rename the file using the current date & time, then move the file to another folder with the script running as a Post Command for the backup task.

The script is called 'Stop_AAP_move_tibs.ps1' which lives in my D:\Powershell folder.  This is called using the following command line.

c:\windows\system32\WindowsPowerShell\v1.0\powershell -ExecutionPolicy RemoteSigned -file pathto\Stop_AAP_move_tibs.ps1

The Arguments shown above are needed as Powershell is prevented from running scripts by default.

The resulting renamed files look as shown in the screen image below, where the original task name is 'NewFullTest' which is then renamed to add the date / time stamp information.
The original destination folder was S:\Test and the new location is S:\New folder (both of which are set using variables at the top of the script).  The script also creates a small text log file for the results of the actions being performed which again is stored in a location set via a variable at the top of the script.

The Powershell script is zipped & attached below.

Note: see webpage here if Powershell is disabled and you want to test the script using either the Powershell ISE or main Powershell command window.  The script has to be run as Administrator in order to perform the actions here.

Steve,

Thanks!  This is an unexpected surprise trust me.  I will give this s try and report back.  I'm confident it works as advertised! :)

Thanks guys for this work around!

I did test the process manually and it's working for me.

Thanks also for the script which I will take a look at later by chance.

Nevertheless I still very much hope that Acronis will put quite some more work into this area as I am still convinced that - even if the new file format brings more speed, reliability etc. - the new logic bears significent risks to lose all backup data. This definitely shouldn't be the case.

I also wouldn't dare to bet that all users will be aware sufficiently of the consequences of the radical change that has happened with V2020 and the .tibx files - and we likely have to recognize that not all have the knowledge, are able or willing to spend hours on investigating, rechecking their setup and testing the same again - but rather expect a solution that is matching their needs - just like it did in the past so very well.

But as said - thanks again so much for your time, efforts, help and the outlined approach which I can live with for the time being.

I very much hope Acronis is somehow rewarding you guys well - as what you do for them is tremendous.

Needs to be said as well...

Nevertheless I still very much hope that Acronis will put quite some more work into this area as I am still convinced that - even if the new file format brings more speed, reliability etc. - the new logic bears significent risks to lose all backup data. This definitely shouldn't be the case.

I fully agree with this!  Hoping this gets worked out as well, as we saw similar issues with incremental and differential chains in the beta and first public release that had recovery issues simply by picking a backup later in the chain (as accustomed to doing in 2019 and earlier) that are sorted out now.

Any "good" full backup should always be able to recover, independent of any other file, chain, metadata, etc.

Likewise, any specific differential backup, tied to a "good" full, should be able to recover as well.

Worse case, we know to keep any "current" 12KB version of the meta data with the existing backup files, but beyond that, there really shouldn't be any other hoops to jump through to be able to recover from fulls where automated scheduling has been used.

Hello All,

I have followed this thread from the beginning because I felt that I misunderstood the new .tibx format.

Thanks to all of your untiring testing and evaluation, I see that I was correct and need to change my backup strategy immediately.

In the past I relied on Full and Differential files for a rock solid and tested recovery plan.  That thinking is no longer valid for TIH2020 and probably beyond.  The whole point with our decades of backup strategies was to rely on a full plus differential to recover from.  With 2020 the full/differential method is out of the question.  I cannot rely on that method, knowing that one corrupted file in the chain, will prevent any possibility of a successful restore. 

We will deal with full backup plans for the short term but in the long term, something has to change.  This is not logical or trustworthy for the future.

Everyone be safe,

Steve F.

I am with Christian on this one - this is a step backward !

I only do full backups which now result in 2 dependent files ... you call that progress ?

Hello Everyone,

thank you for providing your feedback on this change - registered all comments in the internal request TI-190052 Support Independent Full backups. 

Heiner Behrendt wrote:

I am with Christian on this one - this is a step backward !

I only do full backups which now result in 2 dependent files ... you call that progress ?

I agree.  Christian stated this very well back in November:

Christian wrote:

...  So based on your input this is probably expected behaviour - made by design.

If so please allow me to question this design then.

I am still convinced that a full backup should contain everything for recovery - not being dependent on some maybe catalogue file.

But IF such a catalogue file with such high relevance is required - there should be at least means to recover this catalogue file somehow - else I am sorry but would consider this being a severe design gap...

I believe the current design that makes a full backup dependent on other files violates the "Principle of least surprise" which, according to Wikipedia, states 

... a component of a system should behave in a way that most users will expect it to behave; the behavior should not astonish or surprise users

All of the KB articles in the world aren't going to help the user who expects ATI to behave in the (reasonable) way it has in the past (and as competitive products behave).  The articles will be read after a recovery failure; they will explain why the recovery failed.  They may help prevent future failures but will do nothing to repair the ill will caused by an unexpected (and likely uncorrectable) failure. 

The new behavior may be based on a superior redesign of the product which is now more robust and efficient, but that is irrelevant.  The behavior is unexpected.

I have found this topic to be of great interest.  It has answered many of my questions.  

From the time I first installed TIH 2020, I found it difficult to understand and use.  My two daily backup tasks (scheduled, custom, full, with automatic cleanup after 5 iterations) increased up to 5x in duration over my previous versions (10.0 thru 2019).  I spent many hours on my own trying to understand what was going on and I finally created a support ticket with Acronis to try to get a resolution.  What I was told is pretty much what Acronis has posted in this forum topic: “Sorry Mr. Customer it is what it is and you will just have to live with it”. 

I have gone back to using TIH 2019 because I found TIH 2020 to be impractical from a user’s standpoint.  The rules and regulations that have been forced upon the user makes me believe that the TIBX system design was done by a first-year computer science graduate who has no concept of what a user environment is all about. 

Technically, the TIBX design may be great, but from a practical point it is worthless.

I just got bitten where the sun don't by this idiotic "feature" and I'm mad as hell. All my years of TI backup and the one time I need it it's useless. 

I am also concerned by the complexity that Acronis True Image has become.
The reason I ran into this forum thread was from looking up information on why backups are so slow with 2020.
A full backup of several 1 TB SSDs is taking over 12 hours. 
The write speed is only 20 MB/second to a drive that can easily do 150 MB/second
I can see a lot of reading of the backup drive causing 100% active time. WTF
Why the heck so much reading? This is likely related to issues in this thread about how the new format works.

Acronis True Image 2020 is not more efficient than previous versions.
Backup times are horrible with whatever format change was made.

This needs to be made right with people that bought the 2020 product after owning previous versions.
If it is not fixed I am definitely going to let reviewers know. This product as is should not be in the list of best backup products.

I have wasted a lot of time dealing with slow Acronis True Image 2020 backups and other problems. I have the latest version installed. That did not fix any of the performance problems.
I can see from this thread that other people doing backups of lots of data are having the same bad experience as me. I have a fast new i9 cpu machine, 32GB RAM,  several Samsung SSDs and high quality Seagate and WD SCSI high capacity backup drives. I was expecting reasonable backup times with Acronis 2020. Hasn't happened.
I am not usually an online complainer. This deserves complaints. I have been in the computer biz over 20 years.
Anybody in Acronis management listening?

chris dickey wrote:

I am also concerned by the complexity that Acronis True Image has become.
The reason I ran into this forum thread was from looking up information on why backups are so slow with 2020.
A full backup of several 1 TB SSDs is taking over 12 hours. 
The write speed is only 20 MB/second to a drive that can easily do 150 MB/second
I can see a lot of reading of the backup drive causing 100% active time. WTF
Why the heck so much reading? This is likely related to issues in this thread about how the new format works.

Hello Chris,

each case with performance issues should be investigated separately, as the issue is highly tied to the environment. If you have some time for investigation, please install a free trial of Acronis True Image 2021 and open a support ticket. Let me know the ticket ID, so that we can follow the investigation progress. 

One of the known changes in the new archive is increased times of the full validation job, which now performs the more thorough checks and hence increased times. If validation is enabled in your backup plan, you may want trying to disable it and observe the speed of a plain backup job. If the overall completion time is ok without validation, consider switching to Validate the latest diverse backup only option or running Entire backup validation job separately, when the PC is not busy.