Skip to main content

What does the tilde character do in True Image?

Thread solved
Regular Poster
Posts: 20
Comments: 273

As I was setting up a new installation of True Image 2021 I came across the not so familiar Exclusions tab that appears only when setting up a disk or partition backup (not so when working with file or folder backup). I normally don't spend too much time on this tab before I click OK. Most often I would just uncheck the exclusion box so that everything is included (nothing excluded). But I decided to customize the exclusion list this time. This brings me to my question.

What does the tilde character do in True Image?

Specifically...

*.~

What does this rule say?

I'm familiar with wildcard characters such as asterisk and question mark. But I'm not familiar with the tilde character. So I did some research, and found a few sources that suggest it's used in Microsoft Excel essentially as escape sequence for wildcard characters. In other words, it's used to get a match for strings that contains wildcards themselves, such as asterisk and question mark.

Here are two links that explain this.

Link 1 remove because I am not allowed to link.
"You are not allowed to place hyperlinks inside the text, please, clear them."

Link 2 remove because I am not allowed to link.
"You are not allowed to place hyperlinks inside the text, please, clear them."

For example, given a file named...

Summer2021?.jpg

The following pattern would match it:

Summer2021~?.jpg

But this pattern would not match it:

Summer2021?.jpg

If I understand this correctly. I have not tested this, this is what I gathered from reading those two web pages.

If this is right, then what kind of file names is that True Image pattern matching against? If it's used in the same way as in Excel then I guess the files could be names something like this:

Financial Report 2021.?

Financial Report 2020.*

Where the wildcards are used as file name extensions. I have never seen such a file in my life. Not sure it's even possible to create while running Microsoft Windows (maybe if using NTFS under some Unix-like operating system).

0 Users found this helpful
Regular Poster
Posts: 20
Comments: 273

I took a photo of the missing links. I hope this is OK with site administrators. You can see that they are harmless, especially the one to Microsoft Corporation, and no one can click on them by mistake, but will have to type the whole URL to get there.

Forum Star
Posts: 158
Comments: 1523

I always assumed it just meant files with a tilde extension... something assumed to be a temporary file not worth backing up (e.g. TempFile.~)

 

Regular Poster
Posts: 20
Comments: 273

I see... a tilde literal. I'm not sure I have ever seen that though.

There is another rule for temporary files that's applied by True Image by default.

*.tmp

As far as I know, this is the most commonly used extension for temporary files.

Regular Poster
Posts: 20
Comments: 273

If I understand this correctly. I have not tested this, this is what I gathered from reading those two web pages.

If this is right, then what kind of file names is that True Image pattern matching against? If it's used in the same way as in Excel then I guess the files could be names something like this:

Financial Report 2021.?

Financial Report 2020.*

I'm quoting myself here. Files could not be named like that, because it's illegal to use asterisk and question marks in file names in Windows.

The only possible explanation for this exclusion rule is the one offered by Bruno, I think.

On another note, I am currently facing a problem deleting an old Firefox profile folder in AppData, due to it containing a folder with a dot at the end of the name. I have concluded that this is the cause after some research.

Windows was giving me this message when trying to delete it:

An unexpected error is keeping you from renaming the folder.
If you continue to receive this error, you can use the error code to
search for help with this problem.
Error 0x80004005: Unspecified error

Now it's giving me a different error:

Could not find this item
This is no longer located in
C:\Users\Me\AppData\Roaming\Mozilla\Profiles. Verify the item's
location and try again.

I'm not sure how or why this happened. But of the original 3700 something files and 22000 something folders, it's not stuck with 0 files and 3 folders. It's just garbage now, I could just leave it alone. But this shows that Windows does not like it when you use dots in file or folder names. Other operating systems don't have a problem with it though, so I could use a live Linux disk to fix this (it would not be the first time that Linux has to step in to fix a Windows problem). I did not create this folder myself, Firefox did, while running on Windows. It's in storage\default folder. I'm not sure how it did it or why, but I do know I have a matching copy of this entire folder within my Profiles folder. I was restoring my Firefox profile from a previous Windows installation, and so I wanted to delete the original.

The lesson is, don't use dots at end of file and folder names in Windows. It should not allow you to, but you could be making this mistake in another operating system and then bringing the problem over to Windows (which I did not do).

I will accept Bruno's answer to this question. It's the only sensible explanation I can think of. Tilde character at the end of an exclusion rule in True Image means it will match file names with that character as is.

 

Legend
Posts: 105
Comments: 25746

Acronis uses the ~ tilde character when restoring files to indicate incomplete files, so if a restore is interrupted it can result in 1000's of such files with this character in the name or extension, which has in turn led to many forum posts over time.  Otherwise, it does indicate another form of temporary file which normally have no value or meaning to make them worth including!

Regular Poster
Posts: 20
Comments: 273

So if I understood this correctly now, Steve, True Image uses the exclusion rule *.~ in order to skip backing up its own temporary files that were created previously after a failed or interrupted backup operation. Is that right?

I'm not aware of any programs that name their temporary files like this. That doesn't mean they don't exist, of course. (Now I know of at least one such program.)

 

Regular Poster
Posts: 20
Comments: 273

This should be documented here:

Excluding items from backup

https://www.acronis.com/en-us/support/documentation/ATI2021/index.html#…

 

Legend
Posts: 105
Comments: 25746

Samir, there was mention of excluding *.~ in older versions of the ATI user guide but that looks to have been dropped when Acronis revised the text in more recent versions.

Example: from the ATI 9.0 User Guide:

5.3.2 Source files exclusion
The default setting – all files from the selected folders will be included into the
archive.
You can set default filters for the specific types of files you do not wish to back up. For
example, you may want hidden and system files and folders, as well as files with .~, .tmp
and .bak extensions, not to be stored in the archive.
You can also apply custom filters, using the common Windows masking rules. For example,
to exclude all files with extension .exe, add *.exe mask. My???.exe mask will reject all .exe
files with names consisting of five symbols and starting with “my”.
This option is effective for file/folders backup only. When creating a disk/partition image, you
cannot filter out any files.

The mention of tilde characters is minimal but indicates it is recognised as being used for temporary file types.

Regular Poster
Posts: 20
Comments: 273

Thanks Steve! Yes, that appears to be the case. I also looked at some of my older user guides for True Image that I have in DPF format.

True Image 2011 user guide, page 56, chapter 3.10 Excluding items from backup:

You can exclude hidden and system files from a backup by selecting the corresponding check boxes.

By default, the program excludes from backups files with the following extensions: .~, .tmp, and .tib.

But perhaps most interesting is the section on using wildcards to specify what files specifically to remove as part of a clean-up operation.

True Image 2011 user guide, page 201, chapter 9.6.3.3 Specific clean-up options:

The Files setting defines the names of files to clean with System Clean-up wizard and can be used with a search string.

Under the Windows operating system, a search string can represent a full or partial filename. A search string can contain any alphanumeric symbols, including commas and Windows wildcard symbols, and can have values similar to the following:

• *.* – to clean all files with any file names and extensions.
• *.doc – to clean all files with a specific extension – Microsoft document files in this case .
• read*.* – to clean all files with any extensions, and names beginning with "read".
• read?.* – to clean all files having five-letter names and any extensions, names beginning with "read"; the fifth letter is random.

The last search string, for example, will result in the removal of read1.txt, ready.doc files, but readyness.txt will remain with its longer name (excluding the extension)
You can enter several different search strings separated by semicolons; for example:
*.bak;*.tmp;*.~~~ (without spaces between the search strings)

All files with names corresponding to at least one of the search strings will be cleaned.

Using semicolons to separate patterns is a neat trick.

What's most revealing is the screenshot of True Image 2011 that's used in the user guide. It talks specifically about "Temporary file". It's somewhat misleading though. It talks about temporary file removal from existing True Image backups (as part of a clean-up operation), but the screenshot shows folder locations where Windows and other programs (other than True Image, such as "VMwareDnD") would normally store their own temporary files (in addition to those created by True Image perhaps).

Regular Poster
Posts: 20
Comments: 273

Some tilde matching examples

Given following files as example:

File 1.~
File 2~.txt
File 4.txt
~File 3.txt

The file File 1.~ can be matched with this pattern: *.~

Even more simply, the same file can be matched with this pattern: *~

The file ~File 3.txt can be matched with this pattern: ~*

As a comparison with previous file example, this file can be matched with this pattern: ~F*

In other words, you don't need any more characters in your pattern than an asterisk and a tilde if the file name ends with or starts with a tilde. Unless you count on having some files in your file set whose names not only end with a tilde but also are preceded by a dot (as a proper file name extension) and you want to match those files specifically. This is specifically what True Image does with the *.~ pattern.

As one more example, the file File 2~.txt can be matched with this pattern: *2~*

Or even more simply, but beyond the scope of tilde matching, it can also be matched with this pattern: *2*

A possible wildcard matching bug in Windows

The following example is an accidental success I believe (a bug?)...

You can match both File 1.~ and File 2~.txt with this pattern: *~.*

This should not work, I believe... but it does. It should match only File 2~.txt but it appears to be treated as an additional *~ pattern which then would indeed match the File 1.~ file. It appears as if the dot is causing ambiguity. As if it's used as a pattern separator (delimiter), to specify two patterns on the same line.

More on use of tilde character in True Image

Coincidentally, it just so happens that I encountered the tilde characters yesterday as I was restoring my whole OneDrive folder from a True Image disk image backup to its new (old) home (same path as I used previously). As the operation was in progress I saw True Image insert tilde characters in my file names. See examples below.

20200910_224348.mp4~TGDVLBGC
barcode-scanner-history-1593365757819.csv~2LT81MMM
Personal Vault.lnk~S8DKE4K1
Samir's Notebook.url~HB7O89CT

I normally don't work with file and folder level backup and restoration operations in True Image. So this is unfamiliar territory to me.

Once the operation was completed, all these tilde characters along with their ID numbers were removed from the file names. I can now understand the point that Steve made previously. If the operation was to be interrupted or terminated, these files may be left on the disk. If you were then to create a new backup with True Image, these useless files would be included in the backup, unless the pattern *.~ is added in the Exclusions list.

Note that the tilde characters are inserted in the extension part of the file names (after the dot).

Update: True Image uses wrong pattern to match its temp files

I just ran a quick test and found that the default pattern *.~ that True Image uses to exclude its own temporary files doesn't actually work.

I made 4 blank files using the same file names as True Image used yesterday while restoring my OneDrive folder. In addition, I made 4 more blank files that do not have any tilde characters in their names, just to be sure my matching is not targeting unintended files.

So this was my sample file set:

20200910_224348.mp4~TGDVLBGC
barcode-scanner-history-1593365757819.csv~2LT81MMM
Document 1.doc
Document 2.doc
Document 3.doc
Document 4.doc
Personal Vault.lnk~S8DKE4K1
Samir's Notebook.url~HB7O89CT

And here is the result of using *.~ with the DIR command in CMD.

C:\Testing>dir /b
20200910_224348.mp4~TGDVLBGC
barcode-scanner-history-1593365757819.csv~2LT81MMM
Document 1.doc
Document 2.doc
Document 3.doc
Document 4.doc
Personal Vault.lnk~S8DKE4K1
Samir's Notebook.url~HB7O89CT

C:\Testing>dir /b *.~
File Not Found

Using *~* would perhaps match the 4 files with a tilde character, or so I thought. But it did not work... as I mentioned previously, the dot appears to serve as a separator or delimiter.

C:\Testing>dir /b *~*
20200910_224348.mp4~TGDVLBGC
barcode-scanner-history-1593365757819.csv~2LT81MMM
Document 1.doc
Document 2.doc
Document 3.doc
Document 4.doc
Personal Vault.lnk~S8DKE4K1
Samir's Notebook.url~HB7O89CT

I had to use a more elaborate pattern to match them. I used this pattern: *.*~*

C:\Testing>dir /b *.*~*
20200910_224348.mp4~TGDVLBGC
barcode-scanner-history-1593365757819.csv~2LT81MMM
Personal Vault.lnk~S8DKE4K1
Samir's Notebook.url~HB7O89CT

I therefore conclude that *.~ does not do what Acronis thinks it does.

Perhaps it's a remnant from the past when True Image did use single tilde characters as the only character and as a file name extension for its temporary files? Perhaps.

I have a hunch that the ID numbers such a "TGDVLBGC" may have been introduced to tell apart files with tilde characters, for instance if a restoration operation has failed more than once and the program wants to dump a file named "My document.~" more than once. You can't have two files with the same name in the same location. This could be handled by overwriting the older file. But you can see how this could potentially lead to errors, and overall confusion and user frustration?

Depending on what True Image does with these failed files later on, it does make sense to insert an ID to each of them, so that True Image can track which file or files belongs to which failed restoration operation. This could for example be used to make a new attempt and to continue the operation where it left off last time when it was interrupted. Why else would you implement an ID enumeration like this if you don't want to uniquely identify a temporary file either during or after restoring them from a backup archive? You don't need something like this during a restoration operation, assuming it finishes successfully, as each file name in a folder is unique and identifiable already (that's the purpose of the filesystem).

Nevertheless, it should be replaced with *.*~* to serve the (True Image) temp file exclusion purpose correctly in True Image 2021.

Update: Actually... the dot may be just what I suspected, a separator. But not exactly for separating two or more patterns on the same line in the sense "match this OR match this". Rather, it's a separator for the file name extension, naturally. I didn't realize this at first. Windows is not treating "File.ext" as a single file name, but rather as two distinct names.

So in the pattern *.*~* the first asterisk matches any number of characters and any character in a file's name, whereas the part after the dot matches a tilde at any position but only within the extension or suffix of a file name and not the entire file name.

So the even more generalized version of this pattern to match a tilde on either side of the dot in a file name would be *~*.*~* with the dot as separator between the prefix and suffix or file name extension.

I would expect other operating systems to work in the same way with respect to this. I have not tested this though on anything but Windows 10, using CMD and the DIR command.

This also means then that the pattern *.~ is matching any file that has a single tilde character as its file name extension. I have seen no such file created by True Image. If the purpose of this pattern is to match and exclude temporary files created by True Image, then it's failing its purpose. It would for example fail to match "20200910_224348.mp4~TGDVLBGC" given in the example above.