9

I'm on Debian and I have a folder, with thousands of files inside it, that has a size of about 14GB. From this folder I have created two encrypted archives: one is a password protected .7z archive, the other is a TAR archive encrypted with ccrypt (.tar.cpt).

(From now on I will assume that .7z/7zip archive means password protected .7z/7zip archive)

I have noticed that opening the 7z archive and inserting the password, lets me see the actual content in about 1-2 seconds, maybe less (there's almost no human perceptible time difference from opening this file and the same .7z file without encryption). However, to be able to see the TAR archive content, the ccdecrypt command on the .tar.cpt file much longer about 5-6 minutes. After that I can open the TAR archive and see the actual content in about 1-2 seconds. This leaves me with some questions:

1. Is a 7zip archive's content encrypted?

The difference in time for decrypting two files of about the same size (the .7z is only about 100MB smaller because the files are already in compressed formats) makes me think that while ccrypt encrypts all the content of the TAR archive, 7zip only encrypts the archive containing the files, like if it was a folder (which vaguely is), making difficult to break through it, but the actual content of the archive remains unencrypted. So being able to see the content only takes few seconds with a .7z archive (the time to decrypt the containing archive/check the password), while seeing the .tar files implies that all the 14GB content is ccdecrypted. Did I make the right assumption? If not, can someone please explain me how these two encryption methods differ and work? This specific 7zip archive hides the filenames before the password is inserted, if that's somehow relevant.

2. Does that means minor security of the 7zip archive?

If the assumption in the first point is correct, does that imply that a password protected 7zip archive is less secure than a "fully encrypted" .tar.cpt archive? I know that using 7zip leaves a lot of traces/temporary files of the hypothetical extracted/decrypted archive, but my question is about the actual security of the encrypted archive itself. Furthermore, I will encrypt/decrypt the archive only on my desktop PC, while the file will be stored on a server, so even if someone will ever take control of the server (and not of my PC of course), he will not be able to find any traces of a decompressed archive, given that the archive is never been decompressed on the server, right?

3. Is there a true UNIX alternative?

Is there a software typical of the UNIX world that makes possible to encrypt a TAR archive in the 7zip way, so that i can protect stuff not-so-security-critical (if a .tar.cpt is actually more secure than a .7z like I've assumed) in a way that makes possible to see the files in 2 seconds instead of 5 minutes? I've nothing against 7zip, it's a great utility, but I'm wondering if the pure UNIX world has something similar to offer (I suppose certainly yes). Furthermore, creating an uncompressed TAR archive takes much less time than creating his 7zipped counterpart, even at minimum compression settings, so being able to apply the "7zip crypting method" to a .tar file would be really convenient.
Other UNIX encrypting software like gpg, mcrypt and the old unsecure crypt all make the same ccrypt type of encryption (I mean that they encrypt all the content, not considering the encryption method), while gzip, bzip2, etc., if I am not wrong, cannot encrypt nothing at all, given the UNIX tradition "Do one thing and do it well".

Initially, the fact that is possible to extract a single file from an encrypted 7zip archive in a bunch of seconds, more or less like with an unencrypted one, was for me another reason to think that the content wasn't actually encrypted, because I didn't know that even a single file can be decrypted from a .7z (and wasn't passed enough time to let me think that the whole archive was decrypted to give me that single file).
Furthermore, I recorded the extraction times of the encrypted and unencrypted versions of the same 7zip archive, and the difference wasn't noticeable at all. I thought that this was another proof of the content being unencrypted, but I was one more time wrong, because I wasn't considering that modern CPUs have encryption bult-ins for popular algorithms like AES (as suggested in the accepted answer).

The 7zip archive can apparently be set to 'no compression' only through command line, or at least not through GUI with Ark (setting the slider to 'minimum' still compresses the archive). Without compression (-mx=0 switch, from the same answer) the .7z size and making time are, more or less, identical to those of the .tar counterpart.

Even not considering the advantage in time of a .7z header over .tar.cpt to show the file list, ccrypt only uses 1 core for encryption/decryption, which makes the process even slower.

I edited the question to make it more readable. Having the accepted answer appropriate quotes, I hope it's still fully understandable.

mars
  • 93
  • 1
  • 6
  • that is much easier to read - thanks mars. I'll clean up these comments now. Have an upvote :-) – Rory Alsop Nov 25 '16 at 10:17
  • I'm glad to know that helped:) – mars Nov 25 '16 at 15:39
  • Note that [ccrypt](https://security.stackexchange.com/a/102184/165253) is actually [really bad](https://crypto.stackexchange.com/a/24213/54184). – forest May 14 '18 at 01:17
  • I have checked your link, thanks for letting me know. However, since I posted this question I started using encrypted 7z archives already for other reasons (and it seems like it was the right thing to do after all). – mars Jun 12 '18 at 12:20

2 Answers2

7

(I have never used ccrypt. My answer about it is limited to what they claimed in their page)

  1. 7z actually encrypts both the archive header and the actual file content, but with the same password. When using 7z to create an encrypted archive, you have the option to also encrypt the header (through the command-line flag -mhe=on, default is off). As long as you supply a password, the actual file content is always encrypted regardless of whether you encrypt the file header.
    The use of file header by 7z (and zip and rar) is just to allow you to quickly browse through the files/directories stored in the archive, for both encrypted and unencrypted archive. tar does not have any such file list due to historical reasons.
    So your assumption is incorrect.
  2. I can't say for ccrypt, but 7z is safe enough in terms of encryption (as long as you choose a good password). As you have mentioned, 7z uses AES-256 for encryption and AES-256 is safe. 7z is open source and there are no security flaws (malicious or honest) that could tamper encrypted 7z's security.
    Note that you are using a password to encrypt your 7z archive. That means if somebody got your 7z file, he can try every possible passwords one by one until he comes to your password. (That's why you should always choose a good password) 7z uses key stretching (100,000 iterations IIRC) to slow down such process, but if your password is too short or is too common (e.g. "Passw0rd" or "Letmein"), that malicious guy can hit your password quickly. There have been genuine password crackers for encrypted archives which uses this trick.

so even if someone will ever take control of the server (and not of my PC of course), he will not be able to find any traces of a decompressed archive

As long as you don't 7z x archive.7z -pMyPassword or "Extract here", the file should be invisible to other people. This is a common pitfall for automatic cloud sync tools like Dropbox: Any files that you extracted to the sync folder will be uploaded to their server. If you double click on the file in the archive file list, the file will be extracted to some temporary path (/tmp or somewhere in your ~) and the sync tools won't upload it to anywhere.

  1. Yes. It's called 7z :)
    Seriously, .tar archive is nothing more than a large file. You can always compress/encrypt it using 7z. This will produce a .tar.7z file though you are free to rename it. This means multiple files are grouped into a .tar archive, then compressed/encrypted with 7z in whatever way you like. This is particularly useful in *nix administration since tar preserves owner/permission info but 7z doesn't. But in your case if you don't care about these, a simple 7z is more than enough.
    As a sidenote, you don't need to compress with 7z if you don't want. You can set the compression level to 0 (-mx=0 in command line). This way 7z simply packs your file and encrypt them, without doing the time-consuming compression/decompression.

However, even if opening a single file in the 14GB .7z archive isn't as quick as in the .tar archive

It depends on the algorithm used to compress 7z and tar files. There are multiple compression algorithms to compress a file, and each algorithm can be configured to different compression levels. In general, the smaller the output file the longer it takes. Since you mentioned that 7z saves you 100 MB there is a trade off in time.

Also, when you open the file list of .tar, you have already decrypted and decompressed all the files. It's just the matter of getting the bytes you want. But for .7z only the archive header is shown. The actual files have not been decrypted/decompressed yet. If you compare the time taken from double-click on file list to getting the file, tar is almost always the winner.

In fact, the encrypted version should take much longer to be extracted given the added decryption time

This may be true for old computers (>10 years ago). Modern day CPUs are (1) very fast in general, and (2) have built-ins for common crypto algorithms like AES. Disk IO is more of a limiting factor if you are not using SSD.

how can 7zip in only 2 minutes and a half decrypt and extract the whole archive, when ccrypt takes much more to only make the decryption??

Without knowing the filelist and which file you are decrypting/decompressing I can't say for sure. However, since tar must uses solid compression, you need to decrypt and decompress the whole archive in order to get even a single file. Whereas for 7z, solid compression is optional (but is enabled by default in command line 7z; GUI depends on which frontend you use). Each file is compressed and encrypted individually. To get a single file, just get the encrypted bytes according to 7z header, and do the magic only on those bytes for your file. tar is good as a archive format if you always want to extract the whole archive. If you sometimes just need a single file, go with 7z.

is there a way to combine UNIX basic tools to be as powerful as a 7zip archive, that is combining an archiver, compressor, and encrypter

I won't recommend reinventing the wheel / rolling your own when it comes to security. 7zip is free software and even proprietary software supports 7z format. But if you really have to, try gzip-ing individual files, tar them (no compression) and ccrypt the tar. You'll need some non-trivial scripts for that though.

Community
  • 1
Link Ng
  • 186
  • 3
  • Thanks a lot for the detailed answer, is very meaningful. About .7z being slower than .tar at opening a file I was using that example to say that was impossible that the whole archive was extracted in such a short time (not knowing that a single file can be decrypted), and I've explicitly wrote that I suppose is caused by the added decompression time:) Talking about your last point, in truth I can indeed extract a single file from a .tar archive without extracting the whole archive. Am I missing something? :) – mars Nov 24 '16 at 15:01
  • Thanks a lot, do you know if 7zip appeared first on UNIX or Windows? – mars Nov 24 '16 at 15:08
  • About tar: No. You still need to extract the whole archive to get even a single file. What the software does under the hood is to extract everything to temp folder / memory, then copy only the file you select to disk, and let the rest cleared by OS. The file list appears *after* decryption and decompression has been done. – Link Ng Nov 24 '16 at 15:10
  • Extracting the 14GB .tar takes about 1 minute and 10 seconds, clicking the right button and selecting extract on a single file put the file in the specified folder almost in realtime. Shouldn't be passed at least the same minute and 10 seconds to make possible to extract that single file?:) I'm doing that with Ark. – mars Nov 24 '16 at 15:15
  • Another thing: about not being mandatory to compress with 7zip, with compression set to the minimum, .7z archive still is about 100MB smaller than the .tar counterpart, and takes longer to be created (about 1 minute for .tar with 4 cores at about 85% VS about 20 minutes for .7z with 4 cores at 100%). However this is done with Ark through Dolphin file manager, maybe it can be done with no compression only from command the line? Let me try. – mars Nov 24 '16 at 15:42
  • Ok, on the compression thing you were right, I'm sorry for that. From the command line with the -mx=0 argument, it takes about 1 minute to create the uncompressed .7z, and the size is the same as the .tar and the original folder. So apparently through Ark is not so intuitive to do, if not undoable at all :) – mars Nov 24 '16 at 16:23
  • Experimenting a bit I've noted that for .tar.gz is certainly valid what you said, that is if I want to extract a single file, the whole .tar.gz has to be decompressed. But for .tar only, it seems that is not needed. I'll search for some source about that:). For the pure UNIX alternative, maybe I should ask on a more pertinent site like Super User or Linux & Unix. Thanks anyway for the answer:) – mars Nov 24 '16 at 17:29
  • The [Wikipedia page for TAR](https://en.wikipedia.org/wiki/Tar_%28computing%29#Random_access) you posted in the answer says: "But today there are a number of add-on utilities which implement tar file indexing, thus enabling random access, both for raw tar files and for tar files compressed with gzip (which is amenable to indexing). Such an index can be kept in a separate file, appended or prepended to the archive file." I think that is what happens for example when using TAR through Dolphin/Ark instead of inside the command line. – mars Nov 24 '16 at 17:40
3

Assumption 1) is incorrect. Using the password on an 7Z, RAR, ACE or a lot of older archivers allows you access to the file 'tree'. You will still have to extract (decrypt) each file if you want to actually access something. This is valid since the MS-DOS era.

So you practically access an index of your files, nothing more.

Also note, that the decryption key is tested against every file. Archivers only automate this and apply the same key for all files, but you can actually encrypt each file using a different passsword if you want.

-Edit- You can actually study the 7Z code here.

Overmind
  • 8,829
  • 3
  • 19
  • 28
  • So i have ti suppose that 7zip (and others probably) Can decrypt only a specific file in the archive? Isn't this a bit unsecure in someway against encrypting even filenames like in the above command? I think that at this point however, extracting the whole encrypted archive should take much more time that extracting the unencrypted version! I'll try later. – mars Nov 24 '16 at 12:53
  • Yes, you can decrypt and extract a single file independently on the others. There is one exception to this: if the archive is made in solid mode , the encryption is applied over all data and a specific file will be dependent on previous ones being decrypted. Also note that extraction time is dependent on how well optimized the encryption/decryption algorithm is. Key size is also a performance factor. Try comparing Truecrypt and Veracrypt at volume access time as an example. – Overmind Nov 25 '16 at 07:10
  • Added link with 7Z source code; may clear things up if you're a programmer. – Overmind Nov 25 '16 at 07:17
  • With "Key size is also a performance factor" do you mean that an archive with a longer password will take longer to extract? – mars Nov 25 '16 at 15:28
  • No, the key is of given size no matter the password length. – Overmind Nov 28 '16 at 07:24