Compression: effects of varying 7zip parameters

I'm testing 7zip for compressing virtual machine images. I run these with fixed (pre-allocated) disks for performance reasons, and defrag/zero free space before compressing them (this generally makes a significant difference to the final size). I started each test below with the same 8GB VMware virtual disk containing Windows XP, Delphi 7.1 and not a lot else. These tests ran on a dual Xeon machine with 2GB RAM that was not doing anything else while the tests ran. I set the command prompt to display time and date, then ran a batch file with the output redirected to record times and commands. The batch file and raw output are in this file. After running for 40 hours WinRk crashed, leaving a 1GB temp file behind. I'm not interesting in continuing with that program. I've also run tests on other random files.

The command line I used always started with -md= 27 to give a dictionary size of 128MB and use 1.3Gb of RAM while compressing. The - mmt option to use both processors (and almost halve the compression time). The results with multiple dashed numbers all had -mx9 (maximum compression) but varied other parameters. The last command line in order: 7z a -mx9 -mmt -md=27 -ma=2 -mfb= 255) where a= [0|1|2] Sets compressing mode and fb=[5..255] sets number of fast bytes for LZMA. Help on that: Default value is 32 for normal mode and 64 for maximum mode. Usually big number gives a little bit better compression ratio and slower compression process. The Proc % column is processor utilisation - over 100% means the compressing program was using both processors in the dual-CPU machine.

The two bold entries are points 91) and (2) on the chart below. The first one is interesting because it should be the same as -mx1 or -mx5 (since 7z officially only uses 1,5,7,9) but it looks as though it's a single threaded version of -mx5.

The extra ones labelled 9a,9b,9d had command lines contributed by someone in the forums are as follows:

7z a -mx9 -mmt -m0=bcj2 -m1=lzma:d27 -m2=lzma:d20:mf=bt2 -m3=lzma:d20:mf=bt2 -mb0:1 -mb0s1:2 -mb0s2:3
7z a -mx9 -mmt -m0=bcj2 -m1=lzma:d27 -m2=lzma:d20:mf=bt2 -m3=lzma:d20:mf=bt2 -mb0:1 -mb0s1:2 -mb0s2:3
7z a -mx9 -mmt -m0=bcj2 -m1=lzma:d27:fb32:a1 -m2=lzma:d20:mf=bt2 -m3=lzma:d20:mf=bt2 -mb0:1 -mb0s1:2 -mb0s2:3

File Size vs Total Compression Time

The steep drop in size and compression time increases is in the first series of variations - as the compression setting changes from - mx1 to -mx9 in several steps the time taken increases and resulting size drops.

Once I start playing with the compression mode and fast bytes things don't change very much, and worse, they do so fairly unpredictably. So you would probably need to produce several files and choose the smallest. Note that mode 2/32 bytes gives similar compression to 2/64 but is noticeably faster, and that 255 fast bytes gives a bigger file than 128 but takes 50% longer (5 hours!).

I used 7Zip 4.26 beta, WinRK v1.04.1 (alpha), gzip 1.3.5, bzip2 1.0.2.

Comments? Email moz at the geek domain.