Compress your files faster with pbzip2
Everybody uses tgz (tar + gzip) to compress files. Some use bzip2 because it gives higher compress ratio per archived files.
A faster solution is pbzip2 which make use of every core of your CPU, and/or every CPU of your system.If you benchmark a bzip2 archieving procees and in the mean time you use top -P you will see that your system load will
not pass over 1 even if you have multiple cores. That means bzip2 does not scale. Try the same archieving step with
pbzip2 and you’ll see the difference in load and in time that will take to compress/decompress files.
pbzip2 -z file.iso # or —compress, it will archieve file.iso file
pbzip2 -d file.tbz # or —decompress, it will unarchive file.tbz into file.iso
If you want to use pbzip2 with tar ( instead of bzip2) you have the following choices:
1. create an alias from tar to tar —use-compress-prog=pbzip2
alias tar tar —use-compress-prog=pbzip2
2. move bzip2 binary and create a symlink to pbzip2
mv /usr/bin/bzip2 /usr/bin/bzip2.old
ln -s /usr/local/bin/pbzip2 /usr/bin/bzip2
3. use pipe
tar -cf — . | pbzip2 > ../file.tbz
4. use —use-compress-prog parameter of tar (which will also pipe to pbzip2
tar —use-compress-prog=pbzip2 -cf file.tbz .
There is also pigz — parallell gzip — multi thread gzip compression utility.