bzip2 vs pbzip2

[JillSwift]

I have a 4 core processor, and I wanted to take advantage of the unused 3 cores to speed up my cron managed morning backups-to-external-HD. So, I looked around and found pbzip2 - a version of bzip2 that does parallel processing. To make tar use it, I:

 

mv /usr/bin/bzip2 /usr/bin/bzip2.old
ln -s /usr/bin/pbzip2 /usr/bin/bzip2

 

I've already had a good test run this morning (going from just under an hour to just under 15 minutes, yay! :happy: ).

 

However, true to my paranoid self, I'm now concerned this will negatively impact some other aspect of my system... though I can't think of anything.

 

Can anyone else? Or, is there a "best practice" for doing this other than making a link?

 

 

[moved from Software by spinynorman]

[Greg2]

It may cause some problems if you do any dev or rpm work on that system... however I'm not positive. :)

 

I would suggest removing the symlink and try something like this

[greg@halfway temp]$ l
usr/
[greg@halfway temp]$ tar cf usr.tar.bz2 --use-compress-prog=pbzip2 usr/
[greg@halfway temp]$ l
usr/  usr.tar.bz2

That was simply a small test with my single core cpu, and it worked ok. You would have to check it with your cpu.

[JillSwift]

Bummer: I tried that and tar segfaults every time. I've no clue why that would be, given that there's little difference between the link and instructing tar to use a different compressor.

 

I keep experimenting.

 

(BTW, I don't yet know how to make RPMs, so I've not tried that.)

[scarecrow]

AFAIK the only way is recompiling that app with cflags suited for your CPU/hardware.

Mandriva binaries are "just" i586 optimized.

[JillSwift]

The first ting I tried was recompiling bzip2 - there is no support built into the code for parallel processing, thus pbzip2. :)