Jump to content
Sign in to follow this  
JillSwift

bzip2 vs pbzip2

Recommended Posts

I have a 4 core processor, and I wanted to take advantage of the unused 3 cores to speed up my cron managed morning backups-to-external-HD. So, I looked around and found pbzip2 - a version of bzip2 that does parallel processing. To make tar use it, I:

 

mv /usr/bin/bzip2 /usr/bin/bzip2.old
ln -s /usr/bin/pbzip2 /usr/bin/bzip2

 

I've already had a good test run this morning (going from just under an hour to just under 15 minutes, yay! :happy: ).

 

However, true to my paranoid self, I'm now concerned this will negatively impact some other aspect of my system... though I can't think of anything.

 

Can anyone else? Or, is there a "best practice" for doing this other than making a link?

 

 

[moved from Software by spinynorman]

Share this post


Link to post
Share on other sites

It may cause some problems if you do any dev or rpm work on that system... however I'm not positive. :)

 

I would suggest removing the symlink and try something like this

[greg@halfway temp]$ l
usr/
[greg@halfway temp]$ tar cf usr.tar.bz2 --use-compress-prog=pbzip2 usr/
[greg@halfway temp]$ l
usr/  usr.tar.bz2

That was simply a small test with my single core cpu, and it worked ok. You would have to check it with your cpu.

Share this post


Link to post
Share on other sites

Bummer: I tried that and tar segfaults every time. I've no clue why that would be, given that there's little difference between the link and instructing tar to use a different compressor.

 

I keep experimenting.

 

(BTW, I don't yet know how to make RPMs, so I've not tried that.)

Share this post


Link to post
Share on other sites

The first ting I tried was recompiling bzip2 - there is no support built into the code for parallel processing, thus pbzip2. :)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

×
×
  • Create New...