Using rsync to sync mirror [solved]

[ianw1974]

I set up the three following jobs in /etc/cron.daily:

 

[admin@elan cron.daily]$ cat rsync_main
rsync -rv rsync://anorien.csc.warwick.ac.uk/Mandriva/official/2006.0/i586/media/main/* /home/ftp/pub/mirrors/
mandriva/official/2006.0/i586/media/main/

[admin@elan cron.daily]$ cat rsync_contrib
rsync -rv rsync://anorien.csc.warwick.ac.uk/Mandriva/official/2006.0/i586/media/contrib/* /home/ftp/pub/mirrors/
mandriva/official/2006.0/i586/media/contrib/

[admin@elan cron.daily]$ cat rsync_updates
rsync -rv rsync://anorien.csc.warwick.ac.uk/Mandriva/official/updates/2006.0/main_updates/* /home/ftp/pub/
mirrors/mandriva/official/updates/2006.0/main_updates/

 

I then made executable so that cron would run them using "chmod +x scriptname". I checked this morning, and found for some reason, that I was missing a few updates that had been released recently. This made me realise that the script wasn't working correctly, or cron didn't run the job for some reason.

 

Nothing seems to show in any logs for the rsync jobs, so it's making it a little difficult for me to tie down where the problem is. Am I running it the right way, or should I be using an alternative? I'm only mirroring for my own use, and not for people to connect and mirror from me since I don't have the bandwidth for this.

 

Any ideas appreciated.

[aioshin]

the default crontab daily runs at 4:02 am.. maybe at that time the update repo has not been populated yet with the new updated packages

[ianw1974]

One of the updates was released a couple of days ago, but it only got picked up this morning when I ran the script manually.

 

So, I'm not sure it's running my job correctly for some reason, or even if I'm doing it the correct way anyhow. I know there is a rsync daemon, but I think this is to allow people to connect to you, rather than run schedule jobs.

 

I'll check in case it's a path problem, and add the full directory path to the script, but it certainly looks odd as to why it missed running the job and picking an update from two days ago.

 

EDIT:

 

scripts are the same as above except "/usr/bin/rsync -r" now, instead of what it read before.

[aioshin]

rsync by default could be run by any user, so path to rsync should not be a problem... did you not forgot to put the

 

#!/bin/sh

 

on your script.. I noticed the output of those you'd cat.. no #!/bin/sh.. or maybe you'v just exclude it :P

Edited June 2, 2006 by aioshin

[ianw1974]

Yes, I didn't put it. Maybe that's it, I'll try now and see if it makes a difference! doh :o

 

I usually do put it, I must have forgotten it. I guess it's necessary after all. Will update, to see if it works over the next few days.

 

EDIT:

 

OK, now added to the beginning of the script:

 

#!/bin/bash

 

as I use bash normally, and I also noticed some scripts have bash and some have sh. Is it important which one is listed at the top, or not important as to which shell you use?

[aioshin]

which ever will work, I think that's what you should try to use...

[ianw1974]

OK, this now works, forgot to update until now :P

[Steve Scrimpshire]

ianw: on my system, /bin/sh is just a symlink to /bin/bash:

< omar ~ > ls -l /bin/sh
lrwxrwxrwx  1 root root 4 Jul 17 21:33 /bin/sh -> bash*

[ianw1974]

Ah ha, that could explain it then :P. I've never looked into it, but always set as /bin/bash anyhow. The script started working as soon as I added this. I guess they must need this identifier else they fail to run, which is cool. Means I learnt a bit about cron jobs and the lack of them running even though manual was fine :D

[aioshin]

you can also create a file under /etc/cron.d/ .. say /etc/cron.d/rsync_main then with the entries like below

01 20 * * * ian /usr/bin/rsync -rv rsync://anorien.csc.warwick.ac.uk/Mandriva/official/2006.0/i586/media/main/* /home/ftp/pub/mirrors/
mandriva/official/2006.0/i586/media/main/

 

so that would run 8:01 pm daily as been set and executed by user ian.. and that is only a plain file, you dont need to make that executable, but should be created by root.