banjo Posted June 17, 2005 Report Share Posted June 17, 2005 I have an interesting problem with using cp -u. I have an external USB drive with 40 Gig FAT32 (for portability). I plan on using the external drive to keep some backups. I am doing my backups by simply copying selected directories to the external drive.... nothing fancy. I am writing the files using "cp -u" to make the copies a bit faster (the -u switch causes it to copy only new files and files that have newer timestamps than the destination file). I wrote a bunch of my directories to it last night to see how it would work. This went very well for the most part, and the files were copied OK. But there were problems with files that had the same names and only different capitalizations (FAT32 is case insensitive). So I changed the names of some of those files to avoid the problem and did the "cp -u" again. The "new" files were not copied over. I tried this a couple of times with the same result. The last time I tried it, cp returned immediately without really accessing the drive, as though the information had been cached or something. The renamed files did not show up on the external drive. Questions: How does cp decide what is a "new" file? Does changing the name leave the modification time untouched? Do I also need to touch the files? Does cp cache file information? How can it know the timestamp of the destination without reading the external drive? I am running Mandy 9.1 on a ext3 copying files to an external USB with a 40 Gig FAT32. Thanks in advance Banjo (_)=='=~ [moved from Software by spinynorman] Quote Link to comment Share on other sites More sharing options...
banjo Posted June 18, 2005 Author Report Share Posted June 18, 2005 (edited) Ah, me. I fgured this one out. :blush: I started using Unix in about 1975, and cp was just about the second command that I learned after learning cd....... so you would think that I would have it under control after 30 years, but............ Did I mention..... :blush: ? Here is what happened in the Case of the Missing Files. If you use a command such as cp -dpRu /home/user/stuff /mnt/storage/home/user/stuff to copy a directory to an external drive, it copies the entire directory "stuff" to the destination.... including the directory file itself........ OKfine. That makes perfect sense. But the first time I executed the command the destination file, /mnt/storage/home/user/stuff did not exist yet. It appears that if the target directory does not exist, cp is happy enough to copy the entire directory one level up... into.... /mnt/storage/home/user, thus creating the target directory itself. So I ended up with the intended result, /mnt/storage/home/user/stuff with the rest of the directory tree beneath it. The second time I executed the same command, it did find the directory called /mnt/storage/home/user/stuff, so it copied the source directory /home/user/stuff into /mnt/storage/home/user/stuff thus giving me /mnt/storage/home/user/stuff/stuff where the entire directory tree was duplicated.... including the "missing" new files. I simply did not notice that the new duplicate had been created and therefore I could not find the new files. The solution to this embarrasing problem is to issue the correct command to the cp program, which is: cp -dpRu /home/user/stuff /mnt/storage/home/user with no "stuff" on the end. This works with the intended results first time and every time thereafter since all it needs to do is to copy the source directory into the destination directory. Thanks to the MUB for your patience with this 30 year, veteran n00b as he stumbles through the use of basic commands. Linux rocks! :blush: Banjo (_)=='=~ P.S. I also put this into the wrong forum since it is a software issue anyway. :blush: Edited June 18, 2005 by banjo Quote Link to comment Share on other sites More sharing options...
banjo Posted June 19, 2005 Author Report Share Posted June 19, 2005 Not to beat a dead horse or anything, but never ever ever never ever try to back up a Linux system onto a FAT32 file system. It works great for transferring files, but there are so many problems with file incompatibilites that it is not worth the effort. 1). Links are not copied since they cannot be created on the FAT32. 2). Mail cannot be backed up because the file names have invalid characters in them. 3). File ownership is not preserved during the copy, and chown on the FAT32 is not allowed. 4). The FAT32 uses more space to store the files than ext3 (because of cluster issues, I suppose) I am just about to give up on this and wipe the disk and start over. Or maybe I will give this one to the kids for their fnWindows laptops and get a newer, bigger one for me. :D Anyway, if you are thinking of using a FAT32 system as a backup disk, think again. Banjo (_)=='=~ Quote Link to comment Share on other sites More sharing options...
banjo Posted June 21, 2005 Author Report Share Posted June 21, 2005 I thought that I would add an update to this topic since I am discovering things that I did not know before. When talking to a FAT32 partition, there is no facility in the file system to store Linux style file attributes. The only ones that it can save are read/write and perhaps the create and/or modification dates. This makes perfect sense if I had just thought for a minute about how the FAT32 has worked in the past. It is simply a really lame file system. What is confusing at the surface is that Linux fakes up the ownership fields by using the mount properties of the device and displays them for all the files. Hence, the output of ls presents file ownership for the user who mounted the external FAT32. That ownership information is not preserved anywhere. I have confirmed this by logging in as another user, mounting the external drive, and then seeing that all the files on the disk now belong to that other user. So, of course, chown will not work. D'uh. More discoveries to come, until somebody tells me to shut up. Banjo (_)=='=~ Quote Link to comment Share on other sites More sharing options...
AussieJohn Posted June 22, 2005 Report Share Posted June 22, 2005 I think you are going great guns mate. I am following your posts with great interest and I am learning a lot from you. Cheers. John. Quote Link to comment Share on other sites More sharing options...
theYinYeti Posted June 22, 2005 Report Share Posted June 22, 2005 What I often do with "limited" filesystem (ISO-9660, FAT...) is to use them to store tar files. So the backup goes something like this: /place/to/backup# tar cjf /mnt/fat32/BCKP0001.TBZ . Everything is kept as it should (Unix-like) in the tar file. Yves. Quote Link to comment Share on other sites More sharing options...
banjo Posted June 22, 2005 Author Report Share Posted June 22, 2005 Thanks for the tips. I am considering how to use the tar files effectively to do these backups. I am still trying to keep my backups portable. I also like having the files visible on the drive instead of buried in a tar. The other people using this computer can cope with a copy, but might not want to deal with extraction from a tar. There is only one Linux guru in our family.... The files in my Mail directory are being rejected by the FAT32 because of invalid characters in their names - I think it is a colon. But, if I tar them off, I will only be able to extract them back to a Linux box. But that is OK because they would be useless on a WinBox anyway. So I might do that. I now have a script that I use to do the copies for me. It asks me where the USB disk is mounted and then it reads a list of directories from a file, which I call ~/.backupdirs, and copies them -R over to the external drive. Other than the glitches I mentioned before, it seems to work fairly well. I might add some code to look for the Mail directory and tar.gz it just in case. I will never go back to fnWindows permanently anyway, so that should do the job. I still have some permission glitches, but I might be able to get around those by running the script as root. Since ownership is not stored on the external FAT32 that should not be detrimental for any user's ability to retrieve the files. The search goes on. Banjo (_)=='=~ Quote Link to comment Share on other sites More sharing options...
theYinYeti Posted June 23, 2005 Report Share Posted June 23, 2005 If you want a really Cool! backup with files readily available, then look at this: http://mikerubel.org/computers/rsync_snapshots/index.html Yves. Quote Link to comment Share on other sites More sharing options...
banjo Posted June 23, 2005 Author Report Share Posted June 23, 2005 That certainly is a unique and interesting method for backups. Unfortunately I am making my backup on a FAT32 system, and there is no facility for any kind of links at all, much less hard links. It doesn't even store ownership or file permissions. Banjo (_)=='=~ Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.