Gowator Posted April 13, 2007 Report Share Posted April 13, 2007 Any thoughts on using NTFS for the storage drives with ntfs-3g? From what I've been reading about it, ntfs-3g is very good and it doesn't look like I'd have any problems using NTFS for Linux/Windows storage. Sorry... no windows here so I haven't really looked into the 3g stuff ..... One thing to consider though is that if its a backup drive then you want it to be readable by the max amount of PC's.... without needing to start messing about (not saying you need to with 3g since I didn't try but something to consider).... If the worst happens then being able to plug that disk into any computer might be real useful :D Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 13, 2007 Author Report Share Posted April 13, 2007 My thoughts exactly. I just don't know how well using rsync with NTFS works. Quote Link to comment Share on other sites More sharing options...
emmanuel_uk Posted April 13, 2007 Report Share Posted April 13, 2007 maybe an ignorant comment, but what cp, rsync or whatever got to do with ntfs? ntfs-3g would be in the middle between the 2 HD/partitions. The weak link is ntfs3g not rsync. even if ntfs-3g is working ok, is it still beta? you should be driven by biggest size of file you will handle and by how easy to recover / access the file. No idea if fat32 is better than ntfs, but not all live distro have ntfs-3g oops not talking raid anymore, are we? sorry not providing much help here Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 14, 2007 Author Report Share Posted April 14, 2007 I was under the impression that ntfs-3g and rsync would have to work in tandem to sync the files from one NTFS drive to the other. At least, that's what makes sense to me. ntfs-3g is actually in stable release and very workable. According to the site it works flawlessly the majority of the time. This wouldn't be a system run off a live CD - it would be a dedicated server box. And I've ditched the RAID idea for just having a backup drive that I rsync every few days. I'll be changing the name of the thread to reflect this as soon as I add this reply :) The files I will be working with are mostly between 2MB and 5GB. On the smaller end are songs and pictures and the larger end is ISOs for games and distros. NTFS, in my opinion, is better than FAT32, and it's better for what I'll be doing with the files. Quote Link to comment Share on other sites More sharing options...
iphitus Posted April 14, 2007 Report Share Posted April 14, 2007 ntfs-3g just gives you access to your ntfs files from linux, it will work fine with rsync. You could just as easily use the stock linux ntfs module as well, as rsync only needs read-only access to the windows files. James Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 14, 2007 Author Report Share Posted April 14, 2007 (edited) How would it write the files to the backup disk then? Wouldn't it have to be able to write NTFS to backup the files to another NTFS disk? Or am I just confused about what exactly rsync does? Edited April 14, 2007 by Scythe Quote Link to comment Share on other sites More sharing options...
Gowator Posted April 14, 2007 Report Share Posted April 14, 2007 How would it write the files to the backup disk then? Wouldn't it have to be able to write NTFS to backup the files to another NTFS disk? Or am I just confused about what exactly rsync does? no i guess he was just jumping in at the end..... If you want to write to it then you need write .... I personally would probably keep away from NTFS.....It just feels wrong .... I was doing a install for someone yesterday with no CD so I used a install image on a fat part and edited their grub to boot into the live ISO from that drive.... After 1hr messing about (and I had the devs of sidux helping) I said fsck this.... (sorry) and deleted the FAT32 part and replaced it with a ext2 linux part... put the files back.. touched nothing and it worked.... Since its a backup I'd just try and make it the most readable without doing anything.. if you ever need it (like the server goes to pot) you can just boot from for instance a LIVE CD and even make a temp server running from the live CD while you fix/recover.... thats just my 2c though! Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 14, 2007 Author Report Share Posted April 14, 2007 (edited) Ok, I think I'll rehash my intentions for this server to avoid further confusing everyone :) The server will be set up as such: 1.) 10GB HD with the Linux installation (Probably FC6) 2.) 320GB HD shared with a Windows/Linux network via Samba 3.) 320GB HD set in a rack to be used to backup the other drive every few days, otherwise it will be powered off I will use rsync to syncronize the files between 2 and 3. My question is whether having the 320GB drives in NTFS will work well. The shared drive contains pictures, music, TV shows, and ISOs ranging from a couple kilobytes to 4.7GB for the ISOs. FAT32 will not work for the ISOs, hence my desire to use NTFS. As my Windows machines will write to Drive #2, having it in NTFS would be advisable unless I wanted to use the ext3 drivers in Windows. *edit* added the part about Drive #2 being shared via Samba Edited April 15, 2007 by Scythe Quote Link to comment Share on other sites More sharing options...
Gowator Posted April 15, 2007 Report Share Posted April 15, 2007 Good idea:D (to redefine).... I honestly don't see why NTFS wouldn't be OK.... I'm just naturally averse to it because I don't need/use it.... If I'd done it myself I'd feel more confident.... tellin you but I haven't used/tried the 3g... One thing I'd definately do anyway is run a test.... my concerns are really just over the different way ntfs handles dates and permissions because rsynch needs these to perform the differential backup... (i.e if you just want to do newer/unchanged files) Sometimes, like in my grub example these little things aren't immediately apparent... I tend to find the better (more transparent) the process the higher chance you'll miss something so try it out.... Ive done similar things with SAMBA and found I needed some of the more "advanced" options to get the dates and permissions working properly.... but it did work.. it just took a bit of tweaking... Quote Link to comment Share on other sites More sharing options...
iphitus Posted April 15, 2007 Report Share Posted April 15, 2007 Ok, I think I'll rehash my intentions for this server to avoid further confusing everyone :) The server will be set up as such: 1.) 10GB HD with the Linux installation (Probably FC6) 2.) 320GB HD shared with a Windows/Linux network 3.) 320GB HD set in a rack to be used to backup the other drive every few days, otherwise it will be powered off wait up a moment, how is drive 2 being 'shared'? If it's shared across a network using samba, then it's fine for it to be ext3 or any other linux filesystem. The windows clients do not write to the partition, they communicate with samba, and samba writes to the partition. The windows clients never see the real partition, they just need to speak SMB, the samba protocol, to tell samba what to do, and samba, running on the server does the work. James Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 15, 2007 Author Report Share Posted April 15, 2007 Ok, I think I'll rehash my intentions for this server to avoid further confusing everyone :) The server will be set up as such: 1.) 10GB HD with the Linux installation (Probably FC6) 2.) 320GB HD shared with a Windows/Linux network 3.) 320GB HD set in a rack to be used to backup the other drive every few days, otherwise it will be powered off wait up a moment, how is drive 2 being 'shared'? If it's shared across a network using samba, then it's fine for it to be ext3 or any other linux filesystem. The windows clients do not write to the partition, they communicate with samba, and samba writes to the partition. The windows clients never see the real partition, they just need to speak SMB, the samba protocol, to tell samba what to do, and samba, running on the server does the work. James Really? Wow, I had no idea it worked like that. Yes, the drive would be shared via Samba. So if I wanted to use Photoshop files on Drive #2 in Windows and then save the files after I'm done, I can do this without installing the ext3 driver for Windows? That's nuckin futs man! :P Quote Link to comment Share on other sites More sharing options...
Gowator Posted April 15, 2007 Report Share Posted April 15, 2007 You could for instance use Xen and run SW RAID in linux with samba and then the Windows "machine" could use these disks... I do something not completely different but I only use Windows occaisionally so I have a VMWARE session (not good for gaming but that doesn't bother me) and I just use samba to read/write ... You can't dual boot (in the conventional way) and use SAMBA.... the actual "server" has to be running ... My understanding is you wanted the machine to be dual boot so you can run winows and linux on that machine and So if I wanted to use Photoshop files on Drive #2 in Windows and then save the files after I'm done, I can do this without installing the ext3 driver for Windows? If its on the same machine then no (unless you have a virtual machine) If you have a dedicated server running linux and SAMBA then from another machine (or 5000 machines) you can read/write to that machine in windows and linux at the same time.... (and netware, appletalk.... and a near endless list) In other words you don't need to have windows installed at all on a dedicated server... in order to read/write linux and windows.... So the question is are we talking a seperate machine or not.... If we are talking about a seperate machine you can just install linux and SAMBA and read write to it from another machine in windows or linux (or all at the same time).... In this case you can use linux RAID on a single RAID 1 for that machine Even if its just one machine you can for instance use a vmware linux client to act as the server... I do the reverse.. when I occaisionally boot windows its a virtual vmware session and it connects to both the "host machine" and my server via SAMBA... Quote Link to comment Share on other sites More sharing options...
Scythe Posted April 16, 2007 Author Report Share Posted April 16, 2007 This machine will only have Linux on it - I won't be dualbooting with Windows. That's on my main rig. I think I'll stay away from software RAID just so that my backup drive lasts longer. I wouldn't want to use RAID 1 just because of the chance of one failed drive killing the array. After all, I'm doing this for data security, not storage space ;) Quote Link to comment Share on other sites More sharing options...
AussieJohn Posted April 16, 2007 Report Share Posted April 16, 2007 The whole point about Raid-1 is that your data is mirrored to both drives. If one drive fails, sure the raid is broken but your data is still on the working disc. Sensibly you would quickly replace the failed disc and set to restore the raid, this time using the existing good disc as the source. Ideally you would not continue to use the machine until you had replaced the failed disc, because as was said earlier, if both discs were originated at the same time then there is the slight possibility that the nonfailed disc may fail some time soon. I use raid-1 but only while I am completely setting up my OS (Mandriva only) and when I am fairly sure everything is stable and working correctly. When this is complete then I break the raid. This way if I do screw up things badly on my working Os then all I do is restore the raid-1 using the second drive as source, redo the settings in Grub to renable booting into the first drive again and once more I am in business. Next reboot I break the raid again. Sure I could spend hours and hours trying to use the CLI to resolve the muckup I may have made but my way is effortless and gets the result I need, namely a working and effective OS, with the least stress and time. When we get close to a new version as is presently the case then I take a chance to install the new version on the first drive (i.e. my working drive) knowing I have a fully stable and reliable version on the second drive to fall back to if needed. I found this to be extremely handy a few days ago when I wanted to reduce the Picture Partition and use the extra space to increase the Music Partition. I used it for switching the music and picture data about as I made the Partition changes. Never lost a bit of data in the process and all works perfectly. All that Data is on DVDs so it is fully backed up but it would be a pain in the backside and take ages to reinstall it all again. Obviously I am constantly adding Music and Pictures but I add it to a special folder until there is enough for a DVD then I burn it to a DVD. The data in the Backup Folder is also progressively added to the Music and Picture Partitions as it comes in. When the raid is broken then I manually add this new stuff also to the second drive. This way new stuff is reasonably safe until I can burn it to DVD. More often than not I just temporarily restore the raid until fully mirrored, which only takes just over 40 or so minutes while I do somethig else such as make a meal or watch a favourite TV program. I am plannimg to get a SATA hard drive cage so I can place the second drive into it and be able to switch it off therefore increasing its mechanical life significently. Wow, Gowater you certainly have a mammoth amount of data. It must take you months to transfer that data. I can imagine your drives saying "oh no, not again" Lots of good discussion everyone and thanks to Iphitus I have a better understanding of how Samba fits into the picture too. Cheers all. John. Quote Link to comment Share on other sites More sharing options...
Gowator Posted April 16, 2007 Report Share Posted April 16, 2007 (edited) Wow, Gowater you certainly have a mammoth amount of data. It must take you months to transfer that data. I can imagine your drives saying "oh no, not again" Well each photo as a 16-bit TIFF takes about 24MB... and some photo's have 3-4 versions, it soon starts to add up. Mostly (consistency would be good here but ??) I only back up the raw files and processing chain from my processing SW..(usually less than 500kb for the processing chain and 5MB or so for the RAW file) However some stuff the processing chain uses more than one piece of SW (in my case usually bibble->TIFF then lightzone) so then I store a copy as a final 16 -bit tiff as well. I guess its like the old days :D mostly you can just throw away your prints because you can always redevelop and print them from the negs.. but if you spent hours in the darkroom, did some cross processing or some chemicals you don't usually keep about then you want to keep the print too. Specially for you I have an example (well you're most likely to appreciate it:D) http://farm1.static.flickr.com/166/4336728...40dea83c4_b.jpg This took hours of processing (but no editing the sky is REAL....) It was actually taken with a Yellow B&W constrast filter (over a ND8 and polariser) which brings out the layers of the clouds in an almost surreal way... I then had to play with the color temp a LOT to get rid of the yellow and dodge/burn... Edited April 16, 2007 by Gowator Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.