So you’ve maybe read about my issues with OpenSolaris and the Windows Server + Exchange 2010 I wanted to run on it. The base system itself is, as mentioned before, very stable (as you would expect from such a system). With the idea of NOT using Exchange 2010 at home (because of several reasons) and still “hunting” for the perfect storage & backup strategy and yes, I admit, by browsing through this nerdy thread, I played with a configuration in my head that I would like to show.
Imagine the system you know as Belldandy already, running OpenSolaris. The main system is installed on an IDE 160GB (WD / new) HD. There is an Intel Desktop CT 1000 + a Sil3132 chipped eSATA controller stored in both PCIe slots (x1 & x16). Now I would add the following HDs.
-640 Hitachi SATA-II
-640 WD SATA-II
-640 WD SATA-II
-640 WD SATA-II
1000 Seagate SATA-II
1000 Seagate SATA-II
Mirror II – usable: 1000/990GB around about.
4000 STORAGE eSATA Main Storage for very big files (Onnto DataTale “STORAGE”) usable: 2.7TB4000 BACKUP eSATA
Backup of “Storage” (Onnto DataTale “BACKUP”) usable: 2.7TB
-160 Boot IDE
Simply the OpenSolaris on it. Usable: 140GB…………….
12.700GB overall, usable after mirroring & striping: 7.630GB
Yeah, freaky plan and MAYBE a step back, but I still think that maybe ZFS (and even if I run the STORAGE & BACKUP in an external hardware RAID5) is the very best option.
For a secure and reliable storage that would feed my needs perfectly, I would say: Ok – I will NOT use Exchange. I will find another way to accept invitations, sure – easiest would be to run Outlook in a Parallels VM and simply accept it there. Exchange 2010 is pretty good, but I also have concerns to let it run when I’m out of the house for some days. That’s maybe my main painpoint.
So what do you guys think? Is it worth to retry it a last time?
I’ve VERY compatible hardware here incl. meanwhile 6GB of ECC RAM for this box.
Would be great to get a feedback from you!
Ciao
Dennis
See for reference: About my project Belldandy
PS. Why I don’t think about using Debian? Well, I don’t see a very reliable filesystem there that I would like to use. ext3 is too slow for large files, xfs seems also not be the fastest one and ReiserFS is in jail




Chris B.
Hey Dennis,i think it’s definitely a good idea to give your opensolaris-server a “last chance”. I’m not a fan of exchange (for home use at least) anyway. One word concerning filesystems: I don’t think you’d notice the difference in speed between ext3, xfs and zfs. However, the linux filesystems all don’t have checksumming, so that’s the real problem. and reiser of course is no option anymore
. All the best,Chris
Constantin Gonzalez
Hi,I think not letting Exchange dictate your server needs is a very good idea
. I’m sure you’ll find Microsoft-free ways to communicate with your clients…As for the storage, this looks good. You don’t have to manage the two Onnto Boxes, you could set them up as a mirror in ZFS as well and then you’d have data integrity which you don’t in the case you use the Onnto Boxes directly on the Mac.If you don’t need the CD-ROM anymore on Belldandy, I’d suggest mirroring the boot disk as well. Or at least run regular backups from it. Otherwise it’ll become a single point of failure and cause some headaches when it breaks.Cheers, Constantin
Dennis Klein
Wow – posted really late last night and already have comments in the morning
Great guys! (Finally a good sign that someone reads my stuff.@Chris: Yeah ZFS indeed is the driving argument in this idea. I mean, I’m not very experienced in OpenSolaris and without Constantin’s help a few days ago I had crashed the system already
I started the search for a filesystem last night, because I’m much more familiar with Debian than with OpenSolaris, but the filesystem and the hunt for the right one raised concerns in me. Doing this for meanwhile 3 weeks we, I want something I can say “it was worth spending my time for” at the end – means: something that I set up once and let it run until I leave the home for more than a few hours and also be able to simply turn the machine off. I think I don’t have to explain that an own Exchange does make this a bit more complicated – you’ve to switch your MX entries, create the same mailboxes on a 2nd system that is offsite hosted and in the end, you need import the meanwhile incoming mails to your Exchange. Don’t get me wrong, I highly appreciate Exchange, especially for the great iPhone (and iPad) Support, but not sure if it’s worth the headache.Because we’ve 2 cats running through the house and me and wife are really addicted to them, I will not let a computer run when I’m not at home. There’s a very minor risk, I know, that the computer could fail and start a burning, but I want to be 100% sure. That’s just the reason why I avoid having a system running 24/7/365
@Constantin: But RAID != Backup is prayed and written to every wall out there. If I do a mirror of the boxes, I don’t have a backup? Sure, I have the data on 2 independent (more or less
) places, but when I delete the file it’s gone, unless I really create snapshots every few minutes, right? Or maybe I misunderstood this very part of the ZFS “magic”. Because if that is the fact (if snapshots really replace a classic backup), I would start thinking of striping both boxes as a kind of ZFS-mixed-2-RAID5-sets, a RAID50 based on ZFS and the 2 external boxes. I mean, having 5.4GB available on the STORAGE would be a great thing, but only if it’s secure.Regarding CD-ROM, in that way I’m a bit radical
I don’t have built in a ODD into a PC since many months. I’ve a great external USB drive (DVD-R/W/RAM from LG) that I also tend to use on my Mac Pro (as the internal delivered one is very very loud!). Nether the less, I don’t currently have a 2nd 160GB IDE HD
There’s a free IDE port on that cable, true, but I also wouldn’t know how to set it up as mirror. Honestly, I searched on Alternate last night and found a nice SATA-PCI controller from Promise for cheap money (69€), maybe a nice addition for later, but then I would need a bigger case as with this plan, there are already 7 HDs inside (with my Lian Li box in 2x 5.25″ slots, I could max. add up to 9 HDs (3 in the Lian Li + 6 in the case itself). Drifting away from topic – sorry.CiaoDennis
Constantin Gonzalez
It depends on what the purpose of the “backup” is. If it’s against user error, then snapshots are sufficient (and automatic snapshots help a lot). Most of the time, though, “backup” means “disaster recovery”. And that implies an independent, physically separate place to store your data.In that case, you’re right, ZFS won’t help you do backups of the latter sense. But a second RAID box right next to the first one won’t either. If the house burns down the house will burn down with both of the boxes.I don’t have a full backup either, but I do rsync my most important personal documents and projects to an encrypted drive hosted on Strato’s cloud and most of my photos are stored with SmugMug. I can live with losing my music ’cause that could be easily replaced.But I’m seriously thinking of getting a cheap second machine that I can set up in a different room (or in a friend’s place) that does a 1:1 backup over crashplan.com’s utility or through ZFS send/receive (there’s a script that uses ZFS send/receive over encryption that is very neat: http://blogs.sun.com/gbrunett/entry/saving_encrypted_zfs_snapshots_to
Dennis Klein
I also thought about getting an account from BackBlaze, Mozzy or CrashPlan or one of their competitors. On the other hand I could use my Webserver for critical data, too. It’s a dedicated Debian machine and I’ve 500GB free there (which is protected through RAID1), so maybe that’s a cheaper idea and AFAIK I can also ask rsync to sync to my webserver (as I have 2 independent DSL lines I have no worries that it would kill my connection
But this is really meant for CRITICAL data like customer’s projects (PSD files, Excel sheets, Presentations and maybe VERY good to backup: Invoices
).I’m not that sure how to take your answer regarding the striping of the 2 ZFS boxes, so I ask it: Could I do this (or: Should), or better don’t do. I mean, 2 drives could fail (in 2 seperate boxes). On the other hand, I could also put the devices into 2 RAID10 setups (which is possible with the Onnto DataTales), which would give me a mich higher security and 4000GB = 3.8TB or available storage. That would “waste” 2TB for security
Waste in quotes of course.
Dennis Klein
PS. Sorry for all those typos
Seems like I’ve been laying too long in the sun already today…
Constantin Gonzalez
The issue that I see with the Onnto boxes is that if a bit flips and a piece of data goes corrupt in them, you’ll never notice. Then, when you perform the backup, the broken data will be backed up into your backup store where it may possible overwrite good data.The only way to find and correct data corruption is to use ZFS and run regular scrubs. ZFS would then automatically identify broken data and heal it using the redundant information that it has available.So, if the data is to be securely stored on the Onnto boxes, you need to use ZFS as a file system (so it can detect broken blocks) and let ZFS manage the redundancy (so it can repair those broken blocks). Since you only have to Onnto boxes, your only option is to use mirroring at the ZFS level.The RAID protection that the Onnto boxes offer is limited: They can only detect broken _disks_ not individual broken data blocks. And if their RAID firmware fails (it happens to much more expensive hardware, too), their RAID won’t help you either.This is why ZFS loves individual disks: That’s where it can use all of its power without limitations.If you want to keep the Onnto boxes and if you want to take advantage of ZFS, then using RAID5 inside the boxes will give you as much capacity as possible, then mirroring them using ZFS will give you data integrity and self healing. Yes, using RAID 1+0 inside the Onnto boxes would be even more secure, if you want to spend the extra disk.The very best option would be to give back the Onnto boxes, buy 8 individual disks in a JBOD case instead, plus an extra eSATA controller (or two) (which should be cheaper than the Onnto boxes) and use the saved money for a mirrored boot disk
. That would give you more flexibility and less “Verschnitt”, because ZFS woud have total control of the disks and hence better possibilities to react to failures.
Dennis Klein
Hi Constantin,hmm sounds not very good, but I highly appreciate your clear advises and comments on this!Honestly, I don’t want to give the 2nd (just received it last week) box back. I understand that OpenSolaris and ZFS needs some special things to be prepared for. Of course I want to have my data secure, but with the time I start thinking if it’s really worth it. Sure, data should be secured and so I think more about the already mentioned Online-Backup to my web server and a weekly backup on DVD-RAM for my really important company based data.I mean, I want to have a solid and reliable storage, but more and more it get’s supercomplicated. First it needs ECC RAM, ok – bought it + board + CPU and that’s ok. But now it’s simply to much of a “Kopfstand” to get it done.Additionally, I don’t have so much experience with OpenSolaris and maybe in this case, maybe the Win server (which allows me to simply attach these boxes to the Mac – IN EMERGENCY CASE) and let me copy from NTFS to another HD OR a Debian server could serve me better :/This whole process is very disillusioning. Additionally I just noticed, that one of the Seagate HDs just died (yikes!).I’ve checked the documentation, but it seems that the Onnto DataTales do NOT support JBOD. Even though, I’m not sure if it would be such a clever idea to use it.I’m just wondering if everyone had data corruption errors when using ANY other filesystem than ZFS? I mean, sure if can happen and I understand the pros of ZFS compared to other filesystems, but is it really worth this stress?
Constantin Gonzalez
I see checksum errors on my home server about once every 4-6 months. ZFS fixes them automatically for me, while other systems won’t notice (and instead would serve, say, a broken JPEG when it’s too late).Here’s an example: http://blogs.sun.com/constantin/entry/zfs_saved_my_data_rightYes, it becomes complicated once you think stuff really through, but it’s the nature of the problem…Anyway, your time invested into OpenSolaris and ZFS was good – now you know a lot more about data integrity than most Windows or Linux users. Just don’t give up right before the goal
.Cheers, Constantin