What you see here is far from “finished”. The HP ProCurve for example is not yet installed and also the Gateway is still on my ToDo list, but you get the idea.

The big black case is the ESXi server, the smaller silver behind that is the backup server. The rack contains the UPS, the 1U SuperMicro server and the electric wiring – and needs to get the final finish! Also, there’s a Compaq 4 Port KVM switch the CRT & keyboard are connected to that I’ve got for free from Christian.

Speaking of mess – this is the inside of the ESXi server – so much cables running through the box. But I’m sure you guys are curious to see the inside.

From top to bottom:
- LSI Controller (IT Firmware) -> connects to the HP SAS Expander via SFF-8087 cable
- LSI Controller (IR Firmware/Intel brand) -> connects the 3x 640GB via “RAID10E/IME” and is the home for the VMs as by today as the NFS was a bit to slow for my taste – or: this option is a lot faster
- HP SAS Expander -> the beast which connects up to 32 drives
- Intel Pro 1000 CT PCIe x1 NIC

The red light on the bottom right is the USB stick which is shown here. The ESXi software itself is stored there.

So I started setting up Endian on this sweet little box I’ve presented earlier. This thing is DAMN loud Having it sitting on my wifes desk yesterday I thought I need some ear protection – like standing in front of a running jet engine. Here you can see why I have some stuff in the basement I don’t care if it’s loud downstairs.

The box with the name Tortuga (I found it fitting and yes it’s lend from “Pirates of the caribbean” where all sorts of people are staying shortly – evil and good – like in front of a firewall). Well, it starts bad, the 80GB WD drive decided to not work very well, could install it but on the first boot it told me that there is no drive to boot from. Uh? Ok – I had a 160GB WD Caviar Blue IDE drive from my OpenSolaris project laying around unused which fits in perfectly. The next trap was the dual port Intel PCI-X card. Seems like Endian does not like it. Tried to install drivers, but it couldn’t be detected. So I gave up after an annoying hour of trying and replaced it with a DLink GBit/s card I had laying around. No DMZ for me, but that’s ok – I can open the ports I need also by hand

Endian now works fine on this machine and today I will put it into the basement and start configuring it for our network. I will also start switching to a 255.255.0.0 subnet (/16 instead of /24) to better track the IP using devices in our house & the neighbors house.

I guess I will delay the basement overhauling itself by 2 weeks as I’ve already plans for the next WE. My goal for today is: Put “Tortuga” into the rack, use the 2 FritzBoxes as modems as long as I don’t have all the needed cables and put “Blackpearl” (that’s the name of my 2nd Mac Mini) into the rack again, now that it’s equipped with a 32GB SSD and runs pretty fast). I need to get into working with Subversions and I want to start digging into this on Tuesday. That’s a fixed appointment – so the Mini HAS TO RUN that day.

Additionally, I will pick up my (in my mind) broken 1.5TB HD from the store today. Of course, the guys there are telling me that it’s ok.. hmm need to find another job for it. Could it be that only ONE drive (not the series as an identical HD from Samsung works fine in the DataTale) is incompatible? I can’t really believe that and to be honest, I’m a bit annoyed. The store is about 20mins to drive away, so I’ve spend 80mins+ on bringing it there, letting them check it for 1 1/2 weeks and today picking it up and taking it home again. Yes – for nothing.. damn! :/

Oh – and I switched my blog to automatically switch to the “natural” theme. In case you haven’t selected another theme by adding a cookie to your browser, you will notice that. I just think it’s the best designed theme for my blog.

Ciao
Dennis

Why I bought another server (I’ve just sold my DELL PowerEdge R210 a few months ago)? Well, it’s much slower than the DELL was and I bought to be our new firewall. The DELL was simply too expensive to use it for that job

I will install Endian, an extended version of IPCop onto this machine (I will name it “Tortuga”). The two onboard NICs will be used for “red1″ and “red2″. “red” is the interface name for external connection to IPCops (and Endians as it seems). Vodafone has just upgraded my DSL line from 6M to 16M and Telekom will switch me from “T-Home Entertain” to a regular 16M line, which will give me additional 2M of speed which was wasted for two years on not used IPTV.

The other the NIC ports will be used for the “green” network (regular LAN) and an “orange” one (DMZ). Endian allows me to route the traffic through both lines, so for example I will use Vodafone for Port 80, because of their much better routing to my webserver.

If you’re wondering – yes – I’ve returned the Linksys/Cisco RV042 router. It was simply not what I had in mind. For the money I’ve got back, I bought 2x 32GB OCZ SSDs for the Mac Minis which was a GREAT idea, they are now flying instead of playing the network-snails. I can highly recommend to install SSDs into Mac Minis. 32GB is not much, but as ALL stuff is stored on my Onnto DataTale-boxes, I’m fine with that.

Luckily, I’ve got the server with rails, so it will fit into my rack in the basement.

Speaking of the basement. It’s a bit messy inside (again) and I will use the weekend to clean it up and will also redo the cables behind the rack. Because I like it organized, I’ve got 5x 1U accessories for the rack – one blend with a whole, so I can route the cables through it and 4x blends I will use to give the cable mess a structure.

Hopefully I will receive the additional ordered patchcables by Saturday to start doing the overhaul. Believe it or not – the shop really had magenta cables Perfect for my Telekom DSL line!

With moving along to a new (transparent) proxy, content filtering, DNS & a great DHCP with binding MACs to IPs, I’m also changing the subnet of the LAN to 255.255.0.0.

I’m thinking of using the ranges like that:
10.0.0.x = Network devices & Servers (Firewall)
10.0.1.x = DMZ (just one Mac Mini there, no switch needed – yet, rest via VMs)
10.0.33.x = Clients
10.0.35.x = Neighbor clients

A good reason to start using a proxy with content filtering is that our neighbors are using our network via WiFi and they’ve a kid of 6 years who will start using the internet sooner or later. I just want to give them a protection and honestly, I’m also not sad to not see spam, commercial banners etc. anymore.

More about that when I’m done with the overhauling. Yeah – expect photos

What is Belldandy?

Belldandy is a name of a fictional person, used in the Manga “Oh! My goddess“, one of my favorite comics. Reading this post, you will see that I use more japanese comic names for virtual machines.

But what is YOUR Belldandy?

Well, Belldandy is the name of my newest server. I write “newest”, because over the years I tried so much different systems and types of servers, that I’m really happy to announce a very stable server here.

What kind of server is it?

It’s a self-build one, but I used hardware parts that where recommended from Constantin Gonzalez from Sun. I’m using my old but very reliable TAGAN 450W power supply, an ASUS M4A78LT-M AMD mainboard which supports ECC memory and 6GB of ECC RAM. The OS harddisk is a new 160GB IDE HD. It’s not a 19″ server – it’s a regular Midi tower (MESH series by Chieftec).

Why IDE? Isn’t this a rather old standard? What about SATA?

I’m using all 6 onboard SATA ports for the zpool. IDE may be an old standard, but it works fine for the OS itself. With 6GB of RAM I don’t think too much about HD IOs on this HD. It’s more important, that the zpools are bound to a fast bus. I simply found it a waste of an port using a SATA HD for the system.

So you’ve 6 HDs inside the server?

Yes! I’ve installed 6x 1TB WD “Green” HDs into the server. Those are connected to the zpool “storage” which is the main storage of the server.

What is that “zpool”?

zpool is pool of HDs you’ve virtually stacked into one large virtual HD. It’s widely comparable to RAID, but it offers a lot advantages compared to a traditional RAID. I’m using 3x Mirror drives. Each mirror contains 2 HDs with the capacity of 1TB each. Together, because they are internally mirrored, I’ve 50% of the capacity. If you want to learn more about zpool and ZFS, I could highly recommend googling for it – there are tons of interesting and funny material out there.

Why do you mirror? Are your HDs so bad?

No It’s a thing of security for my data. Harddisks are not made forever and the fail. The question is just: When. Having a mirror protects my data from getting lost when one or more HDs are crashing the same time. Of course, if it’s a series issue, maybe I’m also not secured against this – 3 HDs where bought at the same day, but I think it’s a very way which let’s me sleep deep.

Which OS is that? Linux?

No, I’m running OpenSolaris which is something between Linux and UNIX, I would think it contains the best of both worlds. It’s mainly based on Sun’s Solaris but with a GNome environment. It’s a great system, but I’m also rather new to it, so I need to learn a lot about it. It’s amazingly pretty easy to create storages and zpools.

How do you connect to your server from your clients?

I’ve installed netatalk on the OpenSolaris machine to be able to use it with our Macs. It’s pretty much forward. A bad idea is to use SMB and Macs, there seems to be a bug inside the Finder’s SMB implementation. Anyway – it’s cool to have AFP running (netatalk).

Of course, I could always use the SCP protocol to connect to the server if I need to, but that’s more useful for the administration than for normal usage.

Isn’t it loud? Have you stored it in your office?

It’s really too loud as I’ve installed 4 very efficient fans into the server. I had it for a couple of days in our office and it was not very comfortable, so we decided to putit into the basement. I honestly prefer a quiet office (even music sometimes annoys me and I’m working mostly in silence).

You have some external boxes on that desk – what are they for?

The bigger box is a great box – it is my Onnto DataTale. It allows me to put 4 HDs into and comes with a hardware RAID. I’ve currently installed 4x 640GB WD Blue HDs and connected it via eSATA to the server to store temporary data onto. It gives me 1.8TB of usable storage.

What are these other 4 boxes for?

That’s (from left to right) a Seagate Desktop drive, a MacPower enclosure with another 1TB Seagate HD inside and 2 Hitachi external HDs. They are all connected via a USB2-Hub to the server and are set up as an RAIDz1 backup zpool. It gives me 2.7TB of available storage for backing up our data. RAIDz1 is more or less comparable with RAID5.

How do you connect the server to the internet, or is it an offline server for local use only?

It’s connected directly to the internet and out 2 DSL lines. OpenSolaris itself is connected to the Vodafone line which offers a fix IP and NO disconnect every 24h’s, so it’s a real and cheap dedicated line. It gives me 6MBit/s download and 640KBit/s upload. So or so, I don’t run any services on the host system itself for protection. The other line is from the Deutsche Telekom and allows us to surf with 16MBit/s (1MBit/s upstream). This is mostly used for the clients only.

Do you run any virtual machine on the server?

Yes, indeed. I run 2x Debian (“Urd” & “Mila”) (one as development machine for my web design & development, the other as media streaming device (namely Firefly iTunes server or “mt-daapd)). I also run a Windows Server (“Miyuki”) on the server as mailserver. It’s a Windows Server 2008 r2 x64 with an Exchange 2010 installed on top of it. If Microsoft creates a great product, it’s in my mind Exchange. And I say that as Mac user

Which software do you use to virtualize this stuff?

Sun gives away their virtualization software VirtualBox for free – and of course it runs fantastic on their own OpenSolaris. I’m very happy that it runs so well.

On my clients we run Parallels on my Mac Pro for Windows virtualization and my wife is running a VMWare Fusion on her MacBook for Windows 7.

Do you have a firewall installed in your rack?

Yes and no, I’ve a IBM NetVista mounted inside the rack. It’s a very small and slow machine – a Thin Client – but it’s fantastic to run IPCop on. Currently, because it was a long road to get to this very satisfying server, it’s switched off, but I will turn it on again soon, and even if it’s for local proxy, free DNS servers and local DNS (don’t you guys also hate to type IPs for LAN servers? ).

How much HD capacity/usable capacity have you attached to Belldandy?

- 6x 1 TB internal
- 1x 160 GB internal
- 4x 640 GB external
- 4x 1 TB external

Makes around about 12 TB of storage, I can use up to 7,8 TB for real storage. There’s a lot of overhead for security, but I think it’s really worth it. ZFS as filesystem itself is very secure, but in this combination, it’s fantastic and I’m really looking forward to have a permanent server. In case I want to try something else, I will now simply create a new VM to play with instead of killing the server itself.

Do you run Linux anywhere?

I do! “Obelix” – my dedicated webserver runs Debian Lenny, the 2 virtual machines are also both Debian and IPCop is also based on Linux. I would say, I’m using a good mix of the different operating systems.

Many thanks for reading this post! Hope you enjoyed it as much as I enjoyed typing it

Ciao
Dennis