Building a 1U Server, Part 3

Posted by John
on Monday, 17 March 2008

server

System Config

While the final bits arrive for my new server project I might as well put pen to paper on how the server's O/S will be laid out.

The operating system I'm rooting for this time around will be Linux, purely for the flexibility and the stability; the hardware I'll have to run thru it's paces with a CPU testing tool to make sure i've not got a faulty chip along with some brute-force testing on the memory chips, it's going in co-location so I might as well do it as I don't want to drive down to the site everytime it falls over.

I'll be using SSH obviously to connect to it along with a strong keychain to make my logging in easier and the connection encrypted.

I've got a strong iptables config so will put that into play with a custom port for SSH, not the usual port 22 which is standard; the more things you put in the way of potential hackers the better.

And no, I won't be logging in as root, only if I absolutely have to.

System Layout

This is the tricky bit, working out how you're going to lay the system out and run the various services; there really isn't any good answer, no one solution will fit everything.

For me I've decided to use XEN virtualisation to segregate my services out so each one remains separate, not all my eggs in one basket.

So I plan to use XEN and ArchLinux to handle each server stack, e.g.

  • Database Server (MySQL / PostGreSql / Sqlite)
  • PHP Server (Lighttpd + PHP5)
  • Rails Server (Nginx, Thin, Ruby on Rails 2.0)
  • Caching Server (Memcache, or some kind of custom Content Delivery Network)

I haven't nailed down the particular distro of Linux for the core o/s but will be using ArchLinux to handle the 'baby' boxes, the controlling 'big-daddy' may either be Fedora or again ArchLinux.

Now this may sound like overkill, but it does make sense, as it'll allows me to custom-build each Server based on it's proposed use and with ArchLinux I can make sure no extra baggage is added with unneccesary features and services; tuning it to the maximum performance.

It also allows me to do image backups in the future of each server so I can take snapshots of each one, in case I need to restore any at a certain point in time (say a bad update).

ArchLinux?

I've grown to enjoy ArchLinux, mainly for it's neatness and light feature stack, but also because it has no set release number. Unlike other distro's there is no frozen release 7 or 8, you go with the latest core files and that's what you've got; taking the pressure out of always being on-top.

Also the package manager is a little more powerful and because you only install what you need you don't end up with tons of extra software you never even use. Rolling in at a 160mb ISO build compared to 4.4GB for your average Ubuntu or Fedora you know you're only getting the bare-bones.

As Judd Vinet, the founder of the Arch Linux project said: "It (Arch Linux) is what you make it."

Don't get me wrong, I'm not trying to bash any of the other distro's but as you can imagine with this particular project I have to keep things lean so I can maximise power and alleviate future problems when trying to identify awol services.

Final Notes

It's going to be a lot of work and probably a pain in the rear to put together but I've done this long enough with SliceHost and Virtual Hosting so I think I've cut my teeth long enough to do this for real.

Keep you posted how the build goes.

Building a 1U Server, Part 2

Posted by John
on Sunday, 02 March 2008

After a night-long search for parts and voltage configurations I've wound up with a pretty nice shopping list.

Discussion

Started a little discussion over in my favourite Ruby Forum about the subject, really impressed by what some of the guys have offered.

But Power?

Yep that's an issue, the whole 0.5A issue is primarily down to the colo provider. Normal servers run at 1.5 to 2.0A power usage and cost a fair bit to run, however if you can build a server within the 0.5A limit then the cost is less and hence the provider i've chosen charges less.

The trick is building a box which gives you a decent amount of power within the limit, thankfully though with things the way they are there's suddenly a lot of low power options available which still give you a lot of bang for your buck; so the challenge shouldn't be that big a deal.

How Much Power?

So 0.5A is 120 watts (aka 120 volts). Problem is most PSU's don't run at full capacity, so you're probably dealing with 85 watts in all; so be careful what you choose, over-estimating each part to be sure.

If it says only 45w, mark it down as 50w just to give yourself some room.

Ok, What you Buying?

Here's the shopping list...

CPU: AMD A64 3600 X2 AM2
cost: 45w

After tracking down specs from various sources I've found this processor only sucks around 45w of power at full steam. It's judged by it's TDP, Thermal Design Power; the maximum cost the cpu would use running serious apps. The new AM2 socket processors run at a lower power so a good choice to keep the consumption down. The AMD chips also include Cool-n-Quiet which is an essential part of this plan, something that shouldn't be ignored as it keeps the processor running cooler and with lesser power drain under severe conditions.

BOARD: ASUS M2A-VM (HDMI)
cost: unknown (reckon 10w)

Don't know about the connector, may have to get an HDMI > VGA adapter to plug it to my LCD but this board is rated pretty good. It's got an onboard graphics card (ATI), and enough slots for the memory sticks and SATA drives. The nVidia chipsets draw more power so avoid those.

HDD: WESTERN DIGITAL SCORPIO 320GB SATA
cost: 2.5w each (5w total)

These baby's are actually tiny laptop hard drives from Western Digital, my prefered HDD supplier, and run at a lower voltage to be really hit the mark. Normal drives will soak up 8-9w of power and make more heat. These will be setup in a RAID 1 (mirrored) configuration, giving me a dynamic backup copy of the O/S + Files; if one goes down the other takes over and being small they shouldn't heat up too much.

MEMORY: CORSAIR XMS2 DDR2 2GB PC2-6400 x2
cost: 1.2w each (2.4w total)

I haven't had much problem with Corsair sticks in the past, they've usually been pretty reliable so these should fit the bill well. They're low-power chips so shouldn't draw that much power and run fast giving less of a chance of bottlenecking the system at it's most critical point, matched and they should run smoothly.

HEATSINK: Dynatron A48G
cost: 8.4w

This little buddy can easily cool the cpu down to optimum temperatures and blow air thru the side, specifically for 1U cases as we don't have much height to play with; a 1U case only being 1.75" high from the board.

In total the parts i've chosen should total...

2.4w + 5w + 10w + 45w + 8.4 = 70.8w

Nice and with my headroom of Max 120w and Min 85w sounds about right, factor in a couple of fans soaking up 1.5watts each and I should be within the grounds.

Choosing laptop hard-drives also reduces the heat dissipated, as the hotter things run the more power they soak up; especially true for the CPU as at 45'c it'll soak 40w and 60-70 it'll soak 60+w.

Still hunting for the perfect fit, sometimes this really feels like Apollo 13 territory,

Technician: How much power have we got to work with?
John Aaron: Barely enough to run this coffee pot for nine hours

Building a 1U Server

Posted by John
on Saturday, 01 March 2008

Shocked my boss yesterday after telling him i'm building my own rackmount server, well i've built machines for about 20 years and most of those have been custom jobs with 3 running Ubuntu Server for my Rails projects so doing this is the next logical step.

Here's the things I've dug up, which should help you out if you ever decide to go down the same route.

Co-location + Power Limit

For this I've found a company in Manchester (uk) who will host it for me at £32 a-month as long as the power usage doesn't spike over 0.5A

So with a few calculations, that gives me...

0.5a => 120 watts

The company is called RapidSwitch

Case

Next up the case and I've gone for a SuperMicro 1U case, problem is I need specific boards to slide in and so this...

M2N-MX SE Plus

But still looking around for the perfect fit.

Processor

This will probably be an AMD 64 dual core CPU, as they're power efficient, reliable and the O/S i'm choosing (Fedora 8) fits rather well with them.

PSU

For this I'm sourcing the most power efficient model as most PSU's only run at 50-60% operating efficiency.

CPU Heatsink

ThermalTake have a good array at...

Not sure about the availability but your 1U case is 1.75" high so it's a big factor.

HDD

I'm picking 2x 500gb Western Digital Caviar drives, I've always used them, they're not special but reliable.

This'll go in a RAID 1 configuration similar to my NAS box at home, doubling the HDD read speed but lowering the write speed due to the mirroring, reliable though.

Memory

2 sticks of 2gb memory, I'd choose Samsung as they're cheap and just work but having trouble sourcing the OEM sticks, may just go with some corsair sticks.

Power Calculator

Found a power calculator which should give me a judge on how much power I'm gonna be using,

Probably not that accurate but gives me a ballpark figure, with the hardware I've chosen:

  • AMD Athlon 64 X2 5000+ 2600 CPU AM2
  • 2x WD SATA Drives
  • 2x DDR2 2GB Sticks
  • (graphics card on-chipset)
  • 1x 80mm fan, 2x 120mm fans

All comes up to needing a PSU of 177 watts, which fits in with my Sparkle low-power 220w PSU but might be a might too high for the power coming in.

Finally

It's a learning process and one that'll probably not be as easy as it looks but worth the effort for the experience.