Towards the Family Mainframe (by alaric)

Last September, I posted progress on the construction of our domestic mainframe. To recap, the intent is to build a dedicated home server that's as awesome as possible - meaning it's reliable, safe, and easy to maintain. That rules out "desktop tower PC in a cupboard" (accumulates dust bunnies, gets too hot, easily stolen, prone to children poking it); "put a 19" rack somewhere in your house" is better, but consumes a lot of floor footprint and doesn't fix the dust bunny problem. So I've made my own custom steel chassis; fed cold air at pressure via a filter, incorporating a dedicated battery backup system, locked and anchored to the wall, and with lots of room inside for expansion and maintenance.

Since that blog post, I've finished the metalwork, painted it with automotive paint using a spray gun (which was a massive job in itself!), fixed it to the wall, and fitted nearly all of the electronics into it.

A significant delay was caused by the motherboard not working. I sent it back to the shop, and they said it was fine; so I sent the CPU back, and they said THAT was fine; so I sent both back together and it turned out that the two of them weren't compatible in some way that was solved by the motherboard manufacturer re-flashing my BIOS. That's now up and running; I was able to use the HDMI and USB ports on the outside of the chassis to connect up and install NetBSD from a USB stick, then connected it to the network and installed Xen so I can run all my services in virtual machines. It's now running fine and everything else can be done via SSH, but the HDMI and USB ports are there so I can do console administration in future without having to open the case (unless I need to press the reset button, which is inside).

The one thing it's lacking is the management microprocessor. I've prototype this thing on a breadboard and written the software, but need to finish off the PCB and cabling: but it will have an AVR controlling three 10mm RGB LEDs on the front panel, and three temperature/humidity sensors in the inlet and outlet air (and one spare for more advanced air management in future). But the idea is that the three LEDs on the front panel will display useful system status, and the environment sensor data will be logged.

Here's what it looks like from the outside; note the air inlet hose at the top left:

The socket panel on the left hand side worked out pretty well - 240v inlet at the bottom, then on the aluminium panel, three Ethernets, HDMI, and USB (my console cable is still plugged into the HDMI and USB in the photo, which won't usually be the case):

And here's the inside, with lots of space for more disks or other extra hardware; the big black box at the bottom is the battery backup system:

Now I have Xen installed, I'm working on a means of building VMs from scripts, so any VM's disk image can be rebuilt on demand. This will make it easy for me to upgrade; any data that needs keeping will be mounted from a separate disk partition, so the boot disk images of the VMs themselves are "disposable" and entirely created by the script (the one slightly tricky thing being the password file in /etc/). This will make upgrades safe and easy - I can tinker with a build script for a new version of a VM, testing it out and destroying the VMs when I'm done, and then when it's good, remount the live data partition onto it and then point the relevant IP address at it. If the upgrade goes bad, I can roll it back by resurrecting the old VM, which I'll only delete when I'm happy with its replacement. This is the kind of thing NixOS does; but that's for Linux rather than NetBSD, so I'm rolling my own that's a little more basic (in that it builds entire VM filesystems from a script, rather than individual packages, with all the complexities of coupling them together nicely).

I'm using NetBSD's excellent logical volume manager to make it easy to manage those partitions across the four disks. There are two volume groups, each containing two physical disks, so I can arrange for important data to be mirrored across different physical disks (not in the RAID sense, which the LVM can do for me, but in the sense of having a live nightly snapshot of things on separate disks, ready to be hot-swapped in if required). I still have SATA ports and physical bays free for more disks, and the LVM will allow me to add them to the volume groups as required, so I can expand the disk space without major downtime.

So for now it's just a matter of making VMs and migrating existing services onto them, then I can take down the noisy, struggling, cranky old servers in the lounge! This project has been a lot of work - but when I ssh into it from inside the house (over the cabling I put in between the house and the workshop) and see all that disk space free in the LVM and all the RAM waiting to be assigned to domU VMs that I can migrate my current services to, it's all worth it!

Shakespearian Mock Tales (by sarah)

Last night we had the second ever Mock Tales - 2 hours of stories and writing creativeness with sticky non-alcoholic cocktails. They were Shakespeare themed as it was his 400th birthday at the weekend 🙂

I did 20+ pages of my comic book script/story board and now know how this story arc ends, Alaric managed 5000 words of our joint novel/series.

There were 3 of us so I was limited as to how many drinks I could make!

Above are Twelfth Night which was minty, Midsummer Nights Dream which was vanillary, The Scottish Drink which was fruity, and The Tempest which was Sugary.

We also had home made Pizza which was dairy and gluten free - Mary had opinions...

I remembered that when I was in secondary school I became obsessed with The Tempest as a story and drew the whole thing as a comic - I wonder if it survives somewhere? I loved the stop motion animations they did of the shakespeare stories I need to try and find them for the girls.

Jean tried a Tempest before she went off to best - she was a little horrified at it's stickiness - she still drank/ate it.

Recipes are being written down for Salaric Cooking before people start prodding again 🙂

The family mainframe (by alaric)

I'm in the process of consolidation the home fileserver and the public Internet server - currently two separate bits of hardware - into a single physical device, virtualised to support multiple indepedent machine images. Having a single family mainframe will simplify the management of the complex web of computers and services that support our digital life.

For various reasons, the best place to build such a thing is at the office end of my workshop. Even though it's at the "clean" end, this is still a room that is prone to having fine conductive dust in the air, varying humidity and temperature, and (heaven forbid) a leaking roof. Also, I want a case with extensive room for upgrades, and which makes it easy to replace parts. Having used 1U rack-mounting servers for quite some time, I am sick of highly compact servers that are difficult to work with, requiring extensive dismantling to get to parts.

Clearly, I needed a rather special chassis for this new family mainframe, so I bought a load of steel, picked up my tools, and got to work. I've been working on this for months; I initially cut up the metal at home, then visited a friend's workshop to borrow his pneumatic rivet gun and his MIG welder. Since obtaining my own TIG welder, I've been able to continue at home.

The chassis is nearly structurally complete; this weekend, I've been attaching mounting brackets inside it for everything to attach. All that remains is to finish welding the upper panel on, then the whole thing can be cleaned and galvanised, and the exterior painted. Then I can fix it to the wall and start fitting the electronics and electrical systems!

The first thing I did this weekend was to fit mounting brackets for the processor frame. This is taken from a standard ATX case, and is the base plate with standoffs to mount the motherboard, the frame to attach expansion cards to, and the frame to hold the PSU. This is screwed into the chassis, so that I can use an existing frame (rather than having to make one myself), and so I can replace it if needed. The frame is held in place by two locating pins that fit into holes in it, and then two screws through the upper-left corner (I drilled and tapped holes in the top left bracket), and a little spacer at the top right to stop it from flexing:

With the frame in place, it looks like this:

Next came the expansion frames. I may need to add additional hardware inside the chassis in future, but once it's holding a running server and painted and bolted to the wall, I can't really take it down to weld additional brackets into. So I cut off one-inch lengths of square tube, drilled and tapped a hole in the centre of one side, and welded them to the inside of the chassis. I drilled holes in the ends of strips of steel, so they screw into the pairs of brackets, creating a metal strap that can be removed, things mounted onto (via welding etc), and then screwed back into place, without causing major disruption. There are two - one beneath the process frame, above where the UPS will go; and another right at the top, above the environment management system.

Here's the upper one:

And here's the lower one:

The welds were quite difficult, as I had to reach right down into a corner of the chassis. As such, they were either OK or awful, depending on whether I had to use my right (dominant) or left hand:

I also cut and drilled some mounting flanges, which will be what are used to bolt it to the wall:

When I made the sides of the chassis, I welded angle iron onto them, in order to attach said flanges:

(Note the plasma-cut hole, which will be where a removable plate with sockets for Ethernet, VGA, and USB will go).

The mounting flanges are quite thick (the wall is rough and bumpy, so the chassis needs to be spaced slightly from it), so it was good fun welding them to the much thinner angle iron. I think I did an OK job:

Then I mounted the internal frame for mass storage devices, which goes above the processor frame, below the environment management system. It's a metal plate drilled for lots and lots of 3.5" disk drives, which attaches (with screws) to brackets I welded into place:

With all the internal stuff done, I started to weld the top panel in place, which I'd avoided in order to enable me to get access into the top:

Annoyingly, I ran out of argon while doing the tack welds. A TIG welder without shielding gas is a lot like a plasma cutter, and I burnt a nice hole in a shower of sparks. It's only a small hole, so I'll be able to weld over it when I finish the job off.

Unable to do the final welding, I drilled a hole in the eaves, where clean outside air will be drawn in through a duct into the environment management system:

I also hefted the entire thing up to the wall where it will be mounted, propped it in position, leveled it, and drilled through the holes in the flanges to make the holes that will be used to anchor-bolt it in position:

Workshop progress (by alaric)

Setting up my workshop has been a long battle, thanks to the roof leaking (and everything getting covered in slime mold because of the damp, and ivy growing in through the holes in the roof and shedding sticky sap and dead leaves on everything, and a cat getting in there and pooing on the floor...). Things got rusty, everything got grimy, and stuff was moved around willy-nilly to get it out of the water; and then because everything was in the wrong places, stuff couldn't be put away properly, and so things ended up piled wherever they could go. This was quite distressing for me; by nature, I'm a person who makes things, but I've not been able to do anywhere near as much making as I'd like for years, because I didn't have a nice place to work and didn't have easy access to my equipment.

But, now the roof is fixed, and everything's had about a year to dry out. I've had time to continue painting the floor (it's all now covered in paint, but some parts need a second coat); I've cleaned and tidied;0 and thrown away wood and metal that was too rusted to be of any use, and scraped up the remains of cardboard boxes that had dissolved into fungus, and found and removed all the cat turds and scrubbed the floor with disinfectant; thrown away computer parts that were covered in disturbing fungal blooms; and cleaned and tidied the computer desk. I've made racks to store the wood and metal stocks that are still usable, and put down linoleum under the computer desk so the chair can roll around freely (it didn't do too well on the rough concrete) and I have a nice surface to rest my feet on.

I put up a bracket with hooks for my boiler suit and lab coats, by the door, as the place they used to hang now interferes with the material racks:

I've got the compressor installed under the workbench, rather than kicking around the floor:

And my computer desk is all set up nicely; you can't really see it here, but there's a desktop PC, with a nice set of speakers and an amplifier so I can listen to music - or the audio from my wide-band scanner, seen to the right of the monitor, which picks up the FM broadcast band nicely:

What's next? I still need to assemble some shelving, as there's still stacks of flimsy plastic crates holding a lot of stuff. And put a second coat of paint on some bits of the floor (which will be easy to get to when the crates are gone). And I've got a metal garden waste incinerator I was going to turn into a furnace (but which is far too large, and now I've helped somebody else build a furnace that he lets me use), which I need to find a new home for. And I need to find a place to store the festival trolley, which currently kicks around on the workshop floor (getting in the way and offering plenty of shin-scraping opportunity). And I need to finish the meter-and-a-quarter-high heavy steel server chassis standing in the middle of the room (which will be fixed to the wall next to my computer desk when it's finished).

I want to make a new welding bench, too - my current one is curved, as (in the wildness of youth) I tried to put far more welds between the top surface and the frame than was needed, causing it to warp. This means it's very hard to make flat things, as they don't lie flat when I'm lining them up to weld. And, as it stands on four rather thin angle-steel legs, on a rough concrete floor, it wobbles, so isn't much use to mount a vice on. Speaking of vices, while clearing up in the workshop, I came across this beast:

It's a "leg vice"; the long metal leg should be embedded into the ground to steady it. It's a blacksmith's tool, intended to hold something while it's battered with a hammer; thus the exceptionally sturdy construction. If I bolted it to my current bench, then the bench would fall over if I tried to use it in earnest. We found it in the stable where we lived before, which is the ancestral home of one side of my family; I was given it as I thought I could try and get it working, although I was expecting a long task ahead of me to rebuild seized parts. Thankfully, the screw thread was in perfect condition, and penetrating oil and elbow grease got the joint un-seized, and it's now working nicely. It still needs some rust removal to make it more pleasant to touch and to avoid contaminating everything I clamp with rust, but that won't take long.

I costed up materials to make one with a more rigid frame, bolted to the wall at the back and with two legs at the front so that it wouldn't wobble (and, with the wisdom of experience, only stitch-welding it to the frame, as that'll provide more than enough strength without curving it into a bow); making that will cost a little over sixty pounds (plus welding consumables and electricity). The design includes a mounting point for the leg vice, so that it protrudes out into the room (with the top of the vice level with the top of the bench, so it's not in the way of large things going on the bench), and converting the old bench into a shelf under the new bench to provide much-needed handy storage for grinding tools and welding clamps.

However, having just spent a bunch of money on welding gear I'm not going to be in a position to splash out sixty quid on a welding bench for another month or so (let alone buying metal for other metalwork projects needed around the house, such as a set of railings for the front of the house...). Thankfully, we've found a local scrap metal merchant who are willing to let us rummage around for cheap metal! When we get a chance, I'm going to head over there and see if they have any steel plate for the top and box/angle section for the frame and legs... It'll be nice to have a good welding project to focus on with my new TIG welder!

Geo Bake Off – Geologist Despair (by sarah)

I mentioned the Geological Society's Bake Off to Jean - this is the result - she's been planning it for weeks!

The girls are seriously proud of this 7 cake monstrocity.

They have certainly enjoyed eating it 🙂

There is a lot of hidden stuff that went into this cake.

One of the themes was mud which is why there is chocolate orange mud flows 🙂

But there were all sorts of challenges and Jeany decided she wanted to try and complete as many as possible.

So within the river valley there is structure for a cross section.

And then she just got plan creative - with the structure of the cake and I believe some youtube research.

These are the marzi-bones fossil human ancestors or related species buried in a cash by volcanic ash - they may or may not have already been dead when this happened some more excavation will have to occur to find out!

The top layer of the Mud Tower is a chocolate gravel lens between a sandy mud and a volcanic ash.

You can see the colour difference really well in this photo.

Here is Jean cutting open mud tower to reveal whats within.

Spoiler... the chocolate gravel lense.

Here's the river valley with birds foot delta - at this stage the volcano is dormant or extinct.

This is the main part of the cake with Mud Tower and the ammonite loaf as zoomed in bits and the past hidden behind the lush "hill".

Of course there is a hidden volcano and... erm Jurassic Park toilet death scene...

The geologist hammer was another challenge - but being Jean it is a geo-thor hammer so is the wrong shape (to be honest she sneaked a time travel train into it so I was amazed there was no tardis). I did the writing.

Within there is an ammonite - this one was completely and utterly Jean's own idea and it worked and she is soooo happy she is taking it into school tomorrow 🙂

This was the tense moment of cutting in and finding out if the idea had worked. It's a bit flatter than intended but we agreed it's had metamorphic stuff happen to it thanks to the volcanos proximity.

The cake did kind of over flow but that's not surprising - here's how it was made...

Did I mention that she called this cake collective - Geologist Despair.

Volcano before lava.

She did try to put structure inside the volcano but it didn't work that well.

The volcano was fun to put together - she remembered Dino-Mountian I'd made her for her 5th? Birthday 🙂

How the river valley was put together...

One time travel train and it's in a tunnel - the tunnel was the challenge 🙂

And before the tunnel, infact she did a lot of icing moderling for this.

Of course Mary pulled her weight too 🙂 Mainly with rolling out icing and smearing chocolate everywhere!

She did most of the Mud Tower by herself 🙂

Mary put chocolate gravel leaking out of an erroded side and some other bits including sticking out marzi-bones 🙂

Mud cracks were a challenge - Jean went with the existing cake cracks and made the lonely tree which was another of the challenges.

Lonely tree... did I mention the lonely tree?

Other general cakey making pics...

Creating the Marzi-Bones...

I really love this idea 🙂

Creating T-Rex...

This has been EPIC - it took 3 days to make the cakes - Alaric is taking Mud Tower into work tomorrow etc... Both girls have enjoyed it so much and of course we used home grown eggs. The cakes themselves range from chocolate orange to mint to vanilla and strawberry in flavour. There are three icings and marzipan involved and some of the cake is me friendly ie gluten free (the volcano) and some is Mary friendly and so on.

Jean was a little sad as she had meant to put Mary Anning in and a geological map too but she forgot and just don't ask her about how atomically correct her loo death scene is ok.