Information Request II (by sarah)

A little while ago I requested information on cooking and recipes this time it is of a more technical nature - infact what I would love very much is information on the history of computing, electronics or anything loosely related to them 🙂

I want to make Web-Empires blog and glossary a proper useful resource for people to use plus it is sort of related to a novel I am writing!

I hope this isn't me being too cheeky!

If any one can recomend some books I could read as well that would be much appreciated 🙂

Backup system progress (by alaric)

I mentioned an intent to build a backup system this Christmas. Well, I've not had two days to work on it yet; I've just had an hour and a half one evening, in which I've written the local-filesystem backend...

But last night I had a dream about it. In particular, I dreampt of visiting an old friend I've neglected to keep in touch with lately; and at her house she had a rather snazzy little mini-laptop, like an EeePC or something, which had an eSATA connector into which she plugged some humungous (in terms of gibibytes; it was the size of a double CD case, physically) external hard disk upon which she had a filesystem that seemed to work very much like Fossil/Venti, the original inspiration for my backup system - or perhaps like a giant personal Git repository.

In particular, one feature that cropped up in the dream was that the filesystem had a magic file on it which was an RSS feed of recent changes to files.

Which got me thinking about features for version 2 of my backup system (if I get to finish version 1, that is!). My focus is on offline use, for batched backups, but the filesystem in my dream was being used online.

For a start, we could have a log of changes to tags, as in my system tags are the "roots" of the archive system. The creation of new tags, the deletion of tags, or the updating of a tag to point to a new snapshot would all be logged. This could then be used to create the RSS feed.

Secondly, it shouldn't be too hard to write a FUSE interface to the thing for read-only access, presenting a root directory containing an RSS file generated from the log, along with a directory for each tag, which in turn contains a current subdirectory containing the current value of the tag as well as dated subdirectories containing all the past contents of the tag, in ISO date format so they sort correctly. And perhaps an RSS file just listing the history of that tag.

But then the next cool thing would be to allow write access, by using a local disk directory as a staging area, so the current subdirectory can be written to (with the changes being spooled to the local disk). Then when a commit command is given, those changes are merged into current. Which would require a new underlying operation to merge changes in, rather than taking a new snapshot; the difference being that any files or directories missing in the directory tree being snapshotted are 'inherited' from a previous snapshot already in the system, with some mechanism to reflect deletions.

Anyway, unrelated to the dream, it also occurred to me that it'll be neat to support replicated archives; my implementation of the backend architecture will make it easy to take a set of backend instances and merge them into one, with every write going to all of them and reads serviced from the first one that succeeds. It'll also be easy to support staged archives, where a number of read-only backends are checked for existing blocks, but all new blocks go to a nominated writable backend, with another backend adapter. That will allow for generational backup systems, where a local disk is filled up with backups until it reaches a size limit, whereupon its contents are shipped off to a DVD writer (keeping a local cache of what blocks where on there, so the DVD need not be put in unless the contents of blocks are actually needed).

But, all idle speculation for now. I still need another day and three quarters to implement the core of the thing...

Design patterns (by alaric)

I'd thought I'd had this little rant before, but as I went to find the post I'd had it in to link to it from a big ranty post I'm working on, I couldn't find it.

So here it is.

My opinion of design patterns.

Design patterns are common structures that appear in computer programs, but that can't themselves be abstracted into a reusable component.

That's fair enough. I have the "gang of four" book of design patterns, that started the craze, and I think they're all good design patterns in mainstream OO languages; things that can't be actually encapsulated as a class or whatever, but that keep cropping up.

However, the problem is that the patterns community has somehow ended up with some dogmatic folks in it. Who take phrases such as "Design patterns are common structures that appear in computer programs, but that can't themselves be abstracted into a reusable component" - and then attack anybody who claims to have encapsulated a design pattern into a reusable component.

Because design patterns are really only valid in relation to a set of languages. For example, in assembly language (or even traditional BASIC with GOSUB), procedure calls are a design pattern. There is no language construct for a procedure; instead, you have a GOSUB or CALL to a given address, that explicitly saves the calling location somewhere and then jumps, and a RETURN to go back to that saved location. The GOSUB...RETURN pairing is put in there by the programmer; it's a design pattern.

But then higher-level languages have explicit procedure abstractions, and all the fun they entail.

Even object orientation is a design pattern in languages like C. "Put function pointers in a struct, and call them with the struct pointer passed as the first argument". Or including a pointer to a struct of function pointers in every data struct (and call it a virtual method table). But go to Java and that's an explicit language abstraction facility.

The gang-of-four book is taken as the authoritative book of common design patterns (although it is, in my opinion, a book of design patterns for mainstream OO languages). And design patterns are things that cannot be expressed as native abstractions in code. THEREFORE, fools think, it's impossible to write a design pattern in code. And anybody who claims to have done so needs shouting at.

Sadly, as demonstrated in Peter Norvig's presentation on design patterns in dynamic languages, more advanced languages can express abstractions that others can't, and there are plenty of more advanced languages than C++ and Java.

In fact, I think that the appearance of design patterns in a language is probably a hint that the language needs to be more powerful - and suggests how to make it so. What sort of abstraction would make it possible to encode your design patterns?

And so, as I read "Modern C++ Design" by Andrei Alexandrescu (a book on doing good things with C++ templates), I come across the following phrase:

Eventually, Andrei turned his attention to the development of template-based implementations of popular language idioms and design patterns, especially the GoF [gang of four] patterns. This led to a brief skirmish within the Patterns community, because one of their fundamental tenets is that patterns cannot be represented in code. Once it became clear that Andrei was automating the generation of pattern implementations rather than trying to encode patterns themselves, the objection was removed...

Bleargh. It sucks when people get all religious about the interpretation of a given piece of "scripture". Even when people manage to work around it by using illogical arguments ("No, no, it's not a representation of the pattern in code, it's a bit of code that automates the creation of implementations of the pattern..."). Sadly, in other realms, this kind of thing can lead to people being burnt at the stake.

How to use a Maplin USB microscope on a Mac (by alaric)

We received a USB microscope from Maplin for Christmas, from my father and stepmother. It's basically a webcam engine coupled to a different optical system; so there's a small hand-held device on a cable that plugs straight into a USB port. It has inbuilt LED lights, so you push the end right up against a surface, and the casing of the unit holds the surface at the correct focussing distance while illuminating it. There's a manual control to focus, but once it's set, you don't need to fiddle with it.

It's Maplin code N87FX.

Anyway, it claims to work under various flavours of Windows, and certainly doesn't work out of the box on a Mac - but a bit of research (see the USB vendor/device codes in System Profiler, looking them up at http://www.linux-usb.org/usb.ids to find the generic name of the device, googling from there) led me to http://www.pc210.com.cn/downloadhtml/sonix201_PC_Camera_driver_for_MAC_405.html where a suitable driver can be downloaded. The RAR file wouldn't expand under Stuffit, but unrar from pkgsrc on my NetBSD system opened it fine, to reveal two zip files (duh) each containing a .dmg file (duh) which had a different driver in each... I just installed both.

Skype then immediately picked up on it as a webcam. But Photo Booth refuses to talk to anything other than the inbuilt iSight on my laptop, but then I found that the driver installs had also installed an app called Webcam Monitor, which is a simple wrapper around the underlying Quicktime video capture APIs, adapted from Apple's reference source code. It can record video, or a simple ⌘-C will copy a picture to the clipboard for Preview.app to save. It'd be nice to find something where I could just press a capture button and have a snapshot appear in a list which I can then choose a name for and save as a .PNG to automate this, though. Perhaps I ought to do my own quick adaption of Apple's reference source code...

Observe, my stubble!

A stressful upgrade – and a design for a better backup system (by alaric)

My server cluster has been having lots of outages lately - which I traced down, after much experimentation, to probably being the ethernet driver in NetBSD 3.1 (which I was running), since the machine in question (as luck would have it, the NFS/NIS server; mental note, make the other server be a NIS slave so it can run on its own...) seemed to just disappear from the network but be perfectly happy if spoken to over a serial console - but ifconfig wm0 down ; ifconfig wm0 up would then hang it.

So, since a machine with the same Ethernet interface but running NetBSD 4.0 was running fine, and I could see there'd been a lot of commits to the driver between the versions, I decided to upgrade it.

Easy enough, right? Stick in the NetBSD 4 boot CD, boot, select Upgrade. But then problems struck; my /usr and /var are on a RAIDframe mirror set, and the install kernel didn't have RAIDframe installed. So I just let it install into /usr and /var directories under the root directory, and then booted into the new system, mounted the RAID, and copied the new /usr over my old one in order to upgrade all the binaries while leaving /usr/pkg and /usr/local untouched.

Read more »