Web server upgrade (by alaric)

Whew. On Monday I upgraded some of the software on my primary web server, since it was running some old stuff with security holes in.

Annoyingly, the www/apache2 package in NetBSD seemed to now conflict with devel/subversion-base since apache 2 required devel/apr0 while devel/subversion-base required devel/apr and they were conflicting packages. So, I had to upgrade to www/apache22. Fair enough.

One recompile later, and I start apache, and start checking out different web applications I host to see if they all still work...

...and my browser times out. Hmm, OK. I go to an open ssh window to look at the log files, and it's frozen.

I quickly check the network hasn't failed, then resign myself to the fact that my server has just dropped off of the net. It won't even ping, and I can't reach any of the services it forwards in to the backend server either, so the network stack is totally down.

So that evening I head down to the datacentre and take a look... to find that it's died handling the exit() syscall from Apache. Apparently an assertion failure inside knote_destroy or something.

Reboot. Start Apache. Start taking a look at sites.

Kerboom! It dies again in the same way.

Hmmm... Clearly, my three year old NetBSD 2.0 kernel is none too happy with Apache 2.2. It looks like Apache's doing something that triggers a bug in the kernel; knotes are event notification things, so I bet Apache's doing some kind of asynch I/O, and triggering a bug in the kernel code that implements it, causing it to leave the knote state of the process in an invalid state, so that the kernel panics when trying to close down the process state after process termination.

So I reboot it again, stop Apache starting, and leave it at that for the time being. No web service, but everything else works.

Then this evening (the day after), I returned, now with a shiny NetBSD 4.0 install CD in hand. Nervously I backed up some critical directories, then bit the bullet and did an upgrade.

And, to my delight, it was nearly seamless. The NetBSD installer upgraded and rebooted into a nearly perfectly working system. All my existing software, compiled under 2.0, ran fine under 4.0's 2.0 emulation, with the mysterious exception of net/bind9, which wouldn't start. A quick cd /usr/pkgsrc/net/bind9; make install later, and it was starting fine. Even Apache worked without hosing the system!

I had to compile a custom kernel with routing enabled, to allow the NAT that the server provides between the single public IP of the love.warhead.org.uk cluster and the backend server infatuation; then a quick reboot and that was working too.

All in all a successful mission, and it only took an hour or two. I still need to recompile all of my packages, but only to avoid the risk of there being a problem in the 2.0 emulation. While I was there I recompiled bash and sudo, just because it's nice to be able to rely on them.

Error correcting codes (by alaric)

I've just finished reading A Commonsense Approach to the Theory of Error-Correcting Codes. The book does exactly what it says on the tin; it explains error correcting codes in terms of linear modulo-2 algebra, only getting into Galois fields and all that in an appendix.

And, as such, it does little more than scratch the surface. It only goes into Reed-Solomon codes that can correct single word errors, for example. But hey, I'm not complaining - it's done a great job of giving me an intuitive understanding of Hamming codes, cyclic codes (such as CRCs), the single-word-correcting RS codes, and so on. And I've learnt a lot about Linear feedback shift registers.

But it strikes me that the whole field of error correcting code is a bit insular. The maths required to really grasp it are really complex. Finite fields are bizarre things. While lots of people can experiment with things like data compression or basic encryption, error correction codes, the third cornerstone of low-level coding technology, is quite inscrutable.

This sucks.

Read more »

Hyper = non-linear (by sarah)

I have decided that html which conforms to those standard thingies is the equivolent to the plasmids or little circular bits of DNA that bacteria and microorangisms in general expell which can be taken up and incorporated into other organisms either of the some or different species, phylum and some times even kingdoms.

I'm getting a little confused with URLs here and feel that they should somehow fit into this analogy but that might involve re-reading the chapter I've just read again and I dont really want to do that - oh the folly of reading a book on HTML and CSS straight after finishing on on early life and the evolution of life. I think I might have been thinking about links as well and exactly how they are hyper - websites etc... are non-linear and organisms can share DNA across Kingdoms and even take it up from things that have died leading to non-linear evolution - erm.... my heads hurting now.

This post is complete waffle please ignore it.

Code-stealing macros (by alaric)

I'm reading Thinking Forth, which has conveniently been made available as a PDF for free reading, and a reference therein to the ability of languages with metaprogramming facilities to represent things that, in other languages, might need to be part of the language core as user libraries:

Nevertheless a brilliantly simple technique for adding Modula-type modules to Forth has been implemented, in only three lines of code, by Dewey Val Shorre

Read more »

Intermediate languages (by alaric)

One way of making a compiler portable to different host platforms is to make it compile to a processor-independent (or mainly processor independent) low-level language.

GCC, perhaps the most widespread compiler of them all, uses this approach: it produces mainly processor-independent "RTL" (register transfer language), which is close to assembly language in general, but not aligned to any particular processor's implementation thereof. I say mainly processor-independent since earlier stages in the compiler do access target details and produce different RTL for different targets.

Read more »