BlackBerry (by alaric)

Many moons ago I did some work writing apps for BlackBerries. I liked the things at the time; they seemed to be well-built, both from the hardware and software angles.

So when my mobile phone contract came up for renewal (meaning I can get a free new phone if I sign up for another two years), with my existing phone falling to bits and rather crashy, I was pleased to find that I was eligible for a free BlackBerry 8520!

The device is a nice evolution of the BlackBerry I was using back in 2004 or so; rather than the thumb-activated scroll wheel we now have a two-dimensional scroll thing that works like a trackball, but seems to really be the innards of an optical mouse, set up so it sees my thumb moving over a small plastic window. This works well, takes up little space, and has no moving parts apart from the click action when it's pressed in to select something. The one downside of the new hardware is that my original Blackberry had a reflective LCD; it had an optional backlight, but spent most of its time with it switched off, simply reflecting the light incident upon it (in colour!). This didn't make for vibrant, saturated, hues in photos, but it did save a lot of power, and meant that the screen was highly readable in the brightest sunshine.

There's a few rough edges in the software; my model lacks a GPS, and the supplied maps application lets me enter my home and work locations by typing in an address, then gives me the option to locate that from the current GPS position (which of course fails) or to look up the address - which also fails, claiming the state/province cannot be found. If I just put the postcode into the address and nothing else, it works - but only matches on the first part of my postcode, getting a location that's some distance away in my rural area. I'd like to have an option to choose the location I've scrolled the map to by hand, geocoding that doesn't suck, and no menu options about GPSes when I have no GPS, please.

The mail system tries to auto-configure itself. Which is a blessing, and a curse. I bet it's a blessing for many users, and their IT departments, that they can just enter their email address and password, and have the rest fetched. However, I have a funny mail setup; I have lots of different IMAP mailboxes on the same server, with usernames like "alaric-work". And there happens to be a POP3 daemon listening on the machine that hosts my employer's web site. So when I put in my work email address, it notices that the domain part of the email address has an A record, and it has a POP3 server, and that POP3 server has a user called alaric (which is the user part of my work email address) which it can log in as with that password - so it goes ahead and makes me a POP3 account... with the wrong username and wrong mail server. Which would be OK apart from the fact that there's no way of changing the protocol or username on an existing mail account.

The trick, it turns out, is to deliberately put the wrong password in on the initial setup screen. This causes the POP3 login attempt to fail, and the subsequent IMAP one (in order to be compatible with Outlook's autodetection, it tries POP3 before IMAP!) too; it then says it can't automatically configure me, and asks me to list username and mail server. It then proceeds to guess IMAP somehow (it still didn't ask me, and the machine has POP3 as well as IMAP on it), and pow, my IMAP account is set up.

Personal Information Management (by alaric)

One of the neat things computers have become able to do as they become more "personal" - eg, we spend more of our lives operating through them, and they become more portable - is personal information management (PIM).

I remember PIM apps in the early 1990s, running under MS-DOS. Quitting whatever you were doing and starting up a separate app to look at your calendar was a bit unwieldy so they didn't do so well, except perhaps Borland Sidekick, which used some clever tricks to pop itself up over running applications.

But nowadays we have systems like Apple's PIM components in Mac OS X; an address book, todo list manager, crypto keyring and calendar provided with the OS, with nice interfaces for using them yourself and an API for all applications to use the same databases. Mac OS software seamlessly uses the inbuilt PIM infrastructure wherever applicable, for the good of all. It's about as good as it gets, so far.

But it's not perfect.

For a start, the task list is a bit primitive. When I had a Mac, I used the excellent Things.app to manage my tasks, as it supports projects and roles and all that stuff, which helps to keep my hectic life compartmentalised. Since it has a much more rich data model than the OS' inbuilt task list, it has to have its own database to keep it in, but it integrates with the inbuilt one as well as it can. My tasks in Things.app get added to the native task list, without their extra information; and new tasks I add to the native task list appear in Things.app, in the 'inbox' area for unclassified tasks, awaiting my attention to move them to a project.

It'd be nice if the underlying PIM database was flexible, allowing arbitrary properties to be added to objects. Then the native task list viewer could share the same actual task list as third party apps, and it would just ignore the extra information about projects.

But even that would suck a bit. Imagine I also had a task manager app that synched to my smartphone, and let me tag each task with the physical locations I can do it from (eg, DIY must be done from home), so an app on the smartphone can show me tasks filtered by location. I'd then need to juggle two task list apps; both would be a superset of the basic task list app (and would therefore duplicate all the basic display-a-task, deal-with-due-dates, etc logic), but each adding their own extra features. Altering all of the properties of a task would involve finding it in two separate apps. Et cetera.

As it happens, I'm also interested in making more use of knowledge representation techniques in software. Knowledge representations such as Resource Description Framework or Horn clauses have the useful property that information from different sources can be merged, as long as the names of things are agreed upon. They work by storing information, not in tables (like the relational model), nor as a graph structure (as the in-memory data model of most programming languages), but as an unstructured list of statements about objects.

The objects can be literal values - strings, numbers, that sort of thing - or symbols of some kind, used to represent objects that don't (or might not) exist directly within the computer, such as people or concepts. RDF uses URIs as its symbols; Horn clauses (being a mathematical notation) are a bit vaguer as to what a symbol is. Either way, they have to be some identifier for the abstract objects.

Each statement links a number of objects, with a relationship (which is itself an object, usually constrained to be a symbol).

So say we have objects called alaric and cheese (the latter representing the general concept of cheese, rather than any particular lump of cheese), and an object called like, representing the concept of liking something. We might write something like:

  (like alaric cheese)

...to mean "alaric is related to cheese by like". As it turns out, just about anything can be represented as such statements. From the relational model, a table can be converted into a symbol used as a relationship (the table name will usually do), and each row into a relationship of the objects that are the values of the fields of that row (likewise, we could have a relational table called LIKES that lists the IDs of objects that like other objects). Relational models usually use arbitrary integers as the "symbols", and automated mapping into knowledge representations is hard because it's not always explicit if an integer column is an identifier, or the identifier of what (as it could be a foreign key), or just a price or some other actual integer quantity. But with a bit of human guidance, it can usually be done.

However, note that in a relational model, that LIKES table would have two foreign keys into some other table that lists all the objects that can take part in liking. It'd be impossible to say that alaric likes like itself (it's nice to like things!), since like is a table rather than a row in whatever table the foreign key pertains to. The relational model has a very static type system, as people who have tried to map class hierarchies into it often find.

Now, the fun thing about knowledge representations is that the objects are implicit. In a relational system there'd be a table of People or whatever, giving each person an arbitrary integer as a primary key then listing details of that person. You can point at rows in this table and say "there are the people".

But a knowledge model just has lots of facts about each person, sort of spread around. There's no place you can point at and say "there's the person". If you want a list of all the people, then you need to look for all statements saying (is-a-person X) ("X is a person"), which is as close as we get to assigning types (not that (is-a-person X) can't coexist alongside statements such as (is-a-customer X); an object can have lots of types, all overlapping). If you want to delete an object, you need to scan the knowledge base for all statements referring to that object, and delete those.

If my personal information was stored in a knowledge base, then Things.app could share the same basic relationship objects as the inbuilt task list. (is-a-task foo), (title foo "Feed the cat"), (is-urgent foo), (due-on foo (date 2009 12 16)), but also add its own: (is-a-project bar), (title bar "World domination"), (is-part-of foo bar).

Note the use of generic relationships - title gives any object a title. is-part-of is a generic containment relationship. This means that even without knowing about projects and task lists, software can tell that things have names, and that there's some kind of containment relationship to explore. The names we give objects - foo, bar in my example - are arbitrary; so for objects that don't have a natural name, they could just be random strings, like UUIDs; RDF even lets such objects have no name and be referred to implicitly. Objects we want to use widely (such as relationships) will benefit from nice names, however.

It's easy to merge knowledge bases, too. Just pour all the statements into one file. You have to be careful of the same object having a different name on each side, obviously, and how to fix that can only be handled on a case-by-case basis; RDF has the concept of "inverse functional properties" that are meant to uniquely identify an object (such as email addresses for people) that can be used to detect and automatically merge things, but it's not applicable to all situations.

If I merged my task lists from my laptop and my phone for the first time, for example, I'd now have more objects satisfying the query (is-a-task X); they'd be seamlessly merged. If I had synched my devices so they both had the task called foo above, and on one I added some extra statements about foo, they'd be seamlessly merged in. It gets more fun when there's collisions - if I change an existing statement, for example. If my knowledge base is sophisticated and puts unique IDs and timestamps on each statement then it can spot that it's a change and handle that in the merge; if not, then I might end up with both the old and new statements, and the application has to decide how to resolve that.

But all of this doesn't solve the larger problem: if I have several different apps, each of which add more behaviour to tasks, I still need to find the same task in both of them to see all a task holds.

So let's get rid of the apps. Can we make a general "knowledge browser" that's good enough to just edit the knowledge base directly?

This task can be helped a lot by having statements about relationships. I might install a package of pre-written knowledge that states that the like relationship normally relates a person to another person, or concept. It might also state that (like X Y) can be written in English as "X likes Y" (and another rule can state that an object X can be written in English as the title of X, if X has one). All of this can just be more knowledge in the knowledge base. A universal knowledge editor could use these statements to build a user interface, in a very similar vein to a browser using CSS to display some arbitrary XML document.

And yet, I'd still be able to jot down my own made-up relationships. (is-on-my-christmas-list bob). There might be no vocabulary statements defining what this Christmas list thing is about, but a universal editor might well list "is-on-my-christmas-list" when I look up Bob, and would thereafter be able to tab-complete "is-on-my-christmas-list", having seen it already in the knowledge base; it might even notice that existing objects tagged with "is-on-my-christmas-list" are all also tagged "is-a-person" or "is-a-company", and use that to guide editing - but that's a bit more advanced. So I can install pre-written vocabularies that make everything nice, or I can just make my own up as I go along, and maybe write vocabularies for them later.

Knowledge base systems can also be fed rules. Say I have an appointment database, with appointments of the form (is-appointment foo) (on-date foo (date YYYY MM DD)), etc.

I might write a rule of the form "(is-appointment X) and (on-date X (date YYYY MM DD)) and (title X (printf "%s's birthday" Z)) if (birthdate Y (date SOME-YYYY MM DD)) and (title Y Z)". Then anybody (or anything) that has a birthday in the system will cause an appointment to appear called "foo's birthday" on any date that has the same month and day as the birthday.

What if we know the month and day of somebody's birthday, but not their year of birth? We can just write: (birthdate foo (date _ 04 03)) (where _ is a special variable symbol that means 'unknown'). Since searching a knowledge base is a matter of pattern matching, that will match a query for (birthdate Y (date SOME-YYYY MM DD)), and not bind a value to SOME-YYYY.

Finally, they can also read from external data sources. I might tell my knowledge base that (exists X) if X is a pathname that refers to a file or directory that exists, that (is-part-of X Y) if X and Y are pathnames that refer to files or directory that exist, X is a directory, and Y is directly within that directory, that (title X Y) holds if X is a pathname and Y is the last part of it (the filename), etc. The rules can work both ways - if the knowledge manager is directly told (is-a-directory X) and it isn't, then the extension rules can tell it how to create it.

Then, suddenly, my home directory becomes knowledge, too. I can add statements saying that a given directory tree pertains to a given project. Then my knowledge browser UI can tell me about the files I've worked on as part of a project. Perhaps I could teach it how to open up applications to edit different file types. Or teach it how to read the files itself and understand their structure and turn it all into more knowledge.

Now extend that to email messages, browser bookmarks, my phone's call history, my IM history, my Twitter account...

On being oddly dressed (by alaric)

Last Friday, I stood up in front of a hundred or so people and gave a five-minute talk on some software I've spent two years of my life writing. However, I wasn't particularly self-conscious about the fact that I was oddly dressed.

For me, clothes are about:

1. Keeping me warm
2. Carrying my stuff

It's not that their appearance doesn't matter to me - I don't want to be wearing shabby or tatty clothes. I don't want to wear garish bright colours. I like my clothes to more or less match, so I tend to choose solid dark colours when I buy myself clothes, as they're easy to look smart in.

But every now and then I get a comment from somebody that I must be a bit weird to go around wearing a podbelt and an assault vest... and when the weather's bad, I got outside in a full-length heavy cloak. Luckily, saying that sort of thing disqualifies people from me being too interested in their opinions, so it doesn't particularly bother me.

I like carrying lots of stuff with me. I'm equipped for every eventuality. When people get things in their eyes, I'm there with a mirror and tweezers. I have the obligatory geek multi-tool, of course. My first-aid kit has brought comfort to many a cut finger. My little lengths of string have jerry-rigged many a repair. I always have a torch, a compass, a pen, a notepad, a monocular, and a laser pointer to hand; so I can navigate, find things in the dark, read small text on a projector from the back of the room (and then point to the thing I'm asking about with the laser). If a button comes off of something, I sit down, take out my sewing kit, and fix it. In my laptop bag is a pouch full of cables and adapters, which has saved the day on many a late-night data-centre emergency. When it's raining so hard that people are cowering in shop doorways, my cloak keeps me dry; at the conference on Friday, when there were no seats left, it folded up tightly and became a low stool so I could sit comfortably. A week or so ago, when I was driving home from London very late one night and became too tired to continue, I pulled into a dark lay-by and slept underneath it, warm and comfortable even when the temperature plummeted before dawn.

I'm not just hoarding gadgets for the sake of it - I do assess the trade-offs of every extra bit of weight to carry around. Weight in the podbelt isn't an issue as it carries very nicely on my hips, I barely notice the weight of it, but space there is at a premium. Weight in the assault vest is more of an issue, since it pulls at my shoulders. I've tried having just a podbelt, but it's not good to wear while sitting down, so I tended to take it off and sling it over the back of the chair, which makes things harder to get at; and I've tried just having an assault vest, but weight was a problem. The current combination means I can keep lightweight things I often need while sitting down (mobile phone, pens, pads, business cards, laser pointer, etc) on me all the time, while weightier things I tend to need more on the move (keys, wallet, tools, first aid gear) in the podbelt. I have optional extra things I add for specific "missions" that I wouldn't want to carry all the time, too - I have a special tool jacket with loops for screwdrivers and the like which I wear if I'm doing DIY in awkward locations, an extra assault vest with more specialist stuff for when I'm busy being a Cub leader, a black lightweight mesh one with large pockets for hiking (the large pockets accept good quantities of food, GPSes, and the like), a water flask that goes on the podbelt, and a spare podbelt pouch that I'm going to assemble a survival kit in: emergency rations, a survival blanket, that sort of thing.

When I've explained this to people who question the amount of stuff I carry, they say "But what are the odds of all these things happening?". But they happen all the time! So I'm happy being prepared for anything... it makes life a lot less stressful. My clothes and their pockets become an extension of my body; we are, after all, all cyborgs.

Heating an old house (by alaric)

Sarah feels the cold keenly, while I can usually just put on some more warm clothes to deal with British winters. But even I was finding it hard to work in my home office when the temperature went below ten Celcius; fingerless gloves still let me type, but numb fingers increase my error rate, and the pain is distracting.

Part of the problem was that our house is draughty. There were a lot of gaps in the window and door frames, through which daylight could be seen; when it was windy and rainy at the same time, the wind blew rain in through the frame of the large window in my office.

So step one was to fix these. The large office window, it turns out, is somewhat curved, so when my brother in law was visiting, we screwed extra handles to it, pulled it properly closed with levers wedged in the handles, then did up the bolts at top and bottom to force it to stay in shape, which fixed a large source of draughts.

Then I want around a few other choice places, adding draught excluder strips where I could.

Next challenge was to increase the heat. We had only one real source of heat in the house, a wood/coal burning stove at one end of the house. Since it's a long thin house, this was little help for me in my office, right at the opposite end - but it didn't even make enough heat to keep Sarah happy sitting next to it, so she would often use the expensive electric fan heater to keep her temperature up, much to my concern (for if we can't pay the electricity bill, things will quickly become rather unpleasant).

Now, the grate in this fire was rather small compared to the size of the fire itself. The grate had only sides and a front, so had to be pushed back against the firebricks in order to not spill coal out. This meant that air coming in through the vents would tend to rise over the fire and up the chimney, taking heat away without imparting much oxygen to it. Even then, it would slowly wriggle forwards over time, spilling ash and coal down behind, until it came too far forwards for the ash shovel to be pushed underneath it, meaning the fire would choke itself. But as it moved forward, the effective volume fire increased, with a notable improvement in the heat output - even though the fire at the back would be starved of air from beneath, as it sat on a bed of ashes.

While rummaging through piles of random bits of metal lurking about the place from when we moved in, though, I found an iron grating that I suspected might be able to fit in behind the existing grate, enlarging it. Sure enough, it did - and it fitted so perfectly well that I suspect it was actually meant for it. Suddenly it was possible to have a large bed of coal in the fire, with air coming in through the vents from underneath it and being drawn up through; this led to an awesome increase in heat. However, it led back to the same old problem - we now didn't have room to get the ash shovel in underneath to take ash away. And so the fire would slowly choke itself with ash.

So I ordered two metres of 25mm square hot-rolled mild steel from Hindleys, my favourite home-engineering supply house. When it arrived I used my angle grinder to chop off two lengths of the stuff, then used them as spacers on either side of the grate to lift it up an extra inch.

And now the fire's awesome. I can easily get it so hot that it becomes mildly terrifying, an angry yellow glow emanating from the air vents as it roars away, the radiated heat unpleasant to be too near. A few days ago, it actually melted the plastic crates we store our newspaper and kindling in, purely by radiation.

But it's still rather cold in my office.

So we decided to spend some money on the problem, as it was in danger of harming my work. I went down to John Stayte Services, a local purveyor of awesome things. We buy our coal from them, but they also sell propane, butane, related accessories such as heaters and Sievert torches, workwear, and animal feed. To my delight, they had a deal on; a shipment of gas heaters had been damaged due to the shipping container being broken into by illegal immigrants who built a home on top of them for the duration of the voyage... so they were selling a slightly dented heater, along with a cylinder of butane, for £89 when normally a heater alone would cost more than that (and a gas cylinder £50 or so as an initial outlay).

I set it up in my office, lit it... and over the next few hours, the temperature rose from ten degrees to about twenty, with me correspondingly shedding layers of clothing. Since then I've been running the heater on low power, and the temperature's stayed around seventeen degrees; with the stones of the building having been warmed up, it's now not taking much heat to keep it nice and warm.

And so, I can proudly state, for the first time since we moved in, it's actually warm enough at home that we are turning down heat sources so as not to be too hot!

A new laptop (by alaric)

To my great displeasure, my shiny MacBook Pro was stolen from the office in London!

So, I grabbed our finance guy and we went down to the nearest laptop shop and picked up the cheapest thing they had in the shop that would meet my needs: a Hewlett Packard Pavilion dv7.

The first step was replacing Windows 7 with something. As I knew I might have some teething troubles with getting NetBSD installed, so might need to return to Windows to get online, I shrank the Win7 partition down so I could run it dual-boot rather than nuking the whole thing. However, it was worse than I feared - the NetBSD install CD wouldn't even boot - the boot loader came up, then complained it couldn't read the kernel.

Not good.

So I burnt an ISO of Arch Linux, which is the closest to BSD in the Linux world. No good, either - GRUB loaded, and couldn't load the Linux kernel. I downloaded the boot.kernel.org nano-ISO (which then boots over HTTP from a central server) and that booted OK; but many of the Linux installers I tried died of kernel panics during booting. I wondered if the difference was that boot.kernel.org had an ISOLINUIX-based installer rather than GRUB; googling for this, I found out that GRUB sometimes has trouble with some CD drives, and as such, Arch Linux came with the option of an ISOLINUX-based installer CD. So I burnt a copy of that, and pow, it worked!

I installed Arch Linux, but only in a small partition, still intending to try and install NetBSD via non-CDROM means at some point. I soon had X up, and all was quite well, apart from the fact that my wireless Ethernet module (a Broadcom 4315) didn't work. I found a driver and installed it, and then it was recognised, but it still refused to actually do any wifi. Bah humbug.

On a hunch, I re-downloaded the NetBSD installer ISO and burnt a new CD of it... and it worked, this time! Having wasted most of a week trying to get the first one booting then messing with Linux distros. So I left my Arch partition in place (in case I needed it for anything), and set up NetBSD. Got X working again, copied across the home directory I'd made under Arch rather than redoing my dotfiles, and was happy. Except that NetBSD also didn't like my wireless interface, and it had trouble with the ACPI too, so I can't read my battery status or do a suspend properly. I've found a NetBSD driver for Broadcom devices but I've yet to get it to compile (I think it was developed mainly for the macppc port, or I've just not applied the patch properly). When I get my kernel source tree compiling again I'll have a new kernel that also has verbose ACPI debug messages, which will probably help.

I get Xen working, though; it was painless now that NetBSD's boot loader can do multiboot kernels, meaning I didn't need to mess with GRUB; I just installed the xen tools, and a xen kernel, and a NetBSD/xen kernel, then added the following line to /boot.cfg:

  menu=Xen with 1GiB for dom0:load /netbsd-XEN3_DOM0 console=pc;multiboot /xen.gz dom0_mem=1024M

...and rebooted into a Xen dom0. Setting up some Linux domUs for my work has been more exciting: I downloaded Debian filesystem images and a kernel from stacklet.com, but when I bring it up, Debian complains it can't bring up the Xen virtual network interface - and that it's getting lots of disk errors on the Xen virtual block device (although it can actually read the filesystem without any trouble whatsoever). This may not be helped by the fact that the kernel I downloaded doesn't seem to match the /lib/modules directory in the filesystem image, alas. More work is required.

Still, I'm happy to be back on an X workstation. I liked my Mac, but I was feeling hankerings for open source software, and minimalism. I'm running dwm, dmenu, and remind, joined together with awesome shell scripts. I do miss the intergrated personal-information-management tools in Mac OS X; I want to brew up my own database of people/todos/events/etc in some Prolog dialect (eventually replacing remind), so I can express all the relationships between things that I want ("There is an all-day event on date X called 'Y's birthday' if there is a person P with name Y whose birthday has the same day and month as X", sort of thing) - it'd be nice if it could integrate with Thunderbird's address book, so I may have a look at the format of that (or look into a script to sync it to/from an LDAP directory).