Thoughts on Nice Clothes (by )

As I have previously opined, I like practical clothing with lots of pockets; I am generally more interested in practicality (storage capacity, comfort, keeps me warm and dry) than looks in my clothes. So where I get to choose colours or designs, I tend to go for "black" most of the time; the exception being t-shirts and hoodies, where I of course often get things with amusing cultural references or logos of things I'm proud to support. Although I'm agender, my feature-driven taste in clothing is entirely masculine because, sadly, female-gendered clothes tend to be designed for style rather than features...

However, a discussion I was tangentially involved in a while back got me thinking: if I chose to disregard practicality for some reason and wanted to choose clothes purely for what they look like, what would I want to wear?

After much deliberation, I think I like the cut of garment called a Thawb - although it took me a while to find out that's what it's called! But I'm not keen on the white ones as pictured in that Wikipedia article, I'd like a nice dark black/blue/purple one with some kind of angular pattern embroidered on it in white, silver or gold thread. Perhaps a Hilbert curve around the borders, too?

Not that I'd wear it much, mind; I only really dress up for weddings and stuff like that (The jacket I got married in is somewhat similar to a thawb, although shorter; that's served as an inspiration). Although in principle it might be kind of nice to dress up just to lounge around the house and stuff, it would be rare for me to have a day without having to go out in the rain, brave mud, cook, drill holes in walls, etc - not to mention dealing with the unexpected demands that I carry all that stuff in my pockets for!

Designing software (by )

One thing that has often annoyed me in my career as a software designer is coming into conflict with people who want me to do the worst possible job that is "just good enough".

This often comes out in statements like:

  • "There isn't time to design it properly, just hack something so we can start demoing it"
  • "We'll have to rewrite it anyway, requirements always change"
  • "Supporting more features than we need right now is a waste, add them when we need them"
  • "Can't we just do something quick and simple?"

Reading between the lines, they seem to think that "more designing" will mean more complicated software with more features, that will take more time to build.

I think the problem comes from how product management thinks of software - they want a feature, they request the engineers add it (ideally specifying the feature well enough that they actually describe what they want), they get that feature. And there's often some discussion about reducing the scope of the feature by removing some criteria to get it implemented sooner. It seems very much like "more features with more acceptance criteria equals more work".

I'd like to dispell that myth.

I assert that better-designed software.will take less time to write, and will be better due to being more flexible rather than through having more features, and through being easier to extend in future when new requirements come up.

There's a paragraph in the Scheme language standard known as the "Prime Clingerism", and it reads thus:

Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary. Scheme demonstrates that a very small number of rules for forming expressions, with no restrictions on how they are composed, suffice to form a practical and efficient programming language that is flexible enough to support most of the major programming paradigms in use today.

It has been my experience that this approach applies to all forms of software design, not just programming languages. As a software architect, I see designing a feature as more like Michelangelo making the statue of David (famously, he said that the process of making a statue is just to get a big enough block of stone, and remove all the bits of it that aren't the statue).

Rather than thinking in terms of each acceptance criterion in the specification for the feature as a thing that will have to be done separately (so more ACs means more work), I like to work the other way: what's the simplest piece of software that can do all of these things? Most problems have a "core" that it can be boiled down to, plus some "details". I try to design software that has to meet specific requirements by making the "core" with the fewest assumptions and then adding "details" to tailor it to the exact requirements - which tends to mean I can add more details later to fulfill more requirements.

An example

For instance, many years ago, I worked at a CRM SaaS company. A big part of the app was sending out HTML emails to subsets of a user's customers to offer them some special offer, serving up the images in the email from our HTTP server (and tracking who saw the email by including the email message ID and the recipient ID in the image URL), and tracking what links they clicked by replacing URLs in the email body with URLs hosted by our HTTP server that would log the email message ID and recipient ID then redirect the user.

I was given a feature request: people were occasionally forwarding the emails to their friends, and when they did so, their viewing of the email images and their clicking of links would be logged against the original recipient, as their ID was in all the URLs. Our server would see the same person viewing the email lots of times and clicking the links lots of times. Learning how users responded to these messages was a big priority for our customers (we had extensive reporting systems to analyse this data!), so they were tetchy about this, and desired the ability to put "forward to a friend" forms in the email HTML. The recipient would put their friend's email address into this form and hit a button, and our system would send a fresh copy of the email to their friend, with their own recipient ID - so our user would know the forward had happened, and could see the friends' response separately.

Now, the "simplest" solution would be to add an extra feature to our HTTP server, accepting a form submission to a URL containing the recipient and message IDs, extracting an email address from the form submission in the usual manner, creating a new recipient in the DB with the email address (or finding an existing one), recording a forward from the original recipient ID to the "friend"'s ID, and sending a copy of the email to the "friend" - then responding to the form submission with either a hardcoded thankyou page or responding with custom thankyou-page HTML uploaded by our user, or a redirect to a URL set by the user.

A quick chat with the boss revealed that being able to host custom thankyou pages was a very desirable feature, and as that would involved embedded CSS and images (and maybe javascript), I clearly needed some kind of generic user-uploaded-content HTTP serving infrastructure. So I threw that together, letting customers created websites in the application and upload files, and an HTTP server that would server resources from them with a URL containing the website ID and the path of the file "within" that website - and with the ability to tag URLs with email message and recipient IDs for tracking purposes. We already had an engine to "personalise" HTML message content for a specific recipient, which also handled modifying all the URLs to embed email message and recipient IDs, so I used that again for hosting textual content such as HTML, meaning that if a link came in from an email message with message and recipient IDs, they would be fed into all further URLs returned by the system. To prevent spoofing of the IDs in the URLs, I reused the method we used on the image and redirect server URLs: the URL itself was "signed" with a short cryptographic signature when the software generated it.

But rather than hardcoding the forward-to-a-friend feature into that, I did something that took perhaps ten minutes more programming: I allowed an arbitrary number of named "commands" to be included in a URL. The only named command I implemented was "forward" that would do the forward-to-a-friend logic (finding all form submission fields called "forward-to" and using them as recipient email addresses, so you could make forms that accepted lots of addresses; making it loop over all fields with that name rather than just looking for one was the first assumption I removed to simplify the software while making it more flexible) then proceed with handling the HTTP request in the normal way.

But when a requirement came in to support unsubscribe forms, I just added an "unsubscribe" command, which took all of five minutes. Adding an "update" command handler to let people update their details in the master DB took a little longer, just because I had to list all the supported field names in the documentation and include a more lengthy example. And then, because I'd implemented the recipient and email message IDs in the URLs as optional things from the start, adding a "subscribe" command handler that created a new recipient in the DB from scratch, along with a tweak to the UI to let the user get an anonymous URL for any file in their website took an hour or so - and meant that users could now create subscription forms and generate a URL they could publish. I think I also added a "send" command handler to send a copy of a specific email message to the recipient ID in the tracking context; as the "subscribe" command put the new recipient's ID in the tracking context, a URL with a "subscribe" command followed by a "send-" command would handle a subscription and then send a welcome email to the new subscriber...

I added a few other command types that did various other things in the system, and all was good.

Now, I didn't "set out" to design an "overcomplicated super-flexible" HTTP server system that could do all these things when I was asked to add the forward-to-a-friend system. I just spotted that the following assumptions would be really easy to NOT bake into the core of the implementation:

  • Forward to a friend is always to a single friend; it's easy to create multiple form fields with the same name in an HTML form, and easy to make my server loop over any it finds.
  • Every visit to the new generic HTTP server will have a recipient ID and an email message ID; although forward-to-friends will as they will be linked from an email, it's easy to imagine that we might want to host subscription forms in future. So making the tracking IDs in the email optional (or allowing for other kinds of tracking IDs in future) by making the URL structure by a series of tagged fields (it looked something like http://domain/sss/Rxxx/Eyyy/Wzzz/foo.html for a request for file foo.html in website ID zzz for recipient xxx from email yyy, with sss being the anti-tamper signature) was worth the few minutes' work.
  • Requests to the web server will be forward-to-friend requests or just static resources in support of a response page; it was easy (given we already had an ordered list of tagged components in the URL) to added an ordered list of arbitrary commands (with URL path components like forward, subscribe, etc; any component before the Wzzz/filename part was considered a command or tag, and tags started with an uppercase letter).

The "extra work" required to do things that way rather than the "simple" way was tiny. But it meant that the core of the HTTP server was simple to read and didn't need to change as we extended it with more and more commands (by adding them to a lookup table); it made a powerful system that was easy to extend, easy to understand, and capable of things we didn't have to "add as features" (some or our users did quite creative things by stacking commands!). As the HTTP server core and the commands were small separate modules with a clearly-defined interface, they could be understood individually and were well isolated from each others' implementation details, the benefits of which are well documented.

So please help me fight the assumption that putting thought into designing software means it'll be complicated, more effort to implement, and that "it'll need rewriting anyway so there's no point"! Let me do my job well 🙂

Enthusiasm (by )

When I was a child, I was full of enthusiasm - as a keen self-taught engineer I was soaking up knowledge about the wonderful things that could be done, and my future was full of promise; I lacked the tools and money to build many of the things I planned, so focussed on tinkering with software (once I had a computer, programming was free!). But I was confident that I would be able to turn my skills to employment and earn enough to buy tools, and then I'd build so much cool stuff.

However, it took a while to get there, and along the way, I accumulated lots of pressures on my time as well. These days, when I have free time, I'm often too physically exhausted to do much, and that enthusiasm is all gone - nothing seems rewarding any more, and I fritter the free time away.

But it's not always like that. A few times a year, a burst of enthusiasm comes to me (and I think I know how to trigger it deliberately, too).

This weekend, I did a lot of DIY. I worked on the van, tidied the house, sealed the skirting boards in the kitchen (I've learnt how to apply sealant neatly!), caught up on my emails, did some financial admin, wrote up a lot of scrappy notes I had into my filing system (re-organising some bits of it on the way), and caught up on work hours I'd missed in the week due to visiting my sick father in hospital (he hurt his knee, and is recovering well!).

The combination of dealing with emails, filing my notes, and organising my filing system, however, brought back The Enthusiasm.

Which, on the one hand, is great - I used that energy to get a lot more done than I usually do.

On the other hand, it also meant that when I went to bed at 10:30pm (for a 6:30am alarm clock start), I couldn't sleep as my head was buzzing with ideas. I wrote them up in my bedside notepad, which usually releases the pressure of thinking about them, but one of them was exciting (a really nice way of supporting HTTPS in my Web hosting stack), and one of them wouldn't stop going around in my head - I came up with a simpler design for a new desk/shelving system I want to build in my workshop, combining my computer desk and an electronics workbench. This was stuck in my head because I couldn't just think it through to completion and then record it; I was trying to visualise all the fine details to work out how it would fit together, and it wouldn't fit in my head. So about 2am I gave in and went downstairs and fired up OpenSCAD and bashed out a 3D model of it, which also spits out a cutlist of what lengths of square steel tubing and areas of plywood I'll need to make it:

Figuring out all the fine details and seeing how they'd fit together finally relieved the mental pressure, and sometime past 4:30am, I fell asleep... getting somewhat less than two hours' sleep.

Today I've been able to divert the energy to my work, which is great, as that's what I'm supposed to be doing - and using that energy to make up for the fact that I'm dog tired. But I still spent my lunch break writing up my overnight notepad notes into the filing system and doing a few of the tasks I'd thought of, including planning a comprehensive consolidation of my sewing supplies into a proper sewing box plus a tiny sewing kit for emergency repairs, that can go into my bag. And writing this blog post!

I'm confident this is not evidence of bipolar disorder, because I'm fully aware of my slightly manic state, and I'm following tasks through to completion! But it's still not an ideal situation.

I've observed in the past that The Enthusiasm can be invoked by doing "infrastructure" work - updating my filing system, tidying my workshop, building tools, maintaining the van, building software infrastructure, etc. so my plan is thus:

  1. Book out infrastructure days.
  2. Work on infrastructure projects in the mornings. Try to remember the Enthusiasm I've felt before (this blog post will help as a memory jogger) plus sheer willpower to get me started, even if I don't feel like it.
  3. Let my enthusiasm take me where it takes me in the afternoon.
  4. Do this frequently enough that it's not all backed up inside me somewhere, so just a bit comes out at a time in a controlled manner, rather than big multi-day sleepless orgy of creativity.

Wish me luck!

Our radio adventures (by )

My last post was about amateur radio in general, but for those of you who are interested, here's a summary of where we're at and where we're heading.

Jean and I did the Foundation Licence course together and passed, so we have UK Foundation licences. That means we can transmit in nearly every amateur frequency allocation, covering the HF, VHF, UHF and microwave bands discussed in my blog post. We're limited to ten watts of output power (details vary slightly in some bands), which is enough to do most things: at that power level you can communicate over intercontinental distances on HF, especially using Morse or data modes. I'd like to progress to the higher levels of licence, not so much for the increased power levels (although those might be handy), but because at Intermediate level you can use transmitters you have built yourself; and at the Full level, you can operate outside of the UK (as that level of licence meets the requirements of international radio law).

What we've done so far:

  • Obtained FM VHF/UHF handhelds - the ubiquitous Baofeng UV-5Rs - and experimented with external accessories for them; we have a magmount antenna on the car and a UV-5R hooked up to the 12v socket in the car and a fist make, as a cheap mobile rig, and various different headsets, antennas, and pouches to find comfy ways to wear the things out and about.
  • I've tinkered with antenna construction; I built a quarter-wavelength vertical with a radials, hinged so it collapses down flat, for the VHF amateur band. It seems to work pretty well.
  • Chatted to each other on the handhelds. We've done some experiments for the sake of it around town, and used them to usefully keep in touch at a couple of outdoor events.
  • I've played with APRS a bit. I didn't explain that in my last blog post: it's a protocol for announcing position and basic status information in digital data packets over the radio, meaning you can track the positions of other stations near you.
  • Chatted to each other on a local repeater, as an experiment. (Repeaters are set up by volunteers on tall masts so they have a good view of a lot of the local area, and re-broadcast your signal for you, so that two people down on the ground amongst buildings and obstructions can still talk to each other).
  • I've joined in some conversations with strangers on the local repeaters, but it's still a bit nerve-wracking doing so!
  • I've done some experiments with building simple HF antennas and hooking them up to a wide-band receiver I already have, and picked up a few signals - but mainly noise at home.

Things we're interested in doing include:

  • Volunteering with the local RAYNET group, to get some practice in doing useful things with short-range comms (VHF and UHF). In particular, we both want to gain experience with radio etiquette and build up confidence.
  • Talking to the International Space Station, which our licences allow - we have all the required equipment apart from a more directional antenna! We'll probably start with relaying digital data from the ISS digipeater as it doesn't rely on an astronaut being around to say hi.
  • Talking to amateur satellites; requires the same equipment as the ISS, but involves talking to other people on the ground. This will be fun because (a) SPACESHIP!!!! and (b) you get to talk to people much further away.
  • Getting onto HF! Our home isn't great for an HF antenna installation, from my experiments, so I'm interested in "portable operation". This looks fun - there are contests such as SOTA based around making contacts from hilltops and mountaintops, or IOTA for making contacts from islands - and I like the flexibility of being able to set up anywhere.
  • Getting a VHF/UHF antenna on the roof at home, with a base station transceiver. I'd like to be able to sit at my desk and monitor the local repeater, for instance, so that Jean can get in touch with me while she's out and about. Access to our chimney is tricky and my desk is in the workshop which is at the other end of the garden to the house (requiring a long cable or an expensive remote-controlled transceiver), so I might need to build a small mast on top of the workshop: the signal might be good enough for the repeater, at least, and making a truss mast out of aluminium would be a fun welding project.
  • Maybe build a "magnetic loop" HF antenna on the workshop roof; those are supposed to get better results in circumstances such as mine. Experiments must be done!
  • We both want to learn Morse; this opens up the ability to make longer-distance contacts with smaller, cheaper, transmitters, and is kinda fun in its own right.

Amateur radio (by )

The radio spectrum is a complicated and varied thing.

Although some people seem to separate "radio" from "microwaves", I tend to lump them together, so I'm going to consider "the radio spectrum" to be the section of the electromagnetic spectrum from the very lowest frequencies we can usefully use (a few Hertz) up to frequencies that we'd start to count as "infrared light" (several terahertz).

Those bounds are based on practicality - for very low frequencies, the antennas required to efficiently transmit become impractically large and the bandwidth available to communicate gets ridiculously low; at very high frequencies, our electronics cannot process the signals and we have to switch to optical methods, which is a whole different ballgame.

I'm going to give you a whirlwind tour of the radio spectrum - but with an agenda: I'm just setting the scene so I can talk to you about amateur radio... Read more »

WordPress Themes

Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales
Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales