The joys of compressed air (by alaric)

I've always had a hankering for pneumatic tools. The idea is that compressed air is used as a power source for hand tool, rather than an electric motor. This has various advantages - the tool is lighter (motors are heavy), it's safer (electric motors, when stalled, produce a sharp increase in torque that can make the thing break your wrist if you're not lucky, and the motor can then burn out), and it's cheaper (air drive thingies are cheaper than motors) and simpler. This makes a wider range of tools practical and affordable, from drills and power screwdrivers to stranger things like pneumatic files and caulking guns. And compressed air has some unique uses, too - tyre inflaters, blowers for clearing dust away, and spraying liquids for painting or cleaning.

The downside, however, is that you need a source of compressed air. Thankfully, these have been getting more popular in the hobbyist market, and therefore cheaper, and got a bonus from work, and I've wanted one for years, and so, POW! I now own an air compressor!

It's only a small one. I spent just over a hundred pounds, which is more than I've spent on something purely for fun (it's a tool, but not one I need) in ages. I picked up a set of basic air tools with it - a tyre inflater, a blower, a paint sprayer, and a wash sprayer. Sarah's keen on using it to paint things, so once she's had a go with the spray gun, I'm planning on getting her a more artist-grade air brush with fine controls, and she's getting me a pneumatic ratchet (that can do and undo screws and bolts) as a first "proper" tool.

I'm very tempted by a pneumatic nail gun, because they can drive a nail into wood in a single "phut" compared to having to hammer one into place (and the ever-present risk of it going wonky halfway through), which is a huge labour saver - but I don't actually use nails all that often. Perhaps in the next round of workshop roof fixing...

I thought it was broken when I first unpacked it; the manual had a series of steps to follow when first setting it up, and when switching it on (check the crankcase oil level, set the valves to a safe position, etc) - I followed them all and turned the power on, and the motor started roaring away and the tank pressure rising. But the pressure wouldn't go above two bar, and when I turned the motor off, all the air hissed out again. It was getting late so I got ready to go to bed rather than playing more, but while flicking through the manual's troubleshooting section (it looked like a failure of the non-return valve or the safety valve, and I resolved to check both in the morning), I noticed that the instructions for AFTER you've used the compressor included opening a cylinder drain valve to let out any condensation... I felt underneath and, sure enough, the drain valve was wide open! Checking that's closed should be part of the starting process, but wasn't in the instructions...

In the meantime, I've been having fun with the kids making balls levitate in the air stream from the blower 🙂 We can float a ball-pond ball a good twenty to thirty centimetres above the blower nozzle, at an angle of up to forty-five degrees. and 1.5bar; when I have a face shield on and no children crowding around I might try cranking the output pressure up to the full 6-8bar the machine can produce and see if I can float balls at steeper angles, or heavier objects...

Home Automation: Phase 1 (by alaric)

I've always had a nerdish fascination with home automation.

There's been a recent trend towards the "Internet of Things" (a.k.a. "IoT"), which is related, but different. The IoT seems to focus more on home devices talking to centralised Web services, which is a hateful model (we all know why: those central services are outside of your control, so unless you're paying a subscription, you are the product rather than the customer; and if they're shut down, your devices become useless; and they can leak your private information whenever they want; and they can take control of your home; and it all falls apart if your Internet connection goes down; and I'm sure there's others).

No, I want to have my house under computer control - but with those computers under MY control. This is something I've planned for for ages, but as with all hardware projects, getting started is tricky; I need to commit to a final design and then afford to buy the parts, and that's scary because I might find out that the parts don't quite work together in the way I wanted. Unlike with software, hardware hacking requires up-front commitment of resources, that can't be backed out of. Scary!

So the trick is to split the thing into small parts, with flexible interconnections, so I can iteratively prototype parts and then connect them up in due course. And this last week I took the first step - building an announcement system in the living room.

Read more »

Thoughts on Programming and Tracing (by alaric)

I was recently pointed at this interesting article: Learnable Programming.

It's a good read, overturning many assumptions the software industry has picked up over the years, and propagated without thought since.

The first part suggests allowing a programmer to trace the flow of execution of a program graphically, using an interactive timeline. My first thought was that this was all well and good, but would rely on every library in the language annotating every operation with information about how to present it - producing the little thumbnails to go in the timeline, or exposing numeric values that can be plotted onto charts. Also, highlighting the "current" drawing operation in red on the canvas relies on those operations being things that affect a canvas; more abstract operations, such as writing to a database (or even generating images to be encoded directly into a file rather than onto the screen) would require a more explicit "object preview".

However, those are not insurmountable goals. And, perhaps, things that can be built on top of my ideas about logging and tracing, making it possible to use such an interface to go through traces of execution captured from production servers, rather than just within a cute live-coding IDE; the trace entries generated by operations in your libraries could, with the help of a meta-library of trace visualisation rules, generate those little thumbnails. However, it would need to be augmented with dynamic scope information provided by the programming environment itself to know which line of code caused the trace event; the kind of thing one finds in a stack trace.

He asks "Another example. Most programs today manipulate abstract data structures and opaque objects, not pictures. How can we visualize the state of these programs?"; so I suggest that the abstract data structures and opaque objects be annotated with code that summarises their state. Many languages have a notion of "return a string representation of this object", generally aimed at debug logging - Python's repr() versus str(), for instance. Perhaps if we moved to expecting objects to return HTML representations of themselves, we could take a step in that direction.

The second part (and I'm taking some temporal liberties here, as some concepts I've included in the first part are touched upon in the second and vice versa) is also inspiring; it looks at the bigger picture, considering how libraries and code-editing environments can be designed to make it much easier for programmers to identify what operations their libraries are making available to them, rather than requiring the first step to be the reading of documentation. It touches on topics such as the dangers of mutable state (preaching to the converted here!), and the choice of library function names to make code using them clear (I'm also a big fan of smalltalk / Cocoa-style function call syntax, and how it might be brought into the Lisp family of languages...)

I've written before that I think modifying software should be a much more widely-practiced activity; and I think that should be achieved through removing unnecessary obstacles, rather than forcing everyone through complicated programming classes. I'm always interested in more thoughts on how to make that happen!

Look upon my works, ye mighty, and despair (by alaric)

I took it upon myself to translate the following into Lojban:

I met a traveller from an antique land
Who said: Two vast and trunkless legs of stone
Stand in the desert. Near them, on the sand,
Half sunk, a shattered visage lies, whose frown,
And wrinkled lip, and sneer of cold command,
Tell that its sculptor well those passions read
Which yet survive, stamped on these lifeless things,
The hand that mocked them and the heart that fed:
And on the pedestal these words appear:
"My name is Ozymandias, king of kings:
Look on my works, ye Mighty, and despair!"
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away.

I decided I wanted to aim for an idiomatic Lojban translation, rather than a word-for-word translation of the English (which would probably result in quite clumsy Lojban). However, this was a challenge, as it would mean making up new good-sounding idioms for things that I either didn't know, or hadn't actually developed as idioms in Lojban yet!

Here's what I came up with. There's plenty of problems with it, which I'll explain below, in the hope that jbopre will read this and suggest improvements. However, even if you don't know Lojban, please read on; I think you might find the breakdown of it (I give literal English translations of it) interesting:

.i .u'e re lo barda tuple ku se stuzi le cantu'a
.i cpana le canre fa lo se porpi be lo flira
noi .iiru'e lo du'u ke'a frumu ku
     .e lo du'u ke'a tolpluka turni cisma ku
   nibyti'i lo du'u lo zbasu ku pu te smuni
      le se cinmo poi lu'e ke'a za'o renvi
           zi'e poi fu lu'e ke'a xraci'a fi le na jmive
.ije rakci'a fi le zbepi
     fe lu ga'isai mi'e la'oi .ozymandyus.
           .i mi turni lo turni
           .i .a'onaidai ko catlu tu'a mi doi tsali li'u
.i zo'e bi'unai po'o renvi
.i lo .o'ebe'udai pinta canre ku po'o diklo le barda se daspo

Read more »

Opening up Open Source (by alaric)

One of the awesome things about free/open source software (FOSS) is that, as you have access to the source code, you have exactly as much power as the original authors of the software to modify and extend it.

When you are upset about something in closed-source software, or one hosted on the owner's servers on the Internet as a web app or API, all you can do is beg and plead with them, and threaten to take your future business elsewhere; they control the source code, so they have ultimate power. With FOSS software, in theory, the original authors have no special powers.

However, it often doesn't quite work out like that in practice.

Most FOSS software comes onto people's computers as a precompiled binary; they download it and run it. This is convenient and efficient. But if they then want to use their theoretical right to get in there and modify it, they need to do the following things:

1. Learn the programming language(s) it's written in, and the libraries/frameworks/APIs it builds on top of. (This is particularly tricky if they have no previous programming experience).
2. Find and download the sources (which shouldn't be too hard, but can still be tricky).
3. Set up a build environment that can compile the thing. This may involve installing compilers, and development versions of libraries in order to get the header files, and so on.
4. Actually make it compile. A lot of the time, a source package as shipped by the authors won't compile perfectly outright, as you need to apply patches written by the people who maintain the package for your platform, as each platform has their own conventions as to where things are placed in the filesystem, and their interaction with less-standardised bits of infrastructure such as service management frameworks, management of network interfaces, low-level hardware access, and so on.
5. Learn the workings of the codebase. For a large project, this can be VERY daunting, even to a seasoned programmer.
6. Actually design, implement, and debug the change. If the codebase is poorly architected, this can be made unnecessarily difficult, and require refactoring of existing code elsewhere within the codebase.
7. Install the software once it's building. As it's been custom-built, it may not interact nicely with your platform's package manager, so you may end up running it out of /usr/local/bin or ~/bin or similar, and have trouble making managed packages that depend on the one you're tinkering with correctly linking to it, interaction with system configuration tools, and so on.

I think this is harder than it needs to be. It puts a lower bound on the effort required to make even a trivial change that, in many cases, means it's not worth making it; we're as beholden to the whims of the developers of the package as we would be to a closed-source software company, and the supposed benefits of open source are denied to you.

We can break down the barriers into a number of categories, and look at what can be done to solve each.

Learning the programming environment

Whether you're a seasoned programmer or not, any given piece of software you didn't write will involve a set of languages, tools, libraries, and other things, that you may not be familiar with, and these will need to be learnt.

What might help is better standards for automatically-generated API documentation, so you can find the documentation of API functions as they are used in the software you're learning how to edit by just pressing a button in your editor.

But if it became easier, commonplace, and expected for people to dig around in other people's source code, for curiosity or to make their own changes, developers of infrastructure components would feel more compelled to document their interfaces in ways that casual programmers can quickly pick them up, and to make their interfaces simpler and easier to learn, because the expected audience would be less dominated by seasoned programmers.

And, conversely, if more people were casually getting involved in simple programming tasks in order to improve the software they use (if it became easier to do so), then the general programming ability of the population would also rise, giving more people the grounding in basic conventions and concepts required to understand programming tools...

Migrating from a normal installation of the software to a hackable one

This is perhaps the biggest hurdle, and yet, the most amenable to being overcome with better technology. It covers the whole spectrum from finding and downloading the source code, setting up a build environment, getting it building on your platform, and getting it installed as a first-class citizen in the eyes of your package manager.

I can think of two technical fixes to this problem, and the best bit is, they're both things that already exist out there rather than my usual kinds of crazy new reinvent-the-wheel thinking!

Firstly, package managers like Nix make it easy to establish build environments on your own hardware, as the build environment of any package can be requested, and automatically set up for you. Also, they are built around installing software from sources in the first place, and offer downloading pre-built binaries as an optimisation. It's quite easy to adapt a nix expression that builds a software package from downloaded sources to one that builds from a source tarball you've made yourself, and install it into an isolate "profile" in such a way that it's easily kept isolated from other software you have running and might not want to risk being broken by your experimental changes yet, and to roll the change back if it doesn't work out.

I suspect that many traditional package managers are written with users in mind, and not developers, which sounds laudable; but in practice it forces the distinction between user and developer, not allowing the former to easily migrate into the latter. Nix feels written for developers, of course acknowledging that developers are also users and still want to be able to install off-the-shelf prebuilt binaries easily. The inbuilt package manager for Chicken Scheme is likewise developer-friendly, letting you directly build from arbitrary checked-out source trees into a properly installed package; my development process for most Chicken software is to run chicken-install in my in-progress sources as the first port of call to compile and run it, rather than the usual idiom in most languages of compiling and running from the source checkout then "installing" the binaries as an optional, later, step. And yet a "mere" end-user of my software can type chicken-install ugarit, and Chicken will download the latest public Ugarit release from the Internet and install it for them. If they want to join me in hacking Ugarit, they can check out the latest sources from my web site and get stuck in pretty quickly.

Secondly, the move away from compiling software ahead-of-time into distributable binaries, towards on-demand compilation of source code at run time (with caching of compiled forms, of course), means that the normal installation of some software is the source code, there is no need for an external "build environment" to convert your changes into a runnable package, and changes to the source code can be immediately used without going through any kind of build/install phase. Because this makes tinkering with the software so much easier, it can make it become a routine part of using the software, rather than something special done only by special people. The typical Emacs user will have overridden various internal functions of Emacs from within their personal configuration; although many configurations can be done without doing so, customisation through function overriding is so accessible that people routinely customise their Emacs installations in ways that the original developers didn't think (or didn't have time/energy) to add as a configurable option. I wish all open-source projects were written in such an open manner, but it will require a lot of migration away from the batch-compilation model of C, C++, and Java.

Poorly-architected existing code

This is a thorny issue; even if you can easily get into the code of your application and make the changes you want, and you understand all the tools it's built with, it can still be hard to make the changes you want because of one of a number of kinds of inherent "fragility" in the way the software is constructed.

Usually, this boils down to some variant on the idea of some information being repeated all over the code, rather than kept in one place. If your code relies on communicating between its components by using a special file, for instance, and every place where this file is read or written contains its own code to read and write this file directly, then changes such as storing the file in a different place, or adding some extra information to it, or replacing it with access to a database or something, will be difficult. You'll need to find all the places where the file is used, and individually re-write them to reflect your changes. This is laborious, and you might miss some, leading to bugs when those bits of code are run but don't reflect the changes.

However, if the mechanism of accessing this shared state (reading and writing the file) was isolated into one place in the software, with an interface that is used wherever required and reflects only the essentials of the access to the shared state, then that mechanism can easily be changed to another, as long as it still preserves those essentials, relatively painlessly and safely.

Software developers, before they even write a line of code, be thinking about how their software might be changed in future, and make sure that they split it into modules with clean interfaces, to make that easier. As side effects, it also makes their code easier to test and debug, as the interfaces serve to define and clarify the responsibilities and expectations of each module, which makes it easy to write comprehensive tests for the modules.

If this seems like hard work, then you're doing it wrong. We've all heard of code (usually in Java, for some reason) that seems to have taken the Design Patterns book as a checklist of things to do, and features pages and pages of AbstractFactoryWrappers that just indirect everything; the actual code that does the task at hand seems to be scattered thinly amongst all this framework. That's not what I mean by designing your software to be extensible. I just mean splitting it into bits with an explicit interface between each, and makings those interfaces reveal as little as possible about the workings behind them, and putting duplicated code into modules behind interfaces rather than writing the same logic more than once, rather than making it into one big ball of inter-related mud. If you're not saving time by writing software like this in the first place, or if it seems like a burden, then you need to re-think how you write code.

I think programming languages can do a lot to help us write cleaner code, too. I find that when I'm writing C and C++, it's often hard to cleanly pull bits of functionality out into other functions due to the manual memory management which complicates interfaces, and the lack of lexically-scoped first-class functions. Code written in Lispy languages tends to be a lot more easy to refactor as it grows, leading to cleaner interfaces (on average), and the automatic memory management tends to make the interfaces simpler as well.

Also, a culture of open extensibility in software means that extensibility of your code is high in the programmer's mind at all times. Developers of Emacs packages seem to expect bits of their software to be overridden, and write it accordingly.

Conclusion

I think that making programming more accessible has very many good consequences. It gives people more power to get more out of their computers. It gives people more reason to trust computers (and as we move to a more online society, people are forced to place their trust in computers in order to take part; but being forced to place your trust in something you don't trust is a harrowing experience), as they can peer inside the software to see how it works, and fix it if it doesn't. It also means that everyday users of computers have an easy, and natural, transition into learning programming, which is a very rewarding pastime; and more people contributing to open-source projects means we all get to have a better quality of life.

So, open-source software developers, I implore you to consider these points. Try to make your software open and welcoming to newcomers!