Day 2 of making the ladder (by alaric)

Yesterday, I cut out all the bits required to make myself a ladder.

This morning, I set out on foot to visit Machine Mart and Screwfix for some supplies; a set of magnetic welding clamps, some white metal paint for the completed ladder, and these anchor bolts to attach it to the wall. As you can see from the diagram on the box, they just can't wait to be buried in a wall:

Anyway, it's been a long time since I last did any welding (and I've never been great at it), so I started by just tacking a few bits together to check they line up correctly with the wall. I started with the trickiest bit, the angled spacer:

Then I proceeded to drill correctly-sized holes for my anchor bolts into the mounting flanges, using my column drill (a very useful tool). This made a lot of pretty swarf:

Then I started welding the flanges onto the spacers. The new magnetic welding clamps (the red thing in this picture) came in handy here; their function is to accurately hold things at ninety (or forty-five) degree angles:

With all six of the spacers flanged, I could start tack-welding them into place:

My welds on the flat were OK, but my welds inside corners are still pretty ropy; I had to keep chipping the slag off and going over again to fill in gaps. Another problem was that the steel square tubing has rounded corners, meaning that when a spacer has to be welded onto the upright to make a T, two sides of the end of the spacer are to be welded to the curved corner of the upright, making a big gap I had to bridge across somehow. It turns out that surface tension causes the exposed edge of the metal to pull back when it melts, so I had to bodge little slivers of metal left over from the cutting into the gap to stop this happening, resulting in some ugly welds that I may have to grind flat before I paint it - or to make the surfaces I'm going to weld the rungs onto flat enough for them to go on straight! We'll see.

Anyway, having checked the fit against the wall, I could then finish off the welds on all four sides of each junction, completing one side of the ladder. It's intentional that the flanges point in different directions, by the way - they mainly point upwards so the weight of the ladder isn't pulling them away from the wall, except that the one on the angled spacer has to be underneath so I can get in to fit the bolt, and the top one is underneath so the anchor bolt hole isn't too near the top of the wall:

At that point I ran out of time. It won't take me too long to assemble the second side as I already have all the flanges welded onto the spacers (which was quite time-consuming), and I can clamp it all onto the existing side to get the alignment correct without all the cross-checking and measuring I had to do the first time.

After that, I'll need to weld the rungs in place, tidy up the welds, scrub it down with a wire brush and sandpaper to clean off all the welding grime, degrease it with white spirit, and paint it.

Then I need to figure out how I'm going to get it into the wall! I'll need to somehow hold it in place while I mark out the drill holes, which might be tricky without an assistant. Then I have to try to drill the holes where the marks actually are (the bane of my life is carefully marking out a wall, starting to drill, and the bit hitting a stone and suddenly meandering half a centimetre off centre - then having to try and jam a screw in nonetheless, as whatever I'm screwing to the wall has holes in places that can't move to match).

After that, holding it up to the wall and slipping in the anchor bolts should be easy - then tighten them up and I'm done! They have a tightening torque recommendation on the box, so I'll finally get to use my torque wrench, and not be left wondering if I've tightened them enough or risking cracking the wall by over-tightening (which has been a concern with previous anchor bolt expiditions).

Continue to day 3...

Day 1 of making the ladder (by alaric)

This weekend, I am attempting to make a fixed ladder, mounted to the wall of my workshop so I can fix the roof more easily.

First, I had to tidy up in the workshop to make room. I bought a set of four folding chairs for guests which are rather in the way (the chairs, not necessarily the guests), So I screwed a few bits of scrap wood up between the beams to store them:

See all that damp on the ceiling, by the way? That is the enemy! Eradicating that is the long-term goal of this whole mission.

With the chairs out of the way, I could get on with the task at hand. Here's the bit of wall where the ladder will go:

And here's the pile of steel I ordered from Hindleys to make it from:

The ladder is being made mostly from 1" square steel tubing, which needs to be cut into bits of various lengths with my trusty angle grinder. Here's the rungs (longer parts) and the spacers that will join it to the wall:

However, for extra strength and extra not-having-a-sharp-corner-ness, at the bottom it will be joined to the wall by an angled spacer on each side. The spacer will be at forty-five degrees to the vertical (and to the horizontal, thanks to the magic of maths), so I need to cut twenty two and a half degree angles onto the bottoms of the ladder uprights, like so:

Also, I needed to cut out two spacers with twenty two and a half degree angles at one end (to mate with the uprights) and forty five degree angles at the ends which will mount onto the wall, bringing my pile of various lengths of steel tube up to this:

Where the tubes need to attach to the wall, I'm going to weld them to flat steel plates, which will be drilled for the bolts that join them to the wall. All of the tube ends I've cut will be welded, so the fact that they're rough angle-grinder cuts doesn't matter, but these edges will be exposed, so the fact that they are nasty and burred offends me:

But a quick run along the bench grinder each has given them nice clean edges:

Sarah turned up and took a picture of me, just as I was finishing. Check out my lovely protective clothing - I take my safety seriously when dealing with forces that can make metal flow like honey:

Finally, I measure out the layout of the angled spacer, to check I'd done my maths correctly to make it the right length to produce a 30cm spacing, to match the straight spacers, and that the angles were right:

Tomorrow, I will go out shopping to buy some anchor bolts to mount it to the wall (I can't drill the holes in the plates until I know the diameter required), welding magnets so I can make sure the angles are exact as I weld it together (I don't want a wonky ladder), and some metal paint so it won't rust when it's installed - then it's time to start welding. I plan to start by making the sides with the angled and straight spacers so I can check they align perfectly by placing them one on top of the other, before welding the rungs in between them.

Continue to day 2...

Volunteering (by alaric)

I've always wanted to change the world for the better. I'm not sure where this impulse comes from, but it's probably something to do with reading a lot of science fiction and adventure stories as a child. It's always seemed natural to me that the world is a place full of problems, and that they can be solved with a mixture of ingenuity and dtermined hard work.

You can do a lot of world-saving on your own; indeed, that's often the most satisfying kind, as you can see the direct results of your actions in isolation. But it's hard to find opportunities to do so. Problems that have a chink in them a single person can exploit and solve them are rare. Most problems are large and don't have an easy fix (even a cunning one), and a single person's effort against them would be like trying to divert the wind by waving your arms.

So the most immediately rewarding means of improving the world is to join up with others who have a similar idea on what to do, and volunteer to work together with others as a group. This might mean forming a loosely-coupled team, like the contributors of an open-source project; or it might mean joining a more formal organisation.

As it happens, I do both.

On the loosely-coupled team side, I'm involved in the Scheme programming language community, and the Chicken Scheme sect within it; and I have a bunch of open-source projects I publish on my site, Kitten Technologies. Ugarit might help to save the world by helping it to keep better backups and manage its files better, Tangle might help the world not get confused with its cabling, Simple Graphics might make a tiny contribution towards teaching future generations to do awesome things, and if I ever have the time and money to really put some effort into it, ARGON might provide the world with a vastly more awesome platform to build software systems upon.

And on the formal organisation side, I'm the Cub Scout leader for the Cub pack in the village I used to live in. During school term time, every Wednesday I spend an hour in the village hall with two dozen kids aged eight to ten inclusive, trying to broaden their minds and teach them useful life skills. They are a lucky lot, in that that's all I do for them; some Scout groups give children an escape from abuse at home or other terrible situations like that, but my lot come from relatively secure backgrounds, and are well loved, well fed and well educated. Once a year or so I organise a camping weekend, too, where they can learn a bit more self-sufficiency, and have an exciting adventure with their friends.

All of the voluntary stuff I do is very rewarding. When I think about the sorts of things I'd do if I was rich and didn't have to work, a large fraction of it is voluntary work. When I had more money in the past, I actually hired somebody to write open-source software for me, and I'd do it again if I could. I suppose my interest in wearable computers is driven more by my own interest and desire to sharpen my skills than in any benefit to the rest of the world, but I'll still be publishing my designs for others to take inspiration from or to just build their own copy! Volunteering is also a great way to gain new skills, gain confidence, and meet new people. It's my default suggestion for people who feel lonely, bored, or stuck in a rut.

I'm under no illusion that I'm some saintly figure laying down my own interests in order to help others, either. I just happen to enjoy making other people happy, and also enjoy tinkering with certain kinds of technology that happen to be very easy to share, thanks to the Internet. My contributions to the world have generally been insignificant compare to many others, and I often feel sad about that - I feel I am spending too much of my time just keeping my family safe and fed (which is my first priority), and that the kinds of things I'm good at are unfortunately only useful in narrow niches (distributed data storage technology isn't going to solve world hunger on its own). But such is life! This year I am setting aside two weekend days every month for my own projects - for January, this is going to be spent on my infrastructure rather than anything directly useful, as I need to repair the roof of my workshop as part of the ongoing process of making it into a more usable work area, but if I have any time left after that I'm going to work on Ugarit some more (on the sofa in the house - getting an Internet connection down to the workshop is a project for next month).

Volunteering can become unhealthy, however.

I'm struggling with running the Cub pack right now; I am the pack's only leader, and it's more normal for a pack to have two or three leaders. I have helpers who come along on the Wednesdays, which is invaluable as I couldn't safely run the meetings alone, but I still need to organise something every week, and then be there to run it. If I need to take a week off for some reason, I still need to either organise somebody to take over and prepare the required information for them, or contact all of the families to tell them I'm having to cancel an evening (which I really, really, don't like doing). I skirt the edge of a vicious circle - doing something because I feel I have to, and fearing the consequences if I don't, takes the fun out of it; and after executing all my other responsibilities of the day, I often lack the energy to do things I'm not fueled by enthusiasm over, so put them off. That in turn increases the unpleasant stress and pressure the next day until I end up rushing it all at the last minute!

Volunteering where you sit down and do something you feel like doing is safe. Taking on a responsibility is a little more dangerous. I am under no real obligation to keep running the Cub pack - there's no contract - but without a backup leader, if I leave, the pack will collapse, and I don't want that to happen to the lovely kids. Not to mention that I'd miss the little blighters! So I press on, providing a rather less exciting programme than I'd like. I think that it's not a good idea to take on an ongoing voluntary responsibility alone; it should be done as a group, so that the workload can be shared more easily, and when a group member is overloaded with other pressures, they can temporarily or permanently reduce their contribution without major disruption - and otherwise tedious planning and preparation can be fun when it's done as a group. However, sometimes you start doing something as a group and end up doing it alone, or (as in this case) start something with the expectation that others will join you once it picks up some momentum, but they always want to join as helpers rather than taking over from you to some extent - or, alternatively, that you're bad at delegating because you're scared of putting too much pressure on them and losing them.

Speaking of which, I need to go and prepare something for the Cub meeting tomorrow...

...actually, I can do it in my lunch break tomorrow, and go to sleep now instead, as it's forty minutes past my bedtime already. A MUCH better idea. Surely.

A day of drilling (by alaric)

Last Thursday, I had my safety induction for using the laser cutter at Bristol Hackspace, and as my test piece I laser-cut a name tag for Jean. She likes that sort of thing.

However, she requested that it have a hole in the corner, so she can attach it to her school bag. So today we went down to the workshop and I helped her to drill it out.

But I had more drilling to do. A friend asked me on Facebook how she could drill holes through pebbles. It just so happens that Sarah owns a set of diamond core drills, so I borrowed them and had a go, so that I could offer some advice.

I set the bit up in the column drill:

The challenge was in how to hold the stone still while it was being drilled. Irregular shapes are tricky to hold. First off, I tried a simple clamp:

The drilling has to happen under water, to help cool it and to wash away the dust that the stone turns into. I put some scraps of cardboard underneath so that I drilled into that once I was through the stone, rather than the bottom of my box, which would lead to it leaking all over my cluttered workbench:

As soon as the drill cut in, dust whooshed out into the water and made it impenetrably murky, so after a short drilling session, I took the stone out of the water to see what was happening:

It looked good so I tried again, but this time the stone pivoted in the clamp. I tried to clamp it back again but it wouldn't go back at the same angle and kept shifting, so I tried a new approach - gluing it to a piece of wood that was large enough to not be able to rotate inside the box, so I just needed to hold the box steady while drilling:

That worked quite well, but the vibration shook the stone loose after a while, and I had trouble with the wood wanting to float and the stone wanting to sink causing it to flip over in the water. so I glued it more thoroughly (making sure glue came over the side of the stone so it was held in place rather than just stuck in place), and glued a bit of scrap metal to the bottom to stop it floating over:

That worked; now the stone was steady, it was easy to press on all the way. I had to drill a millimetre then back out (with the drill still spinning) to clear the dust out into the water, then press on again. Progress was slow but steady, taking a few seconds to do each millimetre:

Once I felt it go through the bottom of the stone, I had no trouble in peeling the rubbery hot-melt glue back with my fingernails to free the stone. Job done:

So, to conclude:

• Use a column drill.
• Use a diamond core bit.
• Hold the stone steadily in something that conforms to its shape. An ideal technique might be to use something like Plasticene to firmly secure it to the bottom of the box before pouring the water in.
• Drill slowly, backing off to clean the bit every millimetre or so.

When I came back, Sarah was asleep on the sofa:

Aww!

Logging, Profiling, Debugging, and Reporting Progress (by alaric)

I have a hypothesis. The hypothesis is that, in software, the goals of logging stuff for audit purposes, reporting progress to a user on the course of a long-winded process, profiling a system to see where it is spending its time in the long run, and debugging certain kinds of problems (in particular, hangs of multithreaded systems come to mind, although many other kinds of problems too), can all fundamentally be met by a single piece of software infrastructure.

I say this because, largely, I write very similar code to do all four of them, and end up building unnervingly similar-looking ad-hoc solutions, despite the fact that a single well-developed tool could do all of them really well.

Basically, all of them boil down to a bit of code stating what it's about to do and what it's just done at interesting points.

Let's look at each in turn.

Software often writes, either to a log file for unattended bits of software such as daemons, or direct to the user if there is one, lines of text describing the current state of affairs. Generally, it logs when steps in the process are started (to document what it's about to do), when steps end (to document the results), and sometimes during iterations over something or other (although that's really just a special case of the previous cases, as each iteration is itself a step). This is often done for auditing reasons - to find out what operations led up to some state of the system, for a plethora of reasons - and sometimes to offer a kind of primitive progress reporting; you can look at the log to see how far something has gotten.

But we have more sophisticated progress reporting, sometimes. Rather than using a dumb logging facility to say "I'm about to process 1000 objects", then "I have processed N objects so far" periodically and letting the user read the text and do the maths to work out how far through we are, we can use a more evolved progress reporting infrastructure (usually called a "progress bar"), telling it when the process starts (displaying the bar) and ends (removing the bar from the display) and periodically updating it on progress by telling it how many steps we have to do and how many are currently done. Given that higher-level information, it can produce a more intuitive display, and even estimate the time to completion and report a rate of progress.

Although the display is rather different, there is a great commonality in the interface - we are stating the fact that a process has started (and how many steps it will take), and periodically stating how far we have done, and then confirming when we are finished. A high-level progress reporting interface could just as easily generate lines in a log file as display a progress bar on a graphical screen, from the same software interface.

I've also used logging for debugging, quite extensively. Rather than logging user-level messages that relate to the user's model of the state of the system, I can also log messages about the internal state of things "under the covers", and use that to guide my own investigations of problems caused by the internal state going awry. Sometimes I will do that by just printing those messages to a console I can observe, and removing or commenting out the code that does so when I'm not debugging; but more featureful logging infrastructures allow for "debug logging" that is hidden away unless specially requested, meaning I can leave the logging statements present in "production code" and turn them on at run-time, which can be a great boon.

Meanwhile, in profiling and debugging server applications with multiple threads handling requests in parallel, I have often used a particular technique. I've given each thread a thread-local "current status string", and then peppered my code at interesting points with a call to set the current thread's status string with a summary of what it's about to do. I can then easily ask the system to display a dump of all running threads and what they're currently doing. Java makes this easy with threads having a name string and the ability to introspect the tree of threads and thread groups; in a system written in C using different processes rather than threads, I've written my own infrastructure to do it using a shared memory segment with each process getting a fixed-sized slot therein and some lock-free atomic update mechanisms to avoid races between the processes and the sampling tool.

This lets me do two things. Firstly, when the system is grindingly slow, I can quickly see what every thread is up to. Is everyone all blocked in the same operation, all queuing for the same limited resource? Secondly, when something hangs, I can look at the state of the unresponsive threads to see what they're doing. Generally, this shows me what lock they're stalled on, and who has the lock (or, more tellingly, if nobody seems to have the lock). And finally, I can profile the system during heavy load by periodically sampling the status of each thread and then building a histogram of statuses, to see which statuses take up most of the time. However, I had to be careful with this - it worked well if the status string merely recorded what step was in progress, but not if the status string included details of the data being worked upon, because that made the strings different even if the system was at the same step, so they didn't count for the same histogram bucket. The solution was to mark such parameter data in some way (such as by always quoting it with square brackets) so that profiling tools can make themselves blind to them.

Sometimes I've run into trouble with the fact that the same procedure might be called in many different places; so the thread status shows me that I'm using that procedure (and where I am in it) but doesn't tell me why. If I was attaching a debugger I could view the entire stacktrace, but that pauses execution (which may interfere with the delicate race condition I'm hunting down) and it's fiddly to do that for every thread or process in a system to get a good snapshot of the overall state, which is why I prefer the explicit-status approach in the first place. The solution is simple: rather than replacing the entire status string, each procedure should just append their status to the existing string, and remove it when it's done. In Java I did this by capturing the current thread name and then assigning "originalName + processStep" each time, then restoring "originalName" at the end of the process; in C, I did it by recording a pointer to the end of the current string and just writing our new string there, and then setting it to "\0" at the end (being careful not to overrun the buffer!). This turned the status string into a kind of mini-backtrace, but rather than logging each and every procedure call, it only logs things considered important enough to log. When looking at a snapshot of a system with a hundred concurrent requests in progress, this is a great time-saver.

But, clearly, all of the above really just comes down to "reporting the current status of a thread of execution". All that changes is what the infrastructure does with that information. There's no reason why the same notification from the software cannot generate a line in a log file, an on-screen log message to a waiting user, the creation, updating, or removal of a progress bar, the updating of a global scoreboard on what all the threads in the system are up to, and the updating of a profiling histogram of where the system is spending its resources.

So here's my proposal for an API.

(task-start! name [args...]) => task handle

Starts a task, which is some unit of work. Tasks may be dynamically nested within a thread, opening tasks within other tasks. Returns a task handle. The name is a string describing the task, which should itself be static, but may refer to the additional arguments positionally with %<number> syntax.

(task-status! handle status [args...]) => void

Notifies the system that the given task's current status is the given status string, which again may refer to the arguments positionally with %<number> syntax.

(counted-task-start! steps unit-s unit-p name [args...]) => task handle

Starts a task with the given number of counted steps. The singular name of a step is unit-s and the plural name is unit-p, eg "byte" and "bytes". Otherwise, as per task-start!

(counted-task-status! handle steps-completed [status [args...]]) => void

Notifies the system that the given counted task's current status is the given status string, which again may refer to the arguments positionally with %<number> syntax, and that the given number of steps have been completed so far. The status and arguments can be omitted if there's really nothing more to say then the number of steps completed.

(task-end! handle) => void

Ends a task (counted or not), given its handle.

(task-fault! handle message [args...]) => void

Logs a fatal fault (internal failure within the task) of a task. The message should be a static string, referring positionally to the arguments with the usual syntax.

(task-error! handle message [args...]) => void

Logs an fatal error (invalid inputs, or invalid behaviour from a subcomponent, but not a problem with the task itself) encountered by a task. The message is as above.

(task-warning! handle message [args...]) => void

Logs a non-fatal problem encountered by a task. The message is as above.

(task-debug! handle message [args...]) => void

Logs a debug-level event encountered by a task. The message is as above.

Now, this looks like a very ordinary logging framework, with the exception of an explicit hierarchy of nested tasks and the explicit mention of "counted" tasks with a known number of steps. Yet those two additions allow for the same interface to cover all of the above goals.

Seeing how they might generate a log file for auditing is trivial. The explicit knowledge of task nesting lets us give context to common subtasks used in lots of different places, be it by something as simple as indenting each log message according to the subtask nesting depth or creating a an aggregrate "task pathname" by combining all the names of parent tasks into one long string to log.

Generating a profile is also trivial; using the task names and status strings without interpolating the arguments, we can obtain a measure of what bit of the code we're in - either in isolation, or including the entire context of parent tasks, as desired. And we can also generate histograms of the arguments for each different status if we want; if a given subtask takes widely varying amounts of time depending on how it's called, we can find out what arguments make it run slowly to narrow down the problem.

Debugging is helped by turning on the display of debug-level messages in the log, and by making a snapshot of the current status (status and name of the current task and all parent tasks) of each thread/process in the system available for introspecting the current state of the system. That's a useful thing to include in a crash log, too.

But reporting progress to the user is where having a common infrastructure really shines. Rather than needing the explicit construction of "progress dialogs" in interactive applications, the infrastucture could notice when a thread has spent more than a few seconds inside a counted task and produce on automatically. It would display the progress of the highest-level parent counted task of the current task hierarchy, as the overall progress measure of the operation in progress; but if a subtask takes more than a few seconds, then it becomes worthwhile automatically expanding the dialog to list the entire hierarchy of nested counted tasks with progress bars (the very tip of the hierarchy, that has not gained several seconds of age, should be elided to avoid spamming the user with endless wizzing progress bars for short-lived subtasks; only display subtasks that have shown themselves to take user-interesting amounts of time). And the display of non-counted subtasks and their textual statuses within the hierarchy can be turned on as an option for users who want to see "more verbose" information, perhaps along with each task then growing a scrollable log showing that task's history of statuses, warnings, starting subtasks, and so on.

How best to handle multithreaded tasks is an open question, depending really on the threading model of the language you're using. Perhaps tasks should require an explicit parent-task handle to be passed in, to make it clear what the task hierarchy is in the presence of threads; or perhaps newly created threads should inherit their parent's current task. Either way, with threading, it's possible for a task to be parent to more than one active subtask, and a progress reporting user interface will need to handle that, perhaps by dividing the space underneath the parent task into multiple vertical columns for each concurrent subtask, when that level of detail is required.

Also left unspecified is more detail on the "log level" concept; I've just suggested a few (fault, error, warning, status update (called "notice" or "info" in most existing logging systems) and debug, but the system really needs to make some intelligent decision as to who to notify. A server-based interactive application given a fault condition really needs to apologise to the user in vague terms, while sending a detailled crash log and raising an alarm to the system's administrators, for instance. And more complex systems composed of multiple components may have fairly complex rules about who should be notified about what level of event in what context, which I've not even begun to worry about here...

But, to conclude, I think it's a shame that there's so many very different bits of infrastructure that have very similar interfaces and are used in similar ways by software, but with very different goals. I think it'd be great to create a single progress-reporting interface for applications to use, and make different goals into pluggable components beneath that interface!