Pass The Conch (by alaric)

In The Lord of the Flies, the children (marooned on an island and working out how to organise themselves to survive) develop a technique for managing debate: they use a conch shell as a token to represent who currently holds the floor. Without the conch, you can't talk; you have to wait your turn.

Cut to the Real World of Commerce and Industry: in various places I've worked, there's been a number of shared resources which can only be used by one person or agent at a time. Mainly, these have been testing servers - if you are doing performance analyses, or looking for timing-related bugs, you can't have anyone else running jobs on the same server as you, or they'll compete for resources and interfere with your results. Or perhaps there's only one "data area" of some kind, and two attempts to use it at once will lead to catastrophe.

This is usually handed by asking around the office or in IRC: "Is anyone using X?", hoping that anybody who is is still around (as opposed to too busy doing something to notice the request, or leaving a job going while out to lunch). Because of the unreliability of this system, and the inability to integrate it with automatic systems that need to claim resources (such as automatic test systems), I have often wished for a software tool to manage it. Which would, naturally, be called "conch".

Here's my feature wishlist:

• Network-based. A central conch server tracks a set of conches, accessed via a Web interface or a direct protocol. The direct protocol should have a command-line client for scripting, and be trivial to write native client libraries for in programming languages.

• Authenticated. No need for super security, but we want to keep out casual mischief-makers, so require authentication to use the server; to enable easy integration with other workflow apps, support htaccess files, "trust the upstream proxy" (eg, accept HTTP auth usernames and ignore any passwords sent), or running an arbitrary shell command to validate a username/password pair. It might be used across the public Internet, so allow for SSL wrapping the connection. The command line client should, by default, use the username and password from ~/.conch or prompt for them (and save them in ~/.conch) if not specified.

• The ability to create or delete resources, to claim a currently-free resource, to release a resource you hold, or to "force" the release of a resource that somebody else holds (if they forget and go home, etc).

• The ability to list the status of a resource, to list the resources held by a specific user, to list the resources held by yourself, to list all users with resources held, and to list all resources.

• Fine-grained access control (per-user rights limitation) might be handy, but probably not useful for the first draft.

• An IRC bot might be cool - at least for logging resource claims/releases and commands to list current state; maybe for resource claims/releases as well, if users are either trusted by nick or authenticate via a private message.

If this doesn't already exist, it should be easy to build (something could be knocked up with awful and sql-de-lite in a day or two, I bet!)...

Cloud Storage (by alaric)

Currently, you can go to various providers and buy online storage capacity (IMHO, rsync.net is best, after research I did to find an offsite backup host for work). It's more expensive than a hard disk in your computer, and miles slower, but it has one brilliant advantage: it's remote. So it's perfect for backups.

And that's the heart of a free market - storage is cheap to the cloud providers (they just buy disks, and in bulk at that), but their storage has more value to you than your own storage because of it's remoteness. So they can rent it to you at a markup, and you get a benefit, and everyone is happy. Money flows, the economy grows, and one day we'll get to have affordable space tourism et cetera.

But large, centralised, cloud storage providers are attractive targets for people who want to steal data. They become centralised points of failure; if they go bankrupt, lots of people lose their backups. Therefore, it's smart to do your backups to more than one of them, just in case. But that means setting up your systems to talk to each one's interfaces, arranging payment and agreeing to terms and conditions with them all individually, and so on.

Surely this state of affairs can be improved? With ADVANCED TECHNOLOGY?

Well, I think it can, and here's how. Read more »

Lords of a new economy (by alaric)

Pondering Bitcoin, I recently opined:

Who sets the difficulty of the puzzle and all that? The computers in the network do - when the system was created, rules were agreed, and written into the software. As everyone runs software following those rules, anybody solving easier puzzles or trying to award themselves more bounty for doing so will have their bounty-claiming transaction rejected as invalid. To loosen the rules, a majority of the computers in the system will all need to accept the new rules - so it will require consensus from the community.

I've been thinking more about this. Read more »

AURUM (by alaric)

My recent thoughts about bitcoin reminded me of earlier thoughts I'd had about digital currency.

Cryptographic digital currency is a way of transferring value without trusted third parties being involved in every transaction, but within a closed domain, it's easier to go for a trusted party and cut out all the crypto maths. Which is why we have printer credits managed in a central database when we use a shared printer. We may use a digital currency to buy a credit, but once we have credits, we're happy for the owner of the printer to just store our balance in a database and decrement it whenever we print.

And within a company, complex processes are used to transfer money in and out of the company's actual bank account, but budgets within departments are usually allocated by just asking somebody to update a spreadsheet. Money moves within the company using easier, faster, simple methods than bank transfers, writing cheques and letting them clear, or exchanging cryptographic keys.

It's the same story for "ulimit" mechanisms in computer operating systems, and language-level sandboxes, that allocate budgets of things like CPU time and memory space to software running in a computer.

So, when I set out to design AURUM, the resource limit system for ARGON, I decided to make a unified abstraction across all of the above. A process has a budget, which contains arbitrary amounts of arbitrary resources; and it can subdivided that budget into sub-budgets.

That's just an accounting system, though. It needs to integrate with actual resource managers. For something like CPU time, for efficiency, the scheduler probably wants a nice simple machine word reserved for "jiffies in the budget" attached to a process context in a hardcoded way. So the AURUM system probably needs a handler for "run out of jiffies" that takes more from the actual AURUM budget and "prepays" them into the process context - and when the process' balance is requested, knows to ask what's been prepaid and not yet used, so it can report honestly. If the process is stopped, any remaining balance needs to be put back in to be re-allocated to the parent process' budget. And so on.

Similarly, interfaces to actual electronic banking (spend money in a budget by causing an actual bank transfer, or bitcoin transaction, or whatever to happen), and interfaces for incoming budgets from external sources (a bank account interface that fires off a handler when an incoming payment is detected - with that payment as the handler's budget so it can then allocate it appropriately), and so on, can be built.

And a power-constrained mobile device might have joule budgets - and operations such as driving motors, transmitters, and lights might use them up. That would be neat for handheld computers and deep space probes, which can then run less-trusted code in a sandbox with controlled access to expensive resources.

That's all well and good as a way to manage finite resources in a system, but the next level is to take a step back, look at the system as a whole, and see how this facility can be used to do other cool stuff.

This leads naturally to the semi-forgotten discipline of Agoric computing which seeks to make marketplaces and auctions a core tool to solve resource allocation problems. This has scope within an ARGON cluster, if it's shared between multiple organisational units, which can then use budgets purely within AURUM to manage their shared use of the computer resources and to contribute towards its upkeep accordingly.

But, more excitingly, with mechanisms like Bitcoin allowing for money to be transferred across trust boundaries, it starts to become practical to think about allowing our computers to participate in economies between them. What if my desktop PC and servers rented out their space disk space, CPU time, and bandwidth to all comers? And with the money they accumulated from doing so, in turn rented offsite disk space for their backups, and when I gave them a particularly tough job to do, hired some extra CPU and bandwidth to do it, dynamically? Without me having to hand-hold it all as the middleman, pulling my debit card out to pay for resources... If I wanted to do lots of resource-intensive work I might put more money in from my own pocket to give it more to hire extra resources with; if I tend to under-use my system and it makes profits from renting out spare capacity, then I could take cash from it from time to time.

I guess the first step would be to create standard protocols in AURUM for things like auctions and commodity markets, to facilitate transferring between different 'currencies' such as CPU time, bitcoin, fiat currencies, printer credits, disk space, and the like. And a standard interface to bank accounts, where balances and transaction histories can be queried, and transfers requested. A bank account in the context of AURUM would be a third party that holds control of some budget on your behalf, so it should look like an ordinary budget in every way possible. That would make it practical to have software that needs a given resource to find a way, through a registry of trusted markets, to convert them into the resources it wants.

That'd be neat...

Bitcoin security (by alaric)

I've been learning about Bitcoin lately.

It's an electronic currency. I've seen electronic currency before - in the late 90s there were efforts to create them based on virtual banks issuing coins. The coins were basically long random serial numbers which, along with a statement of the value of the coin, were then signed by the bank. The public key of the bank is published, so people can check they're valid coins issued by the bank. The idea was that rather than withdrawing a bunch of notes from the bank, you can ask the bank to mint you a bunch of these signed numbers instead; and anyone who sees them can check their value, and eventually, return them to the bank (which can also check their value in the same way) to get their account credited.

Read more »