Garbage collection (by alaric)

One of the interesting little problems in converting a CPU, some RAM, and a bunch of I/O devices into a useful system is memory management.

Your everyday PC with 512MB of RAM contains about 512 million little memory cells, each of which contains an eight-digit binary number. This is where all running applications, and all the information they are dealing with, must be stored; programs, text, images, and whatnot are all encoded as sequences of these eight-digit numbers (known as bytes) and stuffed into memory.

The problem is - where to put everything? After all, when you double click on a program, the computer looks at the size of the program on disk, and needs that many bytes of memory to load the program into. How does it know which bits are free and which are in use? Different computers have different amounts of memory, and may be running any combination of apps already, so there's no way to reserve a given bit of memory for a particular program; the system has to keep track of what's in use and what isn't in real time. And when you load your app, it will start asking for bits of memory to keep track of the windows it has open, to store your document in, and so on.

Now, the first problem to be solved is how to keep track of which bits of memory are in use and which aren't, in such a way that the computer can efficiently find a block of free memory of a specified size - that problem is harder than it may seem, especially when you consider that multiple threads of execution will be requesting memory at once. But that's not the problem I was pondering as I sat on the train today.

My problem is how to figure out when a block of memory isn't used any more, so that it can be handed back to the system that keeps track of free blocks and reused.

Runtime code generation (by alaric)

Spurred by a thread on comp.os.research, I've been reading up on the Synthesis OS design.

Wiki page about Synthesis

The original thesis

This looks like good work, insipring for ARGON; with HYDROGEN explicitly exposing runtime code generation primitives, like any good FORTH system, and the CHROME metaprogramming stuff is based around run time code generation.

I'd hoped to look at how to optimise the system using runtime generation for common stuff - but I expected to have to experiment later; luckily, this fine fellow has blazed a trail for me!

However, I can't help but feel this is more of a programming language issue than an operating systems issue. The author has written an OS in assembly language, and implemented much of it by generating specialised code on the fly rather than jumping to the general code that then has to analyse its dynamic context every time it's executed to decide which control path to take, and having to load data indirectly rather than having it as an immedate value. Surely this stuff should be generalised into a way of optimising all code, rather than just being used in an operating system? I don't see the fundamental technique as being particularly bound to operating systems - lots of applications could benefit from it.

For ARGON, I'd like to use the metaprogramming facilities I've already planned for CHROME to build a toolkit that makes runtime specialisation optimisation easier. Maybe even look into making it happen automatically, although I suspect that the tradeoff involved may require human intervention at heart. A lot of the benefits could be had anyway by just compiling the code in a custom environment containing the constants, which will be constant-propogated and constant-folded and the used as immediate values - but I think those cunning executable data structures need more thinking about...

ARGON node diagram (by alaric)

Since I'm always rethinking parts of it faster than I can write them up, ARGON lurks to a great extent inside my own head, with the Web site lagging behind a little. This just helps to make it all the more confusing for interested folks, so I've bitten the bullet and produced a shoddy diagram showing how all the services can fit together on a single running node.

http://www.argon.org.uk/argon-node.png.

Here, we see how the HYDROGEN hardware abstraction layer is used by various components to support WOLFRAM, for in-cluster management of shared state; MERCURY, for inter-cluster and in-cluster communication and FLUORINE for interoperability with other network protocols. These act as interfaces to the entity handler user code - your application - invoking your entities to handle incoming network requests, and providing an API for your code to make outgoing network requests.

I've shown how real-time user tasks will be executed by the HELIUM scheduler in response to timers or incoming signals, and how they interact with local hardware to perform their work.

And I've shown how the NEON user interface subsystem uses the human interface hardware APIs provided by HYDROGEN to proxy between the user and the CARBON directory (for browsing) and MERCURY to then interact with entities to request that they provide code for a user interface front end, which is then executed locally by CHROME in order to provide a responsive user experience for accessing remote resources.

Hopefully, this will make my rantings make a lot more sense to people...

CHROME and IRON data typing (by alaric)

IRON schemas and the CHROME type system are intertwined... more accurately, I'd say the latter was a superset of the former.

Let me give a few examples of my requirements for the system. Read more »

ARGON cryptography (by alaric)

I thought I'd like to discuss the design decisions behind the rather brief description of the "security levels" mentioned on the MERCURY page. Don't worry, you don't need to go and read it first - I'll duplicate everything in this post.

To summarise, my design requirement was to implement a cryptograhic architecture for both communications and storage of sensitive data, which allows for:

1. Both parties in a communication may have differing security requirements; the maximum of both need to be met
2. Security levels need to be specified in a way that is future proof; eg, DES would have been unbreakable in World War II, but nowadays it should only be considered suitable for relatively unimportant data
3. Physical communications links may have a certain element of external protection, rendering encryption an unnecessary burden; a cluster of servers locked in a highly screened server room together should be able to communicate sensitive data between each other in the clear, or with a fast lightweight encryption algorithm - after all, an assailant in the room with them could easily interfere with the machines themselves to get them to divulge their keys. However, the same information being shared across the public Internet would need heavy encryption, or be banned altogether.

4. Communications links and data storage devices might have a maximum security level of information they can be trusted to at all, no matter how heavily encrypted it is, because any crypto algorithm is potentially breakable.

   Read more »