Why are networks so hard to build? (by alaric)

One of my sidelines is network management.

Often, the problem is this: you have a bunch of sites, each with zero or more external connections out to the wider Internet (or to people who you provide an Internet connection to), and each with zero or more computers that need some level of network connection (be they servers or workstations). Each computer needs to be able to talk to some subset of the other computers, and maybe able to talk to computers out on the Internet or some other external network, and maybe computers on the Internet or some other external network are able to talk to it. And computers may be on public IP addresses, or on a private IP address; in the latter case, if it can talk to other external networks there needs to have a public IP address (possibly shared with others) that its connections are NATed from, and if incoming connections are allowed, there must be a public IP to which those connections are sent to be "forwarded" into the private IP. We can think of those NAT/forwarding public IPs as "virtual IPs", which don't correspond to a physical computer, but seem to by way of some form of port/address translation.

Also, each computer or external network connection needs some level of reliability. Some have low requirements, and we can happily tolerate perhaps up to a day of outage per year; that's mere 99.7% uptime. The fabled "five nines uptime", 99.999%, equates to a maximum of about 30 minutes of downtime a year. And that downtime isn't just used up by equipment failures; if your network's requirements grow and you need to upgrade things to provide more capacity, you might need some downtime to replace and reconfigure things.

In other words, the problem domain is already complex. But the fun's just starting.

Read more »

R7RS (by alaric)

There has been recent discussion on r6rs-discuss about the r7rs draft charters, most of it arguing from various camps.

I want a Scheme that lets me apply advanced programming language techniques - lightweight Higher-order functions and Hygienic macros rather than Boilerplate code, Continuations rather than a fixed set of predefined Control flow mechanisms, symbols rather than Enumerated types, Functional programming rather than getting tangled with too much state, dynamically-scoped parameters rather than God objects - to my day-to-day tasks. I'm a professional programmer; for a living, I've written code in Java, C, C++, PHP, Perl, Python, Ruby, SQL, AWK, shell and JavaScript, and I'd love to have been able to use Scheme for all of the above. I'm limited more by the usual commercial pressure than by any technical issues with Scheme or the qualities of my favourite implementation, Chicken, so my wishes for R7RS are relatively minor in terms of changing the semantics of the language. What I really want is a Scheme report that will unit the Scheme community, so we can continue to have a wide array of innovative implementations that all have their own strengths and weaknesses - but with much better portability of libraries between them, so they really do start to feel like one language with multiple implementations rather than separate languages.

So I feel that things like module systems and access to networking needs to be standardised, so each implementation doesn't gratuitously have their own syntax for doing the same thing. But these things need to be optional, so implementations are not constrained to be large in order to earn the name "R7RS Scheme".

So I thought I'd step up and propose a solution.

Read more »

Why C++ is not my favourite language (by alaric)

Yesterday, I had a discussion with somebody who's a big fan of C++. He spoke of C++ as letting you write high-level code (with smart pointers, exceptions, destructors and templates conspiring to smartly remove lots of the normal drudgework of C programming), while still being able to perform low-level memory access like C if you want to, and getting highly optimal code. He admits that C++ is complex and takes a lot of time to master, but that you can produce highly elegant and compact code once you've mastered it.

But I think this is broken.

Read more »

n2n revisited (by alaric)

I have spoken before about n2n, the peer-to-peer VPN tool that makes it easy to create efficient virtual networks.

Normal VPN products are really more of a "virtual private cable" than a "virtual private network" - they just establish a point-to-point link over the Internet, requiring a login to set it up and encrypting the traffic. This means you can have a virtual connection to a real private network somewhere; and if a few people connect into that network via VPN links, then there really is a virtual private network between you all, but all going through a central point where all your links meet.

While with n2n, everyone connects to a shared "supernode" that keeps a list of who is connected to the VPN, and from where; then when you want to connect to somebody else, you use the list from the supernode to establish a direct encrypted connection between yourself and them, rather than going through any central point. So it's an actual virtual network out of the box. You can even have more than one supernode running, so that any one can fail; all the supernode does is to provide the directory service.

Also, you don't need to maintain a database of user logins; a supernode can carry any number of virtual networks. When you connect to the supernode, you just tell it the name of the community you want to join, and it will share your connection details with anybody else in the same community - you can make communities up on the fly rather than needing to maintain a central list. Access control is handled by the simple fact that you need to know the correct encryption key for the community you want to join, or your messages will be received garbled by everyone else, and ignored.

Anyway, for a long time, I wanted to get into n2n, but I couldn't as it didn't compile out of the box on NetBSD; but a desire for a better VPN solution at work has led to me getting it working. It wasn't that much work, in the end, as the existing FreeBSD support already had a BSD approach to things.

n2n is distributed via Subversion, so they don't have version tarballs - this is a problem for my NetBSD port. So I decided to mirror it into git with git svn, then forked it as "Kitten n2n", made my NetBSD port, tagged a release, pushed it to github, uploaded a tarball from that tag, and then made a NetBSD package of net/kitten-n2n.

I'll tinker with it for a few more days, then I'll submit it to the NetBSD folks for consideration.

I'll keep pulling in from the official n2n Subversion repo, to pull down patches, and I'll see if they'd like my patches pushed up - as well as NetBSD support, there's a few things I'd like to fix as well (I've spotted passing an integer through a void* by casting, which is slightly dodgy practice and produces warnings on my 64-bit machine, but is easily fixed by passing a pointer to a heap-allocated copy of the integer!)

Designing a global knowledge base (by alaric)

Continuing from my previous posts on HYDROGEN and IRON, I suppose the next thing I can talk about is CARBON.

One thing that users expect out of a system is some form of navigational structure. All but the simplest embedded computer systems have some kind of menu structure; while workstations often have several somewhat confusingly overlaid structures (menus in applications, a "My Documents" hierarchy full of personal stuff, a "Start Menu" or "Applications Folder" full of system-wide resources, and a "My Computer" hierarchy with things like removable drives and their contents, printers, access to the internals of the system, mounted network drives, and the like; then, via a Web browser, a hierarchy of bookmarks and then whatever navigational structures different Web sites out in the world present to you.

So, I clearly needed some concept of "large-scale organisation of resources" in ARGON. After much deliberation, I came up with CARBON. But, as usual, I like to kill several birds with one stone.

Read more »