Who cares about user interfaces? (by alaric)
Where we could be
When a user uses a tool to solve a problem, there's some difficulty inherent in the problem; we call this "essential complexity". For instance, if you want to produce a design for a house, you've got to make the decisions inherent in designing a house; and hopefully the tool can reduce that work by providing useful automation or other forms of amplification of human capability. But every tool also adds complexity, in the form of the user needing to understand the tool as well as understanding their problem. A lever can help you move objects which are heavier than you could lift - but you need to understand leverage, and think about finding a strong enough fulcrum in a suitable place to exert the force you need. We call this added tool-understanding complexity "accidental complexity".
Computers are no different, but we've become so accustomed to handling their accidental complexity that we've started to become blind to it.
Users should be able to interact with computer systems on their own terms, as easily as possible. I think that goal can expanded into a number of more specific goals:
The user should control how things are represented to them. For instance, if the user wants their user interface to use their favourite font, that should be their choice. This is nice for everybody, but it's very important for people with differing abilities, such as dyslexia or poor vision. And it should extend beyond mere stylistic choices to entirely different shapes of user interface, including ones that the creator of the tool is unaware of: screen readers, touch-only interfaces for the blind to use in silence, experimental interfaces based on optic nerve implants, you name it.
The creator of a tool should not be able to restrict how the user uses it, except maybe where required by safety considerations (where the design of road vehicles might make it harder to cause damage with them, for instance).
A user should be able to automate the use of any tool. If you can do something by hand with a tool, you should be able to write a program that can do that thing just as easily. And that program should be able to automate tasks involving multiple tools, even if they're made by enemies.
Software developers are users too, and the difference between them should be blurred. This is implied by the previous point, but requires further elaboration. This is not to say that professional programmers shouldn't exist; people who can work in the problem-domain of computers and mathematics to write tools like programming languages, user-interface infrastructure, cryptography, data compression, and device drivers will be needed, and people who have honed software development as a career will produce better software faster than a dabbling amateur - but the distinction should be no more significant than that. Somebody solving a problem based entirely in a non-technical problem domain should be able to learn to write programs to help them as easily as possible. There will always be some accidental complexity in learning a programming tool, but minimising it should be at the forefront of the minds of the people building those tools. And anybody should be able to migrate from that to being a "professional programmer" by just incrementally learning more about computer science and how computers work and building their experience; there shouldn't be a hard distinction between "serious programming languages" and "scripting languages for beginners".
Tools should integrate where possible, even without the user having to do any programming. Any object, concept, bit of data, or other "thing" that a user would identify as a "thing" within one tool that could arguably be considered as the same kind of thing, or an overlapping kind of thing, as something in another tool should be totally interchangeable. To put that rather vague and general statement in context: if a messaging tool requires you to pick a person to send a message to, and a tool for reading documents lists the authors of a document, you should be able to drag an author from a document into the messaging app to send them a message. To a certain extent this does hold today, due to things like email addresses having a widely known format, but it should apply for all kinds of things; ingredients from recipes to grocery shopping tools, voltages from datasheets and sensors to circuit simulation tools, and so on.
Users should have control over where the things (data, documents, etc) they work on with their tools are (in whatever kind of metaphor of "place" the user interface provides), and how they are organised, and how they are shared; so they can group them to suit their style of working. The management of storage should NOT be under the control of the application - the user's right to choose where things are stored is essential to the user being able to control how they share things (by putting them on removable media, or putting them in secure places, or in public places), and to organise things by putting things created by different applications together in folders grouped by project or year or "In progress" vs "Done" or whatever they want.
Users should be able to look inside things (data, documents, AND tools) to learn how they work, learn how to automate them, and learn from them to make similar kinds of thing themselves. In other words, there should be a View Source option on everything.
How can we get there?
I think that the above imply that the focus should move away from the tools, and towards the problem-domain things the tools operate upon. Users shouldn't have to think of their stuff as being "in" certain applications (other than where said applications are explicitly storage/management tools rather than processing tools); this is not only directly implied by goal 6, but also indirectly by goals 1, 2 and 5.
It also implies that tools should be small, modular, things with expressive problem-domain interfaces exposed to the infrastructure, more like a command-line tool or an API than a desktop application that's just a single entry point that, when invoked, draws a GUI into a window. This enables different user interfaces to interpret the problem-domain interface of the tool in a way appropriate to the user interface itself as per goal 1, and also enables users to automate things are per goal 3, and helping provide a foundation for goal 4.
It implies that there should be some underlying model of "things" in a computer that's practical to implement in terms of the raw 1s and 0s of the computer, while also being easy to manipulate (supporting goals 3 and 4), and easy for non-specialist users to understand (supporting goals 4 and 7). The Unix "everything is a text file" approach was a start in this direction, but it was imperfect and inconsistently applied; things like compiled executables and multimedia data never fit into the model, and lines of text are insufficient for many problem domains, leading to complex things like XML and JSON being built on top. We should have a representation for things that's general enough to represent everything, while avoiding the accidental complexity of syntax and matching parentheses.
And it implies there should be at least one programming language that's easy to pick up the basics of, without requiring boilerplate to write "hello world", while also being suitable for professional-scale programming. It should be seamlessly connected to the common data model, and the interfaces exposed by other tools should be trivially usable within it. Users should be able to write code within whatever kind of context the user interfaces provides - in other words, if you're looking at some things on your screen, you should be able to bring up a programming window and be able to reference those things from your code trivially: by name, or by dragging and dropping them, or clicking on them, or any of the above - depending on your user interface (or user preferences). You should be able to interactively enter, run, and debug some code for a one-off job, and/or save it as a persistent "thing" in your storage system to use it again later. To combine it all together: if a bit of software tells you a current temperature in your room, another bit of software lets you control a heater, and a third thing tells you the current price of electricity - the user should be able to write something barely more complicated than Once a minute, do this: if and only if the temperature_in_room < 30°C and price_of_electricity < 25p/kWh, then the heater should be on.
Does this require everything to be open source? Not necessarily - the user interface engine itself could be closed-source, as could any tools. Users would still be able to write their own software using their exposed interfaces, but they wouldn't be able to look inside those existing bits of software to learn how to make their own, or clone-and-modify things to customise them. Also, there would be a temptation for user interface engines to become commercially important means of control of the entire ecosystem, just like browsers have today; vendors would add new features to their user interfaces and encourage tools to use them, while making it hard for other user interfaces to use them. We don't want the way tools interact with the user interface to enter the same spiral of complexity that browsers have, leading to fewer and fewer implementations - it needs to be simple enough for there to be plenty of implementation to cover everyone's needs and preferences, and easy for people to make their own! So I think we need to be very careful to avoid commercial software taking over the user interface. It's not worked well in the browser or smartphone worlds, nor in the desktop software world before that (although the user interface there is just part of a wider kernel interface so other issues are also at play).
If we can establish such a foundation, we can build user interfaces, applications, and data storage/management/communication tools on top of it, and connect them in arbitrary combinations.
That's a very long way from the current worlds of desktop, mobile, and Web app development. But if we got there, life would be so much easier for users (who would have much more control) and for application developers (who can just implement the functionality specific to their problem, without needing to build user interfaces and basic data management/representation infrastructure).
But that's not to say that such a system can't incorporate the advantages of Web apps, and it could even develop incrementally from current software systems. Defining standards for data representation, and for software tools to explain what they provide as an API, and what inputs they require to do that, and programming language integrations to make those things seamlessly usable from code, can all be done on top of the existing infrastructure. Those standards and integrations will be more complicated if built on top of HTTP and POSIX and so on than if they were built from scratch, but they can be done; and if they take off, then implementations of them in terms of simpler substrates could be made in future, and migrated onto without breaking compatibility.
If that existed, it would be possible to build user interfaces on top of contemporary platforms, that expose those things to the user.
But we face a "collective action problem"; until that ecosystem of tools and interfaces exists, there's no motivation for individual software developers to build for it. To make it work would require a concerted group effort to work on all the parts of an initially usable community of interoperable software. How can we kickstart such a thing? I doubt commercial software development would be terribly interested in it in its own right, and the open source community would be held back by bickering about the details of how to do it, combined with a great deal of "How things are now is fine, actually" sentiment. I could only see it getting started in some world where there isn't already an incumbent world of software to compete with. We might have had this when smartphones and tablet computers appeared on the scene, creating a new market for software where existing software couldn't just be easily ported to a tiny touchscreen; but while some progress was made, it wasn't much.
Perhaps we will have a new user-interface hardware platform, such as VR or wearables or brain implants, that will need new software - and, somehow, we avoid the trap that smartphones fell into of creating a new bespoke software ecosystem coupled to that platform, rather than trying to create a shared one. Or perhaps the infrastructure could be built as a platform for developing bespoke software in vertical niches, and be used in enough niches that they start to overlap and become a cohesive community that starts to threaten the mainstream.
I can't see that happening any time soon, however, so we're probably stuck with incremental improvements for now. Which is a shame - but there's so much scope for incremental improvement, at least!
Tue 8th Jul 2025 7:39 pm 
Subscribe