In which new developer tools are dull

Over on lobste.rs I said that I don’t hold out much hope for another “blue plane” style event in developer tools. In one of Alan Kay’s presentations, he referred to the ordinary way of things as the pink plane, and incremental advances in the state of affairs being movements in that plane. Like the square in Edwin Abbot’s Flatland that encounters a sphere, a development could take us out of the pink plane into the (orthogonal) blue plane. These blue plane ideas are rare because like the square, it’s hard to even conceive of life outside the pink plane.

In what may just be a surprising coincidence, Apple engineers used Blue and Pink to refer to features in evolutionary and revolutionary developments of their operating system.

Software engineering tooling is, for the majority of developers, in a phase of conservative retreat

Build UIs on the web and you probably won’t use a graphical builder, you’ll type HTML and JavaScript (and maybe JSX) into a text editor.

Build native apps and even where there is a GUI builder, you’ll find people recommending against its use and wanting to do things “programmatically” (by which they mean “through typing”, even though the GUI builder tools are another way to construct a program).

In the last couple of decades, interest in CASE tooling has shrunk to conservative interest in text editors with some syntax highlighting, like vim or Atom. Gone even is the “build and run” button from IDEs, to be replaced with command-line invocations of grunt tasks (a fancy phrase meaning shell scripts), npm scripts (a fancy phrase meaning shell scripts) or rake tasks (you get the idea).

Where previously there were live development environments embedded in the deployment environment (and the Javascript VM is almost perfectly designed for that task), there is now console.log and unit tests. The height of advanced interaction with your programming tools are the REPL (an interactive shell) and the Playground/InstaREPL (an interactive shell that echoes stdin and stdout in different places).

For the most part, and I say that to avoid the inevitable commenter who thinks that a counterexample like LabView or Mathematica or that one person they met who uses Expression Blend renders the whole argument broken, developers have doubled down on the ceremony of programming: the typing of arcane text into an 80×24 character display. Now to be fair, text is an efficient and compact graphical representation of a linear sequence of connected concepts. But it is not the only one, nor the most efficient nor most compact, and neither are many software systems linear.

The rewards in making software to make software are scarce.

You can do like IntelliJ do, and make a better version of the 80×24 text entry thing. You can work for a platform vendor, and make their version of the 80×24 thing. You can go and get an engineering grade 6 or above job in Silicon Valley and tell your manager that whatever it is their business does, you’re going to focus on the 80×24 thing (“at scale”) instead.

What you don’t seem to be able to do is to disrupt the 80×24 thing. It’s free (at least as in beer), it’s ubiquitous, and whether or not it’s as good as it could be it certainly seems to be good enough for the people who not only get paid to make bad software, but get paid again to fix it.

Bottom-up teaching

We’re told that the core idea in computer programming is problem-solving. That one of the benefits of learning about computer programming (one that is not universally accepted) is gaining the skill of problem decomposition.

If you look at real teaching of computing, it seems to have more to do with solution composition than problem decomposition. The latter seems to be background noise: here are the things you can build solutions with, presumably at some point you’ll come across a solution that’s the same size and shape as one of your problem components though how is left up to you.

I have many books on programming languages. Each lists the features of the language, and gives minimally complex examples of the use of those features. In that sense, Kernighan and Ritchie’s “The C Programming Language” (section 1.3, the for statement) is as little an instructional in solving problems using a computer as Eric Nikitin’s “Into the Realm of Oberon” (section 7.1, the FOR loop) or Dave Thomas’s “Programming Elixir” (section 7.2, Using Head and Tail to Process a List).

A course textbook on bitcoin and blockchain (Narayanan, Bonneau, Felten, Miller and Goldfeder, “Bitcoin and Cryptocurrency Technologies”) starts with Section 1.1, “Cryptographic hash functions”, and builds a cryptocurrency out of them, leaving motivational questions about politics and regulation to Chapter 7.

This strategy is by no means universal: Liskov and Guttag’s “Program Development in Java” starts out by describing abstraction, then looks at techniques for designing abstractions in Java. Adele Goldberg and Alan Kay described teaching Smalltalk by proposing exploratory projects, designing the objects that model the problem under consideration and the way in which they will communicate, then incrementally filling in by designing classes and methods that have the desired properties. C.J. Date’s “An Introduction to Database Systems” answers the question “why databases?” before introducing the relational model, and doesn’t introduce SQL until it can be situated in the context of the relational model.

Both of these approaches, and their associated techniques (the bottom-up approach and solution construction; the top-down approach and problem decomposition) are useful; the former leads to progress and the latter leads to understanding. But both must be taken in concert, because understanding without progress leads to the frustration of an unsolved problem and progress without understanding is merely the illusion of progress.

My guess is that more programmers – indeed whole movements, when we consider the collective state of things like OOP, functional programming, BDD, or agile practices – are in the “bottom-up only” group than in the “top-down only” or “a bit of both” groups. That plenty more copies of Introduction to Programming in [This Week’s Hot Language] have been sold than Techniques for Making Your Problem Amenable to Computation. That the majority of software really does comprise of solutions looking for problems.

Why I don’t have a favourite programming language

This is my take on Ilya Sher’s similar post, though from a different context. He is mainly interested in systems programming, I have mostly written user apps and backend services, and also some developer tools.

I originally thought that I would write a list of the languages and difficulties I have with them, but I realised that there’s an underlying theme that can be extracted. Programming languages I have used either have too much vendor dependence (I love writing ObjC, but can’t rely on GNUstep when I’m not on Apple), too little interaction with the rest of the software world (I love writing Pharo, but don’t love going through its FFI to use anything else) or, and this is the biggest kicker, I don’t like the development environments.

When I work on JavaScript, my environment is a text editor (something like VSCode or emacs) that has syntax highlighting, maybe has auto-completion…and that’s about it. When I work in something like Java, ObjC or C++, I have a build button, an integrated debugger, and the ability to run tests. And, if I’m lucky, a form designer. When I work in something like Swift or Clojure, I have insta-repls. When I work in Pharo, I have all the live browsers and things you hear about from smug people, but I still have to type code for things you might expect to be ‘live’ in such an environment. I get confused by the version control tools, but that might be because I’m not familiar with image-based development.

It feels like, details of the languages aside, there’s a synthesis of programming language with environment, where the programming language is a tool integrated into the environment just like the compiler and debugger, and the tools are integrated into the programming language, like the Lisp macro system. It feels like environments like Oberon, Lisp machines and Smalltalks all have some of this integration, and that popular programming environments for other languages all have less of it.

I’m not entirely sure what the ideal state is, and whether that’s an ideal just for me or would benefit others. I wrote my MSc thesis on an exploration of this problem, and still have more research to do.

Free Software should welcome contributions by Apple, Google

It started with a toot from the FSF:

Freedom means not #madebygoogle or #madebyapple, it means #madebythousandsoffreesoftwarehackers #GNU

This post is an expansion on my reply:

@fsf as an FSF Associate I’m happy to use software made by Google or made by Apple as long as it respects the four freedoms.

Yes to made by Google or made by Apple

The Free Software Foundation financially supports the Replicant project, a freedom-respecting operating system based on the Android Open Source Project. The same Android Open Source Project that’s made by Google. Google and Apple are both behind plenty of Free Software contributions, both through their own projects such as Android and Swift or contributions to existing projects like the Linux kernel and CUPS. Both companies are averse to copyleft licences like the GPL, but then both companies have large software patent portfolios and histories of involvement in software patent litigation so it may be that each company is actually averse to compromising the defensibility of their patent hoards through licences like GPL3. On the other hand, the Objective-C support NeXT created for GCC was the subject of an early GPL applicability test so in Apple’s case they could well be averse to “testing” the GPL any further.

Whatever their motivations for the stances they’ve taken, Apple and Google do contribute to Free Software and that should be both encouraged and welcomed. If they want to contribute to more projects, create new ones, or extend those freedoms to their existing proprietary code then we advocates of software freedom should encourage them and welcome them. Freedom does not mean “not #madebygoogle or #madebyapple”.

No to controlled by Google or controlled by Apple

While we in software development have never had it so good in terms of software freedom, with all of our tools and libraries being published as free software (usually under the banner of open source), the community at large has never had it so bad, and Google and Apple are at the vanguard of that movement too. The iOS kernel, Darwin UNIX system and Swift programming language may all be open for us to study, share and improve, but they exist in a tightly-controlled walled garden that’s eroding the very concept of ownership and centralising all decisions within the spheres of the two platform providers. This means that even Freedom Zero, the freedom to use the software for any purpose, is denied to anyone who isn’t a programmer (and in fact to the rest of us too: you can study the iOS kernel but cannot replace the kernel on your phone if you make an improvement; you can study Swift but cannot sell an iOS app using any version other than the one blessed by Apple at time of submission).

People often complain at this point that software freedom is only relevant to programmers because you need to be a programmer to study or improve a program given its source code, but that’s not the point. Open Source is only relevant to programmers. Having the freedom to use your computer for any purpose, and to share your software, gives two things:

  1. to some people, “I wish that my software could do this, it doesn’t, but I understand that it is possible to change it and that I could use the changed version” can be the incentive to learn and to enable their own programming skills.
  2. to others, having the freedom to share means having the freedom to share the software with someone who already knows how to program who can then make improvements and share them back with the first person.

Ignoring those possibilities perpetuates the current two-tier system in which programmers have a lot of freedom and everybody else has none. I have argued against the walled garden before, as a barrier to freedom. That is different from arguing against things that are made by the companies that perpetuate the walled gardens, if we can encourage them to change.

Welcome, Apple. Seriously.

The FSF has a long history of identifying itself “against” some IT incumbent, usually Microsoft. It has identified a change in the IT landscape by positioning itself as an underdog “against” Apple and Google. But it should not be against them, it should be with them, encouraging them to consider and support the freedom of their customers.

Recommend me some books or articles

I’ve been looking for something to read on these topics, can you help?

  • a history of the Unix wars (the ‘workstation’ period involving Sun, HP, Apollo, DEC, IBM, NeXT and SGI primarily, but really everything starting from AT&T up to Linux and OS X would be interesting)
  • a business case study on Apple’s turnaround 1997-2001. I’ve read plenty of 1990s case studies explaining why they’ll fail, and 2010s interpretations of why they’re dominant, and Gil Amelio’s “On the Firing Line” which explains his view of how he stemmed the bleeding, but would like to fill in the gaps: particularly the changes from Dec 1997 to the iPod.
  • a technical book on Mach (it doesn’t need to still be in print, I’ll try to track it down): I’ve read the source code for xnu, GNU Mach and mkLinux, Tevanien’s papers, and the Mac OS X Internals book, but could still do with more