More speed, lower velocity

I frequently meet software teams who describe themselves as “high velocity”, they even have graphs coming from Jira to prove it, and yet their ability to ship great software, to delight their customers, or even to attract their customers, doesn’t meet their expectations. A little bit of sleuthing usually discovers the underlying problem.

Firstly, let’s take a look at that word, “velocity”. I, like Kevlin Henney, have a background in Physics, and therefore I agree with him that Velocity is a vector, and has a direction. But “agile” velocity only measures amount of stuff done to the system over time, not the direction in which it takes the system. That story may be “5 points” when measured in terms of heft, but is that five points of increasing existing customer satisfaction? Five points of new capability that will be demoed at next month’s trade show? Five points of attractiveness to prospects in the sales funnel?

Or is it five points of making it harder for a flagship customer to get their work done? Five points of adding thirty-five points of technical debt work later? Five points of integrating the lead engineer’s pet technology?

All of these things look the same in this model, they all look like five points. And that means that for a “high-velocity” (but really low-velocity, high-speed) team, the natural inclination is to jump on it, get it done, and get those five points under their belt and onto the burn down chart. The faster they burn everything down, the better they look.

Some of the presenting symptoms of a high-speed, low-velocity team are listed below. If you recognise these in your team, book yourself in for office hours and we’ll see if we can get you unstuck.

  • “The Business”: othering the rest of the company. The team believes that their responsibility is to build the thing that they were asked for, and “the business” needs to tell them what to build, and to sell it.
  • Work to rule: we build exactly what was asked for, no more, no less. If the tech debt is piling up it’s because “the business” (q.v.) doesn’t give us time to fix it. If we built the wrong thing it’s because “the business” put it at the top of the backlog. If we built the thing wrong it’s because the acceptance criteria weren’t made clear before we started.
  • Nearly done == done: look, we know our rolling average velocity is 20 bushels of software, and we only have 14 furlongs and two femtocandela of software to show at this demo. But look over here! These 12 lumens and 4 millitesla of software are in QA, which is nearly done, so we’ve actually been working really hard. The fact that you can’t use any of that stuff is unimportant.
  • Mini-waterfall: related to work to rule (q.v.), this is the requirement that everyone do their bit of the process in order, so that the software team can optimise for requirements in -> software out and get that sweet velocity up. We don’t want to be doing discovery in engineering, because that means uncertainty, uncertainty means rework, and rework means lower velocity.
  • Punitive estimation: we’re going to rename “ambiguity” to “risk”, and then punish our product owner for giving us risky stories by boosting their estimates to account for the “risk”. Such stories will never get scheduled, because we’ll never be asked to do that one risky thing when we can get ten straightforward things done in what we are saying is the same time.
  • Story per dev: as a team, our goal is to shovel as much software onto the runtime furnace as possible. Therefore we are going to fan out the tasks to every individual. We are each capable of wielding our own shovel, and very rarely do we accidentally hit each other in the face while shovelling.

Figurative Programming and Gloom: the [G]raphical [LOOM]

Donald Knuth is pretty cool. One of the books he wrote that I own and have actually read[*] is Literate Programming, in which he describes (among other things) weaving program text and documentation together in a single narrative.

Two of his books that I own and have sort of dipped into here and there are TeX: the Program, and METAFONT: the Program. These are literate programs, created from webs in which Human text and Computer text are interleaved to tell the story of what the program does.

Human text and computer text, but not images. If you want pictures, you have to carry them around separately. Even though we are highly visual organisms, and many of the programs we produce have significant graphical components, very few programming environments treat images as anything other than external files that can be looked at and maybe previewed. The only programming environment I know of that lets you include images in program source is TempleOS.

I decided to extend the idea of the Literate web to the realm of Figurative Programming. A gloom (graphical loom) web can contain human text, computer text, and image descriptions (e.g. graphviz, plantuml, GLE…) which get included in the human-readable document as figures.

The result is gloom. It’s written in itself, so the easiest way to get started is with the Xcode project at gloomstrap which can extract the proper gloom sources from the gloom web. Alternatively, you can dive in and read the PDF it made about itself.

Because I built gloomstrap first, gloom is really a retelling of that program in a Figurative Programming web, rather than a program that was designed figuratively. Because of that, I don’t really have experience yet of trying to design a system in gloom. My observation was that the class hierarchy I came up with in building gloomstrap didn’t always lend itself to a linear storytelling for inclusion in a web. I expect that were I to have designed it in noweb rather than Xcode, I would have had a different hierarchy or even no classes at all.

Similarly, I didn’t try test-firsting in gloom, and nor did I port the tests that I did write into the web. Instinct tells me that it would be a faff, but I will try it and find out. I think richer expressions of program intention can only be a good thing, and if Figurative Programming is not the way in which that can be done, then at least we will find out something about what to do instead.

[*] Coming up in January’s De Programmatica Ipsum: The Art of _The Art of Computer Programming_, an article about a book that I have _definitely_ read _quite a few bits here and there_ of.

Two books

A member of a mailing list I’m on recently asked: what two books should be on every engineer’s bookshelf? Here’s my answer.

Many software engineers, the ones described toward the end of Code Complete 2, would benefit most from Donald Knuth’s The Art of Computer Programming and Computers and Typesetting. It is truly astounding that one man has contributed so comprehensively to the art of variable-height monitor configurations.

If, to misquote Bill Hicks, “you’ve got yourself a reader”, then my picks are coloured by the fact that I’ve been trying to rehabilitate Object-Oriented Design for the last few years, by re-introducing a couple of concepts that got put aside over the recent decades:

  1. Object orientation; and
  2. Design.

With that in mind, my two recommendations are the early material from that field that I think shows the biggest divergence in thinking. Readers should be asking themselves “are these two authors really writing about the same topic?”, “where is the user of the software system in this book?”, “who are the users of the software system in this book?”, and “do I really need to choose one or other of these models, why not both or bits of both?”

  1. “Object-Oriented Programming: an evolutionary approach” by Brad Cox (there is another edition with Andrew Novobilski as a co-author). Cox’s model is the npm/CPAN model: programmers make objects (“software ICs”), describe their characteristics in a data sheet, and publish them in a catalogue. Integrators choose likely-looking objects from the catalogue and assemble an application out of them.

  2. “Object-Oriented Software Construction” by Bertrand Meyer. Meyer’s model is the “software engineering” model: work out what the system should do, partition that into “classes” based on where the data should naturally live, and design and build those classes. In designing the classes, pay particular attention to the expectations governing how they communicate: the ma as Alan Kay called the gaps between the objects.

Packaging software

I’ve been learning about Debian Packaging. I’ve built OS X packages, RPMs, Dockerfiles, JARs, and others, but never dpkgs, so I thought I’d give it a go.

My goal is to make a suite of GNUstep packages for Debian. There already are some in the base distribution, and while they’re pretty up to date they are based on a lot of “default” choices. So they use gcc and the GNU Objective-C runtime, which means no blocks and no modern objc features. The packages I’m making are, mostly thanks to overriding some choices in building gnustep-make, built using clang, the next-generation objc runtime, libdispatch etc.

The Debian packaging tools are very powerful, very well documented, and somewhat easy to use. But what really impressed me along this journey was CPack. I’ve used cmake before, on a team building a pretty big C++/Qt project. It’s great. It’s easier to understand than autoconf, easier to build correct build rules over a large tree than make, and can generate fast builds using ninja or IDE-compatible projects for Xcode, IntelliJ and (to some extent) Eclipse.

What cpack adds is the ability to generate packages, of various flavours (Darwin bundles, disk images, RPMs, DEBs, NullSoft installers, and more) from the information about the build targets. That’s really powerful.

Packaging software is a really important part of the customer experience: what is an “App Store” other than a package selection and distribution mechanism? It irks me that packaging systems are frequently either coupled to the target environment (Debian packages and OpenBSD ports are great, but only work in those systems), or baroque (indeed autoconf may have gone full-on rococo). Package builder-builders give distributors a useful respite, using a single tool to build packages that work for any of their customers.

It’s important that a CD pipeline produces the same artefacts that your customers use, and also that it consumes them: you can’t make a binary, test it, see that it works, then bundle it and ship it. You have to make a binary, bundle it, ship it, install it, then test it and see that it works. (Obviously the tests I’m talking about here are the “end-to-end”, or “customer environment” tests. You don’t wait until your thing is built to see whether your micro-tests pass, you wait until your micro-tests pass to decide whether it’s worth building the thing.)

I know that there are other build tools that also include packaging capabilities. The point is, using one makes things easier for you and for your customers. And, it turns out, CMake is quite a good choice for one.

Concurrent objects and SCOOP

Representing concurrency in an object-oriented system has been a long-standing problem. Encapsulating the concurrency primitives via objects and methods is easy enough, but doesn’t get us anywhere. We still end up composing our programs out of threads and mutexes and semaphores, which is still hard.

Prior Art

It’s worth skimming the things that I’ve written about here: I’ve put quite a lot of time into this concurrency problem and have written different models as my understanding changed.

Clearly, this is a broad problem. It’s one I’ve spent a lot of time on, and have a few satisfactory answers to if no definitive answer. I’m not the only one. Over at codeotaku they’ve concluded that Fibers are the right solution, though apparently on the basis of performance.

HPC programs are often based on concurrent execution through message passing, though common patterns keep it to a minimum: the batch processor starts all of the processes, each process finds its node number, node 0 divvies up the work to all of the nodes (a message send), then they each run through their part of the work on their own. Eventually they get their answer and send a message back to node 0, and when it has gathered all of the results everything is done. So really, the HPC people solve this problem by avoiding it.

You’re braining it wrong, Graham

Many of these designs try to solve for concurrency in quite a general way. Serial execution is a special case, where you only have one object or you don’t submit commands to the bus. The problem with this design approach, as described by Bertrand Meyer in his webinar on concurrent Object-Oriented Programming, is that serial execution is the only version we really understand. So designing for the general, and hard-to-understand, case means that generally we won’t understand what’s going on.

The reason he says this is so is that we’re better at understanding static relationships between things than the dynamic evolution of a system. As soon as you have mutexes and condition locks and so on, you are forced to understand the dynamic behaviour of the system (is this lock available? Is this condition met?). Worse: you have to understand it holistically (can anything that’s going on at the moment have changed this value?).

Enter SCOOP

Meyer’s proposal is that as serial programs are much easier to understand (solved, one might say, if one has read Dijkstra’s A Discipline of Programming) we should make our model as close to serial programming as possible. Anything that adds concurrency should be unsurprising, and not violate any expectations we had if we tried to understand our program as a sequential process.

He introduced SCOOP (Simple Concurrent Object-Oriented Programming) developed by the Concurrency Made Easy group at ETH Zürich and part of Eiffel. Some of the design decisions he presented:

  • a processor is an abstraction representing sequential execution
  • there is a many-to-one mapping of objects to processors (this means that an object’s execution is always serial, and that all objects are effectively mutexes)
  • where an object messages another on a different processor, commands will be asynchronous (but executed in order) and queries will be synchronous
  • processors are created dynamically and opportunistically (i.e. whenever you create an object in a “separate” and as-yet unpopulated domain)

An implementation of this concurrency model in Objective-C is really easy. A proxy object representing the domain separation intercepts messages, determines whether they are commands or queries and arranges for them to be run on the processor. It inspects the objects returned from methods, introducing proxies to tie them to the relevant processor. In this implementation a “processor” is a serial operation queue, but it could equivalently be a dedicated thread, a thread pulled from a pool, a dedicated CPU, or anything else that can run one thing at a time.

This implementation does not yield all of the stated benefits of SCOOP. Two in particular:

  1. The interaction of SCOOP with the Eiffel type system is such that while a local (to this processor) object can be referred to through a “separate” variable (i.e. one that potentially could be on a different processor), it is an error to try to use a “separate” object directly as if it were local. I do not see a way, in either Swift’s type system or ObjC’s, to maintain that property. It looks like this proposal, were it to cover generic or associated types, would address that deficiency.

  2. SCOOP turns Eiffel’s correctness preconditions into wait conditions. A serial program will fail if it tries to send a message without satisfying preconditions. When the message is sent to a “separate” object, this instead turns into a requirement to wait for the precondition to be true before execution.

Conclusions

Meyer is right: concurrent programming is difficult, because we are bad at considering all of the different combinations of states that a concurrent system can be in. A concurrent design can best be understood if it is constrained to be mostly like a serial one, and not require lots of scary non-local comprehension to understand the program’s behaviour. SCOOP is a really nice tool for realising such designs.

This is something I can help your team with! As you can see, I’ve spent actual years understanding and thinking about software concurrency, and while I’m not arrogant enough to claim I have solved it I can certainly provide a fresh perspective to your team’s architects and developers. Book an office hours appointment and let’s take a (free!) hour to look at your concurrency problems.

A little challenge

A little challenge today: create a JS function that turns its arguments into a list of pairs. Actually, the brief was “using Ramda” but I ended up not doing that:

function basePairwise(xs) {
  if (xs.length == 0) return [];
  if (xs.length == 1) return [[xs[0], undefined]];
  return [[xs[0], xs[1]]].concat(basePairwise(xs.slice(2)));
}

function pairwise(...xs) {
  return basePairwise(xs);
}

One of the nice things about JavaScript (I won’t say that often, so note the date) is the introspective nature: the fact that I get to just look at the way a function was called, rather than saying “you must use exactly these types and this arity always”. Here, I’ve done that with the JS spread operator: I could have used the arguments pseudo-list for only a little more work.

Ultimate Programmer Super Stack: Last day!

I already wrote about the Ultimate Programmer Super Stack, a huge bundle of books and courses on a range of technologies: Python, JS, Ruby, Java, HTML, node, Aurelia… and APPropriate Behaviour, my book on everything that goes into being a programmer that isn’t programming.

Today is the last day of the bundle. Check it out here, it won’t be available for long.

Coming to terms with fewer terms

I was on a “Leadership in Architecture” panel organised by RP International recently, and was asked about problems we face using new techniques like Microservices, serverless and machine learning in the financial technology sector. The biggest blocker I see is the RFP (Request for Proposals), RFI (Request for Information), the MSA (Master Service Agreement), any document with a three-letter acronym. We would do better if they disappeared.

I’m fully paid up in my tithe to the church of “customer collaboration over contract negotiation”, and I believe that this needs to extend beyond the company boundary. If we’re going to spend a few months going back and forth over waving our certifications about, deciding who gets to contact whom within what time, and whether the question they asked constitutes a “bug report” or a “feature request”, then I don’t think it matters whether the development team use two-week sprints or not. We’ve already lost.

We’ve lost because we know that the interactions between the people involved are going to be restricted to the terms agreed during that negotiation. No longer are people concerned about whether the thing we’re making is valuable; they’re concerned with making sure their professional indemnity insurance is up to date before sending an email to the DRI (Definitely Responsibility-free Inbox).

We’ve lost because we had a team sitting on its hands during the negotiation, and used that time “productively” by designing the product, putting epics and stories in a backlog, grooming that backlog, making wireframes, and all of those other things that aren’t working software.

We’ve lost because each incompatibility between the expectation and our intention is a chance to put even more contract negotiation in place, instead of getting on with making the working software. When your RFI asks which firewall ports you need to open into your DMZ, and our answer is none because the software runs outside of your network on a cloud platform, we’re not going to get into discussions of continuous delivery and whether we both read the Phoenix Project. We’re going to get into discussions of whether I personally will warrant against Amazon outages. But here’s the thing: we don’t need the software to be 100% up yet, we don’t even know whether it’s useful yet.

Here’s an alternative.

  1. We, collectively, notice that the software we make solves the problem you have.
  2. We, collectively, agree that you can use the software we have now for a couple of weeks.
  3. We, collectively, discuss the things that would make the software better at solving the problem.
  4. We, collectively, get those things done.
  5. We, collectively, GO TO 2.

Notice that you may have to pay for steps 2-4.

Microservices for the Desktop

In OOP the Easy Way, I make the argument that microservices are a rare instance of OOP done well:

Microservice adopters are able to implement different services in different technologies, to think about changes to a given service only in terms of how they satisfy the message contract, and to independently replace individual services without disrupting the whole system. This […] sounds a lot like OOP.

Microservices are an idea from service-oriented architecture (SOA) in which each application—each microservice—represents a distinct bounded context in the problem domain. If you’re a movie theatre complex, then selling tickets to people is a very different thing from showing movies in theatres, that are coupled loosely at the point that a ticket represents the right to a given seat in a given theatre at a given showing. So you might have a microservice that can tell people what showings there are at what times and where, and another microservice that can sell people tickets.

People who want to write scalable systems like microservices, because they can scale different parts of their application separately. Maybe each franchisee in the theatre chain needs one instance of one service, but another should scale as demand grows, sharing a central resource pool.

Never mind all of that. The real benefit of microservices is that they make boundary-crossing more obvious, maybe even more costly, and as a result developers think about where the boundaries should be. The “problem” with monolithic (single-process) applications was never, really, that the deployment cost too much: one corollary of scale is that you have more customers. It was that there was no real enforcement of separate parts of the problem domain. If you’ve got a thing over here that needs that data over there, it’s easy to just change its visibility modifier and grab it. Now this thing and that thing are coupled, whoops!

When this thing and that thing are in separate services, you’re going to have to expose a new endpoint to get that data out. That’s going to make it part of that thing’s public commitment: a slightly stronger signal that you’re going down a complex path.

It’s possible to take the microservices idea and use it in other contexts than “the backend”. In one Cocoa app I’m working on, I’ve taken the model (the representation in objects of the problem I’m solving) and put it into an XPC Plugin. XPC is a lot like old-style Distributed Objects or even CORBA or DCOM, with the exception that there are more safety checks, and everything is asynchronous. In my case, the model is in Objective-C in the plugin, and the application is in Swift in the host process.

“Everything is asynchronous” is a great reminder that the application and the model are communicating in an arm’s-reach fashion. My model is a program that represents the domain problem, as mentioned before. All it can do is react to events in the problem domain and represent the changes in that domain. My application is a reification of the Cocoa framework to expose a user interface. All it can do is draw stuff to the screen, and react to events in the user interface. The app and the model have to collaborate, because the stuff that gets drawn should be related to the problem, and the UI events should be related to desired changes in the domain. But they are restricted to collaborating over the published interface of the XPC service: a single protocol.

XPC was designed for factoring applications, separating the security contexts of different components and giving the host application the chance to stay alive when parts of the system fail. Those are valid and valuable benefits: the XPC service hosting the model only needs to do computation and allocate memory. Drawing (i.e. messaging the window server) is done elsewhere. So is saving and loading. And that helps enforce the contract, because if I ever find myself wanting to put drawing in the model I’m going to cross a service boundary, and I’m going to need to think long and hard about whether that is correct.

If you want to talk more about microservices, XPC services, and how they’re different or the same, and how I can help your team get the most out of them, you’re in luck! I’ve recently launched the Labrary—the intersection of the library and the laboratory—for exactly that purpose.

Ultimate Programmer Super Stack

There’s a great bundle of polyglot learning taking place over at the Ultimate Programmer Super Stack. My book, APPropriate Behaviour – the things every programmer needs to know that aren’t programming – is featured alongside content on Python, Ruby, Java, JS, Aurelia, Node, startups, and more.

The bundle is just up for a week, but please do check it out: for not much more than you’d probably pay for APPropriate Behaviour you’ll get a whole heap of stuff that should keep you entertained for a while :).