Essence and accident in language model-assisted coding

Posted on 2025-11-24 by Graham

In 1986, Fred Brooks posited that there was “no silver bullet” in software engineering—no tool or process that would yield an order-of-magnitude improvement in productivity. He based this assertion on the division of complexity into that which is essential to the problem being solved, and that which is an accident of the way in which we solve the problem.

In fact, he considered artificial intelligence of two types: AI-1 is “the use of computers to solve problems that previously could only be solved by applying human intelligence” (here Brooks quotes David Parnas), to Brooks that is things like speech and image recognition; and AI-2 is “The use of a specific set of programming techniques [known] as heuristic or rule-based programming” (Parnas again), which to Brooks means expert systems.

He considers that AI-1 isn’t a useful definition and isn’t a source of tackling complexity because results typically don’t transfer between domains. AI-2 contains some of the features we would recognize from today’s programming assistants—finding patterns in large databases of how software has been made, and drawing inferences about how software should be made. The specific implementation technology is very different, but while Brooks sees that such a system can empower an inexperienced programmer with the experience of multiple expert programmers—“no small contribution”—it doesn’t itself tackle the complexity in the programming problem.

He also writes about “automatic programming” systems, which he defines as “the generation of a program for solving a problem from a statement of the problem specifications” and which sounds very much like the vibe coding application of language model-based coding tools. He (writing in 1986, remember) couldn’t see how a generalization of automatic programming could occur, but now we can! So how do they fare?

Accidental complexity

Coding assistants generate the same code that programmers generate, and from that perspective they don’t reduce accidental complexity in the solution. In fact, a cynical take would be to say that they increase accidental complexity, by adding prompt/context engineering to the collection of challenges in specifying a program. That perspective assumes that the prompt is part of the program source, but the generated output is still inspectable and modifiable, so it’s not clearly a valid argument. However, these tools do supply the “no small contribution” of letting any one of us lean on the expertise of all of us.

In general, a programming assistant won’t address accidental complexity until it doesn’t generate source code and just generates an output binary instead. Then someone can fairly compare the complexity of generating a solution by prompting with generating a solution by coding; but they also have to ask whether they have validation tools that are up to the task of evaluating a program using only the executable.

Or the tools can skip the program altogether, and just get the model to do whatever tasks people were previously specifying programs for. Then the accidental complexity has nothing to do with programming at all, and everything to do with language models.

Essential complexity

Considering any problem that we might want to write software for, unless the problem statement itself involves a language model then the language model is entirely unrelated to the problem’s essential complexity. For example, “predict the weather for the next week” hides a lot of assumptions and questions, none of which include language models.

That said, these tools do make it very easy and fast to uncover essential complexity, and typically in the cursed-monkey-paw “that’s not what I meant” way that’s been the bane of software engineering since its inception. This is a good thing.

You type in your prompt, the machine tells you how absolutely right you are, generates some code, you run it—and it does entirely the wrong thing. You realize that you needed to explain that things work in this way, not that way, write some instructions, generate other code…and it does mostly the wrong thing. Progress!

Faster progress than the old thing of specifying all the requirements, designing to the spec, implementing to the design, then discovering that the requirements were ambiguous and going back to the drawing board. Faster, probably, even than getting the first idea of the requirements from the customer, building a prototype, and coming back in two weeks to find out what they think. Whether it’s writing one to throw away, or iteratively collaborating on a design[*], that at least can be much faster now.

[*] Though note that the Spec-Driven Development school is following the path that Brooks did predict for automatic programming (via Parnas again): “a euphemism for programming with a higher-level language than was presently available to the programmer”.

Posted in AI, software-engineering, tool-support | 2 Comments

Tony Hoare and negative space

Posted on 2025-11-12 by Graham

The Primeagen calls it Negative-Space Programming: using assertions to cut off the space of possible programs, leaving only the ones you believe are possible given your knowledge of a program’s state at a point.

Tony Hoare just called it “logic”, which is why we now call it Hoare Logic. OK, no he didn’t. He called it An axiomatic basis for computer programming, but that doesn’t make the joke work. The point is, this is how people were already thinking about program design in 1969 (and, honestly, for a long time before). I’m glad the Primeagen has discovered it, I just don’t think it needs a new name.

Imagine you know the program’s state (or, the relevant attributes of that state) at a point in your program, and you can express that as a predicate—a function of the state that returns a Boolean value—P. In other words, you can assert that P is true at this point. The next thing the computer does is to run the next instruction, C. You know the effect that C has on the environment, so you can assert that it leaves the environment in a new state, expressed by a new predicate, Q.

Tony Hoare writes this as a Hoare Triple, {P}C{Q}. We can write it out as a formulaic sentence:

Given P, When C, Then Q

Given-when-then looks exactly like a description of a test, because it is; although our predicates might not be specific states for the program to be in as in a specific unit test, they might describe sets of states for the program to be in, as in a property-based test.

We can also use the fact that the computer runs program statements in sequence to compose these triples. Given P, when C, then Q; given Q, when D, then R; given R and so on. Now, if we arrange our program so that it only runs C when P is true, and it always runs D next, and so on, we can use the composition rule to say something about the compound statement C;D;etc. If we wrap that compound statement up in a function f(), then we get to say this:

If the caller of f() makes sure that the precondition for calling the function is true before making the call, then f() guarantees that the postcondition is true after the call returns.

This combination of responsibilities and guarantees is a contract. If you start with the contract and work to design an implementation that satisfies the contract, then you are designing by contract. You’re following a discipline of programming, as Dijkstra put it.

Oh, speaking of Dijkstra. We said that you need to know that the computer always runs this part of our program in the sequential way, with the stated precondition, that means it isn’t allowed to jump partway into it with some other, different initial state. If we allowed that then we’d have a much harder time constructing our P at a point in the program, because it would be whatever we think it is if the program follows its normal path, or whatever it is if the program goes to this point from over here, or whatever it is if the program goes to this point from over there, and so on. And our Q becomes whatever we think it is if the computer runs C after the normal path, or whatever it is if the computer goes to this point from over here then runs C, and so on.

Thinking about all of that additional complexity, and persevering with designing a correct program, could be considered harmful. To avoid the harm, we design programs so that the computer can’t go to arbitrary points from wherever the programmer wants. All to support logically-designed programming, or “negative-space programming” as I suppose we’re meant to call it now.

Posted in history, software-engineering | Tagged | Leave a comment

When did people favor composition over inheritance?

Posted on 2025-11-06 by Graham

The phrase “favor composition over inheritance” has become one of those thought-terminating cliches in software design, and I always like to take a deeper look at those to understand where they come from and what ideas we’re missing if we just take the phrase at face value without engaging with the discussion that led to it.

This is one of those aphorisms with a definite origin story (compare with Aphorism Considered Harmful for an example where the parts have clear origins but the whole does not): it’s the second object-oriented design principle in the Design Patterns book from the “Gang of Four” Gamma, Helm, Johnson, and Vlissides. Well, sort of:

Favor object composition over class inheritance.

It comes at the end of a page and a half of justification, and actually just before another page and a half on delegation (an extreme example of object composition), so really in the middle of a three-page discussion. This contrasts inheritance as a “white box” form of reuse, because the inheriting class has full visibility over the implementation details of the inherited class; with composition as a “black box” form of reuse, because the composing object only has access to the interface of the constituent object.

That’s certainly true of Smalltalk objects, and Smalltalk is one of the Gang’s example implementation languages. But even a modestly more recent language like Java has visibility attributes that let a class control what its subtypes can view or change, meaning that any modification in a subclass can be designed before we even know that a subtype is needed. Looking at this the other way, we could also say that languages like Smalltalk and Python that have advanced runtime introspection let a composing object access internal state of its constituent. This part of the argument then is contextually and historically situated, and depends on designers playing nicely within the intent of object designs, even where those aren’t enforced by a language.

Another part is more compelling: inheritance is defined statically at compile time and comes with language support, which makes it easier to use than composition and harder to change. Composition is manually arranged by a programmer assigning the constituent object to a member field and calling its methods in the composing object’s implementation; which is more work but easier to change at runtime. Assign a different object to the field, and get new behavior.

Further, the classes are (assuming the polite agreement above is honored) related only by the public interface of the constituent object, so there’s no implementation dependency. The system’s design depends on the relationships between objects at runtime, rather than the inheritance tree defined at compile time. This is presented as an advantage, which we need to consider in the context of the modern preference to rely more on the compiler as a correctness checker and static analysis tool, and to avoid deferring decisions to runtime that might turn out to be buggy.

Bear in mind that a year earlier, Barbara Liskov and Jeanette Wing proposed this principle:

What does it mean for one type to be a subtype of another? We argue that this is a semantic question having to do with the behavior of the objects of the two types: the objects of the subtype ought to behave the same as those of the supertype as far as anyone or any program using supertype objects can tell.

Within this context, the preference for composition is liberating to the designer: their type isn’t morally a subtype of the thing they’re extending, so they don’t need to restrict themselves to a compatible interface. And indeed, Liskov had already made this point in 1987 in Data Abstraction and Hierarchy, with reference to polymorphic types:

Using hierarchy to support polymorphism means that a polymorphic module is conceived of as using a supertype, and every type that is intended to be used by that module is made a subtype of the supertype. When supertypes are introduced before subtypes, hierarchy is a good way to capture the relationships. The supertype is added to the type universe when it is invented, and subtypes are added below it later.

If the types exist before the relationship, hierarchy does not work as well[…] An alternative approach is to simply allow the polymorphic module to use any type that supplies the needed operations. In this case no attempt is made to relate the types. Instead, an object belonging to any of the related types can be passed as an argument to the polymorphic module. Thus we get the same effect, but without the need to complicate the type universe. We will refer to this approach as the grouping approach.

Liskov goes on to say that when the relationship is identified early in design, “hierarchy is a good way to express the relationship. Otherwise, either the grouping approach…or procedures as arguments may be better”.

That points to a deficiency in the “composition over inheritance” aphorism: those aren’t the only two games in town. If you have procedures as first-class types (like blocks in Smalltalk, or lambdas in many languages), then you might prefer those over composition or inheritance.

Posted in history, ooa/d, OOP | 5 Comments

LLMs and reinforcement learning

Posted on 2025-10-10 by Graham

My reflection on the Richard Sutton interview with Dwarkesh Patel was that it was interesting how much the two participants talk past each other, and fail to find common ground. Particularly that they couldn’t agree on the power of reinforcement learning, when it’s such a large part of the LLM workflow.

To be clear, it isn’t the large language model that engages in reinforcement learning, it’s the person who’s applying the LLM to their task. That’s all that prompt engineering is. Here’s the workflow:

  1. Identify a goal.
  2. Hypothesize a prompt that leads the LLM to satisfy the goal.
  3. Try out the prompt and generate an outcome.
  4. Observe the gap between the outcome and the intended goal.
  5. Repeat steps 1-4 (yes, include step 1, as you might refine the goal or the prompt) until the gap becomes acceptably small.

This process—commonly known in its isolated, bureaucratized form as “science”—is fundamental to the learning experience. It’s the same as John Dewey’s description of reflective thought, as Honey and Mumford’s learning cycle, as the Shewhart cycle (Plan-Do-Check-Act). Start at step four, and it’s the same as Boyd’s Observe-Orient-Decide-Act loop. And it’s fundamental to how LLM-assisted work unfolds. But because the LLM is incapable of manifesting the cycle, it’s left to the person to do all the learning.

Posted in AI | Leave a comment

Prompting software or supporting engineering

Posted on 2025-10-07 by Graham

As we learn to operate these new generative predictive transformers, those of us in the world of software need to work out what we’re doing it for. The way in which we use them, the results we get—and the direction the tools develop—changes dramatically depending on this worldview.

One option is that we’re augmenting or supporting software engineering. Perhaps asking a language model to explain how code works, or getting it to investigate whether there are test cases we haven’t covered, or identifying ambiguities in a user story, or getting it to fill in some boilerplate code that would bore us if we wrote it ourselves.

Another option is that we’re generating software using natural language prompts. Perhaps asking a language model to create an app, or generate an integration between two services, or create a website to achieve some goal.

These are (at least for the moment) very different things.

Posted in AI | Leave a comment

Unintended consequences

Posted on 2025-08-28 by Graham

As the shift in content of this blog has made abundantly clear, for the last five years I’ve been doing a PhD. I’ve also been working full-time, so that research and study has basically taken up all of my spare time. I would wake up early, sit at my computer until it was time to start work, then change computer, then at the end of the work day switch back to the university-issued computer. I knew this was having an adverse impact on my health. Particularly, I could feel that I had far less stamina. Then I got the wake-up call I needed.

My fitness tracker sent me a notification at the beginning of August that my VO2 max—the measure it uses to gauge cardiorespiratory fitness—had dropped to 28.6. The average value for a man of my age is 38, and 28 is the average value for a man over 60. Low cardio fitness, according to the app, is a predictor of various heart diseases, colon cancer, type 2 diabetes, and even dementia and Alzheimer’s.

I’ve been on holiday for about half of the time since then, but my holidays tend to be active as they involve lots of walking and dancing (I go to festivals). On workdays, I’ve added a ten-minute HIIT workout to my morning, and a half-hour walk at lunchtime. Just those changes has seen the (estimated) cardio value in my tracker go up to 30.2 in three weeks. This is still in the low category—31.0 is the boundary to the “below average” category—but it’s a trend in the right direction.

I’ve always defined my “success” by my intellectual contribution to society. But I need to stay healthy to continue making that contribution, and it got past time to take control of my health.

Posted in advancement of the self | Leave a comment

Is Foundation a Utopian vision?

Posted on 2025-07-11 by Graham

Important: while I’m only talking about the Foundation books in vague details here, I will end up summarising a number of key points through the whole series. If you haven’t read them, and intend to, I recommend not reading this post, yet.

Here, at the start of the third TV series of Foundation, I’m actually talking about the series of short stories and novels on which the TV show is based. The TV series takes its leave of the source material in many ways—most of them necessary to produce something watchable in 2025. For example, you don’t get a female character in the novels for a very long time, and you have to wait until the second book to encounter one that Asimov names. Also, atomic ashtrays no longer represent the pinnacle of technological advancement.

The Foundation series of books starts with a collection of short stories about the titular Foundation; indeed the first volume consists of stories that had already been published elsewhere, and which are intended to be read separately but describe events in the same timeline.

Hari Seldon comes up with the field of psychohistory, a branch of mathematics that explains human activity collectively without explaining how any individual acts, in a similar way to how statistical thermodynamics tells you the behaviour of heat transfer in a system without describing the story of a single molecule.

This psychohistory predicts that the great Galactic Empire will crumble, causing millennia of human suffering—or a single millennium, if the Foundation intervene and guide the creation of a new society, using the calculations of psychohistory as a blueprint. Seldon runs the maths, and identifies a number of crisis points that this transition will go through, recording talks for the Foundation to play at the times when these crises occur.

So far, this sounds incredibly Marxist. The Empire takes the place of, well, imperialism, grand in its reach and advances but ultimately destined to destruction through its own contradictions. Psychohistory represents historical materialism, a scientific examination of the currents of history that identifies how these contradictions lead to the collapse of imperialism and set out the blueprint for a transition to a just society. As in Marx’s XVIII Brumaire of Louis Napoleon, there are no great men of history, “they do not make [history] under self-selected circumstances, but under circumstances existing already, given and transmitted from the past.” Though as Napoleon was seen by some (other than Marx, obviously) as the Great Man of European history, so the Mule disrupts the statistical flows of psychohistory.

Then things take a turn, presumably because Marxism fell out of favour (to the extent it ever was in favour) in the America in which Asimov spent most of his life. From Foundation’s Edge onwards, even the Foundation’s psychohistory isn’t the end goal of society, but a Gaia-esque shared consciousness in which all of humanity participates. In its presentation, we learn of this as a sort of harmony-of-nature, in-tune-with-the-universe gestalt existence that sounds very much like a utopian way out of the struggles and conflicts of human existence, and the climatic destruction of a species that doesn’t take its home planet seriously.

Utopian, that is, until we learn the final revelation that Asimov wrote to tie all of his great science fiction series together, in the fifth (and chronologically final, though two prequels follow on) novel, Foundation and Earth. That is, in modern parlance, that the whole lot—the creation of psychohistory by Seldon, the Foundations, and Gaia—were put in motion by what we would now call an Artificial Intelligence, the robot R. Daneel Olivaw. It was Olivaw’s drive to create a better galaxy for humanity that led to the events of the preceding novels, and his vision of how that world should work that culminates in Gaia and the shared consciousness.

The interpretation of this point has probably shifted a lot in the intervening decades, and we even see that shift in other science fiction works. Through Asimov’s Robots series we see AIs as generally benevolent servants of humanity, programmed to do the right thing even if sometimes the programming leads to some surprising events. The main enemy of progress in that series isn’t the robots themselves, it’s the inability of most people to accept that they can exist and work alongside humans. If the spongy, platinum-iridium positronic brain of a robot comes up with the idea of Gaia, then it must be a good idea.

However, fast-forward to the present day—pausing to watch the horrors of the Cybermen on Doctor Who, the Borg on Star Trek, the Matrix, and most episodes of Black Mirror—and we’d have a very different take on “computer decides that the best outcome for humanity is to remove individuality and for everyone to be connected in a shared consciousness.” How often is individuality seen not as a weakness, but as the very point of humanity? Would Asimov add yet another denouement to the Foundation series today? Would he perhaps go back to the original outcome of psychohistory, and the triumph of intellectualism and careful thinking over demagoguery; or would that conclusion be too technocratic for today’s readers?

Posted in Fiction | Leave a comment

Hi, I’m Graham and I like things

Posted on 2025-06-11 by Graham

In my time in special-interest forums, I’ve come to learn that a “fan” of something is someone who doesn’t like it very much. This seems to crop up frequently in relation to long-running science fiction entertainment franchises, leading me to the theory that a “fan” is someone who enjoyed a kid’s show as a kid and is angry at the creators because they don’t enjoy a kid’s show as an adult.

To be a Doctor Who “fan”, you apparently have to know all the ways in which the people who make Doctor Who don’t understand what Doctor Who is and shouldn’t be allowed near it. For example, the Timeless Child arc—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation—is absolute piffle to “fans”, who lament the show’s cancellation before the realization of the Cartmel Master Plan—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation.

To be a Star Wars “fan”, you apparently have to know that the 1977-1983 trilogy of films (the retroactively-subtitled Episode IV through Episode VI) are the only valid Star Wars, and that everything else is Not Star Wars. Particularly 1999-2005 trilogy, who were made by someone who doesn’t truly understand Star Wars; a certain *checks notes* George Lucas.

To be a Star Trek “fan”, you apparently have to know what’s wrong with every episode, why they should never have made any series after (or including) Enterprise, but also why there should always be new Star Trek on the TV.

I also see this with music, where “fans” explain who the worst bands in a genre are; what the worst albums, and tracks, by bands are; why the top ten most-streamed songs are bad; and why the tools that musicians use to make music make musicians make bad music.

Hi, I’m Graham. I like long-running science fiction entertainment franchises, I like music, I like books, I like many things.

Posted in star trek, whatevs | Leave a comment

On Nostalgia for Physical Media

Posted on 2025-02-15 by Graham

While I have access to streaming services that offer most of the music that the labels the services deal with still publish, I also have a significant collection of music on physical media, and do most of my listening to prerecorded music by playing entire albums, in order, from a physical format. I recently shared a not-entirely-serious overview of the nostalgia for physical prerecorded audio formats with some friends, and here’s the annotated version. In what follows, the formats are presented from best to worst.

Compact Disc (CD)

Summary: these are the true audiophile-grade format and any golden ears fool who tells you vinyl is better is deluded.

It’s CD’s blessing, and curse, that it’s the ideal physical format for music. Blessing, because it replaced everything that came before it, and remained dominant until the rise of the MP3/streamed audio file (which offered lower quality than CDs at the time, but greater convenience). Curse, because its perfection leads to sterility: what you get from playing a CD is the music that was recorded onto it, every time. Any noise was on the master tape, and any imperfections are due to your setup. It’s unexciting, so was easy to abandon.

And CD digital audio, the red book standard for encoding stereo sound quantised at 16 bits with 44.1kHz sampling, is good enough. HDCD, SACD, DVD-Audio, and HFPA Blu-ray may all be “better” on the spec sheet, but unless you’re sitting in the centre of a perfectly set up room listening to something that was mastered at very high fidelity, and are young enough that your ears can physically react to the higher frequencies, none of that matters.

There are two downside to CD, the first is that the case problem took so long to solve. The standard jewel cases are too fragile for many applications, and the little tabs that hold the booklet in tend to damage the booklets. The other is that the ability to record to CD took so long to come about, and that the recordable discs are unreliable in many players. That probably worked in the format’s favour for adoption as a pre-recorded format, though, because the labels were very worried that “home taping is killing music” so a read-only delivery medium was their preference.

Minidisc (MD)

Summary: like better CDs except for the limited capacity and the proprietary encoding.

I’ll admit that even though CDs are at the top of my list, MD is my favourite format. A disc is smaller than a CD, robust due to its enclosing cassette, recordable (even with many portable players), contains the track listing on the disc, and is straight-up fun to use.

Getting CD capacity on a smaller disc was achieved by using a compressed audio format called ATRAC (the data capacity of a MD is about half that of a CD). Often this doesn’t lead to any observable quality degradation, but later players and recorders supported the MD-LP format which used more aggressive compression to achieve higher run times, and you can sometimes hear that like listening to a 2000s-era MP3 file. Eventually Hi-MD (high capacity minidisc) solved the problem but in a way that didn’t preserve backwards compatibility.

Prerecorded MDs were uncommon (“home taping is killing music” and MD is a recordable format, so Sony had to introduce the Serial Copy Management System which meant that a second-generation digital copy on MD could not itself be duplicated), indeed most were released by labels that Sony themselves owned. And if you think the little booklets of notes, lyrics, and art you get in a CD are small, they’re absolutely Lilliputian in an MD box.

Vinyl (33rpm LP, 45rpm SP)

Summary: for getting a pleasant feeling from the shitty sound of pops and dusty styli.

The nostalgia that exists around vinyl stems from its long reign as the only game in town. Because the discs are so big (12″ LP, 7″ SP), you get nice sleeve art. The format itself suffers from the fact that dragging a diamond or sapphire across a sheet of PVC generates static electricity, which attracts dust and discharges, and that all of that gets heard as noise in the pickup system. Also it’s easy to smear or scratch a record, and cause more defects on playback.

One thing that the stylus doesn’t particularly care about is the amount of plastic below the groove, which hasn’t stopped revival manufacturers from selling the amount of plastic (180g! 200g! 220g!) as a premium feature.

MP3 player

Summary: I guess these are nostalgia now also.

Shellac

Summary: like shitty vinyl.

A great way to annoy a vegan would be to farm insects so you can turn their exudate into reproductions of musical recordings. And that’s exactly what happened up until about 1948.

By the way, the limited capacity of shellac records (as the reproduction process was even noisier than microgroove vinyl, the records had to spin at 78rpm, chosen because the gears to get the motor to run at 78rpm were already constructed for use in players for talkie movie houses) meant that they wouldn’t fit much music on each side of a disc. Large collections of discs were bound into albums and sold together, and that’s why music albums are called albums. There you go.

Pianola/barrel organ punched paper rolls

Summary: convenient format, awkward-ass players.

Imagine having a whole piano that you don’t play yourself, but that has a clockwork punched paper feeder like a mechanised loom that plays it for you. The rolls were mastered using the reverse process, where someone played a piano that marked up where the holes should go on a blank paper roll. It’s kind of fun to think that the ghost of Scott Joplin is playing the Maple Leaf Rag on your piano.

Wax cylinder

Summary: no, you don’t actually remember this.

Digital Audio Tape (DAT)/ Digital Compact Cassette (DCC)

Summary: if you remember these then well done, you bought the wrong hi-fi.

It’s a shame that these come above the compact cassette in this list, because they fixed some of the problems of that format that we haven’t been able to discuss yet. However, they didn’t fix the main issues: that tape is sequentially fed into the player like a pianola roll, and that when you do that there’s a high chance it gets mangled up in the player.

DCC exists because Sony and Philips had stopped collaborating, and Philips wanted to position themselves against MD. There’s a sounds-too-good-to-be-true story that MD and DCC were demoed at the same event, and that while playback sounded great from both formats, there was an awkward silence punctuated only by the giggles of journalists when the Philips rep switched to a different track which involved fast-winding the tape.

Compact Cassette

Summary: you remember these and they’re really bad.

There’s a recent trend for cassette revival, and it makes me feel weird. The only format that adds so much noise, Dolby invented multiple compansion techniques to try to make the sound listenable. The format where there’s a good chance that when you take a cassette out of the player, the tape stays inside the player and outside the cassette. The format that triggered the “you’re old if you know what this is for” picture of a pencil meme. The format where playing the music rubs the music off the tape.

I have multiple cassette players, but mostly for retro computing reasons. I didn’t enjoy tape “back in the day”, I’m not like to start now.

Endless loop (4-track, 8-track), other cart tape or reel-to-reel tape formats

Summary: worse than compact cassette.

Posted in music | 3 Comments

The gaps between the processes

Posted on 2025-01-25 by Graham

Knowledge management—not just in software engineering and not just digital knowledge management—has long had to account for tacit knowledge: the things that people know, but never say.

“A lesser blog would reproduce the Donald Rumsfeld statement about known unknowns at this point”—Graham Lee.

Where do people pick up this tacit knowledge? In conversations, at conferences, by reading books: it’s not really tacit in that it’s literally never spoken. Tacit knowledge is tacit in that people rely on it in particular scenarios without being explicit that they’re relying on it.

Every software engineering process or methodology works (or not) in the presence of tacit knowledge. How they work in its presence is illuminating, mostly for telling us something about how the people who created the process think about the world.

The waterfall-as-practiced (i.e. not the Royceian “do it twice” version, but the DoD and everyone in the 1980s “do it once” version) model seems to rely on all knowledge being explicit, and available. There are truths about the universe that are your software’s requirements, and if you spend enough time and do a good enough job at requirements gathering you can know what those requirements are before you get to the point where you use those requirements in writing a specification.

Every iterative and incremental model, from the Royceian waterfall-as-described process where you write one to throw away then write it again, through the spiral model, to the various scrums and scrumbuts in practice today, allows for people saying “oh, right, no, not that, because…” and bringing that tacit knowledge into the open. They might not express that that’s why they have iterations, they might say it’s for early and continuous delivery of value or to identify and mitigate risks, but they still do give space for “what I meant to say was…”.

Structured programming approaches expect everything to be written down. A use case (in the original, technical meaning, not the current fancy-way-to-say-“situation” meaning) is a pedantic document that describes all of the interactions someone might have with “the system” in pursuit of a particular goal.

Lightweight approaches expect the bare minimum to be written down, and everything to be elucidated. A user story (in the original, technical meaning, not the current fancy-way-to-say-“Jira ticket” meaning) is a placeholder for a conversation. You’re meant to write the story on one side of an index card, then, as you have the conversations, capture the rest of the information that everybody needs on the other side.

Internal wikis trade on the idea that if only it were really easy to edit a knowledge base, we’d all just write down our tacit knowledge and make it explicit, while also somehow making it salient, searchable, and up to date. Intranets trade on the idea that nobody knows anything outside of their lane and that it’s up to the Department of XYZ to make the Policy for XYing your Z available to all employees or no Z will ever get XYd in the correct way.

Interestingly a recent development in software engineering methodology seems to be both positivist and dependent on tacit knowledge. That’s the “web-scale” methodology, where you never talk to customers, but you run A/B tests, you create metrics, and optimise for those metrics. Positivist, because there is assumed to be a success factor that can be measured numerically and attained by moving the numbers. Tacit, because no customer ever says what they want or why they want it: instead, organisations create numeric goals that represent what a million, or a billion, customers are guessed to want, and strive to make the numbers go up (or down, as appropriate).

I suspect that this approach may be the biggest methodological innovation in software in more than two decades, and also one that methodologists seem to be currently quiet on.

Posted in whatevs | Leave a comment