Is Foundation a Utopian vision?

Posted on 2025-07-11 by Graham

Important: while I’m only talking about the Foundation books in vague details here, I will end up summarising a number of key points through the whole series. If you haven’t read them, and intend to, I recommend not reading this post, yet.

Here, at the start of the third TV series of Foundation, I’m actually talking about the series of short stories and novels on which the TV show is based. The TV series takes its leave of the source material in many ways—most of them necessary to produce something watchable in 2025. For example, you don’t get a female character in the novels for a very long time, and you have to wait until the second book to encounter one that Asimov names. Also, atomic ashtrays no longer represent the pinnacle of technological advancement.

The Foundation series of books starts with a collection of short stories about the titular Foundation; indeed the first volume consists of stories that had already been published elsewhere, and which are intended to be read separately but describe events in the same timeline.

Hari Seldon comes up with the field of psychohistory, a branch of mathematics that explains human activity collectively without explaining how any individual acts, in a similar way to how statistical thermodynamics tells you the behaviour of heat transfer in a system without describing the story of a single molecule.

This psychohistory predicts that the great Galactic Empire will crumble, causing millennia of human suffering—or a single millennium, if the Foundation intervene and guide the creation of a new society, using the calculations of psychohistory as a blueprint. Seldon runs the maths, and identifies a number of crisis points that this transition will go through, recording talks for the Foundation to play at the times when these crises occur.

So far, this sounds incredibly Marxist. The Empire takes the place of, well, imperialism, grand in its reach and advances but ultimately destined to destruction through its own contradictions. Psychohistory represents historical materialism, a scientific examination of the currents of history that identifies how these contradictions lead to the collapse of imperialism and set out the blueprint for a transition to a just society. As in Marx’s XVIII Brumaire of Louis Napoleon, there are no great men of history, “they do not make [history] under self-selected circumstances, but under circumstances existing already, given and transmitted from the past.” Though as Napoleon was seen by some (other than Marx, obviously) as the Great Man of European history, so the Mule disrupts the statistical flows of psychohistory.

Then things take a turn, presumably because Marxism fell out of favour (to the extent it ever was in favour) in the America in which Asimov spent most of his life. From Foundation’s Edge onwards, even the Foundation’s psychohistory isn’t the end goal of society, but a Gaia-esque shared consciousness in which all of humanity participates. In its presentation, we learn of this as a sort of harmony-of-nature, in-tune-with-the-universe gestalt existence that sounds very much like a utopian way out of the struggles and conflicts of human existence, and the climatic destruction of a species that doesn’t take its home planet seriously.

Utopian, that is, until we learn the final revelation that Asimov wrote to tie all of his great science fiction series together, in the fifth (and chronologically final, though two prequels follow on) novel, Foundation and Earth. That is, in modern parlance, that the whole lot—the creation of psychohistory by Seldon, the Foundations, and Gaia—were put in motion by what we would now call an Artificial Intelligence, the robot R. Daneel Olivaw. It was Olivaw’s drive to create a better galaxy for humanity that led to the events of the preceding novels, and his vision of how that world should work that culminates in Gaia and the shared consciousness.

The interpretation of this point has probably shifted a lot in the intervening decades, and we even see that shift in other science fiction works. Through Asimov’s Robots series we see AIs as generally benevolent servants of humanity, programmed to do the right thing even if sometimes the programming leads to some surprising events. The main enemy of progress in that series isn’t the robots themselves, it’s the inability of most people to accept that they can exist and work alongside humans. If the spongy, platinum-iridium positronic brain of a robot comes up with the idea of Gaia, then it must be a good idea.

However, fast-forward to the present day—pausing to watch the horrors of the Cybermen on Doctor Who, the Borg on Star Trek, the Matrix, and most episodes of Black Mirror—and we’d have a very different take on “computer decides that the best outcome for humanity is to remove individuality and for everyone to be connected in a shared consciousness.” How often is individuality seen not as a weakness, but as the very point of humanity? Would Asimov add yet another denouement to the Foundation series today? Would he perhaps go back to the original outcome of psychohistory, and the triumph of intellectualism and careful thinking over demagoguery; or would that conclusion be too technocratic for today’s readers?

Posted in Fiction | Leave a comment

Hi, I’m Graham and I like things

Posted on 2025-06-11 by Graham

In my time in special-interest forums, I’ve come to learn that a “fan” of something is someone who doesn’t like it very much. This seems to crop up frequently in relation to long-running science fiction entertainment franchises, leading me to the theory that a “fan” is someone who enjoyed a kid’s show as a kid and is angry at the creators because they don’t enjoy a kid’s show as an adult.

To be a Doctor Who “fan”, you apparently have to know all the ways in which the people who make Doctor Who don’t understand what Doctor Who is and shouldn’t be allowed near it. For example, the Timeless Child arc—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation—is absolute piffle to “fans”, who lament the show’s cancellation before the realization of the Cartmel Master Plan—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation.

To be a Star Wars “fan”, you apparently have to know that the 1977-1983 trilogy of films (the retroactively-subtitled Episode IV through Episode VI) are the only valid Star Wars, and that everything else is Not Star Wars. Particularly 1999-2005 trilogy, who were made by someone who doesn’t truly understand Star Wars; a certain *checks notes* George Lucas.

To be a Star Trek “fan”, you apparently have to know what’s wrong with every episode, why they should never have made any series after (or including) Enterprise, but also why there should always be new Star Trek on the TV.

I also see this with music, where “fans” explain who the worst bands in a genre are; what the worst albums, and tracks, by bands are; why the top ten most-streamed songs are bad; and why the tools that musicians use to make music make musicians make bad music.

Hi, I’m Graham. I like long-running science fiction entertainment franchises, I like music, I like books, I like many things.

Posted in star trek, whatevs | Leave a comment

On Nostalgia for Physical Media

Posted on 2025-02-15 by Graham

While I have access to streaming services that offer most of the music that the labels the services deal with still publish, I also have a significant collection of music on physical media, and do most of my listening to prerecorded music by playing entire albums, in order, from a physical format. I recently shared a not-entirely-serious overview of the nostalgia for physical prerecorded audio formats with some friends, and here’s the annotated version. In what follows, the formats are presented from best to worst.

Compact Disc (CD)

Summary: these are the true audiophile-grade format and any golden ears fool who tells you vinyl is better is deluded.

It’s CD’s blessing, and curse, that it’s the ideal physical format for music. Blessing, because it replaced everything that came before it, and remained dominant until the rise of the MP3/streamed audio file (which offered lower quality than CDs at the time, but greater convenience). Curse, because its perfection leads to sterility: what you get from playing a CD is the music that was recorded onto it, every time. Any noise was on the master tape, and any imperfections are due to your setup. It’s unexciting, so was easy to abandon.

And CD digital audio, the red book standard for encoding stereo sound quantised at 16 bits with 44.1kHz sampling, is good enough. HDCD, SACD, DVD-Audio, and HFPA Blu-ray may all be “better” on the spec sheet, but unless you’re sitting in the centre of a perfectly set up room listening to something that was mastered at very high fidelity, and are young enough that your ears can physically react to the higher frequencies, none of that matters.

There are two downside to CD, the first is that the case problem took so long to solve. The standard jewel cases are too fragile for many applications, and the little tabs that hold the booklet in tend to damage the booklets. The other is that the ability to record to CD took so long to come about, and that the recordable discs are unreliable in many players. That probably worked in the format’s favour for adoption as a pre-recorded format, though, because the labels were very worried that “home taping is killing music” so a read-only delivery medium was their preference.

Minidisc (MD)

Summary: like better CDs except for the limited capacity and the proprietary encoding.

I’ll admit that even though CDs are at the top of my list, MD is my favourite format. A disc is smaller than a CD, robust due to its enclosing cassette, recordable (even with many portable players), contains the track listing on the disc, and is straight-up fun to use.

Getting CD capacity on a smaller disc was achieved by using a compressed audio format called ATRAC (the data capacity of a MD is about half that of a CD). Often this doesn’t lead to any observable quality degradation, but later players and recorders supported the MD-LP format which used more aggressive compression to achieve higher run times, and you can sometimes hear that like listening to a 2000s-era MP3 file. Eventually Hi-MD (high capacity minidisc) solved the problem but in a way that didn’t preserve backwards compatibility.

Prerecorded MDs were uncommon (“home taping is killing music” and MD is a recordable format, so Sony had to introduce the Serial Copy Management System which meant that a second-generation digital copy on MD could not itself be duplicated), indeed most were released by labels that Sony themselves owned. And if you think the little booklets of notes, lyrics, and art you get in a CD are small, they’re absolutely Lilliputian in an MD box.

Vinyl (33rpm LP, 45rpm SP)

Summary: for getting a pleasant feeling from the shitty sound of pops and dusty styli.

The nostalgia that exists around vinyl stems from its long reign as the only game in town. Because the discs are so big (12″ LP, 7″ SP), you get nice sleeve art. The format itself suffers from the fact that dragging a diamond or sapphire across a sheet of PVC generates static electricity, which attracts dust and discharges, and that all of that gets heard as noise in the pickup system. Also it’s easy to smear or scratch a record, and cause more defects on playback.

One thing that the stylus doesn’t particularly care about is the amount of plastic below the groove, which hasn’t stopped revival manufacturers from selling the amount of plastic (180g! 200g! 220g!) as a premium feature.

MP3 player

Summary: I guess these are nostalgia now also.

Shellac

Summary: like shitty vinyl.

A great way to annoy a vegan would be to farm insects so you can turn their exudate into reproductions of musical recordings. And that’s exactly what happened up until about 1948.

By the way, the limited capacity of shellac records (as the reproduction process was even noisier than microgroove vinyl, the records had to spin at 78rpm, chosen because the gears to get the motor to run at 78rpm were already constructed for use in players for talkie movie houses) meant that they wouldn’t fit much music on each side of a disc. Large collections of discs were bound into albums and sold together, and that’s why music albums are called albums. There you go.

Pianola/barrel organ punched paper rolls

Summary: convenient format, awkward-ass players.

Imagine having a whole piano that you don’t play yourself, but that has a clockwork punched paper feeder like a mechanised loom that plays it for you. The rolls were mastered using the reverse process, where someone played a piano that marked up where the holes should go on a blank paper roll. It’s kind of fun to think that the ghost of Scott Joplin is playing the Maple Leaf Rag on your piano.

Wax cylinder

Summary: no, you don’t actually remember this.

Digital Audio Tape (DAT)/ Digital Compact Cassette (DCC)

Summary: if you remember these then well done, you bought the wrong hi-fi.

It’s a shame that these come above the compact cassette in this list, because they fixed some of the problems of that format that we haven’t been able to discuss yet. However, they didn’t fix the main issues: that tape is sequentially fed into the player like a pianola roll, and that when you do that there’s a high chance it gets mangled up in the player.

DCC exists because Sony and Philips had stopped collaborating, and Philips wanted to position themselves against MD. There’s a sounds-too-good-to-be-true story that MD and DCC were demoed at the same event, and that while playback sounded great from both formats, there was an awkward silence punctuated only by the giggles of journalists when the Philips rep switched to a different track which involved fast-winding the tape.

Compact Cassette

Summary: you remember these and they’re really bad.

There’s a recent trend for cassette revival, and it makes me feel weird. The only format that adds so much noise, Dolby invented multiple compansion techniques to try to make the sound listenable. The format where there’s a good chance that when you take a cassette out of the player, the tape stays inside the player and outside the cassette. The format that triggered the “you’re old if you know what this is for” picture of a pencil meme. The format where playing the music rubs the music off the tape.

I have multiple cassette players, but mostly for retro computing reasons. I didn’t enjoy tape “back in the day”, I’m not like to start now.

Endless loop (4-track, 8-track), other cart tape or reel-to-reel tape formats

Summary: worse than compact cassette.

Posted in music | 3 Comments

The gaps between the processes

Posted on 2025-01-25 by Graham

Knowledge management—not just in software engineering and not just digital knowledge management—has long had to account for tacit knowledge: the things that people know, but never say.

“A lesser blog would reproduce the Donald Rumsfeld statement about known unknowns at this point”—Graham Lee.

Where do people pick up this tacit knowledge? In conversations, at conferences, by reading books: it’s not really tacit in that it’s literally never spoken. Tacit knowledge is tacit in that people rely on it in particular scenarios without being explicit that they’re relying on it.

Every software engineering process or methodology works (or not) in the presence of tacit knowledge. How they work in its presence is illuminating, mostly for telling us something about how the people who created the process think about the world.

The waterfall-as-practiced (i.e. not the Royceian “do it twice” version, but the DoD and everyone in the 1980s “do it once” version) model seems to rely on all knowledge being explicit, and available. There are truths about the universe that are your software’s requirements, and if you spend enough time and do a good enough job at requirements gathering you can know what those requirements are before you get to the point where you use those requirements in writing a specification.

Every iterative and incremental model, from the Royceian waterfall-as-described process where you write one to throw away then write it again, through the spiral model, to the various scrums and scrumbuts in practice today, allows for people saying “oh, right, no, not that, because…” and bringing that tacit knowledge into the open. They might not express that that’s why they have iterations, they might say it’s for early and continuous delivery of value or to identify and mitigate risks, but they still do give space for “what I meant to say was…”.

Structured programming approaches expect everything to be written down. A use case (in the original, technical meaning, not the current fancy-way-to-say-“situation” meaning) is a pedantic document that describes all of the interactions someone might have with “the system” in pursuit of a particular goal.

Lightweight approaches expect the bare minimum to be written down, and everything to be elucidated. A user story (in the original, technical meaning, not the current fancy-way-to-say-“Jira ticket” meaning) is a placeholder for a conversation. You’re meant to write the story on one side of an index card, then, as you have the conversations, capture the rest of the information that everybody needs on the other side.

Internal wikis trade on the idea that if only it were really easy to edit a knowledge base, we’d all just write down our tacit knowledge and make it explicit, while also somehow making it salient, searchable, and up to date. Intranets trade on the idea that nobody knows anything outside of their lane and that it’s up to the Department of XYZ to make the Policy for XYing your Z available to all employees or no Z will ever get XYd in the correct way.

Interestingly a recent development in software engineering methodology seems to be both positivist and dependent on tacit knowledge. That’s the “web-scale” methodology, where you never talk to customers, but you run A/B tests, you create metrics, and optimise for those metrics. Positivist, because there is assumed to be a success factor that can be measured numerically and attained by moving the numbers. Tacit, because no customer ever says what they want or why they want it: instead, organisations create numeric goals that represent what a million, or a billion, customers are guessed to want, and strive to make the numbers go up (or down, as appropriate).

I suspect that this approach may be the biggest methodological innovation in software in more than two decades, and also one that methodologists seem to be currently quiet on.

Posted in whatevs | Leave a comment

In which things are not known

Posted on 2025-01-11 by Graham

In the last episode—Is software engineering a thing?—I (apparently controversially) suggested that software is the reification of thought, and that software engineering is thus the art of reifying thought, and that thus there can’t be any single one-size-fits-all software engineering approach. Let’s dig in.

One of the big questions in a software project, hence one of the big topics in software engineering, is requirements: who wants the software to do something, do we need to pay attention to them, and what do they want it to do? We’re already well into the realm of individuals and interactions—whether the people building the thing and the people telling them what to build can agree on what one of the two groups thinks they mean—and haven’t got as far as building software yet. There’s plenty of software engineering ink spilled in this domain, but it can be hard to even decide whether to agree at a metaphysical level with some of it.

Taking a convenience sample (i.e. what’s the nearest book on my shelf that I think might mention software requirements), Shari Pfleeger’s “Software Engineering: the Production of Quality Software” asks “What is a requirement?” and supplies its own answer:

A requirement is a feature of the system or a description of something the system is capable of doing in order to fulfill the system’s purpose.

Enquiring minds have many questions, but let’s focus on questions pertaining to reality. Does the system have an objective, positive purpose that can be said to exist? Does the requirement support that purpose, or does someone just think or hope that it does? Does the requirement-as-description accurately capture the expectation of the person who thought it?

With this level of reflection, we can still expect a field of software engineering to say something about requirements, and for understanding that to help with constructing software, but not for it to supply a single solution to “how to requirements”. And without that, much of the rest of software engineering naturally bifurcates or multifurcates. For example, verification and validation is about whether the software does what it ought—or whether someone thinks the software does what they think it ought—but we’re back to asking our question of whether we have (or can) accurately capture that.

Posted in software-engineering | Leave a comment

Is software engineering a thing?

Posted on 2025-01-06 by Graham

In the title I’m kindof punning on the word “a” (it’s my blog, and I get to do what I want). Is there a single thing, software engineering, that all people making software should (or could, or would find to be beneficial) do?

It’s a question that’s sat with me all through my doctoral research, and has sat with the field for much longer. In 2007, Diane Kelly argued that there’s a “chasm” between scientific computing and commercial software, and that leaders should get together and identify their differences to come up with discipline-specific approaches to writing software. In other words, that there isn’t a universal “software engineering” that’s the best approach to writing software.

A decade later, Tim Storer described the chasm as a skills gap that needed to be closed. In this view, software engineering is the application of software knowledge to the production of software, and computational scientists don’t have enough of that knowledge to do it correctly.

There’s a whole community of research devoted to uncovering a grand unified theory of software engineering, in analogy to the Grand Unified Theories of physics that unite the electromagnetic and weak and strong nuclear forces. Members of this community (which goes by the name SEMAT: Software Engineering Methods and Theory) start not by constructing their theory but by deconstructing others.

They argue (convincingly) that any particular software engineering methodology is flawed, because it recommends a whole suite of practices but we don’t know which are relevant and useful, or how they interact, we just know that the methodologists managed to trademark their particular grab bag of practices and argue that if you’re not making enough software, you’re not doing it the way they propose. While there might be something to daily stand-ups, or to organising work by sprints, or to holding retrospectives, there’s nothing to Scrum because selecting all of these practices together is entirely arbitrary.

What the SEMAT folks argue for instead is more of a systems approach to software (in a Dana Meadows sense rather than a Jerry Weinberg sense): the team use their way of working to do work to generate a software system that realises requirements to exploit an opportunity identified by stakeholders; which part of that process is the most broken and what can you do to make it less broken than some other part?

I think that’s a great way to think about it, and I also don’t think that a GUT of software engineering will arise from it. To me, software is the reification of thought in a reusable and somewhat abstracted structure: we understand something about a context and try to capture our (or, commonly, someone else’s) understanding of that context in a way that can be automatically calculated using digital electronic equipment. To say that a universal theory of making software exists is to say that a universal theory of understanding thought exists, and we aren’t there.

Many of the open problems in software engineering boil down to not being able to capture thoughts precisely. Software engineering ethics is the inability to define universal rights and wrongs (which may not exist anyway). Software quality management is the inability to agree what the understanding was and whether we’ve captured it correctly. The fact that we don’t agree on whether object-oriented, structured, functional, or some other approach to analysis and design is the best choice is a sign that we don’t agree on how to encode thought in a way that we can think about.

In other words, software construction is thinking about thought, it is meta-thought. And we don’t agree enough on how thought works to be able to get consensus on the best way to think about thought, let alone the best way to encapsulate that thinking about thought in a saleable product.

Posted in software-engineering | 12 Comments

Resolutions

Posted on 2025-01-02 by Graham

Although I didn’t make any resolutions this new year, it’s still a time for change. That’s because I finally submit my D.Phil. thesis (if I’m on time, that will be before January 18th), so I’ve already been putting things in place that mean that I’ll start doing things differently this year, without having to decide that I’m going to do things differently on January 1st (and invariably last until about the 3rd).

In my case, this year is about society. Writing a doctoral thesis means a lot of time alone, so I’ve joined committees for some clubs I’m in, and will be looking to do other things which mean spending more time finding out what other people want to do and helping them to do it.

Posted in whatevs | Leave a comment

On the Consolation of Software Engineering

Posted on 2024-12-02 by Graham

I’m currently reading Boethius’s writing on the consolation of philosophy. Imprisoned awaiting the death penalty in 523 (for treason against King Theodoric), Boethius imagined a conversation with the personification of Philosophy herself, a woman of variable height whose fine dress was torn by various previous philosophers who had snatched tatters from it and imagined that they had the whole thing. The work has been available in English translation since the ninth century, when King Alfred commanded it be translated, though I’m reading a much more recent Penguin Classics translation.

The work leads me to imagine De Consolatione Ingenariae Computatraliae, in which someone is visited by the anthropomorphic personification of software engineering (in my headcanon this is Bruce Boxleitner as Tron), and the visitor uses the Socratic method to explore why the narrator chose to ignore so much knowledge of computing as they cut corners to close ticket after ticket in their career.

Posted in book | Leave a comment

In which a life re-emerges

Posted on 2024-10-31 by Graham

While it’s far from finished, my PhD thesis is now complete: there are no to-do items left, no empty sections, no placeholders. Now the proof-reading, editing and corrections continue in earnest.

I look forward to poking my head out of that rabbit hole too, and finding out what else has been happening in the world since October 2020.

Posted in academia | Leave a comment

The Cataract of Software Delivery

Posted on 2024-06-18 by Graham

There’s this paper from August 1970, called Managing the Development of Large Software Systems, that’s considered something of a classic (either for good or for bad, depending on your worldview). The discussion often goes something like this:

Let’s say I have some software to build, and I think it’s going to take about a year to build it. Few people are going to happily say “go away for a year and tell me when its done”. Instead, most people will want to break down that year into smaller chunks, so they can monitor progress and have confidence that things are on track. The question then is how do we perform this break down?

The waterfall style, as suggested by the Royce sketch, does it by the activity we are doing. So our 1 year project might be broken down into 2 months of analysis, followed by 4 months design, 3 months of coding, and 3 months of testing.

Martin Fowler, Waterfall Process

Within the phase space of Royce paper discussions, there are those who say that he argues for the strict sequential process as Fowler does, based on Figure 2 in the paper. There are those who say that he predicts some iterative, evolutionary, (dare we say it?) agile contribution to the process, based on Figure 4 in the paper. But many agree with Fowler when he says “this paper seems to be universally acknowledged as the source of the notion of waterfall”.

It isn’t. The overall process had already been described in 1956 as a “structured” (ever wonder where the phrase “structured programming” comes from, when it clearly doesn’t refer to having data structures?), “top-down” approach, by Herb Benington in Production of Large Computer Programs. Methodologists contemporary with and later than Royce, including luminaries like Barry W. Boehm and Agilistas like Alan Moran, knew about this paper, so even if we don’t use it in our software mythologies any more, it isn’t obscure and wasn’t obscure in Royce’s time.

Both Benington and Royce write within a context of large-scale government-funded projects: Benington from his experience with SAGE (the Semi-Automatic Ground Environment) and Royce at the aerospace contractor TRW. Both talk about phased approaches with dependencies between tasks (so you can’t do coding until you’ve done design, for example). Both knew about the value of prototyping, though in a mistake that makes Hoare’s introduction of NULL look like loose change, Benington didn’t mention it until 1983:

I do not mention it in the attached paper, but we undertook the programming only after we had assembled an experimental prototype of 35,000 instructions of code that performed all of the bare-bone functions of air defense. Twenty people understood in detail the performance of those 35,000 instructions; they knew what each module would do, they understood the interfaces, and they understood the performance requirements. People should be very cautious about writing top-down specs without having this detailed knowledge, so that the decision-maker who has the “requirement” can make the proper trade-offs between performance, cost, and risk.

To underscore this point, the biggest mistake we made in producing the SAGE computer program was that we attempted to make too large a jump from the 35,000 instructions we had operating on the much simpler Whirlwind I computer to the more than 100,000 instructions on the much more powerful IBM SAGE computer. If I had it to do over again, I would have built a framework that would have enabled us to handle 250,000 instructions, but I would have transliterated almost directly only the 35,000 instructions we had in hand on this framework. Then I would have worked to test and evolve a system. I estimate that this evolving approach would have reduced our overall software development costs by 50 percent.

Herb Benington, Production of Large Computer Programs

Royce, on the other hand, describes the “write one to throw away” approach, in which a prototype informs the design of the final system but doesn’t become part of it:

A preliminary program design phase has been inserted between the software requirements generation phase and the analysis phase. This procedure can be criticized on the basis that the program designer is forced to design in the relative vacuum of initial software requirements without any existing analysis..As a result, his preliminary design may be substantially in error as compared to his design if he were to wait until the analysis was complete. This criticism is correct but it misses the point. By this technique the program designer assures that the software will not fail because of storage, timing, and data flux reasons. As the analysis proceeds in the succeeding phase the program designer must impose on the analyst the storage, timing, and operational constraints in such a way that he senses the consequences. When he justifiably requires more of this kind of resource in order to implement his equations it must be simultaneously snatched from his analyst compatriots. In this way all the analysts and all the program designers will contribute to a meaningful design process which will culminate in the proper allocation of execution time and storage resources. If the total resources to be applied are insufficient or if the embryo operational design is wrong it will be recognized at this earlier stage and the iteration with requirements and preliminary design can be redone before final design, coding and test commences.

Winston Royce, Managing the Development of Large Software Systems

Royce’s goal with this phase was to buttress the phased development approach, which he believed to be “fundamentally sound”, by adding data in earlier phases that informed the later phases. Indeed, if Royce finds any gap in the documentation on a project “my first recommendation is simple. Replace project management. Stop all activities not related to documentation. Bring the documentation up to acceptable standards. Management of software is simply impossible without a very high degree of documentation.”

So we have a phased, top-down, sequential development process in 1956, that survived until 1970 when more phases were added to reduce the risk accepted in lower phases. Good job those lightweight methodologists came along in the 1990s and saved us with their iterative, incremental development, right?

Not quite. Before Boehm knew of Benington’s paper, he’d already read Hosier’s Pitfalls and Safeguards in Real-Time Digital Systems with Emphasis on Programming from June 1961. Hosier presents (in figure 2) a flow chart of program development, including feedback from much of the process and explicit “revision” steps in the control routine, unit programming, and assembly.

It’s not so much that nobody knew about iterative development, or that nobody did it, or that nobody shared their knowledge. Perhaps it just wasn’t in vogue.

Posted in whatevs | 1 Comment