This episode is the first to discuss the 1969 NATO conference on Software Engineering Techniques. We find out that there isn’t a NATO-sponsored international institute of software engineering, though only a little about why; that computer scientists and software engineers have been ignoring each other for almost 60 years now, if not longer; and that if you want to propose a way to ensure software is correct, you can rely on Edsger Dijkstra for a snarky rejoinder.
The podcast is now available on YouTube!
The episode is supported by members of the Chiron Codex Patreon (use this gift link for your first month free), so please do join the community or hit the Ko-Fi button to make a one-off donation.
Links
- Software Engineering Techniques: Report on a conference sponsored by the NATO Science Committee
- The NATO Software Engineering conferences
- Computer History Museum — Iconic Voices of the Digital Age
- Curriculum 68: Recommendations for academic programs in computer science: a report of the ACM curriculum committee on computer science
- BCS Academic Accreditation Guidelines
- Computing Curricula 2020: Paradigms for Global Computing Education
- Research ethics committees: values and power in higher education
- The Humble Programmer
- Infer
- Issue 95: Fortran
Transcript
Hello, and welcome to episode 61 of the Structure and Interpretation of Computer
Programmers podcast. I’m Graham Lee, and this episode, which introduces the 1969
NATO Science Committee Conference on Software Engineering Techniques, is the
fifth part in a mini-series exploring the NATO Software Engineering Conferences.
The episode is sponsored by you, the software engineering community. By the way,
this episode is the first I’ve made since adding the podcast feed to YouTube.
That addition was long overdue, and if you’d rather listen over there or you
have friends who would listen on YouTube, check out the playlist that’s linked
in the show notes.
The 1969 conference, which was held in Rome, is very different from the 1968
conference in Garmisch, which has been the topic of the last four episodes of
the podcast. The 1968 conference was convened because people believed that
software development could be improved, that they didn’t know how it could be
improved, and that engineering practices might provide the answer. Briefly
recapping those episodes, the conference discovered that most software works
fine, thank you very much. The problem comes in projects that are both cutting
edge and large, and that the big ideas for fixing things are to get more and
earlier feedback about software, including interleaving testing and
implementation.
The 1969 conference had two aims. To dig deeper into the technical issues from
the 1968 conference, leaving behind the personnel and management problems that
were discussed in depth at that conference, and which continue to be some of the
biggest blockers to software engineering to this very day. That was one of the
goals, and that was the one that the report editors were most interested in. The
second aim that many attendees had was to advocate for a proposal to create a
NATO-funded International Software Engineering Institute. We know that there
isn’t a NATO-funded International Software Engineering Institute. We know from
Brian Randell’s recollections that the discussions were pretty fractious, and we
also don’t have any record, because he and John Buxton, who were the editors of
the conference report, decided, probably for political reasons, not to cover
those points in the report.
It’s this sort of thing that makes oral histories of a profession, such as those
in software collected and maintained by the Computer History Museum in Mountain
View, critical. The documented record of software engineering leaves out the
inconvenient parts, but those rough edges are where some of the pretty impactful
decisions get made.
What Buxton and Randell did record is the technical basis of the disagreement,
which forms the whole of the first section of the conference report. That is,
the distinction between theory and practice. There are actually three camps in
evidence. Theoreticians, who feel like they aren’t allowed to speak at the
conference because they’re not saying anything practical. Practitioners, who
feel like they aren’t allowed to speak at the conference because they don’t have
any theoretical basis for their claims. And Niklaus Wirth and Edsger Dijkstra,
who seem to be very happy to speak at the conference, who feel like their work
is both theoretically sound and practically useful, and that people need to stop
making the distinction between theory and practice.
The summary of the shape of this disagreement between theory and practice was
given by Christopher Strachey, who was, at the time, directing the programming
research group at Oxford. His address in the conference report is given as 45
Banbury Road, an address which has since been consigned to history. Actually, in
two senses. Firstly, the Computer Science Department is now slightly closer to
town, as it’s in two sites on Parks Road, and the building at 45 Banbury Road is
now used by the History faculty.
Strachey frames the debate with the two questions, can computing science be of
any assistance to software engineering, and what can computing science get out
of software engineering? Before we move on to how that landed with the
conference delegates, let’s play that forward and see how the industry has
responded in the subsequent decades.
Basically, my abbreviated and highly flippant take is that at the time of the
NATO conferences, the Association of Computing Machinery were not only
encouraging university computing departments to do computer science instead of
informatics, cybernetics, information theory, or related topics, but were also
defining computer science in their “Curriculum 68” to be the mathematics of
algorithms, a theoretical discipline that is so separate from practice that when
I was working in Oxford’s computer science department in 2019 as a research
software engineer, they were hiring for a senior prof, and some of the
candidates didn’t even need a computer to do their research.
For most of history, this was broadly the state of affairs. Software people
learned on the job. Some of them started with computer science degrees, but that
mostly didn’t set them apart. Software engineers were hired either through
nonsensical aptitude tests that were mostly geared towards discovering
introverted middle-class men, or programming challenges that determined
something similar to whether or not they could program because there was no
other signal to work with. This situation probably hit its peak with my
generation, the Xennials, who grew up with microcomputers and so entered the
workforce with 20 years of practical experience, and a uniform of combat
trousers and band or conference t-shirts that they adapted from cyberpunk novels.
Approximately a decade later, Generation Y had grown up with computers that were
less accessible, Salow’s paradox finally resolved itself and everybody wanted to
use computers because of the internet. Some applications suddenly became web
scale, and so the theoretical study of algorithms became the in thing. The
situation reversed, and people rushed into the computer science schools, and
rushed out into internships with the technology giants.
All of these approaches are pretty bad at generating a representative workforce,
at the time when computing finally lives up to its promise of entering all
aspects of society, or eating the world as Marc Andreessen put it. So then we
get the coding boot camp generation, where the pendulum swings back towards
practical experience over computer science theory.
Meanwhile, there have been a small number of academic institutions offering
“software engineering” courses (I have one of those degrees) and a few
researchers in computer science department doing “software engineering” research
(my doctorate is one of those). But much of the research isn’t read or acted on
by practitioners, for the straw man reason that Strakey gave in the conference
in 1969: “Well, there’s nothing we can get out of computer science, look at all
this rubbish they’re talking”. People eat up research that supports techniques
they want to use, or that repudiates techniques that they don’t want to use, but
without necessarily reading or otherwise engaging with it beyond the titles and
abstracts. Software engineering research mostly exists to get published in
software engineering journals, and cited by other software engineering
researchers, because that’s how you make a career in software engineering
research.
Computer science degrees do need to have a practical component these days to get
accredited in many places. For example, here in the UK, the BCS—full name BCS,
the Chartered Institute for IT. It used to be called the British Computer
Society—The BCS accredits CS curricula, and requires that degrees have both
theoretical and practical learning outcomes. A lot of the practical contribution
is farmed out to industry internships, or co-created projects that are in
practice led by industrial partners. Similarly, in the US, the ACM curriculum
2020 report now includes software development fundamentals in the CS degree
competencies, and two criteria for software engineering learning outcomes. Those
are reviewing a set of software requirements and designing a system in the UML.
That is two competencies out of the 56 that they list for a full software
engineering degree.
Software engineering is still taught on the job, with consultants doing a lot of
what is called coercive mimesis. That is the business of making teams all do it
one way, because that’s the way that other teams do it. Said consultants get
angry when people want to do it another way, even though the only evidence
supplied by adherence of both of these approaches, either doing it the way the
consultant wants, or doing it the way the team is currently doing it, is
survivor bias.
Okay, tangent aside, back to 1969. Strachey’s problem with the discussion of how
much software engineering can help CS, or vice versa, is that, and I’ve already
quoted this, “it’s rubbish”. Recapitulating an important point from the 1968
conference, he notes that some projects “have been quite astonishingly
successful”. The problem CS has in convincing software engineers to adopt their
techniques, and he uses recursive programming as an example of such a technique,
is that they, the practicing software engineers, “haven’t thought seriously of
doing it. They’ve been told to do it, and they brush it away, apparently because
it hasn’t got the right sort of software support, or because their machine
doesn’t do it easily, or because they don’t know about it.” This reminds me of
almost every single argument both for and against adopting TDD, and probably
applies to every other novel software engineering practice. Some people say you
should do it. Others explain why it doesn’t make sense in their specific
context. Nobody is correct.
Or, as Strachey says, a computer scientist can’t expect a huge software project
to change direction just because they say so. But the project leader can’t
expect the CS department to spin up a 500-person-year project to demonstrate the
benefits of their technique just because they say so either.
The discussion by the rest of the conference attendees of Strachey’s soliloquy
on the theory-practice divide quickly gets lost in the weeds. J.D. Aron of IBM,
sometime president of the IBM Federal Systems Centre, sometime contributor to
the SAGE automated defence platform, sometime editor-in-chief of the IBM series
on System Programming, notes that many projects go wrong because of a lack of
appropriate selection of scientifically recommended methods, when the point that
Strachey was making is that science doesn’t have a mechanism for recommending
the appropriate methods. Roger Needham at Cambridge University asks how big a
project needs to be before it can be considered an appropriate pilot project,
when the point is that even when faced with a pilot project that successfully
demonstrates a technique, people feel empowered to ignore that and make software
the way they like to do it.
Indeed, Dijkstra summarises this last point well. People welcome new software
engineering techniques and tools, as long as they don’t have to change their
thinking habits, programming tools, hardware, tasks or organisational structure.
Those listeners who have seen companies that claim to have undergone agile
transformations and are using SAFe, or who remember the gap between
object-oriented programming as promised in Smalltalk-80 and as delivered in
Java, probably need to reduce the amount you’re nodding your heads along at this
point to avoid sustaining neck injuries.
Ed David from Bell Labs suggests replacing the pilot project with research
investigations of production software projects, which is what IBM research,
Microsoft research and others did and continue to do exceptionally well. It’s
the source of almost all of today’s research into LLMs, as LLM companies tend to
have enough hardware to develop and research LLMs and universities tend not to.
And of course today, researchers can do the same using open source software
projects. Projects that represent the output of huge amounts of effort put into
making software, because people need to use the software, and which allow the
freedom for academics to demonstrate their practices in realistic, because real,
situations. Of course there are real ethical issues related to using a live
project with actual human contributors as a test bed for any old novel idea that
comes out of academia. And it’s not entirely clear that the way academics
navigate those issues is entirely great, but it is important to navigate them.
This is the advert break. It starts now.
This episode is brought to you by me, Graham Lee. But really, by you. Chiron
Codex is a community of people who are learning how to become better software
engineers by adopting AI augmentation in a thoughtful way. We aren’t outsourcing
our understanding to coding assistants like Claude or Codex, but becoming
software engineering centaurs by using AI tools to improve our knowledge and the
quality of our work. Join the community over on Patreon to find out about
interaction patterns that improve your work with AI coding tools, running LLMs
for software development locally, discussions of recent research in the field,
and more. If you’re a software engineer who’s interested in the promise of AI
tools, but sceptical about handing your skills over to the computer, this is the
community for you. Go to patreon.com slash Chiron Codex, that’s
C-H-I-R-O-N-C-O-D-E-X, now for more information and to join. Use the gift link
in the show notes to get your first month of insider access completely free.
Alternatively, you can show your appreciation by donating at Ko-fi, that’s
ko-fi.com slash ko-fi.com. Direct support by my audience is the only revenue I
get for my work as a software engineer and communicator, so your support really
means a lot to me and makes it possible for me to produce this podcast. Thank
you so much.
That was the advert break. It’s over now.
The second section of the report focuses on software specification. Right out of
the gate, the editors ignore one of the more commonly repeated points of
discussion in the 1968 conference, that the specification needs to be iterative
and rely on feedback received during development. They describe a specification
as serving as “the foundation of the subsequent system implementation”. Luckily,
the actual conference attendees were more on the ball. Discussing Seegmüller’s
IBM-derived checklist of system specification, multiple people criticised it for
not offering enough opportunities for a team to restart the specification
process from scratch as they discover new information. Scalzi, who is also from
IBM but a different office in a different country, points out that OS/360 had
been going through evolutionary and incremental development since it was
released.
Subsequent specification discussion, at least as presented in the report, is
split into two sections, specification languages and implementation languages.
I’d wager that many programmers have only really thought about implementation
languages, but it was a really big deal in the 1960s that there were two
different types of language that software teams used. It’s still in fact a
useful distinction, so it’s worth going into more detail.
I’ll start by expressing the limitations of implementation languages, so that
you and I have a shared understanding of the gap into which a specification
language could fit. An implementation language, at the time say FORTRAN or COBOL
or PL/1, but these days something like Swift or Kotlin or Rust, tells you how
the software does what it does, what instructions the computer performs in what
order (albeit in a somewhat abstracted way, at least in the modern case, a point
we’ll get back to when we catch up to the conference reports discussion of
implementation languages), and what memory locations the computer uses to store
variables. From that you can derive an understanding of what the software does,
but that information isn’t front and centre, and it’s a description that’s
focused on what transforms the computer goes through, not what the people and
systems surrounding the software go through.
A specification language then is a language in which you can express a
description of what a software system does in reaction to what the people and
systems surrounding it do, without cluttering it up with information about how
it does it. That means not only eliding information about what variables the
computer stores and operates on, but what algorithms entirely. To paraphrase an
example that Jerome Feldman, who was then at Stanford, gives in the report, you
might just say the software calculates the square root, without mentioning
whether it uses the Newton-Raphson method, or logarithm tables, or some other
approach.
The specification technique that gets most coverage in this report is the Vienna
Development Method, created by people at IBM’s office in Vienna, Austria. That
deserves its whole own episode (and if anyone has literature on the Vienna
Method that they are able to share with me please get in touch using the details
at the end of the episode). The Vienna Method is similar in scope to something
like the Z notation. You describe the data structures in the system and the
operations in the system in terms of the effects on the data structures, and you
design the system’s behaviour as a composition of those operations, or you
design the operations as refinements of a description of the system’s behaviour,
or you do both. Recall the top-down versus bottom-up arguments in the previous
conference that we encountered in episodes 58 and 60. When you have sufficient
detail to understand the design, you use an implementation language to provide
executable examples of the operations that you listed in your specification
language, the operations required to fulfil the specification. Once you have
that, in theory at least, everything works.
Programming teams in the 1960s commonly used different languages for
specification and implementation, even in situations where the specification
language they use could in principle be compiled into executable code, like APL,
which was mentioned in the 1968 report under the name of “the Iverson notation”,
or ALGOL. Part of the reason might have been not trusting or not paying for the
compiler on a particular computer system, or that compiler not even being
available, with programmers hand-compiling the specification and implementing
operations in assembler. That’s an approach that the conference attendees
bemoan, expressing what was then a forlorn hope for a machine-independent
implementation language that nonetheless produces efficient code on every
hardware configuration it supports. Well, eventually Fortran became that, and
Adrian and I discussed Fortran in the August 2026 issue of De Programmatica
Ipsum, for which there’s a link in the show notes. But the sine qua non example
is of course C, which was still a few years away from the Rome attendees, even
those of them who worked at the Bell Labs. And the way that we made sure C
compilers could produce efficient code on every hardware configuration was we
ended up redesigning all of the hardware so that it could produce efficient C.
Actually, going back to APL, Alan Perlis noted that APL and LISP are both very
good as specification languages. Because their adherents, “much like both ardent
Catholics and Communists”, understand their frameworks and provisions so well
that they can think about a system design in terms of those provisions. In other
words, if you describe a desired system to an APL programmer, they immediately
start thinking about the APL they would use to implement that system.
Of the attendees at the conference, apparently only Adin Folkoff, who’s also
from IBM and who worked a lot on APL, thought that English was appropriate as a
specification language. His argument is presented in one of the included
position papers in Section 7 of the report, so we’ll leave him alone for now and
come back to that in a later episode.
For now, let’s take a look at the next section of the report, Section 3 and
Software Quality, then we’ll call it a day for this episode. Section 3 opens
with a fascinating glimpse into the sociology of software releases. ME Hopkins,
yet another IBMer—IBM had 11 delegates at a conference of 63 people—reports that
OS/360 consistently has about 1,000 problem reports in every release. Now why is
that? He suggests some alternatives. That problem report frequency is connected
to the release cycle. That problem report frequency scales with number of field
engineers. Or that engineering management don’t feel comfortable adding new
features to the system until there are fewer than 1,000 problem reports in the
existing scope. Sadly, I haven’t seen evidence that these hypotheses were ever
tested, but that would make for a really important contribution to the
psychology of release schedules and of problem reporting in software.
Within software quality, there are two subsections on “correctness”; the first
being “formal correctness”, by which the editors mean the use of mathematical
proofs to demonstrate that software satisfies its requirements, the second being
“debugging”, by which they mean handling the fact that you were wrong about the
correctness. Niklaus Wirth observes that he designed Pascal with some of the
ideas from formal verification in mind, another of those maddeningly laconic
quotes from a working paper that we don’t get to read in full. This one, which
is called “The programming language Pascal and its design criteria”, does appear
to have been published elsewhere and people have cited it, but my university
library doesn’t have a copy. Again, if you can send it, please get in contact.
The common approach to formal correctness is the one that Tony Hoare originally
described, and that forms the basis of Eiffel’s design by contract. At every
point, list the preconditions that must be true for the next statement to work
correctly, and the postconditions that that statement guarantees to be true
after it works correctly. Your design of an operation is correct if the
preconditions for the first statement in the operation include the situation
that your customer finds themself in, and the postconditions after the last
statement has executed include the results that your customer wants, and that
the outcome of each statement is compatible with successfully executing the next
statement.
At the conference, Hoare suggests another approach to correctness in this
section of the conference report, and it’s one that includes testing. Construct
a proof by induction that if the software is correct in case k, it’s also
correct in case k plus 1, k plus 2, etc. Write a test that shows that the
software is correct in case k, and then you know the software to be correct.
This is somewhat related to what we now call “equivalence partitioning”, though
that’s more showing that the software is correct in one situation, and that
there are a whole class of situations where it behaves the same way. Of course,
mentioning tests at all brings Dijkstra out in hives, so he says “testing shows
the presence, not the absence, of bugs”. In context of responding to Hoare’s
statement (which he might not have been of course, we really have no information
on how the editors compiled the conference report), but in that context, this is
a really hollow statement. Nonetheless, it’s one that Dijkstra was particularly
proud of, so he recycled it in his 1972 Turing Award acceptance speech, “The
Humble Programmer”. And appearing in that speech has led to the quote, gaining a
life as a pithy aphorism for the ages, regardless of its veracity, or indeed its
relevance in its original context.
Onto debugging then. And Aron distinguishes between debugging, as discovering
mistakes a programmer made in their own work, and testing, as discovering
problems that occur on integrating components into a complete system. That means
within Erin’s framework that unit testing would be considered a debugging
technique, and I wonder whether, for some developers, it hasn’t entirely
replaced “traditional” debugging. It almost has for me. Most of my use of what
we call debugging tools is restricted to setting exception breakpoints, and
trying to work backwards to discover how the software got to a point where it
raised an exception, or setting watchpoints and trying to follow how the program
logic affects a particular variable. I make much less use of the live
state-changing behavior of debugger tools than I used to, instead writing tests
that explore interesting conditions, and investigating how the outcome deviates
from my expectations.
Wirth and Perlis fall out over whether “online debugging”—i.e. the thing that I
was just talking about, where you interact with a running program—encourages
sloppy thinking by making it too easy to create an almost working program, then
debug it into correctness. Wirth thinks that it does encourage sloppy thinking.
Perlis thinks that it’s an improvement over previous tools. If you have any
other examples of software engineers gatekeeping that the previous way to do
things was the One True Way, and that the new tools enable dangerous slop
because they’re too easy to use, just leave them in a pile by the door.
Debugging ends with a discussion of automatic analysis tools. This is a topic
that’s similar to the discussion of specification-slash-implementation
languages. One reason that we don’t just program in specification languages and
let the computer work out what to do (any prolog programmers who are listening
can note my email address is given at the end of the episode); it’s difficult to
write efficient solvers for turning the specification into implementation. In
other words, programming is a useful mental skill. While the same has proven to
be true of automated analysis tools, you can easily show that particular
programs are undesirable in that they crash, or run out of memory, or handle
untrusted input, or whatever. It’s proven much harder to write a tool that
demonstrates that a particular program actually does what you or your customer
desires.
There’s still, of course, loads of value in a type of analysis tool that shows a
program definitely isn’t what you want. Back when I worked at Facebook in around
2014, I went to a very enjoyable internal symposium on static analysis tools,
which was held in Sicily. The problem of static Objective-C analysis came up,
and people said that they could only demonstrate correctness within a method,
because things like method swizzling and forwarding mean you can’t trace a
program across message boundaries. I pointed out that while that’s entirely true
in theory, it mostly isn’t true in practice. Approximately 100% of Objective-C
programs don’t use those fancy runtime techniques. And you can treat the
program’s control flow as fully transparent unless you encounter one of those
constructs. The team considered that assessment, and now the Infer analysis tool
supports Objective-C. You’re welcome.
I’m skipping the one page in the report on performance measurements and
improvement, because it mostly discusses old techniques where the program has
access to various clocks, and the programmer has access to hardware monitors.
When the clock is running at multiple gigahertz, watching the blinkenlichter
flicker isn’t a feasible way to understand your software’s performance. We have
similar things these days where processors internally track events in counters
that programs can interrogate. But it’s worth leaving that to episodes
specifically on that topic, and not trying to connect it to this brief and
largely these days outdated segment of the 1969 conference.
So, that’s three out of the seven sections in the conference report. And, I
think, quite a good flavour of how the whole thing pans out. It’s still a shame
that we don’t have the content on the proposal to build Software Engineering
Institute, of course. Stay tuned for discussions on software flexibility, large
systems, education, and what actually proves to be a very interesting collection
of working papers submitted by attendees. In the meantime, you can leave your
thoughts about this episode at the post on sicpers.info, S-I-C-P-E-R-S.info, or
email me, grahamlee at acm.org. Thanks so much for listening. If you enjoyed
this podcast, please share the link with your friends or colleagues, and
consider supporting the podcast on Patreon or Ko-fi. Take care, and we’ll talk
soon.
