Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Sunday, January 8, 2017

Minimum Viable Controller

The book “NeXTstep Programming Step One: Object-Oriented Applications” by Garfinkel and Mahoney said this about Controllers in 1993:

A good rule of thumb is to place as little code in your controller as necessary. If it is possible to create a second controller that is only used for a particular function, do so – the less complicated you make your application’s objects, the easier they are to debug.

Later, on the same page (p131):

Before you start coding, it’s a good idea to sit down and think about your problem.

Both of these pieces of advice still apply. Neither has been universally internalised 24 years later.

posted by Graham at 11:25  

Tuesday, March 18, 2014

Inside-Out Apps

This article is based on a talk I gave at mdevcon 2014. The talk also included a specific example to demonstrate the approach, but was otherwise a presentation of the following argument.

You probably read this blog because you write apps. Which is kind of cool, because I have been known to use apps. I’d be interested to hear what yours is about.

Not so fast! Let me make this clear first: I only buy page-based apps, they must use Core Data, and I automatically give one star to any app that uses storyboards.

OK, that didn’t sound like it made any sense. Nobody actually chooses which apps to buy based on what technologies or frameworks are used, they choose which apps to buy based on what problems those apps solve. On the experience they derive from using the software.

When we build our apps, the problem we’re solving and the experience we’re providing needs to be at the uppermost of our thoughts. You’re probably already used to doing this in designing an application: Apple’s Human Interface Guidelines describe the creation of an App Definition Statement to guide thinking about what goes into an app and how people will use it:

An app definition statement is a concise, concrete declaration of an app’s main purpose and its intended audience.

Create an app definition statement early in your development effort to help you turn an idea and a list of features into a coherent product that people want to own. Throughout development, use the definition statement to decide if potential features and behaviors make sense.

My suggestion is that you should use this idea to guide your app’s architecture and your class design too. Start from the problem, then work through solving that problem to building your application. I have two reasons: the first will help you, and the second will help me to help you.

The first reason is to promote a decoupling between the problem you’re trying to solve, the design you present for interacting with that solution, and the technologies you choose to implement the solution and its design. Your problem is not “I need a Master-Detail application”, which means that your solution may not be that. In fact, your problem is not that, and it may not make sense to present it that way. Or if it does now, it might not next week.

You see, designers are fickle beasts, and for all their feel-good bloviation about psychology and user experience, most are actually just operating on a combination of trend and whimsy. Last week’s refresh button is this week’s pull gesture is next week’s interaction-free event. Yesterday’s tab bar is today’s hamburger menu is tomorrow’s swipe-in drawer. Last decade’s mouse is this decade’s finger is next decade’s eye motion. Unless your problem is Corinthian leather, that’ll be gone soon. Whatever you’re doing for iOS 7 will change for iOS 8.

So it’s best to decouple your solution from your design, and the easiest way to do that is to solve the problem first and then design a presentation for it. Think about it. If you try to design a train timetable, then you’ll end up with a timetable that happens to contain train details. If you try to solve the problem “how do I know at what time to be on which platform to catch the train to London?”, then you might end up with a timetable, but you might not. And however the design of the solution changes, the problem itself will not: just as the problem of knowing where to catch a train has not changed in over a century.

The same problem that affects design-driven development also affects technology-driven development. This month you want to use Core Data. Then next month, you wish you hadn’t. The following month, you kind of want to again, then later you realise you needed a document database after all and go down that route. Solve the problem without depending on particular libraries, then changing libraries is no big deal, and neither is changing how you deal with those libraries.

It’s starting with the technology that leads to Massive View Controller. If you start by knowing that you need to glue some data to some view via a View Controller, then that’s what you end up with.

This problem is exacerbated, I believe, by a religious adherence to Model-View-Controller. My job here is not to destroy MVC, I am neither an iconoclast nor a sacrificer of sacred cattle. But when you get overly attached to MVC, then you look at every class you create and ask the question “is this a model, a view, or a controller?”. Because this question makes no sense, the answer doesn’t either: anything that isn’t evidently data or evidently graphics gets put into the amorphous “controller” collection, which eventually sucks your entire codebase into its innards like a black hole collapsing under its own weight.

Let’s stick with this “everything is MVC” difficulty for another paragraph, and possibly a bulleted list thereafter. Here are some helpful answers to the “which layer does this go in” questions:

  • does my Core Data stack belong in the model, the view, or the controller? No. Core Data is a persistence service, which your app can call on to save or retrieve data. Often the data will come from the model, but saving and retrieving that data is not itself part of your model.
  • does my networking code belong in the model, the view, or the controller? No. Networking is a communications service, which your app can call on to send or retrieve data. Often the data will come from the model, but sending and retrieving that data is not itself part of your model.
  • is Core Graphics part of the model, the view, or the controller? No. Core Graphics is a display primitive that helps objects represent themselves on the display. Often those objects will be views, but the means by which they represent themselves are part of an external service.

So building an app in a solution-first approach can help focus on what the app does, removing any unfortunate coupling between that and what the app looks like or what the app uses. That’s the bit that helps you. Now, about the other reason for doing this, the reason that makes it easier for me to help you.

When I come to look at your code, and this happens fairly often, I need to work out quickly what it does and what it should do, so that I can work out why there’s a difference between those two things and what I need to do about it. If your app is organised in such a way that I can see how each class contributes to the problem being solved, then I can readily tell where I go for everything I need. If, on the other hand, your project looks like this:

MVC organisation

Then the only thing I can tell is that your app is entirely interchangeable with every other app that claims to be nothing more than MVC. This includes every Rails app, ever. Here’s the thing. I know what MVC is, and how it works. I know what UIKit is, and why Apple thinks everything is a view controller. I get those things, your app doesn’t need to tell me those things again. It needs to reflect not the similarities, which I’ve seen every time I’ve launched Project Builder since around 2000, but the differences, which are the things that make your app special.

OK, so that’s the theory. We should start from the problem, and move to the solution, then adapt the solution onto the presentation and other technology we need to use to get a product we can sell this week. When the technologies and the presentations change, we can adapt onto those new things, to get the product we can sell next week, without having to worry about whether we broke solving the problem. But what’s the practice? How do we do that?

Start with the model.

Remember that, in Apple’s words:

model objects represent knowledge and expertise related to a specific problem domain

so solving the problem first means modelling the problem first. Now you can do this without regard to any particular libraries or technology, although it helps to pick a programming language so that you can actually write something down. In fact, you can start here:

Command line tool

A Foundation command-line tool has everything you need to solve your problem (in fact it contains a few more things than that, to make up for erstwhile deficiencies in the link editor, but we’ll ignore those things). It lets you make objects, and it lets you use all those boring things that were solved by computer scientists back in the 1760s like strings, collections and memory allocation.

So with a combination of the subset of Objective-C, the bits of Foundation that should really be in Foundation, and unit tests to drive the design of the solution, we can solve whatever problem it is that the app needs to solve. There’s just one difficulty, and that is that the problem is only solved for people who know how to send messages to objects. Now we can worry about those fast-moving targets of presentation and technology choice, knowing that the core of our app is a stable, well-tested collection of objects that solve our customers’ problem. We expose aspects of the solution by adapting them onto our choice of user interface, and similarly any other technology dependencies we need to introduce are held at arm’s length. We test that we integrate with them correctly, but not that using them ends up solving the problem.

If something must go, then we drop it, without worrying whether we’ve broken our ability to solve the problem. The libraries and frameworks are just services that we can switch between as we see fit. They help us solve our problem, which is to help everyone else to solve their problem.

And yes, when you come to build the user interface, then model-view-controller will be important. But only in adapting the solution onto the user interface, not as a strategy for stuffing an infinite number of coats onto three coat hooks.


None of the above is new, it’s just how Object-Oriented Programming is supposed to work. In the first part of my MVC series, I investigated Thing-Model-View-Editor and the progress from the “Thing” (a problem in the real world that must be solved) to a “Model” (a representation of that problem and its solution in the computer). That article relied on sources from Trygve Reenskaug, who described (in 1979) the process by which he moved from a Thing to a Model and then to a user interface.

In his 1992 book Object-Oriented Software Engineering: A Use-Case Driven Approach, Ivar Jacobson describes a formalised version of the same motion, based on documented use cases. Some teams replaced use cases with user stories, which look a lot like Reenskaug’s user goals:

An example: To get better control over my finances, I would need to set up a budget; to keep account
of all income and expenditure; and to keep a running comparison between budget and accounts.

Alastair Cockburn described the Ports and Adapters Architecture (earlier known as the hexagonal architecture) in which the application’s use cases (i.e. the ways in which it solves problems) are at the core, and everything else is kept at a distance through adapters which can easily be replaced.

posted by Graham at 00:05  

Tuesday, February 25, 2014

Messily Veneered C

A recap: we saw that Model-View-Controller started life as Thing-Model-View-Editor, a way of approaching problems to design Smalltalk user interfaces. As Smalltalk-80 drifted off from its ivory tower, many Smalltalkers were using and talking about MVC, although any kind of consensus on its meaning was limited by the lack of important documentation.

Would this telephone game continue to redefine Model-View-Controller as object-oriented programming grew beyond the Smalltalk-80 community?

Part 3: Objective-MVC

Objective-C is now best known as the programming language Apple promotes for developing iOS and Mac applications. It was first available on the Mac during the 1980s pre-Cambrian explosion of OOP languages: on the Mac alone you could send messages using Smalltalk-80, Object Pascal, Objective-C, ExperCommon Lisp, Allegro Common Lisp, Object Assembler, Object Logo, and there were probably more. Most of the list here was derived from Object-Oriented Programming for the Macintosh, written by Kurt J. Schmucker then of Productivity Products International.

Which brings us back to Objective-C, which is probably the best known of the products produced by Productivity Products. Objective-C did not let programmers use a spatial programming interface, it made them use linear text files.

Objective-C also did not follow the Model-View-Controller paradigm of Smalltalk-80. As described by Brad Cox in this figure adapted from Object-Oriented Programming: an Evolutionary Approach, the PPI programmers were thinking about Models and Views, and then about reusing Views in different UIs.

Objective-C Layer Architecture

So, what happened to the Controllers? Here’s Cox:

This architecture is very similar to the one used in Smalltalk-80, with one exception. […] The outgoing leg of Smalltalk’s user interface is handled by a hierarchy of views much like the ones discussed here. But the incoming leg is implemented by a separate hierarchy of classes, Controllers, that provide the control for the application. […] The need for the separate controller hierarchy is unclear and is the topic of spirited debate even within the Smalltalk-80 community.

Unfortunately the citation for “spirited debate” is personal communication with various Smalltalkers, so we may never know the content.

This gives us three Objective-C modifications to the original Smalltalk-80 concept that persist to this day. One is the move from a user interface paradigm concerning the interactions of individual objects to a layered architecture. The Cox book shows the display screen as a projection of the view layer onto glass, and the models all in their own two-dimensional layer suspended from the views, hanging:

suspended from the presentation level by pointers in much the way that circuit boards are connected to test equipment with a bed-of-nails testing jig.

That still persists in Cocoa MVC, though with a somewhat weak assertion:

The collection of objects of a certain MVC type in an application is sometimes referred to as a layer—for example, model layer.

The second thing is the abstraction of the Presentation layer away from the display primitives.

The other thing that we still don’t have in ObjC is the Controller. Wait, what? Surely Objective-C applications have Controllers. Apple and NeXT both talked about it, and they have objects with the name “Controller” in them. Surely that’s MVC.

For whatever reason, NeXT kept the ObjC-style Views with their handling of both input and output, but also had to reintroduce the idea of a Controller. So they took the same approach followed by others, notably including Ivar Jacobson a few years later, of defining the Controller layer to contains whatever’s left once you’ve worked out what goes into the Model and View layers.

Now there’s only the question of placement. The Controller can’t go where Smalltalk’s Controllers went, because then it would get in the way of the input events which now need to get to the View. It could go between the two:

A controller object acts as an intermediary between one or more of an application’s view objects and one or more of its model objects. Controller objects are thus a conduit through which view objects learn about changes in model objects and vice versa. Controller objects can also perform setup and coordinating tasks for an application and manage the life cycles of other objects.

Here’s a reproduction of the diagram from that document:

Cocoa MVC

Notice that for the Controllers to act as a mediator between Models and Views, the Views actually have to forward action messages on to the Controllers (via delegation, target-action or the responder chain). Had the Controller been left where it was it would already be receiving those events.

Compare that with the diagram derived from Smalltalk MVC:

Smalltalk MVC

It’s clear that these are different concepts that unfortunately share a name.

posted by Graham at 10:31  

Wednesday, February 5, 2014

Missing Vital Content

After reading Moderately Valuable Cliché, reader Nicholas Levin got in touch to recommend that I look at the back of my Smalltalk-80 books. Here’s the blue one.

Books in the Smalltalk-80 series

The first book mentioned in “Other books in the … Series” is the red book, “Smalltalk-80: The Interactive Programming Environment”. The third is the green book, “Smalltalk-80: Bits of History, Words of Advice”. But the middle one? “Smalltalk-80: Creating a User Interface and Graphical Applications”? No-one ever assigned that one a colour. It was never published.

Discussions of the book in comp.lang.smalltalk in 1986 indicate that it was a manifesto for Model-View-Controller, promoting and describing the pattern for building Smalltalk applications. The thread instead recommends looking at Chapter 9 of a book that has been mentioned on this blog before: Object-Oriented Programming: an evolutionary approach by Brad Cox. More on that later.

So the idea behind MVC was well-known in the Smalltalk user community, but the principle was not documented by the Smalltalk team themselves (as far as I know: it’s possible there are documents I haven’t found) until Adele Goldberg left and founded ParcPlace with others at the end of the 1980s.

Levin also pointed me at a couple of other resources. In this interview for the Computer History Museum, Adele Goldberg confirms that the missing book was “to be an applications level book which was basically the model-view-controller metaphor”.

In an interview for the IEEE Global History Network, Goldberg explains how the whole concept of the Dynabook was to allow people to model real-world problems on their portable computers:

That’s what we thought it was, a PDP-10, which is lightweight compared to what we have now. It was going to be there, like a notepad, but it was going to be a computational-based device that allowed you to build models of the world and test your understanding of the world. What students who succeed really do is, they interact with teachers and parents and other people—and it could be other students—who challenge the models they build of their world; and this is true whether it’s science, math, or social studies: that you’re constantly constructing and deconstructing. And when someone really pushes you to think about your models, make the models explicit, talk about those models, and be able to build those models and then challenge those models—I mean, every well educated person I know remembers in their childhood that that was what was going on all the time.

In other words, Smalltalk is the programming equivalent of “[taking] apart engines and [putting] them back together again, to understand how things work”.

The long time over which the books were written and the fact that the system was being designed in reaction to problems found in documenting it (“In order to get the language book done we kind of organized ourselves so that Dan Ingalls and I, we’d have an aspect of the system we’d need to agree on like, what would be the collection classes? How would you provide collections?”) explains why the blue book’s FinancialHistory example can have no MVC, the red book briefly mentions MVC, and the authors of the ParcPlace paper, looking back, can remember it as a key part of the sample. Perhaps that example was eventually fleshed out for the unprinted book, or was developed further for another reason beyond the publication of the blue book.

We also have a possible reason for the subsequent diversity of interpretations of MVC among applications developers. With no primary reference for the original intention behind MVC, as the Addison-Wesley book was unwritten, people guessed at what Model-View-Controller meant from the bits and pieces that were published or discussed in the community.

posted by Graham at 18:58  

Monday, February 3, 2014

Moderately Valuable Cliché

In part 1 of the MVC story, I examined “Thing-Model-View-Editor”, a pattern[*] extracted by Trygve Reenskaug’s work in Smalltalk-76. By the time Smalltalk-80’s hot air balloon set sail from the ivory tower, there was already a structure called Model-View-Controller.

Part Two: MVC and Smalltalk

[*] Bear in mind that I’m retroactively applying the word “pattern” here. Design Patterns as a thing, modelled on the book modelled on the other book, do not belong in this story for another 15 years or so.

This story will be told through the lens of a single overview article: A Description of the Model-View-Controller User Interface Paradigm in the Smalltalk-80 System by Krasner and Pope is the first. The authors give their affiliation as “ParcPlace Systems, Inc.”, a company that built tools on top of Smalltalk. It was founded by ACM president Adele Goldberg, who originally developed Smalltalk along with Alan Kay and others.

Krasner and pope say that MVC is an application of a particular three-way factoring of a software problem into objects:

separating (1) the parts that represent the model of the underlying application domain from (2) the way the model is presented to the user and from (3) the way the user interacts with it.

They go on to say that it was observed that most Smalltalk-76 applications offer the same interaction—images, buttons, text and menus—and that the goal of the MVC factoring in Smalltalk-80 is to handle those interactions in a common way across all applications.

Smalltalk MVC, adapted from Krasner and Pope 1988

The model and the view hold the same positions that they did in TVME: the models represent the thing in a computerey way, and the views represent the model on the bitmap display. “Controllers are used to send messages to the model, and provide the interface between the model with its associated views and the interactive user interface devices (e.g., keyboard, mouse). Each view may be thought of as being closely associated with a controller, each having exactly one model, but a model may have many view/controller pairs.” The controller is to MVC as the editor was to TVME.

So, the controller is the thing that senses and responds to UI events. Because you probably only intend for one thing to happen when you press a key or click a mouse button, only one controller is active at a time. That controller can interpret your command as a requirement to edit or query the model, or to display or change a particular view.

The views can also edit or query the model. So, there’s a need for the model to let the controller know when it was changed by a view, and the views should know when the model was changed by the controller. Enter dependents!

Dependents represent a realisation of what would later be known as the Observer pattern. The model can keep track of a collection of the objects that care about its changes. When it changes, it sends a changed or changed: message to its dependents.

Smalltalk-80 supplies bundled abstract classes for each of Model, View and Controller. The article goes into each of these classes and some of the subclasses that have already been created, showing how inspectors, editors and debuggers in Smalltalk-80 are all built from this factored collection of objects. It also details complete applications, including the FinancialHistory tutorial from “Smalltalk-80: the Language and its Implementation” by Goldberg and Robson.

Looking in the blue book, as it’s known, none of the terms MVC, model, view, nor controller appear in its index. The classes Model, View and Controller are not in the example class index, and the dependents, changed and changed: methods are not in its implementation index. The FinancialHistory object is described throughout the book, but is a subclass of Object and does not participate in any factoring that’s similar to MVC.

It’s probable that this reference is a mistake, and that Krasner and Pope intended to refer to the red book “Smalltalk-80: the Interactive Programming Environment” by Goldberg. This does indeed describe MVC, but only in relationship to the “MVC Inspector”. However, the description of the inspector shows that the pattern was common enough to motivate the construction of bespoke UI:

Many objects in the Smalltalk-80 system are closely related to one another. In particular, the user interface of the system is implemented using subclasses of two classes, class View and class Controller. A View represents ways of presenting information on the screen; a Controller represents ways in which the user interacts with a screen view. Any View is related to a Controller and to another object, the object whose information is accessed in the View. This other object is referred to as a “model.” Whenever you inspect a View, you typically want to inspect its related model or Controller. There is a special inspector in the system for inspecting the two main objects related to a View, as well as the View, whenever a View is sent the message inspect.

Having mentioned the red and blue books, it seems appropriate to introduce the green book, “Smalltalk-80: Bits of History, Words of Advice” edited by Glenn Krasner (the first author of the paper we started at). There is no MVC in the green book, but the three books[*] together are a fascinating snapshot of this time in our industry.

[*] There is a purple book too, but it’s just half of the blue book repackaged.

So Smalltalk-80 MVC is a formalisation of the abstract collaboration between objects documented as Thing-Model-View-Editor. It is an idiom named after the developer tool that was designed to take advantage of the common relationships between these objects. At some point after its initial documentation, authors (Krasner and Pope along with others) decided that this was not merely a factoring, but a “User Interface Paradigm”.

posted by Graham at 00:11  

Tuesday, January 21, 2014

Meaningless Vapid Catchphrase

On the 4th December 2013, I said:

Urge to search the archives for papers on Model-View-Controller and write an essay on its ever-changing meaning in programmer discourse.

Do you have any idea how much work that is? I do, now. So I’m going to cover a very small part of the story here, with a view—your interest and my inclination permitting—to adding to it in subsequent posts.

Part One: you’ve lost your thing

The genesis of MVC comes in 1979 with a pair of memos written by Trygve Reenskaug of the Xerox Learning Research Group. In the first, A Note on Dynabook Requirements, Reenskaug documents an approach to tackling problems in the Smalltalk environment by describing his design for a project management task.

A fun digression at this point is the observation that Reenskaug appears to have described what we now call the user story: a goal-directed statement of what someone’s trying to achieve, as a replacement for a specification of what some theoretical software product should do.

Most problems would start with a rather unclear and often self-contradictory goal. Some examples: I would like to get better control over my finances; I don’t want to be troubled with detailed accounting; I want to settle my account with the butcher; I want to know more about Tarot cards; or I want a small, cheap house with many large, luxurious rooms.

I would expect the user to go more or less subconsciously through a goal-means hierarchy: Certain means are needed to reach a given goal. These means are not immediately available, but constitute a new set of part-goals, each of which needs certain means for their satisfaction, and so on.

An example: To get better control over my finances, I would need to set up a budget; to keep account of all income and expenditure; and to keep a running comparison between budget and accounts. Three new, non-trivial goals that need further consideration.

He goes on to note that given a sufficiently consistent collection of goals, the computer should probably just solve the problem itself. In the absence of evidence that this is possible (Prolog notwithstanding), we’ll need to tell the computer the methods by which the problem is solved.

Anyway, back to MVC. The solution in the context of his project management problem then follows. Along the way he introduces some metaphors that are formalised in a glossary in the second memo, called not Model-View-Controller but Thing-Model-View-Editor: an Example from a planningsystem.

An immediate observation is that where MVC has three parts, TMVE has four: the thing being modelled is explicitly part of the problem. MVC is a way to do computer stuff, TVME is a way to use a computer stuff to solve a problem based in the real world. Of course, you cannot put the thing itself inside the computer, except in the world of Tron which was yet to be released. So instead you find some useful abstraction and represent it in the computer as a model. This model contains both the data and the actions appropriate to the abstraction it represents.

The image below, an adaptation of a figure that appears in both documents, demonstrates that a nebulous thing can be modelled by multiple different abstractions, or that multiple models can represent different parts of the thing.

Models bound a thing for representation in a computer.

Now for any model, there will be one or more views that represent the model in a meaningful way; models do not know how to draw or print themselves. Views can also control the model in ways appropriate to the view’s representation. For example, Reenskaug shows that a view describing a model’s properties could accept changes to those properties, and make the required changes to the model.

Now there’s going to be a complicated object graph in place, with one or more model objects each represented by one or more views, all to solve a problem with one particular thing. The editor is a coordinator for all of these. It acts as a command interface, mediating between the user and this network of objects. As changes are made, the editor coordinates with all of the views.

You will recall that Trygve Reenskaug was on the Learning Research Group at Xerox. A Smalltalk system like the proposed Dynabook is supposed to be a computer that lets people solve their own problems, by being easy to program and to manipulate. Therefore, unlike modern MVC which is a design tool for professional programmers, Thing-Model-View-Editor is a user interface paradigm, describing how people can build their own programs to solve their own problems. The editor may have a doit command to send Smalltalk messages, but sending Smalltalk messages should not be beyond the wit of an interested and engaged user.

This goes some of the way to explaining why “computery stuff” like persistence, networking and the like are not tackled in TVME. They are outside its purview: the computer should be looking after that stuff. Just as the humane interface on a Canon Cat has no save or load buttons, and the computer looks after storage itself, so Dynabooks do not have databases, or filesystems, or memory allocators. These are abstractions of the thing “computer”, but Thing-Model-View-Editor is concerned with the thing that people want to represent in the computer. The computer is not the thing.

posted by Graham at 13:11  

Powered by WordPress