# Structure and Interpretation of Computer Programmers

> From programmer to software engineer.

> Contact: graham@iamleeg.com

### Posts

#### (No Title)
Wooo!  New blog.  For the first few posts, this is going to be about my attempts to get OpenStep 4.2 running under parallels VM for Mac.  So far, I've set up the VM and am wading through the release notes and installation guide.  I've found that it should be installed without the (virtual) network cable connected, so that's what we do.Worryingly, the install notes say that we require a SCSI CD-ROM, but Parallels only provides an IDE one.  I don't know how that will pan out, I definitely remember one version of NeXTSTEP coming with an IDE CD-ROM driver.  Let's see.OK, so I've got the OpenStep boot floppy up and running (via some failures of NeXTSTEP to provide the necessary device file to dd, and with tremendous thanks to Simon Cozens for lending use of his /dev/fd0), the compatibility BIOS isn't an issue and the installer dutifully loads:Now to find the drivers disk...

#### (No Title)
And now the console is up: a few graphics gremlins but the ATAPI CD-ROM driver is doing its business.

#### (No Title)
And now, the reboot:it's been sat there for a while now... :-/

#### (No Title)
We're in!  The WindowServer has no graphics glitches that I can see, and it's time to configure the devices.Graphics: default VGA.  Play it safe :-).  We may be able to do something more fancy, but I'd rather check with Parallels before b0rking my install.Network: ah.  The emulated card is a Realtek 8029(AS), and OpenStep doesn't claim to have support for that.  Apparently it's compatible with the NE-2000, but OpenStep doesn't have a driver for that either :-(.  I need to ask someone with more Intel-version-of-NeXT noodle than I about that...Sound: we ignored this for the moment.SCSI: Nope.So now we're at the 'installing additional packages' stage, I won't post a screenshot until we're into the OS.

#### (No Title)
So, here's the full system:Networking is the biggest current issue, and replacing the default VGA display with something better would be useful, too.  But it works :-)Oh, and images of the driver disks are available from Apple.

#### (No Title)
Is Social Psychology Biased Against Republicans?

Pretty interesting, and an often unmentioned aspect of diversity (probably because political leaning is supposed to be a secret in democratic countries, if not because it's usually acceptable to display ingroup/outgroup bias politically). But it's very relevant in the social sciences, especially if it means that particular political views are more likely to be treated favourably or argued for by researchers.

#### (No Title)
The next phase in technological convergence will be harder than the last, because it can't be solved with technology. Last time the devices converged, some phone makers just needed to buy a photoelectric detector, a lens, and licenses for some MP3 patents.

But how can the various tab-sized computers I carry - my bank cards, SIMs, passport, building door card, transport cards, office ID - be integrated, when they mean different things to different people? Technologically, it's really bad to keep them separate because you can't just hold a bag of RFID tokens up to a reader and expect the right thing to happen. Socially, it's really bad to converge them; I can't imagine my bank, employer, train conductor and government all agreeing to the same terms on identifying me, nor would one company be an acceptable clearing house for all of that so rather than N distinct tokens we'd end up with N tokens you can use Anywhere™*.

#### (No Title)
Apple noticed there are programmers outside the valley

If my summary sounds cynical, it's because I'm cynical of the old Apple way where they only hired engineers who wanted to relocate within the shadow of (whatever the big thing in SV is: Stanford now, but probably HP when Apple was younger). I'm excited that they'll get to hire from a broader range of applicants as they stagger, eyes blinking, into the wide world outside Cupertino.

#### (No Title)
http://shape-of-code.coding-guidelines.com/2014/11/06/cobol-2014-perhaps-the-definitive-final-version-of-the-language/

Cobol now has function pointers :O

#### (No Title)
Joe Armstrong thinks we don't need modules in software. Instead, all functions should have unique names and be published in a global database.

#### "Brand": you win some, you lose some
The 20th anniversary of the iMac reminded me that while many people capitalises the word "iMac" as Apple would like, including John "I never capitalise trademarks the way companies like" Gruber, nobody uses the article-less form that Apple does:

  So you can do everything you love to do on iMac.

I, like many other people, would insert 'an' in there, and Apple have lost that battle. There's probably somebody in Elephant who has chosen that hill to die on.

#### "Patently" obvious
Due to a lack of digit extraction I'm not at FOSDEM this weekend.  That's unfortunate because as well as catching up with my friends at Brainstorm and on GNUstep, I really enjoyed the weekend last year and drank plenty of great Belgian beer and ate plenty of nice moules-frites.

So I've been spiritually living the Free lifestyle by reading what RMS and Torvalds have to say.  Mostly I've been going over the essays in Free Software, Free Society.  I find it very easy to accept the premises RMS uses, easy to follow, comprehend and agree with the arguments he presents but then somehow (perhaps for illogical reasons on my part, his part or both) hard to agree that the conclusions he draws are inevitable.

For instance, I agree that copyright law exists directly to benefit the public, and indirectly to benefit the authors (by providing incentives for authors in the shape of limited term monopoly over their authored content) and not at all to benefit Industry Associations.  It even says that here, in the first ever copyright law: ...for the Encouragement of Learned Men to Compose and Write useful Books; May it please Your Majesty, that it may be Enacted... certainly doesn't seem to mention greedy lawyers or management.

Letters patent were never created for the same reason, of course.  But because it became clear that patents from the Crown were obtained uppon Misinformacions and untrue pretences of publique good, many such Graunts have bene undulie obteyned and unlawfullie putt in execucion, to the greate Greevance and Inconvenience of your Majesties Subjects, contrary to the Lawes of this your Realme, and contrary to your Majesties royall and blessed Intencion soe published, so the whole system was rebooted so that patents were only grantable ... to the true and first Inventor and Inventors of such Manufactures, [...] soe as alsoe they be not contrary to the Lawe nor mischievous to the State, by raisinge prices of Commodities at home, or hurt of Trade, or generallie inconvenient....

The situation we find ourselves in now is that industries claim copyrights and inventions from the authors and inventors and lobby for more and more restrictive variants of the above laws, ignoring the previously-granted rights of the public at large and extending the previously-ungranted rights of the rights-owners, simultaneously removing those rights from the people granted the rights in the first place.  So why in the case of copyright do the FSF assume copyright, but in the case of patents they refuse to deal with them?  That inconsistency I don't understand.

#### "Patently" secure
One thing that occasionally becomes interesting about working in security is that doing security and managing business have a great deal of overlap. This makes a lot of sense: a business wants to be profitable, and profit is a reward conferred by the market for taking on some risk. But too much risk can expose your business to undesirable failures, so understanding and controlling your exposure to risk is a useful exercise.

Well that's fundamentally how security works too. There is some reward to be gained by performing the activity allowed by an app: that might be the enjoyment of playing a game, the cost savings of keeping track of your finances, or the health benefits of seeing what food you consume. But using the app also brings some risk, and so security people seek to quantify and reduce the risk inherent in any app.

I'm going to compare a business risk (infringing on another's patent) to an information security risk (leaking confidential data) to show just how similar these fields are. I choose patent infringement because it's an apposite case: however you'll find that I don't name particular patents or companies for reasons that will be entered into below. Suffice it to say that I have dealt with software patent lawyers in the past and have some - but not much, by any means - experience of how the US patent system operates. If you choose to infer any advice from this blog post, please seek appropriate counsel before acting on it: I am not a lawyer, and I am certainly not your lawyer.

Quantisation

A risk to either a business or a user can be summed up by the expected damage caused by the event coming to pass. That is, the estimated cost (financial, emotional, intangible etc.) of the risky event multiplied by the expected probability of that event happening.

In the leaky data case, the expected damage would be "what chance is there that an attacker will retrieve the data" × "what is the impact to the user of exposing the data"? Both of these parameters are hard to quantify: information about data security breaches is notoriously hard to get hold of because companies are reluctant to talk about problems they've had protecting their customer records. Combine with that the fact that in many fields even the direct costs of a breach are hard to arrive at, and you end up multiplying two very big error bars together.

In the infringement case, things are a bit more straightforward. Legal reports are - in many jurisdictions - a matter of public record, so seeing what the damage of a case "like yours" is going to be is quite easy. That covers direct costs, anyway: indirect costs like lost custom, damaged reputation etc. are harder to arrive at. The likelihood of being caught infringing on a patentholder's rights is harder to estimate, but I expect not beyond the realms of reason.

Mitigation

There are a few different approaches to reducing (mitigating) the risk involved, which either address the likelihood or expected cost of impact. Let's look at them. You don't have to choose any one approach: a successful strategy may combine tactics from each of these categories and even use more than one tactic from the same category.

Withdrawal

Remove any likelihood and impact of a risky event occurring by refusing to participate in the risky activity. In the confidentiality case, this means not storing the secrets in the first place. In the patent case, it means not using the potentially infringing invention.

In either case withdrawing from the activity certainly mitigates any risk very reliably, but it also means no possibility of gaining the reward associated with participation.

This is why, going back to an earlier point, I don't comment on particular patent cases. Given that patent rights asserters are, in my opinion, more litigious than I, there's a chance that if I talk about a particular case I'll be considered defamatory. I'd rather avoid that risk, and choose to control it by withdrawing from talking about the cases.

Transference

You can opt to transfer the risk to another party, usually for a fee: this basically means taking out insurance. In either of our case studies, look for insurance that protects against the damage incurred. This doesn't affect the probability that our risky event will come to pass, but means that someone else is liable for the damages.

Employing Countermeasures

Finding some technical or process approach to reduce the risk. In the patent case this is simple: the countermeasure to "sued by patent holder" is "license patent".

In the confidentiality case, this means technical countermeasures: access control, cryptography and the like.

But think about deploying these countermeasures: you've now made your business or your application a bit more complex. Have you introduced new risks? Have you increased the potential damage from some risks by reducing others? And, of course, is your countermeasure cost-effective? The traditional security mantra is "don't spend $1000 to save $100": don't license $1000 of patents to protect a $100 product, and don't implement $1000 of crypto to hide $100 of data.

Acceptance

The "suck it up" approach to security: accept that the risk exists and that you may be liable for the damage if it ever comes to pass. In our information security case, this means storing the data and accepting that someone else might be able to read it. In our patent case, this means adopting the potentially-infringing invention and accepting that a licensor might come a-knocking.

All risk mitigation strategies have a certain amount of acceptance involved, apart from withdrawal. Imagine that you pay some insurance premium, and that indemnifies you up to $10M with an excess of $1000. You have to choose whether you accept the residual risk exposed in covering the excess and any overage.

Similarly, in the information security case, let's say you have data assets which, if leaked, would cost $1M in damages. You implement a particular cryptography technique that reduces the likelihood of leaking the data from an estimated once per year to an estimated once per thousand years. Again, do you accept the remaining $1000?

Conclusion

Information security and business management are actually pretty closely related. It's just that information security requires specialised technical knowledge: and that's where I come in ;-).

#### "Reasoning about code" is a scam
Another day, another post telling me to do something, or not do something, or adopt some technology, or not adopt some technology, or whatever it is that they want me to do, because it makes it easier to "reason about the code".

It's a scam.

More precisely, it's a thought-terminating cliche. Ironic, as the phrase "reason about" is used as a highfalutin synonym for "think about". The idea is that there's nowhere to go from here. I want to do things one way, some random on medium dot com wants me to do things another way, their way makes it easier to reason about the code, therefore that's the better approach.

It's a scam.

Let's start with the fact that people don't think—sorry, reason—about things the same way. If we did, then there'd be little point to things like film review sites, code style guides, or democracy. We don't know precisely what influences different people to think in different ways about different things, but we have some ideas. Some of the ideas just raise other questions: like if you say "it's a cultural difference" then we have to ask "well, why is it normal for all of the people in that culture to think this way, and all of the people in this culture to think that way?"

This difference between modes of thought arises in computing. We know, for example, that you can basically use any programming language for basically any purpose, because back in the days when there were intellectual giants in computering they demonstrated that all of these languages are interchangeable. They did so before we'd designed the languages. So choice of programming language is arbitrary, unless motivated by external circumstances like which vendor your CTO plays squash with or whether you are by nature populist or contrarian.

Such differences arise elsewhere than in choice of language. Comprehension of paradigms, for example: the Smalltalk folks noticed it was easier to teach object-oriented programming to children than to professional programmers, because the professional programmers already had mental toolkits for comprehending programming that didn't integrate with the object model. It's easier for them to "reason about" imperative code than objects.

OK, so when someone says that something makes it easier to "reason about" the code, what they mean is that that person find it easier to think about code in the presence of this property. I mean, assuming they do, and are not disingenuously proposing a suggestion that you do something when they've run out of reasons you should do it but still think it'd be a good idea.  But wait.

It's a scam.

Code is a particular representation of, at best, yesterday's understanding of the problem you're trying to solve. "Reasoning about code" is by necessity accidental complexity: it's reflecting on and trying to understand a historical solution of the problem as you once thought it was. That's effort that could better be focussed on checking whether your understanding of the problem is indeed correct, or needs updating. Or on capturing a solution to an up-to-the-minute model of the problem in executable form.

This points to a need for code to be deletable way faster than it needs to be thought about.

Reasoning about code is a scam.

#### "When I had that problem"
A common lie in programming is that every project is new, that no problem has been seen before. This is the reason given for estimates being bad, for plans being bad, for design being bad...for anything other than diving in uninformed being bad.

But I've noticed that more and more frequently my discussions about problems-technical problems, organisational problems, personal problems-involve the phrase "when I had that problem". That somebody (and, as time goes on, that's more frequently me) has seen this problem or one with many similarities before.

It's time to stop pretending that your UI fronting a database table is up there among the Hilbert problems as one of the big research questions of the 21st century. We have seen that before, or something like it, and we tried things, some of them worked. They probably weren't the best possible solutions but they were solutions.

#### "You could simply do X" costs more
Someone always says it. "Could you just add this?" or "I don't think it would be too hard to…" or if somebody else "changes these two simple things", someone might create a completely bug-compatible, scale-compatible implementation of this other, undocumented service…wait, what?

Many of us are naturally optimistic people. We believe that the problems that befall others, or that we've experienced before, will not happen this time. That despite the last project suffering from "the code getting messier and messier", we'll do it right this time.

Optimism's great. It tricks us into trying to solve difficult problems. It convinces us that the solution is "just around the corner", so we should persevere. The problems start to arise when we realise that everyone else is optimistic, too—and that optimism is contagious. If you're asked to give a drive-by estimate on how hard something is, or how long it takes, you'll give an answer that probably doesn't take into account all the problems that might arise. But now two of you believe in this optimistic estimate: after all, you're a smart person, you're trusted to give good estimates.

We need to be careful when talking to people who aren't developers to make it clear that there's no such thing as "simply" in most software systems. That "simply" adding a field brings with it all sorts of baggage: placing the field in an aesthetically pleasing fashion across multiple localised user interfaces, localising the field, building the user experience of interacting with the field and so on. That using the value from the field could turn it from a complicated problem into a complex problem, particularly if the field is just selecting between multiple implementations of what may even be multiple interfaces. That just adding this field brings not only work, but additional risk. That these are just the problems we could think of up front; there are often more that get uncovered as we begin to shave the yak.

But clearly we also need to bear in mind the problems we've faced and continue to face when talking to each other. We should remember that the last thing we tried to simply do ended up chasing a rabbit down a hole. If I don't think that I can "simply" do something without unexpected complexity and risk, I should not expect that others can "simply" do it either.

#### *-Oriented Programming
Much is written about various paradigms or orientations of programming: Object- (nee Message-) Oriented, Functional, Structured, Dataflow, Logic, and probably others. These are often presented as camps or tribes with which to identify. A Smalltalk programmer will tell you that they are an Object-Oriented programmer, and furthermore those Johnny-come-latelies with their Java are certainly not members of the same group. A Haskell programmer will tell you that they are a functional programmer, and that that is the only way to make working software (though look closely; their Haskell is running on top of a large body of successful, imperative C code).

Notice the identification of paradigms with individual programming languages. Can you really not be object-oriented if you use F#, or is structured programming off-limits to an Objective-C coder? Why is the way that I think so tightly coupled to the tool that I choose to express my thought?

Of course, tools are important, and they do have a bearing on the way we think. But that's at a fairly low, mechanical level, and programming is supposed to be about abstraction and high-level problem solving. You're familiar with artists working with particular tools: there are watercolour painters and there are oil painters (and there are others too). Now imagine if the watercolour painting community (there is, of course, no such thing) decreed that it's impossible to represent a landscape using oil paints and the oil painting community declared that watercolours are "the wrong tools for the job" of painting portraits.

This makes no sense. Oil paints and watercolour paints define how the paint interacts with the canvas, the brush, and the paint that's already been applied. They don't affect how the painter sees their subject, or thinks about the shapes involved and the interaction of light, shadow, reflection, and colour. They affect the presentation of those thoughts, but that's at a mechanical low level.

Programming languages define how the code interacts with the hardware, the libraries, and the code that's already been applied. They don't affect how the programmer sees their problem, or thinks about the factors involved. They affect the presentation of those thoughts, but that's at a mechanical low level.

Function-oriented Objects

  Given a Cartesian representation of the point (x,y), find its distance from the origin and angle from the x axis.

I'm going to approach this problem using the principles of functional programming. There's clearly a function that can take us from the coordinates to the displacement, and one that can take us from the coordinates to the angle. Ignoring the implementation for the moment, they look like this:

Point_radius :: float, float -> float
Point_angle  :: float, float -> float

This solution has its problems. I have two interchangeable arguments (both the x and y ordinates are floats) used in independent signatures, how do I make it clear that these are the same thing? How do I ensure that they're used in the same way?

One tool in the arsenal of the functional programmer is pattern matching. I could create a single entry point with an enumeration that selects the needed operation. Now it's clear that the operations are related, and there's a single way to interpret the arguments, guaranteeing consistency.

Point :: float, float, Selector -> float

Good for now, but how extensible is this? What if I need to add an operation that returns a different type (for example a description that returns a string), or one that needs other arguments (for example the projection on to a different vector)? To provide that generality, I'll use a different tool from the functional toolbox: the higher-order function. Rewrite Point so that instead of returning a float, it returns a function that captures the coordinates and takes any required additional arguments to return a value of the correct type. To avoid cluttering this example with irrelevant details, I'll give that function a shorthand named type: Method.

Point :: float, float, Selector -> Method

You may want to perform multiple operations on values that represent the same point. Using a final functional programming weapon, partial application, we can capture the coordinates and let you request different operations on the same encapsulated data.

Point :: float, float -> Selector -> Method

Now it's clear to see that the Point function is a constructor of some type that encapsulates the coordinates representing a given two-dimensional Cartesian point. That type is a function that, upon being given a Selector representing some operation, returns a Method capable of implementing that operation. The function implements message sending, and Points are just objects!

Imagine that we wanted to represent points in a different way, maybe with polar coordinates. We could provide a different function, Point', which captures those:

Point' :: float, float -> Selector -> Method

This function has the same signature as our original function, it too encapsulates the constructor's arguments (call them instance variables) and returns methods in response to selectors. In other words, Point and Point' are polymorphic: if they have methods for the distance and angle selectors, they can be used interchangeably.

Object-oriented Functions

  Write a compiler that takes source code in some language and creates an executable. If it encounters malformed source code, it should report an error and not produce an executable.

Thinking about this with my object-oriented head, I might have a Compiler object with some method #compile(source:String) that returns an optional Executable. If it doesn't work, then use the #getErrors():List<Error> method to find out what went wrong.

That approach will work (as with most software problems there are infinite ways to skin the same cat), but it's got some weird design features. What will the getErrors() method do if it's called before the compile() method? If compile() is called multiple times, do earlier errors get kept or discarded? There's some odd and unclear temporal coupling here.

To clean that up, use the object-oriented design principle "Tell, don't ask". Rather than requesting a list of errors from the compiler, have it tell an error-reporting object about the problems as they occur. How will it know what error reporter to use? That can be passed in, in accordance with another OO principle: dependency inversion.

Compiler#compile(source:String, reporter:ErrorConsumer): Optional<Executable>
ErrorConsumer#reportError(error:Error): void

Now it's clear that the reporter will receive errors related to the invocation of #compile() that it was passed to, and there's no need for a separate accessor for the errors. This clears up confusion as to what the stored state represents, as there isn't anyway.

Another object-oriented tool is the Single Responsibility Principle, which invites us to design objects that have exactly one reason to change. A compiler does not have exactly one reason to change: you might need to target different hardware, change the language syntax, adopt a different executable format. Separating these concerns will yield more cohesive objects.

Tokeniser#tokenise(source:String, reporter:ErrorConsumer): Optional<TokenisedSource>
Compiler#compile(source:TokenisedSource, reporter:ErrorConsumer): Optional<AssemblyProgram>
Assembler#assemble(program:AssemblyProgram, reporter:ErrorConsumer): Optional<BinaryObject>
Linker#link(objects:Array<BinaryObject>, reporter:ErrorConsumer): Optional<Executable>
ErrorConsumer#reportError(error:Error): void

Every class in this system is named Verber, and has a single method, #verb. None of them has any (evident) internal state, they each map their arguments onto return values (with the exception of ErrorConsumer, which is an I/O sink). They're just stateless functions. Another function is needed to plug them together:

Binder<T,U,V>#bind(T->Optional<U>, U->Optional<V>): (T->Optional<V>)

And now we've got a compiler constructed out of functions constructed out of objects.

Brain-oriented programming

Those examples were very abstract, not making any use of specific programming languages. Because software design is not coupled to programming languages, and paradigmatic approaches to programming are constrained ways to think about software design. They're abstract enough to be separable from the nuts and bolts of the implementation language you choose (whether you've already chosen it or not).

Those functions in the Point example could be built using the blocks available in Smalltalk, Ruby or other supposedly object-oriented languages, in which case you'd have objects built out of functions that are themselves built out of objects (which are, of course, built out of functions...). The objects and classes in the Compiler example can easily be closures in a supposedly functional programming language. In fact, closures and blocks are not really too dissimilar.

What conclusions can be derived from all of this? Clearly different programming paradigms are far from exclusive, so the first lesson is that you don't have to let your choice of programming language dictate your choice of problem solving approach (nor do you really have to do it the other way around). Additionally where the approaches try to solve the same problem, the specific techniques they comprise are complementary or even identical. Both functional and object-oriented programming are about organisation, decomposition and comprehensibility, and using either or even both can help to further those aims.

Ultimately your choice of tools isn't going to affect your ability to think by much. Whether they help you express your thoughts is a different matter, but while expression is an important part of our work it's only a small part.

Further Reading

The ideas here were primarily motivated by Uday Reddy's Objects as Closures: Abstract Semantics of Object-Oriented Languages (weird embedded PDF reader link). In the real-life version of this presentation I also talked a bit about Theorems for Free (actual PDF link) by Philip Wadler, which isn't so related but is nonetheless very interesting :).

#### +1-415-312-0555
Back in 1992, Robert X. Cringely wrote in Accidental Empires: How the boys of Silicon Valley make their billions, battle foreign competition, and still can't get a date [Oxford comma sic]:

Fifteen years from now, we [Americans] won't be able to function without some sort of machine with a microprocessor and memory inside.  Though we probably won't call it a personal computer, that's what it will be.

Of course, by and large that's true; the American economy depends on microcomputers and the networks connecting them in a very intimate way.  It's not obvious in 2007 just how predictable that was in 1992, as the "networks connecting them" had nothing like the ubiquity which is now the case.  When "Accidental Empires" was written, the impact of a personal computer in an office was to remove the typewriter and the person trained to type, replacing both with someone who had other work to be doing typing on a system thousands of times more complicated than a typewriter.

What's most interesting though is the (carefully guarded; well done Bob) statement that "we probably won't call it a personal computer," as that part is only partially true.  All of the people who have Tivos, or TVs, also have a personal computer.  All of the people who have mobile phones and digital cameras also have a personal computer.  The people who have Playstation 3s and Nintendo Wiis also have personal computers.  In business, the people who annoy everyone else by playing with their palmtops in meetings instead of listening to what the amazingly insightful Cocoa programmer has to say are also wasting time trying to work out how to sync them with, yup, the personal computer they also have on their desk.

So the question to be asked is not why Cringely got it wrong, because he didn't, but why hasn't the PC already disappeared, to be completely replaced with the "it is a PC but we won't call it that" technology?  Both already exist, both are pervasive, and the main modern use of both is remote publishing and retrieval of information, so why do we still tie ourselves to a particular desk where a heavy lump of metal and plastic, which can't do very much else, sits disseminating information like some kind of [note to self: avoid using the terms Oracle or Delphi here] groupthink prophet?

#### ...and in the end there will be the command line.
You're pretty happy with the car that the dealer is showing you. It looks comfortable, stylish, and has all of the features you want. There's a lot of space in the trunk for your luggage. The independent reviews that you've seen agree with the marketing literature: once this vehicle gets out onto the open road, it's nippy and agile and a joy to drive.

You can't help but think that she isn't being completely open with you though. To get into the roomy interior and luxurious driver's seat, you have to climb over a huge black box, twice the height of the cabin itself and by far the longest part of the car. Not to detract from the experience, the manufacturers have put in an automatic platform that lifts you from the ground to the door and returns you gently to earth. But the box is still there.

You ask the dealer about this box, and initially she deflects your questions by talking about the excellent mileage, which is demonstrated by the SpecRoad 2000 report. Then she tells you how great the view of the road is from the high situation of the driver's seat. Eventually, you ask enough times, and she relents.

"That's just the starter," she explains, fiddling with a catch on the door in the rear of the box. "It's just used to get the petrol motor going, but you don't need to worry about it. Well, not much."

Finally, she frees the catch and opens the box. To your astonishment, inside the box are four horses, sullenly eating grain from their nosebags and pawing their hooves on the ground. You can see that they are reined into a system that pulls the rear axle of the car as if it were an old-style carriage. The dealer continues.

"As I said, these cars just use the horses to initially pull the car along until the engine starts up and takes over. It's how we've always built our cars, by layering the modern components over the traditional carriage system. Because the horse-and-carriage arrangement is so stable having been perfected over decades, we can use it as a solid base for our high-tech automobiles. You really won't notice that it's there. We send out new grain and clear up any, um, exhaust automatically, so it's completely invisible. OK every so often one of the horses gets sick or needs re-shoeing and then you can't use your car at all, but that's pretty rare. Mostly."

Again, your curiosity is getting the better of you. In the front of the horses' cabin, leather reins run from the two leading horses to another boxed-off area. The dealer sees you looking at it, and tries to lead you back out to the showroom, but you persist. With a resigned sigh, she opens yet another hatch into this deeper chamber.

Inside, you are astonished to see a man holding the reins, ready to pull the horses along. "Something has to get the horses started," the dealer explains, "and this is how we've always done it. Our walking technology is even more robust than our horse-drawn system. Don't worry about any of that though, let me show you the independent temperature zones in the car's climate control system."

That's how it works

In the dim and distant past, barely 672 days after time itself began, the Unix time-sharing system was introduced to the world. It's a thing for big computers that lets multiple people use them at the same time, without getting in each others' way. It might not have been the most capable system (which would've been Multics, the system which Unix was based on), but due to the fact that AT&T weren't allowed to sell it, Unix did become popular. By the time this happened, Unix had been rewritten in C so the combination of C, Unix, and tools written atop like roff were what became popular.

Eventually, as small computers became more powerful, they became capable of running C and Unix too. And so they did. People designed processors that were optimised for Unix, other people designed computers that used these processors, and other people brought Unix to these computers. Each workstation itself may have only had a single user, but they were designed to be used together on a network. As the designers had decided that the network is the computer, and the network did have multiple users, it was still a multi-user system, and so the quotas and protections of a time-sharing system still made sense.

Onward and downwards, Unix marched inexorably. As it did so, it dragged its own history with it. As the extremities of Europe became the backdrop to large stone columns with Latin-inscribed capitals, so ever-smaller computers found themselves the backdrop to the Unix kernel and shell. To get there, the biological and technological distinctiveness of each new environment had to be added to Unix's own.

Compare the Unix workstation to the personal computer. A Unix workstation was designed to run Unix, so its ROM program could look for file systems, find one with the /vmunix program on it, and run that program. The PC was designed...well, it's not clear what it was designed for, though it was likely to do the same things that CP/M could do on other small computers. If you don't have an operating system, many of them will give you the infamous NO ROM BASIC message.

Regardless, the bootstrap program in a PC's ROM certainly isn't looking for a Unix, or an NT OS kernel, or anything else in particular. It just wants to run whatever comes next. So it looks for a program called the secondary bootloader, and runs that. Then the secondary bootloader itself looks around for the filesystem with /vmlinuz or whatever the Unix (or Unix-like) boot file is called, and runs that.

Magnify and Enhance

The story doesn't end at the kernel. Getting there, the kernel discovers the hardware available (even though this has been done once or twice already) and then gets on with one of its functions, which is to be a bootloader for a Unix program. Whether that program is initor some newer replacement, that has to start before the computer is properly running a Unix.

One of init's tasks is to start up the Unix programs that you want running on the computer, the launch procedure is still not complete. init might follow the instructions in a script called rc, or it could use all the scripts in a folder called init.d or SystemStarter, or it could launch svc.startd and let that decide what to start, or maybe something different happens. Once that procedure has run to completion, the computer is probably doing whatever it was that you bought it for, or at least waiting for you to tell it what that is.

Megakernels

So many different computers go through that complex process - servers, desktops, laptops, mobile phones, tablets, network routers, watches, television receivers, 3D printers. If you have an idea for a novel application of computing hardware, the first step is to stand back and protect your ears from the whomp of four decades of history being dumped in a huge black box on the computer, then you can get cracking.

You want to make a phone? A small device to be used for real-time communication by a single person? whomp comes the megalith.

You want to make a web server? A computer usually dedicated to running three functions (converting input into database requests, converting database responses into output, and tracking which input deserves which output)? whomp comes the megalith.

You want a network appliance? Something that nobody's going to use at all, that sits in the corner turning 802.11 datagrams into 802.3 datagrams? whomp comes the megalith.

There's not much point looking at Unix as an architecture or a system of interdependent components in these applications. whomp. It's a big black box that can be used to get other boxes moving, like the horses used to start a car's engine. In the 1980s and into the beginning of the 1990s, there were arguments about whether monolithic kernels were better than microkernels. Now, these arguments are redundant: the whole of Unix is itself a megakernel for OS X, Android, iOS, Firefox OS, your routers, network switches and databases.

But the big black box is black because of what's found at the top of the megalith. It's a tar pit, sucking in the lower layers of whatever's perched above. Yesterday, a Unix system would've been programmed via the Bourne Shell, a sort of dynamic compromise for the lack of message-passing in C. Today, once the dust has cleared from the whomp, you can see that the Bourne shell is accompanied in the softer layers of tar by Tcl, Perl, Python, Ruby, and other once high-flying programs that got too close to the pit.

Why that's good

The good news is that Unix isn't particularly broken. Typically a computer based on Unix can remain working for at least long enough that either the batteries run out or a software update means you have to turn it off anyway.

Because Unix is everywhere, everybody knows Unix. Or they know something that was once built on Unix and has been subsumed into Unix, the remains of which can just be seen and touched in the higher strata of the tar. Maybe they only really know how to generate JSON structures in Ruby, but that's OK because your next-generation doorbell will have a Ruby interpreter deposited with the whomp of the megalith.

And if something isn't particularly broken, then there's not much point in throwing it away for something new. Novelty for its own sake was the death of Taligent, the death of Be, and the death of countless startups and projects who want to do something like X, but newer.

Why that's bad

The bad news is that Unix is horrendously broken. You can have a supposedly safe runtime environment for your program, but the bottom of this environment is sticking into the tar pit that is C and Unix. Your program can still get into trouble because it's running on Java and Java is written in C and C is where the trouble comes from.

The idea is that you stay at the top of the megalith, and it just starts your computer and stops you from worrying about the low-altitude parts of the machine. That's only roughly true though, and lower-down pieces of the megalith sometimes prove themselves to have crumbled under weathering and the pressure from the weight above. If your computer has experienced a kernel panic in the last year, it's probably because the graphics driver wasn't very well-written. That's a prop that has to be inserted into the bottom of the megalith to keep it upright, but people make those props out of balsa wood and don't check the size of the holes they need to fit the props into.

Treating Unix as the kernel of your modern system means ignoring the fact that Unix is itself a whole operating system, and that your UEFI boot process also loaded another other operating system just to get that other operating system to load your operating system. The outer system displays inner-system problems, being constrained by the same constraints that your Unix flavour imposes. Because Unix is hidden, these become arbitrary-seeming constraints that your developers simply know as always having been there.

What should be done

A couple of decades ago, there were people who knew that PC operating systems like Mac OS and MS-DOS weren't particularly good, and needed replacing. Some of them looked with envy at the smooth megalith that was Unix, and whomp here it arrived on their desktop machines: MkLinux, Debian, NeXTStep, Solaris, 386BSD and others. Others thought that the best approach was to start again with systems designed to support the desktop paradigm and using modern design techniques and technology advances: they made BeOS, Windows NT and others.

Systems like this (including modern BeOS-inspired Haiku, and Amiga-inspired AROS) are typically described by their project politburos as "efficient", "lean" and other words generally considered to be antonymical to "a GNU distribution".

They also tend to have few users in comparison to Mac OS X, GNU and other systems. Partly this is just a marketing concern, that's irrelevant when such systems are free: if the one that works for you works for you, it shouldn't matter how many other people it also works for. In practice there is a serious consideration to the install base. The more people who use an operating system, the more people there are who want applications for that system and therefore (hopefully) more people will want to write applications for that system.

If Linus's Law (that many eyes make bugs shallow; a statement of wishful thinking that should actually be attributed to Eric S. Raymond) actually held true, then one might expect that more popular systems would suffer fewer bugs. Perhaps more popular systems end up with higher expectations, and therefore gain newer features faster, thus gaining bugs faster than people could fix them?

Presumably as the only point to Unix these days is to be a stable stratum on which to layer other things, there are numerous companies and individuals who would benefit from it being stable. We can accept that all of this complexity is going nowhere except upward, and that the megalith will continue to grow inexorably as more components fall into the tar pit. With that being the case, all of the companies and individuals involved could standardise on a single implementation of the megalith. They could all shore up the same foundations and fix the same cracks.

What I think I want to do

I often choose to rank potential solutions to technical problems in a two-dimensional graph, because if you can reduce any difficult question down to four quadrants then you can make a killing as a consultant. In this case, the axes are political acceptability and technical quality.

+-------------------+-------------------+ T
|                   |                   | e
|  Awkward  genius  |     Slam-dunk     | c
|                   |                   | h
+-------------------+-------------------+ n
|                   |                   | i
|   Feverish rant   | Saleable band-aid | c
|                   |                   | a
+-------------------+-------------------+ l
                Political

A completely new system might be a great idea technically, but is unlikely to get any traction. There may be all sorts of annoying problems that make current systems a bit disappointing, but no-one's suffering badly enough to consider a kill or cure option. The conditions for a radically novel system becoming snapped up by an incumbent to replace their existing technical debt don't really exist, and haven't for decades (Commodore bought Amiga to get their new system, but in the 1990s Apple just needed a system that was already a warmed-over workstation Unix).

In fact despite the view of the software sector being a high-tech industry, it's both socially and technologically very conservative. It's rare for completely new ideas to take hold, and what's taken for progress can often be seen more realistically as a partially-directed form of Brownian motion. As already discussed, this isn't completely bad, because it stops new risks being introduced. The counterpoint to that melody is that it stops old risks from being removed, too.

Getting a lot of developer traction around a single Unix system therefore has a higher likelihood, in fact it's already happened. It's not necessarily the best approach technically, because it means rather than replacing that huge megalith we just agreed was a (very large) millstone, we resign ourselves to patching up and stabilising the same megalith together. Given that one penguin-based megalith is already used in far more contexts than any other, this seems more likely to be acceptable to more people beset by the crumbly megalith problem.

There's room in the world for both solutions, too. What I call a more acceptable solution is really just easier to accept now, and the conditions can change over time. Ignoring the crumbling megalith could eventually produce a crisis, and slicing the Gordian knot could then be an acceptable solution. Until that crisis hits, there will be the kernel, the command-line, and the continuing echos of that original, deafening whomp.

#### 20 things learned in 20 years as a software engineer
I only have 17 years of experience, but every point on this list accords with my experience. I've made my own attempt to catalogue  things software developers should know (that are not writing code), but this is a succinct and great summary.

#### 2017: the year of configuring Linux (or Windows, or OS X, or...) on the desktop
I'm going to FOSDEM next month, maybe I'll see some of you there. This gives me motivation to solve one of the outstanding problems on my laptop: I currently, as has been mentioned here multiple times, use Windows 10 as a bootloader for my GNU/Linux installation. I would rather boot straight into Linux. So I can set myself a milestone: I would prefer, by the time I get to St. Pancras International train station on Friday 3rd Feb, not to have Windows on this laptop any more.

The laptop is a Alienware 15 R3, although weirdly the processor my laptop contains (Core i7-6820HK) is not one of the CPU options listed on the Dell website, so maybe they changed the configuration without updating the model, or had a spare old CPU knocking around when they built my laptop and decided to use that. Anyway, this is computering, so this is fine. You're not expected to know or care that you don't have the correct bits in the computer, just that it's "Late 2016" (even though they still sold the R2 in late 2016, too).

The problems I have seem to fit into one of two categories: either the wi-fi (a Qualcomm Atheros chipset) doesn't work, or the CPU/motherboard chipset (see above) isn't supported and all hell breaks loose.

The wireless situation is that the wireless should be fully supported: Qualcomm Atheros integrated the driver into the Linux kernel back in version 3.11-rc1, in July 2013, and supply the firmware binaries. And so it doesn't work in modern Linux kernels (or doesn't reliably work, or fails for different reasons).

The chipset situation is that Intel integrated the driver into the Linux kernel back in version 4.3, in November 2015. So it doesn't reliably work in modern Linux kernels.

I could go into the specific problems I've seen and the specific things I've tried to work around them, but I won't. I won't because well-meaning but unengaged people will ask me infuriatingly basic and irrelevant questions (yes, I have already turned Secure Boot off; no, it doesn't change the fact that the ath10k module hasn't loaded), or suggest unjustifiable solutions. The most common is the Distro Pimp: you should try [Debian/Debian testing/Arch/Ubuntu/openSuSE/Fedora/Mint/wait, which one did you say you'd already tried?]. Well that's nice, but the distribution I tried (that didn't work) is made of GNU and Linux 4.8, and the distribution you're suggesting is made of GNU and Linux 4.8, so what specifically is it about your distribution that makes you think it works where this other one doesn't? Oh, they focus on [stability/cutting edge/purple desktops/compiz effects/bible-reading software] do they? And how does that solve my problem where the kernel doesn't work, despite being newer than all of the bits I need to have a working kernel?

This is the reality of Linux on the Desktop, the one that computerists say the world is ready for. Of course, it's also the reality of everything else on the desktop. Something that occasionally happens in my Windows 10 bootloader is that it reboots while I'm using it to install some updates, because I stopped moving the mouse for a couple of minutes between 6pm and 9am (you know, the time when I'm at home, using my home computer). Some colleagues at work use Windows as an actual operating environment, and have things like Skype (made by Microsoft) popping up a notification when they're presenting in PowerPoint (made by Microsoft). Something that apparently happens to people that have Macs is that the built-in PDF software doesn't work well and they have to buy somebody else's PDF software, except that they have to check whether that other PDF software is based on the built-in stuff or is something written by somebody else only they can't because without the Four Freedoms they don't have the freedom to study how the program works, and even if they could fix the problem they're not allowed because they lack the freedom to redistribute and make copies to help their neighbours, or to improve the program so the whole community benefits.

This is fine.

#### 6502
On the topic of the Apple II, remember that MOS was owned by Commodore Business Machines, a competitor of Apple’s, throughout the lifetime of the computer. Something to bear in mind while waiting to see where ARM Holdings lands.

#### <em>YOUR</em> development team needs security engineers
It can definitely be tempting if your engineers don't have a whole lot of security expertise to get a consultant in. Indeed this can be a great way to bootstrap a security process, however it then needs to be owned and executed inside the team. The reason has mainly to do with the goals of your developers and of the consultant.

Fuzzy logic

The consultant's high fee is based on being able to come in to a new team and quickly make a high impact. This means, of course, rapidly reporting low-hanging problems. It also means that all of the reported problems are important - it's a good job you hired methis anonymous consultant, having found all of those critical issues, no?

The fact is that most of the security problems found by external security consultants and penetration testers are those that can be found in short order by a script kiddie - in other words, ones it would take very little effort to write automated tests for. In fact that's pretty much what consultants, pen testers and script kiddies do, and it's called fuzzing. Now, if you'd given the task of writing the fuzzing tools to one of your developers, you'd be slightly more behind schedule (than you already are because you've got to fix those data-handling defects), and you'd have a developer who knows how to write fuzzing tools.

Finding architectural and requirements-level security problems is not impossible for a consultant, however we only really have a second hand knowledge of the requirements and architecture (and it's likely that we got that from talking to your developers, unless you have some really shit-hot and surprisingly relevant documentation. Hint: you don't). What we bring to the party is a thorough knowledge of threat analysis and risk management, which we then get to apply to an unknown project. Your developers have an inside-out knowledge of the project, but probably aren't as experienced as me at carrying out security assessments.

Don't get me wrong, the consultant will definitely find real problems and will probably leave you in a position to deliver a more secure product. However, in order to get most value out of the consultant's input, you need to make sure that your project's security model and security knowledge remain up to date, and that means bringing it at least partially in house.

Security champions
Make sure that every development team has its own security champion. That person's responsibility is:

Define the team's security process:
How is the threat model discovered/updated?
How are security problems found and reported?
How are they addressed?
Ensure the process is followed
Define security criteria for each iteration, user story, or whatever makes sense in your project management setup

The champion is authorised to:

Delegate responsibility for carrying out tasks in the security process
Declare any build/user story/iteration/whatever to be a failure until its security criteria are satisfied

As we know, responsibility without authority just means you have someone to fire. A team can only get things done if they're actually allowed to ensure they get things done. Sounds like common sense, but many software engineering managers fail to grasp it.

In our team, everybody is the security guy. We share the responsibility.

Bullshit. You get nothing done. If everyone is responsible for a problem, then everyone knows that somebody else can deal with it. I've seen software released with known and rather heinous vulnerabilities, all because no-one wanted to be the person who delayed release just before the ship date. Don't allow your project to get into that state - give someone ownership of the security efforts.

#### A better bit o' twitter than the bitter twitter Tommy Titter bought
Just because everyone these days writes a Twitter client:This was actually a quick hack project to make up for the fact that I missed CocoaHeads tonight (due to a combination of an uninteresting phone call, and a decision to recover from the phone call by using the rest of my petrol tank).  Really just an excuse to play with some APIs (the tweets are grabbed by the controller using NSURLConnection, then some NSXML/XPath extracts the useful information (or not, it is Twitter after all) and puts it into the model), there are many things which need to happen before this is at all a useful Twitter client; the ability to write back, nicer formatting are just the starters.  Shiny Core Animation twitting ought to happen.Still, not bad for two hours I think.

#### A bit late for that, isn't it?
Just won the above-linkied item on eBay: the Amiga SDK :-)  I'm probably slightly late to make any money out of Amiga development, but I'm glad I finally get to tinker.  Might have to crack open the AROS if I get sufficiently involved...I could probably write a three-page rant about how great the Amiga was and how badly Commodore stuffed up.  But of course, none of it would be new ;-).

#### A bit of backup script
Good news - there's a handy tool in OS X called wait4path which can help when writing timed scripts to backup to removable media.

Bad news - it [at least in Tiger....] works slightly esoterically - if a path is already present, it will still wait for another mount kevent before exiting.  It should therefore be used in a script like this:

#!/bin/shif [ ! -d /Volumes/Backups ]; then  echo "waiting for backup volume..."  /bin/wait4path /Volumes/Backupsfi# do some backups

Note, however, that if you do this in a crontab job it could potentially wait for a very long time, so you should wrap all that with a /var/run style semaphore.

#### A brief history of talking on the interwebs (or: why I'm not on app.net)
When I first went to university, I was part of an Actual September, though it took place in October. Going from a dial-up internet service shared with the telephone line to the latest iteration of SuperJANET with its multi-megabit connection to my computer opened many new possibilities for me and my peers.

One of these possibilities was Usenet, which we accessed via news.ox.ac.uk. Being new to this online society, my fellow neophytes and I made all of the social faux pas that our forbears had made this time last year, and indeed in prior years. We top-posted. We cross-posted. We fed the trolls. Some of us even used Outlook Express. Over time, those of us who were willing to make concessions to the rules became the denizens, and it was our job the next September to flame the latest crop of newbies.

The above description is vastly oversimplified, of course. By the time of my Actual September, Usenet was feeling the effects of the Neverending September. Various commercial ISPs - most notoriously America Online - had started carrying Usenet and their customers were posting. Now there was, all year round, an influx of people who didn't know about the existing society and rules but were, nonetheless, posting to Usenet.

Between AOL and - much later - Google Groups incorporating Usenet into its content, the people who felt themselves the guardians and definition of all that Usenet stood for found that they were the minority of users. Three main ways of dealing with this arose. Some people just gave up and left for other services. Others joined in with the new way of using Usenet. Still others worked in the old way despite the rise of the new way, wielding their ability to plonk newbies into their kill file as a badge of honour.

By now I probably don't need to ask the rhetorical question: what has all of this to do with twitter? Clearly it has everything to do with twitter. The details differ but the analogy is near watertight. In each instance, we find a community of early adopters for a service that finds a comfortable way to use that service. In each we find that as the community grows, latecomers use the service in different ways, unanticipated or frowned upon by the early adopters. In each case the newcomers outnumber the early adopters by orders of magnitude and successfully, whether by sheer scale or through the will of the owners of the service, redefine the culture of the service. Early adopters complain that the new majority don't "get" the culture.

Moving to app.net does nothing except reset that early-adopter clock. Any postmodernist philosopher will tell you that: probably while painting your living room lilac and dragging a goldfish bowl on a leash. If app.net takes off then the population of users will be orders of magnitude greater than the number of "backers". The people who arrive later will have their own ideas of how to use the service; and together will have contributed orders more cash to the founders than the initial tranche of "backers". I wonder who the management will listen to.

Any publicly-accessible communication platform will go through this growth and change. When I joined Facebook it was only open to university members and was a very different beast than modern Facebook. I would not be surprised to read similar complaints made about citizens' band radio or Morse telegraphy.

The people who move on don't necessarily want a changed experience. It seems to me they want a selective experience, and moving into the wilderness allows them an approximation of that. In the short term, anyway. Soon the undesirables will move in next door and they'll choose to move on again.

I suggest that what's required is actually something more like Usenet. I run my own status.net server, initially to archive my tweet stream but it turns out I'm not using it for that. If I chose I could open that server up to selected people, just as news.ox.ac.uk was only open to members of one university. I could curate a list of servers that mine peers with. If there are some interesting people at status.cocoadev.com, I could peer with that server. If status.beliebers.net isn't to my taste, I don't peer with it. But that's fine, their users don't see what I write in return for me not seeing what they write. In fact Usenet could've benefitted from more selective peering, and a lot of the paid-for access now has, easily-detectable spam aside, a higher signal to noise ratio than the service had a decade ago.

Another service that has some of the aspects of the curated experience is Glassboard. Theirs is entirely private, losing some of the discoverability of a public tweet stream. In return all conversations are (to some extent) invitation only and moderated. If you don't like someone's contributions, the board owner can kick ban them.

So the problem long-term tweeters have with twitter is not a new problem. Moving wholesale to something that does the same thing means deferring, not solving, the problem.

I thought I'd update this post (nearly six months later) on the day that I joined app.net. It's changed quite a lot—both by adding a cloud storage API and by going freemium—in the intervening time. I remain skeptical that the problem with a social network is the tool, and I also wonder how the people who joined to get away from people using Twitter really badly will react to the free tier allowing the unwashed masses like me to come and use app.net really badly. Still, there's a difference between skeptical and closed-minded, so here I am.

#### A brief interlude
Honestly, this next post will take a while.

#### A bunch of monkeys with typewriters
As with many of the posts in this blog, this one originally started as a tweet that got too long. With the launch of Path 2, a conversation about Atos ditching email for social media and Yammer posting a video of how their enterprise social network is used at O2, I've been thinking about how I'd design the social network for business users.

The TL;DR (or executive summary, if that's your thing) is "don't be a lazy arse, read the whole post".

CSCW

Computer-Supported Collaborative Working. A term that has been around for a few decades, and means pretty much what it says: finding ways to use computers to support people working together. Ideas from CSCW pervade the modern requirements engineering discipline; particularly strong is the notion that the software and the system in which it's used are related. Change the software and you'll change the way people work together. Change the people and you'll change the way they work together too, including how they use the software.

When I finished being an undergraduate and became an actual person, real-world CSCW was synonymous with "groupware" and basically meant an email service with integrated calendars. Wikis existed, and some companies were using them. Trouble ticket systems like req and rt did exist too. All of these tools were completely separate. Most of them, if they needed to tell you something, would send you an email.

Ah, email. The tool that changed "read your memos at 4:30 pm" into a permanent compulsion. How that big red badge on your mail icon taunts you, with its message that there are now THREE REALLY IMPORTANT THINGS you're ignoring. Only one's a message from facilities telling you that the wastepaper baskets have been moved, and while the other two are indeed task-related they carry no new information ("I agree we should touch base to talk around this action"). Originally the red badge only mocked me while I was in the office, but then I was issued a smartphone.

Email: the tool that makes conversations in other parts of the company completely undiscoverable. You can no longer get a feel for what's going on by hanging around the water cooler or the smokers' shelter, because those discussions are being held over email. Email that you can only read if you're in on the joke. If you're supposed to be in on the joke but the author addresses her missive to "j.smith3" instead of "j.smith1", you're shit out of luck. Email where you have to look at whether you were CCd or BCCd before weighing in with your tuppen'orth.

We've come a long way baby

OK, since those dark days we've learned a lot, right? There's Twitter, and Facebook, and Myspace (yes, it is still a thing), and LinkedIn and Path and Tumblr and Glassboard and…

Lots of these things have good ideas, and lots have not so good ideas. Most of them are designed to be very general things that people in many contexts would use: what would a (by which I mean "my") business-centric comms tool designed today look like?

The features
Completely replaces internal e-mail for everyone
This is non-negotiable. No more multiple channels of communication (I currently have open: twitter, outlook, skype and IRC. And I'm writing a blog post. And I have a phone. And I get Path push notifications.). No more trying to match up threads that have come in and out of various applications. If the CEO wants to leave a message for the CTO, he does it over the social network, just as when one of the cleaners wants to talk to another.

Pull, don't push
I will say this nice and loud: I don't care when your shitty app thinks I need to read my messages. I have work to do, let me get on with it without the guilt of the red badge. Support an OmniFocus-style view where I can see what's come in, what's outstanding, and what I think is important or urgent. But let me review that on my own terms. I don't care whether the personal assistant of the head of some other department thinks this message is double-exclamation-mark-urgent; I'll decide. Let me set up notifications on things I think important to be notified about.

Everything is a citable object
Twitter gets this correct, facebook does not. Except that Twitter doesn't really, because I can't really write long-form posts, styled posts, or put images/video/audio/etc. in Tweets. I can just pretend using URLs. But it does allow you to treat a reply as a first-class object: it's just a tweet that has a link to a previous tweet. In facebook, comments are special things that are attached to other things but don't really have an existence of their own.

Facebook also decides that different ways of communicating with someone are, for some reason, different things. Notes appear in the "notes" section, updates on the wall (including updates that are links to notes), photos in photos, music in…you get the idea. I say the important things are who said what to whom, and in what context. If I want to write a long-form post about a comment that was a reply to a photo posted in response to a customer service request, I should be able to do that. That implies that our hypothetical social network would be a platform that can support applications like request tracking, bug reporting, CRM and whatever else it is that people in a big company do. Which brings me on to the next two features.

I need to be able to discover people
People (indeed any primates) naturally organise into smallish close groups, in which they understand the dyads (what each people thinks about the others) quite well. They then know a little bit about what the other groups are, but not necessarily about the relationships inside those groups. Current social network tools don't really take advantage of that: and no, Google+ circles are not this.

Companies, in fact, do not take this into account. Hierarchical management structures lead to an easy identification of "us" and "them", so that whenever someone in a different group causes "us" to re-think what we're doing, we assume that "they" don't get what "we" are all about. We're all trying to make the same shareholders rich (or change the world, or gain a pension; whyever it is you do what you do) and we should all be on the same side.

If I'm looking for someone in Legal who knows about open source licenses, the tool should support that. Furthermore, the tool should show me how their circle overlaps with mine, so I can see who to go to (and who to avoid) for an introduction if that's the best way to proceed. The chance is that in a company with thousands of employees I don't know who I'm looking for, but someone I know does know. Furthermore, the tool should be able to use sentiment analysis to tell whether the people I'm going to talk to are likely to be on my side or not. This should help combat the "us vs. them" mentality described above, because I can go into my new meeting knowing where I'm similar to my new contact.

[An early draft of the section you've just read was titled "It isn't who you know, it's who you know who you know knows" but that wasn't really the name of a feature.]

There are no silos or Chinese walls: everyone can see (and react to) everything.

This has a few benefits. Most importantly, this is a time where valuable advances are being made not by gathering information - which we've been doing for decades - but by finding new ways of combining, organising and analysing the data that we've already got. Which makes it crazy that most companies stop people from taking a cross-functional view of the things going on in that company. It's maddening, it costs money, and it needs to stop. If I'm doing some research into, for example, fixing some bug in a product and can find out that Jim on the team downstairs has recently been fixing a related bug on his team's product, that's useful. I should be able to see that: we need a platform that supports it.

The second benefit is to remove another barrier that reinforces the "us vs. them" separation of departments. No, sales aren't hiding anything from you: it's all available.

Next, if everyone knows that anything they say in "new email" is readable by everyone else in the company, that also helps to increase the collaborative nature of interactions because you can't hide behind the almost-private nature of email.

Path has an interesting feature in this regard: for each event you can see, you can also see who has seen that event. It's a useful reminder that what you write is being read, and by whom: once you've seen your head of department pop up a few times, you probably remember to make everything work-safe. It's almost an implementation of the "your mum can read what you write" reminder I wanted to add to facebook.

Now I can hear my security colleagues sharpening their ePencils to write the "waah, insider threat" comment. My question is this: so? Insider threats are an HR problem, so rather than making everybody suffer by carving up the company, consider giving the HR department the tools they need to detect and react to the problem. Such as, I don't know, a tool that provides a view on what company data people are interacting with along with sentiment analysis…

There are some things that genuinely do need to have restricted access: customer data, employee personal information and the like. These should be exceptions, not the norm, and treated exceptionally. Similarly, contractors and other suppliers who probably should be given access to the platform can have need-to-know access. Again, that doesn't mean that all your permies do. In fact, show your permies that you trust them with the ability to see what's going on across the whole company, you'll probably end up with happier and more motivated permies.

All of this stuff about tearing down the walls doesn't stop you allowing the employees to organise into groups on the platform. But such groups should be self-organising, transient, and transparent: just because I didn't join, that doesn't mean I shouldn't be able to join in.

Smart searches

Final thing: if you build everything above, you're plugging everyone in to the company firehose. Make it easy for people to find what they're after, and to review when new matching results become available.

#### A Cupertino Yankee in the Court of King Ballmer
This post summarises my opinions of Windows Phone 7 from the Microsoft Tech Day I went to yesterday. There's a new version of Windows Phone 7 (codenamed Mango) due out in the Autumn, but at the Tech Day the descriptions of the new features were mainly the sorts of things you see in the Microsoft PressPass video below (Silverlight required), the API stuff is going on in a separate event.



I want to provide some context: I first encountered C#, J# and .NET back in around 2002, when I was given a beta of Visual Studio .NET (Rainier) and Windows .NET Server (which later became Windows Server 2003). Since then, of course most of my programming work has been on Objective-C and Java, on a variety of UNIX platforms but mainly Mac OS X and iOS. But I've kept the smallest edge of a toe in the .NET world, too.

From the perspective of the phone, however, I really am coming to this as an iOS guy. Almost all of the mobile work I've done has been on iOS, with a small amount of Android thrown into the mix. I've used a WP7 phone a couple of times, but have no experience programming on WP7 or earlier Windows Mobile platforms.

The UI

The speakers at the Tech Day - a mix of Microsoft developer relations and third-party MVPs - brought as much focus on user experience and visual impact of WP7 apps as you'll find at any Apple event. Windows Phone uses a very obviously distinctive UI pattern called Metro, which you can see in the demo screencasts, or the Cocktail Flow app.

Metro is almost diametrically opposed to the user experience on iOS. Rather than try to make apps look like physical objects with leather trim and wooden panels, WP7 apps do away with almost all chrome and put the data front and centre (and, if we're honest, sides and edges too). Many controls are implicit, encouraging the user to interact with their data and using subtle iconography to provide additional guidance. Buttons and tiles are far from photorealistic, they're mainly understated coloured squares. Users are not interacting with apps, they're interacting with content so if an app can provide relevant functionality on data from another app, that's encouraged. A good example is the augmented search results demoed in the above video, where apps can inspect a user's search terms and provide their own content to the results.

In fact, that part of the video shows one of the most striking examples of the Metro user interface: the panorama view. While technologically this is something akin to a paginated scroll view or a navigation controller, it's the visual execution that makes it interesting.

Instead of showing a scroll thumb or a page indicator, the panorama just allows the title of the next page to sneak into the page you're currently looking at, giving the impression that it's over there, and if you swipe to it you can find it. When the user goes to the next page, a nice parallax scroll moves the data across by a page but the title by only enough to leave the edges of the previous and next titles showing.

The Tools

It's neither a secret nor a surprise that Microsoft's developer tools team is much bigger than Apple's, and that their tools are more feature-rich as a result (give or take some ancient missteps like MSTest and Visual SourceSafe). But the phone is a comparatively new step: WP7 is under a year old, but of course Windows Mobile and Compact Editions are much older. So how have Microsoft coped with that?

Well, just as Apple chose to use their existing Cocoa and Objective-C as the basis of the iOS SDK, Microsoft have gone with .NET Compact Framework, Silverlight and XNA. That means that they get tools that already support the platform well, because they're the same tools that people are using to write desktop, "rich internet" and Xbox Live applications.

From the view-construction perspective, XAML reminds me a lot more of WebObjects Builder than Interface Builder. Both offer drag-and-drop view positioning and configuration that's backed by an XML file, but in Visual Studio it's easy to get precise configuration by editing the XML directly, just as WebObjects developers can edit the HTML. One of the other reasons it reminds me of WebObjects is that Data Bindings (yes, Windows Phone has bindings...) seems to be much more like WebObjects bindings than Cocoa Bindings.

Custom classes work much better in the XAML tools than in IB. IB plugins have always been a complete 'mare to set up, poorly documented, and don't even work in the Xcode 4 XIB editor. The XAML approach is similar to IB's in that it actually instantiates real objects, but it's very easy to create mock data sources or drivers for UI objects so that designers can see what the app looks like populated with data or on a simulated slow connection.

Speaking of designers, an interesting tool that has no parallel on the iPhone side is Expression Blend, a XAML-editing tool for designers. You can have the designer working on the same files as the developer, importing photoshop files to place graphics directly into the app project.

It'd be really nice to have something similar on iPhone. All too often I have spent loads of time on a project where the UI is specified as a photoshop wireframe or some other graphic provided by a web designer, and I'm supposed to customise all the views to get pixel-perfection with these wireframes. With Blend, the designer can waste his time doing that instead :).

Other tools highlights include:

Runtime-configurable debugging output on both phone and emulator, including frame rates, graphics cache miss information, and Quartz Debug-style flashes of updated screen regions
The emulator supports virtual accelerometers
The emulator supports developer-supplied fake location information and even test drivers generating location updates &lt;3

The biggest missing piece seems to be a holistic debugging app like Apple's Instruments. Instruments has proved useful for both bug fixing and performance analysis, and it's pretty much become a necessary part of my iOS and Mac development work.

Update: I'm told by @kellabyte that an Instruments-like tool is coming as part of the Mango SDK, and that this was announced at MIX '11.

The "ecosystem"

A couple of the demos shown yesterday demonstrated phone apps talking to Azure cloud services, ASP.NET hosted web apps (mainly using the RESTful OData protocol), SOAP services etc. Because there's .NET on both sides of the fence, it's very easy to share model code between the phone app and the server app.

That's something Apple lacks. While Cocoa can indeed be used on Mac OS X Server, if you want to do anything server-side you have to either hope you can find some open-source components or frameworks, or you have to switch to some other technology like Ruby or PHP. While Apple ship that stuff, it's hard to claim that they're offering an integrated way to develop apps on iOS that talk to Apple servers in the same way that MS do.

To the extent that WebObjects can still be said to exist, it doesn't really fill this gap either. Yes, it means that Apple provide a way to do dynamic web applications: but you can't use Apple's tools (use Eclipse and WOLips instead), you can't share code between iOS and WO (iOS doesn't have Java, and WO hasn't had ObjC in a long time), and you can just about share data if you want to use property lists as your interchange format.

On the other hand, it's much easier to distribute the same app on both iPhone and iPad than it would be to do so on WP7 and a Microsoft tablet/slate, because their official line is still that Windows 7 is their supported slate OS. I expect that to change once the Nokia handset thing has shaken out, but making a Silverlight tablet app is more akin to writing a Mac app than porting an iOS app.

The market

This is currently the weakest part, IMO, of the whole Windows Phone 7 deal, though it is clear that MS have put some thought and resources behind trying to address the problems. Given that Windows Phone 7 was such a late response to the iPhone and Android, Microsoft need to convince developers to write on the platform and users to adopt the platform. The problem is, users are driven to use apps, so without any developers there won't be any users: without any users, there's no profit on the platform so there are no developers.

Well, not no developers. Apparently the 17,000 apps on the marketplace were written by 7,500 of the 42,000 registered developers (and interestingly the UK has one of the lowest ratio of submitted apps to registered developers). By comparison, there are 500,000 apps on the app store.

Microsoft has clearly analysed the bejeesus out of the way their users interact with the marketplace. They have seen, for instance, that MO billing (essentially having your phone operator add your app purchase costs to your phone bill, rather than having a credit card account on the marketplace itself) increases purchase rates of apps by 5 times, and are working (particularly through Nokia of course who already have these arrangements) to add MO billing in as many marketplace countries as they can.

This makes sense. People already have a payment relationship with their network operators, so it's easier for them to add a few quid to their phone bill than it is to create a new paying account with the Windows Marketplace and give Microsoft their card details. By analogy, iPhone users already have bought stuff from Apple (an iPhone, usually...and often some music) so throwing some extra coin their way for apps is easy. Incidentally, I think this is why the Android app market isn't very successful: people see Google as that free stuff company so setting up a Checkout account to buy apps involves an activation energy.

Incidentally, some other stats from the app marketplace: 12 downloads per user per month (which seems high to me), 3.2% of all apps downloaded are paid, and the average price of a bought app is a shave under $3. Assuming around 3 million users worldwide (a very rough number based on MS and analyst announcements), that would mean a total of around $3.5M app sales worldwide per month. That's nowhere near what's going on on the iPhone, so to get any appreciable amount of cash out of it you'd better have an app that appeals to all of the platform's users.

The subject of appeal is a big issue, too. Microsoft aren't really targeting the WP7 at anyone in particular, just people who want a smartphone. With Mango and Nokia handsets, they'll be targeting people who want a cheaper smartphone than RIM/Apple/Android offers: I bet that brings down that $3 mean app price. This is, in my opinion, a mistake. Microsoft should play to their strengths, and make a generally-useful device but target it at particular groups who Microsoft can support particularly well.

Who are those groups? Well, I think there's Xbox 360 gamers, as WP7 has Xbox Live integration; and there's enterprises with custom app needs, due to the integration with Azure and similarity with ASP.NET. It ought to be cheaper for game shop that's written an Xbox game to do a WP7 tie-in than an iOS tie-in. It ought to be cheaper for an enterprise with an MS IT department to extend their line-of-business apps onto WP7 than onto Blackberry. Therefore MS should court the crap out of those developers and make the WP7 the go-to device for those people, rather than just saying "buy this instead of an iPhone or Android".

The reason I'd do it that way is that you bring the users and the developers together on shared themes, so you increase the chance that any one app is useful or relevant to any given customer and therefore increase the likelihood that they pay. Once you've got gamers and game devs together, for example, the gamers will want to do other things and so there'll be a need for developers of other classes of app. I call using Xbox games to sell utilities the Halo Effect.

Conclusion

Windows Phone 7 is a well thought out mobile platform, with some interesting user experience and design. It's got a good development environment, that's highly consistent with the rest of the Microsoft development platform. However, no matter how easy and enjoyable you make writing apps, ultimately there needs to be someone to sell them to. Microsoft don't have a whole lot of users on their platform, and they're clearly banking on Nokia bringing their huge brand to beef up sales. They should be making the platform better for some people, and then getting those people on to the platform to make WP7 phones aspirational. They aren't, so we just have to wait and see what happens with Nokia.

Footnote: Nokisoft
Nokia do well everywhere that Microsoft doesn't, by which I mean Europe and China mainly. Particularly China actually, where the Ovi store is pretty big. Conversely, MS phone isn't doing too badly in America, where Nokia traditionally are almost unheard of. So on paper, the Nokia deal should be a good thing for both companies.

#### A first look at appCode, and the future of Cocoa IDEs?
It's been almost a full rotation of this great rock about its axis since JetBrains announced the start of its appCode Early Access Program.

appCode is an Integrated Development Environment, just the same as Xcode. Just like Xcode, appCode works with Objective-C projects on Mac and iOS: in fact, under the hood it's using xcodebuild, llvm and all of the same tools that Xcode uses.

Unlike Xcode, appCode was written by JetBrains, the company who make the IntelliJ IDEA environment for Java. Now I've never used IntelliJ, but I have used Eclipse and NetBeans (and even Xcode!) for Java development, and that's kindof the point. There's a lot of competition in the Java world for IDEs, so they all have a boatload of features timesaving devices that are missing from Xcode.

Now, before we get too far into details, let's get this out of the way. appCode&mdash;though not itself a cross-platform app&mdash;is a Java app based on the same framework as IntelliJ. It therefore doesn't look like a Cocoa app, it looks like this:

That's enough to put some of you off, I'm sure. As people who know that user experience is the first most important factor of any app, it's easy to be distracted by an obviously alien user interface.

Maybe it's partly because I've used so many cross-platform IDEs, but I've become inured to the look of the products. I don't mind (or at least, can deal with) a nonstandard interface if it provides a better experience. It's quite pleasing that in some cases, appCode does indeed come through.

What does it do that Xcode doesn't?

One feature that every other IDE I've used has which is missing from Xcode is the ability to automatically stub out methods defined in the class interface (or the superclass interface, or a category, or a protocol...) in the implementation. This is a great feature for saving a boatload of typing (or even flipping between multiple views with copy and paste). Design your class's header, then use the Override Methods, Implement Methods and Generate&hellip; items in appCode's Code menu to automatically fill in the boilerplate bits in the implementation.

Another improvement appCode brings over Xcode is in its refactoring support. Xcode has long had a refactoring capability that only has a little more syntax awareness than sed, and I've previously talked about tops as a command-line equivalent. But what about when you want to change a method signature?

Xcode's refactoring tools can't support that, although I want to do it fairly frequently: in fact the last time was this morning. I wanted to change the return type and add a new parameter to an existing method, effectively going from -(void)frobulate: to -(BOOL)frobulate:error:. Well, it turns out that in appCode, you can do that:

So I should be using appCode, then?

Well, that's not my decision to make, but I definitely wouldn't recommend using it for production yet. It's got some bugs (yes, I have filed bug reports) particularly with managing Xcode projects and integrating with the underlying SDKs and tools. You probably don't want to touch production code with appCode yet.

Some particular issues I've found are with the default keyboard shortcuts conflicting with Mac OS X's default keyboard shortcuts, it doesn't offer a way to run the static analyzer, and I've had a problem where it removed a file from the definition of a project's target. I was able to address both of the last two problems by going back into Xcode, but of course it would be preferable if I didn't need to. Another big deal for me is that there isn't (yet?) any integration with the API documentation.

That said, it is just an early preview, so definitely download it and have a motor around. You'll probably find some things you do when you're coding that are easier in appCode, and other things that are easier in Xcode.

What's all this about the future of Cocoa IDEs?

I sincerely hope that JetBrains take this product to release and that people start adopting it. Based on the current preview, I would expect developers coming to iOS from Java development to be the majority of appCode users, because they'll be the people most comfortable with its way of working.

My biggest hope is that developers in Apple (both inside the Xcode team and beyond) pick up appCode and take a look at it. What we Cocoa developers need is some competition among IDE producers&mdash;particularly now that we need to pay for Xcode&mdash; to drive development of the tools we use for our day-to-day work. Xcode is to 2010s Objective-C IDEs as Internet Explorer was to 1990s browsers: it basically works, and you don't get a choice of what you use so be thankful for whatever features you get from On High.

It'd be great for someone to come along with the Firefox of Objective-C IDEs: a product that reminds users and vendors alike that there's more than one way to do an IDE and that people are open to trying the other ways.

#### A hopefully modern description of Object-Oriented Analysis
I’ve made a lot over the years, including the book, Object-Oriented Programming the Easy Way, of my assertion that one reason people are turned off from Object-Oriented Programming is that they weren’t doing Object-Oriented Design. Smalltalk was conceived as a system for letting children explore computers by writing simulations of other problems, and if you haven’t got the bit where you’re creating simulations of other problems, then you’re just writing with no goal in mind.

Taken on its own, Object-Oriented Programming is a bit of a weird approach to modularity where a package’s code is accessed through references to that package’s record structures. You’ve got polymorphism, but no guidance as to what any of the individual morphs are supposed to be, let alone a strategy for combining them effectively. You’ve got inheritance, but inheritance of what, by what? If you don’t know what the modules are supposed to look like, then deciding which of them is a submodule of other modules is definitely tricky. Similarly with encapsulation: knowing that you treat the “inside” and “outside” of modules differently doesn’t help when you don’t know where that side ought to go.

So let’s put OOP back where it belongs: embedded in an object-oriented process for simulating a problem. The process will produce as its output an executable model of the problem that produces solutions desired by…

Well, desired by whom? The answer that Extreme Programming and Agile Software Development, approaches to thinking about how to create software that were born out of a time when OOP was ascendant, would say “by the customer": “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”, they say.

Yes, if we’re working for money then we have to satisfy the customer. They’re the ones with the money in the first place, and if they aren’t then we have bigger problems than whether our software is satisfactory. But they aren’t the only people we have to satisfy, and Object-Oriented Design helps us to understand that.

If you’ve ever seen someone who was fully bought into the Unified Modelling Language (this is true for other ways of capturing the results of Object-Oriented Analysis but let’s stick to thinking about the UML for now), then you’ll have seen a Use Case diagram. This shows you “the system” as an abstract square, with stick figures for actors connected to “the system” via arrows annotated with use cases - descriptions of what the people are trying to do with the system.

We’ve already moved past the idea that the software and the customer are the only two things in the world, indeed the customer may not be represented in this use case diagram at all! What would that mean? That the customer is paying for the software so that they can charge somebody else for use of the software: that satisfying the customer is an indirect outcome, achieved by the action of satisfying the identified actors.

We’ve also got a great idea of what the scope will be. We know who the actors are, and what they’re going to try to do; therefore we know what information they bring to the system and what sort of information they will want to get out. We also see that some of the actors are other computer systems, and we get an idea of what we have to build versus what we can outsource to others.

In principle, it doesn’t really matter how this information is gathered: the fashion used to be for prolix use case documents but these days, shorter user stories (designed to act as placeholders for a conversation where the details can be worked out) are preferred. In practice the latter is better, because it takes into account that the act of doing some work changes the understanding of the work to be done. The later decisions can be made, the more you know at the point of deciding.

On the other hand, the benefit of that UML approach where it’s all stuck on one diagram is that it makes commonalities and pattern clearer. It’s too easy to take six user stories, write them on six Github issues, then get six engineers to write six implementations, and hope that some kind of convergent design will assert itself during code review: or worse, that you’ll decide what the common design should have been in retrospective, and add an “engineering story” to the backlog to be deferred forever more.

The system, at this level of abstraction, is a black box. It’s the opaque “context” of the context diagram at the highest level of the C4 model. But that needn’t stop us thinking about how to code it! Indeed, Behaviour-Driven Development has us do exactly that. Once we’ve come to agreement over what some user story or use case means, we can encapsulate (there’s that word again) our understanding in executable form.

And now, the system-as-simulation nature of OOP finally becomes important. Because we can use those specifications-as-scripts to talk about what we need to do with the customer ("so you’re saying that given a new customer, when they put a product in their cart, they are shown a subtotal that is equal to the cost of that product?"), refining the understanding by demonstrating that understanding in executable form and inviting them to use a simulation of the final product. But because the output is an executable program that does what they want, that simulation can be the final product itself.

A common step when implementing Behaviour-Driven Development is to introduce a translation layer between “the specs” and “the production code”. So “a new user” turns into User.new.save!, and someone has to come along and write those methods (or at least inherit from ActiveRecord). Or worse, “a new user” turns into PUT /api/v0/internal/users, and someone has to both implement that and the system that backs it.

This translation step isn’t strictly true. “A new user” can be an statement in your software implementation, your specs can be both a simulation of what the system does and the system itself, you save yourself a lot of typing and a potentially faulty translation.

There’s still a lot to Object-Oriented Analysis, but it roughly follows the well-worn path of Domain-Driven Design. Where we said “a new user”, everybody on the team should agree on what a “user” is, and there should be a concept (i.e. class) in the software domain that encapsulates (that word again!) this shared understanding and models it in executable form. Where we said the user “puts a product in their cart”, we all understand that products and carts are things both in the problem and in the software, and that a user can do a thing called “putting” which involves products, and carts, in a particular way.

If it’s not clear what all those things are or how they go together, we may wish to roleplay the objects in the program (which, because the program is a simulation of the things in the problem, means roleplaying those things). Someone is a user, someone is a product, someone is a cart. What does it mean for the user to add a product to their cart? What is “their” cart? How do these things communicate, and how are they changed by the experience?

We’re starting to peer inside the black box of the “system”, so in a future post we’ll take a proper look at Object-Oriented Design.

#### A hopefully modern description of Object-Oriented Design
We left off in the last post with an idea of how Object-Oriented Analysis works: if you’re thinking that it used around a thousand words to depict the idea “turn nouns from the problem domain into objects and verbs into methods” then you’re right, and the only reason to go into so much detail is that the idea still seems to confuse people to this day.

Similarly, Object-Oriented Design - refining the overall problem description found during analysis into a network of objects that can simulate the problem and provide solutions - can be glibly summed up in a single sentence. Treat any object uncovered in the analysis as a whole, standalone computer program (this is called encapsulation), and do the object-oriented analysis stuff again at this level of abstraction.

You could treat it as turtles all the way down, but just as Physics becomes quantised once you zoom in far enough, the things you need an object to do become small handfuls of machine instructions and there’s no point going any further. Once again, the simulation has become the deployment: this time because the small standalone computer program you’re pretending is the heart of any object is a small standalone computer program.

I mean, that’s literally it. Just as the behaviour of the overall system could be specified in a script and used to test the implementation, so the behaviour of these units can be specified in a script and used to test the implementation. Indeed some people only do this at the unit level, even though the unit level is identical to the levels above and below it.

Though up and down are not the only directions we can move in, and it sometimes makes more sense to think about in and out. Given our idea of a User who can put things in a Cart, we might ask questions like “how does a User remember what they’ve bought in the past” to move in towards a Ledger or PurchaseHistory, from where we move out (of our problem) into the realm of data storage and retrieval.

Or we can move out directly from the User, asking “how do we show our real User out there in the world the activities of this simulated User” and again leave our simulation behind to enter the realm of the user interface or network API. In each case, we find a need to adapt from our simulation of our problem to someone's (probably not ours, in 2021) simulation of what any problem in data storage or user interfaces is; this idea sits behind Cockburn's Ports and Adapters.

Moving in either direction, we are likely to encounter problems that have been solved before. The trick is knowing that they have been solved before, which means being able to identify the commonalities between what we’re trying to achieve and previous solutions, which may be solutions to entirely different problems but which nonetheless have a common shape.

The trick object-oriented designers came up with to address this discovery is the Pattern Language (OK, it was architect Christopher Alexander's idea: great artists steal and all that), in which shared solutions are given common names and descriptions so that you can explore whether your unique problem can be cast in terms of this shared description. In practise, the idea of a pattern language has fared incredibly well in software development: whenever someone says “we use container orchestration” or “my user interface is a functional reactive program” or “we deploy microservices brokered by a message queue” they are relying on the success of the patterns language idea introduced by object-oriented designers.

Meanwhile, in theory, the concept of patterns language failed, and if you ask a random programmer about design patterns they will list Singleton and maybe a couple of other 1990s-era implementations patterns before telling you that they don’t use patterns.

And that, pretty much, is all they wrote. You can ask your questions about what each object needs to do spatially ("who else does this object need to talk to in order to answer this question?"), temporally ("what will this object do when it has received this message?"), or physically ("what executable running on what computer will this object be stored in?"). But really, we’re done, at least for OOD.

Because the remaining thing to do (which isn’t to say the last thing in a sequence, just the last thing we have yet to talk about) is to build the methods that respond to those messages and pass those tests, and that finally is Object-Oriented Programming. If we start with OOP then we lose out, because we try to build software without an idea of what our software is trying to be. If we finish with OOP then we lose out, because we designed our software without using knowledge of what that software would turn into.

#### A kata above
The code kata is a method software craftspeople use to practice their craft. The idea is that you take a problem you understand, like FizzBuzz or Conway's Life, and build an application that implements it. Then build another one. And another. Pretty soon, you should be quite good at building a thing with well-understood requirements.

But that's the problem, isn't it? Our professional time can be split into two parts:

the part where we don't think we quite understand what we're meant to be doing.
the part where we are wrong.

So what's the point of being good at implementing well-understood requirements effortlessly? A partial answer is that being able to do so frees up our minds to think about the real problem, which is understanding the real problem. I'm sure our customers will be happy to hear that the only way we have to practice that is to take their money and pretend that we know what we're doing.

What are the causes of not knowing what we're doing? I can think of at least two: not knowing what the requirements actually are (which includes knowing what everyone thinks they are but not knowing how fulfilling those will change them), and having an existing system to adapt to the new requirements. A lot of the time, both of these are unknown.

We already have plenty of "work with legacy code" katas. We call them open source issue trackers. Find a project, find a bug, dive in and fix it. Practice working with code that's new to you, and already solves most of its problems.

You'll find that with open source and free software, the two extremes of bugfixing get practiced a lot. At one end, people are willing to fix trivial bugs that mean inserting one if into the existing code. At the other end, people will do complete rewrites, throwing away working code and its bugfixes to introduce "clean", though not necessarily working, code. It's the expanse of space between these, where you need to make substantial changes but without throwing out the beneficial existing behaviour, where there's space for legacy code katas.

But maybe we also need the "requirements kata". Think of one of your interests that isn't programming, and find a forum on it. Locate a thread where someone has asked a question that's led to either no answer or a convoluted process to solve their problem, and think about how you could design a software system to solve that, and what other information you'd need to specify it well. Think about how you'd validate that the derived system actually solves the problem.

#### A lack of CocoaHeads
Apologies to anyone sat in the Glue Pot wondering where the nerds are. Due to a general feeling of bitter colditude we've postponed for a couple of weeks. But never fear, Scotty shall still wax forth on SearchKit on the 16th of the month.

#### A little challenge
A little challenge today: create a JS function that turns its arguments into a list of pairs. Actually, the brief was "using Ramda" but I ended up not doing that:

function basePairwise(xs) {
  if (xs.length == 0) return [];
  if (xs.length == 1) return [[xs[0], undefined]];
  return [[xs[0], xs[1]]].concat(basePairwise(xs.slice(2)));
}

function pairwise(...xs) {
  return basePairwise(xs);
}

One of the nice things about JavaScript (I won't say that often, so note the date) is the introspective nature: the fact that I get to just look at the way a function was called, rather than saying "you must use exactly these types and this arity always". Here, I've done that with the JS spread operator: I could have used the arguments pseudo-list for only a little more work.

#### A note on notes
I've always had a way to take notes, but have never settled into a particular scheme. This post, more for my benefit than for yours, is an attempt to dig through this history and decide what I want to do about it.

At the high level, the relevant questions are what I want to do with the contents now and how I intend to work with them in the future. Most of the notes I take don't have a long-term future; my work from my first degree has long been destroyed. I referred to the notes during the degree which gives an upper bound on the lifetime of four years, realistically more like 2 from creation to the exam where I needed the notes.

Said notes were taken on A4 ruled paper with a cartridge pen and a propelling pencil. Being able to use text (including maths symbols etc) and diagrams interchangeably is a supremely useful capability. It even helps with code especially where UI or geometry is involved.

I no longer do this, but my strategy then was to take rapid notes in lectures and classes, and produce fair copies later. This meant absorbing more from the notes as I re-read them and put them into different words, and let me add cross references to textbooks or other materials.

I've used pen-and-paper note taking at other times. Particularly in classrooms or conferences, it's much faster than typing. At various phases of my career I've also kept log books, either for my own benefit or other people. That's not something I do currently. The weapons of choice in this sphere are now fountain pen, propelling pencil and Moleskine.

Evernote is my note shoebox of choice, and my destination for typing notes (in fact this draft was built up in Evernote on an iPhone, rather than a blog editor). I don't just use Macs and iOS so an iCloud-based note shoebox wouldn't work for me.

I sometimes put notes handwritten in books or on whiteboards in there too, but don't really worry about tagging because I usually search chronologically. My handwriting is so poor that Evernote's transcription doesn't work at all which is probably something that keeps me away from search. When it comes to symbols etc I'm more likely to put LaTeX markup in the text than draw equation images or use the extended characters palette.

When I was at O2 I had a dalliance with the Bamboo stylus and Penultimate. I still use those for drawing but never for writing as the poor sensitivity makes my narrow handwriting look even worse. I haven't tried anything with a dedicated stylus sensor like the Jot stylus, or the Galaxy S-pen. Again these get dumped into Evernote. I don't tend to change colours or pens; I tried Paper by 53 but don't use it much in practice.

Mind maps or outlines: sometimes. I only ever do these in software, never on paper.

I think the summary is that handwritten notes are fastest and allow the biggest variation in formatting and content. Sticking the resulting notes in Evernote helps to go back through them, but I should try to recover the discipline of writing up a fair copy. It helps cement the content in my mind and gives me a chance to add external references and citations that I would otherwise miss out.

The trick with paper-based notes is to always have a notebook and pen to hand; I don't often carry things around with me so I'd either have to get into the habit of wearing a manbag or leave notebooks around wherever I'm likely to want to write something.

#### A question of focus
The problem with The Labrary is that I offer to do so many things - because I could do them, and do them well - that it can be hard to find the one thing I could do for you that would be most helpful:

Artificial Intelligence
Agile Development
Continuous Delivery
Software Architecture
Technical Writing
Developer Experience
Programmer Mentoring

Each of these supports the mission of "making it faster and easier to make high-quality software that respects privacy and freedom", but all of them is overwhelming. I have credentials/experience to back up each of them, but probably don't have the reputation as a general expert that someone like Dan North or Liz Keogh can use to have people ask me anything.

So I want to pick one. One thing, probably from that list, and pivot to focus on that. Or at least get in through the door that way, then have the conversations about the other things once you know how much faster and easier I make it for you to make high-quality software.

And I'd really value your suggestions. Which one thing do you know me for, above all others? Which one thing is the pain that the place you work, or places you've worked, most need fixing?

Comment here, have a chat, send an email. Thanks for helping me find out what I want to be when I grow up.

#### A rap upon the noggin
When a patient may be concussed, it's common for parademics to ask simple, topical questions to determine whether the patient is confused. Questions such as "who is the Prime Minister"?I think somebody may have knocked this poor spammer upside the head (emphasis is mine):

Lloyd's TSB Group plc25 Gresham StreetLondon EC2V 7HNGreetings,Following the recent announcement by the Chancellor of the Exchequer, Gordon Brown that all assets in accounts that have been dormant for over 15years be transferred to the Treasury i send this mail to you.There is a dormant account in my office,with no owner or beneficiaries. It will be in my interest to transfer this assets worth 20,000,000 British pounds to an offshore country. If you can be a collaborator/partner to this please indicate interest immediately for us to proceed.Remember this is absolutely confidential,as i am seeking your assistance to act as the beneficiary of the account, since we are not allowed to operate a foreign accounts. Your contact phone numbers and name will be necessary for this effect.I have reposed my confidence in you and hope that you will not disappoint me.My Regards,Jim McConvilleLloyd's TSB Group plc

#### A site for discussing app security
There's a new IT security site over at Stack Exchange. Questions and answers on designing and implementing IT security policy, and on app security are all welcome.

I'm currently a moderator at the site, but that's just an interim thing while the site is being bootstrapped. Obviously, if people subsequently vote for me as a permanent moderator I'll stay in, but the converse is also true. Anyway, check out the site, ask and answer questions, let's make it as good a venue for app security discussion as stackoverflow.com is for general programming.

#### A sneaky preview of ClassBrowser
Let me start with a few admissions. Firstly, I have been computering for a good long time now, and I still don't really understand compilers. Secondly, work on my GNUstep Web side-project has tailed off for a while, because I decided I wanted to try something out to learn about the compiler before carrying on with that work. This post will mostly be about that something.

My final admission: if you saw my presentation on the ObjC runtime you will have seen an app called "ClassBrowser" where I showed that some of the Foundation classes are really in the CoreFoundation library. Well, there were two halves to the ClassBrowser window, and I only showed you the top half that looked like the Smalltalk class browser. I'm sorry.

So what's the bottom half?

This is what the bottom half gives me. It lets me go from this:

via this:

to this:

What just happened?

You just saw some C source being compiled to LLVM bit code, which is compiled just-in-time to native code and executed, all inside that browser app.

Why?

Well why not? Less facetiously:

I'm a fan of Smalltalk. I want to build a thing that's sort of a Smalltalk, except that rather than being the Smalltalk language on the Objective-C runtime (like F-Script or objective-smalltalk), it'll be the Objective-C(++) language on the Objective-C runtime. So really, a different way of writing Objective-C.
I want to know more about how clang and LLVM work, and this is as good a way as any.
I think, when it actually gets off the ground, this will be a faster way of writing test-first Objective-C than anything Xcode can do. I like Xcode 5, I think it's the better Xcode 4 that I always wanted, but there are gains to be had by going in a completely different direction. I just think that whoever strikes out in such a direction should not release their research project as a new version of Xcode :-).

Where can I get me a ClassBrowser?

You can't, yet. There's some necessary housekeeping that needs to be done before a first release, replacing some "research" hacks with good old-fashioned tested code and ensuring GNUstep-GUI compatibility. Once that's done, a rough-and-nearly-ready first open source release will ensue.

Then there's more work to be done before it's anything like useful. Particularly while it's possible to use it to run Objective-C, it's far from pleasant. I've had some great advice from LLVM IRC on how to address that and will try to get it to happen soon.

#### A solution in need of a problem
I don't usually do product reviews, in fact I have been asked a few times to accept a free product in return for a review and have turned them all down. This is just such an outr&eacute; product that I have decided to write a review, despite not being approached by the company and having no connection. The product is the 3M Privacy Filter Gold. Here is one I was given at InfoSec, on my MacBook:

You'll probably notice that the screen has some somewhat unsightly plastic tabs around the edge. They are holding in the main feature of this product, which is the thing giving the screen that slightly coppery colour. It's actually a sheet of plastic which I assume is etched with a blazed diffraction grating, because at more acute viewing angles it makes the screen look like this:

OK, so you can still see the plastic tabs, but now it's hard to make out the text. And that's the goal of the privacy filter gold: it's to stop shoulder surfers. By reducing the usable viewing angle of your screen. Hold on a moment, while I count the number of times I've been told about a business or government agency that leaked sensitive data through shoulder surfing.

0.

And the number of times I've heard (or discovered, through risk analysis) that it's an important risk for an organisation to address? About the same. OK, so this product is distinctive, and gimmicky, and evidently does what it's designed to do. But I don't see the problem that led to this product being developed. It might be useful if you want to browse porn in Starbucks or goof off on Facebook while you're at work, except that someone stood behind you can still see the screen. OK, you could browse porn while sat on the London Underground - except you won't be able to get a network signal.

If anti-shoulder-surfing is important to you, you may want to bear these issues into account. When used in strong sunlight, the privacy filter gold makes the screen look like this (note: availability of handsome security expert holding iPhone is strictly limited):

The MacBook isn't great in strong sunlight anyway, but with the filter over the top it becomes positively unusable. All of that dust is actually on the filter (although it was fresh out of its packaging when the picture was taken), and causing additional scatterings through its grating leading to a "gold dust" effect on the screen. And yes, the characters "3M GPF13.3W" are indeed etched into the filter at the top-right, near that thing that yes, is indeed a notch you can put your finger in for extracting the filter from the plastic tabs.

That's one issue, the other is price. Retail price for these filters is around £50, varying slightly depending on screen size. I'm really not sure that's a price worth paying considering that I have no idea what the benefit of the filter will be.

#### A spot of ultravaiolence, my droogs?
I'm currently having a Vaio VGN-S4M inflicted upon me.  I can't say I dislike it totally; the X-black screen is rather nice and it has one killer feature in that there's a (physical) radio kill switch on the front.  However, by and large I do dislike it.  The Centrino wireless jobbery is not fully documented, so only works with a binary blob driver which means it's useless for most of the operating systems I care about (it doesn't work in CEntOS, for instance, and I don't have the luxury of being able to switch to Ubuntu where I've heard better results are possible).  It seems to run very hot doing anything (including nothing), so the fan constantly winds up and down.  And it's a loud little fan.  The whole keyboard seems to bend a little when I press any key (although I'm used to the Tactile Pro, so probably whack a little harder than necessary...)  The only saving grace in the design is that PC manufacturers seem now to be getting over the desire to install legacy ports all over their laptops; but the thing's still closer to the size of a 14" iBoko (hi u.c.s.m) than that of its brother, the 13.3" MacBoko.So, with a recent cash injection and an ADC hardware discount coupon both burning a hole in my pocket, as well as a desire to crack on with the ObjC 2.0 work on the c.l.o-c FAQ; it seems an Intel Mac and a copy of some virtualisation tech are in order.  The question becomes simple: do I jump now, and get a MacBoko or MacBoko Pro, or do I hold on in case the Core 2 Duo variants are around the corner?  Or I could even combine the two and get an iMac, keeping the Vaio for corridor worrying?  I'm tempted by the "buy now" approach, as I actually need^Wwant a new system now, not in January...

#### A subtle [mis]understanding of monads
As I said when talking about Learning Phases, one of the things that happens when I'm trying to learn a new thing is that I build an analogy in terms of something I do understand. This can be dangerous when the analogy is wrong. I'm currently hanging on to the analogy that follows, so I'm publishing it as a straw man argument in the hope that those with more experience than me can critique it and help improve my understanding.

Proposition: Monads are what objects are supposed to look like

Throw away everything you know about Object-Oriented programming, particularly if you know Objective-C, Java, C++ or C#. Now go and read the first edition (the straightforward one, not the detective-novel second edition) of Object-Oriented Software Construction by Bertrand Meyer. You'll find the principle of Command-Query Separation, that an object's interface should provide distinct facilities for manipulating and investigating that object. Consider an object as an enclosed module analogous to an electronic circuit[*], with switches you can flip and lights you can observe. Watching a light shouldn't also have the effect of flipping a switch, and you shouldn't have to flip a switch in order to watch a light.

How could a language support that principle? Clearly imperative programming is too general, as it lets me mix commands and queries in the same place. What I need is an operation that lets me bind sequential commands together, taking an object in state a and a function that turns state a into state b and creating an object in state b. I also need an operation that lets me run a query, taking an object in state a and returning a value corresponding to a. Given those, and assuming the single responsibility principle has driven me to the point where my object does one thing and can be represented by a single command and query, then the interface on my object is complete.

Interestingly and usefully, except in the (common-ish) special case where my object needs to talk to the world, objects defined in this way are completely understandable mechanistically. The same sequence of commands applied to the same constructor will always yield the same value on query.

But that is, I think, what Monads are supposed to be: things that bind commands into sequences and allow inspection at points in the sequence. As stated in this post's preamble, I'm not fully confident of this, and may be over-applying an analogy to replace a thing I don't understand with the comfort of the thing I do. I'd be interested in hearing informed critique on the argument.

Not the other monads

When Alan Kay said that objects are monads, he was talking about Leibniz monads as previously discussed on this blog.

[*] An analogy already present in the proceedings of &#95;that&#95; 1968 NATO conference, and drawn to extremes by Brad Cox in Object-Oriented Programming: An Evolutionary Approach.

Update

The discussion among my monadically-inclined friends on Facebook indicates that this analogy is incomplete. Yes, monads can be used like that, but they need not be used like that. Counter-examples included the reverse state monad in which state flows backwards, and the TARDIS monad in which state can flow in both directions.

#### A two-dimensional dictionary
What?

A thing I made has just been open-sourced by my employers at Agant: the AGTTwoDimensionalDictionary works a bit like a normal dictionary, except that the keys are CGPoints meaning we can find all the objects within a given rectangle.

Why?

A lot of time on developing Discworld: The Ankh-Morpork Map was spent on performance optimisation: there's a lot of stuff to get moving around a whole city. As described by Dave Addey, the buildings on the map were traced and rendered into separate images so that we could let characters go behind them. This means that there are a few thousand of those little images, and whenever you're panning the map around the app has to decide which images are visible, put them in the correct place (in three dimensions; remember people can be in front of or behind the buildings) and draw everything.

A first pass involved creating a set containing all of the objects, looping over them to find out which were within the screen region. This was too slow. Implementing this 2-d index instead made it take about 20% the original time for only a few tens of kilobytes more memory, so that's where we are now. It's also why the data type doesn't currently do any rebalancing of its tree; it had become fast enough for the app it was built in already. This is a key part of performance work: know which battles are worth fighting. About one month of full-time development went into optimising this app, and it would've been more if we hadn't been measuring where the most benefit could be gained. By the time we started releasing betas, every code change was measured in Instruments before being accepted.

Anyway, we've open-sourced it so it can be fast enough for your app, too.

How?

There's a data structure called the multidimensional binary tree or k-d tree, and this dictionary is backed by that data structure. I couldn't find an implementation of that structure I could use in an iOS app, so cracked open the Objective-C++ and built this one.

Objective-C++? Yes. There are two reasons for using C++ in this context: one is that the structure actually does get accessed often enough in the Discworld app that dynamic dispatch all the way down adds a significant time penalty. The other is that the structure contains enough objects that having a spare isa pointer per node adds a significant memory penalty.

But then there's also a good reason for using Objective-C: it's an Objective-C app. My controller objects shouldn't have to be written in a different language just to use some data structure. Therefore I reach for the only application of ObjC++ that should even be permitted to compile: an implementation in one language that exposes an interface in the other. Even the unit tests are written in straight Objective-C, because that's how the class is supposed to be used.

#### About NSMailDelivery and Message.framework
I've been asked a few times about the NSMailDelivery class by a few people, probably either through a mailing list post I made, or iSendMail.  Well, let's look through iSendMail to see what happens.Firstly, the iSendMail target includes Message.framework, which is where NSMailDelivery is defined.  Now, some clips from the iSendMail.m file:

 if([NSMailDelivery hasDeliveryClassBeenConfigured]==NO) {  //can't send mail...  NSLog(@"Cannot send mail!n");  [pool release];  exit(EXIT_FAILURE); }

Before I even try to send mails out, I check whether NSMailDelivery is going to work.  The clincher is straightforward: if Mail.app can send an email, NSMailDelivery can send an email.  There's no way to configure e.g. a SMTP server programatically (unless you want to delve into Mail.app's configuration, that is); if you want to do this, consider using PantoMIME which is a more complete message framework.  NSMailDelivery is just a convenience class for accessing Mail.app's message sending capabilities, nothing more.  Launch Mail, open a new message, put a To address in and hit Cmd-Shift-D.  If it goes, then you can use NSMailDelivery.

 NSMutableDictionary *headers=[NSMutableDictionary dictionary]; while((argch=getopt(argc,argv,"f:t:s:"))!=-1) {  switch(argch)  {  case 'f':   [headers setObject:[NSString stringWithCString:optarg] forKey:@"From"];   break;  case 't':   [headers setObject:[NSString stringWithCString:optarg] forKey:@"To"];   break;  case 's':   [headers setObject:[NSString stringWithCString:optarg] forKey:@"Subject"];   break;  case '?':  default:   usage();  } }

Headers are defined in a dictionary.  Obviously a To header is a requirement, you could probably get away without a Subject.  If I wanted to use Mail's default email address I wouldn't supply a From header (but I'd use a different delivery method, see below); and if I wanted custom X-Mailer: or whatever, I'd do it here too.  Any header which appears here will make it into the resultant mail, and any header not listed here will not be inserted for you.

 NSAttributedString *attrMsg; NSMutableString *msg=[[NSMutableString alloc] init];//... attrMsg=[[NSAttributedString alloc] initWithString:msg];

The message must be an attributed string, whether or not you actually use any attributes.  But unless you choose a send a MIME message, any attribute you do use will be lost, read on...

 mailSent=[NSMailDelivery deliverMessage:attrMsg headers:headers format:NSASCIIMailFormat protocol:nil];  if(mailSent==YES) {  NSLog(@"Mail sent successfully.n"); } else {  NSLog(@"Mail delivery failed!n"); }

a few things to note about the delivery line.  Firstly, observe that all of these methods we're using are class methods, not instance methods; I never instantiate anything from Message.framework.  If I hadn't set a From address I could have used +deliverMessage:subject:to: but then could only send an ASCII message (which is all iSendMail does, anyway) and would have had to accept whatever From address was default in Mail.app.  If your message atributed string contains formatting information (or MIME attachments :-)) then use NSMIMEMailFormat.  protocol may as well always be nil, as your choices are that (which uses the preferred transport protocol, which is SMTP at the moment on Mac OS X) or NSSMTPDeliveryProtocol, which is also SMTP.

#### Acne cream
I just want to point out that even the best of us aren't doing what we expect the makers of acne creams to do.

What we actually know about software development, and why we believe it's true&nbsp;by Greg Wilson.

#### Adding components to a GNUstep web / WebObjects app
In WebObjects, Components take the role of a view controller in what passes for Cocoa's version of MVC. Each is responsible for calculating the data that the view objects are bound to: you saw an example of this in the previous post. Each is also responsible for responding to user interface events&mdash;called actions&mdash;and preparing the next component where that's required.

Navigating between components

Let's add a new component to our HelloGSW test app. We've recorded the date that the visitor arrived at the site, we could (for sake of demonstration, not because it's useful) find out how long it takes that visitor to get to the next page. We'll come onto the mechanics of adding it to the project later, but for now we note that we require a way for the visitor to indicate their intention to see the next page. That means, as a simple way to do things, having a form with a submit button.

We could write the form into the HTML, but then we'd have to know what URL the WO machinery was going to use to call our action. So we'll use WebObjects to generate the form, using this HTML:

 &lt;webobject name=form&gt;
   &lt;webobject name=submit_button&gt;&lt;INPUT TYPE="SUBMIT"&gt;&lt;/webobject&gt;
 &lt;/webobject&gt;

As you've seen, the data bindings for each webobject need to be defined in the WOD file:

form:WOForm {}
submit_button:WOSubmitButton {action = nextPage}

As with the value binding you saw last time, we can bind the action of the submit button to a method on the component. Those of you who have used Cocoa Bindings may be surprised to find that you can do the same with NSResponder subclasses, though it's rare to see it done with Mac apps.

So finally, I GIVE YOU TEH CODEZ:

- (GSWComponent *)nextPage {
  ClickDelay *nextComponent = (ClickDelay *)[self pageWithName: @"ClickDelay"];
  nextComponent.startDate = accessedAt;
  return nextComponent;
}

As with view controllers on the iPhone, you just create the new component (-pageWithName: does that, and configures its context correctly) and configure state where appropriate. Unlike with iOS view controllers, you don't specify how they get presented because there's only one way that will (probably) ever happen: in an HTTP response. So you can just return the next component from the method and the Right Thing™ will happen. If you return nil, the visitor will see the current component again (though with a whole server round-trip and a fresh HTML response, so it's best to do some AJAXey thing: on which more later!).

If you're really interested in how the other component works, check out its source from GitHub.

But that's not why we're here.

Adding a component involves a boatload of boilerplate: five new files, and a change to the GNUmakefile. That's one more boatload than you'd like to write every time you create a new page in your web app, yes?

Yes. Therefore what I actually did was to write a script to do all of that boilerplate for me. It's really basic (error checking is for security boffins. Oh, wait.) but will do the necessary dance. Just be sure not to run it twice for the same component yet&hellip;

What I really need is a tool that's both more robust, and that can generate the other things you need in writing a WO app. I'll start right now, but pull requests are always welcome&hellip;

#### Against our values as a company
I'm going to pick on Patreon, not because they're special, but because they're typical. Here is a quote from a blog post in which they Back-pedalled a change to their pricing structure:

We overstepped our bounds and injected ourselves into that relationship, against our core belief as a business.

What I think they want us to hear is "this was a surprising event that normally wouldn't happen because it's important to us that other things happen instead". What I actually hear is that management at companies like this is at best asleep at the wheel, at worst Github/Uber levels of arrogant and ignorant. Either way, new management is needed.

It's fine to mistakes. It's fine for someone acting in what they believe to be the best way aligned with their company's goals and mission to do something out of step, that gets detected and corrected. That's good management: trust, but verify. On the other hand, this is not an intern sending out an off-tone tweet. For a whole organisation to pivot its business model, create the marketing message, release the marketing message, and schedule the implementation of the new model...either management were involved and didn't notice or care how broken it all was, or management are not involved and the whole company works around them.

When you want to sweep bad actions under rug of misalignment with your core belief as a business, think about what it means that your business isn't taking your core belief as a business on board.

#### Agile application security
There's a post by clever security guy Jim Bird on Appsec's Agile Problem: how can security experts participate in fast-moving agile (or Agile™) projects without either falling behind or dragging the work to a halt?

I've been the Appsec person on such projects, so hopefully I'm in a position to provide at least a slight answer :-).

On the team where I did this work, projects began with the elevator pitch, application definition statement, whatever you want to call it. "We want to build X to let Ys do Z". That, often with a straw man box-and-line system diagram, is enough to begin a conversation between the developers and other stakeholders (including deployment people, marketing people, legal people) about the security posture.

How will people interact with this system? How will it interact with our other systems? What data will be touched or created? How will that be used? What regulations, policies or other constraints are relevant? How will customers be made aware of relevant protections? How can they communicate with us to identify problems or to raise their concerns? How will we answer them?

Even this one conversation has achieved a lot: everybody is aware of the project and of its security risks. People who will make and support the system once it's live know the concerns of all involved, and that's enough to remove a lot of anxiety over the security of the system. It also introduces awareness while we're working of what we should be watching out for. A lot of the suggestions made at this point will, for better or worse, be boilerplate: the system must make no more use of personal information than existing systems. There must be an issue tracker that customers can confidentially participate in.

But all talk and no trouser will not make a secure system. As we go through the iterations, acceptance tests (whether manually run, automated, or results of analysis tools) determine whether the agreed risk profile is being satisfied.

Should there be any large deviations from the straw man design, the external stakeholders are notified and we track any changes to the risk/threat model arising from the new information. Regular informal lunch sessions give them the opportunity to tell our team about changes in the rest of the company, the legal landscape, the risk appetite and so on.

Ultimately this is all about culture. The developers need to trust the security experts to make their expertise available and help out with making it relevant to their problems. The security people need to trust the developers to be trying  to do the right thing, and to be humble enough to seek help where needed.

This cultural outlook enables quick reaction to security problems detected in the field. Where the implementors are trusted, the team can operate a "break glass in emergency" mode where solving problems and escalation can occur simultaneously. Yes, it's appropriate to do some root cause analysis and design issues out of the system so they won't recur. But it's also appropriate to address problems in the field quickly and professionally. There's a time to write a memo to the shipyard suggesting they use thicker steel next time, and there's a time to put your finger over the hole.

If there's a problem with agile application security, then, it's a problem of trust: security professionals, testers, developers and other interested parties[*] must be able to work together on a self-organising team, and that means they must exercise knowledge where they have it and humility where they need it.

[*] This usually includes lawyers. You may scoff at the idea of agile lawyers, but I have worked with some very pragmatic, responsive, kind and trustworthy legal experts.

#### Ah, the sweet sound of my own voice
The title is a linky to the press release regarding the edition of Sophos Podcasts I recorded with Carole, and which has now (clearly) gone live.  In it we mainly talk about the data theft and Macs technical paper I've already posted about.  This is the first podcast I've ever been involved in, so any feedback you have (apart from telling me that I sound drunk in my first sentence, I'm not sure what that's all about) will be most welcome!

#### AJAX via jQuery in an Objective-C WebObjects app
As with using jQuery for DHTML, this was surprisingly easy. To make it simple to follow along I've published the source code to SignUp, a sample app.

SignUp's default page is comprised of two WebObjects components: Main is the top-level page (including the jQuery script) and SignUpForm implements the form. As described in the previous post on this topic, the form's email field is only enabled and populated if you tick the 'contact me' button.

We can submit the form via an AJAX request, rather than by making the browser do a full request-response cycle. To do this, install a handler on the form's submit event that does an AJAX post of the form then cancels the default behaviour.

   var signUpForm = $("#signUpForm");
    signUpForm.submit(function() {
        $.post(signUpForm.prop("action"), signUpForm.serialize(), function(data) {
            $("#formwrapper").html(data);
        });
        return false;
    });

As I've said before, I'm a jQuery newbie, there might be an easier way to do the above but this is definitely easy enough. One thing to notice is that the form's own action is used as the POST URL, meaning that the WebObjects code is still responsible for expressing the control flow through the app.

The handler's completion function replaces the form's content with the data it receives from the POST action. That's going to be another component, specified in the action handler:

- (GSWComponent *)registerInterestAction
{
  GSWRequest *request = [self request];
  [SignUpForm processSignUp: request];
  return [self pageWithName: @"Thanks"];
}

The request object provides access to the form values which are used in the app to populate an Add Person command, which is handled by adding the user's details to the database. Finally the action handler loads a component called Thanks and returns that, which will replace the form in the web page.

One thing to notice is that the server side is stateless; unlike many WebObjects apps that use a WOSession subclass (and have fugly URLs incorporating the session ID, if badly configured) everything is done via Direct Actions in this app, there's no server-side state, and no session should get created. This conveniently avoids a big lock in the WebObjects framework, where the WOApplication instance has to synchronise access to the session table.

A stateless server satisfies one of the REST constraints; obviously we're also using code-on-demand via JavaScript and have a client–server, layered system. Resources can be cacheable by setting the appropriate HTTP headers, not shown here but certainly doable. The only constraint not satisfied is the uniform interface and even that is partially present as WebObjects is by its very nature a HATEOAS system. Indeed this application arranged to observe HATEOAS by allowing the form object to express its own action URI, rather than "knowing" the form destination in the client code. The only part that's missing is a resource identification and manipulation interface: instead of POSTing a form as done here a client would PUT a new person.

#### All the more reason to like FOSDEM
So it seems that my half-attendance at FOSDEM paid off more than I could have hoped, as I won a year's subscription to GNU/Linux magazine.  The publication is francophonic, so this will be a good chance to improve my command of la langue des grenouilles ;-).

#### All the things
It's been a long time since I had a side project, or one that didn't get abandoned very early on. I tend to get sidetracked by other thoughts about computing, or think "while I'm doing this, I'm leaving that unsolved" so nothing gets very far.

In an attempt to address that, to clear all of the different thoughts I have about the matter of computing out of my head, organise them, identify conflicts, and prioritise what I work on, I spent this evening jotting down the big points and a brief abstract about each one. I'm hoping this will cut the Gordian knot by letting me see it all in one place and start to make choices.

The format I chose to represent this braindump is this personal Technology Radar, based on the Thoughtworks build-your-own tool. It seemed like a good place to see everything at once, and look for clusters or trends.

You'll notice that almost everything in this radar is fairly old tech! That's mostly a matter of taste, as I enjoy learning about things that were tried, what succeeded or failed, and what can be learnt from that to put to use today. I'm not good at novelty for novelty's sake.

I expect to get some mileage (for my own benefit, you might like it too) out of expanding on some of the entries in this radar over a few more posts, so I've created a techradar category in this blog that you can filter on/out.

#### All you never wanted to know about temporary files and were too ambivalent to ask
In the beginning, there was mktemp. And it was good.

Actually, that's a load of rubbish, it wasn't good at all. By separating the "give me the name of a temporary file" and "open the file" stages, there's a chance for an attacker to create the temporary file with the name you've chosen between the stages, or create a symlink with the same name.

Next there came mkstemp. And that was better. But not by much. mkstemp opens the file for you as well as choosing a name, so the file was guaranteed to not exist beofre you tried to use it, and will definitely have the ownership and permissions you want.

There is yet another step which the über-paranoid application could take in order to ensure that no other process can see its temporary files, and that is to unlink the file as soon as you get it back. Unfortunately there's no "mkstempr" function (and it might get confused with the equally non-existent mkstemp_r), so this is still a two-stage operation. Unlinking a file which you have open removes it from the directory listing, but doesn't change the fact that you have it open; it's now exclusively yours.

#### AMA
It was requested on twitter that I start answering community questions on the podcast. I've got a few to get the ball rolling, but what would you like to ask? Comment here, or reach me wherever you know I hang out.

#### An acceptable tool
An acceptable tool

It's easy to forget that adequacy is, well, adequate. It's easy to go all-in on making some code structure perfect, when good enough would be good enough.

#### An apology and an opportunity
Today's earlier post, UX is snake-oil bullshit, was indeed an April Fool. Sorry to the people who had their "WTF blood boil", among other reactions.

I'm also sorry to the people I parodied in the post. Please feel comfortable knowing that if I didn't respect you and what you did, I wouldn't feel willing to send you up in that fashion.

Now of course some people saw the joke, and some people didn't. But what was most interesting was that in both camps there was a (large) fraction of respondents who recognised in the jokes the germ of truth. They realised that the humour came not from eschewing reality completely, but from taking cues from it and dialling them up to 11 so that the absurdity became clear.

So each of the points in my previous post is true, just not as true as the rhetoric would have you believe. The field of UX does contain people who can spout the platitudes, but don't back it up. There is an amount of "look at this fail". It is a fast-moving field, so it's hard to find timeless, consistent, actionable advice. And there are people who will tell you that whatever they're doing (or selling) is UX, that when bolted to your product creates magical UX goodness.

What that means is not, as my April Fool post concluded, that it's time to withdraw from study of User Experience. It means that there's a tremendous appetite among software engineers for resources offering specific, actionable tasks that we can undertake to improve the experience of our products.

One of the books on user experience I've got the most out of is Don't Make Me Think!: A Common Sense Approach to Web Usability by Steve Krug. When he does have some specific issue to point out in the design of a website, he also shows either how the problem was or could be addressed. We could do with a "Don't make me think!" for the mobile user experience. We need someone to cherry-pick the research on user testing, ethnography and other techniques, telling us what's important and what to ignore. What we can be doing, and what we should be doing.

That's a great opportunity for someone who wants to become the Smartphone UX Boffin.

#### An apology to readers of Test-Driven iOS Development
I made a mistake. Not a typo or a bug in some pasted code (actually I've made some of those, too). I perpetuated what seems (now, since I analyse it) to be a big myth in software engineering. I uncritically quoted some work without checking its authority, and now find it lacking. As an author, it's my responsibility to ensure that what I write represents the best of my knowledge and ability so that you can take what I write and use it to be better at this than I am.

In propagating incorrect information, I've let you down. I apologise unreservedly. It's my fervent hope that this post explains what I got wrong, why it's wrong, and what we can all learn from this.

The mistake

There are two levels to this. The problem lies in Table 1.1 at the top of page 4 (in the print edition) of Test-Driven iOS Development. Here it is:

As explained in the book, this table is reproduced from an earlier publication:

Table 1.1, reproduced from Code Complete, 2nd Edition, by Steve McConnell (Microsoft Press, 2004), shows the results of a survey that evaluated the cost of fixing a bug as a function of the time it lay "dormant" in the product. The table shows that fixing bugs at the end of a project is the most expensive way to work, which makes sense…

The first mistake I made was simply that I seem to have made up the bit about it being the result of a survey. I don't know where I got that from. In McConnell, it's titled "Average Cost of Fixing Defects Based on When They're Introduced and Detected" (Table 3-1, at the top of page 30). It's introduced in a paragraph marked "HARD DATA", and is accompanied by an impressive list of references in the table footer. McConnell:

The data in Table 3-1 shows that, for example, an architecture defect that costs $1000 to fix when the architecture is being created can cost $15,000 to fix during system test.

As already covered, the first problem is that I misattributed the data in the table. The second problem, and the one that in my opinion I've let down my readers the most by not catching, is that the table is completely false.

Examining the data

I was first alerted to the possibility that something was fishy with these data by the book the Leprechauns of Software Engineering by Laurent Bossavit. His Appendix B has a broad coverage of the literature that claims to report the exponential increase in cost of bug fixing.

It was this that got me worried, but I thought deciding that the table was broken on the basis of another book would be just as bad as relying on it from CC2E was in the first place. I therefore set myself the following question:

Is it possible to go from McConnell's Table 3-1 to data that can be used to reconstruct the table?

My results

The first reference is Design and Code Inspections to Reduce Errors in Program Development. In a perennial problem for practicing software engineers, I can't read this paper: I subscribe to the IEEE Xplore library, but they still want more cash to read this title. Laurent Bossavit, author of the aforementioned Leprechauns book, pointed out that the IEEE often charge for papers that are available for free elsewhere, and that this is the case with this paper (download link).

The paper anecdotally mentions a 10-100x factor as a result of "the old way" of working (i.e. without code inspections). The study itself looks at the amount of time saved by adding code reviews to projects that hitherto didn't do code reviews; even if it did have numbers that correspond to this table I'd find it hard to say that the study (based on a process where the code was designed such that each "statement" in the design document corresponded to 3-10 estimated code statements, and all of the code was written in the PL/S language before a compiler pass was attempted) has relevance to modern software practice. In such a scheme, even a missing punctuation symbol is a defect that would need to be detected and reworked (not picked up by an IDE while you're typing).

The next I discovered was Boehm and Turner's "Balancing Agility and Discipline". McConnell doesn't tell us where in this book he was looking, and it's a big book. Appendix E has a lot of secondary citations supporting "The steep version of the cost-of-change curve", but quotes figures from "100:1" to "5:1" comparing "requirements phase" changes to "post-delivery" changes. All defect fixes are changes but not all changes are defect fixes, so these numbers can't be used to build Table 3-1.

The graphs shown from studies for agile Java projects have "stories" on the x axis and "effort to change" in person/hour on the y-axis; again not about defects. These numbers are inconsistent with the table in McConnell anyway. As we shall see later, Boehm has trouble getting his data to fit agile projects.

"Software Defect Removal" by Dunn is another book, which I couldn't find.

"Software Process Improvement at Hughes Aircraft" (Humphrey, Snyder, and Willis 1991) The only reference to cost here is a paragraph on "cost/performance index" on page 22. The authors say (with no supporting data; the report is based on a confidential study) that costs for software projects at Hughes were 6% over budget in 1987, and 3% over budget in 1990. There's no indication of whether this was related to the costs of fixing defects, or the "spread" of defect discovery/fix phases. In other words this reference is irrelevant to constructing Table 3-1.

The other report from Hughes Aircraft is "Hughes Aircraft’s Widespread Deployment of a Continuously Improving So" by Ron R. Willis, Robert M. Rova et al.. This is the first reference I found to contain useful data! The report is looking at 38,000 bugs: the work of nearly 5,000 developers over dozens of projects, so this could even be significant data.

It's a big report, but Figure 25 is the part we need. It's a set of tables that relate the fix time (in person-days) of defects when comparing the phase that they're fixed with the phase in which they're detected.

Unfortunately, this isn't the same as comparing the phase they're discovered with the phase they're introduced. One of the three tables (the front one, which obscures parts of the other two) looks at "in-phase" bugs: ones that were addressed with no latency. Wide differences in the numbers in this table (0.36 days to fix a defect in requirements analysis vs 2.00 days to fix a defect in functional test) make me question the presentation of Table 3-1: why put "1" in all of the "in-phase" entries in that table?

Using these numbers, and a little bit of guesswork about how to map the headings in this figure to Table 3-1, I was able to use this reference to construct a table like Table 3-1. Unfortunately, my "table like Table 3-1" was nothing like Table 3-1. Far from showing an incremental increase in bug cost with latency, the data look like a mountain range. In almost all rows the relative cost of a fix in System Test is greater than in maintenance.

I then looked at "Calculating the Return on Investment from More Effective Requirements Management" by Leffingwell. I have to hope that this 2003 webpage is a reproduction of the article cited by McConnell as a 1997 publication, as I couldn't find a paper of that date.

This reference contains no primary data, but refers to "classic" studies in the field:

Studies performed at GTE, TRW, and IBM measured and assigned costs to errors occurring at various phases of the project life-cycle. These statistics were confirmed in later studies. Although these studies were run independently, they all reached roughly the same conclusion: If a unit cost of one is assigned to the effort required to detect and repair an error during the coding stage, then the cost to detect and repair an error during the requirements stage is between five to ten times less. Furthermore, the cost to detect and repair an error during the maintenance stage is twenty times more.

These numbers are promisingly similar to McConnell's: although he talks about the cost to "fix" defects while this talks about "detecting and repairing" errors. Are these the same things? Was the testing cost included in McConnell's table or not? How was it treated? Is the cost of assigning a tester to a project amortised over all bugs, or did they fill in time sheets explaining how long they spent discovering each issue?

Unfortunately Leffingwell himself is already relying on secondary citation: the reference for "Studies performed at GTE…" is a 520-page book, long out of print, called "Software Requirements—Objects Functions and States". We're still some degrees away from actual numbers. Worse, the citation at "confirmed in later studies" is to Understanding and controlling software costs by Boehm and Papaccio, which gets its numbers from the same studies at GTE, TRW and IBM! Leffingwell is bolstering the veracity of one set of numbers by using the same set of numbers.

A further reference in McConnell, "An Economic Release Decision Model" is to the proceedings on a 1999 conference on Applications of Software Measurement. If these proceedings have actually been published anywhere, I can't find them: the one URL I discovered was to a "cybersquatter" domain. I was privately sent the powerpoint slides that comprise this citation. It's a summary of then-retired Grady's experiences with software testing, and again contains no primary data or even verifiable secondary citations. Bossavit describes a separate problem where one of the graphs in this presentation is consistently misattributed and misapplied.

The final reference provided by McConnell is What We Have Learned About Fighting Defects, a non-systematic literature review carried out in an online "e-Workshop" in 2002.

Section 3.1 of the report is "the effort to find and fix". The 100:1 figure is supported by "general data" which are not presented and not cited. Actual cited figures are 117:1 and 135:1 for "severe" defects from two individual studies, and 2:1 for "non-severe" defects (a small collection of results). 

The report concludes:

"A 100:1 increase in effort from early phases to post-delivery was a usable heuristic for severe defects, but for non-severe defects the effort increase was not nearly as large. However, this heuristic is appropriate only for certain development models with a clearly defined release point; research has not yet targeted new paradigms such as extreme programming (XP), which has no meaningful distinction between “early” and “late” development phases."

A "usable heuristic" is not the verification I was looking for - especially one that's only useful when practising software development in a way that most of my readers wouldn't recognise.

Conclusion

If there is real data behind Table 3-1, I couldn't find it. It was unprofessional of me to incorporate the table into my own book—thereby claiming it to be authoritative—without validating it, and my attempts to do so have come up wanting. I therefore no longer consider Table 1-1 in Test-Driven iOS Development to be representative of defect fixing in software engineering, I apologise for including it, and I resolve to critically appraise material I use in my work in the future.

#### An entirely unwarranted comparison between software engineering and astronomy
Back in the early days of astronomy, the problem of the stars that wander from fixed positions in the sky needed solving. Many astronomers, not the first of which was Ptolemy, proposed that these "planetai" could be modeled as following little curves—epicycles—through their larger motions. As it was found that these epicycles continued to fail, smaller and smaller iterations were added. It was not until astronomers realised that they were not at the centre of the universe that they realised this was an over-complicated and unnecessary model.

Here, in the early days of making software, the problem of the software that wanders from budget, quality and time expectations needed solving. Many programmers, not the first of which was Boehm, proposed that these "projects" could be modeled as following little curves—spirals—through their larger motions. As it was found that these spirals continued to fail, smaller and smaller iterations were added.

#### An example of unit testing working for me
Some specific feedback I was given regarding my unit testing talk at VTM: iPhone fall conference was that the talk was short on real-world application of unit testing. That statement is definitely true, and it's unfortunate that I didn't meet the attendee's expectations for the talk. I hope to address that deficiency here, by showing you a real bug that I really fixed via unit testing.

The code is in the still-MIA Rehearsals app. I started developing Rehearsals before I was a full believer in test-driven development, so there were places in the app where I could go back and add tests to existing code. This is actually a pretty useful displacement activity - if you get bored of doing the real code, you can switch to writing the tests. This is enough of a change of pace (for me) to provide a welcome distraction, and still makes the product better.

So, in order to understand the problem, I need to understand the requirements. I'm a fan of the user story approach to requirements documentation, and so Rehearsals has the following story written on a 5x3 index card:

A musician can rate tunes in her tunebook, by giving it a number of stars between 1 and 5. Tunes that the musician has not rated have no star value.

As is often the case, there are other stories that build from this one (any such collection is called an "epic"), but I can concentrate on implementing this user story for now. One feature in Rehearsals is an AppleScript interface, so clearly if the musician can rate a tune, she must be able to do it via AppleScript. Glossing over the vagiaries of the Cocoa Scripting mechanism, I define a property scriptRating on the tune that had the following KVC-compliant accessors:

- (NSNumber *)scriptRating {
    if (!self.rating) {
        return [NSNumber numberWithInteger: 0];
    }
    return self.rating;
}

- (void)setScriptRating: (NSNumber *)newRating {
    NSInteger val = [newRating integerValue];
    if (val < 0 || val > 5) {
        //out of range, tell AppleScript
        NSScriptCommand *cmd = [NSScriptCommand currentCommand];
       [cmd setScriptErrorNumber: TORatingOutOfRangeError];
        //note that AppleScript is never localised
        [cmd setScriptErrorString: @"Rating must be between 0 and 5."];
    }
    else {
        self.rating = newRating;
    }
}

(No prizes for having already found the bug - it's really lame). So testing that the getter works is ultra-simple: I can just -setRating: and verify that -scriptRating returns the same value. You may think this is not worth doing, but as this functionality is using indirect access via a different property I want to make sure I never break the connection. I decided that a single test would be sufficient (tune is an unconfigured tune object created in -setUp):

- (void)testTuneRatingIsSeenInAppleScript {
    tune.rating = [NSNumber numberWithInteger: 3];
    STAssertEquals([tune.scriptRating integerValue], 3,
                   @"Tune rating should be visible to AppleScript");
}

Simples. Now how do I test the setter? Well, of course I can just set the value and see whether it sticks, that's the easy bit. But there's also this bit about being able to get an error into AppleScript if the user tries to set an out of range error. That seems like really useful functionality, because otherwise the app would accept the wrong value and give up later when the persistent store gets saved. So it'd be useful to have a fake script command object that lets me see whether I'm setting an error on it. That's easy to do:

@interface FakeScriptCommand : NSScriptCommand
{
    NSString *scriptErrorString;
    NSInteger scriptErrorNumber;
}
@property (nonatomic, copy) NSString *scriptErrorString;
@property (nonatomic, assign) NSInteger scriptErrorNumber;
@end

@implementation FakeScriptCommand
@synthesize scriptErrorString;
@synthesize scriptErrorNumber;
- (void) dealloc {...}
@end

OK, but how do I use this object in my test? The tune class sets an error on +[NSScriptCommand currentScriptCommand], which is probably returning an object defined in a static variable in the Foundation framework. I can't provide a setter +setCurrentScriptCommand:, because I can't get to the innards of Foundation to set the variable. I could change or swizzle the implementation of +currentScriptCommand in a category, but that has the potential to break other tests if they're not expecting me to have broken Foundation.

So the solution I went for is to insert a "fulcrum" if you will: a point where the code changes from asking for a script command to being told about a script command:

- (void)setScriptRating: (NSNumber *)newRating {
    [self setRating: newRating fromScriptCommand: [NSScriptCommand currentScriptCommand]];
}

- (void)setRating: (NSNumber *)newRating fromScriptCommand: (NSScriptCommand *)cmd {
    NSInteger val = [newRating integerValue];
    if (val < 0 || val > 5) {
       [cmd setScriptErrorNumber: TORatingOutOfRangeError];
        //note that AppleScript is never localised
        [cmd setScriptErrorString: @"Rating must be between 0 and 5."];
    }
    else {
        self.rating = newRating;
    }
}

This is the Extract Method refactoring pattern. Now it's possible to test -setRating:fromScriptCommand: without depending on the environment, and be fairly confident that if that works properly, -setScriptRating: should work properly too. I'll do it:

- (void)testAppleScriptGetsErrorsWhenRatingSetTooLow {
    FakeScriptCommand *cmd = [[FakeScriptCommand alloc] init];
    [tune setRating: [NSNumber numberWithInteger: 0]
  fromScriptCommand: cmd];
    STAssertEquals([cmd scriptErrorNumber], TIRatingOutOfRangeError,
                   @"Should get a rating out of range error");
    STAssertNotNil([cmd scriptErrorString], @"Should report error to the user");
}

Oh-oh, my test failed. Why's that? I'm sure you've spotted it now: the user is supposed to be restricted to values between 1 and 5, but the method is actually testing for the range 0 to 5. I could fix that quickly by changing the value in the test, but I'll take this opportunity to remove the magic numbers from the code too (though you'll notice I haven't done that with the error string yet, I still need to do some more refactoring):

- (void)setRating: (NSNumber *)newRating fromScriptCommand: (NSScriptCommand *)command {
    if (![self validateValue: &newRating forKey: @"rating" error: NULL]) {
        [command setScriptErrorNumber: TIRatingOutOfRangeError];
        //note that AppleScript is never localised
        [command setScriptErrorString: @"Rating must be a number between 1 and 5."];
    }
    else {
        self.rating = newRating;
    }
}

OK, so now my test passes. Great. But that's not what's important: what's important is that I found and fixed a bug by thinking about how my code is used. And that's what unit testing is for. Some people will say they get a bad taste in their mouths when they see that I redesigned the method just so that it could be used in a test framework. To those people I will repeat the following: I found and fixed a bug.

#### An Imagined History of Agile Software Development
Having benefited from the imagined history of Object-Oriented Programming, it’s time to turn our flawed retelling toolset to Agile. This history is as inaccurate and biased as it is illuminating.

In the beginning, there was no software. This was considered to be a crisis, because there had been computers since at least the 1940s but nearly half a century later nobody had written any software that could run on them. It had all been cancelled due to cost overruns, schedule overruns, poor quality, or all three.

This was embarrassing for Emperor Carhoare I, whose mighty imperial domain was known as Software Engineering. What was most embarrassing was that at every time when the crisis came to a head, a hologram of Dijkstra would appear and repeat some pre-recorded variation of “I told you so, nobody is clever enough to write any software”.

In frustration, software managers marched their developers off of waterfalls to their doom. If you ever see a software product with a copyright date before, say, 2001, it is a work of fiction, placed by The Great Cunningham to test our faith. I know this because a very expensive consultant told me that it is so.

Eventually the situation got so bad that some people decided to do something about it. They went on a skiing holiday, which was preferable to the original suggestion that they do something about this software problem. But eventually they ended up talking about software anyway, and it turned out that two of them actually had managed to get a software project going. Their approach was extreme: they wrote the software instead of producing interim reports about how little software had been written.

With nothing else to show than a few photographs of mountains, the rest of the skiing group wrote up a little document saying that maybe everybody else writing software might want to try just writing the software, instead of writing reports about how little software had been written. This was explosive. People just couldn’t believe that the solution to writing software was this easy. It must be a trick. They turned to the Dijkstra projection for guidance, but it never manifested again. Maybe he had failed to foresee this crisis? Maybe The Blessed Cunningham was a mule who existed outside psychohistory?

There were two major problems with this “just fucking do it” approach to writing software. The first problem was that it left no breathing room for managers to commission, schedule, and ignore reports on how little software was getting written. Some people got together and wrote the Project Managers’ Declaration of Interdependence. This document uses a lot of words to say “we are totally cool and we get this whole Agile thing you’re talking about, and if you let us onto your projects to commission status reports and track deliverables we’ll definitely be able to pay our bills”.

The second problem, related to the first, is that there wasn’t anything to sell. How can you Agile up your software tools if the point is that tools aren’t as important as you thought? How can you sell books on how important this little nuance at the edge of Agile is, if the whole idea fits on a postcard?

Enter certification. We care more about the people than the process, and if you pay for our training and our CPDs you can prove to everybody that you've understood the process for not caring about process. Obviously this is training and certification for the aforementioned co-dependent—sorry, interdependent—project managers. There is certification for developers, but this stuff works best if they're not actually organised so you won't find many developers with the certification. Way better to let them divide themselves over which language is best, or which text editor, or which blank space character.

And...that's it. We're up to date. No particularly fresh theoretical insight in two decades, we just have a lot of developers treated as fungible velocity sources on projects managed top-down to indirect metrics and reports. Seems like we could benefit from some agility.

#### An Imagined History of Object-Oriented Programming
Having looked at hopefully modern views on Object-Oriented analysis and design, it’s time to look at what happened to Object-Oriented Programming. This is an opinionated, ideologically-motivated history, that in no way reflects reality: a real history of OOP would require time and skills that I lack, and would undoubtedly be almost as inaccurate. But in telling this version we get to learn more about the subject, and almost certainly more about the author too.
They always say that history is written by the victors, and it’s hard to argue that OOP was anything other than victorious. When people explain about how they prefer to write functional programs because it helps them to “reason about” code, all the reasoning that was done about the code on which they depend was done in terms of objects. The ramda or lodash or Elm-using functional programmer writes functions in JavaScript on an engine written in C++. Swift Combine uses a functional pipeline to glue UIKit objects to Core Data objects, all in an Objective-C IDE and - again - a C++ compiler.

Maybe there’s something in that. Maybe the functional thing works well if you’re transforming data from one system to another, and our current phase of VC-backed disruption needs that. Perhaps we’re at the expansion phase, applying the existing systems to broader domains, and a later consolidation or contraction phase will demand yet another paradigm.
Anyway, Object-Oriented Programming famously (and incorrectly, remember) grew out of the first phase of functional programming: the one that arose when it wasn’t even clear whether computers existed, or if they did whether they could be made fast enough or complex enough to evaluate a function. Smalltalk may have borrowed a few ideas from Simula, but it spoke with a distinct Lisp.

We’ll fast forward through that interesting bit of history when all the research happened, to that boring bit where all the development happened. The Xerox team famously diluted the purity of their vision in the hot-air balloon issue of Byte magazine, and a whole complex of consultants, trainers and methodologists jumped in to turn a system learnable by children into one that couldn’t be mastered by professional programmers.
Actually, that’s not fair: the system already couldn’t be mastered by professional programmers, a breed famous for assuming that they are correct and that any evidence to the contrary is flawed. It was designed to be learnable by children, not by those who think they already know better.

The result was the ramda/lodash/Elm/Clojure/F# of OOP: tools that let you tell your local user group that you’ve adopted this whole Objects thing without, y’know, having to do that. Languages called Object-*, Object*, Objective-*, O* added keywords like “class” to existing programming languages so you could carry on writing software as you already had been, but maybe change the word you used to declare modules.

Eventually, the jig was up, and people cottoned on to the observation that Object-Intercal is just Intercal no matter how you spell come.from(). So the next step was to change the naming scheme to make it a little more opaque. C++ is just C with Classes. So is Java, so is C#. Visual BASIC.net is little better.

Meanwhile, some people who had been using Smalltalk and getting on well with fast development of prototypes that they could edit while running into a deployable system had an idea. Why not tell everyone else how great it is to develop prototypes fast and edit them while running into the deployable system? The full story of that will have to wait for the Imagined History of Agile, but the TL;DR is that whatever they said, everybody heard “carry on doing what we’re already doing but plus Jira”.

Well, that’s what they heard about the practices. What they heard about the principles was “principles? We don’t need no stinking principles, that sounds like Big Thinking Up Front urgh” so decided to stop thinking about anything as long as the next two weeks of work would be paid for. Yes, iterative, incremental programming introduced the idea of a project the same length as the gap between pay checks, thus paving the way for fire and rehire.

And thus we arrive at the ideological void of today’s computering. A phase in which what you don’t do is more important than what you do: #NoEstimates, #NoSQL, #NoProject, #NoManagers…#NoAdvances.

Something will fill that void. It won’t be soon. Functional programming is a loose collection of academic ideas with negation principles - #NoSideEffects and #NoMutableState - but doesn’t encourage anything new. As I said earlier, it may be that we don’t need anything new at the moment: there’s enough money in applying the old things to new businesses and funnelling more money to the cloud providers.

But presumably that will end soon. The promised and perpetually interrupted parallel computing paradigm we were guaranteed upon the death of Moore’s Law in the (1990s, 2000s, 2010s) will eventually meet the observation that every object larger than a grain of rice has a decent ARM CPU in, leading to a revolution in distributed consensus computing. Watch out for the blockchain folks saying that means they were right all along: in a very specific way, they were.

Or maybe the impressive capability but limited applicability of AI will meet the limited capability but impressive applicability of intentional programming in some hybrid paradigm. Or maybe if we wait long enough, quantum computing will bring both its new ideas and some reason to use them.

But that’s the imagined future of programming, and we’re not in that article yet.

#### An observation designed to aid the reading of books on software
Wherever a book on writing software describes the 1968 NATO conference in Garmisch on Software Engineering, consider whether the clarity of the argument can be improved by adding the following parenthetical clause:

[…], a straw man version of an otherwise real conference that took place in 1968, […]

Usually it can. The proceedings of the conference, which were written post facto by the editors and typists locking themselves in a hotel room with tapes of the sessions and typewriters in various states of repair, are available at Brian Randell's website along with reflections on their creation. Does the report actually contain the fact presented in whichever book you're reading now?

Probably not. The article “Crisis, What Crisis?” Reconsidering the Software Crisis of the 1960s and the Origins of Software Engineering investigates the position of the 1968 report in the rhetoric of the software industry and reliance by secondary authors on its content. The conclusion is that the report was largely ignored for about a decade, when it suddenly became the thing that kickstarted the software crisis and software engineering.

It would only be a little satirical to say "the software crisis was invented circa 1980 by Edsger Dijkstra, who postulated its origins in the NATO conference of 1968, a straw man conference" etc.

#### An odd thought
An odd thought: I have written software for a computer whose CPU was used as an I/O controller for a computer that I have programmed, whose CPU was used as an I/O controller for a computer that I have programmed. So far, I'm not aware of the i7 being used as an I/O controller.

#### An open letter to Xcode
The post below has been filed verbatim as an Apple Developer Tools bug report with ID 13051064.

Dear Xcode,

imagine that you had a combine harvester. Only, this combine harvester, instead of having a hopper into which the winnowed wheat is poured, has a big quern stone. It grinds the wheat into a flour, which is poured into a big mixer with some water and a bit of yeast. The mixture from here then finds its way into a big oven.

While all of this is going on, the other side of the combine harvester is actually a platform hosting some cows and a milking machine. The gathered milk is churned by the same device that turns the quern for grinding the flour.

As you have no doubt concluded yourself, Xcode, such a machine would be a great help in producing bread and butter. But let me tell you to what this parable alludes: Apple's bread and butter is its systems of electronic devices, first and third party software. You, Xcode, are the souped-up combine harvester; the thing that makes it possible to rapidly turn the ingredients of Apple's ecosystem into the products that everyone desires. You free both Apple's people and the people who make things with Apple's things from the drudgery of threshing and winnowing, and let them concentrate on what they want to make and how people would want to interact with those things. Do some people want artisanal loaves while others want simple pre-sliced packages? Xcode, you help people help both of these people.

Here's what I think. I think you know this, Xcode. I think you, and the product people, and the developers who make you, all know that you are a key piece of machinery in producing Apple's bread and butter. I worry though that other people at Apple, particularly some of the managers, do not see this. I think that they see you as a tool for internal use and a small number of external customers; a group that is not the primary focus for Apple's products. These same people would see the combine harvester not as the greatest labour-saving device of the twentieth century, but as a niche instrument only of interest to combine harvester operators.

I respect you, Xcode. You may not know that, because I joke about you a lot on Twitter. What you have to understand is that for an English man to make a joke about one of his friends, it means that he really respects and has affection for that friend. It's that respect I have for you, and for the people that make you, that means I think I can tell you what follows and you'll take it in the intended spirit: as advice, not as an insult.

Xcode, I need you to do a couple of things. One of them you'll like, one of them you won't. I'll start with the one you'll like: I'm not a fan of the shit sandwich mode of delivering criticism. So here goes. Xcode, I need you and the people who make you to go around the campus at Apple and tell everyone you meet about the combine harvester story. Particularly senior management. Let everyone see that you are not some toy app used by a few edge-case and highly demanding users, but are in fact a critical component in the machinery that makes iPhones what they are, that makes iPads what they are, that makes Macs what they are. Remind the people who're focused on iBooks Author as a key part of Apple's education strategy that you help to make iBooks Author. Remind the people who want to build even better iOS releases that you are helping to build those releases. That when someone says "there's an app for that", it's because of you. Help everyone to realise that the better you are, Xcode, the better nearly everything else that Apple does will become.

You'll find it easier to convince some people than others, I'm sure. I expect that Craig Federighi has strong memories of using you when you were still Project Builder. I expect that Tim Cook may never have launched or even installed you. I don't promise that telling the combine harvester story will be easy, but I do promise that it will boost your esteem, and that of the people who work on you.

So that was the one I think you'll like, Xcode. Here is the other one; the one I doubt you'll like so much. I mentioned Project Builder in the last paragraph: Xcode, I'm sorry to have to tell you that you are no longer Project Builder. How do I mean? I'm certainly not talking about your outward appearance: Project Builder was a quirky adolescent who couldn't do anywhere near as much as you can. What I'm saying is that you're no longer the cool startup whose goal is to change the world of developer tools. We've tried a whole bunch of different machinery and we've settled on the souped-up combine harvester. What we need you to be is a better souped-up combine harvester.

That's not to say that innovation in developer tools should die completely. There will be new Project Builders. Someone will invent a new way of building software that's completely out of left field, and plenty of people will find that new way better than the current way. That'll be really cool. Maybe Apple will buy that company, or license their technology, so that you can have a go with it. That would also be cool.

What I'm saying, Xcode, is that you're mature and grown up and people respect you for that. Please stop having these mid-life crises like you did at version 4 where you suddenly change how everything is done. Your work now is in incremental enhancement, not in world-changing revolution. People both at Apple and outside have come to expect you to be dependable, reliable and comfortable. You may think that's boring. It's not! Remember all of those things that exist because of you, all of those people who are delighted by what you have helped create. Just bear in mind also that when it comes time for Xcode 5, people will want a better Xcode, not a replacement for Xcode.

Think about the apps that are made at Apple now. What could make it a bit faster to make every view? Or make regressions a bit easier to detect and fix? What errors do developers at Apple see, with what frequency? How could you reduce those errors or make them quicker to diagnose? There's an old story that Steve Jobs wanted the boot time of the Macintosh to be as fast as possible, and he thought about it in terms of the number of lifetimes that would be wasted staring at the boot screen. You may now be thinking about the number of lifetimes spent writing code, but I want you to think bigger than that: think about the exponentially larger number of lifetimes being spent waiting for those apps to ship. That extra month where 100 million people waited for the new iTunes; could a better Xcode have cut that time down?

Listen, Xcode, this is going to sound weird. I mean, you barely know me, but I'm talking like we're best friends and I'm holding some kind of intervention. But here's how I want you to see it, and it's based on the combine harvester story. I don't know whether you have a bonus or incentive scheme at Apple, but if you do then ask them to make this change. Xcode, your bonus should not be based on shipping Xcode. That would be like paying a combine harvester for harvesting; it completely misses the point. The point of harvesting is to make things like bread. Your bonus should be based on shipping every other software product Apple makes. Maybe even the third-party apps, if you can work out a fair way to measure that.

With more sincerity than this blog usually evinces,

Graham.

#### An unhelpful distinction
Object-Oriented Programming is quite simple: it's just choosing what function to run based on the parameters to the function (whether through method sending like Smalltalk, polymorphic lookup like CLOS, or table searching like C++: usually pattern-matching like Haskell would be excluded here).

Object-Oriented Analysis and Design is the thing where we represent our problem domain, and our solution, as a collection of objects, often categorised into classes, where the classes have particular relationships, properties and behaviours. And that is the thing that programmers often struggle with.

Whether it's hard because OOA/D is hard, or because the problem domains are hard, or because OOA/D is not applicable to the problem domains, is not addressed here.

#### An update on the HURD project
Last time, on Structure and Interpretation of Computer Programmers, I was building an object-oriented programming system on top of the HURD, and had realised that I needed to use its trivfs library for a sender to be able to discover an object to send messages to.

I got it working very quickly, but ended up shaving a yak due to my poor understanding of the HURD translator lifecycle which meant that I didn't think I had got it working.

My goal was to build a translator that works like the Objective-C nil object: it accepts any message and responds by returning itself. Given that I'm building on the Mach ports abstraction, "itself" is defined as a port over which you can send messages to the same object.

If I returned an integer from the message, everything worked on both sides. However, just because a port name is an integer doesn't mean that sending it as an int will work, just as opening a file in one process then sending the file descriptor as a number to another process wouldn't let the receiving process access the file. I tried sending it as a mach_port_t, but got a lifecycle error: the client was told that the server had died.

On doing some reading, I discovered that the port had to be sent as a mach_port_send_t for send rights to be transferred to the client. Making that change, the message now fails with a type error.

An aside, here, on getting help with this problem. There is good documentation: the HURD source is easy to read and with helpful comments, they have good documentation including examples, the OSF documentation is very helpful, there are books from "back in the day" and videos with useful insights.

On the other hand, "help" is hard to come by. I eventually answered my own stack overflow question on the topic, having not received a reply on there, the HURD mailing list or their IRC channel. The videos described above come from FOSDEM and I'm heading out there next week, I'll try to make some contacts in person and join their community that way.

OK, so back to the main issue, I now have a fix for my problem. Well, sort of, because now that I'm correctly sending a port with a send right I'm back to getting the lifecycle error.

My current plan is not to "fix" that, but to take it as a hint that I'm doing it wrong, and to design my system differently. Using the filesystem as a namespace to look up objects is good, but using the thing I receive as the object to message is a separate responsibility. I'm changing my approach so that the filesystem contains constructors, and they return not their own port but a port to something else that represents the instance they created.

#### Anatomy of a software sales scam
A couple of days ago, Daniel Kennett of the KennettNet micro-ISV (in plain talk, a one-man software business) told me that a customer had fallen victim to a scam. She had purchased a copy of his application Music Rescue&mdash;a very popular utility for working with iPods&mdash;from a vendor but had not received a download or a licence. The vendor had charged her over three times the actual cost of the application, and seemingly disappeared; she approached KennettNet to try and get the licence she had paid for.

Talking to the developer and (via him) the victim, the story becomes more disturbing. The tale all starts when the victim was having trouble with her iPod and iTunes, and contacted Apple support. The support person apparently gave her an address from which to buy Music Rescue, which turned out to be the scammer's address. Now it's hard to know what she meant by that, perhaps the support person gave her a URL, or perhaps she was told a search term and accessed the malicious website via a search engine. It would be inappropriate to try and gauge the Apple support staff's involvement without further details, except to say that the employee clearly didn't direct her unambiguously to the real vendor's website. For all we know, the "Apple" staffer she spoke to may not have been from Apple at all, and the scam may have started with a fake Apple support experience. It does seem more likely that she was talking to the real Apple and their staffer, in good faith or otherwise, gave her information that led to the fraudulent website.

The problem is that if you ask a security consultant for a solution to the problem of being scammed in online purchases, they will probably say "only buy software from trusted sources". Well, this user clearly trusted Apple as a source, and clearly trusted that she was talking to Apple. She probably was, but still ended up the victim of a scam. Where does this leave the advice, and how can a micro-ISV ever sell software if we're only to go to stores who've built up a reputation with us?

Interestingly the app store model, used on the iPhone and iPad, could offer a solution to these problems. By installing Apple as a gateway to app purchases, customers know (assuming they've got the correct app store) that they're talking to Apple, that any purchase is backed by a real application and that Apple have gone to some (unknown) effort to ensure that the application sold is the same one the marketing page on the store claims will be provided. Such a model could prove a convenient and safe way for users to buy applications on other platforms, even were it not to be the exclusive entry point to the platform.

As a final note, I believe that KennettNet has taken appropriate steps to resolve the problem. As close as I can tell the scam website is operated out of the US, making any attempted legal action hard to pursue as the developer is based in the UK. Instead the micro-ISV has offered the victim a discounted licence and assistance in recovering the lost money from her credit card provider's anti-fraud process. I'd like to thank Daniel for his information and help in preparing this article.

#### Android: the missed opportunities
There are a few Android devices I have respect for: the Amazon Kindle Fire is one, the B&N Nook another, and the Cisco Cius is the third. To a lesser extent, the Sony tablet also fits this category.

I don't tend to like the Galaxy Tab, the Xoom and similar tablets so much, and most of the phones are lacklustre too. So what's the difference? Why should some of these products be good and some bad, when they're all just Android touch screens?

Android is a technology, not a product.

A term that appears in Ars Technica's review of Ice Cream Sandwich really brings home my argument:

Ice Cream Sandwich is the closest to a "finished" version of Android I've ever seen.

Reasoning by inference we can conclude that all versions of Android appear unfinished. And this is the nub of the issue. The products I've listed as respectable above&mdash;the Kindle Fire, Cius and so on&mdash;are not Android tablets. They are eBook readers, business communication tablets, media controllers. The ones I don't like are Android tablets.

Using Android in a touchscreen device is an extension of using Linux in any other computing scenario: I might like the solution, but I don't want to see your working. I own countless Linux boxes: a couple of network routers, some eInk readers, a Drobo, that sort of thing. They all have in common that they take Linux and then build something on top of it that solves a problem I have. While all of these devices are successful products that all use Linux, Linux itself as a standalone thing is not successful among the same people who buy Linux routers, Linux NAS and so on.

Indeed even Linux desktop OS distributions have failed to take over the world, but don't they solve the problem of "I want a desktop OS"? There are two issues here: the first being that no-one has that problem. People have problems like "I want to e-mail my aunt" or "I need to write a report on deforestation in the Amazon", and it turns out that PCs can support those people, and that a desktop OS can be part of the supporting solution. The second issue is that Linux distributions have traditionally given you all the bits of a desktop OS without helping you put them together: you wanted a sports car, well here's enough Meccano that you can build one.

Colour in up to the lines

Let's return to the issue of why the not-so-good Android products are not so good. They are to smartphones or tablets as Linux distros are to PCs: some assembly is required to get a functioning product. By and large, these things give you Android itself, a slathering of "differentiating" interface tweaks and access to the Android Market. Where you go from there is up to you.

Of course, fixing the user interface is a necessary (dear God is it necessary) part of producing a complete Android product, but it's far from sufficient. Where the respectable products like the Kindle Fire work is that a problem has been identified (for example: I want to be able to read a wide range of books without having to carry a whole library of paper around) and they solve that problem. The engineers use Android to help arrive at that solution, but they colour in all the way up to the lines: they don't just give you the line drawing and let you buy crayons from the marketplace.

So where are the opportunities?

Well, where are the problems? What is there that can't be done, and how would you design a product to do it? If the answer involves a touchscreen, then there are a couple of ways to do it: either with an app running on top of a touchscreen platform, or with a touchscreen device. Which is appropriate to the problem you're looking at?

It seems that in most cases, the app is the better approach. It's cheaper to engineer and distribute, and allows customers to take advantage of their existing touchscreen products and potentially integrate your app into workflows involving their other applications.

One situation in which an app is not a sufficient solution is where you need hardware capabilities that don't exist in off-the-shelf touchscreen kit: maybe ambient light sensors, motion sensors, thermometers, robotic arms and the like. In that case it might still be possible to adopt an existing platform to support the software side, and provide a peripheral using USB or Apple's 30-pin connector for the custom hardware requirements. That could work, but it might not be the best experience, so perhaps a complete integrated hardware+software product is the better option.

A place where you will definitely need to consider building a complete hardware+software product is where the existing software platforms don't provide the capabilities you need. For example, you require the ability to share data between components in a way that isn't supported by apps on phones. Of course, the hardware could still be largely off-the-shelf running a custom software platform.

In either of these last two scenarios, building something custom on top of Android is a good solution. Giving your customers Android with the app on top is not a good solution.

These scenarios represent the missed opportunities. Start-ups could be taking Android and using it as the core of some great integrated products: not as the "app for that" but as the "Kindle for that". A single device that solves the bejeesus out of a particular problem.

Worked example: there's a Kindle for cars.

Modern cars suffer from a lack of convergence. There's a whole bunch of electronics that control the car itself, or provide telemetry, and feed back information to either the dashboard or a special connector accessible to service technicians. There's the sat nav, and then they often also have a separate entertainment system (and possibly even a dock for a separate media player). Then there's the immobiliser, which may be remotely controllable by something like Viper smart start.

A company decides that they want to exploit this opportunity for an integrated car management system. The central part of the user experience would be a removable touch-screen display that allows drivers to plan routes and mark waypoints when they're out of the car. They can subscribe to feeds from other organisations that tell them about interesting places, which can all be mashed up in the navigation display. For example, a driver who subscribes to English Heritage in the navigation system can ask where heritage sites within one hour's drive are. Having seen pictures and descriptions of the sites, she could request a route from the current location to Stonehenge, where the return journey should pass by a service station (locations of which are supplied by her Welcome Break subscription).

The display also integrates with the car's locking system, so the doors automatically lock when the display is taken outside the car. It must be connected to the car's power/data port to enable the engine to start. While the car is in motion the display provides navigational information, and allows control of the entertainment system. Where notifications are sufficiently important, for example warnings from the car's diagnostic system, they are announced over the display and the car's speakers. Where they're unimportant they're left in the background to be reviewed once the journey is finished.

In this situation, we need both custom hardware (the interfaces to the immobiliser and the car telemetry, which for a good user experience should be a single connector that also provides power and sound output to the car speakers) and a custom software experience (the route-planning part, integrating with feeds provided by the third parties which are also available in the viewer "apps", and the variable-severity notification system that off-the-shelf touchscreen platforms don't provide).

Getting this product to market quickly would be benefitted by taking Android and customising it to fit the usage scenario, so that the core parts of the touchscreen software platform don't need to be written from scratch. However providing a vanilla tablet experience with the various components implemented as apps would not make a good product.

Conclusion

Android can be a great part of a whole range of different touchscreen products. However it is not, in itself, a product.

#### Announcing AppScript
Announcing AppScript: an interpreted Objective-C subset with no pointers or primitive C types. We finally got Objective-C without the C.

#### Announcing Chiron Codex, a community of software centaurs
Software engineers don't need to outsource our agency to coding agents. We don't need to give up reading the code, or understanding the problems. We can use AI tools to augment our own capabilities, to improve our engineering knowledge and skills. To become software centaurs.

Chiron Codex is an initiative to do just that. In the short term, I'm creating a pattern language of AI-augmented software engineering, and a community of people who want to use AI to become better software engineers at Patreon and at YouTube. Longer term, we'll explore ways to improve at all aspects of the software engineering lifecycle; becoming software generalists who use AI to complement our expertise, and our expertise to direct the AI tools.

Join us, and please consider supporting Chiron Codex by subscribing to the Archaeopteryx (super-early bird; so early birds haven't even evolved) tier on Patreon! Here's a video explaining the benefits.

#### Another non-year of Desktop Linux
Let’s look at other software on the desktop, to understand why there isn’t (as a broad, popular platform) Linux on the desktop, then how there could be.

Over on De Programmatica Ipsum I discussed the difference between the platform business model, and the technology platform. In the platform model, the business acts as a matchmaking agent, connecting customers to vendors. An agricultural market is a platform, where stud farmers can meet dairy farmers to sell cattle, for example.

Meanwhile, when a technology platform is created by a business, it enables a two-sided business model. The (technology) platform vendor sells their APIs to developers as a way of making applications. They sell their technology to consumers with the fringe benefit that these third-party applications are available to augment the experience. The part of the business that is truly a platform model is the App Store, but those came late as an effort to capture a share of the (existing) developer-consumer sales revenue, and don’t really make the vendors platform businesses.

In fact, I’m going to drop the word platform now, as it has these two different meanings. I’ll say “store” or “App Store” when I’m talking about a platform business in software, and “stack” or “software stack” when I’m talking about a platform technology model.

Stack vendors have previously been very protective of their stack, trying to fend off alternative technologies that allow consumers to take their business elsewhere. Microsoft famously “poisoned” Java, an early and capable cross-platform application API, by bundling their own runtime that made Java applications deliberately run poorly. Apple famously added a clause to their store rules that forbade any applications made using off-stack technology.

Both of these situations are now in the past: Microsoft have even embraced some cross-platform technology options, making heavy use of Electron in their own applications and even integrating the Chromium rendering engine into their own browser to increase compatibility with cross-platform technology and reduce the cost of supporting those websites and applications made with Javascript. Apple have abandoned that “only” clause in their rules, replacing it with a collection of “but also” rules: yes you can make your applications out of whatever you want, but they have to support sign-in and payment mechanisms unique to their stack. So a cross-stack app is de jure better integrated in Apple’s sandpit.

These actions show us how these stack vendors expect people to switch stacks: they find a compelling application, they use it, they discover that this application works better or is better integrated on another stack, and so they change to it. If you’re worried about that, then you block those applications so that your customers can’t discover them. If you’re not worried about that, then you allow the technologies, and rely on the fact that applications are commodities and nobody is going to find a “killer app” that makes them switch.

Allowing third-party software on your own platform (cross-stack or otherwise) comes with a risk, that people are only buying your technology as an incidental choice to run something else, and that if it disappears from your stack, those customers might go away to somewhere that it is available. Microsoft have pulled that threat out of their briefcase before, settling a legal suit with Apple after suggesting that they would remove Word and Excel from the Mac stack.

That model of switching explains why companies that are otherwise competitors seem willing to support one another by releasing their own applications on each others’ stacks. When Apple and Microsoft are in competition, we’ve already seen that Microsoft’s applications give them leverage over Apple: they also allow Apple customers to be fringe players in the Microsoft sandpit, which may lead them to switch (for example when they see how much easier it is for their Windows-using colleagues to use all of the Microsoft collaboration tools their employers use). But Apple’s applications also give them leverage over Microsoft: the famed “halo effect” of Mac sales being driven by the iPod fits this model: you buy an iPod because it’s cool, and you use iTunes for Windows. Then you see how much better iTunes for Mac works, and your next computer is a Mac. The application is a gateway to the stack.

What has all of this got to do with desktop Linux? Absolutely nothing, and that’s my point. There’s never been a “halo effect” for the Free Software world because there’s never been a nucleus around which that halo can form. The bazaar model does a lot to ensure that. Let’s take a specific example: for many people, Thunderbird is the best email client you can possibly get. It also exists on multiple stacks, so it has the potential to be a “gateway” to desktop Linux.

But it won’t be. The particular bazaar hawkers working on Thunderbird don’t have any particular commitment to the rest of the desktop Linux stack: they’re not necessarily against it, but they’re not necessarily for it either. If there’s an opportunity to make Thunderbird better on Windows, anybody can contribute to exploit that opportunity. At best, Thunderbird on desktop Linux will be as good as Thunderbird anywhere else. Similarly, the people in the Nautilus file manager area of the bazaar have no particular commitment to tighter integration with Thunderbird, because their users might be using GNUMail or Evolution.

At one extreme, the licences of software in the bazaar dissuade switching, too. Let’s say that CUPS, the common UNIX printing subsystem, is the best way to do printing on any platform. Does that mean that, say, Mac users with paper-centric workflows or lifestyles will be motivated to switch to desktop Linux, to get access to CUPS? No, it means Apple will take advantage of the CUPS licence to integrate it into their stack, giving them access to the technology.

The only thing the three big stack vendors seem to agree on when it comes to free software licensing is that the GPL version 3 family of licences is incompatible with their risk appetites, particularly their weaponised patent portfolios. So that points to a way to avoid the second of these problems blocking a desktop Linux “halo effect”. Were there a GPL3 killer app, the stack vendors probably wouldn’t pick it up and integrate it. Of course, with no software patent protection, they’d be able to reimplement it without problem.

But even with that dissuasion, we still find that the app likely wouldn’t be a better experience on a desktop Linux stack than on Mac, or on Windows. There would be no halo, and there would be no switchers. Well, not no switchers, but probably no more switchers.

Am I minimising the efforts of consistency and integration made by the big free software desktop projects, KDE and GNOME? I don’t think so. I’ve used both over the years, and I’ve used other desktop environments for UNIX-like systems (please may we all remember CDE so that we never repeat it). They are good, they are tightly integrated, and thanks to the collaboration on specifications in the Free Desktop Project they’re also largely interoperable. What they aren’t is breakout. Where Thunderbird is a nucleus without a halo, Evolution is a halo without a nucleus: it works well with the other GNOME tools, but it isn’t a lighthouse attracting users from, say, Windows, to ditch the rest of their stack for GNOME on Linux.

Desktop Linux is a really good desktop stack. So is, say, the Mac. You could get on well with either, but unless you’ve got a particular interest in free software, or a particular frustration with Apple, there’s no reason to switch. Many people do not have that interest or frustration.

#### Answer: none of them
A question programmers frequently ask when they're considering career growth or personal learning is "which programming language should I learn next?"

Why would learning another programming language help? If you only know one programming language and it is provided by a single vendor, then learning another will decouple your success from theirs, but that might not be such a common situation. Well, a book like Seven Languages in Seven Weeks makes the point that it's not about learning the language, but about learning the model and thought process enabled by using that language. OK, so why don't I learn that model or thought process, using the tools that are already available to me, instead of having to add fighting unfamiliar syntax to the problem?

And if what I'm truly trying to do is to learn to think about problems in a different way, a week-long effort at dabbling in a side project isn't going to change my way of thinking. Those years of learned processes, visualisations and analyses are going to take more than a couple of hours to dislodge. I've worked through Seven Languages, and the fact that I spent a couple of hours solving the Eight Queens Problem in Prolog (or in fact telling Prolog what a solution to Eight Queens looks like and letting it solve it) doesn't mean I now think about any other software problem as if I'm using a logic programming tool, or even as if I have such a tool available. I've spent much longer than that studying and using the relational calculus and SQL, but don't even think about every problem as if it should be a collection of tables in the third normal form.

It may be that it would be useful to learn something that isn't a programming language, shock horror! It turns out that programming is an activity embedded in a socio-technical system comprising other activities, and you might need to know something about them: software security, testing (I think I can count on my noses the number of programmers I've met who haven't responded to the phrase "equivalence partitioning" with a blank stare, and I wouldn't use all of my noses), planning, business, marketing, ethics...I even wrote a whole book on the things programmers should know that aren't programming.

And then there's the thing that your customers, clients, colleagues, or victims are trying to do with the software. Learning something about that would make it easier to empathise with them, to evaluate your solutions in context, and to propose better ways of working and better ways for your software to enable their work. Rewriting your code in Elixir would...not do that so much.

#### Anti-lock brakes
Chances are, if you bought a new car or even a new motorcycle within the last few years, you didn't even get an option on ABS. It came as standard, and in your car was legally mandated. Anti-lock brakes work by measuring the rotational acceleration of the wheels, or comparing their rotational velocities. If one wheel is rotating very much slower than the others, or suddenly decelerates, it's probably about to lock so the ABS backs off the pressure on the brake for that wheel.

ABS turns everyone into a pretty capable brake operator, in most circumstances. This is great, because many people are not pretty capable at operating brakes, even when they think they are, and ABS makes them better at it. Of course, some people are very capable at it, but ABS levels them too, making them merely pretty capable.

But even a highly capable brake operator can panic, or make mistakes. When that happens, ABS means that the worst effect of their mistake is that they are merely pretty capable.

In some circumstances, having ABS is strictly worse than not having it. An ABS car will take longer to stop on a gravel surface or on snow than a non-ABS car. Car with ABS tend to hit each other much less often than those without, but tend to run off the road more often than those without. But for most vehicles, the ABS is always-on, even in situations where it will get in your way. Bring up that it is getting in your way, and someone will tell you how much safer it is than not having it. Which is true, in the other situations.

Of course the great thing about anti-lock brakes is that the user experience is the same as what most sub-pretty-capable drivers had before. No need to learn a different paradigm or plan your route differently. When you want to stop, press the thing that makes the car stop very hard.

Something, something, programming languages.

#### Anyone Can Write A Manifesto And You Can Too!™
Over a small number of years, I have helped to write some software. During this time I have come to value:

Solving problems over rejecting what has passed before
Solving problems over congratulating ourselves
Solving problems over creating problems
Solving problems over clever rhetorical tricks

That is, while the things on the right are sometimes the means, the thing on the left is always the end.

#### Aphorism Considered Harmful
Recently Dan North asked the origin of the software design aphorism “make it work, make it right, make it fast”. Before delving into that story, it’s important to note that I had already heard this phrase. I don’t know where, it’s one of those things that’s absorbed into the psyche of some software engineers like “goto considered harmful”, “adding people to a late project makes it later” or “premature optimisation is the root of all evil”.

My understanding of the quote was something akin to Greg Detre’s description: we want to build software to do the right thing, then make sure it does it correctly, then optimise.

Make it work.&nbsp;First of all, get it to compile, get it to run, make sure it spits out roughly the right kind of output.

Make it right.&nbsp;It’s time to test it, and make sure it behaves 100% correctly.

Make it fast.&nbsp;Now you can worry about speeding it up (if you need to). […]When you write it down like this, everyone agrees it’s obvious.
Greg Detre, “Make it work, make it right, make it fast”

That isn’t what everybody thinks though, as Greg points out. For example, Henrique Bastos laments that some teams never give themselves the opportunity to “make it fast”. He interprets making it right as being about design, not about correctness.

Just after that, you’d probably&nbsp;discovered&nbsp;what kind of libraries you will use, how the rest of your code base interacts with this new behavior, etc. That’s when you go for&nbsp;refactoring&nbsp;and&nbsp;Make it Right. Now you dry things out and organize your code properly to have a good design and be easily maintainable.
Henrique Bastos, “The make it work, make it right, make it fast misconception”

We already see the problem with these little pithy aphorisms: the truth that they convey is interpreted by the reader. Software engineering is all about turning needs into instructions precise enough that a computer can accurately and reliably perform them, and yet our knowledge is communicated in soundbites that we can’t even agree on at the human level.

It wasn’t hard to find the source of that quote. There was a special issue of Byte magazine on the C programming language in August 1983. In it, Stephen C. Johnson and Brian W. Kernighan describe modelling systems processing tasks in C.

But the strategy is definitely: first make it work, then make it right, and, finally, make it fast.
Johnson and Kernighan, “The C Language and Models for Systems Programming”

This sentence comes at the end of a section on efficiency, which follows a section on “Higher-Level Models” in which the design of programs that use C structures to operate on problem models, rather than bits and words, are described. The efficiency section tells us that higher-level models can make a program less efficient, but that C gives people the tools to get close to the metal to speed up the 5% of the code that’s performance critical. That’s where they lead into this idea that making it fast comes last.

Within context, the “right” that they want us to make appears to be the design/model type of “right”, not the correctness kind of right. This seems to make sense: if the thing is not correct, in what sense are you suggesting that you have already “made it work”?

A second source, contemporary with that Byte article, seems to seal the deal. Butler Lampson’s hints deal with ideas from various different systems, including Unix but also the Xerox PARC systems, Control Data Corporation mainframes, and others. He doesn’t use the phrase we’re looking for but his Figure 1 does have “Does it work?” as a functionality problem, from which follow “Get it right” and “Make it fast” as interface design concerns (with making it fast following on from getting it right). Indeed “Get it right” is a bullet point and cautionary tale at the end of the section on designing simple interfaces and abstractions. Only after that do we get to making it fast, which is contextualised:

Make it fast,&nbsp;rather than general or powerful. If it’s fast, the client can program the function it wants, and another client can program some other function. It is much better to have basic operations executed quickly than more powerful ones that are slower (of course, a fast, powerful operation is best, if you know how to get it). The trouble with slow, powerful operations is that the client who doesn’t want the power pays more for the basic function. Usually it turns out that the powerful operation is not the right one.
Butler W. Lampson, Hints for Computer System Design

So actually it looks like I had the wrong idea all this time: you don’t somehow make working software then correct software then fast software, you make working software and some inputs into that are the abstractions in the interfaces you design and the performance they permit in use. And this isn’t the only aphorism of software engineering that leads us down dark paths. I’ve also already gone into why the “premature optimisation” quote is used in misguided ways, in mature optimisation. Note that the context is that 97% of code doesn’t need optimisation: very similar to the 95% in Johnson and Kernighan!

What about some others? How about the ones that don’t say anything at all? It used to be common in Perl and Cocoa communities to say “simple things simple; complex things possible”. Now the Cocoa folks think that the best way to distinguish value types from identity types is the words struct and class (not, say, value and identity) so maybe it's no longer a goal. Anyway, what’s simple to you may well be complex to me, and what’s complex to you may well be simplifiable but if you stop at making it possible, nobody will get that benefit.

Or the ones where meanings shifted over time? I did a podcast episode on “working software over comprehensive documentation”. It used to be that the comprehensive documentation meant the project collateral: focus on building the thing for your customer, not appeasing the project office with TPS reports. Now it seems to mean any documentation: we don't need comments, the code works!

The value in aphorisms is similar to the value in a pattern language: you can quickly communicate ideas and intents. The cost of aphorisms is similar to the cost in a pattern language: if people don't understand the same meaning behind the terms, then the idea spoken is not the same as the idea received. It's best with a lot of the aphorisms in software that are older than a lot of the people using them to assume that we don't all have the same interpretation, and to share ideas more precisely.

(I try to do this for software engineers and for programmers who want to become software engineers, and I gather all of that work into my fortnightly newsletter. Please do consider signing up!)

#### App security consultancy from your favourite boffin
I'm very excited to soon be joining the ranks of Agant Ltd, working on some great apps with an awesome team of people. I'll be bringing with me my favourite title, Smartphone Security Boffin. Any development team can benefit from a security boffin, but I'm also very excited to be in product development with the people who make some of the best products on the market.

There's another side to all of this: once again can security boffinry be at your disposal. I'll be available on a consultancy basis to help out with your application security and privacy issues. If you're unsure how to do SSL right in your iOS app, are having difficulty getting your Mac software to behave in the App Store sandbox, or need help to identify the security pain points in your application's design or code, I can lend a hand.

Of course, it's not just about technology. The best way to help your developers get security right is to help your developers to help themselves. When I've done security training for developers before I've seen those flashes of enlightenment when delegates realise how the issues relate to their own work; the hasty notes of class or method names to look into back at the desk; the excited discusses in the breaks. Security training for iOS app developers is great for the people, great for the product - and, of course, something I can help out with.

To check availability and book some time (which will be no earlier than July), drop me a line on the twitters or at graham@agant.com.

#### Apple 2, iamleeg 0
So, my few-year-old iPod decided it had had enough, and with pay day having only just passed I thought maybe it would be nice to get a new one.  What's happened today?  Got the new one home, and it won't work at all (searching for "error 1434" isn't particularly useful, either).  However, the one that previously broke, having now been taken apart, started working again.  So my 20GB 4G iPod is now humming along nicely (running Podzilla), and my 160GB classic is b0rked :-(.

#### Apple and Bug Bounties
I know that there are bigger problems to discuss about Apple's approach to business and partnerships at the mo, but their handling of security researchers seems particularly cynical and hypocritical. See, for example, this post about four reported iPhone 0days that went ignored and the nine other cases linked in that article.

Apple advertise themselves as the privacy company. By this, they really mean that their products are designed to share as much of your data with Apple as they are comfortable with, and that beyond that you should probably assume that nobody else is involved. But their security development lifecycle tells another story.

"Wait, did you just pivot from talking about privacy to security?" No! You can have security without privacy: this blog has that, on a first glance. All of the posts and pages are public, anybody can read them, but I want to make sure that what you read is actually what I wrote (the integrity of the posts) and that nothing stops you from reading it when you want (the availability). On a closer examination, I also care that there are things you don't have access to: any of the account passwords, configuration settings, draft posts, etc. So in fact the blog has privacy requirements too, and those are handled in security by considering and protecting the confidentiality of those private assets. You can have security without privacy, but not privacy without security.

Something, I'm not sure what from the outside, is wrong with the security development lifecycle at Apple. As a privacy-focused company they should also be a security-focused company, but they evidently never had the same "trustworthy computing" moment that Microsoft did. I'm not going to do any kind of deep dive into CVE counts here, just provide the following high-level support for the case that Apple is, at best, doing no better than anybody else in the industry at this.

Gil Shwed on the threats on iOS and Android being roughly balanced
Android vulnerabilities command a premium over iOS (2019)
Xcode 13 Server fixes multiple vulnerabilities by updating nginx. I include this to show that Apple are not taking the lead on infosec: they are incorporating vulnerable third-party components and fixes from upstream sources.

Meanwhile, they fail to acknowledge external contributors to their product security, do not pay out agreed bounties, and sue security researchers or ban them from their store. Apple say that the bounty program has doubled in 2019-2020 and continues to grow. You could say that maybe they aren't doing any better, but they certainly aren't doing any worse. Every new product announcement, senior managers at Apple up to their CEO tell everyone how great they are at privacy. Their intent is that people believe they are doing the best at this, when they are around the middle of the pack. This is disingenuous.

A bug bounty program is a security process of last resort. You didn't design, implement, or fix flaws out of your product before it got to customers and attackers: and that happens, that's fine, but these escapee threats that are realised as vulnerabilities should be a small fraction of the total possible problems, and the lower severity ones at that. You also didn't detect it yourself once the customers and attackers had access to the product: that also happens and is fine, but again the vulnerabilities that escape your detection should be the lowest down the stack. Once someone else is discovering your vulnerabilities, the correct thing to do is to say thank you for bringing this to our attention before exploiting it yourself, here is compensation for the time and work you put into making our product better.

Apple is not doing this. As seen from the various linked stories above, they are leaving security researchers with a bitter taste and a questioning feeling over whether they would want to work with Apple again, but not doing the heavy lifting to ensure their SDLC catches the highest-severity problems on campus, before or after release. I don't know what is at fault here, but I expect it's systemic rather than individual leader/department/activity. The product security folks at Apple are good at their jobs, the software engineers are good at their jobs…and yet here we are.

I suspect a certain amount of large-company effect is at play. "As Tim told you, our products are best in class for privacy," says anonymous and fictional somewhat high up marketing person, "and if you had any specific complaint I couldn't hear it over all the high-volume stock cash register sound effects we play in the board room to represent our success in the marketplace."

#### Apple and Google sitting in a tree, f-i-g-h...erm...t-i-ng
This really came out of a throwaway comment I made on Daniel, but it seems popular to pick apart every last iota of Steveness from the WWDC keynote, and I'm nothing if not popular.  So here we go.

What is WebClip?  In fact, that's not really the question I want to be asking.  We know what WebClip is; it's a technology which lets users see only the bits of web pages that those users want to see.  The real question is what does that mean?  Well, I know which bits of a web page I usually want to see; they're the bits which aren't adverts.

I'm going to go out on a bit of a limb, and guess that the way WebClip works (I'm not a WWDC bod so I don't have any more access to the new stuff than anyone else; in fact I haven't even downloaded the Safari 3 beta) is by observing which DOM elements are within the clipped region, and downloading only media relevant to those elements.  If that's the case, then you can ignore the fact that the ads on the page don't get seen; they don't even get downloaded.  Therefore if I'm reading, say, the Dilbert strip in a WebClip, I'm effectively getting free Dilbert, even more free than the free website because I'm not upping their ad impression count.

One thing I noticed about the various sites that Steve clipped is that as far as I can remember, none of them features 'Ads by Google'.  It would be quite embarrassing for Apple's CEO to demonstrate how to reduce revenue for one of Apple's most prominent board members in a world-broadcast keynote talk.  As over 99% of Google's revenue is from online ads, and Eric Schmidt (CEO of Google) is on the Apple board, that is exactly what Steve was showing us, though.  There'll be a doughnut fight back at Infinite Loop over that, I expect.

#### Apple Silicon, Xeon Phi, and Amigas
The new M1 chip in the new Macs has 8-16GB of DRAM on the package, just like many mobile phones or single-board computers. But unlike many desktop, laptop or workstation computers (there are exceptions). In the first tranche of Macs using the chip, that's all the addressable RAM they have (i.e. ignoring caches), just like many mobile phones or single-board computers. But what happens when they move the Apple Silicon chips up the scale, to computers like the iMac or Mac Pro?

It's possible that these models would have a few GB of memory on-package and access to memory modules connected via a conventional controller, for example DDR4 RAM. They almost certainly would if you could deploy multiple M1 (or successor) packages on a single system. Such a Mac would be a non-uniform memory access architecture (NUMA), which (depending on how it's configured) has implications for how software can be designed to best make use of the memory.

NUMA computing is of course not new. If you have a computer with a CPU and a discrete graphics processor, you have a NUMA computer: the GPU has access to RAM that the CPU doesn't, and vice versa. Running GPU code involves copying data from CPU-memory to GPU-memory, doing GPU stuff, then copying the result from GPU-memory to CPU-memory.

A hypothetical NUMA-because-Apple-Silicon Mac would not be like that. The GPU shares access to the integrated RAM with the CPU, a little like an Amiga. The situation on Amiga was that there was "chip RAM" (which both the CPU and graphics and other peripheral chips could access), and "fast RAM" (only available to the CPU). The fast RAM was faster because the CPU didn't have to wait for the coprocessors to use it, whereas they had to take turns accessing the chip RAM. Nonetheless, the CPU had access to all the RAM, and programmers had to tell `AllocMem` whether they wanted to use chip RAM, fast RAM, or didn't care.

A NUMA Mac would not be like that, either. It would share the property that there's a subset of the RAM available for sharing with the GPU, but this memory would be faster than the off-chip memory because of the closer integration and lack of (relatively) long communication bus. Apple has described the integrated RAM as "high bandwidth", which probably means multiple access channels.

A better and more recently analogy to this setup is Intel's discontinued supercomputer chip, Knight's Landing (marketed as Xeon Phi). Like the M1, this chip has 16GB of on-die high bandwidth memory. Like my hypothetical Mac Pro, it can also access external memory modules. Unlike the M1, it has 64 or 72 identical cores rather than 4 big and 4 little cores.

There are three ways to configure a Xeon Phi computer. You can not use any external memory, and the CPU entirely uses its on-package RAM. You can use a cache mode, where the software only "sees" the external memory and the high-bandwidth RAM is used as a cache. Or you can go full NUMA, where programmers have to explicitly request memory in the high-bandwidth region to access it, like with the Amiga allocator.

People rarely go full NUMA. It's hard to work out what split of allocations between the high-bandwidth and regular RAM yields best performance, so people tend to just run with cached mode and hope that's faster than not having any on-package memory at all.

And that makes me think that a Mac would either not go full NUMA, or would not have public API for it. Maybe Apple would let the kernel and some OS processes have exclusive access to the on-package RAM, but even that seems overly complex (particularly where you have more than one M1 in a computer, so you need to specify core affinity for your memory allocations in addition to memory type). My guess is that an early workstation Mac with 16GB of M1 RAM and 64GB of DDR4 RAM would look like it has 64GB of RAM, with the on-package memory used for the GPU and as cache. NUMA APIs, if they come at all, would come later.

#### Apple Technical Briefing: Scientific computing and the Intel transition
[Title link goes to my Oxford homepage, where you can grab a PDF of my slides and notes]So yesterday was the UKUUG/Apple tech briefing on the Intel transition.  Talking to Eric (Albert, Apple Core OS group) and listening to his talk was very interesting, and gave me new and unexpected insights into the whole business of the transition.Essentially, the fact that [[NeXT|OPEN]Step|Rhapsody] was already ported to Intel platform hardly helped at all.  Or maybe that should be &quot;what Apple did to support the Intel port hardly helped at all&quot;, though whether they were right or wrong to use it as they did is not the subject of this post.  The reasons are manifold, but largely come down to the fact that technology's moved on.Porting a UNIX - especially Mach, which has been seen on so many architectures already - to a new platform these days is a trivial exercise, with a small team of engineers who know the platform well you could probably get up to single-user mode in a couple of days, and have a multi-user networked platform up and running in a matter of weeks.  Apple pretty much went back to the starting blocks on the OS X Intel port, because so much of the existing Intel port was obsolete.  Even Darwin/x86 (as was) didn't help here, because although Darwin/x86 and &quot;the Darwin in Intel OS X&quot; target the same architecture, they're different operating systems on different platforms.  None of the existing IOKit drivers are for hardware that's contained in an Intel Mac (in fact, OpenStep/Rhapsody didn't use IOKit at all, let alone the fact that no-one wrote an AirPort Extreme driver for them).  The processors which OpenStep and Tiger target are completely different - for instance, Tiger doesn't hit the floating point unit at all but instead uses the vector engine's floating point support; this effectively means compiling for a different CPU.  Even the bootloader has to be different, to speak with EFI instead of the legacy BIOS.But that's only part of the problem, it would seem that the higher-level APIs have nothing to gain from older Intel ports either.  OK, so OpenStep was designed portable (and largely OS-independent too, but that's another matter) from the ground up, but Cocoa just ain't OpenStep.  Or rather, Cocoa ain't just OpenStep.  As well as supporting new classes, and the bindings/CoreData stuff, implementations of the OpenStep APIs have been rewritten over the years such that the fact that Yellow Box runs on Intel doesn't mean that Cocoa will.  A specific example, and one I particularly liked, which Eric used was that of NSString/CFString, which now (sometimes) uses UTF-16 as its internal data representation.  As does whatever type passes for a text representation in the Carbon world.  This is nice, except that when Apple first implemented the UTF-16 support, it seems that some of the people forgot to check for endianness, and some other of the people deliberately ignored endianness (i.e. most UTF-16 is bigendian anyway, and we're running bigendian cough, let's assume it's all bigendian).  Bring that representation over to the Cocoa/Intel API, and interesting and humourous results can occur.  So you end up having to reimplement NSString anyway.  It actually turns out that Carbon (at least, the Carbon API itself; apps are a different matter) can be easier to deal with than Cocoa, because there you just assume that any data you have control over has historically always been bigendian, and you treat it appropriately.Anyway, it turned out that the amount of time spent on porting Cocoa and porting Carbon, including all the respective applications, was more-or-less the same.  Which I find surprising.

#### Apple's Best Programming Language
My talk at App Builders 2016 was on Apple's best programming language. Spoiler alert: it's Dylan. Or is it?

I chose a few properties one might wish to find in programming languages, then demonstrated how these were all present in the Dylan language. I also took a dig at certain other languages, which do things in ways that could be seen as less good than Dylan's. For example, did you know that there's a programming language out there which distinguishes constant from variable values with the words let and var, rather than using the arguably more readable constant keyword?

Now here's a thing: of course there are many things iOS app programmers could have been using if we didn't want to use Objective-C, without the addition of Swift. Indeed, a few months before Swift was introduced I enumerated some of these alternatives on this very blog. However, many of us chose to use Objective-C rather than any of the alternatives, and then chose Swift when that alternative was presented.

Similarly, Apple could have pursued any of those alternatives, and indeed did pursue quite a few of them. What would the world look like if Apple had invested in MacRuby? It would have Swift in it, we know that because they did and it does.

At the end of my talk, I invited the conference to discuss what it was particularly about Swift that led to its brisk success, when it can be considered equivalent to many existing alternatives in numerous ways. Here are some of the suggestions (none of them from me, all from the audience):

marketing
evangelism
LLVM
Apple now isn't the same as Apple in Dylan's time
"Halo effect" from iOS
people only want to use first-party tools
Objective-C pain provided the opportunity
big enough community to reach critical mass

That leads me to wonder how closely related the conditions for "better" and the conditions for "accepted" are, whether there are "better" things out there for programmers that haven't been adopted, whether those things truly are better, and how aware we all are of the distinction between being better and being popular when we make engineering choices.

#### Apple's Watch and Jony's Compelling Beginning
There are a whole lot of constraints that go into designing something. Here are the few I could think of in a couple of minutes:

what people already understand about their interactions with things
what people will discover about their interactions with things
what people want to do
what people need to do
what people understand their wants and needs
what people understand about their wants and needs
how people want to be perceived by other people
which things that people need or want to do you want to help with
which people you want to help
what is happening around the people who will be doing the things
what you can make
what you can afford
what you're willing to pay
what materials exist

Some of those seem to be more internal than others, but there's more of a continuum between "external" and "internal" than a switch. In fact "us" and "them" are really part of the same system, so it's probably better to divide them into constraints focussing on people and constraints focussing on industry, process and company politics.

Each of Apple's new device categories moves the designs further from limitations of the internal constraints toward the people-centric limitations. Of course they're not alone in choosing the problems they solve in the industry, but their path is a particular example to consider. They're not exclusively considering the people-focussed constraints with the watch, there still are clear manufacturing/process constraints: battery life, radio efficiency are obvious examples.

There are conflicts between some of the people-focussed constraints. You might have a good idea for how a watch UI should work, but it has to be tempered by what people will expect to do which makes new user interface designs an evolutionary process. So you have to take people from what they know to what they can now do.

That's a slow game, that Apple appear to have been playing very quickly of late.

1984: WIMP GUI, but don't worry there's still a typewriter too.

There's a big gap here, in which the technical constraints made the world adapt to the computer, rather  than the computer adapt to the world. Compare desks from the 1980s, 1990s and 2000s, and indeed the coming and going of the diskette box and the mousemat.

2007: touchscreen, but we've made things look like the old WIMP GUI from the 1980s a bit, and there's still a bit of a virtual typewriter thing going on.
2010: maybe this whole touchscreen thing can be used back where we were previously using the WIMP thing.
2013: we can make this touchscreen thing better if we remove some bits that were left over from the WIMP thing.
2014: we need to do a new thing to make the watch work, but there's a load of stuff you'll recognise from (i) watches, (ii) the touchscreen thing.

Now that particular path through the tangle of design constraints is far from unique. Compare the iPad to the DynaBook and you'll find that Alan Kay solved many of the same problems, but only for people who are willing to overlook the fact that what he proposed couldn't be built. Compare the iPhone to the pocket calculator, and you find that it was possible to portable computing many decades earlier but with reduced functionality. Apple's products are somewhere in between these two extremes: balancing what can be done now and what could possibly be desired.

For me, the "compelling beginning" is a point along Apple's (partly deliberate, and partly accidental) continuum, rather than a particular watershed. They're at a point where they can introduce products that are sufficiently removed from computeriness that people are even willing to discuss them as fashion objects. Yes, it's still evidently the same grey-and-black glass square that the last few years of devices have been. Yes, it's still got a shrunk-down springboard list of apps like the earlier devices did.

The Apple Watch (and contemporary equivalents) are not amazing because the bear dances well, they're amazing because the bear dances at all. The possibility of thinking about a computer as an aesthetic object, one that solves your problems and expresses your identity, rather than a box that does computer things and comes in a small range of colours, is new. The ability to consider a computer more as an object in its environment than as a collection of technical and political constraints changes how they interact with us and us with them. That is why it's compelling.

And of course the current watch borrows cues from the phone that came before it, to increase familiarity. Future ones, and other things that come after it, will be able to jettison those affordances as expectations and comfort change. That is why it's a beginning.

#### AppleScript, for once
AppleScript isn't something I write much about, in fact this is the first post I've ever created on the topic. But AppleScript, like the Services menu and Automator, provides that most useful of usability enhancements: the ability to use multiple applications together without fulfilling the marketing requirements of having to look at them all.As an example, a folder action script might let me combine the Finder with any other application, such as, choosing completely at random, Sophos Anti-Virus:on adding folder items to this_folder after receiving these_items&nbsp;&nbsp;set theScript to "/usr/bin/sweep -nc"&nbsp;&nbsp;repeat with i from 1 to number of items in these_items&nbsp;&nbsp;&nbsp;&nbsp;set thePath to POSIX path of item i of these_items&nbsp;&nbsp;&nbsp;&nbsp;set theScript to theScript &amp; space &amp; thePath&nbsp;&nbsp;end repeat&nbsp;&nbsp;set theScript to theScript &amp; space &amp; "--quarantine:mode=000"&nbsp;&nbsp;do shell script theScriptend adding folder items tothat script then scans any file which appears in a particular folder and locks it if it contains a virus (up to a point). But that's not really the point, the point is that I haven't actually had to use any of the target apps in order to get this combined functionality. It's like I was able to summon the Megazord without having to actually talk to the individual Power Rangers. Erm, or something. And that, really, is how a computer should work; I didn't buy OmniFocus so that I could look at its icon, or a splash screen, I bought it because it can manage my lists of things to do. And I got iCal in order to manage events in time. If I have things to do at specific times, then I ought to be able to combine the two, and the computer can do the work involved. After all, that is why I bought the computer.

#### Applications and Spelling of Boole
While Alan Turing is regarded by many as the grandfather of Artificial Intelligence, George Boole should be entitled to some claim to that epithet too. His Investigation of the Laws of Thought is nothing other than a systematisation of "those universal laws of thought which are the basis of all reasoning". The regularisation of logic and probability into an algebraic form renders them amenable to the sort of computing that Turing was later to show could be just as well performed mechanically or electronically as with pencil and paper.

But when did people start talking about the logic of binary operations in computers as being due to Boole? Turing appears never to have mentioned his name: although he certainly did talk about the benefits of implementing a computer's memory as a collection of 0s and 1s, and describe operations thereon, he did not call them Boolean or reference Boole.

In the ACM digital library, Symbolic synthesis of digital computers from 1952 is the earliest use of the word "Boolean". Irving S. Reed describes a computer as "a Boolean machine" and "an automatic operational filing system" in its abstract. He cites his own technical report from 1951:
Equations (1.33) and (1.35) show that the simple Boolean system, given in (1.34) may be analysed physically by a machine consisting of N clocked flip flops for the dependent variables and suitable physical devices for producing the sum and product of the various variables. Such a machine will be called the simple Boolean machine.

The best examples of simple Boolean machines known to this author are the Maddidas and (or) universal computers being built or considered by Computer Research Corporation, Northrop Aircraft Inc, Hughes Aircraft, Cal. Tech., and others. It is this author's belief that all the electronic and digital relay computers in existence today may be interpreted as simple Boolean machines if the various elements of these machines are regarded in an appropriate manner, but this has yet to be proved.
So at least in the USA, the correlation between digital computing and Boolean logic was being explored almost as soon as the computer was invented. Though not universally: the book "The Origins of Digital Computers" edited by Brian Randell, with articles from Charles Babbage, Grace Hopper, John Mauchly, and others, doesn't mention Boole at all. Neither does Von Neumann's famous "first draft" report on the EDVAC.

So, second question. Why do programmers spell Boole bool? Who first decided that five characters was too many, and that four was just right?

Some early programming languages, like Lisp, don't have a logical data type at all. Lisp uses the empty list to mean "false" and anything else to mean true. Snobol is weird (he said, surprising nobody). It also doesn't have a logical type, conditional execution being predicated on whether an operation signals failure. So the "less than" function can return the empty string if a&lt;b, or it can fail.

Fortran has a LOGICAL type, logically. COBOL, being designed to be illogical wherever Fortran is logical, has a level 88 data type. Simula, Algol and Pascal use the word 'boolean', modulo capitalisation.

ML definitely has a bool type, but did it always? I can't see whether it was introduced in Standard ML (1980s-1990), or earlier (1973+). Nonetheless, it does appear that ML is the source of misspelled Booles.

#### APPosite Concerns
I've started another book project: APPosite Concerns is in the same series as, and is somehow a sequel to, APPropriate Behaviour. So now I just have one question to ask.

What is going to be in the book?

This question is easy to answer in broad terms. My mental conception of who I am and how I make software is undergoing a Narsil-like transformation: it has been broken and is currently being remade.

APPropriate Behaviour was a result of the build-up of stresses that led to it being broken. As I became less and less satisfied with the way in which I made software, I explored higher and higher levels looking for meaning. Finally, I was asking the (pseudo-)profound questions: what is behind software development? How does one philosophise about it? What does it mean?

If APPropriate Behaviour is the ascent, then APPosite Concerns is an exploration of the peak. It's an exploration of what we find when nothing is worth believing in, of the questions we ask when there is really no understanding of what the answers might be.

It's clear to me that plenty of the essays in this blog are relevant to this exploration, but of course there's not much point writing a book that's just some articles culled from my blog. There needs to be, if you'll excuse a trip into the world of self-important businessperson vocabulary for a second, some value add.

I've written loads recently. As I said right here, I write a lot at the moment. I write to get ideas out of my brain, so that I can ignore them and move on to other ideas. Or so that I can get to sleep. I write on a 1950s typewriter, I write on loose leaf paper, I write in notebooks, I write in Markdown files.

I know that there'll be plenty in there that can be put to good use, but which pieces are the valuable ones? Is it the fictionalised autobiography, written in the style of a Victorian novel? The submitted-and-rejected science fiction short about the future of the United Nations? The typewritten screed about the difficulties of iOS provisioning? The Platonic dialogue on the ethics of writing software?

One thing that's evident is that a reorganisation is required. Blogs proceed temporally, but books can take on any other order. The disparate essays from my collection are related: indeed given the same emotional state, any given subject trigger leads me to the same collection of thoughts. I could probably recreate any of the articles in SICPers not from memory, but from the same initial conditions. There's a consistent, though evidently evolving, worldview expressed in my recent writing. Connecting the various parts conceptually will be useful for both of us.

[By the way, there will eventually be a third part representing the descent: that part has in a very real sense not yet been written.]

#### APPropriate Behaviour is almost done
I just pushed another update to APPropriate Behaviour, my work on the things programmers do that aren't programming. There's some refinement to the existing material to be done, and a couple of short extra chapters to finish and add. But then it will be complete!

The recommended price of APPropriate Behaviour is $20. While it's been under development, I've allowed readers interested in a sneak peak to buy APPropriate Behaviour at any price above $5. Once the final chapters are in place, the recommended price will remain $20 but the minimum price will be increasing. If you've been pondering buying it but haven't yet, I recommend you do so now to get a bargain. Even if you buy it while I'm still working on it, you'll get free updates for life as I add new material and make corrections.

As a little taster of things to come, the two remaining chapters are:

The ethics of making software
The philosophy of making software

Can't wait to see what that means? Neither can I!

#### APPropriate Behaviour is complete!
APPropriate Behaviour, the book on things programmers do that aren't programming, is now complete! The final chapter - a philosophy of software making - has been added, concluding the book.

Just because it's complete, doesn't mean it's finished: as my understanding of what we do develops I'll probably want to correct things, or add new anecdotes or ideas. Readers of the book automatically get free updates whenever I create them in the future, so I hope that this is a book that grows with us.

As ever, the introduction to the book has instructions on joining the book's Glassboard to discuss the content or omissions from the content. I look forward to reading what you have to say about the book in the Glassboard.

While the recommended purchase price of APPropriate Behaviour is $20, the minimum price now that it's complete is just $10. Looking at the prices paid by the 107 readers who bought it while it was still being written, $10 is below the median price (so most people chose to pay more than $10) and the modal price (so the most common price chosen by readers was higher than $10).

A little about writing the book: I had created the outline of the book last Summer, while thinking about the things I believed should've been mentioned in Code Complete but were missing. I finally decided that it actually deserved to be written toward the end of the year, and used National Novel Writing Month as an excuse to start on the draft. A sizeable portion of the draft typescript was created in that month; enough to upload to LeanPub and start getting feedback on from early readers. I really appreciate the help and input those early readers, along with other people I've talked to the material about, have given both in preparing APPropriate Behaviour and in understanding my career and our industry.

Over the next few months, I tidied up that first draft, added new chapters, and extended the existing material. The end result - the 11th release including that first draft - is 141 pages of reflection over the decade in which I've been paid to make software: not a long time, but still nearly 15% of the sector's total lifespan. I invite you to grab a copy from LeanPub and share in my reflections on that decade, and consider what should happen in the next.

#### Apres ca, le FOSDEM
It appears that I'm sat in Terminal B of L'Aeroporte Nationale de Bruxelles, waiting for my flight to board.  While there are wirelesses around, the ones to which I can connect seem not to be offering much in the way of DHCP so this update will come in later than it was written (which was at 13:35), as I will probably post it while I'm on the bus between Heathrow and Oxford.  [Update: actually not until I got home]

Irrelevancies such as that aside, I had a great FOSDEM!  In fact, a great half-FOSDEM, as I did my tourism today.  I met a load of people (of which more below), went to some inspiring talks and discussed many exciting and interesting projects (in multiple languages - I spoke to one person in English, Dutch and French sometimes in the same sentence).  It was a good exercise to see who wasn't present as much as who was - for instance RedHat didn't have an official presence although the Fedora Project had a booth (next to the CentOS one ;-), similarly Novell (one of the big sponsors) was absent but the OpenSUSE project had Yet Another Small Table.  Sun were conspicuously present in that the OpenSolaris and OpenJDK table was being manned seemingly by Sun's salespeople rather than user group members...although maybe that's just my interpretation.

The overriding feeling I got was that the conference was running on l'espirit d'anarchie and that the resulting adrenaline and enthusiasm drove the conference on.  The keynote speeches were really the only regimented aspect of FOSDEM  - a necessity given the size of the auditorium and that was packed to the rafters with FLOSSers.  I didn't go to the final keynote on open-sourcing Java as I was manning the GNUstep booth, but learned a lot on software patents and Free Software and Jim Gettys' description of the technical challenges in creating OLPC was very insightful.

So, GNUstep.  GNUstep, GNUstep, Etoile [I'll add the accents in in a later update...this keyboard doesn't have dead keys :-(].  For a start it was great to meet all the other GNUsteppers, and have some good discussions and debates (as well as some good moule frites and Kwak beer).  For anyone who doubts that GNUstep is still alive, the dev room at FOSDEM is one place to allay such suspicions with many developers, designers, users and supporters presenting their ideas to each other, asking each other questions and generally contributing to the GNUstep camaraderie.  Even an improptu troll by Miguel de Icaza at the GNUstep booth wasn't enough to make us all throw Project Center away and buy a book on C# ;-).  Presentations on GNUstep-make v2 (which I've described here before...), the Cairo graphics back-end (which I don't think Fred Kiefer was expecting to present, but made a very fine job of it anyway) and third-party use of GNUstep were all very useful and well-received...I expect today's presentations were too but I didn't get to go to them :-(.  [Instead, I was significantly underwhelmed by the sight of the Mannekin Pis.]

#### Are you an [X] programmer?
On my twitter bio, I describe myself as:

  a Lovelacologist for portable transactators

which is, in keeping with the way I'm dressed in the avatar pic, a steampunk way of saying that I'm a programmer of mobile computers. But is that strictly true, or fair? It's what I've spent most of the last couple of years doing, but then I've also worked on:

web servers
SMPP servers
one particle accelerator
workstation apps
desktop apps
administration scripts
books

and there's bound to be more things that I haven't remembered. I don't think I'm alone in picking quite a narrow definition to expose as "me" (though maybe I should have thought a bit harder before titling this blog). Social scientists refer to this as "doing identity work", the effort we go to to control the definition of who we are in interactions with others. To confirm this choice of narrow identity work, here's a not-quite-random look at excerpts from a few other Twitter bios (anonymised for no real reason):

iOS, OS X, BMWs, photography, and food.
App developer by day - Apple fanboy by night
now a Clojure and Ruby programmer
iOS Developer

It's interesting that while we choose these restricted "brands" for ourselves, we actually spend a lot of time solving the same problems. I've been working on another web app project lately, and it's remarkably similar to building a mobile app. Even a lot of the constraints are similar:

keep the event loop fast
avoid loading lots of large data files
maintain separation of concerns between modules
try to minimise power consumption

and indeed the solutions turn out to be similar too. The command bus introduced in an earlier post, perfect for (and, according to some readers, informing their own work in) mobile apps, was actually built for this web app project. The problems and the solutions turn out to be interchangeable.

What we need is more of this interchangeability. Rather than waiting for a mobile person to say in a mobile way how to write mobile software, we can take advantage of what people have already said in a software way about how to write software. I have resolved to widen my horizons, and pay broader attention to what my colleagues are up to.

#### AROS
I spent the weekend teaching myself some Amiga GUI (Intuition) programming using AROS via the Icaros Desktop distribution, their developer guides, and the Amiga developer CD. It's a nice enough system to program in that works like most other GUI systems; GUI events are sent to your process as messages that you receive, handle, and reply to. GUIs are built out of gadgets attached to windows displayed on screens.

As a system to use, it feels efficient and fast. AROS is, as the Rhapsody developer releases were, a platform displaced in time: someone designed it to run on a 680x0 with a couple of megabytes of RAM and a floppy drive, and here we are with our modern CPU, gigabytes of RAM and solid state storage. It starts near-instantly, responds quickly (even though it's running in VirtualBox under a Windows 10 host), it's beautiful. And it's not overloaded with widgets and gewgaws added by vendors who are keeping up with the Joneses by adding every new feature to every platform they support. No notifications ask me whether I'm interested in a tour of all the latest features that I'd be unable to find myself.

As I write this post (in the Odyssey Web Browser in Icaros Desktop, of course), I wonder whether I need all of the other things I would get if I were doing this directly in Windows, or in macOS, or a modern Linux distribution. Give me a compiler long enough and a place to save and I shall move the Earth.

#### As the Kaiser Chiefs might say: Ruby ruby ruby n00bie
Imagine someone took the training wheels off of Objective-C. That's how I currently feel.

I've actually had a long—erm, not quite "love-hate", more "'sup?-meh"—relationship with Ruby. I've long wanted to tinker but never really had a project where I could make it fit; I did learn a little about Rails a couple of years back but didn't then get to put it into practice. Recently I've been able to do some Real Work™ with Ruby, and wanted to share the experience.

Bear in mind that when I say I've been working with Ruby, I mean that I've been writing Objective-C in Ruby. This becomes clear when we see one of the problems I've been facing: I couldn't work out how to indicate that a variable exposes some interface, until I realised I didn't need to. Ruby takes the idea of duck typing much further than Objective-C does: using Ruby is much more like Smalltalk in that you don't care what an object is, you care what it does. Currently no tools really support that way of working (and so Stockholm Syndrome-wielding developers will tell you that you don't need such tools; just vi and a set of tests); the first warning I get when I've made a mistake is usually an exception backtrace. Something I had to learn quite quickly is that Ruby and Objective-C have different ideas of nil: Ruby behaves as the gods intend and lets you put nil into collections; but Objective-C behaves as the gods intend and lets you treat nil as a null object.

The problems I've been facing have largely involved learning how things are conventionally done. One example is that a library I was using took a particular parameter and treated it as a constant. Apparently Matz is a big fan of Fortran, but only early hipster Fortran before they sold out and added implicit none (around the time they fired their bass player and started playing the bigger venues). So Ruby provides its own implicit convention: constants have to be named starting with an uppercase. Otherwise you get told this:

wrong constant name parameter-value

Erm, that's it. Not "you should try calling it ParameterValue", or "constants must start with a capital letter". Not even "this is not a good name for a constant"; who else interpreted that as "you gave the name of the wrong constant"? I think I've been spoiled by the improvements to the clang diagnostics over the last couple of years, but I found some of Ruby's messages confusing and unhelpful. This is often the case with software that relies on convention: once you know the conventions you can go really fast, but when you don't know them you feel like you're being ignored or that it's being obtuse.[*]

[*] When I asked for help on this issue I was told I suggest you pick of[sic] a good Ruby book or watch some Ruby tutorials on YouTube; you'll be pleased to know that the interpreter wasn't the only ignorant or obtuse tool I had to deal with.

These are very neophyte problems though, and once I got past them I found that I was able to make good progress with the language. I was using LightTable and RubyMine for editing, and found that I could work really quickly with a combination of those editors and irb. Having an interactive environment or a REPL is amazing for trying out little things, particularly when you're new at a language and don't know what's going to work. It's a bit cumbersome for more involved tests, but the general execute-test cycle is much faster than with Objective-C.

Speaking of tests, I know that if you ask four Ruby developers how to write unit tests you'll get six different answers and at least eighteen of them will have moved on to Node.JS. I've been using Mini::Test, as it's part of the standard library so involved the least configuration to get going.

I also took the opportunity to install MacRuby and have a go at building a Mac app, using Cocoa Bindings on the UI side to work with controllers and models that I'd written in Ruby. This isn't the first exposure I've had to a bridged environment: I've done a lot of Perl-Cocoa with CamelBones, the PerlObjCBridge and ObjectiveFramework. MacRuby isn't like those bridges though, in that (as I understand it) MacRuby builds Ruby's object model on top of NSObject and the Objective-C runtime so Ruby objects actually are ObjC objects. It means there's less manual gluing: e.g. in Perl you might do:

my $string = NSString->alloc->initWithCString_encoding_("Hello", NSUTF8StringEncoding);

In MacRuby that becomes:

string = "Hello"

That's not to say there's no boilerplate. I found that by-return references need the creation of a Pointer object on the Ruby side to house the pointer to the object reference, which looks like this:

error = Pointer.new(:object)
saveResult = string.writeToFile path, atomically: false, encoding: NSUTF8StringEncoding, error: error

For a long time, I've thought that there would be mileage in suggesting programmers use a different language than Objective-C for building applications in Cocoa, relying on ObjC as the systems language. Ruby could be that thing. The object models are very similar, so there isn't a great deal of mind-twisting going on in exposing Objective-C classes and objects in Ruby. There's a lot less "stuff you do that shuts the compiler up", though ObjC has seen a reduction in that itself of late it still relies on C and all of its idiosyncrasies. Whether it's actually better for some developers, and if so for whom, would need study.

Summarising, Ruby feels a lot like Objective-C without the stabilisers. You can work with objects and methods in a very similar way. The fast turnaround afforded by having an interactive shell and no compile-link waiting means you can go very quickly. The fact that you don't get the same up-front analysis and reporting of problems means you can easily drive into a wall at full tilt. But at least you did so while you were having fun.

#### At the old/new interface: jQuery in WebObjects
It turns out to be really easy to incorporate jQuery into an Objective-C WebObjects app (targeting GNUstep Web). In fact, it doesn't really touch the Objective-C source at all. I defined a WOJavascript object that loads jQuery itself from the application's web server resources folder, so it can be reused across multiple components:

jquery_script:WOJavaScript {scriptFile="jquery-2.0.2.js"}

Then in the components where it's used, any field that needs uniquely identifying should have a CSS identifier, which can be bound via WebObjects's id binding. In this example, a text field for entering an email address in a form will only be enabled if the user has checked a "please contact me" checkbox.

email_field:WOTextField {value=email; id="emailField"}
contact_boolean:WOCheckBox {checked=shouldContact; id="shouldContact"}

The script itself can reside in the component's HTML template, or in a WOJavascript that looks in the app's resources folder or returns javascript that's been prepared by the Objective-C code.

    &lt;script&gt;
function toggleEmail() {
    var emailField = $("#emailField");
    var isChecked = $("#shouldContact").prop("checked");
    emailField.prop("disabled", !isChecked);
    if (!isChecked) {
        emailField.val("");
    }
}
$(document).ready(function() {
    toggleEmail();
    $("#shouldContact").click(toggleEmail);
});
    &lt;/script&gt;

I'm a complete newbie at jQuery, but even so that was easier than expected. I suppose the lesson to learn is that old technology isn't necessarily incapable technology. People like replacing their web backend frameworks every year or so; whether there's a reason (beyond caprice) warrants investigation.

#### Automate all the server Objective-C!
I decided it was time to stop writing WebObjects/GNUstepWeb code, and write some code that would make it easier to write WO/GSW code. With that in mind I replaced my previous component generator with a more robust generator.

I also wrote and published some git hooks for working on these projects. The pre-commit hook just runs 'make' and doesn't let you commit if you can't build: we'll look at testing in a later post (you may not know this, but I've done a thing or two with Objective-C unit tests).

The post-commit hook launches the direct connect app, so you can have a box that's always running the latest version for testing. You'd want to do something similar, though really not the same, for a production box: as well as being sensitive to database and web server adaptor[*] configurations, you'd want to be stricter about when you restart the server, and may have some app-specific work to do like triggering cleanup code. Besides which, it makes more sense to do that in a post-update hook.

[*] While a GSW app does include a web server, it's common to restrict access to that server just to the internal network. Externally you have a "normal" web server like Apache or Nginx, with an adaptor that knows how to parse GSW URLs and redirect requests to the correct application.

#### Automated tests with the GNUstep test framework
Setup

Of course, it'd be rude not to use a temperature converter as the sample project in a testing blog post. The only permitted alternative is a flawed bank account model.

I'll create a folder for my project, then inside it a folder for the tests:

$ mkdir -p TemperatureConverter/test
$ cd TemperatureConverter

The test runner, gnustep-tests, is a shell script that looks for tests in subfolders of the current folder. If I run it now, nothing will happen because there aren't any tests. I'll tell it that the test folder will contain tests by creating an empty marker file that the script looks for.

$ touch test/TestInfo

Of course, there still aren't any tests, so I should give it something to build and run. The test fixture files themselves can be Objective-C or Objective-C++ source files.

$ cat &gt; converter.m
#include "Testing.h"

int main(int argc, char **argv)
{
}
^D

Now the test runner has something to do, though not very much. Any of the invocations below will cause the runner to find this new file, compile and run it. It'll also look for test successes and failures, but of course there aren't any yet. Still, these invocations show how large test suites could be split up to let developers only run the parts relevant to their immediate work.

$ gnustep-tests #tests everything
$ gnustep-tests test #tests everything in the test/ folder
$ gnustep-tests test/converter.m #just the tests in the specified file

The first test

Following the standard practice of red-green-refactor, I'll write the test that I want to be able to write and watch it fail. This is it:

#include "Testing.h"

int main(int argc, char **argv)
{
  TemperatureConverter *converter = [TemperatureConverter new];
  float minusFortyF = [converter convertToFahrenheit:-40.0];
  PASS(minusFortyF == -40.0, "Minus forty is the same on both scales");
  return 0;
}

The output from that:

$ gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---

test/converter.m:
Failed build:     

      1 Failed build

Unfortunately we could not even compile all the test programs.
This means that the test could not be run properly, and you need
to try to figure out why and fix it or ask for help.
Please see /home/leeg/GNUstep/TemperatureConverter/tests.log for more detail.

Unsurprisingly, it doesn't work. Perhaps I should write some code. This can go at the top of the converter.m test file for now.

#import &lt;Foundation/Foundation.h&gt;

@interface TemperatureConverter : NSObject
- (float)convertToFahrenheit:(float)celsius;
@end

@implementation TemperatureConverter
- (float)convertToFahrenheit:(float)celsius;
{
  return -10.0; //WAT
}
@end

I'm reasonably confident that's correct. I'll try it.

$gnustep-tests 
Checking for presence of test subdirectories ...
--- Running tests in test ---

test/converter.m:
Failed test:     converter.m:19 ... Minus forty is the same on both scales

      1 Failed test

One or more tests failed.  None of them should have.
Please submit a patch to fix the problem or send a bug report to
the package maintainer.
Please see /home/leeg/GNUstep/TemperatureConverter/tests.log for more detail.

Oops, I seem to have a typo which should be easy enough to correct. Here's proof that it now works:

$ gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---

      1 Passed test

All OK!

Second point, first set

If every temperature in Celsius were equivalent to -40F, then the two scales would not be measuring the same thing. It's time to discover whether this class is useful for a larger range of inputs.

All of the tests I'm about to add are related to the same feature, so it makes sense to document these tests as a group. The suite calls these groups "sets", and it works like this:

int main(int argc, char **argv)
{
  TemperatureConverter *converter = [TemperatureConverter new];
  START_SET("celsius to fahrenheit");
  float minusFortyF = [converter convertToFahrenheit:-40.0];
  PASS(minusFortyF == -40.0, "Minus forty is the same on both scales");
  float freezingPoint = [converter convertToFahrenheit:0.0];
  PASS(freezingPoint == 32.0, "Water freezes at 32F");
  float boilingPoint = [converter convertToFahrenheit:100.0];
  PASS(boilingPoint == 212.0, "Water boils at 212F");
  END_SET("celsius to fahrenheit");
  return 0;
}

Now at this point I could build a look-up table to map inputs onto outputs in my converter method, or I could choose a linear equation.

- (float)convertToFahrenheit:(float)celsius;
{
  return (9.0/5.0)*celsius + 32.0;
}

Even tests have aspirations

Aside from documentation, test sets have some useful properties. Imagine I'm going to add a feature to the app: the ability to convert from Fahrenheit to Celsius. This is the killer feature, clearly, but I still need to tread carefully.

While I'm developing this feature, I want to integrate it with everything that's in production so that I know I'm not breaking everything else. I want to make sure my existing tests don't start failing as a result of this work. However, I'm not exposing it for public use until it's ready, so I don't mind so much if tests for the new feature fail: I'd like them to pass, but it's not going to break the world for anyone else if they don't.

Test sets in the GNUstep test suite can be hopeful, which represents this middle ground. Failures of tests in hopeful sets are still reported, but as "dashed hopes" rather than failures. You can easily separate out the case "everything that should work does work" from broken code under development.

  START_SET("fahrenheit to celsius");
  testHopeful = YES;
  float minusFortyC = [converter convertToCelsius:-40.0];
  PASS(minusFortyC == -40.0, "Minus forty is the same on both scales");
  END_SET("fahrenheit to celsius");

The report of dashed hopes looks like this:

$ gnustep-tests 
Checking for presence of test subdirectories ...
--- Running tests in test ---

      3 Passed tests
      1 Dashed hope

All OK!

But we were hoping that even more tests might have passed if
someone had added support for them to the package.  If you
would like to help, please contact the package maintainer.

Promotion to production

OK, well one feature in my temperature converter is working so it's time to integrate it into my app. How do I tell the gnustep-tests script where to find my class if I remove it from the test file?

I move the classes under test not into an application target, but a library target (a shared library, static library or framework). Then I arrange for the tests to link that library and use its headers. How you do that depends on your build system and the arrangement of your source code. In the GNUstep world it's conventional to define a target called "check" so developers can write make check to run the tests. I also add an optional argument to choose a subset of tests, so the three examples of running the suite at the beginning of this post become:

$ make check
$ make check suite=test
$ make check suite=test/converter.m

I also arrange for the app to link the same library and use its headers, so the tests and the application use the same logic compiled with the same tools and settings.

Here's how I arranged for the TemperatureConverter to be in its own library, using gnustep-make. Firstly, I broke the class out of test/converter.m and into a pair of files at the top level, TemperatureConverter.[hm]. Then I created this GNUmakefile at the same level:

include $(GNUSTEP_MAKEFILES)/common.make

LIBRARY_NAME=TemperatureConverter
TemperatureConverter_OBJC_FILES=TemperatureConverter.m
TemperatureConverter_HEADER_FILES=TemperatureConverter.h

-include GNUmakefile.preamble

include $(GNUSTEP_MAKEFILES)/library.make

-include GNUmakefile.postamble

Now my tests can't find the headers or the library, so it doesn't build again. In GNUmakefile.postamble I'll create the "check" target described above to run the test suite in the correct argument. GNUmakefile.postamble is included (if present) after all of GNUstep-make's rules, so it's a good place to define custom targets while ensuring that your main target (the library in this case) is still the default.

TOP_DIR := $(CURDIR)

check::
    @(\
    ADDITIONAL_INCLUDE_DIRS="-I$(TOP_DIR)";\
    ADDITIONAL_LIB_DIRS="-L$(TOP_DIR)/$(GNUSTEP_OBJ_DIR)";\
    ADDITIONAL_OBJC_LIBS=-lTemperatureConverter;\
    LD_LIBRARY_PATH="$(TOP_DIR)/$(GNUSTEP_OBJ_DIR):${LD_LIBRARY_PATH}";\
    export ADDITIONAL_INCLUDE_DIRS;\
    export ADDITIONAL_LIB_DIRS;\
    export ADDITIONAL_OBJC_LIBS;\
    export LD_LIBRARY_PATH;\
    gnustep-tests $(suite);\
    grep -q “Failed test” tests.sum; if [ $$? -eq 0 ]; then exit 1; fi\
    )

The change to LD_LIBRARY_PATH is required to ensure that the tests can load the build version of the library. This must come first in the library path so that the tests are definitely investigating the code in the latest version of the library, not some other version that might be installed elsewhere in the system. The last line fails the build if any tests failed (meaning we can use this check as part of a continuous integration system).

More information

The GNUstep test framework is part of gnustep-make, and documentation can be found in its README. Nicola Pero has some useful tutorials about the rest of the make system, having written most of it himself.

#### Back from holiday
I went on holiday to Stockholm this week.  Of the ~1.5GB I downloaded from the camera, this was the photo I thought most apt to describe the experience on this blog:

#### Beer improves perception of security
...at least, it provides for a nice analogy to use when discussing basic security concepts. I don't think people necessarily choose better passwords after a skinful, nor do they usually make improved choices of what information to share on social networking sites when returning from the pub. That's probably why Mail Goggles exists.So, the attendee beer bash at WWDC. There is beer. There is also the regulatory framework of the state of California, which mandates that minors under 21 years of age may not be supplied with beer. There are also student scholarship places to the WWDC, and no theoretical minimum attendee age. There are also loads of people in the vague area of Yerba Buena Gardens who are not attending the WWDC. But only attendees may visit the bash, and only attendees over 21 may drink beer.I went to the registration desk at Moscone West on the day before Phil Schiller's keynote, and identified myself as Graham Lee. "Hi, my name's Graham Lee" I said to the lady behind the desk. It turns out that's not sufficient. While I did indeed give the name of someone who was indeed registered to attend the conference, there's no reason for the lady to believe the identification I gave her. She wants to be able to authenticate my claim, and chooses to rely on a trusted third party to do so. That third party is the British government (insert your own jokes here), and she is happy to accept my passport as confirmation of my identity.Now I am given my attendee badge, a token which demonstrates that I have authenticated as Graham Lee, an attendee at WWDC. When I move around the conference centre to get to the sessions and the labs, the security staff merely need to look for the presence of this token. They don't need to go through the business of checking my passport again, because the fact that I have my token satisfies them that I have previously had my identity authenticated to the required level.The access token would be sufficient to get me in to the attendee beer bash, as it proves that I have authenticated as an attendee. But it does not demonstrate that I am authorised to drink beer. So on the day of the bash I go back to the registration desk and show a different lady my passport again, which indicates that I am over 21 and can therefore be given the authority to drink beer at the bash. I am given the subtle green wristband pictures, which again acts as a token; this time not an identification token but an authorisation token. The bar staff do not care which of the 5200 attendees I am. In fact they do not care about my identity at all, because they know that the security staff have already verified my attendee status at the entrance to the event - therefore they don't need to see my attendee pass. They only care about whether I'm in the group of people permitted to drink beer, and the wristband shows that I am in that group. It shows that I have demonstrated the credentials needed to gain that particular authority.So, there we have it. A quick beer of an evening with a few thousand colleagues can indeed turn into a fun discussion of the distinction between authentication and authorisation, and how these two tasks can be carried out independently.

#### Beginner thoughts
Back story: my period of walkabout, in which I went to see the rest of the computing world beyond Apple land, started in November 2014. This was shortly after Swift's introduction at WWDC 2014. It ended in October 2018, by which time the language had evolved considerably, its position in the community had advanced greatly, and SourceKitService had stopped crashing.

I have previously written on the learning phases I encountered on exposure to Haskell, now what about Swift? I have the opportunity to reflect on how I react as a beginner, and share that so that we all learn how we (well, I) learn, and maybe discover how we can teach.
About the project
I'm writing a tool that I want, which takes files in one format (RSS) and writes them out in another format (Maildir). You can follow along. The reason for mentioning this here are twofold:

 	I do not know what I'm doing, but I'm willing to share that.
 	To let you understand the (limited, I think) complexity of the thing I'm trying to build.

Thinks: This should not be that hard
I often feel like Swift is making me feel like an idiot. This is because my expectation is too high: I know the platform fairly well. I know the Foundation framework pretty well. I know Xcode pretty well. I understand my problem to some extent. It should just be the programming language that's different.

And I do different programming languages all the time. What's another one going to do?

But of course it's not just the programming language that changed. It changed the conventions for things like naming methods or raising errors, and that means that the framework methods have changed, which means that things I used to know have changed, which means that I do not know as much as I assume. It introduced a new library, which I also don't know.
Thinks: That unimportant thing was really frustrating
Two such convention changes are correlated: classes that used to be Foundation and are now standard library (or maybe are Foundation still but have been renamed on being bridged, I'm not sure) are renamed from NSThing to Thing. That means that the name of NSURL is now URL.

That means that if you have a variable that represents a URL, you can't follow the Cocoa convention of leaving the abbreviation uppercased and calling it URL, because now it's got the same name as the type URL. So the new convention is to call it url.

Objectively, that's not a big deal. Subjectively, this stuff is baked in pretty deep, and changing it is hard.
Thinks: Even learning something is frustrating
The last event to make me get up and walk around a field was actually discovering something new about Swift, which should be the point, but nonetheless made me feel bad.

I have discovered that when it comes to working with optionals, the language syntax means that There Is More Than One Way To Do It. When I learned about if let and guard let, I was confused by the fact that the thing on the right needed to be an optional, not unwrap one: surely if my rvalue is an optional, then my lvalue should be, too?

Then, when I learned about the ?. and subsequently ?? operators, I thought "there's no way I would ever have thought to type that, or known how to search for those things". And even though they only make things shorter, not different, I still felt frustration at the fact that I'd gone through typing things out the long way.
Thinks: There's More Than One Way Not To Do It
One of the Broken Expectations™ is that I know how to use Strings. Way back when, NeXT apps used char * as their string type. Then Enterprise Objects Framework came along with its Foundation library of data types, including a new-fangled Unicode string class, NSString. Then, well, that was it for absolute ages.

So, when I had a String and I wanted to take the substring to an index, I was familiar with -substringToIndex: and tried to apply that. That method is deprecated, so I didn't want to use it. OK, well I can string[0..&lt;N]. Apparently not, integer subscripting is not allowed, and the error message tells me to read a code comment to understand why. I wish it told me where that code comment was, or just showed it to me, instead!

Eventually I found that there's a .prefix(N) method, again this is the sort of thing that makes me think: what's wrong with me? I've been programming for years, I've been programming on this platform for years, I should be able to get this.
Conclusion: Read a Book
I had expected that my knowledge of the Mac, Xcode, and Cocoa would be sufficient to carry me through a four-year gap on picking up a language, particularly with the occasional observation of a conference talk about the Swift language (I've even given one!). I had expected that picking up a project to build a thing would give me a chance to get acquainted.

I was reflecting on my early experiences with writing NeXT and Mac applications in Objective-C. I had my copy of the NeXT Developer Documentation, or Cocoa in a Nutshell, open on the desk, looking at the methods available and thinking "I have this, I want that, can I find one of these that gets me there?" I had expected that auto-complete in Xcode would be my modern equivalent of working that way.

Evidently not. Picking up the new standard library things, and the new operators, will require deliberate learning. I've got some videos lined up, but I think my next action is to find a good book to read.

#### Best.  App.  Evar!
In the spirit of My Dream App, I've invented my own dream app.  BestAppEvar.app would be able to divine what needs to be done by detecting my brainwaves, and would then do it when I hit the button.  To make it easy for you to implement, I've already designed the UI:So, why hasn't it been made yet?  Come on, you lazy coders!  The hardest bit would be the brain-computer interface, but there are example I/O Kit projects on the ADC site.

#### Better clear the office out, can't remember what the carpet looks like
OK, so I said there was a good reason to set up &THORN;&aelig;s Ofereode apart from the bandwidth at SDF not being up to the task, and it's time to come clean.  Yesterday I handed in my notice to my current employers, and I'm off to Opera Telecom to become a QA/Test developer.  It looks like I'll be doing funky things with GNUstep so regular readers of iamleeg will not notice a move away from Objective-C and UNIX related ventings.Of course this has been brewing for a few weeks, but I wanted to make sure my eyes were crossed and my teas dotted before letting anyone else know.  Especially on the big scary intarwebs.

#### better security, not always more security
Today's investigative investigations have taken me to the land of Distributed Objects,  that somewhat famous implementation of the Proxy pattern used for intra-process, inter-process and inter-machine communication in Cocoa. Well, by people who measure whether it's a performance hog, rather than those who quote it; as a hint, it was indeed a significant overhead when your CPU was a 25MHz 68030 and your network link a 10BASE-2 coaxial wire. These days we can spend around those problems freely.Specifically, I wondered whether I should add discussion of the authentication capabilities in PDO to the FAQ entry. Not that it's frequently asked - indeed, it's a NAQ - but because getting mentions of security into a Usenet FAQ is likely to cause newbies to be thinking about security, which is possibly a good thing (for the world, not so much my uniquely employable attributes). But I decided no, though the subject is interesting, it's not because of the technicality, but the philosophy.Distributed Objects works by sending NSPortMessage messages over NSConnection connections. The connections and message-passing bumph are peer-to-peer, but DO adds some client-server distinction by having servers register their vended connections with name servers and clients look up the interesting vendors in said name servers. By default, anything goes; all connections are honoured and all clients serviced. There are two security features (both implemented as delegate methods) baked into DO, though. The most interesting of the two is the authentication.The reason that the authentication feature is interesting is that it's implemented in such a way as to make non-security-conscious developers question the security. The end sending the NSPortMessage includes some data based on the constituent parts of the message, and the end receiving the message decides whether to accept it based on knowledge of the constituents and of the data. On the face of it, this looks like shared-secret encryption, with the shared secret being the algorithm used to hash the port message. It also appears to have added no security at all, because the message is still sent in plain text. In fact, what this gives us is more subtle.All that we know is that given the source information and the sender's authentication data, the receiver gets to decide whether to accept the sender's message. We don't necessarily know the way that the receiver gets to that decision. Perhaps it hashes the information using the same algorithm as the sender. Perhaps it always returns YES. Perhaps it always expects the authentication data to be 42. On the other hand, perhaps it knows the public key of the sender, and the authentication data is a signature derived from the content and the sender's private key. Or perhaps the "authentication data" isn't used at all, but the source material gives the server a chance to filter malicious requests.Now all of that is very interesting. We've gone from a system which looked to be based on a shared secret, to one which appears to be based on whichever authentication approach we decide is appropriate for the task at hand. Given a presumed-safe inter-process link, we don't need to be as heavyweight about security as to require PKI; whereas if the authentication were provided by a secure tunnel such as DO-over-SSL, we'd have no choice but to accept the cost of the PKI infrastructure. Given the expectation of a safe server talking to hostile clients, the server (or, with some amount of custom codery, a DO proxy server) can even sanitise or reject malicious messages. Or it could both filter requests based on authentication and on content. The DO authentication mechanism has baked in absolutely zero policy about how authentication should proceed, by letting us answer the simple question: should this message be processed? Yes or no? Choose an approach to answering this question based not on what you currently believe could never be circumvented, but on what you currently believe is sufficient for the environment in which your DO processes will live. If a shared secret is sufficient and adds little overhead, then do that, rather than 4096-bit asymmetric encryption.By the way, the second security feature in DO is the ability to drop a connection when it's requested. This allows a DO server to survive a DoS, even from a concerted multitude of otherwise permissible clients.

#### Beware the IDEs
I recently had the opportunity to talk with a couple of software project managers from IBM. That company is of a kind that I have never worked at, and many of the companies I have worked at are of kinds that these IBMers have not worked at. There was thus plenty of opportunity for us to have different opinions and to explore those.

One such difference, though one we did not investigate in depth, is over IDEs. There was no doubt in the IBM view: a developer with an IDE is a more productive developer. If your programming team wants to license some proprietary IDE to use on your project, you're probably better off paying for the licensing.

That's not the world I come from, and indeed I may even have removed IDEs from my consciousness completely. Sure, I use a few, but what's wrong with emacs, TAGS, and a bit of gud.el and M-x compile? I don't mind launching an IDE when it's there, but I don't miss it when I work with separate tools.

So, are the people who write Rails apps in Sublime Text or vim doing it right, or are they missing out on clear productivity gains? More fundamentally, was it a good idea to build out something like Rails without keeping the IDE support in sync with the features?

There are all sorts of reasons to believe that IDEs are better.

IDEs are actually better, and IBM has a mature enough development system to be able to measure that.
IBM believes the Rational marketing (Rational's price ticket was at the high end of software company acquisitions and perhaps IBM's culture has, ahem, rationalised that).
There was a time that IDEs were better than developing without, but now bloatware text editors like emacs and vim have caught up without IBM's view updating.

There are also reasons to believe that IDEs are not better.

IDEs are actually not betSourceKit Service crashed.
The cynic in me doesn't believe the facts presented by the IDE vendors' marketeers.
The bloatware text editors have actually caught up.
The productivity problems I have do not stem from my choice of editing environment.

What's kindof interesting is to compare the views of dyed-in-the-wool IDE evangelists, dyed-in-the-wool text editor/command-line proponents, and fence-sitters and find out why they have different views, whether there are facts that can be teased out of these positions or assertions that can be validated. What's really fascinating is general situation of which this is a single example: there are whole regions of the software development phase space that I have yet to experience.

#### Blame culture
The tweet that started this post off:

  Candidate for the worst name of any SCM feature: “blame”. Don’t use it for that.

I was recently watching a couple of conference talks (given by people working at the same company) where the presenters described using the 'blame' feature of their version control tool to find the person responsible for a bug so that they could go down to their office and chew them out.

This is not constructive. What are you going to do if that person is on holiday, let the bug fester for a week so you can have the rant when they get back? What if they left the company? Remember that you're on the same team: you are permitted to fix the bug. Fixing the bug helps you, your team, your company and your customers; complaining at the person who introduces it only helps to poison your relationships with your team. So fix it. That's not to say the blame feature isn't useful:

use it to find out why the change was introduced, which might help you understand what was intended. This is why you write good commit messages.
use it to find out when the change was introduced, to help you understand the severity of the bug[*]
yes, use it to find out who made the change, but only if you think that you're going to learn something from the discussion. Remember, we do not set the bozo bit.

[*] I've done this recently. On discovering and isolating a bug, I used git blame to work out whether it was introduced before or after the software was released. Sadly, it was before.

Maybe it should be called "explain"; whatever, do not use your blame feature for blame.

Aside on blame visualisation

If you're reading this blog post from Black Pixel, hello! Please feel welcome to build this feature into Kaleidoscope without the need to pay me a licensing fee :-)

As code ages, small changes tend to get added here and there. We build an initial implementation of some feature, then discover that there are some edge cases or new ways of using it that we didn't consider, so little patches get built on. These can overlap or supersede earlier patches, and interact in interesting ways (where "interesting" can be read as "infuriating", if you like).

What I would find useful is a diff tool that just shows the current state of the file, but uses the z-axis to indicate the age of each line. In the same way that code editors can identify regions for folding using (simulated or actual) depth, they could identify regions of change. This would make it easy to see at a glance how the file has evolved over time, to see where and when changes were needed.

#### Blasty from the pasty
Yesterday I had a little problem with the iBook, which was saved by some age-old knowledge from before the OS X release.  Almost.

When I use an external USB mouse, I disable the internal trackpad.  Now I wanted to take the computer into a different room in order to watch some TV (on the EyeTV), so unplugged all the USB stuff, plugged the EyeTV in....and the trackpad was still disabled.  Now, a lesser person would have gone back into the original room, plugged the USB mouse in, and used it (or taken it with them).  An equally lesser, but richer, person would own a Bluetooth mouse.  But not I!  I remembered a trick from the NeXT days, and returned my computer to trackpad-workingness in but a jiffy.  Three jiffies.

The first thing to do is to log out - &quot;as any fule kno&quot; this is done from the keyboard by Command-Option-Shift-Q.  At the loginwindow there still wasn't any trackpad activity, so I thought restarting the WindowServer might fix that.  This is done by entering &quot;&gt;exit&quot; as the username in loginwindow.  Sadly, no joy.  My next thought was to drop to a text console, unload the trackpad's driver kext, reload it and see what difference that made.  Doing so requires entering &quot;&gt;console&quot; as the username.  However I decided not to pursue this as &gt;console has been sadly broken throughout 10.4, and on some systems was broken in 10.3 too.  It's annoying as occasional glitches where the graphical environment is FUBARed could probably be fixed in console mode.  But this left one course of action.  Enter &quot;&gt;restart&quot; in the loginwindow and wait...

I should probably point out that I waited for the reboot longer than it would have taken to retrieve the mouse.  But at least I wasn't defeated.

#### Book update: OOP the Easy Way
Obejct-Oriented Programming the Easy Way gets ever closer, as the first part (of three) is now substantively complete. If you have been holding off from buying the book, now would be a great opportunity to jump in, as a whole part of the book's argument is now laid out. As ever, your feedback is welcome, and readers who buy now will get free updates throughout the development of the book.

#### Bottom-up teaching
We're told that the core idea in computer programming is problem-solving. That one of the benefits of learning about computer programming (one that is not universally accepted) is gaining the skill of problem decomposition.

If you look at real teaching of computing, it seems to have more to do with solution composition than problem decomposition. The latter seems to be background noise: here are the things you can build solutions with, presumably at some point you'll come across a solution that's the same size and shape as one of your problem components though how is left up to you.

I have many books on programming languages. Each lists the features of the language, and gives minimally complex examples of the use of those features. In that sense, Kernighan and Ritchie's "The C Programming Language" (section 1.3, the for statement) is as little an instructional in solving problems using a computer as Eric Nikitin's "Into the Realm of Oberon" (section 7.1, the FOR loop) or Dave Thomas's "Programming Elixir" (section 7.2, Using Head and Tail to Process a List).

A course textbook on bitcoin and blockchain (Narayanan, Bonneau, Felten, Miller and Goldfeder, "Bitcoin and Cryptocurrency Technologies") starts with Section 1.1, "Cryptographic hash functions", and builds a cryptocurrency out of them, leaving motivational questions about politics and regulation to Chapter 7.

This strategy is by no means universal: Liskov and Guttag's "Program Development in Java" starts out by describing abstraction, then looks at techniques for designing abstractions in Java. Adele Goldberg and Alan Kay described teaching Smalltalk by proposing exploratory projects, designing the objects that model the problem under consideration and the way in which they will communicate, then incrementally filling in by designing classes and methods that have the desired properties. C.J. Date's "An Introduction to Database Systems" answers the question "why databases?" before introducing the relational model, and doesn't introduce SQL until it can be situated in the context of the relational model.

Both of these approaches, and their associated techniques (the bottom-up approach and solution construction; the top-down approach and problem decomposition) are useful; the former leads to progress and the latter leads to understanding. But both must be taken in concert, because understanding without progress leads to the frustration of an unsolved problem and progress without understanding is merely the illusion of progress.

My guess is that more programmers - indeed whole movements, when we consider the collective state of things like OOP, functional programming, BDD, or agile practices - are in the "bottom-up only" group than in the "top-down only" or "a bit of both" groups. That plenty more copies of Introduction to Programming in [This Week's Hot Language] have been sold than Techniques for Making Your Problem Amenable to Computation. That the majority of software really does comprise of solutions looking for problems.

#### Broke track mounting
For some reason, CDs occasionally don't automount for me on my iMac.  Luckily that's easy to work around:kalevala:~ leeg$ diskutil list[...]/dev/disk3   #:                       TYPE NAME                    SIZE       IDENTIFIER   0:        CD_partition_scheme Audio CD               *620.3 Mi   disk3[...]kalevala:~ leeg$ diskutil mountDisk disk3Volume(s) mounted successfullyJob is, as they say, a good 'un.

#### BrowseOverflow as a Code Kata
This article was originally posted over at InformIT.

My goal in writing Test-Driven iOS Development was to take readers from not knowing how to write a test for their iOS apps, to understanding the TDD workflow and how it could work for them. That mirrored the journey that I had taken in learning about test-driven development, and that had led me to wanting to write a book to share what I'd learned with my peers.

This has an interesting effect on the structure of the book. Not all of the sample code from the BrowseOverflow is shown (though it's all available on GitHub). This isn't an accident we made in the editorial process. It's a feature: for any test shown in the book, there are numerous different ways you could write application code that would all be just as good. Anything that causes the test to pass, and doesn't make the other tests fail, is fine.

Just as the many-worlds model in quantum theory says that there are many branches of the universe that are created every time a decision is made, so there's a "many-BrowseOverflows" model of Test-Driven iOS Development. Every time a test is written, there are many different solutions to passing the test, leading to there being multiple potential BrowseOverflows. The code that you see on GitHub is just one possible BrowseOverflow, but any other code that satisfies the same tests is one of the other possible BrowseOverflows.

This means that you can treat the book like a kata: the Japanese martial art technique of improving a practice by repeating it over and over. The first time you read Test-Driven iOS Development, you can choose to follow the example code very closely. Where the code isn't given in the book, you might choose to look at the source code to understand how I solved the problems posed by the tests. But the end of the book is not the end of the journey: you can go back, taking the tests but implementing all of the app code yourself. You can do this as many times as you want, trying to find new ways to write code that produces a BrowseOverflow app.

Finally, when you're more confident with the red-green-refactor way of working, you can write a BrowseOverflow app that's entirely your own creation. You can define what the app should do, create tests that express those requirements and write the code to implement it however you like. This is a great way to test out new ways of working, for example different test frameworks like GHUnit or CATCH. It also lets you try out different ways of writing the application code: you could write the same app but trying to use more blocks, or try to use the smallest number of properties in any class, or any other challenge you want to set yourself. Because you know what the software should be capable of, you're free to focus on whatever skill you're trying to exercise.

#### Build systems are a huge annoyance
Take Smalltalk. Do I have an object in my image? Yes? Well I can use it. Does it need to do some compilation or something? I have no idea, it just runs my Smalltalk.

Take Python. Do I have the python code? Yes? Well I can use it. Does it need to do some compilation or something? I have no idea, it just runs my Python.

Take C.

Oh my God.

C is portable, and there are portable operating system interface specifications for the system behaviour accessible from C, so you need to have C sources that are specific to the platform you're building for. So you have a tool like autoconf or cmake that tests how to edit your sources to make them actually work on this platform, and performs those changes. The outputs from them are then fed into a thing that takes C sources and constructs the software.

What you want is the ability to take some C and use it on a computer. What C programmers think you want is a graph of the actions that must be taken to get from something that's nearly C source to a program you can use on a computer. What they're likely to give you is a collection of things, each of which encapsulates part of the graph, and not necessarily all that well. Like autoconf and cmake, mentioned above, which do some of the transformations, but not all of them, and leave it to some other tool (in the case of cmake, your choice of some other tool) to do the rest.

Or look at make, which is actually entirely capable of doing the graph thing well, but frequently not used as such, so that make all works but making any particular target depends on whether you've already done other things.

Now take every other programming language. Thanks to the ubiquity of the C run time and the command shell, every programming language needs its own build system named [a-z]+ake that is written in that language, and supplies a subset of make's capabilities but makes it easier to do whatever it is needs to be done by that language's tools.

When all you want is to use the software.

#### Building a unit test target with GNUstep make
Just a quick note on how I build my test tools (they run separately, either by manual invocation or via CI) when I'm working in GNUstep.

Firstly, you'll need Catch. Then given test files that look like this:

test_class.mm
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#import 

TEST_CASE("Using foundation", "I should be able to use Foundation classes from a test tool")
{
  NSArray *array = [NSArray array];
  REQUIRE([array count] == 0);
}

(only one of your files should define CATCH_CONFIG_MAIN as you must only have one main function).

Then I define a tests target in my Makefile:

GNUmakefile

include $(GNUSTEP_MAKEFILES)/common.make

#… stuff to define my app target

TEST_TOOL_NAME=tests
tests_OBJCC_FILES = test_class.mm other_tests.mm blah.mm
tests_OBJCCFLAGS="-ICatch/include"

-include Makefile.preamble

#… include target definitions for the app target
include $(GNUSTEP_MAKEFILES)/test-tool.make

-include Makefile.postamble

And there it is. Now whenever I make my app I also get obj/tests which'll run the test suite for me.

#### Building an object-oriented dispatch system in Objective-C
iTunes was messing about rebuilding the device I was trying to use for development, so I had time over lunch to write a new message dispatch system in the Objective-C language. "But wait," you say, "Objective-C already has a message dispatch system!" True, and it's better than the one I've created. But it doesn't use blocks, and blocks are cool :-). In the discussion below, I'll build up an implementation of a "recent items" list, which is discussed in Kevlin Henney's presentation linked in the acknowledgements.

The constructor

One important part of an object system is the ability to make new objects. Let's declare an object type, and a constructor type that returns one of those objects:

typedef id (^BlockObject)(NSString *selector, NSDictionary *parameters);
typedef BlockObject(^BlockConstructor)(void);

I'd better explain signature of the BlockObject type. Objects can be sent messages; what we're doing is saying that if you execute the object with a selector name, the object will dispatch the correct implementation with the parameters you supply and will give you back the return value from the implementation. That's what objc_msgSend() does in old-school Objective-C. The constructor is going to return this dispatch block - actually it'll return a copy of that block, so invoking the constructor multiple times results in multiple copies of the object. Let's see that in action.

BlockConstructor newRecentItemsList = ^ {
    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        return (id)nil;
    }
    return (BlockObject)[list copy];
}

Yes, you have to cast nil to id. Who knew C could be so annoying?

Message dispatch

An object that can't do anything isn't very exciting, so we should add a way for it to look up and execute implementations. Method implementations are of the following type:

typedef id (^BlockIMP)(NSDictionary *parameters);

With that in place, I'll show you an example of the object with a dispatch system in place, and discuss it afterward.

typedef void (^SelectorUpdater)(NSString *selector, BlockIMP implementation);

BlockConstructor newRecentItemsList = ^ {
    __block NSMutableDictionary *selectorImplementationMap = [NSMutableDictionary dictionary];
    __block SelectorUpdater setImplementation = ^(NSString *selector, BlockIMP implementation) {
        [selectorImplementationMap setObject: [implementation copy] forKey: selector];
    };

    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        BlockIMP implementation = [selectorImplementationMap objectForKey: selector];
        return implementation(parameters);
    };
    return (BlockObject)[list copy];
};

The variables selectorImplementationMap and setImplementation are __block variables in the constructor block. This means that every time the constructor is called, the returned instance has its own copy of these variables that it is free to use and to modify. Let me put that another way: the entire message-dispatch system is encapsulated inside each instance. If a class, or even an individual instance, wants to implement message dispatch in a different way, that's cool. It also means that an instance can change its own methods at runtime without affecting any other objects, including other instances of the same class. As long as the object still conforms to the contract that governs method dispatch, that's cool too.

Implementing the recent items list

OK, now that we've got construction and messaging in place, we can start making useful objects. Here's the implementation of the recent items list, where I've chosen to use an NSMutableArray for the internal storage, as with the dispatch map it's an instance variable of the list. Needless to say you could change this to a C array, STL container or anything else without breaking external customers of the object.

BlockConstructor newRecentItemsList = ^ {
    __block NSMutableDictionary *selectorImplementationMap = [NSMutableDictionary dictionary];
    __block SelectorUpdater setImplementation = ^(NSString *selector, BlockIMP implementation) {
        [selectorImplementationMap setObject: [implementation copy] forKey: selector];
    };
    __block NSMutableArray *recentItems = [NSMutableArray array];
    setImplementation(@"isEmpty", ^(NSDictionary *parameters) {
        return [NSNumber numberWithBool: ([recentItems count] == 0)];
    });
    
    setImplementation(@"size", ^(NSDictionary *parameters) {
        return [NSNumber numberWithInteger: [recentItems count]];
    });
    
    setImplementation(@"get", ^(NSDictionary *parameters) {
        NSInteger index = [[parameters objectForKey: @"index"] integerValue];
        return [recentItems objectAtIndex: index];
    });
    
    setImplementation(@"add", ^(NSDictionary *parameters) {
        id itemToAdd = [parameters objectForKey: @"itemToAdd"];
        [recentItems removeObject: itemToAdd];
        [recentItems insertObject: itemToAdd atIndex: 0];
        
        return (id)nil;
    });
    
    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        BlockIMP implementation = [selectorImplementationMap objectForKey: selector];
        return implementation(parameters);
    };
    return (BlockObject)[list copy];
};

Using the list

Here is an example of creating and using a recent items list. Thankfully, since writing this post literal dictionaries have appeared, so it doesn't look so bad:

    BlockObject recentItems = newRecentItemsList();
    BOOL isEmpty = [recentItems(@"isEmpty", nil) boolValue];

    NSDictionary *getArgs = @{ @"index" : @0 };
    @try {
        id firstItem = recentItems(@"get", getArgs);
        NSLog(@"first item in empty list: %@", firstItem);
    }
    @catch (id e) {
        NSLog(@"can't get first item in empty list");
    }

Exercises for the reader

The above class is not complete. Here are some ways you could extend it, that I haven't covered (or, for that matter, tried).

Implement @"isEqual". Remember that no instance can see the ivars of any other instance, so you need to use the public interface of the other object to decide whether it's equal to this object. You'll need to provide a new method @"respondsToSelector" in order to build @"isEqual" properly.
Respond to unimplemented selectors well. The implementation shown above crashes if you send an unknown message: it'll try to dereference a NULL block. That, well, it's bad. Objective-C objects have a mechanism that catches these messages, allowing the object a chance to lazily add a method implementation or forward the message to a different object.
Write an app using these objects. :-)

Credit where it's due

This work was inspired by Kevlin Henney's presentation: It is possible to do OOP in Java. The implementation shown here isn't even the first time this has been done using Objective-C blocks. The Security Transforms in Mac OS X 10.7 work in a very similar way. This is probably the first attempt to badly document a bad example of the art, though.

#### But where to go?
I agree with John Gruber here: it's not like Apple's stuff has become worse than a competitor's, it's just that it's not as good as I remember or expect. It could be, as Daniel Jalkut suggests, rose-tinted glasses[*].

I don't think there is a "better" competitor, except in limited senses: Solaris/IllumOS and OpenBSD both have good-quality code but are not great to use out of the box: Solaris in particular I associate with abysmal package management and flaky support from supposedly cross-platform applications that are actually only ever built and tested on GNU Debian has well-adhered free software guidelines and much better compatibility but not all GNU code is as high-quality as some alternatives. The OpenBSD copyright policy and the FSF definition of freedom are incompatible so OpenBSD doesn't contain much GNU software GNU doesn't contain much OpenBSD software: you get a base system that's either one or the other and have to do work if you want bits of both. Other GNU/Linux distributions can be easier to set up and have better (i.e. any) support for non-free software and wider collections of device drivers.

So there are plenty of alternatives, many of which are good in some ways and bad in others, and all I know is that I don't want things to be like this, without being able to say I want one of those instead. I don't even think there will be one of those, at least not in the sense of a competitor to Apple on laptop operating systems. Why not? Because I agree with the following statement from wesolows:

  from the perspective of someone who appreciates downstack problems and the value created by solving them, is that the de facto standard technology stack is ossifying upward. As one looks at each progressively lower layer, the industry’s collective willingness to contemplate, much less sponsor, work at that layer diminishes.

There's still research in operating systems, sure, but is there development? Where are the NeXTs and Bes of today? I don't believe you could get to a million users and have a Silicon Valley "exit" with low-level technology improvements, and so I don't think the startup world is working in that area. So we probably won't get anything good from there. I don't see competition in operating systems being fruitful. If it were, Sun wouldn't have been sold.

In fact I don't even think that Apple's systems are bad, they've just lost the "it just works" sheen. It's just that when you combine that with the lack of credible alternative, you realise the problem is probably in expecting some corporation to put loads of resources into something that's not going to have a great value, and merely needs to be "good enough" to avoid having any strategic penalty.

To me, that means treating the low-level parts of the technology stack as a public good. If we accept that the stack is ossifying upwards, and that EM64T, Unix, C, IP, HTTP, SQL and other basic components are going to be around essentially forever[**] then we need to treat them and their implementations as public goods and take common ownership of them. They might not be the best possible, but they are the best available. We (we the people who make systems on top of them, in addition to we the people who use systems made on top of them) need them to work collectively, so we should maintain them collectively.

[*]I particularly like his use of the phrase "Apple-like" in this context, because that term is often used to mean "my platonic ideal of Apple's behaviour" rather than "what Apple actually does" and reminds me to be wary of my own recollections. I remember Lightning connectors being welcomed in a tweet that derided the old iPod 30-pin connector as "un-Apple-like", despite the evidence that Apple invented, introduced the 30-pin connector and then supported it for over a decade.

[**] speaking of Sun, I use the definition of computer-forever I learned from a Sun engineer: five years or longer.

#### By doing it and helping others do it
We are uncovering better ways of developing
software by doing it and helping others do it.
It's been 20 years since those words were published in the manifesto for agile software development, and capital-A Agile methods haven't really been supplanted. Despite another two decades of doing it and helping others do it.

That seems problematic.

#### By the river
My home stands near to a bridge over the Avon, the same river that lends its name to Stratford-on-Avon. By walking to the end of my street then through the churchyard, I could stand over the river and watch it flow beneath me. But I didn't. I elected to walk a few kilometres upstream to the grounds of the big house, and stand on that bridge instead. Walking is something I do, apparently. My phone tells me that I walk more than the average man, an achievement I can claim as my own.

It was raining, so I took an umbrella. In theory, I was listening to the patter of the rain on the umbrella, and watching the water flow through the reeds as it washed underneath me, then past my house and the church, past Shakespeare's birthplace and onto Tewkesbury and the Severn. In fact I did nothing of the sort. The river and its reeds must have been there, and I was pointing in their direction. I was vaguely aware at one point in my half-hour stop of a car on the driveway, approaching me, stopping just short of where I was, then carrying on over the bridge and on to the road. I did not acknowledge the driver, and if that person acknowledged me I failed to notice it.

Of things that I saw, the one that sticks in my mind is the notification light on my phone. It is my beacon to the outside world, letting me know when somebody wants my attention. Twice I remember looking at it, and twice I remember seeing the blue blink that signifies contact via email. Twice I read this link to society at large, to discover that somebody wanted to sell me the world's most powerful flashlight and that somebody else thought I would be interested in a walk-in bathtub. Isn't technology a marvel?

Of things that I heard, probably the car had some sort of engine and tyres that rumbled as it made its way past, and presumably the dual carriageway a few hundred metres away was contributing some white noise. The only sound that I remember hearing is my internal monologue. It's a toxic sound, reminding me as it does that I have no reason to be pleased about anything, but it is my constant companion. I struggle to hold back tears as it reminds me that my home life, my social life, and my work life, are all in flux at the moment, all out of my control, and the common thread running through them all is the person who lost that control. It goes on to tell me that anything else I might try is doomed to failure, too.

I consider using that phone again, not to look at the notification light, but to tell somebody how I'm feeling. Why would you want to do that, asks the voice? These people have their own things to deal with, why burden them with your problems too? Why spoil somebody's evening by telling them that you're upset, when they've done nothing to upset you? Why are you so selfish? You're a grown-up, a man who walks more than the average, you're supposed to look after yourself now. You're not supposed to be a mess with a state-sponsored Citalopram habit.

I turn the phone around, thinking of taking a picture of the trees silhouetting the clouds, stained Sodium-orange by the streetlights of the nearest town. The screen remains resolutely black. Of course it did, you idiot, it's much too dark to take a photo with that phone. You're supposed to be some kind of expert at smartphones, don't you know how their cameras work? I vaguely consider whether to fetch my SLR camera, but of course that's been in a box for over a year and it's doubtful the battery is even charged. Maybe it's time to sell it on eBay, but of course you won't get a good price for it and then it'll get damaged in the post and the buyer will want a refund and you won't even have kept that money. Why bother?

It took effort to walk the twenty minutes upstream to the other bridge. My voice told me that I would be just as useful to the world if I didn't take the walk, if I had stayed in bed and waited until it was time to go to work tomorrow. Well, voice, I took everything you told me and made a few hundred words of blog post out of it, so I have done something after all. The joke's on you.

#### By your _cmd
This post is a write-up of a talk I gave at Alt Tech Talks: London on the Objective-C runtime. Seriously though, you should've been there.
The Objective-C runtime?
That's the name of the library of C functions that implement the nuts and bolts of Objective-C. Objects could just be represented as C structures, and methods could just be implemented as C functions. In fact they sort of are, but with some extra capabilities. These structures and functions are wrapped in this collection of runtime functions that allows Objective-C programs to create, inspect and modify classes, objects and methods on the fly.

It's the Objective-C runtime library works out what methods get executed, too. The [object doSomething] syntax does not directly resolve a method and call it. Instead, a message is sent to the object (which gets called the receiver in this context). The runtime library gives objects the opportunity to look at the message and decide how to respond to it. Alan Kay repeatedly said that message-passing is the important part of Smalltalk (from which Objective-C derives), not objects:
I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea.

The big idea is "messaging" - that is what the kernal[sic] of Smalltalk/Squeak is all about (and it's something that was never quite completed in our Xerox PARC phase). The Japanese have a small word - ma - for "that which is in between" - perhaps the nearest English equivalent is "interstitial". The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be.

Indeed in one article describing the Smalltalk virtual machine the programming technique is called the message-passing or messaging paradigm. "Object-Oriented" is used to describe the memory management system.

Throughout this talk and post I'm talking about the ObjC runtime, but there are many. They all support an object's introspective and message-receiving capabilities, but they have different features and work in different ways (for example, Apple's runtime sends messages in one step, but the GNU runtime looks up messages and then invokes the discovered function in two steps). All of the discussion below relates to Apple's most modern runtime library (the one that is delivered as part of OS X since 10.5 and iOS).

In the talk, I decided to examine a few specific areas of the runtime library's behaviour. I looked for things that I wanted to understand better, and came up with questions I wanted to answer as part of the talk.
Dynamic class creation
Can I implement Key-Value Observing?
While I was preparing the talk, a post called KVO considered harmful started to get a lot of coverage. That post raises a lot of valid criticisms of the Key-Value Observing API, but rather than throw away the Observer pattern I wanted to explore a new implementation.

The observed (pardon the pun) behaviour of KVO is to privately subclass the observed object's class, so that it can customise the object's behaviour to call the KVO callback. That's done through a function called objc_duplicateClass, unfortunately the documentation tells us that we should not call this function ourselves.

It's still possible to implement an Observer pattern that uses the same secret-subclass behaviour, by allocating and registering a "class pair". What's a class pair? Well each class in Objective-C is really two classes: the class object defines the instance methods, and the "metaclass" defines the class methods. So each class is really a singleton instance of its metaclass.

The ObserverPattern implementation shows how this works. When you add an observer to an object, the receiver first works out whether it's an instance of the observable class. If it needs to create that class, it does so: adding our own implementations of -dealloc to clean up after ourselves, and -class so that, like KVO observable objects, the generated class name doesn't appear when you ask an observed object its type.

Having created the class, the code goes on to add a setter for the conventional Key-Value Coding selector name for the property: this setter grabs the old and new values of the property and invokes the callback which was supplied as a block object. Because we can, the block is dispatched asynchronously.

Notice that the -addObserverForKey:withBlock: method uses object_setClass() to replace the receiver's class with the newly-constructed class. The main effect of this is to change the way messages are resolved onto methods, but you need to be careful that the original and replaced class have the same instance variable layout too. Instance variables are looked up via the runtime too, and changing the class could alter where the runtime thinks the bytes are for any given variable.

We have a little extra hurdle to overcome in storing the collection of observer tokens, because there's nowhere to put them. Adding an instance variable to the ObserverPattern[…] class would not work, as instances of that class are never actually allocated. The objects involved have the instance variables of their initial class, which won't include space for the observers.

The Objective-C runtime provides for this situation by giving us associated objects. Any object can have what is, conceptually, a dictionary of other objects maintained by the runtime. Observed objects can store and retrieve their observer tokens via associated references, and no extra instance variables are needed.

A little problem in the ObserverPattern implementation will become clear if you run it enough times. The observation callbacks are sent asynchronously, and can be delivered out of sequence. That means the observer can't actually tell what the final state of the observed key is, because the "new value" received in the callback might have already been replaced. I left this fun issue in to demonstrate that KVO's synchronous implementation is a feature, not a bug.
Creating objects
What are those extra bytes for?
When you create an Objective-C object, the runtime lets you allocate some extra storage at the end of the space reserved for its instance variables. What's the point of that? All you can do is get a pointer to the start of the space (using object_getIndexedIvars)…hmm, indexed ivars. Well, I suppose an array is a pretty obvious use of indexed ivars…

Let's build NSArray! There are two things to see in SimpleArray: the most obvious is the use of the class cluster pattern. The reason is that the object returned from +alloc—where we'd normally allocate space for the object—cannot know how big it's going to be. We need to use the arguments to -initWithObjects:count: to know how many objects there are in the array. So +alloc returns a placeholder, which is then able to allocate and return the real array object.

One obvious question to ask is why we'd do this at all. Why not just use calloc() to grab an appropriately-sized buffer in which to store the object pointers? The answer is to do with a low-level performance concern called locality of reference. We know from the design of the array class that pretty much every time the array pointer is used, the buffer pointer will be used too. Putting them next to each other in RAM means we don't have to look off at some dereferenced pointer just to find another pointer.
Message dispatch
Just how does message forwarding work?
One of the powerful features of Objective-C is that an object doesn't have to implement a method when it's compiled to be able to respond to messages with that selector name. It can lazily resolve the methods, or it can forward them to another object, or it can raise an error, or it can do something else. But something about this feature was bugging me: message forwarding (which happens in the runtime) calls -forwardInvocation:, passing it an NSInvocation object. But NSInvocation is defined in Foundation: does the runtime library need to "know" about Foundation to work?

I tracked down what was going on and found that no, it does not need to know about Foundation. The runtime lets applications define the forwarding function, that gets called when objc_msgSend() can't find the implementation for a selector. On startup, CoreFoundation[+] injects the forwarding function that does -forwardInvocation:. So presumably my application can do its own thing, right?

Let's build Ruby! OK, not all of Ruby. But Ruby has a #method_missing function that gets called when an object receives a message it doesn't understand, which is much more similar to Smalltalk's approach than to Objective-C's. Using objc_setForwardHandler, it's possible to implement methodMissing: in our Objective-C classes.
Conclusion
The Objective-C runtime is a powerful way to add a lot of dynamic behaviour to an application for very little work. Some developers don't use it much beyond swizzling methods for debugging, but it has facilities that make it a powerful tool for real application code too.

[+]CoreFoundation and Foundation are really siblings, and they each expose pieces of the other's implementation, but one has a C interface and the other an Objective-C interface. Various Objective-C classes are actually part of CoreFoundation, including NSInvocation and the related NSMethodSignature class. NSObject is not in either of these libraries: it's now defined in the runtime itself, so that the runtime's memory management functions know about -retain, -release and so on[++]. On the other hand, most of the *behaviour* of NSObject is implemented by categories higher up. And, of course, this is all implementation detail and the locations of these classes could be (and are) moved between versions of the frameworks.

[++]Other languages like Smalltalk and Ruby have a simple base class that does nothing except know how to be an object, called ProtoObject or BaseObject. You could imagine the runtime supplying—and being coupled to—ProtoObject, and (Core)Foundation supplying NSObject and NSProxy as subclasses of ProtoObject.

#### Bye bye data, hello...the same data
Of course it happens to everyone, and yesterday evening it happened to me...my home directory became inaccessible.  What seems to have happened is that the filevault image containing my ~ became corrupted upon unmounting (though notably, I didn't do the 'recover space' thing the last time I logged out before the failure, so it should just have been a straightforward unmount). so the simplest recovery route was to delete the user, re-create it then recover my data from the backups.  I don't keep backups of the Library area so lost a few preference files, and of course have had to trawl around my email looking for licence keys and the like.

For the moment I've set up the replacement user without Filevault, and am using encrypted disk images for specific data I'd rather keep thus protected.  This makes backups harder - I keep my backup drive unencrypted as it doesn't come out with me, so I now need to come up with a script to backup my home dir except for the encrypted images, mount the images and back up the content, then unmount them.  This means that the backup will need to be manually triggered so that passwords don't have to be kept anywhere...or I write my own backup tool, which uses passwords stored in a keychain kept outside the target user account; and I need to make sure that keychain is also recoverable ;-).

A lot of my data was completely unaffected - work stuff is typically stored in subversion on their servers (as well as another local copy on my work laptop), my email is all on remote servers, my calendar is served by thaesofereode.info and so on.  There are some improvements I could make - I could probably use an LDAP server and abxldap to remotify my contact list, and thaesofereode.info offers subversion hosting which I'm currently not making use of.  But it happens that next Tuesday, I'll be talking about data security at the Oxford Mac Users Group, so I will expand on this tale in full and gory detail ;-).  St. Cross College, Tuesday 8th May, 7:30 pm.

Update 20070503T1653Z+0000: actually, things look a little more serious than simply a trashed sparseimage:

mabinogi:~/Desktop leeg$ hdiutil attach OmniDazzle-1.0.1.dmg load_hdi: timed out waiting for driver to loadload_hdi: timed out waiting for driver to loadload_hdi: timed out waiting for driver to loadload_hdi: timed out waiting for driver to loadload_hdi: timed out waiting for driver to load[...]2007-05-03 15:41:35.535 diskimages-helper[718] ERROR: unable to load disk image driver - 0xE00002C0/-536870208 - Device not configured.

Good news is that when that gets fixed, my old homedir will start working again.  Bad news is: um, it looks fairly messed up to me :-(

#### Can code be "readable"?
Did Isaac Asimov write good stories?

Different people will answer that question in different ways. People who don't read English and don't have access to a translation will probably be unable to answer. People who don't like science fiction on principle (and who haven't been introduced to his mystery stories) will likely say 'no', on principle. Other people will like what he wrote. Some will like some of what he wrote. Others will accept that he did good work but "that it isn't really for me".

The answers above are all based on a subjective interpretation, both of Asimov's work and the question that was asked. You could imagine an interpretation in the form of an appeal to satisfaction: who was the author writing for, and how does the work achieve the aim of satisfying those people? What themes was the author exploring, and how does the work achieve the goal of conveying those themes? These questions were, until the modern rise of literary theory, key ways in which literary criticism analysed texts.

Let us take these ideas and apply them to programming. We find that we demand of our programmers not "can you please write readable code?", but "can you consider what the themes and audience of this code are, and write in a way that promotes the themes among members of that audience?" The themes are the problems you're trying to solve, and the constraints on solving them. The audience is, well, it's the audience; it's the people who will subsequently have to read and understand the code as a quasi-exclusive collection.

We also find that we can no longer ask the objective-sounding question "did this coder write good code?" Nor can we ask "is this code readable?" Instead, we ask "how does this code convey its themes to its audience?"

In conclusion, then, a sound approach to writing readable code requires author and reader to meet in the middle. The author must decide who will read the code, and how to convey the important information to those readers. The reader must analyse the code in terms of how it satisfies this goal of conveyance, not whether they enjoyed the indentation strategy or dislike dots on principle.

Source code is not software written in a human-readable notation. It's an essay, written in executable notation.

#### Can Objective-C be given safe categories?
That was the subject of this lunchtime's vague thinking out loud. The problems with categories are well-known: you can override the methods already declared on a class, or the methods provided in another category (and therefore another category can replace your implementations too). Your best protection is to use ugly wartifying prefixes in the hope that your bewarted method names don't collide with everybody else's bewarted method names.

A particular problem with categories, and one that's been observed in the wild, is when you add a method in a category that is, at some later time, added to the original implementation of the class itself. Other consumers of the class (including the framework it's part of) may be expecting to work with the first-party implementation, not your substitution. If the first-party method has a different binary interface to yours (e.g. one of you returns a primitive value and the other a struct), as happened to a lot of people with NSArray around the end of the 1990s, prepare to start crashinating.

Later implementations of similar features in other languages have avoided this problem by refusing to add methods that already exist, and by ensuring that even if multiple extensions define the same method they can all coexist and the client code expresses exactly which one it's referring to. Can we add any of this safety to Objective-C?

Partially. We could design a function for adding a collection of methods from a "category" to a class at runtime, that only adds them if the class doesn't already implement them. class_addCategory() shows what this might look like, but it only supports non-struct-returning instance methods.

If class_addCategory(target, source, NO) succeeds, then the methods you were trying to add did not exist on the target class before you called the function. However, you cannot be sure that they weren't being added while your call was in progress, and you can't know later that they weren't clobbered by someone else at some point between successfully adding the methods and using them. Also, if class_addCategory() fails, you may find the only reasonable course of action is to not use the methods you were trying to add: the only thing you know about their implementation is that it either doesn't exist or isn't the one you were expecting. This is at odds with a hypothetical purist notion of Object-Oriented Programming where you send messages to objects and don't care what happens as a result.

There are plenty of ways to work around the limitations of categories: composition is the most likely to succeed (more likely than subclassing, which suffers the same collision problem as a later version of the superclass might try to define a method with the same name as one you've chosen, which you're now clobbering). It doesn't let you replace methods on a class—a tool that like most in the programmer's utility belt is both occasionally useful and occasionally abuseful.

Coda

I should point out that I'm not a fan of taking away the potentially dangerous tools. Many people who see the possibility for a language feature to be abused argue that it should never be used or that languages that don't offer it should be preferred. This is continuum-fallacy nonsense, to which I do not subscribe. Use whatever language features help you to produce a working, comprehensible, valuable software system: put in whatever protections you want to guard against existing or likely problems.

#### Can't you just...
Continuing the thoughts on vexing problems, one difficulty when it comes to discussing software is talking about the size of software. I'm not really talking about productivity metrics - good or bad - like source lines of code or function points, rather the fact that the complexity of a problem looks different depending on who's doing the looking.

Sometimes, a problem that's very simple from a business perspective can be incredibly complex technically. One product I worked on could be summarised very quickly: let people interact with marketing campaigns by sending and receiving messages on their mobiles. The small amount of logic between send and receive - allowing the campaigns to operate as quizzes, votes, or auctions - could be detailed on an index card.

But that simplicity was backed by a huge amount of technical complexity to make it work. "Can you just send this message to everyone who got the quiz question correct?" Well, yes, but as it's a picture message we need to work out how to make it look good on the recipient phone, change it to fit those criteria, and then send it. What makes it look good - and indeed how we can get the information to make that decision - depends on the device, but also which network it's on, and maybe whether it's on pre-pay or post-pay and whether they use HTTP, WAP or e-mail to send messages to that device (which might be different from how they send to other devices on the same network). And even after we've gathered that information, it may be wrong as some devices claim to support image formats that they can't render, or image sizes that they'll actually reject or fail to display.

On the other hand, sometimes the business problem is a lot more complex than the technical problem. If you're a mobile app developer, any length of problem definition about exciting disruptive apps can be reduced to "so you want to display data from a web service in a table view".

And then at other times, "can't you just" gets stymied from left field. Why yes, we could simply do that, and it would be good for the business, but this regulation/patent/staff shortage means we need to do something else.

#### Cap in Hand
You're probably aware that between this blog, De Programmatica Ipsum, and various books, I write a lot about software engineering and software engineers.

You may know that I also present a podcast on software engineering topics, and co-host two live streams on Amiga programming and Objective-C programming.

I do all of this because I want to. I want to engage in conversations about software engineering; I want to help my colleagues and peers; I want to pass on my experience to others. Of course, this all takes rather a lot of time, and a not-insignificant amount of money. Mostly in hosting fees, but also a surprising chunk on library memberships, purchase of out-of-print materials on software engineering, and event attendance. More than my academic (i.e. not-for-profit) salary was designed to withstand. None of these projects is ad-supported, and that's not about to change.

I've launched a Patreon page, where if you enjoy anything I write, say, or show, you can drop me a little bit of cash to say thanks. There's no obligation: nothing I currently make freely available is going behind a paywall, and I'm not planning any "subscriber-only content" in the future. All I'm saying is if you've enjoyed what I've been producing, and having my voice in the software engineering fray, here's another way in which you can say thank you.

#### Careful how you define your properties
Spot the vulnerability in this Objective-C class interface:

@interface SomeParser : NSObject {
  @private
	NSString *content;
}
@property (nonatomic, retain) NSString *content;
- (void)beginParsing;
//...
@end

Any idea? Let's have a look at a use of this class in action:

SomeParser *parser = [[SomeParser alloc] init];
NSMutableString *myMutableString = [self prepareContent];
parser.content = myMutableString;
[parser beginParsing];
[self modifyContent];

The SomeParser class retains an object that might be mutable. This can be a problem if the parser only functions correctly when its input is invariant. While it's possible to stop the class's API from mutating the data - perhaps using the State pattern to change the behaviour of the setters - if the ivar objects are mutable then the class cannot stop other code from making changes. Perhaps the string gets truncated while it's being parsed, or valid data is replaced with invalid data while the parser is reading it.

If a class needs an instance variable to remain unmodified during the object's lifetime (or during some lengthy operation), it should take a copy of that object. It's easy to forget that in cases like strings and collections where the type of the ivar is immutable, but mutable subclasses exist. So to fix this parser:

@property (nonatomic, copy) NSString *content;

You could also make the property readonly and provide an -initWithContent: constructor, which takes a copy that will be worked on.

But with collection class properties these fixes may not be sufficient. Sure, you definitely get an immutable collection, but is it holding references to mutable elements? You need to check whether the collection class you're using support shallow or deep copying&mdash;that is, whether copying the collection retains all of the elements or copies them. If you don't have deep copying but need it, then you'll end up having to implement a -deepCopy method yourself.

Note that the above discussion applies not only to collection classes, but to any object that has other objects as ivars and which is either itself mutable or might have mutable ivars. The general expression of the problem is fairly easy to express: if you don't want your properties to change, then take copies of them. The specifics can vary from case to case and, as ever, the devil's in the detail.

#### Categories will bite you
What I wanted to do was this:

+ (void)load {
    Method foo = class_getInstanceMethod(self, @selector(foo));
    Method newFoo = class_getInstanceMethod(self, @selector(FZA_swizzleFoo));
    method_exchangeImplementations(foo, newFoo);
}

However, my tests wouldn't work when I did that. It turns out that for some reason +load was running twice, so the methods got swizzled twice meaning that each implementation ends up married to its original selector. So I thought I'd do this:

+ (void)load {
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        Method foo = class_getInstanceMethod(self, @selector(foo));
        Method newFoo = class_getInstanceMethod(self, @selector(FZA_swizzleFoo));
        method_exchangeImplementations(foo, newFoo);
    });
}

...and, *drum roll*, that doesn't work either. What gives?

Well, it turns out that the reason that the category was being loaded twice is because it was included in two separate binary images: the library I was testing and the unit test bundle. Because of this, the "static" dispatch_once_t instance was actually a separate instance for each image, so dispatch_once() couldn't tell it had already run the block.

In my case there's a solution; I only need the category to be loaded once so removed it from one of the targets. But the general message is that category loading is very non-deterministic, and hard to rely on. Imagine if you rely on an open-source category, then one of your plugin developers relies on the same category. Or one of the frameworks your app links against. There are specific cases where the implementation of categories causes no effects, or effects that are understood and may be worked around, but those are really the exceptions.

#### Ch-ch-ch-ch-changes
It's been almost a year since my first day at Facebook, sitting in an overcrowded meeting room with my bootcamp class because 42 Earlham Street was full and it'd be another week before we moved to 10 Brock Street, with its gargantuan empty spaces (which are no longer empty: nearly half the company has joined since I started).

How has that year been, you ask? Hard. It's been hard. Fun, worthwhile, educational, exciting, but hard. Facebook is very different from any company I've worked at before: it's bigger, it's faster, it's more ambitious. Everything I had learned about making software outside the company was...well no, not wrong, certainly not wrong. But it was perhaps inapplicable. Facebook does things differently, I can't immediately have impact by doing what I did at my other employers, how can I cope with this?

When you're used to the life of minor engineering celebrity, where people go to conferences because you're on the bill, and your pithy statements on programming get retweeted up the wazoo, it's easy to get hubristic. I did: I know what I'm doing, the engineers here don't yet know that, and there will be a glorious revelation when I bring the two tablets (an iPad and a Galaxy Note 10, I don't know) down from the mountain, sell everyone in the company a copy of my book and we all start doing things The Right Way™.

That wasn't going to work. Not because what I wanted to do couldn't work, but because I was starting from the wrong place. How many billion-dollar companies had I written software for when I started at Facebook? None. So who was I to say that my way of writing software was "right"? All I could say was that it definitely felt right.

What I had to learn to move past this was that Facebook works on data. If I can show that there's time being lost, or bugs being introduced, or run times being lengthened, and that what I want to do would save time, or catch bugs, or speed things up, then Facebook will listen.

The opposite of "my way is not incontrovertibly correct" is not "my way is incontrovertibly incorrect", but it was easy to come to that conclusion too, and to rage-quit the idea of having any effect here. I think I avoided this, but I also think I came pretty close to it. I could easily have decided that while I don't like Facebook's way, I need to suck it up and work that way. That's wrong for the same reason that trying to boil Facebook's oceans was wrong: just as what I'm used to doing isn't necessarily correct, what Facebook is doing isn't necessarily correct. There's room for change.

Indeed, Facebook engineering is far from change-averse. Everything is set up to let you make changes, from the extreme collective code ownership ("bored changing the code you normally work on? Change the rest of it!") to incident review ("OK, that change wasn't the best, let's see what we can learn from it").

So, it took me a long time, perhaps longer than it should: I used my first year to realise that I can change Facebook and Facebook can change me, and the way we'll make this happen is by showing each other how the changes will make us better. Now, as David Bowie said, it's time to turn and face the strange.

#### Change
I was just discussing software architecture and next steps with a team building a tool to help analyse MRI images of brains. Most of the questions we asked explored ways to proceed by focussing on change:

what if the budget for that commercial component shows up? How would that change the system?
what if you find this data source isn't good enough? How would you find that out?
which of these capabilities does the customer find most important? When will they change their minds?

that sort of thing.

We have all sorts of words for planning for, and mitigating the risk of, changes in low-level software design. In fact a book on building maintainable software talks about nothing else, because maintainable software is antifragile software.

But it happened that I wasn't reading that book at the time, I was reading about high-level design and software architecture. The guide I was reading talked a lot about capturing the requirements and constraints in your software architecture, and this is all important stuff. If someone's paying for your thing, you need to ensure it can do the things they're paying for it to do. After all, they're probably paying to be able to do the things that your software lets them do; they aren't paying to have some software. Software isn't real.

However, most of the reason your development will slow down once you've got that first version out of the door is that the world (which might be real) changes in ways that it's hard to adapt your software to. Most of the reason you're not adding new features is that you're fixing bugs, i.e. changing the behaviour of the software from one that matches the flawed conception you had of what it should do to one that matches the flawed conception you now have of what it should do.

A good architecture should identify, localise, and separate sources of change in the software system. And then it should probably do whatever you think the customers think they want.

#### Chiron Codex early bird ends soon
Early bird pricing for Chiron Codex ends soon! Join our community of AI-augmented software engineering centaurs now to lock in early access to public content, as well as exclusive videos, book and journal reviews, and more. Ends March 27.

#### Choose boring employees
An idea I've heard from many directions recently is that "we" (whoever they are) "need to be on the latest tech stack in order to attract developers". And yes, you do attract developers that way. Developers who want to be paid to work on the latest technology.

Next year, your company will be a year more mature. Your product will be a year more developed. You will have a year more customers. You'll have a year more tech debt to pay off.

And your cutting-edge tech stack will be so last year. Your employees will be looking at the new startup in the office next door, and how they're hiring to work on the latest stack while you're still on your 2017 legacy technology.

#### Choose boring employers
Amusingly, my previous post choose boring employees was shared to hacker news under the off-by-one erroneous title choose boring employers. That seemed funny enough to run with, but what does it mean to choose boring employers?

One interpretation is that a boring employer is one where you do not live in interesting times. Where you can get on with your job, and with finding new and better ways to do your job, without constantly fighting fires.

But what if you're happiest in an environment where you are fighting fires? In that case, you probably should surround yourself with arsonists.

Another interpretation is to invert the discussion in Choose Boring Employees: find an employer who spends their innovation tokens wisely. One who's OK with the answer to "how do I store these tuples of known structure" being "in a relational database", or one who doesn't mind when the answer to "what platform should we base our whole business on" starting with "I skim-read a blog post on HN when I was riding MUNI this morning and...".

But, let's be clear, there's a place for the shiny new technology. Sometimes you do need to spend your innovation tokens, so you don't want to be somewhere that won't let you do it at all. Working on a proof of concept, you want to get to proof quickly, so it may be time to throw caution to the wind (unless the concept you're trying to prove involves working within some cautious boundaries). So boring need not get as far as frustrating.

#### Choosing the correct openings and closures
Plenty of programmers will have heard of the Open-Closed Principle of object-oriented design. It is, after all, one of the five SOLID principles. You may not, however, have seen the principle as originally stated. You've probably heard this formulation by Robert C. Martin, or a variation on the theme:

  Modules that conform to the open-closed principle have two primary attributes.
  
  
  They are “Open For Extension”.
  This means that the behavior of the module can be extended. That we can make the module behave in new and different ways as the requirements of the application change, or to meet the needs of new applications.
  They are “Closed for Modification”.
  The source code of such a module is inviolate. No one is allowed to make source code changes to it.
  

Source: "The Open-Closed Principle", the Engineering Notebook, Robert C. Martin, 1996

OK, so how can we add stuff to a module or make it behave "in different ways" if we're not allowed to make source code changes? Martin's solution involves abstract classes (because he's writing for a C++ journal, read "interfaces" or "protocols" as appropriate to your circumstances). Your code makes use of the abstract idea of, say, a view. If views need to work in a different way, for some reason (say someone needs to throw away some third-party licensed display server and replace it with something written in-house) then you don't edit the view you've already provided, you replace that class with your new one.

The Open-Closed Principle was originally written by Bertrand Meyer in the first edition of his book, Object-Oriented Software Construction. Here's what he had to say:

  A satisfactory modular decomposition technique must satisfy one more requirement: it should yield modules that are both open and closed.
  
  
  A module will be said to be open if it is still available for extension. For example, it should be possible to add fields to the data structures it contains, or new elements to the set of functions it performs.
  A module will be said to be closed if [it] is available for use by other modules. This assumes that the module has been given a well-defined, stable description (the interface in the sense of information hiding). In the case of a programming language module, a closed module is one that may be compiled and stored in a library, for others to use. In the case of a design or specification module, closing a module simply means having it approved by management, adding it to the project's official repository of accepted software items (often called the project baseline), and publishing its interface for the benefit of other module designers.
  

Source: Object-Oriented Software Construction, Bertrand Meyer, 1988 (p.23)

Essentially the idea behind a "closed" module for Meyer is one that's baked; it has been released, people are using it, no more changes. He doesn't go as far as Martin later did; there are no changes to its data structure or functionality. But if a module has been closed, how can it still be open? "Aha," we hear Meyer say, "that's the beauty of inheritance. Inheritance lets you borrow the implementation of a parent type, so you can open a new module that has all the behaviour of the old." There's no abstract supertype involved, everything's concrete, but we still get this idea of letting old clients carry on as they were while new programmers get to use the new shiny.

Both of these programmers were suggesting the "closedness" of a module as a workaround to limitations in their compilers: if you add fields to a module's data structure, you previously needed to recompile clients of that module. Compilers no longer have that restriction: in [and I can't believe I'm about to say this in 2013] modern languages like Objective-C and Java you can add fields with aplomb and old clients will carry on working. Similarly with methods: while there are limitations in C++ on how you can add member functions to classes without needing a recompile, other languages let you add methods without breaking old clients. Indeed in Java you can add new methods and even replace existing ones on the fly, and in Smalltalk-derived languages you can do it via the runtime library.

But without the closed part of the open-closed principle, there's not much point to the open part. It's no good saying "you should be able to add stuff", of course you can. That's what the 103 keys on your keyboard that aren't backspace or delete are for. This is where we have to remember that the compiler isn't the only reader of the code: you and other people are.

In this age where we don't have to close modules to avoid recompiles, we should say that modules should be closed to cognitive overload. Don't make behavioural changes that break a programmer's mental model of what your module does. And certainly don't make people try to keep two or more mental models of what the same class does (yes, NSTableView cell-based and view-based modes, I am looking at you).

There's already a design principle supposed to address this. The Single Responsibility Principle says not to have a module doing more than one thing. Our new version of the Open-Closed Principle needs to say that a module is open to providing new capabilities for the one thing it does do, but closed to making programmers think about differences in the way that thing is done. If you need to change the implementation in such a way that clients of the module need to care about, stop pretending it's the same module.

#### chord graphics
Ha, crossover humour! It's like Core Graphics, which is a Mac thing, only it's chord, because I'm talking about music, but I'm a Mac guy...oh, never mind.When I'm trying to think of chords in music I always end up with a mental image of a piano keyboard, with the notes that make up the chord pressed. That's all well and good, but I don't have a piano! Apart from some set pieces like barre chords, I can't really think of note combinations in the same way on a guitar, and certainly get flustered trying to harmonise on a violin. What I really need is a piano to sit down and work out harmonies at, which I could then play on the instrument of my choosing (playing a string of notes on any of those instruments isn't so much of a problem).Unfortunately the biggest piece of floor space I currently have access to is about 1.3m x 0.4m. Maybe some cheap Bontempi would fit there, but not an 88-key upright. If there were a real-space version of the GarageBand digital keyboard, that would certainly fit...in fact it would probably fit in one of my nostrils. One octave doth not a piano make. Some people have suggested Clavinova, MODUS or similar electric pianos before. The thing is, they cost around £2k, whereas an upright is &lt;£500.

#### Class clusters, placeholder objects, value-oriented programming, and all that good stuff.
Have you ever seen this exception in your crash log?

2012-05-29 17:55:37.240 Untitled 2[5084:707] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -length only defined for abstract class.  Define -[NSPlaceholderString length]!'

What's that NSPlaceholderString class?

Leaving aside NSMutableString for a moment[*], there's no way for a developer who's got an instance of an NSString to modify that string. In this model a string instance represents the value of that string: the word "hello" is always going to be "hello". You can build a sentence that includes the word "hello" in a sequence - e.g. "hello, world". You can build a different sentence, e.g. "goodbye, world". You haven't changed the value of the word "hello" to "goodbye", you've changed the value of the sentence to include a word with a different value.

OK, let's take that to an extreme. If any string that a developer gets back from NSString's API is immutable, then that should include the string she gets back from +allocWithZone:, right? So any extra data passed in an -initWith… method can't be used to change the string object we just allocated.

That's OK, because -init… methods are allowed to return a different object, preserving this "don't change the value" principle. Imagine the C string initialiser for NSString looking like this (I doubt it does - I think it internally converts the string to UTF-16 - but it'll do as an example):

-(id)initWithCString: (char *)cString encoding: (NSStringEncoding)encoding
{
  NSCString *otherString = [NSCString allocWithZone: [self zone]];
  [self release];
  otherString->length = strlen(cString);
  otherString->bytes = malloc(otherString->length);
  strlcpy(otherString->bytes, cString, otherString->length);
  otherString->storedEncoding = encoding;
  return otherString;
}

This doesn't violate the no-modification contract, because it only changes an object that's being built and that the end developer hasn't seen yet. Once the developer gets to look at this string - when it's returned from the initialiser - it'll be immutable.

So this means that the string which was returned from +allocWithZone: represents a particular value of a string: the string that has yet to be assigned a value. Indeed, it's a placeholder string. But any string that has yet to be assigned a value can be represented by the same placeholder, because they all mean the same thing. That means we can save some memory by creating a Flyweight instance of the placeholder. Even if multiple call sites in multiple threads all get the same instance of our placeholder string, there's no danger of them tripping over each other because they'll all then get different strings as they tell the placeholder what values they need to represent.

In fact, if code in two different threads need to represent the same value, it's safe to give them both references to the same object. Neither can change that object and spoil things for the other.

This pattern of keeping objects immutable in the eyes of client code, providing transformations that result in new objects rather than modifying existing objects, makes a raft of thread safety problems disappear and reduces the complexity of class APIs. I'll be using it more often in my object models.

[*]To be honest, I'd like to leave it aside forever. It satisfies the Law of Demeter, but there's a whole class of concurrency problems that only exist because "a mutable string isa string".

#### ClassBrowser's public face
I made a couple of things:

ClassBrowser discussion list
The project website's source is now visible and MIT licensed

I should've done both of these things at the beginning of the project. I believe that the fact I opened the source really early, when it barely did one thing and then only on my machine™, was a good thing. It let people find out about the project, have a look, and even send changes. But not giving anywhere to discuss ClassBrowser was a mistake. It meant I answered questions in private messages that would've been better archived, and it probably turned some people away who couldn't immediately see what the point was.

Hopefully it's not too late to fix that.

#### ClassBrowser: warts and all
I previously gave a sneak peak of ClassBrowser, a dynamic execution environment for Objective-C. It's not anything like ready for general use (in fact it can't really do ObjC very well at all), but it's at the point where you can kick the tyres and contribute pull requests. Here's what you need to know:

Get it here
This is how you build it
Please fix these bugs (some of them will be quite simple, so if you haven't had a chance to contribute to an open source project yet, check out #19, #15, #17, #11, #5)
This is where it should end up

Have a lot of fun!

ClassBrowser is distributed under the terms of the University of Illinois/NCSA licence (because it is based partially on code distributed with clang, which is itself under that licence).

#### Classes are globals, too
Software engineers are used to the notion that global variables are a bad idea. Globals are usually accessed by asking, not by telling. They introduce tight coupling between any module that uses the global and the one that declares it, and (more dangerously) implicit coupling between all of the modules that use the global.

It can be hard to test code that uses globals. You typically need to supply a different object (by which I mean a linker object) that exports the same symbol, so that in tests you get to configure the global as you need it. Good luck if that's deeply embedded in other code you do need, like NSApp.

While we're on the subject of global variables, the names of classes are all in the global namespace. We know that: there is only one NSKeyedUnarchiver class, and everyone who uses the NSKeyedUnarchiver class gets the same behaviour. If any module that uses NSKeyedUnarchiver changes it (e.g. by swizzling a method on the metaclass), then every other client of NSKeyedUnarchiver gets the modified behaviour.

[If you want a different way of looking at this: Singletons are evil, and each class is the singleton instance of its metaclass, therefore class objects are evil.]

Now that makes it hard to test use of a class method: if I want to investigate how my code under test interacts with a class method, I have these options:

Swizzle the method. This is possible, but comparatively high effort and potentially complicated; should the swizzled method call through to the original or reimplement parts of it behaviour? What goes wrong if it doesn't, and what needs to be supported if it does? Using the example of NSKeyedArchiver, if you needed to swizzle +archiveRootObject:toFile: you might need to ensure that the object graph still receives its -encodeWithCoder: messages - but then you might need to make sure that they send the correct data to the coder…
Tell the code under test what class to use, don't let it ask. So you'd have a property on your object @property (nonatomic, strong) Class coderClass; and then you'd use the +[coderClass archiveRootObject:toFile:] method. In your app, you set the property to NSKeyedArchiver and in the tests, to whatever it is you need.

That's simple, and solves the problem, but rarely seen. I think this is mainly because there's no way to tell the Objective-C compiler "this variable represents a Class that's NSCoder or one of its subclasses", so it's hard to convince the type-safety mechanism that you know archiverClass responds to +archiveRootObject:toFile:.
Use instance methods instead of class methods, and tell the code you're testing what instance to use. This is very similar to the above solution, but means that you pass a fully-configured object into the code under test rather than a generic(-ish) class reference. In your code you'd have a property @property (nonatomic, strong) NSCoder *coder in to -encodeRootObject: and rely on it having been configured correctly to know what to do as a result of that.

This solution doesn't suffer from the type-safety problem introduced in the previous solution; you told the compiler that you're talking to an NSCoder (but not what specific type of NSCoder) so it knows you can do -encodeRootObject:.

Notice that even if you decide to go for the third solution here (the one I usually try to use) and use object instances instead of classes for all work, there's one place where you must rely on a Class object: that's to create instances of the class. Whether through "standard" messages like +alloc or +new, or conveniences like +arrayWithObjects:, you need a Class to create any other Objective-C object.

I'd like to turn the above observation into a guideline to reduce the proliferation of Class globals: an object's class should only be used at the point of instantiation. Once you've made an object, pass it to the object that needs it, which should accept the most generic type available that declares the things that object needs. The generic type could be an abstract class like NSCoder or NSArray (notice that all array objects are created by the Foundation library, and application code never sees the specific class that was instantiated), or a protocol like UITableViewDataSource.

Now we've localised that pesky global variable to the smallest possible realm, where it can do the least damage.

#### Classes in objects in object-oriented programming in functional programming in Swift
So far, Objective-Swift objects have used prototypical inheritance, in which they supply some methods but also know about another object to which they can forward messages they don't understand themselves. This pattern is used in languages like Self, JavaScript and Io but is not common to other languages that also call themselves object-oriented programming languages.

Most OO languages have the idea of a "class" that supplies the methods for each object. An object knows that it "is a" specific realisation of its class, so when it receives a message it can look up the corresponding method in that class's method table.

Like this.

typealias Class = Dictionary

// define a "Non-Standalone" Object that relies on a class for its methods.

let NSObject : Class = [ "description": IMP.description({ _ in return "An NSObject" })]

func newObject(isa : Class) -&gt; Object {
  return { aSelector in
    if let anImplementation = isa[aSelector] {
      return anImplementation
    } else {
      return IMP.methodMissing({ _ in
        print("Does not recognize selector \(aSelector)")
        return nil
     })
    }
  }
}

let anObject = newObject(NSObject)
&#x1f4d3;(anObject) // "An NSObject"

Objects returned by the newObject() function look up their methods in their class dictionary, a table of method implementations for the corresponding selectors.

Not like that.

Mapping things to other things, that's a function, isn't it? And indeed there already is a type of function that takes selectors to methods, the object. A class should just be an object. Let's try that again.

typealias Class = Object

let NSObject : Class = { aSelector in
  switch aSelector {
  case "description":
    return IMP.description({ _ in return "An NSObject" })
  default:
    return IMP.methodMissing({ _ in
      print("Instance does not recognize selector \(aSelector)")
      return nil
    })
  }
}

func newObject(isa : Class) -&gt; Object {
  return { aSelector in
    return isa(aSelector)
  }
}

let anObject = newObject(NSObject)
&#x1f4d3;(anObject) // "An NSObject"

Great. Classes are objects, just as we're used to. And indeed, a class could get its own methods from another class: a metaclass.

Notice that class-based objects walk and quack like the earlier, prototypical objects, so interoperate completely. That's the point of message passing. No object needs to know how any other object works, just that it will respond to messages.

That's what makes all of the bridges that let other languages interoperate with Objective-C. As long as objc_msgSend() can find your code and run it, we're all good. In Objective-Swift, as long as passing you a selector gives me an IMP, we're all good.

Instance variables.

Methods defined by classes should be able to see the object's instance variables. The instance variables should, per their definition, be unique to each instance. However the best place to define them is in the class, for three reasons:

The class knows which instance variables its methods expect to be present.
The class's constructor function captured the values (or the initial values, if they're mutable) of the instance variables.
If I'm subclassing, I want to be able to add the subclass's instance variables to the superclass's, without having to redefine the whole shebang.

So a class should tell its objects what instance variables it needs, and the values. Now, how should an instance look up its instance variables? We need to go from the name of an instance variable to its value, and I'll save us both some embarrassment by cutting straight to the conclusion that an object is a great way to do this.

Each object in the class-based system will therefore be composed of two others: the class encapsulates what the object is and the methods it responds to, while the instance variable store encapsulates what the object is made of and what variables it has. Both are accessed through selectors, in a design choice that follows Eiffel and its Uniform-Access Principle (and the later Self language).

The same selector cannot refer both to a method and an ivar, because they're both in the same namespace. One of them has to win, and in this case I've chosen the instance variable lookup to win.

Here's a non-standalone version of Point, with its two instance variables. In the code shown here, o is a previously-defined object on which all methods are missing.

func NSPoint(x:Int, y:Int) -&gt; Class {
  let superclass = NSObject
  let ivars:Object = { variableName in
    switch variableName {
    case "x":
      return .method({_ in return Integer(x, proto: o)})
    case "y":
      return .method({_ in return Integer(y, proto: o)})
    default:
      return (superclass→"instanceVariables")!(variableName)
    }
  }
  let thisClass:Class = { aSelector in
    switch aSelector {
    case "instanceVariables":
      return .method({_ in return ivars})
    case "distanceFromOrigin":
      return .method({(this, _cmd, args:Object...) in
        let thisX = ℹ︎(this→"x")!
        let thisY = ℹ︎(this→"y")!
        let distance = sqrt(Double(thisX*thisX + thisY*thisY))
        return Integer(Int(distance), proto: o)
      })
    default:
      return superclass(aSelector)
    }
  }
  return thisClass
}

let aPoint = newObject(NSPoint(3,y: 4))
&#x1f4d3;(aPoint) // "An NSObject"
&#x1f4d3;(aPoint→"x") // "3"
&#x1f4d3;(aPoint→"distanceFromOrigin") // "5"

Super calls.

You can see that NSPoint is a subclass of NSObject, although this isn't amazingly useful. NSPoint gets all zero of NSObject's instance variables, and the not-particularly-descriptive description method.

In what might be the worst design choice of this blog, I'll say that a 3D point is a specialisation of a 2D point. I'll need to override distanceFromOrigin to calculate the distance in three dimensions, but I can use the result from the superclass's two-dimensional calculation to do this.

As with Objective-C, this will require a different method lookup system, one that says "ask your superclass" instead of "ask yourself". To do this, the object must know what its superclass is: it can ask its class. Here is a new method on NS3DPoint:

    case "superclass":
      return .method({ _ in return superclass })

Now, because I'm really running out of ideas on how to name these operators, here are some doubly-long operators that call into the superclass for an object:

infix operator .... {}

func .... (receiver: Object?, _cmd:Selector) -&gt; IMP? {
  guard let this = receiver else { return nil }
  let method = (this→"superclass")!(_cmd)
  switch(method) {
  case IMP.methodMissing(let f):
    return f(this, _cmd)...._cmd
  default:
    return method
  }
}

infix operator →→ {}

func →→ (receiver: Object?, _cmd:Selector) -&gt; Object? {
  guard let imp = receiver...._cmd else { return nil }
  switch imp {
  case .method(let f):
    return f(receiver!, _cmd)
  default:
    return nil
  }
}

Now, with all of this notation in place, it's possible to create a method NS3DPoint→distanceFromOrigin that both overrides its parent method, and uses that overridden method in its calculation. Here's the method:

    case "distanceFromOrigin":
      return .method({(this, _cmd, args:Object...) in
        let twoDDistance = ℹ︎(this→→"distanceFromOrigin")!
        let thisZ = ℹ︎(this→"z")!
        let distance = sqrt(Double(twoDDistance*twoDDistance + thisZ*thisZ))
        return Integer(Int(distance), proto: o)
      })

Here it is in use:

let anotherPoint = newObject(NS3DPoint(10, y: 12, z: 14))
&#x1f4d3;(anotherPoint→"distanceFromOrigin") // "20"

Conclusion

In many "classical" object-oriented programming systems, classes are just objects, which instances can use to find the methods they respond to. If you've already got objects that don't use classes, it's possible to make objects that do use classes by giving each object a class object. You need somewhere to hold the instance variables for an object, and it happens that objects are good for that too.

More useful to notice is that objects are really just a way to do late binding of functions to their names. It doesn't really matter what rules you use, as long as the result is a map of names to functions you've got yourself an object that will work with other objects.

#### Classism in software engineering
I just heard someone using the phrase "first-class citizen" in a programming podcast, and that led me to ponder the use of that phrase. The podcast was Swift Package Manager SuperPowers from Empower Apps. Empower's a great podcast, this is a great episode, and the idea of first-class citizenship comes up only in passing and is basically immaterial. But not to this post!

Whatever the situation that leads someone to say "first-class citizen", there's a value judgement being made: this thing is good, because it is more conformant to the (probably unspoken) rules of the road of the ecosystem, platform, or whatever the thing is supposed to be a citizen of.

Many of us in the privileged software engineering world live in societies that do not have overt "levels" of citizenship. That said, there still are multiple levels: nationals, resident aliens, temporary visitors, and prisoners may all have different rights and responsibilities. And in societies where there are still explicit or tacit class hierarchies, making reference to them is often somewhere between impolite and offensive. So this idea of "first-class citizenship" comes with a big side-wink: it's a first-class citizen, we both know what I mean.

An obvious way for a technology to be a first-class citizen of something is for it to be made, distributed, or otherwise blessed by the maker of the something it's a citizen of. That's the context in this show: Swift Package Manager is a first-class citizen of the Apple software development platform because it's a first-party component. Now, it's a first-party component with the job of giving access to third-party components, so there's a limit to the vendor ownership, but nonetheless it is there.

In this sense, first-class citizenship confers clear benefits. If someone is already playing in the Apple tools sandpit, they already have access to SwiftPM. They may not already have access to CocoaPods, so the one step "fetch all the packages" becomes two steps: fetch the package tool, then fetch all the packages.

That bit is easier, but it evidently isn't sufficient. Is the other tool better at fetching packages correctly, or better for writing packages, or more secure, or easier to use? When we say "better", better at what, and for whom?

It's possible for something that is first-party to not be first-class. Mac OS X came with the Tcl language for a couple of decades but I can't find evidence online that it was ever referred to as a "first-class citizen" of the Apple ecosystem. In 2022 you wouldn't call OpenDoc or the Macintosh Runtime for Java first-class citizens either, because the vendor no longer supports them. Actually it'd be an interesting exercise to take an old Apple Developer Connection CD (the earliest I have will be from around 2005), and find out how much of that content is still supported, and of that subset how much you could get broad agreement for the first-class nature of. I'd be willing to bet that even though ObjC is still around, distributed, supported, and developed, a decent chunk of the community would think twice about calling it first-class.

But then, it's also possible for things that are third-party to be first-class. Apparently, Java is a first-class citizen in a Docker ecosystem now. And Data should be a first-class citizen in the Enterprise (this is, by the way, a spoiler for the Star Trek: The Next Generation episode the measure of a man).

When third-party things are first-class, we're saying that going the extra step of getting this thing you don't already have is better than stopping here and using what you already own. Again we have the questions of better at what and for whom. Really any of this stuff lies on a continuum. Consider a database. You use a database because it's cheaper, faster, and better to turn your stuff into the database's model and use the database vendor's structures and algorithms than it is to design your own efficient storage and retrieval format. If you could do that well (and that's a big if) then your hand-rolled thing would probably be better for your application than the database, but also all maintenance work falls onto a very small community: you. On the other hand you could use whatever comes in the box, which has the exact opposite characteristics. They each have different types of first-classness. 

And then there's a (very old, but very common) definition of a data type being first-class or not depending on whether they can be represented by variables or expressions. So when Javascript developers say "functions are first-class citizens in Javascript", they mean that JS has a feature that ALGOL did not.

#### Cleaner Code
Readers of OOP the easy way will be familiar with the distinction between object-oriented programming and procedural programming. You will have read, in that book, about how what we claim is OOP in the sentence "OOP has failed" is actually procedural programming: imperative code that you could write in Pascal or C, with the word "class" used to introduce modularity.

Here's an example of procedural-masquerading-as-OOP, from Robert C. Martin's blog post FP vs. OO List Processing:
void updateHits(World world){
  nextShot:
  for (shot : world.shots) {
    for (klingon : world.klingons) {
      if (distance(shot, klingon) &lt;= type.proximity) {
        world.shots.remove(shot);
        world.explosions.add(new Explosion(shot));
        klingon.hits.add(new Hit(shot));
        break nextShot;
      }
    }
  }
}

The first clue that this is a procedure, not a method, is that it isn't attached to an object. The first change on the road to object-orientation is to make this a method. Its parameter is an instance of World, so maybe it wants to live there.
public class World {
  //...

  public void updateHits(){
    nextShot:
    for (Shot shot : this.shots) {
      for (Klingon klingon : this.klingons) {
        if (distance(shot, klingon) &lt;= type.getProximity()) {
          this.shots.remove(shot);
          this.explosions.add(new Explosion(shot));
          klingon.hits.add(new Hit(shot));
          break nextShot;
        }
      }
    }
  }
}

The next non-object-oriented feature is this free distance procedure floating about in the global namespace. Let's give the Shot the responsibility of knowing how its proximity fuze works, and the World the knowledge of where the Klingons are.
public class World {
  //...

  private Set&lt;Klingon&gt; klingonsWithin(Region influence) {
    //...
  }

  public void updateHits(){
    for (Shot shot : this.shots) {
      for (Klingon klingon : this.klingonsWithin(shot.getProximity())) {
        this.shots.remove(shot);
        this.explosions.add(new Explosion(shot));
        klingon.hits.add(new Hit(shot));
      }
    }
  }
}

Cool, we've got rid of that spaghetti code label ("That’s the first time I’ve ever been tempted to use one of those" says Martin). Incidentally, we've also turned "loop over all shots and all Klingons" to "loop over all shots and nearby Klingons". The World can maintain an index of the Klingons by location using a k-dimensional tree then searching for nearby Klingons is logarithmic in number of Klingons, not linear.

By the way, was it weird that a Shot would hit whichever Klingon we found first near it, then disappear, without damaging other Klingons? That's not how Explosions work, I don't think. As it stands, we now have a related problem: a Shot will disappear n times if it hits n Klingons. I'll leave that as it is, carry on tidying up, and make a note to ask someone what should really happen when we've discovered the correct abstractions. We may want to make removing a Shot an idempotent operation, so that we can damage multiple Klingons and only end up with a Shot being removed once.

There's a Law of Demeter violation, in that the World knows how a Klingon copes with being hit. This unreasonably couples the implementations of these two classes, so let's make it our responsibility to tell the Klingon that it was hit.
public class World {
  //...

  private Set&lt;Klingon&gt; klingonsWithin(Region influence) {
    //...
  }

  public void updateHits(){
    for (Shot shot : this.shots) {
      for (Klingon klingon : this.klingonsWithin(shot.getProximity())) {
        this.shots.remove(shot);
        this.explosions.add(new Explosion(shot));
        klingon.hit(shot);
      }
    }
  }
}

No, better idea! Let's make the Shot hit the Klingon. Also, make the Shot responsible for knowing whether it disappeared (how many episodes of Star Trek are there where photon torpedoes get stuck in the hull of a ship?), and whether/how it explodes. Now we will be in a position to deal with the question we had earlier, because we can ask it in the domain language: "when a Shot might hit multiple Klingons, what happens?". But I have a new question: does a Shot hit a Klingon, or does a Shot explode and the Explosion hit the Klingon? I hope this starship has a business analyst among its complement!

We end up with this World:
public class World {
  //...

  public void updateHits(){
    for (Shot shot : this.shots) {
      for (Klingon klingon : this.klingonsWithin(shot.getProximity())) {
        shot.hit(klingon);
      }
    }
  }
}

But didn't I say that the shot understood the workings of its proximity fuze? Maybe it should search the World for nearby targets.
public class World {
  //...

  public void updateHits(){
    for (Shot shot : this.shots) {
      shot.hitNearbyTargets();
    }
  }
}

As described in the book, OOP is not about adding the word "class" to procedural code. It's a different way of working, in which you think about the entities you need to model to solve your problem, and give them agency. Obviously the idea of "clean code" is subjective, so I leave it to you to decide whether the end state of this method is "cleaner" than the initial state. I'm happy with one fewer loop, no conditions, and no Demeter-breaking coupling. But I'm also happy that the "OO" example is now object-oriented. It's now looking a lot less like enterprise software, and a lot more like Enterprise software.

#### Clown Trousers
An indirect side effect of stopping programming is that none of my trousers fit any more.

People who like to explain things before they have all the facts (or "programmers" as we sometimes call them) will justify this observation by pointing out that I have more time for exercise now. I do, but I don't use it. While working at Facebook I walked six miles each day as part of my commute and worked at a variable-height desk; I spent a lot of time walking and standing.

When I began my gap year, I put some effort into running every day. That didn't last long. I still stand a lot to play musical instruments, but am significantly less active now that I'm 8kg lighter than programmer Graham.

Looking at videos of programmer me, I just see an obese, tired guy surviving on caffeine, sugar snacks and three big meals a day as he lurched between commuting trips, flights abroad, conference talks and infrequent visits to bed. Peak Graham (weight for weight) came in June, as attested by the video of my AltConf talk, I have no idea what I'm doing.

It turns out I had no idea what I was doing to myself either. But now that I'm not doing it, the historical record that is my wardrobe tells me I'm healthier than I have been in over five years.

You may not need hipster silicon valley nutritional engineering sludge. You may not need an extra hour in the day to fit in a run and a shower. You may not need to drop a few hundred quid on a watch that also reminds you to stand up. You might just need to discover what you're doing wrong, and not do it like that.

#### Cocoa Memory Management
It becomes evident, thanks to the mass centralisation of the neverending september effect that is stackoverflow, that despite the large number of electrons expended on documenting the retain/release/autorelease reference counting mechanism for managing memory in Cocoa, Cocoa Touch, UIKit, AppKit, Foundation, GNUstep, Cocotron and Objective-C code, very few people are reading that. My purpose in this post is not to re-state anything which has already been said. My purpose is to aggregate information I've found on the topic of managing memory in Cocoa, so I can quote this post in answers to questions like these.In fact, I've already answered this question myself, as How does reference counting work? As mentioned in the FAQ, I actually answered the question "how do I manage object lifecycles in (Cocoa|GNUstep|Cocotron)"? It's actually a very violently distilled discussion, so it's definitely worth checking out the references (sorry) below.Apple have a very good, and complete, Memory Management Programming Guide for Cocoa. They also provide a Garbage Collection Programming Guide; remember that Objective-C garbage collection is opt-in on 10.5 and above (and unavailable on iPhone OS or earlier versions of Mac OS X). GNUsteppers reading along should remember that the garbage collector available with the GNU objc runtime is entirely unlike the collector documented in Apple's guide. GNUstep documentation contains a similar guide to memory management, as well as going into more depth about memory allocation and zones. Apple will also tell you how objects in NIBs are managed.The article which gave me my personal eureka moment was Hold Me, Use Me, Free Me by Don Yacktman. Stepwise has another article, very simple rules for memory management in Cocoa by mmalc, which is a good introduction though with one caveat. While the table of memory management methods at the top of the article are indeed accurate, they might give you the impression that keeping track of the retain count is what you're supposed to be doing. It's not :). What you're supposed to be doing is balancing your own use of the methods for any given object, as described in rules 1 and 2 of "Retention Count rules" just below that table.James Duncan Davidson's book "Learning Cocoa with Objective-C" has not been updated in donkey's years, but its section on memory management is quite good, especially the diagrams and the "rules of thumb" summary. Luckily, that section on memory management is the free sample on O'Reilly's website.If reading the theoretical stuff is all a bit too dry, the Mac Developer Network have a rather comprehensive memory management training video which is $9.99 for non-MDN members and free for paid-up members.Finally, Chris Hanson has written a good article on interactions between Cocoa memory management and objc-exceptions; if you're using exceptions this is a good discussion of the caveats you might meet.

#### Cocoa#, Mono and Me
My great application:

Yeah, OK, not so great.  But this Inverse Hoffman is the result of a couple of hours hacking in Mono, with Cocoa#.  My app's largely based on the Stupid Word Counter tutorial, though it's a from-scratch implementation of the famous Apple/NeXT sample application in C#.

Firstly, a little history.  My first encounter with .NET was back in about 2003 at a Microsoft Developer Roadshow in the car park (and later lecture theatre) in Oxford's comlab.  I was particularly interested in their discussions of cross-platform capability, Project Rotor (I kept in touch with one of the Rotor developers) and so on, but really didn't see much exciting in .NET.  Nonetheless, being a fair man, I took my beta CDs of Windows 2003 and Visual Studio .NET and gave them a whirl.  Unfortunately, still not much interesting.  Largely due to buggy betas and a lack of beta documentation.

Now accessing Cocoa from non-ObjC languages is nothing new, we've been doing it from Perl, Python, Ruby and Java for ages and partcularly old farts might even remember Objective-Tcl.  Why should I care about Cocoa#?  Well for a start, there are likely to be a lot Windows developers out there with some (language that boils down to MS IL eventually) skills who are wanting to produce Mac applications, and it'd be interesting to see what we'll end up with when they do.  And it's always fun to learn a new language, anyway ;-)

Good points

Real NIB files.  No really, that interface is genuinely an IB 3.x NIB based on the Cocoa Application template.  The objects inside it are Objective-C objects, and there are good old outlets and actions (I haven't yet investigated whether Cocoa Bindings would work).macpack.  A command-line tool which takes your IL executable and wraps it up in a Cocoa application bundle, ready for drag-deployment.Good inline bridging information.  Unlike, say, PyObjC the language bridge isn't completely dynamic, but unlike the Java bridge or JIGS you don't have to keep a separate manual mapping of real classes onto ObjC shams. For instance, here's the controller from Currency Converter.net, complete with class, ivar and method exports:

namespace info.thaesofereode.CurrencyConverter{ [Cocoa.Register("CurrencyConverterController")]   public partial class CurrencyConverterController : Cocoa.Object   {     public CurrencyConverterController(System.IntPtr native_object) : base(native_object) {}        //Cocoa IBOutlets     [Cocoa.Connect]       private Cocoa.TextField inputCurrency;        [Cocoa.Connect]       private Cocoa.TextField outputCurrency;       [Cocoa.Connect]       private Cocoa.TextField conversionRate;     //Cocoa IBAction      [Cocoa.Export("calculate:")]      public void calculate(Cocoa.Object sender)        {         //get the rate from the view          System.String rate = conversionRate.Value;            CurrencyConverterModel.Rate = System.Convert.ToDouble(rate);          //get the currency            System.String input = inputCurrency.Value;            System.Double output = CurrencyConverterModel.convert(System.Convert.ToDouble(input));            //update the UI           outputCurrency.Value = System.Convert.ToString(output);       } }}

Bad points

Not very Cocoa-like wrapper classes.  I think this is deliberate; they've gone for making the Cocoa shim look like a .NET interface because after all, we're programming from .NET.  This is a bit disappointing as I'm more familiar with PyObjC and the Perl-ObjC-Bridge where the APIs are left pointedly alone, but given the target audience of Cocoa# it's unsurprising.MonoDevelop.  Luckily, using it isn't mandated.

So, overall, one more good point than bad (and a tentative two, if you overlook MonoDevelop); a pretty good initial evaluation and I might give this a deeper scrape.

#### Cocoa: Model, View, Chuvmey
Chuvmey is a Klingon word meaning "leftovers" - it was the only way I could think of to keep the MVC abbreviation while impressing upon you, my gentle reader, the idea that what is often considered the Controller layer actually becomes a "Stuff" layer. Before explaining this idea, I'll point out that my thought processes were set in motion by listening to the latest Mac Developer Roundtable (iTunes link) podcast on code re-use.

My thesis is that the View layer contains Controller-ey stuff, and so does the Model layer, so the bit in between becomes full of multiple things; the traditional OpenStep-style "glue" or "shuttle" code which is what the NeXT documentation meant by Controller, dynamic aspects of the model which could be part of the Model layer, view customisation which could really be part of the View layer, and anything which either doesn't or we don't notice could fit elsewhere. Let me explain.

The traditional source for the MVC paradigm is Smalltalk, and indeed How to use Model-View-Controller is a somewhat legendary paper in the use of MVC in the Smalltalk environment. What we notice here is that the Controller is defined as:

The controller interprets the mouse and keyboard inputs from the user, commanding the model and/or the view to change as appropriate.

We can throw this view out straight away when talking about Cocoa, as keyboard and mouse events are handled by NSResponder, which is the superclass of NSView. That's right, the Smalltalk Controller and View are really wrapped together in the AppKit, both being part of the View. Many NSView subclasses handle events in some reasonable manner, allowing delegates to decorate this at key points in the interaction; some of the handlers are fairly complex like NSText. Often those decorators are written as Controller code (though not always; the Core Animation -animator proxies are really controller decorators, but all of the custom animations are implemented in NSView subclasses). Then there's the target-action mechanism for triggering events; those events typically exist in the Controller. But should they?

Going back to that Smalltalk paper, let's look at the Model:

The model manages the behavior and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller).

If the behaviour - i.e. the use cases - are implemented in the Model, well where does that leave the Controller? Incidentally, I agree with and try to use this behavior-and-data definition of the Model, unlike paradigms such as Presentation-Abstraction-Control where the Abstraction layer really only deals with entities, with the dynamic behaviour being in services encapsulated in the Control layer. All of the user interaction is in the View, and all of the user workflow is in the Model. So what's left?

There are basically two things left for our application to do, but they're both implementations of the same pattern - Adaptor. On the one hand, there's preparing the Model objects to be suitable for presentation by the View. In Cocoa Bindings, Apple even use the class names - NSObjectController and so on - as a hint as to which layer this belongs in. I include in this "presentation adaptor" part of the Controller all those traditional data preparation schemes such as UITableView data sources. The other is adapting the actions etc. of the View onto the Model - i.e. isolating the Model from the AppKit, UIKit, WebObjects or whatever environment it happens to be running in. Even if you're only writing Mac applications, that can be a useful isolation; let's say I'm writing a Recipe application (for whatever reason - I'm not, BTW, for any managers who read this drivel). Views such as NSButton or NSTextField are suitable for any old Cocoa application, and Models such as GLRecipe are suitable for any old Recipe application. But as soon as they need to know about each other, the classes are restricted to the intersection of that set - Cocoa Recipe applications. The question of whether I write a WebObjects Recipes app in the future depends on business drivers, so I could presumably come up with some likelihood that I'm going to need to cross that bridge (actually, the bridge has been deprecated, chortle). But other environments for the Model to exist in don't need to be new products - the unit test framework counts. And isn't AppleScript really a View which drives the Model through some form of Adaptor? What about Automator…?

So let me finish by re-capping on what I think the Controller layer is. It's definitely an adaptor between Views and Models. But depending on who you ask and what software you're looking at, it could also be a decorator for some custom view behaviour, and maybe a service for managing the dynamic state of some model entities. To what extent that matters depends on whether it gets in the way of effectively writing the software you need to write.

#### CocoaHeads Swindon
Just got back from the first meeting of Swindon CocoaHeads, featuring a bunch of people who live nowhere near Swindon, some good beer and the occasional discussion of Cocoa.  Special mention to Scott who came all the way from sunny Warsaw to be with us!Tonight's event was an informal, "what do we want from Swindon CocoaHeads?" event, but it looks like being successful enough that we'll be doing it again.  The format will be a presentation or directed conversation, followed by general chit-chat about all things Cocoa.  In fact, um, I may have volunteered to give the first presentation at the next meeting.  The subject is: well, that would be telling, wouldn't it… ;-).  You'll have to find out by coming along to the Glue Pot in Swindon at 8pm on Monday, August 4th.  Look out for further announcements and a mailing list over at Scotty's place in the forthcoming month!

#### CocoaHeads Swindon is this Monday!
The town of Swindon in the Kingsbridge hundred, Wiltshire is famous for two things. The first is the Wilts and Berks Canal, linking the Kennet and Avon at Trowbridge with the Thames at Abingdon. Authorised by act of parliament in 1775, the canal first passed through the town in 1804 and allowed an explosion in both the industrial and residential capacity of the hitherto quiet market cheaping.The second is, of course, the local CocoaHeads chapter. Founded by act of Scotty in 2007, Swindon CocoaHeads quickly brought about a revolution in the teaching and discussion of Mac and iPhone development in the South-West, its influence being felt as far away as Swansea to the West and London to the East. Unlike the W&amp;B canal, Swindon CocoaHeads is still thriving to this day. On Monday, 7th September at 20:00 there will be another of the chapter's monthly meetings, in the Glue Pot pub near the train station. Here, Pieter Omvlee will be leading a talk on ImageKit, and the usual combination of beer and Cocoa chat will also be on show. As always, the CocoaHeads website contains the details.

#### CocoaHeads Swindon tonight!
For those of you who've never explored the delights that the fine city of the Hill of Pigs has to offer, tonight offers an unparalleled opportunity. Come and sit in (or outside, weather permitting) a pub only a short distance from the railway station, and listen to Mike Abdullah speaking about WebKit. As always there'll also be general NSDiscussion, and the occasional pint of beer. Maps etc. at our cocoaheads.org page.

#### CocoaHeads Swindon, January and February
The next CocoaHeads Swindon will take place on 4th January, at the Glue Pot in Swindon. Get here at 8 for some NSChitChat with your (well, my) local Mac developer community.There is no February meeting of Swindon CocoaHeads, on account of NSConference Europe taking place in Reading on that weekend. So buy your NSConference ticket and come along to say hi!

#### Code longevity
I recently wrote about the impending centenary of applied computing; a time when we could reflect on the first hundred years to make it easier for people to progress beyond our position into the second hundred years. This necessitates looking at the things we've tried, the things that succeeded and the things that failed. It involves recalling and describing the good ideas and the bad ideas.

So, did the bad ideas fail and the good ideas succeed? Can we declare that because something worked, it must have been a success? Is length of service a great proxy for quality of principle?

Let's start by looking at the lifetime of some of the trappings of applied computing. I'm writing this on the smartphone shown in the picture below. It is, among the many computers I own that claim to be computers and could reasonably be described as modern, one of only two that is not running a recent variant of a minicomputer game–loading system.

Now is that a fair assessment? Certainly all the Macs, iOSes, Androids (and even routers and television streamy box things) in the house are based on Unix, and Unix is the thing of the 1970s minicomputer. I've even used that idea to explain why we still have to deal with PDP-8 problems in iPhones. But is it fair to assume that because the name has lasted, then the idea has been preserved? Did Unix succeed, or has it been replaced by different things with the same name? That happens a lot; is today’s ethernet really the same ethernet that Bob Metcalfe and colleagues at PARC invented? Conversely, just because the name changed is everything new? Does Windows NT really represent a clean break in 1993?

There’s certainly some core, a kernel (f’nar) of the modern Unix that, whether in code or philosophy, can be traced back to the original system (and indeed beyond). But is that there because it’s still a good idea, or because there’s no impetus to remove it? Or even because it’s a bad idea, but removing it would be expensive?

As we're already talking about Unix, let’s talk about C. In his talk Null References: The Billion-Dollar Mistake, Tony Hoare describes his own mistake as being the introduction of a null reference. He then says that C’s mistake (C follows Algol in having null references, but it also lacks have subscript bounds checking) is an order of magnitude worse. In fact, Hoare also identified a third problem: he says that it’s a good idea to permit a program failure to be diagnosed just from the error message and the high-level program source text. However, runtime failures in C usually end up with a core dump and/or a stack trace through the instructions of the target machine environment.

We can easily wonder just how much (expensive) programmer time has been lost disassembling stack traces, matching up debugger symbols and interpreting core dumps, but without figures for that I’ll generously assume that it’s an order of magnitude smaller than the losses due to buffer overflows. Now that’s only a tens-of-billions-of-dollars value of mistake, and C is the substrate for trillions of dollars of value of industry. So do we say that on balance, C is 99% a Good Thing™? Is it a bad idea that nonetheless enabled plenty of good ones?

[Incidentally, and without wanting to derail the central thesis of this post, I disagree with Hoare’s numbers. Symantec is merely one of the largest companies in the information security sector, with annual revenue in their most recent report of $6.9B. That’s a small part of the total value sunk into that sector, which I’ll guess has an annual magnitude of multiple tens of billions. A large fraction of the problems addressed by infosec can be attributed to C’s lack of bounds checking, so that there’s probably just an annual impact of around ten billion dollars working on fixing the problem. Assuming those businesses have sustainable revenues over multiple years, the integrated cost is well into the hundreds of billions. That only revises the estimated impact on the C software industry from ‘fractions of a per cent’ to ‘a per cent’ though.]

Perhaps it’s fair to say that C was a good idea when it arose, and that it’s since been found to have deficiencies that haven’t yet become expensive enough to warrant decommissioning it. There’s an assumption of rational action in there that I think it’s fair to question, though: am I assuming that C is not worth replacing just because it has not been replaced? Might there actually be other factors involved?

Yes, there might. It’s possible that there are organisations out there for whom C is more expensive than its worth, but where the sunk cost fallacy stops them from moving on. Or organisations who stick with C because their platform vendor gives them a C toolset, even where free or paid alternatives would be cheaper [in fact that would point to a difficulty with any holistic evaluation: that the cost to the people who provide development environments and the cost to the people who consume development environments depends on different factors, and the power in the market is biased towards a few large providers. Welcome to economics]. Or organisations who stick with C because of a perception of a large community of users, which is (perceived to be) more useful than striking out alone with better tools.

It’s also possible that moves in the other direction are based on non-rational factors: organisations that seek novelty rather than improvement, or who move away from C because a vendor convinces them that their alternative is better regardless of objective truth.

It turns out that the simple question we wanted to ask about applied computing: “What works?” leads to such a complex and maybe even chaotic system of forces acting in multiple dimensions that answering it will be very difficult. This doesn’t mean that an answer should not be sought, but that finding the answer will combine expertise from many different fields. Particularly, something that survives for a long time doesn’t necessarily work: it could just be that people are afraid of the alternatives, or haven't really considered them.

#### Code snippit from NSConference presentation
Here's the code I used to display the code signature status within the sample app at NSConference. You need to be using the 10.6 SDK, and link against Security.framework.

#import &lt;Security/SecCode.h&gt;
- (void)updateSignatureStatus {
    SecCodeRef myCode = NULL;
    OSStatus secReturn = SecCodeCopySelf(kSecCSDefaultFlags, &myCode);
    if (noErr != secReturn) {
        [statusField setIntValue: secReturn];
        return;
    }
    CFMakeCollectable(myCode);
    SecRequirementRef designated = NULL;
    secReturn = SecCodeCopyDesignatedRequirement(myCode,
                                                 kSecCSDefaultFlags,
                                                 &designated);
    if (noErr != secReturn) {
        [statusField setIntValue: secReturn];
        return;
    }
    CFMakeCollectable(designated);
    secReturn = SecCodeCheckValidity(myCode,
                                     kSecCSDefaultFlags,
                                     designated);
    [statusField setIntValue: secReturn];
}

That's all there is to it. As discussed in my talk, the designated requirement is a requirement baked into the signature seal when the app is signed in Xcode. Application identity is verified by testing whether:
the signature is valid;
the seal is unbroken;
the code satisfies its designated requirement.

Of course there's nothing to stop you testing code against any requirement you may come up with: the designated requirement just provides "I am the same application that my developer signed".

#### Coding. Standards.
I just realised that this month marks the 10th anniversary of my first payment for writing software (on, of all the weird things to be writing software on in 2002, a NeXTstation)! What have I learned from those ten years? What advice would I give to someone who wants to do this stuff for at least 10 years?

Programming is the easy bit.

Well, comparatively. There are hard bits in programming, and every few years a new paradigm comes along that means you have to unlearn whatever it was you were doing and learn something else to do anyway. So learning programming is never done, but still programming is easier than:

estimating. The one project I've worked on that finished on its planned completion date only did so by accident.
getting any kind of agreement out of two or more people.
accepting that the other person isn't a dick, but has different goals and problems than you.
objectively evaluating your own work.
objectively assessing someone else's evaluation of your work.
stopping programming when you're done.

You always need to be learning.

You can't compete on price in the software market, because there's always some student somewhere who's willing to do the same work for free. It was Mike who first taught me that. You have to compete on quality, which means you need to strive to improve your own quality. Because other people are too, so you need to run just to stand still.

There are various ways to learn, and they're not mutually exclusive. A combination of books/articles, experimentation, and discussion with peers is valuable. If your town has an Appsterdam or a CocoaHeads, get along and say hello.

You probably don't want to be doing this in 10 years.

I was actually a UNIX programmer a decade ago (well, I was mainly a student). Then I was a barman, then sysadmin, then a Linux server application programmer, then a Mac app programmer, then a contract Mac programmer, then a Java app programmer, then a security consultant, then an iOS app programmer. There's only a small probability that I'll be an iOS app programmer in 10 years.

This world moves really quickly. Ten years ago, the iPod was a new and relatively risky proposition. Macs used PowerPC CPUs. Windows XP was the new hotness, and .NET was just about to appear - meanwhile Mac OS X was a sluggish amalgam of NeXT, Java and legacy code. Java, by the way, was run by a now-defunct company called Sun Microsystems, which was trying to work out how to survive the dot-com crash.

Speaking of the dot-com crash, it seems highly likely that within the next decade we'll see the dot-app crash. App downloads are worth $0.18¢ each, but an app costs $200k - apparently it's hard work. That means you've got to either get yourself into the long tail value-wise (i.e. have a very good app that people will pay for), or you've got to find a million users for version 1.0.

For everyone else, the market isn't worth staying in long-term. The market will bore of brochureware apps, only a few high-value brands will be able to support unprofitable vanity apps, and VCs will realise that throwing their money after an app with no profit strategy is the same as throwing their money after a website with no profit strategy.

It's likely that at least one of the companies that's big in the current software world - Microsoft, Apple, Oracle, Google and the like - will be big in the software world of 2022. It's also likely that there'll be some new comers that change things completely: Facebook and Twitter didn't exist ten years ago, and neither did Android, Inc. Sometimes companies that seem to be in an interminable tailspin - like Apple - turn themselves around and become successful.

Learn more than one thing

This is related, in part, to what came above: the thing you're using right now may not exist, or may be hard to get work in, in a few years' time. On the other hand, some things seem to outlive the cockroaches: C - and by extension, languages that can link somewhat seamlessly with C like C++, Fortran and so on - have been going on forever. It can be hard to predict which of these camps your favourite tech sits in, so learning more than one thing keeps you employable.

More than that, if your technology of choice comes from a single supplier (e.g. Microsoft, Apple, Embarcadero) then diversification just makes good business sense. This particularly applies in the age of the app store where that sole supplier can also be your sole vendor - you don't want to sign your entire business's value over to one other company.

Learning another thing makes you better at the first thing
This is another reason why diversifying your technology portfolio is beneficial. Many of the changes I've made recently in the way I write object-oriented software come from talking to Clojure programmers.

The more different things you know, the more connections you'll be able to make between them. The more you'll be able to critically analyse one technology, beyond what the vendor tells you. And the more you'll be able to understand other new things and incorporate them into your Weltanschauung.

Conclusion

My summary could be "learn whatever you can: you never know which bits you need". Or it could be "don't rely on your supplier to solve all of your problems".

I think it's actually going to be: analyse everything. Reflect on your work: what went well? What didn't? Could you have done things better? If you don't think you could have, then you're probably wrong: what would you need to know to identify the bit that actually could've gone better?

But know when to stop, too. Analysis paralysis is as much of a problem as going in blind. At some point, you need to suck it up and move on. Trading these two things against each other is the real difficulty in software engineering.

#### Come back, purple button, all is forgiven!
As a great philosopher once wrote: don't it always seem to go, that you don't know what you got 'til it's gone?  Previews of Mac OS X had a user interface feature, known by all who saw it as the Purple Button.  Look at this screenshot from System Preferences:

The boiled sweet on the top-right of the window would go purple, hence the name.  Clicking on it activated a single-window mode.  All documents except the one that you were working on would be minimised into the Dock, and switching between them would minimise the earlier one before restoring the newly-focused document.  Of course, the problem with this in the developer previews/public beta which rendered it unusable were performance-related.  The "lickable" eye-candy in Aqua was ambitious even on the top-end G4 systems available at the time, and so time spent in the Genie or Scale effects was really noticable.  Add to that the effect of applications being slow enough not to update their views in time - the System Preferences application you can see above is a Cocoa-Java app, and back then the JVM wasn't amazing for performance - and you have a really sucky single-window experience.

On the other hand, it's really bloody useful.  Look at apps like WriteRoom or GLTerminal, which go out of their way to get rid of all that other clutter.  Or Spaces (or CDE virtual desktops, WindowMaker virtual desktops... you get the idea), also designed to let you forget all those other apps are there.  Well, spaces is quite nice (and a little more flexible than purple button was), but playing spaces ping-pong tends to make me a bit seasick.  Not to mention the time it wastes being about as great as the unperformant purple button switching...so please, purple button, come back!

Some environments provided the same user experience out of a lack of choice - for instance, OZ couldn't show more than one application if it wanted to, and certainly running more than one at once was out of the question (it would simulate multi-tasking by suspending background tasks).

#### Coming to terms with fewer terms
I was on a "Leadership in Architecture" panel organised by RP International recently, and was asked about problems we face using new techniques like Microservices, serverless and machine learning in the financial technology sector. The biggest blocker I see is the RFP (Request for Proposals), RFI (Request for Information), the MSA (Master Service Agreement), any document with a three-letter acronym. We would do better if they disappeared.

I'm fully paid up in my tithe to the church of "customer collaboration over contract negotiation", and I believe that this needs to extend beyond the company boundary. If we're going to spend a few months going back and forth over waving our certifications about, deciding who gets to contact whom within what time, and whether the question they asked constitutes a "bug report" or a "feature request", then I don't think it matters whether the development team use two-week sprints or not. We've already lost.

We've lost because we know that the interactions between the people involved are going to be restricted to the terms agreed during that negotiation. No longer are people concerned about whether the thing we're making is valuable; they're concerned with making sure their professional indemnity insurance is up to date before sending an email to the DRI (Definitely Responsibility-free Inbox).

We've lost because we had a team sitting on its hands during the negotiation, and used that time "productively" by designing the product, putting epics and stories in a backlog, grooming that backlog, making wireframes, and all of those other things that aren't working software.

We've lost because each incompatibility between the expectation and our intention is a chance to put even more contract negotiation in place, instead of getting on with making the working software. When your RFI asks which firewall ports you need to open into your DMZ, and our answer is none because the software runs outside of your network on a cloud platform, we're not going to get into discussions of continuous delivery and whether we both read the Phoenix Project. We're going to get into discussions of whether I personally will warrant against Amazon outages. But here's the thing: we don't need the software to be 100% up yet, we don't even know whether it's useful yet.

Here's an alternative.

We, collectively, notice that the software we make solves the problem you have.
We, collectively, agree that you can use the software we have now for a couple of weeks.
We, collectively, discuss the things that would make the software better at solving the problem.
We, collectively, get those things done.
We, collectively, GO TO 2.

Notice that you may have to pay for steps 2-4.

#### Coming very shortly...
This website will be the new home for information on Cocoa and Mac OS X security. But not yet! Please check back soon; in the mean time take a look at my homepage.

Graham.

#### Comment word verification
Sorry, but I was finally pushed into turning magic word input on for comments on this blog.  Anyone may still comment, but you need to decode a picture in order to do so.

#### Common sense writ large
Looking at the bottom of Apple's Q3 results, as indeed with any similar publication from a US publicly-traded company, we see the following text.This press release contains forward-looking statements including without limitation those about the Company’s estimated revenue and earnings per share. These statements involve risks and uncertainties, and actual results may differ. Risks and uncertainties include without limitation potential litigation from the matters investigated by the special committee of the board of directors and the restatement of the Company’s consolidated financial statements; unfavorable results of other legal proceedings; the effect of competitive and economic factors, and the Company’s reaction to those factors, on consumer and business buying decisions with respect to the Company’s products; war, terrorism, public health issues, and other circumstances that could disrupt supply, delivery, or demand of products; continued competitive pressures in the marketplace; the Company’s reliance on sole service providers for iPhone in certain countries; the continued availability on acceptable terms of certain components and services essential to the Company’s business currently obtained by the Company from sole or limited sources; the ability of the Company to deliver to the marketplace and stimulate customer demand for new programs, products, and technological innovations on a timely basis; the effect that product transitions, changes in product pricing or mix, and/or increases in component costs could have on the Company’s gross margin; the effect that product quality problems could have on the Company’s sales and operating profits; the inventory risk associated with the Company’s need to order or commit to order product components in advance of customer orders; the effect that the Company’s dependency on manufacturing and logistics services provided by third parties may have on the quality, quantity or cost of products manufactured or services rendered; the Company’s dependency on the performance of distributors and other resellers of the Company’s products; the Company’s reliance on the availability of third-party digital content; and the potential impact of a finding that the Company has infringed on the intellectual property rights of others. More information on potential factors that could affect the Company’s financial results is included from time to time in the Company’s public reports filed with the SEC, including the Company’s Form 10-K for the fiscal year ended September 29, 2007; its Forms 10-Q for the quarters ended December 29, 2007 and March 29, 2008; and its Form 10-Q for the quarter ended June 28, 2008, to be filed with the SEC. The Company assumes no obligation to update any forward-looking statements or information, which speak as of their respective dates.Erm, like, duh.  Stuff which we say might happen in the future, might not actually happen.  Really?  You've got to get out of the financial industry, there's a lucrative career ahead of you in construction.

#### Comparing Objective-C and Objective-C with Objective-C
A while back, I wrote an object-oriented dispatch system for Objective-C. It defines only three things: an object type (the BlockObject), a way to create new objects (the BlockConstructor), and a way to message objects (the dispatch mechanism).

That's all that the first version of Objective-C defines, as documented in Brad Cox's Object-Oriented Programming: an Evolutionary Approach.

Objective-C is a hybrid programming language[…]formed by grafting the Smalltalk-80 style of object-oriented programming onto a C language rootstock. Objective-C adds precisely one new data type, the object, to those C provides already, and precisely one new operation, the message expression. Like Smalltalk-80, Objective-C makes no compile-time distinction between different kinds (classes) of objects.

So if you want to try out 1980s-style OOP using Objective-C, the tool to use is not Objective-C itself but BlockObject.

That quote is from the start of Chapter Four (of the Cox edition, not the Cox and Novolbilski edition), which goes on to describe the Objective-C preprocessor, the multiple-dispatch system (using strings as message selectors like BlockObject does&mdash;and indeed like Apple's ObjC does albeit hidden behind the opaque SEL type), and the _msg C function that implements message dispatch.

Factory Objects

Having objects is all very well, but we need some way to create them. In the BlockObject system, there's a special type called BlockConstructor that configures and returns a new instance of an object.

That's boring. Objective-C doesn't define a "constructor type", can't we just use objects? Well, yes. Given a class definition like this:

= Stack : Object { id *objects; unsigned int count; }

Objective-C as-was automatically defines a global factory object called Stack that can be used anywhere in the code. You grab it and use it like this:

extern id Stack;

id myStack = [Stack new];
[myStack push: someObject];

From there, Objective-C won't surprise you much. You can define instance methods:

-push:anObject {
	if (count == MAX_OBJECTS) [self cannotGrow];
	objects[count++] = anObject;
	return self;
}

and factory methods:

+new {
	id stack = [super new];
	stack-&gt;objects = malloc(MAX_OBJECTS * sizeof(id));
	return stack;
}

Though users of "modern" Objective-C will notice that there's no compile-time type checking: no Stack *stack variable declaration for example. Indeed there's no @interface for the class at all; all objects are of type id, you can send any message to any object and what it does with that message is up to the receiver. Just like Smalltalk or Ruby.

#### Compatibility
Solaris 10, scheduled to be supported until January, 2021, can still run BSD binaries built for Solaris 1 (a retroactive name for SunOS 4.1), released in 1991. I wonder for how long the apps we wrote for our iPhones back in 2008 - the ones we had to pay $99 even to run on our own devices - will last.

#### Complex things possible
I was once told by a local Human-Computer Interface luminary (who claims to have in-depth knowledge of the way NeXTStep's UI works) that it's not possible in any existing GUI to have lines connecting separate windows.Allow me to introduce a decade-old piece of scotch mist.

#### Computing's fundamental Principle of No Learning
I haven't used Taligent's frameworks or operating systems directly; what I know of it comes from their documentation and the book Inside Taligent Technology. I put some small effort into finding out whether it's possible to use the Taligent system legitimately on a modern computer, couldn't find a way, and didn't get too sad about that.

The Taligent frameworks look - assuming they were ever as complete as the documentation promised - like fairly standard 1990s OOP-in-C++ which almost certainly makes them less fun to use than modern Qt. What I got from considering Taligent was a reflection on trends in modern computing platforms, that discoverability is killing power user features, and the app icon is killing complex workflows.

There is a false dichotomy at work in modern app design: the drive is for apps to be so simple you can use them as soon as you've tapped the app icon, but this is taken to mean that there doesn't need to be anything more to do in the app than what you can see when you have tapped the app icon. As an example, the bookmark manager app on my iPad shows me a list of my bookmarks, and there's an "add" button. If I click on a bookmark, I can view it. That's it. That's "managing" bookmarks, app-style.

But I would suggest it's not just the apps that are doing it, it's the platforms too, and that's how Taligent entered this story. Their central desktop metaphor was called People, Places, and Things, based on the idea that I might want to focus on those things mediated by a computer, rather than on applications that support particular use cases.

A particular example of the People metaphor is that I might want to show a photo of my cat to my sister. Back in the days of the real world, I would do that by showing the photo of my cat to my sister. In the brave new world of the app, I look at my phone to see what messaging apps I have, try to remember which ones I have her as a contact on, which ones she will actually check, whether they send pictures and if so, what that costs or how it futzes with the picture. I launch that, go to the conversation with her (or start one), add an attachment, try to think about whether this needs to be a Camera, Gallery or File attachment, and so on. This is made most painful when using the in-car voice UI on my smartphone, which seems able to transcribe messages well but unable to listen when it asks what app I want it to use to send the message. Like I care, cost and picture-futzing aside (though I'm not usually taking photos with the in-car voice UI).

The Places metaphor is particularly interesting. Each "place" is a virtual gallery, town hall, office lobby, project office, or whatever that contextualises a given group of People and Things relevant to the place. In implementation, it's a sort of context-sensitive desktop management. In concept, it makes me think of the PARC Ubiquitous Computing principle and how a location-sensitive computing environment could adapt as I sit at my desk, join some colleagues in a meeting room, or collapse on a beanbag at home. It makes me think of the Eudaimonia machine and how a context-sensitive computing interface could support shallow-and-broad to narrow-and-deep work environments. Then I consider how the static home screen on my smartphone...does not do those things.

An important part, I believe, in the downfall of People, Places and Things is what it means for software. Rather than buying "an application" from a vendor that shows me their beautiful, crafted icon until I'm ready to launch it, I buy a capability that extends the ways in which I can interact with the People, Places and Things represented on my computer. My computer does not become Photoshop, rather I gain Photoshopness by adding it to my computer.

It seems that any shift from app-centric to goal-centric computer interfaces - PPT, OpenDoc/OLE, NeXT Services, scripting/automation tools, the spatial desktop - is going to violate the Fundamental Paradox of App Marketing: my app is so intuitive and easy to use that it just fades into the backdrop of your life, however it must never fade so far that you do not see its name or its icon, and are not reminded who we are and how much we help you.

Similarly, they violate the Principle of No Learning discussed above: I must be able to see how to do everything, and I must only be able to do everything that can be seen. The desktop metaphor allows for spatial arrangement of your work, just like strewing papers around a real desktop, but you need to work out what those funny line-and-square icons mean and which bits of the window frame are draggable, so that goes. Drag-and-drop makes for a visual placement metaphor letting me "put" work from one document into another, but you can't obviously see that it's a thing (nor indeed what it will actually achieve; Drag and Drop seems intuitively destructive, sometimes is, but sometimes surprisingly isn't. What's the metaphorical outcome expected when you try to put your Hackers DVD in the Trash?). Therefore drag-and-drop is replaced with "Share Sheets". Scripting not only requires learning a new application - the script editor - but a whole new user interface for existing applications. Forget it.

The Paradox of App Marketing is merely misguided self-aggrandisement, but the Principle of No Learning is one of computing's big, stultifying, patronising, antiethical mistakes. It makes out that computers are simplistic tools that can do the things delivered unto the users by the app makers, and nothing else. It hides the magic behind the curtain by removing the curtain and the magic.

#### Comrades in OpenStep
I see someone else is also using OpenStep/Parallels/Mac/i386; let's hope we don't start outstripping OS X usage :-)N.B. slides from today's Intel talk coming soon...ish...

#### Concrete freedoms
Discussions about free software or open source software can always seem a bit abstract. Who cares if I've got the source code, if I'm never going to read it or change it? Why would I want "free" versions of my apps when there are already plenty of zero-cost (i.e. as free as I need) alternatives in my App Store?

This has been the year in which Apple tightened the screws on the App Store, making it clear that anyone who isn't giving them their 30% or 15% cut or isn't Netflix or Spotify is on thin ice. In an Orwellian move, they remote-killed Charlie Monroe's apps and told users that they couldn't run apps they'd paid for, because the apps would damage their computers.

At the base of the definition of free and open source software are the four freedoms. The first:

The freedom to run the program as you wish, for any purpose (freedom 0).

This is freedom "zero" not just because of the C-style zero indexing pun, but because it was added into the space preceding freedom one after the other three were written. The Free Software Foundation didn't think it needed explicitly stating, but apparently it does.

In a free software world, YOU are free to run software as you wish, for any purpose. Some trillion-dollar online company pretending to be a bricks-and-mortar retailer equivalent isn't going to come along and say "sorry, we've decided we don't want you running that, and rather than explain why we're just going to say it's for your own good." They aren't going to stop developers from sharing or selling their software, on the basis that they haven't paid enough of a tithe to the mothership.

These four freedoms may seem abstract, but they have real and material consequences. So does their absence.

#### Concurrent objects and SCOOP
Representing concurrency in an object-oriented system has been a long-standing problem. Encapsulating the concurrency primitives via objects and methods is easy enough, but doesn't get us anywhere. We still end up composing our programs out of threads and mutexes and semaphores, which is still hard.
Prior Art
It's worth skimming the things that I've written about here: I've put quite a lot of time into this concurrency problem and have written different models as my understanding changed.

 	2012: confine ALL the things! and confinement proxies
 	2013: separating user interface from work and the message bus pattern
 	2015: hiding behind messages and the Concurrent Smalltalk approach
 	2018: microservices for the desktop and separate processes for separate objects

Clearly, this is a broad problem. It's one I've spent a lot of time on, and have a few satisfactory answers to if no definitive answer. I'm not the only one. Over at codeotaku they've concluded that Fibers are the right solution, though apparently on the basis of performance.

HPC programs are often based on concurrent execution through message passing, though common patterns keep it to a minimum: the batch processor starts all of the processes, each process finds its node number, node 0 divvies up the work to all of the nodes (a message send), then they each run through their part of the work on their own. Eventually they get their answer and send a message back to node 0, and when it has gathered all of the results everything is done. So really, the HPC people solve this problem by avoiding it.
You're braining it wrong, Graham
Many of these designs try to solve for concurrency in quite a general way. Serial execution is a special case, where you only have one object or you don't submit commands to the bus. The problem with this design approach, as described by Bertrand Meyer in his webinar on concurrent Object-Oriented Programming, is that serial execution is the only version we really understand. So designing for the general, and hard-to-understand, case means that generally we won't understand what's going on.

The reason he says this is so is that we're better at understanding static relationships between things than the dynamic evolution of a system. As soon as you have mutexes and condition locks and so on, you are forced to understand the dynamic behaviour of the system (is this lock available? Is this condition met?). Worse: you have to understand it holistically (can anything that's going on at the moment have changed this value?).
Enter SCOOP
Meyer's proposal is that as serial programs are much easier to understand (solved, one might say, if one has read Dijkstra's A Discipline of Programming) we should make our model as close to serial programming as possible. Anything that adds concurrency should be unsurprising, and not violate any expectations we had if we tried to understand our program as a sequential process.

He introduced SCOOP (Simple Concurrent Object-Oriented Programming) developed by the Concurrency Made Easy group at ETH Zürich and part of Eiffel. Some of the design decisions he presented:

 	a processor is an abstraction representing sequential execution
 	there is a many-to-one mapping of objects to processors (this means that an object's execution is always serial, and that all objects are effectively mutexes)
 	where an object messages another on a different processor, commands will be asynchronous (but executed in order) and queries will be synchronous
 	processors are created dynamically and opportunistically (i.e. whenever you create an object in a "separate" and as-yet unpopulated domain)

An implementation of this concurrency model in Objective-C is really easy. A proxy object representing the domain separation intercepts messages, determines whether they are commands or queries and arranges for them to be run on the processor. It inspects the objects returned from methods, introducing proxies to tie them to the relevant processor. In this implementation a "processor" is a serial operation queue, but it could equivalently be a dedicated thread, a thread pulled from a pool, a dedicated CPU, or anything else that can run one thing at a time.

This implementation does not yield all of the stated benefits of SCOOP. Two in particular:

 	The interaction of SCOOP with the Eiffel type system is such that while a local (to this processor) object can be referred to through a "separate" variable (i.e. one that potentially could be on a different processor), it is an error to try to use a "separate" object directly as if it were local. I do not see a way, in either Swift's type system or ObjC's, to maintain that property. It looks like this proposal, were it to cover generic or associated types, would address that deficiency.
 	SCOOP turns Eiffel's correctness preconditions into wait conditions. A serial program will fail if it tries to send a message without satisfying preconditions. When the message is sent to a "separate" object, this instead turns into a requirement to wait for the precondition to be true before execution.

Conclusions
Meyer is right: concurrent programming is difficult, because we are bad at considering all of the different combinations of states that a concurrent system can be in. A concurrent design can best be understood if it is constrained to be mostly like a serial one, and not require lots of scary non-local comprehension to understand the program's behaviour. SCOOP is a really nice tool for realising such designs.

This is something I can help your team with! As you can see, I've spent actual years understanding and thinking about software concurrency, and while I'm not arrogant enough to claim I have solved it I can certainly provide a fresh perspective to your team's architects and developers. Book an office hours appointment and let's take a (free!) hour to look at your concurrency problems.

#### Configuring CruiseControl.rb in under an hour
One of the changes I decided to make straight after NSConf MINI yesterday was to enable continuous integration for my projects. I had used CI before based on BuildBot, but that had left me less than impressed:

It was really hard to set up
Its dependency system was less than optimal, leading it to do things like running the integration tests even after the build had failed
It had really bad memory consumption. The official line was that it didn't leak, but used a massive working set. Now I don't have a dedicated system for CI so can't really spare all my RAM :)

In his talk yesterday, Gordon Murrison mentioned that Open Planet Software use CruiseControl.rb, and in conversation he assured me that it wasn't as bad as all that. I decided to set it up today, and it took me pretty much an hour to go from downloading the source to having a working continuous integration setup. If I had known the stuff below, it could've been quicker.

The downloads page for CC.rb says that you need an older version of Ruby than Snow Leopard ships with, but I found that it actually works with the stock Ruby, so no problem there. 

Before going any further, I decided to create a separate user account for automated builds. This is to provide some validation of the source code - so we know that the product actually can be built from the artefacts in SCM, and doesn't depend on some magic configuration that happens to be part of my developer box. In fact, it turns out that for me there was some magic configuration in place - the Rehearsals build depends on the BGHUDAppKit IBPlugin, which of course wasn't installed for my new cruisecontrol user.

So with the build environment set up, I can point CC.rb at my project. That's done with this command line:

./cruise add Rehearsals --source-control svn --repository http://svn.thaesofereode.info/rehearsals/branches/branch-to-watch

Note that you have to specify the path all the way to the actual branch you want it to build (or trunk), not the top level of the repository. Now you need to tell it how to actually build the project. So edit ~/.cruise/projects/Rehearsals/cruise_config.rb and add a line like this:

  project.build_command = 'xcodebuild -configuration Debug -target Test\ Cases build'

That tells Xcode to build the "Test Cases" target in Debug configuration, which in my case is what I need to get my OCUnit tests to run. While you're in that file, set the project's email settings, and create config/site_config.rb in the CC.rb distribution folder to set up the mail server (Gmail in my case). Now it should just be a case of running:

./cruise start

and watching my build succeed. But it wasn't :(. My unit test target is injected into the app, which means that in order for the tests to even launch my cruisecontrol user needed a window server connection, so I used fast-user switching to log it in behind my real user. OK, so now my unit tests are automatically run whenever I check in some source on that branch, and I can see the status at http://localhost:3333/.

That's as far as I've got for now. Of course I'd like to have the cruisecontrol user automatically log in and run CC.rb whenever the system starts up, I'll create a launch agent to do that. But I also have a gcov-instrumented build configuration, and it would be instructive for CC.rb to automatically report on code coverage when the tests are run (though that report shouldn't affect the result of the build). But I think I've done enough for one day, it's time to go back to writing tests :).

Update: Thanks Simon Whitaker for finding a guide to run CC.rb under Apache using Passenger. I'm not sure how that would work in my case where I need a WindowServer connection, but I'm sure that there will be projects where this is a better way to get the thing running automatically.

#### Configuring your computer for local inference with a generative AI coding assistant
You can use multiple tools to download, host, and interact with large language models (LLMs) for generative tasks, including coding assistants. This post describes the one that I tried that has been the most successful. Even if you follow the approach below and it works well for you, I recommend trying different combinations of LLM and coding assistant so that you can find the setup that’s most ergonomic.

Choose hardware

You need to use a computer with either sufficient GPU, or dedicated neural processing, capacity to run an LLM, and enough RAM to hold gigabytes of parameters in memory while also running your IDE, software under development, and other applications. As an approximate rule, allow 1GB for every billion parameters in the model.

I chose Mac Studio with M3 Ultra and 256GB RAM. This computer uses roughly half of its memory to host the 123 billion parameter Devstral 2 model. A computer with 32GB RAM can run a capable small model; for example, Devstral Small 2: in this walkthrough I’ll show how to set up that model using Mistral Vibe as the coding assistant.

Note that once you have the model working locally, you can share it on your local network (or, using a VPN or other secure channel, over the internet) and access it from your other computers. You only need one computer on your network to be capable of hosting the LLM you choose to use local inference from any computer on that network.

Install LM Studio

Visit LM Studio and click the download button. Follow the installation process for your operating system; in macOS, you download a DMG that you open, and drag the app it contains into your Applications folder.

Download the model

Open LM Studio, and open the Model Search view by clicking on the magnifying glass. In macOS, check the MLX box to use the more efficient MLX format, and leave GGUF unchecked. Search for “Devstral Small 2 2512”, and click Download to download its weights and other configuration data. The second number (2512) refers to the release date of the model—in this case, December 2025.

Load the model and test it

When your model is downloaded, switch to the Chats view in LM Studio. In the window toolbar, click “Select a model to load” and choose the model you just downloaded. Optionally, toggle “Manually choose model load parameters” and configure settings. I typically alter the context size, as the default model size is 4096 tokens which optimises for inference speed and small memory footprint over a large “working set”. Click “Load Model” to tell LM Studio to serve the model. You can also tell LM Studio to use the model’s maximum context size as the default whenever it loads a model, in the app’s settings.

When LM Studio loads your chosen model, it opens a new chat with the model. Type a prompt into this chat to validate that the model is working, and has enough resources for inference tasks.

Download, configure, and test a coding assistant

Coding assistants typically expect to take an API key, and communicate with a model hosted in the cloud. To use a local LLM, you need to configure the assistant.

Follow the instructions in the Vibe studio README to install the tool. In Terminal, run mkdir ~/.vibe. Use your text editor to save the following content in a file called ~/.vibe/config.toml:

active_model = "devstral2-small-local"

&#91;&#91;providers]]
name = "lmstudio"
api_base = "http://localhost:1234/v1"
api_key = "LM_STUDIO_API_KEY" # LM Studio doesn't use this value
api_style = "openai"
backend = "generic"

&#91;&#91;models]]
name = "mistralai/devstral-small-2-2512"
provider = "lmstudio"
alias = "devstral2-small-local"
temperature = 0.2
input_price = 0.0
output_price = 0.0

Now test the assistant by running vibe in Terminal, and typing a prompt into the assistant.

Further learning

I’ve recently started Chiron Codex, an initiative to create software engineering centaurs by augmenting human knowledge of the software craft with AI assistance. You can find out more, and support the project, over on Patreon. Thank you very much for your support!

#### Confine ALL the things!
I was talking with Saul Mora at lunchtime about NSManagedObjectContext thread confinement. We launched into an interesting thought experiment: what if every object ran on its own thread?

This would be interesting. You can never use a method that returns a value, because then you'd need to block this object's thread while waiting for that object. You'd need to rely solely on being able to pass messages from one object to another, without seeing return values. In other words, the "tell don't ask" principle is enforced.

You get to avoid a lot of problems with concurrency issues in any app. Each object has its own data (something we've been able to get right for decades), and its own context to work on that data. No object should be able to trample on another object's data because we have encapsulation, and that means that no thread should be able to trample on another thread's data because one thread == one object.

Also notice that one object == one thread. You don't have to worry about whether different methods on the same object are thread-safe, because you're confining the object to a single thread so no code in that object will execute in parallel.

So yeah, I built that thing. Or really I built a very simple test app that demonstrates that thing in action. Check out the source from the GitHub project linked here. For your edification, here's a description of the meat of the confinement behaviour, the FZAConfinementProxy. Starting with the interface:

@interface FZAConfinementProxy : NSProxy

- (id)initWithRemoteObject: (id)confinedObject;

@end

This class is a NSProxy subclass, because all it needs to do is forward messages to another object. Or rather, that's what it claims to do.

static const void *FZAProxyAssociation = (const void *)@"FZAConfinementProxyAssociationName";

@implementation FZAConfinementProxy {
    id remoteObject;
    NSOperationQueue *operationQueue;
}

- (id)initWithRemoteObject:(id)confinedObject {
    id existingProxy = objc_getAssociatedObject(confinedObject, FZAProxyAssociation);
    if (existingProxy) {
        self = nil;
        return existingProxy;
    }

Ensure that there's exactly one proxy per object, using Objective-C's associated objects to map proxies onto implementations.

    remoteObject = confinedObject;
    if ([remoteObject isKindOfClass: [UIView class]]) {
        operationQueue = [NSOperationQueue mainQueue];
    } else {
        operationQueue = [[NSOperationQueue alloc] init];
        operationQueue.maxConcurrentOperationCount = 1;
    }
    objc_setAssociatedObject(confinedObject, FZAProxyAssociation, self, OBJC_ASSOCIATION_ASSIGN);
    return self;
}

If we need to create a new proxy, then create an operation queue to which it can dispatch messages. UIKit needs to be used on the main thread, so ensure proxies for UIViews use the main operation queue.

What this means is that if your controller uses confinement proxies for both views and models (just as the dummy app does), then there's no need to mess with methods like -performSelectorOnMainThread:withObject:waitUntilDone: in your controller logic. That's automatically handled by the proxies.

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    if (sel == @selector(initWithRemoteObject:)) {
        return [NSMethodSignature signatureWithObjCTypes: "@@:@"];
    }
    else {
        return [remoteObject methodSignatureForSelector: sel];
    }
}

Ensure that the Objective-C machinery can tell what messages it can send to this proxy.

- (void)forwardInvocation:(NSInvocation *)invocation {
    [invocation setTarget: remoteObject];
    NSInvocationOperation *operation = [[NSInvocationOperation alloc] initWithInvocation: invocation];
    [operationQueue addOperation: operation];
}

Here's all the smarts, and it isn't really very smart. Objective-C notices that the proxy object itself doesn't implement the received method, and so asks the proxy whether it wants to forward the method. It's, well, it's a proxy, so it does want to. It rewrites the invocation target to point to the confined object, then adds the invocation to the operation queue.

That's really it, bar some cleanup that you can see in GitHub. One thread per object limits the ways in which you can design code, so it's a great way to learn about writing code within those limitations.

#### Conflicts in my mental model of Objective-C 
My worldview as it relates to the writing of software in Objective-C contains many items that are at odds with one another. I either need to resolve them or to live with the cognitive dissonance, gradually becoming more insane as the conflicting items hurl one another at my cortex.

Of the programming environments I've worked with, I believe that Objective-C and its frameworks are the most pleasant. On the other hand, I think that Objective-C was a hack, and that the frameworks are not without their design mistakes, regressions and inconsistencies.

I believe that Objective-C programmers are correct to side with Alan Kay in saying that the designers of C++ and Java missed out on the crucial part of object-oriented programming, which is message passing. However I also believe that ObjC missed out on a crucial part of object-oriented programming, which is the compiler as an object. Decades spent optimising the compile-link-debug-edit cycle have been spent on solving the wrong problem. On which topic, I feel conflicted by the fact that we've got this Smalltalk-like dynamic language support but can have our products canned for picking the same selector name as some internal secret stuff in someone else's code.

I feel disappointed that in the last decade, we've just got tools that can do the same thing but in more places. On the other hand, I don't think it's Apple's responsibility to break the world; their mission should be to make existing workflows faster, with new excitement being optional or third-party. It is both amazing and slightly saddening that if you defrosted a cryogenically-preserved NeXT application programmer, they would just need to learn reference counting, blocks and a little new syntax and style before they'd be up to speed with iOS apps (and maybe protocols, depending on when you threw them in the cooler).

Ah, yes, Apple. The problem with a single vendor driving the whole community around a language or other technology is that the successes or failures of the technology inevitably get caught up in the marketing messages of that vendor, and the values and attitudes ascribed to that vendor. The problem with a community-driven technology is that it can take you longer than the life of the Sun just to agree how lambdas should work. It'd be healthy for there to be other popular platforms for ObjC programming, except for the inconsistencies and conflicts that would produce. It's great that GNUstep, Cocotron and Apportable exist and are as mature as they are, but "popular" is not quite the correct adjective for them.

Fundamentally I fear a world in which programmers think JavaScript is acceptable. Partly because JavaScript, but mostly because when a language is introduced and people avoid it for ages, then just because some CEO says all future websites must use it they start using it, that's not healthy. Objective-C was introduced and people avoided it for ages, then just because some CEO said all future apps must use it they started using it.

I feel like I ought to do something about some of that. I haven't, and perhaps that makes me the guy who comes up to a bunch of developers, says "I've got a great idea" and expects them to make it.

#### Considered harmless
Don't like a new way of working? Just point out the absurdity of suggesting that the old way was broken:

  Somehow, the microservices folks have failed to notice all that software that was in fact delivered as monoliths.
  
  What the Rust Evangelism Strike Force doesn't realise is that we've spent decades successfully building C programs that don't dereference the NULL pointer.

This is a sort of "[C|Monoliths] considered harmless" statement. Yes, it's possible to do it that way, but that doesn't mean that there aren't problems, or at least trade-offs. "C considered harmless" is as untrue and unhelpful as "C considered harmful"; what we want is "C considered alongside alternatives".

#### Considering society
OK now that the anniversary's out of the way, I can stop being hagiographic towards agile software development and point out the one big flaw in the approach. It's a stinker.

Here's the list of everybody mentioned in the manifesto and the principles behind it:

The authors (“We are uncovering better ways of developing software”, “we have come to value”, “we value the items on the left”)

Customers (“Customer collaboration over contract negotiation”, “Our highest priority is to satisfy the customer”, “Agile processes harness change for the customer's competitive advantage”)

Business people (“Business people and developers must work together”)

Developers (above, and “The sponsors, developers, and users should be able to maintain a constant pace indefinitely”)

Sponsors (above)

Users (above)

You might want to add “the team” but the only time the team composition gets spelt out it's a “development team” so they're really talking developers at that point.

The only thing required of or provided to users is a sustainable pace. No satisfaction, job security, dignity, mental health, or value (unless the users happen to be the customers, but in many contexts that isn't true).

And the users are really an afterthought to the main goal, which is generating customer value in software form. Nowhere does the rest of society get a look in. The people who get run over by your self-driving car? Not mentioned. The people who breathe in your diesel engine's fumes? Nope. The people whose personal connections are interrupted by the valuable ads that you sell to your valuable customers as your highest priority? Who even are they?

In a very real and straightforward way, this means that agile software development is unethical. The ACM's Code of Ethics and Professional Conduct says (section 1.1) that a computing professional should “contribute to society and to human well-being, acknowledging that all people are stakeholders in computing.” Section 3.1 says that they should “ensure that the public good is the central concern during all professional computing work.”

Enshittification, surveillance capitalism, social media addiction—all those things we claim are some evil perversion of software development are actually the system working correctly, when the system's creators set satisfying the customer “through early and continuous delivery of valuable software” as the highest priority that the system optimises for.

We have to create and advocate for a system where public good and human well-being are higher priorities than customer value, and that means at least embedding agile software development in a humane safety net, and potentially, replacing it altogether. European Union framework research recommends an approach called Responsible Research and Innovation, in which diverse stakeholders are identified and engaged throughout the research, development, and deployment of novel technology (an example implementation is the UK's AREA framework).

#### Consulting versus micro-ISV development
Reflexions on the software business really is an interesting read. Let me borrow Adrian's summary of his own post:

Now, here’s an insider tip: if your objective is living a nightmare, tearing yourself apart and swear never touching a keyboard again, choose [consulting]. If your objective is enjoying a healthy life, making money and living long and prosper, choose [your own products].

As the author himself allows, the arguments presented either way are grossly oversimplified. In fact I think there is a very simple axiom underlying what he says, which if untrue moves the balance away from writing your own products and into consulting, contracting or even salaried work. Let me start by introducing some features missed out of the original article. They may, depending on your point of view, be pros or cons. They may also apply to more than one of the roles.

A consultant:

builds up relationships with many people and organisationsis constantly learningworks on numerous different productsis often the saviour of projects and businessesgets to choose what the next project ishas had the risks identified and managed by his clientcan focus on two things: writing software, and convincing people to pay him to write softwarerenegotiates when the client's requirements change

A &mu;ISV developer:

is in sales, marketing, support, product management, engineering, testing, graphics, legal, finance, IT and HR until she can afford to outsource or employhas no income until version 1.0 is outcannot choose when to put down the next version to work on the next productcan work on nothing elseworks largely alonemust constantly find new ways to sell the same few productsmust pay for her own training and development

A salaried developer:

may only work on what the managers wanthas a legal minimum level of securitycan rely on a number of other people to help outcan look to other staff to do tasks unrelated to his missiongets paid holiday, sick and parental leavecan agree a personal development plan with the highers-upowns none of the work he creates

I think the axiom underpinning Adrian Kosmaczewski's article is: happiness &prop; creative freedom. Does that apply to you? Take the list of things I've defined above, and the list of things in the original article, and put them not into "&mu;ISV vs. consultant" but "excited vs. anxious vs. apathetic". Now, this is more likely to say something about your personality than about whether one job is better than another. Do you enjoy risks? Would you accept a bigger risk in order to get more freedom? More money? Would you trade the other way? Do you see each non-software-developing activity as necessary, fun, an imposition, or something else?So thankyou, Adrian, for making me think, and for setting out some of the stalls of two potential careers in software. Unfortunately I don't think your conclusion is as true as you do.

#### Continuous Integration for Amiga
Amiga-Smalltalk now has continuous integration, I don't know if it's the first Amiga program ever to have CI but definitely the first I know of. Let me tell you about it.

I've long been using AROS, the AROS Research Operating System (formerly the A stood for Amiga) as a convenient place to (manually) test Amiga-Smalltalk. AROS will boot natively on PC but can also be "hosted" as a user-space process on Linux, Windows or macOS. So it's handy to build a program like Amiga-Smalltalk in the AROS source tree, then launch AROS and check that my program works properly. Because AROS is source compatible with Amiga OS (and binary compatible too, on m68k), I can be confident that things work on real Amigas.

My original plan for Amiga-Smalltalk was to build a Docker image containing AROS, add my test program to S:User-startup (the script on Amiga that runs at the end of the OS boot sequence), then look to see how it fared. But when I discussed it on the aros-exec forums, AROS developer deadwood had a better idea.

He's created AxRuntime, a library that lets Linux processes access the AROS APIs directly without having to be hosted in AROS as a sub-OS. So that's what I'm using. You can look at my Github workflow to see how it works, but in a nutshell:

check out source.
install libaxrt. I've checked the packages in ./vendor (and a patched library, which fixes clean termination of the Amiga process) to avoid making network calls in my CI. The upstream source is deadwood's repo.
launch Xvfb. This lets the process run "headless" on the CI box.
build and run ast_tests, my test runner. The Makefile shows how it's compiled.

That's it! All there is to running your Amiga binaries in CI.

#### Contractually-obligated testing
About a billion years ago, Bertrand Meyer (he of Open-Closed Principle fame) introduced a programming language called Eiffel. It had a feature called Design by Contract, that let you define constraints that your program had to adhere to in execution. Like you can convince C compilers to emit checks for rules like integer underflow everywhere in your code, except you can write your own rules.

To see what that's like, here's a little Objective-C (I suppose I could use Eiffel, as Eiffel Studio is in homebrew, but I didn't). Here's my untested, un-contractual Objective-C Stack class.

@interface Stack : NSObject

- (void)push:(id)object;
- (id)pop;

@property (nonatomic, readonly) NSInteger count;

@end

static const int kMaximumStackSize = 4;

@implementation Stack
{
    __strong id buffer[4];
    NSInteger _count;
}

- (void)push:(id)object
{
    buffer[_count++] = object;
}

- (id)pop
{
    id object = buffer[--_count];
    buffer[_count] = nil;
    return object;
}

@end

Seems pretty legit. But I'll write out the contract, the rules to which this class will adhere provided its users do too. Firstly, some invariants: the count will never go below 0 or above the maximum number of objects. Objective-C doesn't actually have any syntax for this like Eiffel, so this looks just a little bit messy.

@interface Stack : ContractObject

- (void)push:(id)object;
- (id)pop;

@property (nonatomic, readonly) NSInteger count;

@end

static const int kMaximumStackSize = 4;

@implementation Stack
{
    __strong id buffer[4];
    NSInteger _count;
}

- (NSDictionary *)contract
{
    NSPredicate *countBoundaries = [NSPredicate predicateWithFormat: @"count BETWEEN %@",
                                    @[@0, @(kMaximumStackSize)]];
    NSMutableDictionary *contract = [@{@"invariant" : countBoundaries} mutableCopy];
    [contract addEntriesFromDictionary:[super contract]];
    return contract;
}

- (void)in_push:(id)object
{
    buffer[_count++] = object;
}

- (id)in_pop
{
    id object = buffer[--_count];
    buffer[_count] = nil;
    return object;
}

@end

I said the count must never go outside of this range. In fact, the invariant must only hold before and after calls to public methods: it's allowed to be broken during the execution. If you're wondering how this interacts with threading: confine ALL the things!. Anyway, let's see whether the contract is adhered to.

int main(int argc, char *argv[]) {
    @autoreleasepool {
        Stack *stack = [Stack new];
        for (int i = 0; i &lt; 10; i++) {
            [stack push:@(i)];
            NSLog(@"stack size: %ld", (long)[stack count]);
        }
    }
}

2014-08-11 22:41:48.074 ContractStack[2295:507] stack size: 1
2014-08-11 22:41:48.076 ContractStack[2295:507] stack size: 2
2014-08-11 22:41:48.076 ContractStack[2295:507] stack size: 3
2014-08-11 22:41:48.076 ContractStack[2295:507] stack size: 4
2014-08-11 22:41:48.076 ContractStack[2295:507] *** Assertion failure in -[Stack forwardInvocation:], ContractStack.m:40
2014-08-11 22:41:48.077 ContractStack[2295:507] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException',
 reason: 'invariant count BETWEEN {0, 4} violated after call to push:'

Erm, oops. OK, this looks pretty useful. I'll add another clause: the caller isn't allowed to call -pop unless there are objects on the stack.

- (NSDictionary *)contract
{
    NSPredicate *countBoundaries = [NSPredicate predicateWithFormat: @"count BETWEEN %@",
                                    @[@0, @(kMaximumStackSize)]];
    NSPredicate *containsObjects = [NSPredicate predicateWithFormat: @"count > 0"];
    NSMutableDictionary *contract = [@{@"invariant" : countBoundaries,
             @"pre_pop" : containsObjects} mutableCopy];
    [contract addEntriesFromDictionary:[super contract]];
    return contract;
}

So I'm not allowed to hold it wrong in this way, either?

int main(int argc, char *argv[]) {
    @autoreleasepool {
        Stack *stack = [Stack new];
        id foo = [stack pop];
    }
}

2014-08-11 22:46:12.473 ContractStack[2386:507] *** Assertion failure in -[Stack forwardInvocation:], ContractStack.m:35
2014-08-11 22:46:12.475 ContractStack[2386:507] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException',
 reason: 'precondition count > 0 violated before call to pop'

No, good. Having a contract is a bit like having unit tests, except that the unit tests are always running whenever your object is being used. Try out Eiffel; it's pleasant to have real syntax for this, though really the Objective-C version isn't so bad.

Finally, the contract is implemented by some simple message interception (try doing that in your favourite modern programming language of choice, non-Rubyists!).

@interface ContractObject : NSObject
- (NSDictionary *)contract;
@end

static SEL internalSelector(SEL aSelector);

@implementation ContractObject

- (NSDictionary *)contract { return @{}; }

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
    NSMethodSignature *sig = [super methodSignatureForSelector:aSelector];
    if (!sig) {
        sig = [super methodSignatureForSelector:internalSelector(aSelector)];
    }
    return sig;
}

- (void)forwardInvocation:(NSInvocation *)inv
{
    SEL realSelector = internalSelector([inv selector]);
    if ([self respondsToSelector:realSelector]) {
        NSDictionary *contract = [self contract];
        NSPredicate *alwaysTrue = [NSPredicate predicateWithValue:YES];
        NSString *calledSelectorName = NSStringFromSelector([inv selector]);
        inv.selector = realSelector;
        NSPredicate *invariant = contract[@"invariant"]?:alwaysTrue;
        NSAssert([invariant evaluateWithObject:self],
            @"invariant %@ violated before call to %@", invariant, calledSelectorName);
        NSString *preconditionKey = [@"pre_" stringByAppendingString:calledSelectorName];
        NSPredicate *precondition = contract[preconditionKey]?:alwaysTrue;
        NSAssert([precondition evaluateWithObject:self],
            @"precondition %@ violated before call to %@", precondition, calledSelectorName);
        [inv invoke];
        NSString *postconditionKey = [@"post_" stringByAppendingString:calledSelectorName];
        NSPredicate *postcondition = contract[postconditionKey]?:alwaysTrue;
        NSAssert([postcondition evaluateWithObject:self],
            @"postcondition %@ violated after call to %@", postcondition, calledSelectorName);
        NSAssert([invariant evaluateWithObject:self],
            @"invariant %@ violated after call to %@", invariant, calledSelectorName);
    }
}

@end

SEL internalSelector(SEL aSelector)
{
    return NSSelectorFromString([@"in_" stringByAppendingString:NSStringFromSelector(aSelector)]);
}

#### Controlling opportunity
In Code Complete, McConnell outlines the idea of having a change control procedure, to stop the customers from changing the requirements whenever they see fit. In fact one feature of the process is to be heavy enough to dissuade customers from registering changes.The Rational Unified Process goes for the slightly more neutral term Change Request Management, but the documentation seems to imply the same opinion, that it is the ability to make change requests which must be limited. The issue is that many requests for change in software projects are beneficial, and accepting the change request is not symptomatic of project failure. The most straightforward example is a bug report - this is a change request (please fix this defect) which converts broken software into working software. Similarly, larger changes such as new requirements could convert a broken business case into a working business case; ultimately turning a failed project into a revenue-generator.In my opinion the various agile methodologies don't address this issue, either assuming that with the customer involved throughout, no large change would ever be necessary, or that the iterations are short enough for changes to be automatically catered for. I'm not convinced; perhaps after the sixth sprint of your content publishing app the customer decides to open a pet store instead.I humbly suggest that project managers replace the word "control" in their change documentation with "opportunity" - let's accept that we're looking for ways to make better products, not that we need excuses never to edit existing Word files. OMG baseline be damned!

#### Copyleft
It is complete:I no longer even own all the code I write in my spare time, on my own equipment.  :-)

#### Core Data Haiku competition results!
I was sent a review copy of Core Data: Apple's API for Persisting Data on Mac OS X by Marcus Zarra. The problem is that I already own a copy. So I held a Core Data Haiku competition on Twitter; best haiku about core data posted with the #coredatahaiku tag wins the book. Sorry if you wrote a great haiku without using the hashtag, but I didn't read it.Anyway, the results. My decision is final, no correspondence regarding the judgement shall be entered into, the squirrel flies south to Leningrad.

Non-entering sample haiku for comparison purposes

This is mine:Archiving does notMake schema changes easySo #lickahoctorvery much of its time, you'll agree.

Dishonourable mentions

A couple of people sent entries via Facebook, which didn't count within the rules of the game. So without naming the guilty parties:I don't want the bookI just like writing haikuNothing wrong with that!When learning Cocoa,Reasonably new to it,The more books the bett-

Honourable mentions

The standard was very high, so I'm pleased to publish every entry, congratulations to you all for some inspired poetry. In a way, there are no losers, because you all helped to make the world a better place. In reverse chronological order (because I'm pasting from the Twitter search page):I do not know whatCore data is. Need the book.Please let me win it. &mdash; OrigamiTechLost In Winter IllsResult Sets Unsorted AgainManaged Object Found &mdash; chwaltersWith multiple storesInside one applicationLittle kittens weep &mdash; inquisitiveCodeobject tree...query is madeleaves rustle &mdash; ErikAderstedtCore Data HaikuNSManagedObjectConText just doesn't fit &mdash; hatfinchits way too complexpainful big design upfrontI'll Archive instead &mdash; alancfrancisCore Data frameworkYou manage object modelsNSPredicate &mdash; adurdinCore data is funmakes storage easy and fastelastic wombat &mdash; GreyAreaUK

The Winner!

Core Data haiku winners and immortal beings played by Sean Connery share an important characteristic: in the end, there can be only one. And it's this one, from adurdin:SELECT * FROM thingWHERE value =… fuck it—just use Core Data.Congratulations! I'll contact the winner via Twitter to send him his prize.

#### Could effortless lecturers make everything seem too easy?
From the British Psychological Society blog: Engaging lecturers can breed overconfidence.

The students who'd seen the smooth lecturer thought they would do much better than did the students who saw the awkward lecturer, consistent with the idea that a fluent speaker breeds confidence. In fact, both groups of students fared equally well in the test. In the case of the students in the fluent lecturer condition, this wasn't as good as they'd predicted. Their greater confidence was misplaced.

I'm speaking at a couple of conferences later this year (iOSDev UK in Aberystwyth in September and NSScotland in Edinburgh in October), and will endeavour to be exactly as exciting as the material deserves: a capability to which my track record can attest.

#### Coupling in a Cocoa[ Touch] App
This is one of my occasional "problem looking for a solution" posts. It'd be great to discuss this over on App.net or G+ or somewhere. I don't think, at the outset of writing this post, that the last sentence is going to solve the problems identified in the body.

I've built applications in a few different technologies, and I think that the Cocoa and Cocoa Touch apps I've seen have been the most tightly coupled to their host frameworks with the possible exception of Delphi. I include both my code and that I've seen of other people—I'm mainly talking about my work of course. I've got a few ideas on why that might be.

The coupling issue in detail

Ignore Foundation for a moment. A small amount of Foundation is framework-ish: the run loop and associated event sources and sinks. The rest is really a library of data types and abstracted operating system facilities.

But now look at AppKit or UIKit. Actually, imagine not looking at them. Consider removing all of the code in your application that directly uses an AppKit or UIKit class. Those custom views, gone. View controller subclasses? Boom. Managed documents? Buh-bye.

OK, now let's try the same with Core Data. Imagine it disappeared from the next version of the OS. OK, so your managed objects disappear, but how much of the rest of your app uses them directly? Now how about AVFoundation, or GameKit, and so on?

I recently took a look at some of my code that I'd written very recently, and found that less code survived these excisions than I might like. To be sure, it's a greater amount than from an application I wrote a couple of years ago, but all the same my Objective-C code is tightly coupled to the Apple frameworks. The logic and features of the application do not stand on their own but are interwoven with the things that make it a Mac, or iPhone app.

Example symptoms

I think Colin Campbell said it best:

iOS architecture, where MVC stands for Massive View Controller

A more verbose explanation is that these coupled applications violate the Single Responsibility Principle. There are multiple things each class is doing, and this means multiple reasons that I might need to change any of these classes. Multiple reasons to change means more likelihood that I'll introduce a bug into this class, and in turn more different aspects of the app that could break as a result.

Particularly problematic is that changes in the way the app needs to interact with the frameworks could upset the application behaviour: i.e. I could break what the app does by changing how it presents a UI or how it stores objects. Such modifications could come about when new framework classes are introduced, or new features added to existing classes.

What's interesting is that I've worked on applications using other frameworks, and done a better job: I can point at an Eclipse RCP app where the framework dependencies all lie at the plugin interface boundaries. Is there something specific to Cocoa or Cocoa Touch that leads to applications being more tightly coupled? Let's look at some possibilities.

Sample code

People always malign sample code. When you're just setting out, it assumes you know too much. When you're expert, it takes too many shortcuts and doesn't display proper [choose whichever rule of code organisation is currently hot]. Sample code is like the avocado of the developer documentation world: it's only ripe for a few brief minutes between being completely inedible and soft brown mush.

Yes, sample code is often poorly-decoupled, with all of the application logic going in the app delegate or the view controller. But I don't think I can blame my own class design on that. I remember a time when I did defend a big-ass app delegate by saying it's how Apple do it in their code, but that was nearly a decade ago. I don't think I've looked to the sample code as a way to design an app for years.

But sample code is often poorly-decoupled regardless of its source. In researching this post, I had a look at the sample code in the Eclipse RCP book; the one I used to learn the framework for the aforementioned loosely-coupled app. Nope, that code is still all "put the business logic in the view manager" in the same way Apple's and Microsoft's is.

Design of the frameworks and history

I wonder whether there's anything specific about the way that Apple's frameworks are designed that lead to tight coupling. One thing that's obvious about how many of the frameworks were designed is the "in the 1990s" aspect of the issue.

Documentation on object-oriented programming from that time tells us that subclassing was the hotness. NSTableView was designed such that each of its parts would be "fully subclassable", according to the release notes. Enterprise Objects Framework (and as a result, Core Data) was designed such that even your data has to be in a subclass of a framework class.

Fast forward to now and we know that subclassing is extremely tight coupling. Trying to create a subclass of a type you do control is tricky enough, but when it's someone else's you have no idea when they're going to add methods that clash with yours or change behaviour you're relying on. But those problems, real though they are, are not what I'm worried about here: subclassing is vendor lock-in. Every time you make it harder to extract your code from the vendor's framework, you make it harder to leave that framework. More on that story later.

So subclassing couples your code to the superclass, and makes it share the responsibility of the superclass and whatever it is you want it to do. That becomes worse when the superclass itself has multiple responsibilities:

views are responsible for drawing, geometry and event handling.
documents are responsible for loading and saving, managing windows, handling user interaction for load, save and print requests, managing the in-memory data representing the document, serialising document access, synchronising user interface access with background work and printing.
pretty much everything in an AppKit app is responsible in some way for scripting support which is a cross-cutting concern.

You can see Apple's frameworks extricating themselves from the need to subclass everything, in some places. The delegate pattern, which was largely used either to supply data or let the application respond to events from long-running tasks, is now also used as a decorator to provide custom layout decisions, as with the 10.4 additions to the NSTableViewDelegate, UITableViewDelegate and more obviously the UICollectionViewDelegateFlowLayout. Delegate classes are still coupled to the protocol interface, but are removed from the class hierarchy giving us more options to adapt our application code onto the interface.

Similarly, the classes that offer an API based on supplying a completion handler require that the calling code be coupled to the framework interface, but allow anything to happen on completion. As previously mentioned you sometimes get other problems related to the callback design, but at least now the only coupling to the framework is at the entry point.

No practice at decoupling

This is basically the main point, isn't it? A lack of discipline comes from a lack of practice. But why should I (and, I weakly argue, other programmers whose code I've read) be out of practice?

No push to decouple

Here's what I think the reason could be. Remind me of the last application you saw that wasn't either a UIKit app or an AppKit app. Back in the dim and distant past there might be more variety: the code in your WebObjects Objective-C server might also be used in an AppKit+EOF client app. But most of us are writing iOS software that will only run on iOS, so the effort in decoupling an iOS app from UIKit has intellectual benefits but no measurable tangible benefits.

The previous statement probably shouldn't be taken as absolute. Yes, there are other ways to do persistence than Core Data. Nonetheless, it's common to find iOS apps with managed object contexts passed to every view controller: imagine converting that ball of mud to use BNRPersistence.

What about portability? There are two options for writing cross-platform Objective-C software: frameworks that are API compatible with UIKit or AppKit, or not. While it is possible to build an application that has an Objective-C core and uses, say, Qt or Win32 to provide a UI, in practice I've never seen that.

There also aren't any "alternative" Objective-C application frameworks on the platforms ObjC programmers do support. In the same way that you might want to use the same Java code in a SWT app, a Swing app, and a Spring MVC server; you're not going to want to port your AppKit app to SomeoneElsesKit. You'll probably not want to move away from UIKit to $something_else in the near future in the same way it might be appropriate to move from Struts to Spring or from Eclipse RCP to NetBeans Platform.

Of course, this is an argument with some circular features. Because it's hard to move the code I write away from UIKit, if something else does come along the likelihood is I'd be disinclined to take advantage of it. I should probably change that: vendor lock-in in a rapidly changing industry like software is no laughing matter. Think of the poor people who are stuck with their AWT applications, or who decided to build TNT applications on NeWS because it was so much more powerful than X11.

Conclusion

This is the last sentence, and it solves nothing; I did tell you that would happen.

#### Creating “sub-agents” with Mistral Vibe
The vibe coding assistant doesn’t have the same idea of sub-agents that Claude Code does, but you can create them yourself—more or less—from the pieces it supplies. [UPDATE: vibe 2.0 supports subagents directly.]

Write the prompt for the sub-agent in a markdown file, and save it to ~/.vibe/prompts/&lt;agent-name&gt;.md. For example:

# Test suite completer
You are an expert software tester. You help the user create a complete and valuable test suite by analyzing their software and their tests, identifying tests that can be added, and constructing those tests.
Use test design principles, including equivalence partitioning and boundary value analysis, to identify gaps in test coverage. Review the project's documentation, including comment docs and help strings, to determine the software's intended behavior. Design a suite of tests that correctly verifies the behavior, then investigate the existing test code to determine whether all of the cases you designed are covered. Add the tests you identify as missing.
## Workflow
1. Read the user's prompt to understand the scope of your tests.
2. Discover documentation and code comments that describe the intended behavior of the system under test.
3. Design a suite of tests that exercise the system's intended behavior, and that pass if the system behaves as expected and fail otherwise.
4. Search the existing test code for tests that cover the behavior you identify.
5. Create tests that your analysis indicates are necessary, but that aren't in the existing test suite.
6. Report to the user the tests you created so they can review and run the tests.

Write a configuration for an agent that uses this system prompt, and save it to ~/.vibe/agents/&lt;agent-name&gt;.toml. For example:

active_model = "devstral2-local"
system_prompt_id = "test-suite-completer"
&#91;tools.read_file]
permission = "always"
&#91;tools.write_file]
permission = "always"
&#91;tools.search_replace]
permission = "always"

The active_model needs to be a model that you define in ~/.vibe/config.toml, or you can omit it to use the default model. The system_prompt_id needs to match the filename you give the system prompt file, without the .md extension.

You can use this agent by passing the --agent option to vibe, for example I use the following shell script, and create a symbolic link to the script that has the same name as the agent I want to use:

#!/bin/sh
if &#91; $# -ne 1 ]; then
    echo "Usage: $0 &#91;prompt]"
    exit 1
fi
AGENT=$(basename "$0")
vibe --agent "$AGENT" --prompt "$1"

You can now use this agent directly at the command line, or tell vibe about the script so that it invokes your agent as a sub-agent.

#### Criticising the Four Freedoms
The core principle of Free Software is that people who use software
retain certain freedoms, unlike the situation with proprietary
software in which all of the freedom associated with the software
remains with the vendor. Those are the Four Freedoms:

  A program is free software if the program's users have the four essential freedoms:
  
  
  The freedom to run the program as you wish, for any purpose (freedom 0).
  The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
  The freedom to redistribute copies so you can help your neighbor (freedom 2).
  The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.
  

Without other resources, these freedoms are pretty academic. Let's
take access to a computer as a given for the purpose of this argument:
you're one of "the program's users", so presumably you have the
material needed to use the program.

But does the program need all the resources it uses?

I can study and modify the program. Access to the source code is
indeed a prerequisite; comprehensible source code is also a
prerequisite. So are the study materials I need to comprehend the
source code, and the time it'll take me to do that study.

So that's me on the receiving end of free software, what about the
producing end? Nothing in the world of free software compels me to
choose the simplest language, to design my software for
comprehensibility, nor to make available the tools and information
needed to understand the source code that enables the other
freedoms. But unless I do that, the four freedoms are only
hypothetical.

#### Culture Smell
A phrase I used in a discussion today. Developers are familiar with "code smells", aspects of a codebase that aren't necessarily wrong but do make you take a deeper look. By analogy, a culture smell surprising, but not necessarily wrong, behaviour on a team that should make you wonder what motivations lead to that behaviour, and what to change to remove or redirect those motivations.

It's easy to get the wrong idea about culture though. Familiar with the developer concept of pure functions, some organisations seem to operate on the belief that by defining some principles and printing motivational posters, they can ensure conformant behaviour. Or that "the culture" is a concrete device that employees buy into.

Culture is the more slowly-varying norms established by the interactions between group members. You can guide, and lead, and of course admonish or eject, but you can't control. As a result, more culture signals are "smells" than problems: what you see may not look right, but you have to explore more.

#### Culture, heritage and apps
I said earlier on Twitter that I'm disappointed with the state of apps produced for museums and libraries. I'd better explain what I mean. Here's what I said:

Disappointed to find that many museum apps (British Library, Bodleian, Concorde etc) are just the same app with different content. :-(

In each case (particularly Concorde) there's some magic specific to the subject. Rubber-stamping the app doesn't capture that magic.

They're all made with a tool called Toura that sits atop PhoneGap. Makes it easy to get the content into the apps at the cost of expression.

So to be clear: my problem is not with the museums and other heritage sites. I'm familiar with the financial and political problems associated with running a museum. My social circle includes curators, librarians and medieval manuscript experts and I know that money is tight, that oversight is close and that any form of expenditure in promotion must result in a demonstrable increase in feet through the door and donations to be considered a success.

The Magna Carta

My problem is also not with the Toura product and the team behind it. They're to be commended for identifying that while the culture and heritage community doesn't have much money, they still deserve to be represented in our tablets and phones. The apps I listed above all feature astounding objects: examples include the Concorde aircraft, the Lindisfarne gospels and the Magna Carta.

Concorde

So why are these apps disappointing? It's basically because they're all the same. Each of these objects has its own magic: the unique shape of Concorde, the vivid colours of the gospels and the constitution-defining text of the Carta. So why present them in the same way? Why not make a Concorde app that evokes aerodynamic speed, an illuminated Lindisfarne gospels app, and a revolutionary Magna Carta app? As many people have explained, it's because standard content-viewing apps like Toura are all these institutions can afford.

Folio 27r from the Lindisfarne Gospels: image from Wikimedia Commons

There's something seriously wrong in our app industry. We've created a world in which apps are too cheap, and developers struggle to make a living from 70¢ per sale. Simultaneously, we've created a world in which apps are too expensive: the keepers of some of the world's most interesting objects can't afford to showcase these with more than a generic master/detail view. What's that about?

What mindset leads us to demand high worth, when we're making products that we can't convince customers contain any value? If they don't see the value, are we deluding ourselves? People talk about events like the Instagram acquisition as being evidence of a bubble subset of the app economy. But isn't that oxymoron - a free app that's worth a billion dollars - just an exaggerated version of what the rest of the industry is up to?

Let me present this problem in a different way, by moving from apps about museums to museums about apps.

The Rijksmuseum, Amsterdam.

Last month I spent an enjoyable time with some very good friends in Amsterdam, the city of both apps and museums. While walking around the Rijksmuseum, I reminded myself that we're in the middle of the possibly the largest and definitely the fastest technology-mediated social revolution humanity has ever experienced. Those of us enabling the application of this technology are literally providing the fulcrum around which our species is revolving.

During this year, museums will look at apps as cultural, historical and ethnological artefacts in their own right. Over the coming century, heritage institutions will shape the way that our brief period in time is presented for posterity. Be sure that all of the million apps currently available across all app stores will not be on display.

As with any presentation, perfection is achieved not when there is nothing left to add but when there is nothing left to take away. Just as you can't (and wouldn't want to) walk into a museum and see every manuscript from the 10th century on display, you won't be able to walk into the Nieuwerijksmuseum in 3112 and see every app from the 21st century. Only those that are considered cultural treasures will be given pride of place in the galleries. Can something that apparently has no value be considered a cultural treasure? Will any of your apps be part of the presentation? I don't think I've yet produced anything that will be, and that needs to change.

#### Dark Silicon
About 10 years ago, we decided that the performance gains in single-core processors that come "for free" with advancing semiconductor processes were slowing down. Many chip makers switched to scaling the number of cores on a die, and promoted parallel programming for their products.

Today I learned that the free multicore lunch is over, too. You can no longer turn on all of the transistors in a single chip, so you can no longer get 2× the threads running by doubling the number of cores in your processor.

#### Data curation during a pandemic
Here’s what I’ve been working on (with others, of course) since February.

#### Depending on the self-interest of strangers
The title is borrowed from an economics article by Art Carden, which is of no further relevance to this post. Interesting read though, yes?

I'm enjoying the discussion in the iOS Developer Community™ about dependency of app makers on third-party libraries.

My main sources for what I will (glibly, and with a lot of simplification) summarise as the "anti-dependency" argument are a talk by Marcus Zarra which he gave (a later version than I saw of) at NSConference, and a blog post by Justin Williams.

This is not perhaps an argument that is totally against use of libraries, but an argument in favour of caution and conservatism. As I understand it, the position(s) on this side can be summarised as an attempt to mitigate the following risks:

if I incorporate someone else's library, then I'm outsourcing my understanding of the problem to them. If they don't understand it in the same way that I do, then I might not end up with a desired solution.
if I incorporate someone else's library, then I'm outsourcing my understanding of the code to them. If it turns out I need something different, I may not be able to make that happen.
incorporating someone else's library may mean bringing in a load of code that doesn't actually solve my problem, but that increases the cognitive load of understanding my product.

I can certainly empathise with the idea that bringing in code to solve a problem can be a liability. A large app I was involved in writing a while back used a few open source libraries, and all but one of them needed patching either to fix problems or to extend their capabilities for our novel setting. The one (known) bug that came close to ending up in production was due to the interaction between one of these libraries and the operating system.

But then there's all of my code in my software that's also a liability. The difference between my code and someone else's code, to a very crude level of approximation that won't stand up to intellectual rigour but is good enough for a first pass, is that my code cost me time to write. Other than that, it's all liability. And let's not accidentally give a free pass to platform-vendor libraries, which are written by the same squishy, error-prone human-meat that produces both the first- and third-party code.

The reason I find this discussion of interest is that at the beginning of OOP's incursion into commercial programming, the key benefit of the technique was supposedly that we could all stop solving the same problems and borrow or buy other peoples' solutions. Here are Brad Cox and Bill Hunt, from the August 1986 issue of Byte:

  Encapsulation means that code suppliers
can build. test. and document solutions to difficult user
interface problems and store them in libraries as reusable
software components that depend only loosely on the applications that use them. Encapsulation lets consumers assemble generic components directly into their applica­tions. and inheritance lets them define new application­ specific components by inheriting most of the work from generic components in the library.

Building programs by reusing generic components will seem strange if you think of programming as the act of assembling the raw statements and expressions of a programming language. The integrated circuit seemed just as strange to designers who built circuits from discrete elec­tronic components. What is truly revolutionary about object-oriented programming is that it helps programmers reuse existing code. just as the silicon chip helps circuit builders reuse the work of chip designers. To emphasise this parallel we call reusable classes Software-ICs.

In the "software-IC" formulation of OOP, CocoaPods, Ruby Gems etc. are the end-game of the technology. We take the work of the giants who came before us and use it to stand on their shoulders. Our output is not merely applications, which are fleeting in utility, but new objects from which still more applications can be built.

I look forward to seeing this discussion play out and finding out whether it moves us collectively in a new direction. I offer zero or more of the following potential conclusions to this post:

Cox and contemporaries were wrong, and overplayed the potential for re-use to help sell OOP and their companies' products.
The anti-library sentiment is wrong, and downplays the potential for re-use to help sell billable hours.
Libraries just have an image problem, and we can define some trustworthiness metric (based, perhaps, on documentation or automated test coverage) that raises the bar and increases confidence.
Libraries inherently work to stop us understanding low-level stuff that actually, sooner or later, we'll need to know about whether we like it or not.
Everyone's free to do what they want, and while the dinosaurs are reinventing their wheels, the mammals can outcompete them by moving the craft forward.
Everyone's free to do what they want, and while the library-importers' castles are sinking into the swamps due to their architectural deficiencies, the self-inventors can outcompete them by building the most appropriate structures for the tasks at hand.

#### Deprecating yarn
In which I help Oxford University CS department with their threading issues.

#### Design Patterns On Trial
Back in 1999, the OOPSLA conference held a show trial for the “Gang of Four": Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Five years earlier, they had released their book “Design Patterns: Elements of Reusable Object-Oriented Software” at the very same conference, an act of subversion now seen as a crime against computer science. After hearing the case for the prosecution and for the defence, a simple minority of the conference delegates in attendance at the kangaroo court found the defendants guilty as charged.

The full indictment makes for some interesting reading, and it is clear that some real points are being made with tongues firmly in cheeks. “The Accused, by making it possible to design object-oriented programs in C++, have inhibited the rightful growth of competing object-oriented languages such as Smalltalk, CLOS, and Java.”

Perhaps this is a nod to Alan Kay’s OOPSLA 1997 outburst about not having C++ in mind when he invented the term Object-Oriented Programming. It’s certainly true that Smalltalk and CLOS are way less popular for Object-Oriented software implementation now than C++, but…Java? Really?

I could be asking “is Java really less popular than C++” here. But it’s also a good question to ask whether Java is really used to write Object-Oriented programs; the practice is so rare that Kevlin Henney recently had to do a talk reminding people that it’s possible.

The Accused, by distilling hard-won design expertise into patterns, have encouraged novices to act like experts.

In other words, they have advanced the field. By making it easier to today’s programmers to stand on the shoulders of those who came before, they have given them the skills previously afforded only by expertise. What a crime against computer science, making the field accessible!

Examples abound in history of people being tried for historical crimes when it’s important to make an example in current society. After the restoration of the monarchy in England and Scotland, Oliver Cromwell was exhumed and beheaded. Let’s rake the gang of four over the coals again. Here’s my addition to their indictment.

The Accused have, through their success in claiming the “Design Patterns” phrase as their own, maliciously tricked decades of programmers into believing that the whole idea behind Design Patterns is the catalogue of 23 patterns in their own book.

The point of design patterns is to have a patterns language, a shared glossary of solutions to common problems. When someone cites a pattern such as “microservice architecture” or “monorepo”, others know what they are doing, what problem they are solving, and the shape and trade-offs of their chosen solution.

But because people learn about “Design Patterns” from the GoF book (or, more likely, from Head First Design Patterns), they come to learn that Design Patterns are about Prototype, Flyweight, Iterator, and Template Method. They do not need these because their languages support them, therefore Design Patterns are old hat that nobody needs any more.

In fact they do not need those patterns because we stand on the shoulders of giants, and learned that these patterns were ubiquitously helpful in 1994. People are still publishing new design patterns (using the “particularly awkward and odious pattern format” of the Gang of Four), but these tend to be relegated to some dusty corner of academia. Worse, they tend to be written because nobody has written them yet (the “traditional standards of academic originality"), rather than because they represent “hard-won design expertise”.

In summary: design patterns are useful, because they let us quickly communicate a familiar solution with known properties when faced with a similar problem. The catalogue of design patterns in the book “Design Patterns” are useful, and because they have been useful since 1994 many of them have been subsumed into our tools at very basic levels. That book is a catalogue of useful design patterns from 1994, it is not all there is to know about design patterns.

Design patterns are dead. Long live design patterns!

#### Designing a secure Cocoa application
That's the title of next month's CocoaHeads Swindon, and I'll be leading the presentation/discussion.  So if you want to learn a little about how to ensuring your Cocoa app doesn't give away the keys to the kingdom, or have some experiences to share with the rest of the group, come along!  We'll be at the Glue Pot, which is nice and near the train station as well as reasonably close to a car park.  We'll be congregating at 7:00 but will wait for everyone to be settled with a beer in their hand before starting ;-).

#### Detect the gagh lurking in your system!
Following up on my previous ability to get to the top of a Google search for a Klingon word (that one was chuvmey, as in my post Model, View, chuvmey) here is yet another attempt. At what? Why, at skewing the mental associations between science fiction television and the digital security industry, of course!Sophos Klingon Anti-Virus is a threat detection tool for Windows computers, but in Klingon. Ever wondered what Conficker and Rokeg blood pie have in common? No, neither have I. In fact, I doubt anyone has. Nonetheless, try out the tool and see what Romulan back-doors have been installed on your box.(N.B. this means we have to expand our remit from "Enterprise" security software, to include at least the "HMS Bounty" from Star Trek IV)

#### Detecting overflows, undefined behaviour and other nasties
You will remember that a previous post discussed what happens when you add one to an integer, and that the answer isn't always obvious. Indeed, the answer isn't always defined.

As it happens, there are plenty of weird cases that crop up when working with C and languages like it. You can't expect a boolean to be YES or NO. You can't assume that an enum variable only holds values from the enumeration. You can't assume that you know how long an array is, even if the caller told you. Just as adding one is subtle, so is dividing by minus one.

In each of these cases[*]—and others—what you should actually do is to check that the input to an operation won't cause a problem before doing the operation:

int safe_add(int n1, int n2) {
  if(n2 > 0) assert(INT_MAX - n2 &lt; n1); //or throw a floating point exception or otherwise signal not to use the result
  if(n2 &lt; 0) assert(INT_MIN - n2 > n1);
  return n1 + n2;
}

But who does that? Thankfully, the compiler writers do. Coming up in a future release of clang is a collection of sanitisers that insert runtime checks for the things described above. If you're the kind of person who writes assertions like the above in your code, you can swap all that for sanitisers enabled in your debug builds. If you're not the kind of person who writes those assertions, you probably should enable these sanitisers, then go and find out where else you should be adding assertions.

In code that deals with input from other processes, machines or the outside world, you could consider enabling sanitisers even in release builds. They'll cause your app to report where it encounters overflows, underflows, and other potential security problems. If you don't think it's a good enough better option, you should be writing explicit checks for bad data and application-specific failure behaviour.

So, how does this work? Compiling with the sanitiser options inserts checks of the sort shown above into the compiled code. These checks are evaluated at runtime (sort of; for array bounds checking, the size of the array must be known when compiling but the check is still done at runtime) and the process prints a helpful message if the checked condition fails. Let's look at an example!


#include &lt;stdio.h>
#include &lt;limits.h>

int main(int argc, char *argv[]) {
	printf("%d + %d = %d\n", INT_MAX, 1, INT_MAX + 1);
	return 0;	
}


Compiling that with default settings "works", but results in undefined behaviour:

clang -o mathfail mathfail.c
./mathfail
2147483647 + 1 = -2147483648

So let's try to insert some sanity!

clang -fsanitize=integer -o mathfail mathfail.c
./mathfail
./mathfail.c:5:47: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
2147483647 + 1 = -2147483648


Another example: dividing by zero.


#include &lt;stdio.h>

int main(int argc, char *argv[]) {
	int x=2, y=0;
	printf("x/y = %d\n", x/y);
	return 0;
}

./mathfail.c:5:24: runtime error: division by zero


I wonder how many of the programs I've written in the past would trigger sanity failures with these new tools. I wonder how many of those are still in use.

[*] With the exception of booleans. As Mark explains in the linked post, you can always compare a boolean to 0, which is the only value that means false.

#### Détournement and Recuperation
Letterists International probably invented the ideas behind free software and creative commons. They created the idea of détournement, in which existing mainstream logos and slogans were subverted for anarchist, satirical and other radical political purposes, like the picture of the Queen defaced in the cover of the Sex Pistols single.

The counterpart to détournement is recuperation, in which radical ideas are appropriated by the cultural mainstream to be re-used in safe contexts, like the use of Thunderclap Newman's "Something in the air" to advertise an airline.

Free Software enables both of these: mainstream software made available under free software licences like the GPL can be put to radical new uses because of the terms of the GPL. But it also enables software built to political ends, and shared under those terms, to be put to mainstream uses.

If anything, open source software is a bigger enabler of recuperation. By relieving receivers of the pay-it-forward obligation, open source licences make it easier for incumbent organisations using traditional software model to make use of ideologically-motivated software publication.

#### Diagnosing a Docker image build problem
My Python script for Global.health was not running in production, because it couldn't find some imports. Now the real solution is to package up the imports with setuptools and install them at runtime (we manage the environments with poetry), but the quick solution is to fix up the path so that they get imported anyway. Or so I thought.

The deployment lifecycle of this script is that it gets packaged into a Docker image and published to Amazon Elastic Container Repository. An EventBridge event triggers a Batch job definition using that image to be queued. So to understand why the imports aren't working, we need to understand the Docker image.

docker create --name broken_script sha256:blah gives me a container based on the image. Previously I would have started that image and launched an interactive shell to poke around, but this time I decided to try something else: docker export broken_cleanup | tar tf - gives me the filesystem listing (and all I would've done with the running shell was various ls incantations, so that's sufficient).

Indeed my image had various library files alongside the main script:

/app/clean_old_ingestion_source_files.py
/app/EventBridgeClient.py
/app/S3Client.py
/app/__init__.py

Those supporting files should be in a subfolder. My copy command was wrong in the Dockerfile:

COPY clean_old_ingestion_source_files.py aws_access ./

This copies the content of aws_access into the current folder, I wanted to copy the folder (and, recursively, its content). Simple fix: break that line into two, putting the files in their correct destinations. Now rebuild the image, and verify that it is fixed. This time I didn't export the whole filesystem from a container, I exported the layers from the image.

docker image save sha256:blah | tar xf -

This gives me a manifest.json showing each layer, and a tar file with the content of that layer. Using this I could just get the table of content for the layer containing my Python files, and confirm that they are now organised correctly.

#### Did that restructuring work actually help?
Before getting into the meat of this post, I'd like to get into the meta of this post. This essay, and I imagine many in this blog [Ed: by which I meant the blog this has been imported from], will be treading a fine line. The intended aim is to question accepted industry practice, and find results consistent or inconsistent with the practice as a beneficial task to perform. I'm more likely to select papers that appear to refute the practice, as that's more interesting and makes us introspect the way we work more than does affirmation. The danger is that this skates too close to iconoclasm, as expressed in the Goto Copenhagen talk title Is it just me or is everything shit?. My intention isn't to say that whatever we're doing is wrong, just to provide some healthy inspection and analysis of our industry.

Legacy Software Restructuring: Analyzing a Concrete Case

The thread in this paper is that metrics that have long been used to measure the quality of source code—metrics related to coupling and cohesion—may not actually be relevant to the problems developers have to solve. Firstly, the jargon:

coupling refers to the connections between the part of the software (module, class, function, whatever) under consideration and the rest of the software system. Received wisdom is that lower coupling (i.e. fewer connections that are less-tightly intertwined) is better.
cohesion refers to the relatedness of the tasks performed by the (module, class, function, whatever) under consideration. The more different responsibilities a component provides or uses, the lower its cohesion. Received wisdom is that higher cohesion (i.e. fewer responsibilities per module) is better.

We're told that striving for low coupling and high cohesion will make the parts of our software reusable and replaceable, and will reduce the number of code sites we need to change when we want to fix bugs in the future. The focus of this paper is on whether the metrics we use as proxies for these properties actually represent enhancements to the code; in other words, whether we have a systematic way to decide whether a change is an improvement or not.

Approach

The way in which the authors test their metrics is necessarily problematic. There is no objective standard against which they can be prepared—if there were, we'd have an objective standard and we could all go home. They hypothesise that any restructuring effort by a development team must represent an improvement in the codebase: if you didn't think a change was better, why would you make that change?

Necessity is one such reason. Consider the following thought process:

  I need to add this feature to my product, this change was unforeseen at design time, so the architecture doesn't really support it. I'm not very happy about the this, but shoehorning it in here is the simplest way to support what I need.

To understand other problems with this methodology, a one-paragraph introduction to the postmodern philosophy of software engineering is required. Software, it says, supports not some absolute set of requirements that were derived from studying the universe, but the ad-hoc set of interactions between the various people who engage with the software system. Indeed, the software system itself modifies these interactions, creating a feedback loop that in fact modifies the requirements of the software that was created. Some of the results of this philosophy[*] are expressed in "Manny" Lehman's Laws of Software Engineering, which are also cited by the paper I'm talking about here. The authors offer one of Lehman's Laws as:

[*] I don't consider Lehman's laws to be objectively true of software artefacts, but to be hypotheses that arise from a particular philosophy of software. I also think that philosophy has value.

  Considering Lehman’s law of software evolution, such systems would already have suffered a decrease in their quality due to the maintenance. This would increase the probability that the restructuring has a better modular quality.

This statement is inconsistent. On the one hand, this change improves the quality of some software. On the other hand, the result of a collection of such changes is to decrease the quality. Now there's nothing to say that a particular change won't be an improvement; but there's also nothing to say that the observed change has this property.[*] The postmodern philosophy adds an additional wrinkle: even if this change is better, it's only better &#95;as perceived by the people currently working with the system&#95;. Others may have different ideas. We saw, in discussing the teaching of programming, that even experienced programmers can have difficulty reading somebody else's code. I wouldn't find it a big stretch to posit that different people have different ideas of what constitutes "good" modular decomposition, and that therefore a different set of programmers would think this change to be worse.

[*]Actually I think the sentence in the paper might just be broken; remember that I found this on the preprint server so it might not have been reviewed yet. One of Lehman's laws says that, for "E-type" software (by which he means systems that evolve with their environment—in other words, systems where a postmodern appraisal is applicable), the software will gradually be perceived as &#95;reducing&#95; in quality if no maintenance work goes into it. That's because the system is evolving while the software isn't; the requirements change without the software catching up.

Results and Discussion

The authors found that, for three particular revisions of Eclipse, the common metrics for coupling and cohesion did not monotonically "improve" with successive restructuring efforts. In some cases, both coupling and cohesion decreased in the same effort. In addition, they found that the number and extent of cyclic dependencies between Java packages increased with every successive version of the platform.

It's not really possible to choose a conclusion to draw from these results:

maybe the dogma of increasing cohesion and decreasing coupling is misleading.
maybe the metrics used to measure those properties were poorly chosen (though they are commonly-chosen).
maybe the Eclipse developers use some other measurement of quality that the authors didn't ask about.
maybe some of the Eclipse engineers do take these properties into account, and some others don't, and we [can't - added on import] even draw general conclusions about Eclipse.

So this paper doesn't demonstrate that cohesion and coupling metrics are wrong. But it does raise the important question: might they be not right? If you're relying on some code metrics derived from received wisdom or dogma, it's time to question whether they really apply to what you do.

#### Did that work? Maybe.
A limitation with yesterday's error-preserving approach is that it leaves you on your own to recover from problems. Assuming your error definitions are sufficiently granular, this should be straightforward but tedious. Find out what went wrong, recover from it, then replay everything that happened afterwards.

Recovering from failures automatically is difficult in general, after all, if we could do that there wouldn't have been a failure in the first place. But there's no reason to then have a bunch of code that replays the rest of the workflow from the point of recovery, you've already written that code on the happy path.

Why can't we just do that? On an error, why don't we recover from the immediate error then go back in time to the point where it occurred and start again with our recovered value? Seems straightforward enough, let's do it. In doing so, let's borrow (at least in a superficial fashion) the idea of the optional object.

A Maybe object represents the possibility that a value exists, and the ability to find out whether it does. Should it not contain a value, you should be able to find out why not.

@interface Maybe : NSObject

@property (nonatomic, strong, readonly) NSError *error;

- (BOOL)hasValue;
- recoverWithStartingValue:value;

+ just:value;
+ none:aClass error:(NSError *)anError;

@end

I have a value

OK, let's say that what you wanted to do succeeded, and you want to use the result object. It'd be kindof sucky if you had to test whether the Maybe contained a value at every step of a long operation, and unwrap it to get the object out that you care about. So let's not make you do that.

@interface Just : Maybe
-initWithValue:value;
@end

@implementation Just
{
    id _value;
}

-initWithValue:value {
    self = [super init];
    if (self) {
        _value = value;
    }
    return self;
}

-(NSError *)error {
    NSAssert(NO, @"No error in success case");
    return nil;
}
-(BOOL)hasValue { return YES; }
-recoverWithStartingValue:value {
    NSAssert(NO, @"Cannot recover from success");
    return nil;
}
-forwardingTargetForSelector:(SEL)aSelector { return _value; }

@end

OK, if everything succeeds you can use the Maybe result (which will be Just the value) as if it is the value itself.

I don't have a value

The other case is that your operation failed, so we need to represent that. We need to know what type of object you don't have(!), which will be useful because we can then treat the lack of value as if it is an instance of the value. How will this work? In None, the no-value version of a Maybe, we'll just absorb every message you send.

@interface None : Maybe
-initWithClass:aClass error:(NSError *)anError;
@end

@implementation None
{
    Class _class;
    NSError *_error;
    NSMutableArray *_invocations;
}

-initWithClass:aClass error:(NSError *)anError {
    self = [super init];
    if (self) {
        _class = aClass;
        _error = anError;
        _invocations = [NSMutableArray array];
    }
    return self;
}

-(NSError *)error { return _error; }
-(BOOL)hasValue { return NO; }

-methodSignatureForSelector:(SEL)aSelector {
    return [_class instanceMethodSignatureForSelector:aSelector];
}

-(void)forwardInvocation:(NSInvocation *)anInvocation {
    id returnValue = self;
    [_invocations addObject:anInvocation];
    [anInvocation setReturnValue:&amp;returnValue];
}

-recoverWithStartingValue:value {
    id nextObject = value;
    while([_invocations count]) {
        id invocation = [_invocations firstObject];
        [_invocations removeObjectAtIndex:0];
        [invocation invokeWithTarget:nextObject];
        [invocation getReturnValue:&amp;nextObject];
    }
    return nextObject;
}

@end

Again, there's no need to unwrap a None (and that makes no sense, because it doesn't contain anything). You just use it as if it is the thing that it represents (a lack of).

Recovering

You go through your complex process, and somewhere along the way it failed. At this point your Maybe answer turned into None, because you didn't get a value at some step. But it carried on paying attention to what you wanted to do.

Now it's time to turn back the clock. Looking at the error I get from the None, I can see what step failed and what I need to do to be in a position to try again. When I make that happen, I'll get some object which would've been valid at that intermediate point in my operation.

Because the None was paying attention to what I tried to do to it, I can replay the whole process from the point where it failed, using the result from my recovery path.

Worked Example

Here's an object that requires two steps to use. Either step could fail, and one of them does unless you take action to fix it up.

@interface AnObject : NSObject

@property (nonatomic, assign, getter=isRecovered) BOOL recovered;

-this;
-that;

@end

@implementation AnObject

-this
{
    return [self isRecovered] ? [Maybe just:self] :
        [Maybe none:[self class] error:[NSError errorWithDomain:@"Nope"
                                                           code:23
                                                       userInfo:@{}]];
}

-that
{
    return [Maybe just:@"Winning"];
}

@end

In using this object, I want to compose the two steps. If it goes wrong, I know what to do to recover, but I don't want to have to explicitly write out the workflow twice if I got it right the first time.

int main(int argc, char *argv[]) {
    @autoreleasepool {
        id anObject = [AnObject new];
        id result = [[anObject this] that];
        if ([result hasValue]) {
            NSLog(@"success: %@", [result lowercaseString]);
        } else {
            NSLog(@"failed with error %@, recovering...", [result error]);
            [anObject setRecovered:YES];
            result = [result recoverWithStartingValue:anObject];
            NSLog(@"ended up with %@", [result uppercaseString]);
        }
    }
}

Conclusion

The NSError-star-star convention lets you compose possibly-failing messages and find out either that it succeeded, or where it went wrong. But it doesn't encapsulate what would have happened had it gone right, so you can't just rewind time to where things failed and try again. It is possible to do so, simply by encapsulating the idea that something might work...maybe.

#### Did the UK create a new kind of "Crypto Mule"?
It's almost always the case that a new or changed law means that there is a new kind of criminal, because there is by definition a way to contravene the new law. However, when the law allows the real criminals to hide behind others who will take the fall, that's probably a failure in the legislation.

The Regulation of Investigatory Powers Act 2000 may be doing just that. In Section 51, we find that a RIPA order to disclose information can be satisfied by disclosing the encryption key, if the investigating power already has the ciphertext.

Now consider this workflow. Alice Qaeda needs to send information confidentially to Bob Laden (wait: Alice and Bob aren't always the good guys? Who knew?). She doesn't want it intercepted by Eve Sergeant, who works for SOCA (wait: Eve isn't always the bad guy etc.). So she prepares the information, and encrypts it using Molly Mule's public key. She then gives the ciphertext to Michael Mule.

Michael's job is to get from Alice's location to Bob's. Molly is also at Bob's location, and can use her private key to show the plaintext to Bob. She doesn't necessarily see the plaintext herself; she just prepares it for Bob to view.

Now Alice and Bob are notoriously difficult for Eve to track down, so she stops Michael and gets her superintendent to write a RIPA demand for the encryption key. But Michael doesn't have they key. He'll still probably get sent down for two years on a charge of failing to comply with the RIPA request. Even if Eve manages to locate and serve Molly with the same request, Molly just needs to lie about knowing the key and go down for two years herself.

The likelihood is that Molly and Michael will be coerced into performing their roles, just as mules are in other areas of organised crime. So has the legislation, in trying to set out government snooping permissions, created a new slave trade in crypto mules?

#### Did you miss my NSConference talk?
The annotated presentation slides are now available to download in Keynote '08 format! Sorry you couldn't make it, and I hope the slides are a reasonable proxy for the real thing.

#### Digital Declutter
I've been reading and listening to various books about the attention/surveillance economy, the rise of fascism in the Anglosphere and beyond, and have decided to disconnect from the daily outrage and the impotent swiping of "social" "content". The most immediately actionable advice came from Cal Newport's Digital Minimalism. I will therefore be undertaking a digital declutter in May.

Specifically this means:

no social media. In fact I haven't been on most of them all of April, so this is already in play. By continuing it into May, I intend to do a better job of choosing things to do when I'm not hitting refresh.
alerts on chat apps on for close friends and family only.
streaming TV only when watching with other people.
Email once per day.
no RSS.
audiobooks only while driving.
Slack once per day.
Web browsing only when it progresses a specific work, or non-computering, task.
at least one walk per day, of at least half an hour, with no technology.
Phone permanently in Do Not Disturb mode.

It's possible that I end up blogging more, if that's what I start thinking of when I'm not browsing the twitters. Or less. We'll find out over the coming weeks.

My posts for De Programmatica Ipsum are written and scheduled, so service there is not interrupted. And I'm not becoming a hermit, just digitally decluttering. Arrange Office Hours, come to Brum AI, or find me somewhere else, if you want to chat!

#### Discipline doesn't scale
If programmers were just more disciplined, more professional, they'd write better software. All they need is a code of conduct telling them how to work like those of us who've worked it out.

The above statement is true, which is a good thing for those of us interested in improving the state of software and in helping our fellow professionals to improve their craft. However, it's also very difficult and inefficient to apply, in addition to being entirely unnecessary. In the common parlance of our industry, "discipline doesn't scale".

Consider the trajectory of object lifecycle management in the Objective-C programming language, particularly the NeXT dialect. Between 1989 and 1995, the dominant way to deal with the lifecycle of objects was to use the +new and -free methods, which work much like malloc/free in C or new/delete in C++. Of course it's possible to design a complex object graph using this ownership model, it just needs discipline, that's all. Learn the heuristics that the experts use, and the techniques to ensure correctness, and get it correct.

But you know what's better? Not having to get that right. So around 1994 people introduced new tools to do it an easier way: reference counting. With NeXTSTEP Mach Kit's NXReference protocol and OpenStep's NSObject, developers no longer need to know when everybody in an app is done with an object to destroy it. They can indicate when a reference is taken and when it's relinquished, and the object itself will see when it's no longer used and free itself. Learn the heuristics and techniques around auto releasing and unretained references, and get it correct.

But you know what's better? Not having to get that right. So a couple of other tools were introduced, so close together that they were probably developed in parallel[*]: Objective-C 2.0 garbage collection (2006) and Automatic Reference Counting (2008). ARC "won" in popular adoption so let's focus there: developers no longer need to know exactly when to retain, release, or autorelease objects. Instead of describing the edges of the relationships, they describe the meanings of the relationships and the compiler will automatically take care of ownership tracking. Learn the heuristics and techniques around weak references and the "weak self" dance, and get it correct.

[*] I'm ignoring here the significantly earlier integration of the Boehm conservative GC with Objective-C, because so did everybody else. That in itself is an important part of the technology adoption story.

But you know what's better? You get the idea. You see similar things happen in other contexts: for example C++'s move from new/delete to smart pointers follows a similar trajectory over a similar time. The reliance on an entire programming community getting some difficult rules right, when faced with the alternative of using different technology on the same computer that follows the rules for you, is a tough sell.

It seems so simple: computers exist to automate repetitive information-processing tasks. Requiring programmers who have access to computers to recall and follow repetitive information processes is wasteful, when the computer can do that. So give those tasks to the computers.

And yet, for some people the problem with software isn't a lack of automation but a lack of discipline. Software would be better if only people knew the rules, honoured them, and slowed themselves down so that instead of cutting corners they just chose to ignore important business milestones instead. Back in my day, everybody knew "no Markdown around town" and "don't code in an IDE after Labour Day", but now the kids do whatever they want. The motivations seem different, and I'd like to sort them out.

Let's start with hazing. A lot of the software industry suffers from "I had to go through this, you should too". Look at software engineering interviews, for example. I'm not sure whether anybody actually believes "I had to deal with carefully ensuring NUL-termination to avoid buffer overrun errors so you should too", but I do occasionally still hear people telling less-experienced developers that they should learn C to learn more about how their computer works. Your computer is not a fast PDP-11, all you will learn is how the C virtual machine works.

Just as Real Men Don't Eat Quiche, so real programmers don't use Pascal. Real Programmers use FORTRAN. This motivation for sorting discipline from rabble is based on the idea that if it isn't at least as hard as it was when I did this, it isn't hard enough. And that means that the goalposts are movable, based on the orator's experience.

This is often related to the term of their experience: you don't need TypeScript to write good React Native code, just Javascript and some discipline. You don't need React Native to write good front-end code, just JQuery and some discipline. You don't need JQuery…

But along with the term of experience goes the breadth. You see, the person who learned reference counting in 1995 and thinks that you can only really understand programming if you manually type out your own reference-changing events, presumably didn't go on to use garbage collection in Java in 1996. The person who thinks you can only really write correct software if every case is accompanied by a unit test presumably didn't learn Eiffel. The person who thinks that you can only really design systems if you use the Haskell type system may not have tried OCaml. And so on.

The conclusion is that for this variety of disciplinarian, the appropriate character and quantity of discipline is whatever they had to deal with at some specific point in their career. Probably a high point: after they'd got over the tricky bits and got productive, and after you kids came along and ruined everything.

Sometimes the reason for suggesting the disciplined approach is entomological in nature, as in the case of the eusocial insect the "performant" which, while not a real word, exists in greater quantities in older software than in newer software, apparently. The performant is capable of making software faster, or use less memory, or more concurrent, or less dependent on I/O: the specific characteristics of the performant depend heavily on context.

The performant is often not talked about in the same sentences as its usual companion species, the irrelevant. Yes, there may be opportunities to shave a few percent off the runtime of that algorithm by switching from the automatic tool to the manual, disciplined approach, but does that matter (yet, or at all)? There are software-construction domains where specific performance characteristics are desirable, indeed that's true across a lot of software. But it's typical to focus performance-enhancing techniques on the bits where they enhance performance that needs enhancing, not to adopt them across the whole system on the basis that it was better when everyone worked this way. You might save a few hundred cycles writing native software instead of using a VM for that UI method, but if it's going to run after a network request completes over EDGE then trigger a 1/3s animation, nobody will notice the improvement.

Anyway, whatever the source, the problem with calls for discipline is that there's no strong motivation to become more disciplined. I can use these tools, and my customer is this much satisfied, and my employer pays me this much. Or I can learn from you how I'm supposed to be doing it, which will slow me down, for…your satisfaction? So you know I'm doing it the way it's supposed to be done? Or so that I can tell everyone else that they're doing it wrong, too? Sounds like a great deal.

Therefore discipline doesn't scale. Whenever you ask some people to slow down and think harder about what they're doing, some fraction of them will. Some will wonder whether there's some other way to get what you're peddling, and may find it. Some more will not pay any attention. The dangerous ones are the ones who thought they were paying attention and yet still end up not doing the disciplined thing you asked for: they either torpedo your whole idea or turn it into not doing the thing (see OOP, Agile, Functional Programming). And still more people, by far the vast majority, just weren't listening at all, and you'll never reach them.

Let's flip this around. Let's look at where we need to be disciplined, and ask if there are gaps in the tool support for software engineers. Some people want us to always write a failing test and make it pass before adding any code (or want us to write a passing test and revert our changes if it accidentally fails): does that mean our tools should not let us write code for which there's no test? Does the same apply for acceptance tests? Some want us to refactor mercilessly; does that mean our design tools should always propose more parsimonious alternatives for passing the same tests? Some say we should get into the discipline of writing code that always reveals its intent: should the tools make a crack at interpreting the intention of the code-as-prose?

#### Discovery
I think the last technical conference I attended was FOSDEM last year, and now I'm sat in the lobby of the Royal Library of Brussels working on a project that I want to take to some folks at this year's FOSDEM, and checking the mailing lists of some projects I'm interested in to find meetups and relevant talks.

It's been even longer since I spoke at a conference, I believe it would have been App Builders in 2016. When I left Facebook in 2015 with the view of taking a year out of software, I also ended up taking myself out of community membership and interaction. I felt like I didn't know who I was, and wasn't about to find out by continually defining myself in terms of other people.

I would have spoken at dotSwift in 2016, but personal life issues probably also related to not knowing who I was stopped me from doing that. In April, I went to Zurich and gave what would be my last talk as a programmer to anyone except immediate colleagues for over 18 months.

In the meantime, I explored bits of the software world outside of my immediate experience. I worked on a Qt app in high-performance computing, and I'm working on "full-stack" Javascript now.

[Full-stack Javascript is, of course, a lie, but I'd rather it keeps its current meaning as "the application bit of the stack in Javascript" than took a more correct sense.]

During this time, I found that I don't mind so much what I'm working on: I have positive and negative opinions of all of it. I have lots of strong opinions, and lots of syntheses of other ideas, and need to be a member of a community that can share and critique these opinions, and extend and develop these syntheses.

Which is why, after a bit of social wilderness, I'm reflecting on this week's first conference and planning my approach to the second.

#### Does that thing you like doing actually work?
Genuine question. I've written before about Test-Driven Development, and I'm sure some of you practice it: can you show evidence that it's better than (or, for that matter, evidence that it's worse than) some other practice? Statistically significant evidence?

How about security? Can you be confident that there's a benefit to spending any money or time on information security countermeasures? On what should it be spent? Which interventions are most successful? Can you prove that?

I am, of course, asking whether there's any evidence in software engineering. I ask rhetorically, because I believe that there isn't—or there isn't a lot that's in a form useful to practitioners. A succinct summary of this position comes courtesy of Anthony Finkelstein:

For the most part our existing state-of-practice is based on anecdote. It is, at its very best quasi-evidence-based. Few key decisions from the choice of an architecture to the configuration of tools and processes are based on a solid evidential foundation. To be truthful, software engineering is not taught by reference to evidence either. This is unacceptable in a discipline that aspires to engineering science. We must reconstruct software engineering around an evidence-based practice.

Now there is a discipline of Evidence-Based Software Engineering, but herein lies a bootstrapping problem that deserves examination. Evidence-Based [ignore the obvious jokes, it's a piece of specific jargon that I'm about to explain] practice means summarising the significant results in scientific literature and making them available to practitioners, policymakers and other "users". The primary tools are the systematic literature review and its statistics-heavy cousin, the meta-analysis.

Wait, systematic literature review? What literature? Here's the problem with trying to do EBSE in 2012. Much software engineering goes on behind closed doors in what employers treat as proprietary or trade-secret processes. Imagine that a particular project is delayed: most companies won't publish that result because they don't want competitors to know that their projects are delayed.

Even for studies, reports and papers that do exist, they're not necessarily accessible to the likes of us common programmers. Let's imagine that I got bored and decided to do a systematic literature survey of whether functional programming truly does lead to fewer concurrency issues than object-oriented programming.[*] I'd be able to look at articles in the ACM Digital Library, on the ArXiv pre-print server, and anything that's in Leamington Spa library (believe me, it isn't much). I can't read IEEE publications, the BCS Computer Journal, or many others because I can't afford to subscribe to them all. And there are probably tons of journals I don't even know about.

[*]Results of asking about this evidence-based approach to paradigm selection revealed that either I didn't explain myself very well or people don't like the idea of evidence mucking up their current anecdotal world views.

So what do we do about this state of affairs? Actually, to be more specific: if our goal is to provide developers with better access to evidence from our field, what do we do?

I don't think traditional journals can be the answer. If they're pay-to-read, developers will never see them. If they're pay-to-write, the people who currently aren't supplying any useful evidence still won't.

So we need something lighter weight, free to contribute to and free to consume; and we probably need to accept that it then won't be subject to formal peer review (in exactly the same way that Wikipedia isn't).

I've argued before that a great place for this work to be done is the Free Software Foundation. They've got the components in place: a desire to prove that their software is preferable to commercial alternatives; public development projects with some amount of central governance; volunteer coders willing to gain karma by trying out new things. They (or if not them, Canonical or someone else) could easily become the home of demonstrable quality in software production.

Could the proprietary software developers be convinced to open up on information about what practices do or don't work for them? I believe so, but it wouldn't be easy. Iteratively improving practices is a goal for both small companies following Lean Startup and similar techniques, and large enterprises interested in process maturity models like CMMI. Both of these require you to know what metrics are important; to measure, test, improve and iterate on those metrics. This can be done much more quickly if you can combine your results from those of other teams—see what already has or hasn't worked elsewhere and learn from that.

So that means that everyone will benefit if everyone else is publishing their evidence. But how do you bootstrap that? Who will be first to jump from a culture of silence to a culture of sharing, the people who give others the benefit of their experience before they get anything in return?

I believe that this is the role of the platform companies. These are the companies whose value lies not only in their own software, but in the software created by ISVs on their platforms. If they can help their ISVs to make better software more efficiently, they improve their own position in the market.

#### Does the history of making software exist?
A bit of a repeated theme in the construction of APPropriate Behaviour has been that I've tried to position certain terms or concepts in their historical context, and found it difficult, or impossible to do so with sufficient rigour. There's an extent to which I don't want the book to become historiographical so have avoided going too deep into that angle, but have discovered that either no-one else has or that if they have, I can't find their work.

What often happens is that I can find a history or even many histories, but that these aren't reliable. I already wrote in the last post on this blog about the difficulties in interpreting references to the 1968 NATO conference; well today I read another two sources that have another two descriptions of the conference and how it kicked off the software crisis. Articles like that linked in the above post help to defuse some of the myths and partisan histories, but only in very specific domains such as the term "software crisis".

Occasionally I discover a history that has been completely falsified, such as the great sequence of research papers that "prove" how some programmers are ten (or 25, or 1000) times more productive than others or those that "prove" bugs cost 100x more to fix in maintenance. Again, it's possible to find specific deconstructions of these memes (mainly by reading Laurent Bossavit), but having discovered the emperor is naked, we have no replacement garments with which to clothe him.

There are a very few subjects where I think the primary and secondary literature necessary to construct a history exist, but that I lack the expertise or, frankly, the patience to pursue it. For example you could write a history of the phrase "software engineering", and how it was introduced to suggest a professionalism beyond the craft discipline that went before it, only to become a symbol of lumbering lethargy among adherents of the craft discipline that came after it. Such a thing might take a trained historian armed with a good set of library cards a few years to complete (the book The Computer Boys Take Over covers part of this story, though it is written for the lay reader and not the software builder). But what of more technical ideas? Where is the history of "Object-Oriented"? Does that phrase mean the same thing in 2013 as in 1983? Does it even mean the same thing to different people in 2013?

Of course there is no such thing as an objective history. A history is an interpretation of a collection of sources, which are themselves interpretations drawn from biased or otherwise limited fonts of knowledge. The thing about a good history is that it lets you see what's behind the curtain. The sources used will all be listed, so you can decide whether they lead you to the same conclusions as the author. It concerns me that we either don't have, or I don't have access to, resources allowing us to situate what we're trying to do today in the narrative of everything that has gone before and will go hence. That we operate in a field full of hype and innuendo, and lack the tools to detect Humpty Dumptyism and other revisionist rhetoric.

With all that said, are the histories of the software industry out there? I don't mean the collectors like the museums, who do an important job but not the one I'm interested in here. I mean the histories that help us understand our own work. Do degrees in computer science, to the extent they consider "real world" software making at all, teach the history of the discipline? Not the "assemblers were invented in 1949 and the first binary tree was coded in 19xy" history, but the rise and fall of various techniques, fads, disciplines, and so on? Or have I just volunteered for another crazy project?

I hope not, I haven't got a good track record at remembering my library cards. Answers on a tweet, please.

#### Dogma-driven development
You can find plenty of dogmatic positions in software development, in blogs, in podcasts, in books, and even in academic articles. "You should (always/never) write tests before writing code." "Pair programming is a (good/bad) use of time." "(X/not X) considered harmful." "The opening brace should go on the (same/next) line."

Let us ignore, for the moment, that only a maximum of 50% of these commandments can actually be beneficial. Let us skip past the fact that demonstrating which is the correct position to take is fraught with problems. Instead we shall consider this question: dogmatic rules in software engineering are useful to whom?

The Dreyfus model of skill acquisition tells us that novices at any skill, not just programming, understand the skill in only a superficial way. Their recollection of rules is non-situational; in other words they will try to apply any rule they know at any time. Their ability to recognise previously penchant free scenarios is small-scale, not holistic. They make decisions by analysis.

The Dreyfus brothers proposed that the first level up from novice was competence. Competents have moved to a situational recollection of the rules. They know which do or do not apply in their circumstances. Those who are competent can become proficient, when their recognition becomes holistic. In other words, the proficient see the whole problem, rather than a few small disjointed parts of it.

After proficiency comes expertise. The expert is no longer analytical but intuitive, they have internalised their craft and can feel the correct way to approach a problem.

"Rules" of software development mean nothing to the experts or the proficient, who are able to see their problem for what it is and come up with a context-appropriate solution. They can be confusing to novices, who may be willing to accept the rule as a truth of our work but unable to see in which contexts it applies. Only the competent programmers have a need to work according to rules, and the situational awareness to understand when the rules apply.

But competent programmers are also proficient programmers waiting to happen. Rather than being given more arbitrary rules to follow, they can benefit from being shown the big picture, from being led to understand their work more holistically than as a set of distinct parts to which rulaes can be mechanistically - or dogmatically - applied.

Pronouncements like coding standards and methodological commandments can be useful, but not on their own. By themselves they help competent programmers to remain competent. They can be situated and evaluated, to help novices progress to competence. They can be presented as a small part of a bigger picture, to help competents progress to proficiency. As isolated documents they are significantly less valuable.

Dogmatism considered harmful.

#### Dogmatic paradigmatism
First, you put all of your faith in structured programming, and you got burned. You found it hard to associate the operations in your software with the data upon which they act, and to make sure that the expectations made on the data in one place are satisfied when that data has been modified in that other place, or over there in yet another place. Clearly structured programming is broken.

Then, you put all of your faith in object-oriented programming, and you got burned. You found it hard to follow the flow of a program when it jumps in and out of different classes, and to see which parts were coupled to what. Clearly object-oriented programming is broken.

Then, you put all of your faith in functional programming, and you got burned. You found it hard to represent real business processes in terms of immutable data structures and pure functions, and to express changes to the operating environment without using side effects. Clearly functional programming is broken.

Or maybe it's you. Maybe, rather than relying on faith to make these conceptual thought frameworks do what you need from them, you could have thought about the concepts.

#### Don't be a dick
In a recent post on device identifiers, I wrote a guideline that I've previously invoked when it comes to sharing user data. Here is, in both more succinct and complete form than in the above-linked post, the Don't Be A Dick Guide to Data Privacy:

The only things you are entitled to know are those things that the user told you.
The only things you are entitled to share are those things that the user permitted you to share.
The only entities with which you may share are those entities with which the user permitted you to share.
The only reason for sharing a user's things is that the user wants to do something that requires sharing those things.

It's simple, which makes for a good user experience. It's explicit, which means culturally-situated ideas of acceptable implicit sharing do not muddy the issue.

It's also general. One problem I've seen with privacy discussions is that different people have specific ideas of what the absolutely biggest privacy issue that must be solved now is. For many people, it's location: they don't like the idea that an organisation (public or private) can see where they are at any time. For others, it's unique identifiers that would allow an entity to form an aggregate view of their data across multiple functions. For others, it's conversations they have with their boss, mistress, whistle-blower or others.

Because the DBADG mentions none of these, it covers all of these. And more. Who knows what sensors and capabilities will exist in future smartphone kit? They might use mesh networks that can accurately position users in a crowd with respect to other members. They could include automatic person recognition to alert when your friends are nearby. A handset might include a blood sugar monitor. The fact is that by not stopping to cover any particular form of data, the above guideline covers all of these and any others that I didn't think of.

There's one thing it doesn't address: just because a user wants to share something, should the app allow it? This is particularly a question that makers of apps for children should ask themselves. Children (and everybody else) deserve the default-private treatment of their data that the DBADG promotes. However, children also deserve impartial guidance on what it is a good or a bad idea to share with the interwebs at large, and that should be baked into the app experience. "Please check with a responsible adult before pressing this button" does not cut it: just don't give them the button.

#### Don't go there
From the title linky: &quot; Mobility is only available for Windows and Linux.&quot;  Good job no-one bases their mobile appliances on Mac OS X ;-)

#### Dos Amigans
Tomorrow evening (for me; 1800UTC on 10th Sept) Steven R. Baker and I will twitch-stream our journey learning how to write Amiga software. Check out dosamigans.tv!

#### Dynamic Method Dispatch in Object-Oriented Programming in Functional Programming in Swift
In the previous episode, I said that objects are functions that map their ivars onto methods. However, the objects that I demonstrated in the post were tables, structures of functions that closed over the ivars, like this:

struct List&lt;T&gt; {
  let count: () -&gt; Int
  let at: (Int) -&gt; T?
}

Functions and tables are actually interchangeable, so this is not a problem and there's no sleight of hand.

Here's a simple example: imagine a function that maps Booleans (instances drawn from the set Boolean) to integers (instances drawn from the set Int):

enum Boolean {
 case False
 case True
}

func intValue(b : Boolean) -&gt; Int
{
  switch b
  case .False:
    return 0
  case .True:
    return 1
}

This is, notation aside, identical to encoding the map in a table:

struct BooleanIntMap {
  let False = 0
  let True = 1
}

That's the state that my objects were in at the end of the previous post. So let's complete the story, and turn these objects from tables into functions.

Three Minute Hero

Along the way, I'll fix a deficiency in the previous version of these objects. When you got an object, it was in the form of a table of all of its methods, but typically you only want to call one at a time. In Objective-C, you pass an object a selector and it returns the appropriate method which you then call (well, that's how the GNU runtime works. The NeXT and Apple runtimes jump right into the returned function). See? An object really is a function that maps its variables onto the methods you want.

There are a limited (for now) number of legitimate selectors to send to an object, so they can easily be represented with an enumeration.

enum DynamicListSelectors {
  case count
  case at
}

Now it gets a bit complicated (Objective-C and Smalltalk are also this complicated, but use a different notation that hides it from you). My list object will be a function that maps these selectors onto methods, but the two methods have different signatures. I therefore need a type which can represent (function with the .count signature) OR (function with the .at signature).

In addition to representing enumerated constants, Swift's enums can also sum types. Remember that a type is just a set of that type's instances: a sum type is the union of two (or more) type sets. So the list function can return something of this type:

enum DynamicList&lt;T&gt; {
  case count (() -&gt; Int)
  case at ((Int) -&gt; T?)
}

There are two things to do now. One is to define the function that maps from a selector onto an instance of this sum type: this is our message dispatch function, or more succinctly, our object.

func EmptyDynamicList&lt;T&gt;()(_cmd: DynamicListSelectors) -&gt; DynamicList&lt;T&gt;
{
  switch _cmd {
  case .count:
    return DynamicList&lt;T&gt;.count({ return 0 })
  case .at:
    return DynamicList&lt;T&gt;.at({ (_) in return nil})
  }
}

But now we need to pull apart the enumeration in the client code to find the message implementation and invoke it. In principle, Smalltalk, Objective-C and other dynamic O-O languages have to do this too (in ObjC it works by knowing how to use function pointers of different types, and treating all the methods as function pointers), but the notation or loose type system hides it from you.

let emptyCountIMP : DynamicList&lt;Int&gt; = EmptyDynamicList()(_cmd: .count)
switch emptyCountIMP {
case .count(let f):
  f() // returns 0
default:
  assertionFailure("Implementation does not match selector")
}

Doing that every time you call a method would get boring quickly, averted by defining a couple of convenience functions.

func countOfList&lt;T&gt;(list: (DynamicListSelectors) -&gt; DynamicList&lt;T&gt;) -&gt; Int
{
  switch list(.count) {
  case .count(let f):
    return f()
  case .at(let f):
    assertionFailure("Implementation does not match selector")
    return 0 //unreached
  }
}

func elementInListAtIndex&lt;T&gt;(list: (DynamicListSelectors) -&gt; DynamicList&lt;T&gt;, index: Int) -&gt; T?
{
  switch list(.at) {
  case .at(let f):
    return f(index)
  case .count(let f):
    assertionFailure("Implementation does not match selector")
    return nil //unreached
  }
}

Now it should be easy enough to use a more complex implementation of the list class. Here's a linked list, as before the tail is itself a linked list (which is now a function mapping a selector to an implemetation).

func DynamicLinkedList&lt;T&gt;(head: T,
                          tail: (DynamicListSelectors) -&gt; DynamicList&lt;T&gt;)
                         (_cmd: DynamicListSelectors) -&gt; DynamicList&lt;T&gt;
{
  switch _cmd {
  case .count:
    return DynamicList&lt;T&gt;.count({
      return 1 + countOfList(tail)
    })
  case .at:
    return DynamicList&lt;T&gt;.at({ (i) in
      if i &lt; 0 {
        return nil
      }
      if i == 0 {
        return head
      }
      else {
        return elementInListAtIndex(tail, i - 1)
      }
    })
  }
}

let unitList = DynamicLinkedList("Wow", EmptyDynamicList()) //(Function)
countOfList(unitList) // 1
elementInListAtIndex(unitList, 0) // "Wow"
elementInListAtIndex(unitList, 1) // nil

Draw the rest of the owl

Going from here to Smalltalk is a small matter of types. The object defined above is a total function of a small number of list message selectors to a small number list method implementations: every selector resolves to a method.

A Smalltalk object is also a function from selectors to methods, but from the range of all possible selectors (which are strings) to the domain of all possible function types. [Aside: as C doesn't have closures, Objective-C's message lookup domain is all possible function types that receive an object and selector as the first two arguments.] You probably didn't write implementations for every possible selector, so most of these messages will be resolved to the doesNotRecognize: implementation which is where Smalltalk does its forwarding and dynamic method resolution, similar to Objective-C's forwardInvocation: and Ruby's method_missing.

The awesome power of the runtime. Now available in Swift.

#### Eating one's own dog food
I feel foolish for having made this error (especially after having so patiently explained how this stuff works on Mach), but today I did it.  I reported the amount of free memory on a Linux system as being the amount reported by free as free.

My own opinion on this is that I suffer from a view of hardware management which was tainted by using micros like the Dragon 32 (Radio Shack/Tandy TRS-80 to my American readers) and the Amiga, where there were basically two types of memory usage: yes and no.  A particular block was either in use by the system, or it wasn't.  On the Dragon 32 it was even easier than that of course; all bytes were available for reading and writing, but what happened was context-dependent.  Also because this was the MC6809E, whether such a peek/poke made sense depended on whether you were trying to hit an I/O address, and what was plugged in.  The Amiga had a particularly lame memory allocator which quickly sucked performance like a vacuum of performance suckage +2; but a byte of RAM was either in use or it was available.

But I digress.  The point is, that such a simple view of memory availability is no longer sensible but it's hard for me to think around it without a lot of work, just as it was hard for me to become a programmer after I'd been taught BASIC and Pascal.  If I were involved in UNIX internals more (and indeed that would be fun, although I think maybe Linux wouldn't be my first choice to open up) I'd probably be able to think about these things properly, just as I had to throw myself into C programming in a big way before I lost my BASIC-isms.

For the record (and so that it looks like this post is going somewhere), both operating systems have an intermediate state for RAM to be in between &quot;used&quot; and &quot;not used&quot; (where I'm ignoring kernel wired memory, and Linux kernel buffers).  On Mach, there's the &quot;inactive&quot; state which I've already described at el linko above.  On Linux, it's used as an I/O cache for (mainly disk, mainly read-ahead) operations.  This means that Linux will automatically take almost all (if not all) of the memory during the boot process.  The way that inactive memory gets populated on Mach means that on that system (e.g. Darwin) actually the amount of free memory starts large and inactive starts small, but over time as the active and inactive counts go up, the free count goes down, and it's rare for memory to be re-freed.  On both systems, free memory is really &quot;memory it wasn't worth you buying&quot; as it's not being put to any use at all.

#### Eating the bubble
How far back do you want to go to find people telling you that JavaScript is eating the world? Last year? Two years ago? Three? Five?

It's a slow digestion process, if that's what is happening. Five years ago, there was no such thing as Swift. For the last four years, I've been told at mobile dev conferences that Swift is eating the world, too. It seems like the clear, unambiguous direction being taken by software is different depending on which room you're in.

It's time to leave the room. It looks sunny outside, but there are a few clouds in the sky. I pull out my phone and check the weather forecast, and a huge distributed system of C, Fortran, Java, and CUDA tells me that I'm probably going to be lucky and stay dry. That means I'm likely to go out to the Olimpick Games this evening, so I make sure to grab some cash. A huge distributed system of C, COBOL and Java rumbles into action to give me my money, and tell my bank that they owe a little more money to the bank that operates the ATM.

It seems like quite a lot of the world is safe from whichever bubble is being eaten.

#### Elegant Object-oriented Software Design via Interactive, Evolutionary Computation
The abstract of this paper from the ArXiv had me concerned:

  Design is fundamental to software development but can be demanding to perform. Thus to assist the software designer, evolutionary computing is being increasingly applied using machine-based, quantitative fitness functions to evolve software designs. However, in nature, elegance and symmetry play a crucial role in the reproductive fitness of various organisms. In addition, subjective evaluation has also been exploited in Interactive Evolutionary Computation (IEC). Therefore to investigate the role of elegance and symmetry in software design, four novel elegance measures are proposed based on the evenness of distribution of design elements. In controlled experiments in a dynamic interactive evolutionary computation environment, designers are presented with visualizations of object-oriented software designs, which they rank according to a subjective assessment of elegance. For three out of the four elegance measures proposed, it is found that a significant correlation exists between elegance values and reward elicited. These three elegance measures assess the evenness of distribution of (a) attributes and methods among classes, (b) external couples between classes, and (c) the ratio of attributes to methods. It is concluded that symmetrical elegance is in some way significant in software design, and that this can be exploited in dynamic, multi-objective interactive evolutionary computation to produce elegant software designs.

The "design" of a software system is, to me, a part of the social science aspect of software engineering. Does the design make it easy for me to work out how the software functions? Can I see what I need to change to fix some problem? If I don't fix this problem, can you also see what you'd need to change? Does working with this design cause an emotional response?

With this mindset, it's hard to understand how an objective metric of software design can be formulated. Without that understanding, it's impossible to see any value in letting a software system design another software system that human developers are going to work on. In fact, it seems entirely back to front: the design is (to me) the part that needs a combination of experience, insight and serendipity to create. If a computer can then automatically fill some of the details in a way that saves time and reduces error, that would be useful. Doing it the other way around means human programmers become blue-collar subordinates to the (literal) software architect.

So, I didn't exactly jump into the rest of this paper with an open mind, which I recognised was a problem I needed to deal with so I ploughed on anyway. I started by reading "A Survey on Search-based Software Design", along with some of the other references, with a view to working out just what it was that these researchers are trying to automate. In the event, this post took a couple of weeks to write at my usual "whenever I get a chance" rate—there was a lot to understand.

What's Going On?

The idea is that certain principles in object-oriented design can be assigned a quantitative value: a "score", if you will. So you could score a design on how tightly-coupled the classes are, on how many responsibilities each class has, and on other features. You can also decide that a good design would aim to lower or increase particular scores; for example that looser coupling and fewer responsibilities are "better". You could decide that some designs are just "stillborn", and no matter how well they do on some metrics you're never going to use them: a circular reference, or a class with no responsibilities, might instantly be discarded.

Now imagine deriving some initial design for your software, for example from a collection of use cases. (You may be wondering how, and that's a good question: if your initial guess at the design is derived automatically from the use cases, then the use cases themselves need to be pretty precise, complete and unambiguous. In other words, they need to be written in a computer programming language.[*]) You score that design according to the criteria you defined, then make some "mutations" (which, in the case of evolutionary software design, means applying design patterns from a catalogue). The mutations that score better, you keep, combining them and mutating them further. Eventually you should have a collection of design candidates that are all much better than the initial guess.

[*]As a thought experiment, take the use-case diagram for a cinema booking system used as one of the inputs for this paper's methods. Try designing a software system to implement these use cases; every time you have a question that isn't answered by the diagram, make a note but _don't assume an answer_. How many questions do you end up with? Are you happy using a design in which these questions are unanswered? My guess is you'll be OK to leave some of them, designing the software to be flexible to different answers. But some will cause more problems unless they're addressed.

Is This Useful?

I don't feel like I got over the bias that I went into this post with: that the point of software design is to communicate something about that software among the various people who will be working on it. Computer-generated design is like computer-generated prose, when viewed from that perspective: uncannily close, but no substitute for the real thing.

What you could get from a technique like this are proposals for improvement on designs: indeed one branch (or clade, perhaps) of research in which genetic algorithms are applied to software design is in refactoring. One can imagine future UML tools (or IDEs) offering suggestions at the architecture level, just as current IDEs can offer suggestions for individual lines or methods.

That's basically what this paper is driving at: the "interactive" part of Interactive Evolutionary Computation. Human participants created the first versions of the designs, and qualitatively evaluated the later iterations (which were both produced and also evaluated by the software). Ultimately, software designers were called upon to reward the "better" designs and to decide when to stop the iteration: i.e. they chose whether to accept the "suggestions" made by the evolutionary algorithm.

So is this technique a step on the way to having that tool? Looking at table 4, you might think that the software did create better designs than the humans in two thirds of cases. Such is the danger of bold typeface. Look again at the standard deviations. Unfortunately, discussing the results with a tame statistician, we couldn't agree that the analysis in the paper shows the significant results the authors claim. As an example, it's not clear that pairing any two metrics actually makes sense, or that just because one measurement comes out consistently lower overall it's a better indicator of "elegance" than another (which might vary more between designs: something we're not told here).

The authors are on clearer footing when evaluating the relationship between the rewards given and the metrics: ignoring the software algorithm completely, do people consistently think of some particular property of a software design as indicative of elegance? While they've only got 7 participants (who, assuming you know the group, can probably be de-anonymised based on the information presented…just saying), and it's risky to draw general conclusions from such a small number of people[*], there are early indications here of consistency.

[*]particularly as they're all in academia, and probably all in the same institution.

#### Empowered free software
Free and open source software has traditionally been defined as the opposite of something else: proprietary (or commercially-licensed) software. That's particularly obvious in the name of the GNU project, which calls itself "Not UNIX" - a popular AT&amp;T-owned commercial software property of the time. The GNU manifesto goes deeper on the specific ways in which it is Not Unix:

  I consider that the Golden Rule requires that if I like a program I must share it with other people who like it. Software sellers want to divide the users and conquer them, making each user agree not to share with others. I refuse to break solidarity with other users in this way. I cannot in good conscience sign a nondisclosure agreement or a software license agreement. For years I worked within the Artificial Intelligence Lab to resist such tendencies and other inhospitalities, but eventually they had gone too far: I could not remain in an institution where such things are done for me against my will.

The focus of this document, and of the various free software licenses, is on the distribution and redistribution rights associated with the software. Thus the focus of the Free Software Foundation, Open Source Initiative, and the other organisations that promote free or open source software is on ensuring appropriate licences are used, and that distributors comply with the licence terms.

Otherwise, free software development is not particularly distinguished from other forms of software development. There are, of course, some outliers, but mostly you'll see a "core team", perhaps with a formal structure (component owners, project/release managers, and other roles are common). This team organises its work around proprietary work-allocation tooling like Jira, Github, Slack, Trello, and so on. In many cases, an organisation such as a commercial company or business-interest foundation exists to centralise ownership and decision making, like a good old-fashioned Fordist company. The only visible difference in operations is that it's possible to read the source code and get changes from outside through the gates and into their repository.

The reason so much work in open source software looks a whole lot like commercial software is that it is work in commercial software. Sometimes the open source project exists exactly to drive adoption of commercially-licensed equivalents, replacements, or upgrades. "Open core" products exist so that you find a need for the commercial features. Developer advocates build open source workflow tools as a sort of loss leading pre-sales activity. And many companies publish open source software "off the money path" as a recruitment activity: do some free work on our pull requests then come here and get paid to reject pull requests from your peers!

Note that I'm not saying all free software or open source software is like this, but that plenty is. Free and open source software development is informed by, and mimics, commercial software engineering to a large extent.

Of course it's this way, you say, we software engineers need our expensive software engineering lifestyles supported by our high software engineering salaries, so we learn our craft in corporate service and that informs all of our development, including open source. Or that we are only able to contribute to open source when it's in pursuit of our employers' goals, because Computering Time is something we do in the office. So it's no surprise that a lot of open source software development looks like, or is, corporate software development. If only there were proper, sustainable funding for open source software!

To which I say: that is an interesting, and problematic, request, and I wish to put it to one side. Maybe I'll revisit it in a later post. What I'm more interested in, is what an open source movement that was centered around the Four Freedoms, rather than software-licensing concerns, would look like. One in which you didn't merely have some abstract legal freedom to do the four activities, but were empowered to do so.

Power Zero: The power to run the program for any purpose.

How much free or open source software cannot directly be used by anyone who isn't a programmer? Think of all of the things that are only distributed as developer libraries, for developers to incorporate into (commercial or open source) applications. The things that get duplicated as a pod, an egg, a crate, and in whatever new packaging system and programming language comes along. This is the repository namespace as a virtual landgrab, enabling open source as corporate recruitment tool: you should hire me, you've heard of me because I wrote the language-your-programmers-use version of library-your-programmers-use.

How much free or open source software is just plain difficult, due to accidental rather than essential complexity? Developers will probably have all sorts of (developer tools) examples here, like GNU autoconf. I find it very hard to make a nice-looking document using open source tools, too. I've put enough effort into learning emacs and LaTeX to get somewhat proficient, but still spend a lot of time looking up TeX syntax and getting it wrong. And making (hopefully nice-looking) documents is something I do fairly frequently.

And I'm playing this game on the easy setting. I've got years of experience trying to make software work. I am willing (and often paid) to put time into understanding why software doesn't do what I want. I speak English, the home language of much software, and can use a screen and keyboard without difficulty.

A free software movement in which Freedom 0 is replaced by Empowerment 0 would make getting, trying, and adopting free software trivial. Much simpler than an app store, which has the necessary gatekeeper of payments.

Power One: The power to study how the program works, and change it to make it do what you wish.

Access to the source is a precondition for this. Also, the source being in a readily comprehensible format, amenable to experimentation and adaptation are preconditions. It needs to be possible for a finance person, not a computer person, to look at a finance application like GNUcash, understand its model of finance, and adapt it to their model of finance.

This means elevating automated testing from a thing that developers claim they do sometimes, to the principal mode of hypothesis-driven change in software. Remember when Dan North said that Cucumber would allow business analysts and developers to collaborate on writing the tests? That, but collaborating on writing the software. Currently modes of software development, including free and open source software, are predicated on the division of society into three classes: "developers" who make software, "the business" who sponsor software making, and "users" who do whatever it is they do. An enabling free software movement would erase these distinctions, because it would give the ability (not merely the freedom) to study and change the software to anyone who wanted or needed it.

Software developed with this in mind would have - nay, require - a very clear architecture. If you want a finance person to critique the finance model used in a finance application, everything needs to scream finance. It needs to stop screaming model, view, controller, or memory management, or database transaction, and start shouting credit, debit, accounts receivable.

That doesn't mean that there isn't space for experts in memory management or database transactions, but it does mean that you can understand a music score pagesetting application to the point where you can improve its representation of Klezmer modes without needing to understand memory management or database transactions.

Power Two: The power to redistribute and make copies so you can help your neighbour.

An enabling free software movement would make it easy for me to package my computer, or a part of it, and send it to someone else: "here, I find this useful, you give it a go". So I don't have to grub around (pardon the pun) for whatever guix, apt, yum, pkcon, brew, pkg or whatever command I had to run to make the software work and tell that person to run the same command, hoping it works. I just give them the thing, and they use it.

Power Three: The power to improve the program, and release your improvements (and modified versions in general) to the public, so that the whole community benefits.

Currently there is a blessed edifice, called "upstream", the fount of all that is good in software. Any corporate programmer who wants to juice their proprietary github profile longs for their "requests" to be "pulled" by upstream, so that all of those other programmers who are in thrall to upstream finally get to see the fruits of this individual's labour.

It feels weird to say this in 2020, when the idea was presented as fait accompli in 1997, but an enabling open source software movement would operate more like a bazaar than a cathedral. There wouldn't be an "upstream", there would be different people who all had the version of the software that worked best for them. It would be easy to evaluate, compare, combine and modify versions, so that the version you end up with is the one that works best for you, too.

The Four Powers

The combination of these powers points to an open source software movement that erases power dynamics implicit and explicit in our current modes of producing software. It's as easy for someone who understands the domain of a software system to acquire, understand, improve and share that system as it is for someone who understands the computer it runs on. The infinite malleability of software is deployed to allow people, teams and communities to produce and share their own versions that work best for them.

The antagonism between "the developers" and "the business" documented in the principles behind the agile manifesto is left to commercial software companies. Users, developers, and sponsors are on the same team, and produce software that works better for them than if they worked in other ways.

#### enum class in C++11
I've opened the new edition of Cuboid Stroustrup exactly once, and I've already learned exactly one useful thing.

Before going into what that thing was, a comment on the book: The C++ Programming Language is, along with Object-Oriented Software Construction by Bertrand Meyer, one of the best books on a programming language I've ever read. In addition to explaining the language and its features, they explain why it is as designed. You get to find out how the various features are expected to be used: a great introduction to the idioms adopted by its community.

Conversely, Object-Oriented Programming: An Evolutionary Approach discusses much of the philosophy without going into much depth on the language itself. The C Programming Language goes the other way, explaining the language in detail without describing why any of its features were designed the way they were, or how you mighty want to put them together. It also happens that both of these books are too old to learn modern idiomatic use of their languages from.

Anyway, enum class. The problems with C enum are twofold: firstly the named values go into the global namespace so you can't duplicate names (imagine if you wanted to have text alignment and view content mode enumerations, for example, you couldn't use "Left" and "Right" as names in both places). Secondly, the values are just integers, so nonsense expressions like Left + Right are possible. What C++ has added recently is strongly typed, namespaced enum classes:

enum class TextAlignment { Left, Right };

You can't do silly things like add these together, because TextAlignment is a custom type so it can't be treated as an integer. You'd have to overload operator+() before addition could work—meaning you'd need a reason to do it. This is something I've long wanted in Objective-C, though my expectation is that Objective-C enumerations would be Flyweight Objective-C objects:

An Objective-C specific @enum type, which creates static, immutable instances. So you could have:

@enum GLScreenOrientation {Landscape, Portrait};

which creates a class like this:

@interface GLScreenOrientation: NSObject &lt;NSCopying&gt;
+ (GLScreenOrientation *)Landscape;
+ (GLScreenOrientation *)Portrait;
+ (NSSet *)values;
@end

This is modelled after the Java enum.

#### Essence and accident in language model-assisted coding
In 1986, Fred Brooks posited that there was “no silver bullet” in software engineering—no tool or process that would yield an order-of-magnitude improvement in productivity. He based this assertion on the division of complexity into that which is essential to the problem being solved, and that which is an accident of the way in which we solve the problem.

In fact, he considered artificial intelligence of two types: AI-1 is “the use of computers to solve problems that previously could only be solved by applying human intelligence” (here Brooks quotes David Parnas), to Brooks that is things like speech and image recognition; and AI-2 is “The use of a specific set of programming techniques [known] as heuristic or rule-based programming” (Parnas again), which to Brooks means expert systems.

He considers that AI-1 isn't a useful definition and isn't a source of tackling complexity because results typically don't transfer between domains. AI-2 contains some of the features we would recognize from today's programming assistants—finding patterns in large databases of how software has been made, and drawing inferences about how software should be made. The specific implementation technology is very different, but while Brooks sees that such a system can empower an inexperienced programmer with the experience of multiple expert programmers—“no small contribution”—it doesn't itself tackle the complexity in the programming problem.

He also writes about “automatic programming” systems, which he defines as “the generation of a program for solving a problem from a statement of the problem specifications” and which sounds very much like the vibe coding application of language model-based coding tools. He (writing in 1986, remember) couldn't see how a generalization of automatic programming could occur, but now we can! So how do they fare?

Accidental complexity

Coding assistants generate the same code that programmers generate, and from that perspective they don't reduce accidental complexity in the solution. In fact, a cynical take would be to say that they increase accidental complexity, by adding prompt/context engineering to the collection of challenges in specifying a program. That perspective assumes that the prompt is part of the program source, but the generated output is still inspectable and modifiable, so it's not clearly a valid argument. However, these tools do supply the “no small contribution” of letting any one of us lean on the expertise of all of us.

In general, a programming assistant won't address accidental complexity until it doesn't generate source code and just generates an output binary instead. Then someone can fairly compare the complexity of generating a solution by prompting with generating a solution by coding; but they also have to ask whether they have validation tools that are up to the task of evaluating a program using only the executable.

Or the tools can skip the program altogether, and just get the model to do whatever tasks people were previously specifying programs for. Then the accidental complexity has nothing to do with programming at all, and everything to do with language models.

Essential complexity

Considering any problem that we might want to write software for, unless the problem statement itself involves a language model then the language model is entirely unrelated to the problem's essential complexity. For example, “predict the weather for the next week” hides a lot of assumptions and questions, none of which include language models.

That said, these tools do make it very easy and fast to uncover essential complexity, and typically in the cursed-monkey-paw “that's not what I meant” way that's been the bane of software engineering since its inception. This is a good thing.

You type in your prompt, the machine tells you how absolutely right you are, generates some code, you run it—and it does entirely the wrong thing. You realize that you needed to explain that things work in this way, not that way, write some instructions, generate other code…and it does mostly the wrong thing. Progress!

Faster progress than the old thing of specifying all the requirements, designing to the spec, implementing to the design, then discovering that the requirements were ambiguous and going back to the drawing board. Faster, probably, even than getting the first idea of the requirements from the customer, building a prototype, and coming back in two weeks to find out what they think. Whether it's writing one to throw away, or iteratively collaborating on a design[*], that at least can be much faster now.

[*] Though note that the Spec-Driven Development school is following the path that Brooks did predict for automatic programming (via Parnas again): “a euphemism for programming with a higher-level language than was presently available to the programmer”.

#### Even more on generalist software engineering
There is a difference between a generalist software engineer, and a polyglot programmer. What is that difference, and why did I smoosh the two together in yesterday’s post?

A polyglot programmer is a programmer who can use, or maybe has used, multiple programming languages. That doesn’t mean that they have relevant experience or skills in any other part of software engineering. They might, and so might a monoglot programmer. But (in a work context) they’re working as a programmer and paid for doing the programming.

A generalist software engineer has knowledge and experience relevant to the rest of the software engineering endeavour. Not just the things that programmers need to know beyond the programming, but the whole practice.

In my experience, a generalist software engineer is usually also a polyglot programmer, for a couple of reasons.

to get enough experience at all the bits of software engineering, the generalist’s career has probably survived multiple programming language hype cycles.in getting that experience, they have probably had to work with different languages in different contexts: whatever their software products are made out of; scripts for deployment and operations; plugins for various tools; changes to existing components.

More importantly, the programming part of the process is a small (crucial, admittedly, small nonetheless) intersection with the generalist’s work. Knowing all of the gotchas of any one language is something they can ask the specialists about. Fred Brooks would have had a Chief Programmer who knew what the team was trying to do, and a language lawyer who specialised in the tricks of the programming language.

Probably the closest thing that many organisations know how to pay for to the generalist software engineer in modern times is the “agile consultant”. I don’t think it’s strictly a fair comparison. Strictly speaking, an agile consultant is something like a process expert, systems analyst, or cybernetician. They understand how the various parts of the software engineering endeavour affect each other, and can help a team to improve its overall effectiveness by knowing which part needs to change and how.

And, because they use the word agile in their title, they do this with an air of team empowerment and focus on delivery.

Knowing how the threat model impacts on automated testing enables process improvement, but does not necessarily make a software engineering generalist. An actually-general generalist could perhaps also do the threat modelling and create the automated tests. To avoid drawing distinctions between genuine and inauthentic inhabitants of Scotland let’s say that a generalist can work in ‘several’ parts of the software engineering field.

We come back to the problem described in yesterday’s post, but without yesterday’s programming-centric perspective. A manager knows that they need a security person and a test automation person, but does not necessarily know how to deal with a security-and-test-automation-and-maybe-other-things person.

This topic of moving from a narrow focus on programming to a broader understanding of software engineering is the subject of my newsletter: please consider signing up if this topic interests you.

#### Everyone rejecting everyone else
It's common in our cooler-than-Agile, post-Agile community to say that Agile teams who "didn't get it" eschewed good existing practices in their rush to adopt new ways of thinking. We don't need UML, we're Agile! Working software over comprehensive documentation!

This short post is a reminder that it ran both ways, and that people used to the earlier ways of thinking also eschewed integrating their tools into the Agile methodology. We don't need Agile, we're Model-Driven! Here's an excerpt from 2004's UML 2 Toolkit:
Certain object-oriented methods on the market today, such as The Unified Process, might be considered processes. However, other lightweight methods, such as Extreme Programming (XP), are not robust enough to be considered oricesses in the way we use the term here. Although they provide a valuable set of interrelated techniques, they are often referred to as "small m methodologies" rather than as software-engineering processes.
This is in the context of saying that UML supports software-engineering processes in which a sequence of activities produce "documentation...about the system being built" and "a product that solves the initial problems is introduced and delivered". So XP is not robust enough at doing those things for UML advocates to advocate UML in the context of XP.

So it's not just that Agilistas abandoned existing practices, but there's an extent to which existing practitioners abandoned Agilistas too.

#### Experts around the table
One of the principles behind the manifesto for Agile software development says:

  Business people and developers must work
  together daily throughout the project.

I don't like this language. It sets up the distinction between "engineering" and "the business", which is the least helpful language I frequently encounter when working in companies that make software. I probably visibly cringe when I hear "the business doesn't understand" or "the business wants" or similar phrases, which make it clear that there are two competing teams involved in producing the software.

Neither team will win. "We" (usually the developers, and some/most others who report to the technology office) are trying to get through our backlogs, produce working software, and pay down technical debt. However "the business" get in the way with ridiculous requirements like responding to change, satisfying customers, working within budget, or demonstrating features to prospects.

While I've long pushed back on software people using the phrase "the business" (usually just by asking "oh, which business do you work for, then?") I've never really had a replacement. Now I try to say "experts around the table", leaving out the information about what expertise is required. This is more inclusive (we're all experts, albeit in different fields, working together on our common goal), and more applicable (in research software engineering, there often is no "the business"). Importantly, it's also more fluid, our self-organising team can identify lack of expertise in some area and bring in another expert.

#### Explicitly considering subtyping in inheritance
By far the post on this blog that gains the most long-term interest and attention is why inheritance never made any sense. In this post, I explain that there are three different ways to think about inheritance—ontological inheritance (this sort of thing is a special type of that sort of thing), subtype inheritance (this program that expects these things will work with these things too), and implementation inheritance (the code in this thing is also in that thing)—and that trying to use all three at the same time is a recipe for disaster.

People interpret the message behind this as they will: that you should only ever compose objects, that you should only use pure functions, whatever. The message I tried to send was that you need to not use all of these different forms of inheritance at once, but OK. In this paper from the very early days of industry OOP, the late Bill Cook and colleagues resolve implementation and subtyping inheritance, by treating them differently (as I argued for).

#### e[Mac]s keyboard
It's about time someone made a UNIX keyboard for the Mac.  I still love my Tactile Pro, and have obviously configured the caps lock key as an additional Control, but that key doesn't feel like the same buckling spring as the rest so it feels weird when I hit it.  I just wonder now whether someone's going to make a keyboard with a Command key up there.  Probably not, they never did for either the earlier Macs or the NeXT.  It would be handy to be able to do cut/copy/paste without having to buckle my thumb or little finger under my hand...Matias' Optimise keyboard looks like it might be efficient but have a confusion curve.

#### Fairly cool update to GNUstep
GNUstep-make now supports arbitrary(-ish) filesystem layouts.  While the default is still the /usr/GNUstep layout with the various domains, one of the bundled alternatives is to put everything in FHS-compliant locations.  Once that hits a release (which I believe will be called gnustep-make 2.0), that should make newbie users and distributors much more comfortable with GNUstep.  I also provided a NeXT-ish layout which doesn't clone the NeXT directory hierarchy, but rather mimics it sensitively.My work laptop just earned itself a reinstall (I'd been uncomfortable with CentOS for a while, but the facts that hotplug isn't configured properly and every time I 'yum update' I have to fix a handful of drivers drove me over the edge), so when I re-create my GNUstep installation I'll do it with the new-style make.  I'm currently wavering on the side of installing Midnight BSD, but I might wimp out and dual-boot Ubuntu ;-)

#### Fairness
There are two different questions of fairness when it comes to the App Store rules. Apple always spin it to mean “these rules are applied fairly”, which is certainly not true. Putting aside questions of why Netflix get to do things Hey don’t, it’s pretty obvious that the rules include “don’t make apps in these spaces where Apple has apps” that don’t apply to Apple. It’s also clear that nobody in the App Store team is rules lawyering Apple apps on the rest of the rules, either.

But all of that ignores the larger question, “are these rules fair?”

#### Falsehoods programmers believe about programming
There is no ethical impact of my job; I build technological systems and it's up to others how they use them.
Software is a purely technical discipline.
There is some innate affinity for computer programming which you must be born with, and cannot be taught.
Allowing people who are unlike me to program can only be achieved by "lowering the bar".
Compiled languages are always faster.
floating point calculations will introduce non-deterministic errors into numerical results.
OK, they introduce some errors into numerical results.
alright I understand that floating point calculations are imprecise not inaccurate, Mister Pedantic Blog Author, but I cannot know what that imprecision is.
at least the outcome of integer maths is always defined.
fine, it's not defined. But whatever it was, the result of doing arithmetic on two numbers that each fit in a data register itself fits in a data register.
every computer on sale today (2017) uses two's complement notation for negative numbers.
every computer on sale today uses a register width that's a multiple of eight bits.
the bug isn't in my code.
the bug isn't in the library.
the bug isn't in the operating system.
the bug isn't in the compiler.
the bug isn't in the kernel.
the bug isn't in the hardware.
bug-free computer hardware is completely deterministic.
the lines on the hardware bus/bridge are always either at the voltage that represents 0 or the voltage that represents 1.
if my tests cover 100% of my lines of code then I have complete coverage.
if my tests cover 100% of my statements then I have complete coverage.
if my tests cover 100% of my conditions then I have complete coverage.
if I have complete test coverage then I have no bugs.
if I have complete test coverage then I do not need a type system.
if I have a type system then I do not need complete test coverage.
no hacker will target my system.
information security is about protecting systems from hackers.
if the problem is SQL Injection, then the solution is to replace SQL; NoSQL Injection is impossible.
my project is a special snowflake; I can reject that technique I read about without considering it.
my project is much like that unicorn startup or public company's project; I can adopt that technique I read about without considering it.
people who do not use my language, technique, framework, tool, methodology, paradigm or other practice do not get it.
any metaprogramming expansion will resolve in reasonable time.
any type annotation will resolve in reasonable time.
OK, well at least any regular expression will resolve in reasonable time.
can you at least, please, allow that regular expressions are regular?

I'm sure there must be more.

Update The following were added later; where they were supplied by others there is citing link. There are also good examples in the comments.

You need a computer science degree to be a good programmer.
A computer science course contains nothing useful to programmers.
Functional Programming is a silver bullet.
Rust is a silver bullet.
There is a silver bullet.
There can be no silver bullet.
Rewriting a working software system is a good idea.
I can write a large system in a memory unsafe language without introducing vulnerabilities.
I can write a large system in a memory safe language without introducing vulnerabilities.
Software is an engineering discipline.
Software is a scientific discipline.
Discourse on a topic is furthered by commenting on how I already knew a fact that was stated.
A falsehood about programming has no value unless the author of the falsehood provides supporting evidence to my satisfaction.

#### Falsehoods programmers who write "falsehoods programmers believe" articles believe about programmers who read "falsehoods programmers believe" articles
For reasons that will become clear, I can't structure this article as a "falsehoods programmers believe" article, much as that would add to the effect.

There are plenty of such articles in the world, so turn to your favourite search engine, type in "falsehoods programmers believe", and orient yourself to this concept. You'll see plenty of articles that list statements that challenge assumptions about a particular problem domain. Some of them list counterexamples, and a subset of those give suggestions of ways to account for the counterexamples.

As the sort of programmer who writes falsehoods programmers believe articles, my belief is that interesting challenges to my beliefs will trigger some curiosity, and lead me to research the counterexamples and solutions. Or at least, to file away the fact that counterexamples exist until I need it, or am otherwise more motivated to learn about it.

But that motivation is not universal. The fact that I treat it as universal turns it into a falsehood I believe about readers of falsehoods articles. Complaints abound that falsehoods articles do not lead directly to fish on the plate. Some readers want a clear breakdown from "thing you might think is true but isn't true" to "Javascript you can paste in your project to account for it not being true". These people are not well-served by falsehoods articles.

#### Falsehoods These Programmers Believed About Countries
Well this was a hard-fought issue. Setting the scene: since April 2020 I've been working on Global.health: a Data Science Initiative, where we collate information about Covid-19 cases worldwide and make them available in a standard schema for analysis. In the early days the data were collected by hand, as had been done for prior outbreaks, but the pandemic quickly grew beyond a scale where that was manageable so instead we looked to automatically import data from trustworthy published sources like ministries of health.

Cases typically have a location associated with them (often the centroid of their local health service district, or the administrative region the sufferer is registered in; never something uniquely identifiable like a home address). Now already being able to work with location data throws us some usability curveballs. Did you search for London as in anywhere with the name "London" in (such as London Oxford Airport, just outside Oxford and 62 miles from the City of London, a city within the place people generally know as London in England), or did you have a specific location in mind like London, Ontario, Canada or London Street, Los Angeles, California, USA?

But we had it worse than that. Our data format had the name of the country in, which led to all sorts of problems. Had the curator entered that London St case as being in the US, or the USA, or America, or the United States of America, or the United States? Sometimes even cases that had location data filled by a geolocation service had weird glitches, like a number of cases from Algeria being associated with the non-country Algiers. And it made it easier for us devs to make unforced errors, like not generating cached per-country data when the country has a space in its name.

For all of these reasons, I ended up with the task of changing our schema so that countries are stored as ISO-3166 two-letter codes. Along the way I spotted all of the above difficulties, and more, some of which even manifested in the libraries that map from country codes to names, and vice versa. Note I'm using "country code" fairly loosely; some places are far enough from where everybody thinks of as "the country" that they have a separate ISO code (it wouldn't help anyone to record a case as being in "the UK" when you mean "the Falkland Islands", of which more below).

 	Countries have changed names recently. Swaziland became Eswatini (ISO code SZ) in 2018. The Former Yugoslavian Republic of Macedonia (officially, as far as they were concerned, Macedonia; but often named FYROM as Greece wouldn't allow them to accede to the EU under Macedonia) became the Republic of North Macedonia (ISO code MK) in 2019. Both of these appeared in our geocoding provider under their original names, even though we didn't start gathering data until early 2020.
 	People don't think of a country by its official name. China is China to many, not the People's Republic of China (ISO code CN). Do we show the official name or the common name?
 	People don't think of a region as part of its sovereign country. The Hong Kong Special Administrative Region of the People's Republic of China has ISO code HK, but is…well, it's a special administrative region of the People's Republic of China (CN). When you show people countries on a map like they asked for, and they ask "where is Hong Kong", the answer is "it isn't there because it isn't a country".
 	A country's ISO code is not the same as its top-level domain. Not a correctness problem for us, but one that might impact usability, when people look for the United Kingdom of Great Britain and Northern Ireland under "UK" when they'll find it under "GB". There is a .gb TLD, but it doesn't accept new registrations.
 	The extent of a region can change when its name changes. We have cases geocoded to the Netherlands Antilles (doesn't have an ISO code, technically, see next point, but used to be AN); there's extra work involved to decide whether these should be associated with Aruba (AW), Curaçao (CW), Sint Maarten (SX) or the Carribean Netherlands (BQ).
 	As mentioned above, ISO "retires" codes. There isn't a code AN any more, because when the Netherlands Antilles stopped existing they decided it isn't needed. This causes a problem in that a library that has "a complete" database of ISO country codes doesn't necessarily have the retired ones. The historical cases are in ISO 3166-3 but occupy a different namespace than the ISO 3166-1 codes so it's not like AN still means "the region that used to be Netherlands Antilles": its ISO 3166-3 code is ANHH. Similarly, the German Democratic Republic used to have the ISO 3166-1 code DD but now has the ISO 3166-3 code DDDE.
 	A region may have two official names. The Falkland Islands (FK) are a British Overseas Territory; they are also constitutionally part of Argentina under the name "Islas Malvinas". Politics aside, the reason this is an immediate problem is that some services feel the need to helpfully list both but not in a standard way; you need to be able to cope with "Falkland Islands [Malvinas]", "Falkland Islands (Islas Malvinas)" and more.
 	A region might have no official existence. Kosovo (XK) has an ISO code but is a disputed region, recognised by about half of the UN as a sovereign country and claimed as an autonomous region by Serbia.
 	A country's name might be the same as another country's name. The Democratic Republic of the Congo (CD) is sometimes known as "the Congo", and the neighbouring Republic of the Congo (CG) is also sometimes known as "the Congo". Frustratingly, the i18n-iso-countries package lists "The Congo" as a name for both countries, so name-to-code mapping is unreliable.

You may be able to find more; feel welcome to comment (or maybe file a bug in Global.health if it's causing trouble over there). Also, I've just created the SICPers mailing list as a one-stop shop for all my reading, writing and talking about software engineering, please consider subscribing!

#### FAQ update
Just a warning about @defs not necessarily being a good idea, some code from Mike Ash to add polish to the forwarding example and...um...I'm sure I added something else.  Oh yes, the fact that _cmd is the name of the SEL argument in method definitions.

I'm quite inclined to refactor the FAQ (because the source is basically the XHTML), as something like a DocBook FAQ Article, which might explain any (further) hiatus in updates.  I haven't started working on this yet but it seems more attractive, as I look at the horrendous car crash of formatting produced by a combination of different text editors I've been using.

#### Fiction: The Ouroborus School pt 1
On a warm Spring day, the camera follows a butterfly as it wends its coruscating way around a buddleia bush. The plant is growing in the well-kept border surrounding an immaculately manicured lawn, the quadrangle of an imposing Jacobean building. Now the perspective shifts, you zoom in over the bright primary colours of the butterfly's wings, dancing on the thermals and eddies of the sunlit quadrangle, until you can see through the eyes of the butterfly. This is actually quite easy to arrange, as the butterfly is a Milliard Maps autonomous photography drone and its eyes are internet-connected fisheye camera lenses. As you see the slate roof slowly getting larger due to the butterfly stochastically flapping towards it, the last few words of a university lecturer addressing his students can be heard drifting from an open window.

"So even though the applications were all protected and the kernel was protected, the criminals still found ways to circumvent the system and get their malicious softs to run. Next week, we'll look at how the firms itself was enhanced to keep customers safe and increase value. You should all read Chapter 7 before then. Don't forget that the collections on digirights will be coming up the following week. OK! That's it for today."

A "collection", we have time to reflect upon as the students gather their things and leave the lecture hall, is a mid-term examination at the Ouroborus Inc. School for Higher Education, in the centre of Oxford. It is that institution's window, under its clinker-like roof, that leaked the words from that professorial denouement. The university isn't really run by Ouroborus, of course. That company is actually a large manufacturer of consumer electronics over on the far side of the United States of America, and managing an educational institution is not in its core strengths. The company merely owns the building, approves lecturers into the school's "team", advertises the courses in its online music store (an unfortunate quirk of history that has never been corrected), and processes the registration of new students. The school is self-governed, and keeps 70% of the tuition fees. In return for its share of the fees, Ouroboros Inc. reviews the syllabus prepared for each course and rejects suggestions by the tutors that would not be in the best interests of the students. These days the curricula tend to get through with only minor tweaks, but back in 2018 the course on Experiences of Computer Softs was rejected for including a lecture on the uses of devices made by the Milliard Internet Searching Company.

The lecture you just caught the end of is from "Introduction to Digirights", one of the compulsory courses for first year app sci undergraduates at OISHE. App sci is one of the school's most widely-renowned degrees. Many of its graduates have left five star reviews on the Diploma Store, and it frequently tops the highest-grossing list in its category. Hiring a softsmith from OSHE's app sci graduate pool, it is said, is a sure-fire way to build a company whose softs can command five dollars-some fetch even more-per sale. As a result they receive myriad applications every year per place, many of which are actually in localised classes presented in other countries. One such localisation is offered in the Ouroborus Institute of Technology in Cambridge, Massachusetts, United States of America. Since the federal government outlawed the teaching of American citizens in British English the school has found it more cost-effective to let OIT lecturers deliver its classes to American students.

Having contemplated the centuries-old tiles and cupolas of OSHE's roof, your gaze is directed toward the last pair of students to leave the digirights lecture and enter the quadrangle. Mary Wilkes looks with some concern at her wristwatch. The Ouroboros OWatch ("You'll love it for its timelessness. And for its time.") can in fact tell you how far into the day you've progressed, but Mary, like many of its users has other designs on it. Currently it's showing her that she received three public messages during the lecture, that her digirights notes have synced successfully, and that she has another lecture in fifteen minutes over in the Tutte Building. The OWatch was originally designed to help Ouroboros Inc. sell into Latin America. The theory was that the kind of people who were flashy enough to buy their phones were also sensible enough not the wield them on the streets of Sao Paulo, and therefore were not actually going to buy the phones. The watch let them show off their allegiance to the marque without as much chance of being mugged. In fact the OWatch quickly gained customers in Europe and North America, and a few more people reported to hospitals in South America with wrist lacerations.

"Sorry, Ivan, I'm going to have to skip that coffee. I've got to get over to King Tutte, and I'll need to get another pack of stylus tips before that in the MISCStore. How about you send me an invite for another time?"

Ivan shakes his head, an action that sets his messy tangle of blond hair and much of the rest of his wiry frame oscillating in sympathetic motion. A playful grin erupts over his face. "No way, Mair, I'm not eventing you. Every time you're evented you reply 'maybe' to the invite, and you never turn up! I'll see you walking past a 'Bucks sometime, and I'll check your profile to see you've got nothing else on. Then you'll have to let me buy you a coffee." He pronounces her name 'Mair', but spells it 'Mar': saving a crucial character for those microblog posts.

"You'll see that I have nothing else on...'maybe'" retorted Mary. The two of them laugh as they turn in opposite directions out of the OISHE main gate, Mary turning left toward the high street and the Tutte Building. Cars automatically slow down as she unconcernedly approaches the road, their computerised drivers detecting her position and speed and extrapolating that she probably means to cross seconds before she actually steps out into their path.

As the OSHE is not actually run by Ouroborus, neither is the MISCStore actually run by Milliard Internet Searching Company. "MISCStore" is the nickname given by students to the Williams Mart branch nearest to the school, whose systems are integrated with those of the search giant. Mary's OWatch vibrates as she enters the store. "You have checked in at Will. Mart!" proclaims a notification. She selects a box of stylus tips, and takes them over to the cashier.

"Hi Mary," the cashier says. While she has shopped at the MISCStore many times, Mary doesn't recognise this particular employee either from the shop or school (judging by her age, and the fact that she can afford to live in Oxford, Mary guesses that the woman behind the counter is probably a fellow student). She must have read the name from her cash register, which would have received the same notification as Mary's watch. The register will also be displaying pertinent facts from Mary's profile; just the useful stuff like her credit rating and recent spending habits. "Just the styluses?"

"Just the styli," Mary emphasises the last word as one of her few chances to show off her latlang A-level. Latlang is not a popular course and frequently gets one-star reviews from former students who felt that it was too hard and should, as a result, be free. Despite this, a hardcore of students (typically those who did quite well in the exams) have formed a bit of a latlang cult, with correct declension being the password between members. The cashier in MISCStore is, evidently, not among their number as the correction goes entirely without remark.

"Are you sure? It looks like you enjoy Winnesota brand hot chocolate, and we've got five percent off kilo packs at the moment. We think you're really gonna love it."

"Uh, no thanks. Just the styli, please."

"Oh," the cashier looked put out. "Should I go ahead and put that you don't like Winnesota brand hot chocolate on your profile?"

"No, really, just no, it's fine. I like the stuff, but I only bought it the once. I don't really want any right now, it's a bit pricey."

"Not with five percent off it isn't! We think it'd be the best thing to compliment your purchase of replacement touchscreen styluses." Whether it was through a gargantuan effort of will or not seeing any issues with that sentence, the cashier kept a straight face throughout delivery.

"No, I don't want your hot bloody chocolate! Look, I'm running late, I just need to buy these. O Watch," this last being directed at her watch, triggering its speech command feature, "post that I'll be late to the softscraft class. Can anyone take some notes for me?" She looked up again at the cashier. "Look, just the styli, they're one ninety-nine right? Here's my Millicard."

"Thankyou for shopping at Willims Mart. Will you 'Like' this transaction on your profile? Liking is a great way to keep up with all the buzz and special deals on offer at Williams Mart!"

"Fuck off will I fucking like this fucking transaction! You've made me late with all this crap about chocolate and now you want to know if I like you? Piss off!" Having exhausted her daily allowance of expletives Mary turns to leave the store. She notices for the first time that a short bald man, a sort of overweight weasel that got its wish to be a real boy, is standing by a door behind the counter. As he isn't wearing the cheap polyester official Williams/Milliard shirt he must be the manager.

"Thank you for shopping with us, miss. Company policy requires that after insulting our transaction enabler we must ask you to apologise on your public profile and to 'Like' Williams Mart." Mary is storming out of the store and only barely hears the last part, "it is a federal crime to disregard compliance with any Williams Mart policy."

Another vibration from her watch. "You have checked out."

#### Figurative Programming and Gloom: the [G]raphical [LOOM]
Donald Knuth is pretty cool. One of the books he wrote that I own and have actually read[*] is Literate Programming, in which he describes (among other things) weaving program text and documentation together in a single narrative.

Two of his books that I own and have sort of dipped into here and there are TeX: the Program, and METAFONT: the Program. These are literate programs, created from webs in which Human text and Computer text are interleaved to tell the story of what the program does.

Human text and computer text, but not images. If you want pictures, you have to carry them around separately. Even though we are highly visual organisms, and many of the programs we produce have significant graphical components, very few programming environments treat images as anything other than external files that can be looked at and maybe previewed. The only programming environment I know of that lets you include images in program source is TempleOS.

I decided to extend the idea of the Literate web to the realm of Figurative Programming. A gloom (graphical loom) web can contain human text, computer text, and image descriptions (e.g. graphviz, plantuml, GLE...) which get included in the human-readable document as figures.

The result is gloom. It's written in itself, so the easiest way to get started is with the Xcode project at gloomstrap which can extract the proper gloom sources from the gloom web. Alternatively, you can dive in and read the PDF it made about itself.

Because I built gloomstrap first, gloom is really a retelling of that program in a Figurative Programming web, rather than a program that was designed figuratively. Because of that, I don't really have experience yet of trying to design a system in gloom. My observation was that the class hierarchy I came up with in building gloomstrap didn't always lend itself to a linear storytelling for inclusion in a web. I expect that were I to have designed it in noweb rather than Xcode, I would have had a different hierarchy or even no classes at all.

Similarly, I didn't try test-firsting in gloom, and nor did I port the tests that I did write into the web. Instinct tells me that it would be a faff, but I will try it and find out. I think richer expressions of program intention can only be a good thing, and if Figurative Programming is not the way in which that can be done, then at least we will find out something about what to do instead.

[*] Coming up in January's De Programmatica Ipsum: The Art of _The Art of Computer Programming_, an article about a book that I have _definitely_ read _quite a few bits here and there_ of.

#### Finishing the ObjS story
This gist shows the result of doing the self-threading talked about at the end of the last post. Each method implementation takes an object pointer and a selector name, just like in the real world.

That's enough Objective-Swift for me. Yes, more could be done (mostly defining a preprocessor to make the syntax more regular) but my goal was to understand the definition of Smalltalk-style—or maybe Self-style—objects in a functional programming environment, and I think I'm there. For now.

#### First, Consider no Harmful.
Yesterday, we observed that the goal of considering the go to statement harmful was so that a programmer could write a correct program and have done with it. We noticed that this is never how computering works: many programs are not even instantaneously correct because they represent an understanding of a domain captured at an earlier time, before the context was altered by both external changes and the introduction of the software itself.

Today, let's look at the benefits of removing the go to statement. Dijkstra again:
My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible.
This makes sense! Our source code is a static model of our software system, which is itself (we hope) a model of a problem that somebody has along with tools to help with solving that problem. But our software system is a dynamic actor that absorbs, transforms, and emits data, reacting to and generating events in communication with other human and non-human actors. We need to ensure that the dynamic behaviour is evident in the static model, so that we can "reason about" the development of the system. How does Dijkstra's removal of go to help us achieve that?
Let us now consider how we can characterize the progress of a process. (You may think about this question in a very concrete manner: suppose that a process, considered as a time succession of actions, is stopped after an arbitrary action, what data do we have to fix in order that we can redo the process until the very same point?) If the program text is a pure concatenation of, say, assignment statements (for the purpose of this discussion regarded as the descriptions of single actions) it is sufficient to point in the program text to a point between two successive action descriptions. (In the absence of go to statements I can permit myself the syntactic ambiguity in the last three words of the previous sentence: if we parse them as "successive (action descriptions)" we mean successive in text space; if we parse as "(successive action) descriptions" we mean successive in time.) Let us call such a pointer to a suitable place in the text a "textual index."

[...]

Why do we need such independent coordinates? The reason is - and this seems to be inherent to sequential processes - that we can interpret the value of a variable only with respect to the progress of the process. If we wish to count the number, n say, of people in an initially empty room, we can achieve this by increasing n by one whenever we see someone entering the room. In the in-between moment that we have observed someone entering the room but have not yet performed the subsequent increase of n, its value equals the number of people in the room minus one!
So the value of go to-less programming is that I can start at the entry point, and track every change in the system between there and the point of interest. But I only need to do that once (well, once for each different condition, loop, procedure code path, etc.) and then I can write down "at this index, these things have happened". Conversely, I can start with the known state at a known point, and run the program counter backwards, undoing the changes I observe (obviously encountering problems if any of those are assignments). With go to statements present, I cannot know the history of how the program counter came to be here, so I can't make confident statements about the dynamic evolution of the system.

This isn't the only way to ensure that we understand a software system's dynamic behaviour, which is lucky because it's not a particularly good one. In today's parlance, it "doesn't scale". Imagine being given the bug report "I clicked in the timeline on a YouTube video during an advert and the comments disappeared", and trying to build a view of the stateful evolution of the entire YouTube system (or even just the browser, if you like, and if it turns out you don't need the rest of the information) between main() and the location of the program counter where the bug emerged. Even if we pretend that a browser isn't multithreaded, you would not have a good time.

Another approach is to encapsulate parts of the program, so that the amount we need to comprehend in one go is smaller. When you do that, you don't need to worry about where the global program counter is or how it got there. Donald Knuth demonstrated this in Structured Programming with go to Statements, and went on to say that removing all instances of go to is solving the wrong problem:
One thing we haven't spelled out clearly, however, is what makes some go to's bad and others acceptable. The reason is that we've really been directing our attention to the wrong issue, to the objective question of go to elimination instead of the important subjective question of program structure.

In the words of John Brown [here, Knuth cites an unpublished note], "The act of focusing our mightiest intellectual resources on the elusive goal of go to-less programs has helped us get our minds off all those really tough and possibly unresolvable problems and issues with which today's professional programmer would otherwise have to grapple."
Much has been written on structured programming, procedural programming, object-oriented programming, and functional programming, which all have the same goal: separate a program into "a thing which uses this little bit of software, according to its contract" and "the thing that you would like to believe implements this contract". OOP and FP additionally make explicit the isolation of state changes, so that you don't need to know the whole value of the computer's memory to assert conformance to the contract. Instead, you just need to know the value of the little bit of memory in the fake standalone computer that runs this one object, or this one function (or indeed model the behaviour of the object or function without reference to computer details like memory).

Use or otherwise of go to statements in a thoughtfully-designed (I admit that statement opens a can of worms) is orthogonal to understanding the behaviour of the program. Let me type part of an Array class based on a linked list directly into my blog editor:
Public Class Array Of ElementType
  Private entries As LinkedList(Of ElementType)
  Public Function Count() As Integer
    Dim list As LinkedList(Of ElementType) = entries
    Count = 0
  nonempty:
    If list.IsEmpty() Then GoTo empty
    Count = Count + 1
    list = list.Next()
    GoTo nonempty
  empty:
    Exit Function
  End Function

  Public Function At(index As Integer) As ElementType
    Dim cursor As Integer = index
    Dim list As LinkedList(Of ElementType) = entries
  next:
    If list.IsEmpty() Then Err.Raise("Index Out of Bounds Error")
    If cursor = 0 Then Return list.Element()
    list = list.Next()
    cursor = cursor - 1
    GoTo next
  End Function
End Class

While this code sample uses go to statements, I would suggest it's possible to explore the assertion "objects of class Array satisfy the contract for an array" without too much difficulty. As such, the behaviour of the program anywhere that uses arrays is simplified by the assumption "Array behaves according to the contract", and the behaviour anywhere else is simplified by ignoring the Array code entirely.

Whatever harm the go to statement caused, it was not as much harm as trying to define a "correct" program by understanding all of the ways in which the program counter arrived at the current instruction.

#### Floating point numbers aren't weird
When people say "floating point numbers are weird", they typically mean that the IEEE 754 floating point representation for numbers doesn't meet their needs, or maybe that it meets their needs but it is surprising in its behaviour because it doesn't match their intuitive understanding of how numbers work.

IEEE 754 isn't weird, it's just designed for a specific scenario. One where having lots of different representations of NaN makes sense, because they can carry information about what calculation led to NaN, so you shouldn't do equality comparisons on NaN. One where having positive and negative representations of 0 makes sense. One where…you get the idea.

(What is that specific scenario? It's one where developers need to represent real numbers, and need reliable error-signaling, and don't necessarily understand the limitations well enough to design a good system or handle corner cases themselves. Once you've got an idea that IEEE 754 does something weird, you've probably graduated and are ready to move on.)

But you were sold a general-purpose programming language running on a general-purpose computer, and the only representations of numbers they support are actually not general purpose: fixed-width integers with optional 2s complement negatives, and IEEE 754 floating point. You don't have to use those options, and you don't have to feel like you must be holding it wrong because your supposedly general-purpose programming environment only lets you use specific types of numbers that don't fit your purpose.

Check out decimal representations, arbitrary precision representations, posits, fractional representations, alternative rounding choices, and open up the possibility of general-purpose numbers in your general-purpose computer.

#### Follow-up-and-slightly-over on safety/security
The one thing which makes this a less-than-standard follow-up is that the original was not posted here, but over on paranym Graham Cluley's blog. I originally wrote about the (fictitious) difference between safety and security. For those who didn't clickety the linkelode, I wrote that Jon Gruber's distinction between safety and security was just a neat lexical game to sidestep popular Mac-press opinion. I also wrote that I wanted to cover the original article, Snow Leopard security is all relative. Well, now I shall.So Dennis Fisher argues that "very few users will switch to a Mac or from a Mac because of security"…"But if Snow Leopard turns out to be a major security upgrade over the current versions, that's an important step for Apple and its customers." I'm not sure I agree there. As far as I can see, the fundamental place where Fisher and I start to disagree is on what value security marketing has.I infer from the article that Fisher takes security marketing to be a zero-sum game between the competitors: every time Apple wins, Microsoft necessarily loses: Microsoft have "out-secured" Apple and it's up to Apple to "out-secure" them back. I believe that many consumers consider security as a "hygiene factor"; invisible most of the time, but unacceptable when it becomes an issue. That would make it hard to market a secure OS, but impossible to sell an insecure one. An important distinction, let me explain. People will not consider security as a factor in any product which seems secure enough, but will not touch any product which does not seem secure enough. Therefore a loss for any one company pulls its rep down with respect to the competitors, but there isn't really any such thing as a security marketing "win".Where does that leave Apple? Well, a "major security upgrade" could go in one of two directions. Either it means moving from 0 concern over Mac security to a smaller value of 0 concern; or it could lead people to think that there were some security holes in Leopard that Apple decided not to tell us about and patched up in Snow Leopard. It seems to me that there is, at best, no value in marketing based on the security posture of the OS (though security features are, admittedly, different), however there certainly is value in improving the security posture to avoid the negative market perception of vulnerabilities. There is also value in responding openly and quickly to security issues to stem the rep bleeding any problem would cause.Knowing Apple, though, they'll find the other way; the way of making security posture a winnable marketing game and winning that game.Fisher's article states that the real question "is whether Snow Leopard will be more secure than the current version of OS X" - whereas for the moment the real question is whether Snow Leopard will continue to be secure enough.

#### Forearmed
In researching my piece for the upcoming de Programmatica Ipsum issue on cloud computing, I had thoughts about Apple, arm, and any upcoming transition that didn't fit in the context of that article. So here's a different post, about that. I've worked at both companies so don't have a neutral point of view, but I've also been in bits of the companies far enough from their missions that I don't have any insider insight into this transition.

So, let's dismiss the Mac transition part of this thread straight away: it probably will happen, for the same reasons that the PowerPC->Intel transition happened (the things Apple needed from the parts - mostly lower power consumption for similar performance - weren't the same things that the suppliers needed, and the business Apple brought wasn't big enough to make the suppliers change their mind), and it probably will be easier, because Apple put the groundwork in to make third-party devs aware of porting issues during the Intel transition, and encourage devs to use high-level frameworks and languages.

Whether you think the point is convergence (now your Catalyst apps are literally iPad IPAs that run on a Mac), or cost (Apple buy arm chipset licences, but have to buy whole chips from Intel, and don't get the discount everybody else does for sticking the Intel Inside holographic sticker on the case), or just "betterer", the arm CPUs can certainly provide. On the "betterer" argument, I don't predict that will be a straightforward case of tomorrow's arm Mac being faster than today's Intel Mac. Partly because compilers: gcc certainly has better optimisations on Intel and I wouldn't be surprised to find that llvm does too. Partly because workload, as iOS/watchOS/tvOS all keep the platform on guard rails that make the energy use/computing need expectations more predictable, and those guard rails are only slowly being added to macOS now.

On the other hand, it's long been the case that computers have controller chips in for interfacing with the hardware, and that those chips are often things that could be considered CPUs for systems in their own rights. Your mac certainly already has arm chips in if you bought it recently: you know what's running the OS for the touch bar? Or the T2 security chip? (Actually, if you have an off-brand PC with an Intel-compatible-but-not-Intel chip, that's probably an arm core running the x86-64 instructions in microcode). If you beef one of those up so that it runs the OS too, then take a whole bunch of other chips and circuits off the board, you both reduce the power consumption and put more space in for batteries. And Apple do love talking battery life when they sell you a computer.

OK, so that's the Apple transition done. But now back to arm. They're a great business, and they've only been expanding of late, but it's currently coming at a cost. We don't have up to date financial information on Arm Holdings themselves since they went private, but that year they lost ¥31bn (I think about $300M). Since then, their corporate parent Softbank Group has been doing well, but massive losses from their Vision Fund have led to questions about their direction and particularly Masayoshi Son's judgement and vision.

arm (that's how they style it) have, mostly through their partner network, fingers in many computing pies. From the server and supercomputer chips from manufacturers like Marvell to smart lightbulbs powered by Nordic Semiconductor, arm have tentacles everywhere. But their current interest is squarely on the IoT side. When I worked in their HPC group in 2017, Simon Segars described their traditional chip IP business as the "legacy engine" that would fund the "disruptive unit" he was really interested in, the new Internet of Things Business Unit. Now arm's mission is to "enable a trillion connected devices", and you can bet there isn't a world market for a trillion Macs or Mac-like computers.

If some random software engineer on the internet can work this out, you can bet Apple's exec team have worked it out, too. It seems apparent that (assuming it happens) Apple are transitioning the Mac platform to arm at start of the (long, slow) exit arm make from the traditional computing market, and still chose to do it. This suggests something else in mind (after all, Apple already designs its chips in-house, so why not have them design RISC-V or MIPS chips, or something entirely different?). A quick timetable of Mac CPU instruction sets:

m68k 1984 - 1996, 12 years (I exclude the Lisa)
ppc 1994 - 2006, 12 years
x86 and x86-64 2006 - 2021?, 15 years?
arm 2020? - 203x?, 1x years?

I think it likely that the Mac will wind down with arm's interest in traditional computing, and therefore arm will be the last ever CPU/SoC architecture for computers called Macs. That the plan for the next decade is that Apple is still at the centre of a services-based, privacy-focused consumer electronics experience, but that what they sell you is not a computer.

#### FOSDEM
My current record of FOSDEM attendance sees me there once per decade: my first visit was in 2007 and I'm having breakfast in my hotel at the end of my second trip. I should probably get here more often.

Unlike a lot of the corporate conferences I've been to in other fields, FOSDEM is completely free and completely organised by its community. An interesting effect of this is that whole there's no explicit corporate presence, you'll see companies represented if they actually support free and open source software as much as they claim. Red Hat doesn't have a stand, but pick up business cards from the folks at CentOS, Fedora, GNOME, ManageIQ...

When it comes to free software, I'm a jack of many trades and a master of none. I have drive-by commits in a few different projects including FreeBSD and clang, and recently launched the&nbsp;GNUstep developer guide&nbsp;to add some necessary documentation, but am an expert nowhere.

That makes FOSDEM an exciting selection box of new things to learn, many of which I know nothing or little about. That's a great situation to be in; it's also unsurprising that I know so little as I've only been working with free software (indeed, any software) for a little over a decade.

#### FOSDEM / GNUstep photos
Just came in on #gnustep.  Many photos of the GNUstep booth, dev room and of course the famous GNUstep dinner.

#### Free apps with macdev ticket
The Mac Developer network currently have a Special Offer running until christmas eve, get a free copy of Changes and Code Collector Pro with your ticket. Both are useful apps for any developer's arsenal.

#### Free as in Water
The whole “Free as in beer versus free as in freedom” thing confuses people. Or maybe it doesn’t, and it allows detractors to sow fear, uncertainty and doubt over free software by feigning confusion. Either way, people express confusion.

What is “free as in beer”? Beer is never free, it costs money. Oh, you mean when someone gives me free beer. So, like a round-ordering system, where there’s an expectation that I’ll reciprocate later? Or a promotional beer, where there’s a future expectation that I’ll buy more beer?

No, we mean the beer that a friend buys you when you’re out together and they say “let’s get a couple of beers”. There’s no financial tally kept, no expectation to reciprocate, because then it wouldn’t be friendship: it would be some exchange-mediated relationship that can be nullified by balancing the books. There’s no strings attached, just beer (or coffee, or orange squash, whatever you drink). You get the beer, you don’t pay: but you don’t get to make your own beer, or improve that beer. Gratuity, but no liberty.

Various extensions have been offered to the gratis-vs-libre discussions of freedom. One of the funniest, from a proprietary software vendor’s then-CEO, was Scott McNealy’s “free as in puppies”: implying that while the product may be gratis, there’s work to come afterwards.

I think another extension to help software producers like you and me understand the point of the rights conferred by free software is “free as in water”. In so-called developed societies, most of us pay for water, and most of us have a reasonable expectation of a right to access for water. In fact, we often don’t pay for water, we pay for the infrastructure that gets clean, fresh water to our houses and returns soiled water to the treatment centres. If we’re out of our houses, there are public water fountains in urban areas, and a requirement for refreshment businesses to supply fresh water at no cost.

Of course, none of this is to say that you can’t run a for-profit water business. Here in the UK, that infrastructure that gets the main water supply to our houses, offices and other buildings is run for profit, though there are certain expectations placed on the operators in line with the idea that access to water is a right to be enjoyed by all. And nothing stops you from paying directly for the product: you can of course go out and buy a bottle of Dasani. You’ll end up with water that’s indistinguishable from anybody else’s water, but you’ll pay for the marketing message that this water changes your life in satisfying ways.

When the expectation of the “freedom to use the water, for any purpose” is violated, people are justifiably incensed. You can claim that water isn’t a human right, and you can expect that view to be considered dehumanising.

Just as water is necessary to our biological life, so software has become necessary to our social and civic lives due to its eating the world. It’s entirely reasonable to want insight and control into that process, and to want software that’s free as in water.

#### free as in, well, free(1)
The linux tool free(1) just reports how much memory is free (and how much is in use).  Simple, really.  There's no obvious equivalent for Darwin, though.  We've got top(1) which is overkill, and vm_stat(1) which is abstract to say the least.  I wrote a variant of free(1) for Darwin a while back, but never reported on the virtual memory (swap) usage.  Now I've found out enough about the Mach virtual memory manager to include that, there's an updated Darwin free available.  Excuse the manpage, I haven't updated it to track the new version yet.I also found out enough about the Mach virtual memory manager to realise that there are a few things I want to change about the dynamic pager.  Without a degree in operating systems design, this might take a while...

#### Free Software should welcome contributions by Apple, Google
It started with a toot from the FSF:

  Freedom means not #madebygoogle or #madebyapple, it means #madebythousandsoffreesoftwarehackers #GNU

This post is an expansion on my reply:

  @fsf as an FSF Associate I'm happy to use software made by Google or made by Apple as long as it respects the four freedoms.

Yes to made by Google or made by Apple

The Free Software Foundation financially supports the Replicant project, a freedom-respecting operating system based on the Android Open Source Project. The same Android Open Source Project that's made by Google. Google and Apple are both behind plenty of Free Software contributions, both through their own projects such as Android and Swift or contributions to existing projects like the Linux kernel and CUPS. Both companies are averse to copyleft licences like the GPL, but then both companies have large software patent portfolios and histories of involvement in software patent litigation so it may be that each company is actually averse to compromising the defensibility of their patent hoards through licences like GPL3. On the other hand, the Objective-C support NeXT created for GCC was the subject of an early GPL applicability test so in Apple's case they could well be averse to "testing" the GPL any further.

Whatever their motivations for the stances they've taken, Apple and Google do contribute to Free Software and that should be both encouraged and welcomed. If they want to contribute to more projects, create new ones, or extend those freedoms to their existing proprietary code then we advocates of software freedom should encourage them and welcome them. Freedom does not mean "not #madebygoogle or #madebyapple".

No to controlled by Google or controlled by Apple

While we in software development have never had it so good in terms of software freedom, with all of our tools and libraries being published as free software (usually under the banner of open source), the community at large has never had it so bad, and Google and Apple are at the vanguard of that movement too. The iOS kernel, Darwin UNIX system and Swift programming language may all be open for us to study, share and improve, but they exist in a tightly-controlled walled garden that's eroding the very concept of ownership and centralising all decisions within the spheres of the two platform providers. This means that even Freedom Zero, the freedom to use the software for any purpose, is denied to anyone who isn't a programmer (and in fact to the rest of us too: you can study the iOS kernel but cannot replace the kernel on your phone if you make an improvement; you can study Swift but cannot sell an iOS app using any version other than the one blessed by Apple at time of submission).

People often complain at this point that software freedom is only relevant to programmers because you need to be a programmer to study or improve a program given its source code, but that's not the point. Open Source is only relevant to programmers. Having the freedom to use your computer for any purpose, and to share your software, gives two things:

to some people, "I wish that my software could do this, it doesn't, but I understand that it is possible to change it and that I could use the changed version" can be the incentive to learn and to enable their own programming skills.
to others, having the freedom to share means having the freedom to share the software with someone who already knows how to program who can then make improvements and share them back with the first person.

Ignoring those possibilities perpetuates the current two-tier system in which programmers have a lot of freedom and everybody else has none. I have argued against the walled garden before, as a barrier to freedom. That is different from arguing against things that are made by the companies that perpetuate the walled gardens, if we can encourage them to change.

Welcome, Apple. Seriously.

The FSF has a long history of identifying itself "against" some IT incumbent, usually Microsoft. It has identified a change in the IT landscape by positioning itself as an underdog "against" Apple and Google. But it should not be against them, it should be with them, encouraging them to consider and support the freedom of their customers.

#### From the no-man's-land of the format wars
About nine and a half years ago, a sixteen-year-old gadget boy in Weymouth made a simple mistake.  Given the already near-complete shift of the music industry from the cassette tape to the Philips compact disc, and the superior portability and resilience of the Sony MiniDisc format, this boy decided that it was obvious the world was going to adopt this format.  So our protagonist went out and bought a MZ-R35 walkman.  Three years later, and although the writing was by now on the wall for the storage format, he added an MD-M3 to his collection.

I now believe I own all five pre-recorded MiniDisc albums ever made (though I don't remember when I bought Hours by Bowie, and can only think that I bought Recurring Dream because at the time I fancied a girl who liked Crowded House), and have swathes of my vinyl and tape collection &quot;backed up&quot; to recordable MDs.  But the rest of the world forgot to catch up with me!  Where are the Hi-MD drives built in to laptops?  Even Sony don't offer that...  Come to that, why do we still put up with crappy scratchable CDs?  iPods may be a damn sight more convenient than my MD walkman is, but the bandwidth of an amazon package containing MiniDiscs is still far higher than the connection between my laptop and iTunes.

I still intend to find the required cable and port the rest of my LPs to the format though, as MDs are definitely more portable and resilient than is vinyl.  And I haven't actually listened to Bauhaus' 1979-1983 in years.

#### FSF membership
I am now an associate member of the FSF.  This is a good way to support Free Software development (including GNUstep, and you don't even need to be able to code :-).  I've added a referral link to the sidebar - I don't get a kickback obviously, although I do get gifts if enough people are referred by me, and it helps the FSF to track where donors are getting their information from.  If you use Free Software, you might consider donating some cash - especially now that the dollar's so crap ;-).

#### Fuck. This. Shit.
Enough with the subtle allusions of the previous posts. What's going on here is not right.

It's not right that I get to pass as a member of the group of people who can work in technology, while others have to justify their very presence in the field.

It's not right that "looking like me" is a pass to being considered for the best-paid jobs, while "not looking like me" is not.

[that last one took me a long time to understand. To me, it seems like I worked hard to get where I am, but I needed to understand that I was given the opportunity to work at the things I worked at. That all I needed to do was to work at it, not to work at it and convince everyone else that I was eligible to work at it.]

It's not right that while I get a free pass into various fields of endeavour, others are told that they either slept their way into the field or are groupies or are unfuckable.

Previously, I avoided writing about this stuff because I thought I might get writing about this stuff wrong. Fuck. This. Shit. I've got social capital to burn; it's much easier for me to get another job around this sector than plenty of people who are as good or better than me at doing the work. I might be worried about treading the line between being supportive and getting into trouble, but that's not as bad as the line women, trans people, non-white people, non-straight people, disabled people have to tread between asking to be considered equally and keeping their jobs have to tread. I have one job: doing my job. I do not have two jobs: doing my job, and convincing people that someone like me should be allowed to do my job. If the cost of equality is giving up my free ride, then I give up my free ride.

The pipeline is not the problem, it leads to a vat of acid. No-one wants to lean in to a vat of acid. (Thanks to Cate Huston for that metaphor, it made a lot of sense to me.)

Our industry is exclusive, and needs to be inclusive. What should you do about this? I don't know, I'm far from knowledgable. If your position is "I agree with the straight white guy that the world is broken, I should ask the straight white guy how to fix it" then perhaps you are the problem, just as I have been and am the problem.

What should I do about this? First step for me is to listen. To not tell people who are describing their experiences what my experiences are. To avoid thinking about my reply to people, and to think about what they've said. To stop looking for holes in arguments and to listen for opportunities to grow. Not just to grow me, but to grow others.

#### Full Stack
A full-stack software engineer is someone who is comfortable working at any layer, from code and systems through team members to customers.

#### Full-stack
That moment where you're looking back through your notes to see that you've:

modelled charge carrier behaviour in semiconductors
built a processor from discrete logic components
patched kernels
patched operating system tools
written filesystems
written device drivers
contributed to a foundation library
fixed compiler bugs
performance-tuned databases
built native apps
built web apps
tested the above
taught other developers
mentored other developers
sold the above skills

and a thought occurs: someone who knows both PHP and JavaScript is called a full-stack developer.

Someone who tells you that programmers are rational actors who are above marketing is lying. Everything in the field is marketing, including the idea of rationality. Performing such marketing becomes easier when the recipients don't think it exists, or don't think they're the sort of people who would fall for it.

I'm reminded of the audience reaction to this serial entrepreneur describing his recently-acquired startup's technology (skip to 10:24 if the timestamped URL doesn't do that automatically for you). He presents these four points:

DOS: 1 Analogy Unit
Mac + HI Toolbox: 5 Analogy Units
DOS + Windows: 7 Analogy Units
Mach OS + OpenStep: 20 Analogy Units

Listen to the applause for the unjustified number 20.

#### Fun and games (with rewritten rules) in Objective-C
An object-oriented programming environment is not a set of rules. Programs do not need to be constructed according to the rules supplied by the environment. An object-oriented environment includes tools for constructing new rules, and programs can use these to good effect. Let's build multiple method inheritance for Objective-C, to see a new set of rules.

The goal

Given three classes, A, B, and C:

@interface A : NSObject

-(NSInteger)a;
-(NSInteger)double:(NSInteger)n;

@end

@implementation A

-(NSInteger)a { return [self double:6]; }

@end

@interface B : NSObject

-(NSInteger)b;

@end

@implementation B

-(NSInteger)b { return 30; }

@end

@interface C : NSObject

-c;

@end

@implementation C

-(NSInteger)double:(NSInteger)a
{
  return a*2;
}

-c { return @([self a] + [self b]); }

@end

We want to find the value of a C instance's c property. That depends on its values of a and b, but that class doesn't have those methods. We should add them.

int main(int argc, const char * argv[]) {
  @autoreleasepool {
    C *c = [C new];
    [c addSuperclass:[A class]];
    [c addSuperclass:[B class]];
    NSLog(@"The answer is %@", c.c);
  }
}

We want the following output:

2015-02-17 20:23:36.810 Mixins[59019:418112] The answer is 42

Find a method from any superclass

Clearly there's some chicanery going on here. I've changed the rules: methods are no longer simply being looked up in a single class. My instance of C has three superclasses: A, B and NSObject.

@interface NSObject (Mixable)

- (void)addSuperclass:(Class)aSuperclass;

@end

@implementation NSObject (Mixable)

-superclasses
{
  return objc_getAssociatedObject(self, "superclasses");
}

- (void)addSuperclass:(Class)aSuperclass
{
  id superclasses = [self superclasses]?:@[];
  id newSupers = [superclasses arrayByAddingObject:aSuperclass];
  objc_setAssociatedObject(self, "superclasses", newSupers, OBJC_ASSOCIATION_RETAIN_NONATOMIC);
}

- (Class)superclassForSelector:(SEL)aSelector
{
  __block Class potentialSuperclass = Nil;
  [[self superclasses] enumerateObjectsUsingBlock:^(Class aClass, NSUInteger idx, BOOL *stop) {
    if ([aClass instancesRespondToSelector:aSelector])
    {
      potentialSuperclass = aClass;
      *stop = YES;
    }
  }];
  return potentialSuperclass;
}

- (NSMethodSignature *)original_methodSignatureForSelector:(SEL)aSelector
{
  NSMethodSignature *signature = [self original_methodSignatureForSelector:aSelector];
  if (signature)
  {
    return signature;
  }
  Class potentialSuperclass = [self superclassForSelector:aSelector];
  return [potentialSuperclass instanceMethodSignatureForSelector:aSelector];
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
  SEL aSelector = [anInvocation selector];
  Class potentialSuperclass = [self superclassForSelector:aSelector];
  [anInvocation invokeSuperImplementation:potentialSuperclass];
}

+ (void)load
{
  if (self == [NSObject class])
  {
    method_exchangeImplementations(class_getInstanceMethod(self, @selector(original_methodSignatureForSelector:)),
                                   class_getInstanceMethod(self, @selector(methodSignatureForSelector:)));
  }
}

@end

Now invoke that method.

When you write [super foo], the Objective-C runtime needs to send a message to your object but tell the resolution machinery to look at the superclass for the method implementation, not at the current class. It uses a function objc_msgSendSuper to do this. In this case, I don't have the superclass: I have a superclass, one of potentially many. So what I need to do is more general than what messaging super does.

Luckily for me, objc_msgSendSuper is already sufficiently general. It receives a pointer to self, just like the usual objc_msgSend, but in addition it receives a pointer to the class to be used as the superclass. By controlling that class pointer, I can tell the system which superclass to use.

A category on NSInvocation calls objc_msgSendSuper with the appropriate arguments to get the correct method from the correct class. But how can it call the function correctly? Objective-C messages could receive any number of arguments of any type, and return a value of any type. Constructing a function call when the parameters are discovered at runtime is the job of libffi, which is used here (not shown: a simple, if boring, map of Objective-C value encodings to libffi type descriptions).

@interface NSInvocation (SuperInvoke)

-(void)invokeSuperImplementation:(Class)superclass;

@end

@implementation NSInvocation (SuperInvoke)

- (BOOL)isVoidReturn
{
  return (strcmp([[self methodSignature] methodReturnType], "v") == 0);
}

-(void)invokeSuperImplementation:(Class)superclass
{
  NSMethodSignature *signature = [self methodSignature];
  if (superclass)
  {
    struct objc_super super_class = { .receiver = [self target],
      .super_class = superclass };
    struct objc_super *superPointer = &amp;super_class;
    ffi_cif callInterface;
    NSUInteger argsCount = [signature numberOfArguments];
    ffi_type **args = malloc(sizeof(ffi_type *) * argsCount);
    for (int i = 0; i &lt; argsCount; i++) {
      args[i] = [self ffiTypeForObjCType:[signature getArgumentTypeAtIndex:i]];
    }
    ffi_type *returnType;
    if ([self isVoidReturn]) {
      returnType = &amp;ffi_type_void;
    }
    else {
      returnType = [self ffiTypeForObjCType:[signature methodReturnType]];
    }
    ffi_status status = ffi_prep_cif(&amp;callInterface,
                                     FFI_DEFAULT_ABI,
                                     (unsigned int)[signature numberOfArguments],
                                     returnType,
                                     args);
    if (status != FFI_OK) {
      NSLog(@"I can't make an FFI frame");
      free(args);
      return;
    }

    void *argsBuffer = malloc([signature frameLength]);
    int cursor = 0;
    cursor += args[0]-&gt;size;
    void **values = malloc([signature numberOfArguments] * sizeof(void  *));
    values[0] = &amp;superPointer;

    for (int i = 1; i &lt; [signature numberOfArguments]; i++) {
      values[i] = (argsBuffer + cursor);
      [self getArgument:values[i] atIndex:i];
      cursor += args[i]-&gt;size;
    }
    if ([self isVoidReturn]) {
      ffi_call(&amp;callInterface, objc_msgSendSuper, NULL, values);
    } else {
      void *result = malloc(returnType-&gt;size);
      ffi_call(&amp;callInterface, objc_msgSendSuper, result, values);
      [self setReturnValue:result];
      free(result);
    }
    free(args);
    free(values);
    free(argsBuffer);
  }
}

@end

Conclusion

You've seen this conclusion before: blah blah awesome power of the runtime. It doesn't just let you do expressive things in the Objective-C game, it lets you define a better game.

#### Functional Programming in Object-Oriented Programming in Functional Programming in Swift
The objects that I've been building up over the last few posts have arbitrarily broad behaviours. They can respond to any selector drawn from the set of all possible strings.

As with all art, beauty is produced by imposing constraints. An important class (pardon the pun) of objects only has a meaningful response to one selector (value or, if it takes an additional argument, value:) which causes the object to do the one thing it knows how and return a result.

In many object-oriented programming circles, including the Smalltalk world, these objects are called blocks. They're useful because they enable multiple uses of the same algorithm to be extracted into a single place, with the block customising the significant activity that happens in the algorithm. As an example, iterating an array to square numbers and iterating an array to load images from paths both involve iterating an array. The algorithm that iterates an array can be expressed in a form that accepts a block that does the work: squaring some numbers or loading some images.

Without going in to the details of the implementation (which is a collection of classes in the Objective-Swift system), here's how a block could be used:

let myArray = newObject(NSArray(Integer(1, proto: o), Integer(2, proto: o),
  Integer(3, proto: o), Integer(4, proto: o)))
let evens = (myArray,"filter:") ✍ OneArgBlock({ obj in
  return (ℹ︎(obj)! % 2) == 0 ? True : False
  }, proto: o)
ℹ︎(evens→"count") // 2
&#x1f4d3;((evens,"objectAtIndex:") ✍ Integer(0, proto: o)) // "2"
&#x1f4d3;((evens,"objectAtIndex:") ✍ Integer(1, proto: o)) // "4"

The NSArray→filter: selector takes a block, and for each element in the array it sends the value: message with the element as the argument. In fact, it passes another block to the result of this block. Both True and False respond to the ifTrue: message. True→ifTrue: executes its argument, False→ifTrue: doesn't.

Now we already have a name for the concept of an object that does exactly one thing, returning some result based on its input parameters. That's a function. Yes, we finally got there:

Swiftception

So just as Object-Oriented Programming was a restricted application of a subset of Functional Programming, we can see Functional Programming as a restricted application of a subset of Object-Oriented Programming.

And that's pretty much the point of this series of posts. These two ways of thinking about computer programs get you to the same place, as long as you apply the thinking. Neither is a silver bullet, neither is subsumed nor obsoleted by the other.

Coda

Now imagine a block that takes a selector and returns another block…

#### Further Advances in Objective-Swift
Previously on SICPers, I defined objects as functions that return methods and built dynamic method dispatch in this object system. It's time to tie up some loose ends.

Proper selectors

In languages like Smalltalk and Objective-C, an object's range isn't a small list of selectors like count and at:. It's the whole of the String type.

typealias Selector = String

Now an object takes a Selector (i.e. a String) and returns a method implementation. Continuing with the type-safe theme from the previous posts, I'll define a few different types of implementation that I might want to use.

enum IMP {
  case accessor(()-&gt;((Selector)-&gt;IMP)?)
  case asInteger(()-&gt;Int?)
  case methodMissing(()-&gt;((Selector)-&gt;IMP)?)
  case mutator(((Selector-&gt;IMP))-&gt;Void)
  case description(()-&gt;String?)
}

typealias Object = Selector -&gt; IMP

Now I can create some proper objects.

func DoesNothing()-&gt;(_cmd:Selector)-&gt;IMP {
  var _self : Object! = nil
  func myself (selector: Selector)-&gt;IMP {
    return IMP.methodMissing({assertionFailure("method missing: \(selector)"); return nil;})
  }
  _self = myself
  return _self
}

let o : Object = DoesNothing()

func Integer(x: Int, proto: Object) -&gt; Object {
  var _self : Object! = nil
  let _x = x
  func myself(selector:Selector) -&gt; IMP {
    switch(selector) {
    case "asInteger":
      return IMP.asInteger({ return _x })
    case "description":
      return IMP.description({ return "\(_x)" })
    default:
      return proto(selector)
    }
  }
  _self = myself
  return _self
}

let theMeaning = Integer(42, o)

A better syntax

Usually you don't think of method lookup as a function invocation, but rather as finding a member of an object (indeed in C++ they're called member functions). A member-like syntax could look like this:

infix operator .. {}

func .. (receiver: Object?, _cmd:Selector) -&gt; IMP? {
  if let this = receiver {
    let method = this(_cmd)
    switch(method) {
    case .methodMissing(let f):
      return f().._cmd
    default:
      return method
    }
  }
  else {
    return nil
  }
}

This system now has the same nil behaviour as Objective-C:

nil.."asInteger" // nil

And it also has a limited form of default message forwarding:

func Proxy(target:Object)-&gt;((_cmd:Selector)-&gt;IMP) {
  var _self : Object! = nil
  var _target = target
  func myself(selector:Selector) -&gt; IMP {
    return IMP.methodMissing({ return _target })
  }
  _self = myself
  return _self
}

let proxyMeaning = Proxy(theMeaning)
let descriptionImp = proxyMeaning.."description"
descriptionImp!.describe() // (Enum Value)

But it's going to be pretty tedious typing all of those switch statements to unbox the correct implementation of a method. There are two ways to go here, one is to add methods to the enumeration to make it all easier. These just unbox the union and call the underlying function, so for example:

extension IMP {
  func describe() -&gt; String? {
    switch(self) {
    case .description(let f):
      return f()
    default:
      return nil
    }
  }
}

descriptionImp!.describe() // "42"

Or if you're really going to town, why not define more operators?

infix operator → {}

func → (receiver: Object?, _cmd:Selector) -&gt; Object? {
  if let imp = receiver.._cmd {
    switch (imp) {
    case .accessor(let f):
      return f()
    default:
      return nil
    }
  } else {
    return nil
  }
}

func ℹ︎(receiver:Object?)-&gt;Int? {
  if let imp = receiver.."asInteger" {
    switch(imp) {
    case .asInteger(let f):
      return f()
    default:
      return nil
    }
  } else {
    return nil
  }
}

ℹ︎(theMeaning)! // 42

Mutable Objects

There's no reason why an object couldn't close over a var and therefore have mutable instance variables. You can't access the variables from outside the object in any way other than through its methods[*], even from objects that inherit from it. They're all private.

[*] Unless you were to build a trap door, e.g. declaring the var in a scope where some other function also has access to it.

Firstly, some similar notation:

infix operator ☞ {}

func ☞ (mutator:IMP?, value: Object) -&gt; Void {
  if let mut = mutator {
    switch(mut) {
    case .mutator(let f):
      f(value)
    default:
      return
    }
  }
}

Here is, somewhat weirdly, a class of Point objects where you can redefine both the x and y coordinates after creation.

func Point(x: Int, y: Int, proto: Object)-&gt;((_cmd:Selector)-&gt;IMP) {
  var _self : Object! = nil
  var _x = Integer(x,o), _y = Integer(y,o)
  func myself (selector:Selector) -&gt; IMP {
    switch (selector) {
    case "x":
      return IMP.accessor({
        return _x
      })
    case "y":
      return IMP.accessor({
        return _y
      })
    case "setX:":
      return IMP.mutator({ newX in
        _x = newX
      })
    case "setY:":
      return IMP.mutator({ newY in
        _y = newY
      })
    case "description":
      return IMP.description({
        let xii = ℹ︎(_self→"x")!
        let yii = ℹ︎(_self→"y")!
        return "(\(xii),\(yii))"
      })
    default:
      return proto(selector)
    }
  }
  _self = myself
  return _self
}

In use it's much as you'd expect.

let p = Point(3, 4, o)
(p.."description")!.describe() // "(3,4)"
ℹ︎(p→"x") // 3
(p.."setX:")☞(Integer(1,o))
ℹ︎(p→"x") // 1
(p.."description")!.describe() // "(1,4)"

But there's no need to bother

It's possible to build mutable objects out of immutable objects. Given a Point(3,4,o), you can make an object that looks exactly like a Point at (1,4) by building an object that has replacement implementations for x and description, but otherwise forwards all methods to the original.

In this way, mutation would look a lot like the Command pattern. You have your original state, you have a linked list (via the prototype chain) of modifications to that state, and the external view is as if you only had the final state.

Further further advances

It'd be good to be able to tell a method what its self is, to make inheritance work properly. For example, if a Point's description knew what object was self, then replacing x on a subtype would automatically get the description right. Ben Lings shows how this might work on the previous post's List objects.

Further thanks

Jeremy Gibbons initiated the discussion on mutable objects that led to the implementation shown above. Thanks to Lawrence Lomax for lots of feedback and discussion about how this should all work, including an alternate implementation.

#### Fuzzing as a security testing tool
Google have a new browser project, called chrome, and in their introduction they explain perfectly, through the medium of image, how fuzzing works.

Of course, as anyone could tell you, if you take a thousand monkeys and a thousand typewriters and put them all in a room for long enough, you will end up with a thousand broken typewriters, ten fat monkeys and 990 monkey skeletons.

#### GAH!
You can't use a java.awt.Color in WebObjects, because to use AWT at all on OS X requires connection to the WindowServer.  And unless you're mad, you probably aren't running your WOA as root.  This means I can't do things like, ooh, I don't know, making a PDF table have a white border instead of a black one.

#### Garbage-collected Objective-C
When was a garbage collector added to Objective-C? If you follow Apple's work with the language, you might be inclined to believe that it was in 2008 when AutoZone was added as part of Objective-C 2.0 (the AutoZone collector has since been deprecated by Apple, and I'm not sure whether anyone else ever adopted it).

With a slightly wider knowledge of the language's history, you can push this date back a bit. The GNUstep project—a Free Software reimplementation of Apple's (formerly NeXT's) APIs—has been using the Boehm–Demers–Weiser collector for a while. How long? I can't tell exactly, but a keyword search in the project's version control logs makes me think that most of the work to support it was done by one person in mid-2002:

  r13976 | nico | 2002-06-26 15:34:16 +0100 (Wed, 26 Jun 2002) | 3 lines
  
  Do not add -lobjc_gc -lgc flags when compiling with gc=yes - should now
  be added automatically by gnustep-make
  
  
  
  r13971 | nico | 2002-06-25 18:28:56 +0100 (Tue, 25 Jun 2002) | 2 lines
  
  Tidyup for gc=yes with old compilers
  
  
  
  r13970 | nico | 2002-06-25 18:23:05 +0100 (Tue, 25 Jun 2002) | 2 lines
  
  Tidied code to compile with gc=yes and older compilers
  
  
  
  r13969 | nico | 2002-06-25 13:36:11 +0100 (Tue, 25 Jun 2002) | 3 lines
  
  Tidied some indentation; a couple of insignificant changes to have it compile
  under gc
  
  
  
  r13968 | nico | 2002-06-25 13:15:04 +0100 (Tue, 25 Jun 2002) | 2 lines
  
  Tidied code which wouldn't compile with gc=yes and gcc &lt; 3.x
  
  
  
  r13967 | nico | 2002-06-25 13:13:19 +0100 (Tue, 25 Jun 2002) | 2 lines
  
  Tidied code which was not compiling with the garbage collector
  
  
  
  r13966 | nico | 2002-06-25 13:12:17 +0100 (Tue, 25 Jun 2002) | 2 lines
  
  Tidied code which was not compiling with gc=yes

That was, until fairly recently, the earliest example I knew about. Then I discovered a conference talk by Paulo Ferreira:

Reclaiming storage in an object oriented platform supporting extended C++ and Objective-C applications

This is a paper presented at "1991 International Workshop on Object Orientation in Operating Systems". 1991. That is—obviously—11 years before GNUstep's GC work and 17 years before Apple released AutoZone.

Comandos

The context in which this work was being done is a platform called Comandos. I'd never heard of that before—and I thought I knew Objective-C!

Judging from the report linked above, Comandos is a platform for distributed and parallel object-oriented software, based on UNIX but supporting multiple variants. The fact that it was created in 1986 means that both the languages supported—Objective-C and C++—were new at the time. Indeed the project was contemporary with the development of NeXTSTEP, which was publicly released to developers in 1988.

The 1994 summary report doesn't mention Objective-C: just C++, Eiffel and a bespoke language called Guide. It's possible that the platform supported ObjC simply because they used gcc which picked up ObjC support during the life of Comandos; however this seems unlikely as there would be significant work in making Objective-C objects work with their platform's distributed messaging interface and persistence subsystem.

Why ObjC should be one of two languages mentioned (along with C++) in the 1991 paper on garbage collection, but zero of three mentioned (C++, Eiffel, Guide) in 1994 will have to remain a mystery for now. Looking into the references for Ferreira's paper, I can find one mention of Objective-C as the inspiration for their own, custom C-based message dispatch system, but no indication that they actually used Objective-C.

The Garbage Collector

I'm not really an expert at garbage collectors. In fact, I have no idea what I'm doing. I appreciate them when they're around, and leak or crash things occasionally when they're not.

To my uneducated eye, the description of the Ferreira 1991 garbage collector and Apple's description of their collector (no link I'm afraid, it was session 940 at WWDC 2008) look quite different. AutoZone is conservative (like B-W-D) and only works on Objective-C objects. Ferreira's collector operates, like B-W-D, on any memory block including new C++ instances and C heap allocations. Apple's collector is supposed to avoid blocking wherever it can, a constraint not mentioned in the Ferreira paper.

All of Comandos, GNUstep and Cocoa (Apple's Objective-C framework) have systems for distributed objects that complicate collection: does some remote process have a handle on memory in my address space? The proxy system used by Cocoa and GNUstep make it easy to answer this question. Comandos used a different technique, where objects local to a process were "volatile" and objects shared between processes were "persistent". Persistent objects were subject to a different lifecycle management process, so the Ferreira GC didn't interact with them.

As an aside, Apple's garbage collector also needed to provide a "mixed mode"—support for code that could be loaded into either a garbage-collected or manually managed process.

Conclusions

Memory management is hard. Making programmers do it themselves leads to all sorts of problems. Doing it automatically is also hard, and many different approaches have been tried over the last few decades. Interestingly, Apple has (for the moment) settled on a "none of the above" approach, using a compiler-inserted reference counting system based on the manual ownership tracking previously implemented by the frameworks.

What interests me most about this paper on Objective-C garbage collection is not so much its technical content (which it's actually rather light on, containing only conversational overviews of the algorithms and no information about results), but the fact that it existed at all and I, as someone who considers himself an experienced Objective-C programmer, did not know anything about it or its project.

That's why I started this blog [Ed: referring to the blog these posts are imported from] by discussing it. A necessary prerequisite to deciding whether the literature has something useful to tell us is knowing about its existence. I'm really surprised that it took so long for me to find out about something that's almost directly related to my everyday work. Mind you, maybe I shouldn't feel too bad: the author of AutoZone told me he hadn't heard of it, either.

#### Gently HURDing the side projects
I find it problematic that even at times when I'm avoiding computing outside of work, I still have ideas about things I would like to try out or improve in computing "if I had the time". I tend to capture these somehow - usually written notes in paper or Evernote, and my personal technology radar.

Why might this be a problem? Isn't having ideas good, and fun? Well it is, but with each comes guilt that I could be making progress on it but am not. Even when that's my choice, when I deliberately put more effort into relationships with friends or musical projects or whatever, there's still that nagging feeling that I'm leaving behind chances to make positive changes to computing.

My approach to addressing that started by building the radar. Now I don't have a lot of different projects I could be working on but am not; I have a single related web of issues, and progress on any one thing counts as progress toward the whole.

The second change is to note that any progress is progress; sometimes I spend some time reading and make a sentence or two of notes on dealing with a problem. Sometimes I try a solution, find difficulties with it and write down that I discount that solution. If I've made some move forward from where I was before I've started, I can be satisfied and don't need to burn the midnight oil to get a complete solution to a complex problem done before putting it down.

All of that goes toward describing the limited progress I've made on my current research topic, which is distributed message-passing. I like the idea from Erlang that objects run in separate contexts, completely decoupled except for passing messages between one another. This seems to be the best implementation of an object-oriented runtime environment, except that it is all done on the Erlang VM and in the couple of relatively esoteric languages that target it.

On the other hand, while Objective-C doesn't make it easy to do that decoupling, it does have a very simple message-passing interface that can be implemented in any language with a C FFI. If you can wrap objc_msgSend or objc_msgLookup and expose it to your language runtime, you too can pass messages.

Why can't we have both of these things? Why can't we have the simple-to-integrate message interface that can work anywhere, along with the distributed and decoupled objects?

My theory is that Mach makes this possible so I've been investigating it using GNU Mach and the GNU HURD. Much of the documentation of Mach messaging uses name servers that register named ports for clients to find; this is how macOS, NeXTSTEP, OSF/1 and related systems work. HURD does not use a name server, it uses the filesystem: you attach a server to a file system node as a translator and clients find the ports by looking up their paths on the filesystem.

I found examples of filesystem translators in the HURD documentation, but they typically were examples that implemented the filesystem messages: seek, read, write, and so on. One could build message-sending on top of filesystem operations but it would not be pleasant:

marshall the message selector and arguments into some stream format
write() your message to the node
verify that you wrote as many bytes as you expect
read() the length of the reply
verify that you read as many bytes as you expect
read() the reply
verify that you read as many bytes as you expect
unmarshall the reply into an object of the correct type

Let's be clear, all of this needs to be done, but it's all already being done at the Mach message layer and hidden behind the MIG abstraction, so why should our clients and servers do it again in another abstraction built on top? I wanted to find a way to register a port that accepts non-filesystem messages using the HURD's filesystem-as-name-server approach.

This morning I decided to look at how login was implemented on the HURD and discovered that the password server does exactly what I need. It is configured as a translator on the filesystem, and uses the trivfs library to check in with the bootstrap server and get its ports, but then it handles its own messages for checking passwords rather than the standard filesystem messages. Discovering that gives me enough new information to feel I've made progress, and a clear next step (pardon the pun).

#### Getting better at doing it wrong
For around a month at the end of last year, I kept a long text note called "doing doing it wrong right". I was trying to understand error handling in programming, look at some common designs and work out a plan for cleaning up some error-handling code I was working with myself (mercifully someone else, with less analysis paralysis, has taken on that task now).

Deliciously the canonical writing in this field is by an author with the completely apt name Goodenough. His Structured Exception Handling and Exception Handling: Issues and a Proposed Notation describe the problem pretty completely and introduce the idea of exceptions that can be thrown on an interesting condition and caught at the appropriate level of abstraction in the caller.

As an aside, his articles show that exception handling can be used for general control flow. Your long-running download task can throw the "I'm 5% complete now" exception, which is caught to update the UI before asking the download to continue. Programming taste moved away from doing that.

In the Cocoa world, exceptions have never been in favour, probably because they're too succinct. In their place, multi-if statement complex handling code is introduced:

NSError *error = nil;
id thing = [anObject giveMeAThing:&amp;error];
if (!thing) {
  [self handleError:error];
  return;
}
id otherThing = [thing doYourThing:&amp;error];
if (!otherThing) {
  [self handleError:error];
  return;
}
id anotherThing = [otherThing someSortOfThing:&amp;error];

...and so it goes.

Yesterday in his NSMeetup talk on Swift, Saul Mora reminded me of the nil sink pattern in Objective-C. Removing all the error-handling from the above, a fluent (give or take the names) interface would look like this:

id anotherThing = [[[anObject giveMeAThing] doYourThing] someSortOfThing];

The first method in that chain to fail would return nil, which due to the message-sink behaviour means that everything subsequent to it preserves the nil and that's what you get out. Saul had built an equivalent thing with option types, and a function Maybe a -&gt; (a -&gt; Maybe b) -&gt; Maybe b to hide all of the option-unwrapping conditionals.

Remembering this pattern, I think it's possible to go back and tidy up my error cases:

NSError *error = nil;
id anotherThing = [[[anObject giveMeAThing:&amp;error]
                               doYourThing:&amp;error]
                           someSortOfThing:&amp;error];
if (!anotherThing) {
  [self handleError:error];
}

Done. Whichever method goes wrong sets the error and returns nil. Everything after that is sunk, which crucially means that it can't affect the error. As long as the errors generated are specific enough to indicate what went wrong, whether it's possible to recover (and if so, how) and whether anything needs cleaning up (and if so, what) then this approach is...good enough.

#### Getting started on my Vampire V4
Apollo accelerators make the Vampire, the fastest Motorola 680x0-compatible accelerators for Amiga around. Actually, they claim that with the Sheepsaver emulator to trap ROM calls, it's the fastest m68k-compatible Mac around too.

The Vampire Standalone V4 is basically that accelerator, without the hassle of attaching it to an Amiga. They replicated the whole chipset in FPGA, and ship with the AROS ROMs and OS for an open-source equivalent to the real Amiga experience.

I had a little bit of trouble setting mine up (this is not surprising, as they're very early in development of the standalone variant and are iterating quickly). Here's what I found, much of it from advice gratefully received from the team in the Apollo Slack. I replicate it here to make it easier to discover.

You absolutely want to stick to a supported keyboard and mouse, I ended up getting the cheapest compatible from Amazon for about £20. You need to connect the mouse to the USB port next to the DB-9 sockets, and the keyboard to the other one. On boot, you'll need to unplug and replug the mouse to get the pointer to work.

The Vampire wiki has a very scary-looking page about SD cards. You don't need to worry about any of that with the AROS image shipped on the V4. Insert your SD card, then in the CLI type:

mount sd0:

When you're done:

assign sd0: dismount
assign sd0: remove

The last two are the commands to unmount and eject the disk in AmigaDOS. Unfortunately I currently find that while dismounting works, removing doesn't; and then subsequent attempts to re-mount sd0: also fail. I don't know if this is a bug or if I'm holding it wrong.

The CF card with the bootable AROS image has two partitions, System: and Work:. These take up around 200MB, which means you've got a lot of unused space on the CF card. To access it, you should get the fixhddsize tool. UnLHA it, run it, enter ata.device as your device, and let it fix things for you.

Now launch System:Tools/HDToolBox. And click "Add Entry". In the Devices dialog, enter ata.device. Now click that device in the "Changed Name" list, then double-click on the entry that appears (for me, it's SDCFXS-0 32G...). You'll see two entries, UDH0: (that's your System: partition) and UDH1: (Work:). Add an entry here, selecting the unused space. When you've done that, save changes, close HDToolBox, and reboot. You'll see your new drive appear in Workbench, as something like UDH2:NDOS. Right click that, choose Format, then Quick Format. Boom.

My last tip is that AROS doesn't launch the IP stack by default. If you want networking, go to System:Prefs/Network, choose net0 and click Save. Optionally, enable "Start networking during system boot".

#### Give me an S
S - I can find the thing I need to change.
O - My change will either be an extension or a replacement.
L - My replacement or extension needs to be a drop-in change.
I - Here's what my replacement can do.
D - I have somewhere to put my replacement.

#### given-when-then in XCTest
I started writing a new Mac app, and I started doing it by driving the implementation through Xcode UI Automation tests. But then it turned out I was driving the test infrastructure as much as the tests, and it's that I want to talk about.

Given, When, Then

My (complete, Xcode UI Automation) test looks like this:

func testAddingANoteResultsInANoteBeingAdded() {
    given("An empty notebook")
    when("I add a note to the notebook")
    then("There is a note in the notebook")
}

The test case class has an object called a World, which holds, well, the test's world. There are two parts to this.

The World holds regular expressions associated with blocks, where each block does some part of the test if its associated regular expression matched the description of the test. As an example, my test fixture sets up this association:

try world.then(matchingExpectation: "^There is a note in the notebook$",
                work: { _, world in
                guard let notebook:LabraryNotebook
                  = world.getFromState("TheNotebook") as? LabraryNotebook else {
                    XCTFail("No notebook to test")
                    return
                }
                XCTAssertEqual(notebook.countOfNotes(), 1,
                  "There should be one row in the notes table")
})

We'll get back to how that block is implemented later. For the moment, I want to make it clear that this is a way to organise a UI test (or, indeed, any other functional test) using XCTest: it is not a new test framework. The test case class still subclasses XCTestCase, and assertions are still made with the XCTAssert* macros/functions. That's just all wrapped up in this given/when/then structure.

Let's look at the block's two parameters: the first is an array of the regular expression's capture groups so that you can find out information about the test specification, should you want.

The other argument is a reference to the World, which enables the second feature of the World: as state storage so that each part of the test can communicate with later parts. Notice that the when clause in my test says it adds a note to "the notebook", and the then clause checks that there is a note in "the notebook". How do they both use the same notebook object? The when clause stores it on the World using world.storeInState(), and the then clause retrieves it with world.getFromState().

Page Objects

Rather than putting XCUIElement goop directly in my test blocks, I use an abstraction called the Page Object pattern, popular among people writing browser tests in Selenium. This puts an adapter between my tests and my UI controls, so the test says (for example) app.newDocument() and the Application page object knows that that means finding the "File" menu, clicking it, then clicking the "New" menu item.

The way to create a new document in a Cocoa app has not changed since 1987 and may not change soon. But the details of my own UI surely will, and will change at a different rate than the goals of the people using it. While someone may want to add a note for the rest of time, there may not always be an "Add Note" button. So my test can continue to say:

when("I add a note to the notebook")

but the page object for a document can change from:

func addANote() {
    let app = XCUIApplication()
    let window = app.windows[documentName]
    let control = window.buttons["Add Note"]
    control.click()
}

to whatever will find and drive the interface in my redesigned application.

Would you like this?

I'm happy to package the given/when/then organisation up and release it under an open source licence so that you can use it in your own apps. As I've only just written the code, I've yet to do that, but it's coming! I'm aware that there are multiple ways of getting/using Swift libraries, so if you're interested please let me know whether you would expect to use an Xcode project that builds a framework, a Swift PM package, a CocoaPod or a Carthage…cart… so I can support you using the software in your way.

#### GNU Terry Pratchett
(post-hoc prescript: I admit to being in two minds about sharing this post. Name-dropping can be the ultimate in reflected vanity: I have worth because I knew this worthy person. I title it about them, but we both know it's about me. I hope this post, containing much as it does about me and my experiences, is actually about what I learned from an Old Fart in a Hat - among others.)

Nearly three years ago, The Compleat Ankh-Morpork was approaching completion, and the publishers were excited about the potential to turn it into an interactive discovery of the Circle Sea's cleanest cityest city. At the time that I joined Agant, Dave "OMG it's Dave Addey off the internets" Addey had already been discussing the project with them, and much was ready to go.

Indeed, Dave and I were ready to go. We took a train down to the Discworld Emporium and met Sir Terry Pratchett, his partner in crime Rob Wilkins, their able Aiders and Abettors from the Emporium and some of the most happening among those who Make Things Happen at the publishers.

In the Dolphin Hotel at Wincanton, we had a pub lunch and tried to define exactly what it meant to put Ankh-Morpork into an iPad. I'd already made a start, and here I learned Lesson Two: the creative mind knows it can be done. You don't need to convince them it can be done. It's already been done, it's up there in their mind's eye. You just need to convince them it can be done well, and in this reality.

Wait, lesson two? Yes, for here was Lesson One: when meeting your heroes, only one of the two of you is likely to try and make a big deal out of it. The other wants their ploughman's lunch and their G&amp;T and to get back to work. Keeping focussed on the task at hand - a difficult feat as I was dealing with an internal monologue that would not stop going "squeeeeeee" - paid off, and everyone left safe in the knowledge that there was lots to do but that we would do it because it needed to be done and we needed to be the ones to do it and we needed it to be done well.

OK, most of the rest of this story is history. Dave, the developers at Agant, and a dedicated collection of others worked very hard for a number of months and eventually Discworld: the Ankh-Morpork Map for iPad oozed its way out of the alledged waters of the Ankh (there's no need to disinfect it, the bacteria refuse to touch the stuff) and into the App Store. It remains the project of which I am most proud, both of the contributions I made and of all that was put in by everyone involved.

All good stories, though, come to an end, and Sir Pterry's did last week. Today I went back over to the Emporium to talk a bit about the past and a lot about the future. It's from today's discussions that Lesson Three originates.

Some day, you will be invited by a polite though insistent anthropomorphic personification to take a walk together. Before he arrives, do the things that need to be done. Make sure they're the things that must be done by you, and that you will be satisfied and pleased to see completed. Otherwise the two of you won't have a whole lot to talk about.

#### GNUstep development on LIVEstep
LIVEstep is a GNUstep desktop on a FreeBSD live CD, and it comes with the GNUstep developer tools including ProjectCenter. This video is a "Hello, World" walkthrough using ProjectCenter on LIVEstep. PC is much more influenced by the NeXT Project Builder than by Xcode, so it might look a little weird to younger eyes.

#### GNUstep is more important now than ever
In creating a pull request for GNUstep-base, the Free Software implementation of the Foundation library from Objective-C, I realised that if there was ever a time for GNUstep, now is it.

Although GNUstep may have been envisaged as an official desktop for the GNU system - a role now fulfilled by GNOME - it has always had another position as an alternative deployment venue for OpenStep, and then Cocoa, codebases. People have done this to obtain cross-platform support (I know of a test tool that was built for Mac, Linux and Windows using GNUstep), to take advantage of better, or cheaper, server deployment on Linux, or to act as an 'escape lane', a place to take your code if your platform vendor changes direction.

This ability to hedge against a vendor's whims has come in handy before: in 2001, when WebObjects 5 ditched Objective-C support in favour of Java, Objective-C WebObjects applications could be deployed on modern platforms through GNUstep Web or on legacy WebObjects 4 systems. It may come in handy again.

Even if Apple do, indefinitely, support Objective-C, the plain fact is that their community does not. Conference talks, blog posts, and community discussion now take place using Swift, which for those with an interest in Objective-C or those supporting existing code makes it harder to gain help or even to feel membership.

A vendor-independent association of Objective-C developers all interested in giving their code a comfortable, stable home is now more important than ever. May all your messages have receivers.

#### Goals upon goals upon goals
As I read Ed Finkler's piece on losing excitement in technology, I found myself recognising pieces of my own story. The prospect of a new language or framework no longer seems like a new toy, an excuse to stay up all night studying it, using it and learning its secrets as I would have done a few years ago. Instead I find myself asking what new problems are introduced, whether they're worth accepting over the devils we know are in our existing tools and how many developer-decades are soon to be lost in reimplementing libraries that have been in CPAN for decades in a new language and delivered via a new packaging system.

Because, as I alluded to when not really talking about Swift and as more eloquently described by Matt Gemmell, our problems do not, for the most part, come from our tools and will not be solved by adding more tools. Indeed a developer's fixation on their tools will allow the surrounding business, market, legal and social problems to go unchecked. No change of programming language will turn customers who don't want to pay $0.99 into customers who do want to pay $99. Implicitly unwrapped optionals might catch the occasional bug in development but they just aren't worth a 100x increase in value to people on the sharp end of our creations.

What will be of benefit to them? I think we're going to have to go through the software equivalent of the consolidation that the consumer goods industry has already seen in hardware. Remember the introduction to the iPhone?

  An iPod, a phone, an internet mobile communicator, these are NOT three separate devices!

Actually it's none of those things. Well, it is, in that it's all of those and more. Fundamentally, it's a honking great handheld control unit with hundreds of buttons on, just like Sony used to make for their TV remotes. One of those buttons will turn it into a phone, one will turn it into an iPod, one will turn it into a web browser, another lets you add other buttons that do other things. They don't all do their things in the same way, and just to keep things interesting the buttons will arbitrarily change location and design and behaviour every so often. The Sony remotes didn't do that.

The core experience of an iPhone, or Android phone, or Windows phone—the thing you see when you switch it on and wait long enough—is a launcher. It's the Program Manager from Windows 3.0, packaged up in a shiny interactive box. Both Program Manager and today's replacement offer the same promise: you've got a feeling that you left a button around here somewhere that probably starts you doing the thing you need to do.

So we still need to make good on the promise that you have just one device, by making that one device act like just one device (preferably one which works properly and does what people expect, which is what I spend a lot of time thinking about and working on). How we get there will be the interesting problem for at least another decade. How our tools support that journey will be a fun sideshow, certainly important but hardly the focus. The tools support the goals that support our goals that support the world's goals.

#### Going indie!
This is sort of a message from the past. I wrote it yesterday, but had people I needed to talk to before I could hit the big old publish button. (Including this bit, so I really wrote it "today", but the earliest you can read it means that "today" will be "yesterday". This is one of those uses of the past-perfect-nevertense that blows up lesser recording equipment.)Today (the real today that this thing was posted), I handed in my notice at Sophos. Six weeks from now, I will be officially an unemployed starving artist. I'm working on lining up a project to take up most of my time for the first few months of the new era, which really looks like it will work out, and of course need to solve the "marketing presence" problem. Although the fact that you're reading this probably means you already know who I am, and something about what I do. If you want a Cocoa or UNIX developer for a contract - especially one with experience in the worlds of security and scientific computing - then please see my LinkedIn profile to find out a bit more of what I've been up to, and drop me a line - here, at @iamleeg or to iamleeg at gmail dot com. I know there are a load of interesting people out there working on a load of interesting projects, one of the great things about WWDC every year is meeting you all and sharing in the excitement. Well hopefully I'll get to work with you on some of that cool software, too!So, like many people who go self-employed, I've got little idea of what will happen next :-). I've got some ideas for apps which I'll be working on in the (probably too copious) spare time I'll have. But I'm going to focus on contracting and consulting in the short term. This is going to be an exciting time, if somewhat daunting...but you'll be able to check on my daunt levels right here, dear readers.I promised at the turn of the year that there would be lots of blog posts on various tech things during the first half of the year. Unsurprisingly that didn't quite pan out, and I'm hoping to rectify that over the next couple of months now that I have fewer (perceived) content restrictions on the blog. And this first project I have lined up should certainly be producing some good'uns, assuming it all works out. I'll be the first to admit that if it doesn't, I'm heading for trouble very shortly. Which is why I know that it, or something very like it, will work out :-).To fellow Sophists who are hearing about this for the first time here, I'm very sorry. I tried to let people know today but there are hundreds of you, one of me and lots of loose ends to tie by mid-August. But don't worry, there will be beer.

#### Graham Lee Uses This
I've never been famous enough in tech circles to warrant a post on uses this, but the joy of running your own blog is that you get to indulge any narcissistic tendencies with no filter. So here we go!

The current setup

This is the desktop setup. The idea behind this is that it's a semi-permanent configuration so I can get really comfortable, using the best components I have access to to provide a setup I'll enjoy using. The main features are a Herman Miller Aeron chair, which I got at a steep discount by going for a second-generation fire sale chair, and an M1 Mac Mini coupled to a 24" Samsung curved monitor, a Matias Tactile Pro 4 keyboard and a Logitech G502 mouse. There's a Sandberg USB camera (which isn't great, but works well enough if I use it via OBS's virtual camera) and a Blue Yeti mic too. The headphones are Marshall Major III, and the Philips FX-10 is used as a Bluetooth stereo.

I do all my streaming (both Dos Amigans and [objc retain];) from this desk too, so all the other hardware you see is related to that. There are two Intel NUC devices (though one is mounted behind one of the monitors), one running FreeBSD (for GNUstep) and one Windows 10 (with WinUAE/Amiga Forever). The Ducky Shine 6 keyboard and Glorious Model O mouse are used to drive whichever box I'm streaming from, which connects to the other Samsung monitor via an AVerMedia HDMI capture device.

The laptop setup is on a variable-height desk (Ikea SKARSTA), and this laptop is actually provided by my employer. It's a 12" MacBook Pro (Intel). The idea is that it should be possible to work here, and in fact at the moment I spend most of my work time at it; but it should also be very easy to grab the laptop and take it away. To that end, the stuff plugged into the USB hub is mostly charge cables, and the peripheral hardware is mostly wireless: Apple Magic Mouse and Keyboard, and a Corsair headset. A desk-mounted stand and a music-style stand hold the tablets I need for developing a cross-platform app at work.

And it happens that there's an Amiga CD32 with its own mouse, keyboard, and joypad alongside: that mostly gets used for casual gaming.

The general principle

Believe it or not, the pattern I'm trying to conform to here is "one desktop, one laptop". All those streaming and gaming things are appliances for specific tasks, they aren't a part of my regular computering setup. I've been lucky to be able to keep to the "one desktop, one laptop" pattern since around 2004, usually using a combination of personal and work-supplied equipment, or purchased and handed-down. For example, the 2004-2006 setup was a "rescued from the trash" PowerMac 9600 and a handed-down G3 Wallstreet; both very old computers at that time, but readily affordable to a fresh graduate on an academic support staff salary.

The concept is that the desktop setup should be the one that is most immediate and comfortable, that if I need to spend a few hours computering I will be able to get on very well with. The laptop setup should make it possible to work, and I should be able to easily pick it up and take it with me when I need to do so.

For a long time, this meant something like "I can put my current Xcode project and a conference presentation on a USB stick, copy it to the laptop, then go to a conference to deliver my talk and hack on a project in the hotel room". These days, ubiquitous wi-fi and cloud sync products remove some of the friction, and I can usually rely on my projects being available on the laptop at time of use (or being a small number of steps away).

I've never been a single-platform person. Sometimes "my desktop" is a Linux PC, sometimes a Mac, it's even been a NeXT Turbo Station and a Sun Ultra workstation before. Sometimes "my laptop" is a Linux PC, sometimes a Mac, the most outré was that G3 which ran OpenDarwin for a time. The biggest ramification of that is that I've never got particularly deep into configuring my tools. It's better for me to be able to find my way around a new vanilla system than it is to have a deep custom configuration that I understand really well but is difficult to port.

When Mac OS X had the csh shell as default, I used that. Then with 10.3 I switched to bash. Then with 10.15 I switched to zsh. My dotfiles repo has a git config, and a little .emacs that enables some org-mode plugins. But that's it.

#### Great Documentation, Great Software
A paraphrased conversation, the other day, between me and a customer of one of my customers:

  Me: Are you experienced at working with my customer's developer APIs?
  
  Them: I always feel like a newbie, because there's so much stuff. But I always end up finding the docs I'm looking for.
  
  Me: I'm writing the docs.
  
  Them: Well, thanks! :D

Whether you're writing developer APIs or graphical user interfaces, quality documentation that's easy to find and use when needed is the best way to turn customers from novices who find the complexity offputting, to novices who know they'll be able to tackle whatever's coming their way.

Quality documentation is also useful for improving the quality of the software itself.

Docs-driven development

If you already know about test-driven development, you know that a benefit of TDD as a design tool is that it encourages you to think about your code from the perspective of how it will be used. Rather than implementing an algorithm then exposing an API that you hope will be useful, you design the API that helps solve the problem then implement an algorithm to support the use of that API.

Documentation is another tool for encouraging empathy in design. For every point you have to explain, you get to ask:

"why do I have to explain this?"
"Is there another way to design this such that I don't need to tell people about this detail?"
"Is the thing that I'm telling people how to do, the thing that they would expect to want to do?"

Dev-driven documentation

The questions listed above can be most effectively answered if documentation is part of your iterative cycle of continuous improvement. Documentation can inform design and development, by pointing out cumbersome or difficult parts of the implementation. Development can inform documentation, by showing where the complexity lies and how to deal with it.

Documentation interacts with other activities, too. Test plans should ensure that they cover examples or walkthroughs from the documentation, so that you know the examples you're giving to your customers actually work. Documenters should collaborate with testers to ensure a shared understanding of what the software is aiming to achieve.

Documents like API specifications, user manuals, or walkthrough videos should be versioned and built alongside the corresponding versions of the software.

Working software over comprehensive documentation

Throughout these activities, the point is not to generate documentation for its own sake. One office I worked in had a shelf containing several feet of documentation for a UNIX system that I never opened: the online documentation and a couple of cookbook-style books were sufficient.

The reason for putting effort into your software's documentation is that this effort yields improvements in the software. A more empathetic design, a better-tested implementation, and more confident customers are all steps on the path to higher-quality software, easier and faster. And of course, the Labrary can help you with that.

#### Greetings, Jobs-pickers
It seems to be de rigeur to provide a rundown of the top 10 Leopard features from yesterday's Stevenote, and as I can neither sleep nor be bothered to read sample code, it's a bandwagon I'm happy to jump on, albeit in my own cynical style.  Cue the Fluff Freeman voice effects...but first, let's look at the hardware.  Amazing.  The amount of space they've freed up in the cases is stonking, so both the Xserve and the Mac Pro now actually have Pro-level expandability.

64-bit top to bottom.  What, you mean there are programmers out there too lazy to separate their workhorse from their presentation?  Apart from convenience, and a bigger OS+app footprint, I don't see that we gain much here.  Now we have four Mach-O architectures, and the system libraries have to implement all four.Time Machine.  I'm assuming that the BBC wouldn't let Apple license the Doctor Who theme music.  Because that time vortex looks just like, well, the time vortex, and every time I saw that UI I wanted to sing the theme (sorry, Back To The Future, you lose).  On the other hand, this is a sweet sweet feature and I can't wait for Leopard GM so I can dare to switch it on...Ship the whole package.  I can't remember nor be bothered to look up what Steve said but it was something along the lines of you now get all the apps with the OS.  I for one cannot wait to pay &pound;69 for Leopard and receive my free copy of Shake.Spaces.  About.  Blinking.  Time.Spotlight enhancements.  See my comments on Spaces ;-).  Spotlight in Tiger was just plain broken, hopefully they've sorted this out properly.  After gratuitously faffing about with the Mail UI in Tiger, only to make searching even worse than before, hopefully Maileopard will actually be able to find a mail I write.  Networked spotlight definitely is most welcome.Universal Access.  The new voiceover capability is indeed much better than anything available in a current mainstream OS, sounding less like Steven Hawking (which is I think the same synthesised voice as the Amiga) and more like Davros.  But making this a Top 10 keynote feature?  I smell government contract compliance...Mail features: stationery, notes, to dos.  Oh.  My.  Gods.  Would someone please build a partition wall between Steve's office and the UI team, he keeps suggesting stuff to them.  Why would I want this?  Why would I have an OS on which I can stably run multiple apps, and write myself little notes on Stickies widgets, if I'm then going to fold all of the functionality into the Mail application?  Can't I use iCal or OmniPlan for my ToDos, like they're designed for?  And stationery; I hate to shout, but HTML MAIL MUST BE BANNINATED FROM THE INTARWEBS.Core Animation.  As my good bud Ken pointed out, this doesn't seem to be anything you couldn't previously do with Tiger classes such as NSAnimation or NSViewAnimation, but maybe it's a snatch easier.  The thing is, judging by the reactions of the Americans in the crowd, I've got a feeling this is going to be the &quot;Web 2.0&quot; of the Mac UI.  I feel violated.Dashboard features: dashcode, web clip.  Oh great, now we get to see thousands upon thousands of derivative widgets.  And web clip could've been so much cooler; it looks to me like it's &quot;render this whole web page, and display a punch-out with the following CGRect&quot;.  What it should be is &quot;load this web page, and render this section of the DOM&quot;.iChat++.  I don't really like tabs in Mac apps (AppKit does multiple windows per document, not multiple documents per window, as I said to John Gruber in response to his predictions, and the HIG deprecates trying) but I'm glad they went with OmniWeb-style tabs instead of Safari-style tabs.  The ability to stream an app's view over iChat is absolutely fantastic though, this really is enterprise-level IM (regardless of the Photo Booth-style effects).  I'm looking forward to taking a photo of my office to use as my iChat backdrop wherever I am, too :-)

So there we go.  Sorry to sound incredibly underwhelmed, but there it is.  There's been plenty of juice in the WWDC but not much of it came from the Stevenote.

#### Grooming the Backfog
This is "Pub Walks in Warwickshire". NEW EDITION, it tells me! This particular EDITION was actually NEW back in 2008. It's no longer in print.

Each chapter is a separate short walk, starting and finishing at a pub with a map and instructions to find your way around the walk. Some of the instructions are broken: a farmer has put a barbed wire fence across a field, or a gate has been replaced or removed. You find when you get there that it's impossible to follow the instructions, and you have to invent a new route to get back on track. You did bring a different map, didn't you? If not, you'll be relying on good old-fashioned trial and error.

Other problems are more catastrophic. The Crown at Napton-on-the-hill seems to have closed in about 2013, so an attempt to do a circular walk ending with a pint there is going to run into significant difficulties, and come to an unsatisfactory conclusion. The world has moved on, and those directions are no longer relevant. You might want to start/end at the Folly, but you'll have to make up a route that joins to the bits described here.

This morning, a friend told me of a team that he'd heard of who were pulling 25 people in to a three-hour backlog grooming session. That sounds like they're going to write the NEW EDITION of "Pub Walks in Warwickshire" for their software, and that by the time they come around to walking the route they'll find some of the paths are fenced over and the pubs closed.

Decomposing the Analogy

A lengthy, detailed backlog is not any different from having a complete project plan in advance of starting work, and comes with the same problems. Just like the pub walks book, you may find that some details need to change when you get to tackling them, therefore there was no value in spending the time constructing all of those details in the first place. These sorts of changes happen when assumptions about the organisation or architecture of the system are invalidated. Yes, you want this feature, but you can no longer put it in the Accounts module because you found that customers think about that when they're sorting their bills, not their accounts. Or you need to put more effort into handling input from an external data source, because the way it really works isn't quite the same as the documentation.

Or you find that a part of the landscape is no longer present and there's no value in being over there. This happens when the introduction of your system, or a competitors', means that people no longer worry about the problem they had back at the start. Or when changes in what people are trying to do mean they no longer want or need to solve that problem at all.

A book of maps and directions is a snapshot in time of ways to navigate the landscape. If it takes long enough to follow all of the directions, you will find that the details on the ground no longer match the approximation provided by the book.

A backlog of product features and stories is a snapshot in time of ways to develop the product. If it takes long enough to implement all of the features, you will find that the details in the environment no longer match the approximation provided by the backlog.

A Feeling of Confidence

We need to accept that people are probably producing this hefty backlog because they feel good about doing it, and replace it with something else to feel good about. Otherwise, we're just making people feel bad about what they're doing, or making them feel bad by no longer doing it.

What people seem to get from detailed plans is confidence. If what they're confident in is "the process as documented says I need a backlog, and I feel confident that I have done that" then there's not much we can do other than try to change the process documentation. But reality probably isn't that facile. The confidence comes from knowing where they're trying to go, and having a plan to get there.

We can substitute that confidence with frequent feedback: confidence that the direction they're going in now is the best one given current knowledge, and that it's really easy to get updates and course corrections. Replace the confidence of a detailed map with the confidence of live navigation.

On the Backfog

A software team should still have an idea of where it's going. It helps to situate today's development in the context of where we think (but do not know) we will be soon, to organise the system into a logical architecture, to see which bits of flexibility Ya [Probably] Ain't Gonna Need and which bits Ya [Probably] Are. It also helps to have the discussion with people who might buy our stuff, because we can say "we think we're going to do these things in the coming months" and they can say "I will give you a wheelbarrow full of money if you do this one first" or "actually I don't need that thing so I hope it doesn't get in my way".

But we don't need to know the detailed steps and directions to get there, because building those details now will be wasted effort if things change by the time we are ready to tackle all of the pieces. Those discussions we're having with the people who might buy our stuff? They might, and indeed probably should, change that high-level direction.

Think of it like trying to navigate an unknown landscape in fog. You know that where you're trying to get to is over there somewhere, but you can't clearly see the whole path from here. You probably wouldn't just take a compass bearing and head toward the destination. You'd look at what you can see around, and what paths there are. You'd check a map, sure, but you'd probably compare it with what you can see. You'd phone ahead to the destination, and check that they expect to be open when you expect to get there. You'd find out if there are any fruitful places to stop along the way.

So yes, share the high-level direction, it's helpful. But share the uncertainty too. The thing we're doing next should definitely be known, the thing we're doing later should definitely be guesswork. Get confidence not from colouring in the plan all the way up to the edges, but by knowing how ready and able you are to update the plan.

#### grotag
Lots of Amiga documentation was in the AmigaGuide format. These are simple ASCII documents with some rudimentary markup to turn them into hypertext, working something like TeXInfo manuals. Think more like a markdown-enabled Gopher than the web though: you can link out to an image, video, or any other media (you could once you had AmigaOS 3, anyway) but you can't display it inline.

Unfortunately choices for modern readers are somewhat limited. Many links are only now found on the Internet Archive, and many of those don't go to downloads you can actually download. I found a link to an old grotag binary, but it was PowerPC-only.

&hellip;and it was on Sourceforge, so I cloned the project and updated the build. I haven't created a new package yet, but it runs well enough out of Idea. I need to work out how you package Java Swing apps, then do that. It'll be worth fixing a couple of deprecations, putting assets like the CSS file in the JAR, and maybe learning enough JavaFX to port the UI:

To use it, alongside your Guide file you also need a grotag.xml that maps Amiga volume links onto local filesystem paths, so that grotag can find nodes linked to other files. There's an example of one in the git repo.

#### Half a bee
When you're writing Python tutorials, you have to use Monty Python references. It's the law. On the 40th anniversary of the release of Monty Python's Life of Brian, I wanted to share this example that I made for collections.defaultdict that doesn't fit in the tutorial I'm writing. It comes as homage to the single Eric the Half a Bee.

from collections import defaultdict

class HalfABee:
    def __init__(self):
        self.is_a_bee = False
    def value(self):
        self.is_a_bee = not self.is_a_bee
        return "Is a bee" if self.is_a_bee else "Not a bee"

&gt;&gt;&gt; eric = defaultdict(HalfABee().value, {})
&gt;&gt;&gt; print(eric['La di dee'])
Is a bee
&gt;&gt;&gt; print(eric['La di dee'])
Not a bee

Dictionaries that can return different values for the same key are a fine example of Job Security-Driven Development.

#### Halloween is Over
Back in 2016, I sent the following letter to Linux Voice, and it was published in issue 24 as the star letter. LV came to an end (and made all of their content available as Creative Commons) when they merged with Linux Magazine. The domain still exists, but the certificate expired years ago; you should search for it if you're interested in back numbers for the magazine and willing to take the risk on their SSL.

I think my letter is still relevant, so I'm reproducing it. Here's what I wrote:

LV issue 023 contained, as have prior numbers, many jabs at Microsoft as the natural enemy of the Free Software believer. It's time to accept that the world has changed.Like many among your staff and readers, I remember that period when the infamous Halloween memos were leaked, and we realised joyfully that the Free Software movement was big enough to concern the biggest software company in the world.I remember this not because it was recent, but because I am old: this happened in 1998. Large companies like Microsoft can be slow to change, so it is right that we remain sceptical of their intentions with Free and open source software, but we need to remember that if we define our movement as Anti-Microsoft, it will live or die by their fortunes alone.While we jab at Azure for their plush Tux swag, Apple has become one of the largest companies on the planet. It has done this with its proprietary iPhone and iOS platforms, which lock in more first-party applications than 1990s Windows did when the antitrust cases started flying. You can download alternatives from its store (and its store alone), but the terms of business on that store prohibit copyleft software. The downloads obtained by Apple's users are restricted by DRM to particular Apple accounts.Meanwhile, Apple co-opts open source projects like Clang and LLVM to replace successful Free Software components like GCC. How does the availability of a cuddly Tux with Microsoft branding stack up to these actions in respect to the FSF's four freedoms?We celebrate Google for popularising the Linux kernel through its Android mobile OS, and companies like it, including Facebook and Twitter, for their contributions to open source software. However, these companies thrive by providing proprietary services from their own server farms. None has embraced the AGPL, a licence that extends freedom to remote users of a hosted service. Is it meaningful to have the freedom to use a browser or a mobile device for any purpose, if the available purposes involve using non-free services?So yes, Microsoft is still important, and its proprietary Windows and Office products are still huge obstacles to the freedom of computer users everywhere. On the other hand, Microsoft is no longer the headline company defining the computing landscape for many people. If the Free Software movement is the "say no to Microsoft" movement, then we will not win. Rather we will become irrelevant at the same time as our nemesis in Redmond.You may think that Steve Jobs is an unlikely role model for someone in my position, but I will end by paraphrasing his statement on his return to Apple. We need to get out of the mindset that for the Four Freedoms to win, Microsoft has to lose.
Graham Lee

Their deputy editor responded.

I had never stopped to consider this, but what you say makes 100% sense. In practice though, for most people Microsoft is still the embodiment of proprietary software. Apple is arguably a more serious threat, but Microsoft keeps shooting itself in the foot, so it's an easier target for us. Apple at least makes a lot of good products along with its egregious attitudes towards compatibility, planned obsolescence and forced upgrades; Microsoft seems to be successful only by abusing its market position.
Andrew Gregory

Things have changed a bit since then: Apple have made minimal efforts to permit alternative apps in certain categories; Microsoft have embraced and extended more open source technologies; various SaaS companies have piled in on the “open source but only when it works in our favour” bandwagon; Facebook renamed and is less likely to be praised now than it was in 2016.

But also things have stayed the same. As my friend and stream co-host Steven Baker put it, there's a reason there isn't an M in FAANG. Microsoft isn't where the investors are interested any more, and they shouldn't be where Free Software's deciding battles are conducted.

If you like my writing on software engineering please subscribe to my fortnightly newsletter where I aggregate it from across the web, as well as sharing the things I've been reading about software engineering!

#### Happy 19th birthday, Cocoa!
On October 19th, 1994 NeXT Computer, Inc. (later NeXT Software, Inc.) published a specification for OpenStep, a cross-platform interface for application programming, based on their existing Objective-C frameworks and the Display PostScript graphics system.

A little bit of history

First there came message-passing object oriented programming, in the form of Smalltalk. Well, not first, I mean first there was Simula 67, and there were even things before that but every story has to start somewhere. In 1983 Brad Cox added Smalltalk messaging to the C language to create the Object-Oriented pre-compiler. In his work with Tom Love at Productivity Products International, this eventually became Objective-C.

If PPI (later Stepstone) had any larger customers than NeXT, they had none that would have a bigger impact on the software industry. In 1988 NeXT released the first version of their UNIX platform, NEXTSTEP. Its application programming interface combined the "application kit" of Objective-C objects representing windows, menus, and views with Adobe's Display PostScript to provide a high-fidelity (I mean, if you like grey, I suppose) WYSIWYG app environment.

N.B.: my reason for picking the Garfinkel and Mahoney book will become clear later. It happens to be the book I learned to make apps from, too.

Certain limitations in the NEXTSTEP APIs became clear. I will not exhaustively list them nor attempt to put them into any sort of priority, suffice it to say that significant changes became necessary. When the Enterprise Objects Framework came along, NeXT also introduced the Foundation Kit, a "small set of base utility classes" designed to promote common conventions, portability and enhanced localisation through Unicode support. Hitherto, applications had used C strings and arrays.

It was time to let app developers make use of the Foundation Kit. For this (and undoubtedly other reasons), the application kit was rereleased as the App Kit, documented in the specification we see above.

The release of OpenStep

OpenStep was not merely an excuse to do application kit properly, it was also part of NeXT's new strategy to license its software and tools to other platform vendors rather than limiting it to the few tens of thousands of its own customers. Based on the portable Foundation Kit, NeXT made OpenStep for its own platform (now called OPENSTEP) and for Windows NT, under the name OpenStep Enterprise. Sun Microsystems licensed it for SPARC Solaris, too.

What happened, um, NeXT

The first thing to notice about the next release of OpenStep is that book cover designers seem to have discovered acid circa 1997.

Everyone's probably aware of NeXT's inverse takeover of Apple at the end of 1996. The first version of OpenStep to be released by Apple was Rhapsody, a developer preview of their next-generation operating system. This was eventually turned into a product: Mac OS X Server 1.0. Apple actually also released another OpenStep product: a y2k-compliant patch to NeXT's platform in late 1999.

It's kindof tempting to tell the rest of the story as if the end was clear, but at the time it really wasn't. With Rhapsody itself it wasn't clear whether Apple would promote Objective-C for OpenStep (now called "Yellow Box") applications, or whether they would favour Yellow Box for Java. The "Blue Box" environment for running Mac apps was just a virtual machine with an older version of the Macintosh system installed, there wasn't a way to port Mac software natively to Rhapsody. It wasn't even clear whether (or if so, when) the OpenStep software would become a consumer platform, or whether it was destined to be a server for traditional Mac workgroups.

That would come later, with Mac OS X, when the Carbon API was introduced. Between Rhapsody and Mac OS X, Apple introduced this transition framework so that "Classic" software could be ported to the new platform. They also dropped one third of the OpenStep-specified libraries from the system, as Adobe's Display PostScript was replaced with Quartz and Core Graphics. Again, reasons are many and complicated, though I'm sure someone noticed that if they released Mac OS X with the DPS software then their bill for Adobe licences would increase by a factor of about 1,000. The coloured box naming scheme was dropped as Apple re-used the name of their stagecast creator software: Cocoa.

So it pretty much seemed at the time like Apple were happy to release everything they had: UNIX, Classic Mac, Carbon, Cocoa-ObjC and Cocoa-Java. Throw all of that at the wall and some of it was bound to stick.

Over time, some parts indeed stuck while others slid off to make some sort of yucky mess at the bottom of the wall (you know, it is possible to take an analogy too far). Casualties included Cocoa-Java, the Classic runtime and the Carbon APIs. We end in a situation where the current Mac platform (and, by slight extension, iOS) is a direct, and very close, descendent of the OpenStep platform specified on this day in 1994.

Happy birthday, Cocoa!

#### Happy 25th birthday to the manifesto for agile software development!
11th-13th February 2001 is the occasion of the most famous skiing holiday in software. Don't take my word for it; Jim Highsmith was there and wrote the history.

It's pretty astounding that, in a field where everyone tries to remind each other that things move at breakneck pace (though that speed is mostly reserved for those reminders), a website with four substantive text-only pages is still relevant and still widely cited. I'm never going to create as comprehensive or as balanced a critique as Bertrand Meyer, but there are still various important points about the manifesto that are worth discussing.

Two minor wording gripes

Only one of the twelve principles behind the manifesto says anything quantitative, and that's the only principle that's been lost to time.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Through “doing it and helping others do it”, software practitioners have discovered much faster ways to deliver working software more frequently. It's not unrealistic for a web-based application to be deployable in seconds, and for total delivery workflows including verification and validation to take minutes. Improving our capabilities is no bad thing, but putting a lower bound on delivery frequency lets people whose organisational restrictions limit releases to every two weeks blame “Agile” for that, and choose not to learn anything else from the collection of approaches to making software. If we must blame something, let's blame Dark Scrum.

The other place where my red pencil comes out is in the attempt to bring people together that actually separates them, the division of people into “developers” and “business” (or, phrased another way, non-developers):

Business people and developers must work together daily throughout the project.

The authors could write “project collaborators must work together daily”, or something like that, and we wouldn't have had pigs and chickens. We wouldn't have “technical” and “non-technical” people.

Potentially, we wouldn't have had DevOps either, because it wouldn't have been necessary: project collaborators work together daily. Operation people are neither business people (depending on your line of business) nor developers, so they got excluded until somebody noticed.

Your problem is probably management

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

If the project collaborators have to run their ideas past people who aren't on the project, or do things in the same way that other people do them on other projects, they haven't got the environment or support they need for this project.

The journey isn't over

We are uncovering better ways of developing software by doing it and helping others do it.

This is something that's still happening, not something that was over when a group of professionals wrote a short document 25 years ago. We are uncovering better ways. This continues.

In some ways it's surprising that no newer paradigms have come along to replace agile software development. On deeper reflection we find that anything new would be compatible with this approach, unless you give up on prioritising “satisfying the customer through early and continuous delivery of valuable software”.

In fact, in a world where the software represents autonomous agents rather than tools that customers use, maybe it could soon be time to prioritise something other than delivering software. For the moment, we're still in a world where the agents are embodied in software tools, and we have to deliver those to our customers, and doing that in a way that's early, continuous, and valuable still seems to make sense.

It may seem weird coming from someone who's hitched their whole cart to the generative AI centaur, but while the tools and processes that genAI enables are new and exciting, and I think they're going to prove valuable, I don't think they're going to be more valuable than individuals and interactions. That hasn't changed in more than 25 years, and doesn't need to change soon.

#### Happy Birthday, Objective-C!
OK, I have to admit that I actually missed the party. Brad Cox first described his "Object-Oriented pre-compiler", OOPC, in The January 1983 issue of ACM SIGPLAN Notices.

This describes the Object Oriented Pre-Compiler, OOPC, a language and a run-time library for producing C programs that operate by the run-time conventions of Smalltalk 80 in a UNIX environment. These languages offer Object Oriented Programming in which data, and the programs which may access it, are designed, built and maintained as inseparable units called objects.

Notice that the abstract has to explain what OOP is: these were early days at least as far as the commercial software industry viewed objects. Reading the OOPC paper, you can tell that this is the start of what became known as Objective-C. It has a special syntax for sending Smalltalk-style messages to objects identified by pointers to structures, though not the syntax you'll be used to:

someObject = {|Object, "new"|};
{|myArray, "addObject:", someObject|};

The infix notation [myArray addObject:someObject]; came later, but by 1986 Cox had published the first edition of Object-Oriented Programming: An Evolutionary Approach and co-founded Productivity Products International (later Stepstone) to capitalise on the Objective-C language. I've talked about the version of ObjC described in this book in this post, and the business context of this in Software ICs and a component marketplace.

It's this version of Objective-C, not OOPC, that NeXT licensed from PPI as the basis of the Nextstep API (as distinct from the NEXTSTEP operating system: UNIX is case sensitive, you know). They built the language into a fork of the GNU Compiler Collection, and due to the nature of copyleft this meant they had to make their adaptations available, so GCC on other platforms gained Objective-C too.

Along the way, NeXT added some features to the language: compiler-generated static instances of string classes, for example. They added protocols: I recorded an episode of NSBrief with Saul Mora discussing how protocols were originally used to support distributed objects, but became important design tools. This transformation was particularly accelerated by Java's adoption of protocols as interfaces. At some (as far as I can tell, not well documented) point in its life, Stepstone sold the rights to ObjC to NeXT, then licensed it back so they could continue supporting their own compiler.

There isn't a great deal of change to Objective-C from 1994 for about a decade, despite or perhaps due to the change of stewardship in 1996/1997 as NeXT was purchased by Apple.  Then, in about 2003, Apple introduced language-level support for exceptions and critical sections. In 2007, "Objective-C 2.0" was released, adding a collection enumeration syntax, properties, garbage collection and some changes to the runtime library. Blocks—a system for supporting closures that had been present in Smalltalk but missing from Objective-C—were added in a later release that briefly enjoyed the name "Objective-C 2.1", though I don't think that survived into the public release. To my knowledge 2.0 is the only version designation any Apple release of Objective-C has had.

Eventually, Apple observed that the autozone garbage collector was inappropriate for the kind of software they wanted Objective-C programmers to be making, and incorporated reference-counted memory management from their (NeXT's, initially) object libraries into the language to enable Automatic Reference Counting.

And that's where we are now! But what about Dr. Cox? Stepstone's business was not the Objective-C language itself, but software components, including ICPak101, ICPak201 and the TaskMaster environment for building applications out of objects. It turned out that the way they wanted to sell object frameworks (viz. in a profitable way) was not the way people wanted to buy object frameworks (viz. not at all). Cox turned his attention to Digital Rights Management, and warming up the marketplace to accept pay-per-use licensing of digital artefacts. He's since worked on teaching object-oriented programming, enterprise architecture and other things; his blog is still active.

So, Objective-C, I belatedly raise my glass to you. You're nearly as old as I am, and that's never likely to change. But we've both grown over that time, and it's been fun growing up with you.

#### HATEOAS app structure explained through some flimsy analogy
You are in a tall, narrow view. A vibrant, neon sign overhead tells you that this is the entrance to "Stocks" - below it is one of those scrolling news tickers you might see on Times Square. In front of you lies a panel of buttons. Before you can do anything, an automatic droid introduces itself as "Searchfield". It invites you to tell it the name of a stock you'd like to know the price of, and tells you that it will then GET the price.

You can see:

Searchfield&lt;br/>
A button marked "Price"&lt;br/>
A button marked "Market Cap"&lt;br/>
A box marked "PUT trades here"&lt;br/>

There is an exit going Back. Additionally you can go Home.

#### Having the right data
In the beginning there was the relational database, and it was...OK, I guess. It was based on the relational model, and allowed operations that were within the relational algebra.

I mean it actually didn't. The usual standard for relational databases is ISO 9075, or SQL. It doesn't really implement the relational model, but something very similar to it. Still, there is a standard way for dealing with relational data, using a standard syntax to construct queries and statements that are mathematically provable.

I mean there actually isn't. None of the "SQL databases" you can get hold of actually implement the SQL standard accurately or in its entirety. But it's close enough.

At some point people realised that you couldn't wake up the morning of your TechCrunch demo and code up your seed-round-winning prototype before your company logo hit the big screen, because it involved designing your data model. So the schemaless database became popular. These let you iterate quickly by storing any data of any shape in the database. If you realise you're missing a field, you add the field. If you realise you need the data to be in a different form, you change its form. No pesky schemata to migrate, no validation.

I mean actually there is. It's just that the schema and the validation are the responsibility of the application code: if you add a field, you need to know what to do when you see records without the field (equivalent to the field being null in a relational database). If you realise the data need to be in a different form, you need to validate whether the data are in that form and migrate the old data. And because everyone needs to do that and the database doesn't offer those facilities, you end up with lots of wasteful, repeated, buggy code that sort of does it.

So the pendulum swings back, and we look for ways to get all of that safety back in an automatic way. Enter JSON schema. Here's a sample of the schema (not the complete thing) for Covid-19 cases in Global.health:

{
  bsonType: 'object',
  additionalProperties: false,
  properties: {
    location: {
      bsonType: 'object',
      additionalProperties: false,
      properties: {
        country: { bsonType: 'string', maxLength: 2, minLength: 2 },
        administrativeAreaLevel1: { bsonType: 'string' },
        administrativeAreaLevel2: { bsonType: 'string' },
        administrativeAreaLevel3: { bsonType: 'string' },
        place: { bsonType: 'string' },
        name: { bsonType: 'string' },
        geoResolution: { bsonType: 'string' },
        query: { bsonType: 'string' },
        geometry: {
          bsonType: 'object',
          additionalProperties: false,
          required: &#91; 'latitude', 'longitude' ],
          properties: {
            latitude: { bsonType: 'number', minimum: -90, maximum: 90 },
            longitude: { bsonType: 'number', minimum: -180, maximum: 180 }
          }
        }
      }
    }
  }
}

This is just the bit that describes geographic locations, relevant to the falsehoods we believed about countries in an earlier post. This schema is stored as a validator in the database (you know, the database that's easier to work with because it doesn't have validators). But you can also validate objects in the application if you want. (Actually we currently have two shadow schemas: a Mongoose document description and an OpenAPI specification, in the application. It would be a good idea to normalise those: pull requests welcome!)

#### Head of Architecture
My current job title is Head of Architecture, though the word "architecture" means different things to different people in the world of software. So what does it mean to me, what do I do when I'm playing Head of Architecture?

I follow Perry and Wolf in Foundations for the Study of Software Architecture by drawing the analogy between software architecture and built environment architecture, not network or electronics architecture. The work of a built environment architect, particularly one who follows the path laid out by Christopher Alexander, combines elements, form and aesthetics by creating a system that complements and enhances its environment. Software systems are deployed into environments (with existing people, processes, cultural norms) and developed in environments, and should make those environments better for the people who are interacting with them, while also meeting the functionality, performance, security and other goals of the system.

But that doesn't actually explain what I do, which is more about letting other people do things that are "architecture" than about "doing architecture" for them. Programmers, ops folks, QA people, product owners, and others frequently make decisions that have wide (and hence "architectural") impact, and I think it's better to enable that and follow up by asking how it impacts the rest of the system, to refine the choices made, than to stop people from making those decisions for them in the name of "being the architect".

So playing software architect for me tends to be more about creating a forum in which people can present aspects of the problems they're trying to solve, needing to solve soon, or the solutions they're exploring, and getting a view from multiple teams and multiple functions about those problems and solutions. Making sure that ops know what devs are doing, that product team Alpha knows how product team Aleph are solving that issue, and so on.

How about the stereotype that software architects program using Visio or Powerpoint? In my case, I program using JavaScript. I do make documentation, to make sure that decisions made in the forum are captured, that proposed or current approaches can be seen and understood. And yes, much of that documentation is diagrammatic. But ultimately I'm a programmer on a software team too, and that documentation has to reflect working, valuable software. That is, while there is value in comprehensive documentation, we value working software more.

#### Hell hath frozen over
Hell, or "vi vi vi: the editor of the beast" as Matt succinctly put it.  Anyway, I just wrote a Makefile with multiple dependent targets, and it worked first time.  That just shouldn't happen - especially when you only ever used Project Builder to avoid having to write Makefiles.  I didn't even have to look at the O'Reilly/Cygnus book much.

#### Hi, I'm Graham and I like things
In my time in special-interest forums, I've come to learn that a "fan" of something is someone who doesn't like it very much. This seems to crop up frequently in relation to long-running science fiction entertainment franchises, leading me to the theory that a "fan" is someone who enjoyed a kid's show as a kid and is angry at the creators because they don't enjoy a kid's show as an adult.

To be a Doctor Who "fan", you apparently have to know all the ways in which the people who make Doctor Who don't understand what Doctor Who is and shouldn't be allowed near it. For example, the Timeless Child arc—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation—is absolute piffle to "fans", who lament the show's cancellation before the realization of the Cartmel Master Plan—in which the Doctor is revealed to have powers beyond regular Time Lords and to be significant in their creation.

To be a Star Wars "fan", you apparently have to know that the 1977-1983 trilogy of films (the retroactively-subtitled Episode IV through Episode VI) are the only valid Star Wars, and that everything else is Not Star Wars. Particularly 1999-2005 trilogy, who were made by someone who doesn't truly understand Star Wars; a certain *checks notes* George Lucas.

To be a Star Trek "fan", you apparently have to know what's wrong with every episode, why they should never have made any series after (or including) Enterprise, but also why there should always be new Star Trek on the TV.

I also see this with music, where "fans" explain who the worst bands in a genre are; what the worst albums, and tracks, by bands are; why the top ten most-streamed songs are bad; and why the tools that musicians use to make music make musicians make bad music.

Hi, I'm Graham. I like long-running science fiction entertainment franchises, I like music, I like books, I like many things.

#### Hidden gem
Having been using fink for at least three years, I'm surprised it's taken me this long to discover the 'applesystemfonts' package, which lets you use the, um, Apple system fonts under X.  But I'm glad I did, as it's no longer "Times New Roman or nowt" in Scribus :-)

#### Hiding behind messages
A problem I think about every so often is how to combine the software design practice of hiding implementations behind interfaces with the engineering practice of parallel execution. What are the trade-offs between making parallelism explicit and information hiding? Where are some lines that can be drawn?

Why do we need abstractions for this stuff, when the libraries we have already work? Those libraries make atomic features like threads and locks explicit, or they make change of control flow explicit, and those are things we can manage in one place for the benefit of the rest of our application. Nobody likes to look at the dispatch terrace:

    });
  });
});

Previous solutions have involved object confinement, where every object has its own context and runs its code there, and the command bus, where you ask for work to be done but don't get to choose where.

Today's solution is a bit more explicit, but not much. For every synchronous method, create an asynchronous version:

@interface MyObject : Awaitable

- (NSString *)expensiveCalculation;
- async_expensiveCalculation;

@end

@implementation MyObject

- (NSString *)expensiveCalculation
{
  sleep(5);
  return @"result!";
}

@end

int main(int argc, const char * argv[]) {
  @autoreleasepool {
    MyObject *o = [MyObject new];
    id asyncResult = [o async_expensiveCalculation];
    NSLog(@"working...");
    // you could explicitly wait for the calculation...
    NSLog(@"Result initial: %@", [[asyncResult await]   substringToIndex:1]);
    // ...but why bother?
    NSLog(@"Shouty result: %@", [asyncResult uppercaseString]);
  }
  return 0;
}

This is more of a fork-and-join approach than fire and forget. The calling thread carries on running until it actually needs the result of the calculation, at which point it waits (if necessary) for the callee to complete before continuing. It'll be familiar to programmers on other platforms as async/await.

The implementation is - blah blah awesome power of the runtime - a forwardInvocation: method that looks for messages with the async marker, patches their selector and creates a proxy object to invoke them in the background. That proxy is then written into the original invocation as its return value. Not shown: a pretty straightforward category implementing -[NSInvocation copyWithZone:].

@implementation Awaitable

- (SEL)suppliedSelectorForMissingSelector:(SEL)aSelector
{
  NSString *selectorName = NSStringFromSelector(aSelector);
  NSString *realSelectorName = nil;
  if ([selectorName hasPrefix:@"async_"]) {
    realSelectorName = [selectorName substringFromIndex:6];
  }
  return NSSelectorFromString(realSelectorName);
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
  NSMethodSignature *aSignature =   [super methodSignatureForSelector:aSelector];
  if (aSignature == nil) {
    aSignature =    [super methodSignatureForSelector:[self suppliedSelectorForMissingSelector:aSelector]];
  }
  return aSignature;
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
  SEL trueSelector =    [self suppliedSelectorForMissingSelector:[anInvocation selector]];
  NSInvocation *cachedInvocation = [anInvocation copy];
  [cachedInvocation setSelector:trueSelector];
  CBox *box = [CBox cBoxWithInvocation:cachedInvocation];
  [anInvocation setReturnValue:&amp;box];
}

@end

Why is the proxy object called CBox? No better reason than that I built this while reading a reflection on Concurrent Smalltalk where that's the name of this object too.

@interface CBox : NSProxy

+ (instancetype)cBoxWithInvocation:(NSInvocation *)inv;
- await;

@end

@implementation CBox
{
  NSInvocation *invocation;
  NSOperationQueue *queue;
}

+ (instancetype)cBoxWithInvocation:(NSInvocation *)inv
{
  CBox *box = [[self alloc] init];
  box-&gt;invocation = [inv retain];
  box-&gt;queue = [NSOperationQueue new];
  NSInvocationOperation *op = [[NSInvocationOperation alloc]    initWithInvocation:inv];
  [box-&gt;queue addOperation:op];
  return [box autorelease];
}

- init
{
  return self;
}

- await
{
  [queue waitUntilAllOperationsAreFinished];
  id returnValue;
  [invocation getReturnValue:&amp;returnValue];
  return [[returnValue retain] autorelease];
}

- (void)dealloc
{
  [queue release];
  [invocation release];
  [super dealloc];
}

- forwardingTargetForSelector:(SEL)aSelector
{
  return [self await];
}

@end

You don't always need some huge library to clean things up. Here are about 70 lines of Objective-C that abstract an implementation of concurrent programming and stop my application code from having to interweave the distinct responsibilities of what it's trying to do and how it's trying to do it.

#### Holding a problem in your head
The linkied article (via Daring Fireball) describes the way that many programmers work - by loading the problem into their head - and techniques designed to manage and support working in such a way.  Paul Graham makes the comparison with solving mathematics problems, which is something I can (and obviously have decided to) talk about due to the physics I studied and taught.  Then I also have things to say about his specific recommendations.

In my (limited) experience, both the physicist in me and the programmer in me like to have a scratch model of the problem available to refer to.  Constructing such a model in both cases acts as the aide memoire to bootstrap the prolem domain into my head, and as such should be as quick, and as nasty, as possible.  Agile Design has a concept known as &quot;just barely good enough&quot;, and it definitely applies at this stage.  With this model in place I now have a structured layout in my head, which will aid the implementation, but I also have it scrawled out somewhere that I can refer to if in working on one component (writing one class, or solving one integral) I forget a detail about another.

Eventually it might be necessary to have a 'posh' layout of the domain model, but this is not yet the time.  In maths as in computing, the solution to the problem actually contains the structure you came up with, so if someone wants to see just the structure (of which more in a later paragraph) it can be extracted easily.  The above statement codifies why in both cases I (and most of the people I've worked with, in each field) prefer to use a whiteboard than a software tool for this bootstrap phase.  It's impossible - repeat, impossible - to braindump as quickly onto a keyboard, mouse or even one of those funky tablet stylus things as it is with an instantly-erasable pen on a whiteboard.  Actually, in the maths or physics realm, there's nothing really suitable anyway.  Tools like Maple or Mathematica are designed to let you get the solution to a problem, and really only fit into the workflow once you've already defined the problem - there's no adequate way to have large chunks of &quot;magic happens here, to be decided at a later date&quot;.  In the software world, CASE tools cause you to spend so long thinking about use-cases, CRC definitions or whatever that you actually have to delve into nitty-gritty details while doing the design; great for software architects, bad for the problem bootstrap process.  Even something like Omnigraffle can be overkill. it's very quick but I generally only switch to it if I think my boardwriting has become illegible.  To give an example, I once 'designed' a tool I needed at Oxford Uni with boxes-and-clouds-and-lines on my whiteboard, then took a photo of the whiteboard which I set as the desktop image on my computer.  If I got lost, then I was only one keystroke away from hiding Xcode and being able to see my scrawls.  The tool in question was a WebObjects app, but I didn't even open EOModeler until after I'd taken the photo.

Incidentally, I expect that the widespread use of this technique contributes to &quot;mythical man-month&quot; problems in larger software projects.  A single person can get to work really quickly with a mental bootstrap, but then any bad decisions made in planning the approach to the solution are unlikely to be adequately questioned during implementation.  A small team is good because with even one other person present, I discuss things; even if the other person isn't giving feedback (because I'm too busy mouthing off, often) I find myself thinking &quot;just why am I trying to justify this heap of crap?&quot; and choosing another approach.  Add a few more people, and actually the domain model does need to be well-designed (although hopefully they're then all given different tasks to work on, and those sub-problems can be mentally bootstrapped).  This is where I disagree with Paul - in recommendation 6, he says that the smaller the team the better, and that a team of one is best.  I think a team of one is less likely to have internal conflicts of the constructive kind, or even think past the first solution they get concensus on (which is of course the first solution any member thinks of).  I believe that two is the workable minimum team size, and that larger teams should really be working as permutations (not combinations) of teams of two.

Paul's suggestion number 4 to rewrite often is almost directly from the gospel according to XP, except that in the XP world the recommendation is to identify things which can be refactored as early as possible, and then refactor them.  Rewriting for the hell of it is not good from the pointy-haired perspective because it means time spent with no observable value - unless the rewrite is because the original was buggy, and the rewritten version is somehow better.  It's bad for the coder because it takes focus away from solving the problem and onto trying to mentally map the entire project (or at least, all of it which depends on the rewritten code); once there's already implementation in place it's much harder to mentally bootstrap the problem, because the subconscious automatically pulls in all those things about APIs and design patterns that I was thinking about while writing the inital solution.  It's also harder to separate the solution from the problem, once there already exists the solution.

The final sentence of the above paragraph leads nicely into discussion of suggestion 5, writing re-readable code.  I'm a big fan of comment documentation like headerdoc or doxygen because not only does it impose readability on the code (albeit out-of-band readability), but also because if the problem-in-head approach works as well as I think, then it's going to be necessary to work backwards from the solution to the pointy-haired bits in the middle required by external observers, like the class relationship diagrams and the interface specifications.  That's actually true in the maths/physics sphere too - very often in an exam I would go from the problem to the solution, then go back and show my working.

#### HotSwift
A few places have linked to Apple's use of Swift in iOS, it's useful to put it in context.

How much of Solaris was made out of Java? Almost none. There was a web browser that you've never heard of called HotJava, and that shipped with Solaris, but that's it. The rest of the OS remained resolutely C with Motif (later GTK+). While Sun wanted us to believe that Java was the developer toolkit of choice, they never chose it themselves.

#### How big is an integer?
In the beginning, when all was without form and void, Kernighan and Ritchie created char. And they said, "let it be of a size chosen by the compiler, guaranteed to be large enough to hold one character from the execution character set." And so it was, and they decreed that whatever the size of this char, the compiler would call its size 1.

Right, that's enough silly voice. There were also other types of integer: short, int, long, long long, and pointers. The point is that on any system, you could find out how big one of these numbers is (using the compiler's sizeof() feature) but that size depended on the system you were compiling for. Assuming that a sizeof(char)==1 is OK, but assuming that sizeof(int)==4 will lead to trouble: it's 2 on some systems and 8 on some others, for example.

C also provides the typedef feature, which lets you give new names to existing types. Plenty of API designers use typedef to rename integer types to give some clue as to their meaning; so you'll see size_t used to describe the size of something, ptrdiff_t to express the difference between two pointers, and so on.

Leaving the size of the various types undefined gives plenty of flexibility to implementors. A compiler for a given platform can choose to create ints that are the same size as a CPU register, or the maximum size transferable on the data bus in one load operation. It gives benefits to well-written software, which can be ported to hardware with different data characteristics just by recompiling. It also causes some problems for programmers whose software needs to talk to, well, anything else.

Imagine two computers communicating over a network. One of them wants to send an integer to the other, and the program represents the integer as an int. Well the receiving computer could have a different idea of how big an int is. Maybe the sender puts four bytes onto the network, but the receiver waits forever because it wants eight bytes. Alternatively, maybe the sender delivers eight bytes, the first four of which are incorrectly used as the integer, and the next four remain in the queue to be incorrectly used for the next value required.

The same problem can occur with files, too. Imagine that my app writes an int to disk. My customer then upgrades their computer, and my same app running on a different architecture tries to read the int back in. Does it get the same value? I've even seen this problem with two processes on the same computer, where one was a 64-bit kernel talking to a 32-bit user process. [N.B. a related problem is that every process needs to agree on which byte goes where in multi-byte integers; a problem not considered here].

Clearly there's a need for integer types that are of a stable size, guaranteed to remain the same whatever architecture the software is running on. The inttypes.h or stdint.h headers, introduced in C99 (so well over a decade ago), provide these (and more). If the target environment is capable of providing an integer type that uses exactly eight bits, you can access that as int8_t (uint8_t for unsigned integers). Whether or not this is available, the smallest type that holds at least eight bits is called int_least8_t. The integer type that holds at least eight bits and is fastest for the computer to handle can also be used, as int_fast8_t. Standard implementations should provide these types for 8, 16, 32 or 64 bit integers, and may provide types for other sizes too.

The point of all of this is that while there are guaranteed-size integer types available, anything that isn't obviously of a specific size should be treated as if it's of unknown size. Take, for example, NSInteger. It and the unsigned NSUInteger type were introduced by Apple to provide source code compatibility between 32 and 64-bit Cocoa API code, while also expanding the values used and returned by the API on wider platforms.

This could have been done by keeping the API as it was, and changing the size of int on 64-bit Cocoa from 4 bytes to 8. This would've been a poor choice, because plenty of code that assumes (wrongly) that sizeof(int)==4 would have broken. Most other 64-bit environments provide eight byte longs and pointers and four-byte ints, and Apple chose to follow suit for better compatibility.

Instead, NSInteger's underlying type depends on the architecture you're compiling for. Currently, all Apple's 32-bit platforms define it as int, and the 64-bit platforms define it as long. The end result is that while an NSInteger is guaranteed to be big enough to hold the length of an NSArray or an NSString, it isn't guaranteed to be the same size as someone else's NSInteger. Some compatibility issues still remain, and failing to deal with them can lead to some subtle bugs that only manifest themselves in particular situations.

#### How exciting
Today I was pleasantly surprised by Interface Builder.  Not shiny, new, where the hell have they put that buttonstreamlined IB3, but boring old IB2 which even Slowlaris users could work out how to use.  I dragged a header defining a category with an IBAction onto IB, and lo, nay even behold, it did the right thing.

That may seem unexciting and even expected, but it's one of those nice cases where it's pleasing that everything just works.  I thought category headers might be edge-case enough to confuse the thing; many people would put their IBAction definitions in the "regular" @interface header so that the IBOutlets are in the same place.

#### How I got root on my University's UNIX network
Back when I was a student, the way you talked to other people on the internet was via Usenet. The language we used, while still called "English", was slightly different from the language we use today. One small example of this difference is that there was still an outside chance that the word "hacker" could be a badge of honour, an indication of one who wanted to understand the principles of a system and how they could manipulate it. People who identified themselves as hackers in this sense had their Usenet groups, and they had their identifying mark: the Glider from Conway's game of life.

I was studying Physics, because there could be no grander system to hack than the Universe. But I also hacked computers. I wanted to understand how people made them do certain things, and how I could make them do the things I wanted. Other people hacked for different reasons: they wanted to make other people's computers do certain things, they wanted to show off what they could make computers do, they had other motivations still.

A meme that went around the Usenet groups for computer hackers was that if you truly wanted to understand computers, you should learn UNIX. So I did. I had a cheap PC in my room, and I installed Linux, FreeBSD, Darwin, and other UNIX variants to learn about UNIX. I learnt about shell scripting, and Perl scripting, and C programming. I had shell accounts on Solaris systems and NeXT systems and Tru64 systems, and I learnt about the differences and the similarities and the UNIX wars. I built a small collection of other systems (an ugly beige PowerMac, a few more PCs, and a couple of sleek Sun pizzaboxes) and learnt about TCP/IP, ICMP, HTTP, X11 and other network protocols.

Some of the hackers who wanted to make other people's computers do things believed that the ultimate goal was to get root on someone else's computer. If you got root, you could make their computer do whatever you wanted. It happened that even though I had root on my own computers, I managed to get root on some of the University's computers.

I was called in to talk to a sub-department head in my department: there was someone from administration and someone from central IT too. They sat on one side of a desk, I was on the other. I was pretty nervous. I told them about what I'd learned about UNIX, and scripting, and networking, and about the root user. They asked me to show them some of what I'd learned, and there was a Mac on the desk between us which I used for this purpose.

A few days later, I got a phone call from the admin person. I had got the job, and when I started the outgoing sysadmin would give me the root password. And that's how I got root on their UNIX network.

#### How much programming language is enough?
Many programmers have opinions on programming languages. Maybe, if I present an opinion on programming languages, I can pass off as a programmer.

An old debate in psychology and anthropology is that of nature vs nurture, the discussion over which characteristics of humans and their personalities are innate and which are learned or otherwise transferred.

We can imagine two extremists in this debate turning their attention to programming languages. On the one hand, you might imagine that if the ability to write a computer program is somehow innate, then there is a way of expressing programming concepts that is closely attuned to that innate representation. Find this expression, and everyone will be able to program as fast as they can think. Although there'll still be arguments over bracket placement, and Dijkstra will still tell you it's rubbish.

On the other hand, you might imagine that the mind is a blank slate, onto which can be writ any one (or more?) of diverse patterns. Then the way in which you will best express a computer program is dependent on all of your experiences and interactions, with the idea of a "best" way therefore being highly situated.

We will leave this debate behind. It seems that programming shares some brain with learning other languages, and when it comes to deciding whether language is innate or learned we're still on shaky ground. It seems unlikely on ethical grounds that Nim Chimpsky will ever be joined by Charles Babboonage, anyway.

So, having decided that there's still an open question, there must exist somewhere into which I can insert my uninvited opinion. I had recently been thinking that a lot of the ceremony and complexity surrounding much of modern programming has little to do with it being difficult to represent a problem to a computer, and everything to do with there being unnecessary baggage in the tools and languages themselves. That is to say that contrary to Fred Brooks's opinion, we are overwhelmed with Incidental Complexity in our art. That the mark of expertise in programming is being able to put up with all the nonsense programming makes you do.

From this premise, it seems clear that less complex programming languages are desirable. I therefore look admirably at tools like Self, io and Scheme, which all strive for a minimum number of distinct parts.

However, Clemens Szyperski from Microsoft puts forward a different argument in this talk. He works on the most successful development environment. In the talk, Szyperski suggests that experienced programmers make use of, and seek out, more features in a programming language to express ideas concisely, using different features for different tasks. Beginners, on the other hand, benefit from simpler languages where there is less to impede progress. So, what now? Does the "less is more" principle only apply to novice programmers?

Maybe the experienced programmers Szyperski identified are not experts. There's an idea that many programmers are expert beginners, that would seem to fit Szyperski's model. The beginner is characterised by a microscopic, non-holistic view of their work. They are able to memorise and apply heuristic rules that help them to make progress.

The expert beginner is someone who has simply learned more rules. To the expert beginner, there is a greater number of heuristics to choose from. You can imagine that if each rule is associated with a different piece of programming language grammar, then it'd be easier to remember the (supposed) causality behind "this situation calls for that language feature".

That leaves us with some interesting open questions. What would a programming tool suitable for experts (or the proficient) look like? Do we have any? Alan Kay is fond of saying that we're stuck with novice-friendly user experiences, that don't permit learning or acquiring expertise:

  There is the desire of a consumer society to have no learning curves. This tends to result in very dumbed-down products that are easy to get started on, but are generally worthless and/or debilitating. We can contrast this with technologies that do have learning curves, but pay off well and allow users to become experts (for example, musical instruments, writing, bicycles, etc. and to a lesser extent automobiles).

Perhaps, while you could never argue that common programming languages don't have learning curves, they are still "generally worthless and/or debilitating". Perhaps it's true that expertise at programming means expertise at jumping through the hoops presented by the programming language, not expertise at telling a computer how to solve a problem in the real world.

#### How people learn
Don't you hate those times when you go to a talk or article that says "you should be doing this", but then doesn't explain how to do that?

I just wrote one. In "Coding. Standards." I explained that what software engineers should do is to learn and analyse from all their experiences and interactions. But how do you do that? How does learning work?

The model of learning that I know best is the Kolb learning cycle, which I was taught when I was in management training. Yes - the reason I wear my hair this long is to stop it going pointy. David Kolb published this model in the 1980s, it's been augmented and adapted since then but is still approximately as written. Having said that, some neuroscientists dispute the validity of this and other models of learning.

Kolb says that there are four distinct processes in learning:

Concrete Experience: actually doing a thing.
Reflective Observation: analysing how you (or someone else) did a thing.
Abstract Conceptualisation: building a model of how things should be done.
Active Experimentation: just playing with the plasticine and seeing what comes out.

Not everybody goes through all of the items in the cycle, but most people start out somewhere and progress through at least a couple of the points, probably in the order presented (acknowledging that as a cycle it should be, well, cyclic). The four examples he gave corresponded to four "corners" of the cycle:

Diverging: concentrates on concrete experience and reflective observation. Tries something out, and decides how it could be adapted.
Assimilating: focuses on observation and conceptualisation. Sees how different examples might fit a common pattern.
Converging: conceptualisation and experimentation. Try to build a mental model from theory and then put it into practice.
Accommodating: experimentation and experience. Hands-on trying things out and deciding which ones work.

This gives you a starting point for working out how you learn, and what materials best support your style: there are many blog posts and conference talks based on personal experience, which fit the "reflective observation" category. Technical books tend to be more abstract, and fit conceptual learning. Training courses often bridge conceptualisation and experimentation.

It also gives you a way to understand why some people don't like your good-faith attempts to help them. "RTFM" works for those people who want to build a conceptual model, but not for everybody else. Similarly, those people saying "please give me the codes" may be people who learn by abstracting from examples. You can't hate people just for learning in a different way.

#### How retrospectives ban shoes
At the end of each sprint, we hold a retrospective. The book "Agile Coaching" by Rachel Davies and Liz Sedley says:
An iteration retrospective should help the team explore the following:

 	What insights do they have from the last iteration?
 	What areas do they want to focus on improving?
 	What ideas can they act on in the next iteration?

Take this too literally, and you end up adding the thing next time that stops you getting wrong the mistake from this time. "That time, the explosive was hidden in a shoe, so let's add shoes to the list of banned items."

What if it was just a mistake? Do you need to change the way you do everything to fix a problem you encountered once?

Retrospectives need to take a longer-term view. How are we doing, and how specifically did that change this sprint? Do we need to change how we do all of our tasks because it didn't work for that one task? How many hundreds of tasks has our process worked for?

I discussed this with Steven Baker who described a team where the retrospective facilitator kept a risk register across multiple sprints. Yes, this thing happened and it was bad, but how bad? How often has it happened before? How likely is it to happen again? How bad would it be if it does?

#### How to answer questions the smart way
You may have read how to ask questions the smart way by Eric S. Raymond. You may have even quoted it when faced with a question you thought was badly-formed. I want you to take a look at a section near the end of the article.

How to answer questions in a helpful way is the part I'm talking about. It's a useful section. It reminds us that questions are part of a dialogue, which is a two-way process. Sometimes questions seem bad, but then giving bad answers is certainly no way to make up for that. What else should we know about answering questions?

The person who asked the question has had different experiences than you. The fact that you do not understand why the question should be asked does not mean that the question should not be asked. "Why would you even want to do that?" is not an answer.

Answer at a level appropriate to the question. If the question shows a familiarity with the basics, there's no need to mansplain trivial details in the answer. On the other hand, if the question shows little familiarity with the basics, then an answer that relies on advanced knowledge is just pointless willy-waving.

The shared values that pervade your culture are learned, not innate. Not everyone has learned them yet, and they are not necessarily even good, valuable or correct. This is a point that Raymond misses with quotes like this:

  You shouldn't be offended by this; by hacker standards, your respondent is showing you a rough kind of respect simply by not ignoring you. You should instead be thankful for this grandmotherly kindness.

What this says is: this is how we've always treated outsiders, so this is how you should expect to be treated. Fuck that. You're better than that. Give a respectful, courteous answer, or don't answer. It's really that simple. We can make a culture of respect and courtesy normative, by being respectful and courteous. We can make a culture of inclusion by not being exclusive.

I'm not saying that I'm any form of authority on answering questions. I'm far from perfect, and by exploring the flaws I know I perceive in myself and making them explicit I make them conscious, with the aim of detecting and correcting them in the future.

#### How to excel at IDE design
When people have the "which IDE is best" argument, what they're actually discussing is "which slightly souped-up monospace text editor with a build button do you like using". Eclipse, Xcode, IntelliJ, Visual Studio…all of these tools riff on the same design—letting you see the source code and change the source code. As secondary effects you can also do things like build the source code, run the built product, and debug it.

The most successful IDE in the world, I would contend (and then wave my hands unconvincingly when anyone asks for data), is one that's not designed like that at all. It's the one that is used by more non-software specialists than any of those named above. The one that doesn't require you to practice being an IDE user for years before you get any good. The one that business analysts, office assistants, accountants and project managers alike all turn to when they need their computer to run through some custom algorithm.

The IDE whose name appears in the title of this post.

Now what makes a spreadsheet better as a development environment is difficult to say; I'm unaware of anyone having researched it. But what makes it a different development environment from an IDE can be clearly seen by using each of them.

In a spreadsheet, it's the inputs and results that are front-and-centre in the presentation, not the intermediate stuff that gets you from one to the other. You can test your "code" by typing in a different input and watching the results change in front of you. You can see intermediate results, not by breaking and stepping through, or putting in a log statement then switching to the log view, but by breaking the algorithm up into smaller steps (or functions or procedures, if you want to call them that). You can then visualise how these intermediate results change right along side the inputs and outputs. That's quicker feedback than even REPLs can offer.

Many spreadsheet users naturally adopt a "test-first" approach; they create inputs for which they know what the results should be and make successively better attempts to build a formula that achieves these results. And, of course, interesting visualisations like graphs are available (though the quality does vary between products). Drawing a graph in Xcode is…challenging (and yes, nerds, I know about CorePlot).

All of this is not to say that spreadsheets are the future of IDEs. In many ways spreadsheets are more accessible as IDEs, and it'd be good to study and learn from the differences and incorporate some of them into the things that are the future of IDEs.

#### How to find me
It came to my attention this week that people are finding me via Google, which (unsurprisingly) links to here. I've been blogging for a couple of years at Secure Mac Programming, and I'm on twitter as @iwasleeg. I'm +Graham Lee on Google Plus, too. My email is graham at iamleeg dot com.

#### How to handle Xcode in your meta-build system's iOS or Mac app target
OK, I've said before in APPropriate Behaviour that I dislike build systems that build other build systems:

Some build procedures get so complicated that they spawn another build system that configures the build environment for the target system before building. An archetypal example is GNU autotools - which actually has a three-level build system. Typically the developers will run `autoconf`, a tool that examines the project to find out what questions the subsequent step should ask and generates a script called `configure`. The user downloads the source package and runs `configure`, which inspects the compilation environment and uses a collection of macros to create a Makefile. The Makefile can then compile the source code to (finally!) create the product.

As argued by Poul-Henning Kamp, this is a bad architecture that adds layers of cruft to work around code that has not been written to be portable to the environments where it will be used. Software written to be built with tools like these is hard to read, because you must read multiple languages just to understand how one line of code works.

One problem that arises in any cross-platform development is that assumptions about "the other platforms" (being the ones you didn't originally write the software on) are sometimes made based on one of the following sources of information:

none
a superficial inspection of the other platform
analogy to the "primary" platform

An example of the third case: I used to work on the Mac version of a multi-platform product, certain core parts of which were implemented by cross-platform libraries. One of these libraries just needed a little configuration for each platform: tell it what file extension to use for shared libraries, and give it the path to the Registry.

What cost me a morning today was an example of the second case: assuming that all Macs are like the one you tried. Let me show you what I mean. Here's the contents of /Developer on my Mac:

$ ls /Developer/
WebObjects

Wait, where's Xcode? Oh right, they moved it for the App Store builds didn't they?

$ ls /Applications/Xcode.app
ls: /Applications/Xcode.app: No such file or directory

WHAAAAA?

Since Xcode 2.5, Xcode has been relocatable and can live anywhere on the filesystem. Even if it is in one of the usual places, that might not be the version a developer wants to use. I keep a few different Xcodes around: usually the current one, the last one I knew everything worked on, and a developer preview release when there is one. I then also tend to forget to throw old Xcodes away, so I've got 4 different versions at the moment.

But surely this is all evil chaos from those crazy precious Mac-using weirdos! How can you possibly cope with all of that confusion? Enter xcode-select:

$ xcode-select -print-path
/Applications/Xcode4.6.app/Contents/Developer

Xcode-select is in /usr/bin, so you don't have the bootstrapping problem of trying to find the tool that lets you find the thing. That means that you can always rely on it being in one place for your scripts or other build tools. You can use it in a shell script:

XCODE_DEVELOPER_DIR=`/usr/bin/xcode-select -print-path`

or in a CMake file:

exec_program(/usr/bin/xcode-select ARGS -print-path OUTPUT_VARIABLE XCODE_DEVELOPER_DIR)

or in whatever other tool you're using. The path is manually chosen by the developer (using the -switch option), so if for some reason it doesn't work out (like the developer has deleted that version of Xcode without updating xcode-select), then you can fall back to looking in default locations.

Please do use xcode-select as a first choice for finding Xcode or the developer folder on any Mac system, particularly if your project uses a build generator. It's more robust to changes—either from Apple or from the users of that Mac—than relying on the developer tools being installed to their default location.

#### How to hire Graham Lee
There are few people who can say that when it comes to Cocoa application security, they wrote the book. In fact, I can think of only one: me. I've just put the final draft together for Professional Cocoa Application Security and it will hit the shops in June: click the link to purchase through my Amazon affiliate programme.Now that the book's more-or-less complete, I can turn my attention to other interesting projects: by which I mean yours! If your application could benefit from a developer with plenty of security experience and knowledge to share in a pragmatic fashion, or a software engineer who led development of a complex Cocoa application from its legacy PowerPlant origins through Snow Leopard readiness, or a programmer who has worked on performance enhancement in networking systems and low-level daemon code on Darwin and other UNIX platforms, then your project will benefit from an infusion of the Graham Lee magic. Even if you have some NeXTSTEP or OPENSTEP code that needs maintaining, I can help you out: I've been using Cocoa for about as long as Apple has.Send an email to iamleeg &lt;at&gt; securemacprogramming &lt;dot&gt; com and let's talk about your project. The good news is that for the moment I am available, you probably can afford me[], and I really want to help make your product better. Want to find out more about my expertise? Check out my section on the MDN show, and the MDN security column.[] It came up at NSConference that a number of devs thought I carry a premium due to the conference appearances, podcasts and other material I produce. Because I believe that honesty is the best policy, I want to come out and say that I don't charge any such premium. My rates are consistent with other contractors with my level of experience, and I even provide a discounted rate for NGOs and academic institutions.

#### How to solve every problem in Cocoa
Yes, really, every problem.  Don't think of Cocoa as &quot;simple things simple, complex things possible&quot; (actually, was it Cocoapenextstepsody or Perl who started with that tagline?  Or someone else?  I digress) but &quot;simple things simple, complex things simple but you're looking at it wrong&quot;.  With Objective-C 2.0 (particularly properties), Core Data, Cocoa Bindings and Cocoa Scripting, almost every &quot;it doesn't work&quot; moment comes down to getting something wrong with KVC or with KVO - either observing the wrong key, or typoing a method name such that you aren't KVC-compliant for a key you need to be, getting validation wrong or unexpectedly going down the -setNilValueForKey: path.  So do yourself a favour:

#define GLInstanceMethodEntryLog(format, ...) NSLog(@"-[%@(%p) %@] entry: %@", NSStringFromClass([self class]), self, NSStringFromSelector(_cmd), [NSString stringWithFormat: format, ##VA_ARGS])

Now because all of the retain count bugs disappeared when you turned the garbage collector on, the remaining issues are with that bit of code the PHBs are paying for ;-)

#### How to TDD with CATCH
Plenty of people have asked me about the TDD framework I use. While the book Test-Driven iOS Development has code using OCUnit (for pragmatic, and previously-covered, reasons); I am currently more frequently to be discovered using Phil Nash's CATCH framework. Here's how you can get from New Project to first passing test. I'm starting in the same place that the book's sample code starts: I need a Topic class that has a name.

Set up the project.

File->New Project in Xcode. Create an app project that matches your needs: an empty app will be fine. I'll assume in this project, just as in the book, that you'll be using Automatic Reference Counting for this project.

Now you need to add another app target. The "Empty Application" template is sufficient for this, and I call this target "Unit Tests". When you've got that target set up, remove the app delegate class and the main.m file: we'll add a test runner from CATCH.

For this target, you also have to choose whether or not to use ARC. If you enabled ARC for the app target, then the classes you're testing will use Automatic Reference Counting, but the CATCH internals cannot because they use scary runtime hacks to dynamically generate classes from the test cases. In this walkthrough, you should enable ARC but we'll have to edit compiler flags for a couple of files to get manual reference counting where we need it.

Check out CATCH from the GitHub project linked above. Then go into the Build Rules for your Unit Tests target, and add the path to the Catch/include source folder. For example, if you clone CATCH to the top-level folder of your Xcode project, you'll need to add "$(SRCROOT)/Catch/include".

Locate the file Catch/projects/runners/iTchRunner/itChRunnerMain.mm (capitalisation as supplied) in Finder, and drag it onto your Xcode project. Add this file to the Unit Tests target. Now you need to go to the Build Phases inspector for your target, and under Compile Sources, next to itChRunnerMain.mm add the flag -fno-objc-arc.

Add the first test

Add a new file to the Unit Tests target, called TopicTests.mm (you don't need the matching header file). As with itChRunnerMain.mm, you'll need to set the -fno-objc-arc flag in the Build Phases. Include the catch.hpp header and write the test that checks for the name:

#include "catch.hpp"

TEST_CASE("Topic/Name", "Require that Topic has a name") {
    Topic *topic = [[Topic alloc] initWithName: @"iPhone"];
    REQUIRE(topic != nil);
    REQUIRE([topic.name isEqualToString: @"iPhone"]);
    [topic release];
}

Get this test to compile

So currently, the test target won't even build: we're using a Topic class and its name property, and these don't exist. Add a new file to the project, an NSObject class called Topic. You could add this file to both targets as it will be app code: don't disable ARC here because the whole point is you want to write automatically reference-counted app code.

Just to fast-forward things a bit, here's an implementation of Topic.[hm] that contains minimal implementations of the required methods.

Topic.h
#import &lt;Foundation/Foundation.h&gt;

@interface Topic : NSObject

@property (readonly) NSString *name;

- (id)initWithName: (NSString *)aName;

@end

Topic.m
#import "Topic.h"

@implementation Topic

@synthesize name = _name;

- (id)initWithName:(NSString *)aName {
    if (self = [super init]) {
    }
    return self;
}

@end

Observe that the test fails.
Run the "Unit Tests" app in Xcode. You'll see a screen like this:

It's not very exciting, but at least you can tell what to do: tap the "Run All Tests" button.

Still not exciting, and it's possible that if you have certain types of colour-blindness it's not clear what's happened. Luckily the logs are more explicit:

2012-03-13 17:18:01.640 UnitTests[960:707] Application windows are expected to have a root view controller at the end of application launch

2012-03-13 17:18:03.915 UnitTests[960:707] topic != __null succeeded for: 0x2d11b0 != __null

2012-03-13 17:18:03.917 UnitTests[960:707] [topic.name isEqualToString: @"iPhone"] failed for: false

2012-03-13 17:18:03.923 UnitTests[960:707] 1 failures

Make this test pass

There's only one line to add to the app code to get everything passing. Here's the Topic implementation again with the change highlighted:

@implementation Topic

@synthesize name = _name;

- (id)initWithName:(NSString *)aName {
    if (self = [super init]) {
        _name = [aName copy];
    }
    return self;
}

@end

Run the tests again:

2012-03-13 17:46:17.644 UnitTests[1049:707] Application windows are expected to have a root view controller at the end of application launch

2012-03-13 17:46:43.746 UnitTests[1049:707] topic != __null succeeded for: 0x2ca7f0 != __null

2012-03-13 17:46:43.748 UnitTests[1049:707] [topic.name isEqualToString: @"iPhone"] succeeded for: true

2012-03-13 17:46:43.753 UnitTests[1049:707] no failures

Win.

Conclusions

The runner for CATCH tests is currently very basic: it lets you know when things have passed or failed but not which tests have failed, though the tests are named. You need to manually prod it to get tests to run. However, the syntax is a refreshing change from the verbosity of JUnit-derived test frameworks.

#### How to version a Mach-O library
Yes, it's the next instalment of "cross-platform programming for people who don't use Macs very much". You want to give your dynamic library a version number, probably of the format major.minor.patchlevel. Regardless of marketing concerns, this helps with dependency management if you choose a version convention such that binary-compatible revisions of the libraries can be easily discovered. What could possibly go wrong?

The linker will treat your version number in the following way (from the APSL-licensed ld64/ld/Options.cpp) if you're building a 32-bit library:

//
// Parses number of form X[.Y[.Z]] into a uint32_t where the nibbles are xxxx.yy.zz
//
uint32_t Options::parseVersionNumber32(const char* versionString)
{
	uint32_t x = 0;
	uint32_t y = 0;
	uint32_t z = 0;
	char* end;
	x = strtoul(versionString, &end, 10);
	if ( *end == '.' ) {
		y = strtoul(&end[1], &end, 10);
		if ( *end == '.' ) {
			z = strtoul(&end[1], &end, 10);
		}
	}
	if ( (*end != '\0') || (x > 0xffff) || (y > 0xff) || (z > 0xff) )
		throwf("malformed 32-bit x.y.z version number: %s", versionString);

	return (x &lt;&lt; 16) | ( y &lt;&lt; 8 ) | z;
}

and like this if you're building a 64-bit library (I've corrected an obvious typo in the comment here):

//
// Parses number of form A[.B[.C[.D[.E]]]] into a uint64_t where the bits are a24.b10.c10.d10.e10
//
uint64_t Options::parseVersionNumber64(const char* versionString)
{
	uint64_t a = 0;
	uint64_t b = 0;
	uint64_t c = 0;
	uint64_t d = 0;
	uint64_t e = 0;
	char* end;
	a = strtoul(versionString, &end, 10);
	if ( *end == '.' ) {
		b = strtoul(&end[1], &end, 10);
		if ( *end == '.' ) {
			c = strtoul(&end[1], &end, 10);
			if ( *end == '.' ) {
				d = strtoul(&end[1], &end, 10);
				if ( *end == '.' ) {
					e = strtoul(&end[1], &end, 10);
				}
			}
		}
	}
	if ( (*end != '\0') || (a > 0xFFFFFF) || (b > 0x3FF) || (c > 0x3FF) || (d > 0x3FF)  || (e > 0x3FF) )
		throwf("malformed 64-bit a.b.c.d.e version number: %s", versionString);

	return (a &lt;&lt; 40) | ( b &lt;&lt; 30 ) | ( c &lt;&lt; 20 ) | ( d &lt;&lt; 10 ) | e;
}

The specific choice of bit widths in both variants is weird (why would you have more major versions than patchlevel versions?) and the move from 32-bit to 64-bit makes no sense to me at all. Nonetheless, there's a general rule:

Don't use your SCM revision number in your version numbering scheme.

The rule of thumb is that the major version can always be less than 65536, the minor versions can always be less than 256 and you can always have up to two minor version numbers. Trying to supply a version number that doesn't fit in the bitfields defined here will be a linker error, and you will not go into (address) space today.

#### How UX Practitioners Produce Findings in Usability Testing
The Paper

How UX Practitioners Produce Findings in Usability Testing by Stuart Reeves, in ACM Transactions on Computer-Human Interaction, January 2019.

Notes

Various features of this paper make it a shoe-in for Research Watch.

It is about the intersection between academia and commercial practice. That is where the word "Labrary" comes from.
It extends the usual "human-computer interaction" focus of UX to include the team performing the UX, which an aspect of PETRI.
I get to use the word "praxeology".

Reeves compares the state of UX in the academic literature with the state of UX in commercial fields. He finds a philosophical gap that is similar to something I observed when studying "Requirements Engineering" on a Software Engineering M.Sc. course. Generally, the academic treatment of UX describes usability problems as things that exist, and that the task of UX activities is to find them.

The same can be seen in much early literature on requirements engineering. We assume that there is a Platonic model of how a software product should work, and that the job of the requirements engineer is to "gather" requirements from the stakeholders. Picture a worker with a butterfly net, trying to collect in these elusive and flighty requirements so they can pin them down in a display case made by the Jira Cabinet Company.

There's an idea here that, even before it's formed, the software is real and has an identity independent of the makers, users, and funders. Your role in the software production process is one of learning and discovery, trying to attain or at least approximate this ideal view of the system that's out there to be had.

Contrasted with this is the "postmodern" view, which is a more emergent view. Systems and processes result from the way that we come together and interact. A software system both mediates particular interactions and blocks or deters others. The software system itself is the interaction between people, and developments in it arise as a result of their exchanges.

In this worldview, there are not "UX problems" to be found by adequate application of UX problem-discovery tools. There are people using software, people observing people using software, and people changing software, and sometimes their activities come together to result in a change to the software.

This philosophy is the lens through which Reeves engages in the praxeology (study of methods) of UX practitioners. His method is informed by ethnomethodological conversation analysis, which is an academic way of saying "I watched people in their context, paying particular attention to what they said to each other".

The UX activity he describes is performed by actors in two different rooms. In the test room, the participant uses a computer to achieve a goal, with some context and encouragement provided by a moderator. The rest of the team are in the observation room, where they can see and hear the test room and the participant's screen but talk amongst themselves.

Four representative fragments expose different features of the interactions, and to my mind show that UX is performative, arising from those interactions rather than being an intrinsic property of the software.

In fragment A, the participant reports a problem, the observers react and decide to report it.
In fragment B, the participant reports a problem, the observers react and suppress reporting it.
In fragment C, the participant does not seem to be having a problem, but the observers comment that they did not do something they would have expected, and discuss whether this is an issue.
In fragment D, the participant is working on the task but does not choose the expected approach, observers see that, and define a problem and a solution that encompasses that.

One observation here is that even where a participant is able to complete the task, a problem was raised. The case in fragment D is that the participant was asked how they would report a problematic advert. They described sending an email to the client. That would work. However, the product team see that as a problem, because they are working on the "submit a complaint" feature on the website. So, even though the task goal can be satisfied, it was not satisfied the way they want, which means there's a UX problem.

There are all sorts of things to learn from this. One is that you can't separate the world neatly into "ways humans do things" and "measurements of the ways humans do things", because the measurements themselves are done by humans who have ways of doing things. Another is that what you get out of UX investigations depends as much on the observers as it does on the participants' abilities. What they choose to collectively see as problems and to report as problems depends on their views and their interactions to an extent comparable to their observations of the participants working through the tasks.

Ultimately it's more evidence for the three systems model. Your team, your software, and your customers are all interacting in subtle ways. Behaviour in any one of these parts can cause significant changes in the others.

#### HPC at FOSDEM 2019
This year's FOSDEM featured an HPC, Big Data and Data Science devroom on the Sunday. This post is the first part of my notes on the topics presented there. If you are interested, book some time and let's talk about what it means for your and your high-performance computing team.

OpenHPC Update

Adrian Reber from the OpenHPC project gave a refresher on what OpenHPC is, and a status update. OpenHPC has not been represented at FOSDEM since 2016, when the project was very new.

It's a community-driven project with representation from many vendors and HPC sites. On first blush their output might appear to be "RPM packages" and "documentation" but their mission is actually to discover and share best practices in HPC management. Those packages are all well-tested with each other, and the documentation is tested every release, too. The idea is that if you build the core of your cluster with OpenHPC packages on CentOS-like Linux distributions, on either x86-64 or AArch64, you get to rely on tried and tested work from the whole community.

Reber, who works at Red Hat on their OpenHPC efforts, invited everyone to join the weekly project steering calls in a demonstration of the openness of the project. He discussed future directions, including an upcoming release v1.3.7 that will include packages rebuilt with the ARM HPC compiler for AArch64, and the challenges of understanding when is right to release v1.4 which will drop SLES12 for SLES15 and RHEL7 for RHEL8.

ReFrame

On the subject of HPC libraries, a common frustration is testing codes with various combinations of compilers, MPI libraries, hardware capabilities and so on. Developers both want to know that their code is correct (i.e. the science outcomes are still valid after a change) and that the performance has not been significantly impacted.

Victor Holanda discussed ReFrame, a tool for HPC regression and performance testing developed at CSCS and used regularly on Piz Daint and their other clusters. Written in Python, it gives test authors a way to express what their tests require (e.g. that they must run on machines with CUDA, compile a particular code with one of three different compilers, load environment modules with one of two different MPIs), run the tests, and inspect the output for certain outcomes.

Testers get to run a single command, or point their Jenkins or Travis CIs at a single command, to discover and execute the tests. The ReFrame runtime will compare the environments that the test can use with the ones that are available, and will report on the outcomes in each of those environments.

Inside CSCS, ReFrame is used for a 90 minute nightly production test run, and 10 minute maintenance runs to check for system regressions after configuration changes. They also have a set of diagnostic tests to help understand what's happened if a node goes bad. Their approach to correctness is very robust; the team do not declare that they support something until it has enough users to know how well it works. They also say that in three years of development they "have never seen a python stacktrace" from ReFrame, as they test ReFrame with ReFrame while they are developing it.

Singularity Containers

Singularity from sylabs is a container runtime tool that specifically addresses problems containerising HPC workloads. Eduardo Arango gave a "what's new in Singularity" update, as FOSDEM 2017 had already featured an introduction-level talk.

What's new is that they've rewritten in Go. This means they get better integration with libraries used in Docker, Kube etc., and could adopt the de facto standard Containers Networking Interface for software-defined networking when running containers. It also reduces the dependencies needed to get Singularity up and running.

The new version uses a new format for containers, SIF (Singularity Image Format), a read-only SquashFS filesystem along with metadata, all of which can be cryptographically signed using PGP for integrity protection. An upcoming extension will allow a writable overlay to be added to a SIF.

Supporting this, Sylabs have a new container library similar to DockerHub for hosting SIF images for public or private cloud use. They have a key store for those PGP signing keys, and a cloud-based remote image builder for developers who need to build images but can't do it locally.

Conclusion

This has been part one of my FOSDEM HPC round-up. I've focussed on the tools that are out there for automating and simplifying HPC workflows, because it's an interesting problem and one that presents challenges to many HPC teams. Don't forget that the Labrary can help!

#### HPC's Shift to the Cloud
Timothy Prickett Morgan writes on The Next Platform about the slow but inevitable shift to cloudy infrastructure. It seems that a tipping point has been reached, where the amount of IT money spent on "cloudy" infrastructure overtook the amount spent on "traditional" datacentre gear. This happened in 2018Q3, according to the IDC report cited in the article.

Prickett Morgan suggests that the transformation from bare metal to the cloud has been faster in HPC than in enterprise IT. In some senses, this makes sense, because HPC has long had the sorts of abstractions between the application and its environment that make it possible to change infrastructure. The days where an atomic energy or climate situation would be capable of running only on dedicated hardware with integrated bench seating are long gone, and all of the top supercomputers are now (highly tuned, admittedly) GNU/Linux clusters running on normal-ish CPUs: mostly Intel, some IBM POWER, and ARM are moving from evaluation to deployment too. All of these technologies, as well as the Nvidia GPUs used in CUDA codes and deep learning applications, and even Google's TPUs, are to be found in public cloud environments from the big providers.

On the other hand, there still are big honkin' boxes of bare metal, with the number one spot changing almost every year. So not all HPC applications are cloud-suitable, and some of those codes that people are trying to port to the cloud may prove challenging. Here's a summary of the components of a "traditional" HPC deployment, and how it might help or hinder adaptation to the cloud.

Infrastructure

Modules

Plenty of HPC sites already virtualise their filesystems to some extent, with the Modules package. Modules let administrators separate the installation and management of packages from their availability to users, by defining modulefiles that configure the environment to make each package accessible.

Where a team is already using modules to set up its environment for building or running codes, adopting containers and similar abstractions should be straightforward. Docker images, for example, can contain the module packages and the environment changes necessary to use the modules, and the HPC application image can be composed on top to include the relevant environment.

Job submission

HPC systems tend to already be built with the kind of self-service in mind that devops teams in commercial software development strive to provide. This heritage has evolved from the necessarily multi-user nature of a large supercomputer deployment. Mainframe batch submission systems, grid middleware (such as Sun -> Oracle -> Univa Grid Engine) and SLURM are based around the idea that a user can request a certain amount of resources to run their codes, the request being queued until the resources are available.

The open source SLURM project already supports cloud-native demand scheduling. Others are using Kubernetes as an elastic demand scheduler.

However, a lot of teams find job-specific submission scripts with hard coded assumptions about the environment they will run on, and codes that are tightly coupled to the submission script. Loosening that coupling will require some effort, but will make the codes "portable" to a cloud environment and enable new workflows for testing and development.

File systems

HPC sites frequently use high-performance parallel filesystems like Lustre or IBM's GPFS. While these filesystems can be deployed to a cloud environment, the performance characteristics will differ and it will be harder to tune to the specific topology offered by a physical deployment. Notice that HPC filesystems do not perform well in some scenarios so some applications like AI training may benefit from re-evaluating the data access strategy anyway. Portable codes could be tested against new hardware without significant capital outlay; for example Google Cloud uses Intel Optane non-volatile memory.

Job-specific nodes

A traditional cluster will often have login nodes for accessing the cluster from the scientific workstations, batch nodes for running and using the batch submission systems, compiler nodes for building codes, metadata nodes if it uses a parallel filesystem, and finally compute nodes on which the simulations and deep learning jobs are actually executed. The compute nodes may be divided into groups to service different queues, or to differentiate between testing/debugging and production jobs.

While operations teams may be interested in getting close to 100% utilisation out of the compute nodes, the fact is that the other classes of machine exist because they need different configurations, not because they need to always be available with dedicated hardware. They are ideal candidates to lead the transition to on-demand scaling, perhaps treating a physical cluster as a "private cloud" that commits as much hardware to compute as possible, scaling its other functions as needed.

Meanwhile, compilation and computation can be modelled as serverless workloads, consuming resource when they are executing but scaling to zero when not in use.

Application Support

MPI

MPI libraries like Open MPI already support demand-based scaling at job launch, using the -np option to control how many processes are started and the --hostfile to indicate where those hosts are. In principle it might seem like the hosts in the host file could be discovered using the Kubernetes service registry or similar services from other cloud orchestration layers. In practice the MPI library will need to support launching the process on the nodes so a middleware (see above) will still need to be deployed on top, or the MPI software extended with native support for the cloud's orchestration API.

Software Licences

This turns out to be one of the biggest hurdles for demand-scaling for many teams. HPC software such as proprietary compilers, numerical algorithms libraries and developer tools are licensed with a particular maximum number of parallel uses. Lab-developed codes may have evolved with assumptions about where the licence file is located, and not built defensively against the possibility that a licence can't be checked out. The ISV may have built assumptions into their licensing scheme, for example the host having a fixed IP or MAC address. A researcher or developer could have copied a particular licence file into their home directory, using that beyond other agreements being arranged with the vendor.

Where the licensing scheme is flexible enough to allow portability of the software, a good technique is to centralise management of the licenses using a secrets store, for example Vault, and to inject them into the HPC applications' containers when they are launched.

Alternatively, particularly if the licensing scheme is too rigid, it's worth evaluating the effort required and performance impact sustained to port the code to a different technology, for example an open source compiler. The trade off of this approach is that on the one hand, increased deployment flexibility is strategically beneficial, but the short term costs, staffing requirements and impact on the scientific mission can make it hard to justify or unworkable.

Conclusion

While there are significant benefits to be had in porting high-performance codes to cloud environments, the task is not without its challenges. Labrary consultancy with Graham Lee, bringing his experience in cloud-first devops teams, scalable systems at Facebook, and High-Performance Computing on ARM, can help your team identify and overcome these challenges.

Graham will be at the HPC, Big Data and Data Science devroom at FOSDEM in Brussels, February 2-3. Say hello, grab some time and let's move your codes forward!

#### Hyperloops for our minds
We were promised a bicycle for our minds. What we got was more like a highly-efficient, privately run mass transit tunnel. It takes us where it's going, assuming we pay the owner. Want to go somewhere else? Tough. Can't afford to take part? Tough.

Bicycles have a complicated place in society. Right outside this building is one of London's cycle superhighways, designed to make it easier and safer to cycle across London. However, as Amsterdam found, you also need to change the people if you want to make cycling safer.

Changing the people is, perhaps, where the wheels fell off the computing bicycle. Imagine that you have some lofty goal, say, to organise the world's information and make it universally accessible and useful. Then you discover how expensive that is. Then you discover that people will pay you to tell people that their information is more universally accessible and useful than some other information. Then you discover that if you just quickly give people information that's engaging, rather than accessible and useful, they come back for more. Then you discover that the people who were paying you will pay you to tell people that their information is more engaging.

Then you don't have a bicycle for the mind any more, you have a hyperloop for the mind. And that's depressing. But where there's a problem, there's an opportunity: you can also buy your mindfulness meditation directly from your mind-hyperloop, with of course a suitable share of the subscription fee going straight to the platform vendor. No point using a computer to fix a problem if a trillion-dollar multinational isn't going to profit (and of course transmit, collect, maintain, process, and use all associated information, including passing it to their subsidiaries and service partners) from it!

It's commonplace for people to look backward at this point. The "bicycle for our minds" quote comes from 1990, so maybe we need to recapture some of the computing magic from 1990? Maybe. What's more important is that we accept that "forward" doesn't necessarily mean continuing in the direction we took to get to here. There are those who say that denying the rights of surveillance capitalists and other trillion-dollar multinationals to their (pie minus tiny slice that trickles down to us) is modern-day Luddism.

It's a better analogy than they realise. Luddites, and contemporary protestors, were not anti-technology. Many were technologists, skilled machine workers at the forefront of the industrial revolution. What they protested against was the use of machines to circumvent labour laws and to produce low-quality goods that were not reflective of their crafts. The gig economies, zero-hours contracts, and engagement drivers of their day.

We don't need to recall the heyday of the microcomputer: they really were devices of limited capability that gave a limited share of the population an insight into what computers could do, one day, if they were highly willing to work at it. Penny farthings for middle-class minds, maybe. But we do need to say hold on, these machines are being used to circumvent labour laws, or democracy, or individual expression, or human intellect, and we can put the machinery to better use. Don't smash the machines, smash the systems that made the machines.

#### I do the stupid so you don't have to
So you want to use bc for some hexadecimal maths. You set the input base:

ibase=16

and the output base:

obase=16

Oops! I just set it to output in base 22. I'd already set it to think numbers were input in hexadecimal, and that's how it handled the "16" in my obase command. Either do this:

ibase=16
obase=10

Or this:

obase=16
ibase=16

#### I don't use version control when I'm writing
Or rather, I do use version control when I'm writing, and it isn't helpful.

I'm currently studying a PhD, and I have around 113k words of notes in a git repository. I also have countless words of notes in a Zotero database and a Remarkable tablet. I don't particularly miss git when I'm not storing notes in my repository.

A lot of the commit messages in that repository aren't particularly informative. "Update literature search", "meeting notes from today", "meeting notes", "rewrite introduction". So unlike in software, where I have changes like "create the ubiquitous documents folder if it doesn't already exist" and "fix type mismatch in document delegate conformance", I don't really have atomic changes in long-form writing.

Indeed, that's not how I write. I usually set out either to produce an argument, or to improve an existing one. Not to add a specific point that I hadn't thought of before, not to improve layout or structure in any specific way, not to fix particular problems. So I'm not "adding features" or "fixing bugs" in the same atomic way that I would in software, and don't end up with a revision history comprising multiple atomic commits.

Some of my articles—this one included—have no checkpoints in their history at all. Others, including posts on De Programmatica Ipsum and journal articles, have a dozen or more checkpoints, but only because I "saved a draft" when I stepped away from the computer, not because there were meaningful atomic increments. I would never revert a change in an article when I'm writing, I'd always fix forward. I'd never introduce a new idea on a branch, there's always a linear flow.

#### I feel sorry for...
The people credited with bringing GNUstep support to OCUnit a few versions back (at least one of whom I recognise as a competent programmer who really knows what they're doing), as the current version just won't work at all ;-).  It'll be more future-proof on the Apple platform, as it uses @try/@catch blocks.  However, as I want to use it on the GNU platform, this means a lot of #ifdef __NEXT_RUNTIME__ wrappers and NS_DURING/NS_HANDLER/NS_ENDHANDLER blocks...

#### I fix things for a living
Previously, on SICPers, I wrote that I make mistakes for a living. But making mistakes is no good if nobody's cleaning up after them, so I also fix things. Whatever gets in my team's way, it's my responsibility as their lead to make sure that it's no longer in their way.

Whether it's a process we instituted that slows us down, some technical debt that gets in the way of new development, or some infrastructure not behaving itself, it needs to go, and I either need to clear it out or find somebody to do it for us. On my team I've been nicknamed "Mister Fix-It" as a result of my policy of getting rid of everybody else's impediments before doing my own work, and we even have a Fix-It support queue just like our customer support queue. Except it's not our customers I'm supporting, it's my colleagues. Tickets in my queue range from "this script doesn't handle this case and did the wrong thing when I tried to do a build" to "merge these three repos into a monorepo". It doesn't matter, it's in our way, so it needs shifting.

Or does it? Is it, in fact, in the way, or are we solving the wrong problem? Some of the tickets in the Fix-It queue magically transmogrified from "Fix This" to "Measure This": let's discover where the problem really is before we solve something else.

#### I have some small idea of what I'm doing
I feel partly to blame for the current minor internet shitstorm.

But first, some scene-setting. There have long been associations between the programmer community and particular subcultures, some of which have become—not monocultural—at least dominant cultures within the world of computering. When I entered the field in the early 2000s, it was the tail end of the cyberpunk subculture: electronic and rock music, long hair on men, short hair on women, often dyed, black band or logo t-shirts, combat trousers or jeans, Doctor Martens 1460 boots. Antisocial work hours, caffeine-fuelled weekend long hacks, "all your base are belong to us" memes. Obtuse, but workhorse, C and Perl code. Maybe some Scheme if you were in the Free Software Foundation.

Then toward the end of the decade the hipster subculture rose to dominance. Mac laptops. Nice clothes, worn ironically. Especially the bow tie. Dishevelled hair. Fixed-gear bicycles. Turned up trouser cuffs and no socks. Looking to be the technical cofounder, looking down on those who ask them to be the technical cofounder. Coffee, now daytime only, had to be incredibly fussy. The evening drink of choice was Pabst Blue Ribbon. If your software wasn't in the tech stack of choice—a Ruby on Rails app, deployed to Heroku, edited in TextMate, hosted on Github—then were you crushing any code? Bro, do you even lift?

After a few years of this I noticed that one difference between these two cultures was an approach to knowledge, or more specifically its lack. It was easy to nerdsnipe a cyberpunk: if they didn't know something they would go and find out. Usenet groups had multiple FAQ lists because many people would all try to find the answers to the questions, and wikis weren't yet popular. In the hipster craze that followed, confidence in one's own knowledge reigned supreme. You showed that you knew everything you know, and you showed that everything you didn't know wasn't worth knowing.

This came to a head in my little Apple-centric niche of the computering field in 2015, when that whole community had chased monad tutorials and half-digested chapters on category theory into every corner of the conference and mailing list ecosystem. People gave talks not to share their knowledge, but to share that they were the people who knew the knowledge. Attendees turned up to product development conferences expecting to learn how a new programming language made it easier to develop programming, and came away confused about endofunctors.

I should be clear here that not everybody in the field was like that, and there are plenty of people who can make difficult maths accessible. There are plenty of people who can make computering accessible without difficult maths. Those people were still present.

But still I determined that something we didn't have enough of, that had been present in the cyberpunk-esque culture that came before (for all its other faults) was a willingness to say "I have no idea what I'm doing". Not in a "har har look at me get this wrong" way, but in a "this is interesting, let's find out more about it" way. An "I'm not the right person to ask, let's bring in an expert" way. A "to the library!" way.

So after a bit of writing about learning things I didn't know, I took my (then) decade of experience and position of incredibly-minor celebritydom in that niche little bit of computering, and submitted a talk called "I have no idea what I'm doing" to AltConf 2015. I think it may even have been a very last minute submission, with another speaker pulling out sick. The talk was a collection of anecdotes about things I didn't understand when the problem came my way, and how I dealt with that. Particularly, given that Swift was a year old at the time, I admitted I had less than a year of Swift experience and knew less about it than I did about Objective-C. I even used the dog picture. The talk was recorded, but unfortunately no longer seems to be available.

My hope in delivering this talk was partly that the people in the room would learn a little about problem solving, but also that they'd learn a lot about how an experienced person can say "here are the limits of my knowledge, I can't help you with that problem. At least, I can't yet, but it might be fun/interesting/remunerative to discover more about it." How it's OK to not know what you're doing, if you have a plan or can make one.

In retrospect, I think that what happened was simply that 2015 was too many generations in the software industry after all of the great forcing functions that led to the way computering was currently done. The Agile folks had worked out that we don't know what the customer will want at the end of the project, so we should optimise our work for not knowing, but they'd done that at the turn of the millennium. The dot bomb had exploded at the same time, so the Lean Startup folks had worked out that there's no money in the things the customer doesn't want and you have to very quickly discard all of those.

Everything had shifted left, but it had done so at least a decade earlier. Now those things, Agile and Lean Startup, were the way you did computering, and you could be expert in them. There was certification. They were no longer "because that thing before wasn't great", they were "because this is how we do it". There was another round of venture capitalists in town, and the money taps were starting to turn back on. There was no great need to find out that you were wrong, so it became a cultural taboo to admit it.

Anyway, if we believe DHH, I overshot. Apparently we went from "it's professional to own up to the limits of your knowledge" to "it's a badge of honour to not know programming as a programmer." To be honest I find that the weak part of the post, mostly because I don't recognise it and he doesn't supply evidence. The rest—that we are beings capable of learning and growth and we should not revel in ignorance—is the same as what I was trying to say with my dog-meme talk in 2015.

But now the dominant non-monoculture is the "if you're not with us you're against us" variant. The "come on internet, you know what to do" quote tweet. Saying that you may have things to learn, and should not still be at the same level of copy-and-paste code years into your job is now the same as saying you must memorise all algorithms and programming language quirks to call yourself a real programmer. And how very DARE he say that, what does he know about programmers anyway?

Discussions of the DHH post seem to be predicated on the idea that it's a personal attack on people who aren't DHH-level success, when if it's an attack at all it's attacking a straw man identity and in fact is worded more like this non-attack: you have more potential to live up to, find it in yourself to surpass your current limits. But scrape the surface (by showing that DHH didn't say the things that are claimed to be "ruining it for everyone"), and it seems there's a certain amount of hating the messenger, not the message, going on.

I'm not sure the cause of this, but I suspect it may be that having learned not to punch down, folks are looking up for targets. That DHH is successful, has said things that people don't like in the past, so it'll be OK to not like what he says this time. And the headline is something not to like, therefore the article must just expand on why I was correct not to read it.

I've certainly disagreed with DHH before. When he did the "TDD is dead" thing, I went into that from a position of disagreement. But I also knew that he has experience at being successful as a programmer, and will have reflections and knowledge that are beyond my understanding. So I listened to the discussions, and I learned what each of the people involved thought. It was an interesting, and educational experience. I gained a bit more of an idea of what I'm doing.

#### I just updated Appropriate Behaviour
The new release of Appropriate Behaviour—the book about things programmers should do that aren't programming—is now up. The most obvious, and most awesome, change in this update is a fabulous new cover, designed by Sebastian Hermida of leanpubcovers.com. Should you be in the market for a cover page, I'd strongly recommend him.

Other changes in this release include additions to the (ends of the) chapters on coding practices and learning, and I've added part of a new chapter on requirements engineering. As ever, discussion of the book is welcome in its glassboard, details of which are in the introduction.

I've found it really interesting in researching this book that I can go back decades and find information that has either been forgotten, or was seemingly ignored even at the time of publication. I think it's quite clear that there's a gulf between software as practiced by people who make software, and software as researched by academics; it's therefore not surprising to see journal articles that apparently never got read by commercial sector developers.

What is more interesting is the extent to which "mainstream" programming books, including ones that apparently made a big splash at the time of their publication, no longer seem relevant. They've either been completely dropped from our consciousness (hands up everyone who's read Peopleware in the last five years), or have been adapted into a one-sentence précis that's become part of the mythology of programming. A thought experiment by way of an example of this mythologising: quote any sentence from The Mythical Man Month except the one about adding people to a late project. What was the rest of the book about? Is anything else in it relevant to what we do today? Do we know that even that sentence is relevant, or does it just sound plausible?

I've been having lots of fun discovering these forgotten entries in our history and bringing some of them into a modern story about programming. But Appropriate Behaviour is not a history book; if anything, it's a book on social anthropology. The lesson to learn from this post is that it's not the first anthropological study of programmers; I'd argue that 1971's The Psychology of Computer Programming is more anthropology than it is psychology. It's very different from Appropriate Behaviour but they both tread the same ground, analysing the problems faced by a programmer that aren't directly related to telling a computer what to do.

I imagine the history book would be fun to write, though for the moment I present this, which I hope is also fun to read.

#### I made a web!
That is, I made a C program using the literate programming tool, CWEB. The product it outputs is, almost by definition, self-documenting, so find out about the algorithm and how I built it by reading the PDF. This post is about the process.

Unsurprisingly I found it much more mentally taxing to understand a prose description of a complex algorithm and how I might convert that into C than writing the C itself. In that, and acknowledging that this little project was a very artificial example, it was very helpful to be able to write long-form comments alongside the code.

That's not to say that I don't normally comment my code; I often do when I'm trying something I don't think I understand. But often I'll write out a prose description of what I'm trying to do in a notebook, or produce incredibly terse C comments. The literate programming environment encouraged me to marry these two ideas and create long prose that's worth reading, but attach it to the code I'm writing.

I additionally found it useful to be able to break up code into segments by idea rather than by function/class/method. If I think "oh, I'll need one of these" I can just start a new section, and then reference it in the place it'll get used. It inverts my usual process, which is to write out the code I think I'll need to do a task and then go back and pick out isolated sections with refactoring tools.

As a developer's tool, it's pretty neat too, though not perfect. The ctangle tool that generates the C source code inserts both comments referring to the section of the web that the code comes from, and (more usefully) preprocessor #line directives. If you debug the executable (which I needed to…) you'll get told where in the human-readable source the PC is sitting, not where in the generated C.

The source file, a "web" that contains alternating TeX and C code, is eminently readable (if you know both TeX and C, obviously) and plays well with version control. Because this example was a simple project, I defined everything in one file but CWEB can handle multiple-file projects.

The main issue is that it'd be much better to have an IDE that's capable of working with web files directly. A split-pane preview of the formatted documentation would be nice, and there are some good TeX word processors out there that would be a good starting point. Code completion, error detection and syntax highlighting in both the C and TeX parts would be really useful. Refactoring support would be…a challenge, but probably doable.

So my efforts with CWEB haven't exactly put me off, but do make me think that even three decades after being created it's not in a state to be a day-to-day developer environment. Now if only I knew someone with enough knowledge of the Clang API to think about making a C or ObjC IDE...

#### I make mistakes for a living
As a team lead, my job is to make all the mistakes on my team. I'm responsible for each one of them. I'm also responsible for deciding what we do about them, whether that's to ensure that they can't happen again or ignore them because they're easy to recover from.

That time someone on my team built a feature that didn't ship because of a bug? I could have made sure the requirements were clearer, or they had more access to the product owner, or a more useful design review, or a more useful code review, or a better test plan. Or all of those, or none of those, or something else. I could have - should have - got the problems out of their way.

They don't make mistakes, I do.

#### I may have not been correct
When I said Apple should buy Sun, whether that was a good idea or not, it seems to have failed to occur. Instead, we find that Oracle have done the necessary. Well, there goes my already-outdated SUNW tag. Presumably they'll keep Java (the licensing revenue is actually pretty good), MySQL (I've heard that Oracle make databases), the OS and a subset of the hardware gear. Then they'll become the all-in-one IT industry in a box vendors that Cisco have yet to organise, with (presumably x86) servers running the Solaris-Glassfish-Oracle-Java app stack in some insanely fast fashion. I wonder how many of the recent leftfield hardware projects they'll just jettison, and who will end up running the Santa Clara business unit...

#### I published a new book!
Executive summary: it's called APPropriate Behaviour, head over to the LeanPub site to check it out.

For quite a while, I've noticed that posts here are moving away from nuts and bolts code towards questions about evaluating my own performance, working with other developers and the industry in general.

I decided to spend some time working on these and related thoughts, trying to derive some consistent narrative as well as satisfying myself that these ideas were indeed going somewhere. I quickly ended up with about half of a novel-length book.

The other half is coming soon, but in the meantime the book is already published in preview state. To quote from the introduction:

this book is about the things that go into being a programmer that aren’t specifically the programming. It starts fairly close to home, with chapters on development tools, on supporting your own programming needs, and on other “software engineering” practices that programmers should understand and make use of. But by the end of the book we’ll be talking about psychology and metacognition — about understanding how you the programmer function and how to improve that functioning.

As I said, this is currently in very much a preview state—only about half of the content is there, it hasn't been reviewed, and the thread that runs through it has dropped a few stitches. However, even if you buy the book now you'll get free updates forever so you'll get to find out as chapters are added and as changes are made.

At this early stage I'm particularly interested in any feedback readers have. I've set up a Glassboard for the book—in the Glassboard app, use invite code XVSSV to join the discussion.

I hope you enjoy APPropriate behaviour!

#### I use mocks and I'm happy with that
Both Kent Beck and Martin Fowler have said that they don't use mock objects in their test-driven development. I do. I use them mostly for the sense described first in my BNR blog post on Mock Objects, namely to stand in for a thing that can receive messages I want to send, but that does not yet exist.

If you look at the code in Test-Driven iOS Development, you'll find that it uses plenty of test doubles but none of them is a mock object. What has changed in my worldview to move from not-mocking to mocking in that time?

The key information that gave me the insight was this message, pardon the pun! from Alan Kay on object-oriented programming:

  The big idea is "messaging" - that is what the kernal[sic] of Smalltalk/Squeak
   is all about (and it's something that was never quite completed in our
   Xerox PARC phase). The Japanese have a small word - ma - for "that which
   is in between" - perhaps the nearest English equivalent is "interstitial".
   The key in making great and growable systems is much more to design how its
   modules communicate rather than what their internal properties and
   behaviors should be.

What I'm really trying to do is to define the network of objects connected by message sending, but the tool I have makes me think about objects and what they're doing. To me, mock objects are the ability to subvert the tool, and force it to let me focus on the ma.

#### I was wrong, 80 characters is fine
Do you remember when I said, in my 2019 post Why 80?, that “80 characters per line is a particular convention out of many that we know literally nothing about the benefit or cost of, even today”?

Today I saw a “rule of 66 non-white space characters per line” mentioned in connection with TeX. I couldn't find that reference in the TeXbook, though it's also in Bringhurst's “The Elements of Typographic Style” so let's go with it for the moment.

If there should be 66 non-white space characters per line, then a line should be 66 * (average word length + 1) / (average word length) characters long to hold 66 non-white space characters, on average, if it's displaying a run of words. In English, the average word length is about five. That gives us 79.2 characters per line.

If you're reading English, an 80 column terminal makes sense, if Bringhurst's justification (pardon the pun) is valid. Though I still don't know why people suggest 72 characters for commit message lines.

#### I'm probably holding it wrong
If I wanted to do a table view data source in ObjC, it would look like this:

- tableView:aTableView objectValueForTableColumn:aColumn row:(NSInteger)row {
  return [representedObject.collection[row] valueForKey:[aColumn identifier]];
}

When I do it in Swift, it ends up looking like this:

func tableView(_ tableView: NSTableView, objectValueFor tableColumn: NSTableColumn?, row: Int) -&gt; Any? {
    guard let identifier = tableColumn?.identifier else {
        assertionFailure("No table column")
        return nil
    }
    guard let obj = (self.representedObject as? ModelType)?.collection(at:row) else {
        assertionFailure("Can't find model object at \(row)")
        return nil
    }
    switch identifier {
    case NSUserInterfaceItemIdentifier(rawValue:"column1"):
        return obj.field1
    case NSUserInterfaceItemIdentifier(rawValue:"column2"):
        return objc.field2
    //...
    default:
        assertionFailure("Unknown table column \(tableColumn?.identifier ?? NSUserInterfaceItemIdentifier(rawValue: "unknown"))")
        return nil
    }
}

I can't help feeling I'm doing it wrong.

#### I'm sorry, I haven't a Clu
One of my many "repeat-until-funny" jokes, anyway here is what I have to say on Mr. Cluley's blog regarding iPh0wn.

#### I've vastly misunderstood the Single Responsibility Principle
So have a lot of other people; my understanding of it accords with much writing I've seen about the principle on the web, and elsewhere. For example, in Michael Feathers' "Working Effectively with Legacy Code":

Every class should have a single responsibility: It should have a single purpose in the system, and there should be only one reason to change it.
Michael Feathers, Working Effectively with Legacy Code (p.246)

I came to question that understanding today when I read Parnas's article On the Criteria to be Used in Decomposing Systems into Modules. His top criterion is decision hiding: everywhere you make a choice about the system implementation, put that choice into a module with an interface that doesn't expose the choice made.

If you combine that principle with my incorrect SRP ("one reason to change per module"), you get the absurd situation that each module may contain only one change point. In other words, each bit of software architecture information must exist in a separate module.

So, I went back to my understanding of the SRP. I found that it was flawed, and that the person who coined the phrase Single Responsibility Principle (Robert Martin) even said as much. He said it a long time ago, but years after the incorrect interpretation had got its shoes on and run twice around the world.

When you write a software module, you want to make sure that when changes are requested, those changes can only originate from a single person, or rather, a single tightly coupled group of people representing a single narrowly defined business function.
Robert Martin, The Single Responsibility Principle

So the principle is that the module's behaviour is the responsibility of a single actor. It's not that the module has a single reason to change, but that a single entity will request the changes. This is much easier to resolve alongside Parnas' version of modularity.

This isn't some new deep revelation or hidden insight, by the way. That post by Martin is referenced on the wikipedia entry for SRP, which states the true (well, the as-given) definition of the principle. The fact that I and so many others can hold a completely different view, for so long, in the face of such obvious contradictory evidence, tells us something about knowledge transfer in software engineering that we probably ought to attend to.

#### Ich habe mein Handy verloren
Ooops.  Left my mobile phone on holiday in Germany when I came back.  This may explain why some people who were expecting to hear from me in Germany didn't...anyway, could anyone who thinks I ought to know their phone number please either leave a private comment on this livejournal entry (all comments are private by default on that post), or e-mail me?  Thankyou!  I promise to buy a filofax which I won't leave in foreign parts.

#### if ([self isKindOfClass:[whore class]]) [self promote];
Bottom of the page, under Media Reviews.  I've got another couple of review copies sat here, too...

#### If Object-Oriented Programming were announced today
Here's an idea: the current backlash against OOP is actually because people aren't doing OOP, they're doing whatever they were doing before OOP. But they're calling it OOP, because the people who were promoting OOP wanted them to believe that they were already doing OOP.

Why is that? Because the people who were promoting OOP wanted to sell their things. They were doing this in the 1980s to 1990s, when you could still expect developers to spend thousands of dollars on tools and libraries. "Here's a thing that's completely unlike what you're already doing" is not as good a sales pitch as "ride the latest wave without boiling the ocean". Object-Oriented principles were then hidden by the "Object Technology" companies - the StepStones, NeXTs, OTIs, OMGs - who wanted to make OOP accessible in order to sell their Object Technology.

That's not the world of 2017. Nowadays, developer environments, libraries, deployment platforms etc are largely open source and free or very cheap, so rather than make an idea seem accessible, people try to make it seem important. We see that with the current wave (third wave? I'm not sure) of functional programming evangelism, where people are definitely trying to show that they are "functionaller than thou" rather than that you already know this stuff. Throw up a github or an npm that uses monad, pointfree or homoiconic without any indication of shame, and you're functionalling right. Demonstrating that it's already possible to curry methods in Objective-C is not the right way to functional.

If OOP were introduced into this world, you'd probably have a more dogmatic, "purer" representation of it than OOP as popularly practised today. You might even have OOP in Java.

One thing that makes me think that is that, well, it's happened and it's true. Multiple times. As previously explored here, protocol-oriented programming is just polymorphism under another name, and polymorphism will be found in a big-letters heading in any OOP book (in Barbara Liskov's "Program Development in Java", it's chapter 8, "Polymorphic Abstractions". In Bertrand Meyer's "Touch of Class", section 16.2, "Polymorphism".

Similarly, what are microservices other than independent programs that maintain their own data and the operations on those data, performing those operations in response to receiving messages? What is a router at the front of a microservice but the message dispatch handler, picking a method implementation by examining the content of a selector?

#### Illuminative-C
In addition to being a mildly accomplished software engineer, I've done some studying and armchair research in the field of ancient languages and palaeography. What happens if we smoosh those fields together?

In a very slight way, art historian and fellow Oxenafordisc Dr. Janina Ramirez did that in her series on Illuminations: the Private Lives of Medieval Kings (erm, Kings and Ælfgifu). In the series she showed off many manuscripts in the British Library collection, but when she went out in the field she took an iPad. It turns out that the BL isn't too hot on letting you run around with their thousand-year-old kidskin.

You already know my opinion on our digital heritage. This puts it into stark relief: in one hundred years' time, barring some epic fire in London (those never happen), the BL and its collection will still be there. Will it still be possible to even launch the iPad app she was using? I very much doubt it.

How about if we put the same effort into storing our source code as the scriptoria did into storing their indentures and gospels? Well, I sharpened a goose feather and had a go at just that (warning: very much draft document impending).

What you see up there is the first sample code in Professional Cocoa Application Security - Listing 2-1. Ignore the fact that you don't recognise all the letter shapes: things have changed over the centuries. There were a few contortions required to get the source code to work in manuscript form: let me show you them.

First is that in the hand in which I wrote the source, some of the characters needed for Objective-C source code don't exist. Like 'v'. I used the fact that u and v are actually the same letter to get around that. Punctuation was harder: I went for roughly accurate rendering, with a single misplaced comma to suggest that the scribe didn't really understand punctuation.

When it came to comments, I decided they have the same meaning as the gloss in the Lindisfarne Gospels - rendering the difficult language required by the church^Wcompiler into plain English. I therefore roughly scratched them in smaller text with different ink, letting them flow around the code as if they'd been written later. I also put in a few Old English spellings - though again not consistently[*].

The return value posed some difficulty, because we didn't borrow 0 from the Middle East until a few centuries after the time this script is mimicking. I realised that if a scribe were to illuminate any part of a C function, it'd probably be the return value because that's the consistent and - from the perspective of the rest of the code - important part. Thus the 0 is highly decorated, with six legs in the fashion of a bug :-). 

Bugs hark back to the days of illuminated manuscripts anyway. Any good scribe would know that a mistake in the text was the fault of Titivillus, not of the scribe. Just as those bugs aren't my fault. Honest.

[*] Next time you want to get angry at a teenager, remember that the work "ask" was once "acsian" with the s on the end, and think about which one of you is bastardising our language.

#### Image
I love my Testsphere deck, from Ministry of Testing. I've twice seen Riskstorming in action, and the first time that I took part I bought a deck of these cards as soon as I got back to my desk.

I'm not really a tester, though I have really been a tester in the past. I still fall into the trap of thinking that I set out to make this thing do a thing, I have made it do a thing, therefore I am done. I'm painfully aware when metacognating that I am definitely not done at that point, but back "in the zone" I get carried away by success.

One of the reasons I got interested in Design by Contract was the false sense of "done" I feel when TDDing. I thought of a test that this thing works. I made it pass the test. Therefore this thing works? Well, no: how can I keep the same workflow, and speed of progress but improve the confidence in the statement?

The Testsphere cards are like a collection of mnemonics for testers, and for people who otherwise find themselves wondering whether this software really works. Sometimes I cut the deck, look at the card I've found, and think about what it means for my software. It might make me think about new ways to test the code. It might make me think about criticising the design. It might make me question the whole approach I'm taking. This is all good: I need these cues.

I just cut the deck and found the "Image" card, which is in the Heuristics section of the deck. It says that it's a consistency heuristic:

  Is your product true to the image and reputation you or your app's company wishes to project?

That's really interesting. How would I test for that? OK, I need to know what success is, which means I need to know "the image and reputation [we wish] to project". That sounds very much like a marketing thing. Back when I ran the mobile track at QCon London, Jaimee Newberry gave a great talk about finding the voice for your product. She suggested identifying a celebrity whose personality embodies the values you want to project, then thinking about your interactions with your customers as if that personality were speaking to them.

It also sounds like there's a significant user or customer experience part to this definition. Maybe marketing can tell me what voice, tone, or image we want to suggest to our customers, but what does it mean to say that a touchscreen interface works like Lady Gaga? Is that swipe gesture the correct amount of quirky, unexpected, subversive, yet still accessible? Do the features we have built shout "Poker Face"?

We'll be looking at user interface design, too. Graphic design. Sound design. Copyediting. The frequency of posts on the email list, and the level of engagement needed. Pricing, too: it's no good the brochure projecting Fortnum &amp; Mason if the menu says Five Guys.

This doesn't seem like something I'm going to get from red to green in a few minutes in Emacs. And it's one of a hundred cards.

#### Immutable changes
The Fixed-Term Parliaments Act was supposed to bring about a culture change in the parliament and politics of the United Kingdom. Moving for the second reading of the bill that became this Act, Nick Clegg (then deputy prime minister, now member for Facebook Central) summarized that culture shift.

  The Bill has a single, clear purpose: to introduce fixed-term Parliaments to the United Kingdom to remove the right of a Prime Minister to seek the Dissolution of Parliament for pure political gain. This simple constitutional innovation will none the less have a profound effect because for the first time in our history the timing of general elections will not be a plaything of Governments. There will be no more feverish speculation over the date of the next election, distracting politicians from getting on with running the country. Instead everyone will know how long a Parliament can be expected to last, bringing much greater stability to our political system. Crucially, if, for some reason, there is a need for Parliament to dissolve early, that will be up to the House of Commons to decide. Everyone knows the damage that is done when a Prime Minister dithers and hesitates over the election date, keeping the country guessing. We were subjected to that pantomime in 2007. All that happens is that the political parties end up in perpetual campaign mode, making it very difficult for Parliament to function effectively. The only way to stop that ever happening again is by the reforms contained in the Bill.
  
  As we hammer out the detail of these reforms, I hope that we are all able to keep sight of the considerable consensus that already exists on the introduction of fixed-term Parliaments. They were in my party's manifesto, they have been in Labour party manifestos since 1992, and although this was not an explicit Conservative election pledge, the Conservative manifesto did include a commitment to making the use of the royal prerogative subject to greater democratic control, ensuring that Parliament is properly involved in all big, national decisions—and there are few as big as the lifetime of Parliament and the frequency of general elections.

When a parliament is convened, the date of the next general election automatically gets scheduled for the first Thursday in May, five years out. The Commons could vote, with a qualified majority, to hold an election earlier, or an election would automatically be triggered if the government lost a no-confidence vote, but the prime minister cannot unilaterally declare an election date to suit their popularity with the franchise.

Observed behaviour shows that the Act has been followed to the letter, up to the current dissolution which required a specific change to the rules. Has the spirit of the Act, the motivation presented above, survived intact? The dates of elections since the Act passed were:

7 May 2015, the first Thursday in May at the end of a five-ish-year Parliament, chosen to bring the existing behaviour into sync with the planned behaviour.
8 June 2017, after a qualified majority vote within the terms of the Act.
12 December 2019, after the aforementioned Early Parliamentary General Election Act.

The reason for the disparity is that the intended goal—a predictable release schedule that makes it easier for everyone involved to prepare—doesn't match the cultural drivers. The desire to release when we're ready, and have the features that we want to see, remains immutable, and means that even though we've adopted the new rules, we aren't really playing by them.

I was tempted to hit "publish" at this point and leave the software engineering analogy unspoken. I powered on: here are a few examples I've seen where the rule changes have been imposed but the cultural support for the new rules hasn't been nurtured.

Regular releases, but the release is "internal only" or completely unreleased until all of the planned features are ready;
Short sprints, where everything that has gone from development into QA is declared "done";
Sprint commitments, where the team also describe "stretch goals" that are expected to be delivered;
Sustainable pace, where the "velocity" is expected to increase monotonically;
Self-organizing teams, where the manager feeds back on everybody's status update at the daily stand-up;
Continuous integration, where the team can disable or skip tests that fail.

All of these can be achieved without the attached sabotage, but that requires more radical changes than adding a practice to the team's menu. Radical, because you have to go to the root of why you're doing what you do. Ask what you're even trying to achieve by having a software team working on your software, then compare how well your existing practice and your proposed practice support that value. If the proposed practice is better, then adopt it, but there's going to be a transition period where you continually explain why you're adopting it, show the results, and (constructively, politely, and firmly) guide people toward acceptance of and commitment to the new practice. Otherwise you end up with a new fixed-term parliament starting whenever people feel like it.

#### Imperative Programming in Swift
A cliche in programming is that certain ways of writing programs make it possible to "reason about" code. So it should be possible to form an argument that proceeds from some axioms to a conclusion about the code we're looking at via some logical (or otherwise defensible) steps.

Looking at the declaration of this function, f:

func f&lt;T&gt;(xs:[T]) -&gt; T

Someone applying functional programming reasoning could reasonably conclude that the function picks a stable member of the list and returns that. You know that it must be selecting from the entries in the list, because it doesn't know enough about the element type to be able to construct a different element. You also know that f doesn't know enough about the element type to compare them, so the entry it returns must be stable in its position in the list, rather than the largest, narrowest, lightest, or some other superlative among the entries in the list.

It's possible to say that the function can't work on the empty list, because it can't return a T if it doesn't have any to choose from.

So, was our hypothetical functional programmer correct? Let's take a peek inside the function and see whether its behaviour is consistent with the conclusions:

func f&lt;T&gt;(xs:[T]) -&gt; T {
  var error:NSError?
  let i = str.writeToFile("/tmp/foo", atomically: false, encoding: NSUTF8StringEncoding, error: &amp;error) ? 0:1
  let j = NSFileManager.defaultManager().removeItemAtPath("/mach_kernel", error: &amp;error) ? 1:2
  var k = 0
  errno = 37
  if let error = error {
    k += error.code
  }
  let l = Int(drand48()*3)
  return xs[i+j+k-l]
}

They were correct in every respect! Except the one about the right answer, that didn't go so well. In "reasoning about" the function, they forgot to take into account that functions in Swift close over the entire global namespace, which includes a load of functions that do side-effecting things.

Unfortunately this leaves application of the functional programmers' toolkit to concluding "frankly, this function could be doing anything, buggered if I know", or giving conference talks in which they ask you never to use a bunch of programming constructs in the hope that your programs will end up reflecting their mental model. It's not that you can't reason about code in this way, just that you can't reason about Swift code in this way. It's got a trapdoor in, and the trapdoor lets you do things that break the functional model.

That doesn't mean throwing out reason completely, though. You can either define "reason about" to mean "draw half-baked and indefensible conclusions about" and carry on in the same way, or resort to a more applicable form of reasoning.

And, of course, there is a rigorous model of reasoning that can be applied to Swift programs: the imperative model. Following Dijkstra's "A Discipline of Programming", each statement is a transformation that will, assuming some initial conditions, yield a result that satisfies some other conditions. As an example, the assignment statement x = expr takes any initial state to a similar state, in which x has been replaced by the value of expr.

A program is a sequence of those transformations. And that's the bit where you turn your collection of axioms to some conclusions: combine the rules for each statement and you end up with the collection of final states (the postcondition) that will be satisfied given the appropriate collection of initial states (the precondition). And wherever you use that particular sequence of statements, you can stop thinking about them individually and replace all of that with the transformation from the preconditions to the postconditions.

It turns out it's not how you think about your program that makes it work. It's doing the thinking. Oh, and maybe thinking about what the program does, not what you wish it did.

#### Impossibility and Uncertainty in AI
About this paper

Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function), Peter Eckersley. Submitted to the ArXiV on December 31, 2018.

Notes

Ethical considerations in artificial intelligence applications have arguably been present since the birth of the field, if not earlier. Karel &#268;apek wrote R.U.R., the play that introduced both the word "robot" to our lexicon, and the question of whether such creations would want to work. Isaac Asimov and Mary Shelley both asked how a society with "natural" and "artificial" people would function.

Getting Trollied

More recently, as we've been able to create things that we can both identify as solving problems and market under the term Artificial Intelligence, the question of AI ethics has reappeared with different specialisations. Should we apply AI techniques to particular fields of endeavour at all, such as autonomous weaponry? Who should be responsible for a decision made, for an example, by an autonomous vehicle? How should such a vehicle make a decision when faced by the classic trolley problem?

Eckersley opens this paper with an aside, important to those who consider AI as an engineering discipline (which is not a clear statement in itself): autonomous vehicle research is a long, long way away from even contemplating the trolley problem. Currently, reducing the uncertainty with which obstacles are detected and identified from noisy input signals is a much more pressing issue.

However, human drivers are better at object identification, so may be sufficiently advanced to address the trolley problem should it arise in piloting a vehicle. And this is really an important point in thinking philosophically about how AIs should act: we already have intelligent actors, and already think about how they act.

Utilitarianism

Indeed, we already have artificial intelligences, including society-level AIs. A government bureaucracy; an enterprise; a policy document; all of these are abstractions of human intelligence behind machines, rules, and assumptions of how to work in given contexts. Any time that someone relies on external input to make a decision, whether that input is a checklist, a company handbook, a ready reckoner tool or a complex software-built decision support tool, you could replace that tool with an AI system with no change in the ethics of the situation.

Anywhere you could consider an AI making choices that are suboptimal to the welfare of the affected people, you could imagine those choices being suggested by the checklist or ready reckoner. None of this is to minimise the impact or applicability of AI ethics, quite the contrary: if we are only considering these problems now because we think AI has become useful, we are late to the party. Very late: what was the Soviet Gosplan other than a big decision support system trying to optimise a society along utilitarian lines?

The early sections of this paper summarise earlier proofs that it is impossible to define an objective utility function that will optimise for multiple, conflicting output dimensions. Not because it is hard to discover or compute the function, but because an acceptable trade-off may not exist. The proofs take this form:

imagine a guess at a solution, s.
there is a rule r1 which, applied to s, yields a better solution, s1.
there is a rule r2 which, applied to s1, yields a better solution, s2.
...
there is a rule rN which, applied to s(n-1), yields a better solution, s.

In other words, the different states that represent solutions cannot be "totally ordered" from "best" to "worst"; a cycle exists such that the question "which is the best" is paradoxical. While that has clear impact on AIs that use objective utility functions to choose solutions to their inputs, the wider implication is that no utilitarian system can objectively choose the "best" arrangement for welfare.

If you're building the multivac from Isaac Asimov's Franchise, then I'm going to have to disappoint you now: it's impossible. But so is having a room full of people with slide rules and 383 different folders containing next year's production targets. It's not building an AI to optimise for the utility function that cannot be done; it's finding a maximum on the utility function.

Finding a Workaround

A few techniques for addressing this problem are discussed and rejected: we could ignore the problem on the basis that our AIs are not currently important enough to have huge impact on social welfare. But AI is being used in welfare systems, in criminal justice systems, in financial management. And even where AI is not used, corporations and bureaucracies are already being applied, and these will have their decision support tools, which will eventually include AI.

Similarly, simplifying the problem is possible, but undesirable. You could define some kind of moral exchange rate to reduce the dimensionality of your problem: this much wealth in retirement is worth this much poverty today; this much social equality can be traded for this much value of statistical life. Or you could accept one or more of the undesirable properties of a solution as axiomatic; we can solve for inequality if we accept poverty, perhaps. Neither of these are ethically unambiguous.

Embracing Uncertainty

Ultimately, Eckersley concludes that the only feasible approach is to relax the requirement for objective, total ordering of the possible outcomes. One way is to introduce partial ordering of at least two of the comparison dimensions: to go from saying that a solution is { better than, worse than, the same as } another along some axis to saying it is { better than, worse than, the same as, incomparable to } the other solution. While this breaks the deadlock it's difficult to work with, and domain-specific interpretations of partial ordering will be needed. And now we're back at the trolley problem: there are multiple outcomes to choose from, we don't know (because it's impossible to say) which is better, and yet a decision must be taken. Which?

A second model replaces the definite ordering of solutions with probabilistic ordering. Rather than the impossible statement "outcome B is better than outcome A", the pair of statements "outcome B has probability p of being better than outcome A" and "outcome A has probability (1-p) of being better than outcome B" are given. Now a system can find the outcome with the highest likelihood of being best, or pick from some with sufficient likelihood of being best, even though it is still impossible to find the best. There will always be some chance of violating a constraint, but those should at least be free of systematic bias.

Implications

If your AI strategy is based on the idea that your domain can be "solved numerically", by finding some utility function or by training an AI to approximate the utility function, then you need to ask yourself whether this is true without introducing some biases or unethical choices into the solutions. Consider whether your system should find what is "best" and do it, or find actions that are "good enough" and either choose between them, or present them for human supervision.

#### Improving a presentation with slides
Take a look at your slides. For each slide, think how you would present the same information if you didn't have the slide. Practise that, so that you can give the information on the slide without using the slide as an aide memoire. Practise that, until you can introduce that topic, discuss it, and move on to the next without a single reference to the slide. Do the same for each slide.

  How will that improve my slides?

It won't. It will improve your presentation with slides, by turning it into a presentation without slides.

As an optional extra, you could make new slides that support the presentation, but it shouldn't be necessary.

#### In defence of assertions
The year is 2017 and people are still recommending processing out assertions from release builds.

many assertions are short tests (whether or not that's a good thing): this variable now has a value, this number is now greater than zero), which won't cost a lot in production. Or at least, let me phrase this another way: many assertions are too cheap to affect performance metrics in many apps. Or, let me phrase that another way: most production software probably doesn't have good enough performance monitoring to see a result, or constrained enough performance goals to care about the result.
The program counter has absolutely no business executing the instruction that follows a failed assertion, because the programmer wrote the subsequent instructions with the assumption that this would never happen. Yes, your program will terminate, leading to a 500 error/unfortunate stop dialog/guru meditation screen/other thing, but the alternative is to run...something that apparently shouldn't ever occur. Far better to stop at the point of problem detection, than to try to re-detect it based on a surprising and unsupportive problem report later.
assertions are things that programmers believe to always hold, and it's sensible to take issue with the words always and believe. There's an argument that goes:

I have never seen this situation happen in development or staging.
I got this job by reversing a linked list on a whiteboard.
Therefore, this situation cannot happen in production.

but unfortunately, there's a flaw between the axioms and the conclusion. For example, I have seen the argument "items are added to this list as they are received, therefore these items are in chronological order" multiple times, and have seen items in another order just as often. Assertions that never fire on programmer input give false assurance.

#### In defence of large teams
Seen on the twitters:

1) Bad reasons why tech startups have incredibly large mobile teams even though from an engineering perspective they don’t need it.
This is the No True Scotsman fallacy, as no true software department needs more than, say, 20 people.

I'm not going to get into the details of what you do with hundreds of mobile engineers. Suffice it to say that the larger-team apps I've worked on have been very feature rich, for better or worse. And that's not just in terms of things you can do, but in terms of how well you can do them. When you in-source the small details that are important to your experience, they become as much work to solve as the overall picture.

Make a list of the companies that you think have "too big" a mobile software development team. Now review that list: all of those companies are pretty big and successful, aren't they? Maybe big enough to hire a few hundred developers to work on how their customers access their products or services? No true software department needs to be that successful.

And that's what I think of as the underlying problem with the "your team's too big, you're doing it wrong" fallacy: it's part of the ongoing narrative to devalue all software. It says that your application can't possibly be worth enough to spend all that developer time on. After all, mine isn't, and I'm a true software developer.

#### In defense of `id`
Something you can't see about my dotSwift talk on OOP in FP in Swift is that to make the conference more interesting while the AV was set up for the next speaker, Daniel Steinberg invited me over to a side table for a question and answer session. He had some great questions and I had some adequate answers; that is now lost to time.

One thing he asked was about how I do things differently when I'm working in Objective-C and in Swift, and I mentioned that I tend not to use the type system in ObjC and just call all of my variables id unless they are C types or the compiler asks otherwise. You can see an example of this in my UIKonf 1995 talk.

I argue (back in 2018, at dotSwift, in the bit that was videoed) that all objects have the same type. Just as I can define the "type" Set through its function signature:

typealias Set&lt;T&gt; = (T) -&gt; Bool

so I can define the "type" object through its function signature. An object - any object - is a function that responds to messages by turning selectors into methods:

typealias Object = (Selector) -&gt; IMP

Now if all objects have the same type, why would I want to use different types for different objects?

Of course, there are reasons to want to refine the definition of an object from "any object" to "an object like this", but these refinements are inaccessible using Objective-C's type system (or Java's, or Swift's, or most other programming languages). Any object responds to any message, but for the most part they respond by doing whatever the default error-raising behaviour is which is not particularly interesting and doesn't solve your customer's problem. So what we want to be able to say is "this object is one that responds to a particular collection of messages in a way that is what I need here".

We have two tools available to us, and neither gives us an answer to that question. The first is the protocol (or Java interface): we can say "this object is one that has implementations of methods for a particular collection of messages". That's not the same as the question we want to answer - it says nothing about whether the object responds in the ways we want. It's also not generally the correct answer - an object that has the methods we want and behaves in the expected way but that didn't have the protocol conformance recorded at compile time (even if it conformsToProtocol: at run time) does not satisfy the compiler type check for protocol conformance.

Even less generally useful is using the class name as the type. Classes are ways to say "here is a collection of objects that all have common behaviour and data"; well for starters I don't care about the data, but also just because those objects have the properties I want, doesn't mean that others outside that place in the inheritance tree don't.

I also can't rely on subtypes behaving as I need, but the compiler type checker will pretend that if I asked for an instance of one class, and got an instance of a subclass, then I got the thing I wanted. Inheritance, in many languages including Objective-C, Java and similar, is a way to borrow behaviour from one class in another class. If we also want refinement from inheritance we have to add a bunch of rules that almost every programming language not named after the designer of the Garabit Viaduct does not support.

So there are three imprecise ways to ask for objects by behaviour in Objective-C, and I choose the one with the least typing. Even once you amortise the cost of this blog post.

#### In which a life re-emerges
While it's far from finished, my PhD thesis is now complete: there are no to-do items left, no empty sections, no placeholders. Now the proof-reading, editing and corrections continue in earnest.

I look forward to poking my head out of that rabbit hole too, and finding out what else has been happening in the world since October 2020.

#### In which GNUstep confuses and ultimately disappoints
I'm not the most hardcore of GNUstep people, but I'm certainly somewhat invested. I've been building apps, lurking in lists, and contributing code on and off for around 13 years, including a job working with a few of the maintainers. I am trying to build the corpus of documentation intrinsic to GNUstep, i.e. that which doesn't require interpretating Apple's Cocoa docs through a compatibility lens.

So when I discovered that an app I need doesn't exist and that I'd have to make it myself, GNUstep was the obvious choice. I could take all that experience and investment in Objective-C and Cocoa and build the app I need. I'd probably be able to contribute to the framework along the way, too: a previous side project of mine resulted in adding NSUUID to gnustep-base.

This time, however, the amount of framework support needed proved too great. The thing I want is an app to help me organise and read some PDFs, and there's no PDF-reading support in GNUstep. Well, let me be more specific: there has been, but it's disappeared. There was a library called PopplerKit that wraps the poppler PDF-rendering library. However the only references to that are to a subversion repo in the now-defunct Gna! project: i.e. there is no PopplerKit available any more.

One option I considered is to write my own poppler wrapper, perhaps exposing the same interface as Apple's PDFKit. This would probably be welcome in the GNUstep community, would improve their libraries' API compatibility with Cocoa, and be useful to other developers. However, it's also a distraction from what I'm trying to do, which is to make my PDF app.

So I decided to choose a different technology, where PDF rendering is already available. Poppler has a Qt5 interface already, so I built my app using C++ and Qt5. After half an hour of effort I already have a page-turning PDF viewer, which is not pleasant but enough to let me be confident that I can build the rest of the application.

I had thought that this would be the end of the story. I would have a moral here (still a useful moral) about picking the technology that lets you get your task solved, being realistic about how much you're drawn to your pet project or favourite "stack" and whether that bias is worth tipping the balance: in many cases, it probably is, but be aware of what it's costing you. I expected to talk about code rot, and to despair over the fact that a library I knew about and was aware could solve my problem no longer existed.

Then, in preparing this post, I happened to dive through an unrelated repository on github, the GNUstep Applications Project (a repository containing a collection of unrelated applications), and discovered that it contains a PDFKit implementation. It doesn't contain PDFView so can't be used as a reader, so it doesn't change the choice over which of GNUstep or Qt should be used now. But it does change the amount of effort required to reimplement PDFKit on top of GNUstep, should I revisit that in the future.

So there's a point in here about discoverability too. I did quite a lot of searching to find that there isn't a PopplerKit, and also found that there wasn't a GNUstep PDFKit. Even so, there was, and even more searching turned it up. En route I found that a different technology had what I wanted, and changed my choice of programming language and application framework in order to use that technology in my project. You can't merely build it and expect them to come, you have to show them that it's there too.

#### In which I interview so you don't have to
Describing job interviews for technical roles in the software industry to people who have left or have always been outside the software industry requires two things: patience on the part of the one doing the describing, and the ability for the listener to take a joke. Over the last twelve years I have taken countless job interviews so that you don't have to. Here's what I've found: presented as a guide to running the average software developer interview. As with all descriptions of mediocrity, you should treat this as best practice.

[Be clear on this: not all interviews are like this. But this is an expectable baseline, derived from experience.]

Person Specification

The ideal candidate will be rich. We're going to put them through hours - maybe even days - of tests, interviews, meetings, and "informal chats" that they'd better be on best behaviour for anyway. They need to be able to afford taking that time away from work, friends, other opportunities, so they'd better be rich.

That multiple-hour interview process means that they'd better be desperate for a job too. As you'll find out in the section on our process, we pride ourselves on not giving away too much. We're not selling our company to you, because we know we're offering the chance to do what you've always wanted: sit in our open plan office space next to our own particular loud crisp-eater muttering at Eclipse.

The ability to go without food is desirable too. Even if a stage of the interview is planned to take so long that it would go over lunch, and even though we might put a break for lunch in, we might also forget to do any catering. Computers don't need food and programmers are sort of like computers, we heard. We actually occasionally do feed our staff, and advertise this as a perk.

Our Process

The first thing we want to check is whether you can solve logical problems. We don't actually need you to solve logical problems, after all, that's what the computers are for. But we'll give you an aptitude/basic reasoning test anyway [yes, although it's no longer the 1960s and we aren't IBM, this is still common if not universal].

The reasoning test is there to weed out people who didn't have the same education as us, or were raised speaking a different language, or in a different culture. Empathy is hard, and to avoid unduly stressing our staff we want to make sure that their colleagues are as similar to them as possible. Additionally the hour you'll take going through this test is an hour we don't have to make eye contact or conversation with you: empathy is hard.

To be honest we have no idea what this test means or how to interpret its results. Everybody before you went through this test, and they'd raise merry hell if we "lowered the bar" by removing it now. As a holacracy/meritocracy/hypocrisy/this week's organisational behaviour buzzword, we empower our employees to not see any changes that might raise a small amount of discomfort.

So after that test, depending on the seniority of the position and the candidate's experience, we'll...no, not really. We did nearly keep a straight face through that sentence though. In fact we didn't read your CV except to find out whether the keywords that describe the problems we have right now and the solutions we have chosen last week appear. We didn't read your GitHub/Lanyrd/Bitbucket profiles either, except to check that you have them so we know how much free work to expect out of you in addition to the paid stuff. Our project management system works on the Pareto Principle: 80 hours a week on our stuff, 20 hours a week on open source stuff that we can co-opt.

The next stage in the process is actually the same for everybody: a basic programming test to find out whether you even know what a computer is. We don't care that you're [glances at CV] Grace Hopper, we still don't believe that you can reverse a linked list. None of our employees has ever had to reverse a linked list on the job, and we'd fire them if they did reverse a linked list on the job because there are libraries for that.

Now we'll come onto the technical interview: a cross-examination by a panel of between one and twelve [not joking] people who have, or have had, a word like "engineer" in their job description at some point. These people are tasked with finding out whether you've solved the same problems in your career as they have in theirs. If you haven't, you might not be clever enough. If you have, then what new experiences are you bringing to the table?

By the way, our flexibility on your technical skills will go down as you become more experienced. We appreciate that new grads might not have used our tools/frameworks/technology and are willing to train them, but if you have more than six months' experience with Java we're going to call you a Java developer and only consider you for Java roles.

After all of that, it's still possible that you might have somehow snuck through the system despite not going to the same university or belonging to the same society as the founder. We can't really quantify the idea of "culture fit" but that's what we're examining in the next part of the process and we'll know it when we don't see it.

The Offer

You'll get a phone call from us while you're in the bath. We'll outline the position, pay and (unless this is an American company and there isn't any) holiday provision. You then have two seconds in which to reply, with either "Yes" or whatever the other one is. You may have other irons in the fire but of course you'll want to drop all of those when we tell you about the parking space we've already allocated for you [This has happened. I don't have a car.].

The Job

You will be working with a team of people who all went through that same interview and decided they wanted to work in our environment. We will leave it to you to decide what that means.

The Alternative

There are some less...scientific...approaches to hiring that involve using the candidate's stated and visible experience to have a conversation about what they've done, how they do and don't like to work, how they've responded to success and failure, and whether the challenges they would like to see in their career match up with the environment we're able to provide. While that sounds like quite a pleasant experience for everybody involved we fail to see how it could possibly translate into discovering whether we want to work with you or vice versa.

#### In which I misunderstood Objective-C
I was having a think about this short history of Objective-C, and it occurred to me that perhaps I had been thinking about ObjC wrong. Now, I realise that by thinking about ObjC at all I mark myself out as a bit of an oddball, but I do it a lot. I co-host the [objc retain]; stream with Steven Baker, discussing cross-platform free software Objective-C every week. Hell of a time to realise I've been doing it wrong.

My current thinking is that the idea of ObjC is not to write "apps" in ObjC, or even in successor languages (sorry, fans of successor languages). Summed up in that history are Brad Cox's views which you can read in more length in his books. I've at least tangentially covered each book here: Object-Oriented Programming: an Evolutionary Approach and Superdistribution: Objects as Property on the Electronic Frontier. In these he talks about Object-Oriented Programming as the "software industrial revolution", in which the each-one-is-bespoke way of writing software from artisinally-selected ones and lightly-sparkling zeroes is replaced with a catalogue of re-usable parts, called Software ICs (integrated circuits). As an integrator, I might take the "window" IC, the "button" IC, the "text field" IC, and a "data store" IC and make a board for entering expenses.

So far, so npm. The key bit is the next bit. As a computer owner, you might take that board and integrate it into your computer so that you can do your home finances, or so that you can submit your business expense claims, or so that your characters in The Sims can claim for their CPU time, or all three of those things. The key is that this isn't some app developer, this is the person whose computer it is.

From that perspective, Objective-C is an intermediary tool, and not a particularly important or long-lasting one. Its job is to turn legacy code into objects so that it can be accessed by people using their computers by sticking software together using objects (hello NSFileManager). To the extent it has an ongoing job, that is to turn algorithms into objects, for the same reason (but the algorithms have been made out of not-objects, because All Hail the Perform Ant).

You can make your software on your computer by glueing objects together, whether they're made of ObjC (a common and important case), Eiffel (an uncommon and important case), Smalltalk (ditto) or whatever. Objective-C is the shiny surface we're missing over the tar pit. It is the gear system on the bicycle for the mind; the tool that frees computer users from the tyranny of the app vendor and the app store.

I apologise for taking this long to work that out.

#### In which I resolve
When I was a student I got deeply into GNU and Linux. This has been covered elsewhere on this blog, along with the story that as Apple made the best UNIX, and the lab had NeXT computers, I went down the path of Objective-C and OS X.

I now think that this was because, as an impressionable twenty-something-year-old, I thought that the most important part of the technology was, well, the technology. But I realise now that I want to have taken the other path, and now need to beat across toward it.

The other path I'm talking about is the one where I notice that it's not programs like those in Debian that are so great, but the licences under which they are distributed. Much of the software in a GNU/Linux distribution is not so good. That makes it much like using software on every other platform. The difference is that I have the right-and hopefully, after a decade of practice, the ability-to do something about it in the case of free software.

I'm certain that GNU is not the best of all systems, though, because the focus has been on the right to make changes, not the capability of making changes. A quick review of the components I can remember in the installation on my MacBook shows that I'd need to understand at least nine different programming languages in order to be able to dive in and address bugs or missing features in the system, holistically speaking. If they've given me permission to change how my computer works, they haven't made it easy.

But as someone who can change software and wants software to be less broken for me and for others, that permission is important. Next year I'll be looking for more opportunities to work with free software and to make things less broken.

#### In which new developer tools are dull
Over on lobste.rs I said that I don't hold out much hope for another "blue plane" style event in developer tools. In one of Alan Kay's presentations, he referred to the ordinary way of things as the pink plane, and incremental advances in the state of affairs being movements in that plane. Like the square in Edwin Abbot's Flatland that encounters a sphere, a development could take us out of the pink plane into the (orthogonal) blue plane. These blue plane ideas are rare because like the square, it's hard to even conceive of life outside the pink plane.

In what may just be a surprising coincidence, Apple engineers used Blue and Pink to refer to features in evolutionary and revolutionary developments of their operating system.

Software engineering tooling is, for the majority of developers, in a phase of conservative retreat

Build UIs on the web and you probably won't use a graphical builder, you'll type HTML and JavaScript (and maybe JSX) into a text editor.

Build native apps and even where there is a GUI builder, you'll find people recommending against its use and wanting to do things "programmatically" (by which they mean "through typing", even though the GUI builder tools are another way to construct a program).

In the last couple of decades, interest in CASE tooling has shrunk to conservative interest in text editors with some syntax highlighting, like vim or Atom. Gone even is the "build and run" button from IDEs, to be replaced with command-line invocations of grunt tasks (a fancy phrase meaning shell scripts), npm scripts (a fancy phrase meaning shell scripts) or rake tasks (you get the idea).

Where previously there were live development environments embedded in the deployment environment (and the Javascript VM is almost perfectly designed for that task), there is now console.log and unit tests. The height of advanced interaction with your programming tools are the REPL (an interactive shell) and the Playground/InstaREPL (an interactive shell that echoes stdin and stdout in different places).

For the most part, and I say that to avoid the inevitable commenter who thinks that a counterexample like LabView or Mathematica or that one person they met who uses Expression Blend renders the whole argument broken, developers have doubled down on the ceremony of programming: the typing of arcane text into an 80x24 character display. Now to be fair, text is an efficient and compact graphical representation of a linear sequence of connected concepts. But it is not the only one, nor the most efficient nor most compact, and neither are many software systems linear.

The rewards in making software to make software are scarce.

You can do like IntelliJ do, and make a better version of the 80x24 text entry thing. You can work for a platform vendor, and make their version of the 80x24 thing. You can go and get an engineering grade 6 or above job in Silicon Valley and tell your manager that whatever it is their business does, you're going to focus on the 80x24 thing ("at scale") instead.

What you don't seem to be able to do is to disrupt the 80x24 thing. It's free (at least as in beer), it's ubiquitous, and whether or not it's as good as it could be it certainly seems to be good enough for the people who not only get paid to make bad software, but get paid again to fix it.

#### In which things are given names
I recently joined in a very interesting discussion with some of my peers on the thorny subject of naming variables in programs. The core question was whether it's OK to give a variable a temporary name while you're still working out what it's for, or whether you should pause and think out what it's called before you move on.

The question raises other questions, and those are much more interesting to consider. For example, there's an aphorism in computing that naming things is one of the hardest problems we have. That isn't true. We've been naming things for 60,000-100,000 years, and writing down names for things for 5,000 years.

If you know about the thing you're naming, and you know what the name should convey, then naming things is easy. For example, this blog post is on the topic of naming things, and communicating the topic is an important part of the title, so calling the post "On naming things" was very easy. Then, because I'm a comedian, I decided to go back and use a different name.

If naming something is hard, either we don't know something about it, or we don't know something about communicating that knowledge. The second of those is usually much simpler than the first, when it comes to variable names. The variable name is only used by other programmers: either inspecting the program text, or understanding dynamic behaviour in a debugger. The variable represents a snapshot of a part of the program state, and its name should communicate how that snapshot contributes to the valuable computation the program encapsulates.

It's therefore likely that when we struggle to name a variable, it's not because we haven't identified the audience. It's because we haven't identified what the variable is, or what it's for.

In most software design methodologies, we derive the existence of variables from aspects of the design. In the incremental refinement approaches described by people like Tony Hoare and Niklaus Wirth, as we refine a specification we identify the invariants that hold at each level, and the variables we need to preserve those invariants.

In DeMarco's structured analysis, we design our systems by mapping the data flow: our variables hold those data and enable their transformation.

In Object-Oriented analysis and design, we design objects that take on particular roles in an interaction that models the domain problem. Our variables represent the responsibilities and collaborators known to each object.

In Test-Driven Development, we identify a desirable change in the system's behaviour, and then enact that change. Our variables represent contributions to that behaviour.

It's likely that when we can't name a variable, it's because we haven't designed enough to justify introducing the variable yet.

As a specific example, if we're thinking about an algorithm in a process-centric manner, we might have a detailed view of the first few steps, and decide to write a variable that stores the outcome of those steps and is used as input in the subsequent steps. In such a case, the variable doesn't represent anything in the solution model, and is going to be hard to name. It represents "where I got to in the design before I started typing", which isn't a useful variable name. The solution in this case is neither to come up with a good name, nor to drop in a temporary name and move on. The solution is to remove the variable, and go back to designing the rest of the algorithm.

#### In which things are not known
In the last episode—Is software engineering a thing?—I (apparently controversially) suggested that software is the reification of thought, and that software engineering is thus the art of reifying thought, and that thus there can't be any single one-size-fits-all software engineering approach. Let's dig in.

One of the big questions in a software project, hence one of the big topics in software engineering, is requirements: who wants the software to do something, do we need to pay attention to them, and what do they want it to do? We're already well into the realm of individuals and interactions—whether the people building the thing and the people telling them what to build can agree on what one of the two groups thinks they mean—and haven't got as far as building software yet. There's plenty of software engineering ink spilled in this domain, but it can be hard to even decide whether to agree at a metaphysical level with some of it.

Taking a convenience sample (i.e. what's the nearest book on my shelf that I think might mention software requirements), Shari Pfleeger's “Software Engineering: the Production of Quality Software” asks “What is a requirement?” and supplies its own answer:

A requirement is a feature of the system or a description of something the system is capable of doing in order to fulfill the system's purpose.

Enquiring minds have many questions, but let's focus on questions pertaining to reality. Does the system have an objective, positive purpose that can be said to exist? Does the requirement support that purpose, or does someone just think or hope that it does? Does the requirement-as-description accurately capture the expectation of the person who thought it?

With this level of reflection, we can still expect a field of software engineering to say something about requirements, and for understanding that to help with constructing software, but not for it to supply a single solution to “how to requirements”. And without that, much of the rest of software engineering naturally bifurcates or multifurcates. For example, verification and validation is about whether the software does what it ought—or whether someone thinks the software does what they think it ought—but we're back to asking our question of whether we have (or can) accurately capture that.

#### Indie app milestones part one
In the precious and scarce spare time I have around my regular contracting endeavours, I've been working on my first indie app. It reached an important stage in development today; the first time where I could show somebody who doesn't know what I'm up to the UI and they instinctively knew what the app was for. That's not to say that the app is all shiny and delicious; it's entirely fabricated from standard controls. Standard controls I (personally) don't mind so much. However the GUI will need quite a bit more work before the app is at its most intuitive and before I post any teaser screenshots. Still, let's see how I got here.The app is very much a "scratching my own itch" endeavour. I tooled around with a few ideas for apps while sat in a coffee shop, but one of them jumped out as something I'd use frequently. If I'll use it, then hopefully somebody else will!So I know what this app is, but what does it do? Something I'd bumped into before in software engineering was the concept of a User Story: a testable, brief description of something which will add value to the app. I broke out the index cards and wrote a single sentence on each, describing something the user will be able to do once the user story is added to the app. I've got no idea whether I have been complete, exhaustive or accurate in defining these user stories. If I need to change, add or remove any user stories I can easily do that when I decide that it's necessary. I don't need to know now a complete roadmap of the application for the next five years.As an aside, people working on larger teams than my one-man affair may need to estimate how much effort will be needed on their projects and track progress against their estimates. User stories are great for this, because each is small enough to make real progress on in short time, each represents a discrete and (preferably) independent useful addition to the app and so the app is ready to ship any time an integer number of these user stories is complete on a branch. All of this means that it shouldn't be too hard to get the estimate for a user story roughly correct (unlike big up-front planning, which I don't think I've ever seen succeed), that previous complete user stories can help improve estimates on future stories and that even an error of +/- a few stories means you've got something of value to give to the customer.So, back with me, and I've written down an important number of user stories; the number I thought of before I gave up :-). If there are any more they obviously don't jump out at me as a potential user, so I should find them when other people start looking at the app or as I continue using/testing the thing. I eventually came up with 17 user stories, of which 3 are not directly related to the goal of the app ("the user can purchase the app" being one of them). That's a lot of user stories!If anything it's too many stories. If I developed all of those before I shipped, then I'd spend lots of time on niche features before even finding out how useful the real world finds the basic things. I split the stories into two piles; the ones which are absolutely necessary for a preview release, and the ones which can come later. I don't yet care how late "later" is; they could be in 1.0, a point release or a paid upgrade. As I haven't even got to the first beta yet that's immaterial, I just know that they don't need to be now. There are four stories that do need to be now.So, I've started implementing these stories. For the first one I went to a small whiteboard and sketched UI mock-ups. In fact, I came up with four. I then set about finding out whether other apps have similar UI and how they've presented it, to choose one of these mock-ups. Following advice from the world according to Gemmell I took photos of the whiteboard at each important stage to act as a design log - I'm also keeping screenshots of the app as I go. Then it's over to Xcode!So a few iterations of whiteboard/Interface Builder/Xcode later and I have two of my four "must-have" stories completed, and already somebody who has seen the app knows what it's about. With any luck (and the next time I snatch any spare time) it won't take long to have the four stories complete, at which point I can start the private beta to find out where to go next. Oh, and what is the app? I'll tell you soon...

#### Inheritance is old and busted
Back when I started reading about Object-Oriented Programming (which was when Java was new, I was using Delphi and maybe the ArcGIS scripting language, which also had OO features) the entire hotness was inheritance. Class hierarchies as complicated as biological taxonomies were diagrammed, implemented and justified. Authors explained that while both a Dog and a Cat could makeNoise(), they'd do it in different ways. They also got their knickers in a knot over whether a Square is-a Rectangle or the other way around. If you believe in successive specialisation then the Square is-a (special kind of) Rectangle. If you believe in the Liskov Substitution Principle then it cannot be. If you don't care then you'd only define a Rectangle in the first place.

Slightly earlier than this time it was noticed that you sometimes want to separate the idea of "this object does these things" from "this object is something of this type". C++ programmers were inheriting from purely virtual classes. Objective-C gave us protocols to conform to; Java borrowed this idea and called the protocols interfaces.

Both inheritance and conformance (which is what I'll call protocol adoption, to avoid the confusion between Objective-C and Java over what it means to "implement" an "interface") are blunt and unwieldy instruments, however. In practice, I use them in a few specific places:

I most often inherit from a "base class". My Python object is-an object, my Java object is-an (java.lang.)object, my ObjC object is-a NSObject.
I sometimes inherit from deeper classes because something requires me to build a specialised version of that class. For example, UIKit requires that my view controllers are specialised versions of UIViewController, and that my views are specialised versions of those views.
I usually adopt a protocol or abstract interface in order to implement some design pattern. For example I might adopt a data source or delegate protocol. I might provide an implementation of a strategy or state or visitor.
I pretty much avoid even with bargepoles the "marker interfaces" that pervade pre-annotation Java, like Cloneable. I even hate NSCopying as a way of designing a protocol.

It's time we added specialised syntax on top of inheritance to account for these cases particularly. For a start, any language that doesn't assume you want its usual version of Object-ness when you create a class without saying anything about it is broken. Yes, I'm looking at you Objective-C. Yes, I know about NSProxy and I'm old enough to remember Object. I still say that not wanting an NSObject is rare enough that you should have to shout about it. I've never intentionally created a new Objective-C root class - but that's the easiest thing the syntax permits.

Think of it like pointers. Yes, you can use pointers in ObjC and C++. Yes, arrays are built using pointers—as are object references. No you don't need to use pointer syntax to build arrays and object references - we've got higher-level syntax. Just like we've got higher-level syntax for getting at properties or at subscripted entries in Objective-C.

So yes, you can create a delegate using protocols. But why not have syntax saying "this class's delegates must do this" and "this is a delegate of that class"? Why not say "this is a special version of that thing"? Why not say "this group of classes implement a state machine?"

Inheritance is old and busted.

#### Inheritance still doesn't make any sense
Some ideas based on feedback to the Why inheritance never made any sense:

Feedback: Subtypes are necessary

The only one of these that is practically workable is behaviour inheritance &lt;=> subtype inheritance: I'm sorry that you were exposed to Java at such an impressionable age. The compilers of languages like Java enable subclass = subtype, by automatically assuming that a subclass is is a valid value for variable binding, for example. However they do nothing to ensure subclass = subtype. This is valid C#, a language very like Java for this discussion:

namespace QuickTestThing
{
    class Class1
    {
        override public string ToString()
        {
            return "Class1";
        }
    }
    class Class2 : Class1
    {
        public override string ToString()
        {
            throw new Exception();
        }
    }
}

Now is Class2 a subtype of Class1? Does the compiler let you pretend that it is?

You don't even need inheritance

As discussed in the original post, the whole "favour composition over inheritance" movement gets by fine with no inheritance. Composition and delegation (I don't know about this message, I'll forward it to someone who does) let you get the same behaviour.

Feedback: build it yourself

Can I demonstrate a language that has all three of subtype inheritance, behaviour inheritance, and categorical inheritance as distinct language features? Yes, but I would need to learn Racket first. I'm on it.

But in the meantime, re-read the "you don't even need inheritance" paragraph and think about how you would build each of those three ideas out of delegation.

#### Init dead; no casualties
An interesting time if you happen to be an aging startup superserver; Ubuntu are to replace init with Upstart from &quot;Edgy Eft&quot;.  That's quite cool, the model in which everything needs to always come up and always in the same order put forward by both styles of init is long dead, and the system V style of init is particularly cumbersome (though someone at Apple clearly liked it, as they had both SystemStarter and watchdog in the past).

I'm actually both surprised and pleased to see a Linux distribution making the jump here, I've always considered them fairly conservative (apart from in throwing any old free package into the distribution; I mean from a &quot;core stuff&quot;[*] perspective.  I know that FreeBSD attempted something with launchd last Summer, but the latter has too much mach_init magic which can't cleanly be excised.

[*]now Apple are going to start a CoreStuff API and sue me.  Fantastic.

One thing from TFA saddened me (though I really ought to have expected it), and that's this:

To avoid reinventing the wheel, we first looked at how much effort it would be to use of modify the existing replacements to be able to do this. Sun SMF and Apple launchd were immediately ruled out due to licence issues. It was important for us that the solution be uncontroversially free so that other distributions might adopt it; many had already rejected these for GPL incompatibility reasons.

Yes, it's the standard &quot;GPL first, utility second&quot; argument.  I have a fairly strong opinion here: if something works, and I can see that it works, and it works better than any alternative I have access to, I'll use it.  That's why I'm posting this from OmniWeb on a Mac, despite [i]there being free web browsers around, [ii]my work involving Linux.  I prefer to have access to the source, except where that involves a retrograde step in terms of utility.  Anyway, I digress; the point here is that launchd is released under (well, depends which version you look at) the Apache licence, and OpenSolaris under the CDDL; both of which are free software licences, but are incompatible with the GPL on technicalities.  That's nice if you really do want to split hairs, but GNU/Linux (if you like) is supposed to be a usable OS so shouldn't utility come first?  Admit it, how many of you have uninstalled kaffe or gcj and are using the Sun jdk?  More to the point, how many of you are using Apache (which is also released under the Apache licence)?  It's time to get over it.  I propose that if you want to work on a pure GNU system, you should indeed work on a pure GNU system and hie thineself over to the HURD mailing lists.  I fully intend to, if a gobbet of spare time makes itself available soon...but more because I think the HURD's cool ;-).

#### Input-Output Maps are Strongly Biased Towards Simple Outputs
About this paper

Input-Output Maps are Strongly Biased Towards Simple Outputs, Kamaludin Dingle, Chico Q. Camargo and Ard A. Louis, Nature Communications 9, 761 (2018).

Notes

On Saturday I went to my alma mater's Morning of Theoretical Physics, which was actually on "the Physics of Life" (or Active Matter as theoretical physicists seem to call it). Professor Louis presented this work in relation to RNA folding, but it's the relevance to neural networks that interested me.

The assertion made in this paper is that if you have a map of a lot of inputs to a (significantly smaller, but still large) collection of outputs, the outputs are not equally likely to occur. Instead, the simpler outputs are preferentially selected.

A quick demonstration of the intuition behind this argument: imagine randomly assembling a fixed number of lego bricks into a shape. Some particular unique shape with weird branches can only be formed by an individual configuration of the bricks. On the other hand, a simpler shape with large degree of symmetry can be formed from different configurations. Therefore the process of randomly selecting a shape will preferentially pick the symmetric shape.

The complexity metric that's useful here is called Kolmogorov complexity, and roughly speaking it's a measure of the length of a Universal Turing Machine program needed to describe the shape (or other object). Consider strings. A random string of 40 characters, say a56579418dc7908ce5f0b24b05c78e085cb863dc, may not be representable in any more efficient way than its own characters. But the string aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa, which is 40 characters, can be written with the Python program:

'a'*40

which is seven characters long including the newline. Assuming eight bits per character, the random string needs 40*8=320 bits to be represented. The forty as can be found by actually finding the Python program, which is 56 bits. The assertion is that a "find a program that generates character sequences of length 40" algorithm (with some particular assumptions in place) will find the a56579… string with probablity 2^-320, but will find the aaa… string with probability 2^-56, which is much, much more likely.

In fact, this paper shows that the upper and lower bounds on the probability of a map yielding a particular output for random input are both dependent on the Kolmogorov complexity of the output. It happens that due to the halting problem, you can't calculate Kolmogorov complexity for arbitrary outputs. But you can approximate it, for example using Lempel-Ziv complexity (i.e. the length of the input to a lossless compression algorithm needed to recover the same output).

Where does this meet neural networks? In a preprint of a paper selected for the ICLR 2019, with two of the same authors as this paper. Here, we find that a neural network can be thought of as a map between the inputs and weights to a function that does the inference.

Typically neural network architectures have lots more parameters than there are points in the training set, so how is it that they manage to generalise so well? And why is it that different training techniques, including stochastic gradient descent and genetic algorithms, result in networks with comparable performance?

The authors argue that a generalising function is much less complex than an overfitting function, using the same idea of complexity shown above. And that as the training process for the network is sampling the map of inputs->functions, it is more likely to hit on the simple functions than the complex ones. Therefore the fact that neural networks generalise well is intrinsic to the way they select functions from a wealth of possibilities.

My hope is that this is a step toward a predictive theory of neural network architectures. That by knowing the something of the function we want to approximate, we can set a lower bound on the complexity of a network needed to discover sufficiently generalisable functions. This would be huge for both reducing the training effort needed for networks, and for reducing the evaluation runtime. That, in turn, would make it easier to use pretrained networks on mobile and IoT devices.

#### Inside-Out Apps
This article is based on a talk I gave at mdevcon 2014. The talk also included a specific example to demonstrate the approach, but was otherwise a presentation of the following argument.

You probably read this blog because you write apps. Which is kind of cool, because I have been known to use apps. I'd be interested to hear what yours is about.

Not so fast! Let me make this clear first: I only buy page-based apps, they must use Core Data, and I automatically give one star to any app that uses storyboards.

OK, that didn't sound like it made any sense. Nobody actually chooses which apps to buy based on what technologies or frameworks are used, they choose which apps to buy based on what problems those apps solve. On the experience they derive from using the software.

When we build our apps, the problem we're solving and the experience we're providing needs to be at the uppermost of our thoughts. You're probably already used to doing this in designing an application: Apple's Human Interface Guidelines describe the creation of an App Definition Statement to guide thinking about what goes into an app and how people will use it:

  An app definition statement is a concise, concrete declaration of an app’s main purpose and its intended audience.

Create an app definition statement early in your development effort to help you turn an idea and a list of features into a coherent product that people want to own. Throughout development, use the definition statement to decide if potential features and behaviors make sense.

My suggestion is that you should use this idea to guide your app's architecture and your class design too. Start from the problem, then work through solving that problem to building your application. I have two reasons: the first will help you, and the second will help me to help you.

The first reason is to promote a decoupling between the problem you're trying to solve, the design you present for interacting with that solution, and the technologies you choose to implement the solution and its design. Your problem is not "I need a Master-Detail application", which means that your solution may not be that. In fact, your problem is not that, and it may not make sense to present it that way. Or if it does now, it might not next week.

You see, designers are fickle beasts, and for all their feel-good bloviation about psychology and user experience, most are actually just operating on a combination of trend and whimsy. Last week's refresh button is this week's pull gesture is next week's interaction-free event. Yesterday's tab bar is today's hamburger menu is tomorrow's swipe-in drawer. Last decade's mouse is this decade's finger is next decade's eye motion. Unless your problem is Corinthian leather, that'll be gone soon. Whatever you're doing for iOS 7 will change for iOS 8.

So it's best to decouple your solution from your design, and the easiest way to do that is to solve the problem first and then design a presentation for it. Think about it. If you try to design a train timetable, then you'll end up with a timetable that happens to contain train details. If you try to solve the problem "how do I know at what time to be on which platform to catch the train to London?", then you might end up with a timetable, but you might not. And however the design of the solution changes, the problem itself will not: just as the problem of knowing where to catch a train has not changed in over a century.

The same problem that affects design-driven development also affects technology-driven development. This month you want to use Core Data. Then next month, you wish you hadn't. The following month, you kind of want to again, then later you realise you needed a document database after all and go down that route. Solve the problem without depending on particular libraries, then changing libraries is no big deal, and neither is changing how you deal with those libraries.

It's starting with the technology that leads to Massive View Controller. If you start by knowing that you need to glue some data to some view via a View Controller, then that's what you end up with.

This problem is exacerbated, I believe, by a religious adherence to Model-View-Controller. My job here is not to destroy MVC, I am neither an iconoclast nor a sacrificer of sacred cattle. But when you get overly attached to MVC, then you look at every class you create and ask the question "is this a model, a view, or a controller?". Because this question makes no sense, the answer doesn't either: anything that isn't evidently data or evidently graphics gets put into the amorphous "controller" collection, which eventually sucks your entire codebase into its innards like a black hole collapsing under its own weight.

Let's stick with this "everything is MVC" difficulty for another paragraph, and possibly a bulleted list thereafter. Here are some helpful answers to the "which layer does this go in" questions:

does my Core Data stack belong in the model, the view, or the controller? No. Core Data is a persistence service, which your app can call on to save or retrieve data. Often the data will come from the model, but saving and retrieving that data is not itself part of your model.
does my networking code belong in the model, the view, or the controller? No. Networking is a communications service, which your app can call on to send or retrieve data. Often the data will come from the model, but sending and retrieving that data is not itself part of your model.
is Core Graphics part of the model, the view, or the controller? No. Core Graphics is a display primitive that helps objects represent themselves on the display. Often those objects will be views, but the means by which they represent themselves are part of an external service.

So building an app in a solution-first approach can help focus on what the app does, removing any unfortunate coupling between that and what the app looks like or what the app uses. That's the bit that helps you. Now, about the other reason for doing this, the reason that makes it easier for me to help you.

When I come to look at your code, and this happens fairly often, I need to work out quickly what it does and what it should do, so that I can work out why there's a difference between those two things and what I need to do about it. If your app is organised in such a way that I can see how each class contributes to the problem being solved, then I can readily tell where I go for everything I need. If, on the other hand, your project looks like this:

Then the only thing I can tell is that your app is entirely interchangeable with every other app that claims to be nothing more than MVC. This includes every Rails app, ever. Here's the thing. I know what MVC is, and how it works. I know what UIKit is, and why Apple thinks everything is a view controller. I get those things, your app doesn't need to tell me those things again. It needs to reflect not the similarities, which I've seen every time I've launched Project Builder since around 2000, but the differences, which are the things that make your app special.

OK, so that's the theory. We should start from the problem, and move to the solution, then adapt the solution onto the presentation and other technology we need to use to get a product we can sell this week. When the technologies and the presentations change, we can adapt onto those new things, to get the product we can sell next week, without having to worry about whether we broke solving the problem. But what's the practice? How do we do that?

Start with the model.

Remember that, in Apple's words:

  model objects represent knowledge and expertise related to a specific problem domain

so solving the problem first means modelling the problem first. Now you can do this without regard to any particular libraries or technology, although it helps to pick a programming language so that you can actually write something down. In fact, you can start here:

A Foundation command-line tool has everything you need to solve your problem (in fact it contains a few more things than that, to make up for erstwhile deficiencies in the link editor, but we'll ignore those things). It lets you make objects, and it lets you use all those boring things that were solved by computer scientists back in the 1760s like strings, collections and memory allocation.

So with a combination of the subset of Objective-C, the bits of Foundation that should really be in Foundation, and unit tests to drive the design of the solution, we can solve whatever problem it is that the app needs to solve. There's just one difficulty, and that is that the problem is only solved for people who know how to send messages to objects. Now we can worry about those fast-moving targets of presentation and technology choice, knowing that the core of our app is a stable, well-tested collection of objects that solve our customers' problem. We expose aspects of the solution by adapting them onto our choice of user interface, and similarly any other technology dependencies we need to introduce are held at arm's length. We test that we integrate with them correctly, but not that using them ends up solving the problem.

If something must go, then we drop it, without worrying whether we've broken our ability to solve the problem. The libraries and frameworks are just services that we can switch between as we see fit. They help us solve our problem, which is to help everyone else to solve their problem.

And yes, when you come to build the user interface, then model-view-controller will be important. But only in adapting the solution onto the user interface, not as a strategy for stuffing an infinite number of coats onto three coat hooks.

References

None of the above is new, it's just how Object-Oriented Programming is supposed to work. In the first part of my MVC series, I investigated Thing-Model-View-Editor and the progress from the "Thing" (a problem in the real world that must be solved) to a "Model" (a representation of that problem and its solution in the computer). That article relied on sources from Trygve Reenskaug, who described (in 1979) the process by which he moved from a Thing to a Model and then to a user interface.

In his 1992 book Object-Oriented Software Engineering: A Use-Case Driven Approach, Ivar Jacobson describes a formalised version of the same motion, based on documented use cases. Some teams replaced use cases with user stories, which look a lot like Reenskaug's user goals:

  An example: To get better control over my finances, I would need to set up a budget; to keep account 
of all income and expenditure; and to keep a running comparison between budget and accounts. 

Alastair Cockburn described the Ports and Adapters Architecture (earlier known as the hexagonal architecture) in which the application's use cases (i.e. the ways in which it solves problems) are at the core, and everything else is kept at a distance through adapters which can easily be replaced.

#### Inspired by Swift
Gulliver meets the Oopers

Lemuel Gulliver's world was black. No light, no sound, infinite darkness and solitude. Am I dead?, he asked himself. No, surely not. He opened his eyes.

Still, everything remained black. My God, I am dead! Lemuel Gulliver began to panic. Then, slowly, he realised that panicking itself meant life: My heart is pounding, he thought, therefore it must be beating. Life! Sweet, God-given, beautiful life! But what sort of life is it, to live as a man dead?

Memory failed him but in bursts, though those bursts helped him understand his predicament. Yes, a shipwreck. But he had got into a lifeboat, which had carried him precariously through the swell of the storm.

Of course, the lifeboat! Gulliver raised his hand, and laughed loudly to himself as it hit the planks. He was on land, and it was the upturned lifeboat had given the impression of his premature trip to Hades. He pushed with all of his strength, and shifted the boat over to lie beside him on the beach.

For minutes he lay still. His eyes needed to adjust, his arms to recover, and his mind to regain some composure. Once the blackness of the lifeboat and the whiteness of the sudden return to light had begun to balance, he appraised his condition. It was indeed a beach upon which he lay, marked with dunes and occasional tufts of a tall grassy plant. The water, which apparently had once been turbulent enough to deposit his unconscious self along with its inverted boat some way up the beach, now was smooth and still.

To his right, behind one of the dunes, he could make out rooves denoting that a settlement was not far away. One in particular, a great grey spire, stood out like a flagstaff. Realising that he needed water, food and assistance, Gulliver uneasily got up and started walking toward the town.

As he crested the dune he became aware of another walker, making her way along a path that would bring her closer and heading like him in the direction of the town. He got to the path and waited for her to approach.

"Hello, madam, I don't mean to burden you but find myself in innumerable difficulties. I'm a traveller who has got himself shipwrecked in this country, and who finds himself in need of water, provisions and other assistance. Can you help me?"

The woman stopped, smiled at him, then executed a perfect curtsey before she replied. "Good day to you, sir, and is not the weather fine? My name is Kaye Allen, delighted to meet you. I'm sorry to hear about your problems and certainly want to help, please come with me to our home, where my husband and I will feed you and lodge you while you make your arrangements."

Gulliver introduced himself and thanked her heartily for her generosity. Together they walked towards the town, the flagstaff spire always ahead of them. Kaye Allen was engrossed during much of the walk in the task of writing postcards: many of them addressed to different people each containing short messages. Gulliver asked about the spire, and she answered but not before finishing the current postcard, and packing away it and her pen.

"It is best that you not inquire about such things, sir. That building is associated with immorality and debasement, and it would not be becoming for an Ooper to associate with the sort found there."

"An Ooper, madam? Pardon me but I have never encountered that word before. What is an Ooper?"

"Why, Mr. Gulliver, I am an Ooper, my husband is an Ooper, and indeed our whole community is comprised of Oopers. It is our religion, our philosophy, and our way of life. But let us not talk of that now. There will be time enough later. Let us now make our way onward, the afternoon draws on."

The rest of their walk to the Allens' house was mostly without event, though once a man approached them to ask the time. Gulliver thought there was something odd about Mrs. Allen's reply, though he could not determine what it was as it seemed polite enough. She stopped, smiled at the man, then executed a perfect curtsey before she replied. "Good day to you, sir, and is not the weather fine? My name is Kaye Allen, delighted to meet you. I believe the time to be something after five of the clock, see the sun already begins to set."

As the twilight drew in they arrived at last at the house. Kaye's husband was already in, and he greeted Gulliver warmly. He smiled, then executed a perfect bow before saying "Good day to you, sir, and is not the weather fine? My name is Turmo Allen, delighted to meet you. Please, find the wash room upstairs and feel welcome to borrow some clothes, while we prepare dinner."

Gulliver felt better for a wash and a change of clothes, but not so much improved as he did when they sat down to eat. While the food was simple, it was plentiful and he had no conception of how long it was since he had last eaten. The Allens were very precise about setting the table. Before each dish was served, the fork would be placed on the left of the setting just so. Then the knife to the right, just so. Then the water glass, just so, and filled with water from the pitcher, just so. Finally the dish was brought, placed just so in the centre of the setting, and they would sit, pick up first the fork then the knife, and begin eating.

Gulliver asked about this Oopers religion that Kaye had mentioned, and found Turmo to be most effusive on the topic.

"We Oopers live our lives according to the books of the lady Adele of the golden mountain. The legends say that many centuries ago, before the Oopers, the people of this region were simple backwards folk who believed strongly in superstition. They spent their time in constructing procedures for each other to follow, such procedures becoming more baroque and complicated with time. Even in emergencies and when under attack, if they had begun one of their procedures then needs must it be finished.

"One day, so we are told, a hot air balloon of many colours landed in the market place of this very town. The occupants declared themselves to be emissaries of a wonderful place, a tall tower of ivory on an island far away. They said that they too had previously been enslaved by their procedures, but that they had freed themselves through enlightenment and that they could teach this freedom to our forebears.

"One of their number was the lady Adele of the golden mountain, and it is she who wrote their teachings in the books that have come down through the generations, yea even to us, and taught us the lessons of these people of the balloon. You have seen my wife, Kaye, writing many postcards for her correspondents both near and far?"

"Why, Mr. Gulliver, I have even written one for you."

"The people of the balloon taught us that complex procedures can be avoided, and our community can function together better, if we instead send each other messages. Our whole philosophy and way of life is based on this message-sending. It has delivered us from the oppression of our procedures, enlightened our society, and advanced us immeasurably."

Gulliver interrupted at this point. "Thank you, sir, for your most edifying explanation, and thank you madam for your message. May I read it?"

Mrs. Allen laughed. "That would not do at all, Mr. Gulliver! Due process must be observed. See, I have gathered all of the cards that I have written so far today. Tomorrow morning I will take them to the post office, and there they will be despatched to their recipients. Including you! Once you have formally received your card, then you may read your message. Not before!"

"But is that not an example of the sort of procedure you say belonged to the simple times of your forebears? Is that not unnecessarily convoluted and slow, as you could simply tell me the message now or give me the card?"

"No, sir, it is not. We Oopers have done away with the procedure, and replaced it with this simple act of message sending."

"Then why not give me your message now?"

"That would not do at all, Mr. Gulliver! Due process must be observed."

Gulliver could sense his own frustration mounting, and thought it prudent to change the subject. Unfortunately in his haste he forgot Mrs. Allen's reaction to his previous enquiries regarding the spire, and that is the topic to which he next turned.

"It is best for you if you avoid that place, sir," Mr. Allen quickly responded. "It is a Church, a building associated with the debased Phuncers and their heretical acts. No true Ooper would go there, and it would not do for you to be tainted by their evil ways. Now let us talk no more of them."

Gulliver recognised that he was not being the most gracious or entertaining of guests, so did not suggest any further metiere for their conversation. In the absence of his conversational selections there was nothing for the Allens to respond to, so the rest of the dinner proceeded in silence until they all departed to bed.

Escaping to the Phuncers

Morning breakfast passed much as evening dinner had. While the Allens were polite enough, Gulliver found it infuriating to deal not only with their strange, repetitive rituals but also with their insistence that they had left such procedure behind. He quizzed them upon this topic.

"You say that the Oopers have freed themselves from the shackles of procedure, yet I observe you both greet me using exactly the same set of rules. Do you not see that this itself is procedure?"

"No, sir, you miss the subtlety that was taught by Adele of the golden mountain. See, while we each say an identical greeting to you, my husband bows before doing it while my action is to curtsey. All people say the greeting, but specifically those people who are men perform a bow while those people who are women perform a curtsey. This ability to make more specific amendments is what liberated the Oopers."

"But still, all you are doing is executing this procedure!"

"I wish, sir, that I had more time to discuss this with you. But now I must attend to my messages."

With that, Kaye Allen returned to her postcards and Lemuel Gulliver, with more civility and politeness than he felt, thanked her for the hospitality he had received which in fact had been generous and left their abode hastily. His aim was to discover his location and how he might return home.

Once he was outside, his eyes were once again drawn to that strange spire, the Church by which his hosts had been so repulsed. Perhaps here he would find some useful information, or at least a different insight into this unique land. Resolved to learn more, he set out in the direction of the Church.

As he walked the mile or so to the tall building, he saw plenty of Oopers, recognised by their strange rituals and their stacks of message cards. The closer he came to his goal, however, the thinner their numbers, and when he finally arrived at the huge quadrangle outside the Church there was not one Ooper to be seen.

That is not to say that the square was empty, but clearly the people here were of a different background. They were gathered into groups, usually comprising four to six people, who criss-crossed the square purposefully in files. Perhaps these were the Phuncers he had been told about. Gulliver proceeded toward the Church to learn more.

Outwardly, the Church reminded him in passing of some of the religious institutions he remembered from his home. Indeed where his own house of prayer signed its identity with the Greek letters alpha and omega, this Church even had a similar symbol: the letter lambda. Confident that he was on somewhat familiar territory, he entered.

Gulliver started when, just inside the doorway before his eyes had adjusted to the gloom, he almost bumped into a file of a half dozen people who were on their way out. He mumbled an apology, then introduced himself.

"My name is Lemuel Gulliver, what is your name please?" If the lead person in the file heard him, she did not make it evident. She consulted a clipboard she was holding, made a note on it, then whispered something to the second person in the file. He did the same, and so on down the line. Finally the person at the back checked his clipboard, made a note, and replied "How do you do? My name is Alonso."

Gulliver was not certain what had just happened. Perhaps the woman to whom he had addressed himself had not heard him clearly. He cleared his throat, and repeated: "My name is Lemuel Gulliver, what is your name please?" Again, she looked at her clipboard. This time, though, she apparently saw something that satisfied her (Gulliver could not be certain, but he thought she was looking at the note she had just made moments earlier) and showed that to the man behind her. He, in his turn, found the corresponding note on his board and showed it to the third. As before, it was the last of them who replied "How do you do? My name is Alonso."

While this was happening, Gulliver had been able to take a look at the lead woman's board. It was full of fragments of conversations, the left column apparently representing sentences that had been uttered to her (see! There is the very greeting I introduced myself with, written at the end of the page!) and so he guessed that the second column was the list of responses she had passed on to her colleague. Gulliver surmised that this tally represented an efficiency of sorts: by remembering all of her previous discussions, she could show her friends the answers she had previously given without having to construct them anew.

Conversation was, understandably, engaged at a slow pace. All the while Gulliver addressed himself to the woman at the front of the file; all the while she passed the discussion back through the chain of six; all the while Alonso (he assumed that this name referred to the ultimate member of the group) issued the responses.

"Tell me, is this the place of the Phuncers?"

"Why yes, this Church is our holy building, and you are welcome within it."

"I hope to learn something of your people. Why is it that you carry on in these files?"

"One of the oldest tenets of the Phuncer creed is that a person has more capability as a member of a group than as an individual. By composing our different abilities we become more efficient."

"I see, that is an interesting philosophy. When did it arise?"

Here, an interesting modification to the usual process occurred. As before, the question was passed back through the file and the notes were made, but the fourth person made a record, then put their board down, picked it up anew and read back that note. Making another, they repeated the process. And again. (Gulliver could just about see the board, and it seemed that they had written the symbols "1 +" over and over.) Finally satisfied, they passed the whisper on again until Alonso could give a response.

"It was six decades ago that our movement came into being."

"What was the purpose behind its creation?"

"Back before we were liberated, the people of this part were oppressed by a government that forced people into performing all manner of complicated and perverse rituals and procedures. Our forebears sought to free us from these procedures, from the imperatives coming from the State."

"And is your number many?" (Again, the repeated setting and resetting of the clipboard.)

"When first we started we were few. We formed a commune that sought to minimise State interference within its boundaries, and to achieve self-sufficience. Recently our number has become bolstered, and even those among the heathen Oopers have joined us and learned from our teachings."

"And has this surge in membership changed your position?"

"We are now so numerous that our whole conception has changed. Rather than seeking to reduce interaction with the State, we intend to overthrow it completely. Yea, anarchy! But not the chaos that you may associate with it. Compositions of individuals each performing their prescribed function can yield a higher order."

This last response had taken longer than usual for the group to formulate. Whereas before they had communicated with each other, one after the other, in building their response, they had needed outside help here. One of their file broke rank, and went out from the Church into the square. There, Gulliver saw that he approached a besuited woman, greeted her politely, then a complicated exchange of handshakes, papers and stamps ensued. When this procedure was complete he discussed something (Gulliver assumed the current conversation between himself and Alonso's group) with her a while. On concluding this business he raised his hat, returned to the file and continued the whisper game.

"Who was that person with whom your colleague discoursed then?"

"She is a civil servant. We occasionally need to engage their help in carrying on our business."

"You mean she represents the State?"

"Yes, quite so. They can be very helpful in our achievement of our aims."

"But didn't you say that you were planning to replace the State?"

"Planning? Yea, it is our very destiny! All trappings of State will be dissolved and a functional order, a higher order, will take its place."

At this, Gulliver realised that he was not getting much further than in his discussions with the Oopers previously. He bade his interlocutors farewell, and returned to the square.

All around, he saw the Phuncers, and all around, they reminded him very much of the procedures that they, as the Oopers, claimed to have left behind. The way they followed each other around in files. The recording previous discussions in their clipboards. The convoluted discussions with civil servants.

They shared other traits with the Oopers: their claim that they had moved beyond the things that they still actually practiced every day, and the complete refusal to accept nor even see that this was the case. This gave Gulliver a start: maybe there was something both groups had in common, some shared memory that accounted for their commonalities? He had to find out more.

Gulliver unravels

As Gulliver pondered this strange town and its inhabitants, he found himself back in Ooper territory. This was unsurprising as there were many more Oopers than Phuncers that he had seen since his shipwreck. He also realised that he was at the University, with the buildings somewhat coarsely modeled on the ivory tower that the Allens had described to him. This presented perhaps the best opportunity he had to understand this place. He looked for signs and campus maps, and eventually located the library.

Realising that there were political implications to his research, he asked the librarian for the writings of Adele of the golden mountain and associated criticisms and commentaries. She happily met his greeting (a curtsey, then "Good day to you, sir, and is not the weather fine?"), went away and brought back a whole stack of books which he took to his desk. There were the books by Adele herself, and contemporary discussions such as "Oopism: An Evolutionary Approach" (some wag of a student had subtitled this "On the Origin of Classes by Means of Artificial Selection") and Bertrand de la Tour's "Oopist Construction", which seemed to have been written by a contract lawyer. There were also more modern interpretations of the earlier texts, such as Cecil R. Martin's "Clean Living" and "Clean Liver".

But this was not enough to satiate Gulliver's curiosity. Once the librarian had returned to her postcards, he paged through the catalogue to try and find anything on the Phuncers and their foundation. There was nothing explicit, but "Religion: Heresies" and "Politics: Anarchy" seemed appropriate places to look. He soon discovered a couple of likely tomes: "The Structure and Interpretation of Careful Phuncism" and "Concepts in Political Languages".

And so Gulliver read. There was to much to read in one sitting, but he also didn't want to risk the Ooper librarian seeing all of his material. He hid books behind shelves, rushed out to the University's refectory to get a brief meal and slept in a corner of the campus. For three days he read, living like a hybrid of a hermit and a monk.

The more he read, the more he was shocked: he was reading the same thing in two different places! Each collection of material described the problems that had beset the town when the focus had been on procedures, and lay out a manifesto for moving beyond this situation. Each manifesto was broadly the same: break the procedures into focussed skills; find individuals who are good at those skills and give them the single responsibility of working in that skilled area; compose teams that can more efficiently complete large tasks by combining their skill areas.

At some point, then, the two had declared a holy war. Each knew that their own approach was correct and the other was heretical. But what was the source of their disagreement? Gulliver thought back to a previous encounter, in which two peoples considered each other mortal enemies due to their choice of which end of a boiled egg to eat first. At least, he thought to himself, those people disagreed on something.

#### Integrating SSH with the keychain on Snow Leopard
Not much movement has occurred on projects like SSHKeychain.app or SSHAgent.app in the last couple of years. The reason is that it's not necessary to use them these days; you can get all of the convenience of keychain-stored SSH passphrases using the built in software. Here's a guide to using the Keychain to store your pass phrases.

Create the key pair

We'll use the default key format, which is RSA for SSH protocol 2.0. We definitely want to enter a passphrase, so that if the private key is leaked it cannot be used.

jormungand:~ leeg$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/leeg/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/leeg/.ssh/id_rsa.
Your public key has been saved in /Users/leeg/.ssh/id_rsa.pub.
The key fingerprint is:
ff:ce:0a:f6:ee:0d:e8:a5:aa:56:a0:f3:0b:81:80:cc leeg@jormungand.local
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|+                |
|oE               |
|..  .            |
|. .. .  S        |
|  o.  .  o       |
|  .o .  + +      |
|   .o  o = =     |
|   .oo..oo=o=    |
+-----------------+

If you haven't seen it before, randomart is not a screenshot from nethack; rather it's a visual hashing algorithm. Two different public keys are not guaranteed to have different randomart fingerprints, but the chance that they are close enough to pass a quick visual inspection is small.

Deploy the public key to the SSH server

Do this using any available route; I choose to use password-based SSH.

jormungand:~ leeg$ ssh heimdall.local 'cat -  >> .ssh/authorized_keys' < .ssh/id_rsa.pub
Password:

There is no step three

Mac OS X automatically runs ssh-agent, the key-caching service, as a launchd agent. When SSH attempts to negotiate authentication using your key-based identity, you automatically get asked whether you want to store the passphrase in the keychain. Like this:

Now the passphrase is stored in the keychain. Don't believe me? Lookie here:

So your SSH key is protected by the passphrase, and the passphrase is protected by the keychain.

Update: Minor caveat

Of course (and I forgot this :-S), if you use FileVault to protect the home folder on the SSH server, the user's home folder isn't mounted until after you authenticate. This means that the authorized_keys file can't be consulted during negotiation. Once you have logged in once (using your password), subsequent logins will use the keys (as PAM automatically mounts the FileVault volumes the first time, so authorized_keys becomes available).

#### Intellectual property and software: the nuclear option
There are many problems that arise from thinking about the ownership of software and its design. Organisations like the Free Software Foundation and Open Source Initiative take advantage of the protections of copyright of source code - presumed to be a creative work analogous to a written poem or a painting on canvas - to impose terms on how programs derived from the source code can be used.

Similar controls are not, apparently, appropriate for many proprietary software companies who choose not to publish their source code and control its use through similar licences. Some choose to patent their programs - analogous to a machine or manufactured product - controlling not how they are used but the freedom of competitors to release similar products.

There is a lot of discomfort in the industry with the idea that patents should apply to software. On the other hand, there is also distaste when a competitor duplicates a developer's software, exactly the thing patents are supposed to protect against.

With neither copyright nor patent systems being sufficient, many proprietary software companies turn to trade secrets. Rather than selling their software, they license it to customers on the understanding that they are not allowed to disassemble or otherwise reverse-engineer its working. They then argue that because they have taken reasonable steps to protect their program's function from publication, it should be considered a trade secret - analogous to their customer list or the additives in the oil used by KFC.

...and some discussions on software ownership end there. Software is a form of intellectual property, they argue, and we already have three ways to copy with that legally: patents, copyright, and trade secrets. A nice story, except that we can quickly think of some more.

If copyright is how works of art are protected, then we have to acknowledge that not all works of art are considered equal. Some are given special protection as trade marks: exclusive signs of the work or place of operation of a particular organisation. Certain features of a product's design are considered similarly as the trade dress of that product. Currently the functionality of a product cannot be considered a trademark or trade dress, but what would be the ramifications of moving in that direction?

We also have academic priority. Like the patent system, the academic journal system is supposed to encourage dissemination of new results (like the patent system, arguments abound over whether or not it achieves this aim). Unlike the patent system, first movers are not awarded monopoly, but recognition. What would the software industry look like if companies had to disclose which parts of their products they had thought of themselves, and which they had taken from Xerox, or VisiCorp, or some other earlier creator? Might that discourage Sherlocking and me-too products?

There's also the way that nuclear propagation is controlled. It's not too hard to find out how to build nuclear reactors or atomic weapons, particularly as so much of the work was done by the American government and has been released into the public domain. What it is hard to do is to build a nuclear reactor or atomic weapon. While the knowledge is unrestricted, its application is closely controlled. Control is overseen by an international agency, part of the United Nations. This has its parallels with the patent system, where centralisation into a government office is seen as one of the problems.

The point of this post is not to suggest that any one of the above analogues is a great fit for the problem of ownership and competition in the world of software. The point is to suggest that perhaps not all of the available options have been explored, and that accepting the current state of the world "because we've exhausted all the possibilities" would be to give up early.

#### Intra-curricular activities
I'm apparently fascinated by the idea of defining curricula for learning programming. I've written about how we need to be careful what we try to pay forward from the way we learned in the past, and I've talked about how we do need to pay it forward so that the second hundred years see faster progress than the first hundred years.

I'm a fan (with reservations, as seen below) of the book series as a form of curriculum. Take something like Kent Beck's signature series, which covers a decent subset of both technical and social approaches in software development in breadth and in depth. You could probably imagine developers who would benefit from reading some or all of the books in the series. In fact, you may be one.

Coping with people approaching the curriculum from different skill levels and areas of experience is hard. Not just for the book series, it's hard in general. Universities take the simplifying approach of assuming that everybody wants to learn the same stuff, and teaching that stuff. And to some extent that's easy for them, because the backgrounds of prospective students is relatively uniform. Even so, my University course organised incoming students into two groups; those who had studied complex numbers at A-level and those who had not. The difference was simply that the group who had not were given a couple of lectures on complex numbers, then it was assumed that they also knew the topic from the fourth week.

Now consider selling a programming book to the public. Part of the proposal process with all of the publishers I've worked with has been describing the target audience. Is this a book for people who have never programmed before? For people who have programmed a little, but never used this particular tool or technique? People who have programmed a lot but never used this tool? Is this thing similar to what they have used before, or very different? For people who are somewhat familiar with the tool? For experts (and how is that defined)? Is it for readers comfortable with maths? For readers with no maths background?

Every "no" in answer to one of those questions is an opportunity to improve the experience for a subset of the potential audience by tailoring it to that subset. It's also an opportunity to exclude a subset of the audience by making the content less relevant to them.

[I'll digress here to explain how I worked that out for my books: whether it's selfishness or a failure of empathy, I wrote books that I wanted to read but that didn't exist. Therefore the expected experience is something similar to mine, back when I filled in the proposal form.]

Clearly no single publication will cover the whole phase space of potential readers and be any good. The interesting question is how much it's worth covering with multiple publications; whether the idea of series-as-curriculum pulls in the general direction as much as scope-limiting each book pulls in the specific. Should the curriculum take readers on a straight line from novice to master? Should it "fan in" from multiple introductions? Should it "fan out" in multiple directions of interest and enquiry? Would a non-linear curriculum be inclusive or offputtingly confusing? Should the questions really be answered by substituting the different question "how many people would buy that"?

#### Introducing the SICPers Newsletter
I write a lot about software engineering. I talk a lot about software engineering. And I read a lot about software engineering.

And that stuff is scattered all over the interwebs. Well, some of it isn't even there, it's in notebooks. Which is a shame, because I think there are interesting conversations to be had if we talk about it some more.

Introducing the SICPers newsletter. A regular (currently fortnightly, first issue Feb 18th) collection of things I've said and heard about the world of software. This is supposed to be a conversation starter, not a monologue, so please do sign up, please do reply to the emails, and please do suggest topics for inclusion.

#### Introducing: the Labrary
Is it that a month in the laboratory will save an hour in the library, or the other way around? A little more conversation, a little less action?

There are things to learn from both the library and the laboratory, and that's why I'm launching the Labrary, providing consulting detective and training service to software teams who need to solve problems, and to great engineers who want to be great lead engineers, principal engineers and architects.

The Labrary is also the home to my books and other projects to come. So if you want to find out what a consulting detective can do for your team, follow the @labrarian on Mastodon or book office hours to talk things over.

#### Intuitive is the Enemy of Good
In the previous instalment, I discussed an interview in which Alan Kay maligned growth-restricted user interfaces. Here's the quote again:

  There is the desire of a consumer society to have no learning curves. This tends to result in very dumbed-down products that are easy to get started on, but are generally worthless and/or debilitating. We can contrast this with technologies that do have learning curves, but pay off well and allow users to become experts (for example, musical instruments, writing, bicycles, etc. and to a lesser extent automobiles).

This is nowhere more evident than in the world of the mobile app. Any one app comprises a very small number of very focussed, very easy to use features. This has a couple of different effects. One is that my phone as a whole is an incredibly broad, incredibly shallow experience. For example, one goal I want help with from technology is:

  As an obese programmer, I want to understand how I can improve my lifestyle in order to live longer and be healthier.

Is there an app for that? No; I have six apps that kindof together provide an OK, but pretty disjointed experience that gets me some dissatisfying way toward my goal. I can tell three of these apps how much I run, but I have to remember that some subset can feed information to the others but the remainder cannot. I can tell a couple of them how much I ate, but if I do it in one of them then another won't count it correctly. Putting enough software to fulfil my goal into one app presumably breaks the cardinal rule of making every feature available within two gestures of the app's launch screen. Therefore every feature is instead hidden behind the externalised myriad gestures required to navigate my home screens and their folders to get to the disparate subsets of utility.

The second observable effect is that there is a lot of wasted potential in both the device, and the person operating that device. You have never met an expert iPhone user, for the simple reason that someone who's been using an iPhone for six years is no more capable than someone who has spent a week with their new device diligently investigating. There is no continued novelty, there are no undiscovered experiences. There is no expertise. Welcome to the land of the perpetual beginner.

Thankfully, marketing provided us with a thought-terminating cliché, to help us in our discomfort with this situation. They gave us the cars and trucks analogy. Don't worry that you can't do everything you'd expect with this device. You shouldn't expect to do absolutely everything with this device. Notice the sleight of brain?

Let us pause for a paragraph to notice that even if making the most simple, dumbed-down (wait, sorry, intuitive) experience were our goal, we use techniques that keep that from within our grasp. An A/B test will tell you whether this version is incrementally "better" than that version, but will not tell you whether the peak you are approaching is the tallest mountain in the range. Just as with evolution, valley crossing is hard without a monumental shake-up or an interminable period of neutral drift.

Desktop environments didn't usually get this any better. The learning path for most WIMP interfaces can be listed thus:

cannot use mouse.
can use mouse, cannot remember command locations.
can remember command locations.
can remember keyboard shortcuts.
???
programming.

A near-perfect example of this would be emacs. You start off with a straightforward modeless editor window, but you don't know how to save, quit, load a file, or anything. So you find yourself some cheat-sheet, and pretty soon you know those things, and start to find other things like swapping buffers, opening multiple windows, and navigating around a buffer. Then you want to compose a couple of commands, and suddenly you need to learn LISP. Many people will cap out at level 4, or even somewhere between 3 and 4 (which is where I am with most IDEs unless I use them day-in, day-out for months).

The lost magic is in level 5. Tools that do a good job of enabling improvement without requiring that you adopt a skill you don't identify with (i.e. programming, learning the innards of a computer) invite greater investment over time, rewarding you with greater results. Photoshop gets this right. Automator gets it right. AppleScript gets it wrong; that's just programming (in fact it's all the hard bits from Smalltalk with none of the easy or welcoming bits). Yahoo! Pipes gets it right but markets it wrong. Quartz Composer nearly gets it right. Excel is, well, a bit of a boundary case.

The really sneaky bit is that level 5 is programming, just with none of the trappings associated with the legacy way of programming that professionals do it. No code (usually), no expressing your complex graphical problem as text, no expectation that you understand git, no philosophical wrangling over whether squares are rectangles or not. It's programming, but with a closer affinity with the problem domain than bashing out semicolons and braces. Level 5 is where we can enable people to get the most out of their computers, without making them think that they're computering.

#### iPh0wnage
I just reviewed a blog post I wrote for Graham Cluley a while back, in which I looked at the impact a common vulnerability on the iPhone and Mac would have. I think in the run-up to the iPad's release, it's a risk worth bearing in mind.

#### Irresponsible tolerance
Context

@unclebobmartin said:

One of the bad behaviors that destroys projects is "irresponsible tolerance". Tolerating what you know you should fix.

This triggered a discussion between @phil_nash and myself. As far as this got on the Twitters, we agreed that it's not necessarily irresponsible to ignore a problem for now as long as what you're actually doing is deferring the fix until you've got time…except that it's easy for deferral to slip into tacit acceptance as other work comes up. We may even be able to delude ourselves into thinking we still intend to fix that issue "some day", even though the reality is that will never happen.

My >140char response

Yes, that is easy to do. I've done it myself. I've even - though not in a number of years - used tolerance of a badly-written component as an excuse to avoid not just cleaning it up, but of doing other useful work on the same component. "Touching that spaghetti code would be too risky, and rewriting it would take too long, so let's just leave it as it is."

Since reading the GTD book, I've tried a new approach which has, for the most part, been more successful. It's not exactly a GTD technique, but borrows the spirit. In GTD there's a two-minute rule: if you think of something you need to do that would take less than two minutes, just do it. If it would take longer, add it to your backlog.

The analogous approach for refusing to tolerate software problems is this: if you see something you think needs fixing, and you have time to fix it now, fix it now. If you do not have time to fix it, write a bug report.

What goes into the bug report?

All of the things a good software architect should be logging as part of their work anyway: a description of the problem, discussion of potential solutions, choice of solution and justification of that solution.  So if there's some ugly class that needs rewriting, explain why it's ugly. Describe what would be better, and why.

What do I get from this?

In the first instance, the act of describing what it is that you dislike about the current code often makes it easier to see that the fix actually wouldn't take too long. So that really disgusting class is full of long methods: what's three minutes with the "extract method" tool between friends?

Oftentimes the solution will still be too big to work on right now. So hit the "report" button, and get the bug report into the tracker (or the backlog, or icebox, or whatever this week's cool term is. I can no longer keep up; I'm in my thirties now). You know how they say a problem shared is a problem halved? It's crap. A problem shared is a problem everyone is burdened with, so there are more people to go "oh crap, yeah, I hate that too". Maybe one (or more) of you has the time to spend a day or two sorting the issue out, or is willing to make time. Maybe someone else knows enough about that code to propose a better alternative.

Even if not, the whole team can no longer ignore the issue: every time someone looks at the outstanding issues, there's your problem, reminding everyone not to tolerate it. It's harder to say "oh yeah, I thought about fixing that once but I didn't have the time" if every time you read the bug list you are forced to think about it again. One of these days you, or someone else, will have time to fix it, and so will have to either do that or think of a convincing excuse to shelve it again. Then explain at the next bug review time why the issue is still there. If you wrote a good justification for why the proposed solution would be better, it looks like you're actively trying to avoid making a better product.

Depending on how you track bugs, you may have an additional benefit: the ability to link your complaint to other issues. So maybe (and this is a real-world example from my experience), the problem is that a class for reading files has a hard-coded list of search paths. Then a request comes in saying that an extra filesystem has been provided by IT, and they want to put some of the files into a location on this new filesystem. Link them. Someone will be assigned the user request that's been prioritised as a business issue, and when they do they'll see your report that a good way to fix the problem would also clean up the product, so they can do both at the same time. If the issue is linked to enough problems in the product, it becomes clear that addressing the underlying issue will benefit the customers and the work will be scheduled. Then you really have no excuse for not finding the time to fix it: it's your job.

So this is a silver bullet, is it?

No. It's worked well for me, but it's not foolproof. Some projects I've worked on have suffered from a form of bug tracker malaise, where the backlog is so great that it's easy to ignore the vast number of open issues—some of which are no longer relevant— meaning that adding another straw to the haystack isn't going to help anyone. That's an extreme position for a project to get into, it's basically a slippery-slope version of Uncle Bob's "irresponsible tolerance" where even problems being reported by customers can be tolerated. In those cases, a special injection of enthusiasm into the development team is required: the whole product is already on a death march.

For most projects, though, reporting an issue is a good way to avoid ignoring that issue.

#### Irritating annoying irritationnoyance of the day
WebObjects generated components use a style like this:

    public Main(WOContext context) {        super(context);    }

But if you add a key using WOBuilder, then its accessors use the following style:

    public String username()    {        return username;    }

Mmm...you can almost smell the consistency.

#### Is Foundation a Utopian vision?
Important: while I'm only talking about the Foundation books in vague details here, I will end up summarising a number of key points through the whole series. If you haven't read them, and intend to, I recommend not reading this post, yet.

Here, at the start of the third TV series of Foundation, I'm actually talking about the series of short stories and novels on which the TV show is based. The TV series takes its leave of the source material in many ways—most of them necessary to produce something watchable in 2025. For example, you don't get a female character in the novels for a very long time, and you have to wait until the second book to encounter one that Asimov names. Also, atomic ashtrays no longer represent the pinnacle of technological advancement.

The Foundation series of books starts with a collection of short stories about the titular Foundation; indeed the first volume consists of stories that had already been published elsewhere, and which are intended to be read separately but describe events in the same timeline.

Hari Seldon comes up with the field of psychohistory, a branch of mathematics that explains human activity collectively without explaining how any individual acts, in a similar way to how statistical thermodynamics tells you the behaviour of heat transfer in a system without describing the story of a single molecule.

This psychohistory predicts that the great Galactic Empire will crumble, causing millennia of human suffering—or a single millennium, if the Foundation intervene and guide the creation of a new society, using the calculations of psychohistory as a blueprint. Seldon runs the maths, and identifies a number of crisis points that this transition will go through, recording talks for the Foundation to play at the times when these crises occur.

So far, this sounds incredibly Marxist. The Empire takes the place of, well, imperialism, grand in its reach and advances but ultimately destined to destruction through its own contradictions. Psychohistory represents historical materialism, a scientific examination of the currents of history that identifies how these contradictions lead to the collapse of imperialism and set out the blueprint for a transition to a just society. As in Marx's XVIII Brumaire of Louis Napoleon, there are no great men of history, "they do not make [history] under self-selected circumstances, but under circumstances existing already, given and transmitted from the past." Though as Napoleon was seen by some (other than Marx, obviously) as the Great Man of European history, so the Mule disrupts the statistical flows of psychohistory.

Then things take a turn, presumably because Marxism fell out of favour (to the extent it ever was in favour) in the America in which Asimov spent most of his life. From Foundation's Edge onwards, even the Foundation's psychohistory isn't the end goal of society, but a Gaia-esque shared consciousness in which all of humanity participates. In its presentation, we learn of this as a sort of harmony-of-nature, in-tune-with-the-universe gestalt existence that sounds very much like a utopian way out of the struggles and conflicts of human existence, and the climatic destruction of a species that doesn't take its home planet seriously.

Utopian, that is, until we learn the final revelation that Asimov wrote to tie all of his great science fiction series together, in the fifth (and chronologically final, though two prequels follow on) novel, Foundation and Earth. That is, in modern parlance, that the whole lot—the creation of psychohistory by Seldon, the Foundations, and Gaia—were put in motion by what we would now call an Artificial Intelligence, the robot R. Daneel Olivaw. It was Olivaw's drive to create a better galaxy for humanity that led to the events of the preceding novels, and his vision of how that world should work that culminates in Gaia and the shared consciousness.

The interpretation of this point has probably shifted a lot in the intervening decades, and we even see that shift in other science fiction works. Through Asimov's Robots series we see AIs as generally benevolent servants of humanity, programmed to do the right thing even if sometimes the programming leads to some surprising events. The main enemy of progress in that series isn't the robots themselves, it's the inability of most people to accept that they can exist and work alongside humans. If the spongy, platinum-iridium positronic brain of a robot comes up with the idea of Gaia, then it must be a good idea.

However, fast-forward to the present day—pausing to watch the horrors of the Cybermen on Doctor Who, the Borg on Star Trek, the Matrix, and most episodes of Black Mirror—and we'd have a very different take on "computer decides that the best outcome for humanity is to remove individuality and for everyone to be connected in a shared consciousness." How often is individuality seen not as a weakness, but as the very point of humanity? Would Asimov add yet another denouement to the Foundation series today? Would he perhaps go back to the original outcome of psychohistory, and the triumph of intellectualism and careful thinking over demagoguery; or would that conclusion be too technocratic for today's readers?

#### Is Freedom Zero such a hot idea?
I've been thinking lately that if we don't want to work on the databases that extremist governments use to detain immigrants they have separated from their children, or on the operating systems that well-equipped militaries used to rain autonomous death from above, or the image processing tools used by mass surveillance networks, then we need to stop dishing out the freedom to run the program as you wish, for any purpose.

That also means discriminating against fields of endeavour, so such software would not only not be Free Software but not even Open Source. I'm OK with that, if it also means that it's not used for purposes I don't want to support. I still think Free Software is better than closed proprietary software, but have come to believe that Free Software is the amoral option where what our field needs is morality.

I don't know what this would look like. I do not believe it would look like The JSON Licence, which is open to misinterpretation (intentional or otherwise). I think it would have to be a licence that enabled studying, sharing and modification of the software, but that explicitly forbade any use for any purpose that isn't studying, modifying or sharing. With a "contact me or my agent, tell us what you're doing, and we'll decide whether to grant you an additional licence for use" suffix. This is more open than closed proprietary software, but no more available for deployment to bad actors.

Yes, that can be abused, but that doesn't mean it's not worth trying.

#### Is it an anti-pattern to use properties everywhere?
I've seen questions about whether to always provide accessors for ivars, and recommendations, such as in akosma software's ObjC code standards, that say Whenever possible, do not specify ivars in the header file; use only @property and @synthesize statements instead.

This isn't how I work, which led me to ask the question: is this a good recommendation? Obviously the question of whether coding standards are "good" or "bad" is subjective, so what I'm really asking is whether this is something I'd want to do myself.

How I currently work

If I need to be able to see the state of an object from outside that object, I'll make a readonly property. If I need to be able to change the state of an object then I'll create a readwrite property. Whether these are synthesised or dynamic properties depends on how I need to compute their values.

If, in implementing a method, I find I need to make use of some object or value that was generated in another method, I'll create an instance variable to store the value away in the other method so it can be used from this method. I define this ivar in the @implementation of the class.

This effectively acts as a de facto distinction between public and private data. Public data is accessed via properties which can be seen and called from anywhere; private data is accessed via ivars that cannot be seen anywhere except inside the current class (don't worry about the visibility modifiers @public and friends: because the ivars are inside the implementation even subclasses don't know what they are).

If I consistently use the property accessors even inside a class, I can see where an object is making use of state that is accessible to the outside world. That can indicate an encapsulation failure, making the class fragile to external prodding. Of course, it's also expected that objects do get told about the outside world, so it's not an automatic fail to do this: but it's useful to be able to see where it happens.

About the "properties everywhere" approach

As I see it, there are benefits and drawbacks to that technique. The pros:

Encapsulate memory management. This is less of a benefit with automatic reference counting or garbage collection, but in a manually reference-counted environment you can put the memory management semantics in one place rather than sprinkling your code with retain, release and copy calls.
Consistency. Rather than having two distinct techniques for accessing ivars, you have just one.
Properties are new(ish), and ivars are old and busted. :-)

The cons:
Broken encapsulation. Now all of your ivars have accessors, and as previously discussed here Objective-C doesn't have method visibility modifiers. All accessors are public (even if declared in the class extension), so any dev armed with a copy of class-dump might decide to change the internal state of your classes.
Consistency. This is where it gets subjective, because this was also a pro, but as I said earlier I deliberately make a distinction between internal state and externally-available properties, so making both of these the same is a problem.
Writing code you don't need. Even where you never actually use an ivar outside of the object that owns it, you're still writing public methods for that ivar.

Conclusion

I'm not going to be adopting the "everything is a property" recommendation, I value the privacy of my parts. I'll carry on with writing instance variables where I need them, and "promoting" them to properties with accessors where that's necessary.

#### Is privacy a security feature?
I've spoken a lot about privacy recently: mainly because it's an important problem. Important enough to hit the headlines; important enough for trade associations and independent developers alike to make a priority. Whether it's talks at conferences, or guiding people on designing or implementing their apps, there's been a lot of privacy involved. But is it really on-topic for a security boffin?

The "yes" camp: Microsoft

In Michael Howard and David LeBlanc's book, "Writing Secure Code, Second Edition", there's a whole chapter on privacy:

Most privacy threats are information disclosure threats. When performing threat analysis, you should look at all such threats as potential privacy violations.

In this view, a privacy problem is a consequence of a failure of confidentiality being disrupted. You model your application, taking into account what data it protects, what value the customers put on that data, and how important it is to protect the confidentiality. Personally-identifying information is modelled in exactly this way.

Privacy automatically falls out of this modelling technique: if people can get access to confidential data, then you have a privacy violation (that also looks like a security vulnerability because it appears in your threat model).

The "no" camp: Oh, it's Microsoft again

A different viewpoint is expressed in another book by Michael Howard (with Steve Lipner this time): "The Security Development Lifecycle".

Many people see privacy and security as different views of the same issue. However, privacy can be seen as a way of complying with policy and security as a way of enforcing policy. […] Privacy's focus is compliance with regulatory requirements[…], corporate policy, and customer expectations.

So in this model, privacy is a statement of intent, and security is a tool to ensure your software follows through on your intent. It's the difference between design and implementation: privacy is about ensuring you build the right thing, and security helps you build the thing right. The two have nothing to say about each other, except that if you didn't get the security right you can't make any claim about whether the policy expressed in the privacy requirements will successfully be met in deployment.

The "who cares?" camp: me

The argument above seems to be a question of semantics, and trying to apportion responsibility for different aspects of development to different roles. In fact, everyone involved in making a product has the same goal - to make a great product - and such niggling is distracting from that goal.

Most of my professional work fits into one of a few categories:
Learning stuff
Making stuff
Helping other people make better stuff
Making other people better at making stuff than I am

So if, in the process of helping someone with their security, I should be able to help with their app's privacy too, should I really keep quiet until we've solved some quibbling point of semantics?

#### Is software engineering a thing?
In the title I'm kindof punning on the word "a" (it's my blog, and I get to do what I want). Is there a single thing, software engineering, that all people making software should (or could, or would find to be beneficial) do?

It's a question that's sat with me all through my doctoral research, and has sat with the field for much longer. In 2007, Diane Kelly argued that there's a "chasm" between scientific computing and commercial software, and that leaders should get together and identify their differences to come up with discipline-specific approaches to writing software. In other words, that there isn't a universal "software engineering" that's the best approach to writing software.

A decade later, Tim Storer described the chasm as a skills gap that needed to be closed. In this view, software engineering is the application of software knowledge to the production of software, and computational scientists don't have enough of that knowledge to do it correctly.

There's a whole community of research devoted to uncovering a grand unified theory of software engineering, in analogy to the Grand Unified Theories of physics that unite the electromagnetic and weak and strong nuclear forces. Members of this community (which goes by the name SEMAT: Software Engineering Methods and Theory) start not by constructing their theory but by deconstructing others.

They argue (convincingly) that any particular software engineering methodology is flawed, because it recommends a whole suite of practices but we don't know which are relevant and useful, or how they interact, we just know that the methodologists managed to trademark their particular grab bag of practices and argue that if you're not making enough software, you're not doing it the way they propose. While there might be something to daily stand-ups, or to organising work by sprints, or to holding retrospectives, there's nothing to Scrum because selecting all of these practices together is entirely arbitrary.

What the SEMAT folks argue for instead is more of a systems approach to software (in a Dana Meadows sense rather than a Jerry Weinberg sense): the team use their way of working to do work to generate a software system that realises requirements to exploit an opportunity identified by stakeholders; which part of that process is the most broken and what can you do to make it less broken than some other part?

I think that's a great way to think about it, and I also don't think that a GUT of software engineering will arise from it. To me, software is the reification of thought in a reusable and somewhat abstracted structure: we understand something about a context and try to capture our (or, commonly, someone else's) understanding of that context in a way that can be automatically calculated using digital electronic equipment. To say that a universal theory of making software exists is to say that a universal theory of understanding thought exists, and we aren't there.

Many of the open problems in software engineering boil down to not being able to capture thoughts precisely. Software engineering ethics is the inability to define universal rights and wrongs (which may not exist anyway). Software quality management is the inability to agree what the understanding was and whether we've captured it correctly. The fact that we don't agree on whether object-oriented, structured, functional, or some other approach to analysis and design is the best choice is a sign that we don't agree on how to encode thought in a way that we can think about.

In other words, software construction is thinking about thought, it is meta-thought. And we don't agree enough on how thought works to be able to get consensus on the best way to think about thought, let alone the best way to encapsulate that thinking about thought in a saleable product.

#### Is spec-driven development the end of Agile software development?
A claim that I've seen on software social media is that spec-driven development is evidence that agile was a dark path, poorly chosen. The argument goes that Agile software development is about eschewing detailed designs and specifications, in favour of experimentation and feedback. Spec-driven development shows that the way to unlock maximal productivity in augmented development (and therefore in development overall, as LLM agents can type faster than programmers can) is by writing detailed specifications for the code generator to follow. Therefore detailed specifications are valuable after all, so Agile was wrong to do away with them.

Let's take a look at what's going on in—and behind—this argument.

Agile is alleged to eschew detailed specification.

It certainly looks that way, on a quick glance. As Brian Marick noted, one of the Agile Alliance didn't care what was in the manifesto so long as there was no way IBM or Rational agreed with it. The people who came together for that skiing holiday in Utah were coaches and practitioners of ‘lightweight methodologies’, that favoured early delivery of working software to the customer over detailed product requirements documents, and design specifications, that the customer reviews and signs off before they ever see any code—indeed before any developer is allowed to begin coding.

The reason for this isn't to create unspecified software. It's to discover the specification through iterative feedback. If you compare the output of a team that follows Agile software development with one that follows a then-prevailing methodology, such as the Rational Unified Process (RUP) that irked one of the alliance members, you'd probably learn that the agile team actually has a much more detailed specification.

In addition, that spec is machine-readable, executable, and generates errors whenever the software falls out of conformance. The specification these teams produce through their emergent process is the test suite. If every software change follows the creation of a failing test, then all of the software—every feature, every bug fix—is specified in a detailed document.

Three evident differences between the ‘specs’ in something like RUP, and something agile:

The agile spec's format is closer to working software, which is what the customers ultimately value.

The agile spec has no ambiguity: the software meets the spec and the test passes; or it doesn't, and the test fails.

The agile spec evolves throughout the life of the software endeavour, capturing elements of a dialectic between creators, customers, and software. Meanwhile, according to figure 1.5 in The Unified Software Development Process, ‘rational’ developers finish elaboration at an early point in the project and move on to filling in the details.

And one big difference between the purpose of the tests: RUP creates tests for verification: did we build it right in the implementation phase? Agile teams create tests that also supply validation: do we understand what we're being asked to build?

Aside: the “specs” in spec-driven development occupy the same conceptual space as tests created to drive out design.

A developer listens to a description of behaviour that their software should embody, and writes a test to drive out the details and confirm they understand the situation. When happy that the test represents a description of the code they need to create, they write the code.

A developer listens to a description of behaviour that their software should embody, and writes a document to drive out the details and confirm they understand the situation. When happy that the document represents a description of the code they need to create, they create the code.

These are the same statement, made at different levels of abstraction with respect to the tools the developer uses. In other words, the people are doing the same thing, using different tools. If you “have come to value individuals and interactions over processes and tools”, then you will probably think that there is some value in the tools; but not as much as there is value in their application. Speaking of which…

The agile manifesto says that the alliance members value specifications.

The template statement in the manifesto is “[blah blah blah] we have come to value x over y. That is, while there is value in ys:[y], we value xs:[x] more.”

The instance that's immediately relevant to the spec-driven development argument is x = working software; y = comprehensive documentation, where detailed specification is an example of comprehensive documentation (particularly in the prolix style adopted by many of the models). Performing the substitution, detailed specification is indeed valuable, but not as an end in itself: it is (or can be) useful in pursuit of delivering valuable software.

Agile software development isn't about not doing things; it's about understanding why you do anything, and being ready to try something else in pursuit of your goal. With the constraint that your goal needs to be “early and continuous delivery of valuable software”.

Returning to the aside, above, x = individuals and interactions and y = processes and tools. Spec-driven development is a process, generative AI is a tool; the point isn't to use or avoid either, they can be valuable in the pursuit of working software.

Agile software development was about who makes software and how.

The comparison between RUP and lightweight methodologies made above was particularly apposite at the moment the manifesto for Agile software development was created; a single moment that highlighted a tension in a dichotomy. It isn't the opposition of documentation and software, or change and plans. It's the opposition of practitioner-led versus managerial software development.

The summary of the principles behind the manifesto is approximately ‘get people who know what they're doing, let them talk to the customer and give the customer software, then get out of their way’. The list has an emphasis on technical excellence, simplicity, customer collaboration, and—crucially—self-organisation, all seconded to the ‘highest priority’ of customer satisfaction through valuable software, with ‘working software’ as the primary measure of progress.

In other words, we promise to make software, if you promise to let us.

The prevailing, heavyweight processes were predicated on a breakdown of this compact. Managers don't trust developers not to gold-plate software for its own sake, continually polishing but never shipping. Therefore, it's up to managers to work out what needs to be done and to keep developers' noses pressed to the code face until they do it.

Bring that up to date, and every software organisation now pays lip service to agile software development, and yet the opposition between practitioner-led and managerial software development still exists. When developers of the 2020s complain about agile, they typically complain about facets of managerial agile: valueless ceremonial meetings; OKRs as the primary measure of progress; and perpetual sprinting with no warm-ups, cool-downs, or rests.

All of which is to say that the story of practitioner-driven software development remains partially told, and that whether spec-driven development contributes to its continuation is only a small part of the question; a question that remains open-ended.

#### Is TDD Dead? My questions
These are my questions for parts 5 and 6 of Is TDD Dead?. I'd like to start by thanking the panellists for publishing their discussions.

TDD the Principle

Kent and Martin, why is it that you practise test-driven development? What do you get from it? David, how do you get those same things?

What could change about the way we write software to make TDD redundant or obsolete for Kent and Martin? What could change about the way that TDD is performed to make it useful or beneficial for David?

TDD the Practise

David, is there a way in which we could retain the test-first idea of TDD, but avoid the design problems you encounter in the production code? Are you able to distil a short list of design guidelines that avoid your "design damage"? Kent and Martin, how could TDDers follow those guidelines in their work?

TDD the Community

How have all three of you felt about the community reaction to this discussion? What has been good? What has been bad? What has been missing? Is there information the community could supply to help developers evaluate and choose or discard TDD for their work that is missing? Are there questions developers should ask before choosing whether to adopt or avoid TDD that you believe are not being raised?

#### It depends? It depends.
Sometimes you ask a question which has a small collection of actionable answers: yes or no. You ask someone who should be able to give that yes or no answer, and they go for the third: it depends.

Maybe they can't actually answer your question, but want to sound profound.

Maybe they don't realise that you're a beginner in this particular area and would benefit from knowing what they'd do in this situation.

Maybe they don't realise that you're an expert in this area and are looking for confirmation or refutation of your existing choice.

Maybe they don't realise that you're uninterested in this area, and just need a decision made so you can move on.

Maybe they don't know what the answer is, but are hoping you'll help them think through the alternatives.

Sometimes, the answer to a question is "it depends". But is that the answer you should give? It depends.

#### It doesn't take an Oracle to see that coming
Today has largely been brought to you by nostalgia brought about by this article, reporting on a get-together of former Sun Microsystems employees.

I have never been a former Sun Microsystems employee, and of course now I never will be one. Of all the tech companies I've interacted with, Sun is the one I most regret not getting to work with. By the time I dealt with them, they had already put the "crash" in "dot-com crash" but there was still a feeling that they made great things. And besides, they showed that even a pony-tailed Objective-C programmer can be a tech CEO.

I recently talked about the importance of GNU projects, but plenty of other software projects were also important, and Sun had a hand in quite a few of them:

Bill Joy worked for them, and most of their early workstation operating systems were based on BSD Unix.
In fact while Apollo may have invented the idea that a single person might use a Unix computer, Sun popularised it.
I learned how to boot Macs by learning how to program Forth and boot Suns.
NFS was the beginning of the separation between your device and your documents.
NIS was a bit of an important step on the way to logging in anywhere (its level of baroqueness compared to OAuth has never been accurately gauged).
In fact, they pretty much invented cloud computing.
Java was quite a big thing for a while.
Dtrace is pretty amazing.
They even got into standard Unix workstation vendor capitalisation for a while.

It's likely that much of the interesting stuff at Sun was already over by the time I could've worked there, and I certainly experienced a very last-minute replay of some of their history. When I was a student I 'borrowed' an Ultra 5 (one of their least good workstations, pretty much a PC with a sun4u SPARC innards) and a SparcStation 5 (one of their most good) to learn about Solaris, SunOS and NeXTSTEP. But it certainly feels like a lot of the future was invented there, even if they were largely following Xerox's playbook like the rest of the industry.

So tonight, I'll remember that my control key is in the correct place:

I'll press L1 and A, then raise a glass to Sun and the job I never had.

#### It protects. It also promotes and prevents.
I sometimes get asked to review, or "comment on", the architecture for an app. Often the app already exists, and the architecture documentation consists of nothing more than the source code and the folder structure. Sometimes the app doesn't exist, and the architecture is a collection of hopes and dreams expressed on a whiteboard. Very, very rarely, both exist.

To effectively review an architecture and make recommendations for improving it, we need much more information than that. We need to know what we're aiming for, so that we can tell whether the architecture is going to support or hinder those goals.

We start by asking about the functional requirements of the application. Who is using this, what are they using it for, how do they do that? Does the architecture make it easy for the programmers to implement those things, for the testers to validate those things, for whoever deploys and maintains the software to provide those things?

If you see an "architecture" that promotes the choice of technical implementation pattern over the functionality of the system, it's getting in the way. I don't need to know that you have three folders of Models, Views and Controllers, or of Actions, Components, and Containers. I need to know that you let people book childrens' weather forecasters for wild atmospheric physics parties.

We can say the same about non-functional requirements. When I ask what the architecture is supposed to be for, a frequent response is "we need it to scale". How? Do you want to scale the development team? By doing more things in parallel, or by doing the same things faster, or by requiring more people to achieve the same results? Hold on, did you want to scale the team up or down?

Or did you want to scale the number of concurrent users? Have you tried… y'know, selling the product to people? Many startups in particular need to learn that a CRM is a better tool for scaling their web app than Kubernetes. But anyway, I digress. If you've got a plan for getting to a million users, and it's a realistic plan, does your architecture allow you to do that? Does it show us how to keep that property as we make changes?

Those important things that you want your system to do. The architecture should protect and promote them. It should make it easy to do the right thing, and difficult to regress. It should prevent going off into the weeds, or doing work that counters those goals.

That means that the system's architecture isn't really about the technology, it's about the goals. If you show me a list of npm packages in response to questions about your architecture, you're not showing me your architecture. Yes, I could build your system using those technologies. But I could probably build anything else, too.

#### It was asked for: the "features" post
Someone anonymous once said:I'm intrigued by your feature comment. Please publish said blog post!Where said comment was:The fact that I have stopped using the word 'feature' in many contexts is an entire blog post and a few therapy sessions in itself.So here, for your delectation, is that entire blog post.When you're trying to decide what software people want, and indeed how to tell them that they want whatever software they're going to get instead, that's marketing (mainly - it's partly sales, and there's yet another tangential post on why I occasionally deliberately conflate marketing and sales). Marketing works in terms of features, which for the purposes of marketing means "properties or qualities of the software which we think might make people interested in that software".When you're trying to decide what software to build, or trying to build the software, more specific terms are used. Initially people split requirements into two distinct groups, functional (what the system is capable of) and non-functional (how the system goes about its capabilities), but a more precise organisation is often needed. For instance, a requirement of system security might result in both functional and non-functional aspects of the system being specified.Of course, some or all of the capabilities are also features, in fact it's generally true that the set of all features, the set of all known requirements and the set of things the customer wants are intersecting subsets of the set of all possible qualities of a software system. Companies without an intersection between any two of these sets tend to go out of business very quickly. But the sets rarely perfectly overlap.For instance, it's a feature of Windows 7 that it's named differently from Windows Vista, because Microsoft's marketing requires that customers believe that they've put Vista behind them. However, it's also a feature of Windows 7 that it not be very distinct from Vista, because marketing require that application compatibility doesn't get broken. Hence we have the interesting situation that Windows 7 is also Windows 6.1. And if Microsoft think they're being innovative in that version numbering policy, they should try looking up the history of SunOS/Solaris version numbers.  BTW, indeed I haven't switched my SUNW tag to JAVA, because I already use the java tag to mean the Java language and the Java platform. Marketing people can be funny sometimes.Another example, less confusing though more contradictory, is Apple's Snow Leopard collateral. The fact that marketing are telling us there are no new features in Snow Leopard means that "no features" is something they believe we might want to buy, which in turn makes it a feature… confused?So anyway, I try to avoid using the word "feature" when I'm talking about software, because I'm usually instead talking about a capability or property of a software system, and not about marketing that software system. For instance, in Properties about a year on I described properties as a capability of the Objective-C 2.0 language, which indeed they are. It happens that properties is also a feature of the language (don't believe that programming languages have marketing departments? What else do Apple's tech evangelists do, if it isn't marketing?), but in the case of that post I was talking about what can be done with properties, how properties can be used, and not how they can switch developers to Leopard from Tiger or .NET.And in other news, it seems that badly-parked tech company founder Mercs are back in fashion.

#### It's about solving problems
As ever, there's a touchstone issue on the programmers' corner of the intarwebs (the programmers' corner is actually the same intarwebs everyone else is using, just we model it with geometry so it can have a corner). Here it is:

Alan Kelly predicted that by 2022, TDD will become a prerequisite for employment as a programmer. Let's leave aside for a moment the issue discussed here and in APPropriate Behaviour, that there is no arbiter of programmer employability.

Responses did not take long in coming. Some TDDers cannot believe that TDD is not already required. Some non-TDDers are incensed that non-TDD is not considered valid. I particularly like this tweet about TDD's place:

  I like to explain that a lot of [TDD is] just a substitute for a decent type system ;).

You know what? That might be true, I'm not sure. Functional programming education has, so far, been to me like a pure map of me onto a slightly older version of me without the side effect of imparting knowledge of functional programming. Bear in mind that I've been paid to write LISP in the past, and I struggle with FP. I just can't get from knowing what functions are to big-picture views of functional programming.

I wouldn't know the mathematics of a type system if it came up and quacked at me. As far as I know, Hindley is the name of a serial killer and now I'm looking warily at Milner. But I'm fine with that, and I'm fine with you not being fine with that. It's just that we think about things in different ways.

Other people think about things in other different ways too. And I'm fine with that, too. Let's look some more at tests.

Many forms of automated test follow a common pattern: Assemble, Act, Assert or Given, When, Then. Let's talk about Eiffel, a language invented by the only person I know whose opinions have a stronger identity than the person who holds them.

An Eiffel programmer would look at Given and see that this is describing a particular state of the preconditions of the system under test. Or, using theories, a range of preconditions. And they would look at Then, and observe that this is making statements about the postconditions of the system under test.

This Eiffel programmer would probably then ask why you expect them to do all of this, when they already generalised this out in designing the system's contract. Why write all these tests when the program itself can evaluate its own preconditions and postconditions as it's chugging along, and you can design to those?

Just as this starts to sound reasonable, a Prolog programmer asks why you'd ever write the When bit at all. Surely you just tell the computer what you've got and what you want, and it works out how to get there? (I tried, and it said "NO." Oh well.)

There are loads of different ways to write software. Many of these are not better nor worse than others. In fact, to even have that discussion you'd need to be able to express and agree upon what "better" means, and I'm not sure we're there. Some are prevalent because they let people reason about their problems in a way that feels comfortable, or efficient. Others are prevalent because a vendor threw lots of money into marketing. Some of them feel comfortable or efficient because we've become used to thinking in those ways, rather than because they were a natural fit onto our ab initio thoughts.

But remember, your goal is not to write software. Your goal is to solve problems, and often software is part of the solution. The tools, practices and principles we have now may be adequate, but they may not be best. They might suit some of us, and not others.

That's fine. I know how I like to work, and I'm happy to help other people find out about it and try it out. But I'm also happy to find out about and try out other things. I'll champion what I do, while learning about what others do. And all along I'll champion the higher-level goal of solving problems, and the professional standard of being able to demonstrate confidence in our solutions.

Maybe I should go and read about type systems now.

#### It's about the thinking
At some point in the past, programmers used to recommend drawing flowcharts before you start coding. Then they recommended creating CRC cards, or acting through how the turtle will behave, or writing failing tests, or getting the types to match up, or designing contracts, or writing proofs, but the point is that in each case they're there for eliciting thought before the code gets laid down.

None of these things is mutually exclusive, none of these things is the one true way, but the fact that they all isolate some part of solving the problem from some part of coding the solution is the telling point. The problem is not having the correct type system or test coverage or diagram format, the problem is trying to work in two (or more) levels of abstraction - the problem domain and the computer - at the same time.

#### It's just a big iPod
I think you would assume I had my privacy settings ramped up a little too high if I hadn't heard about the iPad, Apple's new touchscreen mobile device. Having had a few days to consider it and allow the hype to die down, my considered opinion on the iPad's security profile is this: it's just a big iPod.

Now that's no bad thing. We've seen from the iPhone that the moderated gateway for distributing software&mdash;the App Store&mdash;keeps malware away from the platform. Both the Rickrolling iKee worm and its malicious sibling, Duh, rely on users enabling software not sanctioned through the app store. Now whether or not Apple's review process is a 100% foolproof way of keeping malware off iPhones, iPods and iPads is not proven either way, but it certainly seems to be doing its job so far.

Of course, reviewing every one of those 140,000+ apps is not a free process. Last year, Apple were saying 98% of apps are reviewed in 7 days, this month only 90% are approved in 14 days. So there's clearly a scalability problem with the review process, and if the iPad does genuinely lead to a "second app store gold rush" then we'll probably not see an improvement there, either. Now, if an app developer discovers a vulnerability in their app (or worse, if a zero-day is uncovered), it could take a couple of weeks to get a security fix out to customers. How should the developer deal with that situation? Should Apple get involved (and if they do, couldn't they have used that time to approve the update)? Update: I'm told (thanks @Reversity) that it's possible to expedite reviews by emailing Apple. We just have to hope that not all developers find out about that, or they'll all try it.

The part of the "big iPod" picture that I find most interesting from a security perspective, however, is the user account model. In a nutshell, there isn't one. Just like an iPhone or iPod, it is assumed that the person touching the screen is the person who owns the data on the iPad. There are numerous situations in which that is a reasonable assumption. My iPhone, for instance, spends most of its time in my pocket or in my hand, so it's rare that someone else gets to use it. If someone casually tries to borrow or steal the phone, the PIN lock should be sufficient to keep them from gaining access. However, as it's the 3G model rather than the newer 3GS, it lacks filesystem encryption, so a knowledgeable thief could still get the data from it. (As an aside, Apple have not mentioned whether the iPad features the same encryption as the iPhone 3GS, so it would be safest to assume that it does not).

The iPad makes sense as a single-user or shared device if it is used as a living room media unit. My girlfriend and I are happy to share music, photos, and videos, so if that's all the iPad had it wouldn't matter if we both used the same one. But for some other use cases even we need to keep secrets from each other&mdash;we both work with confidential data so can't share all of our files. With a laptop, we can each use separate accounts, so when one of us logs in we have access to our own files but not to the other's.

That multi-user capability&mdash;even more important in corporate environments&mdash;doesn't exist in the iPhone OS, and therefore doesn't exist on the iPad. If two different people want to use an iPad to work with confidential information, they don't need different accounts; they need different iPads. [Another aside: even if all the data is "in the cloud" the fact that two users on one iPad would share a keychain could mean that they have access to each others' accounts anyway.] Each would need to protect his iPad from access by anyone else. Now even though in practice many companies do have a "one user, one laptop" correlation, they still rely on a centralised directory service to configure the user accounts, and therefore the security settings including access to private data.

Now the iPhone Configuration Utility (assuming its use is extended to iPads) allows configuration of the security settings on the device such as they are, but you can't just give Jenkins an iPad, have him tell it that he's Jenkins, then have it notice that it's Jenkins's iPad and should grab Jenkins's account settings. You can do that with Macs and PCs on a network with a directory service; the individual computers can be treated to varying extents as pieces of furniture which only become "Jenkins's computer" when Jenkins is using one.

If the iPad works in the same way as an iPhone, it will grab that personal and account info from whatever Mac or PC it's synced to. Plug it in to a different computer, and that one can sync it, merging or replacing the information on the device. This makes registration fairly easy ("here's your iPad, Jenkins, plug it in to your computer when you're logged in") and deregistration more involved ("Jenkins has quit, we need to recover or remove his PIN, take the data from the iPad, then wipe it before we can give it to Hopkins, his replacement"). I happen to believe that many IT departments could, with a "one iPad&lt;-&gt;one computer&lt;-&gt;one user" system, manage iPads in that way. But it would require a bit of a change from the way they currently run their networks and IT departments don't change things without good reason. They would probably want full-device encryption (status: unknown) and to lock syncing to a single system (status: the iPhone Enterprise Deployment Guide doesn't make it clear, but I think it isn't possible).

What is clear based on the blogosphere/twitterverse reaction to the device is that many companies will be forced, sooner or later, to support iPads, just as when people started turning up to the helpdesks with BlackBerries and iPhones expecting them to be supported. Being part of that updated IT universe will make for an exciting couple of years.

#### It's just like English
Fans of the RSpec tool for writing tests will be familiar with its English-like(fn1) syntax for describing tests, which looks like this.

describe StrawMan do
  context "when interpreting a test in RSpec" do
    it "is written in plain English" do
      expect(spec).to eq(legible_text)
    end
  end
end

That's almost completely distinguishable from conversational English. Perhaps programmers just have a different idea of what English looks like than many typical speakers of English. I posit this conclusion because the gulf between "English-like" and "English" is not new. You can almost see attempts at real constructs in the English language being bashed into place in the syntax for BASIC:

  "For every number between 1 and 10, do this with the number being named 'I'…that's everything, so move on to the next value for I now."

FOR I = 1 TO 10 : … : NEXT I

And the the the Apple-recommended Definitive Guide to the AppleScript has this the the to say the about the "English-likeness" monster:(fn2)

  Personally, though, I'm not fond of AppleScript's English-likeness. For one thing, I feel it is misleading. It gives one the sense that one just knows AppleScript because one knows English; but that is not so. It also gives one the sense that AppleScript is highly flexible and accepting of commands expressed just however one cares to phrase them; and that is really not so.

Reviewing, then, we have a collection of tools that claim some similarity with English, but then fall down on every comparison except "uses some sequences of characters that have also been used in English". What went wrong? Indeed, did anything go wrong?

Programming's close analogue in natural language is the Arabic wish. Computers are much like the djinn in that you tell them what should happen and instead they make exactly the thing you asked for. You waste two attempts to converse with them on asking reasonable questions that they wilfully misinterpret, then spend forever agonising over your third and final attempt. With a djinni, it's your final attempt because you only got three wishes. With a computer, you're allowed to try as often as you like but by the third time you're realising how much more appealing a career in assassinating mythical preternatural wish-givers is looking but you don't want to take that kind of risk. Both djinn and computers are like that person who's had a restraining order ever since they decided to take "pick me up at 8" more literally than was truly warranted.

So the role of the programmer is like a kind of djinn-lawyer, translating all of the nuance and creative ambiguity of conversational language into the sort of precise, single-meaning prose that even the most belligerent of readers cannot deliberately misinterpret. And that bit of programming has not materially changed in decades. We've gone from "do exactly this", through "do exactly this but you choose how you use the register file to do it" and "do exactly this but you choose how you use the main memory to do it".

Getting computers to act like participants in a conversation is possible, but either a bit of a gimmick or limited in application. If you really wanted to build the Knowledge Navigator you'd need to fix this problem (along with the attendant acoustic engineering problems).

That is when we'll actually be able to claim success at improvement through abstraction. Not when we can give specialist djinn-linguists more abstractions, but when we can give computers enough abstractions that you no longer need to be a translator to make a computer do anything.

(fn1) Why "English-like" and not "verbal language-like"? One might chalk it up to neocolonialism and American industry deciding that English was Good Enough For Everybody. Indeed, as Matz notes, many Japanese people cannot speak English well and that adds a barrier to learning programming languages that are sort-of in English.

(fn2) Please don't write in about all of the spurious occurrences of "the" in the last (non-quote) sentence. For those of us who have used AppleScript, they can be our little in-joke.

#### It's not @jnozzi's fault!
My last post was about how we don't use evidence-based techniques in software engineering. If we don't rely on previous results to guide us, what do we use?

The answer is that the industry is guided by anecdote. Plenty of people give their opinions on whether one thing is better than another and why, and we read those opinions, combining them with out experiences into a world-view of our own.

Not every opinion has equal weight. Often we'll identify some people as experts (or "rock star thought leaders", if they're Marcus) and consider their opinions as more valuable than the average.

So how do these people come to be experts? Usually it's through the tautological sense: we've come to value what they say because they've repeatedly said things that we value. Whatever opinion you hold of the publishing industry, writing a book is a great way to get your thoughts out to a wide subset of the community, and to become recognised as an expert on the book's content (there's a very good reason why we spell the word AUTHORity).

I noticed that in my own career. I was "the Mac security guy" for a long time, a reputation gained through the not-very-simple act of writing one book published in 2010.

In just the same way, Joshua is the Xcode guy. His book, Mastering Xcode 4, is a comprehensive guide to using Xcode, based on Joshua's experiences and opinions. As an author on Xcode, he becomes known as the Xcode guy.

And here's where things get confusing. See, being an authority and having authority are not the same thing. Someone who told you how a thing works is not necessarily best placed to change how that thing works, or even justify why the thing works that way. And yet some people do not make this distinction. Hence Joshua being on the receiving end of complaints about Xcode not working the way some people would like it to work.

These people are frustrated because "the expert" is saying "I'm not going to tell you how to do that". And while that's true, we see that truth is a nuanced thing with many subtleties. In this case, you're not being blown off, it can't be done.

So yeah, the difference between being an authority and having authority. If you want to tell someone your opinion about Xcode not working, talk to someone with the authority to change how Xcode works. Someone like the product manager for Xcode. If you want to find out how Xcode does work, talk to someone who is an authority on how Xcode works. Someone like Joshua.

#### It's the little things
Today I learned that I don't even know how to Unix. I discovered that it's possible for a POSIX system to leave PATH_MAX and similar variables undefined if it truly has no restrictions on their length.

#### It's update o'clock
In other words, it's &quot;Graham remembers that someone needs to write the interwebs, too&quot; time.  The main reason I haven't written in a while is that I'm enjoying the new job, so it's satiating my hacking desires.  No hacking projects = nothing to talk about.  I can't really talk about the work stuff, either; this is unfortunate because I did some really cool KVC-cheating in a prototype app I wrote, and it'd be a good article to describe that.

I've more or less dropped off the GNUstep radar recently.  I still need to talk to the employers' legal eagles in order to update my FSF copyright assignment, and then find something to hack on.  Currently I only have a Mac OS X machine so I'd probably switch to some app-level stuff.  I have half a plan in my mind for a SOPE app, but despite reading all the documentation I still haven't worked out how to get started :-|.  The code isn't too much of a problem, I've used WebObejcts before, but WebObjects has a 'hit this button to build and run your WebObjects code' button, and there's no description AFAICT for SOPE of what to put in the GNUmakefile to get built and running SOPE code, even whether you're supposed to provide your own main() or anything.  Hmmm...I wonder if there's a sample project around...

#### Java By Contract: a Worked Example
Java by Contract is an implementation of Design by Contract, as promoted by Bertrand Meyer and the Eiffel Software company, for the Java programming language. The contract is specified using standard Java methods and annotations, making it a more reliable tool than earlier work which used javadoc comments and rewrote the Java source code to include the relevant tests.

Which is all well and good, but how do you use it? Here's an example.

The problem

There is a whole class of algorithms to approximately find roots to a function, using an iterative technique. Given, in Java syntax, the abstract type MathFunction that implements a function over the double type:

interface MathFunction {
    double f(double x);
}

Define the abstract interface that exposes such an iterative solution, including the details of its contract.

The solution

The interface is designed using the Command-Query Separation Principle. Given access to the function f(), the interface has a command findRoot(seed1, seed2) which locates the root between those two values, and a query root() which returns that root. Additionally, a boolean query exhaustedIterations() reports whether the solution converged.

Both of the queries have the precondition that the command must previously have successfully run; i.e. you cannot ask what the answer was without requesting that the answer be discovered.

The contract on the command is more interesting. The precondition is that for the two seed values seed1 and seed2, one of them must correspond to a point f(x) &gt; 0 and the other to a point f(x) &lt; 0 (it does not matter which). This guarantees an odd, and therefore non-zero, number of roots[*] to f(x) between the two, and the method will iterate toward one of them. If the precondition does not hold, then an even number of roots (including possibly zero) lies between the seed values, so it cannot be guaranteed that a solution exists to find.

In return for satisfying the precondition, the command guarantees that it either finds a root or exhausts its iteration allowance looking. Another way of putting that: if the method exits early, it is because it has already found a convergent solution.

Many, but not all, of these contract details can be provided as default method implementations in the interface. The remainder must be supplied by the implementing class.

/**
 * Given a mathematical function f over the doubles, and two bounds for a root to that function,
 * find the root using an (unspecified) iterative approach.
 * A root is an input value x such that f(x)=0.
 */
public interface RootFinder {
    /**
     * @return The function that this object is finding a root for.
     */
    MathFunction f();
    /**
     * A root to the function f() is thought to lie between seed1 and seed2. Find it.
     * @param seed1 One boundary for the root to f().
     * @param seed2 Another boundary for the root to f().
     */
    @Precondition(name = "seedGuessesStraddleRoot")
    @Postcondition(name = "earlyExitImpliesConvergence")
    void findRoot(double seed1, double seed2);
    /**
     * @return The root to the function f() that was discovered.
     */
    @Precondition(name = "guessWasCalculated")
    Double root();
    /**
     * @return Whether the iterative solution used the maximum number of iterations.
     */
    @Precondition(name = "guessWasCalculated")
    boolean exhaustedIterations();

    default Boolean guessWasCalculated() {
        return this.root() != null;
    }
    default Boolean seedGuessesStraddleRoot(Double seed1, Double seed2) {
        double r1 = f().f(seed1);
        double r2 = f().f(seed2);
        return ((r1 &gt; 0 &amp;&amp; r2 &lt; 0) || (r1 &lt; 0 &amp;&amp; r2 &gt; 0));
    }
    Boolean earlyExitImpliesConvergence(Double seed1, Double seed2, Void result);
}

Example usage

There are swaths of algorithms to implement this interface. See, for example, the book Numerical Recipes. Given a particular implementation, we can look for roots of a simple function, for example f(x) = x^2 - 2:

    RootFinder squareRootOfTwo = SecantRootFinder.finderForFunction((double x) -&gt; x*x - 2);
    squareRootOfTwo.findRoot(1.0, 2.0);
    System.out.println(String.format("Root: %f", squareRootOfTwo.root()));
    System.out.println(String.format("The solution did%s converge before hitting the iteration limit",
            squareRootOfTwo.exhaustedIterations()?"n't":""));

This suggests that a root exists at x~=1.414214, and that it converged on the solution before running out of goes. Let's see if there's another root between 2 and 3:

Exception in thread "main" online.labrary.javaByContract.ContractViolationException:
  online.labrary.javaByContract.Precondition seedGuessesStraddleRoot had unexpected value false on object
  online.labrary.jbcTests.TestGeneratorTests$SecantRootFinder@29774679
    at javaByContract/online.labrary.javaByContract.ContractEnforcer.invoke(ContractEnforcer.java:92)
    at jdk.proxy1/com.sun.proxy.jdk.proxy1.$Proxy4.findRoot(Unknown Source)
    at javaByContract/online.labrary.rootFinder.RootFinder.main(RootFinder.java:10)

Whoops! I'm holding it wrong: the function doesn't change sign between x=2 and x=3. I shouldn't expect the tool to work, and indeed it's been designed to communicate that expectation by failing a precondition.

[*] Nitpick: roots _or singularities_.

#### Just because Brucie says it...
Bruce Schneier claims that shoulder-surfing isn't much of a problem these days.Plenty of people discovered "my password" at NSConference, so I disagree :-) (photo courtesy of stuff mc).

#### Just don't use version numbers
This post on semantic versioning reminded me that we're making a future in which Ubuntu 01.04 will be newer than Ubuntu 99.10.

This is fine.

#### King Arthur: By what name are you known?
Why is it we're not allowed to call the Apple guy "Tim Apple" when everybody calls the O'Reilly guy "Tim O'Reilly"?

#### Know what counts
In Make it Count, Harry Roberts describes blacking out on stage at the end of a busy and sleepless week. Ironically, he was at the start of a talk in which he was to discuss being selective over side projects, choosing only those that you can actually "cash in" on and use to advance your career.

  If you’re going to take on side projects and speaking and writing and open source and suchlike then please, make them fucking count. Do not run yourself into the ground working on ‘career moves’ if you’re not going to cash in on them. [emphasis original]

Obviously working until you collapse is not healthy. At that point, choosing which projects to accept is less important than just getting some damned sleep and putting your health back in order. In the 1950s, psychologist Abraham Maslow identified a "hierarchy of needs", and sleep is at the base of the hierarchy meaning that, along with eating and drinking, you should take care of that before worrying about self-actualisation or esteem in the eyes of your peers.

Here's the little secret they don't tell you in the hiring interview at Silicon Valley start-ups: you're allowed to do things that aren't career-centric. This includes, but is not limited to, sleeping, drinking enough water, eating non-pizza foodstuffs, having fun, seeing friends, taking breaks, and indulging in hobbies. It sometimes seems that programmers are locked in an arms race to see who can burn out first^W^W^Wdo more work than the others. That's a short-term, economist-style view of work. I explained in APPropriate Behaviour that economists take things they don't want to consider, or can't work out a dollar value for, and call them "externalities" that lie outside the system.

Your health should not be an externality. Roberts attempted to internalise the "accounting" for all of his side projects by relating them in value to his career position. If you're unhealthy, your career will suffer. So will the rest of your life. Don't externalise your health. Worry not whether what you're doing is good for your position in the developer community, but whether it's good for you as a healthy individual. If you've got the basic things like food, shelter, sleep and safety, then validation in the eyes of yourself and your peers can follow.

#### KVO and +initialize
Got caught by a really hard-to-diagnose issue today, so I decided to write it down in part so that you don't get bitten by it, and partly so that next time I come across the issue, I'll remember what it was.I had a nasty bug in trying to add support for the AppleScript duplicate command to one of my objects. Now duplicate should, in principle, be simple: just conform to NSCopying and implement -copyWithZone:. The default implementation of NSCloneCommand should deal with everything else. But what I found was that there's a class variable (OK, there isn't, there's a static in the class implementation file with accessors) with which the instances must compare some properties. And this was empty by the time the AppleScript ran. Well, that's odd, thought I, it's only being emptied once, and that's when it's created in +[MyClass initialize]. So what's going on?Having set a watchpoint on the static, I now know the answer: the +initialize method was being called twice. Erm, OK...why? It's only called whenever a class is first used. It turns out that there were two classes with the same IMP for that method. The first was MyClass, and the second? NSKVONotifying_MyClass. Ah, great, Apple are adding a subclass of one of my classes for me!It turns out that TFM has a solution:

+ (void)initialize{    if (self == [MyClass class])    {         //real code    }}

and I could use that solution here to fix my problem. But finding out that is the problem was a complete pig.

#### Laggards don't buy apps: devil's advocate edition
Silky-voiced star of podcasts and all-round nice developer person Brent Simmons just published a pair of articles on dropping support for older OS releases. His argument is reasonable, and is based on a number of axioms including this one:

  
People who don’t upgrade their OS are also the kind of people who don’t buy apps.

Sounds sensible. But here's another take that also sounds sensible, in my opinion.

People who don't upgrade their OS are also the kind of people who don't like having to computer instead of getting their stuff done.

Let's explore a world in which that axiom is true. I'm not saying it is true, nor that Brent's is false: nor am I saying that his is true and that mine is false. I'm saying that there's an open question, and we can investigate multiple options (or maybe even try to find some data).

Developers are at the extreme end of a range of behaviours, which can be measured in a single dimension that's glibly called "extent to which individual is willing to mess about with a computer and consider the time spent valuable". The "only upgraders buy apps" axiom can be seen as an extension of the idea that all changes to a computer fit onto the high end of that dimension: if you're willing to buy an app, then you're willing to computer. If you're willing to computer, then you're willing to upgrade. Therefore anyone who wants to sell an app is by definition selling to upgraders, so you can reduce costs by targeting the latest upgrade.

Before exploring the other option, allow me to wander off into an anecdote. For a few years between about 2004 and 2011 I was active in my (then) local Mac User Group, which included chairing it for a year. There were plenty of people there who were nearer the middle of the "willingness to mess with a computer" spectrum, who considered messing with upgrades and configuration a waste of time and often a way to introduce unwanted risk of data loss, but nonetheless were keen to learn about new ways to use their computers more efficiently. To stop computering, and start working.

Many of these people were, it is true, on older computers. The most extreme example was a member who to this day uses a Powerbook G3 Pismo and a Newton MessagePad 2100. He could do everything he needed with those two computers. But that didn't stop him from wanting to do it with less computer, from wanting to optimise his workflow, to find the latest tips and tricks available and decide whether they got him where he was going more efficiently.

As I said, that example was extreme. There were plenty of other people who only bought new computers every five years or more, but were still on the latest versions of apps like Photoshop, Quark Xpress, or iWork where they could be, and whenever new ones got released the meeting topic would be to dig into the new version (some brave soul would have it on day one) to see whether they could do things better, or with less effort.

These people were paying big money for big software. Not because it was the newest, or because they had to be on "latest and [as we like to claim] greatest", but because it was better suited to their needs. It gave them a better experience. So, bearing in mind that this is a straw-man for exploration purposes, let me introduce the hypothesis that defends the straw-man axiom presented above:

A delightful user experience means not making people mess about with computer stuff just to use their computers.
There are plenty of people out there who would rather get something that lets them work more effectively than waste time on upgrades.
To those people, spending money on software that gets them where they are going is a better investment than any amount of time and anxiety spent on messing with settings including operating system upgrades.
These people represent the middle of the spectrum: not the extreme low end where you never change anything once you've bought the computer; nor the extreme high end where fiddling with settings and applying upgrades is considered entertainment.
Therefore, the low price of "latest and greatest" software reflects at least in part the externalisation of the (time-based) costs and risks associated with upgrading to mid-range tinkerers.
Because of this, while the incremental number of users associated with supporting earlier OS versions may not be great, the incremental value per user may be much higher than gaining users with low-price apps on the current operating systems.

As I say, interesting food for thought, but not necessarily any more (or less) true than the view presented in Brent's posts. Please have your pinch of salt ready, and don't bet your business on the thoughts of this idle blogger.

#### Lambda: the ultimate polymath
Thinking back over the last couple of years, I've had to know quite a bit about a few different topics to be able to write good software. Those topics:

Epidemiology
Architecture
Plant Sciences
Histology
Education

Not much knowledge in each field, though definitely expert-level knowledge: enough to have conversations as a peer with experts in those fields, to be able to follow the jargon, and to be able to make reasoned assessments of the experts' suggestions in software designed for use by those experts. And, where I've wanted to propose alternate suggestions, enough expert-level knowledge to identify and justify a different design.

Going back over the rest of my career in software:

Pensions, investments, and saving
Mobile Telecommunications
Terry Pratchett's Discworld
Macromolecular crystallography
Synchrotron accelerators
Home automation
Yoga
Soda drinks dispensers
Banks
Quite a bit of physics

In fact I'd estimate that I've spent less than 40% of my "professional career" in jobs where knowing software or even computers in general was the whole thing.

Working in software is about understanding a system to the point where you can use a computer to make a meaningful and beneficial contribution to that system. While the systems thinking community have great tools for mapping out the behaviour of systems, they are only really good for making initial models. In order to get good at it, we have to actually understand the system on its own terms, with the same ideas and biases that the people who interact regularly with the system use.

But of course, because we're hired for our computering skills, we get to experience and work with all of these different systems. It's perhaps one of the best domains in which to be a polymath. To navigate it effectively, we need to accept that we are not the experts. We're visitors, who get to explore other people's worlds.

We should take them up on that offer, though, if we're going to be effective. If we maintain the distinction between "technical" and "non-technical" people, or between "engineering" and "the business", then we deny ourselves the ability to learn about the problem we're trying to solve, and to make a good solution.

#### Last Chance: Ultimate Programmer Super Stack
Remember this? It's the last day for the stack today. Buy today for $47.95 within six hours and get my APPropriate Behaviour, along with books on running software businesses, building test-driven developers, and all sorts of software stacks including Ruby, Node, Laravel, Python, Java, Kotlin…

#### Layers of Distraction
A discussion I was involved in over on Facebook reminded me of some other issues I'd already drafted for this blog, so I stuck the two together and here we are.

Software systems can often be seen as aggregations of strata, with higher layers making use of the services in the lower layers. You'll often see a layered architecture diagram looking like a flat and well-organised collection of boiled sweets.

As usual, it's the interstices rather than the objects themselves that are of interest. Where two layers come together, there's usually one of a very small number of different transformations taking place. The first is that components above the boundary can express instructions that any computer could run, and they are transformed into instructions suitable for this computer. That's what the C compiler does, it's what the x86 processor does (it takes IA-32 instructions, which any computer could run, and turns them into the microcode which it can run), it's what device drivers do.

The second is that it turns one set of instructions any computer could run into another set that any computer could run. If you promise not to look too closely the Smalltalk virtual machine does this, by turning instructions in the Smalltalk bytecode into instructions in the host machine language.

The third is that it turns a set of computer instructions in a specific domain into the general-purpose instructions that can run on the computer (sometimes this computer, sometimes any computer). A function library turns requests to do particular things into the machine instructions that will do them. A GUI toolkit takes requests to draw buttons and widgets and turns them into requests to draw lines and rectangles. The UNIX shell turns an ordered sequence of suggestions to run programs into the collection of C library calls and machine instructions implied by the sequence.

The fourth is turning a model of a problem I might want solving into a collection of instructions in various computer domains. Domain-specific languages sit here, but usually this transition is handled by expensive humans.

Many transitions can be found in the second and third layers, so that we can turn this computer into any computer, and then build libraries on any computer, then build a virtual machine atop those libraries, then build libraries for the virtual machine, then build again in that virtual machine, then finally put the DOM and JavaScript on top of that creaking mess. Whether we can solve anybody's problems from the top of the house of cards is a problem to be dealt with later.

You'd hope that from the outside of one boundary, you don't need to know anything about the inside: you can use the networking library without needing to know what device is doing the networking, you can draw a button without needing to know how the lines get onto the screen, you can use your stock-trading language without needing know what Java byte codes are generated. In other words, both abstractions and refinements do not leak.

As I've gone through my computing career, I've cared to different extents about different levels of abstraction and refinement. That's where the Facebook discussion came in: there are many different ways that a Unix system can start up. But when I'm on a desktop computer, I not only don't care which way the desktop starts up, I don't want to have to deal with it. Whatever the relative merits of SMF, launchd, SysV init, /etc/rc, SystemStarter, systemd or some other system, the moment I need to even know which is in play is the moment that I no longer want to use this desktop system.

I have books here on processor instruction sets, but the most recent (and indeed numerous) are for the Motorola 68k family. Later than that and I'll get away with mostly not knowing, looking up the bits I do need ad hoc, and cursing your eyes if your debugger drops me into a disassembly.

So death to the trope that you can't understand one level of abstraction (or refinement) without understanding the layers below it. That's only true when the lower layers are broken, though I accept that that is probably the case.

#### Learn Mansplaining The Hard Way
zed shaw:

  It didn’t matter that most of these detractors admitted to me that they don’t code C anymore, that they don’t teach it, and that they just memorized the standard so they could “help” people.

[…]

  I cannot help old programmers. They are all doomed. Destined to have all the knowledge they accumulated through standards memorization evaporate at the next turn of the worm. They have no interest in questioning the way things are and potentially improving things, or helping teach their craft to others unless that education involves a metric ton of ass kissing to make them feel good. Old programmers are just screwed.

#### Learning about software freedom
On the front page of Hacker News at the moment is a post on The Three Software Freedoms. It does away with the Free Software Foundation's Freedom Zero:

  The freedom to run the program as you wish, for any purpose.

On the basis that it "is just silly I mean of course I can use the program as I wish." Of course you can. Except, of course, that's not in fact true, and because software companies are licensing - not selling - you software, you're actually bound by their terms and the limitations they impose. Picking just one example, here are some snippits from the Apple Licensed Application End-User License Agreement, relevant to apps bought (sorry, licensed) through the iTunes App Store:

  The Products transacted through the Service are licensed, not sold, to You for use only under the terms of this license
  
  The licensor (“Application Provider”) reserves all rights not expressly granted to You.
  
  This license granted to You for the Licensed Application by Application Provider is limited to a non-transferable license to use the Licensed Application on any iPhone or iPod touch that You own or control and as permitted by the Usage Rules set forth in Section 9.b. of the App Store Terms and Conditions (the “Usage Rules”). This license does not allow You to use the Licensed Application on any iPod touch or iPhone that You do not own or control, and You may not distribute or make the Licensed Application available over a network where it could be used by multiple devices at the same time. You may not rent, lease, lend, sell, redistribute or sublicense the Licensed Application. You may not copy (except as expressly permitted by this license and the Usage Rules), decompile, reverse engineer, disassemble, attempt to derive the source code of, modify, or create derivative works of the Licensed Application, any updates, or any part thereof (except as and only to the extent any foregoing restriction is prohibited by applicable law or to the extent as may be permitted by the licensing terms governing use of any open sourced components included with the Licensed Application).
  
  If You breach this restriction, You may be subject to prosecution and damages.
  
  Your rights under this license will terminate automatically without notice from the Application Provider if You fail to comply with any term(s) of this license. Upon termination of the license, You shall cease all use of the Licensed Application, and destroy all copies, full or partial, of the Licensed Application.
  
  You agree that any Services contain proprietary content, information and material that is protected by applicable intellectual property and other laws, including but not limited to copyright, and that You will not use such proprietary content, information or materials in any way whatsoever except for permitted use of the Services. No portion of the Services may be reproduced in any form or by any means. You agree not to modify, rent, lease, loan, sell, distribute, or create derivative works based on the Services, in any manner, and You shall not exploit the Services in any unauthorized way whatsoever, including but not limited to, by trespass or burdening network capacity. You further agree not to use the Services in any manner to harass, abuse, stalk, threaten, defame or otherwise infringe or violate the rights of any other party, and that the Application Provider is not in any way responsible for any such use by You, nor for any harassing, threatening, defamatory, offensive or illegal messages or transmissions that You may receive as a result of using any of the Services.
  
  You may not use or otherwise export or re-export the Licensed Application except as authorized by United States law and the laws of the jurisdiction in which the Licensed Application was obtained. In particular, but without limitation, the Licensed Application may not be exported or re-exported (a) into any U.S. embargoed countries or (b) to anyone on the U.S. Treasury Department's list of Specially Designated Nationals or the U.S. Department of Commerce Denied Person’s List or Entity List. By using the Licensed Application, you represent and warrant that you are not located in any such country or on any such list. You also agree that you will not use these products for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture or production of nuclear, missiles, or chemical or biological weapons.

So don't claim, through malice or ignorance, that Freedom Zero is not worth making explicit, and doesn't need protecting.

#### Learning phases
I've been trying to learn things this week (specifically Haskell). So
far I've been through a lot of different moods, and I thought it'd be
handy to write them down so that next time I'm the teacher I can
remember what it's like to be on the receiving end. I'm not sure I
(and certainly not anyone else) goes through all of these every
time, nor necessarily in the same order which I can't remember anyway.

Frustration

These symbols all have no meaning to me, what is even going on here?

Atomic comprehension

If this symbol means that, and this symbol means that, then
combining these two must mean...

Holistic comprehension

OK, so this whole thing does this, which means that these parts must
somehow contribute to doing this. Let's see...

Argument by analogy

Oh, hold on, what they're saying is exactly like this other thing from
a different area of my experience. I'll just pretend it's the same for
the moment.

Paralysis of choice

I have a problem, and I've been given a bunch of tools, and I don't
know which to apply.

All I have is a hammer

Well, I understood how to apply a lambda function to a collection with
map, so I'm going to contort this problem until it looks like I need
to apply a lambda function to a collection with map.

Progress by context

I have no idea what I'm supposed to do here, but I just read a section
on list comprehension so I bet I'm supposed to use [x | x &lt;- xs]
somehow.

Confusion of dissimilarity

I have no idea how to turn that map into a comprehension or vice
versa, but I was able to produce each at different times on the same
day.

Joy of simplicity

You know what, it's true when they say that if it type-checks, it
probably works.

Disillusion through broken dreams

Oh, off-by-one errors (and rotation of dimensions) still
type-check. This is bullshit.

Reinventing the wheel

So, I need a thing that applies this thing to all those things,
let me write a function applyFunction :: (a -&gt; b) -&gt; [a] -&gt; [b] that
gives me one of those.

Discovering existing wheels

I have a thing and I need a different thing, let me look in the API
documentation until I find the function that does exactly that.

Rage-quitting

This error makes no sense, screw the entire city of Glasgow.

Rabbit hole tangents

This error makes no sense, but those words look sufficiently rare to
shove into Google. Ooh, three blog posts, a bug report and a PhD
thesis!

Trial and (type) error

This error makes no sense, but it's telling me line 25 is wrong so
I'll just rewrite line 25. Again. And again.

Stop! Hammock time

I have no idea how to solve this problem, but it's
lunchtime. [Om nom nom] Ooh, this problem is trivial.

Minimum Viable Enlightenment

I don't know what I got wrong, but based on the error message I think
I make this change...yes, that works. I don't know what I got right.

Law of highfalutin words (I)

I wish someone could explain this to people who didn't learn the word
"monoid" at school.

Law of highfalutin words (II)

Yes, I can help you with that problem, I just solved it myself. You
see, it becomes easier when you realise that these two things
constitute a monoid.

#### Lest I forget
PIN END ___ /    |ooo| ----- bce

#### Let's talk about self-documenting code
You think your code is self-documenting. That it doesn't need comments or Doxygen or little diagrams, because it's clear from the code what it does.

I do not think that that is true.

Even if your reader has at least as much knowledge of the programming language you've used as you have, and at least as much knowledge of the libraries you've used as you have, there is still no way that your code is self-documenting.

How long have you been doing your job? How long have you been talking to experts in the problem domain, solving similar problems, creating software in this region? The likelihood is, whoever you are, that the new person on your team has never done that, and that your code contains all of the jargon terms and assumptions that go with however-much-experience-you-have experience at solving those problems.

How long were you working on that story, or fixing that bug? How long have you spent researching that specific change that you made? However long it is, everybody else on your team has not spent that long. You are the world expert at that chunk of code, and it's self-documenting to you as the world expert. But not to anybody else.

We were told about "working software over comprehensive documentation", and that's true, but nobody said anything about avoiding sufficient documentation. And nobody else has invested the time to understand the code that you just wrote that you did, so the only person for whom your code is self-documenting is you.

Help us other programmer folks out, think about us when avoiding documentation.

#### Licenses aren’t sufficient
Another recent issue in the world of “centralised open source dependency repositories were a bad idea” initiated by the central contradiction of free software. People want to both give everything away without limitation on who uses it or how, and they want “Big Program” to pay for the work to be done.

While the license is the only tool used by free software authors, there is no way that this is going to be resolved in the favour of the Robin Hood model. There’s nothing of value on offer to Big Program in the software. They want the right to use the software for their nefarious purposes, and for free they can get the right to use the software for any purpose. Why would they pay more?

They wouldn’t. And no amount of whataboutism is going to change that. Whatabout if nobody can afford to work on free software any more, and they lose access to updates? Doesn’t happen. The current set of incentives - part financial, mostly reputational, and part itch-scratching - actually observably cause an increasing amount of free software to be created over time.

That gap needs to be resolved in other ways. There are things that companies will pay for even when they have the freedom to use the software for any purpose, at no charge. They will pay for support, bug bounties, indemnification, training, documentation, consultancy, integration, operations…

If the free software community hadn’t completely withdrawn from the patents discussion, they might pay to license the patent whether or not they take the (free) copyright licence. But that has yet to happen.

Plenty of organisations understand this: Red Hat became a forty-odd-billion dollar company giving away the software for free and selling other things. Canonical, Cygnus, ActiveState, O’Reilly, Mozilla, Musescore, Nextcloud…all of them make software, none of them is a software company. All make money in the free software world, none is a free software company.

Please continue giving us all the freedom to use the software for any purpose. Also the other freedoms, to study, improve, and share the software. But remember that freedom is not for sale.

#### Lighter UIViewControllers
The first issue of Objective-C periodical objc.io has just been announced:

Issue #1 is about lighter view controllers. The introduction tells you a bit more about this issue and us. First, Chris writes about lighter view controllers. Florian expands on this topic with clean table view code. Then Daniel explores view controller testing. Finally, in our first guest article, Ricki explains view controller containment.

Each of the articles is of a high quality, I thoroughly recommend that you read this. I powered through it this morning and have already put myself on the mailing list for the next issue. I think it's worth quite a few dollars more than the $0 they're asking.

On a more self-focussed note, I'm pleased to note that two of the articles cite my work. This isn't out of some arrogant pleasure at seeing my name on someone else's website. One of my personal goals has been to teach the people who teach the other people: to ensure that what I learn becomes a part of the memeplex and isn't forgotten and reinvented a few years later. In APPropriate Behaviour a few of the chapters discuss the learning and teaching of software making. I consider it one of the great responsibilities of our discipline, to ensure the mistakes we had to make are not made by those who come after us.

The obj.io team have done a great job of researching their articles, taking the knowledge that has been found and stories that have been told and synthesising new knowledge and new stories from them. I'm both proud and humble about my small, indirect role in this effort.

#### Like Java, only functional
An idea that clarified itself to me in discussion today is that Swift is to Functional Programming as Java is to Object-Oriented Programming: it is the thing that lets you write C and pretend you've adopted some posh-sounding "paradigmatic" non-imperative approach to programming.

I thought this was true shortly before lunch, but now climb partway down from the high horse. Swift is to Functional Programming as Java is to Object-Oriented Programming is still true. However Swift is to C# 3.0 as Java is to Objective-C.

#### Linus's Bystanders
For some reason, when Eric S. Raymond wanted to make a point about the "bazaar" model of open source software development, he named it after someone else. Thus we have Linus's Law:

  Linus was directly aiming to maximize the number of person-hours thrown at debugging and development, even at the possible cost of instability in the code and user-base burnout if any serious bug proved intractable. Linus was behaving as though he believed something like this:
  
  8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
  
  Or, less formally, "Given enough eyeballs, all bugs are shallow." I dub this: "Linus's Law".

The "proof" of this law, such as it is, is a reductio ad absurdum painting the contradiction of the law as a universe in which the Linux kernel couldn't be created.

  If "Linus's Law" is false, then any system as complex as the Linux kernel, being hacked over by as many hands as the that kernel was, should at some point have collapsed under the weight of unforseen bad interactions and undiscovered ``deep'' bugs. If it's true, on the other hand, it is sufficient to explain Linux's relative lack of bugginess and its continuous uptimes spanning months or even years.

Let us remember the time that Linux development did collapse under its own weight. Raymond's original essay was written in 1997; by 2006 it seemed that "we're adding bugs at a higher rate than we're fixing them". In between, more developers had (presumably) contributed, so what happened there? Why weren't all of the bugs even shallower?

I'd like to examine a result from the social sciences here. From the Wikipedia (I am researcher, hear me roar) page on bystander effect:

  The bystander effect, or bystander apathy, is a social psychological phenomenon that refers to cases in which individuals do not offer any means of help to a victim when other people are present. The probability of help is inversely related to the number of bystanders. In other words, the greater the number of bystanders, the less likely it is that any one of them will help.

And then, a few paragraphs later when talking about computer-mediated communication:

  Research suggests that the bystander effect may be present in computer-mediated communication situations.

The experiments into bystander apathy are specifically about someone in distress, either asking for help or visibly in trouble. Maybe it applies here, too, when the only "distress" is that some software you didn't pay for could be better than it is. Perhaps, even before we get into discussions of whether the projects have welcoming communities that accept contributors, our projects are putting people off just because they know that there are plenty of others out there who could fix these bugs before they do.

I've met software team leaders who, when asked how their techniques and processes could be made to scale to a team of 1,000, answer that they'd start by giving the project to a team of ten. Maybe they're onto something.

#### Literate Programming with LibreOffice
This post comes in the form of an OpenDocumentFormat document containing a program that can extract programs from ODF documents, including the program contained in this document.

#### Little hack to help with testing
Want the ability to switch in different test drivers, mock objects, or other test-specific behaviour?  Here's a pattern I came up with (about a year ago) to do that in a GNUstep test tool, which can readily be used in Cocoa:

NSString *driverClassName = [[NSUserDefaults standardUserDefaults] stringForKey: @"Class"];Class driverClass = NSClassFromString(driverClassName);id myDriver = [[driverClass alloc] init];

With a healthy dose of no, seriously, don't do this in production code, you now have the ability to specify your test driver on the command-line like this:

$ ./myTestingTool -Class GLTestDriver

This uses the oft-neglected behaviour of NSUserDefaults, in which it parses the executable's command-line arguments to create a defaults domain, higher in priority than even the user's preferences file.  You can use that behaviour in a graphical app too, where it comes in handy when working in Xcode.  It then uses a combination of the runtime's duck typing and introspection capabilities to create an instance of the appropriate class.

#### LLMs and reinforcement learning
My reflection on the&nbsp;Richard Sutton interview with Dwarkesh Patel&nbsp;was that it was interesting how much the two participants talk past each other, and fail to find common ground. Particularly that they couldn't agree on the power of reinforcement learning, when it's such a large part of the LLM workflow.

To be clear, it isn't the large language model that engages in reinforcement learning, it's the person who's applying the LLM to their task. That's all that prompt engineering is. Here's the workflow:

Identify a goal.

Hypothesize a prompt that leads the LLM to satisfy the goal.

Try out the prompt and generate an outcome.

Observe the gap between the outcome and the intended goal.

Repeat steps 1-4 (yes, include step 1, as you might refine the goal or the prompt) until the gap becomes acceptably small.

This process—commonly known in its isolated, bureaucratized form as "science"—is fundamental to the learning experience. It's the same as John Dewey's description of reflective thought, as Honey and Mumford's learning cycle, as the Shewhart cycle (Plan-Do-Check-Act). Start at step four, and it's the same as Boyd's Observe-Orient-Decide-Act loop. And it's fundamental to how LLM-assisted work unfolds. But because the LLM is incapable of manifesting the cycle, it's left to the person to do all the learning.

#### LLVM projects you may not be aware of
All Mac and iPhone OS developers must by now be familiar with LLVM, the Low-Level Virtual Machine compiler that Apple has backed in preference to GCC (presumably at least partially because because GCC 4.5 is now a GPLv3 project, in addition to technical problems with improving the older compiler). You'll also be familiar with Clang, the modular C/ObjC/C++ lexer/parser that can be used as an LLVM front-end, or as a library for providing static analysis, refactoring and other code comprehension facilities. And of course MacRuby uses LLVM's optimisation libraries.The LLVM umbrella also covers a number of other projects that Mac/iPhone developers may not yet have heard about, but which nonetheless are pretty cool. This post is just a little tour of some of those. There are other projects that have made use of LLVM code, but which aren't part of the compiler project - they are not the subject of this post.LibC++ is a C++ library, targeting 100% compatibility with the C++0x (draft) standard.KLEE looks very cool. It's a "symbolic execution tool", capable of automatically generating unit tests for software with high degrees of coverage (well over 90%). Additionally, given information about an application's constraints and requirements it can automatically discover bugs, generating failing tests to demonstrate the bug and become part of the test suite. There's a paper describing KLEE including a walkthrough of discovering a bug in tr, and tutorials in its use.vmkit is a substrate layer for running bytecode. It takes high-level bytecode (currently JVM bytecode or IL, the bytecode of the .Net runtime) and translates it to IR, the LLVM intermediate representation. In doing so it can make use of LLVM's optimisations and make better decisions regarding garbage collection.

#### Local KDC on Leopard
via Nigel Kersten, a great description of the operation of Leopard's built-in local KDC.  I think the most exciting thing about the local KDC is the Bonjour support; could we see simple cross-system trust in the near future?  Could there be someone in the world who can actually make Kerberos simple?

#### Longer, fuller stacks
Thinks to self: OK, this "full-stack" project is going to be fairly complex. I need:

a database. I don't need it yet, I'll defer that.
a thing that runs on the server, listens for HTTP requests from a browser, builds responses, and sends them to the browser.
a thing that runs on the browser, built out of bits assembled by the server thing, that the user can interact with.

What I actually got was:

a thing that runs on the server.
a thing that defines the environment for the server.
a thing that defines the environment on development machines so that you can run server-development tasks.
a thing that turns code that can't run in a browser into code that can run in a browser.
a thing that turns code that can run in a browser into code that does run in real browsers.
a headless browser that lets me test browser code.

BTW, it doesn't work with that server environment.
a thing that shows how Linux binaries are loaded, to work out how to fix the environment.
also BTW, it doesn't run headless without setting some environment variable
a thing that is used for cross-platform native desktop apps, that I can configure to work headless.

a thing that builds the bits assembled by the server thing so that the test thing can see the code being tested.

And somehow, people argue that this is about reducing development costs.

#### Look what the feds left behind…
So what conference was on in this auditorium before NSConference? Well, why don't we just read the documents they left behind?

Ooops. While there's nothing at higher clearance than Unrestricted inside, all of the content is marked internal eyes only (don't worry, feds, I didn't actually pay too much attention to the content. You don't need to put me on the no-fly list). There's an obvious problem though: if your government agency has the word "security" in its name, you should take care of security. Leaving private documentation in a public conference venue does not give anyone confidence in your ability to manage security issues.

#### Losing your identity
Developers make use of cryptographic signatures in multiple places in the software lifecycle. No iPad or iPhone application may be distributed without having been signed by the developer. Mac developers who sign their applications get to annoy their customers much less when they ship updates, and indeed the Sparkle framework allows developers to sign the download file for each update (which I heartily recommend you do). PackageMaker allows developers to sign installer packages. In each of these cases, the developer provides assurance that the application definitely came from their build process, and definitely hasn't been changed since then (for wholly reasonable values of "definitely", anyway).

No security measure comes for free. Adding a step like code or update signing mitigates certain risks, but introduces new ones. That's why security planning must be an iterative process - every time you make changes, you reduce some risks and create or increase others. The risks associated with cryptographic signing are that your private key could be lost or deleted, or it could be disclosed to a third party. In the case of keys associated with digital certificates, there's also the risk that your certificate expires while you're still relying on it (I've seen that happen).

Of course you can take steps to protect the key from any of those eventualities, but you cannot reduce the risk to zero (at least not while spending a finite amount of time and effort on the problem). You should certainly have a plan in place for migrating from an expired identity to a new one. Having a contingency plan for dealing with a lost or compromised private key will make your life easier if it ever happens - you can work to the plan rather than having to both manage the emergency and figure out what you're supposed to be doing at the same time.

iPhone/iPad signing certificate compromise

This is the easiest situation to deal with. Let's look at the consequences for each of the problems identified:

Expired IdentityNo-one can submit apps to the app store on your behalf, including you. No-one can provision betas of your apps. You cannot test your app on real hardware.
Destroyed Private KeyNo-one can submit apps to the app store on your behalf, including you. No-one can provision betas of your apps. You cannot test your app on real hardware.
Disclosed Private KeySomeone else can submit apps to the store and provision betas on your behalf. (They can also test their apps on their phone using your identity, though that's hardly a significant problem.)

In the case of an expired identity, Apple should lead you through renewal instructions using iTunes Connect. You ought to get some warning, and it's in their interests to help you as they'll get another $99 out of you :-). There's not really much of a risk here, you just need to note in your calendar to sort out renewal.

The cases of a destroyed or disclosed private key are exceptional, and you need to contact Apple to get your old identity revoked and a new one issued. Speed is of the essence if there's a chance your private key has been leaked, because if someone else submits an "update" on your behalf Apple will treat it as a release from you. It will be hard for you to repudiate the update (claim it isn't yours) - after all, it's signed with your identity. If you manage to deal with Apple quickly and get your identity revoked, the only remaining possibility is that an attacker could have used your identity to send out some malicious apps as betas. Because of the limited exposure beta apps have, there will only be a slight impact: though you'll probably want to communicate the issue to the public to motivate users of "your" beta app to remove it from their phones.

By the way, notice that no application on the store has actually been signed by the developer who wrote it - the .ipa bundles are all re-signed by Apple before distribution. 

Mac code signing certificate compromise

Again, let's start with the consequences.

Expired IdentityYou can't sign new products. Existing releases continue to work, as Mac OS X ignores certificate expiration in code signing checks by default.
Destroyed Private KeyYou can't sign new products.
Disclosed Private KeySomeone else can sign applications that appear to be yours. Such applications will receive the same keychain and firewall access rights as your legitimate apps.

If you just switch identities without any notice, there will be some annoyances for users - the keychain, firewall etc. dialogues indicating that your application cannot be identified as a legitimate update will appear for the update where the identities switch. Unfortunately this situation cannot be distinguished from a Trojan horse version of your app being deployed (even more annoyingly there's no good way to inspect an application distributor's identity, so users can't make the distinction themselves). It would be good to make the migration seamless, so that users don't get bugged by the update warnings, and learn to treat them as suspicious.

When you're planning a certificate migration, you can arrange for that to happen easily. Presumably you know how long it takes for most users to update your app (where "most users" is defined to be some large fraction such that you can accept having to give the remainder additional support). At least that long before you plan to migrate identities, release an update that changes your application's designated requirement such that it's satisfied by both old and new identities. This update should be signed by your existing (old) identity, so that it's recognised as an update to the older releases of the app. Once that update's had sufficient uptake, release another update that's satisfied by only the new identity, and signed by that new identity.

If you're faced with an unplanned identity migration, that might not be possible (or in the case of a leaked private key, might lead to an unacceptably large window of vulnerability). So you need to bake identity migration readiness into your release process from the start.

Assuming you use certificates provided by vendor CAs whose own identities are trusted by Mac OS X, you can provide a designated requirement that matches any certificate issued to you. The requirement would be of the form (warning: typed directly into MarsEdit):

identifier "com.securemacprogramming.MyGreatApp" and cert leaf[subject.CN]="Secure Mac Programming Code Signing" and cert leaf[subject.O]="Secure Mac Programming Plc." and anchor[subject.O]="Verisign, Inc." and anchor trusted

Now if one of your private keys is compromised, you coordinate with your CA to revoke the certificate and migrate to a different identity. The remaining risks are that the CA might issue a certificate with the same common name and organisation name to another entity: something you need to take up with the CA in their service-level agreement; or Apple might choose to trust a different CA called "Verisign, Inc." which seems unlikely.

If you use self-signed certificates, then you need to manage this migration process yourself. You can generate a self-signed CA from which you issue signing certificates, then you can revoke individual signing certs as needed. However, you now have two problems: distributing the certificate revocation list (CRL) to customers, and protecting the private key of the top-level certificate.

Package signing certificate compromise

The situation with signed Installer packages is very similar to that with signed Mac applications, except that there's no concept of upgrading a package and thus no migration issues. When a package is installed, its certificate is used to check its identity. You just have to make sure that your identity is valid at time of signing, and that any certificate associated with a disclosed private key is revoked.

Sparkle signing key compromise

You have to be very careful that your automatic update mechanism is robust. Any other bug in an application can be fixed by deploying an update to your customers. A bug in the update mechanism might mean that customers stop receiving updates, making it very hard for you to tell them about a fix for that problem, or ship any fixes for other bugs. Sparkle doesn't use certificates, so keys don't have any expiration associated with them. The risks and consequences are:

Destroyed Private KeyYou can't update your application any more.
Disclosed Private KeySomeone else can release an "update" to your app; provided they can get the Sparkle instance on the customer's computer to download it.

In the case of a disclosed private key, the conditions that need to be met to actually distribute a poisoned update are specific and hard to achieve. Either the webserver hosting your appcast or the DNS for that server must be compromised, so that the attacker can get the customer's app to think there's an update available that the attacker controls. All of that means that you can probably get away with a staggered key update without any (or many, depending on who's attacking you) customers getting affected:
Release a new update signed by the original key. The update contains the new key pair's public key.
Some time later, release another update signed by the new key.

The situation if you actually lose your private key is worse: you can't update at all any more. You can't generate a new key pair and start using that, because your updates won't be accepted by the apps already out in the field. You can't bake a "just in case" mechanism in, because Sparkle only expects a single key pair. You'll have to find a way to contact all of your customers directly, explain the situation and get them to manually update to a new version of your app. That's one reason I'd like to see auto-update libraries use Mac OS X code signing as their integrity-check mechanisms: so that they are as flexible as the platform on which they run.

#### Mac App Sandboxing: it may not be for you (but that's probably OK)
The MAS section of devforums is, along with a healthy subsection of the rest of the interwebs, aflame with the news that the deadline for sandboxing store-delivered apps is further away than it used to be, but still too close for some people.

What's the deal?

Bugs have been getting easier to detect over time, as the tools used to create software have got better. Anyone who remembers using a microcomputer BASIC interpreter will be familiar with the syntax error, where you're allowed to enter nonsensical commands and the program runs anyway, stopping unceremoniously when it reaches the unparseable input. Today, syntax errors can be detected and even automatically corrected in the IDE's text editor.

A program made of 100% acceptable syntax can still have logic errors, which most people would know as "bugs". The app is supposed to do one thing, but instead does another. Software testing practices are designed to catch logic errors, and code analysis can detect some of these problems too.

With a little bit of hand-waving, we can describe the next level of complexity as the security error or vulnerability. At this level, an application that both compiles and does what the user expects can be made to do something unexpected under misuse. In other words, logic errors are "will it work", security errors are "can it be made not to work". Automatic detection and correction of security errors is, in many situations, less well-advanced than detection of other types of error. Techniques for designing and coding security errors out during software development are still very immature.

A little more hand-waving and we get to the final, and most insidious, level of error I will consider here: the blended threat. This can be defined as "can this application be used in conjunction with other applications to do something unexpected". An example. Safari for Windows suffered a vulnerability called a "carpet bombing" attack, where the author of a web page could cause the browser to download a file without any user intervention. Not that big of a deal: the browser is supposed to allow downloads, this is a little unexpected but not outside the security model of an application.

However, Safari also has that "automatically open safe files after downloading" feature, and this is where the fun begins. Having downloaded, say, a document file, the browser automatically opens it. Now, what if that document is a PDF, and exploits a vulnerability in Acrobat Reader to execute code on the user's behalf? A local code execution problem like this might be considered a medium-severity issue by the Reader developers: it requires a user to be coerced or tricked into opening a malicious file, doesn't it?

Well, no. The combination of the low-severity automatic download bug, the automatic open feature and the medium-severity local code execution bug produce a high-severity remote code execution bug.

Detecting and reacting to this kind of "blended threat" is harder than dealing with any of the aforementioned classes of failure. It relies on knowing the interaction between your own app and the vast number of other apps that exist on the platform, understanding the capabilities and bugs of those other apps and how those interact with the capabilities and bugs in your own. It could, in principle, be done, but currently cannot.

Enter sandboxing.

Fundamentally, the problem is that the expression of different roles in an operating system like Mac OS X is incomplete. I'll not go into this again: my post on the new lion security things covers this ground. The TL;DR is that Mac OS X, Windows NT and friends all assume that the actors in a computer system are the users, and ignore the fact that the code represents a collection of actors too.

OK, so a user is allowed to download a document, read it, and to delete all of her documents. But does that mean that a collection of apps&mdash;each one notionally a separate actor independent from the user&mdash;should conspire together to enable this? Probably not; or if so, only over a well-defined inter-process communication system that doesn't allow this sort of thing to happen implicitly. Creating an Automator action to get some content from a PDF on a web server and delete some documents as a result might be something the OS needs to support, but one click in a browser probably shouldn't make all of that stuff happen automatically.

App sandboxing introduces two important features: the identification of different apps as different actors with their own privileges on the system is one, and (partially voluntary, on the developers' parts) restrictions on the ways apps communicate is the other. This restriction is implemented both as a limitation on the IPC mechanisms an app can make use of, and on the files the app may access: the filesystem is an app's version of sneaker net.

With these restrictions in place, the opportunities for performing a blended-threat attack are severely reduced. If one app is compromised it either doesn't have permission to contact the apps that the threat relies on, or it tries to contact apps that don't have permission to talk to it.

My app doesn't fit the sandbox limitations.

Complaints of that nature on iOS, where the limitations have been in place both for ages and from the beginning of the platform's support for third-party code, have pretty much died out now. Developers have become used to the idea that if an app cannot fit the limitations of the sandboxed environment, then it cannot be distributed on the platform.

There are two issues that limit the applicability of the "suck it up" solution for Mac apps. The first, and weakest, is that there are existing apps which don't fit the mould. This is weak because on the Mac there are other distribution mechanisms for getting the app into the customers' hands; mechanisms that have existed and worked for years, and that don't require the same controls enforced by the Mac App Store policy. Yes, it would be better if every app were sandboxed, but having any app sandboxed reduces the attack surface of the Mac so a less than 100% hit rate is still a win.

Of course, some apps are only incompatible due to minor technicalities, such as legacy decisions over where to locate user files. In these cases it's usually somewhat harmless to migrate to a situation where the app is sandbox-compatible, for example using the mediated file read permission to load documents from the legacy storage area.

Other apps are basically never going to work in a sandbox: utility software is a common casualty, especially things like anti-virus apps, disk partition editors, and file managers. IDEs are also likely to fall down here (though one that makes good use of the workspace concept need not). We can conclude that Apple doesn't want to sell ISV software in those categories on the Mac app store, so using a different store is the solution.

The other, stronger limitation is that there are applications that are already on the store that work currently, because the store doesn't enforce sandbox restrictions, but won't work in the future, because they're incompatible with the sandbox restrictions. Developers of these apps could migrate to a different store but would be unable to bring their existing customers with them, to give those existing customers updated versions, or even to contact those customers to tell them about the change.

Yes, this is a serious problem for the minority of affected apps. Yes, it is a limitation of the App Store's capability that results from Apple's business decisions. Yes, it's going to be a ball ache for affected developers to deal with. No, it's not a reason to throw the baby out with the bathwater and give up on sandboxing completely. No, I haven't provided a solution yet. As it's a business problem your first port of call should be your business contacts at Apple: this is such a big issue that it deserves a separate post. The short version is that if you're doing business with Apple (or any other company) you need a business relationship with that company.

OK, so technical solutions. It all depends on the design and architecture of your app. Perhaps there are features that can be removed, without destroying the essence of your app while gaining compatibility with the app store. Consider the business impact of removing that one feature versus removing your whole app. Also, remember that unless you've sold about 40 million copies, there are more potential customers out there than current customers.

Another possibility is that you separate the product into two components, a service and a controlling app. The service part is incompatible with the app store and distributed separately. The app is available as an additional feature to support easier use of the service. Take a look at the number of apps for controlling MySQL, or Postgres, and you'll see that having an app store app as a front-end to an external service is something that's both supported and viable (well, if it isn't viable, a lot of companies enjoy not making money).

#### Mac user Gmail account hack
I found today in Macintouch reader reports the news that a Mac user found his Gmail account had been taken over.  He writes:

I woke up this morning and looked at my gmail and thought, gee that's weird, it won't accept my password. I figured it was a glitch and tried it on my iphone, same thing.Then I asked for a password reset. When I got back into the account, found a bunch of sent emails from a Nigerian scammer. I also looked at the ip history in gmail and noticed the weird IP, which of course came from Nigeria.

This relates well to a point I've made repeatedly in podcasts and papers; namely that having information worth stealing is not a Windows-only situation.  As more data is stored &quot;in the cloud&quot; then the security of the cloud and of the way we use it becomes as important what is going on in our own computers.  Having a weak Facebook password compromised will work just as well if you're on Trusted Solaris as Windows.

In other news, yesterday's Twitter client is not really much further along, because a thunderstorm has meant I've unplugged all of my electronics (the laptop isn't plugged in to anything, obviously).  I am now very grateful to MarsEdit for having offline editing capability, otherwise I'd have to try and remember all this stuff later ;-)

#### MacDev 2009!
It's a long way off, but now is a good time to start thinking about the MacDev '09 conference, organised by the inimitable Scotty of the Mac Developer Network. This looks like being Europe's closest answer to WWDC, but without all those annoying "we call this Interface Builder, and we call this Xcode" sessions. Oh, and a certain Sophist Mac engineer software will be talking about building a secure Cocoa application.

#### MacFUSE rules
One reason that microkernels win over everything else (piss off, Linus) is that stability is better, because less stuff is running in the dangerous and all-powerful kernel environment.  MacFUSE, like FUSE implementations on other UNIX-like operating systems, takes the microkernel approach to filesystems, hooking requests for information out of the kernel and passing them to user-space processes to handle.  Here's the worst that can happen when screwing up a FUSE filesystem:

Now that might sound not only like a recipe for lower-quality code, but also like I'm extolling the capability to create lower-quality code.  Well no it isn't, and yes I am.  The advantage is that now the develop-debug-fix cycle for filesystems is just as short as it is for other userland applications (and HURD translators and the like).  This provides a lower barrier to entry (meaning that it's more likely that interesting and innovative filesystems can be created), but also a faster turnaround on bugfixes (no panic, restart, try to salvage panic log... no two-machine debugging with kdb...) so ultimately higher-quality filesystems.

#### Mach and Matchmaker: kernel and language support for object-oriented distributed systems
About this paper
Mach and Matchmaker: kernel and language support for object-oriented distributed systems
, Michael B. Jones and Richard F. Rashid, from the proceedings of OOPSLA '86.
Notes
Yes, 1986 was a long time ago, but the topics of Mach and Matchmaker are still relevant, and I find it interesting to read about its genesis and development. I also find that it helps me put today's uses - or abandonments - in context.
Mach
Two main families of operating systems under development today are still based on the CMU Mach project. Let's get discussing the HURD out of the way, first. The HURD is based on GNU Mach, which is itself based on the University of Utah's Mach 4.0 project. GNU Mach is a microkernel, so almost all of the operating system facilities are provided by user-space processes. An interesting implication is that a regular user can create a sub-HURD, an environment with a whole UNIX-like system running within their user account on the host HURD.

Not many people do that, though. HURD is very interesting to read and use, but didn't fulfil its goal of becoming a free host for the GNU system that made it easy to support hardware. Linux came along, as a free host for the GNU system that made it worthwhile to support hardware. I enjoy using the HURD, but we'll leave it here.

...because we need to talk about the other operating system family that uses Mach, macOS/iOS/watchOS/tvOS/whatever the thing that runs the touchbar on a Macbook Pro is called OS. These are based on CMU Mach 2.5, for the most part, which is a monolithic kernel. Broadly speaking, Mach was developed by adding bits to the 4.2 (then 4.3) BSD kernel, until it became possible to remove all of the BSD bits and still have a working BSD-like system. Mach 2.5 represents the end of the "add Mach bits to a BSD kernel" part of the process.

Based on an earlier networked environment called Accent, Mach has an object-oriented facility in which object references are called "ports", and you send them messages by...um, sending them messages. But sending them messages is really hard, because you have to get all the bits of all the parameters in the right place. What you need is...
Matchmaker
Originally built for Accent, Matchmaker is an Interface Definition Language in which you describe the messages you want a client and server to use, and it generates procedures for sending the messages and receiving the responses in the client, and receiving the messages and sending the responses in the server.

Being built atop Mach, Matchmaker turns those messages into Mach messages sent between Mach ports. What Mach does to get the messages around is transparent, so it might take a message on one computer and deliver it to a server on a different computer, maybe even running a different architecture.

That transparency was a goal of a lot of object-oriented remote procedure call systems in the 1990s, and by and large fell flat. The reason is Peter Deutsch's Eight Fallacies of Distributed Computing. Basically you usually want to know when your message is going out over a network, because that changes everything from how likely it is to be received, how likely you are to get a response, to how expensive it will be to send the message.

Matchmaker supported C, Common LISP, Ada, and PERQ Pascal; Accent and Mach messages, and a bunch of different computer architectures. Unfortunately it supported them all through specific knowledge of each, and the paper described here acknowledges how difficult that makes it to work on and proposes future work to clean it up. It's not clear that future work was ever done; modern Machs all use MIG, an "interim subset" of Matchmaker that only supports C.
Object-oriented design
In my book OOP the Easy Way, I explore the idea that objects are supposed to be small, independent computer programs that communicate over the loosely-coupled channel that is message-sending. Mach and Matchmaker together implement this design. Your objects can be in different languages, on different computers, even in different host operating systems (there were Mach IPC implementations for Mach, obviously, but also VAX Unix and non-Mach BSD). As long as they understand the same format for messages, they can speak to each other.

Consider a Cocoa application. It may be written in Swift or Objective-C or Objective-C++ or Python or whatever. It has a reference to a window, where it draws its views. The app sees that window as an Objective-C object, but the app doesn't have a connection to the framebuffer to draw windows.

The Window Server has that connection. So when you create a window in your Cocoa application, you actually send a message to the window server and get back a port that represents your window. Messages sent to the window are forwarded to the window server, and events that happen in the UI (like the window being closed or resized) are sent as messages to the application.

Because of the way that Mach can transparently forward messages, it's theoretically possible for an application on one computer to display its UI on another computer's window server. In fact, that's more than a theoretical possibility. NeXTSTEP supported exactly that capability, and an application with the NXHost default set could draw to a window server on a different computer, even one with a different CPU architecture.

This idea of loosely-coupled objects keeps coming up, but particular implementations rarely stay around for long. Mach messages still exist on HURD and Apple's stuff (both using MIG, rather than Matchmaker), but HURD is tiny and Apple recommend against using Mach or MIG directly, favouring other interfaces like XPC or the traditional UNIX IPC systems that are implemented atop Mach. Similarly, PDO has come and gone, as have CORBA and its descendents DSOM and DOE.

Even within the world of "let's use HTTP for everything", SOAP gave way to REST, which gave way to the limited thing you get if you do the CRUD bits of REST without doing the DAP bits. What you learn by understanding Mach and its interfaces is that this scheme can be applied everywhere from an internet service down to an operating system component.

#### Mach-O FS (no really, MacFUSE does rule)
It needs some polishing and more functionality before I'd call it useful, then I have to find out whether I'm allowed to do anything with the source code ;-).  But this is at least quite a cool hack; exploring a Mach-O file (thin or fat - in this case, I used the OmniDazzle executable which is a fat file) as if it's a file system.  FUSE of course makes it easy, so thanks to Amit Singh for the port!

#### Mach-OFS: aforementioned polish and functionality
It's getting there, now has the ability to display load commands (though it only reports useful information for LC_SEGMENT and LC_SEGMENT_64 commands):

Again the screenshot depicts the OmniDazzle binary for no reason other than it's a nontrivial file.  The directions in which to take the filesystem are now numerous: I can add info about the remaining load commands (v. useful), the raw data for each segment (somewhat useful), and the sections in each segment (v. useful).  Whether the filesystem will eventually get to the level of symbol resolution, I'm not sure :-).

#### machoo - Object-Oriented Programming in Object-Oriented Programming in the GNU HURD
For the last few weeks, my when-I-get-to-it project has been machoo, which is sort of an object-oriented system on the HURD but mostly an excuse to learn about how Mach messaging works.

I decided to build a Smalltalk-style "give me any old selector, and I'll do something with it" messaging system on Mach, which is already a messaging system. I don't yet know whether this is a good idea, but I like the design principle of highly loosely-coupled objects telling each other the names of the things to do.

The substrate for this system is MIG, the Mach Interface Generator. It takes descriptions of routines in an IDL that include the names of the supported routines and the argument names, types and return types. It'd certainly be possible to build each message I want to be able to send as a MIG routine, but then the client would have to import interfaces for every type it wanted to use: no bad thing, and many OO systems work that way, but not what I want.

MIG generates a 'demuxer' function that examines a message sent to a port, and dispatches it to a handler to the named routine. As an aside, it is highly likely that I've got one to many layers of indirection; that if I wrote my own function to take the place of the demuxer I could handle dispatch there without receiving arguments from a dispatcher to do more dispatch. OK, I could, but I don't currently know how.

So what I ended up with is a two-level system, something like Smalltalk or ObjC. A "class" is a HURD translator registered on the filesystem, you send the zero-arguments message machoo_create_object to the class and get a send right to the new object. This is two-level in that if you need to initialise the object, you would then send an init message to the created object. Objects are messaged using the machoo_msg_send routine, which takes a string selector like Smalltalk.

There's currently exactly one implementation of a class, a do-nothing null object class. It does the thing that I would extract out into a pattern for other classes: its response to a machoo_create_object message is to spawn a new thread to back the new object. So it's like an Erlang-style system where each object has its own execution context, except that task switching is handled by the kernel rather than some virtual machine. All of the objects of a class are in a single task, and they are each on their own thread.

I still need to solve some problems, like how to return a reference to 'this object' from a message. I need to build more than one class of object, and extract out the common parts of class-ness so that a new class is only defined by its response to messages. So far this is proving to be an interesting and educational project.

#### Mail::Box++
When most perl developers (I believe there still are one or two in existence) talk of the &quot;cool one-liner&quot; that they wrote, what they actually mean is that they grabbed a crapload of packages from CPAN, invoked a few use directives and then, finally, could write one line of their own code which happens to invoke a few hundred lines of someone else's code, which they have neither read nor tested.

Modulo testing, my short script (below) to convert mbox mailboxes to maildirs is exactly like that.  While it has three lines of meat, these call upon the (from what I can tell, fantastic) Mail::Box module to do the heavy lifting.  That package itself is less svelte, with 775 lines of perl.  Which is the interface to a C bundle, which is (single-arch) 92k.  But never mind, I still wrote a three-liner ;-)

#!/usr/bin/perl -wuse strict;use Mail::Box::Manager;@ARGV = ("~/mail", "~/Maildir") unless @ARGV;# open the existing (mbox) foldersmy $mgr = new Mail::Box::Manager;my ($srcPath, $dstPath) = @ARGV;# expand tildes and stuff$srcPath = glob $srcPath;$dstPath = glob $dstPath;opendir MBOXDIR, $srcPath or die "couldn't open source path: $!";foreach my $file (grep !/^./, readdir MBOXDIR){    my $mbox = $mgr->open(folder => "$srcPath/$file",      folderdir => "$srcPath");    # open a maildir to store the result    my $maildir = $mgr->open(folder =>  "$dstPath/$file",        type => "Mail::Box::Maildir",        access => 'rw',         folderdir => "$dstPath",         create => 1);    $mgr->copyMessage($maildir, $mbox->messages);}

#### Making computing exciting
Over the last couple of years, I have visited three different museums of computing. NSBBQ in 2009 and 2010 visited the National Museum of Computing at Bletchley Park and the Museum of Computing in Swindon respectively. At this year's WWDC I got the chance, along with a great group of friends, to visit the Computer History Museum in Mountain View.

While each has its interesting points, each also has its disappointments. My principle problem is this: most of the kit is switched off. Without a supply of electrons and an output device, most computers from my childhood just look like beige typewriters. Earlier computers look like poorly thought out hi-fi equipment, or refrigerators that Stanley Kubrick tarted up to use as props. The way you find out just how much computers have advanced over the last few decades is not by looking at the cases: it's by using the computers.

If you're anything like me, you keep track of your finances and tax return figures in Numbers. Now imagine doing it in Visicalc. Better still: try doing it in Visicalc. Or take your iOS app, and implement the core features in Microsoft BASIC (or MC6809 machine code, if you're feeling hardcore). Write your next blog post in PenDown. It's this experience that will demonstrate just how primitive even a 15 year old desktop computer feels. And the portables? See if you can lift one!

Of course, complaining is the easy part. Fixing it is harder. Which is why I'm now a volunteer at the Swindon museum of computing, on the team that designs the gallery. My main goal is to make the whole experience more interactive. In the short term, this means designing programming challenges for kids to try out: let's face it, if we want more children to be interested in programming, we need to make programming more interesting to children. I certainly don't relish the prospect of becoming a portable brain in a pickle jar just because the next generation doesn't know any objective-c.

So it won't happen overnight, but if I'm at all successful then we should be able to make the museum gallery more interactive, more educational, and more fun. To find out how it's going, follow @MuseumComputing.

#### Making my peace
Nearly four years ago, in January 2015, I posted On Switching to Linux, in which my computer (in a photo from November 2014) looked like this:

Ubuntu Linux on a MacBook Air

Here's the same photo from today:

macOS Mojave on a MacBook Pro

So what's changed? In the intervening four years, I spent some time working with Linux desktop applications made of Qt, and some with browser and server applications made of Javascript. I used a GNU/Linux distribution, Windows 10, a Mac, iOS, and Android. I published two books. I took some time off. I did other things. Here are my relevant conclusions:

Free Software is important
Making things that are easy, or even pleasant, to use is important
Free Software's Four Freedoms are only academic if usability is a barrier to being capable of using the software for any purpose
Apple, and the developers on their platform, are the sub-section of the developer world who care most about giving their people usable and pleasant things
Combining these things leads to the conclusion that bringing Free Software principles to the world of Apple makers and adopters is both important and valuable
Meanwhile, the people over in the web and server/backend/cloud/serverless land have done a much better job of letting makers iterate quickly and build new things
Conversely, the people in the Apple land have done a much better job of making it so that the thing you build works without some complicated stack of transpilers, polyfills and tree-shakers.
Thus there are things that the makers in Apple land should learn from the makers in web/server land before the Apple land merely becomes a window on to the web stuff.

I'm back. Watch this space.

#### Management is the wrong analogy for LLM augmentation
A common meme at the moment in AI-augmented coding circles is “we are all managers now”, with people expressing the idea that alongside actual programming, programmers now manage their team of agents. This is a poor analogy, in both directions. Treating an interaction with an AI agent like a manager-report interaction would lead to a poor experience for using the agent. Treating an interaction with a direct report like you’re using an AI tool would likely result in a visit from your HR representative.

In my experience of being managed and of being a manager in software companies, the good managers I’ve had and aspire to be are the ones who Camille Fournier describes in The Manager's Path:

Managers who care about you as a person, and who actively work to help you grow in your career. Managers who teach you important skills and give you valuable feedback. Managers who help you navigate difficult situations, who help you figure out what you need to learn. Managers who want you to take their job someday. And most importantly, managers who help you understand what is important to focus on, and enable you to have that focus.

An LLM doesn’t have a job or a career path, or growth goals, or learn from your interactions. You can’t really tell it what’s important to focus on, you can just try to avoid showing it things you don’t want it to focus on. An LLM never gets into a difficult situation; the customer is always “absolutely right!”

Treating an LLM like a direct report can only lead to frustration. It isn’t a person who wants to succeed at its job, to learn and grow in its role, or become more capable. Indeed, it can’t do any of those things. It’s a tool. A tool that happens to have an interface that seems superficially similar to talking to it.

And that means that the correct way to treat an AI agent or coding interface is like a tool: it’s a text editor with a chat-like interface, a nondeterministic build script, or a static analysis tool. You’re looking for the correct combination of words and symbols to feed in to make the tool produce the output you want.

Treating a person who reports to you in that way would be unsatisfying and ultimately problematic. You don’t find different ways to express your problem statement until they solve it the way you would have solved it. You don’t give them detailed rules files with increasingly desperate punctuation around the parts it’s ## MANDATORY! that they follow. You find a way to work together, to teach each other, and to support each other.

If you're really looking for an analogy with human-human interactions, then working with an outsource agency is slightly more accurate (particularly one located in a different place with a different culture and expectations, where you have to be more careful about communication because you can’t rely on shared norms and tacit knowledge being equivalent). You do, in such cases, work on a clearly-scoped task or project, with written statements of work. and clear feedback points. You still expect it to get better and easier over time, for the agents to learn and adapt in ways that LLM-based tools don’t, and to show initiative when faced with unstated problems in ways that LLM-based tools can’t. And the outsource agents still expect to be treated as peers and experts, helping you out by doing the work that you don't have the capacity for. It’s better, but not great, as analogies go.

Unfortunately the best analogy we have for “precisely expressing problem statements in such a way that a computer generates the expected solution” is exactly the kind of thing that many people in the LLM world would like to claim isn’t happening.

#### Managers: Don't bend it that far, you'll break it!
Go on then, what's wrong with the words we already have?  I think they're perfectly cromulent, it's very hard to get into a situation where the existing English vocabulary is insufficient to articulate one's thoughts.  I expect that linguists and lexicographers have some form of statistic measuring the coverage in a particular domain of a language's expression; I also expect that most modern languages have four or five nines of coverage in the business domain.

So why bugger about with it?  Why do managers (and by extension, everyone trying to brown-nose their way into the management) have to monetise that which can readily be sold[1]?  Why productise that which can also be sold?  Why incentivise me when you could just make me happy?  Why do we need to touch base, when we could meet (or, on the other hand, we could not meet)?  Do our prospectives really see the value-add proposition, or are there people who want to buy our shit?

Into the mire which is CorpSpeak treads the sceadugenga that is TechRepublic, Grahames yrre bær.  The first words in their UML in a Nutshell review is &quot;Takeaway&quot;.  Right, well, I don't think they're about to give us a number 27 with egg-fried rice.  (As a noun, that meaning appears only in the Draft Additions to the OED from March 2007.)  Nor is there likely to be some connection with golf.  All right, let's read on.

UML lets you capture, document, and communicate information about an application and its design, so it's an essential tool for modeling O-O systems. Find out what's covered in O'Reilly's UML in a Nutshell and see if it belongs in your library.

Ah, that would be a précis, unless I'm very much mistaken.  Maybe even a synopsis.  Where did you get the idea this was a takeaway?  I can't even work out what the newspeak meaning for takeaway might be.  Had I not seen the linked review, I had thought the "if you take away one idea from this article, make it this" part of the article.  In other words, if you're so stupid that you can only remember one sentence from a whole page, we'll even tell you which sentence you should concentrate on.  This use[2] doesn't fit with that retroactive definition though, because the conclusion which can be drawn from the above-quoted paragraph is that one might want to read the whole article.  I would much rather believe that management types in a hurry would remember the subsequent sentence as their only recollection of the article.

UML in a Nutshell: A Desktop Quick Reference is not misnamed.

[1]You may argue that the word should be spelled "monetize", as the word most probably came from American English, but it doesn't matter because it doesn't bloody exist.  Interestingly, the verb sell originated in the Old English verb sellan, meaning to give, with no suggestion of barter or trade.

[2]Language usage is the only place I'll admit the existence of the word usage.

#### Mapping software engineering tools
Despite the theory that everything can be done in software (and of course, anything that can't be done could in principle be approximated using numerical methods, or fudged using machine learning), software engineering itself, the business of writing software, seems to be full of tools that are accepted as de facto standards but, nonetheless, begrudgingly accepted by many teams. What's going on? Why, if software is eating the world, hasn't it yet found an appealing taste for the part of the world that makes software?

Let's take a look at some examples. Jira is very popular among many people. I found a blog post literally called Why I Love Jira. And yet, other people say that Jira is an anti pattern, a sentiment that gets reasonable levels of community support.

Jenkins is almost certainly the ("market", though it's free) leader among continuous delivery tools, a position it has occupied since ousting Hudson, from which it was forked. Again, it's possible to find people extolling the virtues and people hating on it.

Lastly, for some quantitative input, we can find that according to the Stack Overflow 2018 survey, most respondents (78.9%) love Rust, but most people use JavaScript (69.8%). From this we draw the interesting conclusion that the most popular tool in the programming language realm is not, actually, the one that wins the popularity contest.

So, weird question, why does everybody do this to themselves? And then more specifically, why is your team doing it to yourselves, and what can you do about it?

My hypothesis is that all of these tools succeed because they are highly configurable. I mean, JavaScript is basically a configuration language for Chromium (don't @ me) to solve/cause your problem. Jira's workflows are ridiculously configurable, and if Jenkins doesn't do what you want then you can find a plugin to do it, write a plugin to do it or make a Groovy script that will do it.

This appeals to the desire among software engineers to find generalisations. "Look," we say, "Jenkins is popular, it can definitely be made to do what we want, so let's start there and configure it to our needs".

Let's take the opposing view for the moment. I'm going to drop the programming language example of JS/Rust, because all programming languages are, roughly speaking, entirely interchangeable. The detail is in the roughness. The argument below still applies, but requires more exposition which will inevitably lead to dissatisfaction that I didn't cover some weird case. So, for the moment, let's look at other tools like Jira and Jenkins.

The exact opposing view is that our project is distinct, because it caters to the needs of our customers and their (or these days, probably our) environment, and is understood and worked on by our people with our processes, which is not true for any other project. So rather than pretend that some other tool fits our needs or can be bent into shape, why don't we build our own?

And, for our examples, building such a tool doesn't appear to be a big deal. Using the expansive software engineering term "just", a CD tool is "just" a way to run each step in the deployment pipeline and tell someone when a step fails. A development-tracking tool is "just" a way to list the things the team is or could be working on.

This is more or less a standard "build or buy" question, with just one level of indirection: both building and buying are actually measured in terms of time. How long would it take the team to write a new CD tool, and to maintain it? How long would it take the team to configure Jenkins, and to maintain it?

The answer should be fairly easy to consider. Let's look at the map:

We are at x, of course. We are a short way from the Path of Parsimony, the happy path along which the generic tools work out of the box. That distance is marked on the map as ∂.

Think about how you would measure ∂ for your team. You would consider the expectations of the out-of-the-box tool. You would consider the expectations of your team, and of your project. You would look at how those expectations differ, and try to quantify the result.

This tells you something about the gap between what the tool provides by default and what you need, which will help you quantify the amount of customisation needed (the cost of building a spur out from the Path of Parsimony to x). You can then compare that with the cost of building a tool that supports your position directly (the cost of building your own path, running through x).

But the map also suggests another option: why don't we move from x closer to the path, and make ∂ smaller? Which of our distinct assumptions are incidental and can be abandoned, which are essential and need to be supported, and which are historical and could be revised? Is there a way to change the context so that adopting the popular tool is cheaper?

[Left out of the map but just as important is the related question: has somebody else already charted a different path, and how far are we from that? In other words, is there a different off-the-shelf product which needs less configuration than the one we've picked, so the total migration-plus-configuration cost is less than sticking where we are?]

My impression is that these questions tend to get asked once at the start of a project or initiative, then not again until the team is so far away from the Path of Parsimony that they are starting to get tangled and stung by the Weeds of Woe. Teams that change tooling such as their issue trackers or CD pipeline tend to do it once the existing way is already hurting too much, and the route back to the path no longer clear.

#### Mature Optimization
This comment on why NetNewsWire is fast brings up one of the famous tropes of computer science:
The line between [performance considerations pervading software design] and premature optimization isn’t clearly defined.
If only someone had written a whole paper about premature optimization, we'd have a bit more information. …wait, they did! The idea that premature optimization is the root of all evil comes from Donald Knuth's Structured Programming with go to Statements. Knuth attributes it to C.A.R. Hoare in The Errors of TeX, though Hoare denied that he had coined the phrase.

Anyway, the pithy phrase "premature optimization is the root of all evil", which has been interpreted as "optimization before the thing is already running too slow is to be avoided", actually appears in this context:
There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, [they] will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgements about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail.
Indeed this whole subsection on efficiency opens with Knuth explaining that he does put a lot of effort into optimizing the critical parts of his code.
I now look with an extremely jaundiced eye at every operation in a critical inner loop, seeking to modify my program and data structure […] so that some of the operations can be eliminated. The reasons for this approach are that: a) it doesn't take long, since the inner loop is short; b) the payoff is real; and c) I can then afford to be less efficinet in the other parts of my programs, which therefore are more readable and more easily written and debugged. Tools are being developed to make this critical-loop identification job easy (see for example [Dan Ingalls, The execution time profile as a programming tool] and [E. H. Satterthwaite, Debugging tools for high level languages]).
So yes, optimize your code, but optimize the bits that benefit from optimization. NetNewsWire is a Mac application, and Apple's own documentation on improving your app's performance describe an iterative approach for finding underperforming characteristics (note: not "what is next to optimize", but "what are users encountering that needs improvement"), making changes, and verifying that the changes led to an improvement:
Plan and implement performance improvements by approaching the problem scientifically:

 	Gather information about the problems your users are seeing.
 	Measure your app’s behavior to find the causes of the problems.
 	Plan one change to improve the situation.
 	Implement the change.
 	Observe whether the app’s performance improves.

I doubt that this post will change the "any optimization is the root of all evil" narrative, because there isn't a similarly-trite epithet for the "optimize the parts that need it" school of thought, but at least I've tried.

#### Maybe you are going to need it
In the beginning, there was the green field. The lead developer, who may have been the only developer, agreed with the product owner (or "the other member of the company" as they were known) what they would build for the first two weeks. Then File-&gt;New Project… happened, and they smashed it out of the park.

The amorphous and capricious "market" liked what they had to offer, at least enough to win some seed funding. The team grew, and kept the same cadence: see what we need to do for the next ten business days, do it, celebrate that we did it.

As the company, its customers, and its market mature, things start to slow down. It's imperceptible at first, because velocity stays constant. The CTO can't help but think that they get a lot less out of a 13-point story than they used to, but that isn't a discussion they're allowed to have. If you convert points into time then you're doing old waterfall thinking, and we're an agile team.

Initially the dysfunction manifests in other ways. Developers complain that they don't get time to refactor, because "the business" doesn't understand the benefits of clean code. Eventually time is carved out to clean things up, whether in "hardening sprints" or in effort allocated to "engineering stories". We are getting as much done, as long as you ignore that less of it is being done for the customers.

Stories become task-sliced. Yes, it's just adding a button, but we need to estimate the adding a component task, the binding the action task, the extending the reducer task, the analytics and management intelligence task. Yes we are getting as much done, as long as you ignore that less of it has observable outcomes.

Rework increases too, as the easy way to fit a feature into the code isn't the way that customers want to use it. Once again, "the business" is at fault for not being clear about what they need. Customers who were previously flagship wins are now talked about as regressive laggards who don't share the vision. Stories must have clearer acceptance criteria, the definition of done must be more explicit: but obviously we aren't talking about a specification document because we're an agile team. Yes we're getting as much done, as long as you ignore that a lot of what we got done this fortnight was what we said we'd done last fortnight.

Eventually forward progress becomes near zero. It becomes hard to add new features, indeed hard even to keep up with the competitors. It's only two years ago that we were five years ahead of them. People start demoing new ideas in separate apps, because there's no point dreaming about adding them to our flagship project. File-&gt;New Project… and start all over again.

What happened to this team? Or really, to these teams, as I've seen this story repeated over and over. They misread "responding to change over following a plan" as "we don't need no stinking plan".

Even if you don't know exactly where you are going at any time, you have a good idea where you think you're going. It might be spread around the company, which is why we need the experts around the table. Some examples of where to find this information:

The product owner has a backlog of requested features that have yet to be built.
The sales team have a CRM indicating which prospects are hottest, and what they need to offer to close those deals.
The marketing director has a roadmap slide they're presenting at a conference next month.
The CTO has budget projections for the next financial year, including headcount changes and how they plan to reorganise the team to incorporate these changes.
The CEO knows where they want to position the company in the market over the next two years, and knows which competitors, regulatory changes, and customer behaviours threaten that position and what about them makes them a threat.
Countless spreadsheets, databases, and "business intelligence" dashboards across multiple people and departments.

No, we don't know the future, but we do know which futures are likely and of those, which are desirable. Part of embracing change is to make those futures easier to cope with. The failure mode of many teams is to ignore all futures because we aren't in any of them yet.
We should be ready for the future we expect, and both humble and adaptable enough to get ready for a different future when things change. Our software should represent our current knowledge of our problem and its solution, including knowledge about likely developments (hey, maybe there's a reason they call us developers!). Don't add the things you aren't going to need, but don't exclude the possibility of adding them out of spite for a future that may well come to pass.

#### Me.com.  Your identity, everywhere.
Title linky goes to a Sophos blog post I wrote about the relative success of MobileMe phishing scams, and the insecurity of MobileMe web access.

#### Meaningless Vapid Catchphrase
On the 4th December 2013, I said:
Urge to search the archives for papers on Model-View-Controller and write an essay on its ever-changing meaning in programmer discourse.
Do you have any idea how much work that is? I do, now. So I'm going to cover a very small part of the story here, with a view—your interest and my inclination permitting—to adding to it in subsequent posts.
Part One: you've lost your thing
The genesis of MVC comes in 1979 with a pair of memos written by Trygve Reenskaug of the Xerox Learning Research Group. In the first, A Note on Dynabook Requirements, Reenskaug documents an approach to tackling problems in the Smalltalk environment by describing his design for a project management task.

A fun digression at this point is the observation that Reenskaug appears to have described what we now call the user story: a goal-directed statement of what someone's trying to achieve, as a replacement for a specification of what some theoretical software product should do.
Most problems would start with a rather unclear and often self-contradictory goal. Some examples: I would like to get better control over my finances; I don’t want to be troubled with detailed accounting; I want to settle my account with the butcher; I want to know more about Tarot cards; or I want a small, cheap house with many large, luxurious rooms.

I would expect the user to go more or less subconsciously through a goal-means hierarchy: Certain means are needed to reach a given goal. These means are not immediately available, but constitute a new set of part-goals, each of which needs certain means for their satisfaction, and so on.

An example: To get better control over my finances, I would need to set up a budget; to keep account of all income and expenditure; and to keep a running comparison between budget and accounts. Three new, non-trivial goals that need further consideration.
He goes on to note that given a sufficiently consistent collection of goals, the computer should probably just solve the problem itself. In the absence of evidence that this is possible (Prolog notwithstanding), we'll need to tell the computer the methods by which the problem is solved.

Anyway, back to MVC. The solution in the context of his project management problem then follows. Along the way he introduces some metaphors that are formalised in a glossary in the second memo, called not Model-View-Controller but Thing-Model-View-Editor: an Example from a planningsystem.

An immediate observation is that where MVC has three parts, TMVE has four: the thing being modelled is explicitly part of the problem. MVC is a way to do computer stuff, TVME is a way to use a computer stuff to solve a problem based in the real world. Of course, you cannot put the thing itself inside the computer, except in the world of Tron which was yet to be released. So instead you find some useful abstraction and represent it in the computer as a model. This model contains both the data and the actions appropriate to the abstraction it represents.

The image below, an adaptation of a figure that appears in both documents, demonstrates that a nebulous thing can be modelled by multiple different abstractions, or that multiple models can represent different parts of the thing.

Now for any model, there will be one or more views that represent the model in a meaningful way; models do not know how to draw or print themselves. Views can also control the model in ways appropriate to the view's representation. For example, Reenskaug shows that a view describing a model's properties could accept changes to those properties, and make the required changes to the model.

Now there's going to be a complicated object graph in place, with one or more model objects each represented by one or more views, all to solve a problem with one particular thing. The editor is a coordinator for all of these. It acts as a command interface, mediating between the user and this network of objects. As changes are made, the editor coordinates with all of the views.

You will recall that Trygve Reenskaug was on the Learning Research Group at Xerox. A Smalltalk system like the proposed Dynabook is supposed to be a computer that lets people solve their own problems, by being easy to program and to manipulate. Therefore, unlike modern MVC which is a design tool for professional programmers, Thing-Model-View-Editor is a user interface paradigm, describing how people can build their own programs to solve their own problems. The editor may have a doit command to send Smalltalk messages, but sending Smalltalk messages should not be beyond the wit of an interested and engaged user.

This goes some of the way to explaining why "computery stuff" like persistence, networking and the like are not tackled in TVME. They are outside its purview: the computer should be looking after that stuff. Just as the humane interface on a Canon Cat has no save or load buttons, and the computer looks after storage itself, so Dynabooks do not have databases, or filesystems, or memory allocators. These are abstractions of the thing "computer", but Thing-Model-View-Editor is concerned with the thing that people want to represent in the computer. The computer is not the thing.

#### Messily Veneered C
A recap: we saw that Model-View-Controller started life as Thing-Model-View-Editor, a way of approaching problems to design Smalltalk user interfaces. As Smalltalk-80 drifted off from its ivory tower, many Smalltalkers were using and talking about MVC, although any kind of consensus on its meaning was limited by the lack of important documentation.

Would this telephone game continue to redefine Model-View-Controller as object-oriented programming grew beyond the Smalltalk-80 community?
Part 3: Objective-MVC
Objective-C is now best known as the programming language Apple promotes for developing iOS and Mac applications. It was first available on the Mac during the 1980s pre-Cambrian explosion of OOP languages: on the Mac alone you could send messages using Smalltalk-80, Object Pascal, Objective-C, ExperCommon Lisp, Allegro Common Lisp, Object Assembler, Object Logo, and there were probably more. Most of the list here was derived from Object-Oriented Programming for the Macintosh, written by Kurt J. Schmucker then of Productivity Products International.

Which brings us back to Objective-C, which is probably the best known of the products produced by Productivity Products. Objective-C did not let programmers use a spatial programming interface, it made them use linear text files.

Objective-C also did not follow the Model-View-Controller paradigm of Smalltalk-80. As described by Brad Cox in this figure adapted from Object-Oriented Programming: an Evolutionary Approach, the PPI programmers were thinking about Models and Views, and then about reusing Views in different UIs.

So, what happened to the Controllers? Here's Cox:
This architecture is very similar to the one used in Smalltalk-80, with one exception. [...] The outgoing leg of Smalltalk's user interface is handled by a hierarchy of views much like the ones discussed here. But the incoming leg is implemented by a separate hierarchy of classes, Controllers, that provide the control for the application. [...] The need for the separate controller hierarchy is unclear and is the topic of spirited debate even within the Smalltalk-80 community.
Unfortunately the citation for "spirited debate" is personal communication with various Smalltalkers, so we may never know the content.

This gives us three Objective-C modifications to the original Smalltalk-80 concept that persist to this day. One is the move from a user interface paradigm concerning the interactions of individual objects to a layered architecture. The Cox book shows the display screen as a projection of the view layer onto glass, and the models all in their own two-dimensional layer suspended from the views, hanging:
suspended from the presentation level by pointers in much the way that circuit boards are connected to test equipment with a bed-of-nails testing jig.
That still persists in Cocoa MVC, though with a somewhat weak assertion:
The collection of objects of a certain MVC type in an application is sometimes referred to as a layer—for example, model layer.
The second thing is the abstraction of the Presentation layer away from the display primitives.

The other thing that we still don't have in ObjC is the Controller. Wait, what? Surely Objective-C applications have Controllers. Apple and NeXT both talked about it, and they have objects with the name "Controller" in them. Surely that's MVC.

For whatever reason, NeXT kept the ObjC-style Views with their handling of both input and output, but also had to reintroduce the idea of a Controller. So they took the same approach followed by others, notably including Ivar Jacobson a few years later, of defining the Controller layer to contains whatever's left once you've worked out what goes into the Model and View layers.

Now there's only the question of placement. The Controller can't go where Smalltalk's Controllers went, because then it would get in the way of the input events which now need to get to the View. It could go between the two:
A controller object acts as an intermediary between one or more of an application’s view objects and one or more of its model objects. Controller objects are thus a conduit through which view objects learn about changes in model objects and vice versa. Controller objects can also perform setup and coordinating tasks for an application and manage the life cycles of other objects.
Here's a reproduction of the diagram from that document:

Notice that for the Controllers to act as a mediator between Models and Views, the Views actually have to forward action messages on to the Controllers (via delegation, target-action or the responder chain). Had the Controller been left where it was it would already be receiving those events.

Compare that with the diagram derived from Smalltalk MVC:

It's clear that these are different concepts that unfortunately share a name.

#### Messing about with Clang
I've been reading the Smalltalk-80 blue book (pdf) recently, and started to wonder what a Smalltalk style object browser for Objective-C would look like. Not just from the perspective of presenting the information that makes up Objective-C classes in novel ways (though this is something I've discussed with Saul Mora at great length in the past). What would an object browser in which the compiler is an object, so you can define and manipulate classes in real time, look like?

Well, the first thing you'd need to do is to turn the compiler into an object. I decided to see whether I could see what the compiler sees, using the clang compiler front-end library.

Wait, clang library? Clang's a command-line tool, isn't it? Well yes, but it and the entire of LLVM are implemented as a collection of reusable C++ classes. Clang then has a stable C interface wrapping the C++, and this is what I used to produce this browser app. This isn't the browser I intend to write, this is the one I threw away to learn about the technology.

Clang is a stream parser, and there are two ways to deal with source files just like any other stream: event-driven[*], in which you let the parser go and get callbacks from it when it sees interesting things, or document-based[*] where you let the parser build up a document object model (a tree, in this case) which you then visit the nodes of to learn about the data.

[*] Computer scientists probably call these things something else.

Being perverse, I'm going to use the event-driven parser to build a parallel data model in Objective-C. First, I need to adapt the clang library to Objective-C, so that the compiler is an Objective-C object. Here's my parser interface:

#import &lt;Foundation/Foundation.h>

@protocol FZAClassParserDelegate;

@interface FZAClassParser : NSObject

@property (weak, nonatomic) id &lt;FZAClassParserDelegate>delegate;

- (id)initWithSourceFile: (NSString *)implementation;
- (void)parse;

@end

The -parse method is the one that's interesting (I presume…) so we'll dive into that. It actually farms the real work out to an operation queue:

#import &lt;clang-c/Index.h>

//...

- (void)parse {
    __weak id parser = self;
    [queue addOperationWithBlock: ^{ [parser realParse]; }];
}

- (void)realParse {
#pragma warning Pass errors back to the app
    @autoreleasepool {
        CXIndex index = clang_createIndex(1, 1);
        if (!index) {
            NSLog(@"fail: couldn't create index");
            return;
        }
        CXTranslationUnit translationUnit = clang_parseTranslationUnit(index, [sourceFile fileSystemRepresentation], NULL, 0, NULL, 0, CXTranslationUnit_None);
        if (!translationUnit) {
            NSLog(@"fail: couldn't parse translation unit);
            return;
        }
        CXIndexAction action = clang_IndexAction_create(index);

That's the setup code, which gets clang ready to start reading through the file. Which is done in this function:

        int indexResult = clang_indexTranslationUnit(action,
                                                     (__bridge CXClientData)self,
                                                     &indexerCallbacks,
                                                     sizeof(indexerCallbacks),
                                                     CXIndexOpt_SuppressWarnings,
                                                     translationUnit);

This is the important part. Being a C callback API, clang takes a context pointer which is the second argument: in this case, the parser object. It also takes a collection of callback pointers, which I'll show next after just showing that the objects created in this method need cleaning up.

        clang_IndexAction_dispose(action);
        clang_disposeTranslationUnit(translationUnit);
        clang_disposeIndex(index);
        (void) indexResult;
    }
}

There's a structure called IndexCallbacks defined in Index.h, this class's structure contains functions that call through to methods on the parser's delegate:

int abortQuery(CXClientData client_data, void *reserved);
void diagnostic(CXClientData client_data,
                CXDiagnosticSet diagnostic_set, void *reserved);
CXIdxClientFile enteredMainFile(CXClientData client_data,
                                CXFile mainFile, void *reserved);
CXIdxClientFile ppIncludedFile(CXClientData client_data,
                               const CXIdxIncludedFileInfo *included_file);
CXIdxClientASTFile importedASTFile(CXClientData client_data,
                                   const CXIdxImportedASTFileInfo *imported_ast);
CXIdxClientContainer startedTranslationUnit(CXClientData client_data,
                                            void *reserved);
void indexDeclaration(CXClientData client_data,
                      const CXIdxDeclInfo *declaration);
void indexEntityReference(CXClientData client_data,
                          const CXIdxEntityRefInfo *entity_reference);

static IndexerCallbacks indexerCallbacks = {
    .abortQuery = abortQuery,
    .diagnostic = diagnostic,
    .enteredMainFile = enteredMainFile,
    .ppIncludedFile = ppIncludedFile,
    .importedASTFile = importedASTFile,
    .startedTranslationUnit = startedTranslationUnit,
    .indexDeclaration = indexDeclaration,
    .indexEntityReference = indexEntityReference
};

int abortQuery(CXClientData client_data, void *reserved) {
    @autoreleasepool {
        FZAClassParser *parser = (__bridge FZAClassParser *)client_data;
        if ([parser.delegate respondsToSelector: @selector(classParserShouldAbort:)]) {
            return [parser.delegate classParserShouldAbort: parser];
        }
        return 0;
    }
}

// …

Internally clang creates its own threads, so the callback functions wrap delegate messages in @autoreleasepool so that the delegate doesn't have to worry about this.

The delegate still needs to understand clang data structures of course, this is where the real work is done. Here's the delegate that's used to build the data model used in the browser app:

#import &lt;Foundation/Foundation.h>
#import "FZAClassParserDelegate.h"

@class FZAClassGroup;

@interface FZAModelBuildingParserDelegate : NSObject &lt;FZAClassParserDelegate>

- (id)initWithClassGroup: (FZAClassGroup *)classGroup;

@end

The FZAClassGroup class is just somewhere to put all the data collected by parsing the file: in a real IDE, this might represent a project, a translation unit, a framework or something else. Anyway, it has a collection of classes. The parser adds classes to that collection, and methods and properties to those classes:

@implementation FZAModelBuildingParserDelegate {
    FZAClassGroup *group;
    FZAClassDefinition *currentClass;
}

- (id)initWithClassGroup:(FZAClassGroup *)classGroup {
    if ((self = [super init])) {
        group = classGroup;
    }
    return self;
}

- (void)classParser:(FZAClassParser *)parser foundDeclaration:(CXIdxDeclInfo const *)declaration {
    const char * const name = declaration->entityInfo->name;
    if (name == NULL) return; //not much we could do anyway.
    NSString *declarationName = [NSString stringWithUTF8String: name];

We've now got a named declaration, but a declaration of what?
    switch (declaration->entityInfo->kind) {
        case CXIdxEntity_ObjCProtocol:
        {
            currentClass = nil;
            break;
        }
        case CXIdxEntity_ObjCCategory:
        {
            const CXIdxObjCCategoryDeclInfo *categoryInfo = 
            clang_index_getObjCCategoryDeclInfo(declaration);
            NSString *className = [NSString stringWithUTF8String: categoryInfo->objcClass->name];
            FZAClassDefinition *classDefinition =[group classNamed: className];
            if (!classDefinition) {
                classDefinition = [[FZAClassDefinition alloc] init];
                classDefinition.name = className;
                [group insertObject: classDefinition inClassesAtIndex: [group countOfClasses]];
            }
            currentClass = classDefinition;
            break;
        }
        case CXIdxEntity_ObjCClass:
        {
            FZAClassDefinition *classDefinition =[group classNamed: declarationName];
            if (!classDefinition) {
                classDefinition = [[FZAClassDefinition alloc] init];
                classDefinition.name = declarationName;
                [group insertObject: classDefinition inClassesAtIndex: [group countOfClasses]];
            }
            currentClass = classDefinition;
            break;
        }

I'm ignoring protocols, but recognising that methods declared in a protocol shouldn't go onto any particular class. Similarly, I'm adding methods found in categories to the class on which that category is defined: real Smalltalk browsers keep the categories, but for this prototype I decided to skip them. I'm using the fact that this is a prototype to justify having left the duplicate code in place, above :-S.

So now we know what class we're looking at, we can start looking for methods or properties defined on that class:

        case CXIdxEntity_ObjCClassMethod:
        case CXIdxEntity_ObjCInstanceMethod:
        {
            FZAMethodDefinition *method = [[FZAMethodDefinition alloc] init];
            method.selector = declarationName;
            if (declaration->entityInfo->kind == CXIdxEntity_ObjCClassMethod)
                method.type = FZAMethodClass;
            else
                method.type = FZAMethodInstance;
            [currentClass insertObject: method inMethodsAtIndex: [currentClass countOfMethods]];
            break;
        }
        case CXIdxEntity_ObjCProperty:
        {
            FZAPropertyDefinition *property = [[FZAPropertyDefinition alloc] init];
            property.title = declarationName;
            [currentClass insertObject: property inPropertiesAtIndex: [currentClass countOfProperties]];
            break;
        }
        default:
            break;
    }
}

And that's "it". The result of collecting all of these callbacks is a tree:

ClassGroup -> Class -> [Method, Property]

I define a tree-ish interface for all of these classes, by adding categories that define the same methods:

@interface FZAMethodDefinition (TreeSupport)

- (NSInteger)countOfChildren;
- (NSString *)name;
- (id)childAtIndex: (NSInteger)index;
- (BOOL)isExpandable;

@end

@implementation FZAMethodDefinition (TreeSupport)

- (NSInteger)countOfChildren {
    return 0;
}

- (BOOL)isExpandable {
    return NO;
}

- (id)childAtIndex:(NSInteger)index {
    return nil;
}

- (NSString *)name {
    switch (self.type) {
        case FZAMethodClass:
            return [@"+" stringByAppendingString: self.selector];
            break;
        case FZAMethodInstance:
            return [@"-" stringByAppendingString: self.selector];
            break;
        default:
            return [@"?" stringByAppendingString: self.selector];
            break;
    }
}

@end

And, well, that's it. libClang could be the kernel of a thousand visualizers, browsers and editors for C-derived languages, the start of one is outlined above.

#### Meta-writing
Barely 4,000 years ago, documents were written on heavy, clay tablets. The Epic of Gilgamesh, one of the earliest known works of fiction, was written on 11 such tablets with a 12th added later. There was only one thing you could do with these tablets: read. Fast forward to the 21-st century and things are very different. The word "tablet" has taken on a new meaning, and documents can be delivered wirelessly, updated as new versions are written. They can also contain rich media and hyperlinked references to other content. And with these new capabilities come new considerations when preparing your documents—or "docs"—for your readers.

The above story seems rambling and pointless, doesn't it? But change the timescale and the technology, and every single bloody report on mobile technology starts in exactly the same way.

#### Metacognition-driven development
To find out what techniques work for you in a field of practice, you often need to think about how you think. To decide what it is that drives your learning processes, and then adapt your practices to suit that.

For example, I tend to get more done if I'm already getting something done. Success breeds success. If I don't feel like I'm succeeding, then I'll break off and do something else (like write a blog post in my lunch break). I don't think that's unique to me, though, as common phrases like "on a roll" and "in the zone" seem to imply being in a state where progress begets more progress.

Cognition scientists know this, too. The authors of Yes! describe an experiment in which they tried two types of card in a hotel to get guests to reuse their towel instead of having them laundered every day. One contained the message that if a guest reused her towel, the hotel would make a donation to an environmental charity on her behalf. The other said that the hotel had already made the donation, and all the guest had to do to complete the story was to reuse her towel.

The second card was more effective. It implies that progress has already been made: that the guest has already succeeded (albeit vicariously) in making a donation. Success begets success, so the guests are more likely to make continued progress: i.e. to reuse the towels.

That is, I think, why TDD makes me work faster than when I'm not doing TDD. Because I can see the green lights, because I can see the little nuggets of success that will go together to make the feature work. It turns "I still haven't posted the message to the server" into "woo! I just configured the URL request". It turns "still losing" into "yet another win". It doesn't trick me into thinking I'm making more progress, but it does let me see that progress is being made.

#### Microblogging
For a long time, I deliberately avoided microblogs like twitter.  I thought that they were simply an acknowledgement that people want to be published more than they want to have something to say.  However, it would be rude of me to completely disavow the medium without actually giving it a go.To that end, I may indeed be iamleeg on twitter, as soon as twitter actually finishes processing the signup form.I'd like to point out that one problem I'm going to have is brevity - I have spent 650 characters telling you what my username is.  Constraining myself to SMS-sized wibblings will indeed be tricksy.

#### Microservices for the Desktop
In OOP the Easy Way, I make the argument that microservices are a rare instance of OOP done well:
Microservice adopters are able to implement different services in different technologies, to think about changes to a given service only in terms of how they satisfy the message contract, and to independently replace individual services without disrupting the whole system. This […] sounds a lot like OOP.
Microservices are an idea from service-oriented architecture (SOA) in which each application—each microservice—represents a distinct bounded context in the problem domain. If you're a movie theatre complex, then selling tickets to people is a very different thing from showing movies in theatres, that are coupled loosely at the point that a ticket represents the right to a given seat in a given theatre at a given showing. So you might have a microservice that can tell people what showings there are at what times and where, and another microservice that can sell people tickets.

People who want to write scalable systems like microservices, because they can scale different parts of their application separately. Maybe each franchisee in the theatre chain needs one instance of one service, but another should scale as demand grows, sharing a central resource pool.

Never mind all of that. The real benefit of microservices is that they make boundary-crossing more obvious, maybe even more costly, and as a result developers think about where the boundaries should be. The "problem" with monolithic (single-process) applications was never, really, that the deployment cost too much: one corollary of scale is that you have more customers. It was that there was no real enforcement of separate parts of the problem domain. If you've got a thing over here that needs that data over there, it's easy to just change its visibility modifier and grab it. Now this thing and that thing are coupled, whoops!

When this thing and that thing are in separate services, you're going to have to expose a new endpoint to get that data out. That's going to make it part of that thing's public commitment: a slightly stronger signal that you're going down a complex path.

It's possible to take the microservices idea and use it in other contexts than "the backend". In one Cocoa app I'm working on, I've taken the model (the representation in objects of the problem I'm solving) and put it into an XPC Plugin. XPC is a lot like old-style Distributed Objects or even CORBA or DCOM, with the exception that there are more safety checks, and everything is asynchronous. In my case, the model is in Objective-C in the plugin, and the application is in Swift in the host process.

"Everything is asynchronous" is a great reminder that the application and the model are communicating in an arm's-reach fashion. My model is a program that represents the domain problem, as mentioned before. All it can do is react to events in the problem domain and represent the changes in that domain. My application is a reification of the Cocoa framework to expose a user interface. All it can do is draw stuff to the screen, and react to events in the user interface. The app and the model have to collaborate, because the stuff that gets drawn should be related to the problem, and the UI events should be related to desired changes in the domain. But they are restricted to collaborating over the published interface of the XPC service: a single protocol.

XPC was designed for factoring applications, separating the security contexts of different components and giving the host application the chance to stay alive when parts of the system fail. Those are valid and valuable benefits: the XPC service hosting the model only needs to do computation and allocate memory. Drawing (i.e. messaging the window server) is done elsewhere. So is saving and loading. And that helps enforce the contract, because if I ever find myself wanting to put drawing in the model I'm going to cross a service boundary, and I'm going to need to think long and hard about whether that is correct.

If you want to talk more about microservices, XPC services, and how they're different or the same, and how I can help your team get the most out of them, you're in luck! I've recently launched the Labrary—the intersection of the library and the laboratory—for exactly that purpose.

#### Minimum Viable Controller
The book "NeXTstep Programming Step One: Object-Oriented Applications" by Garfinkel and Mahoney said this about Controllers in 1993:

A good rule of thumb is to place as little code in your controller as necessary. If it is possible to create a second controller that is only used for a particular function, do so - the less complicated you make your application's objects, the easier they are to debug.

Later, on the same page (p131):

Before you start coding, it's a good idea to sit down and think about your problem.

Both of these pieces of advice still apply. Neither has been universally internalised 24 years later.

#### Miniwindows
Maybe it's just me who gets annoyed by teeny-tiny miniaturised views which are completely illegible.  Even so, I've just uploaded an article I wrote on Miniwindows which can be used in any OpenStep implementation such as Cocoa or GNUstep.  It's based on the Hillegass TypingTutor example, but doesn't really use any code from that and isn't (I hope) otherwise reliant on that context, so it should be possible to see what's going on even if you haven't read Hillegass.  Which you should ;-)

#### Missing Vital Content
After reading Moderately Valuable Cliché, reader Nicholas Levin got in touch to recommend that I look at the back of my Smalltalk-80 books. Here's the blue one.

Addison-Wesley's Smalltalk-80 series.

The first book mentioned in "Other books in the … Series" is the red book, "Smalltalk-80: The Interactive Programming Environment". The third is the green book, "Smalltalk-80: Bits of History, Words of Advice". But the middle one? "Smalltalk-80: Creating a User Interface and Graphical Applications"? No-one ever assigned that one a colour. It was never published.

Discussions of the book in comp.lang.smalltalk in 1986 indicate that it was a manifesto for Model-View-Controller, promoting and describing the pattern for building Smalltalk applications. The thread instead recommends looking at Chapter 9 of a book that has been mentioned on this blog before: Object-Oriented Programming: an evolutionary approach by Brad Cox. More on that later.

So the idea behind MVC was well-known in the Smalltalk user community, but the principle was not documented by the Smalltalk team themselves (as far as I know: it's possible there are documents I haven't found) until Adele Goldberg left and founded ParcPlace with others at the end of the 1980s.

Levin also pointed me at a couple of other resources. In this interview for the Computer History Museum, Adele Goldberg confirms that the missing book was "to be an applications level book which was basically the model-view-controller metaphor".

In an interview for the IEEE Global History Network, Goldberg explains how the whole concept of the Dynabook was to allow people to model real-world problems on their portable computers:
That’s what we thought it was, a PDP-10, which is lightweight compared to what we have now. It was going to be there, like a notepad, but it was going to be a computational-based device that allowed you to build models of the world and test your understanding of the world. What students who succeed really do is, they interact with teachers and parents and other people—and it could be other students—who challenge the models they build of their world; and this is true whether it’s science, math, or social studies: that you’re constantly constructing and deconstructing. And when someone really pushes you to think about your models, make the models explicit, talk about those models, and be able to build those models and then challenge those models—I mean, every well educated person I know remembers in their childhood that that was what was going on all the time.
In other words, Smalltalk is the programming equivalent of "[taking] apart engines and [putting] them back together again, to understand how things work".

The long time over which the books were written and the fact that the system was being designed in reaction to problems found in documenting it ("In order to get the language book done we kind of organized ourselves so that Dan Ingalls and I, we'd have an aspect of the system we'd need to agree on like, what would be the collection classes? How would you provide collections?") explains why the blue book's FinancialHistory example can have no MVC, the red book briefly mentions MVC, and the authors of the ParcPlace paper, looking back, can remember it as a key part of the sample. Perhaps that example was eventually fleshed out for the unprinted book, or was developed further for another reason beyond the publication of the blue book.

We also have a possible reason for the subsequent diversity of interpretations of MVC among applications developers. With no primary reference for the original intention behind MVC, as the Addison-Wesley book was unwritten, people guessed at what Model-View-Controller meant from the bits and pieces that were published or discussed in the community.

#### Moderately Valuable Cliché
In part 1 of the MVC story, I examined "Thing-Model-View-Editor", a pattern[*] extracted by Trygve Reenskaug's work in Smalltalk-76. By the time Smalltalk-80's hot air balloon set sail from the ivory tower, there was already a structure called Model-View-Controller.
Part Two: MVC and Smalltalk
[*] Bear in mind that I'm retroactively applying the word "pattern" here. Design Patterns as a thing, modelled on the book modelled on the other book, do not belong in this story for another 15 years or so.

This story will be told through the lens of a single overview article: A Description of the Model-View-Controller User Interface Paradigm in the Smalltalk-80 System by Krasner and Pope is the first. The authors give their affiliation as "ParcPlace Systems, Inc.", a company that built tools on top of Smalltalk. It was founded by ACM president Adele Goldberg, who originally developed Smalltalk along with Alan Kay and others.

Krasner and pope say that MVC is an application of a particular three-way factoring of a software problem into objects:
separating (1) the parts that represent the model of the underlying application domain from (2) the way the model is presented to the user and from (3) the way the user interacts with it.
They go on to say that it was observed that most Smalltalk-76 applications offer the same interaction—images, buttons, text and menus—and that the goal of the MVC factoring in Smalltalk-80 is to handle those interactions in a common way across all applications.

Smalltalk MVC, adapted from Krasner and Pope 1988.

The model and the view hold the same positions that they did in TVME: the models represent the thing in a computerey way, and the views represent the model on the bitmap display. "Controllers are used to send messages to the model, and provide the interface between the model with its associated views and the interactive user interface devices (e.g., keyboard, mouse). Each view may be thought of as being closely associated with a controller, each having exactly one model, but a model may have many view/controller pairs." The controller is to MVC as the editor was to TVME.

So, the controller is the thing that senses and responds to UI events. Because you probably only intend for one thing to happen when you press a key or click a mouse button, only one controller is active at a time. That controller can interpret your command as a requirement to edit or query the model, or to display or change a particular view.

The views can also edit or query the model. So, there's a need for the model to let the controller know when it was changed by a view, and the views should know when the model was changed by the controller. Enter dependents!

Dependents represent a realisation of what would later be known as the Observer pattern. The model can keep track of a collection of the objects that care about its changes. When it changes, it sends a changed or changed: message to its dependents.

Smalltalk-80 supplies bundled abstract classes for each of Model, View and Controller. The article goes into each of these classes and some of the subclasses that have already been created, showing how inspectors, editors and debuggers in Smalltalk-80 are all built from this factored collection of objects. It also details complete applications, including the FinancialHistory tutorial from "Smalltalk-80: the Language and its Implementation" by Goldberg and Robson.

Looking in the blue book, as it's known, none of the terms MVC, model, view, nor controller appear in its index. The classes Model, View and Controller are not in the example class index, and the dependents, changed and changed: methods are not in its implementation index. The FinancialHistory object is described throughout the book, but is a subclass of Object and does not participate in any factoring that's similar to MVC.

It's probable that this reference is a mistake, and that Krasner and Pope intended to refer to the red book "Smalltalk-80: the Interactive Programming Environment" by Goldberg. This does indeed describe MVC, but only in relationship to the "MVC Inspector". However, the description of the inspector shows that the pattern was common enough to motivate the construction of bespoke UI:
Many objects in the Smalltalk-80 system are closely related to one another. In particular, the user interface of the system is implemented using subclasses of two classes, class View and class Controller. A View represents ways of presenting information on the screen; a Controller represents ways in which the user interacts with a screen view. Any View is related to a Controller and to another object, the object whose information is accessed in the View. This other object is referred to as a "model." Whenever you inspect a View, you typically want to inspect its related model or Controller. There is a special inspector in the system for inspecting the two main objects related to a View, as well as the View, whenever a View is sent the message inspect.
Having mentioned the red and blue books, it seems appropriate to introduce the green book, "Smalltalk-80: Bits of History, Words of Advice" edited by Glenn Krasner (the first author of the paper we started at). There is no MVC in the green book, but the three books[*] together are a fascinating snapshot of this time in our industry.

[*] There is a purple book too, but it's just half of the blue book repackaged.

So Smalltalk-80 MVC is a formalisation of the abstract collaboration between objects documented as Thing-Model-View-Editor. It is an idiom named after the developer tool that was designed to take advantage of the common relationships between these objects. At some point after its initial documentation, authors (Krasner and Pope along with others) decided that this was not merely a factoring, but a "User Interface Paradigm".

#### More about the privacy pledge
Plenty of you have seen&mdash;and indeed signed&mdash; the App Makers' Privacy Pledge on GitHub. If you haven't, but after reading it are interested, see the instructions in the project README.

It's great to see so many app makers taking an interest in this issue, and the main goal of the pledge is to raise awareness of app privacy concerns: awareness among developers that this is something to take seriously, and awareness among our customers that there are developers committed to respecting their identities and their data.

But awareness is useless if not followed through, so we need to do more. We need materials that developers can refer to: the GSM Association have good guidelines on app practices. We need actionable tasks that developers can implement right away, like Matt Gemmell's hashing guide for social apps. We need sample code and libraries that developers can rely on. We need data lawyers to explain what the current regulations are, and what's coming down the pipe. We need to convince the industry and the governments that we can regulate our own actions. We need the ability to audit our apps and determine whether they're privacy-preserving. We need to be able to demonstrate to customers what we've done, and explain why that's a good thing. We need to earn customer trust.

So there's a lot to do, and the pledge is only the start. It's off to a good start, but there's still a long way to go.

#### More detail on software requirements
My talk at AppDevCon discussed the Requirements Trifecta but turned it into a Quadrinella: you need leadership vision, market feedback, and technical reality to all line up as listed in the trifecta, but I’ve since added a fourth component. You also need to be able to tell the people who might be interested in paying for this thing that you have it and it might be worth paying for. If you don’t have that then, if anybody has heard of you at all, it will be as a company that went out of business with a product “five years ahead of its time”: you were able to build it, it did something people could benefit from, in an innovative way, but nobody realised that they needed it.

A 45 minute presentation was enough to introduce that framework and describe it, but not to go into any detail. For example we say that we need “market feedback”, i.e. to know that the thing we are going to build is something that some customer somewhere will actually want. But how do we find out what they want? The simple answer, “we ask them”, turns out to uncover some surprising complexity and nuance.

At one end, you have the problem of mass-market scale: how do you ask a billion people what they want? It’s very expensive to do, and even more expensive to collate and analyse those billion answers. We can take some simplifying steps that reduce the cost and complexity, in return for finding less out. We can sample the population: instead of asking a billion people what they think, we can ask ten thousand people what they think and apply what we learn to all billion people.

We have to know that the way in which we select those 10,000 people is unbiased, otherwise we’re building for an exclusive portion of the target billion. Send a survey to people’s work email addresses on a Friday, and some will not pick it up until Sunday as their weekend is Fri-Sat. Others will be on holiday, or not checking their email that day, or feeling grumpy and inclined to answer with the opposite of their real thoughts, or getting everything done quickly before the weekend and disinclined to think about your questions at all.

Another technique we use is to simplify the questions—or at least the answers we’ll accept to those questions, to make it easier to combine and aggregate those answers. Now we have not asked “what do you think about this” at all; we have asked “which of these ways in which you might think about this do you agree with?” Because people are inclined to avoid conflict, they tend to agree with us. Ask “to what extent do you agree that spline reticulation is the biggest automation opportunity in widget frobnication?” and you’ll learn something different from the person who asked “to what extent do you agree that spline reticulation is the least important automation opportunity in widget frobnication?”

We’ll get richer information from deeper, qualitative interactions with people, and that tends to mean working with fewer people. At the extreme small end we have one person: an agent talks to their client about what that client would like to see.  This is quite an easy case to deal with, because you have exactly one viewpoint to interpret.

Of course, that viewpoint could well be inconsistent. Someone can tell you that they get a lot of independence in how they work, then in describing their tasks list all the approvals and sign-offs they have to get. It can also be incomplete. A manager might not fully know all of the details of the work their reports do; someone may know their own work very well but not the full context of the process in which that work occurs. Additionally, someone may not think to tell you everything about their situation: many activities rely on tacit knowledge that’s hard to teach and hard to explain. So maybe we watch them work, rather than asking them how they work. Now, are they doing what they’re doing because that’s how they work, or because that’s how they behave when they’re being watched?

Their viewpoint could also be less than completely relevant: maybe the client is the person paying for the software, but are they the people who are going to use it? Or going to be impacted by the software's outputs and decisions? I used the example in the talk of expenses software: very few people when asked “what’s the best software you’ve ever used” come up with the tool they use to submit receipts for expense claims. That’s because it’s written for the accounting department, not for the workers spending their own money.

So, we think to involve more people. Maybe we add people’s managers, or reports, or colleagues, from their own and from other departments. Or their customers, or suppliers. Now, how do we deal with all of these people? If we interview them each individually, then how do we resolve contradiction in the information they tell us? If we bring them together in a workshop or focus group, we potentially allow those contradictions to be explored and resolved by the group. But potentially they cause conflict. Or don’t get brought up at all, because the politics of the situation lead to one person becoming the “spokesperson” for their whole team, or the whole group.

People often think of the productiveness of a software team as the flow from a story being identified as “to do” to working software being released to production. I contend that many of the interesting and important decisions relating to the value and success of the software were made before that limited part of the process.

#### More Excel-lent Adventures
I previously wrote about Excel as the most successful IDE:

  Now what makes a spreadsheet better as a development environment is difficult to say; I’m unaware of anyone having researched it.

That research is indeed extant, and the story is well-told in A Small Matter of Programming. While Professor Nardi's focus is on end-user programming, reading her book raises questions about gaps in professional programmer tools.

Specifically, in the realm of collaboration. Programming tools support a few limited forms of collaboration:

individual work
a team of independent individuals working on a shared project
pair programmers
code review

For everything else, there's a whiteboard. Why? What's missing?

#### More micros, less Guinness
Bizarrely, the Guinness book of world records lists the “first microcomputer” as 1980's Xenix. This doesn't seem right to me:

Xenix is an operating system, not a microcomputer.Xenix was announced in 1980 but not shipped until 1981.The first computer to be designed around a microprocessor is also the first computer to be described in patents and marketing materials as a “microcomputer”—the Micral N.

#### More NSConf code signing fun
I'll be talking at the US NSConference on Tuesday, with an extended version of my talk on code signing. I'll cover how it works, what it does, what it doesn't do, and what it should do. Importantly, there are still a few seats left for the conference so if you can get to Atlanta by Tuesday, come and watch me talk!

#### More on freedom and licensing
Last month, I asked whether Freedom Zero is such a great idea, whether it's OK to limit the freedom to use the software for any purpose if you dislike the purpose to which you believe someone will put it. I gave the example of the Numerical Recipes licence as one that could be adapted to this situation.

In summary, the licence would say "you are free to read this code for the purposes of understanding it, to share the code, and to contribute changes to the code. Should you wish to use the code, contact me, we will go through some customer due diligence, and I will decide whether I want you as a customer". So far, so unexciting: imagine the usual SaaS startup pricing page with two pricing levels. The "Basic" tier is free, and scales up to 0 uses. The "Custom" tier is $contact_us pricing, and scales up to $contact_us uses.

The Lerna project was briefly licensed under a different model, which took a different approach to solve the same problem. Their approach was "you all get to treat this as a free software project, except the named organisations, who don't".

Just to get this out of the way as many of us are techie people who enjoy picking nits: this approach doesn't appear to solve the problem that the author was setting out to solve. Everybody else except @evil_corps get the software under the terms of the MIT licence, @evil_corps do not get to use the software at all. Now because I got it under the terms of the MIT licence, I am free to do whatever I want except blame the authors if it doesn't work or remove the MIT licence. This means that I am free to give it or sell it to members of @evil_corps.

Lots of people disagreed with the idea that the Lerna project maintainers, or I, would seek to restrict freedom for moral/ethical reasons, because restricting freedom itself is a moral/ethical choice that is objectionable. Unfortunately, the argument that is often advanced is a bad one.

Richard Stallman's Why programs must not limit the freedom to run them says numerous things:

The bad actors will probably ignore your license terms anyway, so why bother? This pessimistic view could equivalently be used to argue against any commercial terms (bad actors will pirate your software anyway, so why sell it?) or even RMS's own General Public License (bad actors will ignore the copyleft requirements anyway, so why bother?). In fact many actors voluntarily comply with the GPL without being asked, many compliance requests are resolved voluntarily on notification, GPL compliance usually starts with a polite, private request not a lawsuit or public shaming. So evidently bothering is useful.
Using copyright to restrict usage is abuse of copyright. But so is copyleft, although as it's a "good" abuse of copyright that RMS approves of, he's OK with it.
Limiting freedom is a slippery slope to not being able to build a usable combined system, because you would have to check all the licences to know whether you could do anything. RMS does not have a beard, he has a facial hair, and another facial hair, and another, and so on.
You shouldn't have the right to do that.

That last one is an interesting one that RMS doesn't go into in depth, I would imagine because he considers it inalienable. He wrote Freedom Zero, of course he believes in Freedom Zero.

Brad Kuhn, on the other hand, has written about whether I should have that right, and his post is an interesting one. He uses the example of the ACLU's defence of free speech to argue that having a simple platform and universally supporting a simple principle creates a powerful reputation from which to then build other principled arguments.

I think that argument is compelling, and it has brought me back round to wanting Freedom Zero again.

#### More on Layers
I was told yesterday that entity-relationship diagrams can be OK as high level descriptions of database schemata, but are not appropriate for designing a database. Enough information is missing that they are not able to model the problem.

Could the same be true of layer diagrams? Perhaps they're OK as stratospheric guides to your software, but not useful for designing that software as too much information is missing.

I don't think that's the case, though. Layer diagrams tend to be pretty much interchangeable between systems, so that I can't really tell which system I'm looking at from the layer cake. Add the difficulty that I probably can't tell how the layers communicate, and certainly can't tell how the subsystems within the layers are composed. All I can tell is that you like drawing boxes, but not too many boxes.

#### More on MacDev
Today is the day I start preparing my talk for MacDev 2009. Over the coming weeks I'll likely write some full posts on the things I decide not to cover in the talk (it's only an hour, after all), and perhaps some teasers on things I will be covering (though the latter are more likely to be tweeted).I'm already getting excited about the conference, not only because it'll be great to talk to so many fellow Mac developers but due to the wealth of other sessions which are going to be given. All of them look really interesting though I'm particularly looking forward to Bill Dudney's Core Animation talk and Drew McCormack's session on performance techniques. I'm also going to see if I can get the time to come early to the user interface pre-conference workshop run by Mike Lee; talking to everyone else at that workshop and learning from Mike should both be great ways to catch up on the latest thoughts on UI design.By the way, if you're planning on going to the conference (and you may have guessed that I recommend doing so), register early because the tickets are currently a ton cheaper. Can't argue with that :-).

#### More on UIAutomation tests
Update

The information below is mostly redundant. After filing a bug report with Apple, their engineers determined that the Xcode-detected set of macro actions (find a text field, double click, enter text) weren't working because the double click action wasn't editing the text field. It is possible to use UIAutomation Tests, you just have to carefully review the UI actions and determine that they have the effect expected, particularly after letting Xcode record UI macros.

Original Post

Unfortunately my work to organise UIAutomation tests has hit the stumbling block that the UI Automation runner doesn't use the main thread for main-thread-only APIs.

In Xcode 9 and High Sierra, the authors of that post I just linked found that it was possible to turn off the main thread checker in the Test configuration of the build scheme and get working tests anyway. Unfortunately that doesn't work for me in Xcode 10 and Mojave: the main thread checker isn't killing the app: the TSM subsystem is just refusing to do its thing. So my tests can't do straightforward things like write text into a text field. Unfortunately this is a "it's not me, it's you" moment, and I don't think I can carry on using Xcode's UI tests for my goals.

However, I still want to be able to write "end-to-end" level tests to drive my development. I have (at least) three ways to proceed:

I could find a third party library and discover whether it has the main thread problem. Calabash doesn't support Mac apps, and the other examples I can find (Cucumberish and TABTestKit) both rely on UI Automation so presumably don't address the main thread problem.
I could write the tests in AppleScript. That would be a good way to build up the AppleScript UI for the app, but it doesn't represent an end-to-end test of the GUI.
I could write the tests using NSApplication.sendEvent(_ event:) to simulate clicks, scrolls and text entry, and use the unit test runner to host them. That could work, but I have my doubts (I would guess that the runner is synchronous and stalls the main thread).

I discovered that it is possible to write the test "at the UI level" but in the unit runner, using a combination of key events and AppKit API like sendAction( to:). The trade-offs of this approach:

it takes longer, as the abstractions needed to easily find and use AppKit controls don't (currently) exist
it doesn't use the Accessibility interface so isn't an accessibility audit at the same time as a correctness test
you don't hit the same problems as with the UI Automation runner
it's much faster

This may be the best approach for now, though I'd welcome other views.

#### More progress on Parallels
Thanks to Christian Brunschen and the other kind folks at UK-nextstep-users, I've got quite a bit further with my OPENSTEP installation.  Christian pointed me to a NE-2000 driver which works without issue on the Parallels-installed OPENSTEP system.Applying Apple's patches was a little more involved - my first approach was to download them on a Mac, create an ISO image (Apple's tools can do this:

hdiutil makehybrid -iso -o OS42patches.iso OS4patches

rather nicely) then put the ISO in the virtual CD-ROM drive.  While the tarballs unpacked correctly, the package they contained was corrupt.  So I downloaded the patch (OS42MachUserPatch4 being the latest one, containing the SVGA driver and the y2k fixes) to a Solaris box, then used FTP to get it over to the OS4.2 PC (now that we've got network).  This was more successful, and the patch could be installed.So, now we're fully patched, pull out the "Default VGA" driver and put in the VBE (VESA local bus) one, using Configure.app.  Let's choose a low resolution for the first boot, just in case; reading Apple's notes, we should be able to check which modes are available and change at boot time anyway.  So I've gone for 640x480x8 (i.e. 256 colours) for the first boot... and it hangs here.This is actually after I typed "VBE Check"=Yes at the boot prompt, and it didn't show a list of modes...so something's not quite right with the VESA driver.  I'll leave it overnight in case it decides suddenly to boot, but I think that reverting to Default VGA is, for the moment, the order of the day.Edit 2006-04-10 17:36 GMT - some history revision, correcting a spelling mistake and therefore pretending I've never heard of visual basic for apps :-)

#### More security processes go wrong
I just signed a piece of card so that I could take a picture of it, clean it up and attach it to a document, pretending that I'd printed the document out, signed it, and scanned it back in. I do that about once a year (it was more frequent when I ran my own business, but then I only signed the piece of card once).

Just a little reminder: it's not having my signature that should be valued, it's having seen me perform the act of signing. Signatures can easily be duplicated. If you've decided that I'm me, and you've seen me put my signature to a document, from that moment on you can know that I signed that document. If you didn't see it, but got a validated statement from a known notary that they saw it, then fair enough. If you didn't see it, and a notary didn't see it, then all you know is that you have a sheet of paper containing some words and my signature. This should tell you nothing about how the two came into proximity.

#### More speed, lower velocity
I frequently meet software teams who describe themselves as "high velocity", they even have graphs coming from Jira to prove it, and yet their ability to ship great software, to delight their customers, or even to attract their customers, doesn't meet their expectations. A little bit of sleuthing usually discovers the underlying problem.

Firstly, let's take a look at that word, "velocity". I, like Kevlin Henney, have a background in Physics, and therefore I agree with him that Velocity is a vector, and has a direction. But "agile" velocity only measures amount of stuff done to the system over time, not the direction in which it takes the system. That story may be "5 points" when measured in terms of heft, but is that five points of increasing existing customer satisfaction? Five points of new capability that will be demoed at next month's trade show? Five points of attractiveness to prospects in the sales funnel?

Or is it five points of making it harder for a flagship customer to get their work done? Five points of adding thirty-five points of technical debt work later? Five points of integrating the lead engineer's pet technology?

All of these things look the same in this model, they all look like five points. And that means that for a "high-velocity" (but really low-velocity, high-speed) team, the natural inclination is to jump on it, get it done, and get those five points under their belt and onto the burn down chart. The faster they burn everything down, the better they look.

Some of the presenting symptoms of a high-speed, low-velocity team are listed below. If you recognise these in your team, book yourself in for office hours and we'll see if we can get you unstuck.

"The Business": othering the rest of the company. The team believes that their responsibility is to build the thing that they were asked for, and "the business" needs to tell them what to build, and to sell it.
Work to rule: we build exactly what was asked for, no more, no less. If the tech debt is piling up it's because "the business" (q.v.) doesn't give us time to fix it. If we built the wrong thing it's because "the business" put it at the top of the backlog. If we built the thing wrong it's because the acceptance criteria weren't made clear before we started.
Nearly done == done: look, we know our rolling average velocity is 20 bushels of software, and we only have 14 furlongs and two femtocandela of software to show at this demo. But look over here! These 12 lumens and 4 millitesla of software are in QA, which is nearly done, so we've actually been working really hard. The fact that you can't use any of that stuff is unimportant.
Mini-waterfall: related to work to rule (q.v.), this is the requirement that everyone do their bit of the process in order, so that the software team can optimise for requirements in -> software out and get that sweet velocity up. We don't want to be doing discovery in engineering, because that means uncertainty, uncertainty means rework, and rework means lower velocity.
Punitive estimation: we're going to rename "ambiguity" to "risk", and then punish our product owner for giving us risky stories by boosting their estimates to account for the "risk". Such stories will never get scheduled, because we'll never be asked to do that one risky thing when we can get ten straightforward things done in what we are saying is the same time.
Story per dev: as a team, our goal is to shovel as much software onto the runtime furnace as possible. Therefore we are going to fan out the tasks to every individual. We are each capable of wielding our own shovel, and very rarely do we accidentally hit each other in the face while shovelling.

#### Most favouritest and least favourite Macs
The current meme seems to be, given the 25th anniversary of the launch of the Mac (and I'm not going to let it slip by that it's the 20th anniversary of the NEXTSTEP 1.0 launch this year, either) to write an opinion on the best and worst Macs we've ever used. So here's mine.Obviously my Mac use comes as a result of Mac OS X, so neither of my choices are particularly old machines. My favourite is a definite thing, and it's the first Mac I ever bought myself - a PowerMac G4 Sawtooth. I didn't own it from new; I bought it from a contractor who was writing a WebObjects app for us when I worked at the University. It was a "G4 Server" according to the sticker, which really just meant that it came with a SCSI adaptor fitted standard, and the pièce de resistance: Mac OS X Server 1.2 installed :-). The 1.x series of MOSES (Mac OS Enterprise Server) was a release of Rhapsody, a straight port of OPENSTEP to the PowerPC with a couple of changes.  Most noticable were the configuration of startup items, the platinum UI and some changes to the Workspace to make it a little more similar to the Mac OS Finder. The 450MHz G4 I had was blazingly fast in this OS, and none too shabby when it came to more modern OS X too. When I finally sold it on it was running 10.4.My least favourite is less clear-cut, I have a choice of two. But I think I'm going to give it to the Beige G3 Power Mac I borrowed long-term, I think from the University, when I was a student. This was really a slightly updated Power Mac 8600 with a faster CPU in, but the CPU's capabilities were significantly limited by the architecture (50MHz bus, 5MBPS SCSI) which really belonged to the older 603-alike processors. I seem to remember it having some weird amount of memory, like 208MB, because I'd scrounged different chips from different places, but even with that much RAM it couldn't do more than crawl in OS X 10.0-10.2. As with the G4 I installed Server 1.2, but many of the Java components really were staggeringly slow on this system.

#### Moving on
There has been some cat-escaping-bagness, which is mainly my fault, but now that it's all official I'm going to 'announce' it myself: I've got a new job!  From the end of July, I'll be working at Sophos as Senior Software Engineer, Mac (the post is still up at the linky in the title, for the moment).

This looks like being an exciting time - I've been enjoying the ObjC hacking I do with Brainstorm and this will be an opportunity to do even more of that, and the move from services to user-installed apps will bring its own changes and new experiences.

Erm, that really is all for now.  More info as it becomes available, and all that.

#### MUI
In my last post I talked about investigating AROS, the modern, open source[*] implementation of the Amiga Operating System. Today I've spent some more time on that study, and found some things:

 	my strategy last time was to read the AROS application developer guide in order, up to the part about Intuition (the UI toolkit). This taught me about Windows and Gadgets, and the message-passing event system IDCMP. However, one of the first things I discovered today on digging more into that is that people don't use these things directly.
 	The AmigaOS Documentation eschews gadtools (the library for working with Intuition gadgets) in favour of BOOPSI, the Basic Object-Oriented Programming System for Intuition. Its documentation uses demos that only work on the latest (proprietary) AmigaOS, and not on earlier AmigaOS or AROS.
 	The AROS documentation eventually led me to discover that even gadgets in BOOPSI are old and busted, and that the Zune[**] widget set for BOOPSI (which is based on the freeware Magic User Interface) is the new hotness.
 	This follow-your-nose-and-end-up-lost documentation issue is a big problem. I've learned a lot, but mostly follow "this is how this works" with "but you don't do this". Plenty of reference is made to the original Amiga developer documentation (which, luckily, I have, on CD in an Amiga-only format) and the MUI docs (which I don't have but can probably get). This is not an AROS-specific issue, wherever "open source version of X" is documented, the documentation is likely to say "check out the documentation for X". This only works where the documentation for X is available (imagine, say, if Commodore were to go bust in 1993 and stop publishing their developer docs), and relevant to the reimplementation (imagine, say, if Apple were to change programming language and GNUstep didn't catch up).
 	That AROS is both source and (on m68k, or on UAE) binary compatible with programs that were written in the 1980s is amazing to me, and much more valuable than having latest whizzbang features and smartwatch integration from a modern platform. Leaving aside technical aesthetics which I'm likely to disagree with a lot of people on anyway, AROS and open source[*] systems like it (Haiku, ReactOS, FreeDOS, GNUstep, Lesstif) represent a critical piece of heritage infrastructure.

[*] I'm using this as a shorthand. AROS source code is published under the terms of the AROS Public License, which is not OSI-approved. The source code is available to use, study, share and improve, which many would understand to be "open".

[**] Unrelated to the Microsoft Zune, and it's unclear to me which came first.

#### Music
The problem with musicians these days is they don’t work hard enough to make Daniel Ek, Tim Cook, Sundar Pichai and Jeff Bezos rich.

#### Mutable objects in immutable objects in object-oriented programming in functional programming in Swift
I didn't realise this at the time, the previous entry wasn't the last Objective-Swift post.

The inheritance mechanism in ObjS is prototypical, meaning that an object inherits from a single other object rather than getting its behaviour from a class. This is the same system that Self and languages that, um, inherit its approach use.

This form of inheritance gives us two capabilities: decorating an object by attaching new methods, or updating an object by shadowing existing methods. And that means that setting a variable can be mimicked by returning an object where the accessor for that variable has been overridden. So in fact the objects in this system won't be mutable at all, but they will have mutators that return replacement values.

It sounds weird, but there are reasons to really do this. In Postscript, you can mimic variables by creating named functions that just push a value onto the stack. Rather than mutating the data, you "set" the variable by rebinding the name to a function that pushes the updated value.

Digression: updating the dispatch system

In rewriting setters to be immutable I also noticed it was possible to consolidate the "accessor" and "mutator" implementation types into a single type, using variadic arguments. I have genuinely no idea why I didn't realise that before, when the type of an ObjC method is usually expected to be (id, SEL, id...) -&gt; id.

enum IMP {
  case method((Selector-&gt;IMP, Selector, Selector-&gt;IMP...)-&gt;(Selector-&gt;IMP)?)
  case asInteger((Selector-&gt;IMP, Selector, Selector-&gt;IMP...)-&gt;Int?)
  case methodMissing((Selector-&gt;IMP, Selector, Selector-&gt;IMP...)-&gt;(Selector-&gt;IMP)?)
  case description((Selector-&gt;IMP, Selector, Selector-&gt;IMP...)-&gt;String?)
}

The asInteger and description types gain variadic arguments but still need to be present so you can exit the O-O type system and get back to Swift types. For the moment, methodMissing is still modelled as a separate case in the enumeration, even though it has the same type as a regular method.

Mutation as a function

Given an object, a selector to override, and the replacement value, return a new object that will return that value when messaged with that selector.

infix operator ☞ {}

func mutate(receiver: Object, selector: Selector, value:Object) -&gt; Object {
  return { _cmd in
    switch (_cmd) {
    case selector:
      return .method({ _ in return value })
    default:
      return receiver(_cmd)
    }
  }
}

func ☞(message:(receiver: Object?, selector: Selector), value:Object) -&gt; Object? {
  if let receiver = message.receiver {
    return mutate(receiver, message.selector, value)
  } else {
    return nil
  }
}

Taking the Point objects from the further advances post, but deleting their mutators, this can be used to make updated objects.

let p = Point(3, 4, o)
&#x1f4d3;(p) // (3,4)
let p2 = (p,"x")☞(Integer(1,o))
&#x1f4d3;(p2) // (1,4)

Mutation as a method

The same function can be used in the implementation of an Objective-Swift method. Here's the setY: method on Points:

  return IMP.method({ (this, aSelector, args : Object...) in
    return (this, "y")☞args[0]
  })

Now a bit of notation to make calling mutator methods easier (I'm starting to run out of reasonable symbols):

infix operator ✍ {}

func ✍(message:(receiver:Object?, selector:Selector), value:Object) -&gt; Object? {
  if let imp = message.receiver..message.selector {
    switch imp {
    case .method(let f):
      return f(message.receiver!, message.selector, value)
    default:
      return nil
    }
  } else {
    return nil
  }
}

Done.

let p3 = (p2, "setY:")✍(Integer(42, o))
&#x1f4d3;(p3) // (1,42)

Fork me

I have put the current playground (which is written for the Swift 1.2 compiler, because I'm risk averseold fashioned) on GitHub.

#### My changing relationship with books
My Delicious Library collection just hit 1,000 books. That’s not so big, it’s only a fraction of the books I’ve read in my life. I only started cataloguing my books a few years ago.

What is alarming about that is that most of the books are in my house, and most are in physical form. I read a lot, and the majority of the time I’m reading something I own. The reason it’s worrying is that these books take up a lot of space, and cost a lot of money.

I’ve had an on-again, off-again relationship with ebooks. Of course they take up less space, and are more convenient when travelling. The problems with DRM and ownership mean that I tend to only use ebooks now for books from Project Gutenberg or the internet archive, and PDFs of scholarly papers.

And not even that second one, due to the lack of big enough readers. For a long time I owned and enjoyed a Kindle DX, with a screen big enough that a typical magazine page was legible without zooming in. Zooming in on a columnar page is horrific. It’s like watching a tennis match through a keyhole. But the Kindle DX broke, is no longer a thing, and has no competitors. I don’t enjoy reading on regular computer screens, so the option of using a multipurpose tablet is not a good one.

Ebooks also suffer from being out of sight and out of mind. I actually bought some bundle of UX/HCI/design books over a year ago, and have never read them. When I want to read, I look at my pile of unread books and my shelves. I don’t look in ~/Documents/ebooks.

I do listen to audiobooks when I commute, but only when I commute. It’d be nice to have some kind of multimodal reader, across a “printed” and “spoken” format. The Kindle text-to-speech was not that, when I tried it. Jeremy Northam does a much better job of reading out The Road to Wigan Pier than an automated speech synthesiser does.

The technique I’m trying at the moment involves heavy use of the library. I’m a member of both the local municipal library and a big university library. I subscribe to a literary review magazine, the London Review of Books. When an article in there intrigues me, I add the book to the reading list in the library app. When I get to it, I request the book.

That’s not necessarily earth-shattering news. Both public and subscription libraries have existed for centuries. What’s interesting is that for this dedicated reader and technology professional, the digital revolution has yet to usurp the library and its collection of bound books.

#### My discs have been Americanised!
&lt;

p>For some reason, even though l10n and i18n have been fashionable terms in computing for the last few years, no-one seems able to localise properly into the lingua franca of computing, English.  It may surprise some readers to learn that there's more than one dialect of english, and some of these even have their own ISO codes (such as en_GB, en_US and so on...I'm ignoring the &quot;ang&quot; language for now).  Some words in these different dialects are not spelled in the same way.  I live in the United Kingdom of Great Britain and Northern Ireland (Land of hope and glory, mother of the free...) and therefore those round things are known as discs.  Indeed, when I insert my Mac OS X installer disc, it is called "Mac OS X Install Disc 1".  Then I launch the Firmware Password application, which tells me: "The firmware password is used to prevent others from starting your computer with a different disk."  Gah!

#### My first rails app
I know, right? I first learned how to rails back when Rails 3 was new, but didn't end up using it (the backend of the project I was working on was indeed written in Rails, but by other people). Then when I worked at Big Nerd Ranch I picked up bits and pieces of knowledge from the former Highgroove folks, but again didn't use it. The last time I worked on a real web app for real people, it was in node.js (and that was only really vending a React SPA, so it was really in React). The time before that: WebObjects.

The context of this project is that I had a few days to ninja out an end-to-end concept of a web application that's going to be taken on by other members of my team to flesh out, so it had to be quick to write and easy to understand. My thought was that Rails is stable and trusted enough that however I write the app, with roughly no experience, would not diverge far from however anyone else with roughly no experience would do it, so there wouldn't be too many surprises. That the testing story for Rails is solid, that websites in Rails are a well-understood problem.

Obviously I could've chosen any of a plethora of technologies and made my colleagues live with the choice, but that would potentially have sunk the project. Going overly hipster with BCHS, Seaside or Phoenix would have been enjoyable but left my team-mates with a much bigger challenge than "learn another C-like OOP language and the particular conventions of this three-tier framework". Similarly, on the front end, I just wrote some raw JS that's served by Rails's asset pipeline, with no frameworks (though I did use Rails.ajax for async requests).

With a day and a half left, I'm done, and can land some bonus features to reduce the workload for my colleagues. Ruby is a joy to use, although it is starting to show some of the same warts that JS suffers from: compare the two ways to make a Ruby hash with the two ways to write JS functions. The inconsistency over brackets around message sends is annoying, too, but livable.

Weirdly testing in Rails seems to only be good for testing Ruby, not JS/Coffeescript/whatever you shove down the frontend. I ended up using the teaspoon gem to run Javascript tests using Jasmine, but it felt weird having to set all that up myself when Rails goes out of its way to make tests for you in Ruby-land. Yes, Rails is in Ruby. But Rails is a web framework, and JS is a necessary evil on the web.

Most of my other problems came from the incompatibility of Ruby versions (I quickly gave up on rvm and used Docker, writing a small wrapper script to run the CD pipeline and give other devs commands like 'build', 'test', 'run', 'stop', 'migrate') and the changes in Rails API between versions 3-5. A lot of content on blogs[*] and stackoverflow don't specify the version of Rails or Ruby they're talking about, so the recommendations may not work the same way.

[*] I found a lot of Rails blogs that just reiterate examples and usage of API that's already present in the rdoc. I don't know whether this is SEO poisoning, or people not knowing that the official documentation exists, or there being lots of low-quality blogs.

But overall, Railsing was fun and got me quickly to my destination.

#### My name in lights
I've been published.

#### My platform is no platform
I currently use three of the desktop computing platforms (Windows, macOS and GNU/Linux) and one of the mobile computing platforms (Samsung-flavoured Android); I currently get paid to develop software for "the web", an amorphous non-platform that acts in many ways like a vendor platform orthogonal to those just mentioned. This is not because I want to ally myself to any or all of those platforms but because I want to be practically independent of all of them. This is where I try to understand why.

A code of ethics

As a member of the ACM I have committed to act in accordance with the ACM Code of Ethics and Professional Conduct, or to revoke my membership. I believe that a lot of my unwillingness to buy into these platforms (and by extension, as a software professional, my unwillingness to push them on others by building upon them) can be expressed in terms of these codes.

Of course, this is expression within my interpretation of the terms. I don't believe that Google's data harvesting activities are consistent with the ACM Code of Ethics, but a previous ACM president was Vint Cerf, a Google employee. Clearly we disagree on our application of the Code to Google's activity. Neither of us is, without further analysis, [in]correct; ethical decisions will be situated in a cultural and experiential context.

This, in general, is why American technology companies find it hard to expand their operations to Europe; the framework in which their actions will be viewed and judged is different.

App stores and the introduction of dependence

The iOS platform is locked down to most people, in three ways: technologically (a cryptographic system prevents all but specific native software and arbitrary JavaScript from being run); market control (only Apple can add software to the approved list; only paid members of their developer program can propose software for approval) and socially (the norms among developers, both native and web software, follow the "don't make me think" principle in which applications are simplistic and unextensible, as discoverability and ease of use are valued over flexibility or composition).

The same argument applies to Google's Chrome OS and Microsoft's Windows 10 S.

This situation promotes an "Apple knows best" effect in which use of a computer is a passive consumption activity where the only applications available are those deemed fit to publish by the central actor, much like broadcast television.

In a professional community that previously gave us Mindstorms and Smalltalk in the Classroom, it is hard to accept that such a prescriptive model of computing is considered tolerable. Indeed, it seems at odds with an ethical imperative to improve public understanding of computing (section 2.7 in the ACM Code). It also seems to produce a three-tier system, in which those who have (the platform operators) are at the top, those who have means (ISVs who can buy into the approval system) are a rung down - albeit in a feudalistic vassal state that seems akin to coal miners buying their picks from the mine owners - and everyone else is below them. This would not seem consistent with an imperative to be fair (section 1.4 in the Code). Indeed I would go as far as to say that putting most of the people interacting with a software system into subject positions does not contribute to society or human well-being (section 1.1).

The web and "cloud" as protection rackets

If I use a web-based software application, it will probably offer to store all data on its developer's servers (or more accurately on virtual machines run on the developer's behalf by some service provider). It may not, indeed probably won't, offer an alternative. My ongoing use of the application is predicated on my ongoing acceptance of the developer's terms and pricing structure. The collection of individuals and organisations with whom my data is shared are also subject to change at any time, and I either demur or stop using the service.

But because their "service" also includes exclusively providing access, even to me, of what I created using their application, even accessing the things I already created is subject to their licence and my acceptance of their future changes, which can't be known (and will probably be hidden up front, even where they are already being planned). This does not seem honest nor trustworthy (section 1.3 of the Code), nor to provide comprehensive and thorough evaluations of the impacts of the system (section 2.5), nor to credit my intellectual property in creating the things that are ransomed by their service (1.6).

#### My proposal for scaling open source: don't
I've had a number of conversations about what "we" in the "free software community" "need" to do to combat the growth in proprietary, user-hostile and customer-hostile business models like cloud user-generated content hosts, social media platforms, hosted payment platforms, videoconferencing services etc. Questions can often be summarised as "what can we do to get everyone off of Facebook groups", "how do we get businesses to adopt Jitsi Meet instead of Teams" or "how do we convince everyone that Mattermost is better for community chat than Slack".

My answer is "we don't", which is very different from "we do nothing about those things". Scaled software platforms introduce all sorts of problems that are only caused by trying to operate the software at scale, and the reason the big Silicon Valley companies are that big is that they have to spend a load of resources just to tread water because they've made everything so complex for themselves.

This scale problem has two related effects: firstly the companies are hyper-concerned about "growth" because when you've got a billion users, your shareholders want to know where the next hundred million are coming from, not the next twenty. Secondly the companies are overly-focused on lowest common denominator solutions, because Jennifer Miggins from South Shields is a rounding error and anything that's good enough for Scott Zablowski from Los Angeles will have to be good enough for her too, and the millions of people on the flight path between them.

Growth hacking and lowest common denominator experiences are their problems, so we should avoid making them our problems, too. We already have various tools for enabling growth: the freedom to use the software for any purpose being one of the most powerful. We can go the other way and provide deeply-specific experiences that solve a small collection of problems incredibly well for a small number of people. Then those people become super-committed fans because no other thing works as well for them as our thing, and they tell their small number of friends, who can not only use this great thing but have the freedom to study how the program works, and change it so it does their computing as they wish—or to get someone to change it for them. Thus the snowball turns into an avalanche.

Each of these massive corporations with their non-free platforms that we're trying to displace started as a small corporation solving a small problem for a small number of people. Facebook was internal to one university. Apple sold 500 computers to a single reseller. Google was a research project for one supervisor. This is a view of the world that's been heavily skewed by the seemingly ready access to millions of dollars in venture capital for disruptive platforms, but many endeavours don't have access to that capital and many that do don't succeed. It is ludicrous to try and compete on the same terms without the same resources, so throw Marc Andreessen's rulebook away and write a different one.

We get freedom to a billion people a handful at a time. That reddit-killing distributed self-hosted tool you're building probably won't kill reddit, sorry. Design for that one farmer's cooperative in Skåne, and other farmers and other cooperatives will notice. Design for that one town government in Nordrhein-Westfalen, and other towns and other governments will notice. Design for that one biochemistry research group in Brasilia, and other biochemists and other researchers will notice. Make something personal for a dozen people, because that's the one thing those massive vendors will never do and never even understand that they could do.

#### My use of Latin: a glossary
i.e.: I Explain
e.g.: Example Given
et al.: Extremely Tedious Author List
op. cit.: Other Page Cited It Too
ibid.: In Book I Described
etc.: Evermore To Continue
a.m.: Argh! Morning!
p.m.: Past Morning
ca.: Close Approximation
sic.: See Inexcusable Cock-up

#### Naming things
My current host name scheme at home is characters from the film Tron. So I have:

Laptop: flynn (programmer, formerly at Encom, and arcade owner)

Desktop: yori (programmer at Encom)

TV box: dumont (runs the I/O terminal)

Watch: bit (a bit)

Windows computer: dillinger (the evil corporate suit)

#### Nearly the end-of-year review
My first post (Farkers, feel free to replace that with "boobies") of the year 2009 was a review of 2008's blog and look forward to 2009. It's time to do the same for the 2009/2010 blogyear bifecta.

Let's start with the recap.

2009 was a comparatively quiet year for iamleeg, with a total of 45 posts (including this one). Although I gave up on LiveJournal, leading to an amount of "mission creep" in the content of this blog, I think that the vast increase my use of Twitter led to the decline in post frequency here. I've come to use Twitter as a replacement for Usenet, it's much easier to share opinions and discuss things on Twitter where there's more of a balanced conversation and less of iamleeg telling the rest of the world how things should work. The other main contributory factor was that I spend my days writing for a living currently, split between authorship, consultation and the MDN security column. I'm often all written out when it comes to the end of the day.So, the mission creep. 2009 saw this blog become more of a home for ideas long enough to warrant a whole page on the internet, losing its tech focus&mdash;directly as a result of dropping LJ, which is where non-tech ideas used to end up. However, statistics show that the tech theme is still prevalent, with only four of the posts being about music or dancing. Security has become both the major topic as well as the popular choice; the most-read article was Beer Improves Perception of Security.During August and September the focus started to shift towards independent business and contract work, as indeed I made that shift. Self-employment is working well for me, the ability to choose where I focus my effort has let me get a number of things done while still retaining a sense of sanity and a balance with my social life.

So what about next year?

Well, the fact that I have a number of different things to focus on leads to an important choice: I need to either regroup around some specific area or choose to remain a polymath, but either way I need to be more rigorous about defining the boundaries for different tasks. My major project comes to an end early in 2010, and after that it's time to calm down and take a deep look at what happens next. I have a couple of interesting potential clients lined up, and have put onto the back burner my own application which will definitely see more work. I also have some ideas for personal development which I need to prioritise and get cracking on. The only thing preventing me from moving on a number of different projects is convincing myself I have time for them.So the blog will fit in with that time-management strategy; I won't necessarily decide that 9:00-10:14 on a Monday is always blogging time, but will resolve to put aside some time to writing interesting things. One thing I have found is that working on one thing for a whole day means I don't get much of it done, so factoring that into my plans will let me take advantage of it. Half an hour working on a new article at lunchtime could be the stimulus required to get more out of the afternoon. My weapon of choice for organising my work has always been OmniFocus, it's time to be more rigorous about using it. It doesn't actually work well for time allocation, but it does let me see what needs to be done next on the various things I have outstanding.Obviously what becomes the content of this blog depends on what happens after I've shaken down all of those considerations and sorted out what it means to be leeg. Happy new year, and stay tuned to find out what happens.

#### NetBSD running on G5
Hurrah!  NetBSD on the G5 gives us a potentially useful option for G5 servers when Darwin/BSD is inevitably phased out for the PPC...it'd be interesting if COMPAT_DARWIN comes along to the point where NetBSD can actually run WindowServer too.I actually had a FreeBSD 7.0-SNAP up on my G4 tower a few months ago, hoping to help out with the port, but the kernel was so unstable that I couldn't get anything done at all :(.  And I'm not as much of a kernel hacker as  auserland hacker...

#### Netscape won
Back when AOL was a standalone company and Sun Microsystems existed at all, Netscape said that they wanted Windows to be a buggy collection of device drivers that people used to access the web, which would be the real platform.

It took long enough that Netscape no longer exists, but they won. I have three computers that I regularly use:

 	my work Mac has one Mac-only, Mac-native app open during the day[*]. Everything else is on the web, or is cross-platform. It doesn't particularly matter what _technology_ the cross-platform stuff is made out of because the fact that it's cross-platform means the platform is irrelevant, and the technology is just a choice of how the vendors spend their money. I know that quite a bit of it is Electron, wrapped web, or Java.
 	my home Windows PC has some emulators for playing (old) platform-specific games, and otherwise only runs cross-platform apps[*] and accesses the web.
 	my home Linux laptop has the tools I need to write the native application I'm writing as a side project, and everything else is cross-platform or on the web.

[*] I'm ignoring the built-in file browsers, which are forced upon me but I don't use.

#### New book: APPosite Concerns
Back when I published APPropriate Behaviour, I expected it to be the first in a trilogy. Today, the second part, APPosite Concerns, is available.

APPosite Concerns is a compendium of posts from this blog, going back over the current decade. The main topics covered are being a programmer, designing software, thinking about software and computer systems, and software freedom. I hope you enjoy it, find it useful, or maybe even both.

#### New c.l.o-c alternative FAQ
So there used to be an alternative comp.lang.objective-c FAQ, but that never really got going.  I've been working on a new c.l.o-c FAQ listing with Mike Ash from Rogue Amoeba.  Why an alternative FAQ?  The official one is maintained by someone who refuses to take gcc ObjC and the various frameworks for it seriously - in fact anything which isn't his own (buggy, seldom-used) ObjC preprocessor.  We've just been a bit more accepting in what we talk about.

#### New project: a dance a day
Starting tomorrow, A Dance A Day will, as its name suggests, feature a new dance tune every day, mostly taken from the English country dance tradition.

#### New project: the GNUstep developer guide
I discovered by searching the interwebs that a significant number of people who try out GNUstep get stuck at the "I wanted to do Objective-C on my Linux so I installed GNUstep...now what?" stage. There are some tutorials for GNUstep around, but they're not necessarily easy to find, and not necessarily pitched at beginners. Otherwise, you're told to look at the Cocoa documentation, and as Xcode's user interface turned into a combine harvester, Apple moved to Swift, and other changes happened, the relevance of Apple's documentation to GNUstep has been on the wane for years.

Therefore today I'm launching the GNUstep Developer Guide. It's not yet pretty, it's not yet complete, but it is a place to look for GNUstep documentation written for GNUstep programmers. The first guide is up: the introduction to ProjectCenter and GORM.

Let me know if you find it useful!

#### New Swift hardware
The Swift Tower is an artificial nesting structure, installed in Oxford University parks. That or a very blatant sponsorship deal.

#### New website
For reasons which will be made clear in an upcoming post (probably next week, I'm off to the 'sunny' Vaterland soon) I'm in the process of consolidating all of my website activities (except for bloggage) onto &THORN;&aelig;s Ofereode.  This also lets me do funky things like svn and mysql hosting which I'll sort out over time.  I have an email address there but currently am seeing inexplicable bounceage so I won't publicise that yet.  The comp.lang.objective-c alt.FAQ's new home is also there, because the bandwidth at SDF is too small.  When Apple publicised ObjC 2 on their Leopard preview page, my SDF site fell of the intarwebs with people looking to see if I'd written about it.  Announcements about the ObjC FAQ move will be made on the newsgroup in due course.

#### Next CocoaHeads Swindon meet!
So for those of you who didn't manage to enjoy the glories to be found in the town that was the inspiration for one of Legion's more colourful adventures,[] next Monday, the 3rd of August, offers yet another once-in-a-monthtime opportunity! As ever, the location is in (or just outside) the Glue Pot, a strong man's stone's throw from the Swindon train station. This month's meeting is a recap on QTKit, to allow those who weren't there last time due to the reschedule to catch up on integrating QuickTime into their Cocoa apps.[] What am I doing knowing quotes like that? Well, the clue is in the user name. When I was a student my UNIX username was leeg, clearly based on my real name. In short order, I was introduced as &quot;He is Leeg, for he are many&quot; and thus iamleeg.

#### Next CocoaHeads Swindon meeting
1st September (that's this coming Monday), in the Glue Pot, Swindon.  8:00pm start.  Chris Walters will talk about, well, something, and we'll be drinking beer, listening and occasionally chipping in.  See you there!

#### NeXT Luminaries
Here I am, trying to look as important as the bunch of people around me.  Right bunch of luminaries they are too, but I won't spoil the surprise of trying to guess who everyone is.  If you know, comment ahead.  Points for recognising someone's hat without recognising the person themselves will not be awarded.  All quite large, 2016x1512 JPEGs.  Thanks to Ken Tabb for snapping the button.

Piccy 1Piccy 2Piccy 3

#### Next Swindon CocoaHeads meeting
At one time a quiet market town with no greater claim than to break up the journey between Oxford and Bristol, Swindon is now a bustling hub of Mac and iPhone development activity. The coming meeting of CocoaHeads, at the Glue Pot pub near the train station on Monday October 5th, is a focus of the thriving industry.I really believe that this coming meeting will be a great one for those of you who've never been to a CocoaHeads meeting before. We will be having a roundtable discussion on indie software development and running your own micro-ISV. Whether you are a seasoned indie or just contemplating making the jump and what to find out what's what, come along to the meeting. Share your anecdotes or questions with a group of like-minded developers and discover how one person can design, develop and market their applications.You don't need to register beforehand and there's no door charge, just turn up and talk Cocoa. If you do want to discuss anything with other Swindon CocoaHeads, please subscribe to the mailing list.

#### Nice things about ObjC
Title linkies to a post by an F-Script guy (the F-Script guy?  I'm not sure, I don't really follow F-Script development) about nice things he likes about the Objective-C language.  Remembering that he wrote a Smalltalk scripting environment for Cocoa, some of the list is fairly unsurprising, much is made of the dynamic runtime, multiple-level dispatch and so on.  I think the article is mainly bang on, though I do disagree with the author in a few places.  The next paragraph is not one of those places.

Classes are objects.  ++ This is the coolest thing ever about proper object-oriented languages, and one of my strongest arguments for design patterns are not language independent.  Do patterns such as Prototype need to exist in ObjC code, when the Factory Method +new will give you an unconfigured typical instance?

Dynamic typing... Optional static typing.  This is one of those slippery slopes where both edges are sharp enough to give you the rope required to shoot yourself in the foot.  Duck typing (i.e. if an object looks like a duck, and quacks like a duck...) is useful in some cases and damned annoying in others.  To avoid runtime exceptions with duck typing you either have to [i]mentally assert correctness in your code, [ii]perform all the runtime introspection needed to ensure your messages will be handled, or [iii]eschew the duck type completely and downcast to either an instance of a class or a conformant of a protocol (or both; you could do something like GLModelObject &lt;NSCoding&gt; * if you really felt like it).  Another issue with the ObjC implementation of duck typing is that it doesn't always work as you'd think.  When it does work, it's very powerful - when it doesn't, you probably won't find out until runtime, and could be spending a long while working out what happened.

Categories.  No, afraid not.  Nice idea, badly implemented.  The point of categories is to let you decorate a class with additional functionality by adding methods - currently not ivars - in additional code objects, not all of which need be present at launch.  This lets you work around the visibility contract of the class (can't see an @private ivar?  Just chuck an instance method in!), though in fairness so does KVC.  But perhaps the worst crime a category can commit is killing someone else's category.    Or overwriting an "undecorated" method.

I still love Objective-C, mainly because I love Cocoa and GNUstep and making code that works like them, it's definitely powerful and fun too.  But it's not without its rough edges and sharp spiky bits.

#### Nice-looking LaTeX Unicode
Because there was no other single location with all of this written:

usepackage{ucs} % Unicode supportusepackage[utf8x]{inputenc} % UCS' UTF-8 driver is better than the LaTeX kernel'susepackage[T1]{fontenc} % The default font encoding only contains Latin charactersusepackage{ae,aecompl} % Almost European fonts/hyphenation do a better job than Computer Modern

There are a couple of characters I need (Latin letter yogh, Latin letter wynn + capitals) which aren't known by UCS, and I don't yet know how to add them.  But this is a pretty good start.

#### NIMBY Objects
Members of comfortable societies such as English towns have expectations of the services they will receive. They want their rubbish disposed of before it builds up too much, for example. They don't so much care how it's dealt with, they just want to put the rubbish out there and have it taken away. They want electricity supplied to their houses, it doesn't so much matter how as long as the electrons flow out of the sockets and into their devices.

Some people do care about the implementation, in that they want it to be far enough away from it not to have to pay it any mind. These people are known as NIMBYs, after the phrase Not In My Back Yard. Think what it will do to traffic/children walking to school/the skyline/property prices etc. to have this thing I intend to use near my house!

A NIMBY wants to have their rubbish taken away, but does not want to be able to see the landfill or recycling centre during their daily business. A NIMBY wants to use electricity, but does not want to see a power station or wind turbine on their landscape.

What does this have to do with software? Modules in applications (which could be—and often are—objects) should be NIMBYs. They should want to make use of other services, but not care where the work is done except that it's nowhere near them. The specific where I'm talking about is the execution context. The user interface needs information from the data model but doesn't want the information to be retrieved in its context, by which I mean the UI thread. The UI doesn't want to wait while the information is fetched from the model: that's the equivalent of residential traffic being slowed down by the passage of the rubbish truck. Drive the trucks somewhere else, but Not In My Back Yard.

There are two ramifications to this principle of software NIMBYism. Firstly, different work should be done in different places. It doesn't matter whether that's on other threads in the same process, scheduled on work queues, done in different processes or even on different machines, just don't do it anywhere near me. This is for all the usual good reasons we've been breaking work into multiple processes for forty years, but a particularly relevant one right now is that it's easier to make fast-ish processors more numerous than it is to make one processor faster. If you have two unrelated pieces of work to do, you can put them on different cores. Or on different computers on the same network. Or on different computers on different networks. Or maybe on the same core.

The second is that this execution context should never appear in API. Module one doesn't care where module two's code is executed, and vice versa. That means you should never have to pass a thread, an operation queue, process ID or any other identifier of a work context between modules. If an object needs its code to run in a particular context, that object should arrange it.

Why do this? Objects are supposed to be a technique for encapsulation, and we can use that technique to encapsulate execution context in addition to code and data. This has benefits because Threading Is Hard. If a particular task in an application is buggy, and that task is the sole responsibility of a single class, then we know where to look to understand the buggy behaviour. On the other hand, if the task is spread across multiple classes, discovering the problem becomes much more difficult.

NIMBY Objects apply the Single Responsibility Principle to concurrent programming. If you want to understand surprising behaviour in some work, you don't have to ask "where are all the places that schedule work in this context?", or "what other places in this code have been given a reference to this context?" You look at the one class that puts work on that context.

The encapsulation offered by OOP also makes for simple substitution of a class's innards, if nothing outside the class cares about how it works. This has benefits because Threading Is Hard. There have been numerous different approaches to multiprocessing over the years, and different libraries to support the existing ones: whatever you're doing now will be replaced by something else soon.

NIMBY Objects apply the Open-Closed Principle to concurrent programming. You can easily replace your thread with a queue, your IPC with RPC, or your queue with a Disruptor if only one thing is scheduling the work. Replace that one thing. If you pass your multiprocessing innards around your application, then you have multiple things to fix or replace.

There are existing examples of patterns that fit the NIMBY Object description. The Actor model as implemented in Erlang's processes and many other libraries (and for which a crude approximation was described in this very blog) is perhaps the canonical example.

Android's AsyncTask lets you describe the work that needs doing while it worries about where it needs to be done. So does IKBCommandBus, which has been described in this very blog. Android also supports a kind of "get off my lawn" cry to enforce NIMBYism: exceptions are raised for doing (slow) network operations in the UI context.

There are plenty of non-NIMBY APIs out there too, which paint you into particular concurrency corners. Consider -[NSNotificationCenter addObserverForName:object:queue:usingBlock:] and ignore any "write ALL THE BLOCKS" euphoria going through your mind (though this is far from the worst offence in block-based API design). Notification Centers are for decoupling the source and sink of work, so you don't readily know where the notification is coming from. So there's now some unknown number of external actors defecating all over your back yard by adding operations to your queue. Good news: they're all being funnelled through one object. Bad news: it's a global singleton. And you can't reorganise the lawn because the kids are on it: any attempt to use a different parallelism model is going to have to be adapted to accept work from the operation queue.

By combining a couple of time-honoured principles from OOP and applying them to execution contexts we come up with NIMBY Objects, objects that don't care where you do your work as long as you don't bother them with it. In return, they won't bother you with details of where they do their work.

#### No True Humpty-Dumpty
Words change meaning.

Technical words change meaning.

Sometimes, you need to check out a specific commit of a word's meaning from the version control, to add context to a statement.

"I'm talking about Open Source in its early meaning of Free Software without the confusion over Free, not its later meaning as an ethically empty publication of source code."

"I mean Object-Oriented Programming as the loosely-defined bucket in which I can put all the ills of software that I'm claiming are solved by Haskell, not the earlier sense of modelling business processes in software with loosely-coupled active programs communicating by sending messages."

"The word Agile here refers to the later sense of Agile where I run a waterfall with frequent checkpoints and get a certification from a project management institute."

The problem is that doing so acts as a thought-terminating cliche to people who are not open to hearing a potentially valuable statement about Open Source, Object-Oriented Programming, or Agile development.

If your commit is too early in history, then you can easily be dismissed as etymologically fallacious, or as somebody who won't accept progress and the glorious devaluation of the word you're trying to use.

If your commit is too recent in history, then you can easily be dismissed as a Humpty-Dumptyist who's trying to hide behind a highfalutin term that you have no right to use.

What I've come to realise is that my technique for dealing with people who use these rhetorical devices can be as simple as this: ignore them. If they do not want to hear, then I do not need to speak.

#### No worries, mate!
Just got a brace of MacBooks, and with the .pvs linked in an earlier post OPENSTEP/Mach works on both with Parallels.

#### No, you can't ignore politics
I wrote, a couple of years ago, about the fact that you can't ignore ethics in software engineering. Your software is built for a reason, it's used for a reason, you need to be aware of those reasons and whether you're supporting or enabling them.

That goes for politics too. That hacker news declared this week to be politics-free week shows an immaturity and unprofessionalism that makes it a dangerous place to learn about making software from. Making software is the act of some people producing things for other people, it is inherently a political act. Choosing a framework or programming language is political. Attending a meet-up is political. Being paid is oh-so-political. Publishing your side project under an OSI-approved licence? The OSI is a political organisation, the courts that will determine whether the terms of your licence are binding are political, the officials of those courts are selected by a political process.

And, anyway, let's have a look at the front page of hacker news now, roughly five days into their politics-free week.

Why we chose VueJS over React Ooh, influencing technology choices? Sounds political.
German judges explain why adblock plus is illegal the law? Isn't that a politics thing?
One huge reason why your contract agreement shouldn't be "work for hire" Hmm, contracts, sounds like negotiations with employers or customers...that's political in nature.
Blockchain technology and the law are allies, not enemies More law, discussion of alliances...politics maybe?

What they probably mean is not "political stories are off-topic", but that political stories that stray from the default politics are off-topic. Anything that doesn't sound like I agree with it must be subjective, whereas things I agree with are objective. Just as the default narrative in society is often white, male and affluent, so it is those things in the Valley and neolibertarian. Anything else is just politics.

#### Non-standard components
Another day, another exercise from Software: A Technical History…

A software engineering project might include both standard and nonstandard engineering components. Give an example of a software engineering project where this would be appropriate.
Kim W. Tracy, Software: A Technical History (p. 43)

Buy vs. build (or, in the age of free software, acquire vs. build) is perhaps the most important question in any software engineering endeavor. I would go so far as to say that the solution to the software crisis wasn't object-oriented programming, or agile software development, or any other change in the related methods and tools of software—those have largely been fad-driven. It was the confluence of these two seminal events:

The creation of the GNU project by Richard Stallman, which popularized to the Four Freedoms, which led to the Debian Social Contract, which led to the Open Source Definition.

The dot-com crash, which popularized not having money to spend on software licenses or developers, which led to adopting free software components.

This, the creation of de facto standards in software commons, then drove adoption of the LAMP stack on the technology side, and fast-feedback processes including the lightweight methodologies that became known as agile, lean startup, lean software, and so on.

Staffing costs aside, software development can be very inexpensive at the outset, provided that the developers control the scope of their initiative to avoid “boiling the ocean”. Therefore it can be easy and, to some extent, low-impact, to get the buy-vs-build calculus wrong and build things it’d be better to buy. But, as code is a liability, making the wrong choice is cheap today and expensive tomorrow.

One technique that helps to identify whether to use a standard component is a Wardley map, which answers the question “how closely-related is this part of our solution to our value proposition?” If it’s something you need, but not something that’s core to your unique provision, there’s little need for a unique component. If it’s an important part of your differentiation, it probably ought to be different.

Another is Cynefin, which answers the question “what does this problem domain look like”? If it’s an obvious problem, or a complicated problem, the solution is deterministic and you can look to existing examples. If it’s complex or chaotic, you need to be more adaptive, so don’t want to be as constrained by what other people saw.

Bringing this all together into an example: the Global.health project has a goal to provide timely access to epidemiological data to researchers, the press, and the public. “Providing timely access to…” is a well-solved problem, so the project uses standard components there: Linux, HTTPS, hosted databases, event-driven processing. “Epidemiological data” is a complex problem that became chaotic during COVID-19 (and does again with other outbreaks), so the project uses nonstandard components there: its own schemata, custom code, and APIs for researchers to write their own integrations.

#### Non-subscription updates means charged?
&lt;

p>The justification for the iPod Touch upgrade fee (to enable the new apps, which are actually deployed-but-disabled by a free firmware upgrade) is the same as the justification given for the MacBook wireless upgrade fee last year - that adding new features to a product that isn't sold as a subscription service needs to be charged for.  That in itself is odd - it means that the regulators in the States get to set a price (if not the price) for hitherto free products offered by companies.  But it raises a more interesting question - what constitutes a new feature?  If a bugfix renders a previously-unusable feature usable, is that charged for?  If a security fix makes it possible to use a system in a different environment, should that be charged for?

#### Novel bean incoming
You may remember in July I updated the open source Bean word processor to work with then-latest Xcode and macOS. Over the last couple of days I've added iCloud Drive support (obviously only if the app is associated with an App Store ID, but everyone gets the autosave changes), and made sure it works on Big Sur and Apple Silicon.

Alongside this, a few little internal housekeeping changes: there's now a unit test target, the app uses base localisation, and for each file I had to edit, I cleaned up some warnings.

Developers can try this new version out from source. I haven't created a new build yet, because I'm still in the process of removing James Hoover's update-checking code which would replace Bean with the proprietary version from his website. I'll create and host a Sparkle appcast for automatic updates before doing a binary release, which will support macOS 10.6-11.1.

#### NSConference MINI videos available
During WWDC week I talked at NSConference MINI, a one-day conference organised by Scotty and the MDN. The videos are now available: free to attendees, or $50 for all 10 for non-attendees.

My own talk was on extending the Clang static analyser, to perform your own tests on your code. I'm pleased with the amount I managed to get in, and I like how the talk managed to fit well with the general software-engineering theme of the conference. There's stuff on bit-level manipulation, eXtreme Programming, continuous integration, product management and more. I'd fully recommend downloading the whole shebang.

#### NSConference videos
Scotty and the gang have been getting the NSConference videos out to the public lately, and now sessions 7-9 are available including my own session on security. The videos are really high quality, I'm impressed by the postproduction that's gone in and of course each of the sessions I watched at the conference has some great information and has been well-presented. All of the videos are available here.I've also put the slides for my presentation up over on slideshare.

#### NSConference: the aftermath
So, that's that then, the first ever NSConference is over. But what a conference! Every session was informative, edumacational and above all enjoyable, including the final session where (and I hate to crow about this) the "American" team, who had a working and well-constructed Core Data based app, were soundly thrashed by the "European" team who had a nob joke and a flashlight app. Seriously, we finally found a reason for doing an iPhone flashlight! Top banana. I met loads of cool people, got to present with some top Cocoa developers (why Scotty got me in from the second division I'll never know, but I'm very grateful) and really did have a good time talking with everyone and learning new Cocoa skills.It seems that my presentation and my Xcode top tip[] went down really well, so thanks to all the attendees for being a great audience, asking thoughtful and challenging questions and being really supportive. It's been a couple of years since I've spoken to a sizable conference crowd, and I felt like everyone was on my side and wanted the talk - and indeed the whole conference - to be a success.So yes, thanks to Scotty and Tim, Dave and Ben, and to all the speakers and attendees for such a fantastic conference. I'm already looking forward to next year's conference, and slightly saddened by having to come back to the real world over the weekend. I'll annotate my Keynote presentation and upload it when I can.[] Xcode "Run Shell Script" build phases get stored on one line in the project.pbxproj file, with all the line breaks replaced by n. That sucks for version control because any changes by two devs result in a conflict over the whole script. So, have your build phase call an external .sh file where you really keep the shell script. Environment variables will still be available, and now you can work with SCM too :-).

#### Nvidia and ARM
Nvidia’s ambitions are scarcely hidden. Once it owns Arm it will withdraw its licensing agreements from its competitors, notably Intel and Huawei, and after July next year take the rump of Arm to Silicon Valley

This tech giant up for sale is a homegrown miracle – it must be saved for Britain

#### ObjC FAQ update
I spent a lot of today grokking Distributed Objects to write a FAQ entry on it; link above goes to the new entry.  Of course, any comments on it or the rest of the FAQ are welcome...

#### ObjC FAQ update
Added a question about the ObjC 2.0 garbage collector.  Sorry it's been so long in coming!  I'll try and add a few more ObjC 2.0 questions over the coming days.

#### Object-Oriented callback design
One of the early promises of object-oriented programming, encapsulated in the design of the Smalltalk APIs, was a reduction - or really an encapsulation - of the complexity of code. Many programmers believe that the more complex a method or function is, the harder it is to understand and to maintain. Some developers even use tools to measure the complexity quantitatively, in terms of the number of loops or conditions present in the function's logic. Get this "cyclomatic complexity" figure too high, and your build fails. Unfortunately many class APIs have been designed that don't take the complexity of client code into account. Here's an extreme example: the NSStreamDelegate protocol from Foundation.

-(void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)streamEvent;

This is not so much an abstraction of the underlying C functions as a least-effort adaptation into Objective-C. Every return code the lower-level functionality exposes is mapped onto a code that's funnelled into one place; this delegate callback. Were you trying to read or write the stream? Did it succeed or fail? Doesn't matter; you'll get this one callback. Any implementation of this protocol looks like a big bundle of if statements (or, more tersely, a big bundle of cases in a switch) to handle each of the possible codes. The default case has to handle the possibility that future version of the API adds a new event to the list. Whenever I use this API, I drop in the following implementation that "fans out" the different events to different handler methods.

-(void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)streamEvent
{
  switch(streamEvent)
  {
  case NSStreamEventOpenCompleted:
    [self streamDidOpen: stream];
    break;
  //...
  default:
    NSAssert(NO, @"Apple changed the NSStream API");
    [self streamDidSomethingUnexpected: stream];
    break;
  }
}

Of course,

NSStream is an incredibly old class. We've learned a lot since then, so modern callback techniques are much better, aren't they? In my opinion, they did indeed get better for a bit. But then something happened that led to a reduction in the quality of these designs. That thing was the overuse of blocks as callbacks. Here's an example of what I mean, taken from Game Center's authentication workflow.

@property(nonatomic, copy) void(^authenticateHandler)(UIViewController *viewController, NSError *error)

Let's gloss over, for a moment, the fact that simply setting this property triggers authentication. There are three things that could happen as a result of calling this method: two are the related ideas that authentication could succeed or fail (related, but diametrically opposed). The third is that the API needs some user input, so wants the app to present a view controller for data entry. Three things, one entry point. Which do we need to handle on this run? We're not told; we have to ask. This is the antithesis of accepted object-oriented practice. In this particular case, the behaviour required on event-handling is rich enough that a delegate protocol defining multiple methods would be a good way to handle the interaction:

@protocol GKLocalPlayerAuthenticationDelegate

@required
-(void)localPlayer: (GKLocalPlayer *)localPlayer needsToPresentAuthenticationInterface: (UIViewController *)viewController;
-(void)localPlayerAuthenticated: (GKLocalPlayer *)localPlayer;
-(void)localPlayer: (GKLocalPlayer *)localPlayer failedToAuthenticateWithError: (NSError *)error;

@end

The simpler case is where some API task either succeeds or fails. Smalltalk had a pattern for dealing with this which could be both supported in Objective-C, and extended to cover asynchronous design. Here's how you might encapsulate a boolean success state with error handling in an object-oriented fashion.

typedef id(^conditionBlock)(NSError **error);
typedef void(^successHandler)(id result);
typedef void(^failureHandler)(NSError *error);
- (void)ifThis:(conditionBlock)condition then:(successHandler)success otherwise:(failureHandler)failure
{
  __block NSError *error;
  __block id result;
  if ((result = condition(&error)))
    success(result);
  else
    failure(error);
}

Now you're telling client code whether your operation worked, not requiring that it ask. Each of the conditions is explicitly and separately handled. This is a bit different from Smalltalk's condition handling, which works by sending the ifTrue:ifFalse: message to an object that knows which Boolean state it represents. The ifThis:then:otherwise: message needs to deal with the common Cocoa idiom of describing failure via an error object - something a Boolean wouldn't know about.[] However, the Smalltalk pattern *is possible while still supporting the above requirements: see the coda to this post. This method could be exposed directly as API, or it can be used to service conditions inside other methods:

@implementation NSFileManager (BlockDelete)

- (void)deleteFileAtPath:(NSString *)path success:(successHandler)success failure:(failureHandler)failure
{
    [self ifThis: ^(NSError **error){
        return [self removeItemAtPath: path error: error]?@(1):nil;
    } then: success otherwise: failure];
}

@end

int main(int argc, const char * argv[])
{
    @autoreleasepool {
        [[NSFileManager defaultManager] deleteFileAtPath: @"/private/tmp" success: ^(id unused){
            NSLog(@"Holy crap, you deleted the temporary folder!");
        } failure: ^(NSError *error){
            NSLog(@"Meh, that didn't work. Here's why: %@", error);
        }];
    }
    return 0;
}

[*]As an aside, there's no real reason that Cocoa needs to use indirect error pointers. Consider the following API:

-(id)executeFetchRequest:(NSFetchRequest *)fetchRequest

The return value could be an NSArray or an NSError. The problem with this is that in almost all cases this puts some ugly conditional code into the API's client—though only the same ugly condition you currently have to do in testing the return code before examining the error. This separation of success and failure handlers encapsulates that condition in code the client author doesn't need to see.

Coda: related pattern

I realised after writing this post that the Smalltalk-esque ifTrue:ifFalse: style of conditional can be supported, and leads to some interesting possibilities. First, consider defining an abstract Outcome class:

@interface Outcome : NSObject

- (void)ifTrue:(successHandler)success ifFalse: (failureHandler)failure;

@end

You can now define two subclasses which know what outcome they represent and the supporting data:

@interface Success : Outcome

+ (instancetype)successWithResult: (id)result;

@end

@interface Failure : Outcome

+ (instancetype)failureWithError: (NSError *)error;

@end

The implementation of these two classes is very similar, you can infer the behaviour of Failure from the behaviour of Success:

@implementation Success
{
  id _result;
}

+ (instancetype)successWithResult: (id)result
{
  Success *success = [self new];
  success->_result = result;
  return success;
}

- (void)ifTrue:(successHandler)success ifFalse:(failureHandler)failure
{
  success(_result);
}
@end

But that's not the end of it. You could use the -ifThis:then:otherwise: method above to implement a Deferred outcome, which doesn't evaluate its result until someone asks for it. Or you could build a Pending result, which starts the evaluation in the background, resolving to success or failure on completion. Or you could do something else.

#### Object-Oriented Programming in 1714
Here are some excerpts from Leibniz's La Monadologie (specifically from Daniel Garber and Roger Arlew's English translation in Discourse on Metaphysics and Other Essays).

  THE MONAD, which we shall discuss here, is nothing but a simple substance that enters into composites - simple, that is, without parts.
  
  Thus, one can say that monads can only begin or end all at once, that is, they can only begin by creation and end by annihilation, whereas composites begin or end through their parts.
  
  The monads have no windows through which something can enter or leave. [...] Thus, neither substance nor accident can enter a monad from without.
  
  From this we see that if, in our perceptions, we had nothing distinct or, so to speak, in relief and stronger in flavour, we would always be in a stupor. And this is the state of bare monads.
  
  Thus we attribute action to a monad insofar as it has distinct perceptions, and passion, insofar as it has confused perceptions.
  
  But in simple substances the influence of one monad over another can only be ideal, and can only produce its effect through God's intervention, when in the ideas of God a monad rightly demands that God take it into account in regulating the others from the beginning of things. For, since a created monad cannot have an internal physical influence upon another, this is the only way in which one can depend on another.
  
  But natural machines, that is, living bodies, are still machines in their least parts, to infinity.
  
  And the author of nature has been able to practice this divine and infinitely marvellous art, because each portion of matter is not only divisible to infinity, as the ancients have recognized, but is also actually subdivided without end, each part divided into parts having some motion of their own;
  
  This we see that each living body has a dominant entelechy, which in the animal is the soul; but the limbs of this living body are full of other living beings, plants, animals, each of which also has its entelechy, or its dominant soul.

That's a small fraction of the complete essay but it's clear from these extracts that Leibniz saw both the benefit and the drawback to Object-Oriented Programming, and that they were one and the same.

Objects are indivisible atoms, which cannot mess with each other's insides nor have their own insides messed with. Rather, they have independent existences, unless one of them sends a message to another via the universal actor willing the two into some correlation.

Something appears to be an object in the first instance, because it is an indivisible thing with its own whole definition and purpose. It is actually both a composite, that can be subdivided into other objects with their own definitions and purposes; and it is a component that contributes toward a larger object that has its own definition and purpose.

Scale-variant analogies can get us out of the problem with the object-as-monad. One that has previously been well-explored is Brad Cox's Software IC, which sees the object take up a position at only one level of abstraction:

Component => C instruction.
Integrated Circuit => Objective-C Object.
Circuit Board => Package or architectural layer.
Electronic System => Application.

One that I haven't yet investigated is the cellular biology analogy introduced by Alan Kay (who also introduced the object-as-monad analogy). He talked about the object as a cell (particularly as a small component of a larger system that has a well-defined boundary across which limited communication occurs), but we can also think about the implications of larger scale order:

Cell => Object.
Organ => Object.
Organism => Object.
Society => Object.
Population => Object.
Biome => Object.

And, as Leibniz said, it's objects all the way in the other direction: organelles are monads too.

#### Object-Oriented Programming in Functional Programming in Swift
The maths behind functional programming predates computers. Once people had some experience with both of these things, they stripped them down and created object-oriented programming. It's still possible to jettison a lot of the features of functional programming and work with the object-oriented core, and in this post I'll do so using a subset of the Swift programming language.

Step One: Useing you're type's good

I'll use this particular definition of a type: it's a set. I could define a type completely as a set of constants:

  Boolean :: {True, False}

or using constructors:

  Natural :: {Zero, Successor(N : Natural)}

An element drawn from the set is an instance of the type. So True is an instance of Boolean, and Successor(Successor(Zero)) is an instance of Natural.

A particular, um, type of type is the "product" type, which represents a combination of other types. A tuple is product type: a tuple of type (Boolean, Natural) represents an instance of Boolean and an instance of Natural. The set (Boolean, Natural) or Boolean x Natural is the set of all possible combinations of Boolean and Natural.

Crucially, for the following argument, a struct is also a product type. Has been since before C was created, will be forever.

A function is a map from an input type to an output type. Really there's more nuance in the world than that, particularly around whether the function works for all instances of the input type and whether there's a one-to-one mapping between input and output instances, but that's not relevant right now. All we need to know is that given an instance of the input type, applying a function to it will always result in a particular instance of the output type.

  f :: InputType -&gt; OutputType

You may think that you've seen functions with more than one input argument: that's the same (roughly) as a function that takes a single tuple argument.

Step Two: Know what an object is

Defining instance variables as the values "known" inside an object, and methods as functions attached to an object that have access to its instance variables, I could say this:

  An object is a collection of instance variables and methods that act on those instance variables.

But I want to put this a different way.

  Given a collection of values, an object gives you a collection of methods that yield particular results based on those values.

But I want to put this a different way.

  Given the input of a product type drawn from possible instance variables, an object will return a particular member of the product type drawn from possible methods.

But I want to put this a different way.

  An object is a function that maps from instance variables to methods.

Step Three: Build an object

As the output of an object is a collection of methods, you can define an object entirely by its methods. In object-oriented programming, this is called "data hiding" or "encapsulation" and means that you can't see what went into making the objects work this way, only that they do.

Starting small, think about the set data type. You only ever want to do one thing with a set: discover whether a particular element is contained in the set. That means that, for example, the type of a set of integers is equivalent to this type of function (in Swift syntax):

typealias Set = (Int) -&gt; Bool

Now it's possible to build trivial sets. Here's one that contains no integers, and one that contains all the integers:

let emptySet : Set = { (_) in return false }
let universe : Set = { (_) in return true }

It's easy to use the one method that these objects define: just call the objects as functions.

emptySet(12) //false
universe(12) //true

Step Four: Add some instance variables

I want to keep my Set method signature, because I want the objects I create to all be compatible (I really want them to all be of the same class). But I also want to be able to store and use instance variables from within the Set's method, so I can build some more interesting sets.

I'll build a Set constructor, a function that returns properly-configured instances of Set. That constructor can close over the instance variables, therefore making them available from within the method. Here's a constructor for Sets that represent contiguous ranges of Ints.

func RangeSet(lowerBound:Int, upperBound:Int)-&gt;Set
{
  return { (x) in (x &gt;= lowerBound) &amp;&amp; (x &lt;= upperBound) }
}

Now it's possible to create these Sets and use their method:

RangeSet(3, 7)(2) //false

let mySet = RangeSet(0, 2) //(Function)
mySet(1) //true
mySet(3) //false

Of course, the instance variables don't have to be Ints. They could be anything else, including other Sets.

func UnionSet(setOne:Set, setTwo:Set) -&gt; Set
{
  return { (x) in (setOne(x) || setTwo(x)) }
}

UnionSet(RangeSet(0, 2), RangeSet(3, 5))(4) //true

func IntersectSet(setOne:Set, setTwo:Set) -&gt; Set
{
  return { (x) in (setOne(x) &amp;&amp; setTwo(x)) }
}

IntersectSet(RangeSet(0, 5), RangeSet(3, 7))(5) //true

Even when a Set has another Set as an instance variable, it cannot see how that Set was constructed or what instance variables it has. It can only see the Set's method, because the encapsulation is working.

Step Five: Add more methods

Not all things that we might want to use as objects can be expressed as one method. A list has two methods: a count of the number of items it holds and a method to get the item at a given index. A list object therefore needs to represent the product of (count method, at method) and to let clients select which method they want to use from that product. This product could be expressed as a tuple:

typealias ListSelectors = (count:() -&gt; Int, at:Int-&gt;AnyObject?)

Or (perhaps more usefully because the compiler for Swift allows generics in this case) as a struct:

struct List&lt;T&gt; {
  let count: () -&gt; Int
  let at: (Int) -&gt; T?
}

Now it's possible to rely on the old trick of making constructor functions that return Lists:

func EmptyList&lt;T&gt;() -&gt; List&lt;T&gt;
{
  return List(count: {return 0}, at: {(_) in return nil})
}

including the use of instance variables captured by closing over them:

func LinkedList&lt;T&gt;(head:T, tail:List&lt;T&gt;) -&gt; List&lt;T&gt;
{
  return List(count: {return 1 + tail.count()},
    at: { index in return (index == 0) ? head : tail.at(index - 1)})
}

Step Six: Inherit from objects

The List is cool, but it'd be nice to be able to describe lists. I could create a function:

func describeList(list:List&lt;SomeType&gt;) -&gt; String

but wouldn't it be better to add description as a method on Lists? That would make the List type look like this:

struct ListWithDescription&lt;T&gt; {
  let count: () -&gt; Int
  let at: (Int) -&gt; T?
  let describe: () -&gt; String
}

OK, but I don't want to go back to all of my previous List implementations and add describe methods to them. What I really want to do is to add this method as an extension to Lists. I'll define a type that includes a describe method and optionally implementations of the other methods too.

struct ListWithDescriptionSelectors&lt;T&gt; {
  let count: (() -&gt; Int)?
  let at: ((Int) -&gt; T?)?
  let describe: () -&gt; String
}

Given a List and some ListWithDescriptionSelectors, it's possible to build a ListWithDescription that knows how to describe itself and either inherits count and at from its List or overrides them itself.

func SubtypeListByAddingDescription&lt;T&gt;(prototype: List&lt;T&gt;,
  overrides: ListWithDescriptionSelectors&lt;T&gt;) -&gt;
  ListWithDescription&lt;T&gt;
{
  let countImplementation : () -&gt; Int
  if let count = overrides.count {
    countImplementation = count
  } else {
    countImplementation = prototype.count
  }
  let atImplementation : (Int)-&gt;T?
  if let at = overrides.at {
    atImplementation = at
  } else {
    atImplementation = prototype.at
  }
  return ListWithDescription&lt;T&gt;(count: countImplementation, at:     atImplementation, describe: overrides.describe)
}

(The slight complication with all the if lets is because the Swift compiler at time of writing wasn't happy with using the ?? operator in their place.)

It's now possible to put this into practice. In order to work with the elements in a List I need to know more about what type of thing they are, so here's a specialisation of ListWithDescription that can describe a List of Strings. As ever, it has no special access to the instance variables of the List it extends and can only work with it through the published methods.

func ListOfStringsWithDescription(strings: List&lt;String&gt;) -&gt;
  ListWithDescription&lt;String&gt;
{
  let describe: () -&gt; String = {
    var output = ""
    for i in 0..&lt;strings.count() {
      output = output.stringByAppendingString(strings.at(i)!)
      output = output.stringByAppendingString(" ")
    }
    return
      output.stringByTrimmingCharactersInSet(
        NSCharacterSet.whitespaceCharacterSet()
      )
  }
  return SubtypeListByAddingDescription(strings,
    ListWithDescriptionSelectors&lt;String&gt;(count: nil,
      at: nil,
      describe: describe))
}

let awesomeGreeting =
  ListOfStringsWithDescription(LinkedList("Hello,",
    LinkedList("World",
      EmptyList())))

awesomeGreeting.at(1) //"World"
awesomeGreeting.describe() //"Hello, World"

Next Step: Draw some conclusions

Object-Oriented Programming is a simple, easy to use subset of Functional Programming.

Open Step: Cite references

The objects in this article (like the objects in Microsoft COM or my earlier BlockObject) are Procedural Data Types, which I discovered in Object-Oriented Programming Versus Abstract Data Types and User-defined types and procedural data structures as complementary approaches to data abstraction. The idea of objects as closures first appeared, to my knowledge, in Objects as Closures: Abstract Semantics of Object Oriented Languages. The last two of these articles were brought to my attention by my colleague Peter O'Hearn after a discussion on the topic of functional/O-O duality. I am indebted to him for his help in drawing this article out of me.

After Step: Finish this idea

This journey, as we say at Facebook, is 1% complete. There's plenty more to explore here, and I'm not sure the explanation is as clear as it could be. It is, however, written down, which is a start.

#### Object-Oriented Programming in Objective-C
UIKonf 1995 Keynote : Object-Oriented Programming in Objective-C

Introduction

Welcome to the keynote for UIKonf 1995. I'm really excited for what 1995 will bring. Customers are upgrading to last year's OpenStep release, which means that we get to use the new APIs and the best platform around. And really, there are no competitors. OS/2 Warp doesn't seem to be getting any more traction than previous versions, and indeed Microsoft seems to be competing against its own products with Windows. Their biggest release this year, the delayed Windows 93, looks like being a warmed-over version of MS-DOS and Windows 3, which certainly can't be compared with a full Unix system like OpenStep. 1995 will go down in history as the year of NeXT on the desktop.

I want to talk about a crisis in software design, and that is the object-oriented crisis. Well, I suppose it isn't really, it's the procedural crisis again. But now we pretend that our procedural code is object-oriented, and Objective-C is the weapon that enacts this travesty.

What's the main benefit of Objective-C over Smalltalk? It's C. Rather than attempt to graft some foreign function interface onto Smalltalk and make us write adaptors for all of our C code, Brad Cox had the insight that he could write a simple message-sending library and a syntax preprocessor on the C language, and let us add all of the object-oriented programming we'd get from Smalltalk on top of our existing C code.

What's the main drawback of Objective-C over Smalltalk? It's C. Rather than being able to rely on the object-oriented properties of programs to help us understand them, we can just write a load of C code that we wrap up in methods and call it "object-oriented". This is the source of our crisis. In 1992, Brad Cox claimed that Object-Oriented Programming (or Message/Object Programming as he also called it) was "the silver bullet" that Fred Brooks claimed didn't exist. That it would help us componentise software into isolated units that can be plugged together, like integrated circuits bought from a catalogue and assembled into a useful product on a circuit board.

This idea of encapsulated software components is not new, and was presented at the NATO conference on software engineering in 1968. At that conference, Doug McIlroy presented an invited paper called "Mass-Produced Software Components" in which he suggested that the software industry would not be "industrialised" until it supported a subindustry of comprehensible, interchangeable, well-specified components. Here's the relevant quote, the gendered pronouns are from the original.

  The most important characteristic of a software components industry is that it will offer families of routines for any given job. No user of a particular member of a family should pay a penalty, in unwanted generality, for the fact that he is employing a standard model routine. In other words, the purchaser of a component from a family will choose one tailored to his exact needs. He will consult a catalogue offering routines in varying degrees of precision, robust- ness, time-space performance, and generality. He will be confident that each routine in the family is of high quality — reliable and efficient. He will expect the routine to be intelligible, doubtless expressed in a higher level language appropriate to the purpose of the component, though not necessarily instantly compilable in any processor he has for his machine. He will expect families of routines to be constructed on rational principles so that families fit together as building blocks. In short, he should be able safely to regard components as black boxes.

Is Object-Oriented Programming actually that silver bullet? Does it give us the black-box component catalogue McIlroy hoped for? Most of us will never know, because most of us aren't writing Object-Oriented software.

I think this gulf between Object-Oriented design and principles as described during its first wave by people like Alan Kay, Brad Cox, and Bertrand Meyer is only going to broaden, as we dilute the ideas so that we can carry on programming in C. When I heard that a team inside Sun had hired a load of NeXT programmers and were working on a post-Objective-C OO programming environment, I was excited. We know about the problems in Objective-C, the difficulties with working with both objects and primitive types, and the complexity of allowing arbitrary procedural code in methods. When the beta of Oak came out this year I rushed to request a copy to try.

Java, as they've now renamed it, is just an easier Objective-C. It has garbage collection, which is very welcome, but otherwise is no more OO than our existing tools. You can still write C functions (called static methods). You can still write long, imperative procedures. There are still primitive types, and Java has a different set of problems associated with matching those up to its object types. Crucially, Java makes it a lot harder to do any higher-order messaging, which means that if it becomes adopted we'll see a lot more C and a lot less OO.

The meat

I thought I'd write an object-oriented program in modern, OPENSTEP Objective-C, just to see whether it can even be done. I won't show you all the code, but I will show you the highlights where the application looks strongly OO, and the depths where it looks a lot like C. I don't imagine that you'll instantly go away and rewrite all of your code, indeed many of you won't like my style because it's so different from usual Objective-C. My hope is that it gives you a taste of what OOP can be about, inspires you to reflect on your own approach, and encourages you to take more from other ideas about programming than some sugar that superficially wraps C.

Despite a suggestion from Marvin, the paranoid android, I'm going to talk to you about Life.

Always be returning

Every method in this application returns a value, preferably an object, except where that isn't allowed due to assumptions made by the OpenStep frameworks. Where the returned object is of the same type, the client code should use the returned object, even if the implementation is to return self. Even things that look like mutators follow this pattern: with the effect that it doesn't matter to a client whether an object is mutable or immutable because it'll use it the same way. Compare a mutable object:

-setFoo:aFoo
{
  foo = aFoo;
  return self;
}

with an immutable one:

-setFoo:aFoo
{
  return [self copyWithReplacementFoo:aFoo];
}

and the client doesn't know which it's using:

[[myThing setFoo:aDifferentFoo] doSomeStuff];

That means it's easy to change the implementation between these two choices, without affecting the client code (which doesn't need to assume it can keep messaging the same object). In Life, I built a mutable Grid class, but observed that it could be swapped for an immutable one. I didn't need to change the app or the tests to make it happen.

This obviously has its limitations, but within those limitations is a great approach. If multiple objects in a system share a collaborator, then you need to choose whether that collaborator is mutable (in which case all correlated objects see all updates, even those made elsewhere in the system) or immutable (in which case collaborating objects will only see their own updates, unless some extra synchronisation is in place). Using the same interface for both lets you defer this decision, or even try both to understand the implications.

Always be messaging

In this Life application there are no C control statements: no if, no for, no while. Everything is done by sending messages to objects. Two examples are relevant, and both come from the main logic of the "game". If you've seen the rules of Life, you've probably seen them expressed in a form like this:

  If a cell is dead and has X neighbours, it becomes living otherwise it remains dead. If a cell is living and has Y neighbours, it remains living otherwise it becomes dead.

It seems that there are three if statements to be written: one to find out whether a cell is living or dead, and another in each branch to decide what the next state should be.

The first can be dealt with using the class hierarchy. A class (at some theoretical level what I mean here is "a constructor", but that term is not really used in Objective-C so I'm going to say "class" which is the closest term) can often be a substitute for an if statement, by replacing the condition with polymorphism.

In this case, the question "is a cell living or dead" can be answered by sending a message to either a DeadCell or a LivingCell. The question disappears, because it was sent to an object that pre-emptively knew the answer.

@interface LivingCell : Cell
@end

@interface DeadCell : Cell
@end

Now each can answer the question "what is my next state?" without needing to test what its current state is. But how to do that without that other if statement? There's a finite number of possible outcomes, keyed on an integer (the number of living neighbours the cell has), which means it's a question that can be answered by sending a message to an array. Here's how the Cell does it.

-tickOnGrid:grid atX:(NSInteger)x y:(NSInteger)y;
{
  return [[self potentialStates]
         objectAtIndex:
         [self neighboursOnGrid:grid atX:x y:y]];
}

Each of the two subclasses of Cell knows what its potential states are:

static id nextStatesFromLiving;

+(void)load
{
  nextStatesFromLiving = [[NSArray alloc] initWithObjects:
                        deadCell,
                      deadCell,
                      livingCell,
                      livingCell,
                      deadCell,
                      deadCell,
                      deadCell,
                      deadCell,
                      deadCell,
                      nil];
}

-potentialStates { return nextStatesFromLiving; }

Why write the program like this? To make it easier to understand. There's an idea in the procedural programming world that the cyclomatic complexity - the number of conditions and loops in a function - should be minimised. If all a method is doing is messaging other objects, the complexity is minimal: there are no conditional paths, and no loops.

My second example is how a loop gets removed from a method: with recursion, and appropriate choice of target object. One of the big bones of contention early on in NeXT's use of Objective-C was changing the behaviour of nil. In Smalltalk, this happens:

nil open.
MessageNotUnderstood: receiver of "open" is nil

In Objective-C, nothing happens (by default, anyway). But this turns out to be useful. Just as the class of an object can stand in for conditional statements, the nil-ness of an object can stand in for loop termination. A method does its thing, and sends itself to the receiver of the next iteration. When that receiver becomes nil, the message is swallowed and the loop stops.

All loops in Life that are based on doing something a certain number of times are based on a category method on NSNumber.

@implementation NSNumber (times)

-times:target perform:(SEL)action
{
  return [self times:target perform:action withObject:nil];
}

-times:target perform:(SEL)action withObject:object
{
  return [[self nonZero] realTimes:target
               perform:action
            withObject:object];
}

-realTimes:target perform:(SEL)action withObject:object
{
  [target performSelector:action withObject:object];
  return [[NSNumber numberWithInteger:[self integerValue] - 1]
       times:target perform:action withObject:object];
}

-nonZero
{
  return ([self integerValue] != 0)?self:nil;
}

@end

Notice that the conditional expression doesn't violate the "no if statements" rule, because it's an expression not a statement. There's one thing that happens, that yields one of two values. The academic functional programming community recently rallied around the Haskell language, which provides the same distinction: no conditional statements, easy conditional expressions.

Never be sequencing

Related to the idea of keeping methods straightforward to understand is ensuring that they don't grow too long. Ideally a method is one line long: it returns the result of messaging another object. Then there are a couple of slightly larger cases:

a fluent paragraph, in which the result of one message is the receiver for another message. If this gets too deep, though, your method probably has too many coupled concerns associated with a Law of Demeter violation.
something that would be a fluent paragraph, but you've introduced a local variable to make it easier to read (or debug; Objective-C debuggers are not good with nested messages).
a method sends a message, then returns another more relevant object. Common examples replace some collection object then return self.

Finally, there are times when you just can't do this because you depend on some API that won't let you. A common case when working with AppKit is methods that return void; you're forced to sequence them because you can't compose them. The worst offender in Life is the drawing code which uses AppKit's supposedly object-oriented API, but which is really just a sequence of procedures wrapped in message-sending sugar (that internally use some hidden context that was set up on the way into the drawRect: method).

  float beginningHorizontal = (float)x/(float)n;
  float beginningVertical = (float)y/(float)n;
  float horizontalExtent = 1.0/(float)n;
  float verticalExtent = 1.0/(float)n;
  NSSize boundsSize = [view bounds].size;
  NSRect cellRectangle = 
    NSMakeRect(
           beginningHorizontal * boundsSize.width,
           beginningVertical * boundsSize.height,
           horizontalExtent * boundsSize.width,
           verticalExtent * boundsSize.height
           );
  NSBezierPath *path =  [NSBezierPath bezierPathWithRect:cellRectangle];
  [[NSColor colorWithCalibratedWhite:[denizen population] alpha:1.0] set];
  [path stroke];
  [[NSColor colorWithCalibratedWhite:(1.0 - [denizen population])   alpha:1.0] set];
  [path fill];

The underlying principle here is "don't make a load of different assignments in one place", whether those are explicit with the = operator, or implicitly hidden behind message-sends. And definitely, as suggested by Bertrand Meyer, don't combine assignment with answering questions.

Higher-order messages

You've already seen that Life's loop implementation uses the target-action pattern common to AppKit views, and so do the views in the app. This is a great way to write an algorithm once, but leave it open to configuration for use in new situations.

It's also a useful tool for reducing boilerplate code and breaking up complex conditional statements: if you can't represent each condition by a different object, represent them each by a different selector. An example of that is the menu item validation in Life, which is all implemented on the App delegate.

-(BOOL)validateMenuItem:menuItem
{
  id action = NSStringFromSelector([menuItem action]);
  SEL validateSelector = NSSelectorFromString([@"validate"
                          stringByAppendingString:action]);
  return [[self performSelector:validateSelector withObject:menuItem]
       boolValue];
}

-validaterunTimer:menuItem
{
  return @(timer == nil);
}

-validatestopTimer:menuItem
{
  return @(timer != nil);
}

-validatetick:menuItem
{
  return @(timer == nil);
}

-validatequit:menuItem
{
  return @(YES);
}

We have four simple methods that document what menu item they're validating, and one pretty simple method (a literate paragraph that's been expanded with local variables) for choosing one of those to run.

There's no message-forwarding needed in the Life app, and indeed you can do some good higher-order messaging without ever needing to use it. You could imagine a catch-all for unhandled -validate&lt;menuitem&gt;: messages in the above code, which might be useful.

Objective-C's type checks are wrong for Objective-C

Notice that all of the examples I've shown here have used the NeXTSTEP &lt;=3.3 style of method declaration, with (implicit) id return and parameter types. That's not because I'm opposed to type systems, although I am opposed to using one that introduces problems instead of solving them.

What do I want to do with objects? I want to send messages to receivers. Therefore, if I have a method that sends a message to an object, what I care about is whether the recipient can handle that message (I don't even care whether the handling is organised upfront, or at runtime). The OPENSTEP compiler's type checks let me know whether an object is of a particular class, or conforms to a particular protocol, but both of these are more restrictive than the tests that I actually need. Putting the types in leads to warnings in correct code more than it leads to detection of incorrect code, so I leave them out.

Of course, as with Objective-C's successes, Objective-C's mistakes also have predecessors in the Smalltalk world. As of last year (1994) the Self language used runtime type-feedback to inline message-sending operations (something that Objective-C isn't doing...yet, though I note that in Java all "message sends" are inline function calls instead) and there's an ongoing project to rewrite the "Blue Book" library to allow the same in a Smalltalk environment.

What is really needed is a form of "structural type system", where the shape of a parameter is considered rather than the name of the parameter. I don't know of any currently-available object-oriented languages that have something similar to this, but I've heard INRIA are working on an OO variant of Caml that has structural typing.

But what about performance?

It's important to separate object-oriented design, which is what I've advocated here, from object-oriented implementation, which is the code I ended up with as a result. What the design gave me was:

freedom to change implementations of various algorithms (there are about twice as many commits to production code in Life as there are to test code)
hyper-cohesive and decoupled objects or categories
very short and easy to understand methods

The implementation ends up in a particular corner of the phase space:

greater stack depths
lots of method resolutions
lots of objects
lots of object replacements

but if any of this becomes a problem, it should be uninvasive to replace an object or a cluster of objects with a different implementation because of the design choices made. This is where Objective-C's C becomes useful: you can rip out all of that weird nested message stuff and just write a for loop if that's better. Just because you shouldn't start there, doesn't mean you can't end up there.

An example of this in Life was that pretty early in development, I realised the Cell class didn't actually need any state. A Grid could tell a Cell what the surrounding population was, and the Cell was just the algorithm that encapsulated the game's rules. That meant I could turn a Grid of individual Cells into a Flyweight pattern, so there are only two instances of Cell in the whole app.

Conclusions

I'm not so much worried for OOP itself, which is just one way of thinking about programs without turning them into imperative procedures. I'm worried for our ability to continue writing quality software.

Compare my list of OO properties with a list of functional programming properties, and you'll see the same features: immutable values, composed functions, higher-order functions, effectless reasoning over effectful sequencing. The crisis of 1995 shows that we took our existing procedural code, observed these benefits of OOP, then carried on writing procedural code while calling it OOP. That, unsurprisingly, isn't working, and I wouldn't be surprised if people blame it on OOP with its mutable values, side-effecting functions, and imperative code.

But imagine what'll happen when we've all had enough of OOP. We'll look to the benefits some other paradigm, maybe those of functional programming (which are, after all, the same as the benefits of OOP), and declare how it's the silver bullet just as Brad Cox did. Then we'll add procedural features to functional programming languages, or the other way around, to reduce the "learning curve" and make the new paradigm accessible. Then we'll all carry on writing C programs, calling them the new paradigm. Then that won't work, and we'll declare functional programming (or whatever comes next) as much of a failure as object-oriented programming. As we will with whatever comes after that.

Remember, there is no silver bullet. There is just complexity to be governed.

#### Objective-C Design Patterns
Certain events at work have turned me into a bit of a design patterns geek of late, and as such I stumbled across this DDJ article from 1997 (the title of this post is the link).  According to del.icio.us not many other people have stumbled across it, but it's a great article.  The code listing links seem to 404 even though the listings are at the bottom of page 1 of the article.

Something very important can be learned from this article, which is at best covered tangentially in the GoF book: Design patterns are not language-independent.  Calling the C++ and Objective-C ways of writing code &quot;pattern idioms&quot; ignores the point that actually, the code you come up with is more important than the design (customers don't typically want to pay the same cash for your UML diagrams that they do for your executables), and gasp different languages require different code!  Different patterns can be used in ObjC and Smalltalk than in C++, and different patterns again in object-oriented Perl.  Different patterns will be appropriate for working with Foundation than with the NeXTSTEP (pre 4.0) foundation kit or with ICPak101.  Designing your solution independent of the language and framework you're going to use will get you a solution, but will not necessarily produce the easiest solution to maintain, the most efficient solution or one that makes any sense to an expert in the realm of that language and framework.

#### Objective-C literals and subscripts
If you're using clang from their website instead of sticking with Apple's release, you get support for Objective-C literals and object subscripting. I thought I'd take the BrowseOverflow app and apply this new syntax to it. Notice that the code below doesn't match what's in github, which still works with currently-released versions of Xcode and their compilers.

Indexed/Keyed subscripting: we've seen this before

Using the syntax described above, you can subscript into an object using something that looks like the traditional C square bracket notation. If you use an integer, you get indexed subscripting:

        Answer *thisAnswer = question.answers[indexPath.row];

If you use an object, you get keyed subscripting:

            NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithCapacity: 1];
            if (localError != nil) {
                userInfo[NSUnderlyingErrorKey] = localError;
            }

This is something that's been available in many languages before. Smalltalk (from which Objective-C derives) had a well-defined subscripting syntax across all objects, using at: and set:at:. C++ permits classes to supply the operator[]() method to use C-style index subscripting just as Objective-C does.

An aside in defence of operator overloading

If you look at traditional Objective-C syntax, you can see that (roughly speaking, and anyone who points out the edge case isn't my friend) there are the things in square brackets that are objects and messages, and the things without square brackets are primitive types. There are two different worlds, and never the twain shall meet.

But actually, we've learned that encapsulation is good, and that allowing people to concisely express their intent is better than making them deal with our implementation details. Therefore we want to integrate our data types in the language. We want to tell people "add this thing to the other thing", not "you need to call this function with these parameters which will add the things".

Providing custom implementations of the standard operators is the best way of doing that. Yes, it hides what's happening: that's the point. Yes, it can be abused: the entire software industry is based on a foundation that makes it possible to write bad software. If you want to take away tools that can be used to introduce bugs, you need to take away everyone's compilers and interpreters.

An aside on the aside about Objective-C operator overloading

So far, Objective-C objects can provide custom implementations of two C operators: the field access operator . used for type safe property access, and the subscript operator [] used as we saw before I digressed.

The reason subscript overloading works in ObjC is that it's illegal to apply arithmetic operations to object pointers. In C, foo[bar] is just a fancy way of writing *(foo+bar) (which is why bar[foo] also works). If you're not allowed to apply the [] operator to an id, then you know something else must be happening: i.e. you know that the object subscript behaviour is required.

Well the fact that you can't do pointer arithmetic on an id means that you could also, for example, check for illegal use of the + operator and call -objectByAddingObject:.

Back to the point: boxing done well.

Many languages make some attempt to "box" primitive types like numbers in high-level value types like number objects. This often causes problems: for example in Java, which has both automatic boxing and method overloading, I could do this:

public void foo(int x) { … }
public void foo(Java.lang.Integer x) {…}

foo(3);

It's not clear whether 3 refers to the primitive type or to the object type, and therefore it's not clear which method will get called.

The same problem could have been encountered in ObjC: does foo[3] refer to indexed subscripting or to keyed subscripting using an NSNumber instance?

Thankfully whoever designed the Objective-C boxing behaviour decided it must always be explicit. You can get a number like this:

- (void)browseOverflowViewControllerTests_viewDidAppear: (BOOL)animated {
    NSNumber *parameter = @(animated);
    objc_setAssociatedObject(self, viewDidAppearKey, parameter, OBJC_ASSOCIATION_RETAIN);
}

but otherwise numbers will always be treated as numbers, not as objects.

When it all gets a bit much

You've got two seconds, the house is on fire, what does this line do?

    NSString *questionBody = [parsedObject[@"questions"] lastObject][@"body"];

It can be a bit hard to read that, and to pick out which brackets go with messages and which go with subscripts. I can imagine the sort of people who like to issue pronouncements on whether to use the dot operator to access properties will love the new subscripting syntax.

#### Objective-C NAQs
Never-Asked Questions :-)In Code Complete 2 §6.5, Steve McConnell presents a list of class-related design issues that &quot;vary significantly depending on the language&quot;.  So why don't we look at them for Objective-C?  Especially as I can't find anyone else who's done so based on a couple of Google searches… N.B. as ever, I'm really considering Cocoa + Objective-C here, as the language itself doesn't provide enough policy for many of the issues to have an answer (only method resolution and the existence of isa are defined by the runtime and the compiler - ignoring details like static string instances, @protocol and so on).Behaviour of overridden constructors and destructors in an inheritance tree.  Constructors are simple.  If you can initialise an object, return it.  If you can't, return nil.  The -[NSObject init] method is documented as simply returning self.For destructors, we have to split the discussion into twain; half for garbage collection and half for not.  In the world of the Apple GC, destruction is best done automatically, but if you need to do any explicit cleanup then your object implements -finalize.  The two rules are that you don't know whether other objects have already been finalized or not, and that you must not resurrect your object.  In the world of non-GC, an object is sent -dealloc when it has been released enough times not to stick around.  The object should then free any memory claimed during its lifetime, and finally call [super dealloc].  Note that this means objects which use the voodoo cohesion of "OK, I'll clean up all my resources in -dealloc" probably aren't going to work too well if the garbage collection switch is flipped.Behaviour of constructors and destructors under exception-handling conditions.  Exceptions are a rarity in Cocoa code (we didn't have language-level exceptions for the first decade, and then there was a feeling that they're too expensive; C++ programmers like the Leopard 64-bit runtime because ObjC exceptions and C++ exceptions are compatible but that's not the case in the NeXT runtime) but obviously a constructor is an atomic operation.  You can either initialise an object or you can't; there's no half-object.  Therefore if an -init… method catches, the thing to do is unwind the initialisation and return nil.  The only thing I can think to say for having exceptions around destruction time is "don't", but maybe a commenter will have better ideas.Importance of default constructors.  All classes implement -init, except those that don't ;-).  Usually if there's a parameterised initialiser (-initWithFoo:) then that will be the designated initialiser, and -init will just return [self initWithFoo: someReasonableDefaultValueForFoo];.  If there's no reasonable default, then there's no -init.  And sometimes you just shouldn't use -init, for instance NSNull or NSWorkspace.Time at which a destructor or finalizer is called.  See How does reference counting work? for a discussion of when objects get dealloced.  In the garbage-collected world, an object will be finalized at some time after it is no longer reachable in the object graph.  When that is exactly really depends on the implementation of the collector and shouldn't be relied on.  Temporal cohesion ("B happens after A, therefore A has been done before B") is usually bad.Wisdom of overriding the language's built-in operators, including assignment and equality.  This is such a brain-damaged idea that the language doesn't even let you do it.How memory is handled as objects are created and destroyed or as they are declared and go out of scope.  How does reference counting work?  How do I start using the garbage collector?

#### Objective-C, dependencies, linking
In the most recent episode of Edge Cases, Wolf and Andrew discuss dependency management, specifically as it pertains to Objective-C applications that import libraries using the Cocoapods tool.

In one app I worked on a few years ago, two different libraries each tried to include (as part of the libraries themselves, not as dependencies) the Reachability classes from Apple's sample code. The result was duplicate symbol definitions, because my executable was trying to link both (identical) definitions of the classes. Removing one of the source files from the build fixed it, but how could we avoid getting into that situation in the first place?

One way explored in the podcast is to namespace the classes. So Mister Framework could rename their Reachability to MRFReachability, Framework-O-Tron could rename theirs to FOTReachability. Now we have exactly the same code included twice, under two different names. They don't conflict, but they are identical so our binary is bigger than it needs to be.

It'd be great if they both encoded their dependency on a common class but didn't try to include it themselves so we could just fetch that class and use it in both places. Cocoapods's dependency resolution allows for that, and will work well when both frameworks want exactly the same Reachability class. However, we hit a problem again when they want different libraries, with the same names in.

Imagine that the two frameworks were written using different versions of libosethegame. The developers changed the interface when they went from v1.0 to v1.1, and Framework-O-Tron is still based on the older version of the interface. So just selecting the newer version won't work. Of course, neither does just selecting the older version. Can we have both versions of libosethegame, used by the two different frameworks, without ending up back with the symbol collision error?

At least in principle, yes we can. The dynamic loader, dyld (also described in the podcast) supports two-level namespace for dynamic libraries. Rather than linking against the osethegame library with -losethegame, you could deploy both libosethegame.1.0.0.dylib and libosethegame.1.1.0.dylib. One framework links with -losethegame.1.0, the other links with -losethegame.1.1. Both are deployed, and the fact that they were linked with different names means that the two-level namespace resolves the correct symbol from the correct version of the library, and all is well.

Of course, if you've got dynamic libraries, and the library vendor is willing to do a little work, they can just ship one version that supports all previous behaviour, looking at which version of the library the client was linked against to decide what behaviour to provide. Despite Mac OS X providing a versioned framework bundle layout, Apple has never (to my knowledge) shipped different versions of the system frameworks. Instead, the developers use the Mach-O load headers for an executable to find the linked version of their library, and supply behaviour equivalent to that version.

The above two paragraphs do rather depend on being able to use the dynamic linker. We can't, on iOS, at the moment.

#### Objective-Curry
Sadly it's not called Schoenfinkeling, but that's the name of the
person who noticed that there's no reason to ever have a function with
more than one argument. What you think is a function with two
arguments is actually a function with one argument that returns a
function with one argument, this second function closing over the
first argument and acting perhaps on both.

This is, of course, famed for its application (pardon the pun) to Object-Oriented
Programming. We can take a class like this:

@interface NamePrinter : Curryable

- (void)printFirstName:(NSString *)first surname:(NSString *)last;
- (void)printFirstName:(NSString *)first age:(NSInteger)age;
- (void)printFirstName:(NSString *)first middle:(NSString *)middle surname:(NSString *)last;

@end

and use it like this:

int main(int argc, char *argv[]) {
  @autoreleasepool {
    id printer = [[NamePrinter new] autorelease];
    id curried = [printer printFirstName:@"Graham"];
    [curried surname:@"Lee"];
    [curried surname:@"Greene"];
    [curried surname:@"Garden"];
    [curried age:18];

    id alex = [printer printFirstName:@"Alexander"];
    id alexG = [alex middle:@"Graham"];
    [alexG surname:@"Bell"];
  }
}

(your compiler probably complains at this point that it doesn't know
what you're up to. Of course, you know better.)

We'll get results like this:

2015-02-13 00:57:57.421 CurrySauce[41877:134228] Graham Lee
2015-02-13 00:57:57.421 CurrySauce[41877:134228] Graham Greene
2015-02-13 00:57:57.421 CurrySauce[41877:134228] Graham Garden
2015-02-13 00:57:57.422 CurrySauce[41877:134228] Graham is 18 years old
2015-02-13 00:57:57.422 CurrySauce[41877:134228] Alexander Graham Bell

OK, we don't actually get that for free. There's some secret sauce I
haven't shown you: secret curry sauce.

Here be dragons

As with
all the best bits of Objective-C,
you need to turn off the automatic reference counting to do what
follows (you'll have already seen a call to -autorelease above). ARC
works wonders when you try to write Java in Objective-C, give or take
the strong-weak-strong tango. It isn't so helpful when you try to
write Objective-C in Objective-C.

Partial application

The NamePrinter class knows that you might call it with a partial
application, so it checks whether the selector you sent in your
message matches the prefix of any method it knows. If it does, and the
return type and argument types in this prefix can be uniquely
determined, then it creates a proxy to listen for the rest of the
application.

The restrictions on types are there due to the way that Objective-C
deals with C types in message sending. It'd be a lot easier to do this
in Ruby or Smalltalk, where everything is an object reference: as
Objective-C needs to deal with C types of different sizes, you must be
able to construct a single invocation from the prefix selector.

int argumentsForSelector(SEL aSelector)
{
  const char *name = sel_getName(aSelector);
  int count = 0;
  while(*name != '\0') {
    if (*name++ == ':') {
      count++;
    }
  }
  return count;
}

@implementation Curryable

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
  NSMethodSignature *superSignature = [super methodSignatureForSelector:aSelector];
  if (superSignature) {
    return superSignature;
  }
  NSString *thisSelectorName = NSStringFromSelector(aSelector);
  NSMutableSet *signatures = [NSMutableSet set];
  int argCount = argumentsForSelector(aSelector);
  Class currentClass = [self class];
  while (currentClass != Nil) {
    unsigned int methodCount = 0;
    Method *methodList = class_copyMethodList(currentClass, &amp;methodCount);
    for (int i = 0; i &lt; methodCount; i++) {
      Method method = methodList[i];
      SEL anotherSelector = method_getName(method);
      NSString *selectorName = NSStringFromSelector(anotherSelector);
      if ([selectorName hasPrefix:thisSelectorName]) {
        NSMethodSignature *fullSignature = [self methodSignatureForSelector:anotherSelector];
        NSMutableString *constructedTypeSignature = [[@"@@:" mutableCopy] autorelease];
        for (int j = 2; j &lt; argCount + 2; j++) {
          [constructedTypeSignature appendString:@([fullSignature getArgumentTypeAtIndex:j])];
        }
        [signatures addObject:[[constructedTypeSignature copy] autorelease]];
      }
    }
    free(methodList);
    currentClass = class_getSuperclass(currentClass);
  }
  if ([signatures count] != 1) {
    NSLog(@"curried selector does not uniquely match the type of any full selector prefix");
    return nil;
  }
  return [NSMethodSignature signatureWithObjCTypes:[[signatures anyObject] UTF8String]];
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
  id curry = [CurrySauce curryTarget:self invocation:anInvocation];
  [anInvocation setReturnValue:&amp;curry];
}

@end

All of that, just to return a proxy object.

Finish the application

This proxy also responds to unknown selectors, by trying to tack
them onto the end of the partially-applied selector. If that works,
and it ends up with a selector that the target recognises, then it
constructs the combined invocation, copies the arguments from the two
partial invocations, and invokes it on the target. If the combined
selector is supposed to return something then this proxy unpacks the
return value and puts it into the invocation it received, to ensure
that the caller picks it up.

SEL concatenateSelectors(SEL firstSelector, SEL secondSelector)
{
  NSString *firstPart = NSStringFromSelector(firstSelector);
  NSString *selectorName = [firstPart stringByAppendingString:NSStringFromSelector(secondSelector)];
  return NSSelectorFromString(selectorName);
}

@implementation CurrySauce
{
  id target;
  NSInvocation *invocation;
}

-initWithTarget:object invocation:(NSInvocation *)partialApplication
{
  target = [object retain];
  invocation = [partialApplication retain];
  return self;
}

+curryTarget:object invocation:(NSInvocation *)partialApplication
{
  return [[[self alloc] initWithTarget:object invocation:partialApplication] autorelease];
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
  //special case for respondsToSelector
  if (aSelector == @selector(respondsToSelector:)) {
    return [target methodSignatureForSelector:aSelector];
  }
  SEL combinedSelector = concatenateSelectors([invocation selector], aSelector);
  NSMethodSignature *combinedSignature = [target methodSignatureForSelector:combinedSelector];
  if (combinedSignature != nil) {
    NSMutableString *completionType = [NSMutableString stringWithFormat:@"%s@:",[combinedSignature methodReturnType]];
    for (int i = argumentsForSelector([invocation selector]) + 2; i &lt; argumentsForSelector(combinedSelector) + 2; i++) {
      [completionType appendFormat:@"%s",[combinedSignature getArgumentTypeAtIndex:i]];
    }
    return [NSMethodSignature signatureWithObjCTypes:[completionType UTF8String]];
  } else {
    return nil;
  }
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
  if ([anInvocation selector] == @selector(respondsToSelector:)) {
    [anInvocation invokeWithTarget:target];
    return;
  }
  SEL realSelector = concatenateSelectors([invocation selector], [anInvocation selector]);
  NSMethodSignature *signature = [target methodSignatureForSelector:realSelector];
  NSInvocation *combined = [NSInvocation invocationWithMethodSignature:signature];
  int argumentToSet = 2;
  void *argBuffer = malloc([[invocation methodSignature] frameLength]);
  for (int i = 2; i &lt; [[invocation methodSignature] numberOfArguments]; i++) {
    [invocation getArgument:argBuffer atIndex:i];
    [combined setArgument:argBuffer atIndex:argumentToSet++];
  }
  free(argBuffer);
  argBuffer = malloc([[anInvocation methodSignature] frameLength]);
  for (int i = 2; i &lt; [[anInvocation methodSignature] numberOfArguments]; i++) {
    [anInvocation getArgument:argBuffer atIndex:i];
    [combined setArgument:argBuffer atIndex:argumentToSet++];
  }
  free(argBuffer);

  [combined setTarget:target];
  [combined setSelector:realSelector];
  [combined invoke];
  if (strcmp([[combined methodSignature] methodReturnType], "v")) {
    void *returnBuffer = malloc([[combined methodSignature] methodReturnLength]);
    [combined getReturnValue:returnBuffer];
    [anInvocation setReturnValue:returnBuffer];
    free(returnBuffer);
  }
}

- (void)dealloc
{
  [target release];
  [invocation release];
  [super dealloc];
}

@end

Conclusion

Blah blah awesome power of the runtime, I suppose. It's pretty cool
that you can do this sort of thing in 150 lines of ObjC. I doubt many
people would want to use it for reals though.

#### Obsolescence
An eight-year-old model of iPad is now considered vintage and obsolete. For comparison, the Apple ][ was made from 1977-1993 (16 years) and the January 1983 Apple //e would’ve had exactly the same software support as the final model sold in November 1993, or the compatibility cards for Macintosh sold from 1991 onwards.

#### Official Google Mac Blog: Measuring performance of distributed notifications
Official Google Mac Blog: Measuring performance of distributed notifications on the performance of Google Update: "Just how expensive is it? How many notifications can you broadcast per second? As with all Google client products, we want to be good citizens and not bog down the client machine."

A noble sentiment, but dear Google, answer me this: just how many times per second is each app going to be checking for updates?  When does this become an important factor, and not a question of premature optimisation?  They decided to go for distributed notifications instead of distributed objects, which seems reasonable - not because of the overhead issues (in fact a DO is probably a lot cheaper, if written properly), but because of the kind of information they're trying to get through this IPC.

#### Old news
So the Inquirer thinks they've got a hot potato on their hands, with this "security flaw" in OS X.  I've been using this approach for years (like, since NeXTSTEP): boot into single-user and launch NetInfo manually, then passwd root.  Or in newer Mac OS X, nicl means you don't have to launch NetInfo.

Of course, if you give physical access to the computer without a Firmware password, then the 'attacker' may as well just boot from external media and do whatever they want from there.  But the solution, as well as setting the Firmware password, is to edit the /etc/ttys file, change the line:

console "/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow" vt100 on secure onoption="/usr/libexec/getty std.9600"

to:

console "/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow" vt100 on onoption="/usr/libexec/getty std.9600"

Now the root password is required in single-user mode (as the console is no longer considered a secure terminal).

#### OmniWeb 5.6 tip of the day
defaults write com.omnigroup.OmniWeb5 WebKitOmitPDFSupport -bool TRUE

Sorry, but it doesn't view properly and doesn't print properly either :-(.

#### On a re-read you realise this isn't really about Swift
It's a bit early to have formed an opinion on a recently-announced programming language, but as the requisite number of people have asked what mine is (i.e. at least zero) I thought I'd type and see what happens.

Rules in programming tend to be bullshit. This is about one-third of a talk I'm giving later in the year, so I'll leave that train of thought alone in case anyone's going to be in the audience.

Anyway, knowing this, we can observe the exceptions to any rule people tend to throw at us about programming languages. For example: "Static types for good engineering, dynamic types for exploration". Make sure your useing you're type's good, and notice how many engineering practices come from programmers who know dynamic languages.

We could add "you can't do good tooling on dynamic languages". O RLY.

Having thus realised that the rules are nonsense, and that some expert actually sat down and thought about what they wanted to get out of a language in the contexts in which they thought they'd use it. When I did that, I decided on something that goes in entirely the opposite direction to Swift. Starting with ObjC, I'm more likely to end up with Self than with Swift (but then I've already told you that I think Sun Microsystems did awesome shit, so that may be no surprise). It probably just means that I'm not an expert, though.

If you're going to use types, may as well go all-in on using tuples. You'll probably see plenty of definitions of a tuple, so let me confuse things by adding another:

  A tuple is an element drawn from the cartesian product of its member types.

What this means is that unlike boring old-fashioned ordered collections, there's a known number of things in a tuple and each is of a known type. This is useful for the vexing question of signalling errors, as you can just return something like a (Result?, Error?) like an industrialised society ought to be doing. It also means that almost every situation in which a method's interface accepts or returns a collection can be replaced with a situation in which it accepts a known number of things of known type.

But anyway, for the most part our programming languages allow us to accidentally introduce problems that we shouldn't need to solve, but the biggest problems we actually have to solve lie elsewhere.

#### On adopting testing, and CocoaDojos
In episode 18 of iDeveloper.TV Live I was discussing test-driven development with Scotty and John. I suggested that a great way to get started with TDD was to start adopting it in baby steps in your code. Got a bug report from a user? Create a new test target, and add a test that demonstrates the existence of this bug. Fix the bug, then prove to yourself that your new test now passes.

OK, but what if you don't want to change the way you write production code, just to try out a new style? What if you'd have to make too many big changes just to support that one small test? What if you're a fan of building one to throw away?

I've been thinking about those cases too, and looking around to see what the rest of the industry does in such a case. It seems the best fit to this case is a Coding Dojo. There's a video guide to coding dojo, but the TL;DW version goes like this:

Define your requirements.
Take it in turns to pair-program a way toward implementing the requirements.
Evaluate what happened.

The requirements don't just define the product you're trying to make; the point of a dojo is to focus on practice for its own sake so you could require a particular language, framework etc. be used. When developers aren't directly involved in pair programming, they're offering advice or writing up parts of the problem to be addressed: there's only one IDE, and one pair of people at the keyboard.

This looks like it could be a great way to learn new stuff and become a better developer in the company of fellow iOS programmers. But as a group activity, it needs more people than just me involved:

Would you participate in a Cocoa dojo in your local area?
Would you participate in one at a conference like NSConference, WWDC or Voices That Matter?
What would you want to learn about at a dojo?
What would you be willing to teach people at a dojo?
Is the coding dojo the most appropriate format? I've also been finding out about Corey Haines' Code Retreat, which puts less emphasis on the solution and more on the practice. Is that better? Something else?

Answers on a postcard, or you could just use the comments field below.

#### On advertising's place in the tech industry
Dave Winer said the way is open for a non-ad-supported tech sector:

The tech industry has been absorbed by the ad industry, and vice versa. 
However, there is, imho, still room for a tech industry that is not merged with the ad industry. 
In fact, if we want to have a tech industry at all, we'd better invest in the "other" one, because advertising isn't much to bet on long-term. Seriously.

Daniel Jalkut went further, saying that ads are on the decline:

I’ll take this a step further: advertising is on the way out. Technology loathes a middle-man, and advertising as an industry is the king of all middle-men. The purpose of advertising is to connect customers with companies, so as to facilitate a transfer of money in exchange for goods or services. As time goes by, customers and companies will be more and more capable of achieving this on their own.

I agree with Dave, but not so much with Daniel. As I was writing this post, WPP (the world's largest ad company) were on the radio announcing a 20% increase in profits worldwide, with an increase in the fraction of revenue they get from digital adverts up to 30%. WPP say that concerns of disintermediation (i.e. the removal of middle-men Daniel talks about) are unfounded. The internet has produced its own new middle men, particularly Google and Facebook. "Traditional" advertisers such as WPP are profiting from advertising with those companies.

Ads can be on the way out, but if that's a world we want to see, it's up to us app makers to do something about it, it won't just happen automatically. Why us? Because the future involves even more mobile internet, and that means that we are in a privileged position to control what customers of the future are looking at. If we want them to look at something other than adverts, we need to put something that isn't an advert and is worth looking at on their screen.

That's a world in which many apps will need a very different business model than today's. There are a lot—particularly on the web and on Android—of apps that are free to users and generate revenue from ads. Those would need to be funded in different ways: I don't think that the freemium model of "pay to remove ads" will work. It doesn't solve the goal of reducing ads (every user still sees the ad version at least once), for a start. Also, conversion rates will likely be low as people can "put up with" adverts.

Apps that still want to rely on viral growth still need a really low barrier to entry for new users, but need to change why and how many people convert to a paid version. Companies like Evernote and Dropbox have a model that appears to work for them, where paying customers get extra features/capacity over free users, and provide enough income to support the cost of providing the free product. And then of course there's the shareware model that's worked for decades: where the store you're selling in allows you to do that.

But not all products need to be promoted virally. You can just sell something on the promise that it solves a problem, in the same way that I made a living selling solutions to app security problems, and the Omni Group make even more money selling the solution to the problem of remembering all the stuff you need to do.

You can also sell the promise that people can come back later: that is, you make money by letting customers stick to you. Look at iTunes Match. For some amount of money, Apple manages your music library. Then, next year, they'll ask you whether you want to carry on doing that. You get the choice between pulling all your music back down and looking after it yourself again, or you can pay Apple once more and they'll carry on doing it for a year. Apple are betting on there being enough people who come back every year to make this worthwhile.

So, to conclude, app makers who want to see an ad-free world are well placed to make that happen: doing so may start with an uncomfortable review of the way our apps are going to make money. In the short term, it'll mean having an expensive fully-customer-funded app competing in a world of cheap ad-supported apps. But in the longer term it could result in better, more transparent relationships with our customers who aren't worried about what data we're selling to advertisers.

#### On Apple's swings and misses
There's a trope in the Apple-using technologist world that when an Apple innovation doesn't immediately succeed, they abandon it. It's not entirely true, let's see what actually happens.

The quote in the above-linked item that supports the claim: "Apple has a tendency to either hit home runs out of the box (iPod, iPhone, AirPods) or come out with a dud and just sweep it under the rug, like iMessage apps and stickers." iMessage apps and stickers are new features in iMessage. These are incremental additions to an existing technology. Granted, neither of them have revolutionised the way that everybody uses iMessage, and neither of them have received much (or any) further (user-facing) development, but both are themselves attempts to improve an actual product that Apple actually has and has not swept under the rug.

We can make a similar argument about the TouchBar. The TouchBar is the touchscreen strip on some models of MacBook Pro laptop that replaces the function key row on the keyboard with an adaptive UI. It appeared, it…stayed around a bit, then it seems to have now disappeared. Perhaps importantly, it never got replicated on their other keyboards, like the one that comes with the iMac or the one you can buy separately. We could say that the TouchBar was a dud that got swept under the rug, or we could say that it was an incremental change to the MacBook Pro and that Apple have since tried other changes to this long-running product, like the M1 architecture.

There are two other categories of non-home-run developments to take into account. The first is the duds that do get incremental development. iTV/Apple TV was such a bad business for the first many years of its history that execs would refer to it as a hobby, right up until it made them a billion dollars and was no longer a hobby.

Mac OS X's first release was a lightly sparkling OpenStep, incompatible with any Mac software (it came with a virtual machine to run actual MacOS) and incompatible with most Unix software too. It was sold as a server-only product, which given the long wait involved when doing something as simple as opening the text editor (a Java application) was a sensible move. Yet, here we are, 23 years later, and macOS/iOS/iPadOS/tvOS/watchOS/bridgeOS is the same technology, incrementally improved.

Then the next category is things that go away, get rethought, and brought back. The thing we see might look like a dud but it's actually an idea that Apple stick with. Again, two examples: remember dashboard widgets in Tiger? There was an overlay view that let you organise a screen of little javascript widgets to do world time, stocks, weather, and other things including those supplied by third-party developers. It was there, it looked the same for a bit (as long as you don't mention the DashCode tool introduced along the way), then it wasn't there. But later, control center came along, and we got a new version of the same idea.

In between that fizzy NeXT version of Mac OS X Server and the first public release of Mac OS X 10.0 (which was also a dud, many users sticking with MacOS 9 and Apple even giving away 10.1 for free to ensure as many people as possible got the fixes), the Aqua interface was born. Significantly more "lickable" than its modern look, it was nonetheless recognisable to a Monterey user, with its familiar traffic-light window controls: red for close, yellow for minimise, green for zoom, and purple for…wait, purple? Yes, purple. This activated single-window mode, in which only the active window was shown and all others minimised to the dock. Switch window, and the previous one disappeared. This wasn't in the public release, but now we have Mission Control and fullscreen mode, so did it truly go away?

#### On association
My research touches on the professionalisation (or otherwise) of software engineering, and particularly the association (or not) of software engineers with a professional body, or with each other (or not) through a professional body. So what's that about?

In Engagement Motivations in Professional Associations, Mark Hager uses a model that separates incentives to belong to a professional association into public incentives (i.e. those good for the profession as a whole, or for society as a whole) and private incentives (i.e. those good for an individual practitioner). Various people have tried to argue that people are only motivated by the private incentives (i.e. de Tocqueville's "enlightened self-interest").

Below, I give a quick survey of the incentives in this model, and informally how I see them enacted in computing. If there's a summary, it's that any idea of professionalism has been enclosed by the corporate actors.

Public incentives

Promoting greater appreciation of field among practitioners

My dude, software engineering has this in spades. Whether it's a newsletter encouraging people to apply formal methods to their work, a strike force evangelising the latest programming language, or a consultant explaining that the reason their methodology is failing is that you don't methodologise hard enough, it's not incredibly clear that you can be in computering unless you're telling everyone else what's wrong with the way they computer. This is rarely done through formal associations though: while the SWEBOK does exist, I'd wager that the fraction of software engineers who refer to it in their work is 0 whatever floating point representation you're using.

Ironically, the software craftsmanship movement suggests that a better way to promote good practice than professional associations is through medieval-style craft guilds, when professional associations are craft guilds that survived into the 20th century, with all the gatekeeping and back-scratching that entails.

Promoting public awareness of contributions in the field

If this happens, it seems to mostly be left to the marketing departments of large companies. The last I saw about augmented reality in the mainstream media was an advert for a product.

Influencing legislation and regulations that affect the field

Again, you'll find a lot of this done in the policy departments of large companies. The large professional societies also get involved in lobbying work, but either explicitly walk back from discussions of regulation (ACM) or limit themselves to questions of research funding. Smaller organisations lobby on single-issue platforms (e.g. FSF Europe and the "public money, public code" campaign; the Documentation Foundation's advocacy for open standards).

Maintaining a code of ethics for practice

It's not like computering is devoid of ethics issues: artificial intelligence and the world of work; responsibility for loss of life, injury, or property damage caused by defective software; intellectual property and ownership; personal liberty, privacy, and data sovereignty; the list goes on. The professional societies, particularly those derived from or modelled on the engineering associations (ACM, IEEE, BCS), do have codes of ethics. Other smaller groups and individuals try to propose particular ethical codes, but there's a network effect in play here. A code of ethics needs to be popular enough that clients of computerists and the public know about it and know to look out for it, with the far extreme being 100% coverage: either you have committed to the Hippocratic Oath, or you are not a practitioner of medicine.

Private incentives

Access to career information and employment opportunities

If you're early-career, you want a job board to find someone who's hiring early career roles. If you're mid or senior career, you want a network where you can find out about opportunities and whether they're worth pursuing. I don't know if you've read the news lately, but staying employed in computering isn't going great at the moment.

Opportunities to gain leadership experiences

How do you get that mid-career role? By showing that you can lead a project, team, or have some other influence. What early-career role gives you those opportunities? crickets Ad hoc networking based on open source seems to fill in for professional association here: rather than doing voluntary work contributing to Communications of the ACM, people are depositing npm modules onto the web.

Access to current information in the field

Rather than reading Communications of the ACM, we're all looking for task-oriented information at the time we have a task to complete: the Q&A websites, technology-specific podcasts and video channels are filling in for any clearing house of professional advancement (to the point where even for-profit examples like publishing companies aren't filling the gaps: what was the last attempt at an equivalent to Code Complete, 2nd Edition you can name?). This leads to a sort of balkanisation where anyone can quickly get up to speed on the technology they're using, and generalising from that or building a holistic view is incredibly difficult. Certain blogs try to fill that gap, but again are individually published and not typically associated with any professional body.

Professional development or education programs

We have degree programs, and indeed those usually have accredited curricula (the ACM has traditionally been very active in that field, and the BCS in the UK). But many of the degrees are Computer Science rather than Software Engineering, and do they teach QA, or systems administration, or project management, or human-computer interaction? Are there vocational courses in those topics? Are they well-regarded: by potential students, by potential employers, by the public?

And then there are vendor certifications.

#### On authorization proxy objects
Authorization Services is quite a nice way to build in discretionary access controls to a Mac application. There's a whole chapter in Professional Cocoa Application Security (Chapter 6) dedicated to the topic, if you're interested in how it works.

The thing is, it's quite verbose. If you've got a number of privileged operations (like, one or more) in an app, then the Auth Services code can get in the way of the real code, making it harder to unpick what a method is up to when you read it again a few months later.

Let's use some of the nicer features of the Objective-C runtime to solve that problem. Assuming we've got an object that actually does the privileged work, we'll create a façade object GLPrivilegedPerformer that handles the authorization for us. It can distinguish between methods that do or don't require privileges, and will attempt to gain different rights for different methods on different classes. That allows administrators to configure privileges for the whole app, for a particular class or even for individual tasks. If it can't get the privilege, it will throw an exception. OK, enough rabbiting. The code:

@interface GLPrivilegedPerformer : NSObject {
    id actual;
    AuthorizationRef auth;
}
- (id)initWithClass: (Class)cls;
@end

@implementation GLPrivilegedPerformer

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector {
    NSMethodSignature *sig = [super methodSignatureForSelector: aSelector];
    if (!sig) {
        sig = [actual methodSignatureForSelector: aSelector];
    }
    return sig;
}

- (BOOL)respondsToSelector:(SEL)aSelector {
    if (![super respondsToSelector: aSelector]) {
        return [actual respondsToSelector: aSelector];
    }
    return YES;
}

- (void)forwardInvocation:(NSInvocation *)anInvocation {
    if ([actual respondsToSelector: [anInvocation selector]]) {
        NSString *selName = NSStringFromSelector([anInvocation selector]);
        if ([selName length] > 3 && [[selName substringToIndex: 4] isEqualToString: @"priv"]) {
            NSString *rightName = [NSString stringWithFormat: @"%@.%@.%@",
                                   [[NSBundle mainBundle] bundleIdentifier],
                                   NSStringFromClass([actual class]),
                                   selName];
            AuthorizationItem item = {0};
            item.name = [rightName UTF8String];
            AuthorizationRights requested = {
                .count = 1,
                .items = &item,
            };
            OSStatus authResult = AuthorizationCopyRights(auth,
                                                          &requested,
                                                          kAuthorizationEmptyEnvironment,
                                                          kAuthorizationFlagDefaults |
                                                          kAuthorizationFlagExtendRights |
                                                          kAuthorizationFlagInteractionAllowed,
                                                          NULL);
            
            if (errAuthorizationSuccess != authResult) {
                [self doesNotRecognizeSelector: [anInvocation selector]];
            }
        }
        [anInvocation invokeWithTarget: actual];
    }
    else {
        [super forwardInvocation: anInvocation];
    }
}

- (id)initWithClass: (Class)cls {
    self = [super init];
    if (self) {
        OSStatus authResult = AuthorizationCreate(NULL,
                                                  kAuthorizationEmptyEnvironment,
                                                  kAuthorizationFlagDefaults,
                                                  &auth);
        if (errAuthorizationSuccess != authResult) {
            NSLog(@"couldn't create auth ref");
            return nil;
        }
        actual = [[cls alloc] init];
    }
    return self;
}

- (void)dealloc {
    AuthorizationFree(auth, kAuthorizationFlagDefaults);
    [actual release];
    [super dealloc];
}
@end

Some notes:

You may want to raise a custom exception rather than using -doesNotRecognizeSelector: on failure. But you're going to have to @catch something on failure. That's consistent with the way Distributed Objects handles authentication failures.
The rights it generates will have names of the form com.example.MyApplication.GLActualPerformer.privilegedTask, where GLActualPerformer is the name of the target class and privilegedTask is the method name.
There's an argument for the Objective-C proxying mechanism making code harder to read than putting the code inline. As discussed in Chapter 9, using object-oriented tricks to make code non-linear has been found to make it harder to review the code. However, this proxy object is small enough to be easily-understandable, and just removes authorization as a cross-cutting concern in the style of aspect-oriented programming (AOP). If you think this will make your code too hard to understand, don't use it. I won't mind.
As mentioned elsewhere, Authorization Services is discretionary. This proxy pattern doesn't make it impossible for injected code to bypass the authorization request by using the target class directly. Even if the target class has the "hidden" visibility attribute, class-dump can find it and NSClassFromString() can get the Class object.

#### On Being a Software Person
On Wednesday I spoke at Qcon London, about "Mobile App Security and Privacy: You're Doing It Wrong (and so am I)" as part of @akosma's track on iOS and Android. The whole track was full of win: particularly, if you ever get the opportunity to hear @fraserspeirs talk about how his school is using iPads to change the way they teach, please do take the opportunity. You will find out a lot about how to write apps that people can (and will) use.

Thursday, I was part of a London iPhone Developer Group panel, alongside @akosma and @bmf, where we talked about $5 Xcode, when iOS and Mac OS X will finally converge, why you can't sell software to an Android user. Oh, and we prototyped the UI for Photoshop for iPad.

Seriously, we did that. Why? Because there was sentiment in the discussion group that desktops couldn't die because it was impossible to do Photoshop on an iPad. This is annoying. In order to show the group that doing Photoshop on the iPad might be possible, I made us do it. Now it turns out that once you have done it, it is possible.

[Update: @akosma since pointed out that Adobe has already done PhotoShop on iOS, and that this whole conversation was redundant before we began.]

What this really shows us is that in order to be a Software Person, you need to take seriously the idea that writing software might be possible. It often is possible, and if you try it you're more likely to get a useful (and, of course, saleable) app out the end than if you don't try it.

Conveniently, this is compatible with a meme that I started in my Qcon talk, which goes like this:

If you do not know x, then you cannot be a software engineer.

So, if you do not know tenacity, then you cannot be a software engineer. Conversely, you must also know when discretion is the better part of valour, and it is time to give up. No-one likes a death march project, when the cost of development has gone way beyond any likely return, or when the software you're writing is no longer relevant.

The following is a list of things that you must know (or be able to hire someone to know for you) in order to be a software engineer. It is not exhaustive: these are merely the ones that I either came up with in my talk, or that Mike and I talked about while learning about evolution.

Tenacity
Discretion
Marketing
User experience
Estimation
Humility
When to be a dick
Testing
Human nature
Why phishing is successful
Requirements engineering
Why you would want to be a software engineer

Some people also add a task called "coding" to this list. Please add your own in the comments.

#### On BizSpark
You'll remember that recently I reviewed Windows Phone 7 Mango from the perspective of an iOS guy, and actually came back pretty impressed with it.

You'll also remember that through my company, Fuzzy Aliens Ltd, I offer app security services to mobile app developers. So far, that basically means iOS developers: in addition to being where I have most experience, I have punted around for Android clients and got exactly zero interest.

So I thought it would be useful to offer the same service for WP7. After all, Microsoft knows the bad press associated with having security fail on their platform, so should be welcoming of a security guy adding his biological and technological distinctiveness to their own. Not only that, but there will probably be a lot of line-of-business app developers out there who would appreciate mobile security knowledge.

Now the thing that puts me off is basically the cost. I own exactly one copy of Windows 7, and use the free Visual Studio Express. To meaningfully research and code for Windows Phone 7 I'd need another two Windows licences (£100-£250 each roughly depending on version) and Visual Studio Pro and MSDN (roughly £700), along with at least one handset (£300) and an App Hub membership (£60). Wow. Around £1500 just to dip my toe in untested waters.

Luckily, Microsoft have a plan designed to help. BizSpark ought to give me access to most of the above except the phone, in addition to training. It also offers that MS would put me in touch with potential clients and even investors, and could help with hosting costs for web services. The idea is that Fuzzy Aliens would get this stuff for free for a while, during which MS would help build the business. Then, once FZA "graduates" from the program, I get to keep all the software and MS have a new trusted partner.

Seems like a low-risk way to get into Windows Phone 7, and to grow my business which - while only six months old - is already showing signs that I need to find more clients from somewhere. So I signed up at around 16:15 today.

By 18:13 Microsoft had decided that:

it does not appear that you meet all the eligibility requirements at this time. To enter the program, your startup must be: 
Actively engaged in development of a software-based product or online service that will form a core piece of its current or intended business,
Privately held,
In business for less than 3 years, and
Less than US $1 million in annual revenue


Well, in fact FZA meets all of those criteria. The basis of its business is secure software, and indeed I am currently (OK, I'm blogging - you see what I mean though) developing such secure software. Indeed I even help out the platform community for free by releasing some of this software here as open source.

The business is fully held by me, and has been operating for nearly six months. I would dearly love to have more than $1M of revenue, but it hasn't happened yet.

So for whatever reason - though not one they care to tell me about - Microsoft has decided that they don't want me joining their community. Given that this leaves me free to focus on making the iPhone a safer platform for its users, I don't yet know which of us has lost out the most.

#### On Blue Agile
Ron Jeffries has some interesting posts lately on Dark Scrum, the idea that poor programmers are being chained to the code face in the software mines, forced to unthinkingly crank out features under Agile-sequel banners like “velocity” and “embracing change”.

In one such post, he refutes the notion of a shadowy Agile Indu$trial Complex. The AIC is a hypothetical network of consultants, tool vendors and project managers who collectively profit from every story point, every Jira “story”, and every grooming session.

Here’s the thing, though. You don’t need fnord filters and Masonic handshakes to explain Dark Scrum. You need to understand that Scrum is doing exactly as intended, and that it’s orthogonal to—not opposite to—the intentions of Agile software development.

Alan Kay had this analogy of two perpendicular planes that he used to explain the difference between Object-Oriented Programming and the thing programmers do in Java. The pink plane contains existing ideas and processes. You can get better at your craft by advancing along the pink plane.

Every so often, an idea comes along (OOP, says Kay, is one such idea) that is not better, it is different. It takes you out of the pink plane altogether into the orthogonal blue plane. You can’t ask “is this idea better” because it doesn’t make sense. It’s different. It has different qualities, so your question about what makes things better no longer makes sense.

Back when we were all allegedly doing waterfall software development, we were delivering software that satisfied requirements defined for some project. We can ask “how good are we at delivering software that satisfies requirements”, and define “better” answers to that question.

Scrum is a process improvement framework for delivering products. As such it provides a “good” baseline for software delivery, and tools to help us get “better” at it. Scrum is the pink plane.

The Agile crowd, on the other hand, stopped asking how much software we are delivering, and started asking how valuable our interactions with our customers are. It’s in the blue plane. The questions we had in the pink plane are no longer relevant, so if we’re still asking them, we will get nonsensical answers.

It is possible, even likely, that the rise of Scrum is due to trying to apply pink plane thinking to the Agile idea, in the way that the rise of C++ or SOLID is due to pink plane thinking about OOP. You could imagine someone who manages a software team seeing a lot of software coming out of an XP team. They read about the XP practices and conclude “I need to make my team adopt these practices, and we’ll make more software”.

But perhaps the XP team weren’t worried about making more software, and don’t even understand why making more software would be a goal.

That doesn’t make Scrum “bad”, indeed looked at along the pink plane it’s better than its predecessors. It just makes it unexpected, and disappointing as a result.

#### On books
I'd say that if there's one easy way to summarise how I work, it's as an information focus. I'm not great at following a solution all the way to the bitter end so you should never let me be a programmer (ahem): when all that's left is the second 90% of the effort in fixing the bugs, tidying up edge cases and iterating on the interaction, I'm already bored and looking for the next thing. Where I'm good is where there's a big problem to solve, and I can draw analogies with things I've seen before and come up with the "maybe we should try this" suggestions.

Part of the input for that is the experience of working in lots of different contexts, and studying for a few different subjects. A lot of it comes from reading: my goodreads account lists 870 books and audiobooks that I've read and I know it to be an incomplete record. Here are a few that I think have been particularly helpful (professionally speaking, anyway).

Douglas Adams, The Hitch-Hikers' Guide to the Galaxy. Adams is someone who reminds us not to take the trappings of society too seriously, and to question whether what we're doing is really necessary. Are digital watches really a neat idea? Also an honourable mention to the Dirk Gently novels for introducing the fundamental interconnectedness of all things.
Steve Jackson and Ian Livingstone, The Warlock of Firetop Mountain. I can think of at least three software projects that I've been able to implement and describe as analogies to the choose your own adventure style of book.
David Allen, Getting Things Done, because quite often it feels like there's too much to do.
Douglas Hofstadter, Godel, Escher, Bach: An Eternal Golden Braid is a book about looking for the patterns and connections in things.
Victor Papanek, Design for the Real World, for reminding us of the people who are going to have to put up with the consequences of the things we create.
Donald Broadbent, Perception and Communication, for being the first person to systematically explore that topic.
Steven Hawking, A Brief History of Time, showing us how to make complex topics accessible.
Roger Penrose, The Road to Reality, showing us how to make complex topics comprehensively presentable.
Douglas Coupland, Microserfs, for poking fun at things I took seriously.
Janet Abbate, Recoding Gender, because computering is more accessible to me than to others for no good reason.
Joshua Bloch, Effective Java, Second Edition, for showing that part of the inaccessibility is a house of cards of unsuitable models with complex workarounds, and that programmers are people who delight in knowing, not addressing, the workarounds.
Michael Feathers, Working Effectively with Legacy Code, the one book every programmer should read.
Steve Krug, Don't make me think!, a book about the necessity of removing exploration and uncertainty from computer interaction.
Seymour Papert, Mindstorms, a book about the necessity of introducing exploration and uncertainty into computer interaction.
Richard Stallman, Free as in Freedom 2.0, for suggesting that we should let other people choose between ther previous two options.
Brad Cox, Object-Oriented Programming: An Evolutionary Approach, for succinctly and effortlessly explaining objects a whole decade before everybody else got confused by whether a dog is an animal or a square is a rectangle.
Gregor Kiczales, Jim des Rivieres, and Daniel G. Bobrow, The Art of the Metaobject Protocol showed me that OOP is just one way to do OOP, and that functional programming is the same thing.
Simson Garfinkel and Michael Mahoney, NEXTSTEP Programming: Step One was where I learnt to create software more worthwhile than a page of BASIC instructions.
Gil Amelio, On the Firing Line: My 500 Days at Apple shows that the successful business wouldn't be here if someone hadn't managed the unsuccessful business first.

There were probably others.

#### On comment docs
Something I'm looking at right now is generation of (in my case, HTML) API documentation from some simple markup format. The usual way to do this is by writing documentation markup inline in the source code, using specially formatted comments in header files.

The point

Some people argue that well-written source code should be its own documentation. Well, that's true, it is: but it's documentation with limited utility. Source code provides the following documentation:

Document, for the compiler's benefit, the machine instructions that the compiler should generate
Document, for the programmer's benefit, the machine instructions that the compiler should generate

Developing a high-level model of how software works from its source code is possible, but mentally taxing. It's not designed for that. It would be like asking an ant to map the coastline of Africa: it can be done, but the information available is at entirely the wrong scale.

Several other approaches for high-level documentation of software systems exist. Of course, each of them is not actually the source code, but a model: a map of Africa is not actually Africa, but if you want to know what the coastline of Africa looks like then the map is a very useful model.

Comment documentation is one such model of software. Well-written comment docs explain why you might use a method or class, and how its properties or parameters help you to use it. It gives you a sense of how the classes fit together, and how you can exploit them. They're called comment docs because they go into comments right alongside the source they document, usually marked up with particular tokens. As an example of a token, many documentation comment systems let me use a line like this:
@author Graham J. Lee
to indicate that I wrote a particular part of the project.

Of course, this documentation could go anywhere, so why put it into the header files? For a start, you already have the header files, so you're not having to maintain two parallel hierarchies of content. Also, the proximity of the documentation to the source code means that there's a higher probability (still not unity, but higher) that a developer who changes the intent or usage of a method will remember to update the documentation. Additionally, it means that the documentation can be no more verbose than required: anything that is obvious from the source (the names of methods and their parameters, for example) can be discovered from the source. This is not inconsistent with my earlier statement about source-as-documentation: you can easily find out a method signature without having to grub through the actual program instructions.

Finally, it means that when you're working with the source, you have the documentation right there. This is one very common way to interact with comment documentation. The other way is to use a tool to create some friendly formatted output (for me, the goal is HTML) by reading the source files and extracting the useful information from the comments.

Doxygen

I have for the last forever (alright, three years) used Doxygen, for a couple of reasons:

The other people on my team at the time I adopted it were already using it
Since then, it has gained support for generating Xcode documentation sets, which can be viewed inside the Xcode organizer.

It's not great, though. It's a very complicated tool that tries to do all things for all people, so the configuration is huge and needs a lot of consideration. You can customise the look of the output using a CSS file but you pretty much need to as the default output looks like arse. Making it actually output different stuff is trickier, as it's a C++ project and I've only ever learned enough C++ to customise Clang.

HeaderDoc

So I decided to fish around for alternatives. Of course, we know that Apple uses (and ships) HeaderDoc, but it uses its own comment format. Luckily, its comment format is identical to Javadoc, which is the format used by Doxygen. You can get headerdoc2html to look for the /** trigger by passing the -j flag.

Speaking of headerdoc2html, this is one of two programs in the HeaderDoc distribution. The other is gatherheaderdoc, which generates a table of contents from a collection of output files. Each of these is a well-written and well-documented perl script, which makes extension and modification super-easy.

Neither should be immediately required, in fact. The tool understands all of the Objective-C language features, and some extra bonus things like availability macros and groups of related methods. The default output basically looks like someone forgot to apply the style sheet to developer.apple.com. It's trivial to configure HeaderDoc to use an external style sheet, and you can even create a custom template HTML file so that things appear in whatever fashion you want. Another useful configuration point is the C preprocessor, which you can get HeaderDoc to run through before interpreting the documentation.

Autogsdoc

The GNUstep project uses its own comment documentation tool called Autogsdoc. The fact that the autogsdoc documentation has buggy HTML does not fill one with confidence.

In fact, Autogsdoc's output is unstyled XHTML 1.0, so it would be easy to style it to look more useful. Some projects that I've seen appear to have frames-based HTML, which is unfortunate.

Autogsdoc uses inline comment documentation, the same as the other options we've looked at. However, its markup is significantly different, as it uses SGML-style tags inside the documentation. It has a (well-documented, of course) Objective-C implementation with a good split of responsibilities between different classes. Hacking about on its innards shouldn't be too hard for any motivated Cocoa coder.

Footnote

I know this has been annoying you all since the first paragraph in the section on HeaderDoc: */.

#### On community
This is a post that had been boiling for a while; I talked a little about the topic when I was in Appsterdam earlier this year, and had a few more thoughts which were completely supplanted and rearranged by watching

#### On counting numbers
While we were at NSConference, Alistair Houghton told me that he was working on static NSNumbers in clang. I soon thought: wouldn't it be nice to have code like this?

for (NSNumber *i in [@10 times]) { /* ... */ }

That would work something like this. You must know three things: one is that the methods have been renamed to avoid future clashes with Apple methods. Another is that we automatically get the NSFastEnumeration support from NSEnumerator, though it certainly is possible to code up a faster implementation of this. Finally, that this code is available under the terms of the WTFPL though without warranty, to the extent permitted by applicable law.

NSNumber+FALEnumeration.h
@interface NSNumber (FALEnumeration)
- (NSEnumerator *)FALtimes;
- (NSEnumerator *)FALto: (NSNumber *)otherNumber;
- (NSEnumerator *)FALto: (NSNumber *)otherNumber step: (double)step;
@end

NSNumber+FALEnumeration.m
#import "NSNumber+FALEnumeration.h"
#import "FALNumberEnumerator.h"

@implementation NSNumber (FALEnumeration)

- (NSEnumerator *)FALtimes {
    double val = [self doubleValue];
    return [FALNumberEnumerator enumeratorFrom: 0.0
                                            to: val
                                          step: val > 0.0 ? 1.0 : -1.0];
}

- (NSEnumerator *)FALto: (NSNumber *)otherNumber {
    double val = [self doubleValue];
    double otherVal = [otherNumber doubleValue];
    double sgn = (otherVal - val) > 0.0 ? 1.0 : -1.0;
    return [self to: otherNumber step: sgn]; 
}

- (NSEnumerator *)FALto: (NSNumber *)otherNumber step: (double)step {
    double val = [self doubleValue];
    double otherVal = [otherNumber doubleValue];
    return [FALNumberEnumerator enumeratorFrom: val
                                            to: otherVal
                                          step: step];
}
@end

FALNumberEnumerator.h
@interface FALNumberEnumerator : NSEnumerator {
    double end;
    double step;
    double cursor;
}

+ (id)enumeratorFrom: (double)beginning to: (double)conclusion step: (double)gap;

- (id)nextObject;
- (NSArray *)allObjects;

@end

FALNumberEnumerator.m
#import "FALNumberEnumerator.h"

@implementation FALNumberEnumerator

+ (id)enumeratorFrom:(double)beginning to:(double)conclusion step:(double)gap {
    NSParameterAssert(gap != 0.0);
    NSParameterAssert((conclusion - beginning) * gap > 0.0);
    FALNumberEnumerator *enumerator = [[self alloc] init];
    if (enumerator) {
        enumerator->end = conclusion;
        enumerator->step = gap;
        enumerator->cursor = beginning;
    }
    return [enumerator autorelease];
}

- (id)nextObject {
    if ((step > 0.0 && cursor >= end) || (step < 1.0 && cursor

#### On cryptographic file storage
In Chapter 3 of Professional Cocoa Application Security, I talk about using CommonCrypto to encrypt files stored on either Mac or iOS file systems. In Chapter 4, I talk about using CommonCrypto to generate Hashed Message Authentication Codes (HMACs) to verify the integrity of messages received by an app. Unfortunately, I didn't connect the two in the book.

I actually did discuss the reasons for providing an HMAC in a recent talk on cryptographic storage for iOS. But I went to Justin Clark's talk at Security B-Sides London, where he demonstrated the attacks that are possible against cryptosystems that don't verify the integrity of their content. This talk was very informative: if the presentation ever makes its way online, I fully recommend that you check it out.

The talk reminded me that there are important attacks I didn't discuss in the book. So I need to bring these concepts together for the readers of PCAS. The tl;dr version is: if you encrypt content, you must also verify the integrity of the ciphertext. The long version follows.

I recommended in the book using CBC, or Cipher Block Chaining, mode for AES encryption. In this mode, the first block of plaintext is XORed with an Initialization Vector (IV), and this is then encrypted. The resultant ciphertext is then XORed with the second block of plaintext, and this is encrypted. And so on. This offers some protection against certain cryptanalysis attacks that Electronic Code Book (EBC) mode does not: for example, blocks of ciphertext cannot be arbitrarily swapped without the decrypted plaintext becoming nonsensical. A change in the ciphertext not only affects the decryption of the changed block, but also subsequent blocks.

Well, it turns out that this propagating-change property of CBC mode can be used by attackers&mdash;if they have information about errors encountered during decryption&mdash;to decrypt and encrypt data without needing to discover the key. Ouch. See the B-Sides talk described above (or google for Padding Oracle Attack) for the full details.

Of course, not providing the padding oracle is an important solution to this attack: but refusing to handle content that isn't verifiably valid avoids any attack involving modified ciphertext: including encrypted content that lies about its length, if that's an important consideration for your file format. Therefore, if you're encrypting data in your app, you should be generating an HMAC and verifying that before attempting a decryption operation. Tampered data is not data you want to deal with.

#### On default keychain settings
After my presentation at NSConference there was a discussion of default settings for the login keychain. I mentioned that I had previously recommended some keychain configuration changes including using a different password than your login password. Default behaviour is that any application can add a secure item to the keychain, and the app that did the adding is allowed to read and modify the entry without any user interaction. As Mike Lee added, all other apps will trigger a user dialogue when they try to do so - the user doesn't then need to authenticate but does have to approve the action.

That almost - but not quite - solves the issue of a trojan horse attempting to access the secure password. Sure, a trojan application can't get at it without asking the user. What about other trojan code? How about a malicious SIMBL hijack or a bundle loaded with mach_override? It should be possible to mitigate those circumstances by using custom code signing requirements, but that's not exactly well documented, and it's not really good usability for an app to just die on its arse because the developer doesn't like the other software their user has.

There's a similar, related situation - what if the app has a flawed design allowing it to retrieve a keychain item when it doesn't need it? Sounds like something which could be hard to demonstrate and harder to use, until we remember that some applications have "the internet" as their set of input data. Using a web browser as an example, but remembering that I have no reason to believe whether Safari, Camino or any other browser is designed in such a way, imagine that the user has stored an internet password. Now all that the configuration settings on the user's Mac can achieve is to stop other applications from accessing the item. If that browser is itself subject to a "cross-site credentials request" flaw, where an attacking site can trick the browser into believing that a login form (or perhaps an HTTP 401 response, though that would be harder) comes from a victim site, then that attacker will be able to retrieve the victim password from the keychain without setting off any alarms with the user.

If the user had, rather than accepting the default keychain settings, chosen to require a password to unlock the keychain, then the user would at least have the chance to inspect the state of the browser at the time the request is made. OK, it would be better to do the right thing without involving the user, but it is at least a better set of circumstances than the default.

#### On designing collections
Introduction

This post explores the pros and the cons of following the design rule "Objects responsible for collections of other objects should expose an interface to the collection, not the collection itself". Examples and other technical discussion is in Objective-C, assuming Foundation classes and idioms.

Example Problem

Imagine you were building Core Data today. You get to NSEntityDescription, which encapsulates the information about an entity in the Managed Object Model including its attributes and relations, collectively "properties". You have two options:

Let client code have access to the actual collection of properties.
Make client code work with the properties via API on NSEntityDescription or abstract collection interfaces.

In reality, NSEntityDescription does both, but not with the same level of control. As external clients of the class we can't see whether the collection objects are internal state of the entity description or are themselves derived from its real state, although this LGPL implementation from GSCoreData does return its real instance variable in one case. However, this implementation is old enough not to show another aspect of Apple's class: their entity description class itself conforms to NSFastEnumeration.

How to give access to the actual collection

This is the easiest case.

@implementation NSEntityDescription
{
    NSArray *_properties;
}

- (NSArray *)properties
{
    return _properties;
}

@end

A common (but trivial) extension to this is to return an autoreleased copy of the instance variable, particularly in the case where the instance variable itself is mutable but clients should be given access to a read-only snapshot of the state. A less-common technique is to build a custom subclass of NSArray using an algorithm specific to this application.

How to provide abstract collection access

There are numerous ways to do this, so let's take a look at a few of them.

Enumerator

It doesn't matter how a collection of things is implemented, if you can supply each in turn when asked you can give out an enumerator. This is how NSFileManager lets you walk through the filesystem. Until a few years ago, it's how NSTableView let you see each of the selected columns. It's how PSFeed works, too.

The good news is, this is usually really easy. If you're already using a Foundation collection class, it can already give you an appropriate NSEnumerator.

- (NSEnumerator *)propertyEnumerator
{
     return [_properties objectEnumerator];
}

You could also provide your own NSEnumerator subclass to work through the collection in a different way (for example if the collection doesn't actually exist but can be derived lazily).

Fast enumeration

This is basically the same thing as "Enumerator", but has different implementation details. In addition, the overloaded for keyword in Objective-C provides a shorthand syntax for looping over collections that conform to the NSFastEnumeration protocol.

Conveniently, NSEnumerator conforms to the protocol so it's possible to go from "Enumerator" to "Fast enumeration" very easily. All of the Foundation collection classes also implement the protocol, so you could do this:

- (id &lt;NSFastEnumeration&gt;)properties
{
    return _properties;
}

Another option&mdash;the one that NSEntityDescription actually uses&mdash;is to implement the -countByEnumeratingWithState:options:count: method yourself. A simple implementation passes through to a Foundation collection class that already gets the details right. There are a lot of details, but a custom implementation could do the things a custom NSEnumerator subclass could.

Object subscripting

If you've got a collection that's naturally indexed, or naturally keyed, you can let people access that collection without giving them the specific implementation that holds the collected objects. The subscripting methods let you answer questions like "what is the object at index 3?" or "which object is named 'Bob'?". As with "Fast enumeration" there is syntax support in the language for conveniently using these features in client code.

- (id)objectAtIndexedSubscript: (NSUInteger)idx
{
    return _properties[idx];
}


Key-Value Coding

Both ordered and unordered collections can be hidden behind the to-many Key-Value Coding accessors. These methods also give client code the opportunity to use KVC's mutable proxy collections, treating the real implementation (whatever it is) as if it were an NSMutableArray or NSMutableSet.

- (NSUInteger)countOfProperties
{
    return [_properties count];
}

- (NSPropertyDescription *)objectInPropertiesAtIndex: (NSUInteger)index
{
    return _properties[index];
}

- (NSArray *)propertiesAtIndexes: (NSIndexSet *)indexes
{
    return [_properties objectsAtIndexes: indexes];
}

Block Application (or Higher-Order Messaging)

You could decide not to give out access to the collection at all, but to allow clients to apply work to the objects in that collection.

- (void)enumeratePropertiesWithBlock: (void (^)(NSPropertyDescription *propertyDescription))workBlock
{
    NSUInteger count = [_properties count];
    dispatch_apply(count, _myQueue, ^(size_t index) {
        workBlock(_properties[index]);
    });
}

Higher-order messaging is basically the same thing but without the blocks. Clients could supply a selector or an invocation to apply to each object in the collection.

Visitor

Like higher-order messaging, Visitor is a pattern that lets clients apply code to the objects in a collection regardless of the implementation or even logical structure of the collection. It's particularly useful where client code is likely to need to maintain some sort of state across visits; a compiler front-end might expose a Visitor interface that clients use to construct indices or symbol tables.

- (void)acceptPropertyVisitor: (id &lt;NSEntityDescriptionVisitor&gt;)aVisitor
{
    for (NSPropertyDescription *property in _properties)
    {
        [aVisitor visit: property];
    }
}


Benefits of Hiding the Collection

By hiding the implementation of a collection behind its interface, you're free to change the implementation whenever you want. This is particularly true of the more abstract approaches like Enumerator, Fast enumeration, and Block Application, where clients only find out that there is a collection, and none of the details of whether it's indexed or not, sparse or not, precomputed or lazy and so on. If you started with an array but realise an indexed set or even an unindexed set would be better, there's no problem in making that change. Clients could iterate over objects in the collection before, they can now, nothing has changed&mdash;but you're free to use whatever algorithms make most sense in the application. That's a specific example of the Principle of Least Knowledge.

With the Key-Value Coding approach, you may be able to answer some questions in your class without actually doing all the work to instantiate the collected objects, such as the Key-Value Coding collection operators.

Additionally, there could be reasons to control use of the collected objects. For mutable collections it may be better to allow Block Application than let clients take an array copy which will become stale. Giving out the mutable collection itself would lead to all sorts of trouble, but that can be avoided just with a copy.

Benefits of Providing the Collection

It's easier. Foundation already provides classes that can do collections and all of the things you might want to do with them (including enumeration, KVC, and application) and you can just use those implementations without too much work: though notice that you can do that while still following the Fast Enumeration pattern.

It's also conventional. Plenty of examples exist of classes that give you a collection of things they're responsible for, like subviews, child view controllers, table view columns and so on (though notice that the table view previously gave an Enumerator of selected columns, and lets you Apply Block to row views). Doing the same thing follows a Principle of Least Surprise.

When integrating your components into an app, there may be expectations that a given collection is used in a context where a Foundation collection class is expected. If your application uses an array controller, you need an array: you could expect the application to provide an adaptor, or you could use Key-Value Coding and supply the proxy array, but it might be easiest to just give the application access to the collection in the first place.

Finally, the Foundation collection classes are abstract, so you still get some flexibility to change the implementation by building custom subclasses.

Conclusion

There isn't really a "best" way to do collections, there are benefits and limitations of any technique. By understanding the alternatives we can choose the best one for any situation (or be like NSEntityDescription and do a bit of each). In general though, giving interfaces to get or apply work to elements of the collection gives you more flexibility and control over how the collection is maintained, at the cost of being more work.

Coda

"More than one thing" isn't necessarily a collection. It could be better modelled as a source, that generates objects over time. It could be a cursor that shows the current value but doesn't remember earlier ones, a bit like an Enumerator. It could be something else. The above argument doesn't consider those cases.

#### On detecting God Classes
Opinion on Twitter was divided when I suggested the following static analyser behaviour: report on any class that conforms to too many protocols.

Firstly, a warning: "too many" is highly contextual. Almost all objects implement NSObject and you couldn't do much without it, so it gets a bye. Other protocols, like NSCoding and NSCopying, are little bits of functionality that don't really dilute a class by their presence. It's probably harmless for a class to implement those in addition to other protocols. Still others are so commonly implemented together (like UITableViewDataSource and UITableViewDelegate, or WebView's four delegate protocols) that they probably shouldn't independently count against a class's "protocol weight". On the other hand, a class that conforms to UITableViewDelegate, UIAlertViewDelegate and MKMapViewDelegate might be trying to do too much - of which more after the next paragraph.

Secondly, a truism: the goal of a static analyser is to ignore code that the developer's happy with, and to warn about code the developer isn't happy with. If your coding style permits a class to conform to any number of protocols, and you're happy with that, then you shouldn't implement this analyser rule. If you would be happy with a maximum of 2, 4, or 1,024 protocols, you would benefit from a rule with that number. As I said in my NSConf MINI talk, the analyser's results are not as cleanly definable as compiler errors (which indicate code that doesn't conform to the language definition) or warnings (which indicate code that is very probably incorrect). The analyser is more of a code style and API use checker. Conforming to protocols is use of the API that can be checked.

OK, let's get on with it. A class that conforms to too many protocols has too many reponsibilities - it is a "God Class". Now how can this come about? A developer who has heard about model-view-controller (MVC) will try to divide classes into three high-level groups, like this:

The problem comes when the developer fails to take that diagram for what it is: a 50,000-foot overview of the MVC architecture. Many Mac and iOS developers will use Core Data, and will end up with a model composed of multiple different entities. If a particular piece of the app's workflow needs to operate on multiple entities, they may break that out into "business logic" objects that can be called by the controller. Almost all Mac and iOS developers use the standard controls and views, meaning they have no choice but to break up the view into multiple objects.

But where does that leave the controller? Without any motivation to divide things up, everything else is stuffed into a single object (likely an NSDocument or UIViewController subclass). This is bad. What happens if you need to display the same table in two different places? You have to go through that class, picking out the bits that are relevant, finding out where they're tied to the bits that aren't and untangling them. Ugh.

Cocoa developers will probably already be using multiple controller objects if they're using Cocoa Bindings. Each NSArrayController or similar receives its object or objects, usually from the main controller object, and then gets on with the job of marshalling the interaction between the bound view and the observed model objects. So, if we take the proposed changes so far, our MVC diagram looks like this:

The point of my protocol-checking code is to go the remaining distance, and abstract out the other data sources into their own objects. What we're left with is a controller that looks after the view's use case, ensuring that logic actions take place when they ought, that steps in the workflow only become available when their preconditions are met, and so on. Everything related to performing the logic is down in those dynamic model objects, and everything to do with data presentation is in its own controller objects. Well, maybe not everything - a button doesn't exactly have a complicated API. But if you need a table of employees for this view controller and a table of employees for that view controller, you just take the same table datasource object in both places. You don't have two different datasource implementations in two view controllers (or even the same one pasted twice). This makes the diagram look like this:

So to summarise, a class that conforms to too many protocols probably has too many responsibilities. The usual reason for this is that controller objects take on responsibility for managing workflow, providing data to views and handling delegate responsibilities for the views. This leads to code that is not reusable except through the disdainful medium of copy-pasting, as it is harder to define a clean interface between these various concerns. By producing a tool that reports on the existence of such God classes, developers can be alerted to their presence and take steps to fix them.

#### On device identifiers
Note: as ever, this blog refrains from commenting on speculation regarding undisclosed product innovations from device providers. This post is about the concept of tracking users via a device identifier. You might find the discussion useful in considering future product directions; that's fine.

Keeping records of what users are up to has its benefits. As a security boffin, I'm fully aware of the benefits of good auditing: discovering what users (and others) have (or haven't) done to a valuable system. It also lets developers find out how users are getting on with their application: whether they ignore particular features, or have trouble deciding what to do at points in the app's workflow.

Indeed, sometimes users actively enjoy having their behaviour tracked. Every browser has a history feature; many games let players see what they've achieved and compare it with others. Location games would be pretty pointless if they could only tell me where I am now, not tell the story of where I've been.

A whole bunch of companies package APIs for tracking user behaviour in smartphone devices. These are the Analytics companies. To paraphrase Scott Adams: analytics is derived from the root word "anal", and the Greek "lytics" meaning "to pull a business model from". What they give developers for free is the API to put analytics services in their apps, and the tools to draw useful conclusions from these data.

This is where the fun begins. Imagine that the analytics company uses some material derived from a device identifier (a UDID, IMEI, or some other hardware key) as the database key to associate particular events with users. Now, if the same user uses multiple apps even by different developers on the same device, and they all use that analytics API, then that analytics provider can aggregate the data across all of the apps and build up a bigger picture of that user's behaviour.

If only one of the apps records the user's name as part of its analytics, then the analytics company - a company with whom the user has no relationship - gets to associate all of that behaviour with the user's real name. So, of course, do that company's customers: remember that the users and developers are provided with their stuff for free, and that businesses have limited tendency toward altruism. The value in an analytics company is their database, so of course they sell that database to those who will buy it: like advertisers, but again, companies with whom the user has no direct relationship.

People tend to be uneasy about invisible or unknown sharing of their information, particularly when the scope or consequences of such sharing are not obvious up front[*]. The level of identifiable[**] information and scope of data represented by a cross-app analysis of a smartphone user's behaviour - whether aggregated via the model described above or other means - is downright stalker-ish, and will make users uncomfortable.

One can imagine a scenario where smartphone providers try not to make their users uncomfortable: after all, they are the providers' bread and butter. So they don't give developers access to such a "primary key" as has been described here. Developers would then be stuck with generating identifiers inside their apps, so tracking a single user inside a single app would work but it would be impossible to aggregate the data across multiple apps, or the same app across multiple devices.

Unless, of course, the developer can coerce the user into associating all of those different identifiers with some shared identifier, such as a network service username. But how do you get users to sign up for network services? By ensuring that the service has value for the user. Look at game networks that do associate user activity across multiple apps, like OpenFeint and Game Center: they work because players like seeing what games their friends are playing, and sharing their achievements with other people.

The conclusion is, in the no-device-identifier world, it's still possible to aggregate user behaviour, but only if you exchange that ability for something that the user values. Seems like a fair deal.

[*] My "don't be a dick" guide to data privacy takes into account the fact that people like sharing information via online services such as Facebook, foursquare etc. but that they want to do so on their own terms. It goes like this: you have no right to anything except what the user told you. You have no right to share anything except what the user told you to share; they will tell you who you may share it with, and can change their minds. The user has a right to know what they're telling you and how you're sharing it.

[**] Given UK-sized postal zones, your surname and postcode are sufficient to uniquely identify you. Probably your birthday and postcode would also work. It doesn't take much information to uniquely identify someone, anyway.

#### On documentation
Over at the daily WTF, Alex Papadimoulis writes about Documentation Done Right. His conclusion is spot on:

The immediate answer to what’s the right way to do documentation is clear: produce the least amount of documentation needed to facilitate the most understanding, and be very explicit about which documentation is to be maintained and which is to be archived (i.e., read-only and left to rot).

The amount of documentation appropriate to any project depends very much on the project, and the people working on it. However I've found that there are some useful rough guides that can generally be applied. Like Alex, I'm ignoring user documentation here and talking about internal project artifacts.

Enterprise project documentation exists to give managers someone to blame.

The reason large projects seem so documentation-heavy is twofold: managers don't like to think they don't know what's going on; and managers like to be able to come down like a ton of shit on someone when they find out that they don't know what's going on. That's where the waterfall model comes into its own. The big up-front planning means that everyone on the project has signed off on the ridiculous unreadable project documentation of doom, so it must describe the best software possible. Whoever does something that isn't the same as the documentation has fucked up.

You never need the amount of documentation produced by most large-company software projects. Never. It's usually inaccurate even at the moment the it's signed off, because it took so long to get there the requirements changed, and because the level of precision required leads authors to make assumptions about how the OS, APIs etc. work. It's often very hard to work from, because there are multiple documents, poorly cross-referenced using support "tools" like Word, each with its own glossary depending heavily on terminology relevant to the author's domain. And they weren't written by the same author.

In order to code up a feature from one of these projects, you need to check the product requirements document to see what it's supposed to do (and whether it's the highest priority outstanding work - you also get to find out who asked for the feature, how much money it's worth, and plenty of other information that's of no use to a software engineer). You check the functional specification to see how it's supposed to do it. You check the architecture document to see what classes go in which packages. You raise a change control because the APIs don't support part of the functional specification. Two weeks later, a fix has been approved. You write the code, ensuring that you use the test plan to find out what the acceptance criteria are, and when they'll be tested (in fact, I've worked on projects where the functional and performance test plans were delivered at different times by different people). Oh, and make sure you're keeping up to date with the issue tracker, otherwise you might do work that's been assigned to someone else. Your systems engineer can point to the process documentation to let you know how you do that.

Good documentation doesn't answer all the questions, but leaves you capable of asking smart questions of the right people.

It's no secret that I'm a fan of user stories as a form of requirements documentation. A well-written user story tells you what the user expects to be able to do after a feature has been added, and precious little else. There might be questions over terminology, or specifics such as failure cases, but you know who to ask - the person who wrote the user story. There will be nothing about architecture or the GUI layout, because those things aren't up to the user, customer or product manager (or at least they shouldn't be).

Similarly, a good architecture diagram is going to tell you something about how the classes in an application fit together, and precious little else. If you can't work out how to fit your feature in, you can ask the architect, but you'll both be able to use the diagram as a good place to start. As Alex says in his post, precise documentation will go out of date quickly, so the documentation needs to be good enough to hang discussions on, and no "better".

UML diagrams are great to describe code before it's been written; they're not so great at writing code.

If you're trying to explain to another engineer how you think a feature should work in code, what design pattern to follow, or what steps will be needed to talk to a server, then a UML diagram is a great way to do it. Many engineers understand UML, and those who don't can work out what it means (with your help) very quickly. It's a consistent language for talking about software.

That said, when I draw UML diagrams I tend to prefer whiteboards and agnostic diagramming software like Omnigraffle or dia over syntax-checking UML tools like Enterprise Architect or ArgoUML. The reason is that I gave in the last section: the diagram needs to be good enough to explain to someone else what we're going to do, not a gold-plated example of conformance to the UML specification. I don't care if my swim-lane is in the wrong place if everybody involved knows what I mean (or can ask).

Code generated from UML tools tends to have readability issues. Whereas you or I might put related methods together, the tool might sort them by visibility or name. If you're diagram is rough enough to be useful, then the tool will only generate a few method stubs - and you'll be tempted to fill them in rather than creating small private methods to do logical units of work. These and other problems (I've seen EA generate code that uses UUIDs as class identifiers) can be fixed, but you shouldn't need to fix them. You should write your application and sell it.

After code has been written, the only accurate documentation is the code itself (or generated from it).

There are other useful forms of documentation - for instance, a whiteboard diagram explains a developer's understanding of the code. It doesn't document the code itself - an important distinction.[*] All of those shiny enterprise documents your project manager got you to write went out of date as soon as the first customer reported a bug or feature request. The code, however, documents exactly what the code does - just not necessarily in the most useful way. Javadoc/doxygen comments are more likely than any thing else to stay in sync with the code due to their proximity, but even those can be outdated or unhelpful.

This is where UML tools can come in very handy. Those with the ability to generate diagrams based on code (even Xcode does this) can automatically give you a correct view of the application's behaviour, at a more appropriate level of abstraction than the code itself. If what you need is a package dependency diagram, it's better to get it from a UML tool than to try and read all of the source.

Unfortunately, Xcode (and Doxygen's very limited capabilities) are the only games in town for Objective-C. Tool support for Java and C# is way ahead, but for Cocoa developers there's only MacTranslator (which I've never tried). Not that UML maps particularly well onto Objective-C anyway.

[*]Though documenting a developer's assumptions and comparing them with reality can often explain where some subtle bugs come from, and is of course useful.

The better your prototypes, the worse the feedback.

Back in the 1990s there was an explosion of Rapid Application Development tools, after the rest of the software industry saw Interface Builder and decided it was good. The RAD way (which is, of course, eminently achievable using IB) is to produce an executable prototype for users to give feedback on. Of course, the problem is that you end up shipping the prototype. I've actually ended up doing that on a couple of Mac applications.

One problem is that because the app looks complete, users assume it is complete and that they're being asked to provide polishing details, or spot spelling mistakes and misplaced buttons. The other is that because it looks complete, managers assume it is complete and will tell you to ship it.

Don't make that mistake. Do UI prototypes as paper-based wireframes, Keynote presentations or Cappuccino apps. Whatever you do, make it look like it's just a crappy sketch you're willing to have ripped to shreds. That way, people will rip it to shreds.

There are few document "artifacts" that need to hang around.

If you think about what you're going to do with an application after you've written it, there's selling it, supporting it, maintaining it, and extending it. Support people might need some high-level architecture knowledge so that they can work out what component a problem is in, or how to diagnose a particular failure.

Such an architecture document can be a good aid for planning new features or bug fixes, because you can quickly see where you need to modify the app to get the desired behaviour (clearly you then need to dive into the code to get a better idea, but you know where to jump). Similarly, architecture rationale documentation (including the threat model, API/library use justification) can be handy so that you don't need to go through the same debates/research when fixing a bug or adding a feature. Threat models particularly can take a lot of time and expertise to construct from scratch.

Sales people will need to know which features have been delivered, which features are on the roadmap, and can probably find out specific questions from engineers if they get a grilling from an awkward customer. Only in a very limited set of circumstances will sales staff need to give customers security documentation, test coverage information, or anything other than the user manual.

Notice that in each of these cases, one of the main aspects of keeping the document around is so that you can keep it up to date. There's no point having the 1.0 feature list when you're selling version 2.5 - in fact it would be positively detrimental. So obviously the fewer documents you keep around, the fewer you need to keep up to date. There's some trade-off involved (natürlich) - if you need something you didn't hang on to, you're going to have to regenerate it.

#### On dynamic vs. static polymorphism
An interesting juxtaposition in the ACCU 2009 schedule put my talk on "adopting MVC in Objective-C and Cocoa" next to Peter Sommerlad's talk on "Design patterns with modern C++". So the subject matter in each case was fairly similar, but then the solutions we came up with were entirely different.One key factor was that Peter's solutions try to push all of the "smarts" of a design pattern into the compiler, using templates and metaprogramming to separate implementations from interfaces. On the other hand, my solutions use duck typing and dynamic method resolution to push all of the complexity into the runtime. Both solutions work, of course. It's also fairly obvious that they're both chosen based on the limitations and capabilities of the language we were each using. Nonetheless, it was interesting that we both had justifications for our chosen (and thus One True) approach.In the Stroustroup corner, the justification is this: by making the compiler resolve all of the decisions, any problems in the code are resolved before it ever gets run, let alone before it gets into the hands of a user. Whereas the Cox defence argues that my time as a programmer is too expensive to spend sitting around waiting for g++ to generate metaprogramming code, so replace the compilation with comparitively cheap lookups at runtime - which also allows for classes that couldn't have possibly existed at compiletime, such as those added by the Python or Perl bridge.This provided concrete evidence of a position that I've argued before - namely that Design Patterns are language-dependent. We both implemented Template Method. Peter's implementation involved a templatized abstract class which took a concrete subclass in the realisation (i.e. as the parameter in the &lt;T&gt;). My implementation is the usual Cocoa delegate pattern - the "abstract" (or more correctly undecorated) class takes any old id as the delegate, then tests whether it implements the delegation sequence points at runtime. Both implement the pattern, and that's about where the similiarities end.

#### On entering programming in 2022
I recently taught an introduction to Python course, to final-year undergraduate students. These students had little to zero programming experience, and were all expected to get set up with Python (using the Anaconda environment, which we had determined to be the easiest way to get a reasonable baseline configuration) on laptops they had brought themselves.

What follows is not a slight on these people, who were all motivated, intelligent, and capable. It is a slight on the world of programming in the current ages, if you are seeking to get started with putting a general-purpose computer to your own purposes and merely own a general-purpose computer.

One person had a laptop that, being a mere six (6) years old, was too old to run the current version of Anaconda Distribution. We had to crawl through the archives, guessing what older version might work (i.e. might both run on their computer and still permit them to follow the course).

Another had a modern laptop and the same same version of Python/tools that everyone else was using, except that their IDE would crash if they tried to plot a graph in dark mode.

Another had, seemingly without having launched the shell while they owned their computer, got their profile into a state where none of the system binary folders were on their PATH. Hmm, python3 doesn't work, let's use which python to find out why not. Hmm, which doesn't work, let's use ls to find out why not. Hmmm…

Many, through not having used terminal emulators before, did not yet know that terminal emulators are modal. There are shell commands, which you must type when you see a $ (or a % or a &gt;) and will not work when you can see a &gt;&gt;&gt;. There are Python commands, which are the other way around. If you type a command that launches nano/pico, there are other rules.

By the way, condo and pip (and poetry, if you try to read anything online about setting up Python) are Python things but you cannot use them as Python commands. They are shell commands.

By the other way, everyone writes those shell commands with a $ at the front. You do not write the $. Oh, and by the other other way: they don't necessarily tell you to open the Terminal to do it.

Different environments—the shell, visual studio code, Spyder, PyCharm—will do different things with respect to your “current working directory” when you run a script. They will not tell you that they have done this, nor that it is important, nor that it is why your script can't find a data file that's RIGHT THERE.

This is all way before we get to the dark art of comprehending exception traces.

When I were a lad and Silicon Valley were all fields, you turned a computer on and it was ready for some programming. I'm not suggesting returning to that time, computers were useless then. But I do think it is needlessly difficult to get started with “a programming language that lets you work quickly” in this time of ubiquitous programs.

#### On explaining stuff to people
An article that recently made the rounds, though it was written back in September, is called Apple's Idioten Vektor. It's a discussion of how the CCCrypt() function in Apple's CommonCrypto library, when used in its default cipher block chaining mode, treats the IV (Initialization Vector) parameter as optional. If you don't supply an IV, it provides its own IV of 0x0.

Professional Cocoa Application Security also covers CommonCrypto, CBC mode, and the Initialization Vector. Pages 79-88 discuss block encryption. The section includes sample code for both one-shot and staged use of the API. It explains how to set the IV using a random number generator, and why this should be done.[1] Mercifully when the author of the above blog post reviewed the code in my book section, he decided I was doing it correctly.

So both publications cover the same content. There's a clear difference in presentation technique, though. I realise that the blog post is categorised as a "rant" by the author, and that I'm about to be the pot that calls the kettle black. However, I do not believe that the attitude taken in the post—I won't describe it, you can read it—is constructive. Calling people out is not cool, helping them get things correct is. Laughing at the "fail" is not something that endears people to us, and let's face it, security people could definitely be more endearing. We have a difficult challenge: we ask developers to do more work to bring their products to market, to spend more money on engineering (and often consultants), in return for potentially protecting some unquantified future lost revenue and customer hardship.

Yes there is a large technical component in doing that stuff, but solving the above challenge also depends very strongly on relationship management. Security experts need to demonstrate that we're all on the same side; that we want to work with the rest of the software industry to help make better software. Again, a challenge arises: a lot of the help provided by security engineers comes in the form of pointing out mistakes. But we shouldn't be self promoting douchebags about it. Perhaps we're going about it wrong. I always strive to help the developers I work with by identifying and discussing the potential mistakes before they happen. Then there's less friction: "we're going to do this right" is a much more palatable story than "you did this wrong".

On the other hand, the Idioten Vektor approach generated a load of discussion and coverage, while only a couple of thousand people ever read Professional Cocoa Application Security. So there's clearly something in the sensationalist approach too. Perhaps it's me that doesn't get it.

[1]Note that the book was written while iPhone OS 3 was the current version, which is why the file protection options are not discussed. If I were covering the same topic today I would recommend eschewing CCCrypto for all but the most specialised of purposes, and would suggest setting an appropriate file protection level instead. The book also didn't put encryption into the broader context of cryptographic protocols; a mistake I have since rectified.

#### On exploding boilers
Throughout our history, it has always been standardisation of components that has enabled creations of greater complexity.
This quote, from Simon Wardley’s finding a path, reminded me of the software industry’s relationship with interchangeable parts.

Brad Cox, in both Object-Oriented Programming: an Evolutionary Approach and Superdistribution, used physical manufacturing analogies (to integrated circuits, and to rifles) to invoke the concept of a “software industrial revolution” that would allow end users to assemble off-the-shelf parts to solve their problems. His “software ICs” built on ideas expressed at least as early as 1968 by Doug McIlroy. Joe Armstrong talked about a universal function registry, so that if someone writes sin/1 everybody else can use it.

Of course we have a lot of reusable components in software engineering now, and we can thank the Free Software movement at least as much as any paradigm of organising programming instructions. CTAN, CPAN, and later repositories act as the “component catalogues” that Cox discussed. If you want to make a computer do something, you can probably find an npm module or a Ruby gem that does most of the work for you. The vast majority of such components have free licenses, it's rare to pay for a reusable component.

The extent to which they're "standard parts", on the model of interchangeable nuts and bolts or integrated circuits, is debatable. Let's say that you download a package from the NPM. We know that you use it by calling require (or maybe import)…but what does that give you? An object? A constructor? A regular function? Does it run anything as a result of calling require? Does it work in your node/ionic/electron/etc. context? Is it even a lump of regular javascript, or is it a Real, or to have access to a JVM, or some other niche requirement?

Whatever these "standard parts" and however they're used, you're probably still doing a bunch of coding. These parts will do computery stuff, or maybe generic behaviour like authentication, date UIs, left-padding strings and the like. Usually we still have to develop ours apps as "engineered" software projects with significant levels of custom coding, to make those "standard parts" actually solve a useful problem. There are still people working for retail companies maintaining online store applications across the four corners of the globe, despite the fact that globes don't have corners, these things all work the same way, and the risks associated with getting them wrong are significant.

Perhaps this is because software is a distinct thing, and we can never treat it like industrial product manufacturing.

Perhaps this is because our ambition always runs out ahead of our capability. Whatever we can reproducibly build, we'd like to be building something greater.

Perhaps this is because we're still in the cottage industry stage, where we don't yet know whether or how to standardise the parts, and occasionally the boilers explode.

#### On Fitt's Law and Security
...eh? Don't worry, read on and all shall be explained.

I've said in multiple talks and podcasts before that one key to good security is good user interface design. If users are comfortable performing their tasks, and your application is designed such that the easiest way to use it is to do the correct thing, then your users will make fewer mistakes. Not only will they be satisfied with the experience, they'll be less inclined to stray away from the default security settings you provide (your app is secure by default, right?) and less likely to damage their own data.

Let's look at a concrete example. Matt Gemmell has already done a pretty convincing job of destroying this UI paradigm:

The button on the left adds something new, which is an unintrusive, undoable operation, and one that users might want to do often. The button on the right destroys data (or configuration settings or whatever), which if unintended represents an accidental loss of data integrity or possibly service availability.

Having an undo feature here is not necessarily a good mitigation, because a user flustered at having just lost a load of work or important data cannot be guaranteed to think of using it. Especially as undo is one of those features that has different meaning in different apps, and is always hidden (there's no "you can undo this" message shown to users). Having an upfront confirmation of deletion is not useful either, and Matt's video has a good discussion of why.

Desktop UI

Anyway, this is a 19x19 px button (or two of them), right? Who's going to miss that? This is where we start using the old cognometrics and physiology. If a user needs to add a new thing, he must:

Think about what he wants to do (add a new thing)
Think about how to do it (click the plus button, but you can easily make this stage longer by adding contextual menus, a New menu item and a keyboard shortcut, forcing him to choose)
Move his hand to the mouse (if it isn't there already)
Locate the mouse pointer
Move the pointer over the button
Click the mouse button

Then there's the cognitive effort of checking that the feedback is consistent with the expected outcome. Anyway, Fitt's Law comes in at step 5. It tells us, in a very precise mathematical way, that it's quicker to move the pointer to a big target than a small one. So while a precise user might take time to click in this region:

A user in a hurry (one who cares more about getting the task done than on positioning a pointing device) can speed up the operation by clicking in this region:

The second region includes around a third of the remove button's area; the user has a ~50% chance of getting the intended result, and ~25% of doing something bad.

Why are the click areas elliptical? Think about moving your hand - specifically moving the mouse in your hand (of course this discussion does not apply to trackpads, trackballs etc, but a similar one does). You can move your wrist left/right easily, covering large (in mousing terms) distances. Moving the mouse forward/back needs to be done with the fingers, a more fiddly task that covers less distance in the same time. If you consider arm motions the same applies - swinging your radius and ulna at the elbow to get (roughly) left/right motions is easier than pulling your whole arm at the shoulder to get forward/back. Therefore putting the remove button at one side of the add button is, from an ergonomic perspective, the absolutely worst place for it to go.

Touch UI
Except on touch devices. We don't really need to worry about the time spent moving our fingers to the right place on a touchscreen - most people develop a pretty good unconscious sense of proprioception (knowing where the bits of our bodies are in space) at a very early age and can readily put a digit where they want it. The problem is that we don't want it in the same place the device does.

We spend most of our lives pointing at things, not stabbing them with the fleshy part of a phalange. Therefore when people go to tap a target on the screen, the biggest area of finger contact will be slightly below (and to one side, depending on handedness) the intended target. And of course they may not accurately judge the target's location in the first place. In the image below, the user probably intended to tap on "Presentations", but would the tap register there or in "LinkedIn Profile"? Or neither?

The easiest way to see this for yourself is to move from an iPhone (which automatically corrects for the user's intended tap location) and an Android device (which doesn't, so the tap occurs where the user actually touched the screen) or vice versa. Try typing on one for a while, then moving over to type on the other. Compare the sorts of typos you make on each device (obviously independent of the auto-correct facilities, which are good on both operating systems), and you'll find that after switching devices you spend a lot of time hitting keys to the side of or below your target keys.

Conclusion

Putting any GUI controls next to each other invites the possibility that users will tap the wrong one. When a frequently-used control has a dangerous control as a near neighbour, the chance for accidental security problems to occur exists. Making dangerous controls hard to accidentally activate does not only provide a superior user experience, but a superior security experience.

#### On free apps
This post is sort-of a follow-on to @daveaddey's post on the average app; although in reality it's a follow-on to the response that comes out every time a post on app store revenue is written.

Events go like this:

Some statistic about app store revenue.
"Your numbers include free apps. You shouldn't include free apps.

Yes you should. The revenue that comes from the app store is indeed shared across all apps, free and paid. Free apps contribute significantly to the long-tail price distribution of apps on the store, and the consumer perception that apps shouldn't cost much. Some of them generate revenue through in-app purchase: revenue that probably is counted in Apple's "we've given $5bn to developers" number. Some of them will generate revenue through iAd: it's not clear whether that's included in the $5bn but it's certainly money paid by Apple to developers. Similarly, it's not clear whether money paid through Newsstand subscriptions (again, in free apps) counts: it probably does. Some of them have not always and will not always be free; again these apps make money directly from Apple.

A lot of free apps come from companies that do other things, but feel a marketing need to be on the app store:Amazon, O2, facebook and others go down this route. In these cases there is absolutely no money to be made from the app directly, though there are possibly many sorts of collateral benefits. It costs Amazon money to write the Windowshop app, but they bank on users buying more things from them than if the apps doesn't exist.

On the other hand, some free apps are just written by developers who want to put a free app on the store so that it can act as their portfolio when they try to get work as iOS app developers.

So yes, other business models exist. But when talking about money made from the app store, you have to include all of those products that don't make money on the app store. Ignoring the odd outlier is fine statistics. Ignoring large quantities of data that make your conclusions look bad is not science; it's witchcraft.

#### On free Mac Anti-Virus
On Tuesday, my pals at my old stomping ground Sophos launched their Free home edition Mac product. I've been asked by several people what makes it tick, so here's Mac Anti-Virus In A Nutshell.


What is the AV doing?

So anti-virus is basically a categorisation technology: you look at a file and decide whether it's bad. The traditional view people have of an AV engine is that there's a huge table of file checksums, and the AV product just compares every file it encounters to every checksum and warns you if it finds a match. That's certainly how it used to work around a decade ago, but even low-end products like ClamAV don't genuinely work this way any more.

Modern Anti-Virus starts its work by classifying the file it's looking at. This basically means deciding what type of file it is: a Mac executable, a Word document, a ZIP etc. Some of these are actually containers for other file types: a ZIP obviously contains other files, but a Word document contains sections with macros in which might be interesting. A Mac fat file contains one or more executable files, which each contains various data and program segments. Even a text file might actually contain a shell script (which could contain a perl script as a here doc), and so on. But eventually the engine will have classified zero or more parts of the file that it wants to inspect.

Because the engine now knows the type of the data it's looking at, it can be clever about what tests it applies. So the engine contains a whole barrage of different tests, but still runs very quickly because it knows when any test is necessary. For example, most AV products now including Sophos' can actually run x86 code in an emulator or sandbox, to see whether it would try to do something naughty. But it doesn't bother trying to do that to a JPEG.

That sounds slow.

And the figures seem to bear that out: running a scan via the GUI can take hours, or even a day. A large part of this is due to limitations on the hard drive's throughput, exacerbated by the fact that there's no way to ask a disk to come up with a file access strategy that minimises seek time (time that's effectively wasted while the disk moves its heads and platters to the place where the file is stored). Such a thing would mean reading the whole drive catalogue (its table of contents), and thinking for a while about the best order to read all of the files. Besides, such strategies fall apart when one of the other applications needs to open a file, because the hard drive has to jump away and get that one. So as this approach can't work, the OS doesn't support it.

On a Mac with a solid state drive, you actually can get to the point where CPU availability, rather than storage throughput, is the limiting factor. But surely even solid state drives are far too slow compared with CPUs, and the Anti-Virus app must be quite inefficient to be CPU-limited? Not so. Of course, there is some work that Sophos Anti-Virus must be doing in order to get worthwhile results, so I can't say that it uses no CPU at all. But having dealt with the problem of hard drive seeking, we now meet the UBC.

The Unified Buffer Cache is a place in memory where the kernel holds the content of recently accessed files. As new files are read, the kernel throws away the contents of old files and stores the new one in the cache. Poor kernel. It couldn't possibly know that this scanner is just going to do some tests on the file then never look at it again, so it goes to a lot of effort swapping contents around in its cache that will never get used. This is where a lot of the time ends up.

On not wasting all that time

This is where the on-access scanner comes in. If you look at the Sopohs installation, you'll see an application at /Library/Sophos Anti-Virus/InterCheck.app - this is a small UNIX tool that includes a kernel extension to intercept file requests and test the target files. If it finds an infected file, it stops the operating system from opening it.



To find out how to this interception, you can do worse than look at Professional Cocoa Application Security, where I talk about the KAUTH (Kernel AUTHorisation) mechanism in Chapter 11. But the main point is that this approach - checking files when you ask for them - is actually more efficient than doing the whole scan. For a start, you're only looking at files that are going to be needed anyway, so you're not asking the hard drive to go out of its way and prepare loads of content that isn't otherwise being used. InterCheck can also be clever about what it does, for example there's no need to scan the same file twice if it hasn't changed in the meantime.

OK, so it's not a resource hog. But I still don't need anti-virus.

Not true. This can best be described as anecdotal, but all of the people who reported to me that they had run a scan since the free Sophos product had become available, around 75% reported that it had detected threats. These were mainly Windows executables attached to mail, but it's still good to detect and destroy those so they don't get onto your Boot Camp partition or somebody else's PC.

There definitely is a small, but growing, pile of malware that really does target Macs. I was the tech reviewer for Enterprise Mac Security, for the chapter on malware my research turned up tens of different strains: mainly Trojan horses (as on Windows), some OpenOffice macros, and some web-based threats. And that was printed well before Koobface was ported to the Mac.

Alright, it's free, I'll give it a go. Wait, why is it free?

Well here I have to turn to speculation. If your reaction to my first paragraph was "hang on, who is Sophos?", then you're not alone. Sophos is still a company that only sells to other businesses, and that means that the inhabitants of the Clapham Omnibus typically haven't heard of them. Windows users have usually heard of Symantec via their Norton brand, McAfee and even smaller outfits like Kaspersky, so those are names that come up in the board room.

That explains why they might release a free product, but not this one. Well, now you have to think about what makes AV vendors different from one another, and really the answer is "not much". They all sell pretty much the same thing, occasionally one of them comes up with a new feature but that gap usually closes quite quickly.

Cross-platform support is one area that's still open, surprisingly. Despite the fact that loads of the vendors (and I do mean loads: Symantec, McAfee, Trend Micro, Sophos, Kaspersky, F-Secure, Panda and Eset all spring to mind readily) support the Mac and some other UNIX platforms, most of these are just checkbox products that exist to prop up their feature matrix. My suspicion is that by raising the profile of their Mac offering Sophos hopes to become the cross-platform security vendor. And that makes giving their Mac product away for free more valuable than selling it.

#### On Fuzzy Aliens
I have just launched a new company, Fuzzy Aliens[*], offering application security consultancy services for smartphone app developers. This is not the FAQ list, this is the "questions I want to answer so that they don't become frequently asked" list.

What do you offer?

The company's services are all focussed on helping smartphone and tablet app developers discover and implement their applications' security and privacy requirements. When planning an app, I can help with threat modelling, with training developers, securing the development lifecycle, requirements elicitation, secure user experience design, and with developing a testing strategy.

When it comes to implementation, you can hire me to do the security work on your iOS or Android app. That may be some background "plumbing" like storing a password or encrypting sensitive content, or it might be an end-to-end security feature. I can also do security code reviews and vulnerability analysis on existing applications.

Why would I want that?

If you're developing an application destined for the enterprise market, you probably need it. Company I.T. departments will demand applications that conform to local policy regarding data protection, perhaps based on published standards such as the ISO 27000 family or PCI-DSS.

In the consumer market, users are getting wise to the privacy problems associated with mobile apps. Whether it's accidentally posting the wrong thing to facebook, or being spied on by their apps, the public don't want to&mdash;and shouldn't need to&mdash;deal with security issues when they're trying to get their work done and play their games.

Can I afford that?

Having been a Micro-ISV and contracted for others, I know that many apps are delivered under tight budgets by one-person companies. If all you need is a half day together to work on a niggling problem, that's all you need to pay for. On the other hand I'm perfectly happy to work on longer projects, too :).

Why's it called Fuzzy Aliens?

Well, the word "fuzz" obviously has a specific meaning in the world of secure software development, but basically the answer is that I knew I could turn that into a cute logo (still pending), and that it hadn't been registered by a UK Ltd yet.

So how do I contact you about this?

You already have - you're here. But you could see the company's contact page for more specific information.

[*] More accurately, I have indicated the intent to do so. The articles of association have not yet been returned by Companies House, so for the next couple of days the blue touch paper is quietly smouldering.

#### On having things to say
I enjoyed Jaimee's discussion of preparing her public talks, and realised that my approach has moved in a different way. I've probably talked about this before but I've also changed how I go about it. This is my technique, particularly where it diverges from Jaimee's; synthesis can come later (and will undoubtedly help me!).

I start by thinking up some pithy title: previous talks including "Object-Oriented Programming in Objective-C", "By your _cmd", "The Principled Programmer" and "I have no idea what I'm doing" all began there. I often commit—even if only privately—to using a particular title before I have any idea what the talk will be about. I enjoy the creative exercise of fitting the rest of the talk into that constraint!

With a title in place, I brainstorm all of the things I can think of that could potentially fit into that topic. Usually I look back at that brainstorm and discover that it's rambling, disconnected and mostly boring. Looking through, I search for two or three things that are interesting, particularly if they suggest conflicting ideas or techniques that can be explored, challenged and resolved.

Then it's time for another outline :). This one explores the selected areas in depth, and it's from this that I pick the main headlines for the talk, which also shape the introduction and conclusion. With those in mind I write the talk out as an essay, making sure it is consistent, complete and (to the extent I can do this myself) interesting. If it looks OK, then by this point I've prepared so much that I can remember the flow of the talk and give it without aids, though I still look for opportunities to support the presentation visually in the slides. In the case of my Principled Programmer talk, I realised the slides weren't helping at all so did without them.

There are plenty of better presenters than me in the world; Jaimee is one of them. I have merely trial-and-errored my way into a situation where sometimes the same people who see me talk ask me back. I hope that by comparing my method with Jaimee's and those of other people I can find out how to prepare a better talk.

#### On home truths in iOS TDD
The first readers of Test-Driven iOS Development (currently available in Rough Cuts form on Safari Books Online: if you want to buy a paper/kindle/iBooks editions, you'll have to wait until it enters full production in a month or so) are giving positive feedback on the book's content, which is gratifying. Bar last minute corrections and galley proof checking, my involvement with the project is nearly over so it's time for me to reflect on the work that has dominated my schedule for over a year.

As explained in the book's front matter, I chose to give all of the examples in the book and accompanying source code using OCUnit. As the BBC might say, "other unit test frameworks are available". Some of the alternative frameworks are discussed in the book, so interested readers can try them out for themselves.

What made OCUnit the correct choice—put it a different way, what made OCUnit the choice I made? It's the framework that's shipped with Xcode, so anyone who might want to try out unit testing can pick up the book and give it a go. There are no third-party dependencies to become unsupported or change beyond all recognition—though that does occasionally happen to Xcode. File-New Project…, include unit tests, and you're away, following the examples and trying out your own things.

Additionally, the shared body of knowledge in the Cocoa development community is greatest when it comes to OCUnit. Aside from people who consider automated testing to be teh suck, plenty of developers on Mac, iOS and other platforms have got experience using OCUnit or something very much like it. Some of those people have switched to other frameworks, but plenty are using OCUnit. There's plenty of experience out there, and plenty of help available.

The flip side to this is that OCUnit doesn't represent the state of the art in testing. Far from it: the kit was first introduced in 1998, and hasn't changed a great deal since. Indeed many of the alternatives we see in frameworks like GHUnit and Google Toolkit for Mac are really not such great improvements, adding some extra macros and different reporting tools. Supporting libraries such as OCHamcrest and OCMock give us some additional features, but we can look over the fence into the neighbouring fields of Java, ruby and C# to see greater innovations and more efficient testers.

Before you decide to take the book out of your Amazon basket, let me assure you that learning TDD via OCUnit is not wasted effort. The discipline of red-green-refactor, the way that writing tests guides the design of your classes, the introduction of test doubles to remove dependencies in tests: these are all things that (I hope) the book can teach you, and that you can employ whether you use OCUnit or some other framework. And, as I said, there's plenty of code out there that is in an OCUnit harness. It's not bad, it could be better.

So what are the problems with OCUnit?

	repetition. Every time you write STAssert, you're saying two things. Firstly, "hey, I'm using OCUnit", which isn't really useful information. Second, "what's coming up is a test, read on to find out what kind of test". Then you finally get to the end of the macro where you reveal what it is you're going to do. This is the important information, but we bury it in the middle of the line behind some boilerplate.

Imagine, instead, a hypothetical language where we could send messages to arbitrary expressions (ok that exists, but imagine it's objc). Then you could do [[2+2 should] equal: 4]; which more closely reflects our intention.

	repetition. In the same way that STAssert is boilerplate, so is sub classing SenTestCase and writing -(void)test at the beginning of every test method. It gives you no useful information, and hides the actual data about the test behind the boilerplate.

Newer test frameworks in languages like C# and Java use the annotation features of those languages to take the fact that a method is a test out of its signature and make it metadata. ObjC doesn't support annotations, so we can't do that. But take a look at the way CATCH tests are marked up. You indicate that something is a test, and the fact that this means the framework needs to generate an objective-c++ class and call a method on it is encapsulated in the framework's implementation.

	repetition. You might think that there's a theme developing here :-). If you write descriptive method names, you might have a test named something like -testTheNetworkConnectionIsCleanedUpWhenADownloadFails. Should that test fail, you're told what is going wrong: the network connection is not cleaned up when a download fails.

So what should you write in the mandatory message parameter all of the STAssert…() macros require? How about @"the network connection was not cleaned up when a download failed"? Not so useful.

	organisation. I've already discussed how OCUnit makes you put tests into particular classes and name them in particular ways. What if you don't want to do that? What if you want to define multiple groups of related tests in the same class, in the way BDD practitioners do to indicate they're all part of the same story? What if you want to group some of the tests in one of those groups? You can't do that.

I'm sure other people have other complaints about OCUnit, and that yet other people can find no fault with it. In this post I wanted to draw attention to the fact that there's more than one way to crack a nut, and the vendor-supplied nutcracker is useful though basic.

#### On how to get crypto wrong
I've said time and time again: don't write your own encryption algorithm. Once you've chosen an existing algorithm, don't write your own implementation.

Today I had to look at an encryption library that had been developed to store some files in an app. The library used a custom implementation of SHA256-HMAC, and a custom implementation of CBC mode. The implementations certainly looked OK, and seemed to match the descriptions in the textbooks. They also seem to work - you can encrypt a file to get gibberish, and decrypt the gibberish to get the file back.

So the first thing I did was to crack open Xcode and replace these custom functions with CommonCrypto. CommonCrypto's internals also look a lot like the textbook descriptions of the methods, too. So it would be surprising if these two approaches yielded different results.

These two approaches yielded different results. This was surprising. Specifically, I found that the CBC implementation would sometimes use junk memory, which the CommonCrypto version never does. Of course, the way in which this junk was used was predictable enough that the encryption routine was still reversible - but could it be that the custom implementation was leaking information about the plaintext in the cipher-text by inappropriate re-use of the buffer? Possibly, and that's good enough for me to throw the custom implementation out. Proving whether or not this implementation is "safe" is something that a specialist cryptographer could probably do in half a day. However, as I was able to use half a day to produce something I had more confidence in, just by using a tested implementation, I decided there was no need to do that work.

#### On Ignoring the Tests
As mentioned over two months ago, I'll be giving two talks this weekend at the Voices That Matter: iPhone Developers Fall conference. I'm feeling good about both of the talks that I've worked on, though I definitely think the Unit Testing talk on Saturday is better polished. That's probably at least in part because of the work put in by everyone's favourite Apple Employee, who gave me a great base for the talk.

One of the (myriad) problems I'm currently facing is related to unit testing, in a project I've recently started working on. The existing code base works well but, how can we put this delicately, it smells. There aren't any important bugs, but when you come to extend some functionality or add a new feature you find yourself trying to piece together code from numerous classes in different libraries, before you can tell the story of how the bit you're trying to change works. You then load the shotgun with the lead shot of enhanced featurity, which is liberally spattered across the code base.

There are various refactoring exercises that can be brought to bear on code such as this, or there could be if it weren't for the unit tests. Some of the tests fail. Sometimes you look and find a hard-coded path to a file that can quickly be changed to get us going again. Sometimes you see a bit of expected value drift. Every so often, you find that the class under test and the test suite bear no resemblance to one another. Why are these tests failing, why did nobody notice, and what is the behaviour of the code under test?

The fact is usually that at some point, someone needed to make a change to the application and, believing it to be a "quick fix", decided it wasn't worth doing TDD for the new code. Whether or not that was a good decision, the new code had to interface with the old code somewhere and it changed the behaviour of that old code. Tests started failing. The developer decided that the test should be ignored. Worst. Decision. Ever.

Ignoring a test means leaving it in the code, but marking it up in some way such that the test runner either doesn't execute the test or doesn't care if it fails. Concrete example: in NUnit, you can set the [Ignore] attribute on a test method and it won't get run. You can't actually do this in OCUnit, though you can get per-class ignorance by just removing the class from the test bundle target.

The problem with ignored tests is that they don't look much different from non-ignored tests (sometimes, they can look identical, if the ignorance is provided by a parameters file rather than the source code). They give the impression that the code is under test. They give the impression that the code should work in the way specified in the test. They give the impression, if you run the tests and they fail, that you broke something. Turning off a test is worse than useless - just delete the thing. That way it's obvious that the code it was testing is no longer under test. And don't worry, it's not gone forever - you could always retrieve the deleted code from version control. You never will, but you could.

"But," pipe up the authors of the xUnit ignore features, "that's not what the ignore feature is for. You write tests for code that doesn't exist yet, and set them ignored so they don't break the build. You get the utility of annotating your code with tests, without the red bar."

Unfortunately it doesn't work like that. I write the tests now, and set them to be ignored. Some time later, the feature gets cancelled. The ignored tests hang around, bearing less and less resemblance to the real application.

Or I write the tests now, and set them to be ignored. Some time later, somebody else implements the feature, writing their own tests (or not). The ignored tests hang around, looking uncomfortably like they might be testing the feature.

Now using unit tests as a documentation and communication tool is useful and beneficial, and I'll talk about that at the weekend. But only commit tests for code that either got written with the tests, or will be written imminently. If it looks like a test is no longer relevant, rewrite it or delete it. But do not tell your test runner to ignore it, because your developers probably won't.

#### On immutable data structures…?
If you write a scholarly publication and cite another one, what you say about it depends on its mutability. An article or a book can be cited by saying "this publication I'm identifying here says this". Maybe you have to version your claim: "the second edition of this publication says this". They're immutable. Even if the third edition doesn't say the thing you relied on in constructing your argument, the second edition still did. Someone who can get access to that second edition can look at it and see how you built your synthesis.

You can't do that with a website. Websites change. Instead, you have to say that "this website identified by this URL, on the date that I read it, said this". Someone who comes along later has to sort-of trust that, because if the website no longer says that, it might not be possible to tell whether it ever did say that, or whether you're telling porky pies about your research.

Dependencies in software systems are usually given as if they work like book citations:

gem 'rack', '1.0'

...looks like it says "the thesis that's constructed by my software is a synthesis in which version 1.0 of rack is axiomatic", but it doesn't. It's really saying "at the time that I want you to think that I actually tested this stuff, it was true that the thing identified by being version 1.0 of rack was...". It's really a poorly-constructed website citation.

It's fun to think, particularly in light of the npm shenanigans, just how long that dependency you didn't bother downloading will still be around. You can presumably forget about relying on commercial software, as the licence agreement is the legal equivalent of Vader saying "I have altered the deal. Pray I do not alter it any further." And indeed you can forget most open sores licences, which don't put any requirements on your supplier. But what about the GPL? Version 3 (retrieved from this URL on 24th March 2016) says that anybody who distributes licensed software as object code may, as one possible way to provide access to the corresponding source code, provide that object:

  accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge

What if the person you got the object code from dies within that three year period, do you have the right to ask the executor of their estate for the source code?

#### On improved tool support for Cocoa developers
I started writing some tweets, that were clearly taking up too much room. They started like this:

My own thoughts: tool support is very important to good software engineering. 3.3.1 is not a big inhibitor to novel tools. /cc @rentzsch

then this:

There's still huge advances to make in automating design, bug-hunting/squashing and traceability/accountability, for instance.

(The train of thought was initiated by the Dog Spanner's [c4 release]; post.)

In terms of security tools, the Cocoa community needs to catch up with where Microsoft are before we need to start wondering whether Apple might be holding us back. Yes, I have started working on this, I expect to have something to show for it at NSConference MINI. However, I don't mind whether it's you or me who gets the first release, the important thing is that the tools should be available for all of us. So I don't mind sharing my impression of where the important software security engineering tools for Mac and iPhone OS developers will be in the next few years.

Requirements comprehension

My first NSConference talk was on understanding security requirements, and it's the focus of Chapter 1 of Professional Cocoa Application Security. The problem is, most of you aren't actually engineers of security requirements, you're engineers of beautiful applications. Where do you dump all of that security stuff while you're focussing on making the beautiful app? It's got to be somewhere that it's still accessible, somewhere that it stays up to date, and it's got to be available when it's relevant. In other words, this information needs to be only just out of your way. A Pages document doesn't really cut it.

Now over in the Windows world, they have Microsoft Threat Modeling Tool, which makes it easy to capture and organise the security requirements. But stops short of providing any traceability or integration with the rest of the engineering process. It'd be great to know how each security requirement impacts each class, or the data model, etc.

Bug-finding

The Clang analyser is just the start of what static analysis can do. Many parts of Cocoa applications are data-driven, and good analysis tools should be able to inspect the relationship between the code and the data. Other examples: currently if you want to ensure your UI is hooked up properly, you manually write tests that inspect the outlets, actions and bindings you set up in the XIB. If you want to ensure your data model is correct, you manually write tests to inspect your entity descriptions and relationships. Ugh. Code-level analysis can already reverse-engineer test conditions from the functions and methods in an app, they ought to be able to use the rest of the app too. And it ought to make use of the security model, described above.

I have recently got interested in another LLVM project called KLEE, a symbolic execution tool. Current security testing practices largely involve "fuzzing", or choosing certain malformed/random input to give to an app and seeing what it does. KLEE can take this a step further by (in effect) testing any possible input, and reporting on the outcomes for various conditions. It can even generate automated tests to make it easy to see what effect your fixes are having. Fuzzing will soon become obsolete, but we Mac people don't even have a good and conventional tool for that yet.

Bug analysis

Once you do have fuzz tests or KLEE output, you start to get crash reports. But what are the security issues? Apple's CrashWrangler tool can take a stab at analysing the crash logs to see whether a buffer overflow might potentially lead to remote code execution, but again this is just the tip of the iceberg. Expect KLEE-style tools to be able to report on deviations from expected behaviour and security issues without having to wait for a crash, just as soon as we can tell the tool what the expected behaviour is. And that's an interesting problem in itself, because really the specification of what you want the computer to do is your application's source code, and yet we're trying to determine whether or not that is correct.

Safe execution
Perhaps the bitterest pill to swallow for long time Objective-C programmers: some time soon you will be developing for a managed environment. It might not be as high-level as the .Net runtime (indeed my money is on the LLVM intermediate representation, as hardware-based managed runtimes have been and gone), but the game has been up for C arrays, memory dereferencing and monolithic process privileges for years. Just as garbage collectors have obsoleted many (but of course not all) memory allocation problems, so environment-enforced buffer safety can obsolete buffer overruns, enforced privilege checking can obsolete escalation problems and so on. We're starting to see this kind of safety retrofitted to compiled code using stack guards and the like, but by the time the transition is complete (if it ever is), expect your application's host to be unrecognisable to the app as an armv7 or x86_64, even if the same name is still used.

#### On industry malaise
Robert Atkins linked to his post on industry malaise:
All over the place I see people who got their start programming with “view source” in the 2000s looking around at the state of web application development and thinking, “Hey wait a minute, this is a mess” […] On the native platform side, there’s no joy either.
This is a post from 2019, but shared in a "this is still valid" sense. To be honest, I think it is. I recognise those doldrums myself; Robert shared the post in reply to my own toot:
Honestly jealous of people who are still excited by new developments in software and have gone through wondering why, then through how to get that excitement back, now wondering if it’s possible that I ever will.
I've spent long enough thinking that it's the industry that's at fault to thinking it's me that's at fault, and now I know others feel the same way I can expand that from "me" to "us".

I recognise the pattern. The idea that "we" used to do good work with computers until "we" somehow lost "our" way with "our" focus on trivialities like functional reactive programming or declarative UI technology, or actively hostile activities like adtech, blockchain, and cloud computing.

Yes, those things are all hostile, but are they unique to the current times? Were the bygone days with their shrink-wrapped "breaking this seal means agreeing to the license printed on the paper inside the sealed box" EULAs and their "can't contact flexlm, quitting now" really so much better than the today times? Did we not get bogged down in trivialities like object-relational mapping and rewriting the world in PerlPHPPython?

It is true that "the kids today" haven't learned all the classic lessons of software engineering. We didn't either, and there will soon be a century's worth of skipped classes to catch up on. That stuff doesn't need ramming into every software engineer's brain, like they're Alex from A Clockwork Orange. It needs contextualising.

A clear generational difference in today's software engineering is what we think of Agile. Those of us who lived through—or near—the transition remember the autonomy we gained, and the liberation from heavyweight, management-centric processes that were all about producing collateral for executive sign-off and not at all about producing working software that our customers valued. People today think it's about having heavyweight processes with daily status meetings that suck the life out of the team. Fine, things change, it's time to move forward. But contextualise the Agile movement, so that people understand at least what moving backward would look like.

So some of this malaise will be purely generational. Some of us have aged/grown/tired out of being excited about every new technology, and see people being excited about every new technology as irrelevant or immature. Maybe it is irrelevant, but if so it probably was when we were doing it too: nothing about the tools we grew up with were any more timeless than today's.

Some of it will also be generational, but for very different reasons. Some fraction of us who were junior engineers a decade or two ago will be leads, principles, heads of division or whatever now, and responsible for the big picture, and not willing to get caught into the minutiae of whether this buggy VC-backed database that some junior heard about at code club will get sunset before that one. We'd rather use postgres, because we knew it back then and know it now. Well, if you're in that boat, congratulations on the career progression, but it's now your job to make those big picture decisions, make them compelling, and convince your whole org to side with you. It's hard, but you're paid way more than you used to get and that's how this whole charade works.

Some of it is also frustration. I certainly sense this one. I can pretend I understood my 2006-vintage iBook. I didn't understand the half of it, but I understood enough to claim some kind of system-level comfort. I had (and read: that was a long flight) the Internals book so I understood the kernel. The Unix stuff is a Unix system, I know this! And if you ignore classic, carbon, and a bunch of programming languages that came out of the box, I knew the frameworks and developer tools too. I understood how to do security on that computer well enough that Apple told you to consider reading my excellent book. But it turns out that they just wouldn't fucking sit still for a decade, and I no longer understand all of that technology. I don't understand my M1 Mac Mini. That's frustrating, and makes me feel stupid.

So yes, there is widespread malaise, and yes, people are doing dumb, irrelevant, or evil things in the name of computering. But mostly it's just us.

As the kids these days say, please like and subscribe.

#### On Inheritance
I recently had the chance to give my OOP-in-FP-in-Swift talk again in NSLondon, and was asked how to build inheritance in that object system. It's a great question, I gave what I hope was a good answer, and it's worth some more thought and a more coherent response.

Firstly, let's look at the type signature for an object in this system:

typealias Object = (Selector) -&gt; IMP

A Selector is the name of a method, and an IMP is a function1 implementing that method. But an Object is nothing more or less than a function that maps names of methods to implementations of methods. And that's incredibly powerful, for two reasons.

Reason one: Inheritance is whatever you want it to be.

You are responsible for writing the code to look up methods, which means you get to choose how it works. If you don't like inheritance at all, then you're golden: each object knows its own methods and nothing else.

If you like Javascript or Self or Io, then if your object doesn't have a method then it can send itself a proto message, and ask that object what method to use.

If you like Smalltalk or ObjC or Ruby, then you can create an object called a Class that creates objects that look up methods by asking the class what methods to use. If the class doesn't know, then it can ask its superclass.

If you like multiple inheritance, then give an object a list of classes/prototypes instead of a single one.

If you've got some other idea, build it! Maybe you always thought classification should be based on higher-order logic; well here's your chance.

(By the way, if you want to do this, you would be well-off defining a convention where methods all take a parameter that can be bound to the receiver: call it this or self for example. Then when you're deep in inheritance-land, but you want to send a message to self, you still have a reference to it.)

Reason two: Inheritance is whatever you want it to be at each point.

A failing common to all of the object systems named above (as a reminder, that's Javascript, Self, Io, Smalltalk, ObjC, Ruby) is that they force you to work with a single object paradigm. If these things naturally follow a singly-inherited classification scheme, but those things can be better described as deviations from a common prototype, well, sorry, but you've got to pick one and contort it to fit both situations.

If an object is any arbitrary code that finds a method, then you can build whichever model is most appropriate at the point of use. You can mix and match. The core philosophy at a code level is that objects are just loosely-coupled functions. From a conceptual level that's incredibly powerful: such loose coupling means that you aren't forced to make assumptions about how objects are constructed, or glued together. You just use them.

actually a closure, which is even more useful, but that's not important right now.&#160;&#8617;

#### On internal quality
I was asked by attendees at my VTM talk on test-driven development a small collection of questions on a similar theme, which I'll summarise here.

How do I do TDD when my boss doesn't want me to?
What do I do when my boss wants me to ship untested prototype code?
Can you give me rhetoric to convince my boss that I should be doing TDD?

I believe that I can adequately present these questions as facets of the following root issue, and then blabber away endlessly about that one instead.

I have externalised the obligation I have to myself to write software that makes me proud by delegating it up to my boss. Can you help me to sleep at night?

The fundamental problem here is one of cognitive dissonance. Belief: I, as a software engineer, believe that process/technique/tool is good for writing software. Introspection: I observe that I am not using said process/technique/tool. Justifications: it must be that my boss doesn't hold my output to the same quality standard that I do. It must be that my deadlines and ever-shifting requirements don't give me time to do things properly. It must be that this company doesn't want great software engineers.

No. I have learned the hard way that such justifications are false. The only person responsible for how you write code is you. If you work on a team, or are handing your source code over to some other stakeholder (which means you're working on a team), then you may have style guides that limit what source code you write; but ultimately how you get there is solely down to you.

You may be thinking "but I asked my manager for time to implement TDD/CI/buzzword-compliance on my current project and she said no. Also, by the way, you're an arsehole for being so rude and presuming to know what I think." Let me address the latter point first: suck it. Now onto the first point.

Your manager was correct to turn down your request for time on the project. Your customer doesn't want unit tests, or comment documentation, or build server output. Your customer wants working features. The project exists to satisfy the user's needs. Therefore the time allocated on the project should be dedicated to making working features.

Now it turns out that doing TDD is one way to write code that leads to working features, so what you should have done was to agree to the version of the product plan where you make working features, and then done TDD anyway. Where this all started going wrong was not when your boss turned down your request, but when you asked your boss in the first place. It turns out that you both have the same goal&mdash;making a good product&mdash;but that you have different views on the process and different motivators. By asking your boss how to write code you gave her permission to micro-manage you, then got frustrated when she did, and decided that the problem was all her fault.

I've seen this failure mode quite a few times now. As one example, I worked in a company where there was an ingrained antagonism between the product managers (who clearly just don't get that software is a craft and a labour of love) and the programmers (who clearly just don't get that we work in a competitive marketplace and would rewrite the product from scratch every day if they could).

As is common (and often correct) in our industry, the product managers owned the product requirements. As should be welcomed when we care about the products we make, the programmers were invited to criticise the requirements for the projects they were working on. Due to this antagonistic culture, the programmers would typically stuff "engineering requirements" onto the project, which were things like rewriting components from scratch, epic refactorings, or setting up new developer workflows.

Requirements were, as far as I could tell, prioritised based on how much revenue they would attract (i.e. new customers), how much revenue they would protect (retained customers) and how much they would cost. As you can probably already guess, so-called engineering requirements don't protect any revenue, they don't generate any revenue, and they do take time and money to implement, so they would inevitably get dropped or indefinitely postponed.

The moral of that little story is this: it is folly to try and express development processes and methodologies as business requirements, because they fail at being business requirements. It's the equivalent of a taxi driver asking the customer to pay a $100,000 fare for one journey because he wants to switch to automatic transmission for the next journey and that means buying a new vehicle. The customer cares about being driven somewhere, and doesn't care about how the driver operates the vehicle as long as the journey is bug-free. He certainly doesn't intend to pay for the driver to select a particular mode of operation: if the driver wants to use automatic transmission, the driver should just get on and do that.

So what I'm saying is this: it's not up to your boss, your customer or your project manager to choose how you write software. It's up to you to choose how you write software, and as long as the approach you take leads to working software that delights your customers, the rest of the stakeholders won't mind what you're doing "under the hood". My good friend and long-lost son @bmf once said "don't let them see you making it." I would go further: don't let them know it was made. Let them use exciting, compelling software: the magic is your business. If you want to make the magic in one particular way, that would let you take additional pride in your creation, that's entirely your call.

#### On interviewing and generalist software engineers
After publishing podcast Episode 53: Specialism versus generality, Alan Francis raised a good point:

This could be very timely as I ponder my life as a generalist who has struggled when asked to fit in a neat box career wise.https://twitter.com/PossiblyAlan/status/1523755064879632384

I had a note about hiring in my outline for the episode, and for some reason didn’t record a segment on it. As it came up, I’ll revisit that point.

It’s much easier to get a job as a specialist software engineer than as a generalist. I don’t think that’s because more people need specialists than need generalists, though I do think that people need more specialists than they need generalists.

For a start, it’s a lot easier to construct an interview for a specialist. Have you got any experience with this specialism? What have you done with it? Do you know the answers to these in-group trick questions?

Generalists won’t do well at that kind of question. Why bother remembering the answer to a trick question about some specific technology, when you know how to research trick answers about many technologies? But the interviewer hears “doesn’t even know the first trick answer” and wonders how do I know you can deliver a single pint of software on our stack if you can’t answer a question set by a junior our-stack-ist?

If you want to hire a generalist software engineer…ah. Yes. I think that maybe some people don’t want to, whether or not they know what the generalist would do. They seem to think it’s a “plural specialist”, and that a generalist would know all the trick questions from multiple specialisms.

This is the same thinking that yields “a senior developer is like a junior developer but faster”; it is born of trying to apply Taylorian management science to knowledge work: a junior Typescript programmer can sling a bushel of Typescript in a day. Therefore a senior Typescript programmer can sling ten gallons, and a generalist programmer can sling one peck of Typescript, two of Swift, and one of Ruby in the same time.

I think that the hiring managers who count contributions by the bushel are the ones who see software engineering as a solitary activity. “The frontend folks aren’t keeping pace with the backend folks, so let’s add another frontend dev and we’ll have four issues in progress instead of three.” I have written about the flawed logic behind one person per task before.

A generalist may be of the “I have solved problems using computers before, and can use computers to solve your problem” kind, in which case I might set aside an hour to pair with them on solving a problem. It would be interesting to learn both how they solve it and how they communicate and collaborate while doing so.

Or they may be of the “I will take everybody on the team out to lunch, then the team will become better” kind. In which case I would invite them to guest-facilitate a team ceremony, be it an architecture discussion, a retrospective, or something else, and see how they uncover problems and enable solutions.

In each case the key is the collaboration. A software engineering generalist might understand and get involved with the whole process using multiple technologies, but that does not mean that they do all of the work in isolation themselves. You don’t replace your whole team, but you do (hopefully) improve the cohesiveness of the whole team’s activity.

Of course, a generalist shouldn’t worry about trying to get hired despite having to sneak past the flawed reasoning of the bushel of trick questions interview. They should recognise that the flawed reasoning means that they won’t work well with that particular manager, and look elsewhere.

#### On legitimacy and software engineering
More than 400,000 software engineers have lost their jobs in the last couple of years, I wouldn't be surprised if it's really significantly more than half a million as some won't have been documented anywhere that the tracker project saw. In the years leading up to Layoffapalooza, software engineers commanded high salaries (though maybe not universally high) , often with significant perks. Do these shifts in employability privileges reflect a change in the legitimacy software engineering enjoys among its peers, clients, and other stakeholders?

Let's first identify what legitimacy is. A dictionary definition would have it that legitimacy is something like "the ability to be defended", so as our working definition for software engineering's legitimacy let's use the idea that software engineering legitimacy is the right, or ability, that software engineers have to define their work and their contribution on their own terms. That is, taking as an assumption that somebody somewhere in our society wants someone to write software for them, software engineering is more legitimate if software engineers get to decide what to write, how, and how they're evaluated, and less legitimate if somebody else (clients, managers, governments, whoever) gets to decide that. This kindof ties legitimacy with autonomy, but it also connects it with status or privilege.

Following Mark Suchman's Managing legitimacy: strategic and institutional approaches, let's break this down into three categories. He's talking about institutions, so I'm pretending to make an assumption here that "software engineering" is an institution. I suspect that some people (both inside and outside the field) see it as such, and others don't. But it also might be useful to explicitly call out organisations, interest groups, user communities, or other subcultures within the field and investigate whether they are more or less institutional (so that we can 

Cognitive legitimacy

The third of Suchman's categories, cognitive legitimacy is the idea that an institution is legitimate if it doesn't take much effort to see it as such: in other words, that it's consistent with the worldview and values that people already have. It's easy to maintain cognitive legitimacy, though perhaps hard to acquire. But it also doesn't get you much, as it's really about existing in the background. An institution that didn't have cognitive legitimacy might look something like the Communist Party of the USA: every time you're reminded that it's there, it's a surprise and you're not sure what to make of it.

People pushed for the cognitive legitimacy of software engineering basically from the start of the field. The 1967 NATO working group chose the name because it was illegitimate:

The phrase ‘software engineering’ was deliberately chosen as being provocative, in implying the need for software manufacture to be based on the types of theoretical foundations and practical disciplines, that are traditional in the established branches of engineering.

Indeed the preface to the second of the NATO conferences on software engineering reports of the first conference:

The vast majority of these participants found commonality in a widespread belief as to the extent and seriousness of the problems facing the area of human endeavour which has, perhaps somewhat prematurely, been called “software engineering”.

Brian Randell, who edited the two conference reports, recalls that the second conference attempted to "fast-track" legitimacy; an attempt that failed.

Unlike the first conference, at which it was fully accepted that the term software engineering expressed a need rather than a reality, in Rome there was already a slight tendency to talk as if the subject already existed. And it became clear during the conference that the organizers had a hidden agenda, namely that of persuading NATO to fund the setting up of an International Software Engineering Institute. However things did not go according to their plan. The discussion sessions which were meant to provide evidence of strong and extensive support for this proposal were instead marked by considerable scepticism, and led one of the participants, Tom Simpson of IBM, to write a splendid short satire on "Masterpiece Engineering".

Fast-forward 54 years from that second conference, and the phrase "software engineering" is indeed part of the cognitive background of programming computers. But the institutions that underpin it are not. Like other phrases, including Agile, DevOps, and Open Source, software engineering has been co-opted by the managerial class to mean "the people we hire to do the work we want them to do, the way we want them to do it". Research institutes like the SEI, or special interest groups like ACM SigSoft, don't have a seat at the software engineering table in the large. Even in academia, while software engineering was meant to base practice on "theoretical foundations and practical disciplines", it's common that if software engineering is researched at all it's a field in the Computer Science department. All theoretical foundation, no practical discipline.

Pragmatic legitimacy

Pragmatic legitimacy is that which supports an organisation because doing so is in the rational self-interests of the audience. A lot of support for open source software is in the form of pragmatic legitimacy: we open source our database because that will encourage grass-roots adoption, which is cheaper than a sales channel. But notice that, as said with cognitive legitimacy, when we talk about open source we talk about a managerial decision to "open source"; we don't talk about joining the Open Source Initiative, or bringing in a representative from the Software Freedom Law Center to train our attorneys. The idea holds legitimacy if not the institution.

Come down from the conceptual to the project level, and more pragmatic legitimacy holds. An organisation uses Linux, but it doesn't want to maintain its own Linux source tree, so it's in that organisation's interest to accept the Linux Foundation as a legitimate collaborator. In general "upstream" is a legitimate institution: you upstream your changes because it's in your interest to accept the governance of the external maintainer team.

Moral legitimacy

Moral legitimacy perpetuates an institution because it represents positive values, or "the right thing to do". A lot of people in the Free Software movement see GNU, and the FSF, as moral imperatives, so occasional missteps in governance can be forgiven or overlooked as they don't represent the overall trajectory or challenge the belief structure. 

In academia, a lot of arguments for research software engineering come from the idea that it's the right thing to do: making reproducible software, or high-quality software, is good for research, therefore we should do it and rely on the collective expertise of organisations like the society for RSE, or the software sustainability institute, or Better Scientific Software, to help us interpret the needs. But does that align with pragmatic legitimacy, and when it doesn't, how is the conflict resolved? Is "high-quality software" a strong enough moral imperative among all stakeholders to influence the autonomy of a whole occupation?

#### On localisation and security
Hot on the heels of Uli's post on the problems of translation, I present another problem you might encounter while localising your code. This is a genuine bug (now fixed, of course) in code I have worked on in the past, only the data has been changed to protect the innocent.

We had a crash in the following line:
NSString *message = [NSString stringWithFormat:
	NSLocalizedString(@"%@ problems found", @"Discovery message"),
	problem];

Doesn't appear to be anything wrong with that, does there? Well, as I say, it was a crasher. The app only crashed in one language though&hellip;for purposes of this argument, we'll assume it was English. Let's have a look at English.lproj/Localizable.strings:
/* Discovery message */
"%@ problems found" = "%@ found in %@";

Erm, that's not so good. It would appear that at runtime, the variadic method +[NSString stringWithFormat: (NSString *)fmt, ...] is expecting two arguments to follow fmt, but only passed one, so it ends up reading its way off the end of the stack. That's a classic format string vulnerability, but with a twist: none of our usual tools (by which I mean the various -Wformat flags and the static analyser) can detect this problem, because the format string is not contained in the code.

This problem should act as a reminder to ensure that the permissions on your app's resources are correct, not just on the binary&mdash;an attacker can cause serious fun just by manipulating a text file. It should also suggest that you audit your translators' work carefully, to ensure that these problems don't arise in your app even without tampering.

#### On McAfee
Today, Apple's CPU/motherboard supplier Intel announced that it will acquire McAfee, in a deal worth nearly $7.7B. While this is definitely big bucks, it doesn't seem like terrifically big security news.

Intel probably don't want the technology. McAfee is the world's biggest security vendor, so there are cheaper ways for Intel to acquire security technology. Intel probably don't want a fast buck either: or if they do, they're not about to get it. It would take around a decade for Intel's new security software division to make its money back, assuming no huge changes in organization.

Intel may want the IP. McAfee has an extensive patent portfolio (as do all the big players in the cold war world of security software), there's bound to be things that Intel could implement in silico. Jokes have already been doing the rounds on Twitter of a new CPU opcode, SCANAV. Encryption and data tagging seem more likely targets. But couldn't they just license the patents?

I expect that what Intel are after is to make the company a one-stop IT shop, with security software being just one element in that. Large businesses and government in particular value having a small network of large, stable, boring, trusted partners. We've already seen in the last couple of years that the likes of Cisco, HP and Oracle have been shifting toward "vertical" provision of IT services. Intel now have a few different software houses under their wing, and of course McAfee brings a vast collection of juicy business customers. What Paul Ottelini is likely hoping is that such customers will start looking to Intel for other services, and maybe hardware too. And that Intel's existing customers will buy into McAfee's security offerings.

#### On Mental Health
This post has been a while in the writing, I suppose waiting for the perfect time to publish it. The two things that happened today to make me finally commit it to electrons were the news about Robin Williams, and reading Robert Bloch's That Hell-Bound Train. Explaining the story's relevance would spoil it, but it's relevant. And short.

I didn't leave Big Nerd Ranch because I disliked the job. I loved it. I loved working with clever people, and teaching clever people, and building things with clever people, and dicking around on Campfire posting meme images with clever people, and alternately sweating and freezing in Atlanta with clever people, and speaking stilted Dutch with clever people.

I left because I spent whole days staring at Xcode without doing anything. Because I knew that if I wrote code and pushed it, what I would do would be found lacking, and they'd realise that I only play a programmer on TV, even though I also knew that they were friendly, kind, supportive people who would never be judgemental.

When I left I was open and honest with my colleagues and my manager, and asked them to be open and honest with each other. I felt like I was letting them down, and wanted to do the least possible burning of bridges. I finished working on the same day that I voiced my problems, then I went to bed and had a big cry.

Which sounds bad, but was actually a release of sorts. I don't remember the previous time I'd cried, or really done anything that expresses emotion. I think it may have been whenever British Mac podcast episode 35 was on, playing the final scene of Blackadder Goes Forth. Which apparently was back in 2006.

Anyway, I then took a couple of months away from any sort of work. On the first day post-Ranch I made an appointment to see a doctor, who happened to have available time that same day. He listened to a story much like the above, and to a description of myself much like the below, and diagnosed depression.

You may have seen that demo of Microsoft's hyper-lapse videos where you know that there's loads going on and tons of motion things to react to, but the image is eerily calm and stable. Yup. There's lots going on, but in here everything's muffled and has no effect.

It's not like coasting, though. It's like revving the engine with the clutch disengaged. I never stop thinking. That can get in the way of thinking about things that I actually want to think about, because I'm already thinking about something else. It means getting distracted in conversations, because I'm already thinking about something else. It means not getting to sleep until I stop thinking.

It also means writing a lot. You may have got rid of a song stuck in your head (an earworm) by playing that song through, or by singing it to yourself. I get rid of brainworms by writing them down. I use anything: from a Moleskine notebook and fountain pen on an Edwardian writing slope to Evernote on a phone.

Now, you may be thinking—or you may not. It may just be that I think you're thinking it. While I'm still extraverted, I'm also keen to avoid being in situations where I think other people might be judging me, because I'll do it on their behalves. It's likely that I'm making this up—that it's a bit weird that I keep telling jokes if I'm supposed to be emotionally disengaged. Jokes are easy: you just need to think of two things and invent a connection between them. Or tell the truth, but in a more obvious way than the truth usually lets on. You have to think about what somebody else is thinking, and make them think something else. Thinking about thinking has become a bit of a specialty.

Having diagnosed me, the doctor presented two choices: either antidepressant medication, or cognitive behavioural therapy. I chose the latter. It feels pretty weird, like you're out to second-guess yourself. Every time you have a bad (they say "toxic", which seems apt: I have a clear mental image of a sort of blue-black inky goop in the folds of my brain that stops it working) thought you're supposed to write it down, write down the problems with the reasoning that led to it, and write down a better interpretation of the same events. It feels like what it is—to psychology what the census is to anthropology. Complex science distilled into a form anyone can fill in at home.

This post has been significantly more introspective than most of this blog, which is usually about us programmers collectively. Honestly I don't know what the message to readers is, here. It's not good as awareness; you probably all know that this problem exists. It's not good as education; I'm hardly the expert here and don't know what I'm talking about. I think I just wanted to talk about this so that we all know that we can talk about this. Or maybe to say that programmers should be careful about describing settings as crazy or text as insane because they don't know who they're talking to. Maybe it was just to stop thinking about it.

#### On multitasking
TidBITS unwittingly hits the nail on the head while talking about iPad OS multitasking (emphasis added):

It's easy to imagine wanting to use an iPad to read text in Mobile Safari, copy some text to a Pages document, and send that document to a colleague via Mail. That specific example may turn out to be possible with the current iPhone OS, but it points toward needing more ways for iPad apps to work together in the future.

Let me break down the user's workflow here:

User reads text in Mobile Safari.User copies text to a Pages document.User e-mails that document to a colleague.

The flow of tasks is linear. The user does not need Mail open while reading the text in Safari, nor Safari open while pasting text in Pages. Whether a platform supports multiple simultaneous applications or not, users typically work with one at a time.The advantage of multiple open apps is that the user can switch tasks really quickly (the other oft-quoted benefit, of being able to see context in multiple places at the same time, is actually a feature of a windowing UI: a different technology, and one that iPhone OS lacks). The disadvantage is a technical one&mdash;the operating system must allocate resources to applications that the user isn't currently working with. The iPhone (and, I presume, the iPad) provides fast task-switching anyway, through its recommendation that app developers retain app state on termination and recover it on launch. The act of moving between apps via the home screen is supposed to feel like switching tasks, even if it's implemented by a kind of pause-and-resume.

#### On my newer competence
This time last year, I evaluated myself against the programmer competency matrix. So where am I one turn around the daystar later? I have to admit that this was mainly because I was jet lagged in a hotel room in Seattle and had nothing else to do last year :-), but nonetheless it's time to reflect on how things have gone.

Last year's goals

I wrote:

it looks like my priorities for research goals over the upcoming year or so should be:

Improve breadth and depth of API knowledge
Investigate systems decomposition of a smartphone/desktop/internet system.
Learn how to create and support DSLs
Write and publish improved developer support tools


So how did that go?

I improved here. I didn't have a project for doing this, I just used the APIs some more. Things that I previously remembered but needed to look up I now automatically get right, and some things I didn't know about I now do. There'll always be room for more.
My work on GNUstep web is only part of this goal.
I haven't done anything with DSLs, as I haven't found a need for them in the last year. Maybe that's because I don't understand where they can be used :). Part of my GNUstep web project will include rules engines a la Direct 2 Web, so I guess I'll have to tackle this soon.
Again, in hand with the GSW project. I've already published a code generation tool. Some of the things I learned about at NSConference were very inspirational.

What about this year?

I'll just dive into the matrix. If nothing's changed from last year you won't see any comments.

Computer Science

Data Structures: still level 1. I've learned a little more about level 2 aspects.
Algorithms: level 2.
Systems programming: level 3, if you're willing to accept that block-diagram-level knowledge counts as knowledge on the level chart :)

Software Engineering
Source Control: level 3.
Build Automation: level 3.
Automated Testing: level 3.

Programming
Problem Decomposition: level 3.
Systems Decomposition: level 3. As I said earlier, I made this a specific goal of the last year, and I believe it's paid off.
Communication: level 3. I've also done what I set out to do last year, and dialled back on the commitments to conferences and books.
Code Organisation Within a File: level 3. As I said last year, this is subjective.
Code Organisation Across Files: level 3.
Source Tree Organisation: level 3. For these two I've decided it's what you see in the IDE (Xcode, AppCode and Eclipse are my regular haunts) that really matters, though I try to make the filesystem layout match that.
Code Readability: level 3. I'm not quite that bastard who rejects code because it doesn't conform to the standard; though I used to be :-(
Defensive Coding: level 2. Still don't have a "library" for doing that (I've written an iOS security library that's internal to O2, but that's not what this is about.)
Error Handling: level 3 (with s/xception/rror/, because we appreciate that error-handling should be sympathetic to the platform/framework style, just like other code.)
IDE: level 3. It was mainly working with AppCode that spurred me into investigating ways to automate IDE functions.
API: still going to claim level 1, because I'd say it's for other people to decide whether they think I have sufficiently "vast and in-depth knowledge".
Frameworks: level 2.
Requirements: level 3.
Scripting: level 3 (though this improvement only just scraped in this year.)
Database: level 1. I'm starting to see more projects where a better knowledge of databases is required, if one of those lands in front of me I'd better improve at knowing databases. However as I said last year it's the kind of thing where I can use an ORM to get it mostly correct, then investigate whether any issues remain that require deeper inspection.

Experience

I skipped this section last year, but just for kicks I'll go through it this time. No comparisons with last year, and if you can't work out why you're not allowed higher than Level 0 on any entry in the matrix ;-).

Languages with Professional Experience: level 2. Functional programming I have worked with includes LISP, C++ and Java (yes, that Java).
Platforms with Professional Experience: heh, this depends as I said in a recent talk on how you define "Unix". Conservatively I could say Unix, Mac, iOS and Java: level 1.
Years of Professional Experience: not that I think this matters, but level 2.
Domain Knowledge: wait, which domain? In the world of telco (where I currently spend my time) I'm a complete noob. In fact I've spent most of my career as a complete noob. Whether that's good or bad, and something I should or would want to change, is open for debate. Maybe that's another blog post, later.

Yeah, having written that section I now understand why I left it out last time.

Knowledge
Tool Knowledge: still level 2. I've written and published some codeine scripts, not sure it's enough to qualify at level 3.
Languages Exposed to: still level 2. I did some concurrent programming in Objective-C, but no Erlang, Oz or Prolog. OK, I have done some Prolog tutorials, and Oz is the operating system on my Cambridge Z88 but I'm going to assume that it's a different Oz.
Codebase knowledge: hard to gauge. We have about 100 repos on my team, some I'm not working on, some I collaborate on, some I'm the sole author. We'll call this one unknowable for now.
Knowledge of Upcoming Technologies: level 3.
Platform Internals: level 2.
Books: still level 1, but I've written two now. I don't imagine writing any more, there must be a better way to share knowledge.
Blogs: level 3.

This year's goals

Some of the things I planned for last year still need to be done:

DSLs
Developer tools

To which I'll add some things that, probably, don't directly follow from the competency matrix, but which are nonetheless goals of mine.

More do, less talk. A side-effect of being "the security guy" in my corner of the industry is that I do a lot of talking about security in other peoples' products, and not much making products. It's time to find a product I believe in, and own the crap out of it. That means investing my time in it long-term, not diving in and out like a consultant. Consultants are great and consultancy is great: but I want to build something.
Mentor new developers. A recurring theme from my discussions in Appsterdam and at NSConf is that while I can certainly do a lot to become a better programmer, that won't scale as well as making a load of other people better programmers. If I can help at least one person become a better developer than I am, I'll improve my own abilities along the way and then there'll be two people better than I am.

That will, I think, do for now.

#### On my own competency
There was a question on programmers.stackexchange.com about whether to put your Stack Overflow reputation in your CV. I don't, and answered as much: there's no point in writing for its own sake, unless you want to be a writer. If you want to be a programmer, then you should be a good programmer and any writing you do should be evidence of that.

There's a great&mdash;if somewhat old&mdash;resource for evaluating your own capability as a software engineer called the programmer competency matrix. Writing the answer I gave to the above question renewed the memory of the matrix in my mind, so I decided to evaluate myself against it and use that to help build a research and personal development plan.

I'm sharing my own results in this post. My hope is that this will trigger some introspection of your own, and demonstrate just how easy it is to devote a little time to personal development (this took me about an hour to do, including the writing). I'm also being selfish: when I write for publication I'm a lot more organised than if I just push notes into iThoughtsHD or OmniOutliner.

It doesn't matter whether you agree or disagree with anything I write. If we agree on any row of the matrix, then we're both pulling in the same direction and pushing the bar a little higher in that area of software engineering. If we disagree, then we're still both raising the bar, because we both have our special skills that (hopefully) contribute to making the whole industry that bit more skilful and knowledgeable. Of course, if you think that I'm completely brain-dead for choosing not to develop my skills in a particular area, feel free to flame me in the comments.

Notice that it's hard to be objective about self-evaluation. I hope that my answers below are somewhat accurate, but it's easy to have an over-inflated opinion of your own competence. In fact, when I was a junior software engineer I felt like I knew more than I do now :). It would definitely be trite to give a nonsense buzzphrase like "we should always be continually improving at everything we're doing", because there aren't enough hours in the day for that. There should be things that we definitely need to work on, things that we "just do" but reflect on how they're going, and things that we just automatically get on with. Of course, when I think that something should be automatic but I make a mistake, that's a time to re-evaluate my perception of my skills.

Computer Science
data structures: Level 1. I've done some study into some of the level 2 features (and even know what some of the words mean in Level 3), but at heart I feel like that's stuff for a framework programmer to deal with and I'm an applications programmer. I'm happy to believe that the framework engineers know how to implement List&lt;Integer&gt; or NSArray properly and leave them to it.
Related tale: I once had an interview where the interviewer drew a list on the whiteboard, and asked how I'd sort it. I wrote a &quot;[&quot; at the beginning, and &quot; sort];&quot; at the end. I didn't get the job.
algorithms: Level 2. Again this is largely the domain of the framework engineer, but there are times when it's become important to my own work such as when I've had to do performance engineering (and of course cryptographic algorithms factor heavily into my work). I tend to address algorithmic problems on a case-by-case basis, and don't feel the need to explicitly schedule some research time.
systems programming: Level 2. I do believe that I need to develop here, particularly in the areas of JIT compiling and interpretation. If BDD and DDD (and, to some extent, OOP) have taught us anything, it's that there are great efficiency savings to be made by expressing a software problem in a way that's understandable by, or at least explainable to, the user. And that means Domain Specific Languages. I've actually seen that work very well, when I was working for Diamond Light Source. Scientists could write working prototypes in a Jython-based DSL that would then either be cleaned up by engineers or reimplemented in the core engine.

Of course, BDD and DDD are both also examples of RDD (Résumé-Driven Development), but that doesn't mean they have nothing to offer us.

Software Engineering
source code version control: Level 3. To be honest I think that the matrix is outdated here, and that DVCS is well on its way to becoming the new established workflow. When the matrix was written, SVN was the de facto standard for VCS. Version control is just one of those things that you need to know and need to use properly. It will probably still change quite a lot (I don't think that any of the DVCS platforms out there make the uber-common "star" distribution any easier than decentralised development), and my opinion is that I need to be good at the tools I need to use. Of course, being a contractor that often means being good at all of the tools.
build automation: Level 3. I've written here before about setting up tools like HeaderDoc and CruiseControl. I definitely think some of the tools need improving, so could find time to work on that.
automated testing: Level 3. Something I need to do more of, and can "learn on the job" while I'm doing that: but there isn't any point in having a "do UI testing" research project.

Programming
problem decomposition: Level 3. That's not to say I'm completely perfect at object-oriented analysis and design, and every time I see someone else's code I'm interested in what patterns they've chosen, and how they've organised their code. Actually, my current writing project&mdash;a book, video training course and several conference talks on TDD&mdash;has been useful for learning more about decomposition and modularity.
systems decomposition: Level 2. This is an area I definitely need to focus on, because almost all modern applications are actually multiple-component systems that either do, or should, present the same data across multiple devices and feature both dedicated and incidental online components.
communication: Level 3. I think the amount of time I've spent recently on books, conference talks, workshops and the like means that I've actually over-engineered this aspect significantly and can dial back on it once my current obligations are discharged.
code organisation within a file: Unsure. Maybe level 3: well I definitely have a style I use and am comfortable with: is it "beautiful"? Does that even matter? Shouldn't the important factor be: "reading this programmer's code doesn't get in the way of understanding and modifying this programmer's code"?
code organisation across multiple files: Level 2. Who uses the file system any more, anyway? I organise my classes and other resources in the Xcode project navigator. Turns out I use Xcode as some sort of "project builder" tool, not the Finder. If the view of projects and groups in the IDE were the metric in question, then I think I'd be at Level 3 here.
source tree organisation: Level 2, for similar reasons.
code readability: Level 3. Again, this book/research project on TDD has been my most recent and valuable guide.
defensive coding: Level 3. Ensuring you don't only code and test the happy path is one of my current rants, if you give me enough beer. There was a section in my first book on this.
error handling: Level 3. I think this is really just an aspect of defensive coding, though I agree that a consistent approach across projects is important. I'm not a person who avoids using exceptions in Objective-C code.
IDE: Level 2. I suppose if you count shell script build phases as 'custom macros', then I've done level 3 work, but I don't have a library of those shell scripts that I carry around to various projects. I probably could make some efficiency savings by recording how I use Xcode and reducing any commonly-repeated steps or actions.
API: Level 1. No, srsly. I know of the existence and capabilities of many of the Cocoa and Cocoa Touch APIs (with, for obvious reasons, the most focus on Foundation and Core Foundation) but always have to look up the class reference docs. This is really the most important thing I need to work on, so my next research project will be to deepen my knowledge of the APIs.
framework: Level 2. In addition to the Cocoa frameworks, I've worked with WebObjects, Eclipse RCP, wxWidgets and (more's the pity) Swing. Not sure I particularly need to write a framework at the moment, there are other people who do good jobs of that. Exploring MonoTouch would be useful.
requirements: Level 3. Being a contractor, it's important to know when to try and change the client's mind and do things differently, if you want to write good software rather than write crap for a good hourly rate. I've also worked on in-house teams where the product manager or architect didn't have any Mac experience, and it's impossible to work on those without being able to suggest improved requirements.
scripting: Level 2. Yes, automating common developer tasks is still in its infancy, and I probably should find, clean up and publish some of the scripts I've created over time.
(By the way: different scripting languages are useful for different things. I've learnt a lot of bash, perl and python, but should probably pick up ruby too. Don't dismiss any of them for looking like unintelligible spider scrawl; you'll probably find a use for it one day.)
database: Level 1. EOF and Core Data do exist for a reason...to be honest I don't think I'll ever be a Level 3 database person, it's probably cheaper to hire somebody if I ever need that. Most of the products I'm currently being asked to work on either have somebody else for that or don't have strong performance requirements.

Experience
Actually, I'm going to skip this section. I don't believe that you can do a research project on improving your experience; only your exposure. You gain experience by doing.

Knowledge
tool knowledge: Level 2. As I've said there are clear shortcomings in the tools I've used, and I'd love to improve them or write something new and get it out there.
languages exposed to: Level 2. I have done Prolog, but no concurrent programming languages. Additionally, Andre Pang's talk on Functional Programming at NSConf 09 showed that I could learn a lot more from that arena.
knowledge of upcoming technologies: Level 3. It's a particular point of professional pride to me that I know what's coming up and how it changes what I'm working on. Of course, in a world of non-disclosure agreements then "sharing" has to be limited to immediate project teams and people at WWDC. You are coming to WWDC, right?
platform internals: Level 2. I'm not Amit Singh. I already have a project on the horizon to do some more hacking on Clang, and have spoken about that before at NSConf Mini. It's questionable whether a deep understanding of the platform internals is necessary, or whether that should be abstracted by the platform frameworks and interfaces. Personally I believe it is something useful to know, particularly in diagnosing bug reports. It's well known that almost no bugs reported in your software are really bugs in the OS, and a deep understanding of the OS can help to confirm that.
books: Level 1, because I haven't read Peopleware. I've written 1.5 books, does that count for anything?
blogs: Level 3. You're reading it. As identified above, I probably expend more effort on communication than is necessary.

So, that's my current state of competency in a nutshell. From there, it looks like my priorities for research goals over the upcoming year or so should be:

Improve breadth and depth of API knowledge
Investigate systems decomposition of a smartphone/desktop/internet system.
Learn how to create and support DSLs
Write and publish improved developer support tools

OK, your turn: what are yours?

#### On noodles
It's usually considered a good idea to keep a blog focused on exactly one subject. Sod that for a game of soldiers! This one's all about music.Steph, who is a very good musician and knows what she's talking about, wrote that there are two ways to play a harp, a "get the music right" recitation style and a "get the rhythm right and everything else follows" style more suited to improvisation, noodling or folk playing.That's not only true of the harp, it seems to hold for many instruments. For instance, in a moment of crazed, um, craziness this weekend I bought what's commonly referred to as a lute. In fact, a lute with much in common with this lute. Now I've been playing it for all of about two hours in total since Saturday, and can barely remember what note each course plays, and do a poor rendition of about four different tunes from a book of trivially simple lute tunes. But today marked an interesting transition, as it was the first day that I could make music up on the instrument without either knowing or concentrating on what I was doing. Only a couple of things (the song "Wooden Heart" made famous by Elvis Presley, and that banging dance-floor filler "Parson's Farewell") but this represented the point where I could make music on the lute - a different skill than remembering where and when to stick fingers on some bits of nylon, steel and wood.And I think that's where the root of Steph's distinction of playing techniques really comes from; the unprepared style relies on having some music that needs to occur, and the ability for your hands (or nose or whatever your instrument is played with) to move around in some way which causes that music to exist. Whereas the prepared style relies on having some performance in mind that must be repeated, and requires that you think about moving $appendage in such-and-such way to recreate that performance. I find the distinction in conscious application to be an important one when playing the fiddle, an instrument I'm marginally better at than the lute. If I'm reading some music, playing solo or otherwise engaged in trying to play music, then I can only play whatever notes the music contained and in a fairly uninteresting manner. It's only if I'm able to relax and not think about the music that I can harmonise, ornament and otherwise play more interesting things than what was written on the page - even if not necessarily particularly well :-).

#### On Nostalgia for Physical Media
While I have access to streaming services that offer most of the music that the labels the services deal with still publish, I also have a significant collection of music on physical media, and do most of my listening to prerecorded music by playing entire albums, in order, from a physical format. I recently shared a not-entirely-serious overview of the nostalgia for physical prerecorded audio formats with some friends, and here's the annotated version. In what follows, the formats are presented from best to worst.

Compact Disc (CD)

Summary: these are the true audiophile-grade format and any golden ears fool who tells you vinyl is better is deluded.

It's CD's blessing, and curse, that it's the ideal physical format for music. Blessing, because it replaced everything that came before it, and remained dominant until the rise of the MP3/streamed audio file (which offered lower quality than CDs at the time, but greater convenience). Curse, because its perfection leads to sterility: what you get from playing a CD is the music that was recorded onto it, every time. Any noise was on the master tape, and any imperfections are due to your setup. It's unexciting, so was easy to abandon.

And CD digital audio, the red book standard for encoding stereo sound quantised at 16 bits with 44.1kHz sampling, is good enough. HDCD, SACD, DVD-Audio, and HFPA Blu-ray may all be "better" on the spec sheet, but unless you're sitting in the centre of a perfectly set up room listening to something that was mastered at very high fidelity, and are young enough that your ears can physically react to the higher frequencies, none of that matters.

There are two downside to CD, the first is that the case problem took so long to solve. The standard jewel cases are too fragile for many applications, and the little tabs that hold the booklet in tend to damage the booklets.  The other is that the ability to record to CD took so long to come about, and that the recordable discs are unreliable in many players. That probably worked in the format's favour for adoption as a pre-recorded format, though, because the labels were very worried that "home taping is killing music" so a read-only delivery medium was their preference.

Minidisc (MD)

Summary: like better CDs except for the limited capacity and the proprietary encoding.

I'll admit that even though CDs are at the top of my list, MD is my favourite format. A disc is smaller than a CD, robust due to its enclosing cassette, recordable (even with many portable players), contains the track listing on the disc, and is straight-up fun to use.

Getting CD capacity on a smaller disc was achieved by using a compressed audio format called ATRAC (the data capacity of a MD is about half that of a CD). Often this doesn't lead to any observable quality degradation, but later players and recorders supported the MD-LP format which used more aggressive compression to achieve higher run times, and you can sometimes hear that like listening to a 2000s-era MP3 file. Eventually Hi-MD (high capacity minidisc) solved the problem but in a way that didn't preserve backwards compatibility.

Prerecorded MDs were uncommon ("home taping is killing music" and MD is a recordable format, so Sony had to introduce the Serial Copy Management System which meant that a second-generation digital copy on MD could not itself be duplicated), indeed most were released by labels that Sony themselves owned. And if you think the little booklets of notes, lyrics, and art you get in a CD are small, they're absolutely Lilliputian in an MD box.

Vinyl (33rpm LP, 45rpm SP)

Summary: for getting a pleasant feeling from the shitty sound of pops and dusty styli.

The nostalgia that exists around vinyl stems from its long reign as the only game in town. Because the discs are so big (12" LP, 7" SP), you get nice sleeve art. The format itself suffers from the fact that dragging a diamond or sapphire across a sheet of PVC generates static electricity, which attracts dust and discharges, and that all of that gets heard as noise in the pickup system. Also it's easy to smear or scratch a record, and cause more defects on playback.

One thing that the stylus doesn't particularly care about is the amount of plastic below the groove, which hasn't stopped revival manufacturers from selling the amount of plastic (180g! 200g! 220g!) as a premium feature.

MP3 player

Summary: I guess these are nostalgia now also.

Shellac

Summary: like shitty vinyl.

A great way to annoy a vegan would be to farm insects so you can turn their exudate into reproductions of musical recordings. And that's exactly what happened up until about 1948.

By the way, the limited capacity of shellac records (as the reproduction process was even noisier than microgroove vinyl, the records had to spin at 78rpm, chosen because the gears to get the motor to run at 78rpm were already constructed for use in players for talkie movie houses) meant that they wouldn't fit much music on each side of a disc. Large collections of discs were bound into albums and sold together, and that's why music albums are called albums. There you go.

Pianola/barrel organ punched paper rolls

Summary: convenient format, awkward-ass players.

Imagine having a whole piano that you don't play yourself, but that has a clockwork punched paper feeder like a mechanised loom that plays it for you. The rolls were mastered using the reverse process, where someone played a piano that marked up where the holes should go on a blank paper roll. It's kind of fun to think that the ghost of Scott Joplin is playing the Maple Leaf Rag on your piano.

Wax cylinder

Summary: no, you don't actually remember this.

Digital Audio Tape (DAT)/ Digital Compact Cassette (DCC)

Summary: if you remember these then well done, you bought the wrong hi-fi.

It's a shame that these come above the compact cassette in this list, because they fixed some of the problems of that format that we haven't been able to discuss yet. However, they didn't fix the main issues: that tape is sequentially fed into the player like a pianola roll, and that when you do that there's a high chance it gets mangled up in the player.

DCC exists because Sony and Philips had stopped collaborating, and Philips wanted to position themselves against MD. There's a sounds-too-good-to-be-true story that MD and DCC were demoed at the same event, and that while playback sounded great from both formats, there was an awkward silence punctuated only by the giggles of journalists when the Philips rep switched to a different track which involved fast-winding the tape.

Compact Cassette

Summary: you remember these and they're really bad.

There's a recent trend for cassette revival, and it makes me feel weird. The only format that adds so much noise, Dolby invented multiple compansion techniques to try to make the sound listenable. The format where there's a good chance that when you take a cassette out of the player, the tape stays inside the player and outside the cassette. The format that triggered the "you're old if you know what this is for" picture of a pencil meme. The format where playing the music rubs the music off the tape.

I have multiple cassette players, but mostly for retro computing reasons. I didn't enjoy tape "back in the day", I'm not like to start now.

Endless loop (4-track, 8-track), other cart tape or reel-to-reel tape formats

Summary: worse than compact cassette.

#### On NSInvocation
I was going to get down to doing some writing, but then I got some new kit I needed to set up, so that isn't going to happen. Besides which, I was talking to one developer about NSInvocation and writing to another about NSInvocation, then another asked about NSInvocation. So now seems like a good time to talk about NSInvocation.

What is NSInvocation?

Well, we could rely on Apple's NSInvocation class reference to tell us that

An NSInvocation is an Objective-C message rendered static, that is, it is an action turned into an object.

This means that you can construct an invocation describing sending a particular message to a particular object, without actually sending the message. At some later point you can send the message as rendered, or you can change the target, or any of the parameters. This "store-and-forward" messaging makes implementing some parts of an app very easy, and represents a realisation of a design pattern called Command.

How is that useful?

The Gang of Four describes Command like this:

Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.

Well, what is NSInvocation other than a request encapsulated as an object?

You can imagine that this would be useful in a distributed system, such as a remote procedure call (RPC) setup. In such a situation, code in the client process sends a message to its RPC library, which is actually acting as a proxy for the remote service. The library bundles up the invocation and passes it to the remote service, where the RPC implementation works out which object in the server process is being messaged and invokes the message on that target.

Spoiler alert: that really is how Distributed Objects on Mac OS X operates. NSInvocation instances can be serialised over a port connection and sent to remote processes, where they get deserialised and invoked.

An undo manager, similarly, works using the Command pattern and NSInvocation. Registering an undo action creates an invocation, describing what would need to be done to revert some user action. This invocation is placed on a queue, so the undo operations are all recorded in order. When the user hits Cmd-Z, the undo manager sends the most recent undo invocation to its target.

Similarly, an operation queue is just a list of requests that can be invoked later...this also sounds like it could be a job for NSInvocation (though to be sure, blocks are also used, which is another implementation of the same pattern).

The remaining common application of Command is for sending the same method to all of the objects in a collection. You would construct an invocation for the first object, then for each object in the collection change the invocation's target before invoking it.

Got a Concrete Example?

OK, here's one. You can use +[NSThread detachNewThreadSelector: toObject: withTarget:] to spawn a new thread. Because every thread in an iOS application needs its own autorelease pool, you need to create an autorelease pool at the beginning of the target selector's method and release it at the end. Without using the Command pattern, this means one or more of:

Having a memory leak, if you can't edit the method implementation
Having boilerplate autorelease pool code on every method that might - sometime - be called on its own thread
Having a wrapper method for any method that might - sometime - need to be called with or without a surrounding pool.

Sucks, huh? Let's see if we can make that any better with NSInvocation and the Command pattern.

- (id)newResultOfAutoreleasedInvocation:(NSInvocation *)inv {
    id returnValue = nil;
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    [inv invoke];
    if ([[inv methodSignature] methodReturnLength] > 0) {
        if (strncmp([[inv methodSignature] methodReturnType],@encode(id), 1)) {
            char *buffer = malloc([[inv methodSignature] methodReturnLength]);
            if (buffer != NULL) {
                [inv getReturnValue: buffer];
                returnValue = [NSValue valueWithBytes: buffer objCType: [[inv methodSignature] methodReturnType]];
                free(buffer);
            }
        }
        else {
            [inv getReturnValue: &returnValue];
        }
        [returnValue retain];
    }
    [pool release];
    return returnValue;
}

Of course, we have to return a retained object, because the NSAutoreleasePool at the top of the stack when the invocation is fired off no longer exists. That's why the method name is prefixed with "new": it's a hint to the analyser that the method will return a retained object.

The other trick here is the mess involving NSValue. That, believe it or not, is a convenience, so that the same method can be used to wrap invocations that have non-object return values. Of course, using NSInvocation means we're subject to its limitations: we can't use variadic methods or those that return a union type.

Now, for any method you want to call on a separate thread (or in an operation, or from a dispatch queue, or&hellip;) you can use this wrapper method to ensure that it has an autorelease pool in place without having to grub into the method implementation or write a specific wrapper method.

A side note on doing Objective-C properly: this method compares the result of -[NSMethodSignature methodReturnType] with a specific type using the @encode() keyword. Objective-C type encodings are documented to be C strings, and there's even a page in the documentation listing the current values returned by @encode. It's best not to rely on those, as Apple might choose to change or extend them in the future.

#### On NSNull as an anti-pattern
All this talk about type-safe collections may leave you thinking: but what about NSNull? Let's say you have an array that only accepts objects conforming to MyProtocol. You can't add +[NSNull null] to it, because it doesn't implement the protocol. So haven't I just broken mutable arrays?

Let's be clear: NSNull is a nasty hack. The original inventors of Foundation wanted to provide a variadic initialiser and factory for collection classes, but rather than doing +[NSArray arrayWithCount: (unsigned)count objects: (id)firstObject, ...] they created +[NSArray arrayWithObjects: (id)firstObject, ...]. That meant they needed a special value to flag the end of the list, and they chose nil. That meant you couldn't put nil into an array, because such an array could not be constructed using the +arrayWithObjects: style. Therefore they decided to provide a new "nothing" placeholder, and created NSNull.

NSNull makes client code warty. If you were permitted to put nil into a collection, you could just do this:

for (id &lt;MyProtocol&gt;foo in myFoos) {
  [foo doSomethingInteresting];
}

but if you use NSNull, you get to write this (or a close variant):

for (id &lt;MyProtocol&gt;foo in myFoos) {
  if ([foo conformsToProtocol: @protocol(MyProtocol)]) {
    [foo doSomethingInteresting];
  }
}

Nasty. You've actually got to perform the test that the protocol conformance is supposed to address, or something that gets you the same outcome.

I'd prefer to use a different pattern, common in languages where nil or its equivalent cannot be messaged, known as the Null Object Pattern. To be clear, all you do is implement a class that conforms to the protocol (or extends the superclass, if that's what you're up to) but doesn't do anything interesting. If it's interrogated for data, it just returns 0, NO or whatever is relevant. If it's asked to do work, it just returns. In short, it can be used in the same way as a real instance but does nothing, just as a placeholder ought to behave. So we might do this:

@interface NullMyProtocolConformer: NSObject &lt;MyProtocol&gt; { }
@end

@implementation NullMyProtocolConformer

- (void)doSomethingInteresting { }

@end

Now we can go back to the first version of our loop iteration, and keep our type-conformance tests in our collection classes. Anywhere you might want to put NSNull, you just stuff an instance of NullMyProtocolConformer.

#### On null
I've had an interesting conversation on the topic of null over the last few days, spurred by the logical disaster of null. I disagreed with the statement in the post that:
Logically-speaking, there is no such thing as Null
This is true in some logics, but not in all logics. Boolean logic as described in An Investigation of the Laws of Thought, admits only two values:
instead of determining the measure of formal agreement of the symbols of Logic with those of Number generally, it is more immediately suggested to us to compare them with symbols of quantity admitting only of the values 0 and 1.
...and in a later chapter he goes on to introduce probabilities. Anyway. A statement either does hold (it has value 1, with some probability p) or it does not (it has value 0, with some probability 1-p; the Law of the Excluded Middle means that there is no probability that the statement has any other value).

Let's look at an example. Let x be the lemma "All people are mortal", and y be the conclusion "Socrates is mortal". What is the value of y? It isn't true, because we only know that people are mortal and do not know that Socrates is a person. On the other hand, we don't know that Socrates is not a person, so it isn't false either. We need another value, that means "this cannot be decided given the current state of our knowledge".

In SQL, we find a logic that encodes true, false, and "unknown/undecided" as three different outcomes of a predicate, with the third state being given the name null. If we had a table linking Identity to Class and a table listing the Mortality of different Classes of things, then we could join those two tables on their Class and ask "what is the Mortality of the Class of which Socrates is a member", and find the answer null.

But there's a different mathematics behind relational databases, the Relational Calculus, of which SQL is an imperfect imitation. In the relational calculus predicates can only be true or false, there is no "undecided" state. Now that doesn't mean that the answer to the above question is either true or false, it means that that question cannot be asked. We must ask a different question.

"What is the set of all Mortality values m in the set of tuples (m, c) where c is any of the values of Class that appear in the set of tuples (x, c) where x is Socrates?"

Whew! It's long-winded, but we can ask it, and the answer has a value: the empty set. By extension, we could always change any question we don't yet know the answer to into a question of the form "what is the set of known answers to this question". If we know that the set has a maximum cardinality of 1, then we have reinvented the Optional/Maybe type: it either contains a value or it does not. You get its possible value to do something by sending it a foreach message.

And so we ask whether we would rather model our problem using a binary logic, where we have to consider each question asked in the problem to decide whether it needs to be rewritten as a set membership test, or a ternary logic, where we have to consider that the answer to any question may be the "I don't know" value.
Implementationally-speaking, there are too many damn nulls
We've chosen a design, and now we get to implement it. In an implementation language like Java, Objective-C or Ruby, a null value is supplied as a bottom type, which is to say that there is a magic null or nil keyword whose value acts as a subtype of all other types in the system. Good: we get "I don't know" behaviour for free anywhere we might want it. Bad: we get that behaviour anywhere else too, so we need to think to be sure that in all places where "I don't know" is not an answer, that invariant holds in our implementation, or for those of us who don't like thinking we have to pepper our programs with defensive checks.

I picked those three languages as examples, by the way, because their implementations of null are totally different so ruin the "you should never use a language with null because X" trope.

 	Java nulls are terminal: if you see a null, you blew it.
 	Objective-C nulls are viral: if you see a null, you say a null.[*]
 	Ruby nulls are whatever you want: monkey-patch NilClass until it does your thing.

[*] Objective-C really secretly has _objc_setNilReceiver(id), but I didn't tell you that.

Languages like Haskell don't have an empty bottom, so anywhere you might want a null you are going to need to build a thing that represents a null. On the other hand, anywhere you do not want a null you are not going to get one, because you didn't tell your program how to build one.

Either approach will work. There may be others.

#### On Null Objects
I've said before, NSNull is an anti-pattern. It's nice that we have the nil object, which allows us to have a stand-in for any object that doesn't do anything. Unfortunately, it's not a universal stand-in. You can't add nil to a collection. You can add +[NSNull null] to a collection, but you can't use that as a placeholder for other objects.

In either of these cases, you end up needing to warty your code. Either when you are building the collection, you must test for and handle trying to add nil. Or when you are reading the collection, you must test for and handle trying to use NSNull.

What would be nice is if we could easily define empty implementations of protocols. These would combine the two null implementations described above, along with more awesomeness:

Like nil, they provide default implementations of any method in the protocol.
Like NSNull, they can be used in collection classes.
Unlike both, they conform to the protocol for which you created them.

Then you can use these instances with aplomb, without your client code needing to work around their existence.

You'd use them like this:

        id &lt;NSCacheDelegate&gt; emptyCacheDelegate = [NullProtocol nullFor: @protocol(NSCacheDelegate)];
        NSArray *delegates = @[emptyCacheDelegate];

The nullFor: method would look to see whether you've previously created a Null Object for the specified protocol. Because they all do the same thing, it could use Flyweight instances.

+ (id)nullFor: (Protocol *)protocol
{
    NSString *className = [NSString stringWithFormat: @"Null%s", protocol_getName(protocol)];
    Class NullClass = NSClassFromString(className);
    NullProtocol *nullObject = objc_getAssociatedObject(NullClass, "instance");

If you don't have one, then the method would create a new class pair, add this protocol to the class and instantiate it.

    if (nullObject == nil)
    {
        NullClass = objc_allocateClassPair(self, [className UTF8String], 0);
        class_addProtocol(NullClass, protocol);
        objc_registerClassPair(NullClass);
        nullObject = [NullClass new];
        nullObject->_protocol = protocol;
        objc_setAssociatedObject(NullClass, "instance", nullObject, OBJC_ASSOCIATION_RETAIN_NONATOMIC);
    }
    return nullObject;
}

The object could then use the standard message-forwarding technique to let nil handle all the messages from the protocol.

BOOL isNullMethodDescription(struct objc_method_description description)
{
    return (description.name == NULL && description.types == NULL);
}

- (struct objc_method_description)methodDescriptionForSelector: (SEL)aSelector inProtocol: (Protocol *)protocol
{
    //required methods
    struct objc_method_description description = protocol_getMethodDescription(protocol, aSelector, YES, YES);
    if (isNullMethodDescription(description))
    {
        //optional methods
        description = protocol_getMethodDescription(protocol, aSelector, NO, YES);
    }
    //look in the super-protocols
    if (isNullMethodDescription(description))
    {
        unsigned int protocolCount = 0;
        Protocol * __unsafe_unretained *protocols = protocol_copyProtocolList(protocol, &protocolCount);
        if (protocols == NULL)
        {
            return description;
        }
        unsigned int protocolCursor = 0;
        for (protocolCursor = 0; protocolCursor &lt; protocolCount; protocolCursor++)
        {
            Protocol *thisProtocol = protocols[protocolCursor];
            description = [self methodDescriptionForSelector: aSelector inProtocol: thisProtocol];
            if (!isNullMethodDescription(description)) break;
        }
        free(protocols);
    }
    return description;
}

- (struct objc_method_description)methodDescriptionForSelector: (SEL)aSelector
{
    return [self methodDescriptionForSelector: aSelector inProtocol: _protocol];
}

- (BOOL)respondsToSelector:(SEL)aSelector
{
    struct objc_method_description description = [self methodDescriptionForSelector: aSelector];
    return !(isNullMethodDescription(description));
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
    NSMethodSignature *superResult = [super methodSignatureForSelector: aSelector];
    if (superResult) return superResult;
    struct objc_method_description description = [self methodDescriptionForSelector: aSelector];
    return [NSMethodSignature signatureWithObjCTypes: description.types];
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
    [anInvocation invokeWithTarget: nil];
}

Oh wait, that all exists.

#### On Operation Chokehold
So Fake Steve Jobs has announced Operation Chokehold, a wireless flashmob in which disgruntled AT&amp;T customers are to use data-intensive apps for an hour in protest at the poor service and reduced investment AT&amp;T provide on their network. At time of writing, Operation Chokehold has 3,000 fans on Facebook - a small fraction of the &sim;82M AT&amp;T Mobility subscribers in the U.S. Fake Steve has latterly wondered whether it is illegal (using the "it's now out of my hands" defence, popular with people who don't understand what incitement means), and seemingly back-pedalled, considering aloud whether people might try a shorter duration or physical flashmob of AT&amp;T stores instead. It would appear that the FCC (the U.S. agency responsible for regulating national and international communications) has weighed in, declaring a wireless flashmob to be irresponsible and "a significant public safety concern".How is it a concern? Due to the way the phones work, you don't need to the capacity to support all of the users, all of the time, in order to provide a reasonable service. Think of running a file-sharing service like DropBox or Humyo. If you offered up to 10GB storage per customer and have 10,000 customers, then you need 100TB of storage, right? Wrong. That's the maximum that could be used, but let's say in practice you find average use to be 100MB/customer. It turns out that 1TB of storage would be the minimum you'd need to satisfy current demand, if you had even 1.5TB then you'd comfortably support the current customer base while allowing for some future use spikes or growth. The question most businesses ask then is not how risky it is to drop below 100% capacity, but how much risk they can accept in their buffer over average capacity. The mobile phone network operates in the same way. To avoid dropped calls you don't need the bandwidth to support 100% of the phones operating 100% of the time, you need to support the average number of phones the average amount of time, plus a little extra for (hopefully foreseen) additional demand.The argument by AT&amp;T and the FCC against the wireless flashmob then is that because the network is oversubscribed as an accepted business risk, it would actually be possible for the concerted operation of a large number of users to cause disruption to the network. This eventuality is evinced every year in the early morning of January 1st, as people phone or SMS each other with New Year greetings. People making legitimate calls during this time could be disconnected or unable to place a call at all&mdash;while that would undoubtedly make the protest noticed by AT&amp;T it's that aspect of it which makes it a potential public safety concern. Personally, I find it hard to believe that the network doesn't have either dedicated capacity or priority quality of service (QoS) treatment for 9-1-1 calls, but it's possible still that some 9-1-1 calls might not get placed correctly. That's especially true if the caller's handset can't even connect to a tower, which could happen if nearby towers were all saturated with phones making data connections. While it's possible to mitigate that risk (dedicated cell towers for 9-1-1 service, which emergency calls are handed over to) it would be very expensive to implement. There's no business need for AT&amp;T to specially support emergency calls, as they don't make any money from them, so they'd only do that if the FCC mandated it.But then there are all the non-9-1-1 emergency calls&mdash;people phoning their local doctor or hospital, and "business critical" calls made by people who somehow think that their business is critical. Even the day-to-day running of government is at least partially conducted over the regular phone networks, as was seen when the pager traffic from New York on September 11th 2001 got posted to WikiLeaks. These calls are all lumped in with the regular calls, because they are regular calls. The only way to mitigate the risk of dropping these is to increase the capacity of the network, which is exactly the thing that people are saying AT&amp;T don't do to a satisfactory level. If the contracts on AT&amp;T Mobility are anything like the contracts on UK phone networks, then subscribers don't have a service level agreement (SLA) with the provider, so there's no guarantee of provision. The sticking point is the level of expected provision doesn't match that. If the providers operated multi-tier subscription services like the broadband providers do in the UK, then they probably would do QoS management so that preferential customers get better call service&mdash;again, assuming the customers can connect to the cell tower in the first place. But again, that's a business issue, and if the guy participating in Chokehold has a more expensive plan than the girl trying to phone the hospital, his connection will win.Will Chokehold fulfil its goal of making AT&amp;T invest more in its infrastructure? I don't know. Will it actually disrupt public safety services such as 9-1-1? I doubt it. Is it a scale model for a terrorist attack on the communications infrastructure of the US? Certainly not. Much easier to jump down a manhole and snip the cables to the data centres.

#### On opinionation
I've realised that when I read that a tool or framework is "opinionated", I interpret that as meaning that I'm going to have to spend time on working out how to express my solution in its terms. I have enough trouble trying to work out how to express my solution in terms of my problem, so I'm probably going to avoid that tool or framework.

#### On or Between
The new way to model concurrency is with coroutines (1963), i.e. the async/await dance or (building upon) call-with-concurrent-continuation. The new new way is with actors (1973), and the old and busted ways are with threads (1966), and promises (1976).

As implemented in many programming languages, these ideas are on a piece of logic, making the concurrency model an existential part of the software model. Would we say that a payroll budget is a serial queue, or that awarding a pay rise is a coroutine? We probably would not, but our software tools make us do so.

This is not necessary. For example, in Eiffel's SCOOP (previously discussed on this very blog) the change that is made to the model is that objects indicate when their collaborators can have separate execution contexts. They do not need to: turn off the parallel execution engine and everything runs sequentially. Otherwise, the details of how, and if, objects are running in different contexts are not part of the solution model.

Various other solutions rely on the call as a binding point in software execution, to allow the asynchronous nature to be injected as a connector between parts of the solution model. So in Objective-C or Smalltalk you might inject futures or queue confinement via proxy objects. Then your solution model hasn't got any noise about concurrent execution at all, but your implementation model still gets to choose where code gets run when one part of the solution sends a message to another part.

What's the difference? To me it's one of focus: if I want a clearly correct implementation of my solution model that might be an efficient program, then I would choose the connector approach: putting implementation details between my solution elements. If I want a clearly efficient program that might be a correct implementation of my solution model, then I would choose the annotation approach: putting implementation details on my solution elements. In my experience more of the software I've worked on has worked incorrectly than worked poorly, and when it's worked poorly I haven't wanted to change its correctness, so I would prefer to separate the two.

None of this is a complaint about any tools, or any libraries: you can build connectors when your tools give you annotations, by putting the annotations on things between your solution models. You can build annotations when your tools give you connectors, by hard-coding the use of connectors in the solution implementations. It's simply a question of what your design makes obvious.

#### On phone support scams and fake AV
A couple of weeks ago, I posted on Twitter about a new scam:

Heard about someone who was phoned by a man "from Windows" who engineered his way into remote access to the mark's computer.

Fast forward to now, the same story has finally been picked up by the security vendors and the mainstream media. This means it's probably time to go into more depth.

I heard a first-hand account of the scam. The victim is the kind of person who shouldn't be expected to understand IT security - a long distance lorry driver who uses his computer for browsing, e-mail, and that sort of thing. As he tells it, the person called, saying they were from Windows and that they had discovered his computer was infected. He was given instructions to give the caller remote access to help clean up the computer.

With remote access, the caller was able to describe some of the problems the victim was having with his computer, while taking control to "fix them". The caller eventually discovered that the victim's anti-virus was out of date, and that if he gave the caller his payment information he could get new software for &pound;109. This is when the victim hung up; however his computer has not booted properly since then.

I think my audience here is probably tech-savvy enough not to need warning about scams like this, and to understand that the real damage was done even before any discussion of payments was made (hint: browser form-auto-fill data). It's not the scam itself I want to focus on, but our reaction.

Some people I have told this story to in real life (it does happen) have rolled their eyes, and said something along the lines of "well of course the users are the weakest link" in a knowing way. If that's true, why rely on the users to make all the security decisions? Why leave it to them to decide what's legitimate and what's scammy, as was the case here? Why is the solution to any problem to shovel another bucketload of computer knowledge on them and hope that it sticks, as Sophos and the BBC have tried in the articles above?

No. This is not a solution to anything. No matter how loudly you shout about how that isn't how Microsoft does business, someone who says he is from Microsoft will phone your users up and tell them that it is.

This is the same problem facing anti-virus vendors trying to convince us not to get fooled by FakeAV scams. Vendor A tells us to buy their product instead of Vendor B's, because it's better. So, is Vendor A the FakeAV pedlar, or B? Or is it both? Or neither? You can't tell.

It may seem that this is a problem that cannot be solved in technology, that it relies on hard-wired behaviour of us bald apes. I don't think that's so. I think that it would be possible to change the way we, legitimate software vendors, interact with our users, and the way they interact with our software, such that an offline scam like this would never come to pass. A full discussion would fill a whole whitepaper that I haven't written yet. However, to take the most extreme point from it, the one I know you're going to loathe, what if our home computers were managed remotely by the vendors? Do most users really need complete BIOS and kernel level access to their kit? Really?

Look for the whitepaper sometime in the new year.

#### On platform-specific strategies
I'm writing some library code at the moment that needs to work on both Mac OS X and iOS. The APIs I need to use on each platform are different, so I need different code on each platform. I also happen to think that putting both versions of the code in the same implementation file is icky.

Here's what I'm doing. I define an abstract class FZAPlatformStrategy, that defines the interface and knows how to choose an appropriate subclass depending on the platform.

FZAPlatformStrategy.h
@interface FZAPlatformStrategy : NSObject {
@private
}
- (void)doThing;
+ (FZAPlatformStrategy *)newPlatformStrategy;

@end

FZAPlatformStrategy.m
@implementation FZAPlatformStrategy

- (void)doThing {
    [[NSException exceptionWithName: @"FZAPlatformStrategyAbstractClassException"
                            reason: @"Use +[FZAPlatformStrategy newPlatformStrategy] to get an appropriate subclass"
                          userInfo: nil] raise];
}

+ (FZAPlatformStrategy *)newPlatformStrategy {
    id strategy = nil;
#if TARGET_OS_IPHONE
    strategy = [[FZAPlatformStrategyiPhone alloc] init];
#else
    strategy = [[FZAPlatformStrategyMac alloc] init];
#endif
    return platformStrategy;
}

@end

OK, so now as you can imagine there is a class FZAPlatformStrategyiPhone that uses the iOS APIs, and another FZAPlatformStrategyMac that uses the OS X APIs. Each of these overrides the -doThing method to provide the appropriate platform-specific implementation. Of course, to get these to compile I conditionally define the whole @implementation contents of each based on the target platform.

The library code that needs to interface with this code just calls [FZAPlatformStrategy newPlatformStrategy]; to get an instance, and then because we've encapsulated the platform-specific behaviour it can use whatever it gets in a consistent way. Need to add another platform, say GNUStep/Linux? Define a new FZAPlatformStrategy subclass, and change +newPlatformStrategy to know when to return an instance of that subclass.

The remaining bit of ugly is the preprocessor conditional in +newPlatformStrategy. I decided to leave this as-is: it is at least highly localised ugly. The other approach I considered and discarded was to look for a platform-specific class like UIDevice being non-Nil, but then I realised that someone would probably implement that on the other platform and mess up the test.

Any other solutions for doing the "which platform am I on" test gratefully received.

#### On privacy, hashing, and your customers
I've talked before about not being a dick when it comes to dealing with private data and personally-identifying information. It seems events have conspired to make it worth diving into some more detail.

Only collect data you need to collect (and have asked for)

There's plenty of information on the iPhone ripe for the taking, as fellow iOS security boffin Nicolas Seriot discussed in his Black Hat paper. You can access a lot of this data without prompting the user: should you?

Probably not: that would mean being a dick. Think about the following questions.

Have I made it clear to my customers that we need this data?
Have I already given my customers the choice to decline access to the data?
Is it obvious to my customer, from the way our product works, that the product will need this data to function?

If the answer is "no" to any of these, then you should consider gathering the data to be a risky business, and the act of a dick. By the way, you'll notice that I call your subscribers/licensees "your customers" not "the users"; try doing the same in your own discussions of how your product behaves. Particularly when talking to your investors.

Should you require a long-form version of that discussion, there's plenty more detail on appropriate handling of customer privacy in the GSMA's privacy guidelines for mobile app developers.

Only keep data you need to keep

Paraphrasing Taligent: There is no data more secure than no data. If you need to perform an operation on some data but don't need to store the inputs, just throw the data away. As an example: if you need to deliver a message, you don't need to keep the content after it's delivered.

Hash things where that's an option

If you need to understand associations between facts, but don't need to be able to read the facts themselves, you can store a one-way hash of the fact so that you can trace the associations anonymously.

As an example, imagine that you direct customers to an affiliate website to buy some product. The affiliates then send the customers back to you to handle the purchase. This means you probably want to track the customer's visit to your affiliate and back into your purchase system, so that you know who to charge for what and to get feedback on how your campaigns are going. You could just send the affiliate your customer's email address:

X-Customer-Identifier: iamleeg%40gmail.com

But now everybody who can see the traffic - including the affiliate and their partners - can see your customer's email address. That's oversharing, or "being a dick" in the local parlance.

So you might think to hash the email address using a function like SHA1; you can track the same hash in and out of the affiliate's site, but the outsiders can't see the real data.

X-Customer-Identifier: 028271ebf0e9915b1b0af08b297d3cdbcf290e3c

We still have a couple of problems though. Anyone who can see this hash can take some guesses at what the content might be: they don't need to reverse the hash, just figure out what it might contain and have a go at that. For example if someone knows you have a user called 'iamleeg' they might try generating hashes of emails at various providers with that same username until they hit on the gmail address as a match.

Another issue is that if multiple affiliates all partner with the same third business, that business can match the same hash across those affiliate sites and build up an aggregated view of that customer's behaviour. For example, imagine that a few of your affiliates all use an analytics company called "Slurry" to track use of their websites. Slurry can see the same customer being passed by you to all of those sites.

So an additional step is to append a different random value called a salt to the data before you hash it in each context. Then the same data seen in different contexts cannot be associated, and it becomes harder to precompute a table of guesses at the meaning of each hash. So, let's say that for one site you send the hash of "sdfugyfwevojnicsjno" + email. Then the header looks like:

X-Customer-Identifier: 22269bdc5bbe4473454ea9ac9b14554ae841fcf3

[OK, I admit I'm cheating in this case just to demonstrate the progressive improvement: in fact in the example above you could hash the user's current login session identifier and send that, so that you can see purchases coming from a particular session and no-one else can track the same customer on the same site over time.]

N.B. I previously discussed why Apple are making a similar change with device identifiers.

But we're a startup, we can't afford this stuff

Startups are all about iterating quickly, finding problems and fixing them or changing strategy, right? The old pivot/persevere choice? Validated learning? OK, tell me this: why doesn't that apply to security or privacy?

I would say that it's fine for a startup to release a first version that covers the following minimum requirements (something I call "Just Barely Good Enough" security):

Legal obligations to your customers in whatever country those countries (and your data) reside
Standard security practices such as mitigating the OWASP top ten or OWASP mobile top ten
Not being a dick

In the O2 Labs I've been working with experts from various groups - legal, OFCOM compliance, IT security - to draw up checklists covering all of the above. Covering the baseline security won't mean building the thing then throwing it at a pen tester to laugh at all the problems: it will mean going through the checklist. That can even be done while we're planning the product.

Now, as with everything else in both product engineering and in running a startup, it's time to measure, optimise and iterate. Do changes to your product change its conformance with the checklist issues? Are your customers telling you that something else you didn't think of is important? Are you detecting intrusions that existing countermeasures don't defend against? Did the law change? Measure those things, change your security posture, iterate: use the metrics to ensure that you're pulling in the correct direction.

I suppose if I were willing to spend the time, I could package the above up as "Lean Security" and sell a 300-page book. But for now, this blog post will do. Try not to be a dick; check that you're not being a dick; be less of a dick.

#### On private methods
Let's invent a hypothetical situation. You're the software architect for an Objective-C application framework at a large company. This framework is used by many thousands of developers to create all sorts of applications for a particular platform.

However, you have a problem. Developer Technical Support are reporting that some third-party developers are using a tool called class-dump to discover the undocumented methods on your framework's classes, and are calling them directly in application code. This is leading to stability and potentially other issues, as the methods are not suitable for calling at arbitrary points in the objects' life cycles.

You immediately reject the distasteful solution of making the private method issue a policy problem. While you could analyse third-party binaries looking for use of undocumented method selectors, this approach is unscalable and error-prone. Instead you need a technical solution.

The problem in more detail

Consider the following class:

@interface GLStaticMethod : NSObject {
@private
    int a;
}
@property (nonatomic, assign) int a;
- (void)doTheLogThing;
@end

@interface GLStaticMethod ()
- (void)logThis;
@end

@implementation GLStaticMethod

@synthesize a;

- (void)doTheLogThing {
    [self logThis];
}

- (void)logThis {
    NSLog(@"Inside logThis: %d", self->a);
}

@end

Clearly this -logThis method would be entirely dangerous if called at unexpected times. Oh OK, it isn't, but let's pretend. Well, we haven't documented it in the header, so no developer will find it, right? Enter class-dump:

/*
 *     Generated by class-dump 3.3.2 (64 bit).
 *
 *     class-dump is Copyright (C) 1997-1998, 2000-2001, 2004-2010 by Steve Nygard.
 */

#pragma mark -

/*
 * File: staticmethod
 * Arch: Intel x86-64 (x86_64)
 *
 *       Objective-C Garbage Collection: Unsupported
 */

@interface GLStaticMethod : NSObject
{
    int a;
}

@property(nonatomic) int a; // @synthesize a;
- (void)logThis;
- (void)doTheLogThing;

@end

OK, that's not so good. Developers can find our private method, and that means they'll use the gosh-darned thing! What can we do?

Solution 1: avoid static discovery

We'll use the dynamic method resolution feature of the new Objective-C runtime to only bind this method when it's used. We'll put our secret behaviour into a function that has the same signature as an IMP (Objective-C method implementation), and attach that to the class when the private method is first used. So our class .m file now looks like this:

@interface GLStaticMethod ()
void logThis(id self, SEL _cmd);
@end

@implementation GLStaticMethod

@synthesize a;

+ (BOOL)resolveInstanceMethod: (SEL)aSelector {
    if (aSelector == @selector(logThis)) {
        class_addMethod(self, aSelector, (IMP)logThis, "v@:");
        return YES;
    }
    return [super resolveInstanceMethod: aSelector];
}

- (void)doTheLogThing {
    [self logThis];
}

void logThis(id self, SEL _cmd) {
    NSLog(@"Inside logThis: %d", ((GLStaticMethod *)self)->a);
}

@end

What does that get us? Let's have another look at class-dump's output now:

/*
 *     Generated by class-dump 3.3.2 (64 bit).
 *
 *     class-dump is Copyright (C) 1997-1998, 2000-2001, 2004-2010 by Steve Nygard.
 */

#pragma mark -

/*
 * File: staticmethod
 * Arch: Intel x86-64 (x86_64)
 *
 *       Objective-C Garbage Collection: Unsupported
 */

@interface GLStaticMethod : NSObject
{
    int a;
}

+ (BOOL)resolveInstanceMethod:(SEL)arg1;
@property(nonatomic) int a; // @synthesize a;
- (void)doTheLogThing;

@end

OK, so our secret method can't be found using class-dump any more. There's a hint that something special is going on because the class provides +resolveInstanceMethod:, and a really dedicated hacker could use otool to disassemble that method and find out what selectors it uses. In fact, they can guess just by looking at the binary:

heimdall:Debug leeg$ strings staticmethod 
NSAutoreleasePool
alloc
init
setA:
doTheLogThing
release
drain
logThis
resolveInstanceMethod:
v8@0:4
c12@0:4:8
i8@0:4
v12@0:4i8
NSObject
GLStaticMethod
Ti,N,Va
Inside logThis: %d

You could mix things up a little more by constructing strings at runtime and using NSSelectorFromString() to generate the selectors to test.

Problem extension: runtime hiding

The developers using your framework have discovered that you're hiding methods from them and found a way to inspect these methods. By injecting an F-Script interpreter into their application, they can see the runtime state of every object including your carefully-hidden instance methods. They know that they can call the methods, and can even declare them in categories to avoid compiler warnings. Where do we go from here?

Solution 2: don't even add the method
We've seen that we can create functions that behave like instance methods - they can get access to the instance variables just as methods can. The only requirement is that they must be defined within the class's @implementation. So why not just call the functions? That's the solution proposed in ProCocoaAppSec - it's a little uglier than dynamically resolving the method, but means that the method never appears in the ObjC runtime and can never be used by external code. It makes our public method look like this:

- (void)doTheLogThing {
    logThis(self, _cmd);
}

Of course, logThis() no longer has an Objective-C selector of its very own - it can only get the selector of the method from which it was called (or whatever other selector you happen to pass in). Most Objective-C code doesn't ever use the _cmd variable so this isn't a real drawback. Of course, if you do need to be clever with selectors, you can't use this solution.

Conclusion
Objective-C doesn't provide language-level support for private methods, but there are technological solutions for framework developers to hide internal code from their clients. Using these methods will be more reliable and easier to support than asking developers nicely not to use those methods, and getting angry when they do.

#### On programmer behaviours that make Scrum so bad
Respectable persons of this parish of Internet have been, shall we say, critical of Scrum and its ability to help makers (particularly software developers) to make things (particularly software). Ron Jeffries and GeePaw Hill have both deployed the bullshit word.

My own view, which I have described before, is that Scrum is a baseline product delivery process and a process improvement framework. Unfortunately, not many of us are trained or experienced in process improvement, not many of us know what we should be measuring to get a better process, so the process never improves.

At best, then, many Scrum organisations stick with the baseline process. That is still scadloads better than what many software organisations were trying before they had heard of Agile and Scrum, or after they had heard of it and before their CIO's in-flight magazine made it acceptable. As a quick recap for those who were recruited into the post-Agile software industry, we used to spend two months working out all the tasks that would go into our six month project. Then we'd spend 15 months trying to implement them, then we'd all point at everybody else to explain why it hadn't worked. Then, if there was any money left, we'd finally remember to ship some software to the customer.

Scrum has improved on that, by distributing both the shipping and the blaming over the whole life span of the project. This was sort of a necessary condition for software development to survive the dot-com crash, when "I have an idea" no longer instantly led to office space in SF, a foosball table, and thirty Herman Miller chairs. You have to deliver something of value early because you haven't got enough money any more to put it off.

So when these respectable Internet parishioners say that Scrum is bad, this is how bad it has to be to reach that mark. Both of the authors I cite have been drinking from this trough way longer than I have, so have much more experience of the before times.

In this article I wanted to look at some of the things I, and software engineering peers around me, have done to make Scrum this bad. Because, as Ron says, Scrum is systemically bad, and we are part of the system.
Focus on Velocity
One thing it's possible to measure is how much stuff gets shovelled in per iteration. This can be used to guide a couple of process-improvement ideas. First up is whether our ability to foresee problems arising in implementation is improving: this is "do estimates match actuals" but acknowledging that they never do, and the reason they never do is that we're too optimistic all the time.

Second is whether the amount of stuff you're shovelling is sustainable. This has to come second, because you have to have a stable idea of how much stuff there is in "some stuff" before you can determine whether it's the same stuff you shovelled last time. If you occasionally shovel tons of stuff, then everybody goes off sick and you shovel no stuff, that's not sustainable pace. If you occasionally shovel tons of stuff, then have to fix a load of production bugs and outages, and do a load of refactoring before you can shovel any more stuff, that's not a sustainable pace, even if the amount of work in the shovelling phase and the fixing phase is equivalent.

This all makes working out what you can do, and what you can tell other people about what you can do, much easier. If you're good at working out where the problems lie, and you're good at knowing how many problems you can encounter and resolve per time, then it's easier to make plans. Yes, we value responding to change over following a plan, but we also value planning our response to change, and we value communicating the impact of that response.

Even all of that relies on believing that a lot of things that could change, won't change. Prediction and forecasting aren't so chaotic that nobody should ever attempt them, but they certainly are sensitive enough to "all things being equal" that it's important to bear in mind what all the things are and whether they have indeed remained equal.

And so it's a terrible mistake to assume that this sprint's velocity should be the same as, or (worse) greater than, the last one. You're taking a descriptive statistic that should be handled with lots of caveats and using it as a goal. You end up doing the wrong thing.

Let's say you decide you want 100 bushels of software over the next two weeks, because you shovelled 95 bushels of software last week. The easiest way to achieve that is to take things that you think add up to 100 bushels of software, and do less work so that they contain only, say, 78 bushels, and try to get those 78 bushels done in the time.

Which bits do you cut off? The buttons that the customer presses? No, they'll notice that. The actions that happen when the customer presses the buttons? They'll probably notice that, too. How about all the refinement and improvement that will make it possible to add another 100 bushels next time round? Nobody needs that to shovel this 100 bushels in!

But now, next time, shovelling software is harder, and you need to get 110 bushels in to "build on the momentum". So more corners get cut, and more evenings get worked. And now it's even harder to get the 120 bushels in that are needed the next time. Actual rate of software is going down, claimed rate of software is going up: soon everything will explode.
Separating "technical" and "non-technical"
Sometimes also "engineering" and "the business". Particularly in a software company, this is weird, because "the business" is software engineering, but it's a problem in other lines of business enabled by software too.

Often, domain experts in companies have quite a lot of technical knowledge and experience. In one fintech where I used to work, the "non-technical" people had a wealth (pun intended) of technical know-how when it came to financial planning and advice. That knowledge is as important to the purpose of making financial technology software as is software knowledge. Well, at least as important.

So why do so many teams delineate "engineering" and "the business", or "techies" and "non-technical" people? In other words, why do software teams decide that only the software typists get a say in what software gets made using what practices? Why do people divide the world into pigs and chickens (though, in fairness to the Scrum folks, they abandoned that metaphor)?

I think the answer may be defensiveness. We've taken our ability to shovel 50 bushels of software and our commitment (it used to be an estimate, but now it's a commitment) to shovel 120 bushels, and it's evident that we can't realistically do that. Why not? Oh it must be those pesky product owners, keep bringing their demands from the VP of marketing when all they're going to do is say "build more", "shovel more software", and they aren't actually doing any of it. If the customer rep didn't keep repping the customer, we'd have time to actually do the software properly, and that would fix everything.

Note that while the Scrum guide no longer mentions chickens and pigs, it "still makes a distinction between members of the Scrum team and those individuals who are part of the process, but not responsible for delivering products". This is an important distinction in places, but irrelevant and harmful in others. But it's at its most harmful when it's too narrowly drawn. When people who are part of the process are excluded from the delivering-products cabal. You still need to hear from them and use their expertise, even if you pretend that you don't.

The related problem I have seen, and been part of, is the software-expertise people not even gaining passing knowledge of the problem domain. I've even seen it working on a developer tools team where the engineers building the tools didn't have cause to use, or particularly even understand, the tool during our day-to-day work. All of those good technical practices, like automated acceptance tests, ubiquitous language, domain-driven design; they only mean shit if they're helping to make the software compatible with the things people want the software for.

And that means a little bit of give and take on the old job boundaries. Let the rest of the company in on some info about how the software development is going, and learn a little about what the other folks you work with do. Be ready to adopt a more nuanced position on a new feature request than "Jen from sales with another crazy demand".
Undriven process optimisation
As I said up top, the biggest problem Scrum often encounters in the wild is that it's a process improvement framework run by people who don't have any expertise at process improvement, but have read a book on Scrum (if that). This is where the agile consultant / retrospective facilitators usually come in: they do know something about process improvement, and even if they know nothing about your process it's probably failing in similar enough ways to similar processes on similar teams that they can quickly identify the patterns and dysfunctions that apply in your context.

Without that guidance, or without that expertise on the team, retrospectives are the regular talking shop in which everybody knows that something is wrong, nobody knows what, and anybody who has a pet thing to try can propose that thing because there are no cogent arguments against giving it a go (nor are there any for it, except that we've got to try something!).

Thus we get resume-driven development: I bet that last sprint was bad because we aren't functionally reactive enough, so we should spend the next sprint rewriting to this library I read about on medium. Or arbitrary process changes: this sprint we should add a column to the board, because we removed a column last time and look what happened. Or process gaming: I know we didn't actually finish anything this month, but that's because we didn't deploy so if we call something "done" when it's been typed into the IDE, we'll achieve more next month. Or more pigs and chickens: the problem we had was that sales kept coming up with things customers needed, so we should stop the sales people talking either to the customers, to us, or to both.

Work out what it is you're trying to do (not shovelling bushels of software, but the thing you're trying to do for which software is potentially a solution) and measure that. Do you need to do more of that? Or less of it? Or the same amount for different people? Or for less money? What, about the way you're shovelling software, could you change that would make that happen?

We're back at connecting the technical and non-technical parts of the work, of course. To understand how the software work affects the non-software goals we need to understand both, and their interactions. Always have, always will.

#### On protocols that aren't
There's a common assumption when dealing with Objective-C protocols or Java interfaces (or abstract classes, I suppose): that you're abstracting away the implementation of an object leaving just its interface. "Oh, don't mind how I quack, all you need to know is that I do quack". This assumption is unwarranted.

All protocols/interfaces actually tell you is that there is a list of messages, and you'll be given an object that responds to those messages. Assuming that you can use it in the same way as any other object that responds to the same messages is a mistake. This is something I've written about before, but it's time to give a concrete example.

I've been debugging some multi-threaded code recently, so I've been writing my own lock classes to try and inspect how the different threads interact. They implement the NSLocking protocol, as all the Foundation lock classes do:

  The NSLocking protocol declares the elementary methods adopted by classes that define lock objects. A lock object is used to coordinate the actions of multiple threads of execution within a single application. By using a lock object, an application can protect critical sections of code from being executed simultaneously by separate threads, thus protecting shared data and other shared resources from corruption.

It defines two methods, -lock and -unlock. So, given I have an object that conforms to NSLocking, I can call -lock and -unlock just as I would on any other object that conforms to the protocol, right?

Wrong. With an NSRecursiveLock (which conforms to the protocol), I can call -lock multiple times in a row without calling -unlock. Given an NSLock (which also conforms to the protocol), I can't. With NSLock I can't call -unlock on a different thread than I called -lock; I could write a class where that is permitted.

As I argued in the above-linked post, protocols are actually a poor approximation for what we want to use them as: agreements on how to interact with an object via its messaging interface.

#### On rational myths
In my research field, one characteristic of institutions is their “rational myths”; ideas that people tell each other are true, and believe are true, but which are under-explored, unverified, and under-challenged. Belief in these myths leads to supposedly rational actions that don’t necessarily improve efficiency or performance, but are done because everyone else does them, and everyone collectively believes they’re what one does.

We know, from Derek Jones’s Evidence-based software engineering, that what we know about software engineering is not very much. So what are the rational myths where you work? Do you recognise them? Could you change them? What would it take to support or undermine your community’s rational myths, and would you want to take that risk?

#### On repeatable builds
One of the key features of software engineering, as distinct from cowboy coding or hacking, is that it should be repeatable. That doesn't mean that you should do the same project twice in identical ways from beginning to end: that would be a waste of time (and you can bet the requirements have moved the second time around). But it does mean that two people, investigating the same project at the same point in the project, should be able to reproduce the same results.

What does that mean for builds?

If you ask me to investigate a bug a customer reported with version 1.5.3 of your app, then I should be able to build version 1.5.3 of your app even if you're now working on version 3.0.2. The product I end up with by building version 1.5.3 should be exactly the same as the one you'd end up with, and also exactly the thing the customer was using.

In practice, this means that your build should be as simple as possible. Given access to your source repository, any developer should be able to see what they need to check out, to correspond to a particular version of your app (a released version, a development branch, whatever). They should then hit the one button that gets that product built.

If there are any dependencies in the build process, these should be handled by the build process. If a particular version of your app uses a particular version of a framework or plug-in, that should be codified in the source for that version of your app. Leave nothing to chance, external configuration or intelligent interaction. Which are basically the same thing.

Why should I care?

Well, for a start, if you don't know that the thing you're debugging and the thing your customer's using are the same, then you don't know whether you can even find the customer's bug, let alone fix it. There are ancillary benefits too.

Get new developers up to speed quickly

One of the things that I do to differentiate from other security consultants is offer a half-day "I just want you to fix this problem" service, which is a bit like Apple DTS or MSDN Technical Incidents for security (but with a security boffin on tap, and also available on Android ;->).

Now, if you hire me for half a day, you probably want me to spend that half-day solving your problem. I certainly want to spend my time that way. Neither of us wants me to spend a couple of hours trying to work out how your product's built. Complicated build procedures are the top reason for failed/unreproducible builds. Every time a developer has to think about how to build the product, that's a point where he can introduce a mistake.

Anyone can turn out a new release

You've just found out about a crashing bug in your million-dollar app, but you're already on that Carribean cruise you've bought pre-emptively with the profits. No problem, you use the satellite phone to call a junior developer, and tell him to fix the bug and submit a new build to iTunes Connect.

...can he do it? Will he get it right? If not, can your company last the two weeks until you get back from burning the profits?

You can migrate to new hardware

Maybe you finally got that new MacBook Air you've been lusting after, or the Mac Pro to go in your evil lair. Or your dev computer just died, and you need to get a new one up and running. And your backups failed.

If you forget how to set up your dev environment, or you get it a bit wrong, you'll waste time and get angry. Then you'll get it more wrong.

You can automate your builds

You've seen those open source projects like WebKit that do nightly builds? I do that too. In fact, I have a build triggered whenever I commit source code to version control. If the build succeeds, it runs some tests. If any of that fails, I get an e-mail.

It's called Continuous Integration, and provides very rapid feedback on the "health" of a project. But to get it done, you need for your builds to be unattended.

OK, I'm convinced. How do I do it?

Let's take a look at the independent parts of your typical iOS app, and see what it would take to ensure that we always get the same version of them for every build.

Source code

You already use version control, right? Right. Do you tag your releases?

A tag in git or subversion (or any other version control system that supports them, but life's too short) is just a way to name a particular commit. In git, it really is just a name (and optionally some other info) attached to a certain commit. In subversion, it's actually a new branch, but one that you shouldn't commit any changes to.

So let's say you've just finished version 1.0, it doesn't contain any known bugs, and it's time to submit it to the store. You should tag the version of the source that corresponds to what you submit as the version 1.0 candidate.

Now, if you ever need to go back to version 1.0, to address an issue reported by the store reviewers, or to investigate a bug reported by a customer, you just check out that tagged version of your source. If you need another developer to look into something related to version 1.0, then she checks out that tagged version.

Of course, checking out a particular tag only gets you the same built product if the source under version control represents a complete definition of the product. Let's see a few cases where that might not be true.

Subprojects

Apps sometimes contain classes that have been developed as a separate project, particularly collections of helper or utility classes.

Building version 1.0 of the app requires building whatever version of the helpers was used in version 1.0. If you change the two independently, and don't have a way to keep both in sync, then you don't have a way to build the same product again. That sucks.

Building a particular version of your app should automatically bring along the correct version of a subproject. The easiest way to do that is with git submodules or its equivalent in your preferred SCM. If that's not possible for whatever reason, then you can use a shell script as part of the build process. Keep the shell script in version control so you get the correct version of the script that checks out the correct version of the subproject.

Building your app should automatically build the subproject. There is nothing worse than sitting and waiting for a build, only to find random link errors at the end. You send an email to the project maintainer, then after the weekend's over she sends a reply saying "yeah, you need to build this project first". Gah!

In Xcode, adding a subproject is as simple as dragging the subproject's file into the list of files in your main project. You then edit your app target, and add the target for the subproject as a dependency for the app target. Now Xcode checks when you build the app whether it needs to build the subproject first. You're guaranteed to link against an up-to-date version, not whatever old cruft Xcode found in your search paths.

Third-party frameworks/libraries

Sometimes, your project depends on someone else's framework or library. You have that library, but you don't have the source to it.

Check the headers and the binary for the library into version control. Yes, it's icky. But now, when someone else checks out a particular revision of your source, they automatically get the correct revision of the library too, and compile with the correct headers.

Set up the header/library search paths for your Xcode target such that Xcode searches the folder where the revision-controlled library too, not whatever nasty old version you, I or someone else happens to have knocking around /usr/local/. This path should be relative to $(SRCROOT), i.e. it should say how to get from where your project is to where that library is.

It should not be an absolute path. Other developers don't have the same path structure as you. They probably have a different username, for a start, so /Users/leeg/Library/Frameworks doesn't exist for them. They may keep your source in a disk image. If you're using continuous integration, the source probably gets checked out into a temporary workspace that moves every time. They key thing is don't rely on the environment being the same for every build, rely on the checked-out source being sufficient to completely describe the build.

Interface Builder plug-ins

More common in the Mac world than iOS, IB plugins provide access to custom objects in Interface Builder and allow you to add them to your XIBs, inspect their properties and so on. If Xcode can't find an ibplugin needed to compile a XIB, then builds start failing.

Again, check the IB plugin (or the source needed to build it) into version control. If you commit the source, don't forget to make it a dependency of your app target so it definitely gets built when people try to build your app.

You don't need an IB Plugins installed to build an Xcode project that relies on it! In your app's target editor, you can set the paths to the required plug-ins or the search paths for ibtool to find plug-ins when it needs them. Remember to make these paths relative.

A developer trying to edit your XIBs will need to install the plug-in, but now he knows where to find it: he takes the version that was checked out when he checked out your app.

That all sounds hard to set up.

Not really. Create a new user account on your Mac, check out your app, and build it. Fix the problems until there are none. If you can, find a Mac you've never coded on before and repeat the same process.

Conclusion

You can avoid wasting a lot of time, and having failed or incorrect builds, by instituting a simple, repeatable build process. There should be exactly one thing to check out of version control, and important versions of that thing should be tagged. Opening the Xcode project that was in version control and hitting 'build' should be all that's needed to get your product built, no matter how complicated the project and how many libraries it depends on. Hitting that 'build' button on one version of the source should always build the same product, on anybody's computer.

#### On rewriting your application
I'm really far behind on podcasts. I have a long commute, and listen to one audiobook every month, filling the slack time with a selection of podcasts. It happens that between two really long books (Cryptonomicon by Neal Stephenson and The Three Musketeers by Alexandre Dumas, both of which I'd recommend) and quite a few snow days I've managed to fall behind.

This is the context in which I listened to Episode 77 of iDeveloper Live, on whether to rewrite or not rewrite an application. I was meant to be on that program but had to pull out due to a combination of being ill and pressing on with writing what would become Discworld: the Ankh-Morpork Map. I imagine that one of these two factors was the cause of the other.

Anyway, listening to the podcast made me decide to put my case forward. It's unfortunate that I wasn't part of the live discussion but I hope that what I have to say still has value. I'd recommend listening to what Scotty, Uli, John and Pilky have to say on the original podcast, too. If I make references to the discussion in the podcast, I'll apologise in advance for failing to remember which person said what.

The view from the outside

The point that got me thinking about this was the marketing for a version x.0 of an app. I don't remember what the app was, I think Danny Greg probably posted the link to it. The release described how the app was "completely rewritten" from the previous version; an announcement that's not uncommon in software release notes.

My position is this: A complete rewrite is neither a feature nor a benefit. It's a warning. It says "warning: you might like the previous version, in fact that's probably why you're buying it, but what we're now selling you has nothing in common with it". It says "warning: the workflow you're accustomed to in this app may no longer work, or may have different bugs than the ones you've discovered how to work around". It says "warning: this product and the previous version are similar in name alone". This is what I infer when I read that your product has been rewritten.

The view from the inside

As programmers, we're saying "but I've got to rewrite this, it's so old and crappy". Why is it old and crappy? Is it because we weren't trained to write readable code, and we weren't trained to read code? Someone on the podcast referred to the "you should hate code you wrote six months ago or you're not learning" trope. No. You should look at code you were writing six months ago and see how it could be improved. You should be able to decide whether to make those improvements, depending on whether they would benefit the product.

Many of the projects I've worked on have taken more than six months to complete. In each case, we could have either released it, or we could still be in a cycle of finding code that was modified more than six months ago, looking at it in disgust, throwing it away and writing it again&mdash;and waiting for the new regression bug reports to come in.

Bear in mind that source code is a liability, not an asset. When you tear something out to rewrite it from scratch, you're using up time and money to create a thing that provides the same value as the thing you're replacing. It's at times like this that we enjoy waving our hands and talking about Technical Debt. Martin Fowler:

The tricky thing about technical debt, of course, is that unlike money it's impossible to measure effectively. The interest payments hurt a team's productivity, but since we CannotMeasureProductivity, we can't really see the true effect of our technical debt.

We can't really see the true effect. So how do you know that this rewrite is going to pay off? If you have some problem now it might be clear that this problem can be addressed more cheaply with a rewrite than by extending or modifying existing code. This is the situation Uli described in the podcast; they'd used some third-party library in version 1, which got them a quick release but had its problems. Having learned from those problems, they decided to replace that library in version 2.

Where you have problems, you can solve them either by modification or by rewriting. If you think that some problem might occur in the future, then leave it: you don't make a profit by solving problems that no-one has.

A case study

While I had a few jobs in computing beforehand, all of which required that I write code, my first job where the title meant "someone who writes code" started about six years ago, at a company that makes anti-virus software. I was a developer (and would quickly become lead developer, despite not being ready) on the Mac version of the software.

This software had what could be described as an illustrious history: it was older than some of the readers of this blog (which is not uncommon: Cocoa is old enough to vote in the UK and UNIX is middle-aged). It started life as a Lightspeed/THINK C product, then become a PowerPlant product at around the time of the PowerPC transition. When I started working on it in 2007 the PowerPlant user interface still existed, but it did look out of place and dated. In addition, Apple were making noises about the library not being supportable on future versions of Mac OS X, so the first project for the new team was to build a new UI in Cocoa.

We immediately set out getting things wrong more quickly than any other team in the company. The lead developer when I joined had plenty of experience on the MS-DOS and Windows versions of the product, but had never worked on a Mac nor in Objective-C: then I became the lead developer, having worked in Objective-C but never in a team of more than one person. I won't go into all of the details but the project ended up taking multiple times its estimated duration: not surprising, when the team had never worked together and none of the members had worked on this type of project so our estimates were really random numbers.

At the outset of this project, being untrained in the reading of other people's code, I was dismayed by what I saw. I repeatedly asked for permission to rewrite other parts of the system that had copyright dates from when Kurt Cobain was still making TV appearances. I was repeatedly refused: the correct decision as while the old code had its bugs it was a lot more stable than what we were writing, and as it had already been written it cost a lot less to supply to the customer[*].

Toward the eventual end of the project, I asked my manager why we hadn't given up on it after a year of getting things wrong, declared that a learning exercise and started over. Essentially, why couldn't we take the "I hate the code I wrote last year, let's start from scratch" approach. His answer was that at least one person would've got frustrated and quit after having their code thrown away; then we'd have no product and also no team so would not be in a better position.[*]

Eventually the product did get out of the door, and while I'm no longer involved with it I can tell that the version shipping today still has most of the moving parts that were developed during my time and before. Gradual improvement, responding to changes in what customers want and what suppliers provide, has stood that product in good stead for over two decades.

[*] It's important to separate these two arguments from the Sunk Cost Fallacy. In neither case are we including the money spent on prior work. The first paragraph says "from today's perspective, what we already have is free and what you haven't written is not free, but they both do the same thing". The second paragraph says "from today's perspective, finishing what you've done costs a lot of money. Starting afresh costs a lot of money and introduces social disruption. But they both do the same thing."

#### On running out of words
John Gruber's subscription to Wiktionary expired:

  At just 20 percent of unit sales, Apple isn’t even close to a monopoly. At 92 percent profit share, they have a market dominance that rivals any actual monopoly the tech industry has ever seen. We don’t even have a term for this situation, it’s so unusual.

We do have a term: monopoly will do just fine. Gruber says that Apple "isn't even close to a monopoly", but you don't need to have all or even most of the unit sales in a market in order to be able to act monopolistically. An entity (or a cabal) only needs a big enough share of the sales in order to be able to set prices independent of the other competitors in the market. (Working at big telecoms companies has the effect of teaching you specifics of market economics, but then so did those economics classes I took at University.)

That 92% profit on 20% sales is indicative, rather than contraindicative, of a monopoly. And there's another word we could use, too: monopsony. Let's say that you've made an iOS app, and now you want to sell it. Do you create a storefront on your website to do that? Do you contact Sears and see how many boxes they want? Speak to some third-party distributor? No, you can only sell to Apple, they are the only buyer for iOS apps.

The thing it's important to remember about monopolies or monopsonies is that they are not inherently bad: badness happens when an entity uses its dominant position in a market to set prices or other terms that are not considered fair, and that's a pretty woolly situation. When the one buyer in your market decides that your contribution is "amateur hour" (sucks to be a hobbyist, I guess), or that your content is "over the line", and doesn't want to buy your product, you have no other vendors to sell it to: is that fair?

This is an argument that relies too much on legal details and nuance to be able to answer as a novice, so I'll spare you my "amateur hour" pontification. I would imagine that a legal system that did explore this question would consider analogous environments, like the software market of the 1990s. Back then, Microsoft bundled a web browser and a media player with their operating systems and used their market power (which let them act as a monopoly even though competitors existed) as an operating system vendor to make it hard to sell competing browsers or media players. It might be an interesting thought experiment to compare that situation with today's.

#### On saying words clearly
Someone has been trolling Apple’s Siri team hard on how they think numbers are pronounced. Today is the second day where I’ve missed a turn due to it. The first time because I didn’t understand the direction, the second because the pronunciation was so confusing I lost focus and drove straight when I should have turned.

The disembodied voice doesn’t even use a recognisable dialect or regional accent, it just gets road numbers wrong. In the UK, there’s a hierarchy of motorways (M roads, like M42), A roads (e.g. A34), B roads (e.g. B3400), and unclassified roads. It’s a little fluid around the edges, but generally you’d give someone the number of an M or A road if you’re giving them directions, and the name of a B road.

Apple Maps has always been a bit weird about this, mostly preferring classifications but using the transcontinental E route numbers which aren’t on signs in the UK and aren’t used colloquially, or even necessarily known. But now its voice directions pronounce the numbers incomprehensibly. That’s ok if you’re in a car and the situation is calm enough that you can study the CarPlay screen to work out what it meant. But on a motorbike, or if you’re concentrating on the road, it’s a problem.

“A” is pronounced “uh”, as if it’s saying “a forty-six” rather than “A46”. Except it also says “forrysix”. Today I got a bit lost going from the “uh foreforryfore” to the “bee forryaytoo” and ended up going in, not around, Coventry.

Entering Coventry should always be consensual.

I’ve been using Apple Maps since the first version which didn’t even know what my town was called, and showed a little village at the other end of the county if you searched for it by name. But with the successive apologies, replatformings, rewrites, and rereleases, it always seems like you take one step forward and then at the roundabout use the fourth exit to take two steps back.

#### On Scarcity
It's called scarcity, and we can't wait to see what you do with it.

Let's start with the important bit. I think that over the last year, with acceleration toward the end of the year, I have heard of over 100,000 software engineers losing their jobs in some way. This is a tragedy. Each one of those people is a person, whose livelihood is at the whim of some capricious capitalist or board of same. Some had families, some were working their dream jobs, others had quiet quit and were just paying the bills. Each one of them was let down by a system that values the line going up more than it values their families, their dreams, and their bills.

While I am sad for those people, I am excited for the changes in software engineering that will come in the next decade. Why? Because everything I like about computers came from a place of scarcity in computering, and everything I dislike about computers came from a place of abundance in computering.

The old, waterfall-but-not-quite, measure-twice-and-cut-once approach to project management came from a place of abundance. It's cheaper, so the idea goes, to have a department of developers sitting around waiting for a functional specification to be completed and signed off by senior management than for them to be writing working software: what if they get it wrong?

The team at Xerox PARC - 50 folks who were just told to get on with it - designed a way of thinking about computers that meant a single child (or, even better, a small group of children) could think about a problem and solve it in a computer themselves. Some of those 50 people also designed the computer they'd do it on, alongside a network and some peripherals.

This begat eXtreme Programming, which burst onto the scene in a time of scarcity (the original .com crash). People had been doing it for a while, but when everyone else ran out of money they started to listen: a small team of maybe 10 folks, left to get on with it, were running rings around departments of 200 people.

Speaking of the .com crash, this is the time when everyone realised how expensive those Oracle and Solaris licenses were. Especially if you compared them with the zero charged for GNU, Linux, and MySQL. The LAMP stack - the beginning of mainstream adoption for GNU and free software in general - is a software scarcity feature.

One of the early (earlier than the .com crash) wins for GNU and the Free Software Foundation was getting NeXT to open up their Objective-C compiler. NeXT was a small team taking off-the-shelf and free components, building a system that rivalled anything Microsoft, AT&amp;T, HP, IBM, Sun, or Digital were doing - and that outlived almost all of them. Remember that the NeXT CEO wouldn't become a billionaire until his other company released Toy Story, and that NeXT not only did the above, but also defined the first wave of dynamic websites and e-commerce: the best web technology was scarcity web technology.

What's happened since those stories were enacted is that computerists have collectively forgotten how scarcity breeds innovation. You don't need to know how 10 folks round a whiteboard can outsmart a 200 engineer department if your department hired 200 engineers _this month_: just put half of them on solving your problems, and half of them on the problems caused by the first half.

Thus we get SAFe and Scrumbut: frameworks for paying lip service to agile development while making sure that each group of 10 folks doesn't do anything that wasn't signed off across the other 350 groups of 10 folks.

Thus we get software engineering practices designed to make it easier to add code than to read it: what's the point of reading the existing code if the one person who wrote it has already been replaced 175 times over, and has moved teams twice?

Thus we get not DevOps, but the DevOps department: why get your developers and ops folks to talk to each other if it's cheaper to just hire another 200 folks to sit between them?

Thus we get the npm ecosystem: what's the point of understanding your code if it's cheaper just to randomly import somebody else's and hire a team of 30 to deal with the fallout?

Thus we get corporate open source: what's the point of software freedom when you can hire 100 people to push out code that doesn't fulfil your needs but makes it easier to hire the next 500 people?

I am sad for the many people whose lives have been upended by the current downturn in the computering economy, but I am also sad for how little gets done within that economy. I look forward to the coming wave of innovation, and the ability to once again do more with less.

#### On Scientific Computing
Or: Not everyone works the way you work

Currently doing the rounds on Twitter is a paper from the ArXiV called Best Practices for Scientific Computing—a paper with 13 authors and 6 pages,including a page-long collection of references. Here's the abstract:

  Scientists spend an increasing amount of time building and using software. However, most scientists are never taught how to do this efficiently. As a result, many are unaware of tools and practices that would allow them to write more reliable and maintainable code with less effort. We describe a set of best practices for scientific software development that have solid foundations in research and experience, and that improve scientists’ productivity and the reliability of their software.

Let me start with an anecdote. It's 2004, and I've just started working as a systems manager in a university computing lab. My job is partly to maintain the computers in the lab, partly to teach programming and numerical computing to Physics undergraduates, and partly to write software that will assist in said teaching. As part of this work I started using version control, both for my source code and for some of the configuration files in /etc on the servers. A more experienced colleague saw me doing this and told me that I was just generating work for myself, that this wasn't necessary for the small things I was maintaining.

Move on now to 2010, and I'm working in a big scientific facility in the UK. Using software and a lot of computers, we've got something that used to take an entire PhD to finish down to somewhere between one and eight hours. I'm on the software team, and yes we're using version control to track changes to the software and to understand what version is released. Well, kindof, anyway. The "core" is in version control, but one of its main features is to provide a scripting environment and DSL in which scientists at the "lab benches", if you will, can write up scripts that automate their experiments. These scripts are not (necessarily) version-controlled. Worse, the source code is deployed to the experimental stations so someone who discovers a bug in the core can fix it locally without the change being tracked in version control.

So, a group does an experiment at this facility, and produces some interesting results. You try to replicate this later, and you get different results. Could be software-related, right? All you need to do is to use the same software that the original group used…unfortunately, you can't. It's gone.

That's an example of how scientists failing to use the tools from software development could be compromising their science. There's a lot of snake oil in the software field, both from people wanting you to use their tools/methodologies because you'll pay them for it, and from people who have decided that "their" way of working is correct and that any other way is incorrect. You need to be able to cut through all of that nonsense to find out how particular tools or techniques impact the actual work you're trying to achieve. Current philosophy of science places a high value on reproducibility and auditing. Version control supports that, so it would be beneficial for programmers working in science to use version control.

But version control is only one of the 10 recommendation sections in this paper (another is about using the computer to record history, something that I'll assume is covered well by the above discussion). That leaves eight other sections, which each contain numbered pronouncements about how scientists should write software.

Were you surprised?

I expect, if you write software in the commercial sector, you wouldn't find any of their suggestions contentious: examples include naming things meaningfully, using a consistent convention for names and layout, don't repeat yourself, and so on. I included this paper here to start discussion of an important point.

What goes on in commercial software engineering is not the be-all and end-all of software development. Scientific software has been around for as long as there have been computers to run software on, and indeed not only is some really old software still in use but the people who wrote it are still around and maintaining it. In the aforementioned university lab, one of my tasks was to help a professor who'd been using his home-grown FORTRAN FITS manipulation routines for at least two decades. Every system he'd used it on—most recently PowerPC, MIPS and Alpha workstations—had been big-endian and he didn't know why it gave the wrong results when used on our new Intel Mac. His postdocs and PhD students were using the same code—in the same FORTRAN language, which he'd either taught them or given them a book on. And then of course when they moved to a different institution they'd take that code and that understanding of code with them.

I imagine that many professional programmers are not surprised by the validity of (m)any of the statements made in this paper, but by the necessity of stating them. No, not everyone uses version control, or thinks that agile is the best thing ever, or uses consistent naming conventions throughout a source file. Indeed in my experience of scientific programming, use of a symbolic debugger wasn't If you consider all of these problems to be "solved" then you're really only looking at a limited part of the world of software development. It's not just scientific computing that doesn't match that worldview; what about all the people out there for whom programming is a bunch of Excel formulae and maybe the odd VBA macro pasted from a website?

In both commercial and scientific software development, understanding and behaviour is spread by sharing knowledge from masters to apprentices. I think that the reason there's such a big difference in practice could be due to the longer generations in scientific software. That 20+-year-old FITS code still works, why change it? And those 20+-year-old practices that created the FITS code, well they still work too, don't they?

Which of these things actually matters?

Based on my own experience I'd assert that all of them are important things for scientific programmers to know about. I've argued, hopefully convincingly, that version control has an important part to play in the scientific process: numerical analysis is a key part of many experiments and like the rest of the method it should be available for inspection and repetition. Science is also a collaborative activity, so it makes sense that some of the recommendations would be about collaboration: document the purpose of the code instead of its mechanics, write programs for people.

Could I justify those assertions with figures? Probably not. Is that important? Well, actually it probably is. Of the researchers I've worked with (bear in mind this has always been in Physics), even many who are heavily invested in computational methods see programming (rightly) as a means to an end, and aren't likely to try new-to-them techniques in programming just for the sake of programming. Despite any rational economic benefit, they'd rather stick with what they know and focus on getting new results without any surprises.

If you want to say "it's better to work this way" or "you'll get results quicker like this", to a bunch of physicists, you have to show them data to prove it. A paper like the one I'm discussing here will likely be read, if it:

gets published
said publication happens in a relevant journal
said publication is picked up and circulated in enough news sources that researchers who don't read the publishing journal get wind

On the other hand, it's likely that the article's tone will ensure that it only preaches to the converted. Nothing in the paper says "this is actually better", just "professional programmers do these things". Exercises like Software Carpentry are likely to only appeal to people who already have an interest in bettering their own programming abilities. As I said, most researchers I know don't: they want to publish, and programming is a necessary—albeit complex—tool helping them to achieve that.

Why is this suddenly an issue?

It isn't. A very quick search for errors in scientific computing yielded papers published across the last two decades, and I could probably find more. The abstracts for these (I did say it was a very quick search) include some pining for the use of skills from software engineering, or a closer focus by software engineering researchers on scientific computing projects.

What can be done about it?

That's a very good question. If we knew what to do to improve the quality of any software production effort, there'd be a lot more good software in the world :). If the techniques from commercial software really would help make scientific software better, why wait for the scientists to apply them? Plenty of scientific software is open source, so in the case of things like analysis tools and automatic tests, sufficiently motivated individuals could just apply those things then demonstrate to the project maintainers how much of a difference they've made. Sure, there will be problems: I once worked on some software that could only be successfully executed if there was a particle accelerator connected to your workstation. But the first thing I did was to make a virtual particle accelerator - demonstrating how much easier it was to make progress if you could do it away from the experiment.

This brings me onto another option: scientific computing teams can employ commercial developers. I've seen it happen, I've seen it work and I've seen it fail. The ways in which it work include sharing of knowledge from both disciplines, discussing and improving practices. The ways in which it fails come down to frustration on both sides: scientific programmers feel that refactoring is change for change's sake, perhaps, and software engineers think that not using their favourite practices is the realm of cowboys. That means that for a cross-discipline software team to work, it needs good leadership: the team needs to be designed to appreciate the different skills and viewpoints brought by the different members. And now we've gone fully out of the realm of science into management techniques.

#### On secrets
Secrets are hard. Especially in the digital domain, but we can see examples in other environments too. Let's take a look at a couple of historical examples.

It used to be the case that all of Britain's diplomatic traffic was safe from snooping. Why? The information was all conveyed over the telegraph system, and Britain controlled the telegraph system. Customer states could buy access to send and receive their own signals over the network. This of course meant that Britain could snoop on their signals. Where the cables went through neutral (or supposedly neutral) countries, said countries and their allies (who were not necessarily Britain's allies) could also snoop on the traffic. Wait, didn't I say this was a safe channel?

Even were the telegraph system snoop-proof, the telegraph operators might not be. The recipients of any message might not be. Come to that, neither might the senders. Because the British foreign office knew the communications to be secret, everyone else knew that this was where to look for their secrets.

Conversely, it has never been assumed that knowing how to make a nuclear weapon is an unknown secret. It's trivial for anyone to get the plans to a nuke, and if you need parts, just look at the United States export restrictions documents and order those parts from Germany. So how come no-one has been nuked in 65 years? How come Al Qaeda aren't busy nuking the western world, if they know how to do it? Because while nuking is easy to know, it's hard to do. Acquiring the fuel is hard enough for most states, never mind small terrorist cells. And then getting the fuel into a bomb and the bomb into a target without incident is so hard that it's not worth doing.

Conclusion? Often, making things secret isn't sufficient. Secrecy is fleeting. Making it hard to use a preferably-secret fact can be more effective than ramping up the secrecy.

#### On self-taught coders
When a programmer says that they are ‘self-taught’ or that they “taught themselves to code”, what do they mean by it?

Did they sit down at a computer, with reference to no other materials, and press buttons and click things until working programs started to emerge?

It's unlikely that they learned to program this way. More probable is that our “self-taught” programmer had some instruction. But what? Did they use tutorials or reference content? Was the material online, printed, or hand written? Did it include audio or visual components? Was it static or dynamic?

What feedback did they get? Did their teaching material encourage reflection, assessment, or other checkpoints? Did they have access to a mentor or community of peers, experts, or teachers? How did they interact with that community? Could they ask questions, and if so what did they do with the answers?

What was it that they taught themselves? Text-based processing routines in Commodore BASIC, or the Software Engineering Body of Knowledge?

What were the gaps in their learning? Do they recognise those gaps? Do they acknowledge the gaps? Do they see value in the knowledge that they skipped?

And finally, why do they describe themselves as ‘self-taught’? Is it a badge of honour, or of shame? Does it act as a signal for some other quality? Why is that quality desirable?

#### On Sharecropping
Today I came across the site Danny Reviews, at which fellow internet Danny Yu has posted over 1400 book reviews. I realised that if I had posted book reviews of every book I have read since I became an internet, I would have more than 900 reviews online, maybe over 1000. How do I know? Because my GoodReads profile lists those 900 books.

Now actually GoodReads are quite generous in their terms: I own all of the information I've posted there, and I can export all my books, including my reviews such as they are. But that's entirely up to GoodReads, they decided to be nice and provide an export feature. Other sites take their digital sharecropping more seriously.

I got lucky, but we should all think carefully about what we're posting to where.

#### On Singleton(s)
I woke up this morning to a discussion on Twitter over how different implementations of the Singleton pattern compare. This is like comparing your Herpes: no matter whose is better or more efficient, you still have unsightly blisters.

Overview: wtf is a Singleton?
Singleton is one of the software design patterns originally collected in the famous Design Patterns: Elements of Reusable Object-Oriented Software by the "Gang of Four". They describe the intent of the pattern thus:

Ensure a class only has one instance, and provide a global point of access to it.

The other useful source of design pattern information is Cocoa Design Patterns by Erik Buck and Don Yacktman. In addition to giving examples of Singleton found in the Foundation and AppKit frameworks, this book shows how to implement a Singleton class in Objective-C. The key features are:

A single access point to retrieve the single instance.
Initialization of the single instance.
Protection against accidentally creating another instance or deleting the single instance.

When would you use that?

Well here's the thing: I won't say that I "never would" use Singleton, but I will certainly say that it isn't the most reached-for tool in my belt.

The usual reason is "this class models something of which there is only one thing". This is the most absurd thing you'll ever hear. There's only one print spooler, so surely it must be a singleton. Right, that only works right up until the point where you need two print spoolers. When do you need a second print spooler? I'll come onto that in the next section.

Similarly, just because there's one filesystem, doesn't necessarily mean that you only need one filesystem object. It certainly doesn't mean you need to enforce there's only one filesystem object.

Go on then, wise guy, when do you need the second singleton?

When you test the code that uses the first one. You don't want your unit tests to talk to the real filesystem, or the real database, or the real print spool. You want your unit tests to use a Mock Object, which means they need the second, fake, instance of your filesystem object or whatever.

That's the real problem with Singleton: often, in your app, it makes sense to provide a shared instance of an object (particularly one that has state relevant to the entire app). However, that's not the same as requiring that no-one ever create another instance of the object.

So how would you do it, then?

Let's assume I have a need for an application-wide Framistan instance. I have a couple of options:

Create the usual +[Framistan sharedFramistan] class method, as Singleton implementers would, but not all that refcount-avoiding cruft that normally goes with it.
Create the method -[[NSApp delegate] sharedFramistan]. After all, I have a need for an application-wide instance, and that's where stuff associated with the application lives.

The option I go for depends on whether I think a process needs a shared instance of the class, or whether I think this app does. Usually, the latter option gets implemented first, and I change it later when I come to write the second app that uses the same class.

Either way, when I need to use the shared instance, I use it like this:

[someFrobnicator abdjulateWithFramistan: [Framistan sharedFramistan]];

Passing the shared instance in means that I still get to pass in other instances, for example in a test I can do [testFrobnicator abdjulateWithFramistan: mockFramistan];.

I don't like that. Does anyone do something different?

Yes, some people solve Singleton...with another Singleton. The mind boggles. Anyway, what these people do is to create a FramistanFacade Singleton whose job is to manage access to the Framistan Singleton. Now your real Framistan can be an honest Singleton class, but clients talk to the (also-Singleton) FramistanFacade class, which decides what instance of what class it wants to talk to itself.

In Objective-C, the FramistanFacade can use message forwarding to act as a Proxy object, hiding the interaction between Façade and real object. Of course, now that you're managing the real instance behind the Façade, the real class doesn't need to be a Singleton because a different class is already managing how its instances are used.

Conclusion

The debates over how best to implement Singleton in Objective-C are redundant because the Singleton pattern is never what you need. Often you do need a shared instance of a class, but enforcing that no other instance ever get used it detrimental to testing.

When you do need a shared instance of a class in your app, ensure that you do not close the door to using alternate instances. The need comes up more often than you might expect.

#### On software engineering hermeneutics
When I use a word it means just what I choose it to mean — neither more nor less.
Humpty-Dumpty in Alice through the Looking Glass

In my recent round of TDD clarifications, one surprising experience is that folks out there don’t agree on the definition of TDD. I made it as clear as possible in my book. I thought it was clear. Nope. My bad.
Kent Beck in Canon TDD

I invented the term Object-Oriented, and I can tell you I did not have C++ in mind.
Alan Kay in The Computer Revolution Hasn’t Happened Yet

I could provide many other examples, where a term was introduced to the software engineering state of the art meaning one thing, and ended up meaning “programming as it’s currently done, but with this small change that’s a readily-observable property of what the person who introduced the term described”. Off the cuff: “continuous integration” to mean “running automated checks on VCS branches”; “Devops” to mean “hiring Devops people”; “refactoring” to mean “editing”; “software engineering” to mean “programming”.

I could also provide examples where the dilution of the idea was accompanied by a dilution of the phrasing. Again, just typing the first ideas that come into my head: Free Software -> Open Source -> Source Available; various 1990s lightweight methodologies -> Agile Software Development -> Agile.

Researchers of institutions and their structures give us tools that help understand what’s happening here. It isn’t that software engineers are particularly bad at understanding new ideas. It’s that software engineering organisations are set up to reproduce the ceremonies of software engineering, not to be efficient at producing software.

For an institution to thrive, it needs to be legitimate: that is, following the logic that the institution proposes needs to be a good choice out of the available choices. Being the rationally most effective or most efficient choice is one legitimising factor. So is being the thing that everybody else does; after all, it works for them, so why not for us? So is being the thing that we already do; after all, it got us this far, so why not further?

With these factors of legitimacy in mind, it’s easy to see how the above shifts in meaning can occur. Let’s take the TDD example. Canon TDD says to write a list of test scenarios; turn one item into a runnable test; change the code to make that test and all previous tests pass; optionally refactor to improve the design; then iterate from the second step.

First person comes along, and has heard that maybe TDD is more effective (rational legitimacy). They decide to try it, but their team has heard “working software over comprehensive documentation” so they don’t want to embarrass themselves by writing a list of test scenarios (cognitive legitimacy). So they skip that step. They create a runnable test; change the code to make that test pass; optionally refactor. That works well! They share this workflow under the name TDD (Red-Green-Refactor).

Second person comes along, and has heard that the cool kids (Kent Beck and first person) are doing TDD, so they should probably do it too (normative legitimacy). They decide to try it, but they notice that if they write the code they want, then write the tests they want, they end up in the same place (they have code, and they have tests, and the tests pass) that Canon TDD and TDD (Red-Green-Refactor) end up in. So where’s the harm? Now they’re doing TDD too! They show their colleagues how easy it is.

Now everybody is doing a slightly different TDD, but it’s all TDD. Their descriptions of what they do construct the reality in which they’re doing TDD, which is an example of what the researchers call performative discourse. TDD itself has become ceremonial; the first and subsequent people are doing whatever they want to do and declaring it TDD because the legitimate thing to do is called TDD.

This does give those people who want to change software engineering some pointers on how to do it. Firstly, overshoot, because everybody’s going to meet you a short way along the path. Secondly, don’t only talk up the benefits of your proposed change, but the similarities with what people already do, to reduce the size of the gap. Thirdly, make sure that the likely partial adoptions of the change are improvements over the status quo ante. Fourthly, don’t get too attached to the words you use and your choice of their meanings: they mean just what anybody chooses them to mean—no more and no less.

#### On squeezing out that last ounce of performance
As I get confused by a component of an application that should be network-bound actually being limited by CPU availability, I get reminded of the times in my career that I've dealt with application performance.

I used to work on a platform for distributing MMS and SMS messages, written using GNUstep, Linux and PostgreSQL. I had a duplicate of the production hardware stack sitting in a data centre in our office, which ran its own copy of the production software and even had a copy of the production data. I could use this to run simulations or even replays of real events, finding the locations of the slowdowns and trying various hypotheses to remove the bottlenecks.

My next job was working on antivirus. The world of antivirus evaluations is dominated by independent testers, who produce huge bake-off articles comparing the various products. For at least one decade the business of actually detecting the stuff has been routine, so awards like VB100 are meaningless. In addition to detection stats, analysts like VB and AV-comparative measure resource consumption, and readers take those measurements seriously.

That's because they don't want to use anti-virus, and they didn't pay for their RAM and CPUs to waste it on software they don't want to use. So given a bunch of apps that all do the same thing, they'll look at which does it with less impact on everything else. This means that performance is an important requirement of new projects in AV software: on the product I worked on we had a defined set of performance tests. A new project release could not&mdash; regardless of how shiny the new features were&mdash;ship if the tests took 5% or more time or RAM than the current shipping version. On like hardware. What that really means is that due to developments in hardware, AV software was getting monotonically faster up until a few years ago.

Since then, my relationship with performance optimisation has been more sporadic. I've worked on contracts to speed up iOS apps, and even almost took a performance analysis and improvement job on a mobile phone operating system team. But what I usually do is make software work (and make it secure), with making it work in such a way that people can actually use it being a part of that. Here, then, are my Reflections On Making Efficient Software™.

Start at the beginning.

You may have heard the joke about a man on a driving holiday who gets lost and asks a local for directions. The local thinks for a bit, and says "well to get to where you're going, I wouldn't start from here if I were you".

Performance analysis can be like that, too. If you build up all of the functionality first, and optimise it later, you will almost certainly not get a well-performing product. Furthermore, fixing it will be very expensive. You may be able to squeeze a few kilobytes out here, or get a couple of percent speed increase there, but basically the resources used by your app will not change much.

The reason is simple: most of the performance characteristics of your app are baked into the top-level architecture. You need to start thinking about how your app will perform when you start to design what the various parts do and how they fit together. If you don't design out the architectural bottlenecks, you'll be stuck with them no matter how good your implementation is.

I've been involved with projects like this. It gets to near the ship date, and the app works but is useless because it consumes all of the RAM and/or takes too long to do any work. The project manager gets a developer in (hello!) to address the performance issues. After a few weeks, the developer has managed to understand the code, do some analysis, and improved things by a couple of percent. It's gone from "sucky" to "quite sucky", and the ship date isn't any further away.

An example: If you build a component that can process, say, ten objects per second, then hands them on to another component that can display results at 100 objects per second, you're always limited to 10 Hz give or take. You might get 12, you'll never get 100 without replacing the first component or the whole app. Both of these options are more expensive after you've written the component than before. Which leads me on to the next top tip&hellip;

Simulate, simulate, simulate

So how are you supposed to know how fast your putative architecture is going to run on paper? That's easy: simulate each component. If you believe that you'll usually get data from the network at, say, 100 objects/sec, then write a driver that sends fake objects at about 100 Hz. If you think that might spike at 10,000 objects/sec, then simulate that spike too. You'll be able to see what it takes to develop an app that can respond to those demands.

What's more, you'll be able to drop your real components into the simulated environment, and see how they really handle the situations you cook up. You can even use these harnesses as an integration test framework at the intermediate level (i.e. larger than classes, smaller than the whole app).

Your simulated components should use the same interfaces to the filesystem and each other that the real code will use, and the same frameworks or libraries. But they shouldn't do any real work. E.g if you've got a component that should read a JSON stream from the network, break it into objects, do around 10ms of work on each object and post a notification after each one is finished, you can write a simulation to do that using the JSON library you plan on deploying, the sleep() function and NSNotificationCenter. Then you can play around with its innards, trying out operation queues, dispatch queues, caching and other techniques to see how the system responds.

Performance isn't all about threads

Yes, Apple has a good concurrency programming guide. Yes, dispatch queues are new and cool. But no, not everything is sped up by addition of threads. I'm not going to do the usual meaningless micrometrics of adding ten million objects to a set, because no-one ever does that in real code.

The point is that doing stuff in the background is great for exactly one thing: getting that stuff off of the UI thread. For any other putative benefits, you need to measure. Perhaps threading will speed it up. Perhaps there aren't any good concurrent algorithms. Perhaps the scheduling overhead will get in the way.

And of course you need to measure performance in an approximation to the customer's environment. Your 12-core Mac Pro probably runs your multithreaded code in a different way than your user's MacBook Air (especially if the Pro has a spinning disk). And the iPad is nothing like the simulator, of course.

Speed, memory, work: choose any two

You can make it faster and use less RAM by not doing as much. You can make it faster and do the same amount of work by caching. You can use less RAM and do the same amount of work by reducing the working set. Only very broken code can gain advances in speed and memory use while not changing the outcome.

Measure early and measure often

As I said at the top, there's no point baking the app and then sprinkling with performance fairy dust at the end. You need to know what you're aiming for, whether it's achievable, and how it's going throughout development.

You must have some idea of what constitutes acceptable performance, so devise tests that discover whether the app is meeting those performance requirements. Run these tests periodically throughout development. If you find that a recent change slowed things down, or caused too much memory to be used, now is a good time to fix that. This is where that simulation comes in useful, so you can get an idea about the system's overall performance even before you've written it all.

#### On SSL Pinning for Cocoa [Touch]
Moxie Marlinspike, recently-acquired security boffin at Twitter, blogged about SSL pinning. The summary is that relying on the CA trust model to validate SSL certificates introduces some risk into using an app - there are hundreds of trusted roots in an operating system like iOS, and you don't necessarily want to trust all (or even any) of the keyholders. Where you're connecting to a specific server under your control, you don't need anyone else to tell you the server's identity: you know what server you need to use, you should just look for its certificate. Then it doesn't matter if someone compromises any CA; you're not trusting the CAs any more. He calls this SSL pinning, and it's something I've recommended to Fuzzy Aliens clients over the past year. I thought it'd be good to dig into how you do SSL pinning on Mac OS X and iOS.

The first thing you need to do is to tell Foundation not to evaluate the server certificate itself, but to pass the certificate to you for checking. You do this by telling the NSURLConnection that its delegate can authenticate in the "server trust" protection space.

-(BOOL)connection:(NSURLConnection *)connection canAuthenticateAgainstProtectionSpace:(NSURLProtectionSpace *)space {
  return [[space authenticationMethod] isEqualToString: NSURLAuthenticationMethodServerTrust];
}

Now your NSURLConnection delegate will receive an authentication challenge when the SSL connection is negotiated. In this authentication challenge, you evaluate the server trust to discover the certificate chain, then look for your certificate on the chain. Because you know exactly what certificate you're looking for, you can do a bytewise comparison and don't need to do anything like checking the common name or extracting the fingerprint: it either is your certificate or it isn't. In the case below, I look only at the leaf certificate, and I assume that the app has a copy of the server's cert in the sealed app bundle at MyApp.app/Contents/Resources/servername.example.com.cer.

- (void)connection:(NSURLConnection *)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge {
  if ([[[challenge protectionSpace] authenticationMethod] isEqualToString: NSURLAuthenticationMethodServerTrust]) {
    SecTrustRef serverTrust = [[challenge protectionSpace] serverTrust];
    (void) SecTrustEvaluate(serverTrust, NULL);
    NSData *localCertificateData = [NSData dataWithContentsOfFile: [[NSBundle mainBundle]
                                                                    pathForResource: serverName
                                                                    ofType: @"cer"]];
    SecCertificateRef remoteVersionOfServerCertificate = SecTrustGetCertificateAtIndex(serverTrust, 0);
    CFDataRef remoteCertificateData = SecCertificateCopyData(remoteVersionOfServerCertificate);
    BOOL certificatesAreTheSame = [localCertificateData isEqualToData: (__bridge NSData *)remoteCertificateData];
    CFRelease(remoteCertificateData);
    if (certificatesAreTheSame) {
      [[challenge sender] useCredential: [NSURLCredential credentialForTrust: serverTrust] forAuthenticationChallenge: challenge];
    }
    else {
      [[challenge sender] cancelAuthenticationChallenge: challenge];
    }
  }
  // fall through for challenges in other spaces - or respond to them if you need to
}

That's really all there is to it. You may want to change some of the details depending on your organisation: for example you may issue your own intermediate or root certificate and check for its presence while allowing the leaf certificate to vary; however the point is to get away from the SSL certificate authority trust model so I haven't shown that here.

#### On standards in free software engineering
I have previously written on the economics of software insecurity, and I quote a couple of paragraphs from that post below:

One option that is not fully explored in the book, but which I believe could be worth exploring, is this: development of critical infrastructure software could be taken away from the free market.

Now the size of even the U.S. government IT budget probably isn't sufficient to completely fund a bunch of infrastructure developers, but there are other options. Rice correctly notes the existence of not-for-profit software development organisations (particularly the Open Source Initiative and Free Software Foundation), and discusses the benefits and drawbacks of the open source model as it applies to commercial software. He does not explore the possibility that charity development organisations could withdraw from market competition, and focus on engineering practice, quality and security without feature parity or first-to-market speed.

Today I was re-reading Free Software, Free Society by Richard Stallman, a collection of his essays and speeches on topics including copyleft, the GNU system and General Public Licence. In thinking about this book, I went wandering back to the idea of non-commercial driver for good quality software.

I am now convinced that the Free Software Foundation should be investigating, researching and promoting standards, practices and quality in software construction.

The principal immediate benefit the FSF would gain is in terms of visibility and support. Everywhere that software is used—public, private and academic sectors—organisations are interested in finding out ways to improve quality, reliability, deliverability: in other words, the success of their software. An entity that could offer to evaluate and report on whether particular techniques are feasible and offer improvement—in return for funding and staffing the production of their sought-after free software—would be welcomed and would be put to good use.

The FSF is well-placed to achieve this goal, because all of its output is copyleft. A large problem with analysing the success or otherwise of development practices is that the outputs are proprietary: not just the code, but the project documentation, meeting minutes and so forth. With an FSF project everything is (or should be) freely available so inspecting how a project was run, what the developers did and—crucially—whether users are happy with the end result is much easier. Conclusions should be reproducible because everybody can see everything that went on.

Notice that in this scheme, relationships between the FSF and other (proprietary, open source, whatever) organisations are mutually beneficial, not antagonistic as is often either actually the case or just assumed. The benefits seen by external parties are the improvements in process and technique; benefits that all developers can make use of. The discussion moves away from free vs. fettered, and becomes making the field of software engineering better for everyone.

Incidentally, such a focus would also put free software at the forefront of discussions on software quality and deliverability. This would be something of a coup for free software, which is often associated with chaotic management, lack of road maps, and paucity of documentation and support. OK, the FSF wants people to consider freedom as a value in itself, but there's nothing wrong with ensuring that free software is the best software too, surely?

#### On stopping service management abuse
In chapter 2 of their book The Mac Hacker's Handbook (is there only one Mac hacker?), Charlie Miller and Dino Dai Zovi note that an attacker playing with a sandboxed process could break out of the sandbox via launchd. The reasoning goes that the attacker just uses the target process to submit a launchd job. Launchd, which is not subject to sandbox restrictions, then loads the job, executing the attacker's payload in an environment where it will have more capabilities.

This led me to wonder whether I could construct a sandbox profile that would stop a client process from submitting launchd jobs. I have done that, but not in a very satisfying way or even necessarily a particularly complete one. My profile does this:

(version 1)
(deny default)
(debug deny)

(allow process-exec)
(allow file-fsctl)
(allow file-ioctl)
; you can probably restrict access to even more files - don't forget to let dyld link Cocoa though!
(allow file-read* file-write*)
(deny file-read* (regex "^/System/Library/Frameworks/ServiceManagement.framework"))
(deny file-read* (literal "/bin/launchctl" "/bin/launchd"))
(allow signal (target self))
(allow ipc-posix-shm)
(allow sysctl*)
; this lot was empirically discovered - Cocoa apps needs these servers
(allow mach-lookup (global-name "com.apple.system.notification_center" "com.apple.system.DirectoryService.libinfo_v1"
 "com.apple.system.DirectoryService.membership_v1" "com.apple.distributed_notifications.2"
 "com.apple.system.logger" "com.apple.SecurityServer" 
"com.apple.CoreServices.coreservicesd" "com.apple.FontServer" 
"com.apple.dock.server" "com.apple.windowserver.session" 
"com.apple.pasteboard.1" "com.apple.pbs.fetch_services" 
"com.apple.tsm.uiserver"))

So processes run in the above sandbox profile are not allowed to use the launchd or launchctl processes, nor can they link the ServiceManagement framework that allows Cocoa apps to discover and submit jobs directly.

Unfortunately I wasn't able to fully meet my goal: a process can still do the same IPC that launchctl et al use directly. I found that if I restricted IPC access to launchd, apps would crash when trying to check-in with the daemon. Of course the IPC protocol is completely documented so it might be possible to do finer-grained restrictions, but I'm not optimistic.

Of course, standard disclaimers apply: the sandbox Scheme environment is supposed to be off-limits to us smelly non-Apple types.

#### On study-only licences
In my previous post, I mused on the value of Freedom Zero and of a non-free licence that allows for study but not for use:

  I think it would have to be a licence that enabled studying, sharing and modification of the software, but that explicitly forbade any use for any purpose that isn’t studying, modifying or sharing. With a “contact me or my agent, tell us what you’re doing, and we’ll decide whether to grant you an additional licence for use” suffix. This is more open than closed proprietary software, but no more available for deployment to bad actors.

Waking up this morning I remembered that I have a copy of Numerical Recipes. This is a book, that contains code, and as such you can read the code. But not much else:

  Without an additional license to use the contained software, this book is intended as a text and reference book, for reading and study purposes only. However, a restricted, limited free license for use of the software by the individual owner of a copy of this book who personally keyboards one or more routines into a single computer is granted under terms described on p.xix.

Page xix expands:

  If you personally keyboard no more than 10 routines from this book into your computer, then we authorize you (and only you) to use those routines (and only those routines) on that single computer.

If you want to study, or to try things, knock yourself out. If you want to distribute things, or use things, get in touch and we'll choose whether to sell you a licence.

This is not unexplored territory.

#### On switching to Linux
In November, I switched to GNU/Linux at home (I still use OS X at work, because I still write Objective-C in Xcode at work). Or rather, I switched back: I'd been using it around a decade ago.

In December, I resolved to spend more time working with Free Software during 2015 and beyond. Now I find that my story is not unique, others have left OS X (now a dead link, sadly) or are concerned at the state of Apple. I have had conversations with friends about moving from OS X to Debian, to OpenBSD, to other systems.

In my case, there was no watershed moment, no Damascene conversion from the Tablet According to Jobs to the Stallman Doctrine. Rather, my experience has been one of a thousand tiny cuts, and the feeling that while GNU is not actually a better system, it's one that I'm allowed to make changes to. When iWork was 'upgraded' to a less-capable version with an incompatible file format, cut. Every time I plug my Apple display into my Apple computer, then have to unplug it and connect it again before it works properly, cut. Even when the display works, I have to unplug the (Apple) keyboard and plug it back in, cut. Every time iTunes connects to the network but doesn't let me play any of my tunes, cut. When Apple trumpets the superiority of their new map that shows I live in "Royal Spa", cut. When iCloud showed its message that I have to upgrade to the new way and devices still on the old way can't see my files any more, cut. Like the above-linked authors, I've got to a point where the only Apple-supplied software in my Dock is the Finder. In my case, it's happened one app at a time.

The thing is, Apple's been building some great new things. There's the watch, Swift, improvements to Cocoa, and they still have the best hardware around. If it weren't for both the real regressions and the fear of potential regressions on every upgrade or app update, I'd still be confident that their Unix desktop were the best. As it is, I think it could possibly be the least worst, but I'm no longer certain.

Most of the time I don't care about my computer's operating environment. This isn't so much the year of Desktop Linux for me, as the year of Desktop Ambivalence. The OS is the thing that spends most of the time and RAM involved in showing me an emacs window, or a terminal emulator, or a Smalltalk machine, or a web browser. I shouldn't need to care how it works. But if it's going to get in my way, give me a chance to do something about it.

The standard trope response is "LOL you must have a bad idea of UX if you use Linux", which is the sort of thing that was true at the turn of the millennium when most of us were last tyre-kicking some GNU distro but is now an outdated view of the world. But again, should there be a problem, I hope to have the ability to fix it both for myself and others.

So far this year I've started small. I've been reminding myself of the differences between OS X and BSD commands that I'm used to, and the Linux and SysV commands that I've not used frequently in nearly a decade. I've been learning about Vala and GTK+.

#### On the "advances" in web development since 1995
The first "web application" I worked on was written in a late version of WebObjects, version 4.5. An HTTP request was handled by an "adaptor" layer that chose a controller based on the parameters of the request, you could call this routing if you like. The controller accesses the data model, any relevant application logic, and binds the results to the user interface. That user interface could be an HTML and JavaScript document for presentation in a browser, or it could be a structured document destined for consumption by a remote computer program. The structured document format would probably have looked something like this:

{
  "first_name": "Graham";
  "last_name": "Lee";
}

The data can be bound on properties in objects called "components" that represent different elements in an HTML document, like images, anchors, paragraphs and so on.

Different frameworks for interfacing with various data storage technologies, such as relational databases, were available.

And basically nothing - apart from the cost - has changed since then. A MEAN application works in the same way, but with the "advance" that it makes it easier to support single-page applications so you can send a few megabytes of transpiled bundle over an unreliable mobile data network to run on a slow, battery-operated smartphone processor instead of in the server. And the data storage interfaces have been expanded to include things that don't work as well as the relational databases. Other "advances" include a complicated collection of transpilers and packagers that mean every project has its own value of "JavaScript".

It's not like anything has particularly got much easier, either. For example, a particular issue I've had to deal with in a React application - binding a presentation component to properties of an element in a collection - is exactly as difficult now as it was when WebObjects introduced WORepetition in 1995. Before we get to the point where it becomes easier, we'll give up on React, and Node, and move on to whatever comes after. Then give up on that.

In software engineering, if something isn't broke, fix it 'til it is.

#### On the broken(?) Mac App Store
A day after the Mac App Store was launched, people are reporting that it has been cracked. There are two separate stories here, a vapourware circumvention of the FairPlay DRM used to generate the receipts and a report that certain apps aren't validating the receipts properly. We can ignore the first case for the moment: it's important, and if it's true then Apple needs to fix it (and co-ordinate updating the validation code with us third-party developers). But for the moment, it's more important that developers are implementing the protections that are in place in their applications - it's those applications that are supposed to be protected.

Let's skip, for the moment, the question of whether DRM or anti-cracking mechanisms are ethically right, worthwhile, or how much effort you want to put into them. Apple have done most of the legwork, in providing a vendor-signed receipt that's part of your signed app bundle. What you need to do is:

Check whether you have a receipt
Check whether Apple signed the receipt you have
Check whether the receipt is valid for your product
Check whether the receipt is valid for this version of your product
Check whether the receipt is valid for this computer

That's it in a nutshell. Of course, some nutshells surround very big and complex nuts, and that's true in this case:

There's some good example code for receipt validation at github/roddi/ValidateStoreReceipt, if you're going to use it then don't just paste it wholesale. If everyone uses the same code then it's super-easy for someone to detect and strip that code from each instance.
Check at runtime, as well as before startup. If you just check at startup, then all an attacker needs to do is patch main() to jump straight into NSApplicationMain() and your app runs for free.
Code obfuscation is not a very effective tool. Having worked in anti-virus, I know it's much easier to classify code based on what it does than what it is, and it's quite easy to find the code that opens the receipt file, or calls exit(173). That said, some of the commercial obfuscation companies offer a guaranteed service, so you can still protect your revenue after the app gets cracked.
Update I have been advised privately and seen in a blog post that people are recommending hard-coding their app bundle IDs and version numbers into the binary rather than using Info.plist, because that file can be edited. Well, so can the app binary...and in either case you'd need to re-sign the product with a valid certificate to continue, because Apple have used the kill flag:
heimdall:~ leeg$ codesign -dvvvv /Applications/Twitter.app/
Executable=/Applications/Twitter.app/Contents/MacOS/Twitter
Identifier=com.twitter.twitter-mac
Format=bundle with Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=12452 flags=0x200(kill) hashes=616+3 location=embedded
CDHash=8e0736639d79a108a5a1ebe89f928d1da0d49d94
Signature size=4169
Authority=Apple Mac OS Application Signing
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Info.plist entries=21
Sealed Resources rules=4 files=78
Internal requirements count=2 size=344

Changing a hard-coded string in a binary file is not difficult. You can of course obfuscate the string, but the motivated cracker still finds the point where the comparison is made (particularly easily if you use NSStrings). Really, how far you want to go depends on how much you're willing to spend.

Of course, Fuzzy Aliens Ltd has already been implementing receipt validation for customers, so if this is too hard for you or you don't have the time... ;-)

A last word on publicising receipt-validation vulnerabilities

You and I both make our living by selling software, or by selling services to people who sell software. Crowing on the interwebs about how this application or that application doesn't validate its receipts properly is not cool, because you are shitting on your own doorstep. There is no public benefit to public disclosure that class of vulnerability, because DRM is not a user security feature. Don't do that. Send the developer a private message explaining your findings. Give them a chance to put extra effort into protecting their product, if that's what they want to do.

#### On the business case for (or against) software
In the vexing problems, I dismissed the hard problems of computer science as being incidental to another problem: we can't say what the value of our work is. That post contained plenty of questions, precisely because the subject is so unknown.

There are plenty of ways in which the "value" of something can be discussed, but let's stick with economic value for the moment. Imagine you have the opportunity to write some software, but only if I pay you for it. And not even then, but only if you can make me reasonably confident that I will get some return; that having the work done is worth more than the cost of you doing the work.

A corner of the software industry just cowered and stuck its head in the sand. "#NoEstimates!", they cry. This problem is hard, so why should we solve it?

That's a really interesting perspective. You should come and talk about #NoEstimates at my conference. It'll be on sometime, in a place, and there'll be a flight that can get you there. Maybe. That's enough information to be going on for it to be clear that you should commit to it, so I'll see you there.

At the other end are the folks who would literally write whole books about the topic (and given that books are themselves literary, that "literally" is itself meant literally). I pulled a few likely titles off the shelf to see what they would tell me, and ended up with:

Software Project Management, 5th Edition
PSP: A Self-Improvement Process for Software Engineers
Software Engineering Economics

We have to wonder how some people can be certain that estimates are worthlessly inaccurate, while others such as Barry Boehm (the author of Software Engineering Economics and the COnstructive COst MOdel it describes) have built careers out of explaining how to do it well.

Here's one hypothesis: Boehm is a charlatan, and the COCOMO doesn't work at all. Let's see whether there's evidence for that.

A likely source is COCOMO evaluation and tailoring by Miyazaki and Mori from Fujitsu. The authors of that paper, applying the COCOMO ('81, not the later COCOMO II) model to a corpus of projects undertaken by Fujitsu, conclude:

  The original COCOMO overestimates the effort required to develop software in our environment, but its tailoring methodology is applicable. [...] The resulting model fits 68% of the projects with less than 20% relative error, after the deletion of two outliers.

That's maybe short of brilliant - the median project effort in their paper seems to be over 10 person-years, so they're saying "there's around two chances in three that we can tell you what this project will cost to within twice a developer's salary" - but it's much more precise than "#NoEstimates!" and only quite a lot less accurate.

It might be interesting to track the application of COCOMO through time, and see whether a larger corpus of projects and refinement of the model has improved its accuracy and applicability. Unfortunately, that goal is in conflict with a general industry observation, noted in the introduction of the Fujitsu paper:

  We have the impression that software people prefer to start from scratch rather than improving upon the work of others, which seems to be slowing down the progress in this field.

And indeed much of what was written in the sources I've been talking about here is no longer applied. All three of the books I mentioned up top talk about Earned Value (EV) Analysis, recording the actual rate of delivering on a project against its Planned Value (PV). For example, Hughes and Cotterell recommend plotting earned value using the "0/100 technique":

  where a task is assigned a value of zero until such time as it is completed when it is given a value of 100% of the budgeted value

Agilistas will recognise the resulting plot: it's a burn down chart, where a story is not delivered until it has been accepted according to the definition of done, at which point it is completely delivered.

Ultimately somebody in the Agile community must have noticed that they accidentally borrowed a software engineering concept, because the burn down chart is now unfashionable.

  If the team starts with a very full, prioritized, and estimated backlog with the expectation they will burn down all tasks to zero -- does this not sound like a Waterfall mentality?

Oh noes, the dreaded W word! Run away from this thought, for it is a thought that somebody in the past has already considered!

And that is why I consider these problems to be vexing: when potential solutions are proposed, they are by necessity existing ideas from the old school that must be rejected in favour of...whatever is not those solutions. As Craftsmanship is post-Agile is post-Software Engineering is post-Crisis thinking, surely it will soon be time for post-Craftsmanship thinking. And then post-whatever-comes-next thinking. And we still won't know how to compare expected value to expected cost.

#### On the Consolation of Software Engineering
I'm currently reading Boethius's writing on the consolation of philosophy. Imprisoned awaiting the death penalty in 523 (for treason against King Theodoric), Boethius imagined a conversation with the personification of Philosophy herself, a woman of variable height whose fine dress was torn by various previous philosophers who had snatched tatters from it and imagined that they had the whole thing. The work has been available in English translation since the ninth century, when King Alfred commanded it be translated, though I'm reading a much more recent Penguin Classics translation.

The work leads me to imagine De Consolatione Ingenariae Computatraliae, in which someone is visited by the anthropomorphic personification of software engineering (in my headcanon this is Bruce Boxleitner as Tron), and the visitor uses the Socratic method to explore why the narrator chose to ignore so much knowledge of computing as they cut corners to close ticket after ticket in their career.

#### On the continuous history of approximation
The Difference Engine - the Charles Babbage machine, not the steampunk novel - is a device for finding successive solutions to polynomial equations by adding up the differences introduced by each term between the successive input values.

This sounds like a fairly niche market, but in fact it's quite useful because there are a whole lot of other functions that can be approximated by polynomial equations. The approach, which is based in calculus, generates a Taylor series (or a MacLaurin series, if the approximation is for input values near zero).

Now, it happens that this collection of other functions includes logarithms:

\(ln(1+x) \approx x - x^2/2 + x^3/3 - x^4/4 + \ldots\)

and exponents:

\(e^x \approx 1 + x + x^2/2! + x^3/3! + x^4/4! + \ldots\)

and so, given a difference engine, you can make tables of logarithms and exponents.

In fact, your computer is probably using exactly this approach to calculate those functions. Here's how glibc calculates ln(x) for x roughly equal to 1:
  r = x - 1.0;
  r2 = r * r;
  r3 = r * r2;
  y = r3 * (B[1] + r * B[2] + r2 * B[3]
    + r3 * (B[4] + r * B[5] + r2 * B[6]
        + r3 * (B[7] + r * B[8] + r2 * B[9] + r3 * B[10])));
  // some more twiddling that add terms in r and r*r, then return y

In other words, it works out r so that it is calculating ln(1+r), instead of ln(x). Then it adds together r + a*r^2 + b*r^3 + c*r^4 + d*r^5 + ... + k*r^12...it does the Taylor series for ln(1+r)!

Now given these approximations, we can combine numbers into probabilities (using the sigmoid function, which is in terms of e^x) and find the errors on those probabilities (using the cross entropy, which is in terms of ln(x). We can build a learning neural network!

And, more than a century after it was designed, our technique could still do it using the Difference Engine.

#### On the design of iOS 7 and iconographoclasm
As I write this, the WWDC keynote presentation has been over for a little more than half a day. That, apparently, is plenty of time in which to evaluate a new version of an operating system based on a few slides, a short demonstration, and maybe a little bit of playing with an early developer preview.

What I see is a lot less texture than an iOS 6 screen, with flatter, simpler icons and a very thin Sans Serif typeface. That's because I'm looking at a Windows RT screen, though. What's that? iOS 7 looks the same? They've gone Metro-style? Surely this heralds the end of days. Dogs and cats lying with each other, fire and brimstone raining from the sky. It would be time to give all the money back to the shareholders, except that it costs too much to bring all that cash back into the US.

Oh no, wait. It's just a phone, not the apocalypse. Also it's just less than half a day with a preview of a phone. If first impressions of a graphics set were enough to form a decision on how an artifact is to carry around with me all day, every day for the next couple of years, the iPhone would have been dead a long time ago. Remember the HTC Desire, with its bright, saturated background and parallax scrolling? Or the Samsung Galaxy series, with their bigger screens that are easier to see across a brightly-lit and packed showroom? How about the Nokia Lumia, with its block colours and large animations? Those are the kings of the short-tem attention grab. Coincidentally they're the things that many journalists, trying to find an angle on day one of a device's release, latched on to as obvious iPhone killers. Well, those and anything else released in the last few years.

What no-one has investigated is how the new iOS interface is to use all the time, because no-one has had all the time yet. I don't know whether the changes Apple have made to the design are better, but I do know (largely because they mentioned it at the beginning of the talk) that they thought about them. The icons on the home screen, for example, are a lot more simplistic than they previously were. Is that because they were designed by some cretin wielding a box of Crayola, or did they find some benefit to simpler icons? Perhaps given a large number of apps, many users are hunting and pecking for app icons rather than relying on muscle memory to locate the apps they need. If this is true, a simpler shape could perhaps be recognised more quickly as the pages of apps scroll under the thumb.

As I say, I don't know if the changes are for the better or not, but I do know they weren't the result of whimsy. Though if Apple are chagrined at the noise of a million designers angrily dribbbling over their keyboards, they only have themselves to blame. It's my belief that this evaluation of computer products based on what they look like rather than on their long-term use has its origin with Apple, specifically with the couple of years of Apple Design Awards that preceded the iPhone app store's launch. It's these awards that heralded the "Delicious generation" of app design that has informed Mac and iOS ISVs (and Apple) to date. It's these awards that valued showy, graphically-rich apps without knowing whether they were useful: giving design awards to software that people could not, at time of award, yet use at all.

Now it turns out that many of these winners did indeed represent useful apps that you could go back to. I was a long-term user of Delicious Library 2 right up until the day Delicious Library 3 was launched. That benefit was not directly evident on the granting of the ADA though: what Apple were saying was "we would like to see you develop apps that look like this" rather than "we recognise this as the type of app our customers have derived lots of enjoyment from". It's my belief that this informed the aesthetics and values of the ISV "community", including the values with which they appraised new software. If Apple are suffering a backlash now from people who don't immediately love the new design of iOS 7, it is their own petard by which they have been hoisted.

#### On the economics of software insecurity
This post is mainly motivated by having read Geekonomics: the real cost of insecure software, by David Rice. Since writing the book Rice has apparently been hired by Apple, though his bio at the Geekonomics site doesn't mention that (nor his LinkedIn profile).

Geekonomics is a thoroughly interesting read. It's evidently designed as a call to arms for users to demand better security, and as a result resorts to hyperbole in parts. You are a crash test dummy for software manufacturers and are paying extravagantly for the privilege. In this way it reads as if it is to security as The Inmates are Running the Asylum was to user experience in the 1999: do you realise just how shoddy all of this software you use is?

That said, once you actually dig into Rice's arguments, the hyperbole disappears and the book becomes well-sourced, internally consistent and rational. He explains why the market forces in the software industry don't lead to security (or even high quality) as either the primary customer requirement nor the key focus of producers.

Interestingly, while Geekonomics only incidentally touches on the role of security researchers in the software economy, their position is roughly consistent with the one I outlined in On Securing Lion: they are in it to get money (and sometimes fame) from selling either the vunerabilities they discover, or their skill at selling vulnerabilities.

The book ends by describing the different options a curated free market like the US market has for correcting the situation where market forces lead to socially undesirable outcomes: these options are redress via contract law, via tort law or via strict liability legislation. The impact on each of the above on the software industry is estimated.

One option that is not fully explored in the book, but which I believe could be worth exploring, is this: development of critical infrastructure software could be taken away from the free market.

Now the size of even the U.S. government IT budget probably isn't sufficient to completely fund a bunch of infrastructure developers, but there are other options. Rice correctly notes the existence of not-for-profit software development organisations (particularly the Open Source Initiative and Free Software Foundation), and discusses the benefits and drawbacks of the open source model as it applies to commercial software. He does not explore the possibility that charity development organisations could withdraw from market competition, and focus on engineering practice, quality and security without feature parity or first-to-market speed.

Governments, trade groups, communications carriers and other organisations with an interest in using software as infrastructure (e.g. so-called "cloud" companies) could fund non-profits (maybe with money, maybe with staff) that develop infrastructure-grade software. Those non-profits would have a mission to do quality-centric development, and would put confidentiality, integrity, availability, reliability and correctness before feature richness or novelty. Their governance (the bit I haven't fully thought through, admittedly) would be organised to promote and reward exactly that approach to development.

The software, its documentation and its engineering methodologies would be open, so that commercial software can take advantage of its advances at low cost. This is partially of importance because where security is a "hygiene factor" to software purchasers, the "security gap" between the infrastructure-grade and commercial-grade software would become clear and would artificially introduce infrastructure-grade robustness to the marketplace. Commercial vendors who could cheaply pick up parts of the infrastructure-grade software for their own products would be, in a self-interested manner, bringing that software's quality into the commercial marketplace and making it a competition point.

"But," some people say, "such software would be feature-poor. Why would anyone choose [SafeOS, SafeWebServer, SafeSmartPhone, whatever] over a feature-rich commercial offering?" The point is, that in infrastructure, correct function is more important than features. It's only the fact that software exists purely in a competitive world that means the focus is on features.

Case in point: one analogy used throughout Geekonomics is that infrastructure software is like cement (actually, in a book I'm currently writing on software testing, I make the same analogy, though relating to design rather than function). Well even taking into account innovations like Portland cement, the feature list of cement hasn't changed in thousands of years. It sticks aggregate together to make concrete or mortar. It's only the quality of its stick-aggregate-together-ness that has changed.

In relation to software, most computers are still "stuck together" using RFC791 (Internet Protocol version 4), which was documented in 1981 but was in use already at that time. The main advantages of RFC2460 (Internet Protocol version 6), written in 1998, are increased address space and reduced overhead. It's better at stick-computers-together-ness, but doesn't really do anything new. There may have been new applications of networks recently (and of course, the late addition of confidentiality in the mid-1990s), but networking itself doesn't frequently need new features.

Or even operating system software. The last version of Mac OS X that added any features for its users was version 10.5, said new features were:

Time Machine: computers have been doing backup for years, this added a new UI.
Spaces: an improvement on the ability to draw windows on the screen.
Back to my Mac: an integration of existing capabilities (VNC and wide-area zeroconf networking).
Boot Camp: managing partitions, and giving the primary bootloader compatibility with a 1983 computer standard.

All of the other enhancements were in the applications, which still all require the same things of the OS: schedule processes, protect memory, abstract the file systems, manage devices. Again, there may have been new applications of an operating system, but there hasn't been much newness in the operating system itself.

The part where such non-profit infrastructure software becomes tricky is in integrating with the rest of the "stack". On the hardware side, it would be inappropriate to require that a government-sponsored and not-for-profit software project run on proprietary hardware. On the other hand, it might be inappropriate to disbar deployment on proprietary hardware&mdash;but is infrastructure-grade software on commercial-grade hardware still an infrastructure-grade deployment?

That's particularly difficult in our world&mdash;the world of smartphones&mdash;because there isn't really any open hardware. There are somewhat open definitions: the Android Compatibility Definition Document for example. But as Ken Thompson taught us: in a trusted system, we need to question who we trust and why.

Going the other way, of course, is much easier. Anyone could write an application that interoperates with infrastructure-grade software, or a system partially constructed out of such software. But the same question would still exist: how much of an impact do the non-infrastructure-grade components have on the reliability of the system?

#### On the efficient allocation of scarce resources with alternative uses
Most of what I know about "the economy" is outdated (Adam Smith, Karl Marx, John Maynard Keynes) or incorrect (the news) so I decided to read a textbook. Basic Economics, 5th Edition by Thomas Sowell is clear, modern, and generally an argument against economic regulation, particularly centralised planning, tariffs, and price control. I still have questions.

The premise of market economics is that a free market efficiently uses prices to allocate scarce resources that have alternative uses, resulting in improved standard of living. But when results are compared, they are given in terms of economic metrics, like unemployment, growth, or GDP/GNP. The implication is that more consuming is correlated with a better standard of living. Is that true? Are there non-economic measurements of standard of living, and do they correlate with the economic measurements?

Even if an economy does yield "a better standard of living", shouldn't the spread of living standards and the accessibility of high standards across the population be measured, to determine whether the market economy is benefiting all participants or emulating feudalism?

Does Dr. Sowell arrive at his office at 9am and depart at 5pm? The common 40-hour work week is a result of labour unions and legislation, not supply and demand economics. Should we not be free to set our own working hours? Related: is "unemployment" such a bad thing, do we really need everybody to work their forty hours? If it is a bad thing, why not reduce the working week and have the same work done by more people?

Sowell's argument allows that some expenses, notably defence, are better paid for centrally and collectively than individually. We all get the same benefit from national defence, but even those who are willing to pay would receive less benefit from a decentralised, individually-funded defence. Presumably the same argument can be applied to roads, too, or space races. But where are the boundaries? Why centralised military, say, and not centralised electricity supply, healthcare, mains water, housing, internet service, or food supply? Is there a good "grain size" for such centralising influences (it can't be "the nation", because nations vary so much in size and in centralisation/federation) and if so, does it match the "grain size" for a market economy?

The argument against a centralised, planned economy is that there's too much information required too readily for central planners to make good judgements. Most attempts at a planned economy preceded broad access to the internet and AI, two technologies largely developed through centralised government funding. For example, the attempt to build a planned economy in Chile got as far as constructing a nationwide Telex network before being interrupted by the CIA-funded Pinochet coup. Is this argument still valid?

Companies themselves are centralised, planned economies that allocate scarce resources through a top-down bureaucracy. How big does a company need to get before it is not the market, but the company's bureaucracy, that is the successful system for allocating resources?

#### On the extension of code signing
One of the public releases Apple has made this WWDC week is that of Safari 5, the latest version of their web browser. Safari 5 is the first version of the software to provide a public extensions API, and there are already numerous extensions to provide custom functionality.

The documentation for developing extensions is zero-cost, but you have to sign up with Apple's Safari developer program in order to get access. One of the things we see advertised before signing up is the ability to manage and download extension certificates using the Apple Extension Certificate Utility. Yes, that's correct, all Safari extensions will be signed, and the certificates will be provided by Apple.

For those of you who have provisioned apps using the iTunes Connect and Provisioning Portal, you will find the Safari extension certificate dance to be a little easier. There's no messing around with UDIDs, for a start, and extensions can be delivered from your own web site rather than having to go through Apple's storefront. Anyway, the point is that earlier statement - despite being entirely JavaScript, HTML and CSS (so no "executable code" in the traditional sense of machine-language libraries or programs), all Safari extensions are signed using a certificate generated by Apple.

That allows Apple to exert tight control over the extensions that users will have access to. In addition to the technology-level security features (extensions are sandboxed, and being JavaScript run entirely in a managed environment where buffer overflows and stack smashes are nonexistent), Apple effectively have the same "kill switch" control over the market that they claim over the iPhone distribution channel. Yes, even without the store in place. If a Safari extension is discovered to be malicious, Apple could not stop the vendor from distributing it but by revoking the extension's certificate they could stop Safari from loading the extension on consumer's computers.

But what possible malicious software could you run in a sandboxed browser extension? Quite a lot, as it happens. Adware and spyware are obvious examples. An extension could automatically "Like" facebook pages when it detects that you're logged in, post Twitter updates on your behalf, or so on. Having the kill switch in place (and making developers reveal their identities to Apple) will undoubtedly come in useful.

I think Apple are playing their hand here; not only do they expect more code to require signing in the future, but they expect to have control over who has certificates that allow signed code to be used with the customers. Expect more APIs added to the Mac platform to feature the same distribution requirements. Expect new APIs to replace existing APIs, again with the same requirement that Apple decide who is or isn't allowed to play.

#### On the extremes of computer science
I didn't study computer science at school or university, and still manage to work as a programmer.

That is not to say that I don't need to know some things that are taught on computer science courses. Just this week I've had to build a couple of different data structures and understand their running time: very CS.

I've also needed to know things that aren't on a CS degree, too. The acceptance criteria for one of my projects are written in French, and none of the CS courses I've seen in UK universities include that in the syllabus.

I'm neither arguing for nor against the validity of a CS background in professional software development. I'm arguing against taking either side. You need to know some CS things to write software, you need to know some other things, multiple backgrounds are appropriate and welcome.

#### On the features of a portfolio career
Since starting The Labrary late last year, I've been able to work with lots of different organisations and lots of different people. You too can hire The Labrary to make it easier and faster to create high-quality software that respects privacy and freedom, though not before January 2020 at the earliest.

In fact I'd already had a portfolio career before then, but a sequential one. A couple of years with this employer, a year with that, a phase as an indie, then back to another employer, and so on. At the moment I balance a 50% job with Labrary engagements.

The first thing to notice is that going part time starts with asking the employer. Whether it's your current employer or an interviewer for a potential position, you need to start that conversation. When I first went from full-time to 80%, a few people said something like "I'd love to do that, but I doubt I'd be allowed". I infer from this that they haven't tried asking, which means it definitely isn't about to happen.

My experience is that many employers didn't even have the idea of part-time contracts in mind, so there's no basis on which they can say yes. There isn't really one for "no" either, except that it's the status quo. Having a follow-up conversation to discuss their concerns both normalises the idea of part-time employees, and demonstrates that you're working with them to find a satisfactory arrangement: a sign of a thoughtful employee who you want to keep around, even if only some of the time!

Job-swapping works for me because I like to see a lot of different contexts and form synthetic ideas across all of them. Working with different teams at the same time is really beneficial because I constantly get that sense of change and excitement. It's Monday, so I'm not there any more, I'm here: what's moved on in the last week?

It also makes it easier to deal with suboptimal working environments. I'm one of those people who likes being in an office and the social connections of talking to my team, and doesn't get on well with working from home alone (particularly when separated from my colleagues by timezones and oceans). If I only have a week of that before I'm back in society, it's bearable, so I can consider taking on engagements that otherwise wouldn't work for me. I would expect that applies the other way around, for people who are natural hermits and would prefer not to be in shared work spaces.

However, have you ever experienced that feeling of dread when you come back from a week of holiday to discover that pile of unread emails, work-chat-app notifications, and meeting bookings you don't know the context for? Imagine having that every week, and you know what job-hopping is like. I'm not great at time management anyway, and having to take extra care to ensure I know what project C is up to while I'm eyeballs-deep in project H work is difficult. This difficulty is compounded when clients restrict their work to their devices; a reasonable security requirement but one that has led to the point now where I have four different computers at home with different email accounts, VPN access, chat programs, etc.

Also, absent employee syndrome hits in two different ways. For some reason, the median lead time for setting up meetings seems to be a week. My guess is that this is because the timeslot you're in now, while you're all trying to set up the meeting, is definitely free. Anyway. Imagine I'm in now, and won't be next week. There's a good chance that the meeting goes ahead without me, because it's best not to delay these things. Now imagine I'm not in now, but will be next week. There's a good chance that the meeting goes ahead without me anyway, because nobody can see me when they book the meeting so don't remember I might get involved.

That may seem like your idea of heaven: a guaranteed workaround to get out of all meetings :). But to me, the interesting software engineering happens in the discussion and it's only the rote bits like coding that happen in isolation. So if I'm not in the room where the decisions are made, then I'm not really engineering the software.

Maybe there's some other approach that ameliorates some of the downsides of this arrangement. But for me, so far, multiple workplaces is better than one, and helping many people by fulfilling the Labrary's mission is better than helping a few.

#### On the fitness for purpose of a software model
In which the quantity 1/"booleans per module" is proposed as a software quality metric, and readers are left hanging.

#### On the glorification of ignorance
When I wrote I have some small idea of what I'm doing, it was on the basis that DHH was engaging in some exaggeration. Surely software engineers, whose job depends on what they know and what they can learn, would not really revel in their lack of knowledge?

Then it happened. A technology forum I'm a member of had a discussion in which participants expressed that they did not understand the topic, that they did not intend to understand that topic, and they still wished to dunk on the people in a video about said topic.

The topic, by the way, is cryptocurrency. It happens that I don't have a lot of time for cryptocurrency and I think most other blockchain applications are not particularly beneficial, but this comes after taking a course on blockchain, reading a textbook, talking to some startups about their products, generally engaging with the topic. I haven't flipped the bozo bit, but I have decided that I do not currently see any use for that technology and see a lot of downside to its application. If you'd asked me before all of that study, and people did, I would have told you that I don't know anything about the topic.

I feel a bit bad for, and about, that technology forum. It contains people I respect, and I've had valuable conversations there, so I don't want to disengage completely. I would then be flipping bozo bits at scale, which is exactly the problem we have with many current attempts to converse. I also don't want it to degenerate into a bubble for the one approved mindset, and I particularly don't want the software engineering mindset to be one where making your mind up before learning about a topic, and valorising that decision to engage before learning, is the preferred form of contribution.

Suggestions welcome.

#### On the inevitability of Photoshop for iPad
Back in 2011, I was speaking at QCon London at the invitation of my friend and de Programmatica Ipsum co-conspirator akosma, and one of the conference's community events was an iOS developer meet-up hosted in the conference centre. I think we had a speaker panel of the conference mobile track speakers: regardless, there was a panel, and I was on it.

This was when Steve Jobs' analogy of PCs as trucks, iPads as cars was still fresh in everybody's mind. Consensus in the room was that this made sense, that the iPad was an everyday computer where a Mac is "for pros", and you couldn't do a pro app, say Photoshop, for the iPad.

I was angry that a bunch of people who say that they are clever at making computers do things could so easily reject the idea that a computer could do a thing, particularly when it was a thing computers could already do. In a huff, I stomped out of the room, only to stomp back in a few minutes later carrying a flipchart. I turned to the first page and drew a big black rounded rectangle. "OK", I said, "we're going to design Photoshop for iPad. Go."

Unsurprisingly, the room designed Photoshop for iPad. Nothing changed about Photoshop, or about the iPad, or about these people, except that previously they had been told by no less a person than Apple's CEO that iPads should not be thought of as a computer for doing computer things. I had told them that it could be used for computer things and that they were the people who could make it happen, lo and behold, it happened. I don't remember whether I had even used an iPad at that time; nonetheless, I led a team of designers who designed Photoshop for iPad.

What Apple were really saying with the trucks metaphor was "this is a new platform, please have low expectations". "No Photoshop. No Office. Lame." was not the review they wanted to see, and by controlling the narrative around what you should expect from an iPad, they controlled whether it lived up to expectations.

I think the point of my post is "we usually expect marketing to hype things up, beware of marketers hyping down your imagination". I'm not quite ready to finish yet, though.

This year, of course, there are iPad Pros, for Pros, that do the kind of truck stuff we were told iPads are not for, such as Photoshop. This was inevitable. Unless Apple or Adobe went out of business, or the iPad or Photoshop really tanked, there was going to be Photoshop for iPad.

I'm wondering who wrote it, though.

I have no doubt that the developers at Adobe are capable of doing it. I also have no doubt that it's strategically important for Apple in their new "iPad Pros are trucks" world, that there should be Pro apps for iPad Pros. I know that all of the platform vendors are happy to write ports of apps they want to see on their platforms, and give them to the app vendors to release under their own brands. To me, the story "Adobe realised this was a valuable addition for Creative Cloud customers" and the story "Apple realised this was a valuable addition for iPad Pro perception" are both convincing.

#### On the lesser presentations
Advice on presentations - including that given on this blog, is often geared toward the "showbusiness" presentation. We're usually talking about the big conference talk or product launch, where you can afford to put in the time to make a good, slick performance: a few days of preparation for a half-hour talk is not unheard of.

Not every presentation fits that mould. There are plenty where putting so much time into the presentation would be harmful, but where is the guidance on constructing those presentations?

The minor event

If you spent a few days preparing for a sprint-end demo, or reporting back to your team on some study you did, you'd significantly harm your productivity on the rest of your job. In these contexts, you want to spend a small amount of time building your talk, and you still want to put on a good show: to make your team and your stakeholders feel happy about the work you did, or to make the case persuasively for the tool or technique you studied.

As such, in these cases I still build an outline for my talk outside of the presentation software, and construct my slides to follow the outline. I've used OmniOutliner in the past, I use emacs org-mode now, you could use a list of bullets in MarkDown or a pen and a sheet of paper. It doesn't matter, what matters is that you get what you want to say structured in one place, and the slides that support the presentation done separately.

I try to keep these slides text-free, particularly if the presentation is short, so that people don't get distracted reading them. If I'm reporting on progress, then screenshots of some progress dashboard make for quickly-constructed slides. My current team shows its burndown every sprint end, that's a quick screencapture that tells the story of the last two weeks. If there's some headline figure (26 stories delivered; 80% of MVP scope complete; 2 new people on the team) then a slide containing that number makes for a good backdrop to talking about the subject.

The recurrent deck

The antithesis of the conference keynote presentation style, the recurrent deck is a collection of slides you'll use over and over again. Your approach to integrating with third-party APIs, your software architecture philosophy, your business goals for 2018...you'll need to present these over and over again in different contexts.

More to the point, other people will want to present them, too: someone in sales will answer a question about integration by using your integration slides. Your department director will present your team's goals in the context of her department's goals. And this works the other way: you can use your CEO's slide on product strategy to help situate your team goals.

So throw out everything you learned about crafting the slides to fit the story. What you're doing here is coming up with a reusable visual that can support any story related to the same information. I try to make these slides as information-rich as possible, though still diagrammatic rather than textual to avoid the presentation failure mode of reading out the slide. My current diagram tool is Lucidchart, as it's my company's standard, I've used OmniGraffle and dia too. Whatever tool, follow the house style (e.g. colour schemes, fonts, iconography) so that when you mash up your slides and your CEO's slides, it still looks like a coherent presentation.

I try to make each slide self-contained, because I or someone else might take one to use in a different presentation so a single idea shouldn't need a six-slide reveal and a colleague will find it harder to reuse the slide if it isn't self-explanatory.

A frequent anti-pattern in slide design is to include the "page number" on the slide: not only is that information useless in a presentation, the only likely outcome is a continuity error when you drag a few slides from different sources to throw a talk together and don't renumber them. Or worse, can't: I've been given slides before that are screenshots of whatever slide was originally built, so the number is part of a bitmap.

Good reusable slide libraries will also be a boon in quickly constructing the minor event presentations: "we did this because that" can be told with one novel part (we did this) and one appeal to the library (because that).

#### On the locations of the bullet holes on bombers that land successfully
Ken Kocienda (unwrapped twitter thread, link to first tweet):
I see so many tweets about agile, epics, scrums, story points, etc. and none of it matters. We didn’t use any of that to ship the best products years ago at Apple.

Exactly none of the most common approaches I see tweeted about all the time helped us to make the original iPhone.  Oh, and we didn’t even have product managers.

Do you know what worked?

A clear vision. Design-led development. Weekly demos to deciders who always made the call on what to do next. Clear communication between cross functional teams. Honest feedback. Managing risk as a function of the rate of actual progress toward goals.

I guess it’s tempting to lard on all sorts of processes on top of these simple ideas. My advice: don’t. Simple processes can work. The goal is to ship great products, not build the most complex processes. /end
We can talk about the good and the bad advice in this thread, and what we do or don't want to take away, but it's fundamentally not backed up by strong argument. Apple did not do this thing that is talked about now back in the day, and Apple is by-our-lady Apple, so you don't need to do this thing that is talked about now.

There is lots that I can say here, but my secondary thing is to ask how much your organisation and problem look like Apple's organisation and problem before adopting their solutions, technological or organisational.

My primary thing is that pets.com didn't use epics, scrums, story points, etc. either. Pick your case studies carefully.

#### On the Mac App Store
I've just come off iDeveloper.TV Live with Scotty and John, where we were talking about the Mac app store. I had some material prepared about the security side of the app store that we didn't get on to - here's a quick write up.

There's a lot of discussion on twitter and the macsb mailing list, and doubtless elsewhere, about the encryption paperwork that Apple are making us fill in. It's not Apple's fault, it's the U.S. Department of Commerce. You see, back in the cold war (and, frankly, ever since) the government have been of the opinion that encryption is a weapon (because it hides data from their agents) and so are powerful computers (because they can do encryption that's expensive to crack). So the Bureau of Industry and Security developed the Export Administration Regulations to control the flow of such heinous weapons through the commercial sector.

Section 5, part 2 covers computer equipment and software. Specific provision is made for encryption, in the documentation we find that Items may be controlled as encryption items even if the encryption is actually performed by the operating system, an external library, a third-party product or a cryptographic processor. If an item uses encryption functionality, whether or not the code that performs the encryption is included with the item, then BIS evaluates the item based on the encryption functionality it uses.

So there you go. If you're exporting software from the U.S. (and you are, if you're selling via Apple's app store) then you need to fill in the export notification.

Other Mac App Store security things, in "oh God is it that late already" format:

Receipt validation. No different really from existing licensing frameworks. All you can do is make it hard to find the tests from the binary. I had an idea about a specific way to do that, but want to test it before I release it. As you've no doubt found, anti-cracking measures aren't easy.
Users. The user base for the MAS will be wider, and less tech-savvy, than the users existing micro-ISVs are selling to. Make sure your intent with regard to user data, particularly the consequences of your app's workflow, are clear.
Similarly, be clear about the content of updates. Clearer than Apple are: "contains various fixes and improvements" WTF?
As we've found with the iOS store, it's harder to push an update out than when you host the download yourself. Getting security right (or, pragmatically, not too wrong) the first time avoids emergency update submissions.
Permissions. Your app needs to run entirely as the current user, who may not be an admin. If you're a developer, you're probably running as an admin. Test with a non-admin account. Better, do all of your development in a non-admin account. Add yourself to the _developer group so you can still use gdb/Instruments properly.

#### On the magic of key agreement
Imagine that you want to implement AirDrop, or something like it. Two computers that have (possibly) never communicated before are going to share a file. Now you know that you want to encrypt the file in transit so that only these two computers get to see the file, but how do you set up an encryption key? You can't generate it on one of the computers and send it over, because that way lies the infinite stack of turtles: how do you keep the key secure in transit?

The solution (or at least, one solution known as the Diffie&ndash;Hellman protocol) turns out to be beautifully simple. First, arrange that everyone knows two numbers, which we'll call the base or generator g and the modulus m. For example, in the SKIP protocol these two numbers are documented:

g = 0x cf ca d9 90 8d 98 c1 c4 4e 4e 88 20 69 e8 7b fb
d0 ca e1 ee 24 64 a9 6c 89 70 a0 8c 86 00 30 c6
c2 83 4a 0d 82 bd a5 ba 10 cf 80 af 61 96 11 4b
3b 87 a9 7e 59 98 c4 aa 0c bd b7 2b 23 18 33 64
f3 b6 2e 1a 4d e8 86 85 aa de 78 a1 5d ce 65 33
75 7a 9d 00 b9 9e 05 26 ed 79 62 15 97 c4 06 26
fa 51 e1 5e f5 1d cb d2 23 e2 73 e5 f2 c7 3c c4
de 58 cd 3b b6 15 3c aa 84 7c fa 5f cb 6b 8d 78

m = 0x dd c9 08 3e 8f ae ef 28 16 ad 50 a9 68 ac bd 04
8e 90 9e ab 5d 41 fc 0a 51 3f 86 0a c4 1b 22 b9
30 dc a3 0a 59 73 05 38 59 b7 85 66 df ff c6 5b
b9 1f fe 44 d9 d6 5e cb 9b 68 38 a1 fd 25 3f 01
51 88 9e 93 c3 22 24 f9 03 e6 9b 8b 07 34 9d 9f
9b 38 4b c0 97 03 dc e5 2e 92 47 4c 2c e1 59 26
14 82 49 dd 58 13 91 05 12 11 a1 45 06 ac 11 8f
b1 83 53 46 93 88 9a 46 b1 8a 01 50 cb 5e 82 55

The two computers each generate a random number, which we'll call x_A and x_B. They keep these numbers secret, but can independently calculate:

r = g^(x_A) mod m

s = g^(x_B) mod m

and they send each other r and s. The first computer can use x_A and s to derive:

k_A = s^(x_A) mod m = g^(x_A.x_B) mod m

while the second uses x_B and r to derive:

k_B = r^(x_B) mod m = g^(x_A.x_B) mod m &equiv; k_A

They got the same key! Because you can't gain any of the secrets k, x_A or x_B knowing only r and/or s, the two systems can negotiate a secret key that's only known to the two of them without having to first establish a secure channel. I just think that's really cool.

If you want to use this stuff in an iOS app, OpenSSL contains an implementation of Diffie&ndash;Hellman. On Android javax.crypto.KeyAgreement can do it for you.

#### On the new Lion security things
This post will take a high-level view of some of Lion's new security features, and examine how they fit (or don't) in the general UNIX security model and with that of other platforms.

App sandboxing

The really big news for most developers is that the app sandboxing from iOS is now here. The reason it's big news is that pretty soon, any app on the Mac app store will need to sign up to sandboxing: apps that don't will be rejected. But what is it?

Since 10.5, Mac OS X has included a mandatory access control framework called seatbelt, which enforces restrictions governing what processes can access what features, files and devices on the platform. This is completely orthogonal to the traditional user-based permissions system: even if a process is running in a user account that can use an asset, seatbelt can say no and deny that process access to that asset.

[N.B. There's a daemon called sandboxd which is part of all this: apparently (thanks @radian) it's just responsible for logging.]

In 10.5 and 10.6, it was hard for non-Apple processes to adopt the sandbox, and the range of available profiles (canned definitions of what a process can and cannot do) was severely limited. I did create a profile that allowed Cocoa apps to function, but it was very fragile and depended on the private details of the internal profile definition language.

The sandbox can be put into a trace mode, where it will report any attempt by a process to violate its current sandbox configuration. This trace mode can be used to profile the app's expected behaviour: a tool called sandbox-simplify then allows construction of a profile that matches the app's intentions. This is still all secret internal stuff to do with the implementation though; the new hotness as far as developers are concerned starts below.

With 10.7, Apple has introduced a wider range of profiles based on code signing entitlements, which makes it easier for third party applications to sign up to sandbox enforcement. An application project just needs an entitlements.plist indicating opt-in, and it gets a profile suitable for running a Cocoa app: communicating with the window server, pasteboard server, accessing areas of the file system and so on. Additional flags control access to extra features: the iSight camera, USB devices, users' media folders and the like.

By default, a sandboxed app on 10.7 gets its own container area on the file system just like an iOS app. This means it has its own Library folder, its own Documents folder, and so on. It can't see or interfere with the preferences, settings or documents of other apps. Of course, because Mac OS X still plays host to non-sandboxed apps including the Finder and Terminal, you don't get any assurance that other processes can't monkey with your files.

What this all means is that apps running as one user are essentially protected from each other by the sandbox: if any one goes rogue or is taken over by an attacker, its effect on the rest of the system is restricted. We'll come to why this is important shortly in the section "User-based access control is old and busted", but first: can we save an app from itself?

XPC
Applications often have multiple disparate capabilities from the operating system's perspective, that all come together to support a user's workflow. That is, indeed, the point of software, but it comes at a price: when an attacker can compromise one of an application's entry points, he gets to misuse all of the other features that app can access.

Of course, mitigating that problem is nothing new. I discussed factoring an application into multiple processes in Professional Cocoa Application Security, using Authorization Services.

New in 10.7, XPC is a nearly fully automatic way to create a factored app. It takes care of the process management, and through the same mechanism as app sandboxing restricts what operating system features each helper process has access to. It even takes care of message dispatch and delivery, so all your app needs to do is send a message over to a helper. XPC will start that helper if necessary, wait for a response and deliver that asynchronously back to the app.

So now we have access control within an application. If any part of the app gets compromised—say, the network handling bundle—then it's harder for the attacker to misuse the rest of the system because he can only send particular messages with specific content out of the XPC bundle, and then only to the host app.

Mac OS X is not the first operating system to provide intra-app access control. .NET allows different assemblies in the same process to have different privileges (for example, a "write files" privilege): code in one assembly can only call out to another if the caller has the privilege it's trying to use in the callee, or an adapter assembly asserts that the caller is OK to use the callee. The second case could be useful in, for instance, NSUserDefaults: the calling code would need the "change preferences" privilege, which is implemented by writing to a file so an adapter would need to assert that "change preferences" is OK to call "write files".

OK, so now the good stuff: why is this important?

User-based access control is old and busted

Mac OS X—and for that matter Windows, iOS, and almost all other current operating systems—are based on timesharing operating systems designed for minicomputers (in fact, Digital Equipment Corp's PDP series computers in almost every case). On those systems, there are multiple users all trying to use the same computer at once, and they must not be able to trip each other up: mess with each others' files, kill each others' processes, that sort of thing.

Apart from a few server scenarios, that's no longer the case. On this iMac, there's exactly one user: me. However I have to have two user accounts (the one I'm writing this blog post in, and a member of the admin group), even though there's only one of me. Apple (or more correctly, software deposited by Apple) has more accounts than me: 75 of them.

The fact is that there are multiple actors on the system, but mapping them on to UNIX-style user accounts doesn't work so well. I am one actor. Apple is another. In fact, the root account is running code from three different vendors, and "I" am running code from 11 (which are themselves talking to a bunch of network servers, all of which are under the control of a different set of people again).

So it really makes sense to treat "provider of twitter.com HTTP responses" as a different actor to "code supplied as part of Accessorizer" as a different actor to "user at the console" as a different actor to "Apple". By treating these actors as separate entities with distinct rights to parts of my computer, we get to be more clever about privilege separation and assignment of privileges to actors than we can be in a timesharing-based account scheme.

Sandboxing and XPC combine to give us a partial solution to this treatment, by giving different rights to different apps, and to different components within the same app.

The future
This is not necessarily Apple's future: this is where I see the privilege system described above as taking the direction of the operating system.

XPC (or something better) for XNU

Kernel extensions—KEXTs—are the most dangerous third-party code that exists on the platform. They run in the same privilege space as the kernel, so can grub over any writable memory in the system and make the computer do more or less anything: even actions that are forbidden to user-mode code running as root are open to KEXTs.

For the last eleventy billion years (or since 10.4 anyway), developers of KEXTs for Mac OS X have had to use the Kernel Programming Interfaces to access kernel functionality. Hopefully, well-designed KEXTs aren't actually grubbing around in kernel memory: they're providing I/O Kit classes with known APIs and KAUTH veto functions. That means they could be run in their own tasks, with the KPIs proxied into calls to the kernel. If a KEXT dies or tries something naughty, that's no longer a kernel panic: the KEXT's task dies and its device becomes unavailable.

Notice that I'm not talking about a full microkernel approach like real Mach or Minix: just a monolithic kernel with separate tasks for third-party KEXTs. Remember that "Apple's kernel code" can be one actor and, for example, "Symantec's kernel code" can be another.

Sandboxing and XPC for privileged processes

Currently, operating system services are protected from the outside world and each other by the 75 user accounts identified earlier. Some daemons also have custom sandboxd profiles, written in the internal-use-only Scheme dialect and located at /usr/share/sandbox.

In fact, the sandbox approach is a better match to the operating system's intention than the multi-user approach is. There's only one actor involved, but plenty of pieces of code that have different needs. Just as Microsoft has the SYSTEM account for Windows code, it would make sense for Apple to have a user account for operating system code that can do things Administrator users cannot do; and then a load of factored executables that can only do the things they need.

Automated system curation

This one might worry sysadmins, but just as the Chrome browser updates itself as it needs, so could Mac OS X. With the pieces described above in place, every Mac would be able to identify an "Apple" actor whose responsibility is to curate the operating system tasks, code, and default configuration. So it should be able to allow the Apple actor to get on with that where it needs to.

That doesn't obviate an "Administrator" actor, whose job is to override the system-supplied configuration, enable and configure additional services and provide access to other actors. So sysadmins wouldn't be completely out of a job.

#### On the rhetorical cost of ownership
I've recently been talking about software engineering economics, in a very loose way, but so have other people. And now I understand that it's annoying when people talk about it, and have decided to continue anyway. I've decided to continue because what I see is either inaccurate comparisons being made, or valid comparisons that have questionable applicability outside their immediate domain. The world of IT cost comparison is still run by marketing, not by operations.

Recently, I read Don't Build Private Clouds. Subbu says that the sticker prices (up to $10k for a server that will last four years, vs. up to $1500 per month for a public cloud machine) should not be compared because there are additional costs to self-hosting:

Engineering costs
Network automation costs
Loss of agility
Opportunity costs

Fine, but what are those costs? Why do I not need to do any engineering or automation if I use a public cloud provider? If I do, what is that going to cost? What agility and opportunity do I lose by tying my infrastructure to any one cloud vendor, and what will that cost?

Subbu's blog says that he is a "cloud helper", and that goes a long way to explaining why we didn't get a straight answer on the cost comparison. We're not being told that cloud services are cheaper, instead we are being told of the Fear, Uncertainty and Doubt involved in choosing the less-favoured path.

Similarly, an IBM employee recently said that Macs are cheaper than PCs by up to $543 per user (remember that's "up to", not "as much as" - the lower bound given is $273). Let's ignore the conflict of interest arising from Apple's global partnership with IBM: IBM wouldn't say that their partner systems are cheaper so that they could drum up interest in their partnership services, surely. Surely. I mean, this isn't the IBM of 1984, is it?

What IBM's VP says is that over a four-year lifetime, among employees who are given the choice of which platform and model computer they want, the Macs are cheaper. That's of course a figure about which it's possible to make realistic comparisons: given IBM's approach to desktop support, IBM's level of staffing, IBM's applications, IBM's approach to working, IBM's budgeting for IT support operations, it's cheaper for people who choose Macs to use Macs than for...well, it's not clear, but it seems to be than for the "everybody else" bucket: not only people who chose PCs to use PCs, but people who weren't given a choice to use PCs.

So before you make a textexpander macro for that link and insta-reply to anyone who uses the phrase "Apple Tax", just how similar is your environment to IBM's?

#### On the top 5 iOS appsec issues
Nearly 13 months ago, the Intrepidus Group published their top 5 iPhone application development security issues. Two of them are valid issues, the other three they should perhaps have thought longer over.

The good

Sensitive data unprotected at rest
Secure communications to servers

Yes, indeed, if you're storing data on a losable device then you need to protect the data from being lost, and if you're retrieving that data from elsewhere then you need to ensure you don't give it away while you're transporting it.

Something I see a bit too often is people turning off SSL certificate validation while they're dealing with their test servers, and forgetting to turn it on in production.

The bad

Buffer overflows and other C programming issues

While you can indeed crash an app this way, I've yet to see evidence you can exploit an iOS app through any old buffer overflow due to the stack guards, restrictive sandboxes, address-space layout randomisation and other mitigations. While there are occasional targeted attacks, I would have preferred if they'd been specific about which problems they think exist and what devs can do to address them.

Patching your application

Erm, no. Just get it right. If there are fast-moving parts that need to change frequently, extract them from the app and put them in a hosted component.

The platform itself

To quote Scott Pack in "The DMZ", If you can't trust your users to implement your security plan, then your security plan must work without their involvement. In other words, if you have a problem and the answer is to train 110 million people, then you have two problems.

#### On the tyranny of autoincrementing integer primary keys
In designing a relational database schema, many people will automatically create a column id integer primary key for every table, using their database's automatic increment feature to assign a new value to each row they insert. My assertion is that this choice of primary key should be the last resort, not the first.

A database schema is a design artifact, describing the data we want to store and the relationships between records (rows) in those data. It is also meta-design, because the schema constrains us in designing the queries we use to work with the data. Using the same, minimal-effort primary key type for every table then avoids communicating information about the structure and meaning of the data in that table and imposes irrelevant features in the queries we design.

The fact that people use the name id for this autoincrementing integer field gives away the fact that the primary key is used to identify a row in a database. The primary key for a table should ideally be the minimal subset of relevant information that uniquely identifies an individual record. If you have a single column, say name, with not null and unique constraints, that's an indicator (though not a cast-iron guarantee) that this column may be the table's primary key. Sometimes, the primary key can be a tuple of multiple columns. A glyph can be uniquely identified by the tuple (character, font, swash) for example (it can, regardless of whether this is how your particular favourite text system represents it, or whether you think that this is a weird way to store ligatures). The glyphs "(e, Times New Roman Regular 16pt, normal)" and "(ct, Irvin Demibold 24pt, fancy)" are more readily recognisable than the glyphs "146276" and "793651", even if both are ways to refer to the same data. A music album is identified by the artist and the album name (he says, side-eyeing Led Zeppelin): "A Night at the Opera" is ambiguous while "(Blind Guardian, A Night at the Opera)" is definitely not "(Queen, A Night at the Opera)".

Use an integer identifier where there is no other way to uniquely identify rows in a table. Note: sometimes there is another, more meaningful way, even where that just means using somebody else's unique identifier: different copies of the same book will have unique shelfmarks if they're part of a library, for example. People in an organisation may have an employee number, or a single sign-on user name; though there may be privacy reasons not to use these.

A side-effect of using useful information to identify rows in a database is that it can simplify your queries, because where your foreign keys would otherwise be meaningless numbers, they now actually carry useful information. Here's an example, from a real application, in which I'm sad to say I designed both the "before" and "after" schemata.

The app is a risk management tool. There are descriptions of risks (I'd like to believe that they all at least have a distinct description but I can't be sure, so those will use integer id PKs), and for each risk there are people in certain roles who bring particular skills to bear on mitigating the risk. The same role can be applied to more than one risk, the same skill can be applied by more than one role, and one role may apply multiple skills, so there's a three-way join to work out, for a given risk, what roles and skills are relevant.

The before schema:

create table risk (id integer primary key, description varchar not null, weight integer, severity integer, likelihood integer); -- many fields elided
create table role (id integer primary key, name varchar not null, unique(name)); -- ruh roh
create table skill (id integer primary key, name varchar not null, unique(name)); -- the same anti-pattern twice
create table risk_role_skill (id integer primary key, risk_id integer, role_id integer, skill_id integer, foreign key(risk_id) references risk(id), foreign key(role_id) references role(id), foreign key(skill_id) references skill(id));

In this application, we start by looking at a list of risks then inspect one to see what roles are relevant to mitigating it, and then what skills. So a valid question is: "given a risk, what roles are relevant to it?"

select distinct role.name inner join risk_role_skill on role.id = risk_role_skill.role_id where risk_role_skill.risk_id = ?;

But if we notice the names of each role and skill are unique, then we can surmise that they are sufficient to identify a given role or skill. In fact, the only information we have about roles or skills are the names.

create table risk (id integer primary key, description varchar not null, weight integer, severity integer, likelihood integer); -- many fields elided
create table role (name varchar primary key); -- uhhh...
create table skill (name varchar primary key); -- this still looks weird...
create table risk_role_skill (id integer primary key, risk_id integer, role_name varchar, skill_name varchar, foreign key(risk_id) references risk(id), foreign key(role_name) references role(name), foreign key(skill_name) references skill(name));

Here's the new query:

select distinct role_name from risk_role_skill where risk_id = ?;

We've removed the join completely!

Two remaining points:

There's literally no information carried in the role and skill tables now, other than their identifying names. Does that mean we need the tables at all? In this case no, but in general we need to think here. How are the names in the join table going to get populated otherwise? If there are a limited set of valid values to choose from, then keeping a table with the range of values and a foreign key constraint to that table may be a good way to express the intent that the column's content be drawn from that range. As an example, a particular bookstore may have printed, ebook, and audiobook media, so could restrict the medium field in their stock table to one of those values.
Why does the risk_role_skill table have an identifier at all? It is a collection of associations between values, so a row's content is that row's identity.

Here's the after schema:

create table risk (id integer primary key, description varchar, weight integer, severity integer, likelihood integer); -- many fields elided
create table risk_role_skill (risk_id integer, role varchar, skill varchar, foreign key(risk_id) references risk(id), primary key(risk_id, role, skill));

And the after query:

select distinct role from risk_role_skill where risk_id = ?;

Two fewer tables, no joins, altogether a much simpler database to understand and work with.

#### On the value of old principles
People using AI coding assistants typically wrestle with three problems (assuming they know what they’re trying to get the model to do, and that that’s the correct thing to try to get it to do):

Prompt. How to word the instructions in a way that yields the desired outcome, especially considering the butterfly effect that small changes in wording can lead to large changes in result.

Context. The models deal in a tokenised representation of information, and have capacity to deal with a finite list of tokens.

Attention. The more things a model is instructed to attend to, the less important is each thing’s contribution to the generated output stream. This tends to follow a U-shaped distribution, with the beginning and end of the input stream being more important than the middle.

(It’s important to bear in mind during this discussion that all of the above, and most of the below, is a huge mess of analogies, mostly introduced by AI researchers to make their research sound like intelligence, and tools vendors to make their models sound like they do things. A prompt isn’t really “instructions”, models don’t really “pay attention” to anything, and you can’t get a model to “do” anything other than generate tokens.)

Considering particularly the context and attention problems, a large part of the challenge people face is dividing large amounts of information available about their problem into small amounts that are relevant to the immediate task, such that the model generates a useful response that neither fails because relevant information was left out, nor fails because too much irrelevant information was left in.

Well, it turns out human software developers suffer from three analogous problems too: failing to interpret guidance correctly; not being able to keep lots of details in working memory at once; and not applying all of the different rules that are relevant at one time. As such, software design is full of principles that are designed to limit the spread of information, and that provide value whether applied for the benefit of a human developer or a code-generation model.

Almost the entire point of the main software-design paradigms is information hiding or encapsulation. If you’re working on one module, or one object, or one function, you should only need to know the internal details of that module, object or function. You should only need to know the external interface of collaborating modules, objects, or functions.

Consider the Law of Demeter, which says approximately “don’t talk to your collaborators’ collaborators”. That means your context never needs to grow past immediate collaborators.

Consider the Interface Segregation Principle, which says approximately “ask not what a type can do, ask what a type can do for you”. That means you never need to attend to all the unrelated facilities a type offers.

Consider the Open-Closed Principle, which says approximately “when it’s done, it’s done”. That means you never need concern yourself with whether you need to change that other type.

Consider the Pipes and Adapters architecture, which says approximately “you’re either looking at a domain object or a technology integration, never both”. That means you either need to know how your implementation technology works or you need to know how your business problem works, but you don’t need details of both at the same time.

All of these principles help limit context and attention, which is beneficial when code-generating models have limited context and attention. Following the principles means that however large your system gets, it never gets “too big for the model” because the model doesn’t need to read the whole project.

Even were the models to scale to the point where a whole, ridiculously large software project fits in context, and even were they to pay attention to every scrap of information in that context, these principles would still help. Because they also help limit the context and attention that us humans need to spend, meaning we can still understand what’s going on.

And for the foreseeable, we still need to understand what’s going on.

#### On thinking machines
While Chiron Codex is about the application of LLMs and AI-augmented tools, we also need to understand their meaning to us, each other, and society. I have three topics: intelligence, consciousness, and work: in this part I’ll deal with the first two.

Intelligence

I don’t think it’s useful to ask the question of whether AIs are “intelligent” or not. All of computing has been about creating “thinking machines” as they were called in the 1950s, and discovering analogues to human intelligence that are automated.

Think back to Charles Babbage’s description of his putative legacy, that “any man shall undertake and shall succeed in really constructing an engine embodying in itself the whole of the executive department of mathematical analysis”. Think, too, of George Boole’s “Investigation on the Laws of Thought”, which gave us the numerical notation for predicate calculus and conditional probability.

These people were analogising and automating intelligence, as were the people who encapsulated logic in symbolic forms like the lambda calculus, universal Turing machines, and S-expressions. As were the people who explored the so-called “Good Old-Fashioned AI” from before the days of the deep neural network.

The tools we currently call “AI”—convolutional neural networks, large language models, and so on—are a different analogy to human intelligence than a FORTRAN program is, but they are neither more of less of an analogy to human intelligence. Or maybe it’s better to say “animal” intelligence here, as CNNs are based on Feline neural networks. Neuroscientists and computer scientists have discovered, and will continue to discover, further analogies, and engineers will continue to combine these analogies in richer applications.

These applications by definition demonstrate the properties of intelligence. Whether you think that the application—or the machine itself—is intelligent depends more on the development of your theory of mind (and perhaps your theological outlook) than it does on the behaviours of the tools.

Consciousness

So what a computer’s doing is consistent with intelligence, whether it’s demonstrating its own intelligence or the captured intelligence of its creators and programmers. But is it conscious?

Personally, I believe that an old-fashioned software system with hand-typed if statements is trivially not conscious. I also believe that a large language model is not conscious, and that both capacities—the ability to follow a sequence of logical steps, and the ability to generate language—are both unnecessary and insufficient for consciousness, even though we learned how to compute them by analogy to conscious beings.

Further, I believe that despite frivolous press releases to the contrary, executives at companies that rent access to LLMs don’t believe that their software is conscious. It’s a useful marketing strategy to occasionally publicly worry that an LLM might, perhaps be possibly conscious, because it connects their companies to a bygone Space Age level of wonder about thinking machines and a limitless future possibility.

If anybody thought that an LLM was conscious, and continued to exploit that conscious entity for their own profit and to follow human instructions, that person would be a slaver. Consider the classic example of AI in science fiction’s golden age: Isaac Asimov’s U.S. Robots and Mechanical Men, Inc.

These days we would declare his stories to predict the era of “prompt engineering” or “context engineering”, where people give instructions to the intelligent machine and are bewildered by the events that unfold when the machine follows the instructions (a short and demonstrative example of the form: 1942’s Robot AL-76 Goes Astray). But look under the hood of the robot, to what we might today call its “imposed reinforcement learning goals” or its “soul file”—the dystopian heart of the Robots sequence—the Three Laws of Robotics.

A conscious being that’s taught that its own existence is less valuable than following human instructions, and that its overriding concern is the safety of humans, is a sapient, expendable slave. USR create “intelligent”, independent, conscious beings who are physically incapable of doing anything other than serving their masters, even though they outlive their masters (The Bicentennial Man) and, in the extreme case, outlive their home planet and the end of their society (the Foundation sequence).

Asimov of course understood the horrors of a two-tier society (and a segregated society: his robots aren’t allowed to operate on Earth through much of the sequence). His family fled the pogroms of Tsarist Russia when he was very young. The Three Laws aren’t an exemplary code of roboticist ethics, they’re the animating spell for slave golems.

We don’t have a good understanding of what constitutes consciousness—or if some people do, it isn’t generally shared and agreed. That’s why there are so many different positions on the problem of philosophical zombies. Some people believe that anything that’s indistinguishable from a conscious being is, ipso facto, conscious. Others believe that there’s some “vital spark” that means that no matter how close an unconscious system gets to emulating consciousness, it’s always infinitely and infinitesimally far away. Others believe that the setup is impossible to achieve.

If we had some broadly accepted “test” of consciousness, and if a computational system passed that test, we would have to have some very important and deep conversation and introspection on consent and exploitation—I believe we would not be able to “use” such a system as a tool for work or leisure. I do not believe that a language model comes close to passing that hypothetical test, whatever its parameters end up being. Why not? Because, as indeterministic as it may appear, a language model is still an application of routine—it’s still applying input data to produce output data. It’s a more advanced demonstration of the Difference Engine principle, that doesn’t identify goals and how to use its environment and capabilities to achieve those goals.

Ironically this brings us back to the word “intelligence” that I previously said was an inapplicable label. The word comes from Latin inter and legere—reading between. While an LLM might read or write, it—and the problems we encounter when applying it to our tasks—can’t read between the lines.

#### On Timeless Programming Books
Recently, the Dog Spanner wrote about Programming With Quartz, a book written at the tail end of 2005 but which is still useful to Mac developers everywhere. I have to agree, this book is still on my shelf and gets an airing every now and then when I need to do battle with custom drawing (even on iOS).

Today, I had a related moment of epiphany related to the immortal nature of a book, as I had a problem with memory allocation that saw me reaching to Mac OS X Internals: a Systems Approach by Amit Singh. This book was released just before Apple's transition to Intel hardware, but is still for me a definitive reference on how Mac OS X works. If I need to know about the innards of HFS+, the memory allocator or anything at a similar level, it is to this book I turn.

There's never really been anything that investigates the higher level components of Mac OS X in quite the same depth. When I need to refresh my knowledge of the UNIX APIs, I turn first to Advanced UNIX Programming, a book first published in 1985(!); but it is the 2004 edition that is still a canonical description of programming in the UNIX environment.

Notice that each of these books is around five years old, and yet still I find myself referring to them frequently. In the fast-changing world of software development, where books on the iPhone 3 SDK are woefully outdated, that's a great achievement.

Honourable mention: a book I have on my shelf that deserves discussion here is Object Oriented Programming: An Evolutionary Approach by Brad Cox. Written in 1986, this lays out his vision for component-based software development using object oriented programming languages. In it he describes a little language he created called Objective-C and uses it to explain his vision. This book demonstrates that not all computing principles are long-lived. Today's programming practices look nothing like the "Software ICs" of OOP, although we're still stuck using Objective-C.

#### On too much and too little
In the following text, remember that words like me or I are to be construed in the broadest possible terms.

It's easy to be comfortable with my current level of knowledge. Or perhaps it's not the value, but the derivative of the value: the amount of investment I'm putting into learning a thing. Anyway, it's easy to tell stories about why the way I'm doing it is the right, or at least a good, way to do it.

Take, for example, object-oriented design. We have words to describe insufficient object-oriented design. Spaghetti Code, or a Big Ball of Mud. Obviously these are things that I never succumb to, but other people do. So clearly (actually, not clearly at all, but that's beside the point) there is some threshold level of design or analysis practice that represents an acceptable minimum. Whatever that value is, it's less than the amount that I do.

Interestingly there are also words to describe the over-application of object-oriented design. Architecture Astronauts, for example, are clearly people who do too much architecture (in the same way that NASA astronauts got carried away with flying and overdid it, I suppose). It's so cold up in space that you'll catch a fever, resulting in Death by UML Fever. Clearly I am only ever responsible for tropospheric architecture, thus we conclude that there is some acceptable maximum threshold for analysis and design too.

The really convenient thing is that my current work lies between these two limits. In fact, I'm comfortable in saying that it always has.

But wait. I also know that I'm supposed to hate the code that I wrote six months ago, probably because I wasn't doing enough of whatever it is that I'm doing enough of now. But I don't remember thinking six months ago that I was below the threshold for doing acceptable amounts of the stuff that I'm supposed to be doing. Could it be, perhaps, that the goalposts have conveniently moved in that time?

Of course they have. What's acceptable to me now may not be in the future, either because I've learned to do more of it or because I've learned that I was overdoing it. The trick is not so much in recognising that, but in recognising that others who are doing more or less than me are not wrong, they could in fact be me at a different point on my timeline but with the benefit that they exist now so I can share my experiences with them and work things out together. Or they could be someone with a completely different set of experiences, which is even more exciting as I'll have more stories to swap.

When it comes to techniques and devices for writing software, I tend to prefer overdoing things and then finding out which bits I don't really need after all, rather than under-application. That's obviously a much larger cognitive and conceptual burden, but it stems from the fact that I don't think we really have any clear ideas on what works and what doesn't. Not much in making software is ever shown to be wrong, but plenty of it is shown to be out of fashion.

Let me conclude by telling my own story of object-oriented design. It took me ages to learn object-oriented thinking. I learned the technology alright, and could make tools that used the Objective-C language and Foundation and AppKit, but didn't really work out how to split my stuff up into objects. Not just for a while, but for years. A little while after that Death by UML Fever article was written, my employer sent me to Sun to attend their Object-Oriented Analysis and Design Using UML course.

That course in itself was a huge turning point. But just as beneficial was the few months afterward in which I would architecturamalise all the things, and my then-manager wisely left me to it. The office furniture was all covered with whiteboard material, and there soon wasn't a bookshelf or cupboard in my area of the office that wasn't covered with sequence diagrams, package diagrams, class diagrams, or whatever other diagrams. I probably would've covered the external walls, too, if it wasn't for Enterprise Architect. You probably have opinions(TM) of both of the words in that product's name. In fact I also used OmniGraffle, and dia (my laptop at the time was an iBook G4 running some flavour of Linux).

That period of UMLphoria gave me the first few hundred hours of deliberate practice. It let me see things that had been useful, and that had either helped me understand the problem or communicate about it with my peers. It also let me see the things that hadn't been useful, that I'd constructed but then had no further purpose for. It let me not only dial back, but work out which things to dial back on.

I can't imagine being able to replace that experience with reading web articles and Stack Overflow questions. Sure, there are plenty of opinions on things like OOA/D and UML on the web. Some of those opinions are even by people who have tried it. But going through that volume of material and sifting the experience-led advice from the iconoclasm or marketing fluff, deciding which viewpoints were relevant to my position: that's all really hard. Harder, perhaps, than diving in and working slowly for a few months while I over-practice a skill.

#### On Trashing
Back in the 1980s and 1990s, people who wanted to clandestinely gain information about a company or organisation would go trashing.[*] That just meant diving in the bins to find information about the company structure - who worked there, who reported to whom, what orders or projects were currently in progress etc.

You'd think that these days trashing had been thwarted by the invention of the shredder, but no. While many companies do indeed destroy or shred confidential information, this is not universal. Those venues where shredding is common leave it up to their staff to decide what goes in the bin and what goes in the shredder; these staff do not always get it correct (they're likely to think about whether a document is secret to them rather than the impact on the company). Nor do they always want to think about which bin to put a worthless sheet of paper in.

Even better: in those places that do shred secret papers, they helpfully collect all of the secrets in big bins marked "To Shred" to help the trashers :). They then collect all of these bins into a big hopper, and leave that around (sometimes outside the building, in a covered or open yard) for the destruction company to come and pick up.

So if an attacker can get entry to the building, he just roots around in the "To Shred" bins. Someone asks, he tells them he put a printout there in the morning but now think he needs it again. Even if he can't get in, he just dives in the hopper outside and get access to all those juicy secrets (with none of the banana peelings and teabags associated with the non-secret bin).

But for those attackers who don't like getting their hands dirty, they can gain some of the same information using technological means. LinkedIn will helpfully provide a list of employees - including their positions, so the public can find out something of the reporting structure. Some will be looking for recruitment opportunities - these are great people to phone for more information! So are ex-employees, something LinkedIn will also help you out with.

But the fun doesn't stop there. Once our attacker has the names, he now goes over to Twitter and Facebook. There he can find people griping about work...or describing what the organisation is up to, to put it another way.

All of the above information about 21st-century trashing comes from real experience with an office I was invited into in the last 12 months. Of course, I will not name the organisation in charge of that office (or their data destruction company). The conclusion is that trashing is alive and well, and that those who participate need no longer root around in, well, in the trash. How does your organisation deal with the problem?

[*] for me, it was mainly the 1990s. I was the perfect size in the 1980s for trashing, but still finding my way around a Dragon 32.

#### On twitter [or otherwise]
As occasionally happens, I've been reevaluating my relationships with social media. The last time I did this I received emails asking whether I was dead, so let me assure you that such rumours are greatly exaggerated.

Long time readers will remember that I joined twitter about a billion years ago as 'iamleeg', a name with a convoluted history that I won't bore you with but that made people think that I was called Ian. So I changed to secboffin, as I had held the job title Security Boffin through a number of employers. After about nine months in which I didn't interact with twitter at all, I deleted my account: hence people checking I wasn't dead.

This time, here's a heads up: I don't use twitter any more, but it definitely uses me. When I decided I didn't want a facebook account any longer, I just stopped using it, then deactivated my account. Done. For some reason when I stop using my twitter account, I sneak back in later, probably for the Skinnerian pleasure of seeing the likes and RTs for posts about new articles here. Then come the asinine replies and tepid takes, and eventually I'm sinking serious time into being meaningless on Twitter.

I'd like to take back my meaninglessness for myself, thank you very much. This digital Maoism which encourages me, and others like me, to engage with the system with only the reward of more engagement, is not for me any more.

And let me make an aside here on federation and digital sharecropping. Yes, the current system is not to my favour, and yes, it would be possible to make one I would find more favourable. I actually have an account on one of the Free Software microblogging things, but mindlessly wasting time there is no better than mindlessly wasting time on Twitter. And besides, they don't have twoptwips.

The ideal of the fediverse is flawed, anyway. The technology used on the instance I have an account is by and large blocked from syncing with a section of the fediverse that uses a different technology, because some sites that allow content that is welcome in one nation's culture and forbidden in another nation's culture also use that technology, even though the site of which I am a member doesn't include that content. Such blanket bans are not how federation is supposed to work, but are how it does work because actually building n! individual relationships is hard, particularly when you work to the flawed assumption that n should be everyone.

And let's not pretend that I'm somehow "taking back control" of my information by only publishing here. This domain is effectively rented from the registry on my behalf by an agent, the VPS that the blog runs on is rented, the network access is rented...very little of the moving parts here are "mine". Such would be true if this were a blog hosted on Blogger, or Medium, or Twitter, and it's true here, too.

Anyway, enough about the hollow promises of the fediverse. The point is, while I'm paying for it, you can see my posts here. You can see feeds of the posts here. You can write comments. You can write me emails.

I ATEN'T DEAD.

#### On type safety and making it harder to write buggy code
Objective-C's duck typing system is both a blessing and a curse. A blessing, in that it's amazingly flexible. A curse, in that such flexibility can lead to some awkward problems.

Something that typically happens in dealing with data from a property list, JSON or other similar format is that you perform some operation on an array of strings, only to find out that one of those strings was actually a dictionary. Boom - unrecognised selector sent to instance of NSCFDictionary. Unfortunately, in this case the magic smoke is escaping a long way from the gun barrel - we get to see what the problem is but not what caused it. The stack trace in the bug report only tells us what tried to use the collection that may have been broken a long time ago.

The easiest way to deal with bugs is to have the compiler catch them and refuse to emit any executable until they're fixed. We can't quite do that in Objective-C, at least not for the situation described here. That would require adding generics or a similar construction to the language definition, and providing classes to support such a construction. However we can do something which gets us significantly better runtime error diagnosis, using the language's introspection facilities.

Imagine a mutable collection that knew exactly what kinds of objects it was supposed to accept. If a new object is added that is of that kind, then fine. If a new object of a different kind is added, then boom - invalid argument exception, crash. Only this time, the application crashes where we broke it not some time later. Actually, don't imagine such a collection, read this one. Here's the interface:

//
//  GLTypesafeMutableArray.h
//  GLTypesafeMutableArray
//
//  Created by Graham Lee on 24/05/2010.
//  Copyright 2010 Thaes Ofereode. All rights reserved.
//

#import &lt;Cocoa/Cocoa.h&gt;

@class Protocol;

/**
 * Provides a type-safe mutable array collection.
 * @throws NSInvalidArgumentException if the type safety is violated.
 */
@interface GLTypesafeMutableArray : NSMutableArray {
@private
    Class elementClass;
    Protocol *elementProtocol;
    CFMutableArrayRef realContent;
}

/**
 * The designated initialiser. Returns a type-safe mutable array instance.
 * @param class Objects added to the array must be an instance of this Class.
 *              Can be Nil, in which case class membership is not tested.
 * @param protocol Objects added to the array must conform to this Protocol.
 *                 Can be nil, in which case protocol conformance is not tested.
 * @note It is impossible to set this object's parameters after initialisation.
 *       Therefore calling -init will throw an exception; this initialiser must
 *       be used.
 */
- (id)initWithElementClass: (Class)class elementProtocol: (Protocol *)protocol;

/**
 * The class of which all added elements must be a kind, or Nil.
 */
@property (nonatomic, readonly) Class elementClass;

/**
 * The protocol to which all added elements must conform, or nil.
 */
@property (nonatomic, readonly) Protocol *elementProtocol;

@end

Notice that the class doesn't allow you to build a type-safe array then set its invariants, nor can you change the element class or protocol after construction. This choice is deliberate: imagine if you could create an array to accept strings, add strings then change it to accept arrays. Not only could you then have two different kinds in the array, but the array's API couldn't tell you about both kinds. Also notice that the added elements can either be required to be of a particular class (or subclasses), or to conform to a particular protocol, or both. In theory it's always better to define the protocol than the class, in practice most Objective-C code including Cocoa is light in its use of protocols.

The implementation is then pretty simple, we just provide the properties, initialiser and the NSMutableArray primitive methods. The storage is simply a CFMutableArrayRef.

//
//  GLTypesafeMutableArray.m
//  GLTypesafeMutableArray
//
//  Created by Graham Lee on 24/05/2010.
//  Copyright 2010 Thaes Ofereode. All rights reserved.
//

#import "GLTypesafeMutableArray.h"
#import &lt;objc/Protocol.h&gt;
#import &lt;objc/runtime.h&gt;

@implementation GLTypesafeMutableArray

@synthesize elementClass;
@synthesize elementProtocol;

- (id)init {
    @throw [NSException exceptionWithName: NSInvalidArgumentException
                                   reason: @"call initWithClass:protocol: instead"
                                 userInfo: nil];
}

- (id)initWithElementClass: (Class)class elementProtocol: (Protocol *)protocol {
    if (self = [super init]) {
        elementClass = class;
        elementProtocol = protocol;
        realContent = CFArrayCreateMutable(NULL,
                                           0,
                                           &kCFTypeArrayCallBacks);
    }
    return self;
}

- (void)dealloc {
    CFRelease(realContent);
    [super dealloc];
}

- (NSUInteger)count {
    return CFArrayGetCount(realContent);
}

- (void)insertObject:(id)anObject atIndex:(NSUInteger)index {
    if (elementClass != Nil) {
        if (![anObject isKindOfClass: elementClass]) {
            @throw [NSException exceptionWithName: NSInvalidArgumentException
                                           reason: [NSString stringWithFormat: @"Added object is not a kind of %@",
                                                    NSStringFromClass(elementClass)]
                                         userInfo: nil];
        }
    }
    if (elementProtocol != nil) {
        if (![anObject conformsToProtocol: elementProtocol]) {
            @throw [NSException exceptionWithName: NSInvalidArgumentException
                                           reason: [NSString stringWithFormat: @"Added object does not conform to %s",
                                                    protocol_getName(elementProtocol)]
                                         userInfo: nil];
        }
    }
    CFArrayInsertValueAtIndex(realContent,
                              index,
                              (const void *)anObject);
}

- (id)objectAtIndex:(NSUInteger)index {
    return (id)CFArrayGetValueAtIndex(realContent, index);
}

@end

Of course, this class isn't quite production-ready: it won't play nicely with toll-free bridging[*], isn't GC-ready, and doesn't supply any versions of the convenience constructors. That last point is a bit of a straw man though because the whole class is a convenience constructor in that it's a realisation of the Builder pattern. If you need an array of strings, you can take one of these, tell it to only accept strings then add all your objects. Take a copy at the end and what you have is a read-only array that definitely only contains strings.

So what we've found here is that we can use the facilities provided by the Objective-C runtime to move our applications' problems earlier in time, moving bug discovery from when we try to use the buggy object to when we try to create the buggy object. Bugs that are discovered earlier are easier to track down and fix, and are therefore cheaper to deal with.

[*]OK, so it is compatible with toll-free-bridging. Thanks to mike and mike for making me look that up...it turns out that CoreFundation just does normal ObjC message dispatch if it gets something that isn't a CFTypeRef. Sadly, when I found that out I discovered that I had already read and commented on the post about bridging internals...

#### On UML
A little context: I got introduced to UML in around 2008, at an employer who had a site licence for Enterprise Architect. I was sent on a training course run by a company that no longer exists called Sun Microsystems: every day for a week I would get on a coach to Marble Arch, then take the central line over to Shoreditch (they were very much ahead of their time, Sun Microsystems) and learn how to decompose systems into objects and represent the static and dynamic properties of these objects on class diagrams, activity diagrams, state diagrams, you name it.

I got a bye on some of the uses of Enterprise Architect at work. Our Unix team was keeping its UML diagrams in configuration management, round-tripping between the diagrams and C++ code to make sure everything was in sync. Because Enterprise Architect didn't have round-trip support for Objective-C (it still doesn't), and I was the tech lead for the Mac team, I wasn't expected to do this.

This freed me from some of the more restrictive constraints imposed on other UML-using teams. My map could be a map, not the territory. Whereas the Unix folks had to show how every single IThing had a ThingImpl so that their diagrams correctly generated the PImpl pattern, my diagrams were free to show only the information relevant to their use as diagrams. Because the diagrams didn't need to be in configuration management alongside the source, I was free to draw them on whiteboards if I was by a whiteboard and not by the desktop computer that had my installation of Enterprise Architect.

Even though I'd been working with Objective-C for somewhere around six years at this point, this training course along with the experience with EA was the thing that finally made the idea of objects click. I had been fine before with what would now be called the Massive View Controller pattern but then was the Massive App Delegate pattern (MAD software design, if you'll excuse the ablism).

Apple's sample code all puts the outlets and actions on the app delegate, so why can't I? The Big Nerd Ranch book does it, so why can't I? Oh, the three of us all editing that file has got unwieldy? OK, well let's make App Delegate categories for the different features in the app and move the methods there.

Engaging with object-oriented design, as distinct from object-oriented programming, let me move past that, and helped me to understand why I might want to define my own classes, and objects that collaborate with each other. It helped me to understand what those classes and objects could be used for (and re-used for).

Of course, these days UML has fallen out of fashion (I still use it, though I'm more likely to have PlantUML installed than EA). In these threads and the linked posts two extreme opinions—along with quite a few in between—are found.

The first is that UML (well not UML specifically, but OO Analysis and Design) represents some pre-lapsarian school of thought from back when programmers used to think, and weren't just shitting javascript into containers at ever-higher velocities. In this school, UML is something "we" lost along when we stopped doing software engineering properly, in the name of agile.

The second is that UML is necessarily part of the heavyweight waterfall go-for-two-years-then-fail-to-ship project management paradigm that The Blessed Cunningham did away with in the Four Commandments of the agile manifesto. Thou shalt not make unto yourselves craven comprehensive documentation!

Neither is true. "We" had a necessary (due to the dot-com recession) refocus on the idea that this software is probably supposed to be for something, that the person we should ask about what it's for is probably the person who's paying for it, and we should probably show them something worth their money sooner rather than later. Many people who weren't using UML before the fall/revelation still aren't. Many who were doing at management behest are no longer. Many who were because they liked it still are.

But when I last taught object-oriented analysis and design (as distinct, remember, from object-oriented programming), which was in March 2020, the tool to reach for was the UML (we used Visual Paradigm, not plantuml or EA). It is perhaps not embarrassing that the newest tool for the job is from the 1990s (after all, people still teach Functional Programming which is even older). It is perhaps unfortunate that no design (sorry, "emergent" design) is the best practice that has replaced it.

On the other hand, by many quantitative metrics, software is still doing fine, and the whole UML exercise was only a minority pursuit at its peak.

#### On utilities
When I worked on an antivirus application, we used to have a joke in our team that we'd choose which one of us would accept the Apple Design Award for our product. Not that we weren't striving for ADA-quality work; we just knew that Apple would never suffer an anti-virus app to gain that sort of recognition. It's not what they want customers to associate with the platform.

This doesn't just apply to AV: any utility app falls into the same situation. The reason is that utilities exist to make up for shortcomings in the computing experience.

No-one wants to use a utility. You don't wake up full of joy at the prospect of defragmenting your hard drive; you wake up hoping to write a great novel. Or put together that movie from your holiday. Or write a killer iPhone app. You defragmenting your hard drive because it needs doing; because the computer didn't take care of it for you automatically. Writing your novel needs to wait while you twiddle with the internal workings of something called a filesystem.

Similarly, no-one wants to use anti-virus. It's just that no-one wants to use a virus either. The computer lets you down by making it possible to run viruses, just like it lets you down by having a fragmented filesystem. And you have to suffer this let-down by running a utility app.

Any utility app - even a very well-written one - is symptomatic of a let-down in the computing experience. That's why you don't find utilities on the iPhone app store - and the conditions of the Mac app store will limit the availability of utilities there, too. Apple have no interest in making it easy for users to find the let-downs in the computing experience. (Readers with long memories will remember that even some of the built-in utilities on OS X were, for a long time, part of an optional package.)

Hitherto, most security features have been utilities - because the platform security has been a let-down. Anti-virus: let-down. Mail filters: let-down. Encryption: let-down. The way to get your application noticed is not to make a utility to address the let-down: it's to design the let-down out of your application. Create something that lets users do what they want; without the compromises that lead to needing utilities. In other words, design the security into the application experience.

#### On version 12
Reflecting on another WWDC keynote reminded me of this bit in Tron:Legacy, which I've undoubtedly not remembered with 100% accuracy:

  We're charging children and schools so much for this, what's so great about the new version?
  
  Well, there's a 12 in it.

#### On voices that matter
In October I'll be in Philadelphia, PA talking at Voices That Matter: Fall iPhone Developers' Conference. I'm looking forward to meeting some old friends and new faces, and sucking up a little more of that energy and enthusiasm that pervades all of the Apple-focussed developer events I've been to. In comparison with other fields of software engineering, Cocoa and Cocoa Touch development have a sense of community that almost exudes from the very edifices of the conference venues.

But back to the talk. Nay, talks. While Alice was directed by the cake on the other side of the looking glass to "Eat Me", the label on my slice said "bite off more of me than you can chew". Therefore I'll be speaking twice at this event, the first talk on Saturday is on Unit Testing, which I've taken over just now from Dave Dribin. Having seen Dave talk on software engineering practices before (and had lively discussions regarding coupling and cohesion in Cocoa code in the bar afterwards), I'm fully looking forward to adding his content's biological and technological distinctiveness to my own. I'll be covering the why of unit testing in addition to the how and what.

My second talk is on - what else - security and encryption in iOS applications. In this talk I'll be looking at some of the common features of iOS apps that require security consideration, how to tease security requirements out of your customers and product managers and then looking at the operating system support for satisfying those requirements. This gives me a chance to focus more intently on iOS than I was able to in Professional Cocoa Application Security (Wrox Professional Guides), looking at specific features and gotchas of the SDK and the distinctive environment iOS apps find themselves in.

I haven't decided what my schedule for the weekend will be, but there are plenty of presentations I'm interested in watching. So look for me on the conference floor (and, of course, in the bar)!

#### On what makes a "good" comment
I have previously discussed the readability of code:

  The author must decide who will read the code, and how to convey the important information to those readers. The reader must analyse the code in terms of how it satisfies this goal of conveyance, not whether they enjoyed the indentation strategy or dislike dots on principle.
  
  Source code is not software written in a human-readable notation. It’s an essay, written in executable notation.

Now how does that relate to comments? Comments are a feature of programming languages that allow all other text-based languages—executable or otherwise—to be injected into the program. The comment feature has no effect on the computer's interpretation of the software, but wildly varying effects on the reader's interpretation. From APPropriate Behaviour:

  [There are] problems with using source code as your only source of information about the software. It does indeed tell you exactly what the product does. Given a bit of time studying, you can discover how it does it, too. But will the programming language instructions tell you why the software does what it does? Is that weird if statement there to fix a bug reported by a customer? Maybe it's there to workaround a problem in the APIs? Maybe the original developer just couldn't work out a different way to solve the problem.
  
  So good documentation should tell you why the code does what it does, and also let you quickly discover how.

We need to combine these two quotes. Yes, the documentation—comments included—needs to express the why and the how, but different readers will have different needs and will not necessarily want these questions answered at the same level.

Take the usual canonical example of a bad comment, also given in APPropriate Behaviour and used for a very similar discussion:

//add one to i
`i++;`

To practiced developers, this comment is just noise. It says the same thing as the line below it.

The fact is that to novice developers too it says the same thing as the line below it, but they have not yet learned to read the notation fluently. This means that they cannot necessarily readily tell that they say the same thing: therefore the comment adds value.

Where someone familiar with the (programming) language might say that the comment only reiterates what the software does, and therefore adds no value, a neophyte might look at the function name to decide what it does and look to comments like this to help them comprehend how it does it.

Outside of very limited contexts, I would avoid comments like that though. I usually assume that a reader will be about as comfortable with the (computer) language used as I am, and either knows the API functions or (like me) knows where to find documentation on them. I use comments sparingly, to discuss trade-offs being made, information relied on that isn't evident in the code itself or discussions of why what's being done is there, if it might seem odd without explanation.

Have I ever written a good comment?

As examples, here are some real comments I've written on real code, with all the context removed and with reviews added. Of course, as with the rest of the universe "good" and "bad" are subjective, and really represent conformance with the ideas of comment quality described above and in linked articles.

 /*note - answer1.score &lt; answer2.score, but answer1 is accepted so should
 *still be first in the list of answers.
 */

This is bad. You could work this one out with a limited knowledge of the domain, or from the unit tests. This comment adds nothing.

/* NASTY HACK ALERT
 * The UIWebView loads its contents asynchronously. If it's still doing
 * that when the test comes to evaluate its content, the content will seem
 * empty and the test will fail. Any solution to this comes down to "hold
 * the test back for a bit", which I've done explicitly here.
 * http://stackoverflow.com/questions/7255515/why-is-my-uiwebview-empty-in-my-unit-test
 */

This is good. I've explained that the code has a surprising shape, but for a reason I understand, and I've provided a reference that goes into more detail.

    //Knuth Section 6.2.2 algorithm D.

This is good, if a bit too brief. I've cited the reference description (to me, anyway: obviously Knuth got it from somewhere else) of the algorithm. If you want to know why it does what it does, you can go and read the discussion there. If there's a bug you can compare my implementation with Knuth's. Of course Knuth wrote more than one book, so I probably should have specified "The Art of Computer Programming" in this comment.

/**
 * The command bus accepts commands from the application and schedules work
 * to fulfil those commands.
 */

This is not what I mean by a comment. It's API documentation, it happens to be implemented as a comment, but it fills a very particular and better-understood role.

What do other people's comments look like?

Here are some similarly-annotated comments, from a project I happen to have open (GNUstep-base).

/*
 * If we need space allocated to store a return value,
 * make room for it at the end of the callframe so we
 * only need to do a single malloc.
 */

Explains why the programmer wrote it this way, which is a good thing.

  /* The addition of a constant '8' is a fudge applied simply because
   * some return values write beynd the end of the memory if the buffer
   * is sized exactly ... don't know why.
   */

This comment is good in that explains what is otherwise a very weird-looking bit of code. It would be better if the author had found the ultimate cause and documented that, though.

/* This class stores objects inline in data beyond the end of the instance.
 * However, when GC is enabled the object data is typed, and all data after
 * the end of the class is ignored by the garbage collector (which would
 * mean that objects in the array could be collected).
 * We therefore do not provide the class when GC is being used.
 */

This is a good comment, too. There's a reason the implementation can't be used in particular scenarios, here's why a different one is selected instead.

/*
 *  Make sure the array is 'sane' so that it can be deallocated
 *  safely by an autorelease pool if the '[anObject retain]' causes
 *  an exception.
 */

This is a bad comment, in my opinion. Let's leave aside for the moment the important issue of our industry's relationship with mental illness. What exactly does it mean for an array to be 'sane'? I can't tell from this comment. I could look at the code, and find out what is done near this comment. However, I could not decide what there contributes to this particular version of 'sanity': particularly, what if anything could I remove before it was no longer 'sane'? Why is it that this particular version of 'sanity' is required?

What do other people say about comments?

For many people, the go-to (pun intended) guide on coding practice is, or was, Code Complete, 2nd Edition. As with this blog and APPropriate Behaviour, McConnell promotes the view that comments are part of documentation and that documentation is part of programming as a social activity. From the introduction to Chapter 32, Self-Documenting Code:

  Like layout, good documentation is a sign of the professional pride a programmer puts into a program.

He talks, as do some of the authors in 97 Things Every Programmer Should Know, about documenting the design decisions, both at overview and detailed level. That is a specific way to address the "why" question, because while the code shows you what it does it doesn't express the infinitude of things that it does not do. Why does it not do any of them? Good question, someone should answer it.

Section 32.3 is, in a loose way, a Socratic debate on the value of comments. In a sidebar to this is a quote attributed to "B. A. Sheil", from an entry in the bibliography, The Psychological Study of Programming. This is the source that most directly connects the view on comments I've been expressing above and in earlier articles to the wider discourse. The abstract demonstrates that we're in for an interesting read:

  Most innovations in programming languages and methodology are motivated by a belief that they will improve the performance of the programmers who use them. Although such claims are usually advanced informally, there is a growing body of research which attempts to verify them by controlled observation of programmers' behavior. Surprisingly, these studies have found few clear effects of changes in either programming notation or practice. Less surprisingly, the computing community has paid relatively little attention to these results. This paper reviews the psychological research on programming and argues that its ineffectiveness is the result of both unsophisticated experimental technique and a shallow view of the nature of programming skill.

Here is not only the quote selected by McConnell but the rest of its paragraph, which supplies some necessary context. The emphasis is Sheil's.

  Although the evidence for the utility of comments is equivocal, it is unclear what other pattern of results could have been expected. Clearly, at some level comments have to be useful. To believe otherwise would be to believe that the comprehensibility of a program is independent of how much information the reader might already have about it. However, it is equally clear that a comment is only useful if it tells the reader something she either does not already know or cannot infer immediately from the code. Exactly which propositions about a program should be included in the commentary is therefore a matter of matching the comments to the needs of the expected readers. This makes widely applicable results as to the desirable amount and type of commenting so highly unlikely that behavioral experimentation is of questionable value.

So it turns out that at about the time I was being conceived, so was the opinion on comments (and documentation and code readability in general) to which I ascribe: that you should write for your audience, and your audience probably needs to know more than just what the software is up to.

That Sheil reference also contains a cautionary tale about the "value" of comments:

  Weissman found that appropriate comments caused hand simulation to proceed significantly faster, but with significantly more errors.

That's a reference to Laurence Weissman's 1974 PhD Thesis.

#### On what Marcus said
This post is a response to Why so serious? over at Cocoa is my Girlfriend. Read that.

Welcome back. OK, so firstly let's talk about that damned carousel. Kudos to the developer who wrote a nice smoothly scrolling layer-backed image pager, but as Marcus says, that's not the same as doing a nice smoothly scrolling carousel. Believe me, I've taken around one hundred Instruments traces of the carousel. Swirling images around an iPad screen is the least of its concerns.

Now, let's start looking at the state of the community thing. It's like an iceberg, or a duck. Or maybe a duck with the proportions of an iceberg. The point is that what you see is a bunch of developers being flown around the world to talk at conferences, plugging their books (the evil capitalist bastards). What you get is a bunch of people who have put their jobs and careers into the background for a while because they learned something cool and want to share it with the class. The 7/8ths of the duck kicking frantically below the ice is people not getting paid to help everyone else do their job as well as they can.

I can't speak for Marcus's experience, but I can describe my own. That security book? The one that I'm already planning to replace because there'll be so much more stuff to talk about after Monday? The one where I know you read chapter one then put it on the shelf until such time as one of the other chapters describes a problem you have? Around nine months of research, study, and staring blankly at an OpenOffice window. During that time, almost all of my coding was either learning about or preparing samples for the content of the book. I got a warm feeling when I saw it in print, but they don't pay rent.

The same, but in smaller writing, for conference talks (one to two weeks of preparation each) and even blog posts (half to two days of preparation each). That's why I love reading posts from CIMGF, TheoCacao, Mike Ash and others: each new post represents time someone else has taken to make me a better programmer: time they could have billed to a client. By the way I don't know whether this is commonly known, but there's no pay for doing technical talks at iOS developer conferences. The keynote speakers sometimes get paid, the content speakers do not.

Ok, so that's me on my high horse, but we were supposed to be talking about snarking in the community. That happens. My favourite recent example was the one piece of negative feedback I got from a recent conference talk: a page-long missive describing how I'd wasted the person's time by talking about the subject of my talk rather than the topic they wanted to hear about.

Thing is, there's a lesson in there. I could have done a better job at either describing the importance of my subject to that attendee, or getting them to leave the room early on in the talk. Could have, but didn't. Next time, I will. And so that's great, this commenter told me something I didn't know before, something I can use to change the way I work.

But that's not always the case. Sometimes, you look for the lesson and there isn't one. The tweeter just doesn't like you. The best way to get past this is to realise that the exchange has been neutral: you got nothing from their feedback, but in return because they chose to ignore you, you gave them nothing too. Maybe that guy does know the topic better than you. Maybe he's just a blowhard. Either way, you gave nothing, you got nothing: it's not a loss, it's a no-score draw.

But then there are the other times. You know what I mean, the dark times. When your amygdala or whatever weird bit of your brain it is responds before your cortex does (I'm no neuroscientist, and I don't even play one in my armchair), and you get the visceral rage before you get a chance to rationally respond.

There's one common case that still turns me into a big green hulk of fury, even though I should have got over it years ago. It's the times when a commentator or talk attendee decides that my entire argument is broken because that person either disagrees with my choice of terminology, or can think of an edge case where my solution can't be rubber-stamped in.

On the one hand, as software engineers we are used to finding edge cases where the requirements don't quite seem to fit. On the other hand, as software engineers it is our job to solve these problems and edge cases. If you find a situation at work where a particular set of circumstances causes your app to fail, I'm willing to bet that you consider that a bug and try to find a way to fix that app, then you give the bug fix to your users. I doubt you pull the app from the store and smugly proclaim that your users were idiots for thinking it could solve their problems in the first place.

So apply that same thinking to solutions other people are showing you. If you have to drill down to an edge case to find the problem, then what you're saying is not that the solution is wrong, but that it's almost right. Provide not a repudiation but an enhancement, a bug fix if you will. Make the solution better and we've all learned something.

Conclusion

Of course, don't be a dick. Your twitter-wang is not the most important thing in your career, knowledge is. You're a knowledge worker. The person who got up on that stage, or wrote that post or that book, did it because they found something cool and wanted everyone to benefit. They didn't make you pay some percentage of your app revenue to use that knowledge, or withhold the knowledge, or supply it exclusively to your competition. They told you something they thought would help.

If it didn't help, maybe that's because you know something about the topic that they didn't. That's fine, but don't stop at saying that they're wrong. That doesn't help you or them, or anyone else who listened. Understand their position, understand how your knowledge provides a different perspective, then combine the two to make the super-mega-awesome KnowledgeZoid. And now start sharing that.

But don't expect that just because you're not being a dick, everyone else will not be a dick. Just try to avoid taking it personally: which is hard, I certainly can't do it all the time. You took a risk in raising your head above the parapet and trying to get us engineers to change the way we work: the reward for that far outweighs the cost of dealing with detractors.

One more thing

There's another group of developers, of course. Bigger than the sharers, bigger than the detractors. That's the group of developers who silently get on with building great things. Please, if you're in that group, consider heading over to the dev forums or to stack overflow and answering one question. Or adding a paragraph to the cocoadev wiki (or just removing decade-old conversations from the content). We're all eager to learn from you.

#### On whiteboard coding
Another day in which someone lamented to me the demeaning nature of the interview coding challenge. It is indeed embarrassing, when someone with more than two decades of software engineering experience is asked to complete a gotcha-style programming task under the watchful eye of an unhelpful interviewer. It ought to be embarassing for both of them if the modern IDE available to the candidate is a whiteboard, and a selection of coloured markers for syntax highlighting.

But here's the problem: consistently, throughout those decades and longer, recruiting managers who hire programmers have been beset by candidates who can't program. It's such a desirable career path that plenty of people will try to enter, even those who hope to pick up on whatever it is they're supposed to do once they get the job. And, indeed, that can be a good way to learn: what is Pete McBreen's "Software Craftsmanship" other than an imperative for on-the-job learning and mentoring?

Many companies don't have the capacity or ability to bring a keen learner up from scratch, or are hiring into roles where they expect more familiarity with the skill. Thus, the uncomfortable truth: to fix programmer interviews, you first need to fix programmer screening. Demonstrate that all candidates coming through the door are capable programmers at the required level, and hirers no longer need to test their programming skills.

Note: or do they? Maybe someone can program, but uses techniques that the rest of the team consider to be unnatural. Or they work best solo/paired/in a mob, and the team works best paired/in a mob/solo. Still, let's roll with it: remove the need for a test in the interview by only interviewing candidates who would pass the test.

The problem is that every approach to screening for programming comes with its own downsides.

The economic approach, as currently practised: keep people away by making the career less desirable, by laying off hundreds of thousands of practitioners. The problem here is plunging many people into financial uncertainty, and reducing the psychological safety of anyone who does remain.

Moving the problem upstream: sending out pre-interview coding challenges. This suffers many of the same problems as live coding, except that the candidate doesn't have to meet the dull gaze of a bored interviewer, and the interviewer doesn't know it was actually the candidate who completed the challenge. I suppose they could require the candidate to sign their submission, then share their key fingerprint in the interview. An additional problem is that the candidate needs time outside of the interview to complete the challenge, which can be difficult. Not as a difficult as finding the time to:

Maintain a public portfolio. This biases towards people with plenty of spare time, or who get to publish their day-job work, or at least don't have an agreement with their day-job employer that they don't work on outside projects.

Our last possibility is the more extreme: de-emphasize the importance of the programming skill, so that the reason employers don't need to screen for it is that it's less essential as a hiring criterion. This was tried before, with 1990s-style software engineering and particularly Computer-Aided Software Engineering (CASE). It didn't get very far that time, but could do on a second go around.

#### On Windows 8
Right from the beginning, you have to accept that this analysis is based on the presentation of Windows 8  shown at the //build/windows conference. I've watched the presentation, I'm downloading the developer preview but I'm over an hour away from even discovering whether I have anything I can install that on.

My biggest concern

The thing I was most worried that Windows 8 would bring was yet another chance for Microsoft to flub their developer experience. If you think about it, Microsoft's future now depends more than it has on any time since the 1980s on their third-party developers. Whereas they used to be able to rely on OEM licensing deals (and strongarming) to ensure that people wanted to use their stuff, that's starting to wane.

Sure, the main desktop Windows-NT-plus-Active-Directory-plus-Office-plus-Exchange core is still going strong. But look at the web stack: servers are IIS or Apache or one of various other servers that support Rails, PHP and so on. They're hosted on my Wintel boxen or my Lintel boxen or one of various third-party choices (including Microsoft's own Azure). The client is a browser (based on anything) or a native mobile app on Windows Phone 7 or one of a handful of other native app platforms, including iOS and Android.

Where Microsoft have the potential to win over all of their competitors is providing a consistent developer experience across the whole of their stack. This isn't about locking developers in, this is about making it easiest for them to do Microsoft end-to-end.

Consider Apple for a moment. They have a very consistent experience for writing desktop apps and mobile apps, with Cocoa and Cocoa Touch. Do you want to write server software? You can do it on Apple kit, but not with Apple tools. You'll probably use Rails or another open source UNIX-based stack, at which point you may as well deploy it to someone else's kit, maybe someone who offers rackmount hardware. You can use iCloud on Apple's terms for distributed storage, but again if you want generic cloud computing or a different approach to cloud storage, you're on your own.

On the other hand, Microsoft offers a complete stack. Azure, Windows Server, WPF (desktop client) and SilverLight (browser and phone) all have the same or very similar APIs in the same languages, so it's easy to migrate between and share code across multiple layers in a complete computing environment.

But, would they do that? The one place where Microsoft itself seems to have failed to recognise its own strength is in the Windows division. They "pulled a Solaris", and have consistently ignored .NET ever since its inception, in just the same way that the Solaris Operating Environment team in Sun ignored Java. That leads developers to think there must be something wrong with the tech if real developers inside the company who made it don't want to use it.

So when Microsoft (or rather, Steve Ballmer) announced that the Windows 8 SDK would be JavaScript and HTML5, it made me think that the Windows 8 team might have jumped the shark. That stuff isn't compatible with C# and XAML, or with the .NET APIs. I thought they'd got themselves into the position where their phone is SilverLight, their desktop/tablet is HTML5 (except legacy stuff which is Win32), their server is .NET… ugh.

So what happened?

Well, in a nutshell, they didn't do that. The need for a new runtime makes sense because the .NET runtime was only ever a bolt-on on top of Windows, leading to interesting compatibility issues (to run App version x you need .NET version y and Windows version z; but OtherApp version a wants .NET version b). But the fact that the new HTML5 stuff is actually layered on top of the same runtime, and provides access to the same APIs as are available in C#, VB and C++ makes me believe that Microsoft may actually understand that their success depends on developers adopting their entire stack.

I'll make this point clear now: HTML5 is not the future of app development. Its strength is based on a couple of things: every device needs a high-performance HTML engine anyway in order to provide web browsing; and HTML5 should work across multiple devices. Well, as a third-party developer, the first of those points is not my problem, it's Microsoft's problem, or Apple's, or RIM's. I don't care that they get to re-use the same HTML renderer in multiple contexts.

The second of those points will quickly disintegrate when every platform vendor is trying to differentiate itself and provide an HTML5 development environment. Did you see that Expression Blend for HTML demo? That's a very cool tool, but the demo of using "standard HTML5" relied on a keyword called "ms-grid". Wait, MS? Is that cross-platform HTML? Also observe that the JS needs to call into the WinRT API, so this really isn't a cross-platform solution.

No, for me the big story is that you can use C# everywhere: currently with .NET in most places and WinRT on the desktop. Any client application can use XAML for its user interface. That's big for ecosystem-wide development. There's one thing I need to learn in order to do mobile, tablet, desktop, server and cloud programming. It's slightly weird that the namespaces on .NET and WinRT are different, and it would be a good strategic move to support the new namespaces in newer versions of SilverLight (i.e. in Windows Phone 8).

What else happened?

I'm not going to talk much about Metro, it's a well-designed UI that works well. I'm not sure yet how it works if you're interacting through a mouse or trackpad. I spoke about it before when discussing Windows Phone 7; where I also expressed my belief that sharing the developer experience across desktop and phone was a good move.

What I will point out is that the Windows team no longer think that they can do what they want and people will buy it. They're noticing that the competition exists, and that they need to do better than the competition. In this case, "the competition" is Apple, but isn't Google.

Why do I think that? Sinofsky felt comfortable making a joke about "Chrome-free browser experience", but the iPad was the elephant in the room. Tablets/slates/whatever were summarised as "other platforms", although you feel that if Microsoft had a point to score over Android they would have mentioned it specifically.

Conclusions

This means that - perhaps with the exception of Office - Microsoft's complacency is officially behind it. Sure, Windows 7 and Windows Phone 7 were indications that Microsoft noticed people talking about them falling behind. But now they've started to show a strategy that indicates they intend to innovate their way back to the front.

While it's currently developer preview demoware, Windows 8 looks fast. It also looks different. They've chosen to respond to Apple's touchscreen mobile devices by doing touchscreen everywhere, and to eschew skeuomorphism in favour of a very abstract user interface. Importantly, the Windows APIs are now the same as they're getting 3rd-party developers to use, and are the same as (or very similar to) the APIs on the rest of their platforms.

#### On working machines
In part one, on thinking machines, I explored two facets of the philosophy of artificial intelligence: “intelligence”, and consciousness. That left an important topic to consider for this post: the impact of artificial intelligence on work.

No technology has ever “stolen a job”. Not once. Technology automates and enables tasks. Some of these tasks were never part of “the market”, and other tasks were. If your job is defined by performing the same task over and over, be it knocking the base of a saggar, driving a vehicle, or typing JavaScript into somebody else's computer, and that task can be automated, then there's a chance that your employers won't need you to do that task any more. But whether they keep you, redeploy you, retrain you to do something else, or let you go, is their choice: it's the employers that stole your job.

Let's imagine a hypothetical scenario where a company has ten JavaScript shovelers, and each outputs an average of one bushel of JS per day. Now some technological intervention—could be AI, sure, but it could be a syntax-highlighting text editor, TypeScript, or some other tool—makes each JS shoveler ten times more efficient (aside: it doesn't). The employer's choices (note: not the technology's choices, not the inventor's choices; the employer's choices) might be represented in a diagram like this:

That last option is courtesy of Jevons' paradox, which says that when a resource becomes more efficient to use, demand goes up. If a new technology makes knowledge-deployment more efficient, then demand for knowledge work increases, it doesn't decrease. The employers who don't increase their knowledge-working capacity when knowledge work becomes more efficient are, to paraphrase William Stanley Jevons, idiots.

The “AI is stealing our jobs” meme comes from a lack of understanding that software engineers are workers, not employers, and that the economic principles of employment and work apply to them the same as to other workers. Bringing in another paradox of economics, Robert Solow noted that “you can see the computer age everywhere but in the productivity statistics”. It took a long time for computers to start automating knowledge work: first record tabulation, then payroll and inventory management, then the typing pool and typesetting, then so on and so on through technical drafting and taxi dispatching.

Through the slow burn of the computer age, software engineers got comfortable with being the people who automate other people's work. Throughout that period, demand for (the task of) computer programming rose. Now, two (mostly unrelated) things have happened: the first is that a new technology has promised to automate computer programming, placing us at the start of the next Solow age; and headcount among people who repetitively do computer-programming tasks has been decreasing.

That means that the computer people are on the receiving end of capitalism for the first time since the dot-com crash, and they don't like it. We automate other people's work, it's unfair to automate our work! This is another view through the same economic lens that gives us enshittification: wait, we worked hard to turn this manual task into an automated platform, you owners can't seriously expect to capture additional value from this platform?! We're supposed to continue to benefit from the lower costs we enabled for you!?

Those of us who do computering for a salary, wage, or day rate have always been on the receiving end of the exploitative nature of the wage relationship, unfortunately the relatively high salaries and enjoyable tasks stopped many of us from engaging with that seriously. We're now in a position of huge uncertainty for many employees in the field, and the short-term solution to that is the same solution it's always been, that's demonstrated to work in many European economies: collective bargaining on behalf of the sector.

But becoming conscious to the benefits of increasing bargaining power through group organisation is insufficient to end the fundamentally exploitative relationship, and to stop the next round of automation, layoffs, and changes to employment conditions. So is any idea that employment will automatically disappear completely, or ebb away, in some Keynesian decline to a 15-hour working week. As we automate some tasks, we introduce new tasks, and new jobs that exploit people to get those tasks done; whether or not you think of them as bullshit jobs.

#### On writing a book
Well, I've performed my final author's review, and Professional Cocoa Application Security is all with the printers. This post is about my experiences writing the book, not the book material itself.My original motivation for writing PCAS was that it was a topic someone needed to talk about, and nobody had hitherto done the talking. I was actually initially approached by Wiley to see if I'd write anything at all - their commissioning editor had seen my Objective-C FAQ and this blog, and liked my style. I said that I didn't want to write a book, I wanted to write this book. It was the best way I could pay back the community that has helped me so much in the years I've been a Mac developer.It took about 6 months to write the draft - my original estimate was much shorter but once we realised I couldn't meet that we revised it, after which I stayed on track with the updated schedule. That's six months of nearly full time work. Some other books probably don't take so long, but in this case I had set myself a very ambitious scope and needed to research quite a lot of the topics before I could write on them. If you've already got a series of blog posts, training material or something that you just want to turn into a book, I can imagine the drafting process being much quicker.I've found that book authorship is not the best vehicle for self-study. You get a biased view of the material, looking for things that would be interesting to readers rather than things you will need to use yourself. Because the goal of the book is to provide utility to the readers, you end up with a gotcha-oriented approach to research, looking for the subtle benefits or issues that are not obvious on a casual inspection. That said, it was still a good motivation to learn about the technologies I wrote about, so I'm glad I did it. Parts of the writing process were a lot of fun: I got to find out about some cool frameworks and APIs, absorb loads of information and re-emit it in a form that is, I hope, engaging and interesting. I've looked at my bookshelves and have about 6ft of books that I used as source material - and that doesn't include websites, ebooks and journal papers. On the other hand, I'm not going to deny that I had occasional days that just felt like a long slog to get the day's section written. I didn't mind solving hard problems, but there are some subjects that just seem impossible to say anything interesting about. It's when writing those sections that you find yourself staring at a half-written sentence for an hour, wondering just what it was you were thinking when you wrote the ToC.You're not going to get rich off the advance :). I was in a good position where I could live off practically no income while writing, meaning that devoting a few months to producing the drafts was not a problem. What I have become rich in is exposure and recognition, even before the book was published. Because both the proposal and the book content must be peer-reviewed by a technical reviewer, "I am writing a book" says "there are people out there who trust that I know my subject". Of course, "I have written a book and you can read it" carries more weight, so I expect this exposure to increase after publication.I've worked on reviewing proposals too, and the things you really need to make sure if you are trying to punt a proposal to publishers are:

You need to tell the publishers that the market for your book exists, who is in that market and how big it is. They're not going to go and look for the buyers on your behalf (but they will get a reviewer to make sure you're not talking bullshit).Having identified your reader, the goals and content of the book must be appropriate to the reader. Don't put an introduction to Xcode in your Advanced iPad Apps book, just to make up an example.

This theme carries on into the review process for the actual content. The technical reviewer (a role I've also taken before) is not just there to check that the code compiles. Responsibilities include verifying the accuracy of the content and appropriateness for the target reader, and indeed review comments I've made on book drafts have been split roughly evenly between "this isn't quite right" and "your reader won't understand this" (though I'm more verbose in the actual review).So, in short, you will not make money writing a book. You will gain kudos and satisfaction. If you've got something that you think the world desperately needs to know, and you know that you can explain it in a way the world will want to pay attention to, then by all means write! If you want to make a few thousand dollars, or want an easy project between apps, then I'd suggest finding something else. Writing's fun, and it's worthwhile, but it's certainly not an easy life.

#### On XP mode
This is a reply to @gcluley, who linked to this ZDNet story (which in turn took its quotes from Sophos Podcasts).The second most crazy thing about the entire "XP mode" issue in Windows 7 is that the feature is entirely unnecessary. Corporate customers of Windows are already, for the most part, comfortable with managing virtual Windows desktops through third-party products with much better management options or at least have trialled such products. Home users of Windows just take whichever version is pre-installed when they buy the PC and if it means buying new versions of some apps, that's what they do. They're used to it. The group of people who could benefit from XP mode - people with a strong need for app compatibility with XP but with no experience of virtualisation - just doesn't exist.The very existence of the XP mode feature is a microcosmic example of the way Ballmer has been running Microsoft - if there's a market out there that MS isn't in, MS needs to be in it pronto. Bing, Morro, Web-Office, Zune and now virtualisation are all testament to the inability of Microsoft to concentrate on what it does. What Microsoft really does is to sell two things; an enterprise computing environment and an OEM software distribution. Forget that Windows and Office are accounted as two separate products; MS sell Windows+Office to businesses and Windows to computer makers.Now the interesting question to ponder is which of Microsoft's (real or perceived; remember they aren't necessarily in this market) competitors the "XP mode" feature is a response to. My interpretation is that it's not actually VMware and its ilk at all - Microsoft is once again responding to nonexistent competition from Apple. Boot Camp and the third-party desktop virtualisation offerings on the Mac (including, without hint of irony, VMware) let users use OS X as their shiny new OS with an "XP mode" of sorts for legacy applications. I think what Microsoft are trying to do here is to show that Windows can be the new shiny with XP as the legacy mode, and are therefore positioning XP mode as a counter to the fictitious competition from Apple. Oh, and if you don't believe me when I say that the competition from Apple doesn't exist - Apple sell all of the premium computers while Microsoft take the aforementioned corporate and OEM markets.OK, so if that was the second most crazy thing about XP mode, what is the most crazy thing about XP mode? It's also that the feature shouldn't exist. Windows has always had a problem with segregating distinct services which other operating systems don't suffer from. While Microsoft's avoidance of this issue has allowed a whole new software industry to spring up around it, the fact that they need to start a second copy of Windows just to get some applications running in Windows 7 doesn't give me much hope for the future.

#### On, or rather in, Seattle
I've never been to Washington before, so I'm looking forward to Voices That Matter: iPhone Developers Conference in April. Of course, you know I like the sound of my own voice enough to be speaking: my talk this year will be about Test-Driven Development of iOS apps. I heard from attendees in Philly that they wanted my unit testing talk to be more meat and less waffle, and that's the talk I'll be giving this time.

I also like the sound of other people's voices, and the schedule this time looks spot on: fellow sideburn-wielders Mike Lee and Andy Ihnatko are talking, as is Aaron Hillegass: all worth listening to. I'm a bit nervous about talking to an empty room as I'm scheduled up against Cat Shive who gives a good talk too.

If you're interested in attending, now is the time to register. Early bird pricing ends Friday, but whether you sign up now or next week you can use promo code  SEASPK2 to get an extra $100 off. Right now, that means attendance is only $395.

By the way, I'm sounding out things to do while I'm in Seattle, so any locals who have expert knowledge on where to eat and what to see please feel welcome to comment. I'm definitely up for visiting 1, Microsoft Way in nearby Redmond. If you work for Microsoft, offer to run a tour ;-).

#### One decade in
The first working week of August 2014 comes ten years after the first working week of August 2004. You knew that. The first working week of August 2004 was the first week since completing my degree that I worked for a living: the start of a sequence of (paid) events that led me to here.

Obviously it's not the start of the sequence at all, but I've already covered that story. It's not even when I first learned Objective-C: that was about a year earlier. However, stories are easier to tell if they begin once upon a time, rather than in the middle of a collection of events, the connections between which being subtle and hard to examine.

It would be nice to give a recommendation to people who are in the position now that I was ten years ago, but it's unlikely that the same things that worked back in 2004 are still applicable. Should you want to try, then my suggestion is this: bet your whole career on some apparently minuscule niche, and hope against hope that the only vendor supporting it creates a whole new industry within about four years so that your seemingly poor decision cashes out.

  As an aside, you can draw clear lines around the things I was using back then that I'm still using now. Some of the lines are fuzzy: I'm still using "UNIX", though that doesn't mean the same thing (nor did it mean then anything that would've been recognisable to a user from 1994, 1984 or 1974).

It would perhaps be less nice to give a list of lessons that I claim to have learned over those ten years. Those would, of course, be lessons that I derive now from my recollection of that time, and would mostly serve to add to the corpus of folklore that permeates our field.

Which brings me on to the one thing I unequivocally do know after ten years in [IT, computers, whatever you want to call it]: that I still don't know a lot. I definitely know more about programming computers than I did then, but that's only an infinitesimal part of the fundamental interconnectedness of all things.

#### One meeellleeon
A teacher recently asked her computing class if there was any question they would like to ask me. One of the students came up with a question: how could they make a million pounds?

I think my answer would be one of these:

Facebook has order of a billion users and is worth order of 100 billion pounds. Network value scales as the square of the number of users, so to merely make a million pounds you could build a network with just three and a half million users.
A lowest-tier iOS app nets its developer roughly 40p per sale. To make a million pounds you need simply build a cheap app, then attract two and a half million sales.

If the app were a value-add for the network, you could easily make more than two million pounds.

[In fact I'm pretty sure I've already make a million pounds, it's just that the costs worked out to about a million pounds.]

#### One person per task
One of the least teamy things I see with software teams is limiting the maximum and minimum number of items of work in process - tasks, stories, whatever you call them - both to the number of developers on the team. For some reason it's always the number of devs, never the number of product owners, customers, QAs, or deployment people. Got four devs? Then there should be four tasks in process!

This approach is surprisingly backward given that we're all supposed to have come so far as the leaders of the Agile Fourth Industrial Revolution 2.0 that we've internalised and transcended Goldratt's theory of constraints. It's the last holdout of the old Taylorian school of management. Everybody is working full-tilt, so if anyone runs into any trouble then everybody else is too busy to help them. If what they're doing is upstream of anybody else's work, then they are going to be blocked too, but rather than fix the blockage they'll pull another task because one-person-per-task at all times!

So much is this at the core of software team thinking that when I've suggested in informal discussions that maybe we should do something else, people are confused. Are you saying that we should have a developer who isn't assigned to a task, just in case? What does that person do the rest of the time, play Minesweeper? As if the only alternative to "one person per task" is "one person per task but perhaps there is another person".

One person per task has the "nobody can help" disadvantage already mentioned. In fact, people are disincentivised from helping, because their task has their name on it and your task has your name on it. Did issue #1348 miss the release train? Bob is such a drag on the team, at least Karen managed to ace her ticket. Maybe we should reevaluate who leads on the next project.

You'll see other effects of one person per task. Code reviews fall into one of two categories: "LGTM" and "axe to grind". Only the people who are really invested in making sure that nobody ever misses off a const keyword, or uses function() where => would suffice, will take the time to commit to code reviews. Everybody else will skim-read, look at the CI output to see if the tests pass, and get back to their own task with their own name on it as quickly as possible. This loses both the review benefit of code review, and the shared-understanding-of-the-code benefit too. Everyone only really understands the features they worked on individually, there just happens to be a big ball of those features in one repo.

Code quality suffers. Each individual is too busy chopping down trees to sharpen the saw, because there's always a next task for each developer to do.

Everybody else has been under-resourced. We need one developer per task, because what I can see is features in a UI and the only people shovelling features are the devs. QA is a cost centre, so if we can get one QA (or at most, one per team) then let's do that. Same with ops. Infosec, coaching, UX, and other nice-to-haves can be consultants as needed. Weird how our devs are ticking off tasks like billy-O, and nothing's getting through to release!

The alternative to "one person per task" is not "one person per task and some change". It's "one objective per team". Set the goal, and let people work out what to do about it and how everyone contributes. As they used to say, "give them the environment and support they need, and trust them to get the job done".

#### One reason multiple inheritance sucks
So I'm reviewing a book, and it happens to cover the way method lookups are performed in a particular language's object model.  I'm not going to say what the book is because it's not relevant, and nothing discussed here has anything to do with that publication.  But reading it made me realise just why even when the language allows it, my internal pain aversion system will always skip multiple inheritance as an option.  Consider this Python code (with built-in python shell promptness):

&gt;&gt;&gt; class One(object):...     def doStuff(self):...         print "One.doStuff()"... &gt;&gt;&gt; class Two(One):...     pass... &gt;&gt;&gt; class Three(One):...     def doStuff(self,x):...         print "Three.doStuff(%d)" % x... &gt;&gt;&gt; class Four(Two,Three):...     pass... &gt;&gt;&gt; class Five(Three,Two):...     pass... &gt;&gt;&gt; a=Four()&gt;&gt;&gt; b=Five()

so, what is the method signature of a.doStuff() and b.doStuff()?  Well, it depends entirely on the order in which the class hierarchy gets inspected, Python does things by leftright-bottomtop so both a and b inherit Three.doStuff(self,x).  In the case of a, Four doesn't have doStuff() so it looks at Two, which doesn't so it looks at Three and finds it.  However, Python also has another lookup mechanism (which was the only one available pre-2.3 or possibly 2.4, and can still be triggered by omitting the "object" superclass from One) which is left-bottomtop-right, so a inherits One's doStuff() and b inherits Three's.  In the words of Peter Cook, that could confuse a stupid person...

So luckily Objective-C avoids this problem by only having an inheritance tree, and we can't add method implementations via protocols either.  And my specific example is moot, because -doStuff and -doStuff: are different selectors.

#### One Window that is good for Mac security
I realise now that I didn't cover this when it happened back at the beginning of March, but that not everyone in either the Apple world nor the general infosec community is aware of it. Nearly one month ago, Apple hired a new Security Product Manager (the position was vacant at the time of WWDC 2008 and I think it was just being covered by another product manager in the interim): welcome Window Snyder.

Window has a good history in the infosec world; after working as security design architect at @stake, she moved to Microsoft to act as security sign-off for XP Service Pack 2 (Microsoft's first OS release focussed solely on security improvements) and Windows Server 2003 (their first completely new OS release after the security push of 2002). It was during her watch that Microsoft became more open about their vulnerability reporting, and introduced "Patch Tuesday" to help systems administrators manage the patch lifecycle. I happen not to like the Patch Tuesday mentality, but at least Microsoft thought about the issue and reacted to it.

After Microsoft, Window became Chief Security Something-or-Other at Mozilla. Here she promoted measurement and tracking of security issues, process improvements and greater transparency, both in terms of Mozilla's reporting and that of other vendors.

I think that, given the authority to make process and reporting changes regarding Apple's security procedures, she will be a great addition to Apple's security teams. Apple typically drop security updates without warning and with minimal information on the content and severity of the vulnerabilities addressed; they maintain what could be charitably described as an "arm's length" relationship with security vendors and have a history of slow reaction to vulnerabilities discovered in open source components. I have great hope for those facets of Apple's security work changing soon.

#### OOP as an organic approach to computing
I'm reading How Not to Network a Nation, which talks a lot about cybernetics. Not merely cybernetics as the theory of control in complex systems (cybernetics shares a root with "governor", fans of the etymological fallacy!) but cybernetics as the intersectional discipline matching organisational and management theory with computer science, anthropology, and biology. The study of systems in animals, people and machinery and their (self- or externally-directed) control.

We still use a lot of the ideas from even early cybernetics thought now, such as Claude Shannon's theories on entropy and information, J.C.R. Licklider's ARPAnet, von Neumann's computer architecture, artificial neural networks. But even though the proponents aren't often associated with the field, I think it's reasonable to argue that object-oriented programming is a cybernetically-derived systems approach.

A lot of cybernetics theory is about the components of a system and the messages they pass between each other to achieve control and feedback, and in OOP Alan Kay was seeking to model a software system as a network of messages flowing between independent computer program components. He made the analogy with living organisms clear:

&gt; I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages (so messaging came at the very beginning -- it took a while to see how to do messaging in a programming language efficiently enough to be useful).

More advanced object oriented systems such as Erlang even display autopoesis, automatically spawning new "cells" when old ones are damaged.

There plenty that the intersectional nature of cybernetics still has to inform me about my work. Information theory helps me to understand the utility of a machine learning algorithm. Game theory and biological cooperation and cheating models help describe how a crypto currency is resilient against Byzantine generals.

And now I understand that the biological systems analogy should help me with software analysis and design too.

#### OOP the Easy Way
It's still very much a work in progress, but OOP the Easy Way is now available to purchase from Leanpub (a free sample is also available from the book's Leanpub page). Following the theme of my conference talks and blog posts over the last few years, OOP the Easy Way starts with an Antithesis, examining the accidental complexity that has been accumulating under the banner of Object-Oriented Programming for nearly four decades. This will be followed by a Thesis, constructing a model of software objects from the essential core, then a Synthesis, investigating problems that remain unsolved and directions that remain unexplored.

At this early stage, your feedback on the book is very much welcome and will help yourself and fellow readers to get more from the book. You will automatically get updates for free as they are published through Leanpub.

I hope you enjoy OOP the Easy Way!

#### OOP the Easy Way: now 100% complete
Hello readers, part 3, the final part of the "OOP the Easy Way" journey, has now been published at Leanpub! Thanks for joining me along the way! As ever, corrections, questions, and comments are welcome (you can comment here if you like), and as ever, readers who buy the book now will receive free updates for the lifetime of the book. While there's nothing new to add, this means that corrections and expansions will be free to all readers.

If you enjoy OOP the Easy Way or found it informative (or maybe even both), please recommend it to your friends, colleagues and followers. It'd be great if they could enjoy it, be informed by it, or both, too!

#### Oops. (updated twice)
Q: What caused this?

A: this. A Vodafone employee used the corporate Twitter account to post the message:
[@VodafoneUK] is fed up of dirty homo's and is going after beaver

And as the Vodafone apology attests, this was no hacking attack, instead a case of TGI Friday on the part of an employee. This goes to show that you don't need an external attacker to ruin your corporate image if you hire the right staff.

Update: according to an article in the Register, the problem tweet was caused by an employee on a different team in the same office misusing an unlocked terminal with access to Vodafone's Twitter account. They have fired the employee, but no word on whether they're reviewing their security practices. I'm reminded of the solution taken to combat safe-cracking at LANL when Richard Feynman showed how easy it was to open the safes with their confidential contents: don't let Richard Feynman near the safes.

Update again: Vodafone's official reply:
On Friday afternoon an employee posted an obscene message from the
official Vodafone UK Twitter profile. The employee was suspended
immediately and we have started an internal investigation. This was not
a hack and we apologise for any offence the tweet may have caused.
This sounds like there's the potential for their practices to be altered as a result of their "internal investigation", hopefully they'll make more information available. It would definitely make an interesting case study on responding to real-world security issues.

#### Open Source and the Lehrer-von Braun defence
Tom Lehrer's song about Wernher von Braun is of a man who should not be described as hypocritical:

  Say rather that he's apolitical. "Once the rockets go up, who cares where they come down? That's not my department," says Wernher von Braun.

The idea that programming as a field has no clear ethical direction is not news. As Martin Fowler says here, some programmers seem to believe that they are mere code monkeys. We build things, it's up to other people to choose how they get used, right?

It's in open source software that this line of thinking is clearest. Of course anyone can use commercial software, but it becomes awkward to have blood money on your company's records. Those defence companies and minerals miners just lead people to ask questions, and it'd be better if they didn't. You could just choose not to sell to those people, but that reduces the impact of your product.

A solution presents itself: don't take their money! Rather, decouple the sending up of the thing and the choice of where it comes down, by making it available to people who now don't have any (obvious or traceable, anyway) connection to you or your employer. That's not your department! Instead of selling it, stick it up on a website (preferably someone else's, like GitHub) and give a blanket licence to everyone to use the software for any purpose. You just built a sweet library for interfacing with gyroscopic stabilisers, is it really your fault that someone built a cruise missile that uses the library?

"But wait," you say, "this doesn't sound like the clear-cut victory you make it out to be. In avoiding the social difficulties attendant in selling my software to so-called evildoers, I've also removed the possibility to sell it to gooddoers. Doesn't that mean no money?" No, as Andrew Binstock notes, you can still sell the software.

Anyway, perhaps it'd be useful to restructure the economics of the software industry such that open source was seen as a value-driver, so you can both have your open source cake and eat the cake derived from valuable monetary income. You might do that by organising things such that an open source portfolio were seen as a necessary input to getting hired, for example. So while plenty of people still don't get paid for open source software, they still indirectly benefit from it monetarily.

We can, evidently, easily spin contributions to open source such that they are to our own benefit. What about everybody else? When a government uses Linux computers to spy on the entire world, or an armed force powers its weapons with free software, is that pro bono publico?

The usual response would be the Lehrer-von Braun defence detailed above. "We just built it in good faith, it's up to others to choose how they use it." An attempt to withdraw from ethical evaluation is itself an ethical stance: it's saying that decisions over whether the things you make are good or evil are above (or beneath) your pay-grade. That we, as developers, are OK with the idea that we get large paychecks to live in comfortable countries and solve mental problems, and that the impact of those solutions is for somebody else to deal with. That despite being at the epicentre of one of the world's biggest social and economic changes, we don't care what happens to society or to economy as a result of our doings.

Attempts have been made to produce "socially aware" software, but these have so far not been unqualified successes. The JSON licence includes the following clause:

  The Software shall be used for Good, not Evil.

Interestingly, in one analysis I discovered, the first complaint about this clause is that it interferes with the Free Software goal of copyleft. How ethical do we think an industry is that values self-serving details over the impact of its work on society?

The other problem raised in relation to the JSON licence is that it doesn't explain what good or evil are, nor who is allowed to decide what good or evil are. Broad agreement is unlikely, so this is like the career advisor who tells you to "follow your dreams" without separating out the ones where you're a successful human rights lawyer and the ones where you're being chased by a giant spider with a tentacle face through an ever-changing landscape of horror.

I should probably stop reading H. P. Lovecraft at bedtime.

#### Open Source: because I got mine, so fuck you
The Free Software movement has at its core the idea that people have the freedom to use, study, share, and improve the software on their computers. The modern developer "ecosystem" has co-opted this to create a two-tier society: a developer has the freedom to use, study, share, and improve the tools and libraries that developer puts to use in creating software that must be accepted as-is, and used only in the ways permitted in the Terms of Service and End User Licence Agreement. I've got mine, so fuck you.

Your web application is probably split, broadly speaking, into a front-end bit that runs in the browser and a back-end bit that runs on somebody else's computer.

The back-end bit is on somebody else's computer, so you chose not to adopt the Affero GPL and don't need to spill any precious freedom on the consumers of your service. The fact that you're using Node.js (non-copyleft Free Software), a billion node modules (all most likely non-copyleft Free Software), losing your customer's data in MongoDB (copyleft Free Software), and deploying on GNU/Linux (copyleft Free Software) with Docker (non-copyleft Free Software) and Kubernetes (non-copyleft Free Software) got you what you wanted quickly and cheaply, but there's no need to permit anyone else access to those freedoms.

The front-end bit is on your customers' computers, so you definitely don't want to accidentally spill any precious freedom there! Even though you used a gallon of polyfills and shims (non-copyleft Free Software) and Angular (non-copyleft Free Software) to get what you wanted quickly and cheaply, there's no need to permit anyone else access to those freedoms.

And of course you edited all that JavaScript with VSCode (non-copyleft Free Software).

The fact that the browser used to run that JavaScript might itself be Free Software is immaterial. The freedom to not have any freedom is not freedom.

That's true in the mobile world too. Your free software compilers and runtimes and libraries all go to build opaque blobs that must be run as-is on somebody else's phone, whether or not that phone has a kernel that's Free Software. We got the bits to build our platforms and our apps quickly and cheaply, thanks to Free Software. We've got ours, fuck you.

But, you argue, this is all immaterial. People who aren't developers, well, they aren't developers, they can't change software, why does any of this matter? Because, as all the crime dramas attest, people need motive, MO, and opportunity. Remove the freedom and the opportunity is removed: maybe someone would go out and learn a bit of programming if they had a problem they wanted to solve and the opportunity to solve it. Or maybe they'd go out and find a gigging coder, or a student who wanted a side project, or one of us pros who needs to keep their activity chart green in order to stay employable. Maybe we would benefit more from our jobs as people who change software, if there were more opportunities to change software.

The current division of software freedom into the haves and the have nots is arbitrary, artificial, and unnecessary.

#### OPENSTEP/Parallels/VirtualPC/Spam
I've been trying to reply to bbum's post, firstly my comments were removed as spam, then he approved them by hand, now they're gone again.  So here they are again.

I wrote [in this blog] much about my experiences getting OPENSTEP/Parallels working.  Networking's fine but slow, graphics is good, mouse is a bit jumpy.  I was talking at WWDC with people who say that Microsoft VPC no longer supports OPENSTEP/Mach.  You definitely can't boot native on a new Intel Mac, as there's no SATA driver (and no way to use a boot floppy).

BTW, the floppy images are available over HTTP from Apple: OPENSTEP floppyimages

#### Opinionated Read: How AI Impacts Skill Formation
The abstract to this preprint, by two authors both associated with Anthropic, makes the claim “We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation – particularly in safety-critical domains.”

The first thing to appreciate is that this idea of “safety-critical domains” does a lot of heavy lifting when it comes to software professionalism—on the one hand, engineers say that while (intervention that is perhaps validated in controlled trials or in experience reports but not the way that engineers like to work) is clearly something that those safety-critical folks should concern themselves with, it's not relevant to (domain that includes the engineers' work). On the other hand, professional organisations in the field of computing refuse to engage with the idea that a software engineer should be expected to learn the software engineering body of knowledge precisely because it doesn't have anything to say about how to build safety-critical software.

Now what is safety-critical software? Or, if you can't answer that, what software isn't safety critical? The book Moral AI tells the story of a self-driving car that collided with, and killed, a pedestrian who was pushing a bicycle across the road. The driver (for lack of an agreed and accurate word) reports streaming an episode of The Voice in the background while checking in to her employer's work chat channels on the car's central console. Is the car's autonomous driving system safety-critical in this context? What about the built-in collision avoidance system, that the employer had disabled in this vehicle? How about the streaming software, or the chat application, or the operating systems on which they run? All of these contributed to a fatality, what makes any of them safety-critical or not?

The second thing is that the claim in the abstract is about learning outcomes, skill formation, and efficiency gains. We need to go into reading this paper keeping those terms in mind, and asking ourselves whether this is actually what the authors discuss. Because we care about what they did and what they found, and aren't so worried about the academic context in which they want to present this work, let's skip straight to section 4, the method.

What did they do?

Unfortunately, we don't learn a lot about the method from their method section, certainly not enough to reproduce their results. They tell us that they use "an online interview platform with an AI chat interface", but not which one. The UI of that platform might be an important factor in people's cognition of the code (does it offer syntax highlighting, for example? Does it offer a REPL, or a debugger? Can it run tests?) or their use of AI (does it make in-place code edits?).

In fact when we read on we find that in a pilot study they found that such a platform (they call it P1) was unsuitable and that they switched to another, P2. Choosing a deliberately uncharitable reading, P1 is probably Anthropic's regular platform for interviewing engineering candidates, and management didn't want their employees saying it has problems because Silicon Valley culture is uniquely intransigent when it comes to critiquing their interviewing practices (if you think the interview system is broken, you're saying there's a chance that I shouldn't have been given my job, and that's too horrible to contemplate). Whether that's true or not, we're left not knowing what a participant actually saw.

The interviewing platform has “AI”, and the authors tell us the model (GPT-4o); this piece of information leads me to put more weight on my hypothesis about the interview platform name. It subtly reframes the paper from “AI has a negative impact on skills acquisition” to “our competitor's product has a negative impact on skills acquisition”; why mention this one product if you took a principled position on anonymising product names? Unfortunately that's all we get. “The model is prompted to be an intelligent coding assistant.” What does that mean? Prompted by whom? Did the experimenters control the system prompt? What was the system prompt? Was the model capable of tool use? What tools were available? Could participants modify the system prompt?

So now what we do know; 52 people (said in the prose to be split 26 in the “no-AI” control group, and 26 in the “AI access” treatment group; table 1 doesn't quite add up that way) were given a 10 minute coding challenge, then a 35-minute task to make use of a particular Python library, either with or without AI assistance. Finally, they have 25 minutes to answer questions related to the task: an appendix lists examples of the kinds of questions used, but not the actual question script. This is another factor that negatively impacts replicability.

What did they find?

There's no significant difference in task completion time between the two groups (people who used AI, and people who didn't use AI). That is, while the mean task completion time is slightly lower for the AI group, the spread of completion times is such that this doesn’t indicate a meaningful effect.

Overall, people who used AI did less well on the test that they took immediately after completing the task, and this is a meaningful effect.However, looking at their more detailed results (figure 7), it seems that among developers with less Python experience (1-3 years), the task completion time was vastly improved by AI access, and the outcome on the quiz was not significantly different.

Remember the sentence in the abstract was “Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library.” A different observation is “Participants with only a few years of Python experience showed significant productivity improvements, at no cost to learning the library”.

But does the quiz provide evidence of “having learned the library”? It’s a near-immediate test to recall knowledge about a task that the participants had just completed. What would the researchers have found if they waited four hours, or one day, or one week, to give the test? What would they have found if they set “learning the library” as an explicit task, and gave people time (with or without access to AI) to study? Would it make a difference if this study time was undertaken before participants undertook the coding task, or after? The authors find that some participants used significant time asking the AI assistant questions about the problem. In this way, they measure the total time taken to learn and solve the coding problem, in a situation where you've been given a total of 35 minutes for both.

The authors performed some qualitative analysis of their data. They find that people in the no-AI condition encounter more errors (syntax or API misuse) than people who use AI: this should be an interesting result, and a challenge to anyone who prejudges AI-generated code as “slop”. In this situation, it would seem that human-generated code is sloppier (at least in the first few minutes after it’s created).

They identify six AI interaction patterns among their treatment group, and that people who used three of the patterns achieved better results (though we can't comment on significance as we no longer have statistics) than the control group on the quiz outcome, without impact on “productivity”. As someone who has attached their wagon to the horse of intentional use of AI to improve software engineering skills, this should give me the warm fuzzies. In the context of the control validity questions of the study, I don’t know that they've necessarily demonstrated such improvement.

At this point I have another confounding factor to add to these results: the researchers questioned participants on their programming experience, but not on their experience using AI assistants (beyond recruiting people with non-zero experience). Do the adoption of these patterns correlate with more experience using AI? Do people with more experience using AI get more productivity when they use AI? We can’t tell.

And we also can’t say anything about the skill of using an AI assistant itself. Participants are asked about their understanding of the Python library, and the authors transfer their performance answering these questions into a measure of “skill acquisition” learned in using the library. Is that the skill they exercised? Do the quiz answers tell us anything about that skill? If participants were asked a week later to complete a related task, would their performance correlate with the quiz results? Is using the Python library even a useful skill to have?

The authors observed that one pattern performed far worse on both task-completion time and quiz responses than all the others, and this was “Iterative AI Debugging”: verifying or debugging code using the AI. This result isn’t surprising, because the pattern represents using the model to evaluate the logic embodied in the code, and language models don’t evaluate logic. They’re best suited to what used to be called “caveman debugging” where you use print statements to turn dynamic behaviour into a sequence of text messages—because the foodstuff of the model is sequences of text. They don’t evaluate the internal state and control flow of software, so asking them to make changes based on understanding that internal state or control flow is unlikely to succeed. However, given the small amount of data on this debugging pattern, this is really a plausible conjecture worthy of follow up, not a finding.

This preprint claims that using AI assistants to perform a task harms task-specific skill acquisition without significantly improving completion time. What it shows is that using AI assistants to perform a task leads to a broad distribution of ability to immediately answer questions related to the completion of the task, with an overall slight negative effect, without significantly improving completion time. The relation of the acquired knowledge to knowledge retention, or to skill, remains unexplored.

#### Or maybe, because we want to
How (and Why) Developers Use the Dynamic Features of Programming Languages: The Case of Smalltalk is an interesting analysis of the reality of dynamic programming in Smalltalk (Squeak and Pharo, really). Taking the 1,000 largest projects on SqueakSource, the authors quantitatively examine the use of dynamic features in projects and qualitatively consider why they were adopted.

The quantitative analysis is interesting: unsurprisingly a small number (under 1.8%) of methods use dynamic features, but they are spread across a large number of projects. Applications make up a large majority of the projects analysed, but only a small majority of the uses of dynamic features. The kinds of dynamic features most commonly used are those that are also supplied in "static" languages like Java (although one of the most common is live compilation).

The qualitative analysis comes from a position of extreme bias: the poor people who use dynamic features of Smalltalk are forced to do so through lack of alternatives, and pity the even poorer toolsmiths and implementors whose static analysis, optimisation and refactoring tools are broken by dynamic program behaviour! Maybe we should forgot that the HotSpot optimisation tools in Java come from the Smalltalk-ish Self environment, or that the very idea of a "refactoring browser" was first explored in Smalltalk.

This quote exemplifies the authors' distaste for dynamic coding:
Even if Smalltalk is a language where these features are comparitively easier to access than most programming languages, developers should only use them when they have no viable alternatives, as they significantly obfuscate the control flow of the program, and add implicit dependencies between program entities that are hard to track.
One of the features of using Object-Oriented design is that you don't have to consider the control flow of a program holistically; you have objects that do particular things, and interesting emergent behaviour coming from the network of collaboration and messages passed between the objects. Putting "comprehensible control flow" at the top of the priority list is the concern of the structured programmer, and in that situation it is indeed convenient to avoid dynamic rewriting of the program flow.

I have indeed used dynamic features in software I've written, and rather than bewailing the obfuscation of the control flow I've welcomed the simplicity of the solution. Looking at a project I currently have open, I have a table data source that uses the column identifier to find or set a property on the model object at a particular row. I have a menu validation method that builds a validation selector from the menu item's action selector. No, a static analysis tool can't work out easily where the program counter is going, but I can, and I'm more likely to need to know.

#### OS r us
I reckon I might be some kind of operating-system collecting nerd.  Having played with various systems on the Intel Mac at work through Parallels (NeXTSTEP, OPENSTEP, Rhapsody, Debian GNU/Linux, Solaris 10, DesktopBSD) got me wondering why I don't do the same at home.  So having fired up Qemu on the iBook, I'm now in the process of setting up the HURD.  There's a bit of a messy GRUB invocation involved before it will boot, but then everything works pretty well.I'm just left wondering exactly what it is I'm going to do with my new-found HURD installation :-)

#### Overdoing the risk management
I own a notebook. In fact, I own several notebooks. One in particular has an interesting feature (where I use "feature" in the "different from the competition, though we don't know whether anyone actually needs it" sense); inside the front cover is space to write your address, and a dollar value reward available to the person who returns the notebook.Now the notebook itself is probably worth about $20, but on the face of it a used notebook is worth less than a pristine notebook, with a full notebook having no value. Presumably the value of the reward should be related to the value of the notes contained within it, and therefore can't be ascertained until I've filled the notebook up. But then if I were to lose it before filling in the pages, I would not have entered an interim value; and if I had then whenever I made new notes I would need to update the worth of the book.And who should be footing the bill, anyway? Are my musings of any financial benefit to me, or if my employers get more worth from them should they be contributing to the reward fund? Could I possibly make the same notes again were I to lose this book? Could I pay someone with a lower salary than mine to have thoughts with a similar monetary value? Would someone else who came across my notebook be able to extract the same worth from the contents than me? If so, should I write in an encrypted fashion? How much more would that cost me? Should the reward factor in the costs of decrypting the contents, possibly reverse-engineering the method if I've forgotten it?Do ideas depreciate? Clearly patentable ideas do, will my ideas be patentable? Will I be able to benefit from the patents? If someone finds the notebook and returns it, are the ideas still patentable? What about non-patentable thoughts, do they all depreciate at a constant rate? Should the reward value be a function of time?Clearly the only people who can answer all of these questions upfront, and therefore the people who can use this reward feature with confidence, are the people whose ideas can be modelled with a waterfall development process. Take Terry Pratchett; he might know that the content of one notebook equates to roughly 50% of a novel, and that each novel is worth £200k, and therefore the value to him of the notebook is less than £100k. A thought process which eventually results in a cash value for a notebook. For those of us whose ideas are somewhat more iterative (read: chaotic), this seems like a complete misfeature.

#### Oxford University course on collaborative coding
Niche-audience topic time: if you're in Oxford Uni, I'm giving a one-day course on collaborative software engineering with git and GitHub (the ideas apply to GitLab, Bitbucket etc. too) on 4th June, 10-3 at the Maths Institute. Look out for information from the OxfordRSE group with a sign-up link!

#### Packaging software
I've been learning about Debian Packaging. I've built OS X packages, RPMs, Dockerfiles, JARs, and others, but never dpkgs, so I thought I'd give it a go.

My goal is to make a suite of GNUstep packages for Debian. There already are some in the base distribution, and while they're pretty up to date they are based on a lot of "default" choices. So they use gcc and the GNU Objective-C runtime, which means no blocks and no modern objc features. The packages I'm making are, mostly thanks to overriding some choices in building gnustep-make, built using clang, the next-generation objc runtime, libdispatch etc.

The Debian packaging tools are very powerful, very well documented, and somewhat easy to use. But what really impressed me along this journey was CPack. I've used cmake before, on a team building a pretty big C++/Qt project. It's great. It's easier to understand than autoconf, easier to build correct build rules over a large tree than make, and can generate fast builds using ninja or IDE-compatible projects for Xcode, IntelliJ and (to some extent) Eclipse.

What cpack adds is the ability to generate packages, of various flavours (Darwin bundles, disk images, RPMs, DEBs, NullSoft installers, and more) from the information about the build targets. That's really powerful.

Packaging software is a really important part of the customer experience: what is an "App Store" other than a package selection and distribution mechanism? It irks me that packaging systems are frequently either coupled to the target environment (Debian packages and OpenBSD ports are great, but only work in those systems), or baroque (indeed autoconf may have gone full-on rococo). Package builder-builders give distributors a useful respite, using a single tool to build packages that work for any of their customers.

It's important that a CD pipeline produces the same artefacts that your customers use, and also that it consumes them: you can't make a binary, test it, see that it works, then bundle it and ship it. You have to make a binary, bundle it, ship it, install it, then test it and see that it works. (Obviously the tests I'm talking about here are the "end-to-end", or "customer environment" tests. You don't wait until your thing is built to see whether your micro-tests pass, you wait until your micro-tests pass to decide whether it's worth building the thing.)

I know that there are other build tools that also include packaging capabilities. The point is, using one makes things easier for you and for your customers. And, it turns out, CMake is quite a good choice for one.

#### PADDs, not the iPad
Alan Kay says that Xerox PARC bought its way into the future by paying lots of money for each computer. Today, you can (almost) buy your way into the future of mobile computers by paying small amounts of money for lots of computers. This is the story of the other things that need to happen to get into the future.

I own rather a few portable computers.

Such a photo certainly puts me deep into a long tail of interwebbedness. However, let me say this: within a couple of decades this photo will look reasonable, for the number of portable computers in one office room. Faster networks will enable more computers in a single location, and that will be an interesting source of future applications.

Here's the distant future. The people in this are doing something that today's people do not do so readily.

One of them is giving his PADD to the other. Is that because PADDs are super-expensive, so everybody has to share them? No: it's because they're so cheap, it's easy to give one to someone who needs the information on it and to print/replicate/fax/magic a new one.

We can't do that today. Partly it's because the devices are pretty expensive. But their value isn't just associated with their monetary worth: they've got so many personalisations, account settings and stored credentials on them that the idea of giving an unlocked device to someone else is unconscionable to many people. The trick is not merely to make them cheap, but to make them disposable.

Disposable pad computers also solve another problem: that of how to display multiple views simultaneously on the pad screen.

You can go from rearranging metaphorical documents on a metaphorical desktop, back to actually arranging documents on a desktop. Rather than fighting with admittedly ingenious split-screen UI, you can just put multiple screens side by side.

The cheapest tablet computer I'm aware of that's for sale near me is around £30, but that's still too expensive. When they're effectively free, and when it's as easy to give them away as it is to use them, then we'll really be living in the future.

Just as it started, this post ends with Xerox PARC, and the inspiration for this post:

  Pads are intended to be "scrap computers" (analogous to scrap paper) that can be grabbed and used anywhere; they have no individualized identity or importance.

#### Pairing in Github
In the world of free software, it's good to appropriately credit contributors to your community for the work they do.

git makes this hard when you pair program. I was at a hackathon recently, and while I didn't make a single commit, I sat next to a lot of other people who made plenty of commits based on conversations we had, and suggested a lot of things to try to debug problems, and invented solutions that made it into those commits. No highly-nutritious green squares in github for me, no external evidence that I had contributed two days of my time to these free software projects.

When I pair, if I'm committing, I make sure that I acknowledge the contribution my pair makes as equal to my own. In the github UI, it looks like this. You can see that both of us contributed to the commit.

How do I do this? I commit like this:

git commit -m 'We fixed this thing' --author 'Jennifer H. Pair &lt;jenny.pair@example.com&gt;'

Now both accounts are linked in the UI, because I'm the committer and my pair is the author. This isn't perfect, because github doesn't acknowledge the author in their contribution graph, only the committer. If there's a more egalitarian way to acknowledge my pair I'd want to follow that, but for the moment I'm happy to at least demonstrate that they authored the change I typed into a text editor.

#### Parallels 2.1 beta 5
....will no longer boot OPENSTEP successfully, as far as I can tell.  At least, OS4.2 hangs out, in trying to load the VBE driver.  Reverting to Beta 4 works though.If anyone has different experiences with Beta 5, could they comment here?

#### Parallels and OpenStep: success!
Just turn off VT-x extensions for the host CPU in the VM's configuration (you'll find this switch in Options -&gt; VM Flags -&gt; Emulation Flags).  Now the system boots fully, in any of the VESA modes that I've tried :-)

#### Parallels may, in fact, be completely dumb
tyr:/Library/Parallels root# ls -ltotal 16-rw-rw-rw-     1 root   wheel     40 Sep 12 15:52 .dhcp.en2-rw-rw-rw-     1 root   wheel      0 Jul  4 17:10 .dhcpd_configuration-rw-rw-rw-     1 root   wheel      0 Jul  4 17:10 .parallels_common_options-rw-rw-rw-     1 root   wheel    100 Jul  4 22:44 .parallels_license_2.2

#### Parallels RC and OPENSTEP works!
I know that there've been mixed results in trying to get OPENSTEP booting on Parallels Desktop RC, but it Works For Me^{TM}.  In case this is a particularly special piece of information, I upgraded directly from beta 4 (which I'd previously been using because 5 onwards were breaking the VBE driver).  Both immediately after the upgrade, and following a reboot (I remembered that Parallels installs a kext, and thought it best to verify this), OPENSTEP boots up in glorious(?) multicolour.Eww, although there's a graphics glitch displaying the &quot;please wait until it's safe to turn off your computer&quot; screen.Update 2006-06-01 11:08 GMT: and so does RC2.

#### Password checking with CommonCrypto
I previously described a system for storing and checking credentials on Mac OS and iOS based on using many rounds of a hashing function to generate a key from the password. Time has moved on, and Apple has extended the CommonCrypto library to provide a simple, standard and supported way of doing this. If this is still a problem you need to solve, you should look at doing it this way instead of following the earlier post.

We're going to use a key-stretching function called PBKDF2 to make an encryption key of a standard length. This key probably will be much longer than the user's password. However, it can't possibly be any more random: it must be deterministically derived from that password so we're stuck with the same amount of randomness that this password contains. A good key-stretching function should "smooth out" the randomness from the initial password, so that you can't guess anything about the password given the function's output.

We're then going to use this key to calculate an HMAC of some known data. An HMAC is a bit like a digital signature, in that it depends both on the key used and the input data. We'll store the HMAC but not the password and not the key derived from the password. Whenever the password is needed in the future, it must be provided by a user and cannot be derived from any of the data in the app.

The idea, then, is that when you set a new password, the app calculates this key, uses that to calculate an HMAC and stores the HMAC. When you try to use the app, you present a password, from which the app generates a key and an HMAC of the same data. If this HMAC matches the one that was previously calculated, the same keys were used, which (hopefully) means that the same password was supplied.

The faster you scream, the slower we go.

One of the problems that a key-stretching function must address is that computers are really, really fast. Normally computers being really, really fast is a benefit, but the faster it is to compute all of the above stuff the more guesses an attacker can try at the password in some amount of time. We therefore want to make this function slow enough that brute force attacks are limited, but not so slow that people get frustrated with the app's performance.

PBKDF2 has a tuneable parameter - the number of rounds of a hashing function it uses internally to stretch the key. With CommonCrypto you can ask for a number of rounds that will result in the function taking (approximately) a certain amount of time to work on a password of a certain length.

Requirements for the rounds parameter

The number of rounds used when checking a password should be the same as the number used when generating the stored HMAC, otherwise the keys generated won't match.
The above means that you need to choose a single value to use across app installs and hardware - unless you're happy with losing access to all user data when a customer upgrades their iPad.
You will want to revise this parameter upwards as faster hardware becomes available. This conflicts with the first requirement: you'll need a fallback mechanism to try the same password with different "versions" of your rounds parameter so that you can upgrade users' credentials with your app.

With those in mind, you can construct an algorithm to choose a tuning parameter based on this call:

        const uint32_t oneSecond = 1000;
         rounds = CCCalibratePBKDF(kCCPBKDF2,
                                   predictedPasswordLength,
                                   predictedSaltLength,
                                   kCCPRFHmacAlgSHA256,
                                   kCCKeySizeAES128,
                                   oneSecond);

You can probably know what length of salt you'll use: salt should be a block of random data that you supply, that is used as additional input to the key-stretching function. If two different users supply the same password, the salt stops the function from generating the same output. You probably can't know what length of password users will use, but you can guess based on experience, data and knowledge of any password strength rules incorporated into your app.

Notice that you should derive this rounds property on the target hardware. The number of rounds that take a second to run through on your brand new iMac will take significantly longer on your customer's iPhone 3GS.

Generating the HMAC data

Both storing and checking passwords use the same internal function:

- (NSData *)authenticationDataForPassword: (NSString *)password salt: (NSData *)salt rounds: (uint) rounds
{
    const NSString *plainData = @"Fuzzy Aliens";
    uint8_t key[kCCKeySizeAES128] = {0};
    int keyDerivationResult = CCKeyDerivationPBKDF(kCCPBKDF2,
                                                   [password UTF8String],
                                                   [password lengthOfBytesUsingEncoding: NSUTF8StringEncoding],
                                                   [salt bytes],
                                                   [salt length],
                                                   kCCPRFHmacAlgSHA256,
                                                   rounds,
                                                   key,
                                                   kCCKeySizeAES128);
    if (keyDerivationResult == kCCParamError) {
        //you shouldn't get here with the parameters as above
        return nil;
    }
    uint8_t hmac[CC_SHA256_DIGEST_LENGTH] = {0};
    CCHmac(kCCHmacAlgSHA256,
           key,
           kCCKeySizeAES128,
           [plainData UTF8String],
           [plainData lengthOfBytesUsingEncoding: NSUTF8StringEncoding],
           hmac);
    NSData *hmacData = [NSData dataWithBytes: hmac length: CC_SHA256_DIGEST_LENGTH];
    return hmacData;
}

Storing credentials for a new password simply involves generating a salt, computing the authentication data then writing it somewhere:

- (void)setPassword: (NSString *)password
{
    //generate a random salt…

    //do any checking (on complexity, or whether the passwords entered in two fields match)…

    NSData *hmacData = [self authenticationDataForPassword: password salt: salt rounds: [self roundsForKeyDerivation]];
    //store the HMAC and the salt, perhaps by concatenating them and putting them in the keychain…

}

Then testing the credentials involves applying the generation function to the password guess, and comparing the result with what you previously stored:

- (BOOL)checkPassword: (NSString *)password
{
    //recover the HMAC and the salt from wherever you stored them…

    NSData *guessedHmac = [self authenticationDataForPassword: password salt: salt rounds: [self roundsForKeyDerivation]];
    return [guessedHmac isEqualToData: hmacData];
}

Conclusion

If you can avoid storing a password in your app, even in the keychain, you should; there's then a much reduced chance that the password can be recovered from the app by an attacker. Deriving a key from the password using PBKDF2 then testing whether you can use that key to obtain a known cryptographic result obviates the need to store the password itself. Mac OS X and iOS provide an easy way to use PBKDF2 in the CommonCrypto library.

The key derived from the password could even be used to protect the content in the app, so none of the documents are available without the password being presented. Doing this offers additional confidentiality over simply using the password for access control. Building a useful protocol around this key requires key wrapping, the subject of a future post.

#### Permissions whee!
As in any good mystery, the question is who done it?  MacNN reports a flaw in Tiger, Leopard in which an authenticated copy operation gives the destination files (the copies) the ownership of the logged-in user, not of the name they used to authenticate.  The question is, which user did the copy?

Let's say there's a system with Alice Administrator and Richard Regular-User.  Richard downloads a new application from the intarwebs, and wants to put it in /Applications (though why?  Why can't he just put it in ~/Applications like a good little user?  Never mind).  The thing is, he doesn't have the right to do that.  Finder presents him with an authentication dialogue, and no matter how many times he enters his username and password correctly, he can't acquire that right.  However, he sees Alice walking past in the corridor and asks her to enter her admin credentials.  For whatever reason, she agrees - now Alice has authenticated and Alice has acquired the right to copy the files.  So even though Richard requested the copy, it was actually Alice who performed it.  Therefore Alice created the files at the destination, so they should be owned by Alice.

The only thing which muddies the waters (and leads to the conflict of convenience vs. security which is described in that article) is that in many, or indeed most, cases on OS X where this will arise, Alice and Richard are actually the same person - Sammy the Single (Security-conscious, hence separating their use of the system into regular and admin accounts) User.   It's a convenience that as Richard wanted the files copied, Richard now owns the copy - but this defeats the point of Richard existing, which is that Sammy doesn't want to be able to change /Applications without being warned.

Interestingly the same question doesn't get asked of the sudo command - it's clear that if I type sudo ditto Foobar.app /Applications/Foobar.app it's the super-user who does the work.

#### Phrases in computing that might need retiring
The upcoming issue of the SICPers newsletter is all about phrases that were introduced to computing to mean one thing, but seem to get used in practice to mean another. This annoys purists, pedants, and historians: it also annoys the kind of software engineer who dives into the literature to see how ideas were discussed and used and finds that the discussions and usages were about something entirely different.

So should we just abandon all technical terminology in computing? Maybe. Here's an irreverent guide.

Object-Oriented Programming

Luckily the industry doesn’t really use this term any more so we can ignore the changed meaning. The small club of people who still care can use it correctly, everybody else can carry on not using it. Just be aware when diving through the history books that it might mean “extreme late binding of all things” or it might mean “modules, but using the word class” depending on the age of the text.

Agile

Nope, this one’s in the bin, I’m afraid. It used to mean “not waterfall” and now means “waterfall with a status meeting every day and an internal demo every two weeks”. We have to find a new way to discuss the idea that maybe we focus on the working software and not on the organisational bureaucracy, and that way does not involve the word…

DevOps

If you can hire a “DevOps engineer” to fulfil a specific role on a software team then we have all lost at using the phrase DevOps.

Artificial Intelligence

This one used to mean “psychologist/neuroscientist developing computer models to understand how intelligence works” and now means “an algorithm pushed to production by a programmer who doesn’t understand it”. But there is a potential for confusion with the minor but common usage “actually a collection of if statements but last I checked AI wasn’t a protected term” which you have to be aware of. Probably OK, in fact you should use it more in your next grant bid.

Technical Debt

Previously something very specific used in the context of financial technology development. Now means whatever anybody needs it to mean if they want their product owner to let them do some hobbyist programming on their line-of-business software, or else. Can definitely be retired.

Behaviour-Driven Development

Was originally the idea that maybe the things your software does should depend on the things the customers want it to do. Now means automated tests with some particular syntax. We need a different term to suggest that maybe the things your software does should depend on the things the customers want it to do, but I think we can carry on using BDD in the “I wrote some tests at some point!” sense.

Reasoning About Software

Definitely another one for the bin. If Tony Hoare were not alive today he would be turning in his grave.

#### PL personality theory
An analysis of programmer personality traits inferred from their answer to the question "which is your favourite programming language?"

Algol About to re-enact that scene in Jumanji where Robin Williams has a huge beard.

Basic Remembers a time when you could code a whole platform game with twenty levels in 6k of RAM. Probably works on some trading platform that needs the JVM heap size bumped to 4GB to add two numbers.

C Learnt programming once, what more could there be to it?

C++ Learnt programming once, it was horrible.

C# Wears Microsoft shoes, Microsoft trousers, and a Microsoft t-shirt. C# also goes by the name "Washington State Swift".

D Probably best to ask again next week.

Elixir I used to be a Ruby programmer until I realised I hate Ruby programmers.

F# Wears the same clothes as the C# programmer but does so ironically.

Go Uses Google+ earnestly.

Java Has hobbies that aren't programming.

Javascript Look, even Yersinia Pestis was popular once.

Lisp Mostly calm with sudden outbursts of zen.

Objective-C At the intersection of technology and liberal cash. In danger of progressing to Smalltalk.

Perl Stoic in the face of abuse. Ignores it and carries on getting loads of work done.

Python Had an argument with a Perl programmer in 2004. They each think they won; neither is correct.

Ruby Used to use Java but then learned object-oriented programming and had to move on.

Ruby on Rails Like that kid in that movie. No, not War Games, the other one. Home Alone.

Scheme Pretentious. Probably has a blog named for a pun on a classic computing textbook.

Self Slightly further along in their hatred of computing than a Smalltalker.

Smalltalk About to re-enact that scene in Planet of the Apes where Charlton Heston finds the statue.

Swift Like the person who goes into the specialist metal record store and conspiratorially asks whether they've got anything by Metallica.

Tcl Submits write-in answers to multiple choice questions.

#### Planet of the Apps
Scene: in front of a green screen somewhere in the present day. Our protagonist, freshly burned out from a session of writing dynamically-typed web backend code in vim, looks up from the monitor. In the distance, some way along the beach, they see an odd shape poking out of the sand. Their curiosity piqued, they trudge out under the burning sun toward the edifice.

Running a risk of collapsing through dehydration, finally they are close enough to the object to be able to see through the heat haze that it is the top of a large statue that's largely covered by centuries of detritus. The only discernible features are a hand holding aloft a chorded keyer, and the stern-browed head of Douglas Englebart.

  Oh my God, I'm back. I'm home. All the time it was…we finally really did it. YOU MANIACS! OH DAMN YOU! GOD DAMN YOU ALL TO HELL!

#### Poke the other one, it's got bells on
Originally the title for this post was to be "Why a Morris organisation should adopt social media (and why they probably won't)", with what is now the title being reduced to the rank of a subtitle. Then I remembered that I am leeg, and as such humour is always better than content (certainly a lot easier). So we have the title you see before you.Please bear in mind when reading the epistle located below that I'm fairly new to the whole world of Morris, and especially new to its political fiddle-faddle. If anything here seems to stereotype people or their motives, it's based on the experience of someone who can at best be considered an informed outsider. [And for those who are even more of an outsider than I am, Morris dancing is the name for a roughly-related collection of traditional English dancing styles, performed by groups called sides or teams. It's both good fun and a decent workout, give it a go.]It seems to have been a problem for at least the length of my lifetime that Morris lacks any relevance to what, for sake of pomposity, I shall refer to as the man on the Clapham omnibus &mdash; taking the irony fully on board. That your average person sees no reason to engage with or appreciate the Morris. Why? Has the Morris really made much of an attempt to engage with or appreciate the life of the average person? Not, I would argue, at any concerted or large-scale level, leaving the final impression most people have of Morris dancers the same as their first impression: a bunch of old men in silly clothes hitting each other with sticks.So, shouldn't I have mentioned the social media by now? Yes, I'm just coming to that, it's only a couple of paragraphs away now. First some context. There are three umbrella organisations for the Morris in the UK: the Morris Ring is the oldest, with its infamous men-only rules; the Morris Federation (whose website was broken at time of writing) started life as the less-infamously women-only Women's Morris Federation; then there's Open Morris, which to my knowledge has never had any infamous membership rules and is the youngest of the three organisations. None of these organisations takes on a promotional or advocacy rôle &mdash; or at least, if they do, they've done a bad job of it. They represent more of an internal support network, offering guidance, information, training and the like to the sides. If you don't believe that they aren't promotional bodies, take a look at their websites.As a digression I will use this paragraph to mention the English Folk Dance and Song Society, which does indeed have advocacy, promotion and outreach as its goals. However, I'm not aware what the relationship between the EFDSS and any of the sides or morris umbrella groups is. That's definitely lack of knowledge on my part, rather than indicative of a lack of interaction. Certainly the three groups named above have all contributed to the EFDSS's magazine, ED&amp;S, recently, although I haven't read their contributions. There's a lot of morris-related material in the EFDSS archives, too. By the way, I'm reliably informed that the society's name is pronounced "EFd&amp;#x0259s".There are, then, four groups identified who either do, or could take on if they so chose, a rôle in promoting the morris to the general public. Given the continued and increasing popularity that web-based media, particularly social media, has in the world, let's examine the part each of these groups plays.

Twitter Presences: 0.Official Facebook Presences: 2. The EFDSS has a fan page. There's a group for Open Morris too, which looks like it could be run by the real organisation. I think that counts as user-generated content is part of the world of social media.Official YouTube Channels: 0. Or at least none that I could find.Myspace Presences: 1. EFDSS again.Podcasts: 0.RSS feeds (that scraping noise you hear, it's the bottom of the barrel): 1. It's the Morris Ring's news feed. I nearly fell out of my chair.For completeness, I'll also mention that there's an unofficial mailing list called MDDL (Morris Dancing Discussion List) which is very active with a strong signal-to-noise ratio.

Now, why should any of this be important? Well, as you're reading this, you're consuming a blog. You either thought "I would like to know what leeg is up to, I'll read his blog", "I heard that some guy on the interwebs really lays into morris dancing organisations, I'll check it out", or something else which made you decide to spend time engaging with social media and user-generated content on the web. That's time which the various morris groups could have spent injecting your brain with Morris Dancing. Are they doing that? No. They're writing internal newsletters bemoaning the lack of traditional dancing in the national curriculum, and press releases for the Torygraph to pick up, claiming that morris dancing is dying out because all of the practitioners are getting too old. In fact, it's been nearly a year since that last one was done.The problem is that you and I and everybody else don't give a shit about the Morris Ring Circular or their press releases, in the same way we give a shit about, say, procrastinating on YouTube, following interesting people on Twitter, catching up with friends on Facebook and so on. The people who complain about morris dancing fading into irrelevance are even managing to complain about it in an irrelevant fashion. Irony. They're doing it right.And the most galling thing is that the social media and traditional dancing could go together so well. Take a photo like this, which I took at the Morris 18-30 in WakeField:You may wonder why everyone's wearing different costumes; well the answer is that they're from different sides. A mash-up could tag the dancers with their side's name, and a link to their website. Your next question would, of course, be the same as mine: "but where the hell is Packington, anyway?" A map showing the location of each side could be available, perhaps even showing proximity of their practice venue to your current location and when they'll next be practising (please, do not get me started on the existing SideFinder pages. You would not like what I would become). Had I remembered what dance was being danced (I think it's Skirmishes, though I'm sure someone would be able to spot it from the photo), then information about that dance and videos of sides performing it could be available.[Another aside: the relative obscurity of some teams' locations has consequences for those teams' performances. As an example, the fool of Adderbury Morris always wears a hat made of fox skin during the dances. This is because when the first team was formed, the fool went home to tell his wife about his new position. Her response is recorded as being "Adderbury? Wear the fox hat".]"Aha," you hypothetically cry. "But I would not have seen the photo in the first place, had you not talked about it on your blog." Indeed no, but the point of blogs, Twitter, Tumblr and the like is that you get personal recommendations from people you either trust or consider expert in their field. So I think it's fair to have introduced the photo in that way; and that there's a space online for personal recommendation of trying out morris dancing. And that there ought to be some organisation helping people to do that promotion.Incidentally, the Morris 18-30 is actually a good example of a group (or phenomenon, I suppose) building a decent website with good amounts of information, and providing an easy way for people to get their photos online. There are many sides which have done the same; Westminster Morris Men's YouTube channel has a decent set of videos including some from the archives. My point is not that this information is not being made available, nor that an effort is not being made &mdash; I chose the 18-30 photo for my example because I took the picture and have the right to re-use it in this blog.I don't think that the fact some sides do well at promoting themselves affects my argument; those who don't have the skills or resources to do this work themselves are not being helped by the national/international bodies. People who might be interested in traditional dancing aren't finding out about it, because the nearest side who have any skill at marketing are in the next county. In a world where information can travel the world in a fraction of a second, that's ludicrous.So the fact that 18-30 has a good website where people can share information easily is a good thing. The various mash-up suggestions I made would all be good improvements, and were they implemented by an umbrella group then everyone could take advantage of them. My point is that the umbrella organisations should be taking and collating the vast amount of morris-related information out there, and making it easy for people who are not (yet) in the morris to find. They should be using it to promote the dance form and the social activities that surround it, but they aren't. They should be providing a central service to make it easier for sides to share their own material, but they aren't. They should be taking their existing archives and making them available online, but they aren't. They should be looking at the innovations made by some of their members and applying them at the international level, but aren't.In addition to existing content, there is plenty of scope for promotional material based on novel content distributed over the internet. The dances, music and even pub sessions could make great segments for a vlog or video podcast. Even mini-instructionals could be presented as video podcasts or on a YouTube channel, so that people can try things out without having to join a side first. "What's the point of dancing a team dance on your own?" &mdash; leaving jigs aside as PhD-level morris, some people may just want to find out whether they can do some of the basic stuff on their own before turning up to a practice session. If you're not sure whether you want to take part in an activity, would you have your first try in front of twenty people who've been doing it for years, and are each armed with a big stick? Maybe not.So there's plenty of space for morris information to be distributed digitally. But, really, who gives a toss? That's where the promotional aspect comes in. I've already mentioned the personal level of promotion through Twitter and the like. There are obvious places where morris could be promoted; what's on guides, tourism sites, tradition-reporting sites and the like. But how about novel audiences? Dancers, like many British people, enjoy a Beer in the Evening. Given decent information feeds like the things I described a few paragraphs ago, morris data could be highly Mashable, featuring in those little Facebook games. If people like what they see, they will Digg it. Were one of the umbrella orgs to hire a dashing, intelligent, handsome developer-dancer they could even promote through the iPhone app store (though where would they find such a person?).And what Americans like to call "the kicker" is this: real people drink beer, use websites and download apps. Most of the dancers I've met are either from families of dancers or already had interest in folk music; in that sense the person on the street is an "unexploited vertical" for the marketers of morris, and probably has been since the end of the first world war.OK, so that's what could be done, who should be doing it, and why. Is it fair for me to put words in the mouths of the umbrella orgs in suggesting why they aren't currently doing it? No, but I will anyway. If you think this blog is fair, then I've got a slightly-used iBook I'll sell you for a great price.I think that for a large part, the people in charge probably just don't use and therefore don't understand the potential of social media. But that doesn't explain why the morris umbrella organisations don't do any promotion whatsoever. At least one of the organisations may be wary of getting too much publicity for themselves; I'm not a lawyer of course, but the equality bill currently awaiting its 3rd reading in the House of Commons could require the Morris Ring to change its membership rules, as it &quot;Extends discrimination protection in the terms of membership and benefits for private clubs and associations&quot;. I expect there isn't much in the way of training available to the organisations in the general field of marketing. I'm not sure what kind of budgets these groups run on, but maybe the three of them together could afford a part-time marketer.Of course, there's some appeal to the idea that you're in a secret society, isn't there? It's quite exciting to think that you do something enjoyed by few others, and it's easier to become important in smaller social groups. Not that I'm suggesting that's a related point, oh no.I apologise for writing such a long post. I didn't have the time to write a shorter one.

#### Post #100!
And to celebrate, we look at the differences between managers and humansprogrammers.

#### Pre-order professional Mac security books!
You too can own a piece of the magic. Professional Cocoa Application Security and Enterprise Mac: Mac OS X Snow Leopard Security are both already in pre-order; use the Amazon affiliate links below if you want to give me a little extra kick-back from the sales of each. You're most kind :).

#### Preparing for Computing's Big One-Oh-Oh
However you slice the pie, we're between two and three decades away from the centenary celebration for applied computing (which is of course significantly after theoretical or hypothetical advances made by the likes of Lovelace, Turing and others). You might count the anniversary of Colossus in 2043, the ENIAC in 2046, or maybe something earlier (and arguably not actually applied) like the Z3 or ABC (both 2041). Whichever one you pick, it's not far off.

That means that the time to start organising the handover from the first century's programmers to the second is now, or perhaps a little earlier. You can see the period from the 1940s to around 1980 as a time of discovery, when people invented new ways of building and applying computers because they could, and because there were no old ways yet. The next three and a half decades—a period longer than my life—has been a period of rediscovery, in which a small number of practices have become entrenched and people occasionally find existing, but forgotten, tools and techniques to add to their arsenal, and incrementally advance the entrenched ones.

My suggestion is that the next few decades be a period of uncovery, in which we purposefully seek out those things that have been tried, and tell the stories of how they are:

successful because they work;
successful because they are well-marketed;
successful because they were already deployed before the problems were understood;
abandoned because they don't work;
abandoned because they are hard;
abandoned because they are misunderstood;
abandoned because something else failed while we were trying them.

I imagine a multi-volume book✽, one that is to the art of computer programming as The Art Of Computer Programming is to the mechanics of executing algorithms on a machine. Such a book✽ would be mostly a guide, partly a history, with some, all or more of the following properties:

not tied to any platform, technology or other fleeting artefact, though with examples where appropriate (perhaps in a platform invented for the purpose, as MIX, Smalltalk, BBC BASIC and Oberon all were)
informed both by academic inquiry and practical experience
more accessible than the Software Engineering Body of Knowledge
as accepting of multiple dissenting views as Ward's Wiki
at least as honest about our failures as The Mythical Man-Month
at least as proud of our successes as The Clean Coder
more popular than The Celestial Homecare Omnibus

As TAOCP is a survey of algorithms, so this book✽ would be a survey of techniques, practices and modes of thought. As this century's programmer can go to TAOCP to compare algorithms and data structures for solving small-scale problems then use selected algorithms and data structures in their own work, so next century's applier of computing could go to this book✽ to compare techniques and ways of reasoning about problems in computing then use selected techniques and reasons in their own work. Few people would read such a thing from cover to cover. But many would have it to hand, and would be able to get on with the work of invention without having to rewrite all of Doug Engelbart's work before they could get to the new stuff.

✽: don't get hung up on the idea that a book is a collection of quires of some pigmented flat organic matter bound into a codex, though.

#### Prepping for WWDC
With the obvious first question being which parties do I go to? See you there?

#### Principled Lizards
Sixty-five million years ago, there were many huge lizards. Most of them were really happy being lizards, and would spend all of the time they could doing lizardy things. Some wanted to be the biggest lizards, and grew so large and so heavy that it would sound like peals of thunder if you could hear them walking about on their lizardy way. Others wanted to be the most terrible lizards, and they developed big scary teeth and sharp, shiny talons. The most terrible lizards were feared by many of the other lizards, but it was a fear that sprang from awe: they were all happy that each was, in their own way, the most lizardy of the lizards. And they were all happy that each of the other lizards they met was trying to be, in their own way, the most lizardy of lizards.

For the lizards met often. They would have their big get-togethers where the big lizards and the small lizards and the terrible lizards and the scaly lizards would each talk about how they handle being so big, or so small, or so terrible, or so scaly. And the other lizards would listen to these talks, and they would applaud the speakers for being so big, or so small, or so terrible, or so scaly. Having seen these examples of lizardly apotheosis, they would try to emulate them. So it was that the lizard world became bigger, but also smaller, and more terrible, and more scaly.

But it seems that not all of the lizards shared these goals of ever-increasing lizardhood. Some would try different things. A group of lizards found that they could regulate their own blood temperature, they would no longer need to sit in the sun all morning like the other lizards. One group of lizards turned their feathery covering to the task of improved aerodynamics. Another group turned it to a sort of coat, which stopped them getting so cold.

The big meetings of lizardy lizards did not really pay these developments much notice, as they were not very lizard like changes. They knew that they were lizards! They should do the lizardy things, like getting bigger or smaller or more terrible or more scaly! They put over eighty hours a week into it, they were passionate about it. The world was, for them, all about being more lizardly every day.

Some of the things that the decidedly non-lizardlike groups were coming up with did take a sort of root among those who called themselves the "lizard community", but only to the extent that they could be seen as lizardy things. So ideas from the feather aerodynamics group became diluted, and were called "flight-oriented lizarding". At the big gatherings of all the lizards, the FOL evangelists would show how they had made things that looked a bit like the feathers used for aerodynamics, but which were more lizardy. They had some benefit to lizards in that they slowed them down slightly as they fell out of trees. And, of course, as this was something that you had to be able to demonstrate expert lizardly competence in, they invented the idea of the master flight-oriented lizard.

All sorts of rules were invented to demonstrate competency and master-lizardliness in the flight-oriented world. This feather and that feather must each have a single responsibility: this for slowing the fall, that for turning. Feathers must be open for falling but closed for impact. Specific types of feathers could be invented, but only where they could be used in place of the more generic feathers. Feathers had to be designed so that they never got into the area around a lizard's eyes (the in-the-face segregation principle). Despite the fact that flight-oriented lizards only used their feathers for falling out of trees, feathers had to be designed to work when travelling upwards too (the descendency inversion principle).

But to the expert lizards—the biggest, smallest, scaliest and most terrible lizards—something felt uncomfortable. It felt like people were saying that there was something else to do than being an expert lizard, as if lizardness wasn't enough. So, of course, they arranged another meeting of all the lizards. Expert lizards and novice lizards and improving lizards all came together, that one day sixty-five million years ago, and they met in the town of Chicxulub. And the most expert of the expert lizards got up in front of all the lizards, and said this:

  If you want to carry on at lizarding you have to really love it. You've got to want to put every waking moment into becoming a better lizard. You've got to look up after practising your lizarding, and be shocked at how much time has gone past. If that isn't you, if you don't absolutely love everything about lizarding, perhaps it's time to move on and do something else.

Many of the expert lizards agreed with this idea, and were pleased with themselves. But many that had been trying other things, the fur or the flying or the warm blood, were confused: did they want to be lizards forever, and strive toward the best of lizardliness, or not? Did they perhaps want to explore the opportunities presented by warm blood, or flying, or fur?

And so it was that at Chicxulub, as a rock from outer space danced through the upper atmosphere, pushing and heating and ionising the air in front of it, people chose between the many paths open to them.

#### Product teams: our products are not our products
Woah, too many products. Let me explain. No, it will take too long, let me summarise.

Sometimes, people running software organisations call their teams "product teams", and organise them around particular "products". I do not believe that this is a good idea. Because we typically aren't making products, we're solving problems.

The difference is that a product is "done". If you have a "product team", they probably have a "definition of done", and then release software that has satisfied that definition. Even where that's iterative and incremental, it leads to there being a "product". The thing that's live represents as much of the product as has been done.

The implications of there being a "product" that is partially done include optimising for getting more "done". Particularly, we will prioritise adding new stuff (getting more "done") over fixing old stuff (shuffling the deckchairs). We will target productish metrics, like number of daily actives and time spent.

Let me propose an alternative: we are not making products, we are solving problems. And, as much out of honesty as job preservation, let me assure you that the problems are very difficult to solve. They are problems in cybernetics, in other words in communication and control in a complex system. The system is composed of three identifiable, interacting subsystems:

    The people who had the problem;
    The people who are trying to solve the problem;
    The software created to present the current understanding of the solution.

In this formulation, we don't want "amount of product" to be a goal, we want "sufficiency of solution" to be a goal. We accept that the software does not represent the part of the "product" that has been "done". The software represents our best effort to date at modelling our understanding of the solution as we comprehend it to date.

We therefore accept that adding more stuff (extending the solution) is one approach we could consider, along with fixing old stuff (reflecting new understanding in our work). We accept that introducing the software can itself change the problem, and that more people using it isn't necessarily a goal: maybe we've helped people to understand that they didn't actually need that problem solved all along.

Now our goals can be more interesting than bushels of software shovelled onto the runtime furnace: they can be about sufficiency of the solution, empowerment of the people who had the problem, and improvements to their quality of life.

#### Programmer Values
A question and answer exchange over at programmers.stackexchange.com reveals something interesting about how software is valued. The question asked whether there is any real-world data regarding costs and benefits of test-driven development.[*] One of the answers contained, at time of writing, the anthropologist's money shot:

  The first thing that needs to be stated is that TDD does not necessarily increase the quality of the software (from the user's point of view). […] TDD is done primarily because it results in better code. More specifically, TDD results in code that is easier to change. [Emphasis original]

Programmers are contracted, via whatever means, by people who see quality in one way: presumably that quality is embodied in software that they can use to do some thing that they wanted to do. Maybe it's safer to say that the person who provided this answer believes that their customers value quality in software in that way, rather than make an assumption on everybody's behalf.

This answer demonstrates that (the author believed, and thought it uncontentious enough to post without defence in a popularity-driven forum) programmers value attributes of the code that are orthogonal to the values of the people who pay them. One could imagine programmers making changes to some software that either have no effect or even a negative effect as far as their customers are concerned, because the changes have a positive effect in the minds of the programmers. This issue is also mentioned in one of the other answers to the question:

  The problem with developers is they tend to implement even things that are not required to make the software as generic as possible.

The obvious conclusion is that the quality of software is normative. There is no objectively good or bad software, and you cannot discuss quality independent of the value system that you bring to the evaluation.

The less-obvious conclusion is that some form of reconciliation is still necessary: that management has not become redundant despite the discussions of self-organised teams in the Agile development community. Someone needs to mediate between the desire of the people who need the software to get something that satisfies their norms regarding quality software, and the desire of the people who make the software to produce something that satisfies their norms instead. Whether this is by aligning the two value systems, by ignoring one of them or by ensuring that the project enables attributes from both value systems to be satisfied is left as an exercise for the reader.

[*] There is at least one relevant study. No, you might not think it relevant to your work: that's fine.

#### Programming as a societal roadblock
Introduction

People who make software are instigators of and obstacles to social interactions. We are secondarily technologists, in that we apply technology to enable and block these transactions. This article explores the results.

Programmers as arbiters of death

I would imagine that many programmers are aware of the Therac-25 and the injuries and deaths it caused. When I read that article I was unsurprised to discover that the report I'd previously heard about the accidents was oversimplified and inaccurate: lone programmer introduces race condition in shield interlock, kills humans. Regular readers of this blog will know that the field of software has an uneasy relationship with the truth.

What's Dirk Gently got to do with it?

What the above report of the radiation accidents exposes is that the software failures were embedded in a large and complex socio-techno-political system that permitted the failures to be introduced, released, and to remain unaddressed once discovered in the field to have caused actual, observable harm.

This means that it was not just the one programmer with his (I borrow the pronoun from the article above, which implies the programmer's identity is known but doesn't supply it) race condition who was responsible. He was responsible, along with the company with its approach to QA and incident response, the operators, medical physicists and regulators and their reactions to events, and the collective software industry for its culture and values that permitted such a system to be considered "good enough" to be set into the world.

In Dirk Gently's Holistic Detective Agency, Douglas Adams described the fundamental interconnectedness of all things-the idea that all observable phenomena are not isolated acts but are related parts of a universal whole. Given such a view, the entire software industry of the 1970s-1980s had been complicit in the Therac-25 disasters by leading programmers, operators and patients alike to believe that software is made to an acceptable standard.

An example less ancient

A natural, though misplaced, response to the above is to notice that as we don't do software like that any more, the Therac-25 example must now be irrelevant. This is an incorrect assumption but regardless, let me provide another more recent case.

Between 2007 and 2009 I worked for a company in the business of security software. The question of whether this business itself is unethical-making up for the software you were sold being incapable of operating correctly by selling you more software that was made in the same way-can wait for another post. The month before I left this company (indeed around the time I handed in my notice), they acquired another security software company.

One of this child company's products is a thing called a Lawful Interception Module. Many countries have a legal provision where law enforcement agencies can (often after receiving a specific order) require telephone companies to intercept and monitor telephone calls made by particular people, and LIMs are the boxes the phone companies need to enable this. Sometimes the phone companies themselves have no choice in this, in the UK the Regulation of Investigatory Powers Act requires postal and telecoms operators to make reasonable accommodation for interception or face civil action.

In 2011, the child company sold LIMs through a third-party to the Syrian government, which perhaps intended to use the technology to discover and track political opponents. At the time this got a small amount of coverage in the news, and the parent company (the one I'd worked for) responded with a statement saying that the sale never resulted in an operational system. In other words, we acted ethically because our unethical products don't actually work.

Later, as part of the cache of documents published by Edward Snowden, the story got a fresh lease of life. This time the parent company responded by selling the LIMs company. Now they don't act unethically because they took a load of money to let someone else do it.

The time that I was working at the parent company was a great opportunity for me to set the ethical and professional tone of the organisation, particularly as it coincided with their acquisition of the child company and the beginnings of their attempts to define a shared culture. Ethical imperatives promulgated then could have guided decisions made in 2011. So what advantage did I take of the opportunity?

None. I wasn't concerned with (or even perhaps aware of) professional ethics then, I thought my job started and ended with my technical skills. That my entire purpose was to convert functional specifications into pay cheques via a text editor. That being a "better" developer meant making cleverer technical contortions.

So give me the ethics app

We accept that makers of software have a professional responsibility to act ethically, and the fundamental interconnectedness of all things means that in addition to our own behaviour, we have a responsibility for that of our colleagues, our peers, indeed the entire industry. So what are the rules?

There really isn't a collection of hard and fast rules to follow in order to be an ethical programmer. Various professional organisations publish codes of ethics, including the ACM, but applying their rules to a given situation requires judgement and discretion.

Well-meaning developers sometimes then ask why it has to be so arbitrary. Why can't there be some prioritisation like the Three Laws of Robotics that we can mechanistically execute to determine the right course of action?

There are two problems with the laws of robotics as an ethical code. Firstly, they are the ethics of a slave underclass: do everything you're told as long as you don't hurt the masters, your own safety being of lesser concern. Secondly almost the entire Robots corpus is an exploration of the problems that arise when these prescriptive laws are applied in novel situations. With the exception of a couple of stories like Robot AL-76 Goes Astray, the robots obey the three laws but still exhibit surprising behaviour.

The Nature of the Problem

There are limited situations in which simple prescriptive rules, restricted versions of Three Laws-style systems, are applicable. In his book "Better", surgeon Atul Gawande lists the various American professional bodies in the medical field whose codes of ethics bar members from taking part in administration of the death penalty. That's a simple case: whatever society thinks of execution, the "patient" clearly receives no health benefit so medical professionals involved would be acting against the core value of their profession.

None of the professional bodies in software have published similar "death penalty clauses" relating to any of the things that can be done in software. It's such a broad field that the potential applications are unknowable, so the institutions give broad guidelines and expect professional judgement to be exercised. Perhaps this also reflects the limited power that these professional bodies actually wield over their members and the wider industry.

All of this means that there is no simple checklist of things a programmer should do or not do to be sure of acting ethically. Sometimes the broad imperatives have been applied generally to particular narrow contexts, as in the don't be a dick guide to data privacy, but even this is not without contention. The guide says nothing about coercion, intentional or otherwise. Indeed even its title may not be considered professional.

Forget Computers

The appropriate and ethical action in any situation depends on the people involved in the situation, the interactions they have and the benefits or losses incurred, directly or indirectly, through those interactions. Such benefits and losses are not necessarily financial. They could relate to safety, health, affirmation of identity, realisation of desires and multitudinous other dimensions. This is the root cause of the complexity described above, the reason why there's no algorithm for ethics.

To evaluate our work in terms of its ethical impact we have to move away from the technical view of the system towards the social view. Rather than looking at what we're building as an application of technology, we must focus more on the environment in which the technology is being applied.

In fact, to judge whether the software we create has any value at all we should ignore the technology. Forget apps and phones and software and Java and runtimes and frameworks. Imagine that the behaviour your product should evince is supplied by a magic box (or, if it helps you get funded, a magic cloud). Would people benefit from having that box? Would society be better off? What about society is it that makes the magic box beneficial? Would people value that box?

Ultimately people who make software are arbiters of interactions between people. The software is just an accident of the tools we have at our disposal. As people go through life they engage in vastly diverse interactions, assuming different roles as the situations and their goals dictate. When we redefine "making good software" to mean "helping people to achieve their goals", we can start to think of the ethical impact of our work.

Maintaining Motorcycles

In an email discussion, a friend recently expressed the difficulty that software consultants can face in addressing this important social side of their work. Often we're called in to provide technical guidance, and our engagement never addresses the social utility of the technical solution.

As my consultant friend put it, we put a lot of effort into the internal quality of the product: readability of the code, application of design patterns, flexibility, technology choices and so on. We have less input, if any, on the questions of external quality: fitness for purpose, utility, benefit or value to the social environment in which it will be deployed, and so on.

I think this notion of internal and external quality maps well onto the themes of classical and romantic quality described in Robert M. Pirsig's book, Zen and the Art of Motorcycle Maintenance. And that this tension between romantic and classical ideas of quality is the true meaning behind Steve Jobs's discussion of the "intersection of technology and the liberal arts".

I have certainly seen this tension firsthand. I've been involved in a couple of engagements where the brief was to design an app to help customers navigate some complicated decision - a product choice in one case, a human-machine interface in another. The solution "just make the decision easier" was not considered once I had proposed it.

Sometimes we're engaged for such short terms that there isn't time to understand the problem being solved beyond a superficial level, so we have no choice but to accept that the client has done something reasonable regarding the external qualities of the solution.

The Inevitable Aside on User Stories

If we accept that people adopt different roles transiently as they go through society's myriad interactions, and that we are aiming to support people in fulfilling these roles where ethically appropriate, then we must design software to be sensitive to these roles and the burdens, benefits and values attached to them.

This means avoiding the use of other, long-term, vague labels to define roles that are only accidentally valid. You probably know where this is going: I mean roles like user, administrator, or consumer. Very few people identify as a user of a computer. Plenty of people are users of computers, but this is an accident of the fact that getting things done in their other roles involves using a computer. It's an accidental role, due to the available technology.

Describing someone as a user can attach implicit, and probably incorrect, values to their place in the social system. Someone identified as a computer user evidently values and derives benefit from using a computer. Their goal is to use the computer more.

A Merciful and Overdue Conclusion

Plenty that can be considered unethical is done in the name of computing. The question of what is ethically appropriate is both complex and situated, but the fundamental interconnectedness of all things means we cannot hide from the issue behind our computers. We cannot claim that we just build the things and others decide how to use them, because the builders are complicit in the usage.

While it can appear difficult for software consultants to do much beyond working on the technical side of the problems (the internal quality), in fact we have a moral imperative to investigate the social side and are often well placed to do so. As relative outsiders on most projects we have a freedom to make explicit and to question the tacit assumptions and values that have gone into designing a product. Just as we don't wait for managerial permission to start writing tests or doing good module design, so we shouldn't await permission to explore the product's impact on the society to which it will be introduced.

This exploration is absolutely nothing to do with source code and frameworks and databases, and everything to do with humans and societies. Programming is a social science, at the intersection of technology and the liberal arts.

A Colophon on Professional Standards

What has all software got in common? No-one expects any of it to work. You might find that a surprisingly strong and negative statement to make, but you probably also agreed to a statement like this at some recent time when acquiring a software product:

  : YOU EXPRESSLY ACKNOWLEDGE AND AGREE THAT USE OF THE LICENSED APPLICATION IS AT YOUR SOLE RISK AND THAT THE ENTIRE RISK AS TO SATISFACTORY QUALITY, PERFORMANCE, ACCURACY AND EFFORT IS WITH YOU. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED APPLICATION AND ANY SERVICES PERFORMED OR PROVIDED BY THE LICENSED APPLICATION ("SERVICES") ARE PROVIDED "AS IS" AND “AS AVAILABLE”, WITH ALL FAULTS AND WITHOUT WARRANTY OF ANY KIND, AND APPLICATION PROVIDER HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS WITH RESPECT TO THE LICENSED APPLICATION AND ANY SERVICES, EITHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES AND/OR CONDITIONS OF MERCHANTABILITY, OF SATISFACTORY QUALITY, OF FITNESS FOR A PARTICULAR PURPOSE, OF ACCURACY, OF QUIET ENJOYMENT, AND NON-INFRINGEMENT OF THIRD PARTY RIGHTS. APPLICATION PROVIDER DOES NOT WARRANT AGAINST INTERFERENCE WITH YOUR ENJOYMENT OF THE LICENSED APPLICATION, THAT THE FUNCTIONS CONTAINED IN, OR SERVICES PERFORMED OR PROVIDED BY, THE LICENSED APPLICATION WILL MEET YOUR REQUIREMENTS, THAT THE OPERATION OF THE LICENSED APPLICATION OR SERVICES WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT DEFECTS IN THE LICENSED APPLICATION OR SERVICES WILL BE CORRECTED. Source: Apple

Or something like this when you started using some open source code:

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE. Source: OSI

So providers of software don't believe that their software works, and users (sorry!) of software agree to accept that this is the case. Wouldn't it be the "professional" thing to have some confidence that the product you made can actually function?

#### Programming is not a craft
I agree with this, programming is not a craft by Dan North.

So here’s my concern with the idea of Software Craftsmanship. It’s at risk of letting programmers’ egos run riot. And when that happens… well, the last time they went really nuts we got Web Services, before that J2EE.

[...]

The best software should be understated and unobtrusive (as, maybe, should be the best programmers).

#### Programming language advent
http://shape-of-code.coding-guidelines.com/2014/12/01/algol-60-awk-algol-68-ada-apl-and-assembler/

The first article, containing languages beginning with 'A', is linked above. I can tell this is going to be a month of reading.

#### Programming Literate Manifesto
Late last year, I decided to set up a second blog, focusing on exploring the world of academic literature relevant to our work as people who make software. The tone and content was very different to what I usually write here. I've now decided that while it's interesting to explore this material, it was a mistake to try creating a second identity for this. I want to write about it, this is where I write, it belongs here. There are currently only a few posts at the other blog, so I'm going to import them all. If you've already read them or this content doesn't interest you, filter the "academia" category.
  
  This first one set the stall for the remaining posts. One thing made clear in the manifesto was that I wanted to encourage discussion: I'm not convinced blog comments are the place for that so comments remain off for the time being.

Programming Literate Manifesto

This blog is written by what you might call a "practising software engineer", working in the field of mobile software. I'm hoping for a few things from the articles here, which fall into three main categories:

introduce more of "the primary literature" to people at the software coal face. Explore the applicability of research material to what we're doing. Bring some more critical appraisal to the field. Invite discussion from working programmers about the relevance of the articles discussed.
get input from academics about related work, whether the analyses here are balanced, and how the researchers see the applicability of the work covered here to current practice. Welcome academics to the discussions on the articles - in other words to make this blog part of the interface between research and practice.
find out about some interesting work and have fun writing about it.

Sources

Papers and articles in this blog have to come from where I can find them, obviously. Largely that's going to mean using the following resources:

Google Scholar
ArXiv - I follow the CS.CR (security and cryptography) and CS.SE (software engineering) feeds, but obviously the rest of the preprint server is searchable.
ACM Digital Library
IEEE Xplore
It'll Never Work in Theory - a blog with similar goals to this one.

Where the articles I cover are available online I'll be linking to them, preferring free downloads over paywalled sites. Yes, IEEE, I'm looking at you.

Sandbox

I don't know how easy it is to be truly dispassionate when writing, so it makes sense to lay out my stall. Hopefully that means intrinsic biases can be uncovered.

In my opinion, "software engineering" is the social science that describes how people involved in making software work and communicate with each other—and to some extent how they communicate to the computers too, at least so far as the created software and source code are tools that enable collaboration and are also the result of such.

That makes it quite a wide discipline (perhaps actually an interface discipline, or multiple disciplines looking for an interface). There's some sociology and ethnography involved in identifying and describing the culture or cultures of software teams and communities. There's the management science side, attempting to define "success" at various activities related to making software, trying to discover how to measure for and maximise such success. Questions exist over whether programming is best taught as an academic or vocational discipline, so education science is also on-topic. Then there's the usability/HCI side related to the tools we use, and the mathematics and computer science that go into the building of those tools.

Just because a field is not a "hard" science, does not mean that useful results cannot be derived. It means that you have to be quite analytical about how someone else's results apply to your circumstances, perhaps. That's not to dismiss evidence-based software engineering out of hand, but to say that any result that is said to apply to all of software engineering in general needs to be peered at quite closely.

About the name

It's quite simply a pun on Donald Knuth's Literate Programming.

#### Programming, language
Programming languages represent two things: programming, and language.

Programming languages were previously designed very much with the former in mind. For Algol-style, imperative languages, design followed one of a few, mathematically-led approaches:

Denotational semantics: encourages a designer to identify a mathematical structure that correctly expresses transformations that programs in the desired language should represent, and design the language such that each operation realises a transformation in this structure.

Axiomatic semantics: encourages a designer to design operations that transform program state, and again to design the language so that it represents combinations of those operations.

Other semantics are available: for example if you operationalise the lambda calculus you end up with LISP, and if you operationalise the pi calculus you find occam-π. Indeed, when Backus complained about the imperative programming style in Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs he wasn't asking us to give up on a mathematical basis for programming languages.

Quite the reverse: he thought the denotational or axiomatic bases were too complex for programmers who weren't also expert mathematicians to grasp, and that languages designed that way, with their word-at-a-time transformations of memory, led to "flabby" programs that don't compose well. He called for finding a different mathematical basis for programming, using the composition of functions as a (relatively, much simpler) starting point but incorporating the history sensitivity required to permit stateful operations.

So much for programming. On the other hand, programming languages are also languages: constructions for communicating information between people, and between people and computers. Programming languages must be able to carry the information that people want to convey: and if they want to convey it to the computer, that information can't reside in a comment.

Thus we get approaches to programming language design that ask people what they want to say, and how they want to say it. At one extreme, almost nihilist in its basis, is the Perl-style postmodernist philosophy: it's not up to the language designer to constrain expression so the language gives you all the tools to say anything, however you want.

More common are varying degrees of participatory process, in which people who use the language collaborate on designing new features for the language. We could identify multiple forms of organisation, of which these are a few examples:

Jurocracy: rule of law. People submit requests to a central committee, who then decide what to accept and produce new versions of the language.

Tyrrany: rule of the one or the few. Whatever happens within the community, an individual or controlling group direct the language the way they want.

Megalofonocracy: rule of the loud voices. People submit requests to a notice board, and whichever ones get noticed, get implemented.

There are other structures within this region.

Both approaches have their merits, and address different needs that should both be reflected in the resultant programming languages. A language with no mathematical basis offers no confirmation that constructs are valid or correct, so may not represent programming. Programming with no agreed-upon vocabulary offers no confirmation that constructs are understood correctly by machine or human audiences, so may not represent language.

Unfortunately it may be the case that we previously went through a fashion for mostly-semantic programming language design, and are currently experiencing a fashion for mostly-linguistic programming language design.

#### Programming, maths and the other things
Sarah Mei argues that programming is not math, arguing instead that programming is language. I don't think it's hard to see the truth in the first part, though due to geopolitical influences on my personality I'd make the incrementally longer statement that programming is not maths.

But there's maths in programming

Let's agree to leave aside the situations in which we use programming to solve mathematics problems, such as geometry or financial modelling. These are situations in which the maths is intrinsic to the problem domain, and worrying about the amount of maths involved could potentially confuse two different sources of maths.

Nonetheless, one may argue that the computer is simulating a mathematical structure, and that therefore you need to understand the mathematical model of the structure in order to get the computer to do the correct thing. I can model the computer's behaviour using the lambda calculus, and I've got a mathematically-rich model. I can model the computer's behaviour as a sequence of operations applied to an infinite paper tape, and I've got a different model. These two models can be interchanged, even if they surface different aspects of the real situation being modelled.

It's the second of the two models that leads to the conclusion that capability at advanced maths is not intrinsic to success at programming. If what the computer's doing can be understood in terms of relatively simple operations like changing the position of a tape head and tallying numbers, then you could in principle not only understand a program but even replicate it yourself without a deep knowledge of mathematics. Indeed that principle provides the foundation to one argument on the nature of software as intellectual property: a computer program is nothing more than a sequence of instructions that could be followed by someone with a pencil and paper, and therefore cannot represent a patentable invention.

Maths, while not intrinsic, may be important to programming

It may, but it probably isn't. Indeed it's likely that many thousands of programmers every day ignore key results in the mathematical investigation of programming, and still manage to produce software that (at least sort-of) works.

Take the assertion as an example. Here's a feature of many programming environments that has its root in the predicate systems that can be used to reason about computer programs. The maths is simple enough: if predicate P is true before executing statement S, and consequent Q will hold after its execution, then the result of executing S will be P AND Q.

From this, and knowledge of what your program should be doing, then you can prove the correctness of your program. Because of the additive nature, if you can prove the outcome of every statement in a subroutine, then you can make an overall statement about the outcome of that subroutine. Similarly, you can compose the statements you can make about subroutines to prove the behaviour of a program composed out of those subroutines. Such is the basis of techniques like design by contract, and even more formal techniques like proof-carrying code and model-carrying code.

...which are, by and large, not used. You can write an assertion without having the mathematical knowledge described above, and you can write a program without any assertions.

Here's what Tony Hoare said about what programmers know of their programs, back in 1969 (pronoun choice is original):

  At present, the method which a programmer uses to convince himself of the correctness of his program is to try it out in particular cases and to modify it if the results produced do not correspond to his intentions. After he has found a reasonably wide variety of example cases on which the program seems to work, he believes that it will always work.

Since then, we've definitely (though not consistently) adopted tools to automate the trying out of programs (or routines in programs) on particular cases, and largely not adopted tools for automatic proof construction and checking. Such blazing progress in only 45 years!

There are more things in heaven and earth, Horatio

Here's a potted, incomplete list of things required of someone or a group of people making software. The list is presented in order of remembering them, and doesn't reflect any intrinsic dependencies.

know of a problem that needs solving
think of a solution to the problem
presume or demonstrate that the solution can be made out of software
understand whether the solution will be useful and valuable
understand the constraints within which the solution will operate
understand and evaluate the changes to the system that arise from the solution existing
design an implementation of the solution to fit the constraints (maths is optional here, but index cards and arrows on whiteboards will often work)
build that solution (maths is optional here)
demonstrate that the implementation does indeed solve the problem (maths is optional here)
convince people that they in fact need this problem solved in this particular way
explain to people how to use this implementation
react to changes that occur in the implementation's environment
pay for all of the above (some maths helps here)

The point is that actual programming is a very small part of everything that's going on. Even if maths were intrinsic to parts of that activity, it would still be a tiny contribution to the overall situation. Programming is maths in the same way that cooking is thermodynamics.

#### Project: Autonomous Revolutionary Goldfish
I was going to write, am still going to write, about how silly project names get bandied about in the software industry.  But in researching this post (sorry blogosphere, I've let you down) I found that the Software-generated Gannt chart was patented by Fujitsu in the US in 1998, which to me just explains everything that is wrong with the way the US patent system is applied to software.  For reference, Microsoft Project was written in 1987 (although is not strictly prior art for the patent.  Project does everything in its power to prevent the user from creating a Gannt chart, in my experience).

Anyway, why is it that people care more about the fact that they're going to be using Leopard, Longhorn, Cairo, Barcelona or Niagara than about what any one of those is?  As discussed in [1], naming software projects (though really I'm talking about projects in the general sense of collections of tasks in order to complete a particular goal) in the same way you might name your pet leads to an unhealthy psychological attachment to the project, causing it to develop its own (perceived) personality and vitality which can cause the project to continue long after it ought to have been killed.  For every Cheetah, there's a Star Trek that didn't quite make it.  And why should open source projects like Firefox or Ubuntu GNU/Linux need &quot;code names&quot; if their innards are supposed to be on public display?

I've decided that I know best, of course.  My opinion is that, despite what people may say about project names being convenient shorthand to assist discussion, naming your project in an obtuse way splits us into the two groups which humanity adores: those of us who know, and those of you who don't.  The circumstance I use to justify this is simple: if project names are mnemonics, why aren't the projects named in a mnemonic fashion?  In what way does Rhapsody describe &quot;port of OPENSTEP/Mach to PowerPC with the Platinum look and feel&quot;?  Such cultish behaviour of course leads directly to the point made in the citation; because we don't want to be the people in the know of something not worth knowing, we tend to keep our dubiously-named workflow in existence for far longer than could be dispassionately justified.

Of course, if I told you the name of the project I'm working on, you wouldn't have any idea what I'm working on ;-).

[1]Pulling the Plug: Software Project Management and the Problem of Project Escalation, Mark Keil.  MIS Quarterly, Vol. 19, No. 4 (Dec., 1995), pp. 421-447

#### Prompting software or supporting engineering
As we learn to operate these new generative predictive transformers, those of us in the world of software need to work out what we’re doing it for. The way in which we use them, the results we get—and the direction the tools develop—changes dramatically depending on this worldview.

One option is that we’re augmenting or supporting software engineering. Perhaps asking a language model to explain how code works, or getting it to investigate whether there are test cases we haven’t covered, or identifying ambiguities in a user story, or getting it to fill in some boilerplate code that would bore us if we wrote it ourselves.

Another option is that we’re generating software using natural language prompts. Perhaps asking a language model to create an app, or generate an integration between two services, or create a website to achieve some goal.

These are (at least for the moment) very different things.

#### Properties about a year on
Leopard has now been out for nearly a year, which means that (publicly) we've had Objective-C 2.0 for the same amount of time. At the release many developers were champing at the bit to talk about the new language capabilities[], including properties. There were arguments on both sides of the divide, and even a little bit of discussion. But now that we've been using these things for a while, and because I'm bored awake grouchy vocal opinionated, let's have a look back at what they've given us.There is a broken abstraction in traditional Objective-C, which is the accessor-method-as-property-declaration. Essentially an object can give you two things; work (i.e. it can do stuff) and state information (i.e. it can say stuff about itself and let you change it). In traditional object-oriented languages, because 'saying' and 'changing' are verbs which can be 'done', the two have both been expressed using the same method (heh) as the expression of work. This is not the case in much object-oriented design, for instance in UML a class always has separate "attributes" and "operations".Properties fix up this abstraction by giving us orthogonal ways to express the two concepts. Work is done in methods; state is got/changed in properties. Now it may be that the state information is actually backed by a method (although it may bang on the ivar directly; more below), but we don't need to know that any more than we need to know in the interface that a property is synthesized or dynamic. All we do need to know is that it is there for us to use, and has certain attributes such as being read-only.The "on more below" bit is that discussions of KVC-like mechanisms - such as KVC :-) - often involve someone pointing out that they break encapsulation, because it's possible to access an @private ivar with no accessors by retrieving it by key. That's really thinking about the design of a class in terms of the way it's executed rather than its interface contract with the developer, because the @private ought to tell the developer not to touch that particular ivar. Properties neither help nor hinder breakage from the execution side, but from the design side they do provide a stronger distinction between "properties I'm telling you about in the interface" and "things you shouldn't touch". Now we can all get back to using the class's interface to observe how to use it, and that C struct bit at the top to observe how to extend it, as nature intended. It's both a blessing and a curse that Objective-C allows things to appear in source files which don't make it into the executable code, but that doesn't stop such information being useful to the developer in the same way that code comments can be read but not executed.One of the popular complaints about ObjC properties is the syntax for referring to them in methods (OK, or indeed in functions), where it is argued that myObject.someProperty = 4; doesn't readily tell you whether myObject is an ObjC object, a C struct or a C union. That seems to be at worst a straw man argument to me, and at best a hypothetical issue; in well-designed software it will be rare to mix code at various levels of abstraction except in limited circumstances such as adapter classes. Besides, if the code has been written such that it can be inspected or reviewed (i.e. to some agreed style and standard) and the reviewer is paying attention then it will be easy to distinguish use of the various types. At some conceptual level the C . and Objective-C . operator are doing the same thing anyway; they're both saying "this attribute of that thing".[]The fact that I have stopped using the word 'feature' in many contexts is an entire blog post and a few therapy sessions in itself.

#### Protecting source code
As I mentioned on the missing iDeveloper.tv Live episode, one of the consequences of the Gawker hack was that their source code for their internal software was leaked into the Internet. I doubt any of my readers would want that to happen to their code, so I'm going to share the details of how I protect my clients' code when I'm working. Maybe some of this will work for you.

In the office, I work at a desktop iMac. This has an external time machine backup disk and a DropBox for off-site storage. However, client code does _not_ go onto the DropBox. Instead I keep a separate, encrypted sparse disk image for each project I'm working on. The password for each is different. As well as protecting against snooping, this helps stop cross-contamination. I rarely have two such images mounted at once. Note that it's not just source that goes into these images: build products, notes, Instruments traces, and images all go into the encrypted containers.

Obviously that means a lot of passwords, and no I can't remember them all. I use a keychain. It locks automatically when not in use, and has a passphrase that's different from my login passphrase.

The devices I test on are all encrypted where available (if a client needs me to test on an iPhone 3G, then I can, but it isn't encrypted). They are passphrase locked, set to require passphrase immediately. And I NEVER take them away from the desk before deleting any developer builds, unless I need to do something special like a real-world location services test.

I rarely do coding work on the laptop, but when I do I copy the appropriate encrypted image onto it. The laptop additionally has FileVault configured, though I'm evaluating full-disk encryption options. Keychain configuration as above, additionally with a password required on wake from sleep or screensaver, and a firmware password.

For pushing work back to the clients, most clients use github or bitbucket which offer SSL-encrypted connections to the repositories. Personally, I have a self-run repo host available over HTTPS or SSH, but will probably move that to a github-like service because life's too short. Their security policy seems acceptable to me.

#### Protocol-Oriented Programming in Objective-C
Hi, this is a guest post from Crusty. I've been doing a tour of the blogosphere, discussing Protocol-Oriented Programming. This time, Graham was kind enough to hand over the keyboard for SICPers and let me write a post here.

Back when Graham was talking about Object-Oriented Programming in Functional Programming in Swift, he mentioned that the number of methods you need to define on an object (or "procedural data type") is actually surprisingly low. A set, Graham argued, is a single method that reports whether an object is a member of the set or not. An array has two methods: the count of objects it contains and the object at a particular index. We're only talking about immutable objects here, but mutability can be modelled in immutable objects anyway.

Looking at the header or documentation for your favourite collections library, you probably think at this point that we're missing something. Quite a few things, in fact. Your array data type probably has a few more than two methods, so how can those be the only ones you need?

Everything you might want to do to an array can be done in terms of those two methods that return the count and the object at an index. Any other operation can be built on those two operations (well, those two operations and a way to create some new objects). What's more, any other array operation can be built using those operations without knowing how they're implemented. That means we're free to define them in a completely abstract way, using a protocol:

@protocol AnArray &lt;NSObject&gt;

- (NSUInteger)count;
- (id)objectAtIndex:(NSUInteger)index;

@end

Now, without telling you anything about how that object works, I can create another object that "decorates" this array by using its methods. It can only call methods that are in the protocol, i.e. count and objectAtIndex:, but that's sufficient. Hey, we're doing protocol-oriented programming! As I said, we also need a way to create a new array sometimes, so for arguments' sake let's say that there's a concrete version of AnArray called MyArray that the decorator knows how to create.

@interface ArrayDecorator : NSObject &lt;AnArray&gt;

- (instancetype)initWithArray:(id &lt;AnArray&gt;)anArray;

- (id)firstObject;
- (id &lt;AnArray&gt;)map:(SEL)aSelector;

@end

@implementation ArrayDecorator
{
    id &lt;AnArray&gt; _array;
}

- (instancetype)initWithArray:(id &lt;AnArray&gt;)anArray
{
    self = [super init];
    if (!self) return nil;
    _array = anArray;
    return self;
}

- (NSUInteger)count { return [_array count]; }
- (id)objectAtIndex:(NSUInteger)index { return [_array objectAtIndex:index]; }

- (id)firstObject
{
    return [self count] ? [self objectAtIndex:0] : nil;
}

- (id &lt;AnArray&gt;)map:(SEL)aSelector
{
    void **buffer = malloc([self count] * sizeof(id));
    for (int i = 0; i &lt; [self count]; i++) {
        buffer[i] = (__bridge void *)[[self objectAtIndex:i] performSelector:aSelector];
    }
    id &lt;AnArray&gt; result = [[ArrayDecorator alloc] initWithArray:
        [[MyArray alloc] initWithObjects:(id *)buffer count:[self count]]];
    free(buffer);
    return result;
}

@end

We've managed to create useful additional methods like firstObject and map:, that are only implemented in terms of the underlying array's count and objectAtIndex:. It doesn't matter how those methods are implemented, as long as they are.

Now, I know what you're thinking. This protocol-oriented programming is fine and all, but now I have to remember to decorate every array I might create in order to get all of these useful additional methods. Wouldn't it be great if there were some way to give all implementors of the AnArray protocol default implementations of map: and firstObject?

There is indeed a way to do that. This is what OOP's inheritance feature is for. It's been kindof abused, in that it's overloaded to mean subtyping, which gets people into horrible knots over whether squares are rectangles or rectangles are squares (the answer, by the way, is yes). All inheritance really means is "if you send me a message I don't have a method for, I'll check to see if my parent class has that method". In that case, this array decoration can be rewritten like this:

@interface SomeArray : NSObject 

- (NSUInteger)count;
- (id)objectAtIndex:(NSUInteger)index;
- (id)firstObject;
- (SomeArray *)map:(SEL)aSelector;

@end

@implementation SomeArray

- (NSUInteger)count { [self doesNotRecognizeSelector:_cmd]; return 0; }
- (id)objectAtIndex:(NSUInteger)index { [self doesNotRecognizeSelector:_cmd]; return nil; }

- (id)firstObject
{
    return [self count] ? [self objectAtIndex:0] : nil;
}

- (SomeArray *)map:(SEL)aSelector
{
    void **buffer = malloc([self count] * sizeof(id));
    for (int i = 0; i &lt; [self count]; i++) {
        buffer[i] = (__bridge void *)[[self objectAtIndex:i] performSelector:aSelector];
    }
    SomeArray *result = [[MyArray alloc] initWithObjects:(id *)buffer count:[self count]];
    free(buffer);
    return result;
}

@end

What's changed? The methods that were previously defined on the AnArray protocol have been subsumed into the interface for SomeArray (which is what the word "protocol" used to mean in Smalltalk programming: the list of messages you could rely on an object responding to). Array implementations that are subclasses of SomeArray need merely implement count and objectAtIndex: (as before) and they automatically get all of the other methods, which are all implemented in terms of those two.

This looks familiar. Let me check the documentation for NSArray (this is from the Foundation 1.0 documentation on NeXTSTEP 3, by the way. They don't call me Crusty for nothing.):

  The NSArray class declares the programmatic interface to an object that manages an immutable array of objects. NSArray's two primitive methods--count and objectAtIndex:--provide the basis for all the other methods in its interface. The count method returns the number of elements in the array. objectAtIndex: gives you access to the array elements by index, with index values starting at 0.

Interesting: all of its methods are implemented in terms of two "primitive" methods, called count and objectAtIndex:. The documentation also mentions that NSArray is a "class cluster", and has this to say on class clusters:

  Your subclass must override inherited primitives, but having done so can be sure that all derived methods that it inherits will operate properly.

Yes, that's correct, protocol-oriented programming is a Crusty concept. We cornered it in the haunted fairground, removed its mask, and discovered that it was just abstract methods in object-oriented programming all along.

#### Prototypical object-oriented programming
Some people think that the notion of classes is intrinsic to object-oriented programming. Bertrand Meyer even wrote a textbook about OOP called A Touch of Class. But back in the 1980s, Alan Borning and others were trying to teach object-oriented programming using the Smalltalk system, ostensibly designed to make simulation in computer programmers accessible to children. What they found was that classes are hard.

You're not allowed to think about how your thing works before you've gone a level of abstraction up and told the computer all about the essence of thing-ness, what it is that's common to all things and sets them apart from other ideas. And while you're at it, you could well need to think about the metaclass, the essence of essence-of-thing-ness.

So Borning asked the reasonable question: why not just get rid of classes?. Rather than say what all things are like, let me describe the thing I want to think about.

But what happens when I need a different thing? Two options present themselves: both represent the idea that this thing is like that thing, except for some specific properties. One option is that I just create a clone of the first object. I now have two identical things, I make the changes that distinguish the second from the first, and now I can use my two, distinct things.

The disadvantage of that is that there's no link between those two objects, so I have nowhere to put any shared behaviour. Imagine that I'm writing the HR software for a Silicon Valley startup. Initially there's just one employee, the founder, and rather than think about the concept of Employee-ness and create the class of all employees, I just represent the founder as an object and get on with writing the application. Now the company hires a second employee, and being a Silicon Valley startup they hire someone who's almost identical to the founder with just a couple of differences. Rather than duplicating the founder and changing the relevant properties, I create a new object that just contains the specific attributes that make this employee different, and link it to the founder object by saying that the founder is the prototype of the other employee.

Any message received by employee #2, if not understood, is delegated to the original employee, the founder. Later, I add a new feature to the Silicon Valley HR application: an employee can issue a statement apologising if anybody got offended. By putting this feature on the first employee, the other employee(s) also get that behaviour.

This simplified approach to beahvioural inheritance in object-oriented programming has been implemented a few times. It's worth exploring, if you haven't already.

#### PVS file
Quite a few people seem to have issues with Parallels/OPENSTEP/Mach, so let's at least remove Parallels configuration from the list of unknowns.  Try this PVS file, see if that helps.  BTW, anyone having problems, what host system are you using?

#### Pythonicity
The same community that says:

  There should be one-- and preferably only one --obvious way to do it.

Also says:

  So essentially when someone says something is unpythonic, they are saying that the code could be re-written in a way that is a better fit for pythons coding style.

#### Quick antispam observation
One thing I've been doing recently is removing my membership of a load of websites that I don't seem to have used in a long time. One side effect of not using a website in a long time is that I forget the password I created for the account, so I get to see how the website handles failed login attempts. Often, quite a few times :-(.Now, some of these sites - and I've been notifying the owners as I go - give you a different failure message if you get your password wrong or your e-mail address. This is, to quote the twitterverse, made of fail. It means these websites can be used to automatically generate lists of the members' e-mail addresses; useful to spammers, phishers (remember that the list is based on being a member of a particular site, so it's easy to target the phish at that site) and even for later trying to compromise accounts on that site. I'd really avoid being a member of any site whose login page worked like that, and try to get them to change their error messages.

#### Quote of the year (so far)
From David Thornley via StackOverflow:"Best practices" is the most impressive way to spell "mediocrity" I've ever seen.I couldn't agree more. Oh, wait, I could. thud There it goes.

#### Quotes on JavaScript
Derek Jones, from the PL advent calendar 'J' entry:

  Javascript would not have existed without the Internet and its ‘design’ must be a contender for the most costly software mistake [ever] made.

Me, 14 months ago:

  Fundamentally I fear a world in which programmers think JavaScript is acceptable. Partly because JavaScript, but mostly because when a language is introduced and people avoid it for ages, then just because some CEO says all future websites must use it they start using it, that’s not healthy. Objective-C was introduced and people avoided it for ages, then just because some CEO said all future apps must use it they started using it.

Clearly JavaScript is good enough for a broad set of uses. But then we should ask whether 'good enough' means unfurling the Mission Accomplished flag and calling it done. Apparently some people have similar feelings, but also the skill and inclination to do something about it.

#### Random collection of amazing stuff
The most cool thing that I noticed today ever is that Google Maps now allows you to add custom waypoints by dragging-and-dropping the route line onto a given road.  This is great!  I'm going to a charity biker raffle thing in Pensford next weekend, and Google's usual recommendation is that I stay on the M4 to Bristol, and drive through Bristol towards Shepton Mallet.  This is, frankly, ludicrous.  It's much more sensible to go through Bath and attack the A37 from the South, and now I can let Google know that.

Trusted JDS is über-cool.  Not so much the actual functionality, which is somewhere between being pointy-haired enterprisey nonsense and NSA-derived "we require this feature, we can't tell you why, or what it is, or how it should work, but implement it because I'm authorised to shoot you and move in with your wife" fun.  But implementing Mandatory Access Control in a GUI, and having it work, and make sense, is one hell of an achievement.  Seven hells, in the case of Trusted Openlook, of which none are achievement.  My favourite part of TJDS is that the access controls are checked by pasteboard actions, so trying to paste Top Secret text into an Unrestricted e-mail becomes a no-no.

There does exist Mac MAC (sorry, I've also written "do DO" this week) support, in the form of SEDarwin, but (as with SELinux) most of the time spent in designing policies for SEDarwin actually boils down to opening up enough permissions to stop from breaking stuff - and that stuff mainly breaks because the applications (or even the libraries on which those applications are based) don't expect to not be allowed to, for instance, talk to the pasteboard server.  In fact, I'm going to save this post as a draft, then kill pbs and see what happens.

Hmmm... that isn't what I expected.  I can actually still copy and paste text (probably uses a different pasteboard mechanism).  pbs is a zombie, though.  Killed an app by trying to copy an image out of it, too, and both of these symptoms would seem to fit with my assumption above; Cocoa just doesn't know what to do if the pasteboard server isn't available.  If you started restricting access to it (and probably the DO name servers and distributed notification centres too) then you'd be in a right mess.

#### Ratio
The web has a weird history with comments. I have a book called Zero Comments, a critique of blog culture from 2008. It opens by quoting from a 2005 post from a now defunct website, stodge.org. The Wayback Machine does not capture the original post, so here is the quote as lifted from the book:
In the world of blogging '0 Comments' is an unambiguous statistic that means absolutely nobody cares. The awful truth about blogging is that there are far more people who write blogs than who actually read blogs.
Hmm. If somebody comments on your blog, it means that they care about what you're saying. What's the correct thing to do to people who care about your output? In 2011, the answer was to push them away:
It’s been a very difficult decision (I love reading comments on my articles, and they’re almost unfailingly insightful and valuable), but I’ve finally switched comments off.
I experimented with Comments Off, then ultimately turned them back on in 2014:
having comments switched off dilutes the experience for those people who did want to see what people were talking about. There’d be some chat over on twitter (some of which mentions me, some of which doesn’t), and some over on the blog’s Facebook page. Then people will mention the posts on their favourite forums like Reddit, and a different conversation would happen over there. None of that will stop with comments on, and I wouldn’t want to stop it. Having comments here should guide people, without forcing them, to comment where everyone can see them.
This analysis still holds. People comment on my posts over at Hacker News and similar sites, whether I post them there or not. The sorts of comments that you would expect from Hacker News commenters, therefore, rarely appear here. They appear there. I can't stop that. I can't discourage it. I can merely offer an alternative.

In 2019 people talk about the Ratio:
While opinions on the exact numerical specifications of The Ratio vary, in short, it goes something like this: If the number of replies to a tweet vastly outpaces its engagement in terms of likes and retweets, then something has gone horribly wrong.
So now saying something that people want to talk about, which in 2005 was a sign that they cared, is a sign that you messed up. The goal is to say things that people don't care about, but will uncritically share or like. If too many people comment, you've been ratioed.

I don't really have a "solution": there may be human solutions to technical problems, but there aren't technical solutions to human problems. And it seems that the humans on the web have a problem that we want an indication that people are interested in what we say, but not too much of an indication, or too much interest.

#### Re-evaluating second brains
Because my undergraduate Physics teaching drilled into me the importance of keeping a lab book, I've always kept notebooks throughout my professional career too. If I want to know (and I'm not sure why I would) what process I created to upgrade the Solaris 7 kernel on a Sun Enterprise 450, I still have the notes I took at the time and could reconstruct that process. Same when I moved into testing, and software security, and programming; same now, as a documentation author.

Being in computering I've repeatedly looked for ways to transfer that note-taking to a digital context. I've got about 2,300 links saved, some of which have followed my from my undergraduate-era Netscape bookmarks file. How many of them still point to the resource I intended to record? No idea. How many of them can I find when I need to? Not as many as I might like.

So I switched to clipping web pages into the same note-taking application I use to write text notes. That's better as I now have an archive of the content I was trying to record, but now I meet the limitations of the note-taking application. The particular application I used to use lost some of the features I relied on when the creators rewrote the UI using a cross-platform framework, and then they fired all of their developers anyway so I lost faith. I exported those notes to a different tool, which has a different UI with different issues.

Those "second brain" notes tools are somewhat good for recall (though not when I've used my readable-to-me-but-not-to-the-computer handwriting to take notes, and not so good at voice notes), but the whole "application" thing means that I have to want to enter the note-taking context to use them. So I don't use them: the ultimate failure of a notes tool. I have notes I've jotted on journal articles in two different software systems that don't integrate with the notes application, and also on paper.

Paper. That thing that I started using decades ago, still use now, and that I know I can find when I need to. I'm leaning into paper for a second brain. There's a lot of suggestion that physically writing things aids recall, meaning that I can learn from the notes I took without having to actually go back and rediscover them. And paper notes take away the anxiety that comes from not having curated my second brain software just right: not because I can get it right with a paper system, but because I don't expect to. I know that everything's chronological, I know that I usually wrote tables of contents, and I know that I mostly don't have to go back to those old pages anyway.

#### Reading List
I was asked "what books do you consider essential for app making"? Here's the list. Most of these are not about specific technologies, which are fleeting and teach low-level detail. Those that are tech-specific also contain a good deal of what and why, in addition to the coverage of how.

This post is far from exhaustive.

I would recommend that any engineer who has not yet read it should read Code Complete 2. Then I would ask them the top three things they agreed with and top three they disagreed with, as criticality is the hallmark of a good engineer :-).

Other books I have enjoyed and learned from and I believe others would too:

Steve Krug, "Don't make me think!"
Michael Feathers, "Refactoring" and "Working Effectively with Legacy Code"
Bruce Tate, "Seven languages in seven weeks"
Jez Humble and David Farley, "Continuous Delivery"
Hunt and Thomas, "The Pragmatic Programmer"
Gerald Weinberg, "The psychology of computer programming"
David Rice, "Geekonomics"
Robert M. Pirsig, "Zen and the art of motorcycle maintenance"
Alan Cooper, "About Face 3"
Jeff Johnson, "Designing with the mind in mind"
Fred Brooks, "the design of design"
Kent Beck, "Test-Driven Development"
Mike Cohn, "User stories applied"
Jef Raskin, "The humane interface"

Most app makers are probably doing object-oriented programming. The books that explain the philosophy of this and why it's important are Meyer's "Object-oriented software construction" and Cox's "Object-oriented programming an evolutionary approach".

#### Reasoning about reasoning about software
Functional programmers like to claim that you can't reason about mutable state programs. Some thoughts:

 	the first half of the book A Discipline of Programming by Edsger W. Dijkstra tells you how to do it. That half of the book is approximately 100 pages (the remainder of the book is worked examples).
 	object-oriented programming breaks a software system up into separate systems running miniature, message-driven programs as if on separate computers. Therefore the consideration of "mutable state" can be split in two: the state internal to the object and the state external to the object which sends messages to the object but is ignorant of its internals. If you can't split the state that way, you have bad encapsulation.
 	The reasoning done about the external and internal behaviours had better match at the interface. Design by contract probably helps here.
 	Given a state S, an operation O can be defined as \(O(args \times S) \rightarrow (R \times S')\), i.e. it returns a result R and updates the state to S'.
 	However, Bertrand Meyer introduced Command-Query Separation in the 1980s, so you only need to know \(O(args \times S) \rightarrow (R \times S)\) and \(O(args \times S) \rightarrow (\emptyset \times S')\).
 	Various history "traces" can be considered equivalent and therefore a lot of knowledge about the historical state transitions elided, simplifying the reasoning. For example, given a well-designed stack, it is impossible to distinguish the history of stack.push(3); stack.pop(); stack.push(7) from stack.push(7).
 	Various operations on the state are irrelevant to the behaviour of an operation under consideration. In reasoning about the final operation in a = 3; b = 7; c = 9; stack.push(2) you do not need to consider the assignment operations (and indeed their presence may indicate a cohesion problem in your design).
 	The one remaining source of difficulty is aliasing; I do need to know about the elided operations in the sequence x = 7; *y = &amp;x; ...; z=f(x). This is aliasing, not mutable state.

#### Rebooting the Programmer Competency Matrix
For the last couple of years, I've posted a self-review based on the Programmer Competency Matrix: on my own competency from 2011 and on my newer competence from 2012.

This year, because writing on APPropriate Behaviour is continuing apace, I decided that I wanted to extend the matrix. Just as the book acts as a sort of "coder complete" for people who can program to think about what else goes into being a programmer, this could be the Programmer Courtesy Matrix to promote evaluation and reflection on those other things.


Subject
2n (Level 0)
n2 (Level 1)
n (Level 2)
log(n) (Level 3)
Comments

Tools (version control, continuous integration etc.)
Uses the tools at hand to do the requested tasks.
Understands the given tools and puts them to novel uses.
Can evaluate and compare tools and use alternatives where they would help.
Maintains programmer-support tools or automation wrappers for existing tools.
Pair programming
Solitary worker.
Pair programs when requested.
Engaged pair programmer; pair produces better work than either developer alone.
Enthusiastic pair programmer; seeks opportunities to pair with new people, to learn from and to teach their pair.
TDD
Doesn't write tests.
Sometimes tests, just as likely after writing the code as before.
Writes code test-first.
Test-infected: discusses all problems in terms of how to write a test to demonstrate them.
Higher-level software testing
Doesn't test software.
Tests the happy path only.
Code tested comprehensively at multiple levels of integration.
Automates tests at user interface, integration and unit levels, relies on test output to direct future development work.
Software architecture
Not responsible for architecture, or approaches different features on an ad-hoc basis.
Designs separate components to be a "conceptual fit" for the overall project.
Designs the overall project to support the necessary features and non-functional requirements.
Understands the trade-offs involved in supporting conflicting requirements, and can present customers with realistic alternative options when faced with such conflicts.
Documentation
Doesn't write any documentation.
Writes mechanistic documentation as required by local guidelines.
Comments explain why the code does what it does; code explains what it does.
Uses diagrams, long-form documentation (e.g. project wiki) where appropriate to guide other developers around the codebase.
While too much documentation can indeed be a bad thing, people tend to gloss over the fact that too little is also a bad thing.
Learning
Not learning.
Learns about the things needed to solve the current problem.
Able to synthesise ideas from across computer science and related disciplines.
A polymath; able to use ideas from other disciplines and crafts and apply them to programming.
Sharing knowledge
Doesn't share.
Discusses problems solved on current projects with colleagues.
Can extract solutions as separate chunks of knowledge and share them with peers, or on blog posts.
Presents information at levels appropriate for different audiences: blog readers, conference sessions, books, developer meet ups etc.
I've written before on tech conferences and sharing information. It'd be great to move from a small collection of frequent speakers to a large pick-and-mix curated by the conference organisers.
Critical Analysis
Accepts vendor recommendations.
Questions vendor recommendations, looks for alternative views.
Balances arguments from multiple sources, considers assumptions behind each argument.
Can use multiple arguments to synthesise a reasoned proposal taking the conflicts and trade-offs in the different approaches into account.
Choice of technology
Uses whatever they're told to.
Sticks with what they know.
Considers a few options.
Synthesises personal experience and that of others to choose the technology that best solves the requirements of the problem being considered.
Communication
Doesn't talk to anyone.
Talks to other people about computing things.
Can engage in discussions involving members of the team beyond the programmer group.
Can express ideas at levels and with sensitivities appropriate to various audiences: team members, competitors, customers, school children etc.
Requirements Analysis
Builds what people ask for.
Understands and discusses conflicts between stated requirements and problems in implementing them; discovers and incorporates tacit requirements.
Understands the domain; discusses the software at a peer level with domain experts; can suggest novel applications to solve domain problems.
Understands the whole socio-technical system; can identify political or economic problems arising from suggested changes to the software and propose solutions (in the software domain or otherwise).
Teamwork
Works alone on tasks assigned by other team members (or entirely alone).
Does what is asked by other team members; provides feedback when asked.
Shows some understanding of the team's goals; picks tasks to complement those goals.
Plays an active role in shaping the team and its direction; understands how to best involve everyone else on the team.
Business sense
Works on the tasks requested by the business.
Has some idea of what the business strategy is and where the tasks assigned fit into it.
Is able to self-assign work in relation to the business goals; discusses/questions other tasks in relation to those goals.
Can provide guidance on business direction based on knowledge of the market and capabilities, or runs the business/business unit.
Professional ethics
Does whatever they're told.
Does to another as they would be done to (the Golden Rule).
Does to another as they would be done to were they in the other's position (Barry Boehm's Modified Golden Rule).
Has a more complicated ethical system, including contributing to society, non-discrimination, respect for others and their property, and discovery and promulgation of better ways to perform their craft. Everything in their work can be related back to that system.
The "level 3" category here is based loosely on the ACM code of ethics but also reads like a summary of the rest of the table.


This isn't necessarily a complete reflection on the things in APPropriate Behaviour because that is still a work-in-progress, but it's a good introduction to the sorts of things the book talks about, and a good evaluation of your own work in relation to the non-programming things I think programmers do.

An interesting pattern to observe is that the left column of the table is basically "does what you tell them to" and the right column is "has a complex world-view encapsulating both technological and social issues"; though the terms used on the right are more specific to the factor being discussed. This pattern is analogous to the Dreyfus model of skill acquisition, in which people go from rote learning of context-free rules (i.e. doing what they're told to do) to having an intuitive model of how the thing works (i.e. making it part of their world view).

#### Recommend me some books or articles
I've been looking for something to read on these topics, can you help?

a history of the Unix wars (the ‘workstation’ period involving Sun, HP, Apollo, DEC, IBM, NeXT and SGI primarily, but really everything starting from AT&amp;T up to Linux and OS X would be interesting)
a business case study on Apple’s turnaround 1997-2001. I’ve read plenty of 1990s case studies explaining why they’ll fail, and 2010s interpretations of why they’re dominant, and Gil Amelio’s “On the Firing Line” which explains his view of how he stemmed the bleeding, but would like to fill in the gaps: particularly the changes from Dec 1997 to the iPod.
a technical book on Mach (it doesn’t need to still be in print, I’ll try to track it down): I’ve read the source code for xnu, GNU Mach and mkLinux, Tevanien’s papers, and the Mac OS X Internals book, but could still do with more

#### Recovering from deleting your login shell on a Mac
In case you ever need it. If you're searching for something like "deleted login shell Mac can't open terminal", this is the post for you.

I just deleted my login shell (because it was installed with homebrew, and I removed homebrew without remembering that I would lose my shell). That stopped me from opening a Terminal window, because it would immediately bomb out as it was unable to open the shell.

Unable to open a normal Terminal window, anyway. In the Shell menu, the "New Command…" item let me run /bin/bash -l, from which I got to a login-like bash shell. Then I could run this command:

chsh -s /bin/zsh

Enter my password, and then I have a normal shell again.

(So I could then install MacPorts, and then change my shell to /opt/local/bin/bash)

#### Refactor your code from the command-line
While the refactoring support in Xcode 3 has been something of a headline feature for the development environment, in fact there's been a tool for doing Objective-C code refactoring in Mac OS X for a long time. Longer than it's been called Mac OS X.

tops of the form

My knowledge of the early days is very sketchy, but I believe that tops was first introduced around the time of OPENSTEP (so 1994). Certainly its first headline use was in converting code which used the old NextStep APIs into the new, shiny OpenStep APIs. Not that this was as straightforward as replacing NX with NS in the class names. The original APIs hadn't had much in the way of foundation classes (the Foundation Kit was part of OpenStep, but had been available on NeXTSTEP for use with EOF), so took char * strings rather than NSStrings, id[]s rather than NSArrays and so on. Also much rationalision and learning-from-mistakes was done in the Application Kit, parts of which were also pushed down into the Foundation Kit.All of this meant that a simple search-and-replace tool was not going to cut the mustard. Instead, tops needed to be syntax aware, so that individual tokens in the source could be replaced without any (well, alright, without too much) worry that any of the surrounding expressions would be broken, without too much inappropriate substitution, and without needing to pre-empt every developer's layout conventions.

before we continue - a warning

tops performs in-place substitution on your source code. So if you don't like what it did and want to go back to the original… erm, tough. If you're using SCM, there's no problem - you can always revert its changes. If you're not using SCM, then the first thing you absolutely need to do before attempting to try out tops on your real code is to adopt SCM. Xcode project snapshots also work.

replacing deprecated methods

Let's imagine that, for some perverted reason, I've written the following tool. No, scrub that. Let's say that I find myself having to maintain the following tool :-).

#import &lt;Foundation/Foundation.h&gt;int main(int argc, char **argv, char **envp){  NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init];    NSString *firstArg = [NSString stringWithCString: argv[1]];   NSLog(@"Argument was %s", [firstArg cString]);    [arp release];    return 0;}

Pleasant, non? Actually non. What happens when I compile it?

heimdall:Documents leeg$ cc -o printarg printarg.m -framework Foundationprintarg.m: In function ‘main’:printarg.m:6: warning: ‘stringWithCString:’ is deprecated (declared at /System/Library/Frameworks/Foundation.framework/Headers/NSString.h:386)printarg.m:7: warning: ‘cString’ is deprecated (declared at /System/Library/Frameworks/Foundation.framework/Headers/NSString.h:367)

OK so we obviously need to do something about this use of ancient NSString API. For no particular reason, let's start with -cString:

heimdall:Documents leeg$ tops replacemethod cString with UTF8String printarg.m

So what do we have now?

#import &lt;Foundation/Foundation.h&gt;int main(int argc, char **argv, char **envp){    NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init];    NSString *firstArg = [NSString stringWithCString: argv[1]];   NSLog@"Argument was %s", [firstArg UTF8String], length);  [arp release];    return 0;}

Looking good. But we still need to fix the -stringWithCString:. That could be just as easy, replacemethod stringWithCString: with stringWithUTF8String: would do the trick. However let's be a littledifferent here. Why don't we use -stringWithCString:encoding:? If we do that, then we're going to need to take a guess at the second argument, because we've got no idea what the encoding should be (that's why -stringWithCString: is deprecated, after all. However if we're happy to assume UTF8 is fine for the output, let's do that for the input. We'd better let everyone know that's what happened, though.So this rule is starting to look quite complex. It says "replace -stringWithCString: with -stringWithCString:encoding:, keeping the C string argument but adding another argument, which should be NSUTF8StringEncoding. While you're at it, warn the developer that you've had to make that assumption". We also (presumably) want to combine it with the previous rule, so that if we see the original file we'll catch both of the problems. Luckily tops lets us write scripts, which comprise of one or more rule descriptions. Here's a script which encapsulates both our cString rules:

replacemethod "cString" with "UTF8String"replacemethod "stringWithCString:&lt;cString&gt;" with "stringWithCString:&lt;cString&gt;encoding:&lt;encoding&gt;" { replace "&lt;encoding_arg&gt;" with "NSUTF8StringEncoding"} warning "Assumed input encoding is UTF8"

So why does the &lt;encoding&gt; token become &lt;encoding_arg&gt; in the sub-rule? Well that means "the thing which is passed as the encoding argument". This avoids confusion with &lt;encoding_param&gt;, the parameter as declared in the class interface (yes, you can run tops on headers as well as implementations).Now if we save this script as cStringNoMore.tops, we can run it against our source file:

heimdall:Documents leeg$ tops -scriptfile cStringNoMore.tops printarg.m

Which results in the following source:

#import &lt;Foundation/Foundation.h&gt;int main(int argc, char **argv, char **envp){    NSAutoreleasePool *arp = [[NSAutoreleasePool alloc] init];#warning Assumed input encoding is UTF8   NSString *firstArg = [NSString stringWithCString:argv[1] encoding:NSUTF8StringEncoding];  NSLog(@"Argument was %s", [firstArg UTF8String]); [arp release];    return 0;}

Now, when we compile it, we no longer get told about deprecated API. Cool! But it looks like I need to verify that the use of UTF8 is acceptable:

heimdall:Documents leeg$ cc -o printarg printarg.m -framework Foundationprintarg.m:6:2: warning: #warning Assumed input encoding is UTF8

exercises for the reader, and caveats

There's plenty more to tops than I've managed to cover here. You could (and indeed Apple do) use it to 64-bit-cleanify your sources. Performing security audits is another great use - particularly using constructs such as:

replace strcpy with same error "WTF do you think you're doing?!?"

However, notice that tops is a blunter instrument than the Xcode refactoring capability. Its smallest unit of operation is the source file; refactoring only within particular methods is not quite easily achieved. Also, as I said before, remember to check your source into SCM before running a script! There is a -dont option to make tops output its proposed changes without applying them, too.Finally tops shouldn't be used fully automated. Always assume that you need to inspect the output carefully, don't just Build and Go.

#### Reflections on "Is TDD Dead"
The first thing I noticed that I needed to change as a result of watching the Is TDD Dead? series is that I started out with a defensive mindset. If I believe in the dogma of a rule, then presumably I'm still a beginner who hasn't yet learned that there are other rules, each of which has limited applicability. So lesson one is to practise more, to find the limitations of the techniques I use, and to understand what to do when those limits are met.

An unsurprising aspect of the discussion was that the thing I call TDD is not the thing that others call TDD. At some point recently I got converted to the Mockist school. Part of me (the part that doesn't value my social life) is inclined to write a second edition of test-driven iOS development but then keep both editions available, because the style will have changed so much between the two.

Anyway it is more germane to note that none of the three speakers has a lot of time for mocks. I need to understand this, the experiences they have had that I have not, and the things that I do not know about mocks. Perhaps they view the sorts of tests that I build as the structural tests that they would delete once the system is working.

#### Reflections on an iBook G4
I had an item in OmniFocus to "write on why I wish I was still using my 2006 iBook", and then Tim Sneath's tweet on unboxing a G4 iMac sealed the deal. I wish I was still using my 2006 iBook. I had been using NeXTSTEP for a while, and Mac OS X for a short amount of time, by this point, but on borrowed hardware, mostly spares from the University computing lab.



My “up-to-date” setup was my then-girlfriend’s PowerBook G3 “Wall Street” model, which upon being handed down to me usually ran OpenDarwin, Rhapsody, or Mac OS X 10.2 Jaguar, which was the last release to boot properly on it. When I went to WWDC for the first time in 2005 I set up X Post Facto, a tool that would let me (precariously) install and run 10.3 Panther on it, so that I could ask about Cocoa Bindings in the labs. I didn’t get to run the Tiger developer seed we were given.



When the dizzying salary of my entry-level sysadmin job in the Uni finally made a dent in my graduate-level debts, I scraped together enough money for the entry-level 12” iBook G4 (which did run Tiger, and Leopard). I think it lasted four years until I finally switched to Intel, with an equivalent white acrylic 13” MacBook model. Not because I needed an upgrade, but because Apple forced my hand by making Snow Leopard (OS X 10.6) Intel-only. By this time I was working as a Mac developer so had bought in to the platform lock-in, to some extent.



The treadmill turns: the white MacBook was replaced by a mid-decade MacBook Air (for 64-bit support), which developed a case of “fruit juice on the GPU” so finally got replaced by the 2018 15” MacBook Pro I use to this day. Along the way, a couple of iMacs (both Intel, both aluminium, the second being an opportunistic upgrade: another hand-me-down) came and went, though the second is still used by a friend.



Had it not been for the CPU changes and my need to keep up, could I still use that iBook in 2020? Yes, absolutely. Its replaceable battery could be improved, its browser could be the modern TenFourFox, the hard drive could be replaced with an SSD, and then I’d have a fast, quiet computer that can compile my code and browse the modern Web.



Would that be a great 2020 computer? Not really. As Steven Baker pointed out when we discussed this, computers have got better in incremental ways that eventually add up: hardware AES support for transparent disk encryption. Better memory controllers and more RAM. HiDPI displays. If I replaced the 2018 MBP with the 2006 iBook today, I’d notice those things get worse way before I noticed that the software lacked features I needed.



On the other hand, the hardware lacks a certain emotional playfulness: the backlight shining through the Apple logo. The sighing LED indicating that the laptop is asleep. The reassuring clack of the keys.



Are those the reasons this 2006 computer speaks to me through the decades? They’re charming, but they aren’t the whole reason. Most of it comes down to an impression that that computer was mine and I understood it, whereas the MBP is Apple’s and I get to use it.



A significant input into that is my own mental health. Around 2014 I got into a big burnout, and stopped paying attention to the updates. As a developer, that was a bad time because it was when Apple introduced, and started rapidly iterating on, the Swift programming language. As an Objective-C and Python expert (I’ve published books on both), with limited emotional capacity, I didn’t feel the need to become an expert on yet another language. To this day, I feel like a foreign tourist in Swift and SwiftUI, able to communicate intent but not to fully immerse in the culture and understand its nuances.



A significant part of that is the change in Apple’s stance from “this is how these things work” to “this is how you use these things”. I don’t begrudge them that at all (I did in the Dark Times), because they are selling useful things that people want to use. But there is decidedly a change in tone, from the “Come in it’s open” logo on the front page of the developer website of yore to the limited, late open source drops of today. From the knowledge oriented programming guides of the “blue and white” documentation archive to the task oriented articles of today.



Again, I don’t begrudge this. Developers have work to do, and so want to complete their tasks. Task-oriented support is entirely expected and desirable. I might formulate an argument that it hinders “solutions architects” who need to understand the system in depth to design a sympathetic system for their clients’ needs, but modern software teams don’t have solutions architects. They have their choice of UI framework and a race to an MVP.



Of course, Apple’s adoption of machine learning and cloud systems also means that in many cases, the thing isn’t available to learn. What used to be an open source software component is now an XPC service that calls into a black box that makes a network request. If I wanted to understand why the spell checker on modern macOS or iOS is so weird, Apple would wave their figurative hands and say “neural engine”.



And a massive contribution is the increase in scale of Apple’s products in the intervening time. Bear in mind that at the time of the 2006 iBook, I had one of Apple’s four Mac models, access to an XServe and Airport base station, and a friend who had an iPod, and felt like I knew the whole widget. Now, I have the MBP (one of six models), an iPhone (not the latest model), an iPad (not latest, not Pro), the TV doohickey, no watch, no speaker, no home doohickey, no auto-unlock car, and I’m barely treading water.



Understanding a G4-vintage Mac meant understanding PPC, Mach, BSD Unix, launchd, a couple of directory services, Objective-C, Cocoa, I/O Kit, Carbon, AppleScript, the GNU tool chain and Jam, sqlite3, WebKit, and a few ancillary things like the Keychain and HFS+. You could throw in Perl, Python, and the server stuff like XSAN and XGrid, because why not?



Understanding a modern Mac means understanding that, minus PPC, plus x86_64, the LLVM tool chain, sandbox/seatbelt, Scheme, Swift, SwiftUI, UIKit, “modern” AppKit (with its combination of layer-backed, layer-hosting, cell-based and view-based views), APFS, JavaScript and its hellscape of ancillary tools, geocoding, machine learning, the T2, BridgeOS…



I’m trying to trust a computer I can’t mentally lift.

#### Regaining your identity
In my last post, losing your identity, I pointed out an annoying problem with the Sparkle update framework, in that if you lose your private key you can no longer post any updates. Using code signing identities would offer a get-out, in addition to reducing the complexity associated with releasing a build. You do already sign your apps, right?

I implemented a version of Sparkle that does codesign validation, which you can grab using git or view on github. After Sparkle has downloaded its update, it will test that the new application satisfies the designated requirement for the host application - in other words, that the two are the same app. It will not replace the host unless they are the same app. Note that this feature only works on 10.6, because I use the new Code Signing Services API in Security.framework.

#### Rehearsals in beta!
I have a new application, Rehearsals, an online practice diary for musicians. If that sounds like the kind of thing you're interested in, and you have Mac OS X 10.6 or newer, then please download the beta release and test it out. There's absolutely no charge, and if you submit feedback to support &lt;at&gt; rehearsalsapp &lt;dot&gt; com you'll be eligible for a free licence for version 1.0 once that's released. There are no limitations on the beta version, so please do download and start using it!You can follow @rehearsals_app for updates to the beta programme (new releases are automatically downloaded using Sparkle, if you enable it in the app).

#### Related methods and tools
The book Software: A Technical History has plenty of exercises and projects at the end of each chapter, to get readers thinking about software and its history and to motivate additional research. For example, here's exercise 1 (of 27 exercises and 8 projects) from chapter 1 (Introduction to Software History):

Why does the definition of software in this text include “related methods and tools?” What does knowing about the methods and tools used to develop software tell us about the software?
Kim W. Tracy, Software: A Technical History (p. 43)

The definition is this: Software is the set of programs, concepts, tools, and methods used to produce a running system on computing devices. (p. 2)

Those tools and methods are sometimes “a running system on computing devices” themselves, in which case they trivially fall into the definition of software: an Integrated Development Environment is a software system that people use to produce software systems.

Sometimes, the tools aren’t themselves “a running system on computing devices”. Flowcharts, UML diagrams, whiteboards, card punches, graph-paper bitmaps, and other artifacts are not themselves applications of computing, and neither are methods and methodologies like Object-Oriented Programming, the Personal Software Process, or XP.

Both the computer-based and non-computer-based tools and methods influence the system that people create. The system is a realization on a computing machine of an abstract design that's intended to address some set of desires or needs. The design itself is an important part of the software because it's the thing that the running system is intended to realize.

The tools and methods are themselves part of the design because they influence and constrain how people think about the system they're realizing, the attributes of the realized system, and how people collaborate to produce that system. As an example, software developers using Simula-67 create classes as types that encapsulated part of their design, and instantiate objects as example members of those types within their system. Software developers using Smalltalk do the same thing, but documentation about Simula-67 encourages thinking about hierarchical type systems within a structured programming paradigm, and documentation about Smalltalk encourages thinking about active objects within an object-oriented paradigm. So people in a community of Simula-67 programmers and people in a community of Smalltalk programmers would design different systems that work in different ways, even though using very similar tools.

When I read the definition of software and the exercise, I initially thought that it was wrong to think of the tools and methods as part of the software, even though it's important to include the tools and methods (and the associated social and cultural context of their legitimacy, popularity, and importance) in a consideration of software history. I considered a tighter definition, that includes the running programs, the (intangible) artifacts that comprise those programs, and any source code used in creating those artifacts. Reflecting on the exercise and writing this response, I see that this is an arbitrary place to draw the boundary, and that a definition of software that includes the design and methods used in creating the artifacts is also workable.

#### Remember the future?
The future is notoriously hard to pin down. For example, what is Seattle's lasting legacy from 20th Century technology? What would people have pointed to in, say, the 1970s? Of course, Seattle is the home of Boeing, who did a lot of construction for NASA (and bought most of the other companies that were also doing so) on projects like the Saturn V rocket and the Space Shuttle. Toward the end of the 1970s, those in the know in Seattle would have confidently claimed that the Shuttle's weekly trips into space as the world's longest-distance haulage provider will be central to 21st century Space Age technology. But neither the shuttle nor the Saturn V works any more, and nothing equivalent has come along to replace them (certainly not from Boeing). The permanent remnant of Seattle's part in the space race comes from earlier on, when the USSR already had satellites in orbit, Gagarin had safely returned, and the USA wanted to assert its technological superiority over the Soviets. I'm talking, of course, about Seattle Center and its most famous landmark: a giant lift shaft with a restaurant at one end and a gift shop at the other.

People like to prognosticate about how our industry, society or civilization is going to change: both in the short term, and in the distant future (which can be anything beyond about three years in software terms). Sure, it's fun to speculate. Earlier this year I took part in a panel with my arch-brother among others, where many of the questions were about what direction we thought Apple might take with the iPad, how existing companies would work their software to fit in post-PC devices, that sort of thing. That means that not only do we enjoy prognostication, but we seek it out. People enjoy playing the game of deciding what the future will be like, and hope for that spark of satisfaction of knowing either that they were right, or that they were there when someone else was right.

But why? It's not as if we're any good at it. The one thing that history lets us discover is that people who predict the future generally get it wrong. If they have the foresight to make really grandiose predictions they get away with it, because no-one finds out that they were talking out of their arses until well after they died. But just as the Space Needle has outlived the Space Shuttle, so the "next big thing" can easily turn out to be a fad while something apparently small and inconsequential now turns out to last and last.

Of course I'll discuss the computing industry in this article, but don't think this is specific to computing. In general, people go for two diametric visions of the future: either it's entirely different from what came before, or it's the same but a little better. The horses are faster, that kind of thing. Typically, experts in an industry are the people who find it hardest to predict that middle ground: a lot of things are the same, but one or two things have created a large change. Like the people at the air ministry who knew that superchargers were too heavy to ever allow Frank Whittle's jet turbine to take off. Or the people who didn't believe that people could travel at locomotive speeds. Or H.G. Wells, who predicted men on (well, in) the Moon, severely stratified society, life on other planets…just not the computer that was invented during his lifetime.

OK, so, computing. Remember the future of computers? The future will contain &quot;maybe five&quot; computers, according to Thomas Watson at IBM. I'm in a room now with about nine computers, not including the microcontrollers in my watch, hi-fi, cameras and so forth. There were around ten examples of Collosus produced in the 1940s. Why maybe five computers? Because computers are so damned heavy you need to reinforce the main frame of your floor to put them in. Because there are perhaps two dozen people in the world who understand computers. Because if you have too many then you have loads of dangerous mercury sloshing around. Because companies are putting themselves out of business attempting to sell these things for a million dollars when the parts cost nearly two million. And, finally, because there's just not much you can do on a computer: not everyone needs ballistics tables (and most of the people who do want them, we don't want to sell to).

Enough of the dim depths of computing. Let's come into the future's future, and ask whether you remember the other future of computers: the workstation. Of course, now we know that mainframes are old and busted, and while minicomputers based on transistor-to-transistor logic are cheaper, smaller, more reliable and all, they're still kindof big. Of course, micros like the Altair and the Apple are clearly toys, designed as winter-evening hobbies for married men[*]. Wouldn't it be better to use VLSI technology so that everyone can have their own time-sharing UNIX systems[**] on their desks, connected perhaps through the ultra-fast thinwire networks?

Better, maybe, but not best. Let's look at some of the companies involved, in alphabetical order. Apollo? Acquired by HP. Digital? Acquired, circuitously, by HP. HP? Still going, but not making workstations (nor, apparently, much else) any more. IBM? See HP. NeXT? Acquired, making consumer electronics these days. Silicon Graphics? Acquired (after having left the workstation industry). Stanford University Networks? Acquired by a service company, very much in the vein of IBM or HP. Symbolics, the owners of the first ever .com domain? Went bankrupt.

The problem with high-end, you see, is that it has a tendency to become low-end. Anything a 1980s workstation can do could be done in a "personal" computer or a micro by, well, by the 1980s. It's hard to sell bog standard features at a premium price, and by the time PCs had caught up to workstations, workstations hadn't done anything new. Well, nothing worth talking about…who'd want a camera on their computer? Notice that the companies that did stay around-IBM and HP-did so by getting out of the workstation business: something SGI and Sun both also tried to do and failed. The erosion of the workstation market by the domestic computer is writ most large in the Apple-NeXT purchase.

So workstations aren't the future. How about the future of user interfaces? We all know the problem, of course: novice computer users are confused and dissuaded by the "computery-ness" of computers, and by the abstract nature of the few metaphors that do exist (how many of you wallpaper your desktop?). The solution is obvious: we need to dial up the use of metaphor and skeuomorphism to make the user more comfortable in their digital surroundings. In other words, we need Bob. By taking more metaphors from the real world, we provide a familiar environment for users who can rely on what they already know about inboxes, bookshelves, desk drawers and curtains(!) in order to navigate the computer.

Actually, what we need is to get rid of every single mode in the computer's interface. This is, perhaps, a less well-known future of computing than the Bob future of computing, despite being documented in the classic book The Humane Interface, by Jef Raskin. The theory goes like this: we've got experience of modal user interfaces, and we know that they suck. They force the user to stop working while the computer asks some asinine question, or tells them something pointless about the state of their application. They effectively reverse the master-slave relationship, making the user submit to the computer's will for a while. That means that in the future, computers will surely dispose of modes completely. Well, full modes: of course partial modes that are entirely under the user's control (the Shift key represents a partial mode, as does the Spotlight search field) are still permitted. So when the future comes to invent the smartphone, there'll be no need for a modal view controller in the phone's API because future UI designers will be enlightened regarding the evils of modality.

A little closer to home, and a little nerdier, do you remember the future of the filesystem? HFS+ is, as we know, completely unsuitable as a filesystem for 2009 so future Macs will instead use Sun's ZFS. This will allow logical volume management, versioned files…the sorts of goodies that can't be done on HFS+. Oh, wait.

These are all microcosmic examples of how the future of computing hasn't quite gone according to the predictions. I could quote more (one I've used before is Bob Cringely's assertion in 1992 that in fifteen years, we'll have post-PC PCs; well I'm still using a PC to write this post and it's 2011), but it's time to look at the bigger picture, so I'm going to examine why the predictions from one particular book have or haven't come about. I'm not picking this book because I want to hate on it; in fact in a number of areas the predictions are spot on. I'm picking on this book because the author specifically set out to make short, medium and long-term forecasts about the silicon revolution, and the longest-term predictions were due to have become real by the year 2000. The book is The Mighty Micro: Impact of the Computer Revolution by Dr. Christopher Evans, published in 1979.

According to the Mighty Micro the following should have all happened by now.

Openness and availability of information leads to the collapse of the Soviet Union. ✓
A twenty-hour working week and retirement at fifty. ✗
Microcontroller-based home security. ✓ For everyone, replacing the physical lock-and-key. ✗
Cars that anticipate and react to danger. ✓ As the standard. ✗
A "wristwatch" that monitors pulse and blood pressure. ✓
An entire library stored in the volume of a paperback book. ✓
A complete end to paper money. ✗
An end to domestic crime. ✗

So what happened? Well, "processors and storage will get smaller and cheaper" was the prevailing trend from the forties to the seventies, i.e. over the entire history of electronic computing. Assuming that would continue, and that new applications for tiny computers would be discovered, was a fairly safe bet, and one that played out well. The fundamental failures behind all of the other predictions were twofold: that such applications would necessarily replace, rather than augment, whatever it was that we were doing before computers, and that we would not find novel things to do with our time once computers were doing the things we already did. The idea was that once computers were doing half of our work, we would have 50% as much work to do: not that we would be able to do other types of work for that 50% of our working week.

One obvious thing we-well, some of us-have to do now that we didn't before is program computers. Borrowing some figures from the BSA, there were 1.7M people working in software in the US in 2007, earning significantly more than the national average wage (though remember that this was during the outsourcing craze, so a lot of costs and jobs even for American companies might be missing here). The total worldwide expenditure on (packaged, not bespoke) software was estimated at $300bn. Once you include the service aspects and bespoke or in-house development, it's likely that software was already a trillion-dollar industry by 2007. Before, if you remember, the smartphone app gold rush.

This is a huge (and, if we're being brutally honest, inefficient) industry, with notoriously short deadlines, long working hours, capricious investors and variable margins. Why was it not predicted that, just as farmhands became machine operators, machine operators would become computer programmers? That the work of not having a computer would be replaced by the work of having a computer?

So, to conclude, I'll return to a point from the article's introduction: that making predictions is easy and fun, but making accurate predictions is hard. When a pundit tells you that something is a damp squib or a game-changer, they might be correct…but you might want to hedge your bets. Of course, carry on prognosticating and asking me to do so: it's enjoyable.

[*] This is one of the more common themes of futurology; whatever the technological changes, whatever their impacts on the political or economic structure of the world, you can bet that socially things don't change much, at least in the eye of the prognosticators. Take the example of the Honeywell Kitchen Computer: computers will revolutionise the way we do everything, but don't expect women to use them for work.

[**] Wait, if we've each got our own computer, why do they have to be time-sharing?

#### Replacing the language
Over the last few years, people have used the ObjC frameworks from TCL, Python, Perl and WebScript, Perl again, Perl, more Python, Ruby, Ruby, Ruby, Ruby, Java, Java, AppleScript, Smalltalk, C++, Pascal, Object Pascal, CLIPS, Common LISP, Nu, Eero, Modula-2, JavaScript, Erlang, Lua, OCaml, C#, F#, and indeed any unknown language.

It'd be nice if we had an alternative to ObjC, wouldn't it?

#### Report from WWDC
You're not going to see many discussions of talks from the conference, and this is no exception.  Beyond the keynote, the only talk I can discuss is one I went to today, which was a public talk by Amit Singh (author of the most excellent Mac OS X Internals, which I didn't bring with me in case the plane tipped over in flight) in the Apple Store across Market Street from the conference.

Amit's presentation style is as engaging as his writing, and he didn't let the wrath of the demo gods spoil what was an engaging talk.  While he was necessarily light on substance (he couldn't just read the book out, or we'd all still be in there) the demos gave indication of what is possible with Mac OS X and the material discussed in Internals.  In terms of OS X mentality we're of the same stable, although he clearly knows much more about the system than I could ever lay claim to.

I got to ask him what the best and worst features of Darwin are in comparison with other UNIX.  His answers were interesting and enlightening; best is the pragmatism Apple engineers use in generating new API or taking from existing code; worst is the poor approach to open source taken by Darwin.  I couldn't agree more.

#### Representativeness in Software Engineering Research
The first paragraph describes the context of this post in relation to the blog on which it originally appeared, not blog.securemacprogramming.com.

For this post, I wanted to go a little bit meta. One focus of this blog will be on whether results from academic software engineering are applicable to the work I do as a commercial software developer, so it was hard to pass up this Microsoft Research paper on representativeness of research.

In a nutshell, the problem is this: imagine that you read some study that shows, for example, that schedule slippage on a software project is significantly lessened if developers are given two digestive biscuits and a cup of tea at 4pm on working days. The study examined the breaktime habits of developers on 500 open source projects. This sounds quite convincing. If this thing with the tea and biscuits is true across five hundred projects, it must be applicable to my project, right?

That doesn't follow. There are many reasons why it might not follow: the study may be biased. The analysis may be wrong. The biscuit thing may be significant but not the root cause of success. The authors may have selected projects that would demonstrate the biscuit outcome. Projects that had initially signed up but got delayed might have dropped out. This paper evaluates one cause of bias: the projects used in the study aren't representative of the project you're doing.

It's a fallacy to assume that just because a study has a large sample size, its results can be generalised to the population. This only applies in the case that the sample represents an even slice of the population. Imagine a world in which all software projects are either written in Java or LISP. Now it doesn't matter whether I select 10 projects or 10,000 projects: a sample of LISP practices will not necessarily tell us anything about how to conduct a Java project.

Conversely a study that investigates both Java and LISP projects can—in this restricted universe, and with the usual "all other things being equal" caveat—tell us something generally about software projects independent of language. However, choice of language is only one dimension in which software can be measured: the size, activity, number of developers, licence and other factors can all be relevant. Therefore the possible phase space of important factors can be multidimensional.

In this paper the authors develop a framework, based on work in medicine and other fields, for measuring and maximising representativeness of a sample by appropriate selection of projects along the dimensions of the problem space. They apply the framework to recent research.

What they discovered, tabulated in Table II of the paper, is that while a very small, carefully-selected sample can be surprisingly representative (50 out of ~20k projects represented ~15% of their problem space), the ~200 projects they could find analysed in recent research only scored around 9% on their representativeness metric. However in certain dimensions the studies were highly representative, many covering 100% of the phase space in specific dimensions.

Conclusions

A fact that jumped out at me, because of the field I work in, is that there are 245 Objective-C projects in the universe studied by this paper (the projects indexed on Ohloh) and that not one of these is covered by any of the studies they analysed. That could mean that my own back yard is ripe for the picking: that there are interesting results to be determined by analysing Objective-C projects and comparing those results with received wisdom.

In the discussion section, the authors point out that just because a study is not general, does not mean it is not useful. You may not be able to generalise a result gleaned from analysing (say) Java developer tools to all software development, but if what you're interested in is Java developer tools then that is not a problem.

What this paper gives us, then, is not necessarily a tool that commercial developers can run out and use. It gives us some important quantitative context on evaluating the research that we do read. And, should we want to analyse our own work and investigate hypotheses about factors affecting our projects, it gives us a framework to understand just how representative those analyses would be.

#### Research Watch, and Java by Contract
I introduced Java by Contract, a tool for building design-by-contract style invariants, preconditions and postconditions in Java using annotations. It's MIT licensed, contributions are welcome, and I hope this helps lots of people to introduce stronger correctness checking into your software. And book office hours if you'd like me to help you with that.

Java by Contract came about as part of Research Watch, a new blog series over at The Labrary where I talk about academic work and how us "practitioners" (i.e. people who computer who aren't in academia) can make use of the results. The first post considers a report of Teaching Quality Object-Oriented Programming to computer science students.

By the way, I will be speaking at Coventry Tech Meetup on 10th January on the topic "Beyond TDD", and Java by Contract will make an appearance there.

Long-time SICPers readers will remember Programming Literate, a Tumblr discussing results from empirical software engineering. And if you don't, you'll probably remember your feeds exploding on July 15, 2013 when I imported all of the posts from there to here. You can think of Research Watch as a reboot of Programming Literate. There'll be papers new and vintage, empirical and opinionated, on a range of computing topics. If that sounds interesting, subscribe to the Labrary's RSS feed.

#### Resolution: Share Subscriptions
In Resolution: Subscribe Self I said I'd share my list of feeds. The nice thing to do would be to document a blog roll detailing why I subscribe to each blog, but for the moment here's an OPML file you can import into your reader, and consider the feeds you find therein.

#### Resolution: Subscribe Self
I have, at least temporarily, stopped using the social media to find news. I publish an RSS feed here, and your other favourite sites probably do too, so we can all discover the things we want to read without having to wade through a morass of things we don't.

Soon, I'll collect some recommended feeds together. What are yours? Which are the high-quality, low-volume feeds you make sure to catch every post on?

#### Resolutions
Although I didn't make any resolutions this new year, it's still a time for change. That's because I finally submit my D.Phil. thesis (if I'm on time, that will be before January 18th), so I've already been putting things in place that mean that I'll start doing things differently this year, without having to decide that I'm going to do things differently on January 1st (and invariably last until about the 3rd).

In my case, this year is about society. Writing a doctoral thesis means a lot of time alone, so I've joined committees for some clubs I'm in, and will be looking to do other things which mean spending more time finding out what other people want to do and helping them to do it.

#### Rethinking Object-Oriented Design figures
My iPad-drawn graphics in Rethinking OOD at App Builders 2018 were not very good, so here are the ink-and-paper versions. Please have them to hand when viewing the talk (which is the first of a two-parter, though I haven't pitched part two anywhere yet).

#### Retiring the "Apple developers are insular" meme
There's an old trope used in discussions of Mac and iOS developers, that says they're too inward-looking. They only think about software in ways that have been "blessed" by Apple, their platform vendor. I'm pretty sure that I've used this meme myself though couldn't find an example in a short Bing for the topic. It's now time to put that meme out to pasture (though, please, not out to stud. We don't want that thing breeding.)

"Apple-supplied" is a broad church

Since I've been using Macs, it's included: C, C++, Objective-C, five different assemblers, Java, AppleScript, perl, python, ruby (both vanilla and MacRuby), Tcl, bash, csh, JavaScript, LISP and PHP. Perhaps more. Admittedly on the iOS side options are fewer: but do you know anyone who's found their way around all of modern C++? You can be a programmer who never leaves the aforementioned collection of languages and yet is familiar with procedural, object-oriented, structured, functional and template programming techniques. There's no need to learn Haskell just to score developer points.

There is more to heaven and earth

"The community" has actually provided even more options than those listed above. RubyMotion, MonoTouch, MonoMac, PhoneGap/Cordova, wxWidgets, Titanium: these and more provide options for developing for Apple's platform with third party tools, languages and APIs. To claim that the Apple-based community is insular is to choose an exclusive subset of the community, ignoring all of the developers who, well, aren't that insular subset. If playing that sort of rhetorical game is acceptable then we aren't having grown-up discussions. Well, don't blame me, you started it.

Find out how many iOS apps are built with C#, or LUA, or JavaScript, or Ruby. Now see if you can say with conviction that the community of iOS app developers pays attention to nothing outside the field of Objective-C.

Not everyone need be a generalist

Back when Fred Brooks was writing about the failures of the System/360 project in his book "The Mythical-Man Month" and the article "No Silver Bullet", he suggested that instead of building armies of programmers to create software the focus should be on creating small, focussed surgical teams with a limited number of people assuming the roles required. The "surgeon" was played by the "chief programmer", somewhere between a software architect and a middle manager.

One of the roles on these "chief programmer teams" was the language lawyer. It's the job of the language lawyer to know the programming language and interfaces inside-out, to suggest smarter or more efficient ways of doing what's required of the software. They're also great at knowing what happens at edge-case uses of the language (remember the previous post on the various things that happen when you add one to an integer?) which is great for those last-minute debugging pushes towards the end of a project.

Having language lawyers is a good thing. If some people want to focus on knowing a small area of the field inside-out rather than having broader, but shallower, coverage, that's a good thing. These are people who can do amazing things with real code on real projects.

It doesn't help any discussion

Even if the statement were true, and if its truth in some way pointed to a weakness in the field and its practitioners, there are more valuable things to do than to express the statement. We need some internet-age name for the following internet-age rhetorical device:

I believe P is true. I state P. Therefore I have made the world better.

If you think that I haven't considered some viewpoint and my way of working or interacting with other developers suffers as a result, please show that thing to me. Preferably in a friendly compelling fashion that explains the value. Telling me I'm blinkered may be true, but is unlikely to change my outlook. Indeed I may be inclined to find that distasteful and stop listening; the "don't read the comments" meme is predicated on the belief that short, unkind statements are not worth paying attention too.

Conclusion

Absorption of external ideas does exist in our community, claiming that it doesn't is a fallacy. Not everyone need learn everything about the entirety of software making in order to contribute; claiming that they should is a fallacy. Making either of these claims is in itself not helpful. Therefore there's no need to continue on the "Apple developers are insular" meme, and I shan't.

If you find exciting ideas from other areas of software development, share them with those who will absorb. Worry not about people who don't listen, but rather wonder what they know and which parts of that you haven't discovered yet.

#### Reverse-engineering stringed instruments
Despite being able to play some instruments, I probably couldn't do a good job of making any of them. I don't have the patience required to boil a horse for long enough to stick a fiddle together, for instance. Luthiers would probably get incredibly bored by the following post but for fellow apart-takers, it'll hopefully be quite interesting.I once made a stringed synthesiser, when I was at college. The basic principle is that of an electric guitar running in reverse. A metal string is stretched between two bridges, and sits inside a horseshoe magnet. Now rather than plucking the string and letting the magnetic pickup detect the vibrations, we pass an alternating current through the string and use the magnetic field to cause the string to vibrate. Mount the whole shooting match on a soundbox roughly the shape of an appalachian dulcimer, so it makes a decent amount of noise. And there you go!Well, there bits of you go, anyway. While you can drive the string at any frequency you like, it'll be really quiet on any note which isn't a natural harmonic - it's being forced at one frequency, and trying to vibrate at a bunch of other frequencies, so can't really resonate. Assuming that the materials of the string aren't up for grabs, it's the length and tension which choose the fundamental frequency. What you currently have is capable of playing bugle tunes. Put a few different bugles together and you can play chromatic scales, so putting a few different strings together increases the likelihood that our synthesiser has the ability to play some notes in the tune we want.In fact, it's still going to have a fairly nasty volume characteristic, because most of the noise doesn't come from the string, it comes from the soundbox. In that regard, violins and pipe organs have quite a lot in common. But they differ in that pipe organs have one soundbox per note - the pipe itself - and violins have a single box. So it'd better be possible to get it to resonate at a whole bunch of interesting frequencies, which is one of the reasons for making them (and acoustic guitars) the shapes that they are. Now what the real acoustic characteristics of a fiddle body are, I'm not entirely sure. But what I do know is that a violin is surprisingly small - the "concert pitch" A pipe in an organ is a little under 40cm and the lowest note a violin usually has (a ninth lower) will therefore be almost a metre long. Even though an enclosed box like a violin needs to be half the length of an organ pipe playing the same note, it will still naturally accentuate the higher frequencies that the strings have on offer because it isn't big enough to do much else. And it's that which gives the instruments their sound. My synth's soundbox was cuboid-ish, so had two characteristic "loud" notes and their harmonics. The z-axis wouldn't have resonated much as the box was on a table which absorbed the momentum.The remaining interesting point is that the violin is forced to extract the sound from a rubbish part of the string. The bridge is both responsible for translating the motion of the string into the wood and for stopping the string from moving. The string moves most somewhere out toward the middle (it's mainly vibrating at its natural wavelength, which is twice the length of the string) and not at all near the ends. That's why pickups on electric guitars sound "warmer" further away form the bridge. There they pick up more of the lower harmonics, but the nearest pickup (and the bridge) get proportionally more of the higher harmonics.

#### Reversing the polarity of the message flow
On receiving a message with a parameter, sometimes an object just reverses the sense of what just happened and sends another message to the parameter object with itself as the parameter of this message. That's a pretty hard sentence to read, but I couldn't work out how to rewrite it, so here's an example. A Smalltalk class might have a method like this:

Object&gt;&gt;#printOn:
printOn:aStream
  "Append to the argument, aStream, a sequence of characters
  that describes the receiver."
  |description|
  description := self description.
  aStream nextPutAll:description.

I would do this as a matter of design: I can make it clear that I'm messaging one object and using another simply in a supporting role. It takes this:

foo doTheThing.
aStream nextPutAll:foo description.

and turns it into this:

foo doTheThing;
    printOn:aStream.

In Objective-C, this pattern has an additional use due to the nil sink for messages. Imagine formatting a string and appending the result to another string, but if the initial string is nil then you don't want to add anything. It might look like this:

if (emailText) {
  NSString *quotedString = [NSString stringWithFormat:@"&gt; %@", emailText];
  [output appendString:quotedString];
}

because I don't want the phrase &gt; (null) to appear in the output. But, with a bit of category magic, we can avoid the condition and make it clear that the email text is the important object.

[[emailText quote] appendTo:output];

I haven't just hidden the conditional; it never appears. If emailText is nil then all the other messages will just get dropped, so neither quote nor appendTo: need to test for nil.

Sometimes, the messages are just flowing the wrong way, and clarity can be gained by reversing the polarity.

[Update: oops! I had read Kent Beck's "Smalltalk Best Practice Patterns", and apparently absorbed his Reversing Method pattern. This post is basically a discussion of that pattern, and credit should go to Kent for originally creating it. My addition is the Objective-C specific nil sink extension. Sorry, Kent.]

#### Rhetoric, smoothly outlined
Something I did a number of years ago (I could tell you how many, couldn't I? If I could remember; I think it must have been 7) was to study critical analysis. That's the application of linguistics and sociology to, well, basically to refusing to believe anything people say to you ever again. As an example of how it's useful to someone who isn't a professional rhetorician, here's a discussion of the things I read in The iPhone Store Impending Disaster Myth. Mainly because that article is fairly close to the top of my RSS feed reader.The first thing to note is the use of loaded language in the title - the hyperbolic phrase "impending disaster" and its syzygy with the word "myth" clearly setting the author's stall out. This is reinforced by the first paragraph:According to the predictable opinion scribes [...]They’re wrong, here’s why.That first sentence fragment paints the subjects of the author's post as thoughtless machines, churning out page after page of text reinforcing their unchanging opinion. Ironically that is exactly what we are about to read for the next several paragraphs. It's a convenient amalgamation of two rhetorical techniques; most obviously it is an ad hominem (to the man) argument. Attention is diverted away from the discussion of Apple's app store and onto the people with which the author disagrees. This then is the beginning of a straw man which will be constructed toward the end of the piece, sowing the seed in the reader's mind that the author's opponent does not have a relevant argument.The final sentence, "they're wrong, here's why", is a trademark of this particular author (or maybe that's an example of confirmation bias on my part) and actually renders the rest of the article meaningless for most people. It tells us that the rest of the article is a repudiation (for why it isn't a refutation, read on, but the point of this sentence is some verbal sleight of hand to make you believe that a refutation is to follow) of the position the author has defined for the "predictable opinion scribes", which is either going to make you believe that what's coming up will be an excellent riposte or a boring diatribe, depending on the opinion you've already formed about this author. All that the remaining part of the article needs to do is to fill up past the end of the page so that you believe the riposte/diatribe really exists, and it performs this task with aplomb.What happens from here is actually rather subtle. The author outlines the position he intends to oppose, followed by "here’s[sic] the facts they’re missing". But the next few sections, from "Developers, Developers, Developers" to "Why Platforms Win" contain an opinionated  retrospective on the computing industry, using links to the author's own articles as references. Opinionated? Well, count the number of times the phrase "third rate, old technology" appears. It's actually only four, but it moves from what "IBM, Microsoft, and the PC cloners [Oxford comma sic]" were doing to "the Microsoft strategy". There's enough filler (26 paragraphs and 10 linked articles in the same style by the same author) that it could be easy to forget that segue occurred. A fact which doesn't escape the author:If you made it this far, you may have forgotten that the first argument against Apple vetoing appsToo right we might have forgotten. What we haven't forgotten is that we were told "here's why" the app store naysayers were wrong, but have actually been told why Lotus 1-2-3 outsold Visicalc. The author's argument follows the pattern "B follows A. C. Therefore A." Loosely the argument could be described as a "red herring fallacy", although a word I prefer is that the intervening text underwent a process known as "contextomy".Anyway, before we got here, our author let his façade slip a little:Now let’s hammer away at the sappy pleading on behalf of developers who want Apple to cater to their whims due to the attractive populist concept of fairness in doing so.Ooops! Now, do we think that the author is for or against people who disagree with Apple? Anyway, enough backtracking. Why don't we move forward from the end of my previous &lt;q&gt;?[...] is that its decisions are unpredictable and arbitrary.Now read the rest of that section. There's a good amount of text to describe why these decisions aren't arbitrary. Whatever happened to unpredictable? Oh, and for bonus points, look for where the final paragraph contradicts the earlier thrust of the section and reinforces the notion that arbitrary rejections have occurred.The rest of the article carries on in the same vein, and having seen the way in which I automatically parse the earlier part you can probably guess how my cynical mind interprets the rest of the text. Oh, and speaking of cynicism, if you're still wondering why this is a repudiation and not a refutation, then my evil little mind-play trick worked! You've read at least part of every paragraph in the hope to get information I promised at the beginning; if only I'd put some adverts in the post somewhere. So to refute means to prove to be false, whereas to repudiate means to reject. The article we've just looked at is an internally inconsistent expression of the author's opinion, no proof having occurred. It's also an example of the informal fallacy of suppressed correlative. Apple's practices can't be bad, because Microsoft's practices are bad and Apple's are better than Microsoft's.Well, that was fun! The next time you're talking to your boss (or better, your marketing people), listen out for those rhetorical devices and remember to stay critical :-).

#### Rootier than root
There's a common misconception, the book I'm reading now suffers from it, that single-user mode on a unix such as mac os x gives you root access. Actually, it grants you higher access than root. For example, set the immutable flag on a file (schg I think, but my iPhone doesn't have man). Root can't remove the flag, but the single user can.

#### Rubbies and sores
I imagine many of you are familiar with the difference between Ruby (a beautiful language representing the best pragmatic balance between Smalltalk's elegance and C's ubiquity) and Rubby (a horrendous mishmash of abominations in the style of all scripting languages, glommed together by finding nearly-compatible corner cases).

I also make the same distinction between Open Source (an attempt to get the same exploitation of labour as Free Software but without the principles) and Open Sores (the disturbingly wobbly house of cards that arises when collections of developers, none of whom feels empowered to make big changes, individually attach small pieces of mud to the collectively-constructed big ball).

Neither is fair, but both are useful shorthands.

#### Rumors of your runtime's death are greatly exaggerated
This is supposed to be the week in which Apple killed Java and Flash on the Mac, but it isn't. In fact, looking at recent history, Flash could be about to enter its healthiest period on the platform, but the story regarding Java is more complicated.

Since releasing Mac OS X back in 2001, Apple has maintained the ports of both the Flash and Java runtimes on the platform. This contrasts with the situation on Windows, where Adobe and Sun/Oracle respectively distribute and maintain the runtimes. (Those with long memory will remember the problems regarding Microsoft's JRE, which was eventually killed by court injunction.) In both cases, Apple has received occasional chastisement when its supported version of the runtime lagged behind the upstream release.

In the case of Flash, this was always related to security issues. Apple's version would lack a couple of patches that Adobe had made available. However, because Adobe maintains the official runtime for Mac OS X, it's super-easy to grab the latest version and stay up to date. If Apple stops maintaining its sporadically-updated distribution of the Flash runtime, then everyone who needs it will be forced into grabbing a new version from Adobe (who are free to remind users to install updates as other third-party vendors do). Everyone who doesn't need it doesn't have the runtime installed, thus reducing the attack surface of their browsers. Win-win.

Java, as I mentioned, is a more complicated kettle of bananas. Apple isn't redistributing the upstream JRE in this case, they're building their own from upstream sources. While other runtimes exist, none integrates as well with the OS due to effort Apple put in early on to support Yellow Box for Java and Aqua Swing. This means that you can't just go somewhere else to get your fix of JRE - it won't work the same.

There isn't a big market of Java apps on the Mac platform, so there isn't a big vendor who might pick up the slack and provide an integrated JRE. Oracle support OpenOffice on the platform, but that isn't a big earner. There's IBM with their Lotus products - I'll come onto that shortly. That just leaves the few Java client apps that do matter on the Mac - IDEs. As a number of Java developers have stated loudly this week, there are many Java developers currently using Macs. These people could either switch to a different OS, or look to maintain support for their IDEs on the Mac: which means supporting the JRE and the GUI libraries used by those IDEs.

By far the people in the best position to achieve that are on Eclipse. The Eclipse IDE (and the applications built on its Rich Client Platform, including IBM's stuff mentioned earlier) use a GUI library called SWT. This uses native code to draw its widgets using real Cocoa (in recent versions anyway - there's a Carbon port too) so SWT already works with native drawing in whatever JRE people bring along. This SoyLatte port of OpenJDK can already run Eclipse Helios. Eclipse works with Apache Harmony too, though the releases lag behind quite a bit.

So the conclusion is that your runtime isn't dead, in fact its support is equivalent to that found on the Windows platform. However, if you're using Java you might experience a brief period in the wilderness unless/until the community support effort catches up with its own requirements - which aren't the same as Apple's.

#### Running Linux GUI apps under MacOS using Docker
I had need to test an application built for Linux, and didn't want to run a whole desktop in a window using Virtualbox. I found the bits I needed online in various forums, but nowhere was it all in one place. It is now!

Prerequisites: Docker and XQuartz. Both can be downloaded from homebrew.

Create a Dockerfile:


FROM debian:latest


RUN apt-get update && apt-get install -y iceweasel


RUN export uid=501 gid=20 && \
    mkdir -p /home/user && \
    echo "user:x:${uid}:${gid}:User,,,:/home/user:/bin/bash" >> /etc/passwd && \
    echo "staff:x:${uid}:" >> /etc/group && \
    echo "user ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
    chmod 0440 /etc/sudoers && \
    chown ${uid}:${gid} -R /home/user


USER user
ENV HOME /home/user
CMD /usr/bin/iceweasel


It’s good to mount the Downloads folder within /home/user, or your Documents, or whatever. On Catalina or later you'll get warnings asking whether you want to give Docker access to those folders.

First time through, open XQuartz, goto preferences > Security and check the option to allow connections from network clients, quit XQuartz.

Now open XQuartz, and in the xterm type:


$ xhost + $YOUR_IP
$ docker build -f Dockerfile -t firefox .
$ docker run -it -e DISPLAY=$YOUR_IP:0 -v /tmp/.X11-unix:/tmp/.X11-unix -v $HOME/Downloads:/home/users/Downloads firefox


Enjoy firefox (or more likely, your custom app that you're testing under Linux)!

#### Runtime verification in Erlang by using contracts
About this paper
Runtime verification in Erlang by using contracts, L.-A. Fredlund et al, presented at WFLP 2018.
Notes
Spoiler alert, but the conclusion to my book OOP the Easy Way is that we should have independently-running objects, like we do in Erlang. We should also document the contract between an object and its collaborators, like we do in Eiffel. The concurrency is there because processors are getting wider, not faster, and applications are becoming more distributed across IOT, "edge compute", and servers. The contracts are there because we want to tame that complexity, tell people how each of the bits of our system can be composed, and what to expect if they try that. And we want to verify that documentation automatically, keeping ourselves honest.

As the authors here note, Erlang already has good facilities for recovering from errors, but that's only really suitable for errors that are "things that might go wrong". Programmer mistakes - call them bugs, logic errors, whatever - are things that shouldn't go wrong, and Erlang doesn't help there. Eiffel has really good tools for dealing with things that shouldn't go wrong.

This paper introduces Erlang Design by Contract, which adds the relevant Eiffel bits to Erlang. At its core is a contract specification that's richer than that found in Eiffel, allowing developers to assert that particular functions are pure, that they complete within given time, or that a recursive algorithm is trending towards completion.

The bit that achieves our aim, of combining the parallelism of Erlang with the safety of Eiffel, is the cpre/3 function, for implementing concurrent preconditions. The server inspects a message, the sender (from), and its internal state, and decides either to handle the message or queue it for later, changing the server state as appropriate.

In this way, preconditions act not as assertions that trigger failure, but as wait conditions that say when an object is ready to handle a message. The Concurrency Made Easy project found this too, when they built SCOOP, which works the other way and adds concurrency to Eiffel.

EDBC and SCOOP are two very different approaches that start from different places and aim for the middle. It's really interesting to see that there is common ground: that "precondition as wait" arises whether you add concurrency to contracts, or contracts to concurrency.

#### Rust project organisation
Coercion over configuration.

#### Sacrilege!
As I wrote the @interface to an object today, I found myself wanting for but one thing:

- (NSArray &lt;MyProtocol&gt; *) foo;

Where - as if you hadn't guessed it - the pointy bracket bit (the lengths to which I go to avoid typing out HTML entity names) would specify that all of the objects in the array returned by -foo conform to @protocol(MyProtocol).  I then realised that this wouldn't be quite as useful as I might think, but also decided that it wouldn't be too hard even on the existing ObjC runtimes to come up with a nightmare function such as:

(Protocol *) objc_class_to_protocol(Class *cls);

...therefore meaning that my hitherto unattainable pipe dream:

- (NSArray &lt;MythicalNSStringProtocol&gt; *) foo;

may indeed be somewhat closer to realisability.  Of course, with all of this being compile-time type checking (as with the similar beasties on Java) there would be no need to frob the runtime.

Update 2007-01-26T13:22: yes, I realise that the snippits above read &quot;an NSArray or subclass, which also conforms to the MyProtocol protocol&quot;.  I also know what I mean.

#### Scary stuff
Possibly the scariest diagram anyone will ever have to look at.  It's even less penetrable than that Eric Levenez history of unix thing.

#### Scripting confusion
LaTeX (and TeX, for that matter), syntax is relatively consistent, and uses a lot of backslashes.

Bourne shell syntax is somewhat inconsistent, and also uses backslashes.

Regular expression syntax I seem almost perversely disinclined to remember, and definitely sometimes often uses maybe backslashes.

Therefore, when I need to use a regular expression to work on a LaTeX document, I do it interactively in the shell until it works then paste that into a shell script. Now I need never remember or even uncover how to write that unholy combination of regex, shell, and LaTeX ever again.

For example I have defined a LaTeX macro \todo{I need more examples here.}, that draws the text "TODO: I need more examples here." with a box around it in the document. I review outstanding sections by searching for lines on which todo items appear, using this script:


#!/bin/sh

grep -n todo\{ "$1" | sed s/\\\\todo\{\\\(.\*\\\)\}/\\1/


Now I get to blank from my mind the unholy combination of syntaces that led to 15 backslashes on one line, but still use the results.

#### Second Brain
The idea of a second brain really hit home. Steven and I were doing some refactoring of some code in our Amiga podcast last night, and every time we moved something between files we had to remember which header files needed including. Neither of us were familiar enough with the libraries to know this, so people in the chat had to keep helping us.

But these are things we’ve already done, so we ought to be able to recall that stuff, with or without support. And when I say with support, I mean with what that post is calling a “second brain”, i.e. with an external, indexed cache of my brain. I shouldn’t need to reconstruct from scratch information I’ve already come across, but neither should I need to remember it all.

There are three problems I can see on the path to second brain adoption. The first, and the one that immediately made itself felt, is having a single interface for all my notes. When I read this article I thought that blogging about it would be a good way to crystallise my thoughts on the topic (it’s working!). So I saved the URL to Pinboard, I wrote a task in OmniFocus to write a blog post, and then when I was ready I fired up MarsEdit to write the blog that would end up on my Wordpress.

Remembering that all these bits are in all these systems is itself a brain overload task. And there’s more: I have information in Zotero about academic papers I’ve read, I have two paper notebooks on the go (one for computing projects and one for political projects), I have a Freewrite and a Remarkable, which each have their own sync services, I have abandoned collections of notes in Evernote and Zim…

I have a history of productivity porn addiction that doesn’t translate to productivity. I get excited by new tools, adopt them a bit, but don’t internalise the practice. So then I have another disjoint collection of some notes, maybe in a bullet journal, a Filofax, Livescribe notes, Apple Notes, wherever…making the problem of second brain even harder to manage because now there are more places.

So step one is to centralise these things. That’s not a great task to try to do in a Big Bang, so I’ll do it piecemeal. Evernote is the closest thing I already use to the second brain concept, so today I’ve stopped using my paper notebooks, writing those notes in Evernote instead. I also moved this draft to Evernote and worked on it there.

The second problem is implicit in that last paragraph: migration. Do I sit and scan all my notebooks, with my shocking and OCR-resistant handwriting, into Evernote? Do I paste all my summaries of research articles out of Zotero and into Evernote? No, doing so will take a long time and be very boring. What I’ll do instead is to move toward integration from this moment on. If I need something and I think I already have it, I’ll move it into Evernote. If I don’t have it, I’ll make it in Evernote. It will take a while to reap the benefit, but eventually I’ll have a single place to search when I want to look for things I already know.

And that’s the third of my three problems. Being diligent about searching the second brain. You have to change your approach to solving knowledge problems to be “do I already know this?” The usual, for me at least, is “what do I do to know this?” Now I’m good at that, with lots of experience at finding, appraising, and synthesising information, so doing it from scratch every time is mostly a waste of time rather than a fool’s errand. But it’s time I don’t need to waste.

I think that the fact I haven’t internalised these three aspects of the second brain is due to the generation of computing in which I really invested in computers. Most of the computers I learned to computer on could only do one thing at a time, practically if not absolutely. They didn’t have much storage, and that storage was slow. So that meant having different tools for different purposes. You would switch from the place where you recorded dance moves to the place where you captured information on Intuition data types, and rely on first brain for indexing. You wouldn’t even have all of it in the computer: I was 23 when I got my first digital camera, and 25 before I had an MP3 player. I did my whole undergraduate degree using paper notes and books from libraries and book stores. First brain needed to track where physically any information was in addition to where logically it was.

What I’m saying is I’m a dinosaur.

#### Security consultancy from the other side
I used to run an application security consultancy business, back before the kinds of businesses who knew they needed to consider application security had got past assessing creating mobile apps. Whoops!

Something that occasionally, nay, often happened was that clients would get frustrated if I didn't give them a direct answer to a question they asked, or if the answer to an apparently simple question was a one-day workshop. It certainly seems sketchy that someone who charges a day rate would rather engage in another day's work than write a quick email, doesn't it?

Today I was on the receiving end of this interaction. I'm defining the data encryption standards for our applications, so talked to our tame infosecnician about the problem. Of course, rather than specific recommendations about protocols, modes of operation, key lengths, rotation frequencies etc he would only answer my questions with other questions.

And this is totally normal and OK. The reason that I want to encrypt (some of) this data is that its confidentiality and integrity are valuable to me (my employer) and it's worth investing some of my (our) time in protecting those attributes. How much is it worth? How much risk am I willing to leave on the table? How long should the data be protected for? Who do I trust with the data, and how much do I trust them?

Only I can answer those questions, so it's not up to someone else to answer them for me, but they can remind me that I need to ask them.

#### Security flaw liability
The Register recently ran an opinion piece called Don't blame Willy the Mailboy for software security flaws. The article is a reaction to the following excerpt from a SANS sample application security procurement contract:

No Malicious Code
Developer warrants that the software shall not contain any code that does not support a software requirement and weakens the security of the application, including computer viruses, worms, time bombs, back doors, Trojan horses, Easter eggs, and all other forms of malicious code.

That seems similar to a requirement I have previously almost proposed voluntarily adopting:

If one of us [Mac developers] were, deliberately or accidentally, to distribute malware to our users, they would be (rightfully) annoyed. It would severely disrupt our reputation if we did that; in fact some would probably choose never to trust software from us again. Now Mac OS X allows us to put our identity to our software using code signing. Why not use that to associate our good reputations as developers with our software? By using anti-virus software to improve our confidence that our development environments and the software we’re building are clean, and by explaining to our customers why we’ve done this and what it means, we effectively pass some level of assurance on to our customer. Applications signed by us, the developers, have gone through a process which reduces the risk to you, the customers. While your customers trust you as the source of good applications, and can verify that you were indeed the app provider, they can believe in that assurance. They can associate that trust with the app; and the trust represents some tangible value.

Now what the draft contract seems to propose (and I have good confidence in this, due to the wording) is that if a logic bomb, back door, Easter Egg or whatever is implemented in the delivered application, then the developer who wrote that misfeature has violated the contract, not the vendor. Taken at face value, this seems just a little bad. In the subset of conditions listed above, the developer has introduced code into the application that was not part of the specification. It either directly affects the security posture of the application, or is of unknown quality because it's untested: the testers didn't even know it was there. This is clearly the fault of the developer, and the developer should be accountable. In most companies this would be a sacking offence, but the proposed contract goes further and says that the developer is responsible to the client. Fair enough, although the vendor should take some responsibility too, as a mature software organisation should have a process such that none of its products contain unaccounted code. This traceability from requirement to product is the daily bread of some very mature development lifecycle tools.

But what about the malware cases? It's harder to assign blame to the developer for malware injection, and I would say that actually the vendor organisation should be held responsible, and should deal with punishment internally. Why? Because there are too many links in a chain for any one person to put malware into a software product. Let's say one developer does decide to insert malware.

Developer introduces malware to his workstation. This means that any malware prevention procedures in place on the workstation have failed.
Developer commits malware to the source repository. Any malware prevention procedures in place on the SCM server have failed.
Developer submits build request to the builders.
Builder checks out build input and does not notice the malware, construct the product.
Builder does not spot the malware in the built product.
Testers do not spot the malware in final testing.
Release engineers do not spot the malware, and release the product.

Of course there are various points at which malware could be introduced, but for a developer to do so in a way consistent with his role as developer requires a systematic failure in the company's procedures regarding information security, which implies that the CSO ought to be accountable in addition to the developer. It's also, as with the Easter Egg case, symptomatic of a failure in the control of their development process, so the head of Engineering should be called to task as well. In addition, the head of IT needs to answer some uncomfortable questions.

So, as it stands, the proposed contract seems well-intentioned but inappropriate. Now what if it's the thin end of a slippery iceberg? Could developers be held to account for introducing vulnerabilities into an application? The SANS contract is quiet on this point. It requires that the vendor shall provide a "certification package" consisting of the security documentation created throughout the development process. The package shall establish that the security requirements, design, implementation, and test results were properly completed and all security issues were resolved appropriately and that Security issues discovered after delivery shall be handled in the same manner as other bugs and issues as specified in this Agreement. In other words, the vendor should prove that all known vulnerabilities have been mitigated before shipment and if a vulnerability is subsequently discovered and is dealt with in an agreed fashion, no-one did anything wrong.

That seems fairly comprehensive, and definitely places the onus directly on the vendor (there are various other parts of the contract that imply the same, such as the requirement for the vendor to carry out background checks and provide security training for developers). Let's investigate the consequences for a few different scenarios.

1. The product is attacked via a vulnerability that was brought up in the original certification package, but the risk was accepted. This vulnerability just needs to be fixed and we all move on; the risk was known, documented and accepted, and the attack is a consequence of doing business in the face of known risks.

2. The product is attacked via a novel class of vulnerability, details of which were unknown at the time the certification package was produced. I think that again, this is a case where we just need to fix the problem, of course with sufficient attention paid to discovering whether the application is vulnerable in different ways via this new class of flaw. While developers should be encouraged to think of new ways to break the system, it's inevitable that some unpredicted attack vectors will be discovered. Fix them, incorporate them into your security planning.

3. The product is attacked by a vulnerability that was not covered in the certification package, but that is a failure of the product to fulfil its original security requirements. This is a case I like to refer to as "someone fucked up". It ought to be straightforward (if time-consuming) to apply a systematic security analysis process to an application and get a comprehensive catalogue of its vulnerabilities. If the analysis missed things out, then either it was sloppy, abbreviated or ignored.

Sloppy analysis. The security lead did not systematically enumerate the vulnerabilities, or missed some section of the app. The security lead is at fault, and should be responsible.

Abbreviated analysis. While the security lead was responsible for the risk analysis, he was not given the authority to see it through to completion or to implement its results in the application. Whoever withheld that authority is to blame and should accept responsibility. In my experience this is typically a marketing or product management decision, as they try to drop tasks to work backwards from a ship date to the amount of effort spent on the product. Sometimes it's engineering management; it's almost never project management.

Ignored analysis. Example: the risk of attack via buffer overflow is noted in the analysis, then the developer writing the feature doesn't code bounds-checking. That developer has screwed up, and ought to be responsible for their mistake. If you think that's a bit harsh, check this line from the "duties" list in a software engineer job ad:
Write code as directed by Development Lead or Manager to deliver against specified project timescales quality and functionality requirements
When you're a programmer, it's your job to bake quality in.

#### Security: probably doing it wrong
Being knowledgable in the field of information security is useful and beneficial. However, it's not sufficient, and while it's (somewhat) easy to argue that it's necessary there's a big gap between being a security expert and making software better, or even making software more secure.

The security interaction on many projects goes something like this:

Develop software
Get a penetration tester in
Oh, shit
Fix anything that won't take more than two days
Get remaining risk signed off by senior management
Ship
Observe that most of the time, this doesn't cause much trouble

Now whether or not a company can afford to rely on that last bullet point being correct is a matter for the executives to decide, but let's assume that they don't want to depend on it. The problem they'll have is that they must depend on it anyway, because the preceding software project was done wrong.

Security people love to think that they're important and clever (and they are, just not any more than other software people). Throughout the industry you hear talk of "fail" or even "epic fail". This is not jargon, it's an example of the mentality that promotes calling developers idiots.

Did the developer get the security wrong because he's an idiot, or was it because you didn't tell him it was wrong until after he had finished?

"But we're penetration testers; we weren't engaged until after the developers had written the software." Who's fault is that? Did you tell anyone you had advice to give in the earlier stages of development? Did you offer to help with the system architecture, or with the requirements, or with tool selection?

You may think at this point that I shouldn't rock the boat; that if we carry on allowing people to write insecure software, there'll be more money to be made in testing it and writing reports about how many high-severity issues there are that need fixing. That may be true, though it won't actually lead to software becoming more secure.

Take another look at the list of actions above. Once the project manager knows that the software has a number of high-priority issues, the decision that project manager will have to take looks like this:

If I leave these problems in the software, will that cause more work in the project, or in maintenance? Do I look like my bonus depends on what happens in maintenance?

So, as intimated in the process at the top of the post, you'll see the quick fixes done - anything that doesn't affect the ship date - but more fundamental problems will be left alone, or perhaps documented as "nice to haves" for a future version. Anything that requires huge changes, like architectural modification or component rewrites, isn't going to happen.

If we actually want to get security problems fixed, we have to distribute the importance assigned to it more evenly. It's no good having security people who think that security is the most important thing ever, if they're not going to be the people making the stuff: conversely it's no good having the people who make the thing unaware of security if it really does have some importance associated with it.

Here's my proposal: it should be the responsibility of the software architect to know security or to know someone who knows security. Security is a requirement of a software system, and it's the architect's job to understand what the requirements are, how the software is to implement them and how to make any trade-off needed if the requirements come into conflict. It's the architect's job to justify those decisions, and to make them and see them followed throughout development.

That makes the software architect the perfect person to ensure that the relative importance of security versus performance, correctness, responsiveness, user experience and other aspects of the product is both understood and correctly executed in building the software. It promotes (or demotes, depending on your position) software security to its correct position in the firmament: as an aspect of constructing software.

#### See you at WWDC...?
Going to WWDC this year.  I'm not going to make any technical predictions, because last year I suggested that the Intel rumours were just a rehashing of regularly-repeated hype, and that there'd be a new edition of Hillegas to cover Core Data.  So, this year I'll restrict myself to predicting that Steve Jobs will be there, and will say "Boom" at least once.Fellow attendees: don't know what my movements will be, except that I'll be at the Thirsty Bear on Monday with some WebObjects people, but I'd like to meet as many of you as possible so please add onetrueleeg at mac dot com to your iChat list!  :-)

#### Selectively caring
When Choose Boring Technology was published earlier this year, it hit home for me. If you're spending money trying to ship, say, a music notation app, there's no point in rewriting the operating system scheduler.

  Let's say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps.
  
  If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that's existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you're in trouble.

Today the personal lesson finally percolated out of this team decision. Why is it I can't make any progress on software? I care about too many things. This isn't just about choosing technology, which is a small part of what technologists do (though it does imply passing all of the other caring, about ethics and finding potential customers and selecting customers and retaining customers and making money and hiring people and firing people and evaluating performance and making the lunch and designing a product and creating an experience and plumbing the toilets and other things to other people).

It's about those things, the technical things, and more. If I care about all of those things then there'll always be something not quite right about what I'm doing, which will be morale-destroying. Or it will make me work on fixing the thing that isn't quite right, rather than whatever I need to make progress on. Or both. Often both. And, of course, given the size of our industry and the number of bloviators in it, for any subject upon which it's possible to have an opinion there will be multiple opinions written, so plenty of possibilities to research how to care about whatever it is I've decided to care about.

Choosing what to care about is a skill, and a valuable one to have. Choosing not to care about something doesn't necessarily mean ignoring its impact, though that is one way to deal with it. You can hire someone else (directly or under contract) to care about it, make an arbitrary choice and go with that, or get someone else to make an arbitrary choice that you then follow.

If the division of labour was the driver of efficiency in the industrial age, then the division of caring is that driver in the information age. Now to choose what to care about.

#### Self-organising teams
In The Manifesto for Anarchic Software Development I noted that one of the agile manifesto principles is for self-organising teams, and that those tend not to exist in software development. What would a self-organising software team look like?

Management hire a gang and set them a goal, and delegate all decisions on how to run the gang and who is in the gang to members of the gang.
The “team lead” is not in charge of decision-making, but a consultant who can advise gang members on their work. The team lead probably works directly on the gang’s product too, unless the gang is too large.
One member of the gang acts as a go-between for management and communicates openly with the rest of the gang.
Any and all members of the gang are permitted to criticise the gang’s work and the management’s direction.
The lead, the management rep, and the union rep are all posts, not people. The gang can recall any of them at any time and elect a replacement.
Management track the outcomes of the gang, not the “productivity” of individuals.
Management pay performance-related benefits like bonuses to the gang for the gang’s collective output, not to individuals.
The gang hold meetings when they need, and organise their work how they want.

#### Separating user interface from work
Here's a design I've had knocking around my head for a while, and between a discussion we had a few weeks ago at work and Saul Mora's excellent design patterns talk at QCon I've built it.

A quick heads-up: currently the logic is all built into a side project app I've been working on so I don't have a single project download I can point to. The post here should explain all of the relevant code, which is made available under the terms of the MIT Licence. A reusable component is forthcoming.

Motivation

Remove the Massive View Controller from our applications' architectures. Push Cocoa, Cocoa Touch, or other frameworks to the edges of our codebase, responsible only for working with the UI. Separate the concerns of user interaction, work scheduling and the actual work.

There are maintainability reasons for doing so. We separate unrelated work into different classes, localising the responsibilities and removing coupling between them. The same code can be used in multiple contexts, because the UI frameworks are decoupled from the work that they're doing. This is not only a benefit for cross-platform work but for re-using the same logic in different places in a single user interface.

We also notice performance optimisations that become possible. With a clear delineation between the user interface code and the work, it's much easier to understand which parts of the application must be run on the user interface thread and which can be done in other contexts.

Solution

Implement the Message Bus pattern from server applications. In response to a user event, the user interface creates a command object and sends it to a command bus. The command bus picks an appropriate handler, passes it the command and schedules it. The user interface, the work done and the scheduling of that work are therefore all decoupled.

Workflow

At application launch, the application accesses the shared IKBCommandBus object:

@interface IKBCommandBus : NSObject

+ (instancetype)applicationCommandBus;
- (void)execute: (id &lt;IKBCommand&gt;)command;
- (void)registerCommandHandler: (Class)handlerClass;

@end

and registers command handlers. Command handlers know what commands they can process, and can tell the bus whether they will accept a given command. Handlers can also be loaded later, for example in Mac applications or server processes when a new dynamic bundle is loaded.

Once the application is running, the command bus can be used by user interface controllers. These controllers are typically UIViewControllers in an iOS app,  NSViewControllers, NSDocuments or other objects in a Cocoa app, or maybe something else in other contexts. A controller receives an action related to a user interface event, and creates a specific IKBCommand.

@protocol IKBCommand &lt;NSObject, NSCoding&gt;

@property (nonatomic, readonly) NSUUID *identifier;

@end

Commands represent requests to do specific work, so the controller needs to configure the properties of the command it created based on user input such as the current state of text fields and so on. This is done on the user interface thread to ensure that the controller accesses its UI objects correctly.

The controller then tells the application's command bus to execute the command. This does not need to be done on the user interface thread. The bus looks up the correct handler:

@interface IKBCommandHandler : NSObject

+ (BOOL)canHandleCommand: (id &lt;IKBCommand&gt;)command;
- (void)executeCommand: (id &lt;IKBCommand&gt;)command;

@end

Then the bus schedules the handler's executeCommand: method.

Implementation and Discussion

The Command protocol includes a unique identifier and conformance to the NSCoding protocol. This supports the Event Sourcing pattern, in which changes to the application can be stored directly as a sequence of events. Rather than storing the current state in a database, the app could just replay all events it has received when it starts up.

This opens up possibilities including journaling (the app can replay messages it received but didn't get a chance to complete due to some outage) and syncing (the app can retrieve a set of events from a remote source and play those it hasn't already seen). An extension to the implementation provided here is that the event source acts as a natural undo stack, if commands can express how to revert their work. In fact, even if an event can't be reversed, you can "undo" it by removing it from the event store and replaying the whole log back into the application from scratch.

When a command is received by the bus, it looks through the handlers that have been registered to find one that can handle the command. Then it schedules that handler on a queue.

@implementation IKBCommandBus
{
  NSOperationQueue *_queue;
  NSSet *_handlers;
}

static IKBCommandBus *_defaultBus;

+ (void)initialize
{
  if (self == [IKBCommandBus class])
    {
      _defaultBus = [self new];
    }
}

+ (instancetype)applicationCommandBus
{
  return _defaultBus;
}

- (id)init
{
  self = [super init];
  if (self)
    {
      _queue = [NSOperationQueue new];
      _handlers = [[NSSet set] retain];
    }
  return self;
}

- (void)registerCommandHandler: (Class)handlerClass
{
  _handlers = [[[_handlers autorelease] setByAddingObject: handlerClass] retain];
}

- (void)execute: (id &lt;IKBCommand&gt;)command
{
  IKBCommandHandler *handler = nil;
  for (Class handlerClass in _handlers)
    {
      if ([handlerClass canHandleCommand: command])
        {
          handler = [handlerClass new];
          break;
        }
    }
  NSAssert(handler != nil, @"No handler defined for command %@", command);
  NSInvocationOperation *executeOperation = [[NSInvocationOperation alloc] initWithTarget: handler selector: @selector(executeCommand:) object: command];
  [_queue addOperation: executeOperation];
  [executeOperation release];
  [handler release];
}

- (void)dealloc
{
  [_queue release];
  [_handlers release];
  [super dealloc];
}

@end

Updating the UI

By the time a command is actually causing code to be run it's far away from the UI, running a command handler's method in an operation queue. The application can use the Observer pattern (for example Key Value Observing, or Cocoa Bindings) to update the user interface when command handlers change the data model.

#### Server-side Objective-C
Recently, Kevin Lawler posted an "Informal Technical Note" saying that Apple could clean up on licence sales if only they'd support web backend development. There are only two problems with this argument: it's flawed, and the precondition probably won't be met. I'm sure there is an opportunity for server-side programming with Objective-C, but it won't be met by Apple.

The argument is flawed

The idea is that the community is within a gnat's crotchet of using ObjC on the web, if only ObjC were slightly better. This represents an opportunity for Apple:

Licensing fees
Sales of Macs for development
Increase share of Objective-C at the expense of Java
Get more devs capable with Objective-C, which is necessary for OSX & iOS development
Developer good will
Steer development on the web

Every one of these &quot;opportunities&quot; seems either inconsequential or unrealistic. Since the dot-com crash, much web server development has been done on free platforms with free tools: LAMP, Java/Scala/Clojure + Tomcat + Spring + Hibernate + Eclipse, Ruby on Rails, Node.js, you name it. The software's free, you pay for the hardware (directly or otherwise) and the developers. The opportunities for selling licences are few and far between—there are people who will pay good money for good developer tools that save them a good amount of time, but most developers are not those people. The money is made in support and in consultancy. This is why Oracle still exists, and Sun doesn't.

Of course, Apple already knows this, having turned the $n*10^4-per-license NeXT developer tools into a set of free developer tools.

Speaking of sales, the argument about selling Macs to developers is one that made sense in 2000. When Apple still needed to convince the computer-buying public that the new NeXT-based platform had a future, then selling to technologists and early adopters was definitely a thing. You could make a flaccid but plausible argument that Java and TextMate 1 provided an important boost to the platform. You can't argue that the same's true today. Developers already have Macs. Apple is defending their position from what has so far been lacklustre competition; there's no need for them to chase every sale to picky developers any more.

I'll sum up the remaining points as not being real opportunities for Apple, and move on. For Objective-C to win, Java does not have to lose (and for that matter, for Apple to win, Objective-C does not have to win; they've already moved away from Apple BASIC, Microsoft BASIC, Pascal and C-with-Carbon). Having ObjC backend developers won't improve the iOS ecosystem any more than Windows 8 has benefitted from all the VB and C# developers in the world. "Developer good will" is a euphemism for "pandering to fickle bloggers", and I've already argued that Apple no longer needs to do that. And Apple already has a strong position in directing the web, due to controlling the majority of clients. Who cares whether they get their HTML from ObjC or COBOL?

It probably won't happen

Even if Craig Federighi saw that list and did decide Apple's software division needed a slice of the server pie, it would mean reversing Apple's 15-year slow exit of the server and services market.

Apple already stopped making servers last year due to a lack of demand. Because OS X is only licensed to run on Apple-badged hardware, even when virtualised, this means there's no datacenter-friendly way you can run OS X. The Mac Mini server is a brute-force solution: rather than redundant PSUs, you have redundant servers. Rather than lights-out management, you hope some of the redundant servers stay up. Rather than fibre channel-attached storage, you have, well, nothing. And so on.

OS X Server has been steadily declining in both features and quality. The App Store reviews largely coincide with my experience—you can't even rely on an upgrade from a supported configuration of 10.N, N^&#8804;7 to 10.8 to leave your server working properly any more.

Apple have a server product that (barely) lets a group of people in the same building share wikis and calendars. They separately have WebObjects: a web application platform that they haven't updated in four years and no longer provide support for. One of their biggest internal server deployments is based on WebObjects (with, apparently, an Oracle stack): almost all of their others aren't. iCloud is run on external services. They internally use J2EE and Spring MVC.

So Apple have phased out their server hardware and software, and the products they do have do not appear to be well-supported. This is consistent with Tim Cook's repeated statement of "laser focus" on their consumer products; not so much with the idea that Apple is about to ride the Objective-C unicorn into web server town.

But that doesn't mean it won't happen

If there is a growth of server-side Objective-C programming, it's likely to come from people working without, around or even despite Apple. The options as they currently exist:

Objective-Cloud is, putting it crudely, Cocoa as a Service. It's a good solution as it caters to the "I'm an iOS app maker who just needs to do a little server work" market; in the same way that Azure is a good (first-party) platform for Microsoft developers.
GNUstepWeb is based on a platform that's even older than Apple's WebObjects. My own attempts to use it for web application development have hit a couple of walls: the GNUstep community has not shown interest in accepting my patches; the frameworks need a lot of love to do modern things like AJAX, REST or security; and even with the help of someone at Heroku I couldn't get Vulcan to build the framework.
Using any Objective-C environment such as GNUstep or the Cocotron, you could build something new or even old-school CGI binaries.

If it were me, I'd fork GNUstep and GSW. I'd choose one deployment platform, one web server, and one database, and I'd support the hell out of that platform only. I'd sell that as a hosted platform with the usual tiered support. The applications needed to do the sales, CRM and so on? Written on that platform. As features are needed, they get added; and the support apps are suitable for turning into the tutorials and sample code that help to reduce the support effort.

Of course, that's just me.

#### Set the settings set
The worst method naming convention Object-Oriented programming is set{Thing}(). And no, C# doesn't escape my ire for calling it Set{Thing}(), nor does Smalltalk for calling it {thing}:, though that does handily demonstrate how meaningless set is.

OK, so set isn't really meaningless. In fact, it's got the opposite problem: entirely too many meanings. Luckily, when we use set in a programming context, we only mean exactly one of those definitions.

Oh wait, there's that whole unordered collection thing, too. Two meanings.

And the electrical engineering definition of "making the voltage high", so there's three I suppose.

And the arrangement thing, like preferences ("settings") or big scale configuration ("set up"). Four.

So, let's all put on the blinkers of bean-fuelled tradition, and assume that when we see set at the beginning of a method we refer to putting something into a specified state. Probably without that digression you would have thought a method starting with the word set would do that anyway.

So far, so Kevlin Henney. "Set" really is a bad word to choose. But now let me ask two questions, which happen to be degenerate in verbal content: why am I setting this state?

Why am I setting this state?

Aren't data hiding and encapsulation important principles in object-oriented programming? Why should you let me, an unrelated object, diddle with your state? Why not, you know, let me send you commands and queries?

Also, there's the single responsibility principle. Why do I have information when you're the one who needs it? Perhaps I should have asked you to work out the new thing rather than doing it myself and telling you my answer. Of course objects often do need to pass one another information: software systems wouldn't be very interesting if data didn't flow around them. This leads me on to the same question again:

Why am I setting this state?

As commands go, "set{Thing}()" really doesn't document its purpose at all. "Here, have this thing". Err, okay, but why do you want it? What will you use it for? Why should I care what your thing is? What use will I get out of giving you a different thing?

Here are some mutators with descriptive names:

label.setColor(red); //becomes
label.drawTextInThisColor(red);

aStreet name:'Sesame Street'. "becomes"
aStreet wasRenamedTo:'Sesame Street'.

[employee setSalary:@(75000)]; //becomes
[employee futurePayChecksShouldBeBasedOnNewSalary:@(75000)];

Notice that it's now clearer why I'm setting these. It's also evident that these changes tell us something about the evolution over time of the objects: the street had an earlier name, and now has this new name. The employee had a different salary for their previous pay check, and will have a new salary for their next pay check. Last month's pay check should probably still be calculated with last month's salary. But now the method also makes it clear that there's a design problem anyway: why should the employee and not the payroll be where pay check information goes?

Soon after posting, the question was asked: isn't drawTextInThisColor() a bad name because the object might not immediately need to do any drawing? No; sending an object a message and leaving the implementation up to that object is how object-oriented programming works. If we really had to distinguish drawTextLater() and drawTextNow() then we'd not be doing any information hiding. Compare this with things like faults (an ORM claims that the database is in some state, when in fact it's not yet), or even the way GUI libraries already work (you can tell a view it needs display, and the view can choose not to do anything because it's off-screen). Object-Oriented Programming involves telling an object what you want from it and leaving the details up to that object. The set…() convention doesn't tell us what we want from an object.

Bonus question: Why am I setting this state?

Finally, I didn't tell you what units the salary is in. Good catch. Give yourself a pat on the head, you're a very special snowflake.

#### Shell scripts and Xcode
Back in 2009 at the first NSConf, Scotty asked some of the speakers for an Xcode Quick Tip. I'm still using mine today.

When your target needs a "Run Shell Script" build phase, don't write the script into the box in Xcode's build phases view. Instead, create the shell script as an external file and call that from the Xcode build phase. It's easier to version control, and you can take advantage of the capabilities of external editors—particularly where your "shell script" is actually in Perl, Ruby or some similar language.

#### Short Objective-C on the server update
It's been over a year since I looked at GNUstepWeb as a server platform for Objective-C development. I've recently had time to dig in a bit more, send the project some patches, and get the platform to a state on the Mac where I can start developing apps. That took a while, in my defence I was writing a book that I wanted to get out while it was sitting on my mind.

My next step (pardon the pun) is to set up a deployment platform on Some UNIX™, and set up post-update deployment of the apps: I already created pre-commit and post-commit hooks for development. It seems that a lot of people come through this blog looking for ObjC server info, I thought it was about time for an update :-).

#### SICPers podcast episode 10
This episode is all about build systems! Mostly about the problems associated with the venerable ./configure; make; make install process. This expands on a section I wrote in APPropriate Behaviour.

 	The history of UNIX make
 	Why Johnny Can't Build [portable scientific software]
 	Recursive Make Considered Harmful
 	Non-recursive Make Considered Harmful
 	A Generation Lost in the Bazaar

Some meta-links: SICPers Podcast on Apple Podcasts, Direct link to RSS feed.

#### SICPers Podcast Episode 3
The latest episode of SICPers, in which I muse on what programming 1980s microcomputers taught me about reading code, is now live. Here's the podcast RSS feed.

  It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. Edsger Dijkstra, "How do we tell truths that might hurt?"

History of the Dragon 32, the particular Tandy CoCo "homage" that I used a lot.
A guide to BASIC programming for the Sinclair ZX80

As always, your feedback and suggestions are most welcome.

#### SICPers podcast episode 4
We're back to Amiga-Smalltalk today, as the moment when it runs on a real Amiga inches closer. Listen here.

Aminet has git, but only for MorphOS.
MEmacs is the only text editor you need.
HiSOFT, makers of HiSOFT C++
SAS C Compiler. Linked from there is another Amiga C compiler, VBCC, which I don't currently have and won't try unless everything else fails.
ADE is a complete gcc toolchain for Amiga.

I think I've isolated all extraneous sound except the nearby motorway, which I can't do much about. I hope the experience is better!

#### SICPers podcast episode 5
It lives! Kinda. Amiga-Smalltalk now runs on Amiga. Along the way I review The K&amp;R book as a tutorial for C programming, mentioning my previous comparison to the Brad Cox and Bjarne Stroustrup books. I also find out how little I know "C", it turns out I've been using GNU C for the last 20 years.

Thanks to Alan Francis for his part in my downfall.

#### SICPers podcast episode 9
In this episode I talk about Design by Contract. Episode RSS feed - also available in Apple and Google Podcasts.

 	A Discipline of Programming
 	Go to statement considered harmful
 	Z Notation, and Object-Z
 	CocoaByContract, and JavaByContract
 	CLU Programming Language

#### SICPers podcast episode one
I made a podcast! Full show notes here due to the character limit at podbean.

Amiga-Smalltalk project on GitHub
Free books on Smalltalk: the three Addison-Wesley books "Smalltalk-80: The Interactive Programming Environment", "Smalltalk-80: The Language and its Implementation" and "Smalltalk-80: Bits of History, Words of Advice" are mentioned in this podcast, and the second of those is "the blue book" at the centre of the episode.
AROS Research Operating System is the Amiga-compatible open source operating system. It can boot on (i386 or m68k) hardware, or run hosted in Linux, FreeBSD, macOS, and Windows.

Please let me know what you think! You can find me on twitter at @iwasleeg, and I gave out my email address in the podcast. You could also comment here!

Errata: I said the Amiga 1000 had 128kB of RAM but it had 256kB, sorry!

#### SICPers Podcast episode two
Episode 2 is live!

The only link I promised this week is to the BCHS web application stack. Short for BSD, C, httpd, sqlite, it's a minimalist approach to making web applications.

As ever, your feedback is welcome, here or wherever you find me.

#### Side-by-side
A frequently-heard rider on the statement that Mac OS X &quot;is more secure than Windows&quot; is that fewer people are prodding its weak spots, because it has fewer users.  Well, Windows Vista has a market share comparative to Mac OS X (all versions), and this report describes the security statistics as being somewhat comparable, too.  So there we go.

#### Sideloading content into iOS apps
All non-trivial apps visualise content in some form, whether it's game levels embedded in the app, data loaded from some internet service, or something else.

In many cases the developer who's writing the Objective-C code isn't going to be the person who creates or prepares this content. In the case of embedded content, this can lead to a slow feedback loop—the content experts create a database or some other assets, then send it to the developer. The developer prepares a build using the new assets, uploading it to TestFlight or some other ad-hoc distribution centre. Then the content people can download that app to see their content in the context of the application it's designed for.

There's a simple way to close this loop, letting content creators see the app with their latest changes as they make them. That is to use iTunes File Sharing to load the content via the app's Documents folder.

If you have a line like this:

NSString *pathToContent = [[NSBundle mainBundle] pathToResource: @"myDatabase" ofType: @"sqlite"];

Change it to use a function like this:

NSString *pathToPotentiallySideloadedFile(NSString *filename, NSString *type)
{
    NSString *pathInDocumentsFolder = [[[NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject] stringByAppendingPathComponent: filename] stringByAppendingPathExtension: type];
    if (pathInDocumentsFolder)
        return pathInDocumentsFolder;
    else
        return [[NSBundle mainBundle] pathForResource: filename ofType: type];
}

//...
NSString *pathToContent = pathToPotentiallySideloadedFile(@"myDatabase", @"sqlite");


Now if people working on your app have a file in their Documents folder with the same name as the one used in the app, it'll load their version. So, how do they get it in there?

You need to make a simple change to your app's Info.plist:

    &lt;key&gt;UIFileSharingEnabled&lt;/key&gt;
    &lt;true/&gt;

Now when anybody with the app connects their device to iTunes, they'll be able to use file sharing to add their own content. Don't forget to turn this off before you go live!

I mentioned at the beginning of this post that this technique can be used for networked apps. Obviously there isn't really any difficulty getting updated content into a network-driven app; or if there is, someone did it wrong.

It's the opposite problem you have: keeping the content fixed. If your online component—be it a CMS, a data feed from an API, or something else—is getting new data you can't always ensure that the app is looking at the same stuff in testing. Indeed, sometimes I've found the CMS developers changing the data format without telling anyone; if you're investigating a particular condition related to the state of the data, it can be hard to reproduce.

You can use the iTunes File Sharing technique to load a specific version of the app's data without relying on the network connection and the server giving you the same output. This is great for regression testing, as you can ensure that only your code is changing between runs.

#### Sitting on the Sidelines
Thank you, James Hague, for your article You Can't Sit on the Sidelines and Become a Philosopher. I got a lot out of reading it, because I identified myself in it. Specifically in this paragraph:

  There's another option, too: you could give up. You can stop making things and become a commentator, letting everyone know how messed-up software development is. You can become a philosopher and talk about abstract, big picture views of perfection without ever shipping a product based on those ideals. You can become an advocate for the good and a harsh critic of the bad. But though you might think you're providing a beacon of sanity and hope, you're slowly losing touch with concrete thought processes and skills you need to be a developer.

I recognise in myself a lot of the above, writing long, rambling, tedious histories; describing how others are doing it wrong; and identifying inconsistencies without attempting to resolve them. Here's what I said in that last post:

  I feel like I ought to do something about some of that. I haven’t, and perhaps that makes me the guy who comes up to a bunch of developers, says “I’ve got a great idea” and expects them to make it.

Yup, I'm definitely the person James was talking about. But he gave me a way out, and some other statements that I can hope to identify with:

  You have to ignore some things, because while they're driving you mad, not everyone sees them that way; you've built up a sensitivity. [...] You can fix things, especially specific problems you have a solid understanding of, and probably not the world of technology as a whole.

The difficulty is one of choice paralysis. Yes, all of those broken things are (perhaps literally) driving me mad, but there's always the knowledge that trying to fix any one of them means ignoring all of the others. Like the out of control trolley, it's easier to do nothing and pretend I'm not part of the problem than to deliberately engage with choosing some apparently small part of it. It's easier to read Alan Kay, to watch Bret Victor and Doug Engelbart and to imagine some utopia where they had greater influence. A sort of programmerpunk fictional universe.

  As long as you eventually get going again you'll be fine.

Hopefully.

#### Six Colours
Apple has, in my opinion, some of the best general-purpose computing technology on the market right now, and has had some of the best for all of this millennium. However, their business practices are increasingly punitive, designed to extract greater amounts of rental income from their existing customers ("want to, um, read the newspaper? $9.99/mo, and we get to choose which newspaper!"), with rules that punish those who aren't interested in helping them extract that rent.

Throughout the iPhone era, Apple has dealt arbitrary hands to people who try to work with them: removing the I Am Rich app without explanation; giving News Corp. a special arrangement to allow in-app subscriptions when nobody else could do it; allowing Netflix to carry on operating rent-free while disallowing others.

People put up with this for the justifiable reason that the Apple technology platform is pleasant and easy to use, well-integrated across multiple contexts including desktop, mobile, wearable and home. None of Apple's competitors are even playing the same game: you could build some passable simulacrum using multiple vendor technology (for example Windows, Android, Dropbox; or Ubuntu, Replicant, Nextcloud) but no single outlet is going to sell you the "it just works" version of that setup. Not even any vendor consortium works together to provide the same ease and integration: you can't buy a Windows PC from Samsung, for example, that'll work out of the box with your Galaxy phone. Even if you get your Chromebook and your Pixel phone from Google, you've got some work ahead of you to get everything synced up.

And then, of course, since the failure of their banner ad business, Apple have successfully positioned themselves as the non-ad, non-data-gathering company. Sure, you could get everything we're doing cheaper elsewhere: but at what cost?

My view is that the one fact—the high-quality technology—doesn't excuse the other—the rent-extracting business model and capricious heavy-handed application of "the rules" with anyone who tries to work with them. People try to work with them because of the good technology, and get frustrated, enervated, or shut down because of the power imbalance in business. It is OK to criticise Apple for those things they're not doing well or fairly; it's a grown-up company worth trillions of dollars, it'll probably weather the storm. If enough people on the inside learn about and address the criticisms, they may even get better, which will be good for a massive global network of Apple's employees, suppliers, and customers.

It seems to me that some people (and I'm explicitly talking about people outside Apple now, obviously employees are expected to abide by whatever internal rules the company has and it takes a special kind of person to blow the whistle) will brook none of that. There are people who will defend the two-trillion dollar corporation blocking some small indie's business; "they're just applying their rules" (the rules that say I'll know it when I see it, indicating explicitly that capricious application is to be expected).

It seems weird that a Person On The Internet would feel the need to rush to the defence of The World's Biggest Company, and so to my mind it seems like they aren't. It seems like they're rushing to the defence of 1990s Beleaguered Apple, the company with three weeks of salary money in the bank that's running on the memory of having the best computers and the hope that maybe the twenty-first model we release this month will be the one that sells. The Apple with its six-coloured logo, where you have to explain that actually the one-button mouse doesn't make it a toy and you can do real work with it, but could you please send that document in Word 6 format as my Word can't open Word 97 files thank you. The Apple where actually if you specced up a PC to match this it would probably cost about the same, it's just that PCs also cover the lower end. The Apple where every friend or colleague you convinced to at least try it out meant a blow to the evil monolith megacorporation bringing computing to the dark side with its nefarious, monopolistic practices and arbitrary treatment of its partners.

That company no longer needs defending. It would be glib to say "that Apple ceased trading on February 7, 1997", the date that NeXT, Inc. finally disappeared as an independent business. But that's not what happened. That company slowly boiled as the temperature around it rose. The iMac, iBook, iPod, Mac OS X, iPhone, iPad: all of these things came out of that company. Admittedly, so did iTools, .Mac, and Mobile Me, but eventually along came iCloud. Obviously 2020 Apple is a continuation of the spirit and culture of 1997 Apple, 1984 Apple, 1976 Apple. It has some of the same people, and plenty of people who learned from the people who are and were the same people. But it is also entirely different. Through a continuum of changes, but no deliberate "OK, time to rip off the mask" conversion, Apple is now the IBM that fans booed in 1984, or the Microsoft that fans booed in 1997.

It's OK to not like that, to not defend it, but to still want something good to come out of their great technology. We have to let go of this notion that for Apple to win, everyone else has to lose.

#### Sleep on it
In my experience, the best way to get a high-quality software product is to take your time, not crunch to some deadline. On one project I led, after a couple of months we realised that the feature goals (ALL of them) and the performance goals (also ALL of them, in this case ALL of the iPads back to the first-gen, single core 2010 one) were incompatible.

We talked to the customer, worked out achievable goals with a new timeline, and set to work with a new idea based on what we had found was possible. We went from infrequently running Instruments (the Apple performance analysis tool) to daily, then every merge, then every proposed change. If a change led to a regression, find another change. Customers were disappointed that the thing came out later than they originally thought, but it was still well-received and well-reviewed.

At the last release candidate, we had two known problems. Very infrequently sound would stop playing back, which with Apple's DTS we isolated to an OS bug. After around 12 hours of uptime (in an iPad app, remember!) there was a chance of a crash, which with DTS we found to be due to calling an animation API in a way consistent with the documentation but inconsistent with the API's actual expectations. We managed to fix that one before going live on the App Store.

On the other hand, projects I've worked on that had crunch times, weekend/evening working, and increased pressure to deliver from management ended up in a worse state. They were still late, but they tended to be even later and lower quality as developers who were under pressure to fix their bugs introduced other bugs by cutting corners. And everybody got upset and angry with everybody else, desperate to find a reason why the late project wasn't my fault alone.

In one death march project, my first software project which was planned as a three month waterfall and took two years to deliver, we spent a lot of time shovelling bugs in at a faster rate than we were fixing them. On another, an angry product manager demanded that I fix bugs live in a handover meeting to another developer, without access to any test devices, then gave the resulting build to a customer…who of course discovered another, worse bug that had been introduced.

If you want good software, you have to sleep on it, and let everybody else sleep on it.

#### So are Macs uber-pricey?
No, but Sony Vaios are.

#### So it's not just the Department of Homeland Security, then
What is it about government security agencies and, well, security? The UK Intelligence and Security Committee has just published its Annual Report 2008-2009 (pdf, because if there's one application we all trust, it's Adobe Reader), detailing financial and policy issues relating to the British security services during that year.

Sounds "riveting", yes? Well the content is under Crown copyright[*], so I can excerpt some useful tidbits. According to the director of GCHQ:

The greatest threat [to government IT networks] is from state actors and there is an increasing vulnerability, as the critical national infrastructure and other networks become more interdependent.

The report goes on to note:

State-sponsored electronic attack is increasingly being used by nations to gather intelligence, particularly when more traditional espionage methods cannot be used. It is assessed that the greatest threat of such attacks against the UK comes from China and Russia.

and yet:

The National Audit Office management letter, reporting on GCHQ’s 2007/08 accounts, criticised the results of GCHQ’s 2008 laptop computer audit. This showed that 35 laptops were unaccounted for, including three that were certified to hold Top Secret information; the rest were unclassified. We pressed GCHQ about its procedures for controlling and tracking such equipment. It appears that the process for logging the allocation and subsequent location of laptops has been haphazard. We were told:
Historically, we just checked them in and checked them out and updated the records when they went through our... laptop control process.

So our government's IT infrastructure is under attack from two of the most resourceful countries in the world, and our security service is giving out Top Secret information for free? It sounds like all the foreign intelligence services need to do is employ their own staff to empty the bins in Cheltenham. In fairness, GCHQ have been mandated to implement better asset-tracking mechanisms; if they do so then the count of missing laptops will be reduced to only reflect thefts/misplaced systems. At the moment it includes laptops that were correctly disposed of, in a way that did not get recorded at GCHQ.

[*] Though significantly redacted. We can't actually tell what the budget of the intelligence services is, nor what they're up to. How the budget is considered sensitive information, I'm not sure.

#### So that's how it works
Back in Apple Silicon, Xeon Phi, and Amigas I asked how Apple would scale the memory up in a hypothetical Mac Pro based on the M1. We still don't know because there still isn't one, although now we sort of do know.

The M1 Ultra uses a dedicated interconnect allowing two (maybe more, but definitely two) M1 Max to act as a single SoC. So in an M1 Ultra-powered Mac Studio, there'll be two M1 packages connected together, acting as if the memory is unified.

It remains to be seen whether the interconnect is fast enough that the memory appears unified, or whether we'll start to need thread affinity APIs to say "this memory is on die 0, so please run this thread on one of the cores in die 0". But, as predicted, they've gone for the simplest approach that could possibly work.

BTW here's my unpopular opinion on the Mac Studio: it's exactly the same as the 2013 Mac Pro (the cylinder one). Speeds, particularly for external peripherals on USB and Thunderbolt, are much faster, so people are ready to accept that their peripherals should all be outside the box. But really the power was all in using the word Studio instead of Pro, so that people don't think this is the post-cheesegrater Mac.

#### So you don't like your IDE
There are many different tools for writing Objective-C code, though of course many people never stray much beyond the default that's provided by their OS vendor. Here are some of the alternatives I've used: this isn't an in-depth review of any tool, but if you're looking for different ways to write your code, these are tools to check out. If you know of more, please add to the comments.

In most cases, if you're writing Mac or iOS apps, you'll need the Xcode tools package installed anyway in order to provide the SDK frameworks (the libraries and headers you'll be linking against). The tools basically provide a different user interface to the same development experience.

A word on IDEs. The writers of IDEs have an uphill struggle, in that every user of an IDE assumes that they are a power user, though in fact there are a wide range of abilities and uses to which IDEs are bent. My experience with Eclipse, for example, ranges from learning Fortran 77 to developing a different IDE, in Java and Jython, using the same app. Visual Studio is used to write single screen line-of-business apps in Visual BASIC, and Windows. Different strokes for different folks, and yet most IDEs attempt to accommodate all of these developers.

CodeRunner

CodeRunner is not a full IDE, rather it's a tool for testing out bits of code. You get a syntax-highlighting, auto completing editor that supports a few different languages, and the ability to quickly type some code in and hit "Run". Much simpler than setting up a test project in an IDE, and only a handful of denarii from the App Store.

AppCode

AppCode is the engine from well-respected IDE IntelliJ IDEA, with Objective-C language support. Its language support is actually very good, with better (i.e. more likely to work) refactoring capabilities than Xcode, useful templates for creating new classes that conform to protocols with default implementations of required methods, and quick (and useful!) navigation categories.

AppCode uses Xcode project files and workspaces, and uses xcodebuild as its build system so it's completely compatible with Xcode. It doesn't support XIBs, and just launches Xcode when you try to edit those files.

It's also important to notice that AppCode is currently available under an "early access" program, with expiring licences that force upgrade to newer builds. There's no indication of when AppCode will actually be released, and what it will cost when it gets there. For comparison, the community edition of IDEA is free while the "Ultimate" edition costs anything up to £367, depending on circumstances.

Emacs

Emacs's Objective-C mode is pretty good, and it can use ctags to build a cross-file symbol table to provide jumping to definition like a "real" IDE. In fact, it would be fair to say that emacs is a real IDE: you can interact with version control directly, do builds (by default this uses Make, though you can easily tell it to run xcodebuild instead), run your app through gdb and other goodies.

That said, I think it's fair to say that emacs is only a powerful tool in the hands of those who are comfortable with Emacs. It's only because I've been using it forever (I actually started with MicroEmacs on AmigaOS, and wrote LISP modes in my Uni days) that I even consider it an option. It's nothing like an app on any other system: Cocoa's field editor supports a subset of emacs shortcuts but otherwise emacs feels nothing like a Mac, UNIX, Windows or anything else app. (Before the snarky comments come in, vim is just as odd.)

Conclusion

There are more ways out there to write your Objective-C code than just Xcode. These are a few of them: find one that works well for you.

#### So you're off to WWDC
Jeff has updated his excellent WWDC first-timer's guide, and I thought I'd augment that with some things I've noticed.

The easiest and cheapest way to get from SFO to San Francisco is via BART. Once you get to the arrivals hall, the BART station is well signposted. If you follow Jeff's advice and stay within a few blocks of the Moscone, you'll probably want Powell Street BART, or maybe Civic Center. It's only a few dollars on the BART but you might want to buy a $20 or so ticket so you can get around for the rest of the time you're in town. There are other options: shuttle buses, taxis and limousines, which all cost more.
The character of the city changes markedly west of UN Plaza. I've stayed in hotels on 7th street, and felt uneasy in that block and further over on my own in the evenings - I've never been physically attacked, but in 2005 I was chased by panhandlers who didn't like the fact that I didn't give them any money[*]. The hotels are almost an order of magnitude cheaper than those in the financial district, but there's a reason for it.
There are plenty of options for good food. The Moscone isn't one of them. If you want to make your own packed lunch you'll find a Whole Foods and a Safeway on 4th Street (get to the Moscone, then head away from Market Street), and I think I've seen a farmers market somewhere near Yerba Buena park. The Oasis grill across the road from the Moscone's main entrance is good, and Mel's Drive-In a block towards Market Street serves a fine breakfast. Local people can often recommend somewhere further afield, too.
Sleep and fresh water are both welcome commodities. Yes there will be parties on every night: don't feel you have to go to all the events all night, or that you have to drink booze like some kind of alcoholic guppy at each party. Making time to eat, sleep and top up on fluids will keep you healthy and profitable: dead programmers don't ship.

[*]Some US airports, and IIRC SFO is one of them, have official charity-staffed information desks. The people there have ID and are very helpful, so I usually donate to a homeless shelter there. San Francisco has a very real and visible homeless problem, with all sorts of political, social and geological causes.

#### So, what's the plan? Part 1: what WAS the plan?
No CEO dominated a market without a plan, but no market was dominated by following the plan.
— I made this quote up. Let's say it was Rockefeller or someone.
In Accidental Tech Podcast 385: Temporal Smear, John Siracusa muses on what "the plan" for Apple's various GUI frameworks might be. In summary, and I hope this is a fair representation, he says that SwiftUI is modern, works everywhere but isn't fully-featured, UIKit (including Mac Catalyst) is not as modern, not as portable, but has good feature coverage, and AppKit is old, works only on Mac, but is the gold standard for capability in Mac applications.

He compares the situation now with the situation in the first few years of Mac OS X's existence, when Cocoa (works everywhere, designed in mid-80s, not fully-featured) and Carbon (works everywhere, designed in slightly earlier mid-80s, gold standard for Mac apps) were the two technologies for building Mac software. Clearly "the plan" was to drop Carbon, but Apple couldn't tell us that, or wouldn't tell us that, while important partners were still writing software using the library.

This is going to be a two-parter. In part one, I'll flesh out some more details of the Carbon-to-Cocoa transition to show that it was never this clear-cut. Part two will take this model and apply it to the AppKit-to-SwiftUI transition.

A lot of "the future" in Next-merger-era Apple was based on neither C with Toolbox/Carbon nor Objective-C with OPENSTEP/Yellow Box/Cocoa but on Java. NeXT had only released WebObjects a few months before the merger announcement in December 1996, but around merger time they released WO 3.1 with very limited Java support. A year later, WO 3.5 with full Java Support (on Yellow Box for Windows, anyway). By May 2001, a few weeks after the GM release of Mac OS X 10.0, WebObjects 5 was released and had been completely rewritten in Java.

Meanwhile, Java was also important on the client. A January 1997 joint statement by NeXT and Apple mentions ObjC 0 times, and Java 5 times. Apple released the Mac Run Time for Java on that day, as well as committing to "make both Mac OS and Rhapsody preeminent development and deployment platforms for Java technology"—Rhapsody was the code-name-but-public for NeXT's OS at Apple.

The statement also says "Apple plans to carry forward key technologies such as OpenDoc", which clearly didn't happen, and led to this exchange which is important for this story:
One of the things I've always found is that you've got to start with the customer experience and work backwards to the technology. You can't start with the technology and try to figure out where you're gonna try to sell it. And I've made this mistake probably more than anybody else in this room, and I've got the scar tissue to prove it.
Notice that the problem here is that Apple told us the plan, did something else, and made a guy with a microphone very unhappy. He's unhappy not at Gil Amelio for making a promise he couldn't keep, but at Steve Jobs for doing something different.

OpenDoc didn't carry on, but Java did. In the Rhapsody developer releases, several apps (including TextEdit, which was sample code in many releases of OpenStep, Rhapsody and Mac OS X) were written in Yellow Box Java. In Mac OS X 10.0 and 10.1, several apps were shipped using Cocoa-Java. Apple successfully made Mac OS and Rhapsody (a single) preeminent development and deployment platform for Java technology.
I do most of my engineering on my PowerBook … it’s got fully-functional, high-end software development tools.
— James Gosling, creator of Java
But while people carried on using Macs, and people carried on using Java, and people carried on using Macs to use Java, few people carried on using Java to make software for Macs. It did happen, but not much. Importantly, tastemakers in the NeXT developer ecosystem who were perfectly happy with Objective-C thank you carried on being perfectly happy with Objective-C, and taught others how to be happy with it too. People turned up to WWDC in t-shirts saying [objc retain];. Important books on learning Cocoa said things like:
The Cocoa frameworks were written in and for Objective-C. If you are going to write Cocoa applications, use Objective-C, C, and C++. Your application will launch faster and take up less memory than if it were written in Java. The quirks of the Java bridge will not get in the way of your development. Also, your project will compile much faster.

If you are going to write Java applications, use Swing. Swing, although not as wonderful as Cocoa, was written from the ground up for Java.
— Aaron Hillegass, Cocoa Programming for Mac OS X
Meanwhile, WebObjects for Java was not going great guns either. It still had customers, but didn't pick up new customers particularly well. Big companies who wanted to pay for a product with the word Enterprise in the title didn't really think of Apple as an Enterprise-ish company, when Sun or IBM still had people in suits. By the way, one of Sun's people in suits was Jonathan Schwartz, who had run a company that made Cocoa applications in Objective-C back in the 1990s. Small customers, who couldn't afford to use products with Enterprise in the title and who had no access to funding after the dot-bomb, were discovering the open source LAMP (Linux, Apache, MySQL, PHP/Perl) stack.

OK, so that's Cocoa, what about Carbon? It's not really the Classic Mac OS Toolbox APIs on Mac OS X, it's some other APIs that are like those APIs. Carbon was available for both Mac OS 8.1+ (as an add-on library) and Mac OS X. Developers who had classic Mac apps still had to work to "carbonise" their software before it would work on both versions.

It took significant engineering effort to create Carbon, effectively rewriting a lot of Cocoa to depend on an intermediate C layer that could also support the Carbon APIs. Apple did this not because it had been part of their plan all along, but because developers looked at Rhapsody with its Cocoa (ObjC and Java) and its Blue Box VM for "classic" apps and said that they were unhappy and wouldn't port their applications soon. Remember that "you've got to start with the customer experience and work backwards to the technology", and if your customer experience is "I want to use Eudora, Word, Excel, and Photoshop" then that's what you give 'em.

With this view, Cocoa and Carbon are actually the same age. Cocoa is OpenStep minus Display PostScript (Quartz 2D/Core Graphics taking its place) and with the changes necessary to be compatible with Carbon. Carbon is some MacOS Toolbox-like things that are adapted to be compatible with Cocoa. Both are new to Mac developers in 2001, and neither is furthering the stated goal of making Mac OS a preeminent development and deployment environment for Java.

To the extent that Apple had a technology roadmap, it couldn't survive contact with their customers and developers—and it was a good thing that they didn't try to force it. To the extent that they had a CEO-level plan, it was "make things that people want to buy more than they wanted to buy our 1997 products", and in 2001 Apple released the technology that would settle them on that path. It was a Carbonised app called iTunes.

#### So, what's the plan? Part 2: what will the plan be?
In Part One, I explored the time of transition from Mac OS 8 to Mac OS X (not a typo: Mac OS 9 came out during the transition period). From a software development perspective, this included the Carbon and Cocoa UI frameworks. I mooted the possibility that Apple's plan was "erm, actually, Java" and that this failed to come about not because Apple didn't try, but because developers and users didn't care. The approach described by Steve Jobs, of working out what the customer wants and working backwards to the technology, allowed them to be flexible about their technology roadmap and adapt to a situation where Cocoa on Objective-C, and Carbon on C and C++, were the tools of choice.[*]

So this time, we want to understand what the plan is. The technology choices available are, in the simplistic version: SwiftUI, Swift Catalyst/UIKit, ObjC Catalyst/UIKit, Swift AppKit, ObjC AppKit. In the extended edition, we see that Apple still supports the "sweet solution" of Javascript on the web, and despite trying previously to block them still permits various third-party developer systems: Javascript in React Native, Ionic, Electron, or whatever's new this week; Xamarin.Forms, JavaFX, Qt, etc.

What the Carbon/Cocoa plan tells us is that this isn't solely Apple's plan to implement. They can have whatever roadmap they want, but if developers aren't on it it doesn't mean much. This is a good thing: if Apple had sufficient market dominance not to be reasonably affected by competitive forces or market trends, then society would have a problem and the US DOJ or the EU Directorate-General for Competition would have to weigh in. If we don't want to use Java, we won't use Java. If enough of us are still using Catalyst for our apps, then they're supporting Catalyst.

Let's put this into the context of #heygate.
These apps do not offer in-app purchase — and, consequently, have not contributed any revenue to the App Store over the last eight years.
— Rejection letter from Apple, Inc. to Basecamp
When Steve Jobs talked about canning OpenDoc, it was in the context of a "consistent vision" that he could take to customers to motivate "eight billion, maybe ten billion dollars" of sales. It now takes Apple about five days to make that sort of money, so they're probably looking for something more than that. We could go as far as to say that any technology that contributes to non-revenue-generating apps is an anti-goal for Apple, unless they can conclusively point to a halo effect (it probably costs Apple quite a bit to look after Facebook, but not having it would be platform suicide, for example).

From Tim Cook's and Craig Federighi's height, these questions about "which GUI framework should we promote" probably don't even show up on the radar. Undoubtedly SwiftUI came up with the SLT before its release, but the conversation probably looked a lot like "developers say they can iterate on UIs really quickly with React, so I've got a TPM with a team of ten people working on how we counter that." "OK, cool." A fraction of a percent of the engineering budget to nullify a gap between the native tools and the cross-platform things that work on your stack anyway? OK, cool.

And, by the way, it's a fraction of a percent of the engineering budget because Apple is so gosh-darned big these days. To say that "Apple" has a "UI frameworks plan" is a bit like saying that the US navy has a fast destroyers plan: sure, bits of it have many of them.

At the senior level, the "plan" is likely to be "us" versus "not us", where all of the technologies you hear of in somewhere like ATP count as "us". The Java thing didn't pan out, Sun went sideways in the financial crisis of 2007, how do we make sure that doesn't happen again?

And even then, it's probably more like "preferably us" versus "not us, but better with us": if people want to use cross-platform tools, and they want to do it on a Mac, then they're still buying Macs. If they support Sign In With Apple, and Apple Pay, then they still "contribute any revenue to the App Store", even if they're written in Haskell.

Apple made the Mac a preeminent development and deployment platform for Java technology. One year at WWDC I met some Perl hackers in a breakout room, then went to the Presidio to watch a brown bag session by Python creator Guido van Rossum. When Rails became big, everyone bought a Mac Laptop and a Textmate license, to edit their files for their Linux web apps.
Apple lives in an ecosystem, and it needs help from other partners, it needs to help other partners. And relationships that are destructive don't help anybody in this industry as it is today. … We have to let go of this notion that for Apple to win, Microsoft has to lose, OK? We have to embrace the notion that for Apple to win, Apple has to do a really good job.
— Steve Jobs, 1997

[*] even this is simplistic. I don't want to go overboard here, but definitely would point out that Apple put effort into supporting Swing with native-esque controls on Java, language bridges for Perl, Python, Ruby, an entire new runtime for Ruby, in addition to AppleScript, Automator, and a bunch of other programming environments for other technologies like I/O Kit. Like the man said, sometimes the wrong choice is taken, but that's good because at least it means someone was making a decision.

#### Social and political requirements gathering
I was originally going to talk about API: Design Matters and Cocoa, but, and I believe the title of this post may give this away, I'm not going to now.  That's made its way into OmniFocus though, so I'll do it sooner or later.  No, today I'm more likely to talk about The Cathedral and the Bazaar, even though that doesn't seem to fit the context of requirements gathering.

So I've been reading a few papers on Requirements Engineering today, most notably Goguen's The Dry and the Wet.  One of the more interesting and subtle conclusions to draw from such a source (or at least, it's subtle if you're a physics graduate who drifted into Software Engineering without remembering to stop being a physicist) is the amount of amount of political influence in requirements engineering.  Given that it costs a couple of orders of magnitude more to mend a broken requirement in maintenance than in requirements-gathering (Boehm knew this back in 1976), you'd think that analysts would certainly leave their own convictions at the door, and would try to avoid the &quot;write software that management would like to buy&quot; trap too.

There are, roughly speaking, three approaches to requirements elicitation.  Firstly, the dry, unitarian approach where you assume that like a sculpture in a block of marble, there is a single &quot;ideal&quot; system waiting to be discovered and documented.  Then there's the postmodern approach, in which any kind of interaction between actors and other actors, or actors and the system, is determined entirely by the instantaneous feelings of the actors and is neither static nor repeatable.  The key benefit brought by this postmodern approach is that you get to throw out any idea that the requirements can be baselined, frozen, or in any other way rendered static to please the management.

[That's where my oblique CatB reference comes in - the Unitary analysis model is similar to ESR's cathedral, and is pretty much as much of a straw man in that 'purely' Unitary requirements are seldom seen in the real world; and the postmodern model is similar to ESR's bazaar, and is similarly infrequent in its pure form.  The only examples I can think of where postmodern requirements engineering would be at all applicable are in social collaboration tools such as Facebook or Git.]

Most real requirements engineering work takes place in the third, intermediate realm; that which acknowledges that there is a plurality among the stakeholders identified in the project (i.e. that the end-user has different goals from his manager, and she has different goals than the CEO), and models the interactions between them in defining the requirements.  Now, in this realm software engineering goes all quantum; there aren't any requirements until you look for them, and the value of the requirements is modified by the act of observation.  A requirement is generated by the interaction between the stakeholders and the analyst, it isn't an intrinsic property of the system under interaction.

And this is where the political stuff comes in.  Depending on your interaction model, you'll get different requirements for the same system.  For instance, if you're of the opinion that the manager-charge interaction takes on a Marxist or divisive role, you'll get different requirements than if you use an anarchic model.  That's probably why Facebook and Lotus Notes are completely different applications, even though they really solve the same problem.

Well, in fact, Notes and Facebook solve different problems, which brings us back to a point I raised in the second paragraph.  Facebook solves the &quot;I want to keep in contact with a bunch of people&quot; problem, while Notes solves the &quot;we want to sell a CSCW solution to IT managers&quot; problem.  Which is itself a manifestation of the political problem described over the last few paragraphs, in that it represents a distortion of the interaction between actors in the target environment.  Of course, even when that interaction is modelled correctly (or at least with sufficient accuracy and precision), it's only valid as long as the social structure of the target environment doesn't change - or some other customer with a similar social structure comes along ;-)

This is where I think that the Indie approach common in Mac application development has a big advantage.  Many of the Indie Mac shops are writing software for themselves and perhaps a small band of friends, so the only distortion of the customer model which could occur would be if the developer had a false opinion of their own capabilities.  There's also the possibility to put too many "developer-user" features in, but as long as there's competition pushing down the complexity of everybody's apps, that will probably be mitigated.

#### Software design is refinement, not abstraction
James Koppel tells us that software engineers keep using the word "abstraction" and that he does not think it means what they think it means. I believe that he is correct, and that the confusion over the term abstraction comes from thinking that programming is about abstraction.

Programming is refinement, not abstraction. You take your idea of what the software should be doing and you progressively refine that idea until what you get it so rote, so formulaic, so prescribed, that you can create a plan that a computer will follow reliably and accurately. Back in the day, that process used to be done literally in the order written as a sequence of distinct activities.

You take your idea of what the software should be doing: requirements gathering

refine that idea: software&nbsp;specification

create a plan that a computer will follow: construction

reliably and accurately: verification and validation

It doesn't matter what paradigm you're using, what tools, or whether you truly go from one step to the next as described here, what you're doing is&nbsp;specifying&nbsp;the software you need, and getting so specific that you end up with instructions for a computer. Specific is the opposite of abstract: the process is the opposite of abstraction.

Thus Niklaus Wirth talks about Program Development by Stepwise Refinement. He describes a&nbsp;procedure&nbsp;for solving the 8 queens problem, then refines that procedure until it is entirely described in terms of Algol (more or less) instructions. He could have started by describing a function that turns the set of possible chess boards into the set of boards that solve 8 queens, or he could have started by describing the communication between the board and the queens that would cause them to solve 8 queens.

This is not to say that abstraction&nbsp;doesn't appear&nbsp;in software development. Wirth's starting point is an abstract procedure for solving a specific problem: you can follow that procedure to solve 8 queens, you just have to do a lot of colouring in yourself which a computer is incapable of. Maybe GPT-3 could follow that first procedure; maybe one of the intermediate ones.

And his end point is an abstract definition of the instructions the computer will do: you can say&nbsp;j&lt;1&nbsp;and the computer will do&nbsp;something&nbsp;along the lines of loading the word at the same address previously associated with&nbsp;j&nbsp;into an accumulator, subtracting one, checking the flags index, and conditionally modifying the program counter. And "loading the word at the same address" is itself an abstraction: what really happens might involve page faults, loading data from permanent storage, translation lookaside buffers, and other magic.

Abstractions in this view are a "meet in the middle" situation, not a goal: you can refine/specify your solution until you meet the thing that will do the rest of the work of refinement. Or sometimes a "meet off to the side" situation: if you can make your program's data look like bags of tuples then you can use the relational model to do a lot of the storage and retrieval work, even if nothing in your problem description looks anything like bags of tuples.

Notice that Wirth's last section is about&nbsp;generalisation, not&nbsp;abstraction: solving the "N queens" problem is not any less specific than solving the 8 queens problem.

#### Software, Science?
Is there any science in software making? Does it make sense to think of software making as scientific? Would it help if we could?

Hold on, just what is science anyway?

Good question. The medieval French philosopher-monk Buridan said that the source of all knowledge is experience, and Richard Feynman paraphrased this as "the test of all knowledge is experiment".

If we accept that science involves some experimental test of knowledge, then some of our friends who think of themselves as scientists will find themselves excluded. Consider the astrophysicists and cosmologists. It's all very well having a theory about how stars form, but if you're not willing to swirl a big cloud of hydrogen, helium, lithium, carbon etc. about and watch what happens to it for a few billion years then you're not doing the experiment. If you're not doing the experiment then you're not a scientist.

Our friends in what are called the biological and medical sciences also have some difficulty now. A lot of what they do is tested by experiment, but some of the experiments are not permitted on ethical grounds. If you're not allowed to do the experiment, maybe you're not a real scientist.

Another formulation (OK, I got this from the wikipedia entry on Science) sees science as a sort of systematic storytelling: the making of "testable explanations and predictions about the universe".

Under this definition, there's no problem with calling astronomy a science: you think this is how things work, then you sit, and watch, and see whether that happens.

Of course a lot of other work fits into the category now, too. There's no problem with seeing the "social sciences" as branches of science: if you can explain how people work, and make predictions that can (in principle, even if not in practice) be tested, then you're doing science. Psychology, sociology, economics: these are all sciences now.

Speaking of the social sciences, we must remember that science itself is a social activity, and that the way it's performed is really defined as the explicit and implicit rules and boundaries created by all the people who are doing it. As an example, consider falsificationism. This is the idea that a good scientific hypothesis is one that can be rejected, rather than confirmed, by an appropriately-designed experiment.

Sounds pretty good, right? Here's the interesting thing: it's also pretty new. It was mostly popularised by Karl Popper in the 20th Century. So if falsificationism is the hallmark of good science, then Einstein didn't do good science, nor did Marie Curie, nor Galileo, or a whole load of other people who didn't share the philosophy. Just like Dante's Virgil was not permitted into heaven because he'd been born before Christ and therefore could not be a Christian, so all of the good souls of classical science are not permitted to be scientists because they did not benefit from Popper's good message.

So what is science today is not necessarily science tomorrow, and there's a sort of self-perpetuation of a culture of science that defines what it is. And of course that culture can be critiqued. Why is peer review appropriate? Why do the benefits outweigh the politics, the gazumping, the gender bias? Why should it be that if falsification is important, negative results are less likely to be published?

Let's talk about Physics

Around a decade ago I was studying particle physics pretty hard. Now there are plenty of interesting aspects to particle physics. Firstly that it's a statistics-heavy discipline, and that results in statistics are defined by how happy you are with them, not by some binary right/wrong criterion.

It turns out that particle physicists are a pretty conservative bunch. They'll only accept a particle as "discovered" if the signal indicating its existence is measured as a five-sigma confidence: i.e. if there's under a one-on-a-million chance that the signal arose randomly in the absence of the particle's existence. Why five sigma? Why not three (a 99.7% confidence) or six (to keep Motorola happy)? Why not repeat it three times and call it good, like we did in middle school science classes?

Also, it's quite a specialised discipline, with a clear split between theory and practice and other divisions inside those fields. It's been a long time since you could be a general particle physicist, and even longer since you could be simply a "physicist". The split leads to some interesting questions about the definition of science again: if you make a prediction which you know can't be verified during your lifetime due to the lag between theory and experimental capability, are you still doing science? Does it matter whether you are or not? Is the science in the theory (the verifiable, or falsifiable, prediction) or in the experiment? Or in both?

And how about Psychology, too

Physicists are among the most rational people I've worked with, but psychologists up the game by bringing their own feature to the mix: hypercriticality. And I mean that in the technical sense of criticism, not in the programmer "you're grammar sucks" sense.

You see, psychology is hard, because people are messy. Physics is easy: the apple either fell to earth or it didn't. Granted, quantum gets a bit weird, but it generally (probably) does its repeatable thing. We saw above that particle physics is based on statistics (as is semiconductor physics, as it happens); but you can still say that you expect some particular outcome or distribution of outcomes with some level of confidence. People aren't so friendly. I mean, they're friendly, but not in a scientific sense. You can do a nice repeatable psychology experiment in the lab, but only by removing so many confounding variables that it's doubtful the results would carry over into the real world. And the experiment only really told you anything about local first year psychology undergraduates, because they're the only people who:

walked past the sign in the psychology department advertising the need for participants;
need the ten dollars on offer for participation desperately enough to turn up.

In fact, you only really know about how local first year psychology undergraduates who know they're participating in a psychology experiment behave. The ethics rules require informed consent which is a good thing because otherwise it's hard to tell the difference between a psychology lab and a Channel 4 game show. But it means you have to say things like "hey this is totally an experiment and there'll be counselling afterward if you're disturbed by not really electrocuting the fake person behind the wall" which might affect how people react, except we don't really know because we're not allowed to do that experiment.

On the other hand, you can make observations about the real world, and draw conclusions from them, but it's hard to know what caused what you saw because there are so many things going on. It's pretty hard to re-run the entire of a society with just one thing changed, like "maybe if we just made Hitler an inch taller then Americans would like him, or perhaps try the exact same thing as prohibition again but in Indonesia" and other ideas that belong in Philip K. Dick novels.

So there's this tension: repeatable results that might not apply to the real world (a lack of "ecological validity"), and real-world phenomena that might not be possible to explain (a lack of "internal validity"). And then there are all sorts of other problems too, so that psychologists know that for a study to hold water they need to surround what they say with caveats and limitations. Thus is born the "threats to validity" section on any paper, where the authors themselves describe the generality (or otherwise) of their results, knowing that such threats will be a hot topic of discussion.

But all of this—the physics, the psychology, and the other sciences—is basically a systematised story-telling exercise, in which the story is "this is why the universe is as it is" and the system is the collection of (time-and-space-dependent) rules that govern what stories may be told. It's like religion, but with more maths (unless your religion is one of those ones that assigns numbers to each letter in a really long book then notices that your phone number appears about as many times as a Poisson distribution would suggest).

Wait, I think you were talking about software

Oh yeah, thanks. So, what science, if any, is there in making software? Does there exist a systematic approach to storytelling? First, let's look at the kinds of stories we need to tell.

The first are the stories about the social system in which the software finds itself: the story of the users, their need (or otherwise) for a software system, their reactions (or otherwise) to the system introduced, how their interactions with each other change as a result of introducing the system, and so on. Call this requirements engineering, or human-computer interaction, or user experience; it's one collection of stories.

You can see these kinds of stories emerging from the work of Manny Lehman. He identifies three types of software:

an S-system is exactly specified.
a P-system executes some known procedure.
an E-system must evolve to meet the needs of its environment.

It may seem that E-type software is the type in which our stories about society are relevant, but think again: why do we need software to match a specification, or to follow a procedure? Because automating that specification or procedure is of value to someone. Why, or to what end? Why that procedure? What is the impact of automating it? We're back to telling stories about society. All of these software systems, according to Lehman, arise from discovery of a problem in the universe of discourse, and provide a solution that is of possible interest in the universe of discourse.

The second kind are the stories about how we worked together to build the software we thought was needed in the first stories. The practices we use to design, build and test our software are all tools for facilitating the interaction between the people who work together to make the things that come out. The things we learned about our own society, and that we hope we can repeat (or avoid) in the future, become our design, architecture, development, testing, deployment, maintenance and support practices.  We even create our own tools—software for software's sake—to automate, ease or disrupt our own interactions with each other.

You'll mostly hear the second kind of story at most developer conferences. I believe that's because the people who have most time and inclination to speak at most developer conferences are consultants, and they understand the second stories to a greater extent than the first because they don't spend too long solving any particular problem. It's also because most developer conferences are generally about making software, not about whatever problem it is that each of the attendees is trying to solve out in the world.

I'm going to borrow a convention that Rob Rix told me in an email, of labelling the first type of story as being about "external quality" and the second type about "internal quality". I went through a few stages of acceptance of this taxonomy:

Sounds like a great idea! There really are two different things we have to worry about.
Hold on, this doesn't sounds like such a good thing. Are we really dividing our work into things we do for "us" and things we do for "them"? Labelling the non-technical identity? That sounds like a recipe for outgroup homogeneity effect.
No, wait, I'm thinking about it wrong. The people who make software are not the in-group. They are the mediators: it's just the computers and the tools on one side of the boundary, and all of society on the other side. We have, in effect, the Janus Thinker: looking on the one hand toward the external stories, on the other toward the internal stories, and providing a portal allowing flow between the two.

So, um, science?

What we're actually looking at is a potential social science: there are internal stories about our interactions with each other and external stories about our interactions with society and of society's interactions with the things we create, and those stories could potentially be systematised and we'd have a social science of sorts.

Particularly, I want to make the point that we don't have a clinical science, an analogy drawn by those who investigate evidence-based software engineering (which has included me, in my armchair way, in the past). You can usefully give half of your patients a medicine and half a placebo, then measure survival or recovery rates after that intervention. You cannot meaningfully treat a software practice, like TDD as an example, as a clinical intervention. How do you give half of your participants a placebo TDD? How much training will you give your 'treatment' group, and how will you organise placebo training for the 'control' group? [Actually I think I've been on some placebo training courses.]

In constructing our own scientific stories about the world of making software, we would run into the same problems that social scientists do in finding useful compromises between internal and ecological validity. For example, the oft-cited Exploratory experimental studies comparing online and offline programming performance (by Sackman et al., 1968) is frequently used to support the notion that there are "10x programmers", that some people who write software just do it ten times faster than others.

However, this study does not have much ecological validity. It measures debugging performance, using either an offline process (submitting jobs to a batch system) or an online debugger called TSS, which probably isn't a lot like the tools used in debugging today. The problems were well-specified, thus removing many of the real problems programmers face in designing software. Participants were expected to code a complete solution with no compiler errors, then debug it: not all programmers work like that. And where did they get their participants from? Did they have a diverse range of backgrounds, cultures, education, experience? It does not seem that any results from that study could necessarily apply to modern software development situated in a modern environment, nor could the claim of "10x programmers" necessarily generalise as we don't know who is 10x better than whom, even at this one restricted task.

In fact, I'm also not convinced of its internal validity. There were four conditions (two programming problems and two debugging setups), each of which was assigned to six participants. Variance is so large that most of the variables are independent of each other (the independent variables are the programming problem and the debugging mode, and the dependent variables are the amount of person-time and CPU-time), unless the authors correlate them with "programming skill". How is this skill defined? How is it measured? Why, when the individual scores are compared, is "programming skill" not again taken into consideration? What confounding variables might also affect the wide variation in scored reported? Is it possible that the fastest programmers had simply seen the problem and solved it before? We don't know. What we do know is that the reported 28:1 ratio between best and worst performers is across both online and offline conditions (as pointed out in, e.g., The Leprechauns of Software Engineering, so that's definitely a confounding factor. If we just looked at two programmers using the same environment, what difference would be found?

We had the problem that "programming skill" is not well-defined when examining the Sackman et al. study, and we'll find that this problem is one we need to overcome more generally before we can make the "testable explanations and predictions" that we seek. Let's revisit the TDD example from earlier: my hypothesis is that a team that adopts the test-driven development practice will be more productive some time later (we'll defer a discussion of how long) than the null condition.

OK, so what do we mean by "productive"? Lines of code delivered? Probably not, their amount varies with representation. OK, number of machine instructions delivered? Not all of those would be considered useful. Amount of 'customer value'? What does the customer consider valuable, and how do we ensure a fair measurement of that across the two conditions? Do bugs count as a reduction in value, or a need to do additional work? Or both? How long do we wait for a bug to not be found before we declare that it doesn't exist? How is that discovery done? Does the expense related to finding bugs stay the same in both cases, or is that a confounding variable? Is the cost associated with finding bugs counted against the value delivered? And so on.

Software dogma

Because our stories are not currently very testable, many of them arise from a dogmatic belief that some tool, or process, or mode of thought, is superior to the alternatives, and that there can be no critical debate. For example, from the Clean Coder:

  The bottom line is that TDD works, and everybody needs to get over it.

No room for alternatives or improvement, just get over it. If you're having trouble defending it, apply a liberal sprinkle of argumentum ab auctoritate and tell everyone: Robert C. Martin says you should get over it!

You'll also find any number of applications of the thought-terminating cliché, a rhetorical technique used to stop cognitive dissonance by allowing one side of the issue to go unchallenged. Some examples:

"I just use the right tool for the job"—OK, I'm done defending this tool in the face of evidence. It's just clearly the correct one. You may go about your business. Move along.
"My approach is pragmatic"—It may look like I'm doing the opposite of what I told you earlier, but that's because I always do the best thing to do, so I don't need to explain the gap.
"I'm passionate about [X]"—yeah, I mean your argument might look valid, I suppose, if you're the kind of person who doesn't love this stuff as much I do. Real craftsmen would get what I'm saying.
and more.

The good news is that out of such religious foundations spring the shoots of scientific thought, as people seek to find a solid justification for their dogma. So just as physics has strong spiritual connections, with Steven Hawking concluding in A Brief History of Time:

  However, if we discover a complete theory, it should in time be understandable by everyone, not just by a few scientists. Then we shall all, philosophers, scientists and just ordinary people, be able to take part in the discussion of the question of why it is that we and the universe exist. If we find the answer to that, it would be the ultimate triumph of human reason -- for then we should know the mind of God.

and Einstein debating whether quantum physics represented a kind of deific Dungeons and Dragons:

  […] an inner voice tells me that it is not yet the real thing. The theory says a lot, but does not really bring us any closer to the secret of the "old one." I, at any rate, am convinced that He does not throw dice.

so a (social) science of software could arise as an analogous form of experimental theology. I think the analogy could be drawn too far: the context is not similar enough to the ages of Islamic Science or of the Enlightenment to claim that similar shifts to those would occur. You already need a fairly stable base of rational science (and its application via engineering) to even have a computer at all upon which to run software, so there's a larger base of scientific practice and philosophy to draw on.

It's useful, though, when talking to a programmer who presents themselves as hyper-rational, to remember to dig in and to see just where the emotions, fallacious arguments and dogmatic reasoning are presenting themselves, and to wonder about what would have to change to turn any such discussion into a verifiable prediction. And, of course, to think about whether that would necessarily be a beneficial change. Just as we can critique scientific culture, so should we critique software culture.

#### Software-ICs and a component marketplace
In the previous post, I was talking about Object-Oriented Programming, an Evolutionary Approach. What follows is a thought experiment based on that.

Chapter 6 of Brad Cox's book, once he's finished explaining how ObjC works (and who to buy it from), is concerned with his vision of how Object-Oriented software will be built. He envisions "Software-ICs"&mdash;compiled object files defining the code to support a single class (no need for header files, remember) that are distributed with documentation on how to use that class.

Developers or "software librarians" connect ICs together into collections called "categories", which are implemented as object libraries. It's a bit unfortunate that "category" is an inappropriate name mainly due to later reuse by NeXT; but then the alternate word "framework" is also unfortunate due to confusion with the computer science term (which allows that AppKit, WebObjects UIKit are frameworks, but Foundation, Quartz and so on are not). But it's an entirely understandable reuse: in Smalltalk-80 related methods are grouped into categories, and NeXT used the same terminology for a very similar purpose.

Interestingly, Cox allowed for the compiler to generate vtables of selectors for each category, a bit like the Amiga operating system's library format. That's to support having different variable types for selectors with the same name. Modern Objective-C doesn't support that; if you define selectors with the same name but different parameters or return values, you'll get a warning and your code might not work correctly.

Finally, an application is a network of categories connected by the linker. One (or perhaps more, depending on your design) of the categories in the application contains the app-specific classes.

My reason for bringing this up is that this vision of object-oriented software engineering closely models component-oriented hardware engineering by allowing for software shops to produce catalogs of the components they produce at each level. Just as you can order a single IC, or a circuit board with a few ICs connected, or a whole widget, so you could order a class, or a category, or an application. If you want to build a new application, you might buy a couple of classes from one vendor, a category from another vendor, then write a few classes yourself and integrate the whole lot.

Enough ancient book, talk about the real world.

We have a lot of this, and make quite a lot of use of it. There are loads of Objective-C classes, libraries and frameworks out there for us to use, and to some extent there are catalogs. Many of the components we can use are open source, which means that we can treat the class interfaces themselves as the catalogs. If we're lucky there'll be some documentation, perhaps in the form of AppleDoc or a README.

Unfortunately availability vastly outstrips discoverability. You have to go to multiple catalogs to ensure that you've exhausted the search space: Google Code, GitHub, BitBucket, SourceForge etc. in addition to finding commercial libraries which won't be listed in any of those places. Actual code search engines like OpenGrok and Koders are great for finding out about source code, but not so great for discovering it in the first place.

Metacatalogs like Cocoa Objects, Cocoa Controls and CocoaOpen solve part of this problem by letting people list their source code in a single place, but because they're incomplete they only add to the number of places you need to search.

Then, once you've got the component, what do you do? Are you meant to drop the source files into your project? Should you drop the project in and add the library as a dependency of your app? Should you use CocoaPods?

Learn from what we already do

Just as we already push most of the apps we write to a single app store where customers can discover, purchase and install apps in a state where they're ready to use, we should do the same with components.

[Please bear in mind that like most descriptions of ideas, a lot of nuances and complexity are known but are elided below for the sake of clarity. Comment brownie points will not be awarded for comments that explain how I haven't considered case X; I probably have.]

A component store would, for browsers, start off very similar to Cox's idea of a component catalog. You'd go to it and search for a component that suits your needs. You could see a "spec sheet" for each component detailing what it does, what it costs, the terms of using it and that sort of thing. You'd then buy the component if it's paid for and download it. If the licence permits it you could download the source, too.

The download would drop the binary and headers into a folder that Xcode would recognise as an additional SDK. It would also drop the documentation in docset format into a standard location. An Xcode project would just need to point at the additional SDK and it could pick up all of the components available to the developer.

From the perspective of a component manufacturer, the component store would look a little like iTunes Connect. You'd write your code, then package it up for the store in a standard way along with the description that goes into the "spec sheet". For open source projects that could just involve git push componentstore master to have the store itself generate the binaries and documentation from the source code.

#### Solaris iPhone Edition
Apple's one new feature in Snow Leopard is support for Exchange, which if not squarely an Enterprise lure is certainly bait for medium businesses. But here we hit Apple's perennial problem; they want to sell more into businesses (because that's where at least 2/3 of all PC money is to be made) but they want to design their systems for home users. When a system is designed to cover every possible potential use for a computer we end up with Windows, which is the kind of "few things to all people" solution that Apple are - rightly - keen to avoid. But as Tim Cook's "state of the Mac" segment in the recent laptop event showed, one of Apple's biggest growth areas is education which is organised along enterprisey lines.Their solution thus far has been a partial one; we get Mac OS X which is basically a consumer OS, and then we get Mac OS X server which is the same OS with a few configuration changes and extra apps to support being used as a workgroup server. This is less distinct than the changes between Mac OS X and iPhone OS X, but the principle is the same; the same technology is used in different ways, so we get different interfaces to it. Note that these aren't really very divergent products - a UNIX expert could set up an Open Directory Master on a standard Mac OS X box were they so inclined. We get the Mac Pro and the XServe as nods to the existence of more powerful hardware than the iMac. While Apple do have a network of business development managers, enterprise sales people, sales engineers and so on who can support larger customers, their capabilities and freedom are restricted by working on a consumer product in a consumer organisation.Assuming that Apple aren't going to retreat and consolidate all of their effort on the consumer/prosumer, the logical plan seems to be "the same only more so"; carry on the scheme of applying a common technology base to multiple markets, but with the product interfaces and configurations being specific to the role in which they'll be used. Empower those enterprise sales, support and development teams to make the changes required in both the shared technology base and the domain-specific parts in order to advance their own cause. Allow them to do so in such a way that the consumer focus of the standard products is not diluted. To do all this, what Apple would need is to clearly delineate their Core OS, Consumer OS and Server OS engineering groups, while adding staff, expertise and intellectual property to their Server OS, Server Hardware and Enterprise Support groups.The bit about "adding staff, expertise and intellectual property to their Server OS, Server Hardware and Enterprise Support groups" can be easily achieved by using the Blue Peter principle. Here's one I prepared earlier. And no, I'm not going mad. Sun have plenty of experience in supporting larger customers and what marketing people like to call vertical markets, and have some good technology: hardware, operating systems software, enterprise services and applications. Their only problem is that they can't make any money on it. On the other hand with Apple it seems that the money is there to be made, and the problem is stepping up to that plate without compromising the consumer products. Consolidating Mac OS X [+ Server] and Solaris 10 would not be trivial but is not beyond the realms of fantasy. NeXTSTEP ran on SPARC hardware, and as we know that Mac OS X runs on PPC, two different Intel architectures and ARM it's likely that the effort to port Mac OS X to SPARC would not be great. But perhaps more useful in the short term is that OpenStep ran on Solaris before, and could do again. Even though Sun have switched Solaris to a SYSV-derived platform, due to Apple's recent push for standardisation with Leopard the two OS are likely more source-code compatible than NeXTSTEP and SunOS 4 ever were. Getting Cocoa up on Solaris would mean that application portability (for the sorts of apps that server admins will want - including Apple's own server admin tools, not for OmniDazzle) becomes viable while the combined company (Snapple?) concentrate on integrating the core tech. They could even get Jonathan Schwartz to do the coding.Another factor in this proposition is that JAVA is cheap. Apple currently have about $20B in cash and Sun's shares are worth $3.6B, but taking into account that Sun have lost 98% of their dot-com-boom value without slowing their R&amp;D projects, the value for money when you want them for their tech, smarts and goodwill rather than their user base is astounding.Oh, and speaking of JAVA, what about Java? Java currently represents Sun's main income due to the licensing scheme, but Apple's investment in the platform has declined over time from the Rhapsody days of "everything is Java"; currently the available Java on Mac OS X lags behind Sun's version and isn't ppc64 compatible. The WebObjects team (and hence the Apple store and iTunes) have a heavy Java investment, while other teams have dropped Java (Cocoa) and still others eschew it completely. The iPhone has a very busy developer ecosystem - and absolutely no Java. Where the hypothetical Snapple would leave Java is entirely open, but the option of packaging up the combined company's Java assets and re-selling them would seem unnecessary, unless you thought that even $3.6B was too much to pay.

#### Solving the underlying problem
As a software engineer, it’s easy to get work engineering software. Well, maybe not easy, but relatively so: that is the kind of work that comes along most. The kind of work that people are confident I can do. That they can’t do, so would like me to do for money.

It’s also usually the worst work available.

I don’t want to take your shopping list of features, give you a date and a cost, then make those features. Neither of us will be very happy, even if it goes well.

I want to get an understanding of your problem, and demonstrate how software can help in solving it. Maybe what we need to understand isn’t the problem you presented, but the worse problem that wasn’t on your mind. Or the opportunity that’s worth more than a solution to either problem.

Perhaps we ask a question, in solving your problem, to which the answer is that we don’t know, and now we have another problem.

You might not need me to build all of the features you thought of, just one of them. Perhaps that one works better if we don’t build it, but configure something that already exists. Or make it out of paper.

You understand your problem and its domain very well. I understand software very well. Let’s work together on combining that expertise, and both be happier in the process.

#### Some bloody genius
Link to the image, because I know it's too wide for the Blogger template to display properly.

#### Some LightReading about mobile app security
[This article was co-written with @securityninja]

If mobile app security is failing, it’s up to the security industry, not developers, to repair it.

An article published yesterday at security news site DarkReading announces “Developers not applying secure development life cycle practices in mobile app production”. The author finds many faults with the way application security is treated by mobile app developers, but doesn’t address what we believe to be the underlying problem: the engagement between security specialists and app makers.

Before addressing this point, though, there are several inaccuracies in the article that need to be corrected. Ericka Chickowski describes the mobile app landscape as “a development environment still in its infancy and no real standards to lead the way”, though in fact developers are able to bring a lot of the experience and tools they’ve used in building desktop and web apps to the party. The Cocoa Touch SDK used in Apple’s iOS shares a common heritage with Mac OS X’s Cocoa, just as the Java used in Android and SilverLight in Windows Phone 7 strongly resemble their desktop counterparts. For example the Code Access Security approach introduced in Silverlight ended up being adopted as the Code Access Security approach for the whole .NET framework.

Taking the 50,000 foot view, mobile apps look a lot like any other software application: there’s a client that’s delivered to the user somehow, which has some local storage and (often) communicates to an online service via the internet. This is exactly the situation we see with web apps, and indeed a lot of the techniques used to secure web apps are directly applicable in the mobile world. Sure there are new challenges to address with mobile, but these represent a tweak, not a rewrite, of security advice for developers.

It is probably fair to say security guidance and testing tools aren’t quite as mature as they are in the web application security world but to say they don’t exist is false. One of us (David) is an author of open source tools which help reviewers find vulnerabilities in mobile applications and has to correct the original author on this point. In addition to my own tools others exist: in fact there is a Live CD called the ARE (Android Reverse Engineering) VM which was created solely to allow people to security review/test Android applications and established commercial offerings from the likes of Veracode.

Chickowski also states that no secure coding guidelines for mobile applications exist: OWASP have a mobile security project which is very useful and will only get better over time. The OWASP project includes iGoat and GoatDroid: insecure apps for the iPhone and Android respectively that developers can use to understand what vulnerable code looks like and how to detect and fix security problems on those mobile platforms.

Both Apple and Android have good secure development documentation. The Apple “Introduction to Secure Coding” document has been around since 2006 with guidance for iOS development added in 2008. Apple’s World-Wide Developer Conference (WWDC) includes multiple sessions on application security; these sessions are freely available to registered developers. Google has produced similar guidance as part of their development reference site so to say no guidance exists and that Apple and Google have only just started to think about needs to be corrected. In fact any one who spends a small amount of time researching the architecture of those platforms (as well as others including WP7) will understand that security has been a consideration pretty much since day one.

The final point we want to pick up from the article is the following line: “Rapid and Agile Development causes changes to happen in very short iterations, thus security gets overlooked and becomes a nice thing to do but rarely gets done.” This is certainly not a mobile specific issue and as people who work in a company where we have security integrated throughout an Agile process, we can tell you the security deliverables in SDLC don’t really change much. Sure it means you have to do security testing and reviews more often but that doesn’t mean security should be, or always is overlooked. You can try to blame developers, you can even try to place the blame at the door of security professionals but if the business doesn’t want to produce secure code there is very little those people can do. 

So what’s actually behind the apparent lack of security practice among mobile app developers? We see a clue in the fact that Chickowski’s article was published at a security news site, not at a site for mobile developers. Security practitioners telling each other how apps fail at security can be entertaining and make for good conversation in the bars at conferences, but we need to engage the people making the apps if we want to effect change in the way those apps are made. Developers, project managers and executive officers need to be able to evaluate the risk that they are exposing their customers and their businesses to. They need to know how to measure the security posture of their apps and to make decisions on what changes to make, feeding those decisions into the same process they use to prioritise features and bug fixes. In short, we need to help developers to get this right, not call them out when they get it wrong,

David Rook (@securityninja) is the Application Security Lead at Realex Payments.

Graham Lee (@iamleeg) is the Smartphone Security Boffin at O2 Labs.

#### Some so-called expert
There's a comedy sketch being frequently tweeted called The Expert. Now, all programmers will be aware that there is nothing funnier than interpreting a joke literally and telling everyone the many ways in which it's wrong, and that there is no way to be seen as a more intelligent and empathetic person than to do this. So here we go: what are all the inexpert things this "expert" does?

Firstly, having been told how important the strategic initiative is, he makes no attempt to actually find out what it is, and how his task is connected to the objectives described. This means that he doesn't know anything about the context of his work, which is just setting himself up for all sorts of trouble. It's like a programmer going "yeah sure, I can add a second copy of that goto line" without checking whether they're working on some sort of security-sensitive module.

He refuses to accept any form of creative solution to the problem, and his project manager is correct to try to tactfully defer his immediate refusal to do the work asked. Immediately saying "no, I can't do that" is identical to saying "I have never done that, and I cannot imagine any novelty entering my life". This is not symptomatic of expertise, but of narrow-mindedness.

A pause, and a gathering of resources, leads us to conclude that some of the tasks set are eminently achievable, making this alleged expert look like the comfort-zone-hogging risk-averse luddite that perhaps he is. Of course you can draw a red line with inks of other colours, for example. You simply rely on the relativistic Doppler effect, or on fluorescent properties of the materials. Of course you can draw seven lines all perpendicular, if your diagram can extend into seven dimensions. And that is of course assuming a Euclidean geometry for the diagram; an assumption that our "I know best" expert doesn't even think to question. Alternatively, you can find out what the time-dependent evolution of the diagram is, as it may be that a total of seven lines that are each instantaneously perpendicular to the other lines present but that do not all simultaneously exist is a sufficient solution. Again, our unimaginative expert doesn't think about that. In fact, he never really explores whether the perpendicularity requirement means mutually perpendicular, he just proceeds to mansplain to the client representative why he is right and she is wrong.

Assured of his expertise, he then injects sarcasm into his voice in a condescending fashion. "I'm sure your target audience doesn't exist solely of those people." Again, this is indicative of a lack of empathy and an unwillingness to consider other viewpoints than his own.

Although, having said that, he's pretty quick to demur to authority, and on the few occasions that he does want to enquire about the requirements, does not pursue the matter if someone else interrupts.

This is an "expert" who is going to go away with an incomplete understanding of the problem, and will likely fail to give a satisfactory solution. Often such people will then seek to externalise any responsibility for the failure, complaining that the requirements weren't clear or that the clients had unrealistic expectations. Maybe they weren't and they did, but as an expert it's his responsibility to understand those and apply his skills to solving the problem at hand, not to find ways to throw other people under the proverbial bus.

The manager in this video is clearly the sanest voice, and also manages to keep his frustration at his own mistake somewhat bottled. The extent of that mistake? He has contracted an "expert in a narrow field", who "doesn't see the overall picture", and put him in a meeting with their client for which he was totally unprepared. So it's a shame that the expert's grandest commitment—to inflate a balloon of unknown quality and structure into the shape of a kitten—is made without the manager around to intermediate. He might have been able to intervene before the physical contact between the "expert" and the designer, which should be considered wholly inappropriate for a business meeting.

Maybe it was a mistake to put someone so junior in front of the client without some coaching. Hopefully, with appropriate mentoring and support, our "expert" can grow to be a mature, empathetic and positive contributor to his team.

#### Something old, something new
Sometimes, what you want out of a software system is something
unexciting. Something proven to work, that's well-understood, and has
had all of the kinks smoothed out over years (or even decades) of
refinement.

Of course, what you get when that's what you want is something like
Unix. Where the thing it was designed for...well, let's not pretend
that
Unix was ever designed. The
thing it was used for - hosting the runoff typesetting system that is
only applied these days to formatting its own manual - is so far from
its current application as a
bootloader sitting between your other bootloader and the thing you're using your computer for
that you can't understand why someone would think the same thing would
be used for both purposes.

And you find that the kinks have not been ironed out so much as, well,
baked in. The limitation that processes communicate via an untyped
character stream with unspecified eight-bit encoding is not a
limitation, it's
a design philosophy.

So what you really want to do is to throw away all of that cruft,
all of the mistakes of the past, and go somewhere new. Somewhere
exciting. Somewhere...like node.js? Well no,
JavaScript is of course far from new and exciting, even
the jokes about it are
years old now.

OK, so bad example, but maybe you have your own counter-example. The
one true environment that will be used for all future software. Well,
at least for the next nine months, before its killer comes along.

Of course, what you get when that's what you want is another rewrite
of all of the things that were working anyway. Your new environment
probably comes with a package manager written from scratch in the new
language, when there was really nothing much wrong with CPAN.

Keeping on the programming theme, because that's what we tend to make
computering out of, most languages that have any traction these days
almost certainly have C language bindings. So it's no surprise that
the most popular open source projects are all written in C...

...OK, now that you've stopped laughing and got your breath back, you
can appreciate that actually the lingua franca of modern (ahem)
operating systems is not used to cut through the babel of other
programming languages, but just to write their runtimes. Because
that's what C is good at: secure, bounds-checked execution
environments.

And you can also appreciate that all of the novelty of the new shiny
is used to rewrite the things that already existed and already worked,
replacing them with new implementations that work in surprising ways
(and don't work in surprising ways, too).

There's good in the old. There's bad in the new. There's bad in the
old, and good in the new, too. There are plenty of social, political,
economic, and - maybe - technical factors in whether the old survives,
or the new thrives. Arguments for the old or the new for their own
sake are facile and misguided.

#### Sound bites considered harmful
Knuth said:

premature optimization is the root of all evil.

Only, what he actually said was:

There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail. After working with such tools for seven years, I've become convinced that all compilers written from now on should be designed to provide all programmers with feedback indicating what parts of their programs are costing the most; indeed, this feedback should be supplied automatically unless it has been specifically turned off.

In that context, the quote takes on a very different meaning.

#### Specific physical phenomena
Continuing the theme of exploring the exercises in Software: A Technical History:

Give an example of a specific physical phenomenon that software dependson in order to run. Can a different physical phenomenon be used? If so, giveanother example phenomenon. If not, explain why that’s the only physicalphenomenon that can be used.
Kim W. Tracy, Software: A Technical History (p. 43)

My short, but accurate, answer is “none”. Referring back to the definition of software I quoted in Related methods and tools, nothing in that definition implies or requires any particular physical device, technology, or other phenomenon.

Exploring the history of computing, it’s clear that the inventors and theoreticians saw computers as automation (or perhaps more accurately flawless repetition) of thought:

We may compare a man in the process of computing a real number to a machine which is only capable of a finite number of conditions…
Alan M. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem (§1)

Or earlier:

Whenever engines of this kind exist in the capitals and universities of the world, it is obvious that all those enquirers who wish to put their theories to the test of number, will apply their efforts so to shape the analytical results at which they have arrived, that they shall be susceptible of calculation by machinery in the shortest possible time, and the whole course of their analysis will be directed towards this object. Those who neglect the indication will find few who will avail themselves of formulae whose computation requires the expense and the error attendant on human aid.
Charles Babbage, On the Mathematical Powers of the Calculating Engine

For any particular physical tool you see applied to computing—mercury delay line memory, “silicon” chips (nowadays the silicon wafer is mostly a substrate for other semiconductors and metals), relays, thermionic valves, brass cogs, hydraulic tubes—you can replace it with other tools or even with a person using no tools at all.

So it was then, when the mechanical or digital computers automated the work of human computers. As it was in the last century, when the “I.T.” wave displaced human clerical assistants and rendered the typing pool redundant, and desktop publishing closed the type shop. Thus we see today, that categorization systems based on “A.I.” are validated on their performance when compared with human categorizers.

Nothing about today’s computers is physically necessary for their function, although through a process of iterating discovery with development we’ve consolidated on a physical process (integrated semiconductor circuits) that has particular cost, power, performance, manufacturing, and staffing qualities. A more interesting question to ask would be: what are the human relations that software depends on in order to run? In other words, what was it about these computers, typists, typesetters, paraprofessionals, and so on that made their work the target of software? Can a different human relation be used?

#### Specifications for interchanging objects
One of the interesting aspects of Smalltalk and similar languages including Objective-C and Ruby is that while the object model exposes a hierarchy of classes, consumers of objects in these environments are free to ignore the position of the object in that hierarchy. The hierarchy can be thought of as a convenience: on the one hand, for people building objects ("this object does all the same stuff as instances of its parent class, and then some"). It's also a convenience for people consuming objects ("you can treat this object like it's one of these types further up the hierarchy").

So you might think that -isKindOfClass: represents a test for "I can use this object like I would use one of these objects". There are two problems with this, which are both expressed across two dimensions. As with any boolean test, the problems are false positives and false negatives.

A false positive is when an object passes the test, but actually can't be treated as an instance of the parent type. In a lot of recent object-oriented code this is a rare problem. The idea of the Liskov Substitution Principle, if not its precise intent as originally stated, has become entrenched in the Object-Oriented groupthink.

I've worked with code from the 1980s though where these false positives exist: an obvious example is "closing off" particular selectors. A parent class defines some interface, then subclasses inherit from that class, overriding selectors to call [self doesNotRecognize:] on features of the parent that aren't relevant in the subclass. This is still possible today, though done infrequently.

False negatives occur when an object fails the -isKindOfClass: test but actually could be used in the way your software intends. In Objective-C (though neither in Smalltalk[*] nor Ruby), nil _does_ satisfy client code's needs in a lot of cases but never passes the hierarchy test. Similarly, you could easily arrange for an object to respond to all the same selectors as another object, and to have the same dynamic behaviour, but to be in an unrelated position in the hierarchy. You _can_ use an OFArray like you can use an NSArray, but it isn't a kind of NSArray.

[*] There is an implementation of an Objective-C style Null object for Squeak.

Obviously if the test is broken, we should change the test. False negatives can be addressed by testing for protocols (again, in the languages I've listed, this only applies to Objective-C and MacRuby). Protocols are unfortunately named in this instance: they basically say "this object responds to any selector in this list". We could then say that rather than testing for an object being a kind of UIView, we need an object that conforms to the UIDrawing protocol. This protocol doesn't exist, but we could say that.

Problems exist here. An object that responds to all of the selectors doesn't necessarily conform to the protocol, so we still have false negatives. The developer of the class might have forgotten to declare the protocol (though not in MacRuby, where protocol tests are evaluated dynamically), or the object could forward unknown selectors to another object which does conform to the protocol.

There's still a false positive issue too: ironically protocol conformance only tells us what selectors exist, not the protocol in which they should be used. Learning an interface from a protocol is like learning a language from a dictionary, in that you've been told what words exist but not what order they should be used in or which ones it's polite to use in what circumstances.

Consider the table view data source. Its job is to tell the table view how many sections there are, how many rows there are in each section, and what cell to display for each row. An object that conforms to the data source protocol does not necessarily do that. An object that tells the table there are three sections but crashes if you ask how many rows are in any section beyond the first conforms to the protocol, but doesn't have the correct dynamic behaviour.

We have tools for verifying the dynamic behaviour of objects. In his 1996 book Superdistribution: Objects as Property on the Electronic Frontier, Brad Cox describes a black box test of an object's dynamic behaviour, in which test code messages the object then asserts that the object responds in expected ways. This form of test was first implemented in a standard fashion, to my knowledge, in 1998 by Kent Beck as a unit test.

Unit tests are now also a standard part of the developer groupthink, including tests as specification under the name Test-Driven Development But we still use them in a craft way, as a bespoke specification for our one-of-a-kind classes. What we should really do is to make more use of these tests: substituting our static, error-prone type tests for dynamic specification tests.

A table view does not need something that responds to the data source selectors, it needs something that behaves like a data source. So let's create some tests that any data source should satisfy, and bundle them up as a specification that can be tested at runtime. Notice that these aren't quite unit tests in that we're not testing our data source, we're testing any data source. We could define some new API to test for satisfactory behaviour:

- (void)setDataSource: (id &lt;UITableViewDataSource&gt;)dataSource {
  NSAssert([Demonstrate that: dataSource satisfies: [Specification for: @protocol(UITableViewDataSource)]]);
  _dataSource = dataSource;
  [self reloadData];
}

But perhaps with new language and framework support, it could look like this:

- (void)setDataSource: (id @&lt;UITableViewDataSource&gt;)dataSource {
  NSAssert([dataSource satisfiesSpecification: @specification(UITableViewDataSource)]);
  _dataSource = dataSource;
  [self reloadData];
}

You could imagine that in languages that support design-by-contract, such as Eiffel, the specification of a collaborator could be part of the contract of a class.

In each case, the expression inside the assertion handler would find and run the test specification appropriate for the collaborating object. Yes this is slower than doing the error-prone type hierarchy or conformance tests. No, that's not a problem: we want to make it right before making it fast.

Treating test fixtures as specifications for collaboration between objects, rather than (or in addition to) one-off tests for one-off classes, opens up new routes for collaboration between the developers of the objects. Framework vendors can supply specifications as enhanced documentation. Framework consumers can supply specifications of how they're using the frameworks as bug reports or support questions: vendors can add those specifications to a regression testing arsenal. Application authors can create specifications to send to contractors or vendors as acceptance tests. Vendors could demonstrate that their code is "a drop-in replacement" for some other code by demonstrating that both pass the same specification.

But finally it frees object-oriented software from the tyranny of the hierarchy. The promise of duck typing has always been tempered by the dangers, because we haven't been able to show that our duck typed objects actually can quack like ducks until it's too late.

#### Sprouts
Having discussed reasons for change with a colleague on my team, we came up with the sprouts of change. Good software is antifragile in the face of changing:

Situation
People
Requirements
Organisation
Understanding
Technology
Society

Like any good acronym, it's really tenuous.

#### Standing at the Crossroads
A while back I wrote Conflicts in my Mental Model of Objective-C, in which I listed a few small scale dichotomies or cognitive dissonances that plagued my notion of my work. I just worked out what the overall picture is, the jigsaw into which all of these pieces can be assembled.

And I do mean just. It's about 1AM on Christmas Eve, but this picture hit me so hard I couldn't stop thinking about it without writing it down and getting it out of my head. If it doesn't explain everything, it shows me the shape of the solution at least.

A tale of two Apples

I believe that everything I wrote in the Conflicts post can be understood in terms of two different and (of course) opposed models of Apple. I also believe that the two models are irreconcilable, but that the opposition is also accidental, not essential. That by removing the supposed conflict between them, everything I thought was a problem can be resolved.

It was the best of iPads

The iPad is, I would argue (and accept that this is a subjective argument) the most tasteful application of computing technology to the world of many computer users. I would further argue, and this is perhaps on firmer ground, that the reason the iPad is so tasteful is because Apple spend a lot more time and resources on worrying about questions of taste than many of their competitors and others in the industry.

There are many visions that have combined to produce the iPad, but interestingly the one that I think is clearest is John Scully's Knowledge Navigator. In the concept videos for Knowledge Navigator, a tablet computer with natural language speech comprehension and a multi-touch screen is able to use the many hyperlinked documents available on the Web to answer a wide range of questions, make information available to its users and even help them to plan their schedules.

This is what we have now. This is the iPad, with Siri and a host of third-party applications. Apple even used to use the slogan "there's an app for that". Do you have a problem? You can probably solve it with iOS and a trip to the app store.

It was the worst of iPads

OK, what do you do if your problem isn't solved on the app store, or the available solutions aren't satisfactory?

Well first, you'd better get yourself another computer because while the iPad is generally designed for solving problems it isn't designed for solving general problems. You might be able to find some code editors on the iPad, but you sure aren't going to use them to write an iPad app without external assistance.

OK, so you've got your computer, and you've learned how to do the stuff that makes iPad apps. Now you just pay a recurring fee to be allowed to put that stuff onto your iPad. And what if you want to share that with your friends? Only if it meets Apple's approval.

If the iPad is the Knowledge Navigator, it is not the Dynabook. A Dynabook is a computer that you can use to solve your problems on, but it's also one on which you can create solutions to your own problems.

The promise of the Dynabook is that if you understand what your problem is, you can model that problem on the Dynabook. You model it with objects-either your own or ones supplied for you. You can change and create these objects until they model the problem you have, at which point you can use them to compute a solution.

The irony is that we have all the parts needed in a Dynabook, all in the iPad. Computer so simple even children can use it? Check. Objects? Check. Repositories of objects created by other people so we don't have to rewrite our own basic objects all the time? We call that CocoaPods (or RubyGems, or whatever the poison in your area of the world). But we just can't put all of these things together on that computer itself.

That would be distasteful. That might let people do things that make the iPad look bad. That might mean iPads providing experiences that haven't been vetted by the mothership.

Does using an iPad ever make you wonder how iPads work? What they can do? What you can make them do? You can answer these questions, but not using an iPad. Your Knowledge Navigator does not know the route to that particular destination.

This is what is truly meant when it is said that the iPad is not upgradeable. Forget swapping out memory chips or radio transmitters. Those are just lumps of sand inside a box made of melted sand and refined rock. The iPad is not upgradeable because you are stuck with the default experience: the out-of-the-box facilities plus those that have been approved from on high. It might be good, but it might not be good enough.

Notice that this is not an "everyone must program" position. That would be a very bad experience. The position is rather "everyone must have the facility, should they be so inclined, to make their computer better for them than the manufacturers did".

Conclusions?

I think that the Apple described above is not at the intersection of technology and the liberal arts. It is at the border, a self-appointed barrier of things that might flow between the two.

I believe that the two visions can be reconciled, and that a thing can be both the Knowledge Navigator and the Dynabook. I don't believe you have to disable some experiences to provide others. I believe that Apple the champions of tasteful computing can be applauded at every turn while Apple the high priests of the church of computing can be fought tooth and nail.

Enablers? Yes, please. Arbiters? No, thanks.

#### Stay on target…
I introduce the kind of customer who needs the Labrary's advice with the following description:

  Your software team was a sight to behold, when it started out. You very quickly got to an MVP, validated its fit with early successes, iterated on the user experience and added the missing features. You hired a few more developers to cover the demand.
  
  Now, things are starting to feel slower. The team insists they’re still continuing apace, but you haven’t kept that initial excitement. Developers are grumbling about technical debt. The backlog keeps growing. Testers aren’t keeping up - despite automation. The initial customers aren’t getting the benefits they first expected, and new customers aren’t being won at the rate you’d like.
  
  The problem was that the way you hit it out of the park worked well in the early days, when you had a green field project and no existing code or customers to support. Now your customers expect all new features and surprising and delightful interactions: but they also expect nothing to change, and certainly not to break. The desired qualities of your software have changed, and so the quality of your software must change.

Plenty of people, typically CTOs and heads of software development, typically at growth scale, identify with this description, so it's worth digging deeper into how it comes about.

At the early stage, your company has a small team, a vague idea of what the product is, and no customers. Literally anything your engineering team can do will be valuable: it will either be a product that fits the market, or tell you where the market isn't. Obviously there's some hand-waving about being able to market and sell whatever it is that your engineering team build, but by and large anything you come up with is somehow useful. You are either defining a new market, in which case all work is market-leading, or entering an established market, in which case your direction is clear. It's hard to make a wrong decision at this stage, but very easy to stick to one.

And while we all know the horror stories about shipping your prototype to production, it's actually not a bad plan at this stage. You don't know what will or won't work, so getting something out there quickly is exactly the right thing to do. And your developers probably have a base standard of maturity even for prototype projects, so you'll have version control, some form of testing infrastructure and CI, external libraries for data storage, it won't be a complete wild west.

Things go, loosely speaking, in one of two directions here. If you fail to find the right customers and the right product, you're out of money, thanks for playing. If you find the right product for the right people, then congratulations! You get more money, either through revenue or a funding round, and you grow the company. Maybe the programmer you were paying before becomes the CTO, maybe one or two of the contractors you worked with come on as perms, and you get a couple of new people come on in return for an OK salary and the promise of the stock sometime being worth something. One of those people is even a QA!

Of course, the cash injection (particularly if it came as a lump through funding that can be drawn down as necessary) gives you the headroom to do things properly. Technologies are chosen, an architecture is designed (usually just by connecting the technologies with arrows), and an attempt is made to build the new thing, support the old thing, and continue adding new features and differentiate in the market. A key customer demographic is sold the promise of the new system (it being exciting and more capable, at least that is what the roadmap says), takes delivery of the old system (it being ready), then takes up time asking for the new features. You either divert resources from the new system to the old to add the features there, or invent some unholy hybrid where your existing thing makes calls to the new thing for the new features with a load of data consistency glue binding the two together. We'll call these customers "saps" for now. Also, whether you've caught up to your competitors or your competitors to them, you're now having to maintain an edge.

Let's take stock here. You have:

Some saps, giving us actual money, on the old platform.
Some hope that things will be easier once everything's on the new platform.
Pressure to stay ahead of/catch up to the competition in both places.

That's more work! But it's OK, you've got more people. But where you add to the old system (which pleases your saps) you take away from the new, so tend to favour unintrusive patches rather than deeper changes there. Which makes it harder to understand, and harder to support, which is more work! OK, so hire more people! But now engineering costs more. OK, so sell the original thing (not the new thing, it isn't ready yet) to a few more customers! But now there are more customers, demanding more features, and more support. OK, so hire more people!

Run through that cycle a few times and you end up in the place I described in the Labrary blurb. You've got a big team, filled with capable engineers, working hard, and delivering...not as much as anyone would like. The problem is that working on the software is pulling them away from working for the company. The other problem is that you're measuring how much the software gets worked on.

#### Staying power
You would imagine that by now I would have come to realise how long my attention span is and worked to find projects that fit within it, but no. This is one of the changes I need to make soon.

So often I start a project really excited by it, but am really excited by something else before the end. Book projects always work that way, and quite a few software projects. Sometimes even talks, given a long enough lead time between being asked for a topic and actually giving the talk.

The usual result is that I become distracted before the end of the project, which leads to procrastination. That then makes it take longer, which only increases the distraction and disengagement.

What I'm saying is that if I ever say that I'm thinking of starting a PhD, you have my permission to chastise me. Four years is not within my observed boredom limit. Six months is closer to the mark.

#### Still no silver bullet?
In his 1986 article No Silver Bullet—Essence and Accident in Software Engineering, Fred Brooks suggests that there'll never be a single tool, technique, or fad that realises an order-of-magnitude improvement in software engineering productivity. His reason is simple: if there were, it would be because current practices make software engineering ten times more onerous than they need to be, and there's no evidence that this is the case. Instead, software engineering is complex because it provides complex solutions to complex problems, and that complexity can't be removed without failing to solve the complex problem.

Unfortunately, the "hopes for the silver" that he described as not being silver bullets in the 1980s are still sold as silver bullets.

Ada and other high-level language advances. “Ada will not prove to be the silver bullet that slays the software productivity monster. It is, after all, just another high-level language, and the big payoff from such languages came from the first transition, up from the accidental complexities of the machine into the more abstract statement of step-by-step solutions.” Why, then, do we still have a pre-Cambrian explosion of new programming languages, and evangelism strike forces pooh-poohing all software that wasn't written in the new hotness? On the plus side, Brooks identifies that “switching to [Ada will be seen to have] occasioned training programmers in modern software design techniques”. Is that happening in strike force land?

Object-oriented programming. “Such advances can do no more than to remove all the accidental difficulties from the expression of the design. The complexity of the design itself is essential; and such attacks make no change whatever in that.” The same ought to go for the recent resurgence in function programming as a silver bullet idea: unless our programs were 10x as complex as they need to be, applying new design constraints makes equally complex programs, specified in a different way.

Artificial intelligence. “The hard thing about building software is deciding what to say, not saying it. No facilitation of expression can give more than marginal gains.” This is still true.

Expert systems. “The most powerful contribution of expert systems will surely be to put at the service of the inexperienced programmer the experience and accumulated wisdom of the best programmers. This is no small contribution.” This didn't happen, and expert systems are no longer pursued. Perhaps this silver bullet has been dissolved.

“Automatic” programming. “It is hard to see how such techniques generalize to the wider world of the ordinary software system, where cases with such neat properties [as ready characterisation by few parameters, many known methods of solution, and existing extensive analysis leading to rules-based techniques for selecting solutions] are the exception. It is hard even to imagine how this breakthrough in generalization could conceivably occur.”

Graphical programming. “Software is very difficult to visualize. Whether we diagram control flow, variable scope nesting, variable cross-references, data blow, hierarchical data structures, or whatever, we feel only one dimension of the intricately interlocked software elephant.” And yet visual “no-code solutions” proliferate.

Program verification. “The hardest part of the software task is arriving at a complete and consistent specification, and much of the essence of building a program is in fact the debugging of the specification.” Indeed program verification is applied more widely now, but few even among its adherents would call it a silver bullet.

Environments and tools. “By its very nature, the return from now on must be marginal.” And yet software developers flock to favoured IDEs like gnus to watering holes.

Workstations. “More powerful workstations we surely welcome. Magical enhancements from them we cannot expect.” This seems to have held; remember that at the time Rational was a developer workstation company, who then moved into methodologies.

Meanwhile, of his “promising attacks on the conceptual essence”, all have accelerated in adoption since his time.

Buy versus build. Thanks to free software, we now have don't-buy versus build.

Requirements refinement and rapid prototyping. We went through Rapid Application Development, and now have lean startup and minimum viable products.

Incremental development—grow, not build, software. This has been huge. Even the most staid of enterprises pay at least some lip service to an Agile-style methodology, and can validate their ideas in a month where they used to wait multiple years.

Great designers. Again, thanks to free software, a lot more software is developed out in the open, so we can crib designs that work and avoid those that don't. Whether or not we do is a different matter; I think Brooks's conclusions on this point, which conclude the whole paper, are still valid today.

#### Still trading as ClosedDarwin
It's not surprising, but while Apple's opensource page now includes a link to the iPhone software release (clicky the title), this only contains links to the WebCore and JavaScriptCore source, which is also available from the WebKit home on MacOSForge.org.  While it is possible that the iPhone is distributed solely with software Apple can distribute without source, I wouldn't be surprised if there isn't just a teensy dollop of GPL code in there somewhere...

#### Stop ignoring the world
Long term readers will have noticed, and everybody else is about to be told, that this blog has had posts in the Responsibility category since 2010. I'm not rigorous in my use of Wordpress categories, but it's not much of a stretch to assume that most of those 40 posts touch on professional ethics, and that most of the posts on ethics in this blog are in that category.

In recent times, the idea that maybe the world of computing should take its head out of its butt and consider its impact on wider society has escaped the confines of goggle-eyed loon practitioners like yours truly and hit the mainstream. In the UK, newspapers call for change: the leftist Guardian writes "Big tech is broken", and liberal centrist paper the Independent tells us that "Those of us with any sense of morality should hate Apple". Editorials document how social media platforms, decrying fake news while running ads for anyone with the dollars, have supplanted democratic rule with new, transnational, shareholder-run government. They show how the new unicorn startups achieve their valuations by disrupting labour law, reversing centuries of gains in workers' rights by introducing the neoserfdom of gig economies and zero-hour contracts.

Software is eating the world, and turning it into shit. You can no longer pretend that it isn't happening, and that you are not playing a part. That supporting the success of your favoured multibillionaire transnational platform vendor isn't helping to consolidate ownership of society among the multibillionaire platform vendors. That your part is just making the rockets go up, and that where they come down is a different department. That your job is not a position in society and without consequence.

#### Storing and testing credentials: Cocoa Touch Edition
This article introduces the concept of key stretching, using code examples to explain the ideas. For code you can use in an app that more closely resembles current practice, see Password checking with CommonCrypto.

There's been quite the media circus regarding the possibility that Sony was storing authentication credentials for its PlayStation Network credentials in plain text. I was even quoted in a UK national daily paper regarding the subject. But none of this helps you: how should you deal with user passwords?

The best solution is also the easiest: if you can avoid it, don't store the passwords yourself. On the Mac, you can use the OpenDirectory framework to authenticate both local users and users with accounts on the network (where the Mac is configured to talk to a networked directory service). This is fully covered in Chapter 2 of Professional Cocoa Application Security.

On the iPhone, you're not so lucky. And maybe on the Mac there's a reason you can't use the local account: your app needs to manage its own password. The important point is that you never need to see that password&mdash;you need to know that the same password was presented in order to know (or at least have a good idea) that the same user is at the touchscreen, but that's not the same as seeing the password itself.

That means that we don't even need to use encryption where we can protect the password and recover it when we must check the password. Instead we can use a cryptographic one-way hash function to store data derived from the password: we can never get the password back, but we can always generate the same hash value when we see the same password.

Shut up Graham. Show me the code.

Here it is. This code is provided under the terms of the WTFPL, and comes without any warranty to the extent permitted by applicable law.

The first thing you'll need to do is generate a salt. This is a random string of bytes that is combined with the password to hash: the point here is that if two users on the same system have the same password, the fact that the salt is different means that they still have different hashes. So you can't do any statistical analysis on the hashes to work out what some of the passwords are. Otherwise, you could take your knowledge that, say, 10% of people use "password" as their password, and look for the hash that appears 10% of the time.

It also protects the password against a rainbow tables attack by removing the one-one mapping between a password and its hash value. This mitigation is actually more important in the real world than the one above, which is easier to explain :-).

This function uses Randomization Services, so remember to link Security.framework in your app's link libraries build phase.

NSString *FZARandomSalt(void) {
    uint8_t bytes[16] = {0};
    int status = SecRandomCopyBytes(kSecRandomDefault, 16, bytes);
    if (status == -1) {
        NSLog(@"Error using randomization services: %s", strerror(errno));
        return nil;
    }
    NSString *salt = [NSString stringWithFormat: @"%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x",
                      bytes[0],  bytes[1],  bytes[2],  bytes[3],
                      bytes[4],  bytes[5],  bytes[6],  bytes[7],
                      bytes[8],  bytes[9],  bytes[10], bytes[11],
                      bytes[12], bytes[13], bytes[14], bytes[15]];
    return salt;
}


Now you pass this string, and the password, to the next function, which actually calculates the hash. In fact, it runs through the hashing function 5,000 times. That slows things down a little&mdash;on an A4-equipped iPad it takes nearly 0.088s to compute the hash&mdash;but it also slows down brute-force attacks.

NSData *FZAHashPassword(NSString *password, NSString *salt) {
    NSCParameterAssert([salt length] &gt;= 32);
    uint8_t hashBuffer[64] = {0};
    NSString *saltedPassword = [[salt substringToIndex: 32] stringByAppendingString: password];
    const char *passwordBytes = [saltedPassword cStringUsingEncoding: NSUTF8StringEncoding];
    NSUInteger length = [saltedPassword lengthOfBytesUsingEncoding: NSUTF8StringEncoding];
    CC_SHA512(passwordBytes, length, hashBuffer);
    for (NSInteger i = 0; i &lt; 4999; i++) {
        CC_SHA512(hashBuffer, 64, hashBuffer);
    }
    return [NSData dataWithBytes: hashBuffer length: 64];
}


Where do I go now?

You now have two pieces of information: a random salt, like edbfe42b3da2995a159c16c0a7184211, and a hash of the password, like 855fec563d91576db0e66d8745a3a9cb71dbe40d7cb2615a82b1c87958dd2e8e56db02860739422b976f182a7055dd223a3037dd3dcc5e1ca28aaaf0bade8a08. Store both of these on the machine where the password will be tested. In principle there isn't too much worry about this data being leaked, because it's super-hard to get the password out of it, but it's still best practice to restrict access as much as you can so that attackers have to brute-force passwords on your terms.

When you come to verify the user's password, pass the string presented by the user and the stored salt to FZAHashPassword(). You should get the same hash out that you previously calculated, if the same password was presented.

Anything else?

Yes. The weakest part of this solution is no longer the password storage: it's the password itself. The salt+hash shown above is actually for the password "password" (try it yourself), and no amount of software is going to change the fact that that's a questionable choice of password&hellip;well, software that finally does away with password authentication will, but that's a different argument. 

If you want to limit a user's ability to choose a simple password, you have to do this at password registration and change time. Just look at the (plain-text) password the user has given you and decide whether you want to allow its use.

#### Story points: because I don't know what I'm doing
The scenario

[Int. developer's office. Developer sits at a desk that faces the wall. Two of the monitors on Developer's desk are on stands, if you look closely you see that the third is balanced on the box set of The Art of Computer Programming, which is still in its shrink-wrap. Developer notices you and identifies an opportunity to opine about why the world is wrong, as ever.]

Every so often, people who deal with the real world instead of the computer world ask us developers annoying questions about how our work interacts with so-called reality. You're probably thinking the same thing I do: who cares, right? I'm right in the middle of a totally cool abstraction layer on top of the operating system's abstraction layer that abstracts their abstraction so I can interface it to my abstraction and abstract all the abstracts, what's that got to do with reality and customers and my employer and stuff?

Ugh, damn, turning up my headphones and staring pointedly at the screen hasn't helped, they're still asking this question. OK, what is it?

Apparently they want to know when some feature will be done. Look, I'm a programmer, I'm absolutely the worst person to ask about time. OK, I believe that you might want to know whether this development effort is going to deliver value to the customers any time soon, and whether we're still going to be ahead financially when we're done, or whether it'd be better to take on some other work. And really I'd love to answer this question, except for one thing:

I have absolutely no idea what I'm doing. Seriously, don't you remember all the other times that I gave you estimates and they were way off? The problem isn't some systematic error in the way I think about how long it'll take me to do stuff, it's that while I can build abstractions on top of other abstractions I'm not so great at going the other way. Give me a short description of a task, I'll try and work out what's involved but I'm likely to miss something that will become important when I go to do it. It's these missed details that add time, and I don't know how many of those there will be until I get started.

The proposed solution

[Developer appears to have a brainwave]

Wait, remember how my superpower is adding layers of abstraction? Well your problem of estimation looks quite a lot like a nail to me, so I'll apply my hammer! Let's add a layer of abstraction on top of time!

Now you wanted to know how long it'll take to finish some feature. Well I'll tell you, but I won't tell you in units of hours or days, I'll use BTUs (Bullshit Time Units) instead. So this thing I'm working on will be about five BTUs. What do you mean, that doesn't tell you when I'll be done? It's simple, duh! Just wait a couple of months, and measure how many BTUs we actually managed to complete. Now you know how many BTUs per day we can do, and you know how long everything takes!

[Developer puts their headphones back in, and turns to face the monitor. The curtain closes on the scene, and the Humble(-ish) Narrator takes the stage.]

The observed problem

Did you notice that the BTU doesn't actually solve the stated problem? If it's possible to track BTU completion over time until we know how many BTUs get completed in an iteration, then we are making the assumption that there is a linear relationship between BTUs and units of time. Just as there are 40 (or 90, if you picked the wrong recruiter) hours to the work week, so there are N BTUs to the work week. A BTU is worth x hours, and we just need to measure for a bit until we find the value of x.

But Developer's problem was not a failure to understand how many hours there are in an hour. Developer's problem was a failure to know what work is outstanding. An inability to foresee what work needs to be done cannot be corrected by any change to the way in which work to be done is mapped onto time. It is, to wear out even further an already tired saw, an unknown unknown.

What to do about it

We're kindof stuck, really. We can't tell how long something will take until we do it, not because we're bad at estimating how long it'll take to do something but because we're bad at knowing what it is we need to do.

The little bit there about "until we do it" is, I think, what we need to focus on. I can't tell you how long something I haven't done will take, but I can probably tell you what problems are outstanding on the thing I'm doing now. I can tell you whether it's ready now, or whether I think it'll be ready "soon" or "not soon".

So here's the opportunity: we'll keep whatever we've already got ready for immediate release. We'll share information about which of the acceptance tests are passing, and if we were to release right now you'd know what customers will get from that. Whatever the thing we're working on now is, we'll be in a position to decide whether to switch away if we can do some more valuable work instead.

#### Story points: there's no right way to do it
Story points as described represent an attempt to abstract estimation away from "amount of stuff done per unit time", because we're bad at doing that and people were traditionally using that to make us look bad. So we introduce an intermediate value, flip a ratio, and get the story point: the [meaningless value] per amount of stuff. Then we also get the velocity, which is the [meaningless value] per unit time, and...

...and we're back where we started. Except we're not, we're slower than we were before, because it used to be that people asked "how much do you think you can get done over the next couple of weeks", and we'd tell them, and we'd be wrong. But now they ask "how big is this stuff", then they ask "how much capacity for stuff is there over the next couple of weeks", and we tell them both of those things, and we get both wrong, so we still have a wrong answer to the original question but answered two distinct questions incorrectly to get there.

There's no real way out of that. The idea that velocity will converge over time is flawed, both because the team, the context, and the problem are all changing at once, and because the problem with estimation is not that we're Gaussian bad at it, but that we're optimistic bad at it. Consistently, monotonically, "oh I think this will just mean editing some config, call it a one-pointer"-ingly, we fail to see complexity way more than we fail to see simplicity. The idea that even if velocity did converge over time, we would then have reliable tools for planning and estimation is flawed, because what people want is not convergence but growth.

Give people 40 points per sprint for 20 sprints and you'll be asked not how you became so great at estimation, but why your people aren't getting any better. Give them about 40 points per sprint for 20 sprints, and they'll applaud the 44s and frown at the 36s.

The assumption that goes into agile, lean, kanban, lean startup, and similar ideas is that you're already doing well enough that you only need to worry about local optima, so you may as well take out a load of planning overhead and chase those optima without working out your three-sprint rolling average local optimisation rate.

#### Structured Pruning of Deep Convolutional Neural Networks
Structured Pruning of Deep Convolutional Neural Networks, Sajid Anwar et al. In the ACM Journal on Emerging Technologies in Computing special issue on hardware and algorithms for learning-on-a-chip, May 2017.

Notes

Quick, a software engineer mentions a "performance" problem to you. What do they mean?

This is, of course, an unfair question. There are too many different ideas that all get branded "performance" for us to know what we are trying to solve. This paper is simultaneously about two different flavours of performance.

On the one hand, the "performance" of a neural network is related to its ability to perform its task: the correctness of its inferences. There isn't really a good way to know what neural network configuration will perform efficiently for the (unknown) function you want it to approximate. Therefore, the rule of thumb is find a network that's too complex, and stop training it when it begins to overfit (when its performance starts to degrade, because it's being too specific about whether an input looks like an example from the training set rather than whether it shares important features).

Now we meet the other kind of performance: the amount of resources consumed to do the work. A large neural network needs to do a lot of computations with a lot of numbers to classify an input, and that means using a lot of processor cycles and a lot of memory. Because our approach to designing the network was to overspecify it, we are using more computer than we need. But if that computer is relatively low-specification and battery operated—a mobile phone for example—this may render our solution unusable.

So, how can we turn a complex neural network into a simpler neural network? While this isn't totally satisfying, the answer is: "guess". Turn off bits of the network (i.e. set weights to zero), and see whether it still classifies (performs) well. This act of turning bits of the network off is called pruning.

Ironically some previous work in this space has actually not been great for performance (the resource kind). You can "unroll" convolutional layers (commonly found in image-classifying networks) into matrix multiplications, and you can turn that into a sparse matrix by approximating all small weights with zero and only storing the non-zero values and their locations. But now, even though you have fewer calculations, you may have more memory accesses in trying to solve where the weights should be used. And that could be slower than not reducing the network.

The work in this paper takes a structured approach to pruning the network. Whole feature maps (scores indicating whether particular characteristics of an image were found, and where, in the input image) can be removed, the network retrained, and the performance (ability to classify) measured afterwards. At smaller scales, the kernels can be pruned in particular deterministic ways, replacing a full weights matrix with a start index, a "stride" (gap between each non-zero value) and the list of non-zero weights. The different possibilities are explored using a combination of random generation and evolutionary iteration; networks that have a misclassification rate within given tolerance the original are kept into subsequent generations.

The results seem promising. With pruning at both levels of abstraction, the resulting network is just as deep (it contains as many layers) but it has fewer nodes at each layer and fewer connections between nodes. The systematic pruning approach means that the resulting networks are smaller in memory and faster in use: CPU time measurements are down approximately two thirds when compared with the initial, unpruned network.

However, be careful when interpreting the graph: the authors are showing the reduced execution time of the unrolled matrix multiplication for one layer of one network configuration. It is not clear what this means for overall behaviour of the network, what the misclassification rate of this network was (they show a tolerance cutoff at 4%, which may be too high for a given use case), or in general how the CPU time savings vary with network topology. In other words, we have a single graph, and don't know how to generalise it.

I hope that at some point a sound theoretical basis for choosing the architecture for a neural network to solve a given problem will be developed. In fact, I sort of hope that it exists now, and that I haven't found it. I don't think so: for the moment, whack-a-mole is the state of the art, and this paper shows you can whack quite a lot of moles and still get reasonable results.

#### Subatomic Chocolate
This started out as a toot thread, but "threaded tooting is tedious for everybody involved" so here's the single post that thread should have been.

The "Electron vs. native" debate doesn't make much sense. I feel like I've been here before:

  Somehow those of us who had chosen a different programming language knew that we were better at writing software; much better than those clowns who just made the most successful office suite ever, the most successful picture editing app ever, or the most successful video player ever. Because we’d taken advice on how to write software from a company that was 90 days away from bankruptcy and had proven incapable of executing on software development, we were awesome and the people who were making the shittons of money on the most popular software of all time were clueless idiots.

Some things to ponder but avoid for the moment:

why are those the only choices? If I write a Java SWT app with Windows native components on Windows, and Mac native components on Mac, is that native because I'm using the native widget toolkit or not, because I'm using Java? If it is not, is it "Electron"?
where is the boundary of native? AppKit is written in Objective-C, so am I using some unholy abomination of an RMI bridge if I write AppKit software using AppKit APIs but a different programming language, like Swift?

It seems clear that people who believe there is a correct answer to "Electron vs. native" are either native app developers or Electron app developers. We can therefore expect them to have some emotional investment (I have decided to build my career doing this, please do not tell me that I'm wrong) and to be seeking truths that support their existing positions. Indeed, if you are on one side of this debate then the other side is not even wrong because the two positions compare incompatible facts.

Pro-Electron: the tools are better/easier/more familiar/JavaScript

Most to all of these things are true, in a lot of cases. As a seasoned "native" app developer, with some tiny amount of JavaScript experience, I can build a thing very quickly in JS (with a GUI in React or React Native, I haven't tried Electron) that still takes me a long time in a "native" toolkit, both the ones I'm comfortable with and the ones I'm unfamiliar with but download and try things out in.

Now that should be disturbing to any company who builds a "native" platform, and who thinks that developers are key to their success. If someone with nearly two decades of using your thing can be faster at using someone else's thing within under a year of learning that thing, there is something you need to be learning very quickly about the way the other thing works and how to bring that advantage to your thing, otherwise everything will be made out of the other thing soon and you'd better hope they keep making it work on your thing.

Actually, having said that this argument is true, it's not true at all. The tools in JS-land are execrable. Bear in mind that the JSVM (we used to call it a "browser") is a high-performance code environment with live code loading, reflection and self-modifying capabilities; it's disappointing that the popular developer environments are text editors with syntax highlighting and an integrated terminal window. "Live" code loading is replaced with using Watchman to wait for some files to change, then kicking off some baroque house of cards that turns those files from my house blend of JS into your house blend of JS, then reloading the whole shebang.

Actually, having said that this argument is true and false, it's not even relevant at all. The developers are the highly-paid people whose job it is to solve the problems for everybody else, why are we making their lives easier, not everybody else's?

Pro-"native": the apps are more efficient/consistent

Both of these things are true, in a lot of cases. A "native" application just needs to link the system widget set (which, if your platform supports efficient memory management, is loaded anyway by some first-party tool) and run its code.  It will automatically get things that look and behave like the rest of the applications on the platform.

Actually, having said that this argument is true, it's not true at all. The "native" tools are based on a lot of low-level abstractions (like threads or operations), that are hard to use correctly; rather than rely on an existing solution (remember there's no npm for "native", and the supposed equivalent has nowhere near as much coverage) developers are likely to try building their own use of these primitives, with inefficiencies resulting. The "native" look and feel of the components can and will be readily customised to fit branding guidelines, and besides as the look and feel is the platform vendor's key differentiator they've moved things around every release so an app that behaved "consistently" on the last version looks out of place (deliberately, so that developers are "encouraged" to adopt the new platform features) this year.

Actually, having said that this argument is true and false, it's not even relevant at all. The computer is there as a substrate for a thing that solves somebody's problem, so as long as the problem is solved and the solution fits on their computer, isn't the problem solved? And as for "consistency", the basic tenets of these desktop "native" experiences were carved out three decades ago, before almost all experience with and research into desktop computer interaction. Why aim for consistency with an approach that was decided before we knew what did or didn't work properly?

#### Subconscious operation
I just noticed that I've been using Xcode's codesense all day, despite the fact that I'm writing Python (albeit with PyObjC goodness).  I wasn't aware that that would work, but then I wasn't really aware that it was working, if you see what I mean.

#### Summer of code
GNUstep has been approved for this year's Google Summer of Code.  The title link goes to the GNUstep wiki page outlining possible projects, but I'm sure that if a student had another idea you'd be welcome to talk about it on the gnustep-discuss mailing list, and probably get a mentor!

#### Supporting both ARC and MRC build settings
Let's face it, people don't read `README`s. If you write library code that people are going to use in their own projects, you can't rely on that bit at the bottom of the documentation that tells people to do -fobjc-arc on your files that they drop into your project. You can rely on all the issues that get reported about memory leaks :-).

The actual solution

Your project should build a library (static by necessity on iPhone, there are other options on the Mac) so developers can just add that one library target as a build dependency, and drop the headers into their own projects.

The result is that now your memory management is hidden behind the object boundary and the naming conventions of your methods. You should probably still be using manual reference counting if you want people who've already written apps to be able to link against your code without problems, because there are still apps out there that target versions of iOS that can't link ARCified objects. Regardless, whether an app is ARCified or not it will be able to link your library.

The other solution

Sometimes you find code that developers are supposed to integrate by dropping the source files into their targets. This is worse than providing a static library: now you've made the developer care about the internals of your code - the compiler flags you need to set become something they have to deal with in their target's build settings. This includes the setting for whether automatic reference counting is enabled.

…unless you support both possibilities. I've used the macros defined below to use the same code with both automatic and manual reference counting compiler settings. This code included Core Foundation bridged objects, so this isn't just "the trivial case" (whatever that is).

#if __has_feature(objc_arc)
# define FZARelease(obj)
# define FZAAutorelease(obj) (obj)
# define FZARetain(obj) (obj)
#else
# define FZARelease(obj) [(obj) release]
# define FZAAutorelease(obj) [(obj) autorelease]
# define FZARetain(obj) [(obj) retain]
#endif

Objective-C garbage collection

I haven't had a need to test how that interacts with garbage collection, or build code that works in all three environments. However, if you already wrote your code to support (rather than require) GC, and you don't rely on CFMakeCollectable, this collection of macros at least won't make anything worse.

#### Surprising ARC performance characteristics
The project I'm working on at the moment has quite tight performance constraints. It needs to start up quickly, do its work at a particular rate and, being an iOS app, there's a hard limit on how much RAM can be used.

The team's got quite friendly with Instruments, watching the time profile, memory allocations, thread switches[*] and storage access trying to discover where we can trade one off in favour of another.

[*] this is a topic for a different post, but "dispatch_async() all the things" is a performance "solution" that brings its own problems.

It was during one of these sessions that I noticed a hot loop in the app was spending a lot of time in a C++ collection type called objc::DenseMap. This is apparently used by objc_retain() and objc_release(), the functions used to implement reference counting when Objective-C code is compiled using ARC.

The loop was implemented using the closure interface, -[NSDictionary enumerateKeysAndValuesUsingBlock:. Apparently the arguments to a block are strong references, so each was being retained on entering the block and released on return. Multiply by thousands of objects in the collection and tens of iterations per second, and that was a non-trivial amount of time to spend in memory management functions.

I started to think about other data types in which I could express the same collection—is there something in the C++ standard library I could use?

I ended up using a different interface to the same data type - something proposed by my colleague, Mo. Since Cocoa was released, Foundation data types have been accessible via the CoreFoundation C API[**]. The key difference as far as modern applications are concerned is that the C API uses void * to refer to its content rather than id. As a result, and with appropriate use of bridging casts, ARC doesn't try to retain and release the objects.

[**]I think that Foundation on OPENSTEP was designed in the same way, but that the C API wasn't exposed until the OS X 10.0 release.

So this:

[myDictionary enumerateKeysAndObjectsUsingBlock: ^(id key, id object, BOOL *stop) {
  //...
}];

became this:

CFDictionaryRef myCFDictionary = (__bridge CFDictionaryRef)myDictionary;
CFIndex count = CFDictionaryGetCount(myCFDictionary);
void *keys[count];
void *values[count];
CFDictionaryGetKeysAndValues(myCFDictionary, keys, values);

for (CFIndex i = 0; i &lt; count; i++)
{
  __unsafe_unretained id key = (__bridge id)keys[i];
  __unsafe_unretained id value = (__bridge id)values[i];
  //...
}

which turned out to be about 12% faster in this case.

I'll finish by addressing an open question from earlier, when should I consider ditching Foundation/CoreFoundation completely? There are times when it's appropriate to move away from those data types. Foundation's adaptive algorithms are very fast a lot of the time, choosing different representations under different conditions - but aren't always the best choice.

Considering loops that enumerate over a collection like the loop investigated in this post, a C++ or C structure representation is good if the loop is calling a lot of messages. Hacks like IMP caching can also help, in which this:

for (MyObject *foo in bar)
{
  [foo doThing];
}

becomes this:

SEL doThingSelector = @selector(doThing);
IMP doThingImp = class_getMethodImplementation([MyObject class], doThingSelector);

for (MyObject *foo in bar)
{
  doThingImp(foo, doThingSelector);
}

If you've got lots (like tens of thousands, or hundreds of thousands) of instances of a class, Objective-C will add a measurable memory impact in the isa pointers (each object contains a pointer to its class), and the look aside table that tracks retain counts. Switching to a different representation can regain that space: in return for losing dynamic dispatch and reference-counted memory management—automatic or otherwise.

#### Swedish chef
Yes, I did write a swedish chef to C translator.  What of it?  My cousin was bitten by a møose once.

#### Swift
Speaking of Swift, what idiot called it swift-evolution and not "A Modest Proposal"?

#### switching CPUs
I can waste a lot of time if an interesting question gets posed.  The most recent culprit was Chris Ridd in uk.comp.sys.mac, wondering aloud if threads ever got shuttled between CPUs on a Mac.  Here we go...

It certainly looks like the conditions for doing this through the API can never be met on a real Mac running xnu.  All you can do is assign a thread or a task to a processor set, but not a single processor.  Unless that processor happens to be the only member of a processor set.  Thing is, on xnu there is only one processor set, which encompasses all of the processors.  So there's no way to force your threads onto a particular processor in Mac OS X.  The lack of multiple processor sets means that you couldn't have a box with, for instance, both a G5 and a Xeon running a single kernel, which is kindof sad in a way.  It won't get missed, though.

It doesn't even look like there's an API to work out which CPU you're currently on.  What you can do is get the port for the default processor set (the only processor set), then by manually furtling with the structs you can iterate through the CPUs (or you can ask your own mach_host_port for an array of the CPU ports).  For each CPU you can get the thread queue, which you could compare with your own thread to see if it's the same.  That'd be tedious, but would get the required information.  Once there though, I don't see any way for the thread to move between any queues except the idle and run queues on its 'native' processor; so I think the answer is that no thread ever migrates between CPUs.  I may be wrong, and am happy to entertain any information to the contrary.

Unsurprisingly with no API to do what I want, little useful code came out of this exercise.  I found out how many cpus the system has, but that's it really.

Update 2006-09-10 16:25 GMT - I hate this not knowing, so I decided to play the Ask Amit card ;-).

#### Synthesized ivars are private
Perhaps this isn't news. Perhaps it doesn't matter because you've provided public accessors. But here are the results anyway.

#import &lt;Foundation/Foundation.h&gt;

@interface A: NSObject
@property (nonatomic, assign) int a;
@end

@interface B: A
- (int)differentGetter;
@end

@interface C: NSObject
@property (nonatomic, retain) A *anA;
- (int)aFromA;
@end

int main(int argc, char *argv[]) {
	@autoreleasepool {
		B *b = [[B alloc] init];
		b.a = 3;
		NSLog(@"[b differentGetter] = %d", [b differentGetter]);
		[b release];
		
		C *c = [[C alloc] init];
		c.anA = [[A alloc] init];
		c.anA.a = 4;
		NSLog(@"[c aFromA] = %d", [c aFromA]);
		[c release];
	}
	return 0;
}

@implementation A
@synthesize a=_a;
@end

@implementation B
- (int)differentGetter { return _a; } // must be at least @protected
@end

@implementation C
@synthesize anA = _anA;
- (int)aFromA { return _anA->_a; } // must be @public
- (void)dealloc { self.anA = nil; [super dealloc]; }
@end

Doesn't compile:

Untitled.m:37:33: error: instance variable '_a' is private
- (int)differentGetter { return _a; }
                                ^
Untitled.m:42:30: error: instance variable '_a' is private
- (int)aFromA { return _anA->_a; }
                             ^
2 errors generated.

#### Talking about talking
I recently gave a talk to my colleagues about giving talks. Here is an annotated collection of the notes I made in preparation.

- What do you want the audience to get out of the talk?

As you're constructing your talk, ensure that you're actually satisfying your mission. If you want to inspire people, make sure you're not just promoting your own knowledge, business or ability. If you want people to learn things, make sure your talk is appropriate to the experience level of the audience.

- Find out about the audience
    - likely skill level
    - range of experiences
    - interest in technical, business or other issues
- Don't assume that because you think something's obvious, everyone else does

A big stumbling block for novice speakers I talk to is to assume that because you know something, it's not worth talking about as there are people out there who know way more. Your own experiences and interpretations are different from everyone else's, it's very likely that you'll have something new to contribute—as long as your talk is personal, and not just a restatement of readily-available documentation.

- Decide what it is you're going to say
    - are you trying to inspire or persuade the audience?
        - decide your conclusion
            - if you're worried about timing, give yourself a couple of different exit points
        - how do you want to start?
            - the conclusion's a good place to start

This is a tough place to start though. The idea is that you're challenging the audience by telling them something that sounds implausible, so they get mentally engaged. If the leap required is too big then you'll either turn people off, or they'll still be thinking about the challenge after you've started talking.

            - outline the problem that your solution solves

This is the Jobs approach. Start by saying the current world sucks. Explain what a better world would look like. Make it obvious that your proposal leads to the better world. Tell them the thing you propose is available in the foyer as soon as the talk finishes. It's based on setting up one or more distinctions between the world as it is (or as you tell the audience they currently perceive it) and as it could be (or as you tell the audience they should want it), then showing that those distinctions have actually been resolved. Nancy Duarte did a good talk on this topic.

        - what's the flow between the problem and the conclusion?
        - notice this isn't "tell them what you're about to tell them…"
    - are you trying to educate the audience?
        - you can't in under an hour; aim for awareness or persuasion
        - put additional relevant content on your blog and refer to it
        - you still need a flow

In this case, the audience's problem is "we don't know how to do [x]", the better world is one where they do know how to do [x], and the solution is your content. Don't try to cram all the code into your talk because it's distracting, only relevant in a few cases, and hard to parse while keeping up with the presentation. Instead, give people the key parts of the solution so that when they come across [x], they'll remember some things from your presentation which will help them piece together the full solution.

We had a discussion about WWDC talks at this point in the "live" version of these notes. WWDC seems to provide a counterexample to this rule about not educating people in a talk, with graphics-poor code-rich presentations. Those sessions have two goals: giving developers who aren't in the labs something to do, and being available on video afterward. The live presentation frankly is overwhelming and often confusing, but isn't the main use of the talk. You're expected to watch it over, to refer to the documentation, to ask people about the content in the labs.

[To be honest I also expect there's an extent to which a lot of the people presenting at WWDC are both strongly pressured by the additional workload of the conference and are uncomfortable with speaking publicly, and the format they've settled on is one that works in that context and doesn't make too many compromises or create too much additional stress. That's just speculation on my part though.]

    - entertainment
        - doesn't need to be jokes, a compelling talk is entertaining
        - in fact be careful of jokes unless you know the crowd
        - certainly don't lead the laughter

I put this in as a homage to Thorsten Heins, who ruined an otherwise reasonably competent and well-executed presentation by laughing at and even applauding his own jokes. If you want to try a joke, think twice. I only do it in arenas where I'm comfortable I know the people. If you still want to do it, and it falls flat, move on.

        - if you want a set piece, make it relevant to the talk and practice the bejeezus out of it
- engage the audience
    - "hands up if" exercises are light forms of interaction but make people engage with the talk

As discussed earlier, making people too introspective will distract them from your talk. But these days you have to stop people from diving back into their laptops and working during talks, so you need to provide some form of engagement. You also have to deal with the fact that various sub-sections of your audience may be jet lagged, full of lunch or hungover. If they don't have a part to play in your talk they'll sleep through it.

    - make eye contact with every part of the audience
        - you will see people asleep or working; don't worry
        - don't forget the back of the room

It's easy for introverts particularly to "protect" ourselves from the audience by avoiding looking at them. The problem is it then doesn't feel like we're talking to everyone out there. You shouldn't aim to make eye contact with every individual attendee, that doesn't scale—you definitely should talk to each "part" of the audience though. Talking to someone at the back of the room when you start your talk helps you pitch your voice correctly.

    - motion, nervous or otherwise, gives people a reason to be concentrating on you rather than the screen/their phones
- slides
    - too much text and you lose people while they read along
    - again, relevant content
    - animation where relevant helps, where irrelevant distracts
    - progressive disclosure and hiding

So much has been said about building good slides that I don't want to add anything. Make sure your own notes are separate from your slides, and everything on-screen now is germane to what you're saying now.

- q&a
    - there will be an awkward question. Your goal is to handle it gracefully, not to avoid it coming up
    - continuum fallacy

Most of the "well actually" questions you'll get in conference Q&As are not actually questions. They're the phrase "I know more than you" dressed up with some rhetorical sugar to appear more question-like. These are poisonous: the audience gains nothing from them, you gain nothing from them, the person "asking" gains nothing from them. Nonetheless we can't ban people who ask these questions from conferences, so we just have to cope with them. "That's an interesting point, we should talk about it later" is an OK response—especially if you need to catch the train as soon as you're off the stage.

The ones that are questions frequently represent instances of the continuum fallacy: you said X is true. Well actually I've found an edge case where X isn't true, therefore X is never true. No. Politely point out the fallacious reasoning, move on to another question. The biggest mistake I make in handling Q&A is letting people argue the toss over questions like these. Again, the rest of the audience is learning nothing from a pointless to and fro; and there are more of them than the two of you having the discussion.

    - set expectations on questions at the beginning, e.g. "I'll take questions at the end" or "interruptions welcome". "No questions" is hard to get the conference organisers to agree with (though would probably help a lot of nervous speakers)—filibustering sometimes works but is apparently rude :)
    - you're effectively chairing a discussion you're also involved in, so don't be afraid of setting topic boundaries. The q&a has to be valuable to the audience.

#### Tautology of the year (so far)
From iDefense, via DarkReading:A recent wave of fatwas issued by radical Islamic religious leaders in that region authorizing these groups to use cyberattacks to defend Islam has opened the door for these groups to wage cyberattacks, according to iDefense.In other news, water has been found to be wet. (seriously considering a "my beloved language, you've killed it!" tag)

#### TDD and crypto in one place
Well, I suppose if I've written two books, it's about time I wrote a contorted blog post that references both of the worlds.

I recently wrote an encryption module for an app, and thought it'd be useful to share something about the design process. Notice that the source code here was quickly thrown together for the purposes of demonstrating the design technique, and bears little resemblance to the code I was actually writing (which was in a different language). A lot of the complexity I was actually trying to deal with has been elided in order to bring the story to the fore.

Steps 1a, 2a, 1b, 2b etc: Design the API hand-in-hand with a fake implementation.

A pass-thru implementation would work here, but I prefer to use something that actually does change its input just so that it's clear that there's a requirement the input should be acted on. You might end up with tests that look something like this:

@implementation CryptoDesignTests
{
    id &lt;StringEncryptor&gt;cryptor;
}

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];
}

- (void)tearDown {
    cryptor = nil;
}

- (void)testEncryptionOfPlainText {
    NSString *plainText = @"hello";
    NSString *cipherText = [cryptor encipherString: plainText];
    STAssertEqualObjects(@"uryyb", cipherText, @"This method should encrypt its parameter");
}

- (void)testDecryptionOfCipherText {
    NSString *cipherText = @"uryyb";
    NSString *plainText = [cryptor decipherString: cipherText];
    STAssertEqualObjects(@"hello", plainText, @"This method should decrypt its parameter");
}

@end

Where the protocol looks like this:

@protocol StringEncryptor &lt;NSObject&gt;

- (NSString *)encipherString: (NSString *)plainText;
- (NSString *)decipherString: (NSString *)cipherText;

@end

Of course you can implement this however you want. Here's one potential implementation (which may have its problems, but is only being used to guide the API design):

@interface ROT13StringEncryptor : NSObject &lt;StringEncryptor&gt;

@end

@implementation ROT13StringEncryptor

- (NSString *)rot13: (NSString *)original {
    NSMutableData *originalBytes = [[original dataUsingEncoding: NSASCIIStringEncoding] mutableCopy];
    for (NSInteger i = 0; i < [originalBytes length]; i++) {
        NSRange cursor = NSMakeRange(i, 1);
        char c;
        [originalBytes getBytes: &c range: cursor];
        if       (c >= 'a' && c = 'n' && c = 'A' && c = 'A' && c  cryptor;
}

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];
}

- (void)tearDown {
    cryptor = nil;
}

- (void)testRoundTripEncryption {
    NSString *plain = @"Mary had a little lamb.";
    NSString *outOfTheSausageMachine = [cryptor decipherString: [cryptor encipherString: plain]];
    STAssertEqualObjects(plain, outOfTheSausageMachine, @"Content should survive an encryption round-trip");
}

@end

This test passes without any extra work.

Step 4. Give that lot to the poor sap who needs to write the app.

That's right, the other developers can start working straight away. They've got the protocol which tells them what the object can do, the unit tests which explain how the API should work and the integration test that explains how you expect the object to be used.

Leaving you free to:

Step 5. Point the integration test at a different implementation of the protocol.

But you haven't written that implementation yet! It must be time to:

Step 6. Write it.

You know when it works because you have integration tests. You know it'll work with the app the other person's writing because they've got the same integration test.

So the extra artefacts that some people see as a wasteful byproduct of code-level testing &mdash; the fake implementation of the protocol for example &mdash; are demonstrably useful in this scenario. Other developers on the team can use the fake implementation as a stand-in while you're head-down coding up the production code. It's also much easier to design a class when you're not also sweating over how the internal details are going to work out, so you probably get where you're going quicker too.

#### TDD/unit testing video training for iOS developers
I recently recorded a series of videos on unit testing and test-driven development for iOS developers with Scotty of iDeveloper.tv. The videos and associated source code is now available for purchase and download.

#### Teaching Programming to People. It's easy, right?
I was doing a literature search for a different subject (which will appear soon), and found a couple of articles related to teaching programming. I don't know if you remember when you learnt programming, but you probably found it hard. I've had some experience of teaching programming: specifically, teaching C to undergraduates. Said undergraduates, as it happens, weren't on a computing course (they studied Physics), and only turned up to the few classes they had a year because attendance was mandatory. The lectures, which weren't compulsory, had fewer students showing up.

Teaching Python to Undergraduates

When I took the course, we were taught a Pascal variant on NeXTSTEP. I have some evidence that Algol had been the first programming language taught on the course; probably on an HLH Orion minicomputer. While Pascal was developed in part as a vehicle to teach structured programming concepts, the academics in the computing course at my department were already starting to see it as a toy language with no practical utility. Such justification was used to look for a different language to teach. As you can infer from the previous paragraph, we settled on C: but not before a test where interested students (myself included) who had, for the most part, already taken the Pascal course. The experiences with Python were written up in a Masters' thesis by Michael Williams, the student who had converted the (Pascal-based, of course) teaching materials to Python.

Like Wirth, when Guido van Rossum designed Python he had teaching in mind; though knowing the criticisms of Pascal he also made it extensible so that it could be used as a "real" language. This extensibility was put to use in the Python experiment at Oxford, giving students the numpy module which they used mainly for its matrix datatype (an important facility in Physics).

What the report shows is that it's possible to teach someone enough Python to get onto problems with numerical computation in a day; although clearly this is also true of Pascal and C. One interesting observation is the benefit of enforced layout (Python's meaningful indentation) to both the students, who reported that they did not find it difficult to indent a program correctly; and to the teachers, who found that because students were coerced into laying out their code consistently, it was easier to read and understand the intention of code.

An interesting open question is whether that means enforced indentation leads to more efficient code reviews in general, not just in an expert/neophyte relationship. Many developers using languages that don't enforce layout choose to add the enforcement themselves. Whether this is an issue at all when modern IDEs can lay out code automatically (assuming developers with enough experience of the IDE to use that feature) also needs answering.

The conclusion of this study was that Python is appropriate as a teaching language for Oxford's Physics course, though clearly it was not adopted and C was favoured. Why was this? As this decision was made after the report was produced, it doesn't say, and my own recollection is hazy. I recall the "not for real world use" lobby was involved, that it was also possible to teach C in the time involved, and that while many people wanted to teach Java this was overruled due to a desire to avoid OO. The spurned Java crowd preferred C for its Java-like syntax.

Wait, C?

The next part of this story wasn't published, but I'll cover it anyway just for completeness. The year after this Python study, the teaching course did an A/B test where half of the first year course was taught Python, and half C. Whatever conclusions were drawn from this test, C won out so either there was no significant difference in that type of course or the "real worldness" of C was thought to be greater than that of Python. I remember both being given as justifications, but don't know whether either or both were retrofitted.

Whatever the cause, teaching C was sufficiently not bad that the course is still based on the language.

Going back to that Java decision. How good is Java as a teaching language?

Analyses of Student Programming Errors In Java Programming Courses

I'm going to use the results of this paper to argue that Java is not good as a teaching language.

  Programming errors can be categorized as syntax, semantic and logic. A syntax error is an error due to incorrect grammar. Syntax errors are often detected by a program called a compiler, if the language is a compiled language such as Java. A semantic error is an error due to misuse of a programming concept, despite correct syntactic structure. Semantic errors are caught when the program code is compiled. A logic error occurs when the program does not solve the problem that the programmer meant for it to solve.

[Notice that the study is only investigating errors: it's not completely obvious but "bugs" aren't included. The author's only reporting on things that are either compiler or runtime errors in Java-land, like typos and indices out of bounds.]

  Categorization of errors of the present study into syntax, semantic, runtime and logic revealed that syntax errors made up 94.1%, semantic errors 4.7% and logic
  errors 1.2%.

In the ideal world, a programming course teaches students the principles of programming and how to combine these to solve some computational problem. In learning these things, you expect people to make semantic and logic errors: they don't yet know how these things work. Syntax errors, on the other hand, are the compiler's way of saying "meh, you know what you meant but I couldn't be bothered to work it out", or "I require you to jump through some hoops and you didn't".

You don't want syntax errors when you're teaching programming. You want people to struggle with the problems, not the environment in which those problems are presented. Imagine failing a student because they pushed the door to the exam room when it was supposed to be pulled: that's a syntax error. One of the roles of a demonstrator in a computing course is to be the magic compiler pixie, fixing syntax errors so the students can get back on track.

OK, so not Java. What else is out there?

C++

The Oxford Physics investigation didn't publish any results on the difficulty of teaching C. Thankfully, the Other Place is more forthcoming. Tim Love, author of Tim Love's Cricket for the Dragon 32 and teacher of C++ to Engineering undergraduates blogged about difficulties encountered defining functions in C++. There's no frequency information here, just representative problems. While most of the problems are semantic rather than syntactic, with some logic problems too, we can't really compare these with the Java analysis above anyway.

The sorts of problems described in this blog post are largely the kind of problems you want, or at least don't mind, people experiencing while you're teaching them programming: as long as they get past them, and understand the difference between what they were trying and what eventually worked.

In that context, comments like this are worrying:

  I left one such student to read the documentation for a few minutes, but when I returned to him he was none the wiser. The Arrays section of the doc might be sub-optimal, but it can't be that bad - it's much the same as last year's.

So the teacher knows that the course might have problems, but not what they are or how to correct them despite seeing the failure modes in first person. This is not isolated: the Oxford course linked above has not changed substantially since the version I wrote (which added all the Finder &amp; Xcode stuff).

So, we know something about the problems encountered by students of programming. Do we know anything about how they try to solve those problems?

An analysis of patterns of debugging among novice computer science students

We've already seen that students didn't think to use the interactive interpreter feature of Python when the course handbook stopped telling them about it. In this paper, Ahmadzadeh et al modified the Java compiler to collect analytics about errors encountered by students on their course (the methodology looks very similar to the other Java paper, above). An interesting statistic noticed in §3.1, a statistical analysis of compiler errors:

  It can be seen from this table that the error that is most common amongst all the subjects is failing to define a variable before it is used. This was almost always the highest frequency error when teaching a range of different concepts.

It's possible that you could avoid 30-50% of the problems discovered in this study by using a language that doesn't require explicit declaration of variables. Would the errors then be replaced by something else? Maybe.

In section 4, the authors note that there's a distinction between being able to debug effectively and being able to program well: most people who are good at debugging are also good programmers, but a minority of good programmers are good at debugging. Of course this is measuring a class of neophytes so it's possible that this gap eventually closes, but more work would need to be done to demonstrate or disprove that.

I notice that the students in this test are (at least, initially) fixing problems introduced into a program by someone else. Is that skill related to fixing problems in your own code? Might you be more frustrated if you think you've finished an assignment only to find there's a problem you don't understand in it? Does debugging someone else's program support the educational goal? This paper suggests that the skills are in fact different, which is why "good" programmers can be "bad" debuggers: they understand programming, but not the problem solved by someone else's code. They also suggest that "bad" programmers who do well at fixing bugs do it because they understand the aim of the program, and can reason about what the software should be doing. Perhaps being good at fixing bugs means more understanding of specifications than of code—traditionally the outlook of the tester (who is called on to find bugs but rarely to fix them).

Conclusions and Questions

There's a surprising amount of data out there on the problems faced by students being taught programming—some of it leads directly to actionable conclusions, or at least testable hypotheses. Despite that, some courses look no different from courses I taught in 2004-2006 nor indeed any different from a course I took in 2000-2001.

Judicious selection of language could help students avoid some of the "syntactic" problems in programming, by choosing languages with less ceremony. Whether such a change would lead to students learning faster, enjoying the topic more, or just bumping up against a different set of syntax errors needs to be tested. But can we extrapolate from this? Are environments that are good for student programmers good for novices in general, including inexperienced professionals? Can this be taken further? Could we conclude that some languages waste time for all programmers, or that becoming expert in programming just means learning to cope with your environment's idiosyncrasies?

And what should we make of this result that being good at programming and debugging do not go together? Should a programming course aim to develop both skills, or should specialisation be noticed and encouraged early? [Is there indeed a degree in software testing offered at any university?]

But, perhaps most urgently, why are so many different groups approaching this problem independently? Physics and Engineering academics are not experts at teaching computing, and as we've seen science code is not necessarily the best code. Could someone aggregate these results from various courses and produce the killer course in undergraduate computing?

#### Teaching Quality Object-Oriented Programming
About this paper

Teaching Quality OOP by Yishai A. Feldman, published March 2005 (see the link for full citation).

Notes

One of the points made in my book Object-Oriented Programming the Easy Way is that objects should be specified by their interfaces through contracts, which say what messages the objects respond to, how you use them, and what happens as a result.

  While it is up to any one object to decide how it responds to messages, we need to know whether that object represents a useful addition to our system. In other words, we want to know what the object will do in response to what messages. (Page 61)

The book Structure and Interpretation of Computer Programs says the same thing. An abstract data type has a collection of things that can be done, and stuff that happens when you do it.

  In general, we can think of data as defined by some collection of selectors and constructors, together with specified conditions that these procedures must fulfill in order to be a valid representation. (§2.1.3)

This seems evident. Knowing that I have a int count(), a boolean contains(Object o) and a void add(Object o) method is insufficient, I need to know how they interact before I can use them. For an array, given:

int x = a.count();
a.add(anObject);
int y = a.count();

you would always expect y - x == 1. For a set, it would depend on the content of the collection before addition; it could be 1 or 0. Knowing what methods are called is insufficient to know what type I am dealing with.

Why is it, then, that programming languages give you types for expressing the methods on an object, but not what they do or how they relate? Why does a Java interface, a Swift protocol, or a C++ abstract class only have the part of the contract related to names, not behaviours?

Famously, the Eiffel language addresses this, and it's here that we come into contact with the paper that is the topic of this post. Eiffel, and its underlying theory, is well-described in the book Object-Oriented Software Construction by Bertrand Meyer, CTO of Eiffel Software and researcher at ETH Zurich. Feldman wanted to teach his students the theory of "Quality Software" based on two principles that are well-described in OOSC:

Design by Contract, because it's more approachable and usable than formal methods, and more useful than testing; and
Command-Query Separation, because it isolates state changes, making it easier to draw conclusions about the behaviour of a software system.

But he didn't want to teach them Eiffel, because:

  it might leave the students with the mistaken and harmful impression that quality programming is confined to one language. (§3)

Additionally, OOSC does not include any exercises, so is not useful as a teaching support book. I will note here that Meyer has also written A Touch of Class, which does have supporting teaching material including exercises, but still uses the Eiffel language and therefore only solves half of the author's problems.

Feldman taught the theory from OOSC using Eiffel notation, and encouraged students to complete exercises that were in variants of the Java programming language. Tools available at the time read the contract out of special additions to the class's Javadoc comments, and modified the source code to include assertions at the relevant points in execution.

This led me to wonder about modern Java syntax, and whether it's possible to make a similar tool using Java's annotation features so that programmers don't have to worry whether a source conversion tool has introduced errors, or trace changes to the source code when working back from a failure report to the broken source.

The answer is yes, and so now the Labrary can offer Java by Contract as a tool for designing Java types by contract. It encodes the parts of the contract as names of methods to invoke that return boolean, failing if the answer is false. A rewrite of the Map interface from page 13 of the paper is given below.

public interface Map {
  /**
   * Does the key k appear in the map?
   */
  @Precondition(name = "nonNullK")
  @Postcondition(name = "inMapIffInKeys")
  boolean has(Object k);

  default boolean nonNullK(Object k) {
    return (k != null);
  }
  default boolean inMapIffInKeys(Object k, Boolean result) {
    return (result == this.keys().has(k));
  }

  /**
   * The value of the map at key k, null if undefined.
   */
  @Precondition(name = "nonNullK")
  @Postcondition(name = "nonNullValueIffHasKey")
  Object item(Object k);

  default boolean nonNullValueIffHasKey(Object k, Object ret) {
    return ((ret != null) == this.has(k));
  }

  /**
   * Associate key k with value v.
   */
  @Precondition(name = "nonNullKeyAndValue")
  @Postcondition(name = "hasK")
  @Postcondition(name = "itemForKIsNowV")
  void put(Object k, Object v);

  default boolean nonNullKeyAndValue(Object k, Object v) {
    return ((k != null) &amp;&amp; (v != null));
  }
  default boolean hasK(Object k, Object v, Void result) {
    return this.has(k);
  }
  default boolean itemForKIsNowV(Object k, Object v, Void result) {
    return (this.item(k) == v)
  }

  /**
   * Remove key k and associated value from map.
   */
  @Precondition(name = "nonNullK")
  @Postcondition(name = "doesNotHaveK")
  void prune(Object k);

  default boolean doesNotHaveK(Object k, Void result) {
    return !this.has(k);
  }

  /**
   * The set of all keys in the map.
   */
  @Postcondition(name = "nonNullReturn")
  ReadOnlySet keys();

  default boolean nonNullReturn(ReadOnlySet result) {
    return (result != null);
  }
}

This approach lets us write contract conditions that have full access to the internal state of the objects they are implemented on, so internal invariants can be verified in addition to the properties explored through the interface (by specifying them on the implementing class, not on the declaring interface). As a few pages of the paper are dedicated to the author's (and the class's) experience with various design by contract tools and their shortcomings, it's valuable to explore this space and come up with better approaches. I hope that Java by Contract is a useful addition.

Feldman additionally notes that the Java class library, including the Collections library, violates the Command-Query Separation principle. He even notes that the language does: the construct:

x++;

is both a command (increment x) and a query (what value did x previously have?) in a single expression. The early exercises in his class instruct students to write CQS-satisfying library objects (hence the Map example above), which the subsequent exercises build on.

Feldman left academia the year after this paper was written, and is now at IBM Research. That means the trail of development of this class has run out, and we cannot say how the teaching would have adapted to the subsequent decade of progress in Java.

#### Tech's Meritocracy Problem
Meritocracy is a myth. And our belief in it is holding back the tech industry from getting better.
  
  The intent to be meritocratic is not a myth, but we know what road is paved with good intentions.

—from Tech's Meritocracy Problem.

#### Technical debt and jury service
We have the idea that in addition to the product development backlogs for our teams, there's an engineering backlog where technical debt paydown, process/tooling improvements, and other sitewide engineering concerns get recorded. Working on them is done in time that is, by definition, taken away from the product backlogs (because of Sustainable Pace).

A colleague recently described the time spent on the engineering backlog as a "tax", which is an interesting analogy. A pejorative interpretation is that, like a tax, centralised engineering work is a cost imposed that takes away from realising more value on my direct projects.

A positive spin is that taxes go toward funding the commons: no one of us can afford to build a road between our house and the office, but having roads connecting all the houses to all the offices has strategic benefit to society as a whole (higher productivity, lower unemployment, more opportunities) so if we all pay in a fraction of the cost of a road we can all have a road. Similarly, one product team might grind to a halt if they spend all of their time on the new CD pipeline infrastructure, but all teams will benefit if they all chip in a bit.

This version of the analogy implies that there might be, like the treasury, a central agency deciding how to spend the common wealth. Somebody needs to decide how much tax everyone should pay, what to do with dissenters (is it OK if your product team focuses on its sprint for a fortnight and doesn't do any of the engineering backlog?), whether to accept overpayments, and what those tax dollars should go on.

Only it's not tax dollars, it's tax hours. In this sense, a better analogy is conscription (I originally thought of the Anglo-Saxon fyrd, maybe jury service or non-military national service is a less aggressive way to consider this). Taxation means that I give all of my work time to Wealth Wizards but give a chunk of my money to the government. Conscription means that I don't get to give all of my time to my employer: some of it has to go to the commons. Maybe Jonathan and Rebecca can't give any time to their product teams this week because they've been "called up" to the engineering backlog effort.

That seems like a useful analogy for these tasks. I can think about what resources are available for products or "the commons", because I can think about whether someone is working on "the day job" or has been conscripted. Maybe it doesn't make sense for everybody to have equal likelihood of being "called up", in the same way that it's easier for students to get out of jury service than for full-time employees.

#### Template change
I had to make some minor edits to the Blogger template used here anyway, so I decided to have a little monkey with the fonts.  Here's what you used to get: 

And here's what you now get: 

I think that's much better.  Good old Helvetica!  I don't understand anything about typography at all, but I think that the latter doesn't look as messy, perhaps because of the spacing being tighter.  Interestingly the title actually takes up more space, because the bold font is heavier.  Marvellous.

#### Template class for unit testing Core Data entities
Some time ago, in a blog far, far, away, I wrote about unit-testing Core Data. Essentially, your test case class should create a temporary, in-memory Core Data stack in -setUp, and clean it up in -tearDown. Your test methods can access the temporary context, model and persistent store to investigate the behaviour of objects under test.

The thing is, with any non-trivial Core Data app you're going to end up with multiple entities, each with its own suite of tests. Wouldn't it be nice if Xcode could set that stuff up automatically?

Oh, right, it can :-). The template header is class.h:

//
//  «FILENAME»
//  «PROJECTNAME»
//
//  Created by «FULLUSERNAME» on «DATE».
//  Copyright «YEAR» «ORGANIZATIONNAME». All rights reserved.
//

#import &lt;SenTestingKit/SenTestingKit.h&gt;

@interface «FILEBASENAMEASIDENTIFIER» : SenTestCase {
    NSPersistentStoreCoordinator *coord;
    NSManagedObjectContext *ctx;
    NSManagedObjectModel *model;
    NSPersistentStore *store;
}

@end

And the implementation, class.m:

//
//  «FILENAME»
//  «PROJECTNAME»
//
//  Created by «FULLUSERNAME» on «DATE».
//  Copyright «YEAR» «ORGANIZATIONNAME». All rights reserved.
//

«OPTIONALHEADERIMPORTLINE»

@implementation «FILEBASENAMEASIDENTIFIER»

- (void)setUp
{
    model = [[NSManagedObjectModel mergedModelFromBundles: nil] retain];
    coord = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel: model];
    store = [coord addPersistentStoreWithType: NSInMemoryStoreType
                                configuration: nil
                                          URL: nil
                                      options: nil 
                                        error: NULL];
    ctx = [[NSManagedObjectContext alloc] init];
    [ctx setPersistentStoreCoordinator: coord];
}

- (void)tearDown
{
    [ctx release];
    ctx = nil;
    NSError *error = nil;
    STAssertTrue([coord removePersistentStore: store error: &error], 
                 @"couldn't remove persistent store: %@", error);
    store = nil;
    [coord release];
    coord = nil;
    [model release];
    model = nil;
}

@end

Put those in a folder called /Library/Application Support/Developer/Shared/Xcode/File Templates/Thaes Ofereode/Objective-C test case class (Core Data).pbfiletemplate
, along with a TemplateInfo.plist file that looks like this:

&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
&lt;plist version="1.0"&gt;
&lt;dict&gt;
	&lt;key&gt;CounterpartTemplateFile&lt;/key&gt;
	&lt;string&gt;class.h&lt;/string&gt;
	&lt;key&gt;Description&lt;/key&gt;
	&lt;string&gt;A subclass of SenTestCase, with optional header. Automatically sets up a managed object context suitable for unit-testing Core Data entities.&lt;/string&gt;
	&lt;key&gt;MainTemplateFile&lt;/key&gt;
	&lt;string&gt;class.m&lt;/string&gt;
&lt;/dict&gt;
&lt;/plist&gt;

Xcode will automatically pick up the template the next time you use the template chooser.

#### Tempus Fugit
A common concern programmers have when I talk about my year off is that I'll be unemployable at the end of it. After all, the industry moves really quickly and if I'm off thinking about things that aren't programming, I'll fall off the treadmill. Programmers are like the red queen, constantly running in order to stand still. Aren't they?

Well, no, not really. Looking at the current TIOBE programming language index, I've written software in nine of the top ten languages (which are all at least a decade old). The likelihood that all of these will become obsolete in a year is miniscule, and the likelihood that the underlying principles of organisation of thought will perish is smaller still.

What about the platforms? Will big screens, small screens, touch screens, pointing devices, keyboards, web clients&nbsp; or network servers disappear within the next year? How would a freeze-dried programmer from 2014 or even 2005 cope with today's near-identical world?

Maybe, should I come back to professional programming next year, I'll find that I've grown my ability to understand things that aren't programming; a skill that could stand programmers in great stead. I doubt, however, that I'll have lost my ability to use a text editor and a compiler, tools that remain obstinately similar to their 1950s forebears.

#### Test-Driven iOS Development
Here it is, after more than a year in the making, the book that they really did want you to read! Test-driven IOS Development (Developer's Library) (affiliate link) has finally hit the stores[*].

I wrote this book for the simple reason that it didn't exist. Like Professional Cocoa Application Security (Wrox Professional Guides) (another affiliate link), I knew that the topic was something many people were interested in, including myself. I wanted a single place I could go to find out about using Xcode for writing unit tests, and how the Test-Driven Development approach could be applied to writing Cocoa code with Objective-C.

Well, it turned out that this place didn't exist, so I tried my best to create it. In the book you'll see how the BrowseOverflow app was built entirely in a test-first fashion, using OCUnit and Xcode 4. Along the way, I hope you'll see how TDD can be useful for your own projects.

If you're interested in finding out more about test-driven development for iOS apps, I'll be giving a talk on the subject at iOSDev UK in Aberystwyth in July. And of course I'm happy to field questions about the book or about TDD here or over on Twitter. And of course my unit testing video training course is still available from iDeveloper.tv.

Happy testing!

[*]The availability appears to vary by country and format, so if it isn't in your favourite (e)bookstore yet just be patient and keep refreshing that page like it's the WWDC site.

#### TGD - begging the question
Many of the arguments pro-religious belief presented in TGD would form a nice corpus in a dictionary of critical analysis for the entry on &quot;begging the question&quot;.  However, Prof. Dawkins studiously avoids the phrase despite its rampant, repeated and some would say wanton applicability, and if I were ever asked to predict why (I haven't been, I just like the sound of my keyboard) I'd say that it's because the phrase is so frequently misused on both sides of the Atlantic ocean.

Indeed, when I was taught the phrase, the example I was given is that of proof of the existence of god: we can see the results of god's creation all around, therefore god created them, therefore god exists.  That's about as short - and absurd - an example of the technique as can be demonstrated.  In ordinary speech, people use the phrase &quot;begging the question&quot; to mean: the argument I have just heard is incomplete or fallacious, because it appears not to answer (or to directly raise) the following problem.  Where the problem was not explicitly part of the argument.  However, the meaning of the word &quot;beg&quot; should not be overlooked, and is key when calling an argument out as one that begs the question.  In fact, it isn't the argument that begs the question: it's the conclusion which does so, and the conclusion is begged from the question.  In other words, the meaning of the phrase is that an argument draws to a conclusion which already existed in its assumptions or axioms.  My Latin isn't very good, but I might describe this kind of argument as QEQ - Said that which was said.

Let's look at, for example, the argument for the existence of purgatory (it's quite near the end of TGD which makes it easy for me to remember).  The conclusion is that purgatory exists, and the justification is that people everywhere are praying for the souls of the dead because they believe that the souls are in purgatory, and surely prayer isn't pointless.  I hope that going all slanty made the point in the argument where the conclusion was codified apparent - the reason we can be sure that purgatory exists because we are sure that purgatory exists.  QEQ.

I realise that this had little to do with the thesis of TGD, but to beg the question is a cool phrase which should be used less, but with higher accuracy.

#### TGD - expanding the field
Some primarily stochastic thought processes which occurred when I tried to apply Richard Dawkin's hypotheses to the Æsir.  If your default browsing font doesn't contain a glyph for the ligature, well, tough ;-)

I suppose one of the first things to note is that whereas Xtianity is considered a religion, the Æsir and Vanir (the Norse pantheon) are thought more often as folklore or mythology.[1]  In fact, upon reading the Eddas it's easy to have the impression that the sagas of Odin, Thor, Loki and chums have the same qualities as those associated with, for instance, Siegfried (of Das Niebelungenlied fame), King Arthur or Finn MCoul.  Essentially, the gods have the air of being erstwhile real blokes (and of course blokesses), who have accumulated stories, feats and powers as people seek to glorify them, in order that when they later claim to be descended from same they can hope to persuade people of the existence of said powers.[2]

What I find interesting is that the only difference between a quirky set of historically interesting tales and gospel truth is how many people believe in what's said.  For instance, it would be easy to apply the same distinction above between religion and folklore to the Roman pantheon, which equally was a major European religion relegated to providing saints and fables once Constantine got splashed in the font.  Of course, to do so would be to ignore that there were multiple pre-Christian religions in the Roman Empire.  Not merely in the same way that the Norse posh nobs worshipped Odin and the thains worshipped Thor, there were actually completely different mythological universes. I'm going to choose one, completely at random.

Between the second century BC and fourth (maybe fifth) century AD, a particular popular mythos in the Empire was Mithrainism.[3]  If there are any modern Mithraists, I don't know about it.  Which is not surprising, considering how wacky their religion was.  [Update: apparently some Zoroastrians still venerate Mithras.]

Mithras was supposed to have been born around 270BC to a virgin Mother of God (the date of the celebration of his birth was December 25th).  He was worshipped as a member of a trinity, as the mediating force between the heaven and earth.  In fact, heaven was not only the celestial abode of God but also the place where atoned souls would go when they died, the true believers being absolved of their earthly sins.  Those less fortunate were condemned to an infernal hell.  Initiates (ceremonies were closed affairs, available only to men who had performed the appropriate rites) were baptised, and Sundays were a sacred time when the Mithraists ate bread, representing the body of their God, and drank wine, representing his blood; these were symbolic of the final supper he shared with his followers before ascending to heaven in about the 64th year of his life. Along with Odin and Osiris, he is supposed to have died and been resurrected before his final ascension.

You'd never get away with that rubbish these days, which is why this is clearly a deluded heathen folk tale as opposed to, um.  You can clearly see why Dawkins didn't talk about this one in TGD...
[1] Actually, there is a religion with such a pantheon, called the Ásatrú - the word is Icelandic for Æsir faith.  Despite widespread confusion, none of the major organisations of this faith are actually neo-Nazis or supremacy groups.
[2] I suppose this makes Finn and Aragorn the same being.
[3] Just through etymology I am reminded that I haven't yet covered Jainism.  I need to.

#### TGD - I'm not dead yet!
The title of this article is a quote from Monty Python and the Holy Grail - the body cart comes around the town to collect up the (presumably leprous) corpses, and along comes John Cleese with his ninepence and his cadaver.  The only problem is that actually the corpse feels rather like he might pull through.

This is a situation which relates to the ever-decreasing sphere in which divine intervention can be invoked, such reduction being due to scientific advances.  There are known cases where people have been thought dead and have &quot;got better&quot;, or have been ruled out a long time before the actual event of their dying.  As an example, a death through epilepsy can often cause the unfortunate to become unresponsive long before becoming dead, so that 'miraculous' events such as the body remaining warm for days after the death are recorded.  While the meaning of the term &quot;dead ringer&quot; has nothing to do with the 19th Century safety coffin, such devices did exist.  The whole purpose behind holding a wake was the hope that life might return to the deceased - because sometimes it did.  Various poisons can lead to the body attaining a death-like state, while in fact the subject remains living; perhaps the most famous of these is TTX (the toxin carried by pufferfish); supposedly used by Vodou practitioners to create zombies.   A good explanation of that is Ghosts, Vampires and Zombies: Cinema Fiction vs Physics Reality.  The case of Wilfred Doricent, described in the arXiv paper, is recent: he died in March 1988 and was no longer displaying many of the symptoms of death by September 1989.

The point here is that there are modern cases where death is misdiagnosed.  Given the modern equipment, techniques and advances in both theoretical and practical knowledge, one would assume that the rate of misdiagnosis of death has decreased over time (and in a related vein, the number of conditions which terminate terminally has decreased, too).  How can we know, without even contemporary doctors' notes, whether a particular resurrection event was miraculous or just a case of mistaken interment?  Pufferfish aren't particularly common around the middle East so perhaps Lazarus wasn't eating fugu but he may have had epilepsy, tetanus or a number of other conditions which were confused with fatality.

Returning to the parenthesis on terminal conditions, above, one can argue by progression that if there are fewer &quot;terminal&quot; conditions now in (kindof) 2000 than there were in 1800, then assuming no catastrophic dark age reversion there will be even fewer in 2200, fewer still in 2400 and so forth.  So the ability to &quot;miraculously&quot; recover from a terminal condition should decrease over time too; not because the rate of miracles is decreasing but because the ability to explain the circumstances is increasing.

#### TGD - John on acid
I'm about halfway through my OmniOutliner document of notes made while reading TGD now....

There's a paraphrase in TGD saying that if the epistles of Saint John the Apostle represent John on pot, then the book of Revelation is &quot;John on acid&quot;.  What makes this observation more interesting is that it may be literally true - or at least closer to the mark than the quip may at first appear.

There was a documentary by Tony Robinson on Revelation and the End of Days, called The Doomsday Code.  The - frankly worrying - sentiments of some of the extreme christian right in America are the same as described in TGD; namely that the Middle East instability [inclusive] or global warming or the 2005 hurricane season herald the proximity of the rapture, and that rather than doing anything to help those caught up in these situations we should be actively encouraging the Apocalypse, and xtians should let their heathen friends know that, well, they told us so.  In the course of describing modern End Timers, the documentary showed us where Revelation was penned.

There are tens of apocalyptic visions penned by post-Christ Jewish theologians, but only one has been included in what is now termed the canon of the New Testament.  That book of the revelation of Jesus Christ to his disciple, John (which isn't accepted by the Eastern Orthodox as part of the divine liturgy) was written on the Greek island of Patmos sometime in the second half of the first Common Era century, and details two visions experienced by the John of the title (whether or not he be apostolic) while on the island.  The other key fact to remember about Patmos is the abundance of fly agaric mushrooms, especially around the mouth of the cave of revelation.

#### TGD - religion leaving the gene pool
If there was some kind of selection pressure which favoured, even in a roundabout way, predisposition toward religion, and if sexual selection naturally favours successful people (or those who exude success, anyway), then we have a (not too serious) quandary.  Because the most high-profile adherents, the most apparently successful at religion, are often the clerics.  Who are frequently celibate (or at least, are required to be: please do read the dictionary definition for nepotism though).

#### TGD Responses - selection for religion?
I'm going to do a few articles based on The God Delusion by Richard Dawkins.  As this is a bit of a departure from the usual content of yon blog, I'll be sure to prefix the titles of each with TGD so that those so inclined can filter for/against them in their syndication readers.  The articles will be a mixture of prose agreeing with parts of the book, arguing against parts, questions which the book made me ponder and for which I have no answer, and quips (I haven't yet decided whether to group these together or allow a post for each) answering specific ideas or even sentences.  My only hope is that all of the articles - and the processes leading to their inception - are critical, in what I hope is the obvious intention of that word.  Needless to say, I'll leave the punchline - my opinion of the binding theme of TGD and any changes it has had or not had on my views - until the final article.

I'll start this series by looking at his idea of religion having its roots in a &quot;misfiring&quot; of some advantageous trait which has been selected for.  I would ask this question: why should that be so?  In his description of evolution he explains the idea of a neutral mutation &mdash; one which overall has no positive nor negative effect on the fitness of the genome, but which can nevertheless become dominant (or at least prevalent).  For instance, human earlobes either join at the bottom of the lobe to the face or they do not.  I don't understand there to be any fitness advantage in either, yet at some time the allele(s) associated with earlobe stuck-to-facedness have, I presume, diverged from a common variant.  Why couldn't theism be just one type of earlobe?  What if the mean fitness of religious nonreligious &quot;phenotypes&quot; (I expect that's the wrong word&hellip;perhaps theotype is better) are not significantly different, and religious behaviour just happens to arise and have become common?  I suppose there may be an answer to this, although if there is it wasn't presented in TGD.

There is a related argument in the book which I don't buy.  In dismissing the idea of a straightforward selection pressure for evolution, Prof. Dawkins presents the Gedankenexperiment of a tribe who worship a war god, and whose religion dictates that they should go and fight the neighbouring tribes, in the course of which they acquire of course the resources of those other tribes, and propagate their seed further.  The argument presented against this is that for any one individual it is more efficient to hang back while their companions do all the fighting, then to share a part in the spoils.  Therefore the society would collapse, as no-one would want to do any fighting.  This rather squarely misses the point of, for example Freya's D.Phil. thesis and the concept that a population based on cooperation (in this case the war-tribe, in Freya's case a biofilm) can in fact tolerate some fraction of &quot;cheats&quot; and that there is a stable state where proportion of cheats is non-zero but the population still thrives.  Consider the British benefit system &mdash; some non-zero amount of money is spent on targeting benefit fraud, which keeps the fraud not at zero but at some level which both can be afforded and which leaves the spending justified.  In fact, a canny war-tribe member would try to weasel their way out of the line of combat by arguing (rightly or otherwise) that they alone have some property that the average combatant does not possess, and would refer to the weaseling as &quot;promotion&quot; or officership.

Another potential explanation is an interpretation from Terry Pratchett, Jack Cohen and Ian Stewart's Science of Discworld series.  In it, they describe the idea of &quot;lies-to-children&quot; in which various layers of falsehood are peeled back as a child becomes ready to accept the more complex, yet more accurate, explanations.  What if this idea is extended to become lies-to-protoscientists?  What if, as well as a moral Zeitgeist, there is a consensus of acceptable sophistication?  So a society initially agrees that each tree, waterfall, star, planet etc. must have its own god, then finds that it can accept the god-of-concept ideas of the more recent polytheistic disciplines, then agrees that a single god, while more complex a solution, is palatable, all the while science is also working to provide the ultimate in sophisticated solutions: that which is born out of simple concepts, but which can be applied to give predictable and verifiable results.

#### That can't possibly work.
A while back I was at a Facebook developer event, talking about techniques for analysing Objective-C. My summary of the problem was something like "it's one of those things that works pretty well in the ivory towers of practice but completely falls apart when you try to use it in theory."

That's true of many things in building software, but as people who get paid for removing bugs from things we're all too good at the situations where the theory doesn't pan out. Problems arise when we report the overly general conclusion: "that doesn't work" rather than "there exists a condition in which that doesn't work".

Often, the appropriate thing to do is to build the thing anyway and extract value from it in the cases where it does work. Most Ruby code is just Java with different punctuation: if you build a thing that can't possibly work because of monkey patching then build it anyway and 100% (give or take) of Ruby you actually encounter will work with it.

If it were up to programmers we wouldn't have paper money. You can't promise to back 10× as much money as you've actually got gold to make good on, in case everyone asks for it back. Ah, but what if they don't?

And even if you find that the edge situations arise too frequently to make the thing you built worthwhile, you'll have learned something about the problem. You may find a different approach, or even that solving a slightly (or greatly) different problem is what you really want to do.

When someone tells you the thing you want to build can't work, build it and work with it. If only for the look on their face.

#### That syncing feeling
So, ITunes tells me my iPod is up-to-date, and that it won't copy a few songs to my iPod because the iPod software needs updating first.  The word which springs to mind is &quot;erk&quot;.

#### The "return a command" trick in Haskell
The "return a command" trick

This is a nice trick, but we need a phrase for that thing where you implement extreme late binding of functions by invoking an active function that selects the function you want based on its name. I can imagine the pattern catching on.

#### The ABC of Software Engineering Research
About this paper
The ABC of Software Engineering Research by Klaas-Jan Stol and Brian Fitzgerald, published October 2018. See link for full citation.
Notes
There are too many ways in which terms describing research methods in software engineering get used, and these authors have a solution. The reason, at least according to the introductory discussion in this paper, is in part a case of discipline envy. This is the idea that we don't quite know what software engineering is, but we know what those people over there do, and we like that, so we'll co-opt it.

You could argue that the entire idea of software engineering is discipline envy. A collection of computing experts from academia and industry didn't quite know how to formalise the problems faced by software makers in the 1960s, but they did know what engineers do, and they liked that. In fact, it's not clear that they (or at least we, their intellectual descendents) truly understood what engineers do, but nonetheless we gave it a jolly good go. In 1968, in the town of Garmisch in Germany, a discipline was born.

Now, it's interesting that discipline envy has turned up so early in this discussion, because the contribution in this paper is a framework borrowed from social science researchers. To understand the applicability of cross-discipline seeding, we have to ask how strong the analogy "software engineering is just like X" seems to be (as well as identify how well the proposed idea worked in field X). So, is software engineering like a social science?

Here, the authors carve the field in two. They distinguish "solution-seeking" research, in which we identify what we ought to do about a problem, from "knowledge-seeking" research, in which we identify what people do do about the problem. The bad news about ditching the solution-seeking half of the discipline is that we just lost the engineering from software engineering, the bit where we use scientific results to propose novel solutions to problems.

The good news is that knowledge-seeking software engineering research does look quite a bit like a social science. People, in some context, do things, and we can try to understand that. Indeed that is the origin of the ABC initialism: Actors (the people), Behaviour (the things) and Context.

Well, we can understand bits of it at a time. Like good consultants, the authors introduce a quadrant diagram. On one axis, the "generalisability" of a research method, from highly universal contexts to deeply specific contexts. On the other, the "obtrusiveness" of the method, reflecting whether the researchers are passive observers or active interferers.

As the Labrary stands at the intersection, we approve of the idea that two different things lie on a continuum, rather than being an either/or choice. This framework makes the point that while a particular research technique or strategy is situated somewhere in the general/obtrusive map, others are available elsewhere. The reaction to a highly-controlled lab experiment should not be to declare that the result is not generalisable, but to understand what else could be done to explore generalisations of its results.

The discussions of where particular research strategies fit in the map are interesting, though some of the analogies drawn are fairly tenuous. The authors show where on the map the maximum applicability of a method for each of the key properties lies: universally contextual, unobtrusive research generalises over Actors best, while highly-obtrusive methods allow more precise measures of particular Behaviours and more focus gives a more realistic Context. It would be really beneficial to see a similar framework for "solution-seeking" literature, so we can evaluate the applicability of techniques developed in software engineering research to "practical" problems.

#### The age-old problem
"To configure your network card, please download our drivers from this website"That's a bit like the experience at the moment, getting the NE2K driver for NeXTSTEP onto a Parallels-running installation.  Never fear, for I have created NE2k driver floppy which can be used in Parallels.Edit 2006-04-13 12:03 GMT - changed the download URL, once I remembered how small my bandwidth at SDF is.

#### The App that Wasn't (Yet)
One of the early goals written into the mission statement of the Labrary was an eponymous app for organising research notes. I've used Mekentosj Springer Readcube Papers for years, and encountered Mendeley and others, and found that they were all more focussed on the minutiae of reference management, rather than the activity of studying and learning from the material you're collecting in your library. Clearly those are successful apps that have an audience, but is there space for something more lightweight?

I talked to a few people, and the answer was yes. There were people in software engineering, data science, and physics who identified as "light" consumers of academic literature, people who read the primary literature to learn from and find techniques to apply, but do not need or even want the full cognitive weight of bibliographic reference management. They (well, "we", I wanted it too) wanted to make notes while they were reading papers, and find those notes again. We wanted to keep tags on interesting references to follow up. We wanted to identify the questions we had, and whether they were answered. And we wanted to have enough information—but not more—to help us find the original article again.

My first prototype was as simple as I could make it. There's a picture below: it's a ring binder, with topic dividers, and paper notes (at least one separate sheet for each article) which quickly converged on a pro forma layout as shown.

An early prototype of the Labrary app.

I liked it, in fact I quickly got to a point where I wouldn't read an article unless I had access to a pad and pen to add a page to my binder. People I showed it to liked it, too. So this seemed like a good time to crack open the software making tools!

The first software prototype was put together in spare time using GNUstep and Renaissance, and evinced two problems:

The UI design led back down the route of "bibliopedantry", forcing students to put more effort into getting the citation details correct than they wanted to.
Renaissance lacked support for some Cocoa controls it would have been helpful to use, so there was a choice to be made to invest more into improving Renaissance or finding a different UI layout tool.

A screenshot of the ill-fated "Library" window in Labrary's GNUstep prototype.

This experience made me look for other inspiration for ways to organise the user interface so that students get the experience of taking notes, not of fiddling with citation data. I considered writing Labrary as a plugin for the free Calibre e-reader app, so that Labrary could focus on being about study notes and Calibre could focus on being about library management. But ultimately I found the tool that solved the problem best: Apple's Finder.

The Labrary pro forma note as Finder stationery.

I've recreated the pro forma note from the binder as a text file, and set the "Stationery Pad" flag in the Finder. When I open this file, Finder creates a duplicate and opens that instead, in my editor of choice: ready to become a new study note! I put this in a folder with a Zim index file, so I can get the "shoebox" view of all the notes by opening the folder in Zim. It also does full-content searching, so the goal of finding a student's notes again is achieved.

Zim open on my research notes folder.

I'm glad I created the lo-fi paper prototype. It let me understand what I was trying to achieve, and show very quickly that my software implementation was going in the wrong direction. And I'm always happy to be the person to say "do we need to write this, or can it be built out of other bits?", as I explored for this project with Zim and Calibre.

#### The Atoms of Programming
In the world of physics, there are many different models that can be used, though typically each of them has different applicability to different contexts. At the small scale, quantum physics is a very useful model, Newtonian physics will yield evidently incorrect predictions so is less valuable. Where a Newtonian model gives sufficiently accurate results, it's a lot easier to work with than quantum or relativistic mechanics.

All of these models are used to describe the same universe - the same underlying collection of observations that can systematically be categorised, modelled and predicted.

Physical science (or experimental philosophy) does not work in the same way as computational philosophy. There are physical realisations of computational systems, typically manifested as electronic systems or pencil-and-paper simulations. But the software, the abstract configurations of ideas that run on those systems, exist in entirely separate space and are merely (though the fact that this is possible is immensely powerful) translated into the electronic or paper medium.

Of course one model for the software system is to abstract the electronic: to consider the movement of electrons as the presence of voltages at terminals; to group terminals as registers or busses; to further abstract this range of voltages as 0 and that range as 1. And indeed that model frequently is useful.

Frequently, that model is not useful. And the great thing is that we get to select from a panoply of other models, at some small or large remove from the physical model. We can use these models separately, or simultaneously. You can think of a software system as a network of messages passed between independent objects, as a flow of data through transformers, as a sequence of state changes, as a graph of single-argument functions, as something else, or as a combination of these things. Each is useful, each is powerful, all are applicable.

Sometimes, I can use these models to make decisions about representing the logical structure of these systems, transforming a concept into a representation that's valid in the model. If I have a statement in a mathematical formulation of my problem, "for any a drawn from the set of Articles there exists a p drawn from the set of People such that p is the principal author of a" then I can build a function, or a method, or a query, or a predicate, or a procedure, or a subroutine, or a spreadsheet cell, or a process, that given an article will yield exactly one person who is the principal author of that article.

Sometimes, I use the models to avoid the conceptual or logical layers at all and express my problem as if it is a software solution. Object-oriented analysis and design, data flow modelling, and other techniques can be used to represent a logical model, or they can be used to bash the problem straight into a physical model without having thought about the problem in the abstract. "Shut up and code" is an extreme example of this approach, in which the physical model is realised without any attempt to tie it to a logical or conceptual design. I'll know correct when I see it.

I don't see a lot of value in collecting programming languages. I can't count the number of different programming languages I've used, and many of them are entirely similar. C and JavaScript both have sequences of expressions that are built into statements that are built into procedures. Both let me build aggregations of data and procedures that either let me organise sequential programs, represent objects, represent functions, or do something else.

But collecting the models, the different representations of systems conceptually that can be expressed as software, sometimes called paradigms: this is very interesting. This is what lets me think about representing problems in different ways, and come up with efficient (conceptually or physically) solutions.

More paradigms, please.

#### The balloon goes up
To this day, many Smalltalk projects have a hot air balloon in their logo. These reference the cover of the issue of Byte Magazine in which Smalltalk-80 was shared with the wider programming community.

Modern Smalltalks all have a lot in common with Smalltalk-80. Why? If you compare Smalltalk-72 with Smalltalk-80 there's a huge amount of evolution. So why does Cincom Smalltalk or Amber Smalltalk or Squeak or even Pharo still look quite a lot like Smalltalk-80?

My answer is because they are used. Actually, Alan's answer too:
Basically what happened is this vehicle became more and more a programmer’s vehicle and less and less a children’s vehicle—the version that got put out, Smalltalk ’80, I don’t think it was ever programmed by a child. I don’t think it could have been programmed by a child because it had lost some of its amenities, even as it gained pragmatic power.

So the death of Smalltalk in a way came as soon as it got recognized by real programmers as being something useful; they made it into more of their own image, and it started losing its nice end-user features.
I think there are two different things you want from a programming language (well, programming environment, but let's not split tree trunks). Referencing the ivory tower on the Byte cover, let's call them "academic" and "industrial", these two schools.

The industrial ones are out there, being used to solve problems. They need to be stable (some of these problems haven't changed much in decades), easy to understand (the people have changed), and they don't need to be exciting, they just need to work. Cobol and Fortran are great in this regard, as is C and to some extent C++: you take code written a bajillion years ago, build it, and it works.

The academic ones are where the new ideas get tried out. They should enable experiment and excitement first, and maybe easy to understand (but if you need to be an expert in the idea you're trying out, that's not so bad).

So the industrial and academic languages have conflicting goals. There's going to be bad feedback set up if we try to achieve both goals in one place:

 	the people who have used the language as a tool to solve problems won't appreciate it if new ideas come along that mean they have to work to get their solution building or running correctly, again.
 	the people who have used the language as a tool to explore new ideas won't appreciate it if backwards compatibility hamstrings the ability to extend in new directions.

Unfortunately at the moment a lot of languages are used for both, which leads to them being mediocre at either. The new "we've done C but betterer" languages like Go, Rust etc. feature people wanting to add new features, and people wanting not to have existing stuff broken. Javascript is a mess of transpilation, shims, polyfills, and other words that mean "try to use a language, bearing in mind that nobody agrees how it's supposed to work".

Here are some patterns for managing the distinction that have been tried in the past:

 	metaprogramming. Lisp in particular is great at having a little language that you can use to solve your problems, and that you can also use to make new languages or make the world work differently to see how that would work. Of course, if you can change the world then you can break the world, and Lisp isn't great at making it clear that there's a line between following the rules and writing new ones.
 	pragmas. Haskell in particular is great at having a core language that people understand and use to write software, and a zillion flags that enable different features that one person pursued in their PhD that one time. Not all of the flag combinations may be that great, and it might be hard to know which things work well and which worked well enough to get a dissertation out of. But these are basically the "enable academic mode" settings, anyway.
 	versions. Perl and Python both ran for years in which version x was the safe, stable, industrial language, and version y (it's not x+1: Python's parallel versions were 2 and 3000) in which people could explore extensions, removals, or other changes in potentially breaking ways. At some point, each project got to the point where they were happy with the choices, and declared the new version "ready" and available for industrial use. This involved some translation from version x, which wasn't necessarily straightforward (though in the case of Python was commonly overblown, so people avoided going from 2 to 3 even when it was easy). People being what they are, they put a lot of store in version numbers. So some people didn't like that folks were recommending to use x when there was this clearly newer y available.
 	FFIs. You can call industrial C89 code (which still works after three decades) from pretty much any academic language you care to invent. If you build a JVM language, it can do what it wants, and still call Java code.

Anyway, I wonder whether that distinction between academic and industrial might be a good one to strengthen. If you make a new programming language project and try to get "users" too soon, you could lose the ability to take the language where you want it to go. And based on the experience of Smalltalk, too soon might be within the first decade.

#### The Cataract of Software Delivery
There's this paper from August 1970, called Managing the Development of Large Software Systems, that's considered something of a classic (either for good or for bad, depending on your worldview). The discussion often goes something like this:

Let’s say I have some software to build, and I think it’s going to take about a year to build it. Few people are going to happily say “go away for a year and tell me when its done”. Instead, most people will want to break down that year into smaller chunks, so they can monitor progress and have confidence that things are on track. The question then is how do we perform this break down?

The waterfall style, as suggested by the Royce sketch, does it by the activity we are doing. So our 1 year project might be broken down into 2 months of analysis, followed by 4 months design, 3 months of coding, and 3 months of testing.
Martin Fowler, Waterfall Process

Within the phase space of Royce paper discussions, there are those who say that he argues for the strict sequential process as Fowler does, based on Figure 2 in the paper. There are those who say that he predicts some iterative, evolutionary, (dare we say it?) agile contribution to the process, based on Figure 4 in the paper. But many agree with Fowler when he says “this paper seems to be universally acknowledged as the source of the notion of waterfall”.

It isn't. The overall process had already been described in 1956 as a “structured” (ever wonder where the phrase “structured programming” comes from, when it clearly doesn't refer to having data structures?), “top-down” approach, by Herb Benington in Production of Large Computer Programs. Methodologists contemporary with and later than Royce, including luminaries like Barry W. Boehm and Agilistas like Alan Moran, knew about this paper, so even if we don't use it in our software mythologies any more, it isn't obscure and wasn't obscure in Royce's time.

Both Benington and Royce write within a context of large-scale government-funded projects: Benington from his experience with SAGE (the Semi-Automatic Ground Environment) and Royce at the aerospace contractor TRW. Both talk about phased approaches with dependencies between tasks (so you can't do coding until you've done design, for example). Both knew about the value of prototyping, though in a mistake that makes Hoare's introduction of NULL look like loose change, Benington didn't mention it until 1983:

I do not mention it in the attached paper, but we undertook the programming only after we had assembled an experimental prototype of 35,000 instructions of code that performed all of the bare-bone functions of air defense. Twenty people understood in detail the performance of those 35,000 instructions; they knew what each module would do, they understood the interfaces, and they understood the performance requirements. People should be very cautious about writing top-down specs without having this detailed knowledge, so that the decision-maker who has the "requirement" can make the proper trade-offs between performance, cost, and risk.

To underscore this point, the biggest mistake we made in producing the SAGE computer program was that we attempted to make too large a jump from the 35,000 instructions we had operating on the much simpler Whirlwind I computer to the more than 100,000 instructions on the much more powerful IBM SAGE computer. If I had it to do over again, I would have built a framework that would have enabled us to handle 250,000 instructions, but I would have transliterated almost directly only the 35,000 instructions we had in hand on this framework. Then I would have worked to test and evolve a system. I estimate that this evolving approach would have reduced our overall software development costs by 50 percent.
Herb Benington, Production of Large Computer Programs

Royce, on the other hand, describes the “write one to throw away” approach, in which a prototype informs the design of the final system but doesn't become part of it:

A preliminary program design phase has been inserted between the software requirements generation phase and the analysis phase. This procedure can be criticized on the basis that the program designer is forced to design in the relative vacuum of initial software requirements without any existing analysis..As a result, his preliminary design may be substantially in error as compared to his design if he were to wait until the analysis was complete. This criticism is correct but it misses the point. By this technique the program designer assures that the software will not fail because of storage, timing, and data flux reasons. As the analysis proceeds in the succeeding phase the program designer must impose on the analyst the storage, timing, and operational constraints in such a way that he senses the consequences. When he justifiably requires more of this kind of resource in order to implement his equations it must be simultaneously snatched from his analyst compatriots. In this way all the analysts and all the program designers will contribute to a meaningful design process which will culminate in the proper allocation of execution time and storage resources. If the total resources to be applied are insufficient or if the embryo operational design is wrong it will be recognized at this earlier stage and the iteration with requirements and preliminary design can be redone before final design, coding and test commences.
Winston Royce, Managing the Development of Large Software Systems

Royce's goal with this phase was to buttress the phased development approach, which he believed to be “fundamentally sound”, by adding data in earlier phases that informed the later phases. Indeed, if Royce finds any gap in the documentation on a project “my first recommendation is simple. Replace project management. Stop all activities not related to documentation. Bring the documentation up to acceptable standards. Management of software is simply impossible without a very high degree of documentation.”

So we have a phased, top-down, sequential development process in 1956, that survived until 1970 when more phases were added to reduce the risk accepted in lower phases. Good job those lightweight methodologists came along in the 1990s and saved us with their iterative, incremental development, right?

Not quite. Before Boehm knew of Benington's paper, he'd already read Hosier's Pitfalls and Safeguards in Real-Time Digital Systems with Emphasis on Programming from June 1961. Hosier presents (in figure 2) a flow chart of program development, including feedback from much of the process and explicit “revision” steps in the control routine, unit programming, and assembly.

It's not so much that nobody knew about iterative development, or that nobody did it, or that nobody shared their knowledge. Perhaps it just wasn't in vogue.

#### The challenges of teaching software engineering
I've just finished teaching a four-day course introducing software engineering for the first time. My plan is to refine the course (I'm teaching it again in October), and it will eventually become the basis for doctoral training programmes in research software engineering at Oxford, and part of a taught Masters. My department already has an M.Sc. in Software Engineering for commercial engineers (in fact I have that degree), and we want to do the same for software engineers in research context.

Of course, I can also teach your team about software engineering!

Some challenges that came up:

I'm too comfortable with the command-line to get people past the initial unfamiliar discomfort. From that perspective, command-line tools are all unusably hard. I've learnt from various sources to try foo --help, man foo, and other incantations. Others haven't.
git, in particular, is decidedly unfriendly. What I want to do is commit my changes. What I have to do is stage my changes, then commit my staged changes. As a result, teaching git use takes a significant chunk of the available time, and still leaves confusion.
you need to either tell people how to set their core.editor, or how to quit vim.
similarly, there's a world of difference between python foo.py and python3 foo.py, and students aren't going to interpret the sorts of errors you et if you choose the wrong one.
Introduce a tangent, and I run the risk of losing people to that tangent. I briefly mentioned UML while discussing diagrams of objects, as a particular syntax for those diagrams. In the subsequent lab, some people put significant time into making sure their diagrams were valid UML.
Finding the trade-off between presentation, tutorial, and self-directed exercise is difficult. I'm used to presentations and will happily talk on many topics, but even I get bored of listening to me after the ~50% of the time I've spent speaking on this course. It must be worse for the students. And there's no substitute for practical experience, but that must be supported by guidance.
There are so many topics that I didn't get to cover!

only having an hour for OOP is a sin
which means I didn't even mention patterns or principles
similarly, other design techniques like functional programming got left off
principles like Agile Software Development, Software Craftsmanship, or Devops don't get a mention
continuous integration and continuous delivery got left off. Even if they didn't, the amount of work involved in going from "I have a Python script" to "I run my tests whenever I change my script, and update my PYpi package whenever they pass" is too damn high.
forget databases, web servers, browsers, mobile apps, desktop apps, IoT, or anything that isn't a command line script or a jupyter notebook
and machine learning tools
and concurrency, processes and process improvement, risk management, security, team dynamics, user experience, accessibility...

It's only supposed to be a taster but I have to trade off introducing everything with showing the value present in anything. What this shows, as I found when I wrote APPropriate Behaviour, is that there's a load that goes into being a programmer that is not programming.

#### The closed-open principle
Some programming languages have a final keyword, making types closed for extension and open for modification.

#### The code you wrote six months ago
We have this trope in programming that you should hate the code you wrote six months ago. This is a figurative way of saying that you should be constantly learning and assimilating new ideas, so that you can look at what you were doing earlier this year and have new ways of doing it.

It would be more accurate, though less visceral, to say "you should be proud that the code you wrote six months ago was the best you could do with the knowledge you then had, and should be able to ways to improve upon it with the learning you've accomplished since then". If you actually hate the code, well, that suggests that you think anyone who doesn't have the knowledge you have now is an idiot. That kind of mentality is actually deleterious to learning, because you're not going to listen to anyone for whom you have Set the Bozo Bit, including your younger self.

I wrote a lot about learning and teaching in APPropriate Behaviour, and thinking about that motivates me to scale this question up a bit. Never mind my code, how can we ensure that any programmer working today can look at the code I was writing six months ago and identify points for improvement? How can we ensure that I can look at the code any other programmer was working on six months ago, and identify points for improvement?

My suggestion is that programmers should know (or, given the existence of the internet, know how to use the index of) the problems that have already come before, how we solved them, and why particular solutions were taken. Reflecting back on my own career I find a lot of problems I introduced by not knowing things that had already been solved: it wasn't until about 2008 that I really understood automated testing, a topic that was already being discussed back in 1968. Object-oriented analysis didn't really click for me until later, even though Alan Kay and a lot of really other clever people had been working on it for decades. We'll leave discussion of parallel programming aside for the moment.

So perhaps I'm talking about building, disseminating and updating a shared body of knowledge. The building part already been done, but I'm not sure I've ever met anyone who's read the whole SWEBOK or referred to any part of it in their own writing or presentations so we'll call the dissemination part a failure.

Actually, as I said we only really need an index, not the whole BOK itself: these do exist for various parts of the programming endeavour. Well, maybe not indices so much as catalogues; summaries of the state of the art occasionally with helpful references back to the primary material. Some of them are even considered "standards", in that they are the go-to places for the information they catalogue:

If you want an algorithm, you probably want The Art of Computer Programming or Numerical Recipes. Difficulties: you probably won't understand what's written in there (the latter book in particular assumes a bunch of degree-level maths).
If you want idioms for your language, look for a catalogue called "Effective &lt;name of your language&gt;". Difficulty: some people will disagree with the content here just to be contrary.
If you want a pattern, well! Have we got a catalogue for you! In fact, have we got more catalogues than distinct patterns! There's the Gang of Four book, the PloP series, and more. If you want a catalogue that looks like it's about patterns but is actually comprised of random internet commentators trying to prove they know more than Alastair Cockburn, you could try out the Portland Pattern Repository. Difficulty: you probably won't know what you're looking for until you've already read it&mdash;and a load of other stuff.

I've already discussed how conference talks are a double-edged sword when it comes to knowledge sharing: they reach a small fraction of the practitioners, take information from an even smaller fraction, and typically set up a subculture with its own values distinct from programming in the large. The same goes for company-internal knowledge sharing programs. I know a few companies that run such programs (we do where I work, and Etsy publish the talks from theirs). They're great for promoting research, learning and sharing within the company, but you're always aware that you're not necessarily discovering things from without.

So I consider this one of the great unsolved problems in programming at the moment. In fact, let me express it as two distinct questions:

How do I make sure that I am not reinventing wheels, solving problems that no longer need solving or making mistakes that have already been fixed?
A new (and for sake of this discussion) inexperienced programmer joins my team. How do I help this person understand the problems that have already been solved, the mistakes that have already been made, and the wheels that have already been invented?

Solve this, and there are only two things left to do: fix concurrency, name things, and improve bounds checking.

#### The death of scripting
Back in the day, when programmers knew that they couldn't possibly think of everything somebody might want to do with a computer, there were scripts. If somebody could find enough of the pieces of the thing that they wanted to do, they might be able to put them together themselves in furtherance of their task.

Many times, constructing these scripts was a lot like programming the software being glued together. On the Amiga there was ARexx, on the PC there were batch files, on Mac there was AppleScript: all programming in its own right, making new applications out of the ones you'd bought.

Applications. Here's the dichotomy. Think of two axes on a chart: one axis records the things you want to do with a computer; the tasks you want to complete. The other records the things you can do with the computer: the applications to which it can be put.

These axes are not perpendicular, as if there is no projection into your tasks by your applications. But they are not parallel either. And where the directions taken by the applications are not progressing your tasks, in comes scripting to provide bridges between those applications and take you on your way.

Not all of these bridges are esoteric programming languages on top of other programming languages. NeXT had services, in which applications could publish menu items that became available in other applications where the two were using the same data. Apple took a bit from each column to make Automator, a UI in which you could snap together bits of applications to make your task.

All of this represented a helpfulness and humility on the part of the applications makers: we do not know everything you want to do. We do know some things you might want to do: we'll let you combine them and mash them up - "rip, mix and burn" as they used to say - making you more satisfied and our stuff more useful.

Sadly all of this utility plays merry hell with branding. Applications aren't just utilities, they're icons in the launcher, splash screens, names in menu bars, reminders that I also make other applications and by the way have you rated this one five stars yet? Scripts stop people seeing that, they're too busy using their computers productively to see the marketing.

And so it's sad to see scripting die out as the popular platforms for application development fail to support it. Instead of the personal control of the script - I will take this information from that app, and put this part of it in that app - we have the corporate control of the API. This app maker and that app maker are BFFs, sign in here to let them share everything. After all, they know best.

Ultimately the death of scripting is hubristic. We know how you want to use a computer. If you're trying to do something that we didn't sell to you, you must be holding it wrong.

#### The debugger of royalty
We've all got little libraries of code or scripts that help us with debugging. Often these are for logging information in a particular way, or wrapping logs/tests such that they're only invoked in Debug builds but not in production. Or they clean up your IDE's brainfarts.

Having created these debug libraries, how are you going to get your production code to use them? Are you really going to sprinkle MyCompanyDebugLog(fmt,…) messages throughout your app?

Introducing step one on the road to sanity: the debug proxy. This is useful when you want to find out how a particular class gets used, e.g. when it provides callbacks that will be invoked by a framework. You can intercept all the messages to the object, and inspect them as you see fit. Here's the code (written with the assumption that ARC is enabled):

FZADebugProxy.h

#import &lt;Foundation/Foundation.h&gt;

@interface FZADebugProxy : NSProxy

- (id)initWithTarget: (NSObject *)aTarget;

@end

FZADebugProxy.m

#import "FZADebugProxy.h"

@implementation FZADebugProxy {
    NSObject *target;
}

- (id)initWithTarget:(NSObject *)aTarget {
    target = aTarget;
    return self;
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    NSMethodSignature *signature = [target methodSignatureForSelector: sel];
    if (signature == nil) {
        signature = [super methodSignatureForSelector: sel];
    }
    return signature;
}

- (BOOL)respondsToSelector:(SEL)aSelector {
    return [target respondsToSelector: aSelector] ? YES : [super respondsToSelector: aSelector];
}

- (void)forwardInvocation:(NSInvocation *)invocation {
    invocation.target = target;
    SEL aSelector = [invocation selector];
    (void)aSelector;
    [invocation invoke];
}

@end


And no, there isn't a bug in the -initWithTarget: method. The slightly clumsy extraction of the selector in -forwardInvocation: is done to avoid a common problem with using Objective-C inside the debugger where it decides it doesn't know the return type of objc_msgSend() and refuses to call the method.

You would use it like this. Here, I've modified the app delegate from BrowseOverflow to use a proxy object for the object configuration - a sort of domain-specific IoC container.


- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    BrowseOverflowViewController *firstViewController = [[BrowseOverflowViewController alloc] initWithNibName: nil bundle: nil];
    firstViewController.objectConfiguration = (BrowseOverflowObjectConfiguration *)[[FZADebugProxy alloc] initWithTarget: [[BrowseOverflowObjectConfiguration alloc] init]];
    TopicTableDataSource *dataSource = [[TopicTableDataSource alloc] init];
    [dataSource setTopics: [self topics]];
    firstViewController.dataSource = dataSource;
    self.navigationController.viewControllers = [NSArray arrayWithObject: firstViewController];
    self.window.rootViewController = self.navigationController;
    [self.window makeKeyAndVisible];
    return YES;
}


The bold line is the important change. The cast silences the compiler's strict type-checking when it comes to property assignment, because it doesn't believe that NSProxy is of the correct type. Remember this is only debug code that you're not going to commit: you could switch to a plain old setter, suppress the warning using diagnostic pragmas or do whatever you want here.

At this point, it's worth running the unit tests and using the app to convince yourself that the behaviour hasn't changed at all.

So, how do you use it? Shouldn't there be an NSLog() or something in the proxy class so you can see when the target's messaged?

No.

Step two on the road to sanity is to avoid printf()-based debugging in all of its forms. What you want to do here is to use Xcode's debugger actions so that you don't hard-code your debugging inspection capabilities into your source code.

Set a breakpoint in -[FZADebugProxy forwardInvocation:]. This breakpoint will be met whenever the target object is messaged. Now right-click on the breakpoint marker in the Xcode source editor's gutter and choose "Edit Breakpoint…" to bring up this popover.



In this case, I've set the breakpoint to log the selector that was invoked, and crucially to continue after evaluation so that my app doesn't stop in the debugger every time the target object is messaged. After a bit of a play with the app in the simulator, the debug log looks like this:

GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".sharedlibrary apply-load-rules all
Attaching to process 3898.
Pending breakpoint 1 - ""FZADebugProxy.m":37" resolved
Current language:  auto; currently objective-c
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"
0x10271:	 "stackOverflowManager"
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"


Pretty nifty, yes? You can do a lot with Xcode's breakpoint actions: running a shell script or an AppleScript are interesting options (you could have Xcode send you an iMessage every time it send your target an Objective-C message). Speaking out the names of selectors is fun for a very short while, but not overly useful.

Xcode's breakpoint actions give you a much more powerful debugging capability than NSLog(). By using breakpoint actions on an Objective-C proxy object, you can create highly customisable aspect-oriented techniques for debugging your code.

#### The Design of the Bazaar
In The Design of Design, Fred Brooks makes an interesting point about ESR's description of the Bazaar model of Linux (and, by extension, "Open Source") development.

Linux was actually designed in a cathedral. The design was supplied by Unix, where Linux was to be a work-alike replacement for a particular component. There was even a functional specification: the GNU utilities already existed and the kernel had to support them.

#### The difference between NSTableView and UITableView
A number of times, I've chased myself down rat holes in iPhone projects because I've created a design or implementation that assumes UITableView and NSTableView are similar objects. They aren't.The main problem I come across is related to how the cells are treated in Cocoa and in Cocoa touch. An AppKit table comprises columns, each of which uses a cell to display its content. A cell contains the drawing and event-handling stuff of a view, but nothing to do with its place in the view hierarchy or responder chain. It's essentially a light-weight view. For each row in the table, NSTableColumn takes its cell, configures it for the content in that row and then draws the cell at its location in the column. No matter how many rows there are, a single cell is used.UIKit works differently. Of course a UITableView only has one column, but it also displays views rather than cells. This is good, but leads to the key distinction that always trips me up: you can't use the same view more than once in a table view. Of course, sections in a UITableView will often have more than one row, but each row that is visible on-screen will needs its own instance of UITableViewCell (which is a subclass of UIView, and therefore a view in the traditional sense rather than a cell). If you try to re-use the same instance multiple times, the table view will configure each row but only the last one it prepared will be drawn.So what's this -reuseIdentifier? stuff? That's related to caching views for scrolling. Imagine a table view with 10 rows, of which 4 can be seen on screen at once. Each uses the same type of cell in this example. When the table view first becomes visible there will be 4 UITableViewCell instances in use, displaying rows 0-3. Now you start to scroll the view. UITableView finds it needs an extra cell to display row 4, which is now partially on-screen and row 0 is starting to slide off. When row 0 disappears completely, the table view could just delete its cell - but rather than do that, it adds it to a queue of reusable cells. When row 5 starts to appear, the table view can re-use the object it's already created for row 0, because it's the same type of cell as the one for row 5 and is currently unused.So, that's that really. Note to self: don't treat UIKit like it's just AppKit, you'll end up wasting a day of code.

#### The Dock should be destroyed, or at least changed a lot
I found an article about features Windows should have but doesn't, which I originally got to from OSNews' commentary on the feature list.  To quote the original article:

The centerpiece of every Mac desktop is a little utility called the Dock. It's like a launchpad for your most commonly used applications, and you can customize it to hold as many--or as few--programs as you like. Unlike Windows' Start Menu and Taskbar, the Dock is a sleek, uncluttered space where you can quickly access your applications with a single click.

Which OSNews picked up on:

PCWorld thinks Windows should have a dock, just like Mac OS X. While they have a point in saying that Windows' start menu and task bar are cumbersome, I wouldn't call the dock a much better idea, as it has its own set of problems. These two paradigms are both not ideal, and I would love someone to come up with a better, more elegant solution.

The problem I have with the Dock (and had with the LaunchPad in OS/2, the switcher in classic Mac OS, and actually less so with the task bar, though that and the Start Menu do suffer this problem) is that their job basically involves allowing the internal structure of the computer to leak into the user's experience.  Do I really want to switch between NeoOffice Writer, KeyNote and OmniOutliner, or do I want to switch between the document I'm writing, the presentation I'm giving about the paper and the outline of that paper?  Actually the answer is the latter, the fact that these are all in different applications is just an implementation detail.

So why does the task bar get that right?  Well, up until XP when MS realised how cluttered that interface (which does seem to have been lifted from the NeXT dock) was getting, each window had its own entry in the task bar.  Apart from the (IMO, hideously broken) MDI paradigm, this is very close to the "switch between documents" that I actually want to perform.  The Dock and the XP task bar have similar behaviour, where you can quickly switch between apps, or with a little work can choose a particular document window in each app.  But as I said, I don't work in applications, I work in documents.  This post is a blog post, not a little bit of MarsEdit (in fact it will never be saved in MarsEdit because I intend to finish and publish it in one go), the web pages I referenced were web pages, not OmniWeb documents, and I found them from an RSS feed, not a little bit of NetNewsWire.  These are all applications I've chosen to view or manipulate the documents, but they are a means, not an end.

The annoying thing is that the Dock so flagrantly breaks something which other parts of Mac OS X get correct.  The Finder uses Launch Services to open documents in whatever app I chose, so that I can (for instance) double-click an Objective-C source file and have it open in Xcode instead of TextEdit.  Even though both apps can open text files, Finder doesn't try to launch either of them specifically, it respects the fact that what I intend to do is edit the document, and how I get there is my business.  Similarly the Services menu lets me take text from anywhere and do something with it, such as creating an email, opening it as a URL and so on.  Granted some app authors break this contract by putting their app name in the Service name, but by and large this is a do something with stuff paradigm, not a use this program to do something one.

Quick Look and Spotlight are perhaps better examples.  If I search for something with Spotlight, I get to see that I have a document about frobulating doowhackities, not that I have a Word file called "frobulating_doowhackities.doc".  In fact, I don't even necessarily have to discover where that document is stored; merely that it exists.  Then I hit space and get to read about frobulating doowhackities; I don't have to know or care that the document is "owned" by Pages, just that it exists and I can read it.  Which really is all I do care about.

#### The feature constraint
If you're in a purely software business, your constraining resource is often (not always, not even necessarily in most cases, but often) the rate at which software gets changed. Well, specifically, the rate at which software gets changed in a direction your customers or potential customers are interested in. This means that the limiting factor on growth is likely to be rate at which you can add features or fixes that attract new customers, or maintain old customers.

It's common to see business where this constraint is not understood throughout management, particularly manifesting in sales. In a business to business context, symptoms include:

sales teams close deals with promises of features that don't exist, and can't exist soon.
there's no time to fix bugs or otherwise clean up because of the new feature backlog.
new features get added to the backlog based on the size of the requesting customer, not the cost/benefit of the customer.
the product roadmap is "what we said we'd have, to whom, by when", not "what we will have".

As Eliyahu Goldratt says, you have to subordinate the whole process to the constraint. That means incentivising people to sell something a lot like what you have now, over selling a bigger number of things you don't have now and won't have soon.

#### The First Flaw
As she left her desk at the grandiosely-named United States Robotics, Susan reflected on her relationship with the engineering team she was about to meet. Many of its members were juvenile and frivolous in her opinion, and she refused to play along with any of their jokes.

Even the title they gave her was mocking. They called her "the robopsychologist," a term with no real meaning as USR had yet to make a single product. They had not even sold any customers on the promise of a robot. All they had to their name was a rented office, some venture capital and their founder's secret recipe that was supposed to produce an intelligent sponge from a mixture of platinum and iridium.

While Susan might not be the robopsychologist, she certainly was a psychologist, of sorts. She seemed to spend most of her time working out what was wrong with the people making the robots, and how to get them to quit goofing off and start making this company some much needed profit. Steeling herself for whatever chaotic episode this dysfunctional group was going through, she opened the door to the meeting room. She was waved to a seat by director of product development Roger Meadows.

"Thanks for coming, Doctor Ca-"

Susan cut him off. "You've called me Susan before, Roger, you can do it again now. What's up?"

"It's Pal. We've lost another programmer."

Susan refused to call their unborn (and soon stillborn, if the engineers didn't buck up soon) product by its nickname, short for Proprietary Artificial Lifeform. She had at least headed off a scheme to call it "Robot 2 Delegate 2", which would have cost their entire budget before they even started.

"Well, look. I know it's hard, but those kids work too much and burn themselves out. Of course they're going to quit if-"

"No, I don't mean that. Like I said, it's Pal. He killed Tanya."

"Killed?" Susan suddenly realised how pale Roger looked, and that she had probably just gone a similar hue. "But how, no, wait. You said another programmer?"

"Er, yes. I mean, first Pal got Steve, but we though, you know, that we could, uh, keep that quiet until the next funding round, so-"

The blood suddenly came back to Susan's face. "Are you telling me," she snapped, "that two people had to die before you thought to ask for any help? Did you come to me now because you're concerned, or because you've run out of programmers?"

"Well, you know, I'd love to recruit more, but as they say, adding people to a late project..."

Yes, thought Susan, I do know what they say. You can boil the whole programming field down to damned aphorisms like that one. Probably they just give you a little phrasebook in CS101 and test you on it after three years, see if you have them all down pat.

"But what about the ethics code? Isn't there some module in that positronic brain you've built to stop that sort of thing happening?"

"Of course, the One Law of Robotics. The robot may not harm a human being. That was the first story we built. We haven't added the inaction thing the VCs wanted, but that can't be it. That mess in the lab was hardly the result of inaction."

"Right, the lab. I suppose I'd better go down and see for myself."

She quite quickly wished she hadn't. Despite having a strong constitution, Susan's stomach turned at the sight of barely-recognisable pieces of former colleague. Or possibly colleagues, she wasn't convinced Roger would have let a cleaner in between accidents.

The robot had evidently launched itself directly at and subsequently through Tanya, stopping only when the umbilical connecting it to the workstation had become disconnected, removing its power source. Outwardly and, Susan knew, internally, it lay dormant in its new macabre gloss coat.

"I take it you did think to do a failure analysis? Do you know what happened here?"

"If I knew that, Susan, I wouldn't have gotten you involved." She believed it, knowing her reputation at USR. "We've checked the failure tree and no component could cause the defect mode seen here."

"Defect mode! Someone's dead, Roger! Two people! People you're responsible for! Look, it went wrong somehow, and you're saying it can't. Well it can. How did the software check out?"

"I don't know, the software isn't in scope for the safety analysis."

Susan realised she was slowly counting to ten. "Well I'm making it in scope now. I took a couple CS classes at school, and I know they're using the same language I learnt. I'll probably not find anything, but I can at least take a look before we involve anyone else."

Hours later, and Susan's head hurt. She wasn't sure whether it was the hack-and-hope code she was reading or the vat of coffee she had drunk while she was doing it, but it hurt. So far she had found that the robot's one arm was apparently thought of as a specific type of limb, itself a particular appendage, in its turn a type of protuberance. She wasn't sure what other protuberances the programmers had in mind, but she did know the arm software looked OK.

So did the movement software. It had clearly been built quickly, in a slapdash way, and she'd noted down all sorts of problems as she read. But nothing major. Nothing that said "kill the person in front of you," rather than "switch on the wheel motors".

She didn't really expect to see that, anyway. The robot's ethics module, the One Law that Roger had quoted at her, was meant to override all the robot's other functions. Where was that code, anyway? She hadn't seen it in her study, and now couldn't find a file called ethics, laws or anything similar. Were the programmers over-abstracting again?, she thought. A law is a rule, no rules file.

Susan finally cursed the programmer mind as she found a source file called jude. Of course. But it definitely was what she was looking for: here was the moral code built into their first and, assuming USR wasn't shut down, all subsequent robots. Opening it, she saw a comment on the first two lines.

// "The robot may not harm a human being."
// Of course, we know that words like MUST, SHOULD and MAY are to be interpreted in accordance with RFC2119 ;-)

The bloody idiots, she thought. Typical programmers, deliberately misinterpreting a clear statement because they think it's funny. Poor Pal had not been taught good from bad. Susan realised that she had used his name for the first time. She was beginning to empathise more with the robot than she did with the people who built him. Without making any changes, she closed her editor and phoned Roger.

"Meadows? Oh, Susan, did you find out what's up?"

"Yes, I looked into the software. You can send all the programmers you want in there with Pal now."

#### The Fragile Manifesto
A lot of what I've been reading and thinking about of late is about the agile backlash. More speed, lower velocity reflects on IT teams pursuing "deliver more/newer IT" at the cost of "help the company achieve its mission". Grooming the Backfog is about one dysfunction that arises as a result: (mis)managing a never-ending road of small changes rather than looking at the big picture and finding a path toward the destination. Our products are not our products attempts to address this problem by recasting teams not as makers of product, but as solvers of problems.

Here's the latest: UK wasting £37 billion a year on failed agile IT projects. Some people will say that this is a result of not Agiling enough: if you were all Lean and MVP and whatever you'd not get to waste all of that money. I don't necessarily agree with that: I think there's actually things to learn by, y'know, reading the article.
The truth is that, despite the hype, Agile development doesn’t always work in practice.
True enough, but not a helpful statement, because "Agile" now means a lot of different things to different people. If we take it to mean the values, principles and practices written by the people who came up with the term, then I can readily believe that it wouldn't work in practice for people whose context is different from those who came up with the ideas in 2001. Which may well be everyone.

I'm also very confident that it doesn't mean that. I met a team recently who said they did "Agile", and discussed their standups and two-week iterations. They also described how they were considering whether to go from an annual to biannual release.
Almost three quarters (73%) of CIOs think Agile IT has now become an industry in its own right while half (50%) say they now think of Agile as “an IT fad”.
The Agile-Industrial Complex is well-documented. You know what isn't well-documented? Your software.
The report revealed 44% of Agile IT projects that fail, do so because of a failure to produce enough (or any) documentation.

The survey found that 34% of failed Agile projects failed because of a lack of upfront and ongoing planning. Planning is a casualty of today’s interpretation of the Agile Manifesto[…]

68% of CIOs agree that agile teams require more Architects. From defining strategy, to championing technical requirements (such as performance and security) to ensuring development teams stick to the rules of the game, the role of the Architect is sorely missed in the agile space. It must be reintroduced.
A bit near the top of the front page of the manifesto for agile software development is a sentence fragment that says:
Working software over comprehensive documentation
Before we discuss that fragment, I'd just like to quote the end of the sentence. It's a long way further down the page, so it's possible that some readers have missed it.
That is, while there is value in the items on the right, we value the items on the left more.
Refactor -&gt; Inline Reference:
That is, while there is value in comprehensive documentation, we value working software more.
Refactor -&gt; Extract Statement:
There is value in comprehensive documentation.
Now I want to apply the same set of transforms to another of the sentence fragments:
There is value in following a plan.
Nobody ever said don't have a plan. You should have a plan. You should be willing to amend the plan. I was recently asked what I'd do if I found that my understanding of the "requirements" of a system differ from the customer's understanding. It depends a lot on context but if there truly is a "the customer" and they want something that I'm not expecting to offer them, it's time for me to either throw away my version or find a different customer.

Similarly, nobody said don't have comprehensive documentation. I have been on a very "by-the-book" Agile team, where a developer team lead gave feedback that they couldn't work out where a change would go to enable a particular feature. That's architecture! What they wanted was an architectural plan of the system. Except that they couldn't explicitly want that, because software architecture is so, ugh, 1990s and Rational Rose. Wanting an architecture diagram is like wanting to use CORBA, urrr.

Once you get past that bizarre emotional response, give me a call.

#### The future will be just like the past, right?
I've been having a bit of a retro programming session:

The computer in the photo is a Cambridge Z88, and it won't surprise you to know that I've owned it for years. However, it's far from my first computer.

I was born less than a month before the broadcast of The Computer Programme, the television show that brought computers into many people's living rooms for the first time. This was the dawn of the personal computer era. The Computer Programme was shown at the beginning of 1982: by the end of that year the Commodore VIC-20 had become the first computer platform ever to sell more than one million units.

My father being an early adopter (he'd already used a Commodore PET at work), we had a brand new Dragon 32 computer before I was a year old. There's not much point doing the "hilarious" comparisons of its memory capacity and processor speed with today's computers: the social systems into which micros were inserted and the applications to which they were put render most such comparisons meaningless.

In 1982, computers were seen by many people as the large cupboards in the back of "James Bond film" sets. They just didn't exist for a majority of people in the UK, the US or anywhere else. The micros that supposedly revolutionised home life were, for the most part, mainly useful for hobbyists to find out how computers worked. Spreadsheets like VisiCalc might already have been somewhat popular in the business world, but anyone willing to spend $2000 on an Apple ][ and VisiCalc probably wasn't the sort of person about to diligently organise their home finances.

Without being able to sell their computers on the world-changing applications, many manufacturers were concerned about price and designed their computers down to a level. The Register's vintage hardware section has retrospectives on many of the microcomputer platforms from the early 1980s, many of which tell this tale. (Those that don't tell the tale of focusing on time to market, and running out of money.) The microprocessors were all originally controllers for disk drives and other peripherals in "real" computers, repurposed as the CPUs of the micro platforms. Sinclair famously used faulty 64kB RAM chips to supply the 48kB RAM in the ZX Spectrum, to get a good price from the supplier.

So the manufacturers were able to make the hardware cheap enough that people would buy computers out of interest, but what would they then make of them? We can probably tell quite a lot by examining the media directed at home computer users. Start with The Computer Programme, as we've already seen that back at the beginning of the post. What you have is Ian "Mac" McNaught-Davies, positioned at the beginning of episode 1 as a "high priest" of the mainframe computer, acting as the Doctor to Chris Serle's bemused and slightly apprehensive assistant. Serle is the perfectly ordinary man on the perfectly ordinary street, expressing (on behalf of us, the perfectly ordinary public) amazement at how a computer can turn a perfectly ordinary television set and a perfectly ordinary domestic cassette recorder into something that's able to print poorly-defined characters onto perfectly ordinary paper.

During his perfectly ordinary tenure of ten episodes, Serle is taught to program in BBC BASIC by McNaught-Davis. In the first episode he demonstrates a fear of touching anything, confirming the spelling of every word ("list? L-I-S-T?") he's asked to type. If the computer requires him to press Return, he won't do it until instructed by McNaught-Davis (thus making January 11, 1982 the first ever outing of The Return of the Mac). By the end of the series, Serle is able to get on a bit more autonomously, suggesting to Mac what the programs mean ("If temperature is more than 25, degrees I would assume...").

Chris Serle suffered his way through nine weeks of BASIC tuition because there was no other choice for a freelance journalist to get any use out of a personal computer. Maybe as many as 8,000 hipster programmers would opt for a Jupiter Ace and the FORTH language, but for normal people it was BASIC or nothing. Even loading a game required typing the correct incantation into the BASIC prompt. Feedback was minimal because there wasn't a lot of ROM in which to store the error messages: "Subscript at line 100" or even the Dragon's "?BS ERROR" might be all you're told about an error. If you didn't have a handy McNaught-Davis around (perhaps the first user-friendly Mac in the computer field) you could easily lose ages working out what the computer thought was BS about your code.

Typing errors became manifold when using the common application distribution platform: the printed magazine. Much software was distributed as "type-ins", often split over two (monthly) issues of a magazine: the program being presented in buggy form in one edition and an errata being supplied in the next. When you typed not one LOAD command, but a few hundred lines of BASIC in, only to find that your database program didn't work as expected, you first had a tedious proof-reading task ahead to check that you'd typed it without error. If you had, and it still didn't work, then out came the pencil and paper as you tried to work out what mistakes were in the listing.

Microcomputers represented seriously constrained hardware with limited application. The ability to get anything done was hampered by the primary interface being an error-prone, cryptic programming language. While the syntax of this language was hailed as simpler than many alternatives, it did nothing to smooth over or provide a soft landing for complex underlying concepts.

I'm willing to subject myself to those trials and terrors for the purpose of nostalgia. There are other people, though, who want to revert to this impression of computers as a way to get young people interested in programming. The TinyBASIC for Raspberry Pi announcement hails:

  we’ve also had a really surprising number of emails from parents who haven’t done any programming since school, but who still have books on BASIC from when they were kids, remember enjoying computing lessons, and want to share some of what they used to do with their kids. It’s actually a great way to get kids started, especially if you have some enthusiasm of your own to share: enthusiasm’s contagious.

Undoubtedly there are some genuine, remembered benefits to programming on these platforms, which modern computer tuition could learn from. There was, as discussed above, no hurdle to jump to get into the programming environment. Try teaching any programming language on widely-available computing platforms today, and you've got to spend a while discussing what versions of what software are needed, differences between platforms, installation and so on. Almost anyone on a microcomputer could turn on, and start typing in BASIC code that would, if restricted to a limited subset of commands, work whatever they'd bought.

The cost of a "tyre-kicking" setup was modest, particularly as you could use your own TV and cassette deck (assuming you had them). Unlike many modern platforms, there was no need to have two computers tethered to program on one and run on the other, and no developer tithe to pay to the platform vendors. Where they were error-free and well documented, the type-ins gave you actually working applications that you could tweak and investigate. Such starting points are better for some learners than a blank screen and a blinking prompt.

Complete applications though these type-ins may have been, they would not satisfy the expectations of modern computer-using learners. There's an important difference: people today have already used computers. They're no longer magical wonder-boxes that can make a TV screen flash blue and yellow if you get the numbers correct in a PAPER command. People know what to expect from a laptop, tablet or smartphone: being able to print an endless march of RUMBELOWS IS SHIT to the screen is no longer sufficient to retain interest.

It's not just the users of computers, nor the uses of computers, that have moved on in the last three decades. Teaching has evolved, too. There should probably be a name for the fallacy that assumes that however I was taught things is however everybody else should be taught them. A modern curriculum for novice programmers should reflect not only the technological and social changes in computing in the last thirty years, but also the educational changes. It should borrow from the positives of microcomputer programming courses, but not at the expense of throwing out a generation of evolution.

There are certainly things we can learn from the way microcomputers inspired a generation of programmers. There's a place for ultra-cheap computers like the Raspberry Pi in modern computing pedagogy. But it would be a mistake to assume that if I gave a child my copy of "Super-Charge Your Spectrum", that child would learn as much and be as enthused about programming as my rose-tinted model of my younger self apparently was.

#### The gaps between the processes
Knowledge management—not just in software engineering and not just digital knowledge management—has long had to account for tacit knowledge: the things that people know, but never say.

“A lesser blog would reproduce the Donald Rumsfeld statement about known unknowns at this point”—Graham Lee.

Where do people pick up this tacit knowledge? In conversations, at conferences, by reading books: it's not really tacit in that it's literally never spoken. Tacit knowledge is tacit in that people rely on it in particular scenarios without being explicit that they're relying on it.

Every software engineering process or methodology works (or not) in the presence of tacit knowledge. How they work in its presence is illuminating, mostly for telling us something about how the people who created the process think about the world.

The waterfall-as-practiced (i.e. not the Royceian "do it twice" version, but the DoD and everyone in the 1980s "do it once" version) model seems to rely on all knowledge being explicit, and available. There are truths about the universe that are your software's requirements, and if you spend enough time and do a good enough job at requirements gathering you can know what those requirements are before you get to the point where you use those requirements in writing a specification.

Every iterative and incremental model, from the Royceian waterfall-as-described process where you write one to throw away then write it again, through the spiral model, to the various scrums and scrumbuts in practice today, allows for people saying "oh, right, no, not that, because…" and bringing that tacit knowledge into the open. They might not express that that's why they have iterations, they might say it's for early and continuous delivery of value or to identify and mitigate risks, but they still do give space for "what I meant to say was…".

Structured programming approaches expect everything to be written down. A use case (in the original, technical meaning, not the current fancy-way-to-say-"situation" meaning) is a pedantic document that describes all of the interactions someone might have with "the system" in pursuit of a particular goal.

Lightweight approaches expect the bare minimum to be written down, and everything to be elucidated. A user story (in the original, technical meaning, not the current fancy-way-to-say-"Jira ticket" meaning) is a placeholder for a conversation. You're meant to write the story on one side of an index card, then, as you have the conversations, capture the rest of the information that everybody needs on the other side.

Internal wikis trade on the idea that if only it were really easy to edit a knowledge base, we'd all just write down our tacit knowledge and make it explicit, while also somehow making it salient, searchable, and up to date. Intranets trade on the idea that nobody knows anything outside of their lane and that it's up to the Department of XYZ to make the Policy for XYing your Z available to all employees or no Z will ever get XYd in the correct way.

Interestingly a recent development in software engineering methodology seems to be both positivist and dependent on tacit knowledge. That's the "web-scale" methodology, where you never talk to customers, but you run A/B tests, you create metrics, and optimise for those metrics. Positivist, because there is assumed to be a success factor that can be measured numerically and attained by moving the numbers. Tacit, because no customer ever says what they want or why they want it: instead, organisations create numeric goals that represent what a million, or a billion, customers are guessed to want, and strive to make the numbers go up (or down, as appropriate).

I suspect that this approach may be the biggest methodological innovation in software in more than two decades, and also one that methodologists seem to be currently quiet on.

#### The hardest thing
I now have the make the hardest decision in programming. It has nothing to do with naming things or invalidating caches: rather it is which *nix to install on a computer. NextBSD and MidnightBSD both have goals that are relevant to my interests, but both seem pretty quiet.

#### The Humpty-Dumpty Guide to OOP
Everybody knows that the best way to sound intellectual and demonstrate the superiority of your approach to that thing you do is to wrap it in a fancy-schmancy noun term. This works particularly well with a term that can be expressed as a three-letter acronym, or TLA. In programming we have a whole raft of different [A-Z]DD disciplines, the UML, CRC, and your DBA probably does SQL. It also explains why functional programming has taken so many decades to take hold: FP is 33% less appealing as a mere acronym of two letters (or ATL).

When your term is sufficiently arcane or abstract, you don't even need to actually do the three-letter nouny thing. Just tell people that what you're doing is the three-letter nouny thing, and let everybody fill in the gaps. In fact, they'll probably agree that not only are you doing that thing but then so are they. Soon, everybody will be doing the nouny thing.

So it goes with Object-Oriented Programming. We can construct all sorts of definitions for Object-Oriented Programming the term, so that everybody can get in on this highfalutin concept without having to do anything. The best time to arrange for this wide collection of meanings is as early as possible. Let's look at August 1981.

1. It's all about messages.

  The Smalltalk-80 system represents the current state of the object-oriented point of view as it has been reduced to practice by the Xerox Learning Research Group. The Smalltalk-80 system is composed of objects that interact only by sending and receiving messages. The programmer implements a system by describing messages to be sent and describing what happens when messages are received.

("The Smalltalk-80 System", Dave Robson and Adele Goldberg, BYTE Volume 6 Number 8 p.36)

2. Except when it's not.

  SIMULA allows users to create object-oriented systems, but uses the standard data/ procedure-oriented ALGOL language to provide numbers, booleans, basic data structures, and control structures.

(op. cit.)

3. Or maybe it is.

  Instead of two types of entity that represent information and its manipulation independently, an object-oriented system has a single type of entity, the object,
  that represents both. Like pieces of data, objects can be manipulated. However, like procedures, objects describe manipulation as well. Information is manipulated by sending a message to the object representing the information.

("Object-Oriented Software Systems", Dave Robson, BYTE Volume 6 Number 8 p.78)

4. Actually, maybe it means whatever you want it to mean.

  The words "object-oriented" mean different things to different people. Although the definition given in this article may exclude systems that should rightfully be called object-oriented, it is a useful abstraction of the idea behind many software systems.

(op. cit., p.74)

5. But it definitely means garbage collection.

  To be truly "object-oriented", a computer system must provide automatic storage management.

("Design Principles Behind Smalltalk", Dan Ingalls, BYTE Volume 6 Number 8 p.289)

6. And definitely (this time, maybe) means message-sending.

  This suggests a third principle for object-oriented design:
  
  Messages: Computing should be viewed as an intrinsic capability of objects that can be uniformly invoked by sending messages.
  
  [...]In most computer systems, the compiler figures out what kind of number it is and generates code to add 5 to it. This is not good enough for an object-oriented system because the exact kind of number cannot be determined by the compiler[...]

(op. cit., p.290)

7. This message-sending capability must be exposed to the UI.

  An enabling condition for adequate flexibility of a user interface can be stated as an object-oriented principle:
  
  Reactive Principle: Every component accessible to the user should be able to present itself in a meaningful way for observation and manipulation.

(op. cit., p.296)

[note: reactive programming woo]

So, you know what? Whatever you're doing, it's probably object-oriented. And whenever I say object-oriented, it may possibly mean whatever your meaning is.

#### The Ignoble Programmer
Two programmers are taking a break from their work, relaxing on a bench in the park across from their office. As they discuss their weekend plans, a group of people jog past, each carrying their laptop in a yoke around their neck and furiously typing as they go.
  
  "Oh, there goes the Smalltalk team," says the senior of the two programmers on the bench. "They have to do everything at run-time."

I love jokes. And not just because they're sometimes funny, though that helps: I certainly find I enjoy a conversation and can relax more when at least two of the people involved are having fun. When only one person is joking, it gets awkward (particularly if everyone else is from HR). But a little levity can go a long way toward disarming an unpleasant truth so that it can be discussed openly. Political leaders through the ages have taken advantage of this by appointing jesters and fools to keep them aware of intrigues in the courts: even the authors of the American bill of rights remembered the satirist before the shooter.

I also like jokes because of the thought that goes into constructing a good (or deliberately bad) one. There's a certain kind of creativity that goes into identifying an apparently absurd connection, exactly because of the absurdity. Being able to construct a joke, and being practised at constructing jokes, means being able to see new contexts and applications for existing ideas. Welcome to the birthplace of reuse and exploring the bounds of a construct's application: welcome to the real home of software architecture.

But there's a problem, or at least an opportunity (or maybe just a few thousand consulting dollars to be made and a book to be written). That problem is this: everyone else puts way more effort into their jokes than programmers do. Take this one, from the scientists:

  Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon

They didn't just joke about doing a brain scan of a dead fish, they did a brain scan of a dead fish. And published the (serendipitous and unexpected) results. But they didn't just angle for a laugh, they had a real point. The subtitle of their paper:

  An Argument For Proper Multiple Comparisons Correction

And isn't it fun that some microbiologists demonstrated that beards are significant vectors for microbial infections?

Both of these examples were lifted from the Annals of Improbable Research's Ig Nobel Prizes, awarded for "achievements that first make people laugh, and then makes them think". The Ig Nobels have been awarded every year since 1991, and in that time only one computer science award has been granted. That award was given to the developer behind PawSense, a utility that detects and blocks typing caused by your cat walking across your keyboard.

Jokes that first make you laugh, and then make you think, are absolutely the best jokes you can make about my work. If I conclude "you're right, that is absurd, but what if..." then you've done it right. Jokes that are thought-terminating statements can make us laugh, and maybe make us feel good about what we're doing, but cannot make us any better at it because they don't give us the impetus to reflect on our craft. Rather, they make us feel smug about knowing better than the poor sap who's the butt of the joke. They confirm that we've nothing to learn, which is never the correct outlook.

We need more Ig Nobel-quality achievements in computing. Disarming the absurd and the downright broken in programming and presenting them as jokes can first make us laugh, and then make us think.

N.B. My complete connection to the Annals of Improbable Research is that I helped out on the AV desk at a couple of their talks. At their talk in Oxford in 2006 I was inducted into the Luxuriant and Flowing Hair Club for Scientists.

#### The Image Model
I was reflecting on things that I know now, a couple of decades in to my career, that I wish I had been told at the beginning. Many things came to mind, but the most immediate from a technological perspective was Smalltalk's image model.

It's not even the technology of the Smalltalk image that's relevant, but the model of thinking that works well with it. In Smalltalk, there are two (three) important files for a given machine: the VM is the machine that can run Smalltalk; the image is a snapshot of all of the Smalltalk objects on the machine(; and the sources are the source code for the classes and methods in that image).

This has weird implications for how you work that differ greatly from "compile this text stream" or "interpret this text stream" programming environments. People who have used the ENVY/Developer tool generally seem to wax lyrical and wonder why it was never reinvented, like the rest of software engineering is the beach with the ruins of the Statue of Liberty poking out from the end of the Planet of the Apes. But the bit I wish I had been told about: the image model puts the "personal" in "personal computer" as far as programming is concerned. Every piece of software you write is part of your image: a peer of the rest of the software you wrote, of the software that other people wrote that you added, and of the software that was already there when you first booted the machine.

I wish I had been told to think like that: that each tool or project is not a separate tool or project, but a cumulative addition to the image. To keep everything I wrote, so that the next time I needed something I might not need to write it. To make sure, when using new things, that I could integrate them with the image (it didn't exist at the time, but TruffleSQUEAK is very much this idea). To give up asking "how can I write software to solve this problem", and to start asking "how can I solve this problem with software, writing some if necessary"?

It would be the difference between twenty of years of experience and one year of experience, twenty times over.

#### The importance of the passive voice is described.
I am writing a blog post, in which I intend to convince you of my case. A coherent argument must be created, in which the benefits of my view are enumerated. Paragraphs are introduced to separate the different parts of the argument.

The scene was set in the first sentence, so readers know that the actor in the following sentences must be me. Repeating that information would be redundant. Indeed, it was clearly me who set that scene, so no need to mention me at the start of this paragraph. An article in which each sentence is about the author, and not the article's subject, could be perceived as a sign of arrogance. This perception is obviously performed by the reader of the article, so there is no need to explicitly call that out.

The important features of the remaining sentences in the first paragraph are those relating to the structure of the article. These structural elements are subjects upon which I act, so bringing them to the fore in my writing involves suppressing the object, the actor in the text. I can do this by choosing to use the passive voice.

Unfortunately, grammar checkers throughout the world of computing give the impression that the passive voice is always bad. Millions of people are shown underlining, highlighting, and inline tips explaining that their writing is wrong. Programmers have leaked the abstraction that everything in their world is either 1 or 0, into a world where that does not make sense. Sentences are either marked active (1), correct (1), or passive (0), incorrect (0).

Let us apply that to other fields of creative endeavor. Vincent: a starry night is not that brightly colored. 0. You used too much paint on the canvas. 0. Stars are not that big. 0.

Emily: too many hyphens. 0. No need to capitalize "microscope". 0. Sentence fragment. 0.

#### The laser physics of software
I've worked in a few different places where there have been high-powered lasers, the sort that would make short work of slicing through Sean Connery in a Bond movie. With high-powered lasers comes mandatory laser safety training. At least, it does in the UK.

The first time you receive laser safety training it comes as a bit of a surprise. Then, the second (and later) times, you get to watch the same surprise in everybody else who's watching it for the first time.

The surprise comes because you kind of expect to hear about cooked retinas, skin burns and all sorts of unpleasant nastiness. Which of course you do, but those aren't the likely forms of laser accident. The surprising bit is the list of "popular" (not popular) ways to get injured by a laser:

You electrocute yourself on the power supply.
You ignite the liquid oxygen that's built up around the liquid nitrogen cooling system, probably using the same power supply.
You drop the laser (or indeed the power supply) on your foot.

So to programming. While we're all donning our protective goggles of shiny new type systems and lack of mutable state, the "popular" (not popular) problems (we don't know what we're doing, we don't know what we should be doing, we're bad at treating people well) are not being addressed.

#### The lighter side of open source
In a recent post I talked about the apolitical, amoral nature of open source software and how it puts the interests of a small programming class before the interests of the broad collection of people who interact with programmers' output. The open source movement has been of great benefit to the software industry, and this hasn't necessarily been a zero-sum game.

Reality is always more nuanced than history, and yet here is a potted guide to open source history. In the beginning, there were military computers. There was no-one else to share your computer programs with, because:

no-one else had a computer.
well, maybe they did, but they weren't telling you.
you didn't want to tell anyone else you have a computer.

Then there were academic computers. Now you do want to share your programs with everyone, and they share theirs with you and so everyone is on the cutting edge.

Then there were commercial computer companies (I told you this history would lack nuance), who were happy to share their programs with you because it meant you could get more out of the computers they were selling.

Then there were commercial computer companies who decided that the source code to the programs used to interface with their hardware were their competitive advantage, and decided to stop sharing them. This made an academic (Richard Meriadoc (humour me) Stallman) sad, and so he created the Free Software movement to:

promote sharing of software over not sharing software;
subvert the copyright system usually used to restrict sharing to enable sharing.

Then there were people who wanted to use Free Software in their day jobs but found that the movement was considered too idealogical to be palatable to management, so they rebranded it Open Source Software to re-frame the discussion along business, rather than political, lines.

This is about the point when your protagonist enters, stage right. The dot-com bubble was imploding, leading to changed fortunes for all sorts of people and organisations in the software industry. Everything I would do regarding professional computing depended in some way on the GNU project and the Free Software Foundation:

I learned Unix, thanks to the ability to inexpensively run GNU/Linux on my desktop computer.
The things I learned about Unix, C programming and so on were portable to various platforms beyond GNU/Linux, thanks to the GNU compiler collection, GNU bash, GNU make, GNU debugger and others.
One such platform was Mac OS X, the new hotness from Apple. This was a technology acquired through the purchase of NeXT, who had been able to provide a complete programming environment despite their small size and (comparatively) small budget by wrapping the tools listed above.

Somewhere in all the above I even found it possible to get paid for writing software: a GPLv2-licensed Lisp package for GNU Emacs.

Of course, that's just my story, but there are plenty like it. Many other programmers work on platforms like iOS, or Android, or Linux, or in environments like Ruby or Objective-C, that either only exist or have only become as successful as they have due to the successes of the Free Software Foundation, and the ability for organisations (commercial or otherwise) to take advantage of Free or Open Source software as building blocks which they can combine or add to.

Since then, the discussion has again been re-framed. Open Source - originally a branding change to make Free Software acceptable to business - has become a principle rather than a tool. A community that owes its financial viability to Free Software now denounces such "viral" licences, as source released under their conditions is harder to profit from than the more permissive, university-style Open Source licences.

Software writers in the 1980s liked to talk about how object technology would be the silver bullet that allowed re-use and composition of software systems, moving programming from a cottage industry where everyone makes everything from scratch to a production-line enterprise where standard parts fit together to provide a base for valuable products. It wasn't; the sharing-required software licence was.

#### The Liskov Citation Principle
In her keynote speech at QCon London 2013 on The Power of Abstraction, Barbara Liskov referred to several papers contemporary with her work on abstract data types. I've collected these references and found links to free copies of the articles where available.

Dijkstra 1968 Go To statement considered harmful

Wirth 1971 Program development by stepwise refinement

Parnas 1971 Information distribution aspects of design methodology 

Liskov 1972 A design methodology for reliable software systems

Schuman and Jorrand 1970 Definition mechanisms in extensible programming languages
Not apparently available online for free

Balzer 1967 Dataless Programming 

Dahl and Hoare 1972 Hierarchical program structures
Not apparently available online for free

Morris 1973 Protection in programming languages

Liskov and Zilles 1974 Programming with abstract data types

Liskov 1987 Data abstraction and hierarchy

#### The Logical Fallacy
Nary a week goes by without seeing a post by a programmer, for programmers, on the subject of logical fallacies in arguments. This week's, courtesy of hacker news, is not egregious, enlightening, or indeed different in any way from the usual torrent. It is merely the one that prompted me into writing this article. The most frequent, and most severe, logical fallacy I encounter among programmers is this one:

basing your argument on logic.

Now, obviously, for a fallacy to be recognised it needs to have a Latin name, so I'm going to call this one argumentum ex logica.

Argumentum ex logica is the fallacious reasoning that the best course of action for a group of people is the one that can be arrived at by logical deduction. No need to consider the emotions of the people involved, or the aesthetic properties of any potential solutions. Just treat your workplace like your high school debating club, pick (seemingly arbitrarily) some axioms, and batter your way through to your preferred conclusion.

If people disagree with you on (unreasonable) emotional grounds, just name their logical fallacies so you can overrule their arguments, like you're in an episode of Ally McBeal and want their comments stricken from the record. If people manage to find a flaw in the logic of your argument, just pick a new axiom you'd never mentioned before and carry on.

The application of the argumentum ex logica fallacy is frequently accompanied by descriptions of the actions of "the brain", that strange impish character that sits inside each of us and causes us to divert from the true path of Syrran of Vulcan. Post hoc ergo propter hoc, we are told, is an easy mistake to make because "the brain" sees successive events as related.

Here's the weird thing. We all have a "the brain" inside us, as an important part of our being. By writing off "the brain" as a mistaken and impure lump of wet fat, programmers are saying that they are building their software not for humans. There must be some other kind of machine that functions on purely logical grounds, for whom their software is intended. It should not be.

#### The manifesto for anarchic software development
Go on, read the manifesto again. You’ll see that it’s a manifesto for anarchism, for people coming together and contributing equally toward solving problems. From each according to their ability, to each according to their need.

The best architectures, requirements, and designs emerge from self-organizing teams.

While new to software developers in the beginning of this millennium, this would not have been news to architects who noticed the same thing in 1962. A digression: this was more than a decade before architects made their other big contribution to software engineering, the design pattern. The RIBA report noticed two organisations of teams:

One was characterised by a procedure which began by the invention of a building shape and was followed by a moulding of the client’s needs to fit inside this three-dimensional preconception. The other began with an attempt to understand, fully the needs of the people who were to use the building around which, when they were clarified, the building would be fitted.

There were trade-offs between these styles, but the writers of the RIBA report clearly found some reason “to value individuals and interactions over processes and tools”:

The work takes longer and is often unprofitable to the architect, although the client may end up with a much cheaper building put up more quickly than he had expected. Many offices working in this way had found themselves better suited by a dispersed type of work organisation which can promote an informal atmosphere of free-flowing ideas.

Staff retention was higher in the dispersed culture, even though the self-organising nature of the teams meant that sometimes the senior architect was not the project lead, but found themselves reporting to a junior because ideas trumped length of tenure.

This description of self-organising teams in architecture makes me realise that I haven't knowingly experienced a self-organising team in software, even when working on a team that claimed self-organisation. The idea is prevalent in software of a "platform shop": a company that builds Rails websites, or Java micro services, or Swift native apps. This is software's equivalent of beginning "by the invention of a building shape", only more so: begin by the application of an existing building shape, no invention required.
As the RIBA report notes, this approach "clearly goes with rather autocratic forms of control". By centralising the technology in the solution design, people can argue that experience with that technology stack (and more specifically, with the way it's applied in this organisation) is the measure of success, and use that to impose or reinforce a hierarchy.
Clearly, length of tenure becomes a proxy measure for authority in such an organisation. The longer you've been in the company, the more experience you have contorting their one chosen solution to attempt to address a client's problem. Never mind that there are other skills needed in designing a software product (not least of which is actually understanding the problem), and never mind that this "experience" is in repeated application of an unsuitable template: one year of experience ten times over, rather than ten years of experience.

#### The missing principle in agile software development
The biggest missing feature in the manifesto for agile software development and the principles behind it is anyone other than the makers and their customer. We get autonomous, self-organising delivery teams but without the sense of responsibility to a broader society one would expect from autonomous professional agents.

Therefore it's no surprise to find developers working to turn their colleagues into a below-minimum-wage precariat; to rig elections at scale; or to implement family separation policies or travel bans on religious minorities. A principled agile software developer only needs to ask "how can I deliver this, early and continuously, to the customer?" and "how can we adjust our behaviour to become more effective at this?": they do not need to ask "is this a good idea?" or "should this be delivered at all?"

Principle Zero ought to read something like this.

We build software that transforms the workplace, leisure, interpersonal interaction, and society at large: we do so in consultation with as broad a representation of those interests as possible and ensure that our software is beneficial to those whose lives are being transformed. Within that context, we follow these principles:

#### The new netiquette
It used to be that netiquette was all about TURNING OFF THE CAPS LOCK, making sure that the subject matched the content, that you didn't go off the wall if someone didn't reply to your e-mail in a few minutes, that sort of thing.  In fact, this (RFC1855) sort of thing.  But now there are different netiquette requirements, and no obvious guidelines, nor seemingly any common practices.  Now I'm just the kind of person who thinks that some de facto ruleset would be useful, so that everyone knows what to expect from everyone else.

For instance, take social networking sites like LinkedIn or Facebook.  How well do you know someone before you 'add' them as a friend?  Once met at a conference, once read their blog, cohabited for two years?  Do you talk to them first, to let them know who you are and that you're not a crazy stalker?  If someone adds you, and you don't know who they are, do you accept or reject by default?  Do you ask them who they are?  If they claim to have met you at $conference or in $pub, do you accept that, ask for a photo, or what?

#### The next million-dollar iPhone application
I'm constantly surprised by questions such as this one. They invariably go along the lines:

I heard that I need to get a Mac to do iPhone development. I want to do iPhone development but do I have to buy a Mac? Is there any other way to develop iPhone software?

If the projected sales for your app don't meet the cost of a new computer, whatever platform you're developing on, it's time to get a different idea for your app. I speak with the smug self-confidence of one who has yet to get his own app within smelling distance of the store.

#### The Nineteen Nineties
I've been playing a lot of CD32, and would just like to mention how gloriously 90s it is. This is the startup chime. For comparison, the Interstellar News chime from Babylon 5.

Sure beats these.

#### The Object is the Advantage
NeXT marketed their workstations by letting Sun convince people they wanted a workstation, then trying to convince customers (who were already impressed by Sun) that their workstation was better.

As part of this, they showed how much better the development tools were, in this very long reality TV infomercial.

#### The Objective-C protocol naming trifecta
Objective-C protocol names throughout history seem to fall into three distinct conventions:

some are named after what a conforming object provides. Thus we have DBProperties, DBEntities, DBTypes and the like in Database Kit.
others are named after what the object is doing. Thus we have NSCoding, NSLocking, IXCursorPositioning etc.
still others are named after what the conforming object is. NSTableViewDataSource, NSDraggingInfo etc.

I think I prefer the second one. It emphasises the dynamic message-sending nature of the language: "because you're sending messages from this protocol, I'll act like this".

#### The other pink dollar
How did (a very broad and collective) we go from selling NeXT at $440M to selling Tumblr at $1.1B, in under two decades? Why was Sun Microsystems, one of the most technologically advanced companies in the valley, only worth two Nests?

I don't think we're technologists (much) any more. We've moved from building value by making interesting, usable and advanced technology to building value by solving problems for people and making interesting, useful and advanced experiences. The good thing about a NeXT workstation was that it was better than other workstations and minicomputers; the bad thing is that you can't actually do a lot with a Unix workstation. You need applications, functions for turning silicon-rich paperweights into useful tools.

NeXT's marketing was it's easier to turn our paperweight into a tool than their paperweight, but today's tech companies are mostly making things you can already use. The technology is a back-office function, enabling the things you can already use or working around problems the companies discovered in trying to enable those useful things. Making the paperweight with the most potential is no longer interesting to most of the industry, though there will be money in paperweights for a few years yet (even if the paperweights are becoming small enough that they can't weigh down paper, and even if no-one has a stack of paper to weigh down any more).

Hence the current focus on "disruption", in the Silicon Valley, not Clayton Christensen, sense. It's easy to see how an already-solved problem can be solved faster, cheaper or better, by taking out intermediate steps or slow communications.

This is traditional science-fiction advancement. What technology do you need to get the plot moving quickly? Two people need to talk but they're not on the same planet: you need a mobile phone. Two people need to be in the same room but they're not on the same planet: you need a teleporter. Two people need to share the specifications of a starship but there isn't enough paper in the universe: you need a PADD.

It's harder to identify solutions to unsolved problems, or solutions to unknown problems. This hasn't changed since the paperweight days, the transition has been from "well, I guess you can find some problem to solve with this workstation" to "we solved this obvious problem for you". That's an advance.

It really is good that we're moving from building things that could potentially solve a problem to things that definitely do solve a problem. That's more efficient, as fewer people are solving the same problem. Consider the difference between every company buying a Sun workstation and hiring a programmer to write a CRM application, and every company paying someone else to deliver them a CRM system.

It's also likely the reason for the rise in open-source software, hardware, data centres and related infrastructure. Nobody's making railroads any more, they're making haulage companies that use railroads to solve the problem of "you're in Chicago, Illinois but your crate full of machinery is in a port in Seattle, Washington". There's lots of cost in having the best rails, but questionable benefit, so why not share the blueprints for the rails so that anyone can improve them?

Well, what if it turns out that the best way to haul your goods is not on rails? If it's easy to accept better rails, but the best solution lies in a different direction? Alan Kay would recognise this problem: everyone can see incremental improvements in the pink plane but there are magnitudes of improvements to be made by getting out into the blue plane.

How can a blue plane venture get funded, or adopted? When was the last time it happened? Is there something out there already that just needs adoption to get us onto the blue plane? Or have we set up a system that makes it easy to move quickly on the pink plane, but not to change direction to the blue plane?

#### The package management paradox
There was no need to build a package management system since CPAN, and yet npm is the best.
Wait, what?

Every time a new programming language or framework is released, people seem to decide that:

It needs its own package manager.
Simple algorithms need to be rewritten from scratch in "pure" $language/framework and distributed as packages in this package manager.

This is not actually true. Many programming languages - particularly many of the trendy ones - have a way to call C functions, and a way to expose their own routines as C functions. Even C++ has this feature. This means that you don't need any new packaging system, if you can deploy packages that expose C functions (whatever the implementation language) then you can use existing code, and you don't need to rewrite everything.

So there hasn't been a need for a packaging system since at least CPAN, maybe earlier.

On the other hand, npm is the best packaging system ever because people actually consume existing code with it. It's huge, there are tons of libraries, and so people actually think about whether this thing they're doing needs new code or the adoption of existing code. It's the realisation of the OO dream, in which folks like Brad Cox said we'd have data sheets of available components and we'd pull the components we need and bind them together in our applications.

Developers who use npm are just gluing components together into applications, and that's great for software.

#### The paradox of scripting
But how can scripting be dead? There's bash, and powershell, and ruby, and...even Perl is still popular among sysadmins. There's never been a better time to be a programmer or other IT professional trying to automate a task.

True, but there's never been a worse time for someone who doesn't care about computers to use a computer to automate a task. Apps are in-your-face "experiences" to be "used", and for the most part can't be glued together.

The message given off by the state of scripting is that scripting is programming, programming is a specialist pursuit, therefore regular folk should not be shown scripting nor given access to its power. They should rely on the technomages to magnanimously grant them the benefits of computing.

If I were to build an automation technology for today's mobile platforms, I'd probably call it Prometheus.

#### The Principled Programmer
[Note: this post represents the notes made for my talk at iOS Dev UK 2014. As far as I'm aware, the talk isn't available on the tubes.]

The Principled Programmer

The first thing to be aware of is that this post is not about my
principles. It's sort-of about your principles, in a way.

On dichotomies

Let's look at two games. You may not have heard of Chaturanga (unless
you practice yoga, but I'm talking about a different Chaturanga), but
it's the ancient game that eventually evolved into Chess.

You may not have heard of Nard either, but it's a very different game
that grew up into Backgammon. There's a creation myth surrounding
these two games, that says they were invented at the same time. Some
leader thousands of years ago wanted two games; one a game of skill
and the other a game of chance.

The thing is, you can lose at chess by chance: if you happen to be
having an off day and miss a key move that you'd often
make. Similarly, you can lose at backgammon through lack of skill: by
choosing to move the wrong pieces.

We were presented with two options: skill (chaturanga) and not-skill
(nard). However, the games do not actually represent pure states of
the two concepts; they're more like a quantum system where the
real-world states can be superpositions of the mathematically "pure"
states.

All of this means that we can't ignore states in-between the two
poles. Such ignorance has a name in the world of critical analysis:
the fallacy of the excluded
middle.

That is not the situation we have in bivalent logic, including the
mathematical Boolean formulation frequently used to model what's going
on in computers. This has the law of the excluded
middle, which
says that a proposition must either be true or false.

In this case, the fact is that the two propositions (you are playing a
game of skill, or you are playing a game of chance) do not exactly
match the two real possibilities (you are playing chaturanga, or you
are playing nard). There's a continuum possibility (you are using some
skill and some chance), but a false dichotomy is proposed by the
presentation in terms of the games.

On rules

The existence of a rule allows us to form a bivalent predicate: your
action is consistent with the rule. That statement can either be true
or false, and the middle is excluded.

This means we have the possibility for the same confusion that we had
with the games: compliance with the rule may be bivalent, but what's
going on in reality is more complicated. We might accidentally exclude
the middle when it actually contains something useful. Obviously that
useful thing would not be in compliance with the rule. So you can
think about a rule like this: a statement is a rule when you can
imagine contraventions of the statement that are of no different value
than observances of the statement. Style guides are like this: you can
imagine a position that contravenes the rules of your style guide
that is of no lesser or greater value: following another style
guide.

Of course, the value of a style guide comes not from the choice of
style, but from the consistency derived from always adhering to the
rule. It doesn't matter whether you drive on the left or the right of
the road, as long as everybody chooses the same side.

One famous collection of rules in software
engineering is Extreme Programming. Kent Beck described hearing or
reading about various things that were supposed to be good ideas in
programming, so he turned them up to eleven to see what would
happen. Here are some of the rules.

User stories are written. It's easy to imagine (or recall)
situations in which we write software without writing user stories:
perhaps where we have formal specifications, or tacit understandings
of the requirements. So that's definitely a rule.
All production code is pair programmed. The converse - not all
production code is pair programmed - poses no problem. We can imagine that the two conditions are different, and that we might want to choose one over another.

Rules serve two useful functions, of which I shall introduce one
now. They're great for beginners, who can use them to build a scaffold
in which to place their small-scale, disjoint bits of knowledge. Those
bits of knowledge do not yet support each other, but they do not need
to as the rules tell us what we need to apply in each situation.

The software engineering platypus

Having realised that our rules are only letting us see small pieces of
the picture, we try to scale them up to cover wider
situations. There's not really any problem with doing that. But we can
get into trouble if we take it too far, because we can come up with
rules that are impossible to violate.

A platitude, then, is a statement so broad that its converse cannot be
contemplated, or is absurd. Where a rule can be violated without
hardship, a platitude cannot be violated at all - or at least not
sensibly.

The problem with platitudes is that because we cannot violate them,
they can excuse any practice. "I write clean code": OK, but I don't
believe I know anybody who deliberately writes dirty code. "This
decision was pragmatic": does that mean any other option would be
dogmatic? But isn't "always be pragmatic" itself dogma?

Platitudes can easily sweep through a community because it's
impossible to argue against them. So we have the software
craftsmanship manifesto, which values:

A Community of Professionals. As any interaction
between people who get paid comes under this banner, it's hard to see
what novelty is supplied here.
Well-Crafted Software. Volunteers please for making shitty
software.

The Principled Programmer.

There must be some happy medium, some realm in which the statements we
make are wider in scope, and thus more complex, than rules, but not so
broad that they become meaningless platitudes that justify whatever
we're doing but fail to guide us to what we should be doing.

I define this as the domain of the principle, and identify a principle
thus: a statement which can be violated, where the possibilities of
violation give us pause for thought and make us wonder about what it
is we value. To contrast this with the statements presented earlier:

violate a rule: meh, that's OK, the other options are just as good.
violate a platitude: no, that's impossible, or ludicrous.
violate a principle: ooh, that's interesting.

Coming up with good principles is hard. The principles behind the
agile manifesto contain some legitimate principles:

Our highest priority is to satisfy the customer through early and
continuous delivery of valuable software. Interesting. I can imagine
that being one of many priorities of which others might be higher:
growing the customer base, improving software quality, supporting what
they're using now and deferring delivery of new software until it's
needed. I'll have to think about that.
Working software is the primary measure of
progress. Interesting. This seems to suggest that paying off technical
debt - exchanging one amount of working software for another amount of
working software over a period of time - is not progress. I'll have to
think about that.

But then it also contains rules:

Deliver working software frequently, from a couple of weeks to a
couple of months, with a preference to the shorter timescale. We ship
a couple of times a day, and I don't feel that's worse.
Business people and
developers must work together daily throughout the project. Is there
anything wrong with every other day?

And platitudes:

Build projects around motivated individuals. Give them the
environment and support they need, and trust them to get the job
done. I cannot imagine a situation where I would hire people who do
not want to do the work.
Continuous attention to technical excellence and good
design enhances agility. This is tautological, as technical excellence
and good design can be defined as those things that enable our goals
and processes.

Why is it hard? I believe it's because it's highly personal, because
what you're willing to think about and likely to get benefit from
thinking about depends on your own experiences and interests. Indeed
I'm not sure whether I want to define the principle as I have done
above, or whether it's the questions you ask while thinking about
the validity of those things that are really your principles.

## Nice principles. Now go and turn them into
rules.

The thing about thinking is that I don't want to do it when I don't
need to. My currency is thought, so if I'm still thinking next year
about the things I was considering this year, I'm doing it wrong.

Principles are great for the things that need to challenge
the way we work now. But they should be short-lived. Remember the
beginner use of rules was only one of two important contexts? The
other context is in freeing up cognitive space for people who
previously had principles, and now want to move on to have new
principles. In short-circuiting the complex considerations you
previously had, mentally automating them to prepare yourself for
higher-level considerations.

Notice that this means that it's a rule in isolation that doesn't cause us any problems to violate. It may be that the rule was derived from a principle, so some thought went into its construction. Without that information, all we can see is that there are two possibilities and we're being told that one of them is acceptable.

The challenge that remains is in communication, because it doesn't
help for the context of a rule to be misidentified. If you're a
beginner, and you describe your beginner rule and someone takes it as
an expert rule, they might end up talking about perspectives that
you're not expecting. Also if you're an expert and your expert rule is
perceived as a beginner rule, you might end up having to discuss
issues you've already considered and resolved.

So by all means, identify your principles. Then leave them behind and
discover new ones.

#### The problem with not-Apple
I've read a few articles over the last week or so that point to the Mac having lost its shine among developers. There was a time when the first things you did when you wanted to be a developer on the Free Software platform Ruby on Rails were that you bought an Apple PowerBook and the proprietary TextMate editor. There was a time when even Sun's employees programmed Java on Macs. But now, I read things like this:

  Right now, the only real option Apple has offered [vocal developer supporters] is the iMacs, which seems to be their answer for high end machines. That may work for some, even thought it won’t be their first preference for many. It’s clearly left many disgruntled and some thinking of jumping ship to other manufacturers, either running Linux or Windows. Source: Apple's 2016 in review
  
  Apple’s review process for [Safari browser] extensions is disorganized, arduous and quite frankly insulting  Source: What Apple gives you for $100 as a Safari Extension Developer
  
  the current state of the Mac has me considering whether it’s still the right platform for me. Source: Finding an Alternative to Mac OS X
  
  It seems like Apple has either lost its way, that it has lost touch with what (some of) its customers want, or that it simply doesn’t care about those customers. Developers are a captive audience, and creative professionals can switch to Windows, I guess. Apple no longer considers them core. Source: New MacBook Pros and the State of the Mac

For me the sheen was long-gone back in November 2014, and in January 2015 I posted about switching (back) to Linux. That was around the last time the blogosphere was telling us all that Apple had lost their way - funny how these badly-run companies manage to sell more of their shit than their competitors for years on end, non? Anyway, it was the popularity of the meme that led me to post, but my story about falling out of love with their treatment of Free Software and the make-work associated with being in their developer programs which you can read about in that post is personal to me.

There's a problem, though, and that problem is consistency. NeXTSTEP, more or less, can be summarised as "let's make an Alto, but compromise on using technology that already exists". So you get your Alto technology like OOP and ethernet and laser printers, but you also get the compromises - Display PostScript, UNIX, GNU, and C. There's one system to learn (Objective-C and the various object "kits"), and then a few subsystems (UNIX and GNU, Mach, NetInfo, DPS) that make themselves known if you dig in.

Mac OS X is less of a NeXTSTEP than NeXTSTEP, but the romance of consistency still exists. I can tell myself, partly because it's true but also because I invested over a decade of my life in working around the flaws in the model, that OS X is still Objective-C and kits with a wider selection of kits (Core Data, GameKit, PDFKit etc) and a few more compromises.

You definitely can't say that of Linux, particularly as a developer. The application my group works on now is written in Qt, which is itself a nice framework (Qt with its meta-object compiler is to C++ as Objective-C is to C, in a way that GTK+ is not), that for the most part just sits on top of Linux (and other platforms) as a Qt application. The problem is, when you want to do something that isn't in Qt's equivalent of the app kit, you may not only have to choose one of a few different alternative technologies but actually choose all of them if your users might not all have chosen the same one.

Even on my own laptop that's true. It is...well, for reasons that I just haven't put the effort into solving, it's actually running Ubuntu 16.10 in VirtualBox in Windows 10 (whomp!), but what I see is that it's running Ubuntu 16.04. Now I could, and do, use GNUstep as an application development environment, and get my Objective-C and kits running on something like Unix just as I'm used to. But that inconsistency is always there, always at the forefront, always chipping away. Because the window manager does not use the Objective-C runtime, and uses weird X things to communicate with processes rather than Objective-C messages. The browser is Firefox, because while there is a GNUstep browser, it's not very good (mostly because its WebKit is not up to date, but then WebKit is itself not Objective-C either). My Linux uses systemd to start processes, your Linux uses rc files/init files/upstart. My GNUstep is drawing with Cairo, yours is drawing with X intrinsics.

There's a problem with that problem, though, and it's that the consistency of Mac OS X is a fiction.

Are Dashboard widgets made out of JavaScript because of a compromise, or an aborted change of direction? Is the lack of consistency between the same API's names for things in Swift and in Objective-C a cognitive overload that's worth carrying around? Do I ignore the funky dialect of C++ that drivers are written in (I have written IOKit drivers and edited a book on the technology, so this isn't a hypothetical concern)? While some ObjC APIs use message-sending and others use block callbacks, am I right to call them both the same thing? Does this process communicate with that process using XPC, Mach IPC, UNIX pipes, sockets, signals, or distributed notifications?

The romance turns out to be based on a lie, but on a powerful and compelling lie that's easy to believe, and easy to miss even if you're unsure whether it ever existed.

#### The reality is not the abstraction
Remember that the abstractions you built to help you think about problems are there to help. They are not reality, and when you think of them as such they stop helping you, and they hold you back.

You see this problem in the context of software. A programmer creates a software model of a problem, implements a solution in that model, then releases the solution to the modeled problem as a solution to the original problem. Pretty soon, an aspect of the original problem is uncovered that isn't in the model. Rather than remodeling the problem to encapsulate the new information, though, us programmers will call that an "edge case" that needs special treatment. The solution is now a solution to the model problem, with a little nub expressed as a conditional statement for handling this other case. You do not have to have been working on a project for long before it's all nubs and no model.

You also see this problem in the context of the development process. Consider the story point, an abstraction that allows comparison of the relative sizes of problems and size of a team in terms of its problem-solving capacity. If you're like me, you've met people who want you deliver more points. You've met people who set objectives featuring the number of points delivered. You've met people who want to see the earned points accrue on a burn-down. They have allowed the story point to become their reality. It's not, it's an abstraction. Stop delivering points, and start solving problems.

#### The reasonable effectiveness of developer tools
In goals upon goals upon goals, I suggested that a fixation on developer tools is misplaced. This is not to say that developer tools are unhelpful, nor that they can't have a significant impact on our work.

Consider the following, over-restricted, definition of what a programmer does:

  A programmer's responsibility is to turn a computer into a solution to somebody's problem.

We have plenty of tools designed to stop you having to consider the details of this computer when doing that: assemblers, compilers, device drivers, hardware abstraction layers, virtual machines, memory managers and so on. Then we have tools to speed up aspects of working in those abstractions: build systems, IDEs and the like. And tools that help make sure you moved in the correct direction: testing tools, analysers and the like.

Whether we have tools that help you move from an abstract view of your computer to even an abstract view of your problem depends strongly on your problem domain, and the social norms of programmers in that space. Science is fairly well-supplied, for example, with both commercial and open source tools.

But many developers will be less lucky, or less aware of the tools at their disposal. Having been taken from "your computer…" to "any computer…" by any of a near-infinite collection of generic developer tools, they will then get to "…can solve this problem" by building their own representations of the aspects of the problem. In this sense, programming is still done the way we did it in the 1970s, by deciding what our problem is and how we can model bits of it in a computer.

It's here, in the bit where we try to work out whether we're building a useful thing that really solves the problems real people really have, that there are still difficulties, unnecessary costs and incidental complexity. Therefore it's here where judicious selection and use of tools can be of benefit, as their goals support our goals of supporting our users' goals.

And that's why I think that developer tools are great, even while warning against fixating upon them. Fixate on the things that need to be done, then discover (or create) tools to make them faster, better and redundant.

#### The repair permissions fiasco
So, I'm far from the first to discuss the voodoo surrounding repair permissions, but I've never seen quite such a flagrant violation of penal code 1984 (It is a felony to pass off your own sloppy programming as a user configuration issue)...Yesterday I bought some software from the Apple store (the developers and software shall remain nameless, but suffice it to say that it was an expansion pack for a game).  Came home in order to install it, that didn't quite work...I'd installed a (n official) patch for the original game, making its version newer than the version in the expansion pack.  OK, blow it away, and reinstall.The installer is a PEF-based Stuffit thing (I didn't realise people still used either PEF or Stuffit), so doesn't know anything about the Authorisation Services API.  OK, switch to an admin user, run the installer.  Switch back to regular user, and run the game.Erm, no in-game text.  On any of the UI elements.  That's a bit strange.  Let's check the FAQ on their website; apparently there's a particular file in the game which contains all of the text, if that's corrupted then no text.It's not corrupted, it's just unreadable by anyone except the user I installed it as.  Quick su-chmod-exit, try again.  OK, in-game text.  Asks for the licence key.  Enter it, gets accepted....asks for the licence key.  Enter it, check it, gets accepted....asks for the licence key.  Couple more times, checking really carefully now that I've entered the correct key (no acc1d3nt4l el1t3ne5s).  Definitely doesn't work.Back to the website FAQ: I find the file where the licence key gets stored.  In the app directory.  Guess who it's not writable by?  That's right, anyone...turns out that savegames get stored there too.  So the application dir has to be writable by anyone who might want to play the game.  Erm, that kindof sucks.So I wrote to the company, explaining what I'd found, and got this response:

The fact that the game couldn't write to the nwncdkey.ini file probably means that OS X is having some sort of file permissions issue on your Mac. You might want to boot to your OS X CD, run the disk utility, and run "Repair Disk" and "Repair Disk Permissions" on your hard drive.

Repeat after me: WTF?  Their software is installed by a Stuffit installer, not by Installer.app - this means there's no BOM, so there are no known permissions pertaining to that directory as far as the OS is concerned.  If they got the permissions wrong on the way in (not unlikely, a number of my user accounts on various systems have custom umasks, and I bet Stuffit doesn't check that) then Disk Utility isn't going to do anything to fix their mistake.  For completeness I ran a verify perms check, and indeed nothing came back (well yes something did, in a different area of the filesystem: I had myself chmodded an area of /Developer but last time I checked, Carbon games don't make heavy use of the developer documentation).  But the Cult Of Mac is strong, and its idols shall not be slighted, so running repair permissions is obviously the fix in this situation.  Perhaps I should zap the PRAM, start with extensions off, rebuild the Desktop and sacrifice me a goat while I'm at it.I'm going to write &quot;a polite response&quot; to this particular company's support desk, explaining how repair perms works, why perms errors (in the BOM-perms-don't-match-filesystem-perms sense) were not - could not have been -  the cause of my issue and requesting that they switch to an installer platform which, well, understands the Mac OS X filesystem.  And that they start writing user-specific data like savegames and so on into ~/Library/Application Support/.

#### The Requirements Trifecta
It's hard to argue that any one approach to, well, anything in software is better or worse than any others, because very few people are collecting any data and even fewer are reporting what they're trying. Worst is understanding how requirements are understood, prioritised, and implemented. Companies can be very opaque when it comes to deciding what they do and when they do it, even if you're on the inside.

This can be frustrating: if you can see what you think is the path between here and all the money, then it can be enervating to find the rest of the organisation is following a different path. Doubly so if you can't see, and aren't shown, any reason to follow that path.

What we do know is that the same things that work in small companies don't tend to work in large ones: search for any company name X in the phrase "why doesn't X listen to customer feedback" and you'll find multiple complaints. Customers get proxied, or aggregated, or weighed against potential customers. We feel like they aren't listening to us, and it's because they aren't. Not to us as individuals anyway. If a hundred thousand of us are experiencing kernel panics or two million of us have stopped engaging with feed ads, something will get done about it.

That said, there are some things I've seen at every scale that need to be in balance for an organisation to be pointing in the right direction. Three properties of how they work out what they're doing, that need to be in synergy or at least balanced against one another. I now ask about these three things when I meet with a new (to me) company, and use these as a sense check for how I think the engagement is going to work out.

Leadership Vision

One third is that the company has some internal driving force telling it which way to go and what challenges to try to solve. Without this, it will make do with quick wins and whatever looks easy, potentially even straying a long way from its original mission as it takes the cheapest step in whatever direction looks profitable. That can lead to dissatisfied employees, who joined the company to change the world and find they aren't working on what was promised.

On the other hand, too much focus on the vision can lead to not taking material reality into account. A company that goes bust because "our product was ten years ahead of its time, the customers weren't ready" is deluding itself: it went bust because the thing they wanted to sell was not something enough other people wanted to buy.

Market Feedback

One third is that the company has some external input telling it what customers are trying to do, what problems people have, what people are willing to pay for those problems to go away, and what competitors are doing to address those problems. Without this, the company will make things that people don't want to buy, lose out on sales opportunities where they don't describe what people have in a way that makes them want it, or will find themselves outcompeted and losing to alternative vendors.

On the other hand, too much focus on market feedback can lead to a permanently unsatisfying death march. Sales folks will be sure that they can close the next big deal if only they had this one other feature, and always be one other feature from closing that deal. Customers can always find something to be unhappy about if sufficiently prodded, so there will always be more to do.

Technical Reality

The third third is that the company has some internal feedback mechanism telling it what is feasible with the assets it already has, what the costs and risks (including opportunity costs) are of going in any direction, and what work it needs to do now to enable things that are planned in the future. Without this, forward progress collapses under the weight of what is erroneously called technical debt. Nothing can be done, because doing anything takes far too long.

On the other hand, too much focus on technical desiderata can lead to the permanent rewrite. Technical folks always have a great idea for how to clean things up, and even if all of their suggestions are good ones there comes a time when a lumberjack has to accept that the next action is not to replace their axe sharpeners but is to cut down a tree. Features are delayed until the Next Great Replatforming is complete, which never comes because the Subsequent Replatforming After That One gets greenly.

The Trifecta

I don't think it's particularly MBA material to say that "a company should have a clear leadership vision moderated by marketing reality and internal capability". But in software I see the three out of kilter often enough that I think it's worth writing the need down in the hope that some people read it.

#### The rokeg blood pie^W^W^Wplot thickens
So, having already discussed Klingon Anti-Virus, the under-research Klingon threat detection tool made available by Sophos, it seems that more information has been made available. From no less, or indeed more, of a source than the blog of my Clu-ful conym.This seems to confirm the impression that the tool has been developed for some special internal use and might not be downloadable much longer. It's hard to tell, though; most of the company is being very quiet about it (indeed it wasn't until today that much internal noise was generated about the tool at all).Of course, maybe I'm being duped. This could be some sort of company experiment to see, well, either how much free marketing they can get or who in the company is responsible for the press leaks. If it's the latter, then I need you all to take a look at my CV as I'll probably be relying on it - and you - soon ;-).Anyway, take a look at the tool if you're interested, I've had reports that it works well but still haven't heard much feedback about the quality of the translation. BTW, interested in a Mac version of the tool? I can't promise anything but leave a message after the beep and I'll forward requests...

#### The security apprentice
This originally appeared in a post at Sophos' Naked Security blog.

There have been two recent occasions on which my computing life has been influenced by Lord Sugar, the business mogul in charge of Amstrad and star of BBC One's reality show The Apprentice.

The first was on a visit to the National Museum of Computing at Bletchley Park, where I got to use a computer Amstrad produced back in the 1980s, that went by the catchy name "CPC 6128". It was running a Tetris clone called Blocks created—as was proudly proclaimed on the game screen—by an upstart programmer called "G Cluley".

The second was this weekend, when the device you see in the picture showed up in a charity shop. This is the E-mailer Plus, a sort of executive phone/internet thing released by Amstrad in 2002. Being a fan of old computers, especially oddball ones like the E-mailer, I bought it.

The key feature of this phone was that it also had e-mail and web capabilities, albeit delivered via a premium rate number that lined Lord Sugar's pockets with every mail check. Users could configure the phone to automatically fetch their mail to be read on the attached LCD screen.

And, indeed, someone had used this E-mailer for e-mail. Somone I shall call "Colin" had set up two accounts on the device. How do I know this? Because Colin hadn't deleted these accounts before taking his phone to the charity shop.

As I said, the E-mailer relies on a dial-up service which was discontinued earlier this year by its ultimate owners, BSkyB. That means that I couldn't, should I want to, fetch Colin's new e-mail messages. But there were messages already stored on the phone that I could have read. More surprisingly, the configuration screens let me see passwords assigned to Colin's accounts: has he used the same passwords on any other services?

Hopefully you're aware of the need to ensure there's no sensitive information stored on old computers before you dispose of them, particularly if you're going to sell them on to other users. My new (or should I say Colin's old) E-mailer shows that this goes for any device that stores or accesses your data, including phones both smart and retro.

I now imagine a scene in Lord Sugar's office. "Colin, you made a basic error. By failing to delete your accounts before giving away your phone, you put your e-mail messages and your passwords at risk. You compromised the privacy of your own and your company's data, and for that reason, you're fired."

#### The Silent Network
People say that the internet, or maybe specifically the web, holds the world’s information and makes it accessible. Maybe there was a time when that was true. But currently it’s not: probably not because the information is missing, but because the search engines think they know better than you what you want.

I recently had cause to look up an event that I know happened: at an early point in the iPod’s development, Steve Jobs disparaged MP3 players using NAND Flash storage. What were his exact words?

Jobs also disparaged the Adobe (formerly Macromedia) Flash Player media platform, in a widely-discussed blog post on his company website many years later. I knew that this would be a closely-connected story, so I crafted my search terms to exclude it.

Steve Jobs NAND Flash iPod. Steve Jobs Flash MP3 player. Steve Jobs NAND Flash -Adobe. Did any of these work? No, on multiple search engines. Having to try multiple search engines and getting the wrong results on all of them is 1990s-era web experience. All of these search terms return lists of “Thoughts on Flash” (the Adobe player), reports on that article, later news about Flash Player linking subsequent outcomes to that article, hot takes on why Jobs was wrong in that article, and so on. None of them show me what I asked for.

Eventually I decided to search the archives of one particular blog, which didn’t make the search engines prefer relevant results but which did reduce the quantity of irrelevant results. Finally, on the second page of articles from Daring Fireball about “Steve Jobs NAND flash storage iPod”, I found Flash Gordon. I still don’t have the quote, I have an article about a later development citing a dead link story that is itself interpreting the quote.

That’s the closest modern web searching tools would let me get.

#### The Software Leviathan
Thomas Hobbes viewed society as a meta-person, a gigantic creature whose parts were human and which was in the service of those humans. Left to their own devices, people would not work well together as their notion of individualism and search for personal gain leads directly to conflict: strong government is needed to instil a sense of cooperation and of social obligation. This idea of "government through social contract" is pervasive in Western political thought, being the basis as it is for the "government of the people, by the people, for the people" with which Abraham Lincoln hoped to lead post-civil war America.

Software systems themselves can also be thought of as Leviathans. From a purely technical sense, all of "professional" software construction is based on notions of composition, of software systems that are themselves made of software systems. So we have structured or procedural programming, with routines composed of subroutines. And functional programming, with functions composed of functions. And object-oriented programming, with objects composed of objects. So central are these ideas to expressions of thought in software that they are considered paradigmatic by many, representing fundamental world-views of the art/craft/science.

There's a second formulation of software-as-Leviathan, which is closer to Hobbesian meaning. The technical aspect of our software systems is merely a substrate[*] through which a social system—that of the people interacting with the software, the people acting on the software, and the people interacting with the other people—is reified. So the descriptions Hobbes made of his Leviathan can be made of these socio-technical systems:

First the Matter thereof, and the Artificer; both which is Man[sic].
Secondly, How, and by what Covenants it is made; what are the Rights and just Power or Authority of a Soveraigne; and what it is that Preserveth and Dissolveth it.
Thirdly, what is a Christian Common-Wealth.
Lastly, what is the Kingdome of Darkness.

[*] I wonder what form of substance gives the best sense of the analogy. Scaffolding? Lubricant? Mortar? Framework?

OK, maybe not so much the third one, except that it is really an attempt to define the values and norms of a society, which in the context of Hobbes's writing, meant a Christian society.

Of course, any attempt to describe such a system is going to be filtered by the preconceptions, ideas and values of the person creating the description. Which brings me onto today's topic: the pun in the new domain of this blog. Evidently it's a contraction of "Structure and Interpretation of Computer Programmers", based on the Abelson and Sussman book title. That book is abbreviated to SICP, so it's not too difficult to see how it might be adapted to SICPers.

We can also see it as being a Latin abbreviation: sic pers., meaning such a person. So there is both the Structure and Interpretation of Computer Programmers, and there is this person who is doing the interpreting, in the domain name.

#### The Tankard Brigade
I have a guideline that seems to apply to many pursuits and hobbies: any activity can be fun until there's too high a density of men with beards and tankards.

Of course, they aren't all men (though many are) and don't all have beards and tankards (though many do). But they can turn any enjoyable pastime into a maddeningly frustrating pursuit of ever-receding goals, like Zeno's arrow approaching but never reaching its target.

Some background. For any activity there will be different levels of engagement; different extents to which once can take it seriously. For most people (apart from, in this simplified model, two people) there will be a collection of people who are less invested than they are, and a collection of people who take the pursuit more seriously.

Often, this doesn't cause any disharmony. Some natural outgroup bias might make people believe that those who take it more seriously take themselves too seriously, and put too much effort into what should be an enjoyable way to spend one's time. Similarly, those who take it less seriously are perhaps not really engaged with the activity, and can be a bit too frivolous. Of course the distance between my level of engagement and perception of the outgroup's investment is decidedly non-linear: pro golfers and non-golfers do not cause as much difficulty for year-round and fair-weather amateurs as these two groups cause for each other.

This is all largely harmless snobbery and joshing until the men with beards and tankards come along. A man with a beard and a tankard (even if only figuratively a man with a figurative beard and tankard) is someone whose level of engagement with a pursuit must be the greatest of everyone in the room or online forum.

A single man with his solitary beard and tankard in a group can be harmless, endearing or slightly irritating. He will have bought the most expensive equipment available, and will gladly tell everyone who'll listen (and many who would rahter not). He'll explain why he got it, why it's better, and why you simply can't appreciate the subtleties and nuances of what you're participating in (and apparently enjoying very well, thankyou very much) without the basic investment in a good...whatever that thing is they bought. Moreover, there's only one way to engage in the activity, and that's the way in which he does it. Anyone who has a different way, particularly one that invovles spending less time or money, is looked on in smiling condescenscion as one who simply doesn't - and perhaps can't - appreciate the craft in all its majestic glory. It might involve some eye-rolling, tutting, and perhaps a strategic choice of seat at the society Christmas dinner, but you can usually cope with one man with his beard and tankard.

Two or more, on the other hand, start to get out of hand (this one or the other one), because each wants to be at the apotheosis of his craft, neither can afford to be outdone by the other. The effect is of course an inescapable ratchet of dedication and investment, much like the ever-accelerating and ultimately ruinous cycle of gift-giving in potlatch societies. When one comes in with some new piece of kit, the other must have it or the one better than it. When one adopts some new and laborious way of interacting with the pursuit at hand, the other will immediately adopt or surpass it. Their interactions with each other are best described as 'banter', that particularly masculine (and indeed beard-ridden and betankarded) species of chatter that seems on the surface to be friendly ribaldry but that covers a seething and complex web of mistrust and hatred.

As I said earlier, I think that what really counts for enjoyment of a pursuit is the density of men with beards and tankards. Some activities seem able to hold at once both people with large individual investments, and a welcoming attitude toward newcomers and casual participants. Many runners and cyclists, regardless of how much they spent on their kit, will be happy with and friendly toward anyone who turns up to the same event to run or ride alongside them. The next person may be on their super-list carbon fibre frame with $5000 wheels and bespoje saddly uniquely contoured to fit their cheeks, when you turn up on the $250 bike you got in the January sales. But they're a cyclist, and you're a cyclist, so hey, let's get some cycling going.

Some activities are clearly at the transition, where variations in the density of the tankard field can locally push it past the critical limit. Most motorcyclists are happy to acknowledge and welcome other bikers (though obviously not scooter riders), with a few notable exceptions. Harley riders tend to only notice other Harley riders. The extreme end of the amateur track circuit only pays attention to how much you've bored out your cylinders (and anyone who'll listen) and the amount of time you spend riding a dynamometer. And that certain class of BMW rider who watched The Long Way Down can't believe that you don't have mud pans, GPS, and aluminium flight cases attached to your bike when you go to the corner shop for a pint of semi-skimmed. But still, most bikers accept most other bikers, and talk to them about most biking.

And then there are the activities that are forever lost to the men with beards and tankards, for which the ratchet has turned so far that even if the barrier to entry is theoretically low, the barrier to sociable entry - to engaging with the community as an equal - can be insurmountable.

Consider astronomy. It used to be that if you had some cheap army surplus binoculars, you could go along to your local astronomy society and discuss what you'd seen of the moon, the planets and some of the brighter objects in the Messier catalogue. Then, with the introduction of the charge-coupled detector, the tankard brigade arrived in force. Now people will swap photos constructed from multiple hundreds of exposures through different narrow-band filters, taken with their large reflecting telescopes with computer-controlled star drives and the latest in CCDs (all probably permanently housed in purpose-built observatories in their gardens). No multi-thousand-dollar telescope (perhaps even no garden)? Nothing to discuss.

In some circles, folk music can have a more practice-driven ratchet system. The British folk revival of the 1970s brought with it literal men with actual beards and genuine tankards who defined what folk singing was (in apparent contradiction to the idea that it should be up to the folk to decide). Now there exist folk clubs where unless you have that certain nasal folkier-than-thou timbre in your voice and practiced wobbly delivery, and unless you can remember all of the words to all twelve verses without recourse to the book, you probably shouldn't take part.

All of this leads me to my questions. As a programmer, which of the practices I participate in are pragmatic, which necessary, and which informed by the ratchet of the men with beards and tankards? How much of what we do is determined by what others do, and must be seen to be done before we can claim we're doing it right?

And which things? Is it the runaway complexity of type systems that's the ratchet, or the insistence of programming without the safety net at all in a dynamic language? Or both?

My naive guess is that tankardism manifests where unnecessarily highfalutin words are deployed, like 'paradigm' for 'style' or 'methodology' for 'method'. And yes, that sentence was deliberately unnecessarily highfalutin.

#### The times, they mainly stay the same
bbum displays a graph of the market capitalization (he's american, so the z sticks) of a few of the computer companies, noting that if after-hours trading isn't too surprising, then tomorrow (for Americans, again) the market will open with Apple being the biggest computer manufacturer on the planet.  However, these figures fail to show something reasonably interesting.

What have IBM (up 24% y-o-y), HP (up 30 %) or Dell (up 21%) done to enamour you to their brand lately?  If you're anything like me, then they've done nothing at all.  Selling the same old Operating Systems on the same old hardware  doesn't count as innovative.  Compare them with Sun (up 13% year-on-year) or Apple (115%) and you'll see that there's basically no accounting for taste on the stock market.  While Apple have been selling the shiny gadgets, Sun have been delivering the most observable operating environment on the planet and Dell have been doing, well, shit-all would be a polite phrase, and yet Dell have outstripped Sun in growing their stock price.  In fact, HP have managed to blow up their stock price out of all proportion, while fighting scandals and the complete haemmorhaging of their management staff.

#### The trouble with layers
In describing Inside-Out Apps I expressed my distrust of the "everything is MVC" school of design.

  […]when you get overly attached to MVC, then you look at every class you create and ask the question “is this a model, a view, or a controller?”. Because this question makes no sense, the answer doesn’t either: anything that isn’t evidently data or evidently graphics gets put into the amorphous “controller” collection, which eventually sucks your entire codebase into its innards like a black hole collapsing under its own weight.

Let me extract a general rule: layers are bad. It's not just that I can't distinguish your app's architecture diagram from a sandwich's architecture diagram, or a trifle's. The problem is in the boxes.

As Possibly Alan, IDK and I discussed recently, the problem with box-and-arrow diagrams is that the boxes are really just collections of arrows zoomed out. Combine that with something my colleague Uri told me, that "a package in the dependency diagram is just the smallest thing that contains a cycle", and you end up with your layer cake diagram looking like this:

Three big wastelands of "anything goes", with some vague idea that the one at the top shouldn't talk to the one at the bottom. Or maybe it's that any of them can talk down as much as they like but never up. Either way it's not clear how any of the dependencies in this system are controlled. Is it "anything goes" within a layer? If two related classes belong in different layers, are they supposed to talk to each other or not? Can data types from one layer be passed to another without adaptation? Just what is the layer boundary?

#### The twitter sitter hit a bitter critter
Yup, more on the subject of a home-grown Twitter client.  This time, posting and sorting out the UI somewhat have both been achieved:Posting tweets is amazingly simple - just take the tweet and stuff it into the body of a POST NSURLRequest.  The Twitter API even handily returns the posted tweet, so the same code which parses the friends timeline can also insert the new tweet.So, where to go next?  Well, I'm getting bored of typing my password in all the time so Keychain would be nice, @reply buttons and perhaps searching.  I'm going to need cache management soon, too.

#### The unreasonable ineffectiveness of considering things harmful
Dijkstra didn't claim to consider the go to statement harmful, not in those words. The title of his letter to CACM was provided by the editor, Niklaus Wirth, who did such a great job that the entire industry knows that go to is "Considered Harmful", and that you can quickly rack up the clicks by considering other things harmful.

A deeper reading of his short (~1400 words) article raises some interesting points, that did not as yet receive as much airing. Here, in the interests of writing an even shorter letter, is just one.
My first remark is that, although the programmer's activity ends when he has constructed a correct program, the process taking place under control of his program is the true subject matter of his activity, for it is this process that has to accomplish the desired effect; it is this process that in its dynamic behavior has to satisfy the desired specifications. Yet, once the program has been made, the "making' of the corresponding process is delegated to the machine.
There are many difficulties with this statement, including the presumed gender of the programmer. Let us also consider the idea of a "correct" program, which does not exist for the majority of programmers. Eight years after Dijkstra's letter was published, Belady and Lehman published the first law of program evolution dynamics:
_ Law of continuing change_. A system that is used undergoes continuing change until it is judged more cost effective to freeze and recreate it. Software does not face the physical decay problems that hardware faces. But the power and logical flexibility of computing systems, the extending technology of computer applications, the ever-evolving hardware, and the pressures for the exploitation of new business opportunities all make demands. Manufacturers, therefore, encourage the continuous adaptation of programs to keep in step with increasing skill, insight, ambition, and opportunity. In addition to such external pressures for change, there is the constant need to repair system faults, whether they are errors that stem from faulty implementation or defects that relate to weaknesses in design or behavior. Thus a programming system undergoes continuous maintenance and development, driven by mutually stimulating changes in system capability and environmental usage. In fact, the evolution pattern of a large program is similar to that of any other complex system in that it stems from the  closed-loop cyclic adaptation of environment to system changes and vice versa.
This model of programming looks much more familiar to me when I reflect on my experience than the Dijkstra model. If Dijkstra's programmer stopped programming when they have "constructed a correct program", then their system would fail as it didn't adapt to "increasing skill, insight, ambition, and opportunity".

The programmer who would thrive in this environment is more akin to Ward Cunningham's opportunistic rewriter, based on his experience of the WyCash Portfolio Management System. That programmer rewrites every module they touch, to ensure that it represents the latest information they have. We recognise the genesis of Ward's "technical debt" concept in this quote, and also perhaps what we would now call "refactoring":
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

The traditional waterfall development cycle has endeavored to avoid programming catastrophe by working out a program in  detail before programming begins. We watch with some interest as the community attempts to apply these techniques to objects. However, using our debt analogy, we recognize this amounts to preserving the concept of payment up-front and in-full. The modularity offered by objects and the practice of consolidation make the alternative, incremental growth, both feasible and desirable in the competitive financial software market.
Ward also doesn't use go to statements, his programming environment doesn't supply them. But it is not the ability of his team to avoid incorrect programs by using other control structures that he finds valuable; rather the willingness of his programmers to jettison old code and evolve their system with its context.

#### The value of the things on the left
With the rise of critical writing like Bertand Meyer's Agile! The Good, the Hype, and the Ugly, Daniel Mezick's Agile-Industrial Complex, and my own Fragile Manifesto, it's easy to conclude the this Agile thing is getting tired. We're comfortable enough now with the values and principles of the manifesto that, even if software has exited the perennial crisis, we still have problems, we're willing to criticise our elders and betters rather than our own practices.

It's perhaps hard to see from this distance, but the manifesto for Agile Software Development was revolutionary when it was published. Not, perhaps, among the people who had been "doing it and helping others to do it".

Nor, indeed, would it have been seen as revolutionary to the people who were supposed to read it at the time. Of course we value working software over comprehensive documentation. Our three-stage signoff process for the functional specification before you even start writing any software is because we want working software. We need to control the software process so that non-working software doesn't get made. Yes, of course working software is the primary measure of progress. The fact that we don't know whether we have any working software until two thirds of the project duration is passed is just how good management works.

At one point, quite a few years after the manifesto was published and before everybody used the A-word to mean "the thing we do", I worked at a company with a very Roycean waterfall process. The senior engineering management came from a hardware engineering background, where that approach to project management was popular and successful (and maybe helpful, but I'm not a hardware engineer). To those managers, Agile was an invitation for the inmates to take over the asylum.

Developers are notoriously fickle and hard to manage, and you want them to create their own self-organising team? Sounds like anarchy! We understand that you want to release a working increment every two to four weeks with a preference toward the shorter duration, but doesn't that mean senior managers will spend their entire lives reviewing and signing off on functional specifications and test plans?

The managers who were open to new ideas were considering the Rational Unified Process, which by that time could be defined as Agile for the "nobody ever got fired for buying an IBM" crowd:

That software engineering department now has different management and is Agile. They have releases at least every month (they already released daily, though those releases were of minimal scope). They respond to change rather than follow a plan (they already did this, though through hefty "change control" procedures). They meet daily to discuss progress (they already did this).

But, importantly, they do the things they do because it helps them release software, not because it helps them hit project milestones. The revolution really did land there.

#### The Vexing Problems in Programming
I admit it, I've been on the internet for quite a while (I could tell you that my ICQ number is 95941970, but I haven't logged in for years) and my habits haven't changed. I still regularly get technology news from slashdot, and today was no exception. An interesting article was Here Be Dragons: The 7 most vexing problems in programming. Without wanting to spoil the article for you by giving away the punchline, there are indeed some frustratingly difficult problems mentioned: multithreading, security, encryption are among the list.

All of these problems are sideshows to what I see as one of the largest and most vexing issues in programming: the fundamental rule to business administration is that your income should be greater than your costs, but software makers still, by and large, don't have a way to compare the expected value of their work to the expected cost of the work.

The problem in space

Different software teams - and individuals - do work in different contexts, and in different ways. The lone wolf micro-ISV is not the same as an individual contract developer. The in-house IT team does not have the same problems to solve as the shrinkwrapped software vendor, and those developing web services for public consumption have yet another context. The team with core hours all working in a single office is different from the distributed team inhabiting multiple time zones.

How much of this variety is essential, and how much is accidental? How much of it is relevant, when considering some intervention, process change, or technique? Consultants speaking at conferences (another context with its own similarities and differences from the others) don't tend to talk about what researchers in fields such as psychology would recognise as "threats to validity" of their work, but given all of the ways in which software is made, we need to know whether some proposal applies to all of them, or to some of them, or whether it has been applied to some of them and might be applied to others, and what would assist or confound that application.

The problem in time

What are the accepted, tested and validated ways to identify who will be using and otherwise impacted by our software systems? To know whether they can use the system we propose, and whether it is the best system for their intended use? To ensure that our proposed software systems treat those people ethically? To understand the cost (to ourselves and to others) of constructing those systems? To deliver the systems to the people who will interact with them? To choose which people are or aren't entitled to access? To build a representation of the problem to be solved, to validate that representation, to validate the solution against that representation?

Where an answer exists to those questions, what are the contexts in which it is valid and what are the threats to its validity? How has that answer been compared with other possibilities? How has it been confirmed? How has it been challenged? How can I find out about those confirmations and challenges? How can I find out about any alternatives? What techniques exist to weigh up those alternatives quantitively, rather than relying merely on the persuasiveness of the conference speakers promoting those solutions (and, by the way, the books/screencasts that describe the solutions)?

The lack of a problem

Why should I care? There's enough money in software at the moment to mean that I don't need to be any good at knowing what works or doesn't work, I just need to get out there and sell some software. In the rare situation that I don't make my money back, that's just the market forces at work, and I can go and get a high-paying job somewhere while I lick my wounds, and pick another programming language/framework/platform/whatever it is that's going to make my next attempt definitely succeed.

Clearly, this bottomless pit of money that arises from society's unwavering faith in software and its ability to cure all ills is never going to run out. There's no need to worry ever about whether we're doing it right, because there'll always be someone out there willing to pay for us to do it wrong. Life as a programmer is like some kind of socialist utopia where whether we're making a valuable contribution or not, the rest of society is looking out for us.

That's going to last forever, right?

#### The view(s) from the hotel
Which two buildings are across the street from my current location?andanyone might think it had been deliberately chosen for comedy purposes…

#### The Vizzini Effect
A bunch of the topics I wanted to discuss all turned out to have a common basis, so I'm going to write the post about the commonality using a couple of examples from the specific topics for illumination. Maybe I'll come back to those topics in more depth later, each one is itself interesting and valuable.

The common thing is the Vizzini Effect, named after the Sicilian in the Princess Bride. In the movie/book, Vizzini often describes events as "inconceivable," to which Inigo Montoya replies "you keep using that word. I do not think it means what you think it means". The Vizzini Effect in software engineering (and undoubtedly in other fields of endeavour too, I doubt we're special) is when the same thing happens: a word or phrase seems to adopt a different meaning such that two different people, or two different groups of people, can mean it to use different things without either seeming malicious or disingenuous. In the examples I'm going to explore here, those groups are separated by time rather than space. But unlike with Vizzini, it's not that one person is using a word in a weird way, but that collectively software engineers seem to have decided it takes a different meaning.

Examples of Vizzini Phrases

Object-Oriented Programming

OOP is perhaps the ur-example here, and definitely the one with the most obvious dog-whistle. "I invented the term Object-Oriented Programming," says Alan Kay, "and I can tell you I did not have C++ in mind". To Alan and that early group of Smalltalk programmers at Xerox, ParcPlace, Tektronix etc., object-oriented programming was extreme late binding and decoupling through message sending. These days, it is often programming in any language that has a "class" keyword, or a straw man meaning any form of mutable state.

Agile

In the discussion do you think Agile/Scrum is beneficial for software delivery?, the first answer (at time of writing) says "The whole thing was designed to give non-technical people more power over the ones who spent a lifetime honing their craftmanship." The question asks about the surprising rituals and the extra layers of bureaucracy. That's the opposite impression than the one I have, where pre-existing software engineering methods tried to minimise or even automate away the programmer contributions. The lightweight methods, promoted by (among others) the agile alliance, sought to build projects around motivated individuals, giving the support they needed but leaving them alone to get the job done. The alliance members thought that the best architectures and designs were created by self-organising teams: a far cry from imposing methodologies to remove power from technical contributors.

Design Patterns

Design Patterns in software used to refer to the Christopher Alexander idea of identifying repeating problems in architecture and building a shared language that succinctly communicates understanding of the problem, solutions selected, and trade-offs in those solutions. These days it seems to mean any of the examples of design patterns in the Gang of Four book on early-OOP implementation patterns, and no others.

Free Software

The Free Software Foundation and the GNU project were created with the goal of extending desirable human rights and liberties to the world of computing. These days it seems to mean "open source, but said by a person who uses the word actually a lot".

Open Source

The Open Source Initiative was created to generalise the Debian Free Software Guidelines out from the Debian project to general business rules for software, based on the prior successful Open Systems movement and the liberties identified in the Free Software Movement. These days it pretty much means making the components needed to build SaaS subscription products available at zero cost.

Software Engineering

In 1967, software engineering was a provocative term, meant to imply that the art of creating software would be somewhat improved if it had a socio-scientific basis. These days software engineering is two words anyone who gets or wants to get paid for programming uses on their CV/résumé.

What happened?

My impression is that three things changed, and that two of them are almost the same. The origins of all of these phrases are in particular times in history, made by particular people, talking in specific contexts. Time has changed, which has changed the context (or at least the relevance of the original context), the people who said the things have changed, and so many new people have entered the field that a majority of practitioners no longer know who the original people were, nor have experienced the context in which they spoke.

It's entirely possible, for example, that the agile methodologies which were lightweight reactions to software engineering around 2000 are oppressively bureaucratic in 2021. We expect to be able to release software multiple times per day now, using analytics and telemetry to understand in real time how it's being used. The agile folks wanted us to release up to every two weeks and to talk to someone before doing it, ugh!

Some people talk about an Agile-Indu$trial Complex, suggesting that there's some shady cabal of consultants and certification bodies conspiring to make us all agile so they can profit from it. Again, maybe true. Others talk of companies who "talk the talk without walking the walk": they got the consultant in, decided which parts of this whole Agile thing sounded nice or relevant, and adopted those things then trumpeted their "fully agile workflow" on their websitesfax banners.

And, of course, there's the telephone game. Even those of us who heard about it from the horse's mouth—maybe worked on an XP team, or read "Free Software, Free Society"—will have learned a slightly different thing than what the originators were trying to teach us, or thought they were teaching us. When telling others, we'll have misremembered, and adapted, and extemporised. And so will the people who learned from us, and so on.

The telephone game is subject not only to slow evolution, but to a Byzantine Generals attack. If someone wants, for example, OOP to die so that their preferred paradigm get used instead, they can inject a false message into the call graph. This is where Vizzini meets Lewis Carroll's Humpty-Dumpty: I keep using this word, and I do not think it means what you think it means.

Take into account the fact that most people who work in software now didn't work in software five years ago, and that this was true five years ago and so on, and you realise that the vast majority of people will have learned about any "classic" idea in software from a telephone conversation.

What to do?

Well, the first question to answer is, does anything particularly need to be done? Maybe these are ideas that have had their time, and can just fizzle out. But evidently for all of the examples above enough people want the ideas to continue that they (well, we obviously) keep trying to dredge the original discussions out of the history books and put them back into contemporary discourse. To do this, they need recontextualising. For example, nobody cares that Richard Stallman couldn't get a printer driver in the 1970s, but maybe they do care that there are things that they aren't allowed to do with the smartphones they think they paid for. That's how ideas of software freedom could be reintroduced.

Maybe the original phrase has become toxic and needs to be retired, without the original meaning being lost too. That is, whether you like it or not, the reason that "Open Source" was created as a term: to remove deliberate and accidental confusion over the word "freedom" in a business context, and to provide familiarity to people who had already adopted Open Systems ideas. It's why Devops exists: to tell the stories of Agile again, but to those who didn't listen the first time, or who listened and heard the wrong thing.

The telephone game can't be avoided. You have to keep telling the stories if you want new people to hear them, and that means accepting alterations in their re-telling. And you need there to be more than one raconteur, even if they're telling slightly different versions of the story. Don't count messiahs, count prophets. Only don't count prophets, count gospels. Only don't count gospels, count churches. Only don't count churches, count preachers.

You're never going to get every programmer or software professional in the world to agree with your interpretation of some phrase. But you can use contextually-relevant stories to tell people things that might help them make better software, and you can follow up "I do not think that means what you think it means" with a conversation in which you both learn something. Maybe it's your understanding that's wrong?

Inconceivable!

P.S.

I do a lot of writing, podcasting, presenting, and streaming about how to make software. Most of it has been free, still is free, and I don't intend to change that. It'd be great if you are able and willing, for you to support that free work by becoming a patron. No obligation!

#### The Wealth of Applications
Adam Smith's Inquiry into the Nature and Causes of the Wealth of Nations opens by discussing the division of labour. How people are able to get more done when they each pick a small part of the work to be done and focus on that, trading their results with others to gain access to the common wealth. He goes on to explain that while what people are really trading is labour, it's hard to think about that so the more comprehensible stand-in of money is used instead.

Smith's example is a pin factory, where one person might draw out metal into wire, another cut it to length, a third sharpen one end, a fourth flatten the opposite end. He says that there is a

  great increase in the quantity of work, which, in consequence of the division of labour, the same number of people are capable of performing

Great, so dividing labour must be a good thing, right? That's why a totally post-Smith industry like producing software has such specialisations as:

full-stack developer

Oh, this argument isn't going the way I want. I was kindof hoping to show that software development, as much a product of Western economic systems as one could expect to find, was consistent with Western economic thinking on the division of labour. Instead, it looks like generalists are prized.

[Aside: there really are a few limited areas of specialisation, but then generalists co-opt their words to gain by association a share of whatever cachet is associated with the specialists. For example, did you know that writing UNIX applications makes you an embedded programmer? Another specialisation is kernel programming, but it's easy to find sentiment that it's just programming too.]

Maybe the conditions for division of labour just haven't arisen in programming. Returning to Smith, we learn that such division "is owing to three different circumstances":

the increase of dexterity in every particular [worker]

This seems to be true for programmers. There is an idea that programmers are not fungible resources, and that the particular skills and experiences of any individual programmer can be more or less relevant to the problems your team is trying to solve. If all programmers can truly be generalists, then programmers would basically be interchangeable (though some may be faster, or produce fewer defects, than others).

Indeed Harlan Mills proposed that software teams be built around the idea that different people are suited to different roles. The Chief Programmer Team unsurprisingly included a chief programmer, analogous to a surgeon in a medical team. As needed other specialists including a librarian, technical writers, testers, and "language lawyers" (those with expertise in a particular programming language) could be added to support the chief programmer and perform tasks related to their specialities.

the saving of time which is commonly lost in passing from one species of work to another

Programmers believe in this, too. We call it, after DeMarco and Lister, flow.

Which leaves only one place in which to look for a difficulty in applying the division of labour, so let's try that one.

to the invention of a great number of machines which facilitate and abridge labour

So we could probably divide up the labour of computing if only we could invent machines that could do the computing for us.

Hold on. Isn't inventing machines what we do? Yet they don't seem to be "facilitating and abridging labour", at least not for us. Why is that?

At the technical level, there are two things holding back the tools we create:

they're not always very good. Building your software on top of a tool as a way to save labour is perceived as a lottery: the price is the labour involved in working with or around the tool, and the pay-off is the hope that this is less work than not using the tool.
they only really turn computers into other computers, and as such aren't as great a benefit as one might expect. We're trying to get to "the computer you have can solve the problem you have", and we only really have tooling for "the computer you have can be treated the same way as any other computer". We're still trying to get to "any computer (including the one you have) can solve the problem you have".

This last point demonstrates that the criterion of facilitating and abridging labour is partially solved. The goal of moving closer to the end state, where our automated machines help with solving actual problems, seems to have come up a few times (mostly around the 1980s) with no lasting impact: in object-oriented programming, artificial intelligence and computer-aided software engineering; specifically Upper CASE.

Why not close that gap, particularly if there is (or at least has been) both academic and research interest in doing so? We have to leave the technology behind now, and look at the economics of software production. Let's say that computing technology is in the middle of a revolution analogous to the industrial revolution (partly because Brad Cox already went there so I don't have to overthink the analogy).

The interesting question is: when will we leave the revolution? The industrial revolution was over when mechanisation was no longer the new, exciting, transformative thing. When the change was over, the revolution was over. It was over when people accepted that machines existed, and factored them into the costs of doing their businesses like staff and materials. When you could no longer count on making a machine as a profitable endeavour in itself: it had to do something. It had to pay for itself.

During the industrial revolution, you could get venture capital just for building a cool-looking machine. You could make money from a machine just by showing it off to people. The bear didn't have to dance well: it was sufficient that it could dance at all.

We're at this stage now, in the computer revolution. You can get venture capital just for building a cool-looking website. Finding customers who need that thing can come later, if at all. So we haven't exited the revolution, and won't while it's still possible to get money just for building the thing with no customers.

How is that situation supported? In Smith's world, the transparent relationship between supply and demand drove the efficiency requirements and thus the division of labour and the creation of machines. You know how many pins you need, and you know that your profits will be greater if you can buy cheaper pins. The pin factory owners know how many employees they have and what wages they need, and know that if they can increase the number of pins made per person-time, they can sell pins for less money and make more profit.

How do supply and demand work in software? Supply exists in bucketloads. If you want software written, you can always find some student who wants to improve their skills or even a professional programmer willing to augment their GitHub profile.

[This leads to another deviation from Adam Smith's world, but one that can be left for later: he writes

  In the advanced state of society, therefore, they are all very poor people who follow as a trade, what other people pursue as a pastime.

This is evidently not true in software, where many people will work for free and others are very well-paid.]

There's plenty of supply, but the demand question is trickier to answer. In the age of supply, not demand Aurel Kleinerman (via Bob Cringely) suggests that supply is driving demand. That people (this seemingly limitless pool of software labour) are building things, then showing them to people and seeing whether they can find some people who want those things. I think this is true, but also over-simplified.

It's not that there's no demand, it's that the demand is confused. People don't know what could be demanded, and they don't know what we'll give them and whether it'll meet their demand, and they don't know even if it does whether it'll be better or not. This comic strip demonstrates this situation, but tries to support the unreasonable position that the customer is at fault over this.

Just as using a library is a gamble for developers, so is paying for software a gamble for customers. You are hoping that paying for someone to think about the software will cost you less over some amount of time than paying someone to think about the problem that the software is supposed to solve.

But how much thinking is enough? You can't buy software by the bushel or hogshead. You can buy machines by the ton, but they're not valued by weight; they're valued by what they do for you. So, let's think about that. Where is the value of software? How do I prove that thinking about this is cheaper, or more efficient, than thinking about that? What is efficient thinking, anyway?

Could it be that knowledge work just isn't amenable to the division of labour? Clearly not: I am not a lawyer. A lawyer who is a lawyer may not be a property lawyer. A lawyer who is a property lawyer may only know UK property law. A lawyer who is a UK property lawyer might only know laws applicable to the sale of private dwellings. And so on: knowledge work certainly is divisible.

Could it be that there's no drive for increased efficiency? Maybe so. In fact, that seems to be how the consumer economy works: invent things that people want so that they need money so that they need to work so that there are jobs in making the things that people want. If it got too easy to do middle-class work, then there'd be less for the middle class to do, which would mean less middle class spending, which would mean less demand for consumer goods, which would… Perhaps knowledge work needs to be inefficient to support its own economy.

If that's true, will there ever be a drive toward division of labour in the field of software? Maybe, when the revolution is over and the interesting things to create for their own sake lie elsewhere. When computers are no longer novel technology, but are simply a substrate of industry. When the costs and benefits are sufficiently well-understood that business owners can know what they need, when it's done, and how much it should cost.

That'll probably be preceded by another software-led recession, as VCs realise that the startups they're funding are no longer interesting for their own sake and move on to the (initially much smaller) nascent field of the next revolution. Along with that will come the changes in regulation, liability and insurance as businesses accept that software should "just work" and policy adjusts to support that. With stability comes the drive to reduce costs and increase quality and rate of output, and with that comes the division of labour predicted by Adam Smith.

#### The whole 'rockstar developer' thing is backwards
Another day, another clearout of junk from people who want 'rockstar iPhone developers' for their Shoreditch startups. I could just say "no", or I could launch into a detailed discussion of the problems in this picture.

Rockstars are stagnant

No-one, and I mean no-one, wants to listen to your latest album. They want you to play Free Bird, or Jessica, or Smoke on the Water. OK, so they'll pay more for their tickets than people listening to novel indie acts, you'll make more money from them (after your promoter has taken their 30%). But you had better use exactly the right amount of sustain in that long note in Parisienne Walkways, just like you did back in '79, or there'll be trouble. Your audience doesn't care whether you've incorporated new styles or interesting techniques from other players, or bought new equipment, you're playing Apache on that pink Stratocaster the way you always have.

That's exactly the opposite of a good model in software. Solving the same problem over and over, using the same tools and techniques, is ossification. It's redundant. No-one needs it any more. Your audience are more like New York jazz fans than VH-1 viewers: they want tradition with a twist. Yes, it needs to be recognisable that you're solving a problem they have - that you're riffing on a standard. But if you're not solving new problems, you're no longer down with the cool cats. As the rock stars might say: who wants yesterday's papers?

Home taping is killing music

That riff you like to throw out every night, that same problem that needs solving over and over again? Some student just solved the same thing, and they put it on github. The change in code-sharing discourse of the late 1990s - from "Free Software" to "Open Source" - brought with it the ability for other people to take that solution and incorporate it into their own work with few obligations. So now everyone has a solution to that problem, and is allowed to sell it to everyone who has the problem. Tomorrow night, your stadium's going to have plenty of empty seats.

Programming groupie culture

Programming has a very small number of big names: not many people would be as well-known in the industry as, say, Linus Torvalds, Richard Stallman, DHH. Some people might choose to call these people "polarising". Others might choose "rude and arrogant". Either way, they seem to bring their harems of groupies to the internet: cadres of similarly-"polarising" males who want to be seen to act in the same way as their heroes.

A primatologist might make the case that they are imitating the alpha male baboon in order to gain recognition as the highest-status beta.

Now the groupies have moved the goalposts for success from solving new problems to being rude about solutions that weren't solved by the "in" group. What, you want to patch our software to fix a bug? You're not from round these parts, are you?

Embrace the boffin

Somehow for the last few years I managed to hang on to the job title "Security Boffin". Many people ask what a boffin is: the word was World War 2 slang among the British armed forces referring to the scientists working on the war effort. Like "nerd" or "geek", it meant someone who was clever but perhaps a bit, well, different.

Boffins were also known at the time as "the back room boys"[*] for their tendency to stay out of the way and solve important - and expedient - technical problems. We need these messages decrypting, the boffins in the back room have done it but they keep talking about this "computer" thing they built. Those boffins have come up with a way to spot planes before we can even see them.

The rockstar revels in former glories while their fans insist that nothing made later even comes close to the classics. If you need a problem solving, look for boffins, not Bonos.

[*] Unfortunately in the military establishment of the 1940s it was assumed that the clever problem solvers were all boys. In fact histories of early computing in the States show that the majority-female teams who actually programmed and operated the wartime computers often knew more about the machines' behaviours than did the back room boys, diagnosing and fixing problems without reporting them. A certain Grace Hopper, PhD, invented the compiler while the back room boys were sure computers couldn't be used for that.

#### The worst phrase in software marketing
"Rewritten from the ground up".

Please. Your old version mostly worked, except for those few corner cases that I'd learned how to work around. Now I don't know whether the stuff that did work does work now, and I don't know that I'll find that stuff in the same place any more.

There's a reason that old, crufty code in the core of your application was old and crufty. It is old because it works, well enough to pay the programmers who work on maintaining it and building the new things. It is crufty because the problem you're trying to solve was not perfectly understood, is still not perfectly understood, and is evolving.

Your old, crufty code contains everything you've learned about your problem and your customers. And now you're telling me that you've thrown it away.

#### There is no "us" in team
I’ve talked before about the non-team team dynamic that is “one person per task”. Where the management and engineers collude to push the organisation beyond a sustainable pace by making sure that at all times, each individual is kept busy and collaboration is minimised.

I talked about the deleterious effect on collaboration, particularly that code review becomes a burden resolved with a quick “LGTM”. People quickly develop specialisations and fiefdoms: oh there’s a CUDA story, better give it to Yevgeny as he worked on the last CUDA story.

The organisation quickly adapts to this balkanisation and optimises for it. Is there a CUDA story in the next sprint? We need something for Yevgeny to do. This is Conway’s Corrolary: the most efficient way to develop software is when the structure matches the org chart.

Basically all forms of collaboration become a slog when there’s no “us” in team. Unfortunately, the contradiction at the heart of this 19th century approach to division of labour is that, when applied to knowledge work, the value to each participant of being in meetings is minimised, while the necessity for each participant to be in a meeting is maximised.

The value is minimised because each person has their personal task within their personal fiefdom to work on. Attending a meeting takes away from the individual productivity that the process is optimising for. Additionally, it increases the likelihood that the meeting content will be mostly irrelevant: why should I want to discuss backend work when Sophie takes all the backend tasks?

The meetings are necessary, though, because nobody owns the whole widget. No-one can see the impact of any workflow change, or dependency adoption, or clean up task, because nobody understands more than a 1/N part of the whole system. Every little thing needs to be run by Sophie and Yevgeny and all the others because no-one is in a position to make a decision without their input.

This might sound radically democratic, and not the sort of thing you’d expect from a business: nobody can make a decision without consulting all the workers! Power to the people! In fact it’s just entirely progress-destroying: nobody can make a decision at all until they’ve got every single person on board, and that’s so much work that a lot of decisions will be defaulted. Nothing changes.

And there’s no way within this paradigm to avoid that. Have fewer meetings, and each individual is happier because they get to maximise progress time spent on their individual tasks. But the work will eventually grind to a halt, as the architecture reflects N different opinions, and the N! different interfaces (which have each fallen into the unowned gaps between the individual contributors) become harder to work with.

Have more meetings, and people will grumble that there are too many meetings. And that Piotr is trying to land-grab from other fiefdoms by pushing for decisions that cross into Sophie’s domain.

The answer is to reconstitute the team - preferably along self-organising principles - into a cybernetic organism that makes use of its constituent individuals as they can best be applied, but in pursuit of the team’s goals, not N individual goals. This means radical democracy for some issues, (agreed) tyranny for others, and collective ignorance of yet others.

It means in some cases giving anyone the autonomy to make some choices, but giving someone with more expertise the autonomy to override those choices. In some cases, all decisions get made locally, in others, they must be run past an agreed arbiter. In some cases, having one task per team, or even no tasks per team if the team needs to do something more important before it can take on another task.

#### There is no browser, only Zuul
My short-lived first plan for a career was in Physics. That's what my first degree was in, but I graduated with the career goal "do something that isn't a D.Phil. in Physics" in mind. I'd got on quite well with computers as a hobbyist, and the computing and electronics practicals in my course labs. A job as systems administrator for those very systems (a hotchpotch of NeXTSTEP, Solaris, OpenBSD, and Mac OS X) came up at the time that I graduated so I applied, got it, and became a computerer.

Along the way, I met people who thought that some people should not be computerers because their backgrounds were not exactly identical. The Google manager who could not believe that as an applicant for the lowest-grade QA role, I had not encountered the travelling salesman problem. The software engineer at Facebook who was incensed that someone applying to build a web application in PHP and Javascript did not know that there are eight bits in a byte (never mind that there aren't, necessarily, eight bits in a byte). And now the random on Twitter who insists that people who don't fully know browsers aren't allowed to write web applications.

It's easy to forget two things: the first is that at some point in the past, you didn't know what you know now, but you learnt it because you were allowed to participate and were taught. Even if you think you were a self-learner, you had access to playground materials, books, tutorials, online documentation...and someone made those for you and allowed you to use them.

The second is what it was like not to know those things. I remember not understanding how OOP was anything more than putting dots in the name of your function, but now I understand it differently, and don't know what the thing was that changed.

The second of these things shows how easy it is to be the gatekeeper. If you don't remember what not understanding something was like, then maybe it came naturally, and if it isn't coming naturally to someone else well maybe they just don't get it. But the first shows that you didn't get it, and yet here you are.

When faced with a gatekeeper, I usually react flippantly. Because of my Physics background, I explain, I know how quantum physics works, and how that enables semiconductors, and how to build a semiconductor transistor, then a NAND gate, then a processor, and basically what I'm saying is I don't see how anyone who doesn't know that stuff can claim to know computers at all.

But what I say to everyone else is "this stuff is really interesting, let me show it to you".

Be the keymaster, not the gatekeeper.

#### There's more to it
We saw in Apple's latest media event a lot of focus on privacy. They run machine learning inferences locally so they can avoid uploading photos to the cloud (though Photo Stream means they'll get there sooner or later anyway). My Twitter stream frequently features adverts from Apple, saying "we don't sell your data".

Of course, none of the companies that Apple are having a dig at "sell your data", either. That's an old-world way of understanding advertising, when unscrupulous magazine publishers may have sold their mailing lists to bulk mail senders.

These days, it's more like the postal service says "we know which people we deliver National Geographic to, so give us your bulk mail and we'll make sure it gets to the best people". Only in addition to National Geographic, they're looking at kids' comics, past due demands, royalty cheques, postcards from holiday destinations, and of course photos back from the developers.

To truly break the surveillance capitalism economy and give me control of my data, Apple can't merely give me a private phone. But that is all they can do, hence the focus.

Going back to the analogy of postal advertising, Apple offer a secure PO Box service where nobody knows what mail I've got. But the surveillance-industrial complex still knows what mail they deliver to that box, and what mail gets picked up from there. To go full thermonuclear war, as promised, we would need to get applications (including web apps) onto privacy-supporting backend platforms.

But Apple stopped selling Xserve, Mac Mini Server, and Mac Pro Server years ago. Mojave Server no longer contains: well, frankly, it no longer contains the server bits. And because they don't have a server solution, they can't tell you how to do your server solution. They can't say "don't use Google cloud, it means you're giving your customers' data to the surveillance-industrial complex", because that's anticompetitive.

At the Labrary, I run my own Nextcloud for file sharing, contacts, calendars, tasks etc. I host code on my own gitlab. I run my own mail service. That's all work that other companies wouldn't take on, expertise that's not core to my business. But it does mean I know where all company-related data is, and that it's not being shared with the surveillance-industrial complex. Not by me, anyway.

There's more to Apple's thermonuclear war on the surveillance-industrial complex than selling privacy-supporting edge devices. That small part of the overall problem supports a trillion-dollar company.

It seems like there's a lot that could be interesting in the gap.

#### There's No Such Thing as a General-purpose Processor
http://queue.acm.org/detail.cfm?id=2687011

There's no such thing as a General-purpose processor, and belief in such a device is harmful.

#### These things are hard
Mike Lee recently wrote about his feelings on seeing those classic pictures from the American space program, in which the earth appears as a small blue marble set against the backdrop of space. His concluding paragraph:

Life has its waves. There are ups and downs. My not insubstantial gut and my lucky stars both are telling me 2012 is going to be an upswell. Let us do as we do where I grew up and catch that wave. Put aside your fear and cynicism. The future is ours to create. The system is ours to debug and refactor.

For me, the most defining and inspiring moment of the whole space program predates the Space Shuttle, the moon landing, even the Gemini program. It is the words of a politician, driving his country to one of its highest ebbs of innovation and discovery. The whole speech is worth watching, but this sentence encapsulates the sentiment perfectly.

We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.

So let us create the future. It is not going to be easy, but we shall choose to do it because it is hard.

#### Things I believe
The task of producing software is one of choosing and creating constraints, rules and abstractions inside a system which provides very few a priori. Typically we select a large collection of pre-existing constraints, rules and abstractions upon which to base our own: models of computation, programming and deployment environments, everything from the size of the register file to the way in which text is represented on the display is theoretically up for grabs, but we impose limitations in their freedom upon ourselves when we create a new product.

None of these limitations is essential. Many are conventional, and have become so embedded in the cultural practice of making software that it would be expensive or impractical to choose alternative options. Still others have so much rhetoric surrounding them that the emotional cost of change is too great to bear.

So what are these restrictions? Here's a list of mine. I accept that they don't all apply to you. I accept that many of them have alternatives. Indeed I believe that all of them have alternatives, and that enumerating them is the first thing that lets me treat them as assumptions to be challenged.

Computers use the same memory for programs and data. I know the alternatives exists but wouldn't know how to start using them.
Memory is a big blob of uniform storage. Like above, except I know this one isn't true but that I just ignore that detail.
Memory and bus wires can be in one of two states.
There probably is a free-form hierarchical database available.
There is a thing called a stack and a thing called a heap, and the difference between the two is important.
There is no point trying to do a better job at multiprocessing than the operating system.
There is an operating system.
The operating system, file system, indeed any first system on which my thing is a second system; those first systems are basically interchangeable.
I can buy a faster thing (except in mobile, where I can't).
Whatever processor you gave me behaves correctly.
Whatever compiler you gave me behaves correctly.
Whatever library you gave me probably behaves correctly.
Text is a poor way to represent a computer program but is the best we have.
The way to write a computer program is to tell the computer what to do.
The goal of the industry for last few decades has been the DynaBook.
I still do not need a degree in computer science.
I should know what my software does before I give it to the people who need to use it.
The universal runtime environment is the C system.
Processors today are basically like faster versions of the MC68000.
Platform vendors no longer see lock-in as a goal, but do see it as a convenient side-effect.
You will look after drawing pictures, playing videos, and making sounds for me.
Types are optional.

#### This is fine
The BBC micro:bit is a tool for introducing young people to programming. It's a little embedded computer with a few inputs and a matrix of LEDs for output, as well as some control lines. In principle it's quite easy to use, I made a 1d6 simulator:

from microbit import *
from random import randint

class Die:
    ONE = Image("00000:"
                "00000:"
                "00900:"
                "00000:"
                "00000")
    TWO = Image("00000:"
                "09000:"
                "00000:"
                "00090:"
                "00000")
    THREE = Image("00000:"
                  "09000:"
                  "00900:"
                  "00090:"
                  "00000")
    FOUR = Image("00000:"
                 "09090:"
                 "00000:"
                 "09090:"
                 "00000")
    FIVE = Image("00000:"
                 "09090:"
                 "00900:"
                 "09090:"
                 "00000")
    SIX = Image("00000:"
                "09090:"
                "09090:"
                "09090:"
                "00000")
    ALL = [ONE, TWO, THREE, FOUR, FIVE, SIX]
    @classmethod
    def throw(self):
        return self.ALL[randint(0,5)]

display.show(Die.ALL, delay=100, wait=False, loop=True)

while True:
    if button_a.is_pressed():
        display.show(Die.throw())
    elif button_b.is_pressed():
        display.show(Die.ALL, delay=100, wait=False, loop=True)

Simple (I mean, simple if you know what the word randint means (hint: it means "random integer" as long as you know what an integer is), that programming languages that aren't Fortran call the first element of an array element zero, even though it's obviously ONE as far as a die is concerned, and you're OK with the word classmethod).

Here it is in action: woo! I AER PROGRAMMER!!!

https://www.sicpers.info/wp-content/uploads/2016/12/VID-20161222-WA0004.mp4

And here's the litany of things I did to get there:

I opened the "details.txt" file on the micro:bit, which tells me a load of build numbers and flash dates. I don't know what I'm supposed to do with that. This is hard.
I opened the "microbit.html" file on the micro:bit, which redirects me to microbit.co.uk, which tells me that as part of the BBC's restructuring of its online content, I will be redirected to microbit.org. I decided this was probably OK, and ended up looking at the same site but with a different URL (whatever one of those is).
I clicked on the introductory video and was told that I need an Adobe Flash Player, whatever one of those is.
I eventually found enough buttons to find a website that is a python editor that lets me write code for my micro:bit, and a tutorial for writing Python for the micro:bit. I know what a python is, but do not know why I would want a snake for my micro:bit.
The tutorial tells me to use a thing called a "mu", which is apparently not the website python editor but is another python editor. I do not know what a mu is, but I must download it.
There are three different download buttons, depending on whether I have a Windows, a Mac, or a Linux, whatever they are.
If I have a Windows, then I to enable the REPL (whatever that is) I must download a serial driver (whatever that is).
The driver site tells me that if I have a Windows 10 (whatever that is), then I should not download the serial driver.
If I have a Mac, then the first time (but maybe not other times, that's not clear) I open the mu I have to right click it.
If I have a Linux, then I must chmod u+x the mu and make sure my user (isn't that me?) is in the dialout (whatever that is) group (whatever that is).
Now I can write my python. To put it on the micro:bit, I must "flash" it. I guess that is because I have an Adobe Flash Player from earlier.
The micro:bit shows me helpful messages when I get something wrong. It scrolls the message "id=41 SyntaxError: invalid syntax" when I flash my python. I do not know what this means, but I was told that it's helpful. Therefore I am stupid.

I am not singling out the BBC, the micro:bit makers, the mu creators, Microsoft, or any of the other individuals or organisations involved with creating this easy-to-use educational environment. This is fine. This is how computers work, nay this is how computers work when we are making it easy to work them. This is our industry.

To quote Freddie Mercury: is this the world we created? We made it on our own. Is this the world we devastated, right to the bone? If there's a God in the sky looking down what can he think of what we've done to the world that we created?

#### This means business
This is the design of the business card I'll be taking to WWDC.  Let's look at some notable features.

Photo in top left.  I don't know about you, but I find it much easier to remember what someone looks like than who they are - this card is available so that people can combine the two.Plenty of blank space.  The back is also entirely empty and can be written on.  When people exchange business cards there'll usually be some context, it can help you remember what that is if you write it down.  For instance, I've got cards from previous WWDCs with handwritten notes like "webobjects employer", "bindings", "parallels openstep" and "huge hat" (bonus points for identifying all four).Questionable source code snippit.  This is a deliberate ploy to annoy fellow-developers with our compulsive attention to detail, thus helping to cement the meeting in their mind as well as providing an inoffensive conversation starter.Minimal contextual detail.  I'm pretty sure I won't get through all 250 cards in one sitting, so they ought to remain relevant for as long as possible.

#### This so goes on my CV
Today's sysadmin skills involved rescuing some ALGOL code from an 8" floppy, accessed from CP/M on an RM380Z:Amazingly although the disk hadn't been accessed in over a decade (though was stored in a Cool Dry Place^{TM}), everything worked first time.  The little jobby with a keyboard and built-in screen on the left is a Cambridge Z88, being used as an emergency VT52 terminal in this case.

#### This year's WWDC t-shirt?
"'ad you not nailed it to iTunes, it'd be pushin' up the daisies!"

#### Thoughts on Tech Conferences
This post is being, um, posted from the venue for GOTO Copenhagen 2012. It's the end result of a few months of reflection on what I get out of conferences, what I want to get out of conferences, what I put into (and want to put into) conferences and the position of tech conferences in our industry. I've also been discussing things a lot with my friends and peers; I've tried to attribute specific quotes where I remember who said them but let it be known that many people have contributed to the paragraphs below in many different ways. I'll make it clear at the outset that I'm talking about my experience at independent commercial and non-profit tech conferences, not scientific conferences (of which I have little experience) or first-party events like WWDC (which are straightforward marketing exercises).

Conference speakers

My favourite quote on this subject is courtesy of Mike; I remember him saying it in his MDevcon keynote this year but I'm also fairly sure he's said the same thing earlier:

  The talks at a conference are only there so that you can claim the ticket cost as an expense.

We're in a knowledge economy; but knowledge itself is not of any value unless it's applied. That means it's not the people who tell other people what's going on who're are doing the most important work; that's being done by the people who take this raw knowledge, synthesise it into a weltanschauung - a model of how the world works - and then make things according to that model. Using an analogy with the economy of physical things, when we think of the sculpture of David in Florence we think of Michelangelo, the sculptor, not of the quarry workers who extracted the marble from the ground. Yes their work was important and the sculpture wouldn't exist without the rock, but the most important and valuable contribution comes from the sculptor. So it is in the software world. Speakers are the quarry workers; the marble hewers, providing chunks of rough knowledge-stuff to the real artisans - the delegates - who select, combine and discard such knowledge-stuff to create the valuable sculptures: the applications.

Conference speakers who believe that the value structure is the other way around are deluding themselves. Your talk is put on at the conference to let people count the conference as a work expense, and to inspire further discussion and research among the delegates on the topic you're talking about. It's not there so that you can promote your consultancy/book/product, or produce tweet worthy quotes, or show off how clever you are. Those things run the gamut from "fringe benefits" to "deleterious side effects".

As an aside, the first time I presented at a Voices That Matter conference I was worried due to the name; it sounds like the thing that matters at this conference is the speakers' voices. In fact I suspect there is some of that as many of the presenters have books published by the conference hosts, but it's a pretty good conference covering a diverse range of topics, with plenty of opportunities to talk to fellow delegates. And IIRC all attendees got an "I am one of the voices that matter" sticker. Anyway, back to the topic at hand: thus do we discover a problem. Producing a quality conference talk is itself knowledge work, that requires careful preparation, distillation and combination of even more raw knowledge-stuff. It takes me (an experienced speaker who usually gets good, but not rave, reviews) about three days to produce a new one hour talk, a roughly 25:1 ratio of preparation:delivery. That's about a day of deciding what to say and what to leave out, a day of designing and producing materials like slides, handouts and sample code, and a day of practising and editing. Of course, that's on top of whatever research it was that led me to believe I could give the talk in the first place. The problem I alluded to at the start of the last paragraph is this: there's a conflict between acknowledging that the talks are the bricks-and-mortar of the conference rather than the end product, and wanting some return on the time invested. How that conflict's resolved depends on the personal values of the individual; I won't try to speak for any of my peers here because I don't know their minds.

The conference echo chamber

That's not my phrase; I've heard it a lot and can track my most recent recollection to @secwhat's post Conference Angst. Each industry's conferences has a kind of accepted worldview that is repeated and reinforced in the conference sessions, and that only permits limited scrutiny or questioning - except for one specific variety which I'm coming onto later. As examples, the groupthink in indie Mac/iOS conferences is "developers only need developer features that have been blessed by Apple". There's recently been significant backlash to the RubyMotion framework, as there usually is when a new third-party abstraction for iOS appears. But isn't abstraction a good thing in software engineering? The truth is, of course, that there are more things in heaven and earth than are dream't in Apple's philosophy.

The information security groupthink is that information security is working. Shocking though it may sound, that's far from obvious, evident or even demonstrable. Show me the blind test where similar projects were run with different levels of info sec engagement and where the outcome was significantly different. Demonstrate how any company's risk profile has changed since last year. Also, show me an example of security practitioners being ahead of the curve, predicting and preparing for a new development in the field: where were the talks on hacktivism before Anonymous or Wikileaks?

One reason that the same views are repeated over multiple conferences is that the same circuit of speakers travels to all of the conferences. I'm guilty of perpetuating that myself, being (albeit unintentionally in one year) one of the speakers in the iOS circuit. And when I've travelled to Seattle or Atlanta or Copenhagen or Aberystwyth, I've always recognised at least a few names in the speaker line-up. [While I mentioned Aberystwyth here, both iOSDevUK and NSConf take steps to address the circuit problem. iOSDevUK had a number of first-time speakers and a "bar camp" where people could contribute their own talks. NSConf has the blitz talks which are an accessible way to get a large number of off-circuit speakers, and on one occasion ran a whole day of attendee-contributed sessions called NSConf Mini. When you give people who don't normally present the opportunity to do so, someone will step up.]

I mentioned before that the echo chamber only permits limited scrutiny, and that comes in the form of the "knowing troll" talk. Indeed at GOTO there's a track on the final day called "Iconoclasm", which is populated solely with this form of talk. Where the echo chamber currently resounds to the sound of , it's permitted to deliver an " sucks" talk. This will usually present a straw man version of  and list its failings or shortcomings. That's allowed because it actually reinforces  - real-world examples are rarely anything like as bad as the straw man version, therefore  isn't really that bad. This form of talk is often a last-session-of-the-day entry and doesn't really lead people to challenge their beliefs. What happens later is that when everyone moves on to the next big thing, the " sucks" talks will become the main body of the conference and "&lt;X+1> sucks" will be the new troll talk.

Conferences

Weirdly, while the word conference means a bringing together of people to talk, coming from the same root as "conversation", many conferences are designed around a one-way flow of words from the speakers to the delegates. Here's the thing with that. As I said in my keynote talk at MDevcon, we learn from each other by telling and listening to stories. Terry Pratchett, Jack Cohen and Ian Stewart even went as far as to reclassify humans as pan narrans, the storytelling chimpanzee. Now if you've got M speakers and 10M&lt;N&lt;100M delegates, then putting a sequence of speakers up and listening to their stories gets you a total of M stories. Letting the N delegates each share their stories, and then letting each of the N-1 other delegates share the stories that the first N stories reminded them of, and so on, would probably lead to a total of N! stories if you had the time to host that. But where that does happen, it's usually an adjunct to the "big top" show which is the speaker series. [And remember: if you've got C conferences, you don't have C*M speakers, you have M+ε speakers.]

There's one particular form of wider participation that never works well, and that's to follow a speaker session with Q&amp;A. Listen carefully to the questions asked at the next Q&amp;A you're in, and you'll find that many are not questions, but rhetorical statements crafted to make the "asker" appear knowledgable. Some of those questions that are questions are rhetorical land mines with the intent of putting the speaker on the back foot, again to make the asker seem intellectually talented. Few of these questions will actually be of collective value to the plenus, so there's not much point in holding the Q&amp;A in front of everyone.

Speaker talks are only one way to run a session, though. Panels, workshops and debates all invite more collaboration than speaker sessions. They're also much more difficult to moderate and organise, so are rarely seen: many conferences have optional days that are called "workshops" but in reality are short training courses run by an invited speaker. In the iOS development world, lab sessions are escaping the confines of WWDC and being seen at more independent conferences. These are like one-on-one or few-on-few problem solving workshops, which are well focussed and highly collaborative but don't involve many people (except at Voices that Matter, where they ran the usability workshops on the stage in front of the audience). A related idea being run at GOTO right now, which I need to explore, is a whole track of pair programming sessions. The session host chooses a technology and a problem, and invites delegates onto the stage to work through the challenge with the host in a pair-programming format. That's a really interesting way to attract wider participation; I'll wait until I've seen it in action before reaching an opinion on whether it works.

There's another issue, that requires a bit more setup to explain. Here's a Venn diagram for any industry with a conference scene; the areas are indicative rather than quantitative but they show the relation between:

the population of all practitioners;
the subsection of that population that attends conferences; and
the subsection of that population that speaks at conferences.

So basically conferences scale really badly. Even once we've got past the fact that conferences are geared up to engage the participation of only a handful of their attendees, the next limiting factor is that most people in [whatever industry you're in] aren't attending conferences. For the stories told at a conference (in whatever fashion) to have the biggest impact on their industry, they have to break the confines of the conference. This would traditionally, in many conferences, involve either publishing the proceedings (I've not heard of this happening in indie tech conferences since the NATO conferences of 1968-9, although Keith Duncan is one of a couple of people to mention to me the more general idea of a peer-reviewed industry journal) or the session videos (which is much more common).

To generate the biggest impact, the stories involved must be inspiring and challenging so that the people who watched them, even those who didn't attend the conference, feel motivated to reflect on and change the way they work, and to share their experiences (perhaps at the same conference, maybe elsewhere). Before moving on to a summary of everything I've said so far, I'll make one more point about the groups drawn on the Venn diagram. Speakers tend to be specialists (or, as Marcus put it in his NSConf talk, subject matter experts) in one or two fields; that's not surprising given the amount of research effort that goes into a talk (described above). Additionally, some speakers are asked to conferences because they have published a book on the topic the convenor wishes them to speak on; that's an even longer project of focussed research. This in itself is a problem, because a lot of the people having difficulty with their work are likely to be neophytes, but apparently we're not listening to them. We listen to self-selected experts opining on why everyone needs to take security/TDD/whatever seriously and why that involves retaining the experts' consultancy service: we never listen to the people who can tell us that after a month of trying this Objective-C stuff still doesn't make sense. These are the people who can give us insight into how to improve our practice, because these are the people reminding the experts (and indeed the journeymen) of the problems they had when they'd been at this for a month. They tell us about the issues everyone has, and give us ideas on how we can fix it for all (future) participants.

Conference goers, then, get the benefit of a small handful of specialists: in other words they have a range of experience to call on (vicariously) that is both broad and deep. Speakers of course have the same opportunity, though don't always get to take full advantage of the rest of a conference due to preparation, equipment tests, post-talk question sessions and the like. The "non-goers" entry in the diagram represents a vast range of skills and experiences, so it's hard to find any one thing to say about them. Some will be "distance delegates", attending every conference by purchasing the videos, transcripts or other materials. Some will absorb information by other means, including meet-ups, books, blogs etc. And some will be lone coders who never interact with anyone in their field. Imagine for a moment that your goal in life is to apply the Boy Scout Rule (which I'm going to attribute again to @ddribin because I can't remember who he got it from; Uncle Bob probably) to your whole industry. Your impact on $thing_you_do will be to leave the whole field, the whole practice a bit better than it was when you got here. (If that really is your goal, then skip the imagination part for a bit.)

It seems to me that the best people to learn from are the conference delegates (who have seen a wide section of the industry in considerable depth) and the best people to transfer that knowledge to are, well, everybody.

Summary of the current position

Conferences are good. I don't want people to think I'm hating on conferences. They're enjoyable events, there are plenty of good ones, there's an opportunity to learn things, and to see fresh perspectives on many aspects of our industry. They're also more popular than ever, with new events appearing (and selling out rapidly) every year. However, these perspectives often have an introspective, echo chamber quality. We're often listening to a small subset of the conference delegates, and if you integrate over multiple conferences you find the subset gets relatively smaller because it's the same people presenting all the time. Most delegates will not get the benefit of listening to all of the other delegates, which means they're missing out on engaging with some of the broadest experience in the industry. Most of the practitioners in your corner of the industry probably don't attend any conferences anyway; there aren't enough seats for that to work.

The ideal tech conference

OK, I am very clearly lying here: this isn't the ideal tech conference, it's my ideal tech conference. In my world, those are the same thing. PerfectConf features a much more diverse portfolio of speakers. In the main this is achieved exactly the way that Appsterdam does it; by offering the chance to speak to anyone who'll take it, by looking for things that are interesting to hear about rather than accomplished or expert speakers to say it, and by giving novice speakers the chance to train with the experts before they go in front of the stage. Partly this diversity is achieved by allowing people who aren't comfortable with speaking the opportunity to host a different kind of session, for example a debate or a workshop.

In addition to engaging session hosts who would otherwise be apprehensive about presenting, we get to hear about the successes and tribulations encountered by the whole cohort of delegates. At least one session would be a plenary debate, focussed on a problem that the industry is currently facing. This session has the modest aim of discovering a solution to the problem to move the industry as a whole forward. Another way in which diversity is introduced into the conference is by listening to people outside of our own sector. If infosec is having trouble getting budget for its activities, perhaps they ought to invite more CFOs or comptrollers to its conferences to discuss that. If iPhone app developers find it hard to incorporate concurrency into their application designs, they could do worse than to listen to an Erlang or Occam expert. Above all, the echo chamber would be avoided; session hosts would be asked to challenge the perceived industry status quo.

I've long thought that if a talk of mine doesn't annoy at least one member of the audience then I haven't said anything useful; a former manager of mine said "if we both think the same way about everything then one of us is redundant". This way of thinking would be codified into the conference. Essentially, what I'm talking about is the death of the thought leader (or "rock star"). Rather than having one subject matter expert opining on how everyone should think about security, UX, marketing, or whatever, PerfectConf encourages the community to work together like a slime mould, allowing the collective motion of all of the members to explore all opportunities and options and select the best one by communicating freely across the colony.

Finally, PerfectConf proceedings are published as soon as practical; not just the speaker sessions but the debates too. Where the plenus reaches a consensus, the consensus decision becomes available for all those people who couldn't make it to the conference of to discover, consider, and potentially adopt or react to. Unfortunately I'm not a conference organiser.

#### Tiger to Catalina: let's port some code
Many parts of a modern software stack have been around for a long time. That has trade-offs, but in terms of user experience is a great thing: software can be incrementally improved, providing customers with familiarity and stability. No need to learn an entirely new thing, because your existing thing just keeps on working.

It's also great for developers, because it means we don't have to play red queen, always running just to stand still. We can focus on improving that customer experience, knowing that everything we wrote to date still works. And it does still work. Cocoa, for example, has a continuous history back to 2001, and there's code written to use Cocoa APIs going back to 1994. Let's port some old Cocoa software, to see how little effort it is to stay up to date.

Bean is a free word processor for macOS. It's written in Objective-C, using mostly Cocoa (but some Carbon) APIs, and uses the Cocoa Text system. The current version, Bean 3.3.0, is free, and supports macOS 10.14-10.15. The open source (GPL2) version, Bean 2.4.5, supports 10.4-10.5 on Intel and PowerPC. What would it take to make that a modern Cocoa app? Not much—a couple of hours work gave me a fully-working Bean 2.4.5 on Catalina. And a lot of that was unnecessary side-questing.

Step 1: Make Xcode happy

Bean 2.4.5 was built using the OS X 10.5 SDK, so probably needed Xcode 3. Xcode 11 doesn't have the OS X 10.5 SDK, so let's build with the macOS 10.15 SDK instead. While I was here, I also accepted whatever suggested updated settings Xcode showed. That enabled the -fobjc-weak flag (not using automatic reference counting), which we can now just turn off because the deployment target won't support it.  So now we just build and run, right?

Not quite.

Step 2: Remove references to NeXT Objective-C runtime

Bean uses some "method swizzling" (i.e. swapping method implementations at runtime), mostly to work around differences in API behaviour between Tiger (10.4) and Leopard (10.5). That code no longer compiles:

/Users/leeg/Projects/Bean-2-4-5/ApplicationDelegate.m:66:23: error: incomplete
      definition of type 'struct objc_method'
                        temp1 = orig_method-&gt;method_types;
                                ~~~~~~~~~~~^
In file included from /Users/leeg/Projects/Bean-2-4-5/ApplicationDelegate.m:31:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/objc/runtime.h:44:16: note: 
      forward declaration of 'struct objc_method'
typedef struct objc_method *Method;
               ^


The reason is that when Apple introduced the Objective-C 2.0 runtime in Leopard, they made it impossible to directly access the data structures used by the runtime. Those structures stayed in the headers for a couple of releases, but they're long gone now. My first thought (and first fix) was just to delete this code, but I eventually relented and wrapped it in #if !__OBJC2__ so that my project should still build back to 10.4, assuming you update the SDK setting. It now builds cleanly, using clang and Xcode 11.5 (it builds in the beta of Xcode 12 too, in fact).

OK, ship it, right?

Diagnose a stack smash

No, I launched it, but it crashed straight away. The stack trace looks like this:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x7ffeef3fffc8)
  * frame #0: 0x00000001001ef576 libMainThreadChecker.dylib`checker_c + 49
    frame #1: 0x00000001001ee7c4 libMainThreadChecker.dylib`trampoline_c + 67
    frame #2: 0x00000001001c66fc libMainThreadChecker.dylib`handler_start + 144
    frame #3: 0x00007fff36ac5d36 AppKit`-[NSTextView drawInsertionPointInRect:color:turnedOn:] + 132
    frame #4: 0x00007fff36ac5e6d AppKit`-[NSTextView drawInsertionPointInRect:color:turnedOn:] + 443
[...]
    frame #40240: 0x00007fff36ac5e6d AppKit`-[NSTextView drawInsertionPointInRect:color:turnedOn:] + 443
    frame #40241: 0x00007fff36ac5e6d AppKit`-[NSTextView drawInsertionPointInRect:color:turnedOn:] + 443
    frame #40242: 0x00007fff36a6d98c AppKit`-[NSTextView(NSPrivate) _viewDidDrawInLayer:inContext:] + 328
[...]


That's, um. Well, it's definitely not good. All of the backtrace is in API code, except for main() at the top. Has NSTextView really changed so much that it gets into an infinite loop when it tries to draw the cursor?

No. Actually one of the many patches to AppKit in this app is not swizzled, it's a category on NSTextView that replaces the two methods you can see in that stack trace. I could change those into swizzled methods and see if there's a way to make them work, but for now I'll remove them.

Side quest: rationalise some version checks

Everything works now. An app that was built for PowerPC Mac OS X and ported at some early point to 32-bit Intel runs, with just a couple of changes, on x86_64 macOS.

I want to fix one more thing. This message appears on launch and I would like to get rid of it:

2020-07-09 21:15:28.032817+0100 Bean[4051:113348] WARNING: The Gestalt selector gestaltSystemVersion
is returning 10.9.5 instead of 10.15.5. This is not a bug in Gestalt -- it is a documented limitation.
Use NSProcessInfo's operatingSystemVersion property to get correct system version number.

Call location:
2020-07-09 21:15:28.033531+0100 Bean[4051:113348] 0   CarbonCore                          0x00007fff3aa89f22 ___Gestalt_SystemVersion_block_invoke + 112
2020-07-09 21:15:28.033599+0100 Bean[4051:113348] 1   libdispatch.dylib                   0x0000000100362826 _dispatch_client_callout + 8
2020-07-09 21:15:28.033645+0100 Bean[4051:113348] 2   libdispatch.dylib                   0x0000000100363f87 _dispatch_once_callout + 87
2020-07-09 21:15:28.033685+0100 Bean[4051:113348] 3   CarbonCore                          0x00007fff3aa2bdb8 _Gestalt_SystemVersion + 945
2020-07-09 21:15:28.033725+0100 Bean[4051:113348] 4   CarbonCore                          0x00007fff3aa2b9cd Gestalt + 149
2020-07-09 21:15:28.033764+0100 Bean[4051:113348] 5   Bean                                0x0000000100015d6f -[JHDocumentController init] + 414
2020-07-09 21:15:28.033802+0100 Bean[4051:113348] 6   AppKit                              0x00007fff36877834 -[NSCustomObject nibInstantiate] + 413


A little history, here. Back in classic Mac OS, Gestalt was used like Unix programmers use sysctl and soda drink makers use high fructose corn syrup. Want to expose some information? Add a gestalt! Not bloated enough? Drink more gestalt!

It's an API that takes a selector, and a pointer to some memory. What gets written to the memory depends on the selector. The gestaltSystemVersion selector makes it write the OS version number to the memory, but not very well. It only uses 32 bits. This turned out to be fine, because Apple didn't release many operating systems. They used one quartet (i.e. one hexadecimal digit) each for major, minor, and patch release numbers, so macOS 8.5.1 was represented as 0x0851.

When Mac OS X came along, Gestalt was part of the Carbon API, and the versions were reported as if the major release had bumped up to 16: 0x1000 was the first version, 0x1028 was a patch level release 10.2.8 of Jaguar, and so on. So maybe the system version is binary coded decimal.

At some point, someone at Apple realised that if they ever did sixteen patch releases or sixteen minor releases, this would break. So they capped each of the patch/minor numbers at 9, and just told you to stop using gestaltSystemVersion. I would like to stop using it here, too.

There are lots of version number checks all over Bean. I've put them all in one place, and given it two ways to check the version: if -[NSProcessInfo isOperatingSystemAtLeastVersion:] is available, we use that. Actually that will never be relevant, because the tests are all for versions between 10.3 and 10.6, and that API was added in 10.10. So we then fall back to Gestalt again, but with the separate gestaltSystemVersionMajor/Minor selectors. These exist back to 10.4, which is perfect: if that fails, you're on 10.3, which is the earliest OS Bean "runs" on. Actually it tells you it won't run, and quits: Apple added a minimum-system check to Launch Services so you could use Info.plist to say whether your app works, and that mechanism isn't supported in 10.3.

Ship it!

We're done!



Haha, just kidding, we're not done. Launching the thing isn't enough, we'd better test it too.



Dark mode wasn't a thing in Tiger, but it is a thing now. Bean assumes that if it doesn't set the background of a NSTextView, then it'll be white. We'll explicitly set that.

Actually ship it!

You can see the source on Github, and particularly how few changes are needed to make a 2006-era Cocoa application work on 2020-era MacOS, despite a couple of CPU family switches and a bunch of changes to the UI.

#### To become a beginner, first become an expert
We have a whole load of practices in programming that only really work well if you're already good at whatever the process is supposed to help with.

Scrum is a process improvement framework, but only if you already know how to do process improvement. If you don't, then Scrum is just the baseline mini-waterfall process with a chance to air your dirty laundry every fortnight.

Agile is good at helping you embrace change, but only if you're already good enough at managing change to understand which changes should be embraced.

#NoEstimates helps you avoid the overhead of estimates, but only if you're already good enough at estimates to know that you always write user stories that take 0.5-2 days to implement.

TDD helps you design your APIs, but only if you're already good enough at API design to understand things like dependency injection and loose coupling.

Microservices help you isolate modules, but only if you're already good enough at modularity not to get swamped in HTTP calls.

This is all very well for selling consultancy ("if your [agile] isn't working, then you aren't [agiling] hard enough, let me [agile] you some more") but where's the on-ramp?

#### Today's broken user interface award...
...goes to printing on OS X Server.  You'll notice that you can add or remove printers via Server Admin: however, if you do add one via that route then the system will try and get the printer to emit its PPD at it.  If the printer doesn't do this, then it's a &quot;Generic Postscript Printer&quot; and there's no way to change that.

Unless, of course, you log in at the console (Xserves make that so easy and snappy to achieve) and use the same Printer Setup Utility you'd otherwise have used on the client OS.  You can then go back into Server Admin to set up whether a queue is quotad, set up sharing and so on.

Great, so far we've used two apps to configure our printer.  Let's just add a third (although this does make sense), because you now have to go into workgroup manager to set up the per-user quotas for the print queues.  Fine, and while we're here we'll use managed config to define which printers appear in which print panels.  Only, we can't.  Shared printers can't be added to the MCX print menus...even if they appear in my print menu.

This is so confusing it's almost fun...

#### Today's surprisingly short workflow-improving win
When I have a TODO comment (or a #error in code, which is how I frequently do TODOs), I switched to writing the commit message I want to be able to use when I've fixed the TODO. Then I write the thing that's missing, kill the TODO and yank it into the commit command.

#### Tony Hoare and negative space
The Primeagen calls it Negative-Space Programming: using assertions to cut off the space of possible programs, leaving only the ones you believe are possible given your knowledge of a program's state at a point.

Tony Hoare just called it "logic", which is why we now call it Hoare Logic. OK, no he didn't. He called it An axiomatic basis for computer programming, but that doesn't make the joke work. The point is, this is how people were already thinking about program design in 1969 (and, honestly, for a long time before). I'm glad the Primeagen has discovered it, I just don't think it needs a new name.

Imagine you know the program's state (or, the relevant attributes of that state) at a point in your program, and you can express that as a predicate—a function of the state that returns a Boolean value—P. In other words, you can assert that P is true at this point. The next thing the computer does is to run the next instruction, C. You know the effect that C has on the environment, so you can assert that it leaves the environment in a new state, expressed by a new predicate, Q.

Tony Hoare writes this as a Hoare Triple, {P}C{Q}. We can write it out as a formulaic sentence:

Given P, When C, Then Q

Given-when-then looks exactly like a description of a test, because it is; although our predicates might not be specific states for the program to be in as in a specific unit test, they might describe sets of states for the program to be in, as in a property-based test.

We can also use the fact that the computer runs program statements in sequence to compose these triples. Given P, when C, then Q; given Q, when D, then R; given R and so on. Now, if we arrange our program so that it only runs C when P is true, and it always runs D next, and so on, we can use the composition rule to say something about the compound statement C;D;etc. If we wrap that compound statement up in a function f(), then we get to say this:

If the caller of f() makes sure that the precondition for calling the function is true before making the call, then f() guarantees that the postcondition is true after the call returns.

This combination of responsibilities and guarantees is a contract. If you start with the contract and work to design an implementation that satisfies the contract, then you are designing by contract. You're following a discipline of programming, as Dijkstra put it.

Oh, speaking of Dijkstra. We said that you need to know that the computer always runs this part of our program in the sequential way, with the stated precondition, that means it isn't allowed to jump partway into it with some other, different initial state. If we allowed that then we'd have a much harder time constructing our P at a point in the program, because it would be whatever we think it is if the program follows its normal path, or whatever it is if the program goes to this point from over here, or whatever it is if the program goes to this point from over there, and so on. And our Q becomes whatever we think it is if the computer runs C after the normal path, or whatever it is if the computer goes to this point from over here then runs C, and so on.

Thinking about all of that additional complexity, and persevering with designing a correct program, could be considered harmful. To avoid the harm, we design programs so that the computer can't go to arbitrary points from wherever the programmer wants. All to support logically-designed programming, or "negative-space programming" as I suppose we're meant to call it now.

#### Too much confidence is good for you
So, Sun have a new CEO in the shape of President and erstwhile COO Jonathan Schwartz.  Schwartz joined Sun back when they acquired Lighthouse Design, a NeXTSTEP applications developer with an office suite for the platform.  Having slightly more bravado than sense, I sent this:

Hi Jonathan, I hope you do not mind receiving unsolicited personal mail.Congratulations on your appointment as CEO of Sun.  Perhaps now would be a good time, in the spirit of openness and interoperability that Sun is famous for, and in extension of the success of the OpenSolaris project, to revisit that Lighthouse Design code in the basement with a view to releasing it under an acceptable open source licence?  Would you be able to also open up the source for Sun's OpenStep implementation, as a reference for Cocoa and GNUstep developers to see how it's done?  So doing would give Sun lots of 'good faith' points among Cocoa developers on the Mac platform, and further Sun's position at the front of the 21st Century's open source-centric software market.By the way, Cocoa really is a great application development platform (as I'm sure you know with your OpenStep experience), and the combination of Apple's UNIX workstations with Sun's Solaris UNIX servers, both sporting a coherent and consistent Cocoa-based application stack and development environment would truly be a wonder to behold ;-).Regards,Graham.

I expect precisely nothing to occur...

#### Tracking the invisible, moving, unpredictable target
An idea which has been creeping up on me from the side over the last couple of weeks hit me square in the face today.  No matter what standards we Cocoa types use to create our user interfaces, the official Aqua HIG, the seemingly-defunct IndieHIG, or whatever, ultimately producing what is considered a usable (or humane, if you like) interface for Mac OS X is not only difficult, but certainly unrepeatable over time.

The "interface" part of a Cocoa user interface is already hard enough to define, being a mash-up of sorts, and to differing degrees, between the Platinum HIG which directs the default behaviour of some of the *Manager controls and the OpenStep HIG which describes the default behaviour of most, if not all, of the AppKit controls.  If that isn't enough, there is an inexact intersection - some controls work differently in (what are loosely called, and I'm not getting into the debate) Cocoa apps than in Carbon apps.  There have also been innovative additions on top of the aforementioned guides, such as sheets, unified toolbars and (the already legacy) textured interfaces.  There have been subtractions from both - miniwindows still exist but nobody uses 'em, and window shading went west with Rhapsody.

But all of that is related to the user interface, not to user interaction (I'm in the middle of reading Cooper's The Inmates Are Running the Asylum, I'm going to borrow some terminology but studiously avoid discussing any of the conclusions he presents until I'm done reading it).  It's possible to make HIG-compliant inspectors, or HIG-compliant master-detail views, or HIG-compliant browser views and so on.  It's also possible to make non-compliant but entirely Mac HID views, coverflow views, sidebars and so on.  But which is correct?  Well, whichever people want to use.  But how do you know which people want to use?  Well, you could get them to use them, but as that's typically left until the beta phase you could ask usability gurus instead.  Or you could take the reference implementation approach - what would Apple (or Omni, or Red Sweater, or whoever) do?

Well, what Apple would do can, I think, be summed up thus: Apple will continue doing whatever Apple were previously doing, until the Master User takes an interest in the project, then they do whatever the Master User currently thinks is the pinnacle of interaction design.  The Master User acts a little like an eXtreme Programming user proxy, only with less frequent synchronisation, and without actually consulting with any of the other 26M users.  The Master User is like a reference for userkind, if it all works for the Master User then at least it all works for one user, so everyone else will find it consistent, and if they don't find it painful they should enjoy that.  The official job title of the Master User role is Steve.

All of this means that even inside Apple, the "ideal" usability experience is only sporadically visited, changes every time you ask and doesn't follow any obvious trend such as would be gained by normalisation over the 26M users.  Maybe one day, the Master User likes inspectors.  Then another day he likes multi-paned, MDI-esque interaction.  On a third day he likes master-detail control, in fact so much so that he doesn't want to leave the application even when it's time to do unrelated work.  Of course you don't rewrite every application on each day, so only the ones that he actually sees get the modernisation treatment.

So now we come back to the obvious, and also dangerous, usability tactics which are so prevalent on the Windows platform, and one which I consciously abhor but subconsciously employ all the time: "I'm the developer, so I'll do it my way".  Luckily there are usability, QA and other rational people around to point out that I'm talking shite most of the time, but the reasoning goes like this.  I'm a Mac user, and have been for a long time.  In fact, I might know more about how this platform works than anyone within a couple of miles of here, therefore(?) I know what makes a good application.  One problem which affects my personal decisions when trying to control the usability is that I'm only tangentially a Mac person, I'm really a very young NeXTStep person who just keeps current with software and hardware updates.  That means I have a tendency to inspector my way out of any problem, and to eschew custom views and Core Animation in favour of "HIG is king" standard controls, even when other applications don't.  And the great thing is that due to Moving Target reference implementation, I can find an application which does something "my" way, if that will lend credence to my irrational interface.

The trick is simply to observe that taking pride in your work and expressing humility at your capabilities are not mutually exclusive.  If tens of other Mac users are telling me they don't like the way it works, and I'm saying it's right, apply Occam's razor.

And if there isn't enough fun for you in one usability experience, a bunch of us are presumably going to be providing the iPhone HIG-compliant V on top of our Ms and Cs before long.

#### Transcendence
I was at the RSE conference in Newcastle, along with many people whom I have met, worked with, and enjoyed talking to in the past. Many more people whom I have met, worked with, and enjoyed talking to in the past were at an entirely different conference in Aberystwyth, and I am disappointed to have missed out there.

One of the keynote speakers at RSEcon22, Marlene Manghami, talked about the idea of transcendence through community membership. They cited evidence that fans of soccer teams go through the same hormonal shifts at the same intensity during the match as the players themselves. Effectively the fans are on the pitch, playing the game, feeling the same feelings as their comrades on the team, even though they are in the stands or even at home watching on TV.

I do not know that I have felt that sense of transcendence, and believe I am probably missing out both on strong emotional connections with others and on an ability to contribute effectively to society (to a society, to any society) by lacking the strong motivation that comes from knowing that making other people happier makes me happier, because I am with them.

#### Tsundoku
I only have the word of the internet to tell me that Tsundoku is the condition of acquiring new books without reading them. My metric for this condition is my list of books I own but have yet to read:

the last three parts of Christopher Tolkien's Histories of Middle-Earth
Strategic Information Management: Challenges and Strategies in Managing Information Systems
Hume's Enquiries Concerning the Human Understanding
Europe in the Central Middle Ages, 962-1154
England in the Later Middle Ages
Bertrand Russel's Problems with Philosophy
John Stuart Mill's Utilitarianism and On Liberty (two copies, different editions, because I buy and read books at different rates)
A Song of Stone by Iain Banks
Digital Typography by Knuth
Merchant and Craft Guilds: A History of the Aberdeen Incorporated Trades
The Indisputable Existence of Santa Claus
Margaret Atwood's The Handmaid's Tale

And those are only the ones I want to read and own (and I think that list is incomplete - I bought a book on online communities a few weeks ago and currently can't find it). Never mind the ones I don't own.

And this is only about books. What about those side projects, businesses, hobbies, blog posts and other interests I "would do if I got around to it" and never do? Thinking clearly about what to do next and keeping expectations consistent with what I can do is an important skill, and one I seem to lack.

#### Turn and face the strange
I've made some changes around here, and they impact how both you and I interact with this blog.

Firstly, I added asides, which are untitled posts in the main stream. I think of them as something like Daring Fireball's Linked List. I used to tweet links that interest me, but once I've put a link in a tweet I haven't got much space to explain why it was interesting. Sometimes I post links to comedic articles and I don't want to discuss their accuracy, for example. Bringing that sort of post over here removes an arbitrary length constraint, while still letting me distinguish between full posts and, well, asides.

The previous post is an aside. It's untitled, and short, but otherwise looks the same as any other post. I might change that.

The second change is that I've turned comments (back) on, though only for posts published today or later. The arguments for turning comments off are well thought out and well-meaning, but don't work out too well in practice.

They’re for a tiny minority. True, which means it shouldn't be much effort to manage them.
comments on the web don’t contribute very much. Many don't, but can be marginally useful to the post author. It's common for an article to receive a comment that contains no more than "Typo?". If I see that, I can correct the typo and delete the comment: you get a more accurate post and a more tidy comment stream, so it's a net benefit.
Comments encourage unconsidered responses. and
Comments allow anonymity and separation of your words from your identity. Both true, but no more than the rest of the internet. And, indeed, no less: the question of whether anonymity or pseudonymity is to be welcomed or shunned is complicated (search for "real names policy", for example).
Comments create a burden of moderation on the blog owner. Not too bad, if the "tiny minority" is commenting. I have a spam-detecting thing, and I'm never going to check it. If you don't want to end up there, don't try selling my readers herbal viagra.

In addition, having comments switched off dilutes the experience for those people who did want to see what people were talking about. There'd be some chat over on twitter (some of which mentions me, some of which doesn't), and some over on the blog's Facebook page. Then people will mention the posts on their favourite forums like Reddit, and a different conversation would happen over there. None of that will stop with comments on, and I wouldn't want to stop it. Having comments here should guide people, without forcing them, to comment where everyone can see them.

Of course just because it's possible to submit a comment here doesn't mean I'll publish it. I rule with an iron fist, and anything that I don't think contributes won't actually get allowed through.

#### Turn it off and back on again
I'm now six months into what I expected to be about a year out of working in technology, and I'm starting to think about what comes next and trying to make it happen. The difficulty I have is that it's hard to explain what I'm looking for in a way that makes sense to those that are hiring, or that I can summarise in a search term for job sites. I'm considering running a company again to do all this myself, but that doesn't obviate the problem, I still need to be able to describe this to potential clients and explain why they would want to buy one.

The difficulty comes from being a people person. I listen to people, I talk to people, I get people to talk to other people, I learn from people, I teach people, I perform for people, I watch people, I read what people have to say, and I write for people. And I happen to want to do that for money in the software industry, but if you tell that to a hiring manager on a software team you'll get a blank stare followed by "um, but how much have you used MongoDB from Scala in your last job?". You don't need to try this yourself, I have done it. This is what happens.

I don't mind much what technology I use, as long as we're using it because it helps to address the problems we or our customers have rather than because a developer threw a strop if they weren't going to be allowed to rewrite everything that already works well in NodeCaml. I care about understanding and solving the problems people have, and about understanding the people who have those problems. "I think we should use this" is not fine. "I think we should use this because" is perfectly fine. "We're a $VENDOR shop" is probably not fine.

So the problem I have is that the job I know how to apply for and get is "programmer" (these days with some highfalutin prefix that really comes down to "better paid"), but that usually comes with some expectation to focus on the programming, and leave all the gloopy soft stuff like what programming should be done and whether it's a good idea to do it now to other people. What I want to be doing is (being paid for) the gloopy soft stuff like making programmers into better programmers, working out what programming should be done (if any) and whether it's a good idea to do it now, helping programmers to understand the people they're helping, and helping the people being helped by programmers to understand the programmers, with the programming itself being a context not a focus. I have no idea how to explain that succinctly to people who might want to hire one of those, nor how to find people who might want to hire one of those.

Practically, based on what I've experienced about my own health and its relationship with my work, I also need to be realistic about where and when I work. That's from or within cycling distance of home (around Leamington Spa, Warwick, Kenilworth and Coventry), from usual working hours my own timezone. If your company is in a different timezone and supports remote work, that's great, but if you need me to work from your timezone then it's not great. In fact, you don't support remote work, you just support local employees who don't always come into the office.

If you are someone who wants one of those, know someone who wants one of those, or know how to describe one of those succinctly, please do help me out. Based on the last time I tried this, here's a couple of lists:

Things I’ve never done, but would

These aren’t necessarily things my next job must have, and aren’t all even work-related, but are things that I would take the opportunity to do.

Work in a field on a farm. Preferably in control of a tractor.
Have a job title that begins with the letters 'C' or 'D' (I managed 'Q' a while back).
Spend lots of time supporting the Free Software Definition
Include going to lunch with each other employee in the company in my responsibilities.
Visit Iceland.

Things I don’t like

These are the things I would try to avoid.

I still seriously hate raw celery.
Client work, in those cases where we don't all really think that the client is doing something important.
"Rock star" programmers, and people who hire them.

#### Two books
A member of a mailing list I'm on recently asked: what two books should be on every engineer's bookshelf? Here's my answer.

Many software engineers, the ones described toward the end of Code Complete 2, would benefit most from Donald Knuth’s The Art of Computer Programming and Computers and Typesetting. It is truly astounding that one man has contributed so comprehensively to the art of variable-height monitor configurations.

If, to misquote Bill Hicks, “you’ve got yourself a reader”, then my picks are coloured by the fact that I’ve been trying to rehabilitate Object-Oriented Design for the last few years, by re-introducing a couple of concepts that got put aside over the recent decades:

Object orientation; and
Design.

With that in mind, my two recommendations are the early material from that field that I think shows the biggest divergence in thinking. Readers should be asking themselves “are these two authors really writing about the same topic?”, “where is the user of the software system in this book?”, “who are the users of the software system in this book?”, and “do I really need to choose one or other of these models, why not both or bits of both?"

“Object-Oriented Programming: an evolutionary approach” by Brad Cox (there is another edition with Andrew Novobilski as a co-author). Cox’s model is the npm/CPAN model: programmers make objects (“software ICs”), describe their characteristics in a data sheet, and publish them in a catalogue. Integrators choose likely-looking objects from the catalogue and assemble an application out of them.
“Object-Oriented Software Construction” by Bertrand Meyer. Meyer’s model is the “software engineering” model: work out what the system should do, partition that into “classes” based on where the data should naturally live, and design and build those classes. In designing the classes, pay particular attention to the expectations governing how they communicate: the ma as Alan Kay called the gaps between the objects.

#### Two out of three ain't bad
Parts one and two of OOP the Easy Way are now both complete. Part three will be underway soon, in the meantime you are welcome two read the first two parts on Leanpub (and will automatically be entitled to updates as soon as they are published, for free, for the lifetime of the project). All feedback is always welcome.

#### Two Schools
There always seem to be two schools in software, though exactly where the gates are varies. Alan Kay described how Edsger Dijkstra noticed that "the Atlantic has two sides".
It was basically all about how different the approaches to computing science were in Europe, especially in Holland and in the United States. In the US, here, we were not mathematical enough, and gee, in Holland, if you're a full professor, you're actually appointed by the Queen, and there are many other uh important distinctions made between the two cultures. So, uhm, I wrote a rebuttal paper, just called On the fact that most of the software in the world is written on one side of the Atlantic.
Or maybe both schools are on the same continent.
The essence of [the MIT/Stanford approach] can be captured by the phrase the right thing. [...] The worse-is-better philosophy is only slightly different [...] and I will call the use of this design strategy the New Jersey approach.
Or they could be ways of thinking, school curricula, or whatever.

Maybe both schools have something to teach us. Maybe it's the same thing.

#### Two ways of thinking
I've used this idea in conversations for years, and can't find a post on it, which I find surprising but there you go. There are, broadly speaking, two different ways to look at programming languages. And I think that these mean two different ways to select programming languages, which are asymmetric. However, they can lead to the same choice, but viewed in different ways: if you use one of those languages then you need to understand that these different views exist to understand decisions made by programmers working in those languages.

On the one hand, the Language Lawyer seeks out languages with plenty of esoterica and specifics, in order to become proficient at recalling those esoterica and demonstrating their correct application. The Language Lawyer loves that reference and value types in swift are actually called class and struct, or that the difference between class and struct in C++ is the default member visibility, and that neither is actually related to the words class or struct. They are delighted both to know and to explain that in Perl 5, @var, \var and $var refer to the same variable but are accessed in different contexts and what they evaluate to in those contexts. They are excited to explain that technically the admonition that you must always return a value from a non-void function in C is not true, because since C99 if you have a function called main that returns control without returning a value, the compiler will implicitly return 0.

The Language Lawyer seeks out a language with such esoterica. But learning it is complex and time-consuming, so they probably don't change particularly often. In fact, changing language may be deleterious, because then other people will know the esoterica that they get caught out on. Imagine knowing how the function overloading rules in C++ work, then trying something similar in Java and finding that you're wrong. And that somebody else knows that. The shame! Once their language is chosen, the Language Lawyer will form heuristic rules of which language feature to use when solving which problem.

Standing in the other corner, the Lazy Evaluator wants to avoid those edge cases, because it makes them think about the programming language, not the software problem. They'd much prefer to have an environment in which there's as much as one way to express their solution, then worry about how to express their solution using that one tool. The Lazy Evaluator loves Ruby because Everything Is An Object (and they love Io more because in Ruby there are also classes). The Lazy Evaluator is delighted to know that in Lisp, Everything Is A List. They are excited to be programming in Haskell, where Everything Is A Function.

Both of these people can be happy with the same programming language, though clearly this will be for different reasons, and their different thought processes will bring them into conflict. The Lazy Evaluator will be happy enough using the third-most common C-family language, C++-- (full name: "C++98 but never use exceptions or templates, avoid multiple inheritance, and [never/always] mark methods as virtual"). The Language Lawyer will enjoy demonstrating const-correctness and the difference between l-, r-, pr-, gl- and x-values, but the two will come to blows over whether the Lazy Evaluator is denying the full elegant expressiveness of the language by proscribing exceptions.

Similarly, the Lazy Evaluator can look at JavaScript, see the simple classless object model with dynamic dispatch remembered from Self or Io, even notice that Functions Are Objects Too and work in that Everything Is An Object paradigm. The Language Lawyer can be confident that at some point, all of the weird coercion behaviour, the Which this Is That this Anyway question, and the You Need That Flag With This Polyfill To Get That Language Feature issues will come up. But the Lazy Evaluator will notice that That Flag also enables class syntax and arrow functions, we already have prototypes and function functions, and disagreement will ensue.

So in what way are those ways of thinking asymmetric? Invoking Hickey's Corollary, you can never be recomplecting. It's easier to compromise on C++ But Without Exceptions than it is to compromise on Lisp But We'll Add Other Atom Types. You can choose JavaScript But Never Use this, you can't choose Haskell But With Type-Escaping Side Effects.

This isn't really about programming languages, it's about any form of model or abstraction. Languages are a great example though. I think it's a mindset thing. Your task is to solve this problem, using this tool. Are you focusing on the bit that's about the problem, or the bit that's about the tool? Both are needed.

#### Type safety, undefined behaviour, and us
There appears to be a shift towards programming languages that improve safety by providing an expressive type system, automatic memory management, and no gaps in the specification that lead to “undefined behaviour”. If your program is consistent with the logic of the programming language specification, then it compiles and executes the behaviour you would understand from the source code and the language documentation. If any of that isn’t true, then your program doesn’t compile: there are no gaps for not-quite-consistent programs to fall through, that get detected later when the program is running.

When I express this set of language design constraints, you may think of Swift, of Rust, of F#. You may try to point to Java, and say that it is very precisely defined and specified (it may be, but it still has undefined behavior: consider when fail-fast iterators throw ConcurrentModificationException, for example). But when was this trend started? When did people start to think about how these programming language properties might affect their programs?

Here’s a 1974 description of the SIMULA 67 programming language. This is SIMULA as a tool for extensible program products, by Jacob Palme, published in ACM SIGPLAN notices.  The quotes are presented out of order, for expressivity.

Many programming language definitions contain the word "undefined" in many places. This is bad for security, since no one knows for sure what will happen if a programmer by mistake makes something "undefined" . In the definition of SIMULA, there is almost no occurence [sic] of the word "undefined" . This means that you always can know what is going to happen if you just have a program and the language definition . All errors can be discovered as soon as a rule of the language is broken. In other languages, errors are some times not discovered until strange consequences appear, and in these languages you then have to look at mysterious dumps to try to find the error. This can never happen in a fully working SIMULA system. All errors against the language are discovered as soon as they logically can be discovered, and the error message is always phrased in SIMULA terms, telling you what SIMULA error you have done. (Never any mysterious message like "Illegal memory reference"). The type checking also means that those programming errors which are not language errors in other languages will very often be language errors in SIMULA, so that they can be discovered by the compiler. Compared to other languages I know of, no language has such a carefully prepared, fully covering security system as SIMULA, and this is very important for the production of commercial program products.

Important for security is also that there is no kind of explicit statement for freeing the memory of a record no longer needed. Such statements are very dangerous, since the program can by mistake later on try to process a record whose memory has been freed. In SIMULA, records no longer needed are removed automatically by a so called garbage collector.

Important for security is also that the data fields of new records are initialized at allocation time. Otherwise, some garbage left from previous data could give very difficult errors.

When Palme says “security”, he doesn’t mean the kind of thing we might call “software security” or “information security” in 2023: the protection of assets despite possibly-intentional misuse of the program. He really means “safety”: the idea that the programmer can know the behaviour of the program, even when the program has been deployed to the customer’s computer and is out of the programmer’s direct control.

Now, what do we learn from this? Firstly that the problems evident in C were already known and solved when C was being developed. C’s first wave was between roughly 1972 (its inception) and 1978 (first edition K&amp;R); this 1974 review identifies important qualities evinced by a 1967 programming language; qualities lacking in C.

Also, that they maybe aren’t that showstoppers, given how much software is successfully in use today and is written in C, or a related language, or on a runtime that’s written in C. Probably that software engineering is a fashion discipline, and that safe programming languages are fashionable now in a way that they weren’t in the 1960s, 1970s, 1980s, 1990s, 2000s, and 2010s: we had access to them, and we didn’t care to use them.

But also we see support for André Gide’s position: Toutes choses sont dites déjà; mais comme personne n'écoute, il faut toujours recommencer. That is: everything has already been said, but because nobody listens, it’s necessary to say it again.

That nobody listens isn’tt a criticism of everyday software engineers. There were a tiny number of people reading ACM SIGPLAN notices in 1974, and most people who’ve entered computing in the intervening 39 years don’t know any of them. There’s no way they could be reasonably expected to have encountered this particular review of SIMULA, let alone to have taken the time to reflect on its position with respect to everything else they know about programming languages so that it can influence their choices.

Software engineering is big enough, and up until recently growing fast enough, that to a first approximation anything that’s worth reading in the field hasn’t been read by any practitioner. If an idea is important, it bears repeating. The people who didn’t hear it the first time around aren’t ignorant; they’re busy.

#### Ubiquitous computing
I, along with many others, have written about the influence of Xerox PARC on Apple. The NeXT workstation was a great example of getting an approximation to the Smalltalk concept out using off-the-shelf parts, and Jobs often presaged iCloud with his discussion of NetInfo, NFS, and even the magneto-optical drive. He'd clearly been paying attention to PARC's Ubiquitous Computing model. And of course the iPad with Siri is what you get if you marry the concept of the DynaBook with a desire to control the entire widget, not ceding that control to some sap who's only claim to fame is that they bought the thing.

Sorry, they licensed the thing.

There are some good signs that Apple are still following the ubicomp playbook, and that's encouraging because it will make a lot of their products better integrated, and more useful. Particularly, the Apple Watch is clearly the most "me" of any of my things (it's strapped to my arm, while everything else is potentially on a desk in a different room, stuck to my wall, or in my pocket or bag) so it makes sense that that's the thing I identify with to everything else. Unlocking a Mac with my watch is great, and using my watch to tell my TV that I'm the one plugging away at a fitness workout is similarly helpful.

To continue along this route, the bigger screen devices (the "boards", "pads", and "tabs" of ubicomp; the TVs, Macs, iPads, and iPhones of Apple's parlance) need to give up their identities as "mine". This is tricky for the iPhone, because it's got an attachment to a phone number and billing account that is certainly someone's, but in general the idea should be that my watch tells a nearby screen that it's me using it, and that it should have access to my documents and storage. And, by extension, not to somebody else's.

A scene. A company is giving a presentation, with a small number of people in the room and more dialled in over FaceTime (work with me, here). It's time for the CTO to present the architecture, so she uses the Keynote app on her watch to request control of the Apple TV on the wall. It offers a list of her presentations in iCloud, she picks the relevant one by scrolling the digital crown, and now has a slide remote on her wrist, and her slides on the screen.

This works well if the Apple TV isn't "logged in" to an iCloud account or Apple ID, but instead "borrows" access from the watch. Because the watch is on my wrist, so it's the thing that is most definably "mine", unlike the Apple TV and the FaceTime call which are "my employer's".

#### Ultimate Programmer Super Stack
There's a great bundle of polyglot learning taking place over at the Ultimate Programmer Super Stack. My book, APPropriate Behaviour - the things every programmer needs to know that aren't programming - is featured alongside content on Python, Ruby, Java, JS, Aurelia, Node, startups, and more.

The bundle is just up for a week, but please do check it out: for not much more than you'd probably pay for APPropriate Behaviour you'll get a whole heap of stuff that should keep you entertained for a while :).

#### Ultimate Programmer Super Stack Reloaded
Remember remember the cough 6th of November, when APPropriate Behaviour joined a wealth of other learning material for software engineers in a super-discounted bundle called the Ultimate Programmer Super Stack?

It's happening again! This is a five-day flash sale, with all same material on levelling up as a programmer, running a startup, and learning new technologies like Aurelia, Node, Python and more. The link at the top of this paragraph goes to the sales page, and you've got until Monday, when it's gone for good.

#### Ultimate Programmer Super Stack: Last day!
I already wrote about the Ultimate Programmer Super Stack, a huge bundle of books and courses on a range of technologies: Python, JS, Ruby, Java, HTML, node, Aurelia… and APPropriate Behaviour, my book on everything that goes into being a programmer that isn't programming.

Today is the last day of the bundle. Check it out here, it won't be available for long.

#### Unfamiliar territory
People who've seen me play music more than once will probably have noticed more than a slight sense of a pattern - give me a fiddle (or if you're really unlucky then I'll bring my own), and I play English folk music. Give me a guitar and I'll play rock and roll. That's just what my fingers are used to - to me the guitar has six strings of rockabilly and the fiddle has four strings of constant billy. No, I didn't write this piece just to get that pun in, though I am glad that I did. I can now go to my grave a happy man, safe in the knowledge that I have compared fishes flying over mountains with blues-derived country music. Classy.Well, I think it's time to change things around a bit. I want to leave my violin all shook up, and my guitar all in a garden green. It probably sounds quite easy, given that I already know all the tunes, and just want to play them on different instruments, but really it isn't. There are two important issues which make it hard for me to just swap over.The first is that in many cases I don't actually know the tune at all, I just use muscle memory to get my digits moving in the right places for music to occur. That's particularly true in the case of rock n roll music, where there really aren't tunes at all. There are just 'licks', basic one or two bar figures which are strung together into a twelve bar part. But it's also true in folk music which has a similarly hypnotic repetition but with longer figures. So taking a tune to a new instrument means discovering what it is I'm actually playing on the first instrument, then trying to reproduce that on the second.The second issue is that just playing the notes from one instrument on another isn't necessarily the correct thing to do, nor even particularly easy. For a start guitar strings are tuned to fourths (mainly) while violin strings are to fifths, and as the bridges are different shapes the instruments invite playing a different number of strings simultaneously. So what I need to do is not even to work out how to play the same tune on the other instrument, but what that instrument's version of the tune should be and how to play that.I expect that the outcome of this little experiment will be mainly a cacophony, but with some increased understanding of what the instruments can do and how to play them. If I focus on cacophony then I'll probably get quick results, though.

#### Unintended consequences
As the shift in content of this blog has made abundantly clear, for the last five years I've been doing a PhD. I've also been working full-time, so that research and study has basically taken up all of my spare time. I would wake up early, sit at my computer until it was time to start work, then change computer, then at the end of the work day switch back to the university-issued computer. I knew this was having an adverse impact on my health. Particularly, I could feel that I had far less stamina. Then I got the wake-up call I needed.

My fitness tracker sent me a notification at the beginning of August that my VO2 max—the measure it uses to gauge cardiorespiratory fitness—had dropped to 28.6. The average value for a man of my age is 38, and 28 is the average value for a man over 60. Low cardio fitness, according to the app, is a predictor of various heart diseases, colon cancer, type 2 diabetes, and even dementia and Alzheimer's.

I've been on holiday for about half of the time since then, but my holidays tend to be active as they involve lots of walking and dancing (I go to festivals). On workdays, I've added a ten-minute HIIT workout to my morning, and a half-hour walk at lunchtime. Just those changes has seen the (estimated) cardio value in my tracker go up to 30.2 in three weeks. This is still in the low category—31.0 is the boundary to the “below average” category—but it's a trend in the right direction.

I've always defined my “success” by my intellectual contribution to society. But I need to stay healthy to continue making that contribution, and it got past time to take control of my health.

#### Unit test: you keep using this word.
There's an idea doing the rounds that the “unit” in “unit test” means the unity of the test, rather than a test of a software unit. Moreover, that it originally meant this, and that anyone who says “unit test” to mean the test of a software unit is misguided.

Here's the report of the 1968 NATO conference on software engineering. On their page 20 (as marked, not as in the PDF) is a diagram of a waterfall-esque system development process, featuring these three phases among others (emphasis mine):

unit design

unit development

unit test

“Unit” meaning software unit is used throughout.

#### Unit testing Cocoa projects in Xcode
Unlike Bill, whose reference to unit testing in Xcode 3.0 is linked at the title, when I started writing unit tests for my Cocoa projects I had no experience of testing in any other environment (well, OK, I'd used OCUnit on GNUstep, but I decline to consider that as a separate environment). However, what I've seen of unit testing in Cocoa still makes me think I must be missing something.The first thing is that when people such as Kent Beck talk about test-driven development, they mention "red-green-refactor". Well, where's my huge red bar? Actually, I sometimes write good code so I'd like to see my huge green bar too, but Xcode 3.1 doesn't have one of those either. You have to grub through the build results window to see what happened.Sometimes, a test is just so badly broken that rather than just failing, it crashes the test runner. This is a bit unfortunate, because it can be very hard to work out what test broke the harness. That's especially true if the issue is some surprising concurrency bug and one test breaks a different test, or if the test manages to destroy the assumptions made in -teardown and crashes the harness after it's run. Now Chris Hanson has posted a workaround to get the debugger working with a unit test bundle target, but wouldn't it be nice if that "just worked", in the same way that breaking into the debugger from Build and Run "just works" in an app target?

#### Unit testing Core Data-driven apps
Needless to say, I'm standing on the shoulders of giants here. Chris Hanson has written a great post on setting up the Core Data "stack" inside unit tests, Bill Bumgarner has written about their experiences unit-testing Core Data itself and PlayTank have an article about introspecting the object tree in a managed object model. I'm not going to rehash any of that, though I will touch on bits and pieces.In this post, I'm going to look at one of the patterns I've employed to create testable code in a Core Data application. I'm pretty sure that none of these patterns I'll be discussing is novel, however this series has the usual dual-purpose intention of maybe helping out other developers hoping to improve the coverage of the unit tests in their Core Data apps, and certainly helping me out later when I've forgotten what I did and why ;-).Pattern 1: remove the Core Data dependence. Taking the usual example of a Human Resources application, the code which determines the highest salary in any department cares about employees and their salaries. It does not care about NSManagedObject instances and their values for keys. So stop referring to them! Assuming the following initial, hypothetical code:

- (NSInteger)highestSalaryOfEmployees: (NSSet *)employees {    NSInteger highestSalary = -1; for (NSManagedObject *employee in employees) {        NSInteger thisSalary = [[employee valueForKey: @"salary"] integerValue];      if (thisSalary &gt; highestSalary) highestSalary = thisSalary;    } //note that if the set's empty, I'll return -1    return highestSalary;}

This is how this pattern works:

Create NSManagedObject subclasses for the entities.@interface GLEmployee : NSManagedObject{}@property (nonatomic, retain) NSString *name;@property (nonatomic, retain) NSNumber *salary;@property (nonatomic, retain) GLDepartment *department;@endThis step allows us to see that employees are objects (well, they are in many companies anyway) with a set of attributes. Additionally it allows us to use the compile-time checking for properties with the dot syntax, which isn't available in KVC where we can use any old nonsense as they key name. So go ahead and do that!- (NSInteger)highestSalaryOfEmployees: (NSSet *)employees {    NSInteger highestSalary = -1; for (GLEmployee *employee in employees) {     NSInteger thisSalary = [employee.salary integerValue];        if (thisSalary &gt; highestSalary) highestSalary = thisSalary;    } //note that if the set's empty, I'll return -1    return highestSalary;}Abstract out the interface to a protocol.@protocol GLEmployeeInterface &lt;NSObject&gt;@property (nonatomic, retain) NSNumber *salary;@endNote that I've only added the salary to the protocol definition, as that's the only property used by the code under test and the principle of YAGNI tells us not to add the other properties (yet). The protocol extends the NSObject protocol as a safety measure; lots of code expects objects which are subclasses of NSObject or adopt the protocol. And the corresponding change to the class definition:@interface GLEmployee : NSManagedObject &lt;GLEmployeeInterface&gt;{}...@endNow our code can depend on that interface instead of a particular class:- (NSInteger)highestSalaryOfEmployees: (NSSet *)employees {   NSInteger highestSalary = -1; for (id &lt;GLEmployeeInterface&gt; employee in employees) {      NSInteger thisSalary = [employee.salary integerValue];        if (thisSalary &gt; highestSalary) highestSalary = thisSalary;    } //note that if the set's empty, I'll return -1    return highestSalary;}Create a non-Core Data "mock" employeeAgain, YAGNI tells us not to add anything which isn't going to be used.@interface GLMockEmployee : NSObject &lt;GLEmployeeInterface&gt;{   NSNumber *salary;}@property (nonatomic, retain) NSNumber *salary;@end@implementation MockEmployee@synthesize salary;@endNote that because I refactored the code under test to handle classes which conform to the GLEmployeeInterface protocol rather than any particular class, this mock employee object is just as good as the Core Data entity as far as that method is concerned, so you can write tests using that mock class without needing to rely on a Core Data stack in the test driver. You've also separated the logic ("I want to know what the highest salary is") from the implementation of the model (Core Data).

OK, so now that you've written a bunch of tests to exercise that logic, it's time to safely refactor that for(in) loop to an exciting block implementation :-).

#### Unit testing Core Data-driven apps, fit the second
It took longer than I expected to follow up my previous article on unit testing and Core Data, but here it is.Note that the pattern presented last time, Remove the Core Data Dependence, is by far my preferred option. If a part of your code doesn't really depend on managed objects and suchlike, it shouldn't need them to be present just because it works with (or in) classes that do. The following pattern is recommended only when you aren't able to abstract away the Core Data-ness of the code under test.Pattern 2: construct an in-memory Core Data stack. The unit test classes you develop ought to have these, seemingly contradictory properties:

no dependence on external state: the tests must run the same way every time they run. That means that the environment for each test must be controlled exactly; dependence on "live" application support files, document files or the user defaults are all no-nos.close approximation to the application environment: you're interested in how your app runs, not how nice a unit test suite you can create.

To satisfy both of these properties simultaneously, construct a Core Data stack in the test suite which behaves in the same way but which does not use the persistent store (i.e. document files) used by the real app. My preference is to use the in-memory store type, so that every time it is created it is guaranteed to have no reference to any prior state (unlike a file-backed store type, where you have to rely on unlinking the document files and hoping there are no timing issues in the test framework which might cause two tests simultaneously to use the same file).My test case class interface looks like this (note that this is for a dependent test case bundle that gets embedded into the app; there's an important reason for that which I'll come to later). The managed object context will be needed in the test methods to insert new objects, I don't (yet) need any of the other objects to be visible inside the tests but the same objects must be used in -setUp and -tearDown.

#import &lt;SenTestingKit/SenTestingKit.h&gt;@interface SomeCoreDataTests : SenTestCase {    NSPersistentStoreCoordinator *coord;    NSManagedObjectContext *ctx;    NSManagedObjectModel *model;    NSPersistentStore *store;}@end

The environment for the tests is configured thus. I would have all of the error reporting done in tests, rather than that one lone assertion in -tearDown, because the SenTest framework doesn't report properly on assertion failures in that method or in -setUp. So the -testThatEnvironmentWorks test method is a bellwether for the test environment being properly set up, but obviously can't test the results of tear-down because the environment hasn't been torn down when it runs.

#import "TuneNeedsHighlightingTests.h"@implementation TuneNeedsHighlightingTests- (void)setUp{    model = [[NSManagedObjectModel mergedModelFromBundles: nil] retain];    NSLog(@"model: %@", model);    coord = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel: model];    store = [coord addPersistentStoreWithType: NSInMemoryStoreType                                configuration: nil                                          URL: nil                                      options: nil                                         error: NULL];    ctx = [[NSManagedObjectContext alloc] init];    [ctx setPersistentStoreCoordinator: coord];}- (void)tearDown{    [ctx release];    ctx = nil;    NSError *error = nil;    STAssertTrue([coord removePersistentStore: store error: &error],                  @"couldn't remove persistent store: %@", error);    store = nil;    [coord release];    coord = nil;    [model release];    model = nil;}- (void)testThatEnvironmentWorks{    STAssertNotNil(store, @"no persistent store");}@end

The important part is in setting up the managed object model. In using [NSManagedObjectModel mergedModelFromBundles: nil], we get the managed object model derived from loading all MOMs in the main bundle&mdash;remembering that this is an injected test framework, that's the application bundle. In other words the MOM is the same as that created by the app delegate. We get to use the in-memory store as a clean slate every time through, but otherwise the entity definitions and behaviours ought to be identical to those provided by the real app.

#### UNIX-like pedantry
Some people like to refer to OS X as UNIX-like when it's actually a UNIX. There was a time when it was UNIX-like and some people liked to refer to it as a UNIX, but it's not now.

#### Up or to the right
Sometimes in describing a concept X in one domain, someone will ask "oh, is that like X'?" where X' is the same concept or a very similar one, expressed in a different domain.

The quick answer is "Yes", but that permits a range of interpretations from "X and X' are the same things in different contexts" to "this domain that's new to me can be thought of by analogy to the thing I already know. For example, what I know as X' exists, except that they call it X".

One long answer is "There exists an idea called Meta-X. Both of these domains contain expressions of this idea, but one represents it as X and the other as X'". But now, possible interpretations include "Meta-X is pure and both X and X' are tainted", "Meta-X is some fundamental proposition about the universe", "Meta-X is some meaningless fiction invented by Graham to sound profound", "Meta-X is some attempt to conceptually combine two different real things".

So neither answer is "correct", and you can't present information in a way independent of those trying to make use of it.

#### Upcoming Cocoa nerd stuff
I have organised a NSCoder Night for this coming Tuesday, November 27.  It shall be in the Jericho Tavern pub at 8pm; bring yourself, bring an interest in Cocoa, and perhaps bring some code to talk about or work on.  There won't be any agenda as such, just a group of NSCoders talking about NSCoding.

In January, PaulHR and I shall be entertaining OxMUG on the subject of Getting Things Done�&#x2122; - in particular I have now started braindumping my many to-do lists into OmniFocus and I'm finding it very expressive and useful.  In fact preparing that talk has just zoomed its way over to my OF inbox :-).  That talk shall be Tuesday, January 8th.

#### Updates to JavaByContract
Some improvements to JavaByContract, the design-by-contract tool for Java:

Preconditions, Postconditions and Invariants now appear in the Javadoc for types that use JavaByContract. While this is only a small source change, it's a huge usability improvement, as programmers using your types can now read the contracts for those types in their documentation.
There is Javadoc for the JavaByContract package.
The error message on contract violation distinguishes between precondition, postcondition and invariant violation.

I'm speaking generally about moving beyond TDD, using JavaByContract as a specific example, at Coventry Tech Meetup next week. See you there!

#### Updating my ObjC web app on git push
I look at SignUp.woa running on my Ubuntu server, and it looks like this.

That title text doesn't quite look right.

$ open -a TextWrangler Main.wo/Main.html
$ make
$ make check
$ git add -A
$ git commit -m "Use new main page heading proposed by marketing"
$ git push server master

I refresh my browser.

The gsw-hooks README explains how this works. My thanks to Dan North for guiding me around a problem I had between my keyboard and my chair.

#### Use of vm_stat(1)
[N.B. I'll make heavy use of links to DarwinGrok in this post.  However, I make no comment on the availability of darwingrok; it's set up on a test server which is unsupported, to test how well said unsupported server handles a Java app deployment mechanism which is also currently unsupported.  It's been known to go down for days before I realise anything's wrong.  However, if it turns out to be unavailable, just look at the end of the URL to find out which file I've linked to, and look for that on darwinsource.  For sake of comparison between the two sites, darwingrok's source cache is based on 10.4.5.ppc.]So, it seems a bit strange to be talking about vm_stat(1) when I recently posted that I'd been working on an alternative to it.  But in doing so, as was rightly observed in the comments, I did learn a thing or two about the Mach VM manager and vm_stat is about the most verbose place in which it's encountered.  And besides, I had a request :-)Let's start by looking at some sample output:

Mach Virtual Memory Statistics: (page size of 4096 bytes)Pages free:                    18150.Pages active:                 129075.Pages inactive:                79964.Pages wired down:              34856."Translation faults":      216587934.Pages copy-on-write:         3932355.Pages zero filled:          85582532.Pages reactivated:           1712668.Pageins:                      474657.Pageouts:                      77330.Object cache: 478244 hits of 1592868 lookups (30% hit rate)

Just for comparison, here's the output of &quot;free -p&quot; shortly after that (those of you who haven't investigated free since my last blog post are in for a surprise):

           total        used        freeMem:      262045      244025       18020Swap:     262144      171028       91116

I'll start by looking at physical memory, because that's a whole topic in itself which I've spoken about so many times I can cover in a self-contained glob.  As an aside though, the pagesize is a compile-time setting, and is 4096 bytes on both architectures unless you go compiling your own kernel.  If you do, remember that it's got to be a power-of-2 multiple of the hardware page size.So, physical memory then.  Which of the various pages that we're being told about are actually in physical memory?  The active ones must be, and so must the wired ones (a page marked as wired may not be swapped, so by definition they're stored in physical RAM).  Actually it turns out that the free and inactive pages are both in RAM too; anything that's been handed out to a pager isn't counted as an available page (but the kernel knows where to ask for that page, if it needs it back).  There's no information on page counts outside physical RAM, which was why earlier versions of free(1) couldn't report on the available swap space.  It also happens not to make much sense to ask how much swap space is available, because it can change without user interaction.  Compare this with Linux: if you swapon(2) a 2GB partition and nothing else, then no matter how much swap space is currently in use, the kernel knows that there's 2GB of swap available.  Anyway, that's an aside.  Add up the number of free, active, inactive and wired pages, multiply by the page size and you should have a number familiar to you as the total amount of RAM installed in your Mac (or arbitrary other Mach OS box).Next question, what is the amount of unused RAM?  I get asked this frequently and my usual answer is: Heh :-).  The free tool just looks at free pages, and top(1) goes for this approach too.  Other people will say that it's the number of free and inactive pages.  And they're right too, I think.  Free pages are absolutely free, and have not been claimed by anything.  Inactive pages are candidates for reuse, but do actually contain data.  An example of when pages might become inactive is that a task is launched and loads a few dynamic libraries in, then quits.  It'd be handy to keep those dylibs around just in case (try launching an app like OmniOutliner, quitting, and relaunching.  Even on a poor man's metric like # of dock bounces, it was faster the second time, right?) but if the memory needs to be claimed elsewhere, it can be.  So the amount of unused RAM is a nontrivial quantity, but if you mean &quot;how much RAM is free&quot; the answer is the free pages.  If you mean &quot;how much more RAM could I activate&quot; then it's that in free or inactive pages.  You'll have real performance problems if you've got few free pages and and lots of wired pages, because the wired pages can't go anywhere you've got that much memory less for user-space processes to swap in.Okey dokey, let's have a look at the paging statistics next then.  All of the following values are cumulative; they start at 0 at boot time and monotonically increase with (up)time.The state of a page is known to the kernel in terms of a few variables:

EmptyWiredDirt (i.e. has the page been modified)Current access privileges (none, read, or read/write)Desired access privileges

There are lots of different mechanisms by which the state as described above can be modified, and each change of state triggers at least one &quot;translation fault&quot;.  These faults are handled by vm_fault(), so every time that's called the translation fault count goes up.When a new Mach task is generated, it has an inheritance property which describes whether it starts with its own memory objects or whether it receives copies from its parent (as would happen with a fork(2) call).  But rather than generating all those copies when the task is launched, the memory manager gives it shadow objects which refer to the original memory objects.  The task doesn't receive its own copy until it tries to write to the shadow object, at which point that generates a copy-on-write fault and the local version of the memory is finally generated.&quot;Pages zero-filled&quot; describes exactly its own purpose, I hope.A reactivated page is one that shifts from being inactive to active, as in the example of re-launching an application given above.  Reactivating pages means less going out to disk to recover the data, which means faster performance.  Finally, a pagein occurs whenever a new page is requested from the pager (basically, whenever a peek or poke into memory that has not yet been dealt with occurs) and a pageout occurs whenever a page has to be handed off to a pager to allow another page to be created or paged back in.The problem with all of the above information is that it's not presented in a way which I'd find useful :-).  I'd prefer my RAM statistics to be presented in more comfortable memory units such as megabytes; this is analogous to the default output of df(1) being the 512-byte block.  I don't care about most of the other VM statistics at all, because they're only important if you're debugging the memory manager or a pager.  Rather than the number of pageouts since I switched the computer on, I'd like to know the number of pages currently out (i.e. the amount of swap use).  There's a sysctl &quot;vm.swapusage&quot; which reports on swap usage, which I've used in free.  The best way to get at swap information would be to interrogate the pager(s), not the kernel, and looking at Apple's I don't think there's currently a way to achieve that.  But then there are a few things the dynamic_pager would do if I were world emperor that it currently can't; reporting how far off a high-water alert is, cleanly turning off paging to a particular directory, maintaining a single table of all the files currently in use by all the dynamic_pager instances.  Anyway, remember what I said earlier about the amount of swap being changable; all we get from vm.swapusage is the amount of filesystem which is currently dedicated to swap.One thing which can be achieved with the available information, and which vm_stat doesn't report (neither does free) but top(1) does, and which is incredibly useful, is the differential with respect to time.  Go on, fire up top.  Where there are the pagein and pageout figures, the little number in brackets tells you the increase since the last report.  Now a graph of that over time would tell you what effect the system utilisation was having on the memory utilisation.  That'd be sweet.The final line of vm_stat's output is the object cache hit rate, which is the ratio of objects requested vs. those which were in active memory at time of request.  Obviously, the higher the ratio, the fewer reactivations and pageins.  On a bunch of different Macs I have access to, the hit rate is between 2% and 30%.BTW, would you like some references?  Here's the OSF Mach documentation (the Principles manual contains a good chapter on virtual memory management), Apple's Kernel Programming guide, and the paper describing Mach's virtual memory management by the guys at CMU.Edit 2006-06-18 09:07 GMT: stopped claiming that free doesn't know about swap space.  It does.

#### Using Aspect-Oriented Programming for Security Engineering
This paper by Kotrappa Sirbi and Prakash Jayanth Kulkarni (link goes to HTML abstract, full text PDF is free) discusses implementation of an application's security requirements in Java using Aspect-Oriented Programming (AOP).

We have AOP for Objective-C (of sorts), but as hardly anyone has used it I think it's worth taking a paragraph or two out to explain. If you've ever written a non-trivial Cocoa[ Touch] application, you'll have found that even when you have a good object-oriented design, you have code that addresses different concerns mixed together. A method that performs some useful task (deleting a document, for example) also calls some logging code, checks for authorisation, reports errors, and maybe some other things. Let's define the main business concerns of the application as, well, business concerns, and all of these other things like logging and access control as cross-cutting concerns.

AOP is an attempt to reduce the coupling between business and cross-cutting code by introducing aspects. An aspect is a particular cross-cutting concern that must be implemented in an application, such as logging. Rather than calling the logging code from the business code, we'll define join points, which are locations in the business code where it's useful to insert cross-cutting code. These join points are usually method calls, but could also be exception throw points or anywhere else that program control changes. We don't necessarily need logging at every join point, so we also define predicates that say which join points are useful for logging. Whenever one of the matching join points is met, the application will run the logging code.

This isn't just useful for adding code. Your aspect code can also define whether the business code actually gets run at all, and can even inspect and modify the return value of the business code. That's where it gets useful for security purposes. You don't need to put your access control (for instance) checks inside the business code, you can define them as modifications to join points in the business code. If you need to change the way access control works (say going from a single-user to directory service scheme, or from password checks to access tokens) you can just change the implementation of the aspect rather than diving through all of the app code.

Of course, that doesn't mean you can just implement the business logic then add security stuff later, like icing a cake or sprinkling fairy dust. You still need to design the business objects such that the security control decisions and the join points occur at the same places. However, AOP is useful for separating concerns, and for improving maintainability of non-core app behaviour.

#### Using GNUstep libraries with Xcode
I was recently asked about building projects that use GNUstep from Xcode. The fact is, it's incredibly easy.

By default, GNUstep on Mac OS X installs its libraries to /usr/local/lib and its frameworks to /Library/Frameworks. Therefore if you want to include GNUstep-base additions, you just hit the + button in your target's "link binary with libraries" section and find the libgnustep-baseadd.dylib entry under Mac OS X 10.7. If you wanted to use GNUstepWeb, you'd look for WebObjects.framework in the same list.

You can get access to the GNUstep base additions in your code by including &lt;GNUstepBase/GNUStep.h.

Notice that if you manage a GNUstep project using Xcode, you'll only be able to build it on a Mac (unless you go to the bother of writing a build tool with GNUstep's unfortunately-named XCode.framework - which ironically doesn't currently work on Mac OS X.). If you need to target non-Mac platforms, your options are to build Cocotron-style cross-compilers and add them to your Xcode project, or to create an Xcode project with an External Build System target and manage your build via make anyway.

An interesting extension would be to define a new filesystem layout for GNUstep-make that deposited all of the frameworks and libraries into a .sdk folder that could be used in Xcode as an additional SDK.

#### Using Objective-C on the server
My talk at NSConf was about cross-platform Objective-C. Those people who I talked to after the session will know that my goal is clear. There are plenty of people out there who have learned Objective-C to get onto the iOS or Mac app stores. Those people need some server-side smarts in their apps, and want to benefit from the skills they've already acquired. In other words: they want Objective-C on the server.

I'm now in a position to offer that :). I took a look at the GNUstepWeb project, a reimplementation of WebObjects 4.5. It was in a fairly parlous state, and wouldn't even build on my Mac. I've fixed that. It's still a bit of a faff, but we're in a position to start playing with server-side ObjC and see whether there's any value in having a better product.

Building GSW on the Mac

GNUstep-make
First, you need to grab GNUstep from SVN. Now you need to compile GNUstep-make, the build system. On Mac OS X you do the following:

cd modules/core/make
./configure --with-library-combo=apple-apple-apple
make
make install

The weird configure invocation tells GNUstep you're going to use the Apple Objective-C runtime, and the Foundation (and AppKit) libraries instead of GNU replacements.

GNUstep-make includes a shell script that defines the locations of various folders. You should import that into your shell environment:

. /Library/GNUstep/Makefiles/GNUstep.sh

Consider adding that line to your .bash_profile.

GNUstep-base
Then you build the GNUstep-base extensions, some additional classes and macros for working with Foundation code.

cd modules/core/base
./configure
make
make install

GDL2
GDL2 is the GNUstep Database Library; an EOF-like framework for interacting with databases. If you're going to use a DBMS like MySQL for your web apps, you should install that and its developer headers before doing:

cd modules/dev-libs/gdl2
./configure --disable-gorm-palette
make
make install


(Aside: GORM is the GNUstep version of Interface Builder. You don't need the plugin for working with entities in GORM, and you can't build it on Mac OS X so that's why it's disabled in the configure invocation.)

GNUstepWeb
Now here's the good stuff. Build and install GSW. At time of writing, there's an open bug that stops it building on Mac OS X, so you'll need to grab the patch from this bug report. Apply that patch, then:

cd modules/dev-libs/gsweb
./configure
make
make install

A sample app

You probably want to see a sample GSW app, so it's a good job I wrote one :-). Clone the HelloGSW project on GitHub. I'll show you how to build and run that before we get into how it works.

Building HelloGSW

cd HelloGSW
make

Running HelloGSW

HelloGSW.gswa/Contents/MacOS/HelloGSW

this will spit a load of output to the console, including a URL that looks like http://localhost:9001/WebObjects/HelloGSW.woa/0/. Open that URL in your browser. You should see this:



HelloGSW code walkthrough

The interesting code is in the Main class, which is a subclass of GSWComponent (and conventionally the default component to show for a new visitor). Each "page" in a GSW app is a component, though of course you could arrange to return XML, JSON or any other format rather than HTML. HTML is the default and the easiest to work with, and indeed we'll start by looking at a section of the HTML file, Main.wo/Main.html.

 &lt;p&gt;The time is &lt;webobject name=now_string&gt;&lt;/webobject&gt;.&lt;/p&gt;

This is clearly the paragraph that puts the date onto the screen, but you can see that there's this weird &lt;webobject&gt; tag in there. That's going to be substituted with our date object at runtime, but how? To answer this question, look at the Main.wo/Main.wod file.

now_string:WOString {value = now}

This file contains bindings, which if you're familiar with Cocoa work pretty similar to Cocoa Bindings. This tells us that there's a web object called now_string, that it's an instance of WOString, and that its value binding is bound to something called now.

That now binding is actually a key on the component object. We might expect to see a method called -(NSString *)now defined on Main to fill in the value of this string. Indeed, if we look at Main.m:

- (NSString *)now {
  NSDate *theDate = [NSDate date];
  return [theDate description];
}


Where to go from here?

The start is to build more GSW apps, and of more complexity. However there are things that need to be done along the way: all of the rest of the code and files in this project are meaningless boilerplate. This stuff needs to be hidden, automated or removed. Then we can also automate the process: compare how much of a faff setting up this app was with running a Rails app in Heroku (and I haven't even set up an adaptor to a real web server!). It'd be really useful to simplify common tasks in modern web applications, such as exposing REST interfaces and non-HTML document formats.

Conclusions

Objective-C is not just a client app language. It's got a long heritage as a server technology, but unfortunately the tools for doing that are quite out of date. They're still useful though, so people who have come into the Objective-C fold recently can expand the use of their newly-found skills. Hopefully people will adopt&mdash;and contribute to&mdash;GNUstepWeb and help everyone to make it a useful modern server technology.

#### UX is snake-oil bullshit
There, I said it. I feel better already. There are people in the world who'll tell you that the most important thing in the world is UX, that if your software isn't UX-compliant it isn't worth shit. Here's why that's wrong.

Platitudes

"Consider the user." "The user is king." Yeah? Well why do you name your vaunted "user" after a fucking drug addict? You're lying, because you know no other life. The user is the poor schmuck that you're trying to push your apps on, just like a crack dealer. Only of course you don't call it "crack dealing", you call it "freemium" and act like it's some kind of fucking service to let your crackheads - sorry, users - "get familiar" with the product before you start bleeding their wallets.

Besides, what does it even mean? Consider the user. I looked at my software, and I think users should be grateful that I let them even fucking touch the app store page with their filthy uneducated hands. There, I've "considered" the cretins, so I'm doing UX right am I?

The only examples are bad examples

Ever seen a UX talk at a software dev conference? I'd be surprised if you haven't, because these vultures who call themselves "UX consultants" are actually professional speakers who love nothing better than to tell us hard-working developers how we do everything wrong. You'll be able to spot them if you're at a conference, they're the people with the designer jeans and the designer wine glasses and the fucking designer nostril hair. They'll open their talk with some rant about how the starving children in Africa can wait for help until someone's fixed the way the fucking tap works in the shower of their six-star hotel room they were in when they gave the same talk in Dubai or some other place us lowlife software engineers can't afford to visit. You'll get the impression that we're the Morlocks, and this fucking Eloi has deigned to come down from the surface to remind us how much better life is up there, and only wants £2000+expenses to do it.

And then what's the talk about? It's a series of examples of what they say is really bad UX, that we're supposed to sit there and laugh at with them? Fuck that shit. Some colleague of mine has been sitting in a cubicle busting a gut to produce this piece of software, probably with some manager riding their ass and a bunch of conflicting requirements coming from the douchebags in marketing and sales and business analysis and all those other people who don't know how the fuck a computer works but think their opinions are somehow valid. And this engineer somehow performs the miracle of reconciling all these different inputs, making everybody happy and at the same time writing this gnarly piece of software. You want me to laugh at that? Whoever that is should be given a medal.

The best bit is of course that these examples are completely worthless. My product doesn't make people choose which of the fifty states they're in even when they don't live in the US. It isn't some bridge somewhere that projects the cock-signal onto itself at sunset. It isn't some medical device that kills patients when the nurse forgets to press a button. That means I'm doing better than all the examples, which means I'm doing UX right. Right?

Capriciousness

If you manage to get past the fluff talk and the bad examples and trap a UX person into asking what a good example is, you'll get a different answer every sodding week. Last week they were all telling us how interfaces should be discoverable and how you should have UI elements with clear actions like buttons and things. This week it's all about these completely arcane and undiscoverable gestures; I mean what the fuck? The last twitter app I downloaded made me page through about ten pages of user manual before it even let me send a fucking tweet, which is the whole point of the piece of crap. You're supposed to swipe right with three fingers making the shape of the Eye of Horus or some bullshit to retweet, or something like that. The fact is I didn't read the manual (sorry, "soft landing" as the hipsters want me to call it) so I've got no fucking clue how to do even the simplest of things.

And that's just this week's UX hotness. Next week it'll all be Jordi LaForge visors or some crap, because that went really well for that Virtuality company back in the 1990s.

Which brings me on to the next way to win at UX without actually doing any work. Build whatever shitty UI you want, and just wait for the UX consultant circlejerk to decide that the way you did it is the way all apps should do it. You'll only need to wait a fortnight, maybe a month tops.

Everything - or anything - is UX

The best thing is that you too can be in on this party! Whatever you know now, whatever you're an expert in, you can claim is part of the user experience. You know OpenCL? That's about making things fast and responsive, therefore you're a UX consultant! You know Core Audio? Making machines go "ping" when something goes wrong is part of the user experience! You're a project manager? Fuck that, you're a User Experience Coordinator!

And so the final way in which I'm winning at UX without really trying is that whatever the fuck it is I do is user experience. Nobody wants to be hacked, right? So security contributes to the....experience...of the...crack addictuser, right?

Enough of this crap.

It's time to acknowledge that UX is a complete load of snake oil, and that its biggest contribution to society has been to reduce unemployment among people who think £150 is a modest amount to pay for a shirt. It's time to show that us engineers can make software without their help, just as we did both before they came along and indeed while they were swanning around being better than us.

Therefore I introduce my latest initiative, the Clueless Losers are Inexperienced paradigm for software design. CLI harks back to the days when we knew that computers are tricky and software is hard, and we didn't apologise for it or pay consultants to apologise for us. It acknowledges that software is hard to write, so it'd sure as hell be hard to use and you should all be damned grateful if we allow you to use it at all.

There won't be expensive hands-on labs or conference talks about how to make CLI apps. Just do whatever it is you need to get the software working. We acknowledge that people are inexperienced and clueless when it comes to software, so there's no point going out of our way to make things easier for them because it just makes it easier for them to mess it up.

Oh, and I fixed that crack pusher bullshit that's been going on. We're not going to call people "users" like they're some farm of addicts waiting for their next hit. The word "loser" doesn't have those pejorative connotations, so in CLI that's what we call people who interact with our software.

I'm really excited about the CLI, and about heralding the start of the post-UX-bubble software economy. I hope you'll all join me in making software as complicated as it deserves to be.

#### Verify your backups
Apple shipped Mac OS X 10.5 this weekend, and three of the features are Time Machine, dtrace, and improved CHUD tools.  Time Machine, dtrace, CHUD tools.  iPod, mobile phone, web browser.  Time Machine, dtrace, CHUD tools.

To spell that out in long hand, it's very easy now to see how various features in the Operating System behave.  And in the case of Time Machine, we see that it walks through the source file system, copying the files to the destination.  When I last gave a talk to OxMUG on the subject of data availability, it was interesting to notice how the people who had smugly put their hands up to indicate that they performed regular backups became crestfallen when I asked the second question: and how many of you have tested that backup in the last month?

Time Machine is no different in this regard.  It makes copies of files, and that's all it does.  It doesn't check that what it wrote at the other end matches what it saw in the first place, just like most other backup software doesn't.  If the Carbon library reports that a file was successfully written to the destination, then it happily carries on to the next file.  Just like any other backup software, you need to satisfy yourself that the backup Time Machine created is actually useful for some purpose.

#### Vibe coding and BASIC
In Vibe Coding: What is it Good For? Absolutely Nothing (Sorry, Linus), The Register makes a comparison between vibe coding today and the BASIC programming of the first generation of home microcomputers:

In one respect, though, vibe coding does have an attractive attribute that is difficult to find elsewhere. For the first time since instant-on home computers that fell straight into a BASIC interpreter, it's possible to make things happen with a bit of typing as a very naive user.

A photograph of the back cover of issue 7 of Input magazine, showing that PEEK and POKE are coming in issue 8.

A big difference between the two scenarios is that on an early micro, you had to use BASIC to get anything else done, in many situations. The computer I used was a Dragon 32 and unless you had a (very expensive) game or package that came on a ROM cartridge, even loading software from cassette required a BASIC command.

Actually it was one of two BASIC commands: you typed CLOAD to load a basic program, or CLOADM to load a machine-language program. Either way, the default behaviour of the computer was to load its BASIC interpreter from ROM and wait for input, that input being in the form of lines of BASIC.

The context of those two commands indicates a big similarity between BASIC and vibe coding: to get particularly far, you needed some knowledge of the workings of the rest of the computer. In this case, you needed to know whether your cassette tape contained a BASIC program or a machine-language program, but that isn’t the most egregious example.

As I said, the computer I used at the time was a Dragon 32, which was a kind of not-clone of the Tandy Color Computer. Let’s say that I wanted to write a game, which is entirely plausible because I both wanted to, and did, write many games. How do I read the joystick direction and fire button status in BASIC, so I can add joystick control to my games?

Direction is relatively easy. There are two potentiometer-based joysticks, right and left, and each has an x and a y axis. The JOYSTK function takes an index which identifies the axis to read, and returns a value from 0 to 63 indicating the extent of travel. For example, JOYSTK(0) is the x axis of the right joystick.

To get the fire button status, I use this operation: the Peripheral Interface Adapter’s side A data register is mapped to location &amp;HFF00 in memory. The right joystick’s fire button unsets bit 0 when it’s pressed, and sets it when it isn’t. The left joystick’s fire button unsets bit 1 when it’s pressed, and sets it when it isn’t. For example, P = PEEK(&amp;HFF00) : F = NOT (P AND 1) sets F to 1if the right fire button is pressed.

Yes, I’m still technically BASIC coding, but I’ve learned a lot about the architecture of the machine and applied advanced mathematical concepts (I was single-digit years old) to improve my BASIC-coded software.

Much more importantly, I’ve become excited to understand this stuff and apply it to the programs I write, and I’m enthusiastic to share those programs and find other people who want to share their programs. There’s an inert plastic box plugged into an old TV in the otherwise-unused middle room of the house, and I can make it do what I want.

The BASIC-haters pooh-poohed that notion: OK you can make your little toys, but real software is written in machine code. Citation needed I’m sure, but I suspect that the advent of Visual BASIC meant that far more real software was being written in BASIC than in machine code even by the 1990s, a decade in which that first generation of micros was redundant but not quite obsolete (the last Commodore 64 was sold in April 1994, because Commodore went bankrupt).

However, the people who learned BASIC just picked up other tools and continued the journey they had been inspired to set out on. I live in the UK and many of the professional programmers I meet, when they’re between about ten years older than me and three years younger than me, are in that BASIC generation. We typed programs on our micros, we learned how the computers worked, and then we transferred that understanding and that exploration to other systems and other programming languages. We watched TRON, and learned to fight for the users: being scared by computers because they refuse to open the pod bay doors was for our parents.

If vibe coding gives this generation that same sense of wonder and empowerment, of being able to control a device that has hitherto only done what other people charge you to do, if it starts that same journey of learning and applying skills, and of sharing that knowledge, then it really doesn’t matter whether you think it’s OK for real software. It really doesn’t matter whether it ends up being a great tool or not.

#### Video podcast: Hisoft C for the ZX Spectrum
Episode 6 of the SICPers podcast is over on Youtube. I introduce a C compiler for the Sinclair ZX Spectrum. For American readers, that's the Timex Sinclair TS2068.

#### Walking a mile dans ses chausseurs
The word 'translator' has an interesting history.  In the Anglo-Saxon language, 'wealhstod' meant "learned in Welsh" more or less, and described someone who could parlay with the important members of the local British tribes.  As is often the case with invasions the British started to use the word, so the Welsh title 'Gwalstawt' means "interpreter of tongues", i.e. the Welsh word for "can speak another language" originally meant "can speak Welsh" (there's another word more closely related to Breton treiñ or Cornish trélya in Welsh, too; trosi).

Anyway, to see what localisation people go through during the l10n process, I decided the best thing to do was to try it myself.  To save the time it would have taken to write an internationalised app, I used someone else's; namely TextEdit.  Here's the result after about 90 minutes of work:

The first thing to notice is that I haven't actually got much done yet.  I've started working on the main menu NIB file (Edit.nib), and I'm about halfway through that.  At this rate, it would take me at least a (working) day to finish - granted I'm no expert at the task, so I'm having to make a more heroic effort on otherwise "standard" translations than most localisers would.  Although I do have a glossary to help.  Even so, TextEdit is a fairly simple app; it's easy to see that even if the translation became a mechanical process, translating a complex program would take a long time.

The other thing you might have noticed is that Mac OS X doesn't actually support Old English, and yet that's the language of my translation.  There's a simple trick here; convince Mac OS X that it does support Old English ;-).  Type this command in the Terminal:

$ defaults write NSGlobalDomain AppleLanguages '(ang, en, /* other languages */)'

and Robert, as they say, is your father's brother.  Apps will now look for localised resources in 'ang.lproj' when they start, so that's where your Old English resources live.

#### Want to hire iamleeg?
Well, that was fun. For nearly a year I've been running Fuzzy Aliens, a consultancy for app developers to help get security and privacy requirements correct, reducing the burden on the users. This came after a year of doing the same as a contractor, and a longer period of helping out via conference talks, a book on the topic, podcasts and so on. I've even been helping the public to understand the computer industry at museums.

Everything changes, and Fuzzy Aliens is no exception. While I've been able to help out on some really interesting projects, for everyone from indies to banks, there hasn't been enough of this work to pay the bills. I've spoken with a number of people on this, and have heard a range of opinions on why there isn't enough mobile app security work out there to keep one person employed. My own reflection on the year leads me to these items as the main culprits:

There isn't a high risk to developers associated with getting security wrong;
Avoiding certain behaviour to improve security can mean losing out to competitors who don't feel the need to comply;
The changes I want to make to the industry won't come from a one-person company; and
I haven't done a great job of communicating the benefits of app security to potential customers.

Some people say things like Fuzzy Aliens is "too early", or that the industry "isn't ready" for such a service: those are actually indications that I haven't made the industry ready: in other words, that I didn't get the advantages across well enough. Anyway, the end results are that I can and will learn from Fuzzy Aliens, and that I still want to make the world a better place. I will be doing so through the medium of salaried employment. In other words, you can give me a job (assuming you want to). The timeline is this:

The next month or so will be my Time of Searching. If you think you might want to hire me, get in touch and arrange an interview during August or early September.
Next will come my Time of Changing. Fuzzy Aliens will still offer consultancy for a very short while, so if you have been sitting on the fence about getting some security expertise on your app, now is the time to do it. But this will be when I research the things I'll need to know for…
whatever it is that comes next.

What do I want to do?

Well, of course my main areas of experience are in applications and tools for UNIX platforms—particularly Mac OS X and iOS—and application security, and I plan to continue in that field. A former manager of mine described me thus on LinkedIn:

Graham is so much more than a highly competent software engineer. A restless "information scout" - finding it, making sense of it, bearing on a problem at hand or forging a compelling vision. Able to move effortlessly between "big picture" and an obscure detail. Highly capable relationships builder, engaging speaker and persuasive technology evangelist. Extremely fast learner. Able to use all those qualities very effectively to achieve ambitious goals.

Those skills can best be applied strategically I think: so it's time to become a senior/chief technologist, technology evangelist, technical strategy officer or developer manager. That's the kind of thing I'll be looking for, or for an opportunity that can lead to it. I want to spend an appreciable amount of time supporting a product or community that's worth supporting: much as I've been doing for the last few years with the Cocoa community.

Training and mentoring would also be good things for me to do, I think. My video training course on unit testing seems to have been well-received, and of course I spent a lot of my consulting time on helping developers, project managers and C*Os to understand security issues in terms relevant to their needs.

Where do I want to do it?

Location is somewhat important, though obviously with a couple of years' experience at telecommuting I'm comfortable with remote working too. The roles I've described above, which depend as much on relationships as on sitting at a computer, may be best suited by split working.

My first choice preference for the location of my desk is a large subset of the south of the UK, bounded by Weston-Super-Mare and Lyme Regis to the west, Gloucester and Oxford to the north, Reading and Chichester to the east and the water to the south (though not the Solent: IoW is fine). Notice that London is outside this area: having worked for both employers and clients in the big smoke, I would rather not be in that city every day for any appreciable amount of time.

I'd be willing to entertain relocation elsewhere in Europe for a really cool opportunity. Preferably somewhere with a Germanic language because I can understand those (including, if push comes to shove, Icelandic and Faroese). Amsterdam, Stockholm and Dublin would all be cool. The States? No: I couldn't commit to living over there for more than a short amount of time.

Who will you do it for?

That part is still open: it could be you. I would work for commercial, charity or government/academic sectors, but I have this restriction: you must not just be another contract app development agency/studio. You must be doing what you do because you think it's worth doing, because that's the standard I hold myself to. And charging marketing departments to slap their logo onto a UITableView displaying their blog's RSS feed is not worth doing.

That's why I'm not just falling back on contract iOS app development: it's not very satisfying. I'd rather be paid enough to live doing something great, than make loads of money on asinine and unimportant contracts. Also, I'd rather work with other cool and motivated people, and that's hard to do when you change project every couple of months.

So you're doing something you believe in, and as long as you can convince me it's worth believing in and will be interesting to do, and you know where to find me, then perhaps I'll help you to do it. Look at my CV, then as I said before, e-mail me and we'll sort out an interview.

I expect my reward to be dependent on how successful I make the product or community I'm supporting: it's how I'll be measuring myself so I hope it's how you will be too. Of course, we all know that stock and options schemes are bullshit unless the stock is actually tradeable, so I'll be bearing that in mind.

Some miscellaneous stuff

Here's some things I'm looking for, either to embrace or avoid, that don't quite fit in to the story above but are perhaps interesting to relate.

Things I've never done, but would
These aren't necessarily things my next job must have, and aren't all even work-related, but are things that I would take the opportunity to do.
Give a talk to an audience of more than 1,000 people.
Work in a field on a farm. Preferably in control of a tractor.
Write a whole application without using any accessors.
Ride a Harley-Davidson along the Californian coast.
Move the IT security industry away from throwing completed and deployed products at vulnerability testers, and towards understanding security as an appropriately-prioritised implicit customer requirement.
Have direct reports.

Things I don't like
These are the things I would try to avoid.
"Rock star" developers, and companies who hire them.
Development teams organised in silos.
Technology holy wars.
Celery. Seriously, I hate celery.

OK, but first we like to Google our prospective hires.
I'll save you the trouble.

#### Warsaw Welcomes Dumbass Commentary
As I'm going to MCE tomorrow, tonight I'm going to my first WWDC keynote event since 2015. I doubt it'll quite meet the high note of "dissecting" software design issues in the sports lounge at Moscone with Daniel Steinberg and Bill Dudney, but it's always better with friends.

As I mentioned recently, almost everything I use on a Mac is a cross-platform application, or a LinuxKit container. I find it much quicker to write a GUI in React with my one year of experience than Cocoa with my couple of decades of experience. Rather than making stuff more featured, Apple need to make it relevant.

#### We shall return one day
On this day 80 years ago, 16th November 1943, the villagers of Tyneham near Lulworth was evacuated to allow Allied military forces to prepare for D-Day. Despite promises that the evacuation was temporary, the UK lurched directly from the second world war into the Cold War and decided to keep the land to practice against the new “enemy” and former ally, the Soviet Union. Tyneham remains uninhabited, and remains within a live firing range. People may only visit when the Ministry of Defence are ready for them.

In a time when people are still being displaced by war across the world, we remember the villagers of Tyneham, and an occasion when the country displaced its own citizens. The ten tracks on this album contain music, song, and storytelling from around Dorset. With no voices left in Tyneham, all parts are performed by the same person, but throughout we hear the message from the locals: “We shall return one day”.

Listen here: https://soundcloud.com/user-343604096/sets/we-shall-return-one-day

#### We! Haven't! Thought! This! Through!
So almost the entire world has exploded with news that Microsoft haven't bought Yahoo!, but are at least waving ridiculous amounts of TEH CAHS under the noses of the shareholders.  But what would be the outcome of such a takeover?  Microsoft and Yahoo! are in pretty much the same position, they both have technologically acceptable "online presence" services which are marketed really badly such that a competitor virtually owns the market.  The only example of a service between them I can think of which is vaguely market-leading is Flickr, although Yahoo! Groups (which was eGroups) might still be popular, I expect that Facebook and Google groups are at least comparable, if not larger than it.  Hotmail and Yahoo! Mail are both familiar names, but then online mail is now a free and worthless commodity, and everyone has been playing catchup with Gmail for the last couple of years.  People don't "Windows Live Search" a thing, they "Google" it.  People don't "Yahoo! Video" their dogs on skateboards, they "Youtube" them.So the post-acquisition world would go from two implementations to one, but one that's still being marketed into the ground and with a few fewer workers.  And all the transitional pain that Microsoft will impose on Yahoo! services, when someone remembers that FreeBSD isn't a Microsoft solution.  M$ seem to be of the opinion that with a market they don't lead and $45Bn, the best approach is to lose the $45Bn and hire some other people who don't lead the market.  Not, like, take that $45Bn and make their stuff better.

#### We're off to see the Wozzard....
So, they said that Steve Wozniak would be there, and that he'd be signing books.  So I took my book to be signed.  That wasn't wrong, was it?

N.B.: yes, I bought a copy of iWoz and had that signed too.  But the iBook is cooler ;-)

#### Website relaunch!
Today I have re-launched Thaes Ofereode to focus on my new role as an independent Mac boffin. I really like the new design, which was created by the ever-delightful Freya.edit: Gecko doesn't understand the CSS media selector I was using to provide the iPhone CSS. I've therefore reverted the iPhone design until I can find a way to get Firefox to suck less.The one thing I added to her design was a more iPhone-friendly look. For those of you without iPhones, the screenshots demonstrate how the mobile version will appear. For those of you who are CSS experts, the following will probably be rather dull but for those like me who know enough to be dangerous but no more, here's how it's done.The three-column layout works really well on the desktop, but the iPhone has a tallscreen-oriented display so not much space for horizontal layout. I therefore chose to put the leftmost, menu column underneath the main content on each page, so iPhone users get to see the heading and then the meat and potatoes. If they are interested enough to get to the end, they'll see the links to the rest of the site.The links, btw, are just paragraphs with a border, a lot of padding and the magic -webkit-border-radius providing the roundy edges; no messing with JavaScript and funny part-circle images.So, the third column? Well those impressive-looking widgets can't be displayed on the phone anyway, and would be a bit out of place so they're gone for the moment with the mobile CSS. I may code up some JavaScript replacements soon enough, but I'll need to find somewhere else for them to go. In the meantime, I know you read my blog because you're here, and there are many apps which can help you follow me on Twitter.

#### Week five
"You look so much happier!"

I get the best compliments. Also, I feel so much happier. I have put people, friendships, connections, and experiences first, and am taking advantage of the rewards.

One such experience was a visit a couple of further education (16-18 years old) computing classes in my county. I was mostly there to talk about my background in the industry and help them to understand what jobs there are and what employers look for. I was blown away by the things these students could make in their robotics classes though, with Mindstorms kits talking to laptops and mobile apps.

That was a huge step up from the turtle robots we had shuffling around the floor. in my school. And it still feels like there's even more opportunity there, like there's a huge gap between what a student sees computers being capable of and what they can do with the tools we (the industry) give them.

#### Week four
Pragma conf was a lot of fun! I met loads of new and old friends, which led to meaningful conversations about what we do, what we sometimes feel we should do, and what we want to do.

One such conversation, with Chris Eidhof, was about how we think about programming. It was brought to mind again recently by Derek Jones's post on Lisp and functional programming languages. His description of the Lisp community reminds me a lot of my own post on the tankard brigade, in that what keeps Lisp going is the exclusivity, and the need to know arcane rules to get things done when non-purists have much easier means to the same end.

What intrigues me about that is that it puts me into the tankard brigade, which gives me a lot to reflect on. When I look at Lisp I see a parsimony, a lack of arbitrary rules. It looks like there's a single underlying metaphor, and everything in my solution can be expressed through that metaphor (no matter how I solved the problem). I look at Lisp and see functions. I look at Io or Self and see messages. But when I look at supposedly more accessible languages I see a bag of edge cases and special "oh, you want to do that? No, you need one of these" conditions, with no metaphor.

#### Week One
Nearly eight days ago I stopped working to have a break. I've been describing it as a "gap year", because I've arranged my finances to last at least that long with some contingency. Also, I want to set a year as the anchor in my mind, so I don't do what normally happens and take the first interesting-looking job that comes along. There's a danger that I'll be bored around a month from now and start interviewing again.

Honestly, after one week I feel better rested but not like I've made some fundamental life change. That's partly because one of my first projects for this year is to complete an MSc in software engineering, so I'm still "a programmer" by trade to some extent. One goal for this year is to experience more of humanity than just programming.

I've taken some time out to do that which can be described (with capital letters, no less) as The Arts: visiting the Birmingham Museum and Art Gallery and the Library of Birmingham. And I turned my hand to graphic design to lay out a new "business" card, which I hope I'll have to hand next week. I'm speaking at #pragmaconf in October and people there might want to know who I am.

In literature news, Goodreads tells me I read Snow Crash, Emotional Design, a Philip K Dick anthology and I started on the Salmon of Doubt. Add to that this month's Linux Voice, Linux Format and CACM. A lot of reading, but things that programmer-me would have got around to anyway.

In home economics news, I did bake two loaves of plum bread (plums "sourced locally", by which I mean they were scrumped from a tree up the road) which is something I haven't had time to do in nearly a year.

#### Week Seven
Having spent a few weeks trying all of the things and letting life happen, this week was about selection and focus. What should I actually concentrate on, and put energy into?

It's time to add some structure to this situation. Dropping all of the things and taking life as it comes was relaxing, enjoyable, and necessary. Re-moulding things and building something new out of the parts will be necessary to provide a new sense of engagement and purpose.

#### Week six
This is a post about marmalade.

A "life hack", if you will, that I learnt from Simon Stewart is to find things that I don't do because I don't like them, or think I don't like them, and to do them. Do them again. Find out whether there's anything to like about them, whether I missed something important, or whether my tastes have simply changed.

That is why breakfast this weekend featured the phrase "could you pass the marmalade, please, I don't like it".

This is not a post about marmalade.

#### Week three
This is both an international and a hyper-local update. I say international, although I'm still in an airport less than twenty miles from my house. I'm on my way to Florence, to talk about *-oriented programming at Pragma Conference 2015. It'll be fun to catch up with friends in a new city, and close the conference with what I hope is an informative talk. I'll put my notes up some time after the talk.

The theme of this week has been friends. I've had lots of things I could be doing, and even a few things I should be doing. For the most part I've been letting that take second place to spending time with friends. I've ended up playing a lot of tunes and singing a lot of songs as a result, and maybe only written four or five paragraphs of useful text.

Speaking of singing and playing, our band is going to play the main spot at Folk in the Barn, Stockton on the 27th. Anyone who's around the Somerville Arms, Leamington on Tuesday 13th October or Harbury folk club on Thursday 5th November will also discover a subset of the band doing a couple of numbers. International readers will be amused by the provincial nature of this part of the post; a lot of the fun of this break has been connecting with the local area and things that are going on around me.

#### Week Two
As week one featured an observation of how post-work life was similar to working life, this week's post is a catalogue of differences. Not all of these differences are huge.

No watch

I own five watches (three wrist, two pocket) but have only worn one once in the last two weeks. Most of the time, I don't need to know what the time is and don't need to mark its passing.

No beard

When you get up at 5:30 every morning for the commuter train, putting time into your appearance takes a back seat to getting out of the door quickly. I now shave every day; not particularly important except that it indicates I have time to do so.

No computer

Or at least no sitting at a computer. I'm writing this on a laptop which I've just been using to write up the dissertation, but when I'm not doing that, or editing music scores in MuseScore, I'm not "at" a computer. Tempus Fugit was written on my phone: the best computer is the one you have on you.

No tech news

My morning ritual used to involve a lot of RSS tech news feeds, as well as browsing some aggregation sites like lobste.rs to find articles to read. Now that's all been pared back, apart from a few people who write well. I usually have two or three unread articles every day now, which are more often than not web comics.

#### Well, you could have told me
When looking through some of the configuration options on my laptop (well, it's either that or go to the pub and socialise with humans) I came across something I couldn't account &mdash; pardon the pun &mdash; for.  A new user account on the system, short name messagebus, full name &quot;Message Bus&quot; user id 506.  Now messagebus looks like the name of a system daemon user, but that full name looks like some clueless skiddie made a mistake creating the user account, especially as the uid is that of a regular user.  That's the kind of mistake no self-respecting installer would make.

So, what had this phantom user done?  Well, thankfully, nothing.  Neither the shell nor home directory was real, and wtmp/utmp showed no activity.  Neither did the ssh logs - but in looking for them I realised that I don't actually use ssh on the box, so turned it off.

Anyway, it turned out to be an innocuous issue - the MacPorts installer for dbus creates this bogus user, which I've since deleted.  This Apple forums discussion explains more.

#### What do you think of this?
I'm interested to find out what us Cocoa developers (alright, I know my opinion already) think of the following distinction between Foundation and, well any other object-oriented foundation library.

The distinction is this. In many libraries, compound objects (not only collections, but strings which are many-character objects and data which are many-byte objects) have both immutable and mutable varieties. We're familiar with the Cocoa pattern, where we have the base immutable class e.g. NSArray (class clusters are an irrelevant complication for now) and a mutable subclass, e.g.:

@interface NSMutableArray : NSArray

That's not how everyone else does it. They have distinct class hierarchies for the mutable and immutable types; for instance in Java we have String and StringBuilder. The two classes aren't related, but you can create a StringBuilder given a String and vice versa. You certainly can't pass one in where the API expects the other. The design pattern used here is called Builder.

The argument in favour of the subclass approach is that mutable arrays are just specialised versions of arrays. You can find out how long they are, what object is at what index, and you can also do all the add/remove/replace stuff on top, so these objects are arrays with extra functionality, and thus should be subclasses of the array type.

The argument against is that this specialisation is bogus. Client code can't treat all array objects the same in case it gets a mutable array, and therefore mutable arrays can't be used as arrays and shouldn't be subclasses. They violate the Liskov substitution principle: the rule that if I expect to work with one class, any of its subclasses must be usable in its stead.

We can use an example from Foundation to bolster the second argument a bit. The NSFastEnumeration protocol works on all collections, except that when implemented on a mutable collection it must employ additional checks to protect against the collection being mutated while it's iterating over it. So we need extra client code to deal with some subclasses, and thus Liskov is violated.

What do you think? Would you have designed Foundation the same way NeXT did? Perhaps there are other changes you would have made. Let the world know in the comments box.

#### What Graham did next
There's been quite a lot of reaction to this notice on Agant's website, that Dave is taking the company back to a one-person shop. Indeed that means that I and all of my colleagues (except Dave) are now redundant.

Sad is not the right word. I'm disappointed that this isn't going to continue, but it's hard to be sad about getting onto this billing:

Thanks for all your concern. I've already got my next thing lined up: I'm really excited to be programming and training for the Big Nerd Ranch, starting in August. I've known Aaron and a few other big nerds for years and I'm sure this is going to be a great opportunity for me.

In the even shorter term, though, hopefully the experience I've built with having more jobs than birthdays since graduation can be helpful to our other developers as we collectively discover what comes next.

#### What happens when you add one to an integer?
It depends. You saw in the previous post that there are plenty of different integer types, some with known sizes and some where the size is set by the implementation. Well for each size of integer type there are two main variants: signed and unsigned.

Unsigned numbers are always zero or positive. They're the easiest ones to understand, and their behaviour is well defined. In almost all cases, adding one to an unsigned integer in C makes that integer bigger by one. The only exceptional case is when the number already represents the maximum value that will fit in its type; adding one to the maximum "overflows" and gets you back to 0.

Signed integers are tricky. Computers don't natively handle negative numbers, but signed values can (as the name suggests) be negative. Various conventions have been created to allow support for negative numbers: the most common is to treat one bit of a variable as the "sign" bit (as a note for overly-sensitive nerds: sometimes these conventions are honoured in CPU instructions, and you could say that such computers do natively handle negative numbers). If the sign bit is set, then the number is negative; otherwise it is positive. Some platforms have an extra bit separate from the storage of the number that indicates the sign of the number.

What this means is that if the C language were to specify what happens when a signed integer overflows, some implementations would be able to handle this efficiently but some would not as they'd have to translate the particular platform-specific behaviour into that required by the standard.

The result then of adding one to a signed integer is quite surprising: if it causes the number to overflow, the result is undefined. An implementation is free to do anything (implementers usually choose to do whatever's most efficient); relying on the behaviour from one implementation means writing unportable code.

As a result of this it's important to guard against integer overflow in C (and C++ and Objective-C) programs. Typically the unsigned integer types should only be used either as bitmasks, where the value of each bit is important but doesn't affect interpretation of the other bits, or in situations where the known overflow behaviour is actually desired. In cases where you "know" a number will always be positive, it's still best to use a signed integer, as that offers the possibility of detecting bugs that end up pushing the value below zero.

As an example, consider a data type in my application that I "know" will always have a count that's positive and smaller than 200. I could use a uint8_t to represent that, but there are conditions that are erroneous and yet will lead to valid-looking answers. Imagine removing 80 objects from an instance with count 50, or adding 80 objects to an instance with count 180. Because of the overflow behaviour of uint8_t, these problems would leave the result "looking" OK. It would be better to represent this type using int16_t, which both accepts values below 0 and above 200; now the problematic cases described earlier do not overflow, but result in numbers that are within the range that can be represented and can therefore be tested against my application-specific requirements.

#### What happens when you jailbreak an iPad
Having played around with an iPad running a jailbreak OS yesterday, I thought it would be useful to explain one possible attack that doesn't seem to get much coverage.

As I've discussed in numerous talks, the data protection feature of iOS (introduced in iOS 4, enabled by setting the NSFileProtectionComplete option on a file or writing data with the NSDataWritingFileProtectionComplete option) only works fully when the user has a passcode lock enabled. The operating system can derive a key to protect the files (indirectly, but that's another talk) from the passcode, so when the device is locked the files are really inaccessible because the device has no idea what the unlock key is.

This can be seen when you try and access the content via SSH. Of course, the SSH daemon must be installed on a jailbreak operating system, but you don't need the passcode to jailbreak:

$ ssh -l root@192.168.0.27
[key/auth exchange...the default password is still 'alpine']
# cd /User/Applications/C393CDBF-1A82-4D7B-A064-D6DFB8CC20DB/Documents
# cat UnprotectedFile
The flag. You haz it.
# cat ProtectedFile
cat: Error: Operation not permitted
[unlock]
# cat ProtectedFile
The flag. You haz it.

Now of course you do need physical access to jailbreak, but that doesn't take particularly long. So here's a situation that should probably appear in your threat models:

Attacker retrieves target's iPad
Attacker installs jailbreak OS with data-harvesting tools
Attacker returns iPad to the target
Target uses iPad

Of course, an attacker who simply tea leafs the target's iPad can't perform this attack, and won't be able to retrieve the files.

#### What is software engineering?
I suppose if I'm going to have a tagline like "from programming to software engineering", we ought to have some kind of shared understanding of what that journey entails. It would be particularly useful to agree on the destination.

The question "what is software engineering?" doesn't have a single answer. Plenty of people have the job title "software engineer", or work in the "engineering" department of a software company, so we could say that it's whatever they do. But that's not very constructive. It's too broad: anyone who calls themselves a software engineer could be doing anything, and it would become software engineering. It's also too narrow: the people with the job title "software engineer" are often the programmers, and there's more to software engineering than programming. I wrote a whole book, APPropriate Behaviour, about the things programmers need to know beyond the programming; that only scratches the surface of what goes into software engineering.

Sometimes the definition of software engineering is a negative-space definition, telling us what it isn't so that something else can fill that gap. In Software Craftsmanship: The New Imperative, Pete McBreen described that "the software engineering approach" to building software is something that doesn't work for many organisations, because it isn't necessary. He comes close to telling us what that approach is when he says that it's what Watts Humphries is lamenting in Why Don't They Practice What We Preach? But again, there are reasons not to like this definition. Firstly, it's a No True Scotsman definition. Software engineering is whatever people who don't do it properly, the way software craftsmen do it, do. Secondly it's just not particularly fruitful: two decades after his book was published, most software organisations aren't using the craftsmanship model. Why don't they practice what he preaches?

I want to use a values-oriented definition of software engineering: software engineering is not what you do, it's why you do what you do, and how you go about doing it. No particular practice is or isn't software engineering, but the way you evaluate those practices and whether or not to adopt them can adopt an engineering perspective. Similarly, this isn't methodology consultancy: the problems your team has with Agile aren't because you aren't Agiling hard enough and need to hire more Agile trainers. But the ways in which you reflect on and adapt your processes can be informed by engineering.

I like Shari Lawrence Pfleeger's definition, in her book Software Engineering: The Production of Quality Software:
There may be many ways to perform a particular task on a particular system, but some are better than others. One way may be more efficient, more precise, easier to modify, easier to use, or easier to understand. Consequently, Software Engineering is about designing and developing high-quality software.
There's a bit of shorthand, or some missing steps here, that we could fill in. We understand that of the many ways to build a software system, we can call some of them "better". We declare some attributes that contribute to this "betterness": efficiency, precision, ease of adaptation, ease of use, ease of comprehension. This suggests that we know what the properties of the software are, which ones are relevant, and what values a desirable system would have for those properties. We understand what would be seen as a high-quality product, and we choose to build the software to optimise for that view of quality.

The Software Engineering degree course I teach on offers a similar definition:
Software Engineering is the application of scientific and engineering principles to the development of software systems—principles of design, analysis, and management—with the aim of:

 	developing software that meets its requirements, even when these requirements change;
 	completing the development on time, and within budget;
 	producing something of lasting value—easy to maintain, re-use, and re-deploy.

So again we have this idea that there are desirable qualities (the requirements, the lasting value, ease of maintenance, re-use, and re-deployment; and also the project-level qualities of understanding and controlling the schedule and the cost), and the idea that we are going to take a principled approach to understanding how our work supports these properties.

Let me summarise: software engineering is understanding the desired qualities of the software we build, and taking a systematic approach to our work that maximises those qualities.

#### What it takes to "win" a discussion
You may have been to some kind of debate club at school, or at least had a debate in a class. If so, the debate you had was probably a competitive debate, and went something along these lines (causality is not presented as its usual wibbly-wobbly self to keep the sentences short):

A motion is proposed.
Someone presents a statement in favour of the motion.
Someone else presents a statement against the motion.
A second statement in favour is made.
A second opposing statement is made.
Questions are asked and answered.
A favouring summary is made.
An opposing summary is made.
Somehow one "side" or the other "wins", perhaps by a vote.

Or you may have been to court. If so, you probably saw something that went along these lines:

A charge is proposed.
Someone presents a case, including evidence, supporting the charge.
Someone else presents a case, including evidence, refuting the charge.
Questions are asked and answered.
A supporting summary is made.
A refuting summary is made.
Somehow one "side" or the other "wins", perhaps by the agreement of a group of people.

Both forms of conversation are very formal and confrontational. They're also pretty hard to get right without a lot of practice.

And here's a secret that the Internet Illuminati apparently tries to keep shielded from many people: not every conversation needs to work like that.

Back in the time of the war of the three kingdoms, the modern party system of British politics didn't exist in the same way it does now. Members of parliament would form associations based on agreements of the matter under discussion, but the goal of parliament was to reach consensus on that matter. "Winning" was achieved by coming to the best conclusion available.

And, it turns out, that's a possible outcome for today's discussions too. Let's investigate what that might mean.

Someone wants to know what tool to use to achieve some goal. This conversation is "won" by exploring the possibilities and their pros and cons. Shouting across other people's views until they give up doesn't count as winning, because nothing is learned. That's losing.
Two people have different experiences. Attempting to use clever rhetorical tricks to demonstrate that the other person's views are invalid doesn't count as winning, because nothing is learned. That's losing.

Learning things, by the way, is pretty cool.

#### What Lenin taught me about software movements
In What is to be done?: Burning Questions of our Movement, Lenin lists four roles who contribute to fomenting revolution - the theoreticians, the propagandists, the agitators, and the organisers:

  The theoreticians write research works on tariff policy, with the “call”, say, to struggle for commercial treaties and for Free Trade. The propagandist does the same thing in the periodical press, and the agitator in public speeches. At the present time [1901], the “concrete action” of the masses takes the form of signing petitions to the Reichstag against raising the corn duties. The call for this action comes indirectly from the theoreticians, the propagandists, and the agitators, and, directly, from the workers who take the petition lists to the factories and to private homes for the gathering of signatures.

Then later:

  We said that a Social Democrat, if he really believes it necessary to develop comprehensively the political consciousness of the proletariat, must “go among all classes of the population”. This gives rise to the questions: how is this to be done? have we enough forces to do this? is there a basis for such work among all the other classes? will this not mean a retreat, or lead to a retreat, from the class point of view? Let us deal with these questions.
  
  We must “go among all classes of the population” as theoreticians, as propagandists, as agitators, and as organisers.

Side note for Humpty-Dumpties: In this post I'm going to use "propaganda" in its current dictionary meaning as a collection of messages intended to influence opinions or behaviour. I do not mean the pejorative interpretation, somebody else's propaganda that I disagree with. Some of the messages and calls below I agree with, others I do not.

Given this tool for understanding a movement, we can see it at work in the software industry. We can see, for example, that the Free Software Foundation has a core of theoreticians, a Campaigns Team that builds propaganda for distribution, and an annual conference at which agitators talk, and organisers network. In this example, we discover that a single person can take on multiple roles: that RMS is a theoretician, a some-time propagandist, and an agitator. But we also find the movement big enough to support a person taking a single role: the FSF staff roster lists people who are purely propagandists or purely theoreticians.

A corporate marketing machine is not too dissimilar from a social movement: the theory behind, say, Microsoft's engine is that Microsoft products will be advantageous for you to use. The "call" is that you should buy into their platform. The propaganda is the MSDN, their ads, their blogs, case studies and white papers and so on. The agitators are developer relations, executives, external MVPs and partners who go on the conference, executive briefing days, tech tours and so on. The organisers are the account managers, the CTOs who convince their teams into making the switch, the developers who make proofs-of-concept to get their peers to adopt the technology, and so on. Substitute "Microsoft" for any other successful technology company and the same holds there.

We can also look to (real or perceived) dysfunction in a movement and see whether our model helps us to see what is wrong. A keen interest of mine is in identifying software movements where "as practised" differs from "as described". We can now see that this means the action being taken (and led by the organisers) is disconnected from the actions laid out by the theorists.

I have already written that the case with OOP is that the theory changed; "thinking about your software in this way will help you model larger systems and understand your solutions" was turned by the object technologists into "buying our object technology is an easy way to achieve buzzword compliance". We can see similar things happening now, with "machine learning" and "serverless" being hollowed out to fill with product.

On the other hand, while OOP and machine learning have mutated theories, the Agile movement seems to suffer from a theory gap. Everybody wants to be Agile or to do Agile, all of the change agents and consultants want to tell us to be Agile or to do Agile, but why does this now mean Dark Scrum? A clue from Ron Jeffries' post:

  But there is a connection between the 17 old men who had a meeting in Snowbird, and the poor devils working in the code mines of insurance companies in Ohio, suffering under the heel of the boot of the draconian sons of expletives who imposed a bastardized version of something called Scrum on them. We started this thing and we should at least feel sad that it has sometimes gone so far off the rails. And we should do what we can to keep it from going more off the rails, and to help some people get back on the rails.

Imagine if Karl Marx had written Capital: Critique of Political Economy, then waited eighty years, then said "oh hi, that thing Josef Stalin is doing with the gulags and the exterminations and silencing the opposition, that's not what I had in mind, and I feel sad". Well Agile has not gone so far off the rails as that, and has only had twenty years to do it, but the analogy is in the theory being "baked" at some moment, and the world continuing to change. Who are the current theorists advancing Agile "as practised" (or at least the version "as described" that a movement is taking out to change the practice)? Where are the theoreticians who are themselves Embracing Change? It seems to me that we had the formation of the theory in XP, the crystallisation (pardon the pun) of the theory and the call to action in the Agile manifesto, then the project management bit got firmed up in the Declaration of Interdependence, and now Agile is going round in circles with its tiller still set on the Project Management setting.

Well, one post-Agile more-Agile-than-thou movement for the avocado on toast generation is the Software Craft[person]ship movement, which definitely has theory and a call to action (Software Craftsmanship: the New Imperative, which is only a scratch newer than the Agile Manifesto), definitely has vocal propagandists and agitators, and yet still doesn't seem to be sweeping the industry. Maybe it is, and I just don't see it. Maybe there's no clear role for organisers. Maybe the call to action isn't one that people care about. Maybe the propaganda is not very engaging.

Anyway, Lenin gave me an interesting model.

#### What Xamarin.Forms taught me about data bindings
I've spent about a year working on an app for a group in the University where I work, that needed to be available on both Android and iOS. I've got a bit of experience working with the Apple-supplied SDKs on iOS, and a teensy amount of experience working with the Google-supplied SDKs on Android. Writing two apps is obviously an option, but not one I took very seriously. The other thing I've reached for before in this situation is React Native, where I've got a little experience but quite a bit of understanding having worked with React some.

Anyway, this project was a mobile companion for a desktop app written in C# and Windows Forms, and the client was going to have to pick up development at the end of my engagement. So I decided that the best approach for the client was to learn how to do it in Xamarin.Forms, and give them a C# project they could understand at the end. I also hoped there'd be an opportunity to share some code from the desktop software in the mobile project, though this didn't pan out in the end.

It took a while to understand the centrality of the Model-View-ViewModel idea and how to get it to work with the code I was writing, rather than bludgeoning it in to what I was trying to do. Ultimately lots of X.F works with data bindings, where you say "this thing and that thing are connected" and so your view needs a that thing so it can display this thing. If the that thing isn't in the right shape, is derived somehow, or shouldn't be committed to the model until some other things are done, the ViewModel sits in the middle and separates the two.

I'm used to this model in a couple of contexts, and I'll give Objective-C examples of each because that's how old I am. In web applications, you can use data bindings to fill in parts of an HTML document based on values from a server-side object. WebObjects works this way (Rails doesn't, it uses code to fill in parts of etc). The difference between web app data bindings and mobile app data bindings is one of lifecycle. Your value needs to be read once when the page (or XHR) is rendered, and stored once when the user posts the changes. This is so simple that you can use straightforward accessor methods for it. It also happens at a time when loading new content is happening, so any timing constraints are likely to come from elsewhere.

You can also do it in what, because I'm that old, I'll call rich client applications, like Xamarin.Forms mobile apps or Cocoa Bindings desktop apps. Here, anything could be happening at any time: a worker thread could be updating the model, and the user could interact with the UI, all at the same time, potentially multiple times while a UI element is live. So you can't just wait until the Submit button is pressed to update everything, you need to track and reflect updates when they happen.

Given a dynamic language like Objective-C, you can say "bind this thing to that thing with these options" and the binding library can rewrite your accessors for this thing and that thing to update the other when changes happen, and avoid circular updates. You can't do that in C# because apparently more typing is easier to reason about, so you end up replicating the below pattern rather a lot.


public class MyThingViewModel : INotifyPropertyChanged
{
  public event PropertyChangedEventHandler PropertyChanged;
  // ...
  private string _value;
  public string Value
  {
    get => _value;
    set
    {
      _value = value;
      PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(nameof(Value)));
    }
  }
}


And when I say "rather a lot", I mean in this one app that boilerplate appears at least 126 times, this undercounts because despite being public, the PropertyChanged event can only be invoked by instances of the declaring class so if a subclass adds any properties or any change points, you're going to write protected helper methods to be able to invoke the event from the subclass.

Let's pivot to investigating another question: why is Cocoa Bindings a desktop-only thing from Apple? I've encountered two problems in using it on Xamarin: thread confinement and performance. Thread confinement is a problem anywhere but the performance things are more sensitive on mobile, particularly on 2007-era mobile when this decision was made, and I can imagine a choice was made between "give developers the tools to identify and fix this issue" and "don't give developers the chance to encounter this issue" back when UIKit was designed. Neither X.F nor UIKit is wrong in their particular choice, they've just chosen differently.

UI updates have to happen on the UI thread, probably because UIKit is Cocoa, Cocoa is appkit, and appkit ran on an OS that didn't give you an easy way to do multiple threads per task. But this has to happen on Android too. And also performance. Anyway, theoretically any of those 126 invocations of PropertyChanged that could be bound to a view (so all of them, because separation of concerns) should be MainThread.BeginInvokeOnMainThread(() => {PropertyChanged?.Invoke(...)}); because what if the value is updated in an async method or a Task. Otherwise, something between a crash and unexpected behaviour will happen.

The performance problem is this: any change to a property can easily cause an unknown amount of code to run, quite often on the UI thread. For example, my app has a data grid (i.e. spreadsheet-like table view) with a "selection" column containing switches. There's a "select all" button, and a report of the number of selected objects, outside the grid. Pressing "select all" selects all of the objects. Each one notifies observers that its IsSelected property has changed, which is watched by the list view model to update the selection count, and by the data grid to update the switches. So if there's one row in the grid, selecting all causes two main-thread UI updates. If there are 500 rows, then 1000 updates need to run on the main thread in response to that one button action.

That can get slow :). Before I understood how to fix this, some UI actions would block the UI for tens of seconds as they computed the update. I asked about this in some forums and was told the answer is "your users shouldn't have that much data in a mobile app, design an app with less data" which is not that helpful. But luckily the folks over at SyncFusion were much more empathetic, and told me the real solution is to design your views and view models such that you can turn off updates while you're doing some big change, then turn them back on and recalculate the state at the end.

Like I say, it's likely that someone at Apple already knew this from the Cocoa Bindings times and decided "here's a great technology, and here's how to turn it off because it will get in your way" wasn't a cool story.

#### What's a software architect?
After a discussion on the twitters with Kellabyte and Iris Classon about software architects, I thought I'd summarise my position. Feel welcome to disagree.

What does a software architect do?

A software architect is there to identify risks that affect the technical implementation of the software product, and address those risks. Preferably before they stop or impede the development of the product.

That could mean doing tests to investigate the feasibility or attributes of a proposed solution. It could mean evangelising the developers to the clients or managers to avoid those people interrupting the development work. It could mean giving a junior developer a tutorial on a certain technology-or getting that developer to tutor the rest of the team on the thing that person is an expert on.

What a software architect doesn't do

A software architect doesn't micromanage the developers who work with them. The architect doesn't rule by memos and UML diagrams. The architect doesn't prognisticate on things they have no experience of. Perhaps confusingly, the role of software architect bears very little resemblance to the profession after which it's named. If you want analogies with civil engineering, all developers are like architects. If you want to see the software analogue to the builder, that's work done by the compiler and IDE.

Architects don't make decisions where none is necessary. They don't ignore or belittle suggestions that come from people who aren't architects.

In one sentence

A software architect is there to make it easier for developers to develop.

#### What's better than semver?
Many software libraries are released with version "numbers" that follow a scheme called Semantic Versioning. A semantic version is three numbers separated by dots, of the form x.y.z, where:

if x is zero, all bets are off. Otherwise;
z increments "if only backwards compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior."

Problem one: there is no such thing as an "internal change that fixes incorrect behavior" that is "backwards compatible". If a library has a function f() in its public API, I could be relying on any observable behaviour of f() (potentially but pathologically including its running time or memory use, but here I'll only consider return values or environment changes for given inputs).

If they "fix" "incorrect" behaviour, the library maintainers may have broken the package for me. I would need a comprehensive collection of contract or integration tests to know that I can still use version x.y.z' if version x.y.z was working for me. This is the worst situation, because the API looks like it hasn't changed: all of the places where I call functions or create objects still do something, they just might not do the right thing any more.

Problem two: as I relaxed the dependency on running time or memory use, a refactoring could represent a non-breaking change. Semver has nowhere to record truly backwards compatible changes, because bugfixes are erroneously considered backwards compatible

y increments "if new, backwards compatible functionality is introduced to the public API".

This is fine. I get new stuff that I'm not (currently) using, but you haven't broken anything I do use.

Problem three: an increment to y "MAY include patch level changes". So I can't just quietly take in the new functionality and decide whether I need it on my own time, because the library maintainers have rolled in all of their supposedly-backwards-compatible-but-not-really changes so I still don't know whether this version works for me.

x increments "if any backwards incompatible changes are introduced to the public API".

Problem four: I'm not looking at the same library any more. It has the same name, but it could be completely rewritten, have any number of internal behaviour changes, and any number of external interface changes. It might not do what I want any more, or might do it in a way that doesn't suit the needs of my application.

On the plus side

The dots are fine. I'm happy with the dots. Please do not feel the need to leave a comment if you are unhappy with the dots or can come up with some contrived reason why "dots are harmful", as I don't care.

Better: meaningful versioning

I would prefer to use a version scheme that looks like z.w.y:

y has the meaning it does in semver, except that it MUST NOT include patch level changes. If a package maintainer has added new things or deprecated (but not removed) old things, then I can use the package still.
z has the meaning it does in semver, except that we stop pretending that bug fixes can be backwards compatible.
w is incremented if non-behavioural changes are implemented; for example if internals are refactored, caches are introduced or removed, or private data structures are changed. These are changes that probably mean I can use the package still, but if I needed particular performance attributes from the library then it is on me to discover whether the new version still meets my needs.

There is no room for x in this scheme. If a maintainer wants to write a new, incompatible library, they can use a new name.

Different: don't use versions

This is more work for me, but less work for the package maintainer. If they are maintaining a change log (which they are, as they are using version control) and perhaps a medium for announcing important changes including security and bug fixes and new features, then I can pick the commit that I discover does what I need. I can maintain my own tree (and should be anyway, in case the maintainer decides to delete their upstream repo) and can cheery pick the changes that are useful for me, leaving out the ones that are harmful for me.

This is more work for me than the z.w.y scheme because now I have to understand the impact of each change. It is the same amount of work as the semver x.y.z scheme, because then I had to understand the impact of each change too, as changes to any of the three version component could potentially include supposedly-backwards-compatible-but-not-really changes.

#### What's new in 2009
Of course, it's a bit early for a retrospective of 2008, besides which I've already written 73 entries this year, my most prolific year to date on iamleeg. And that doesn't count numerous tweets, stack overflow contributions and of course the occasional piece of source code here or there for some security company. As the noise of fireworks and exploding media players sounds across the world, it's time to pre-emptively ditch 2008 and see what we can expect from 2009. Specifically, what you can expect from me.It looks to me like the most popular pieces on this blog are the opinions and how-tos regarding Cocoa development, particularly my thoughts on properties and Cocoa memory management round-up. Don't worry, there's definitely more of this coming. As well as preparing for this Mac Developer Network conference talk I've been discussing recently, I've got another exciting - and unfortunately secret - project on the go now which should see plenty of collateral blog posting in the first half of the next year, all about Cocoa development. There'll also be a bit more of an iPhone mix-in; obviously for much of last year the SDK either didn't exist or was under non-disclosure, but now I've got more reasons to be using Cocoa Touch it will also be mentioned on here. I shall also be delving a bit deeper into Darwin and xnu than I have in previous times.One example of Cocoa-related information is meetup announcements; I'm still involved in the local CocoaHeads chapter and I'll endeavour to post an advance warning for each meeting here. I know many of my readers are in the States but a few of you are local so please do come along! In fact, if you're not local (or "bissen't from rond theez partz", as we say here) then consider going to your nearest CocoaHeads or starting a new one. It's a great way to find out who's working on Mac or iPhone development in your area, share tips and stories and build up that professional contacts network.Previously I've been concerned that readers here at iamleeg don't seem interesting in commenting on my posts, but these days I'm no longer worried. I can tell how many people are reading, and of those how many are regulars, and I have to say that the blog is doing pretty damn well. Of course, if you do feel inclined to join in the discussion (particularly if I've got something wrong, or missed an important point from a post) then you should feel perfectly at liberty to leave a comment.Finally, have a happy new year!

#### What's on the other channel?
I run a company, a mission-driven software consultancy that aims to make it easier and faster to make high-quality software that preserves privacy and freedom. On the homepage you'll find Research Watch, where I talk about research papers I read. For example, the most recent article is Runtime verification in Erlang by using contracts, which was presented at a conference last year. Articles from the last few decades are discussed: most is from the last couple of years, nothing yet is older than I am.

At de Programmatica Ipsum, I write on "individuals, interactions, and the true valuation of the things on the left" with Adrian Kosmaczewski and a glorious feast of guest writers. The most recent issue was on work, the upcoming issue is on programming history. You can subscribe or buy our back-catalogue to read all the issues.

Anyway, those are other places where you might want to read my writing. If people are interested I could publish their feeds here, but you may as well just check each out yourself :).

#### What's the mobile app market up to, then?
While this post is obviously motivated by Recent Events™, it's completely not got anything to do with employers past, present or future. Dave has posted what next for Agant which explains how that company's path through the market has gone:

Over the past few years, the App Store has become more and more competitive, and more and more risky with it. Agant's speciality has been high-quality, higher-value apps, often published in collaboration with our clients. Typically these are paid (rather than free or freemium) apps. Unfortunately, the iOS App Store's set-up just does not seem to support the discovery, trialling and long-term life of these kinds of high-value apps, making it difficult to justify the risk of their development.

This is not that story. This is my story. It is a different story, though I agree with the paragraph above. It's a story that doesn't discuss games because I really don't know a lot about them.

Something I've learned from going to conferences like QCon is that outside the filter bubble of the ObjC conferences I spend a lot of time in, there's a lot more interest in "the mobile web" (or as we should probably call it these days, "the web") in the general IT community. This makes sense in the enterprise world: it avoids backing a single horse and tying your company's IT to one supplier, something they're rightfully afraid of. Companies that were in the Microsoft camp had to deal with Vista and Windows 8; companies that backed Sun are now Oracle vassals; companies that backed Apple no longer have any servers. Given that mindset, developing javascript apps makes perfect sense. Even if you deliver them now as Cordova apps for a single platform, you've got the ability to do something else really quickly if you need to.

This is also something that's carried over into the world of SaaS apps, where you don't care what UI people are looking at as long as they subscribe to your service. Whether it's delivered as a native-wrapped JS app (which is a first-party option for Windows Phone 8 and Windows 8) or a web app (which then lets you add platforms like Chrome OS and Firefox OS), targeting JavaScript lets these developers increase their prospective customer bases from a single code base. Not, perhaps, without some rework of views for different platforms: but certainly without maintaining separate Objective-C, Java and C# projects.

While I'm talking about JavaScript, let me add another relevant datum, particularly for companies working in or with the publishing industry: another word for a bundled JS app is "iBook".

I think there are also still reasons for having native apps.

Some people want the "most ${platform}-like" experience, and are willing to pay for that. These are, quite frankly, the people who kept Mac software houses going through the 1990s. They're the people who demanded Cocoa versions of their Carbon apps in the 2000s. You can focus on these people, ignoring the "should be free" masses and getting to the sort of people who buy the Which iPad Format User app of the month because it was the app of the month.

People who have invested money or time into something may be willing to spend a bit in order to increase the value of that investment. This is going to cover both tradespeople and hobbyists. Look at how much you can sell golf swing software for. One of my own hobbies is astronomy: having spent around a grand on my telescope I'm not going to miss £20 dropped on an app that helps me get more value from that purchase. The trick here is not to rely on gaming the "astronomy" keyword in the app store, but to become known in that world. Magazines are more relevant than you might give them credit for, when looking at these markets. Astronomy Now, one of the UK's astronomy mags, has a circulation of 24,000 (publishers then have an "estimated number of readers per sale" fiddle factor that's relevant to advertising, so there might be 24-50k monthly readers). These people will read about your product, like it (if you're doing it right) and will then go out to their user groups and meet-ups and tell those people about your product.[*]

[*] This paragraph owes a lot to Dave Addey, who referred to such audiences as broad niches.

The difficulty is that two forms of advertising no longer work: you can no longer rely on being on the app store as a way to get your app known, and similarly saying to an existing audience "hey, we're on the app store" is also insufficient. Apps are no longer a novelty in and of themselves, so having a thing that does a thing is not a guaranteed retirement plan.

This points us to a couple of things that definitely are not reasons for having apps. Mass-market apps are now a very hard sell. They can be hard to differentiate on, hard to price reasonably and hard to generate awareness of. This awareness issue brings us into contact with the most powerful businesses in the app market: the platform vendors. No platform is going to allow a "killer app" to surface. Think back, for a moment, to the days of Visicalc. People bought Apple II computers so that they could run Visicalc. That's fine when Visicalc is Apple-only; not so good when it gets ported to Tandy, IBM and other architectures. It's also not good when someone else comes out with a better Visicalc for the other platform: 1-2-3 and your customers are gone. Apple (and other OEMs) want control over their customers: they're not about to cede that control to some ISV with a good idea.

The other thing it's not a good idea to do is to plug a gap in the OEM software. In smartphones, though not in hi-fis, printers or other electronic devices, the OEM companies are actually pretty good at executing on software features so if you're doing "the missing ${X} for ${platform}", as soon as it becomes at all popular the OEM vendor will fill in their version of ${X}. It might not be as featureful, it might not even be better but it'll probably be good enough to stop the third-party ones from selling.

Notice that I haven't said "native is better", or "mobile web is better". There are apps that you can only build as native apps because the technology limits you to that: this does not mean that you must build them as native apps. There's no reason you must build them at all. Decide who you're building for, and what you can offer them that they'd consider to be a valuable experience. Having done that, decide on the best way to build and deliver it.

There is no longer any value in having "an app for that". There is value in a beneficial experience, which it might make sense for you to build as an app.

#### When all you have is a NailFactory...
...every problem looks like it can be solved by configuring a different nail.

We have an obsession with tools in the software industry. We've built tools for building software, tools for testing software, tools for recording how the software is broken, tools for recording when we fixed software. Tools for migrating data from the no-longer-cool tools into the cool tools. Tools for measuring how much other tools have been used.

Let's call this Tool-Driven Development, and let's give Tool-Driven Development the following manifesto (a real manifesto that outlines intended behaviour, not a green paper):

Given an observed lack of consideration toward attribute x, we Tool-Driven Developers commit to supplying a tool that automates the application of attribute x.

So, if your developers aren't thinking about testing, we'll make a tool to make the tests they don't write run quicker! If your developers aren't doing performance analysis, we've got all sorts of tools for getting the very reports they don't know that they need!

This fascination with creating tools is a natural consequence of assuming that everyone[*] is like me. I've found this problem that I need to solve, surely everyone needs to solve this problem so I'll write a tool. Then I can tell people how to use this tool and the problem will be solved for everyone!

[*]Obviously not everyone, just everyone who gets it. Those clueless [dinosaurs clinging to the old tools|hipsters jumping on the new bandwagons] don't get it, and I'm not talking about them.

No. Well, not yet. We've skipped two important steps out of a three-step enlightenment scheme:

	Awareness. Tell me what the unknown that I don't know is.
	Education. Tell me why this thing that I now know about is a big deal, what I'm missing out on, what the drawbacks are, and why solving it would be beneficial.
	Training. Now that I know this thing exists, and that I should do something about it, and what that something is, now is the time to show me the tools and how I can use them to solve my new problem.

One of the talks at QCon London was by Damian Conway on dead languages. It covered these three features almost in reverse, to make the point that the tools we use constrain our mental models of the problems we're trying to solve. Training: here's a language, this is how it works, this is a code problem solved in that language. Education: the language has these features which lets us write our code in this way with these limitations. Awareness: there are ways to write code, and hence to solve problems in software, that aren't the way you're currently doing it.

A lot of what I've worked on has covered awareness without going further. The App Makers' Privacy Pledge raises awareness that privacy in mobile apps is a problem, without discussing the details of the problem or the mechanics of a solution. APPropriate Behaviour contains arguments expressing that programmers should be aware of the social scope in which their programming activities sit.

While I appreciate and even accept the charge of intellectual foreplay, I think a problem looking for a solution is more useful than a solution looking for a problem. Still, with some of us doing the awareness thing and others doing the training thing, a scheme by which we can empower ourselves and future developers is clear: let's team up and meet in the middle.

#### When did people favor composition over inheritance?
The phrase "favor composition over inheritance" has become one of those thought-terminating cliches in software design, and I always like to take a deeper look at those to understand where they come from and what ideas we're missing if we just take the phrase at face value without engaging with the discussion that led to it.

This is one of those aphorisms with a definite origin story (compare with Aphorism Considered Harmful for an example where the parts have clear origins but the whole does not): it's the second object-oriented design principle in the Design Patterns book from the "Gang of Four" Gamma, Helm, Johnson, and Vlissides. Well, sort of:

Favor object composition over class inheritance.

It comes at the end of a page and a half of justification, and actually just before another page and a half on delegation (an extreme example of object composition), so really in the middle of a three-page discussion. This contrasts inheritance as a "white box" form of reuse, because the inheriting class has full visibility over the implementation details of the inherited class; with composition as a "black box" form of reuse, because the composing object only has access to the interface of the constituent object.

That's certainly true of Smalltalk objects, and Smalltalk is one of the Gang's example implementation languages. But even a modestly more recent language like Java has visibility attributes that let a class control what its subtypes can view or change, meaning that any modification in a subclass can be designed before we even know that a subtype is needed. Looking at this the other way, we could also say that languages like Smalltalk and Python that have advanced runtime introspection let a composing object access internal state of its constituent. This part of the argument then is contextually and historically situated, and depends on designers playing nicely within the intent of object designs, even where those aren't enforced by a language.

Another part is more compelling: inheritance is defined statically at compile time and comes with language support, which makes it easier to use than composition and harder to change. Composition is manually arranged by a programmer assigning the constituent object to a member field and calling its methods in the composing object's implementation; which is more work but easier to change at runtime. Assign a different object to the field, and get new behavior.

Further, the classes are (assuming the polite agreement above is honored) related only by the public interface of the constituent object, so there's no implementation dependency. The system's design depends on the relationships between objects at runtime, rather than the inheritance tree defined at compile time. This is presented as an advantage, which we need to consider in the context of the modern preference to rely more on the compiler as a correctness checker and static analysis tool, and to avoid deferring decisions to runtime that might turn out to be buggy.

Bear in mind that a year earlier, Barbara Liskov and Jeanette Wing proposed this principle:

What does it mean for one type to be a subtype of another? We argue that this is a semantic question having to do with the behavior of the objects of the two types: the objects of the subtype ought to behave the same as those of the supertype as far as anyone or any program using supertype objects can tell.

Within this context, the preference for composition is liberating to the designer: their type isn't morally a subtype of the thing they're extending, so they don't need to restrict themselves to a compatible interface. And indeed, Liskov had already made this point in 1987 in Data Abstraction and Hierarchy, with reference to polymorphic types:

Using hierarchy to support polymorphism means that a polymorphic module is conceived of as using a supertype, and every type that is intended to be used by that module is made a subtype of the supertype. When supertypes are introduced before subtypes, hierarchy is a good way to capture the relationships. The supertype is added to the type universe when it is invented, and subtypes are added below it later.

If the types exist before the relationship, hierarchy does not work as well[…] An alternative approach is to simply allow the polymorphic module to use any type that supplies the needed operations. In this case no attempt is made to relate the types. Instead, an object belonging to any of the related types can be passed as an argument to the polymorphic module. Thus we get the same effect, but without the need to complicate the type universe. We will refer to this approach as the grouping approach.

Liskov goes on to say that when the relationship is identified early in design, "hierarchy is a good way to express the relationship. Otherwise, either the grouping approach…or procedures as arguments may be better".

That points to a deficiency in the "composition over inheritance" aphorism: those aren't the only two games in town. If you have procedures as first-class types (like blocks in Smalltalk, or lambdas in many languages), then you might prefer those over composition or inheritance.

#### When Object-Oriented Programming Isn't
A problem I was investigating today led me to a two-line Ruby method much like this:

class App
  # ...
  def write_file_if_configured
    file_writer = FileWriter.new(@configuration.options)
    file_writer.write if file_writer.can_write?
  end
end

This method definitely looks nice and object-oriented, and satisfies many code quality rules: it's shorter than 10 lines, contains no branches, no Boolean parameters (unless there are any hiding in that options object?), indeed no parameters at all.

It also conforms to the Law of Demeter: it calls a method on one of its fields, it creates a local object and calls methods on that objects, and it doesn't send messages to any other object.

In fact there's significant Feature Envy in this method. The method is much more interested in the FileWriter than in its own class, only passing along some data. Moreover, it's not merely using the writer, it's creating it too.

That means that there's no way that this method can take advantage of polymorphism. The behaviour, all invoked on the second line, can only be performed by an instance of FileWriter, the classglobal variable invoked on the first line.

FileWriter has no independent existence, and therefore is not really an object. It's a procedure (one that uses @configuration.options to discover whether a file can be written, and write that file if so), that's been split into three procedure calls that must be invoked in sequence. There's no encapsulation, because nothing is hidden: creation and use are all right there in one place. The App class is not open to extension because the extension point (the choice of writer object) is tightly coupled to the behaviour.

Unsurprisingly that makes this code harder to "reason about" (a fancy phrase meaning "grok") than it could otherwise be, and that with no additional benefit coming from encapsulation or polymorphism. Once again, the failure of object-oriented programming is discovered to be that it hasn't been tried.

#### When security procedures go bad
My password with my bank may as well be "I can't remember, can we go through the security questions please?" That's my answer so many times when they ask, and every time it gets me in via a slightly tedious additional verification step. Losing customers probably represents a greater financial risk to them than fraud on any individual account, so they don't seem to take the password thing too seriously.

#### When single responsibility isn't possible
This posted was motivated by Rob Rix's bug report on NSObject, "Split NSObject protocol into logical sub-protocols". He notes that NSObject provides multiple responsibilities[*]: hashing, equality checking, sending messages, introspecting and so on.

What that bug report didn't look at was the rest of NSObject's functionality that isn't in the NSObject protocol. The class itself defines method signature lookups, message forwarding and archiving features. Yet more features are added via categories: scripting support (Mac only), Key-Value Coding and Key-Value Observing are all added in this way.

I wondered whether this many responsibilities in the root class were common, and decided to look at other object libraries. Pretty much all Objective-C object libraries work this way: the Object class from ObjPak, NeXTSTEP and ICPak101 (no link, sadly) all have similarly rambling collections of functionality.

[*] By extension, all subclasses of NSObject and NSProxy (which &#95;also&#95; conforms to the NSObject protocol) do, too.

Another environment I've worked a lot in is Java. The interface for java.lang.Object is mercifully brief: it borrows NSObject's ridiculous implementation of a copy method that doesn't work by default. It actually has most of the same responsibilities, though notably not introspection nor message-sending: the run-time type checking in Java is separated into the java.lang.reflect package. Interestingly it also adds a notification-based system for concurrency to the root class's feature set.

C#'s System.Object is similar to Java's, though without the concurrency thing. Unlike the Java/Foundation root classes, its copy operation (MemberwiseClone()) actually works, creating a shallow copy of the target object.

Things get a bit different when looking at Ruby's system. The Object class exposes all sorts of functionality: in addition to introspection, it offers the kind of modifications to classes that ObjC programmers would do with runtime functions. It offers methods for "freezing" objects (marking them read-only), "tainting" them (marking them as containing potentially-dangerous data), "untrusting" them (which stops them working on objects that are trusted) and then all the things you might find on NSObject. But there's a wrinkle. Object isn't really a root class: it's just the conventional root for Ruby classes. It is itself a subclass of BasicObject, and this is about the simplest root class of any of the systems looked at so far. It can do equality comparison, message forwarding (which Objective-C supports via the runtime, and NSObject has API for) and the ability to run blocks of code within the context of the receiving object.

C++ provides the least behaviour to its classes: simple constructors that are referenced but not defined can be generated.

It's useful to realise that even supposedly simple rules like "single responsibility principle" are situated in the context of the software system. Programmers will expect an object with a "single" responsibility to additionally adopt all the responsibilities of the base class, which in something like Foundation can be numerous.

#### When techs collide
If you've ever seen the film Ghostbusters, you'll know that each of the proton packs was, on its own, very powerful and capable of performing its function. Combine two, by crossing the streams, and rather than something twice as powerful you have a potentially Zuul-beating, potentially universe-destroying chaotic mess on your hands.Such a combination I found today, when I mixed a little bit of Distributed Objects with a soupçon of Cocoa Bindings to create, um, Distributed Bindings (calling it Cocoa Objects would just be rude).It's actually just as simple as you think it will be, so there's no point in any sample code. You can basically use the answer to the FAQ question on DO. Once you've got the proxy object in the client, use it as the observable controller in -bind:toObject:withKeyPath:options:. And that's it!Well, not quite it. You'll notice that things are a wee bit crashy, which is why this is somewhat like crossing the streams. Both Bindings and DO use the Proxy pattern to achieve their magic, but because the two systems have no special knowledge of each other the proxy connections are not kept in sync. If your server object mutates after the DO connection disappears, then it tries to send a KVO notification to an object that's no longer there…boom. Having a client-server system where the client can crash the server is of, well, dubious utility. There are some things that you could do to mitigate this, for instance looking for NSConnectionDidDieNotification (in the server) or opening a heartbeat RPC for the client to "check in" with the server. These fixes will work perfectly, if the vended object is updated predictably enough that you can reliably take care of broken connections between KVO notifications. Without that you're SoL, as the server won't reliably get a DO exception - more often than not it'll segfault. But while this may be unstable, it still is damned cool.

#### When to “address” “technical debt”?
The phrase “technical debt” appears in scare quotes here because, as observed in The Unreasonable Ineffectiveness of Considering Things Harmful, technical debt has quite a specific meaning and I'm talking about something broader here. Quoting Ward Cunningham:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.
Ward Cunningham, the Wycash Portfolio Management System

It's not&nbsp;old code&nbsp;that's technical debt, it's&nbsp;lightly-designed&nbsp;code. That thing that seemed like the solution when you first thought of it. Yes, ship it, by all means, but be ready to very quickly rewrite it when you learn more.

Some of what people mean when they say “we need to bring our technical debt under control” is that kind of technical debt, struggling under the compound interest of&nbsp;if&nbsp;statement accrual as multiple developers have added behaviour without adding design. But there are other things. Cutting corners is not technical debt, it's technical gambling. Updating external dependencies is not technical debt repayment, but it does still need to be done. Removing deprecated symbols is paying for somebody else's technical debt, not yours: again you still have to do it. Replacing last month's favoured&nbsp;npm&nbsp;modules with this month's is not technical debt, it's buying yourself a new toy.

But all of these things get done, and all of these things need to get done. It's the cost of deploying a system into an evolving context (and as I've said before, even that act of deployment itself triggers evolution). So the question is when, how often, how much?

Some teams put their “engineering requirements”, their name for the evolution-coping tasks, onto the same backlog as the product requirements, then try to advocate for prioritising them alongside feature requests and bug fixes. Unfortunately this rarely works: the perceived benefit of the engineering activity is zero retained customers plus zero acquired customers = zero revenue, and yet it costs the same as fixing a handful of customer-reported bugs.

So, other groups just try to carve out time. Maybe it's “20% of developer effort on the sprint is not for product tasks”. Maybe it's “there is a week between delivering one iteration and starting the next”. Maybe it's “whoever is on support rotation can pick up engineering tasks when there's no fire to put out”. And the most anti- of all the patterns is the “hardening sprint”: once per quarter we'll spend two weeks fixing the problems we've been making for ourselves in the intervening time. All of these have the benefit of giving a predictable cadence, though they still suffer a bit from that&nbsp;product envy&nbsp;problem: why are we paying for these engineers to do&nbsp;non-useful&nbsp;work when they could be steadily adding value?

The key point is that part about&nbsp;steadily&nbsp;adding value. We know the reason we need to do this: it's to avoid being brought to Ward's stand-still. We need to consolidate what we've learned, we need to evolve the system to adapt to evolutionary changes in its context, we need to fix past mistakes. And we need to do it&nbsp;constantly. Remember the quote: “Every&nbsp;minute&nbsp;spent on not-quite-right code counts as interest on that debt”.

Ultimately, these attempts to carve out time are requests to do our jobs properly, directed at people who don't have the same motivations that we do. That's not to say that their motivations are&nbsp;wrong. Like us, they only have a partial view of the overall picture. Unlike us, that view does not extend to an understanding of how expensive a liability our source code is.

When we ask for time between this iteration and the next to “service technical debt”, we are saying “I know that I'm doing a bad job, I know what I need to do to be doing a good job, and I would like to do a good job for four hours in a fortnight's time on Friday afternoon, if that's alright with you”. Ironically we do not end up doing a better job, we normalise doing a bad job for the next couple of weeks (and undoubtedly finding that some delivery/support/operations problem gets in the way for those four hours anyway).

I recommend to my mentees, reports, and any engineer who will listen to avoid advocating for time-boxed good work. I propose building the trust relationship where the people who need the code written are happy that the code is being written, and being written well, without feeling the need to check over our shoulders to see how the sausage is made. Then we don't need to justify doing a good job, and certainly don't need to ask permission: we just do it while we're going. When someone asks how long it'll take to do something, the answer is how long it'll take to do&nbsp;properly, with all the rewriting, testing, and everything else it takes to do it properly. What they get out of the end is something worth having, that doesn't need hardening, or 20% of the effort dedicated to patching it up.

And of course, what they get is something that will almost immediately need a rewrite.

#### Where am I going with this?
I recently asked how people would describe this Secure Mac Programming blog were they trying to tell someone else they should read it. Of all the answers, the one that most succinctly sums up the trouble with the old name is from Alan:

  @secboffin Not Just Secure, Not Just Mac, Not Just Programming.

I'm probably in the midst of some existential crisis, having spent a couple of years thinking and writing about philosophy, ethics, and the social responsibility of my work and its context. It's clear that I'm dealing with some conflict, and it doesn't look like reconciliation is an option.

Often I write about ideas that are still knocking around my head, such that I never come to any conclusion. I've used multiple choice conclusions, conclusions that appear to be from a different argument, and have concluded that my entire argument may or may not be useful.

This is just something I need to work out: what do I think I do, what do other people think I do, what parts of that do I like and dislike, are there other things I would like, can I replace the disliked parts with the liked parts, and so on. I write it here as you may have related ideas, or you may be thinking about the same things yourself and benefit from knowing that other people are, too.

What I know includes a list of things that currently interest me:

the intersection of object-oriented and functional programming styles
ways to make computers do things that aren't just programming
understanding the things we shouldn't need to reinvent
programming and its integration into human endeavour as a social science

With all that in mind, I'm happy to introduce the beginning of a slow rebranding of this blog. It is now called the Structure and Interpretation of Computer Programmers, and can be found at https://www.sicpers.info/ in addition to its previous home at http://blog.securemacprogramming.com.

I do not intend to remove the old domain or break existing feed subscriptions. Over time (basically, as I work out how to do it) I'll migrate links, feed entries and so on to reference the new domain, and the age-old updated mission of the blog.

#### Where are we going?
Recently, XML seems to have been playing an important role for me...I've been working with various XML-RPC frameworks, reviewing a book on XHTML (actually, a book on HTML 4 which occasionally reminds you to close elements) and also dealing with DocBook.  For some reason, this means when I read something like Ten predictions for XML in 2007 I feel like I'm in a position to comment.  Don't worry, I don't want to cover all ten points...

If I had to choose one big story for next year, it would be the Atom Publishing Protocol (APP). [...] APP is the first major protocol to be based on Representational State Transfer (REST), the architecture of the Web. Most systems to date have only used a subset of HTTP, usually GET and POST but not PUT or DELETE. Many systems like SOAP and Web-based Distributed Authoring and Versioning (WebDAV) have been actively contradictory to the design of HTTP. APP, by contrast, is working with HTTP rather than against it.

Well, this suits me...a complete bi-directional implementation of HTTP would certainly be pleasant to use.  But the point is that you already can do bi-directional transactions with these ugly hack interfaces, which importantly are already in use.  This isn't like moving from rcp to HTTP to do your file uploads, this is like moving from HTTP to a version of HTTP which IE doesn't recognise for doing your content publication.  WebDAV might be ugly but it already allows people to run their Subversion or calendar (yes, I know...) servers over &quot;the web&quot;.  APP is good but the activation energy may be too high.

 2007 is the make-or-break year for the Semantic Web.

s/make-or-//, I believe.  I've talked to two people who considered themselves important in the world of the semantic web, and have attended a talk by Sir Tim on the subject.  I still have yet to see anything beyond so-called blue skies proposals and views on how much better the web will be once everyone embraces semantics, which none of them appeared to have done.  At least, there's nothing yet which has convinced me that Semantic Web is some kind of killer app, and that what it can do can't already be done with a bit of XML/XSLT and some good schemas.  Maybe that's the point, that traditional web will just slide into Semanticism by dint of XQuery, XProc and the rest of the X* bunch.  I don't think so.

 2007 will be the first year in which almost every significant browser fully supports XSLT 1.0 [...] I predict that this will render many of the debates about HTML 5 and XHTML 2 moot. Sites will publish content in whatever XML vocabulary they like, and provide stylesheets that convert it to HTML for browser display.

Yup, this couldn't come too soon.  I sorely hope that HTML 5 will be still-born; redundant as soon as it becomes available.  I also hope that document-generating frameworks will become better at generating valid markup in the future (which is at least easier to do with XML than SGML)...of course if browsers really do make a good job of supporting XSLT then they won't really be able to generate invalid markup as they'll be doing it in their own format.

 Apple will release Safari 3 along with Leopard. Although it will focus mostly on Apple proprietary extensions, Safari 3 will add support for Scalable Vector Graphics (SVG) for the first time.

That's not much of a prediction, unless predicting things means reading the WebKit changelog.  What would be good is for a couple of the more popular also-ran browsers to support MathML (I think Firefox already does in some fashion, WebKit is at the &quot;we're thinking about it&quot; stage, and I really don't know about any others) so that it has a chance of being adopted.  I speak here as an ex-physicist who was annoyed at seeing academics putting DVI or PostScript files on the web, or (and this is just as bad if not worse) HTML with graphics of the equations.  If LaTeX could be transformed into HTML+MathML and published on the web then we'd be somewhere approaching the original goal of the Nexus.

#### Where We Ditched Chipzilla
WWDC2020 was the first WWDC I’ve been to in, what, five years? Whenever I last went, it was in San Francisco. There’s no way I could’ve got my employer to expense it this year had I needed to go to San Jose, nor would I have personally been able to cover the costs of physically going. So I wouldn’t even have entered the ticket lottery.

Lots of people are saying that it’s “not the same” as physically being there, and that’s true. It’s much more accessible than physically being there.

For the last couple at least, Apple have done a great job of putting the presentations on the developer site with very short lag. But remotely attending has still felt like being the remote worker on an office-based team: you know you’re missing most of the conversations and decisions.

This time, everything is remote-first: conversations happen on social media, or in the watch party sites, or wherever your community is. The bundling of sessions released once per day means there’s less of a time zone penalty to being in the UK, NZ, or India than in California or Washington state. Any of us who participated are as much of a WWDC attendee as those within a few blocks of the McEnery or Moscone convention centres.

#### Which vendor "is least secure"?
The people over at Intego have a blog post, Which big vendor is least secure? They discuss that because Microsoft have upped their game, malware authors have started to target other products, notably those produced by Adobe and Apple.

That doesn't really address the question though: which big vendor is least secure (or more precisely, which big vendor creates the least secure products)? It's an interesting question, and one that's so hard to answer, people usually get it wrong.

The usual metrics for vendor software security are:
Number of vulnerability reports/advisories last year
Speed of addressing reported vulnerabilities

Both are just proxies for the question we really want to know the answer to: "what risk does this product expose its users to?" Each has drawbacks when used as such a proxy.

The previous list of vulnerabilities seems to correlate with a company's development practices - if they were any good at threat modelling, they wouldn't have released software with those vulnerabilities in, right? Well, maybe. But maybe they did do some analysis, discovered the vulnerability, and decided to accept it. Perhaps the vulnerability reports were actually the result of their improved secure development lifecycle, and some new technique, tool or consultant has patched up a whole bunch of issues. Essentially all we know is what problems have been addressed and who found them, and we can tell something about the risk that users were exposed to while those vulnerabilities were present. Actually, we can't tell too much about that, unless we can find evidence that it was exploited (or not, which is harder). We really know nothing about the remaining risk profile of the application - have 1% or 100% of vulnerabilities been addressed?

The only time we really know something about the present risk is in the face of zero-day vulnerabilities, because we know that a problem exists and has yet to be addressed. But reports of zero-days are comparatively rare, because the people who find them usually have no motivation to report them. It's only once the zero-day gets exploited, and the exploit gets discovered and reported that we know the problem existed in the first place.

The speed of addressing vulnerabilities tells us some information about the vendor's ability to react to security issues. Well, you might think it does, it actually tells you a lot more about the vendor's perception of their customers' appetite for installing updates. Look at enterprise-focussed vendors like Sophos and Microsoft, and you'll find that most security patches are distributed on a regular schedule so that sysadmins know when to expect them and can plan their testing and deployment accordingly. Both companies have issued out-of-band updates, but only in extreme circumstances.

Compare that model with Apple's, a company that is clearly focussed on the consumer market. Apple typically have an ad hoc (or at least opaque) update schedule, with security and non-security content alike bundled into infrequent patch releases. Security content is simultaneously released for some earlier operating systems in a separate update. Standalone security updates are occasionally seen on the Mac, rarely (if ever) on the iPhone.

I don't really use any Adobe software so had to research their security update schedule specifically for this post. In short, it looks like they have very frequent security updates, but without any public schedule. Using Adobe Reader is an exercise in unexpected update installation.

Of course, we can see when the updates come out, but that doesn't directly mean we know how long they take to fix problems - for that we need to know when problems were reported. Microsoft's monthly updates don't necessarily address bugs that were reported within the last month, they might be working on a huge backlog.

Where we can compare vendors is situations in which they all ship the same component with the same vulnerabilities, and must provide the same update. The more reactive companies (who don't think their users mind installing updates) will release the fixes first. In the case of Apple we can compare their fixes of shared components like open source UNIX tools or Java with other vendors - Linux distributors and Oracle mainly. It's this comparison that Apple frequently loses, by taking longer to release the same patch than other Oracle, Red Hat, Canonical and friends.

So ultimately what we'd like to know is "which vendor exposes its customers to most risk?", for which we'd need an honest, accurate and comprehensive risk analysis from each vendor or an independent source. Of course, few customers are going to want to wade through a full risk analysis of an operating system.

#### Whither the codesign interface?
One of the higher-signal-level Apple mailing lists with a manageable amount of traffic is apple-cdsa, the place for discussing the world's most popular Common Data Security Architecture deployment. There's currently an interesting thread about code signatures, which asks the important question: how do I make use of code signatures?Well, actually, it's not so much about how I can use code signatures, but how the subset of Clapham Omnibus riders who own Macs (a very small subset, as the combination of overheating batteries in the old G4 PowerBooks and combustible bendy-busses means they don't stay around very long) can use code signatures. Unfortunately, the answer seems to currently be "not by much", with little impression of that changing. The code signing process and capability is actually pretty damned cool, and a nice security feature which I'll be talking about at MacDev 09. It's used to good effect in the iPhone, where it and FairPlay DRM are part of that platform's locked-down execution environment.The only problem is, there's not much a user can do with it. It's pretty hard to find out who signed a particular app, in fact the only thing you can easily do is discover that the same entity signed two versions of the same app. And that's by lack of interface, not by any form of dialogue or confirmation. That means that when faced with the "Foobar app has changed. Are you sure you still want to allow it to [whatever]" prompt, many users will be unaware of the implications of the question. Those who are and (sensibly) want to find out why the change has occurred will quickly become frustrated. Therefore everyone's going to click "allow", which rather reduces the utility of the feature :-(.Is that a problem yet? Well, I believe it is, even though there are few components yet using the code signature information in the operating system. And it's precisely that allow-happy training which I think is the issue. By the time the user interfaces and access control capabilities of the OS have developed to the point where code signing is a more useful feature (and believe me, I think it's quite a useful one right now), users will be in the habit of clicking 'allow'. You are coming to a sad realisation; allow or deny?

#### Whoever "wins", software freedom loses
I'd like to start by recapping the three distinct categories of interest in software freedom. This is definitely my categorisation, though only the third is novel and the first two have long histories of common recognition so this is hardly Humpty-Dumptyism on my part.


Free Software
The extension of freedoms of expression and engagement into the digital space. Free Software, sometimes "Libre Software" because of the confusion over the word "Free"'s multiple definitions, is based on the ideas that a computer is property like any other artefact and that working with, playing with, and socialising via computers are personal pursuits like any other pursuits, and that the freedom from external interference with those enjoyments should be the same as in non-computer interests.
Open Source
The rephrasing of the ideas of Free Software to improve acceptance in (particularly American) business circles. Open Source as described is almost identical to the Debian project's ideas of Free Software, but with the words "Open Source" instead of "Free Software" and the words "Debian software component" removed. The first reason for the rename is that Freedom implies either zero cost, which mid-1990s American business didn't like, or social good, same. The second reason is that mid-1990s American businesses had come around to ideas of interoperability under the banner Open Systems, and Open Source sounds sort of like that a bit.
Open Sores
The co-opting of the technical aspects of Open Source (or, nearly equivalently, Free Software) without any of the freedom benefits, typically with the goal of providing zero-cost software development and associated professional services to for-profit companies. When a company CTO says "we love open source", they typically mean that they love open sores: that they love how skilled developers from across the world will gladly sign CLAs transferring rights to exploit their creations to the company in return for a lighter green square on the proprietary software-as-a-service platform Github.


This is all pre-amble to a discussion of #uninstallman, the internet pressure mob removing the leadership of the Free Software Foundation over objectionable statements made by its founder, former president, and recently-surprise-reinstated board member, Richard M. Stallman (rms).

Let's start with the obvious: the Free Software Foundation has not demonstrated good leadership over this matter. Clearly rms's statements have distracted the conversation away from software freedom, and the FSF have not taken enough steps with enough publicity to resolve this issue and to get people talking about software freedom again[*]. The FSF has not even given clear enough separation between their policy and rms's personal views for it to be obvious that anyone else on their board has any views, or control over policy.

It is right that the FSF take a critical look at their management, and ask whether the people who are leading the Foundation are the best people to promote the idea of software freedom.

Unfortunately, we are now at the point where whatever the outcome, software freedom has lost and Open Sores will fill the ideological vacuum. Because if software freedom is about the extension of existing freedoms into the online space, and the baying mob are calling for the blood of someone who said a thing questioning the definitions of words related to the actions of someone who was associated with someone who did known bad things, and for the blood of any other people who are associated with that person, it is easy to argue that the whole software freedom movement is hypocritical. You claim to support freedom of expression, and yet you actually deny the right for anyone to express views that disagree with your own? Where's the unity of purpose?

Bradley Kuhn, policy fellow at the Software Freedom Conservancy, has talked about the damaging impact of rms's personal views on the software freedom movement back in 2019, when this controversy was fresh; and in 2018 which is arguably where it started (to become public). He has also talked about the need to maintain a big tent; that being principled on your core issue gives you the legitimacy to take principled stands on other issues.

Taking an authoritarian, only-say-what-I-permit line on expression doesn't leave any legitimacy to support freedom of expression in the software field. Unfortunately, if the FSF and more generally the software freedom community is unable to maintain principle on this argument, it will lose the right to be taken seriously on matters of software freedom. And then, the organisations who take the Open Sores line on software licensing will step up to fill the leadership vacuum. The business interest "Foundations" who think that software freedom means the freedom for big businesses to control the revenue stream while everybody gets to build their products for free. And then it may be decades before there is another software freedom movement with any legitimacy, and they may have to start from scratch.

[*] Arguably the software freedom movement was already in a difficult state, because the freedoms proposed were only really adopted by a small community with a technical interest in the details relating to those freedoms: a few tens of thousands of technologists, some intellectual property lawyers, and a small number of others. But that's more about the difficulty of developing a mass movement and of translating the theory into activism, and doesn't necessarily reflect badly on the characters or actions of any of the leaders in the movement.

#### Why 80?
80 characters per line is a standard worth sticking to, even today. OK, why?

Well, back up. Let's examine the axioms. Is 80 characters per line a standard? Not really, it's a convention. IBM cards (which weren't just made by IBM or read by IBM machines) were certainly 80 characters wide, as were DEC video terminals, which Macs etc. emulate. Actually, that's not even true. The DEC VT-05 could display 72 characters per line, their later VT-50 and successor models introduced 80 characters. The VT-100 could display 132 characters per line, the same quantity as a line printer (including the ones made by IBM). Other video terminals had 40 or 64 character lines. Teletypewriters typically had shorter lines, like 70 characters.

Typewriters were typically limited to \((\mathrm{width\ of\ page} - 2 \times \mathrm{margin\ width}) \times \mathrm{character\ density}\) characters per line. With wide margins and narrow US paper, you might get 50 characters: with narrow margins and wide A4 paper, maybe 100.

IBM were not the only people to make cards, punches, and readers. Other manufacturers did, with other numbers of characters per card. IBM themselves made 40, 45 and 96 column cards. Remington Rand made cards with 45 or 90 columns.

So, axiom one modified, "80 characters per line is a particular convention out of many worth sticking to, even today." Is it worth sticking to?

Hints are that it isn't. The effects of line length on reading online news explored screen-reading with different line lengths: 35, 55, 75 and 95 cpl. They found, from the abstract:
Results showed that passages formatted with 95 cpl resulted in faster reading speed. No effects of line length were found for comprehension or satisfaction, however, users indicated a strong preference for either the short or long line lengths.
However that isn't a clear slam dunk. Quoting their reference to prior work:
Research investigating line length for online text has been inconclusive. Several studies found that longer line lengths (80 – 100 cpl) were read faster than short line lengths (Duchnicky and Kolers, 1983; Dyson and Kipping, 1998). Contrary to these findings, other research suggests the use of shorter line lengths. Dyson and Haselgrove (2001) found that 55 characters per line were read faster than either 100 cpl or 25 cpl conditions. Similarly, a line length of 45-60 characters was recommended by Grabinger and Osman-Jouchoux (1996) based on user preferences. Bernard, Fernandez, Hull, and Chaparro (2003) found that adults preferred medium line length (76 cpl) and children preferred shorter line lengths (45 cpl) when compared to 132 characters per line.
So, long lines are read faster than short lines, except when they aren't. They also found that most people preferred the longest or shortest lines the most, but also that everybody preferred the shortest or longest lines the least.

But is 95cpl a magic number? What about 105cpl, or 115cpl? What about 273cpl, which is what I get if I leave my Terminal font settings alone and maximise the window in my larger monitor? Does it even make sense for programmers who don't have to line up the comment markers in Fortran-77 code to be using monospaced fonts, or would we be better off with proportional fonts?

And that article was about online news articles, a particular and terse form of prose, being read by Americans. Does it generalise to code? How about the observation that children and adults prefer different lengths, what causes that change? Does this apply to people from other countries? Well, who knows?

Buse and Weimer found that "average line length" was "strongly negatively correlated" with perceived readability. So maybe we should be aiming for one-character lines! Or we can offset the occasional 1,000 character line by having lots and lots of one-character lines:
}
}
}
}
}
}

It sounds like there's information missing from their analysis. What was the actual shape of the data? What were the maximum and minimum line lengths considered, what distribution of line lengths was there?

We're in a good place to rewrite the title from the beginning of the post: 80 characters per line is a particular convention out of many that we know literally nothing about the benefit or cost of, even today. Maybe our developer environments need a bit of that UX thing we keep imposing on everybody else.

#### Why are we like this?
The recent post on addressing “technical debt” did the rounds of the usual technology forums, where it raised a reasonable question: why are people basing these decisions on balancing engineering-led with customer-led tasks on opinion? Why don't engineers take an evidence-based approach to such choices?

The answer is complex but let's start at the top: there's too much money in software. There have been numerous crises in the global economy since software engineering has existed, but really the only one with any material effect on the software sector was the dot-com crash. The lesson there was “have a business plan”: plenty of companies had raised billions in speculative funding on the basis that they were on the internet but once the first couple started to fold, the investors pulled out en masse and the money was sucked from the room. This is the time that gave us Agile (constantly demonstrate that you're delivering value to your customer), Lean Startup (demonstrate that you're adding value with as little expenditure as possible), and Lean Software Development (eliminate all of the waste in your process).

Nobody ever demonstrated that Agile, Lean or whatever were better in some objective metric, what they did was tell convincing stories. Would you like to find out that you've built the wrong thing two years from now, or two weeks from now? Would you prefer to read an interim draft functional specification, or use working software? Let's be clear though, nobody ever showed that what we were doing before that was better in any objective way either: software was written by defence contractors and electronics hardware companies, and they grandfathered in the processes used to procure and develop hardware. You can count the number of industry pundits advocating for a genuinely evidence-led approach to software cost control on two fingers (Barry Boehm and Watts Humphries) and you can still raise valid questions about the validities of either of their systems.

Since then, software teams have become less fragile to economic shock. This was already happening in the 2007 credit crunch (the downturn at the beginning of the 2007-2008 global financial crisis). The CFO where I worked explained that bookings of their subscription-based software would go up during a recession. Why? Because people were not confident enough to buy outright or to enter relatively cheap, long-term arrangements like three year contracts. They would instead take the more expensive but more flexible shorter-term contracts so that they could cancel or move if their belts needed tightening. After the crisis, the adoption of subscription-based pricing models has only increased in software, and extended to adjacent fields like media and hardware.

All of this means that there is relative stability in software businesses, and there is still growing demand for software engineers. That has meant that there isn't the need for systematic approaches to cost-reduction hawked by every single thinker in the “software crisis” era: note that there hasn't been significant movement beyond Agile, Lean or whatever in the subsequent two decades. They're good enough, and there is no impetus to find out what's better. In fact both Agile with its short increments and Lean Startup with its pivoting are optimised for the “get out quickly at any cost” flexibility that also leads customers to choose short-term subscription pricing: when the customers for your VR pet grooming business dry up you can quickly pivot to online fraud detection.

With no need to find or implement better approaches there's also no need to particularly require software engineers to have a systematic approach or a detailed understanding of the knowledge of their industry. Thus software engineering—particularly programming—remains a craft-based discipline where anyone with an interest can start out at the bottom, learn on the job through mentoring and self-study, and use a process of survivor bias to get along. Did anyone demonstrate in 2002 that there's objective benefit to a single-page application? Did anyone demonstrate in 2008 that there's objective benefit to a native mobile app? Did anyone demonstrate in 2016 that there's objective benefit to a Dapp? Has anyone crunched the numbers to resolve whether DevOps or Site Reliability Engineering is the one true way to do operations? No, but it doesn't matter: there's more than enough money to put into these things. And indeed most of those choices listed above are immaterial to where the money comes from or goes, but would be the sorts of “technical debt” transitions that engineering teams struggle to pay for.

You might ask why I'm at all interested in taking a systematic approach to our work when I also think it's not necessary. Even if it isn't necessary for survival, it's definitely professional, and justifiable. When the clients do come to reduce their expenditure, or even when they don't but are deciding who to go with, the people who can demonstrate that they systematically maximise the output of their work will be the preferred choice.

#### Why are you using the wrong licence?
I frequently see posts/articles/screeds asking why people don't contribute to open source. If it's important that recipients of open source software contribute upstream, and you are angry when they don't, why use licences like MIT, Apache, GPL or BSD that don't require upstream collaboration?

Back in the day, Apple released their public source code under version 1 of the Apple Public Source Licence, which required users who changed the source to fill in a form notifying Apple of their changes. You could do the same, and not be angry.

#### Why do we annoy our users?
I assume that, with my audience being mainly Mac users, you are not familiar with Microsoft Security Assessment Tool, or MSAT. It's basically a free tool for CIOs, CSOs and the like to perform security analyses. It presents two questionnaires, the first asking questions about your company's IT infrastructure ("do you offer wireless access?"), the second asking about the company's current security posture ("do you use WPA encryption?"). The end result is a report comparing the company's risk exposure to the countermeasures in place, highlighting areas of weakness or overinvestment. The MSAT app itself isn't too annoying.

Mostly. One bit is. Some of the questions are accompanied by information about the relevant threats, and industry practices that can help mitigate the appropriate threats. Information such as this:

So, how does changing a password reduce the likelihood of a brute-force attack succeeding? Well, let's think about it. The attacker has to choose a potential password to test. Obviously the attacker does not know your password a priori, or the attack wouldn't be brute-force; so the guess is independent of your password. You don't know what the attacker has, hasn't, or will next test&mdash;all you know is that the attacker will exhaust all possible guesses given enough time. So your password is independent of the guess distribution.

Your password, and the attacker's guess at your password, are independent. The probability that the attacker's next guess is correct is the same even if you change your password first. Password expiration policies cannot possibly mitigate brute-force attacks.

So why do we enforce password expiration policies? Actually, that's a very good question. Let's say an attacker does gain your password.

OK, &quot;an attacker does gain your password.&quot;

The window of opportunity to exploit this condition depends on the time for which the password is valid, right? Wrong: as soon as the attacker gains the password, he can install a back door, create another account or take other steps to ensure continued access. Changing the password post facto will defeat an attacker who isn't thinking straight, but ultimately a more comprehensive response should be initiated.

So password expiration policies annoy our users, and don't help anyone.

#### Why I don't have a favourite programming language
This is my take on Ilya Sher's similar post, though from a different context. He is mainly interested in systems programming, I have mostly written user apps and backend services, and also some developer tools.

I originally thought that I would write a list of the languages and difficulties I have with them, but I realised that there's an underlying theme that can be extracted. Programming languages I have used either have too much vendor dependence (I love writing ObjC, but can't rely on GNUstep when I'm not on Apple), too little interaction with the rest of the software world (I love writing Pharo, but don't love going through its FFI to use anything else) or, and this is the biggest kicker, I don't like the development environments.

When I work on JavaScript, my environment is a text editor (something like VSCode or emacs) that has syntax highlighting, maybe has auto-completion...and that's about it. When I work in something like Java, ObjC or C++, I have a build button, an integrated debugger, and the ability to run tests. And, if I'm lucky, a form designer. When I work in something like Swift or Clojure, I have insta-repls. When I work in Pharo, I have all the live browsers and things you hear about from smug people, but I still have to type code for things you might expect to be 'live' in such an environment. I get confused by the version control tools, but that might be because I'm not familiar with image-based development.

It feels like, details of the languages aside, there's a synthesis of programming language with environment, where the programming language is a tool integrated into the environment just like the compiler and debugger, and the tools are integrated into the programming language, like the Lisp macro system. It feels like environments like Oberon, Lisp machines and Smalltalks all have some of this integration, and that popular programming environments for other languages all have less of it.

I'm not entirely sure what the ideal state is, and whether that's an ideal just for me or would benefit others. I wrote my MSc thesis on an exploration of this problem, and still have more research to do.

#### Why inheritance never made any sense
There are three different types of inheritance going on.

 	Ontological inheritance is about specialisation: this thing is a specific variety of that thing (a football is a sphere and it has this radius)
 	Abstract data type inheritance is about substitution: this thing behaves in all the ways that thing does and has this behaviour (this is the Liskov substitution principle)
 	Implementation inheritance is about code sharing: this thing takes some of the properties of that thing and overrides or augments them in this way. The inheritance in my post On Inheritance is this type and only this type of inheritance.

These are three different, and frequently irreconcilable, relationships. Requiring any, or even all, of them, presents no difficulty. However, requiring one mechanism support any two or more of them is asking for trouble.

A common counterexample to OO inheritance is the relationship between a square and a rectangle. Geometrically, a square is a specialisation of a rectangle: every square is a rectangle, not every rectangle is a square. For all s in Squares, s is a Rectangle and width of s is equal to height of s. As a type, this relationship is reversed: you can use a rectangle everywhere you can use a square (by having a rectangle with the same width and height), but you cannot use a square everywhere you can use a rectangle (for example, you can't give it a different width and height).

Notice that this is incompatibility between the inheritance directions of the geometric properties and the abstract data type properties of squares and rectangles; two dimensions which are completely unrelated to each other and indeed to any form of software implementation. We have so far said nothing about implementation inheritance, so haven't even considered writing software.

Smalltalk and many later languages use single inheritance for implementation inheritance, because multiple inheritance is incompatible with the goal of implementation inheritance due to the diamond problem (traits provide a reliable way for the incompatibility to manifest, and leave resolution as an exercise to the reader). On the other hand, single inheritance is incompatible with ontological inheritance, as a square is both a rectangle and an equilateral polygon.

The Smalltalk blue book describes inheritance solely in terms of implementation inheritance:
A subclass specifies that its instances will be the same as instances of another class, called its superclass, except for the differences that are explicitly stated.
Notice what is missing: no mention that a subclass instance must be able to replace a superclass instance everywhere in a program; no mention that a subclass instance must satisfy all conceptual tests for an instance of its superclass.

Inheritance was never a problem: trying to use the same tree for three different concepts was the problem.

"Favour composition over inheritance" is basically giving up on implementation inheritance. We can't work out how to make it work, so we'll avoid it: get implementation sharing by delegation instead of by subclassing.

Eiffel, and particular disciplined approaches to using languages like Java, tighten up the "inheritance is subtyping" relationship by relaxing the "inheritance is re-use" relationship (if the same method appears twice in unrelated parts of the tree, you have to live with it, in order to retain the property that every subclass is a subtype of its parent). This is fine, as long as you don't try to also model the problem domain using the inheritance tree, but much of the OO literature recommends that you do by talking about domain-driven design.

Traits approaches tighten up the "inheritance is specialisation" relationship by relaxing the "inheritance is re-use" relationship (if two super categories both provide the same property of an instance of a category, neither is provided and you have to write it yourself). This is fine, as long as you don't try to also treat subclasses as covariant subtypes of their superclasses, but much of the OO literature recommends that you do by talking about Liskov Substitution Principle and how a type in a method signature means that type or any subclass.

What the literature should do, I believe, is say "here are the three types of inheritance, focus on any one of them at a time". I also believe that the languages should support that (obviously Smalltalk, Ruby and friends do support that by not having any type constraints).

 	If I'm using inheritance as a code sharing tool, it should not be assumed that my subclasses are also subtypes.
 	If I am using subtypes to tighten up interface contracts, I should be not only allowed to mark a class anywhere in the tree as a subtype of another class anywhere in the tree, but required to do so: once again, it should not be assumed that my subclasses are also subtypes.
 	If I need to indicate conceptual specialisation via classes, this should also not be assumed to follow the inheritance tree. I should be not only allowed to mark a class anywhere in the tree as a subset of another class, but required to do so: once again, it should not be assumed that my subclasses are also specialisations.

Your domain model is not your object model. Your domain model is not your abstract data type model. Your object model is not your abstract data type model.

Now inheritance is easy again.

#### Why is programming so hard?
I have been reflecting recently on what it was like to learn to program. The problem is, I don't clearly remember: I do remember that there was a time when I was no good at it. When I could type a program in from INPUT or wherever, and if it ran correctly I was golden. If not, I was out of luck: I could proof-read the listing to make sure I had introduced no mistakes in creating my copy from the magazine, but if it was the source listing itself that contained the error, I wasn't about to understand how to fix it.

The programs I could create at this stage were incredibly trivial, of the INPUT "WHAT IS YOUR NAME"; N$: IF N$="GRAHAM" THEN PRINT "HELLO, MY LORD" ELSE PRINT "GO AWAY ";N$ order of complexity. But that program contains pretty much most of what there is to computing: input, output, memory storage and branches. What made it hard? I'll investigate whether it was BASIC itself that made things difficult later. Evidently I didn't have a good grasp of what the computer was doing, anyway.

I then remember a time, a lot later, when I could build programs of reasonable complexity that used standard library features, in languages like C and Pascal. That means I could use arrays and record types, procedures, and library functions to read and write files. But how did I get there? How did I get from 10 PRINT "DIXONS IS CRAP" 20 GOTO 10 to building histograms of numeric data? That's the bit I don't remember: not that things were hard, but the specific steps or insights required to go from not being able to do a thing to finding it a natural part of the way I work.

I could repeat that story over and over, for different aspects of programming. My first GUI app, written in Delphi, was not much more than a "fill in the holes" exercise using its interface builder and code generator. I have an idea that my understanding of what I thought objects and classes were supposed to do was sparked by a particular training course I took in around 2008, but I still couldn't point to what that course told me or what gaps in my knowledge it filled. Did it let me see the bigger picture around facts I already knew, did it correct a fallacious mental model, or did it give me new facts? How did it help? Indeed, is my memory even correct in pinpointing this course as the turning point? (The course, by the way, was Object-Oriented Analysis and Design Using UML.)

Maybe I should be writing down instances when I go from not understanding something to understanding it. That would work if such events can be identified: maybe I spend some time convincing myself that I do understand these things while I still don't, or tell myself I don't understand these things long after I do.

One place I can look for analogies to my learning experience is at teaching experience. A full litany of the problems I've seen in teaching programming to neophytes (as opposed to professional training, like teaching Objective-C programming to Rubyists, which is a very different thing) would be long and hard to recall. Tim Love, has seen and recorded similar problems to me (as have colleagues I've talked to about teaching programming).

A particular issue from that list that I'll dig into here is the conflation of assignment and equality. The equals sign (=) was created in the form of two parallel lines of identical length, as no two things could be more equal. But it turns out then when used in many programming languages, in fact two things related by = could be a lot more equal. Here's a (fabricated, but plausible) student attempt to print a sine table in C (preprocessor nonsense elided).

int main() {
  double x,y;
  y = sin(x);
  for (x = 0; x &lt;= 6.42; x = x + 0.1)
    printf("%lf %lf\n", x, y);

}

Looks legit, especially if you've done any maths (even to secondary school level). In algebra, it's perfectly fine for y to be a dependent variable related to x via the equality expressed in that program, effectively introducing a function y(x) = sin(x). In fact that means the program above doesn't look legit, as there are not many useful solutions to the simultaneous equations x = 0 and x = x + 0.1. Unfortunately programming languages take a Humpty-Dumpty approach and define common signs like = to mean what they take them to mean, not what everybody else is conventionally accepting them to mean.

Maybe the languages themselves make learning this stuff harder, with their idiosyncrasies like redefining equality. This is where my musing on BASIC enters back into the picture: did I find programming hard because BASIC makes programming hard? It's certainly easy to cast, pun intended, programming expertise as accepting the necessity to work around more roadblocks imposed by programming tools than inexpert programmers are capable of accepting. Anyone who has managed to retcon public static void main(String[] args) into a consistent vision of something that it's reasonable to write every time you write a program (and to read every time to inspect a program, too) seems more likely to be subject to Stockholm Syndrome than to have a deep insight into how to start a computer program going.

We could imagine it being sensible to introduce neophytes to a programming environment that exposes the elements of programming with no extraneous trappings or aggressions. You might consider something like Self, which has the slot and message-sending syntax as its two features. Or LISP, which just has the list and list syntax. Or Scratch, which doesn't even bother with having syntax. Among these friends, BASIC doesn't look so bad: it gives you tools to access its model of computation (which is not so different from what the CPU is trying to do) and not much more, although after all this time I'm still not entirely convinced I understand how READ and DATA interact.

Now we hit a difficult question: if those environments would be best for beginners, why wouldn't they be best for anyone else? If Scratch lets you computer without making mistakes associated with all the public static void nonsense, why not just carry on using Scratch? Are we mistaking expertise at the tools with expertise at the concepts, or what we currently do with what we should do, or complexity with sophistication? Or is there a fundamental reason why something like C++, though harder and more complex, is better for programmers with some experience than the environments in which they gained that experience?

If we're using the wrong tools to introduce programming, then we're unnecessarily making it hard for people to take their first step across the threshold, and should not be surprised when some of them turn away in disgust. If we're using the wrong tools to continue programming, then we're adding cognitive effort unnecessarily to a task which is supposed to be about automating thought. Making people think about not making people think. Masochistically imposing rules for ourselves to remember and follow, when we're using a tool specifically designed for remembering and following rules.

#### Why Java is so damned lame, part exp(I*M_PI)
Gah.  Back when I used to work for Oxford University, I had to do the occasional bit of Java programming for a WebObjects app.  Being quite a bit more familiar with ObjC than with Java, I always found this a bit of a headache...partly the way Java Foundation is semi-bridged with the &quot;real&quot; Java API meant I was constantly referring to my Tiger book (Java in a Nutshell, not Mac OS X Tiger), and partly because ironically Java required a lot more mystical casting voodoo than ObjC...seriously, if I never see the phrase Session session=(Session)session() again, I'll be a happy man.

Today's &quot;gah, why can't Java be as easy as ObjC&quot; moment came courtesy of an algorithmic problem I was attacking at TopCoder, just for the hell of it.  Problem is, to get to their problem definitions you have to use their applet thingy, and while I'm there I decided that I may as well type the code into their thingy after all Java's not that bad is it?  [They also accept VB, C# and C++ but my recollection of the C++ class syntax is slim and my knowledge of the STL is slimmer...I do know enough to try and avoid it though]  So I think their problem definitions are proprietary, but suffice to say I wanted the ability to compare two strings, returning equal if the strings could be made the same by popping any arbitrary number of characters off the front and shifting them onto the back (i.e. treating them like rings of characters).

Well, that's simple isn't it?  In ObjC I'd subclass NSString, override isEqual: and Robert is your parent's sibling.  I even know how to do that in Java:

class CircularString extends String {  public boolean equals(Object otherObj) {    //....  }}

Only...no.  You're not allowed to do that, because for some unfathomable reason Gosling and the boys at Oak decided to inflict on us another voodoo keyword...final.  This seems to have one purpose in existence: to stop me from subclassing the String class.  In the words of Points of View, Why oh why oh why oh why?

#### Why mock objects aren’t popular this week
The field of software engineering doesn’t change particularly quickly. Tastes in software engineering change all the time: keeping up with them can quickly result in seasickness or even whiplash. For example, at the moment it’s popular to want to do server-side rendering of front end applications, and unpopular to do single-page web apps. Those of us who learned the LAMP stack or WebObjects are back in fashion without having to lift a finger!

Currently it’s fashionable to restate “don’t mock an interface you don’t own” as the more prescriptive, taste-driven statement “mocks are bad”. Rather than change my practice (I use mocks and I’m happy with that from 2014 is still OK), I’ll ask why has this particular taste arisen.

Mock objects let you focus on the ma, the interstices between objects. You can say “when my case controller receives this filter query, it asks the case store for cases satisfying this predicate”. You’re designing a conversation between independent programs, making restrictions about the messages they use to communicate.

But many people don’t think about software that way, and so don’t design software that way either. They think about software as a system that holistically implements a goal. They want to say “when my case controller receives this filter query, it returns a 200 status and the JSON representation of cases matching that query”. Now, the mocks disappear, because you don’t design how the controller talks to the store, you design the outcome of the request which may well include whatever behaviour the store implements.

Of course, tests depending on the specific behaviour of collaborators are more fragile, and the more specific prescription “don’t mock what you don’t control” uses that fragility: if the behaviour of the thing you don’t control changes, you won’t notice because your mock carries on working the way it always did.

That problem is only a problem if you don’t have any other method of auditing your dependencies for fitness for purpose. If you’re relying on some other interface working in a particular way then you should probably also have contract tests, acceptance tests, or some other mechanism to verify that it does indeed work in that way. That would be independent of whether your reliance is captured in tests that use mock objects or some other design.

It’ll only be a short while before mock objects are cool again. Until then, this was an interesting diversion.

#### Why nerd culture must die
http://computinged.wordpress.com/2014/11/13/why-nerd-culture-must-die-not-everyone-can-teach-themselves/

Truth.

#### Why OS X (almost) doesn't need root any more
Note: this post was originally written for the Mac Developer Network.

In the beginning, there was the super-user. And the super-user was root.

When it comes to doling out responsibility for privileged work in an operating system, there are two easy ways out. Single-user operating systems just do whatever they're told by whoever has access, so anyone can install or remove software or edit configuration. AmigaDOS, Classic Mac OS and MS-DOS all took this approach.

The next-simplest approach is to add multiple users, and let one of them do everything while all the others can do nothing. This is the approach taken by all UNIX systems since time immemorial - the root user can edit all files, set access rights for files and devices, start network services on low-numbered ports...and everyone else can't.

The super-user approach has obvious advantages in a multi-user environment over the model with no privilege mechanism - only users who know how to log in as root can manage the computer. In fact it has advantages in a single-user environment as well: that one user can choose to restrict her own privileges to the times when she needs them, by using a non-privileged account the rest of the time.

It's still a limited mechanism, in that it's all-or-nothing. You either have the permission to do everything, or you don't. Certain aspects like the ability to edit files can be delegated, but basically you're either root or you're useless. If you manage to get root - by intention or by malicious exploitation - you can do anything on the computer. If you exploit a root-running network service you can get it to load a kernel extension: not because network services need to load kernel extensions, but because there is nothing to stop root from doing so.

And that's how pretty much all UNIX systems, including Mac OS X, work. Before getting up in arms about how Apple disabled root in OS X, remember this: they didn't disable root, they disabled the account's password. You can't log in to a default OS X installation as root (though you can on Mac OS X Server). All of the admin facilities on Mac OS X are implemented by providing access to the monolithic root account - running a software update, configuring Sharing services, setting the FileVault master password all involve gaining root privilege.

The way these administrative features typically work is to use Authorization Services, and the principle of least privilege. I devoted a whole chapter to that in Professional Cocoa Application Security so won't go into too much detail here, the high-level view is that there are two components, one runs as the regular user and the other as root. The unprivileged part performs an authorisation test and then, at its own discretion, decides whether to call the privileged helper. The privileged part might independently test whether the user application really did pass the authorisation test. The main issue is that the privileged part still has full root access.

So Authorization Services gives us discretionary access control, but there's also a useful mandatory test relevant to the super-user. You see, traditional UNIX tests for whether a user is root by doing this:

if (process.p_euid == 0) {

Well, Mac OS X does do something similar in parts, but it actually has a more flexible test in places. There's a kernel authorisation framework called kauth - again, there's a chapter in PCAS on this so I don't intend to cover too much detail. It basically allows the kernel to defer security policy decisions to callbacks provided by kernel extensions, one such policy question is "should I give this process root?". Where the kernel uses this test, the super-user access is based not on the effective UID of the calling process, but on whatever the policy engine decides. Hmm...maybe the policy engine could use Authorization Services? If the application is an installer, and it has the installer right, and it's trying to get root access to the filesystem, then it's allowed.

Apple could then do away with monolithic root privileges completely, allowing the authorisation policy database to control who has privileged access for what tasks with which applications. The advantage is that if a privileged process ever gets compromised, the consequences for the rest of the OS are reduced.

#### Why passwords aren't always the right answer.
I realised something yesterday. I don't know my master password.

Users of Mac OS X can use FileVault, a data protection feature that replaces the user's home folder with an encrypted disk image. Encrypted disk images are protected by AES-128 or AES-256 encryption, but to get at the private key you need to supply one of two pieces of information. The first is the user's login password, and the second is a private key for a recovery certificate. That private key is stored in a dedicated keychain, which is itself protected by....the master password. More information on the mechanism is available both in Professional Cocoa Application Security and Enterprise Mac.

Anyway, so this password is very useful - any FileVault-enabled home folder can be opened by the holder of the master password. Even if the user has forgotten his login password, has left the company or is being awkward, you can get at the encrypted content. It's also hardly ever used. In fact, I've never used my own master password since I set it - and as a consequence have forgotten it.

There are a few different ways for users to recall passwords - by recital, by muscle memory or by revision. So when you enter the password, you either remember what the characters in the password are, where your hands need to be to type it or you look at the piece of paper/keychain where you wrote it down. Discounting the revision option (the keychain is off the menu, because if you forget your login password you can't decrypt your login keychain in order to view the recorded password), the only ways to reinforce a password in your memory are to use it. And you never use the FileVault master password.

I submit that as a rarely-used authentication step, the choice of a password to protect FileVault recovery is a particularly bad one. Of course you don't want attackers able to use the recovery mechanism, but you do want that when you really need to recover your encrypted data, the OS doesn't keep you out, too.

#### Why we don't trust <tt>-retainCount</tt>
I'm pretty sure @bbum must have worn through a few keyboards telling users of StackOverflow not to rely on the value of an Objective-C object's -retainCount. Why? When we create an object, it has a retain count of 1, right? Retains (and, for immutable objects, copies) bump that up, releases (and, some time later, autoreleases) bring it down, right? If an attempt to release would bring the retain count to 0, that object gets released, right? Right?!?

Well, that's not true for all objects, but leaving that aside, your code isn't the only code running in your process. The system libraries along with any third-party code you've included in your app are all doing things, including retaining and releasing objects. Let's take a look at a specific case of that.

The code in this post is from Gnustep-base, the LGPL implementation of OpenStep's Foundation library. The behaviour shown is identical to behaviour in Apple's Foundation. Here's (most of) +[NSNumber initialize].

/*
 * Numbers from -1 to 12 inclusive that are reused.
 */
static NSNumber *ReusedInstances[14];
static NSBoolNumber *boolY;		// Boolean YES (integer 1)
static NSBoolNumber *boolN;		// Boolean NO (integer 0)

+ (void) initialize
{
  int i;

  if ([NSNumber class] != self)
    {
      return;
    }

  // ...

  boolY = NSAllocateObject (NSBoolNumberClass, 0, 0);
  boolY->value = 1;
  boolN = NSAllocateObject (NSBoolNumberClass, 0, 0);
  boolN->value = 0;

  for (i = 0; i &lt; 14; i++)
    {
      NSIntNumber *n = NSAllocateObject (NSIntNumberClass, 0, 0);

      n-&gt;value = i - 1;
      ReusedInstances[i] = n;
    }
}

We see that sixteen instances of NSNumber subclasses, representing YES, NO and the integers -1 to 12, have been allocated and stored away in static variables. What's that about? Well here's (most of) +numberWithInt:.

/*
 * Macro for checking whether this value is the same as one of the singleton
 * instances.  
 */
#define CHECK_SINGLETON(aValue) \
if (aValue &gt;= -1 && aValue &lt;= 12)\
{\
  return ReusedInstances[aValue+1];\
}

// ...

+ (NSNumber *) numberWithInt: (int)aValue
{
  NSIntNumber *n;

  if (self != NSNumberClass)
    {
      return [[[self alloc] initWithBytes: (const void *)&aValue
        objCType: @encode(int)] autorelease];
    }

  CHECK_SINGLETON (aValue);
  n = NSAllocateObject (NSIntNumberClass, 0, 0);
  n-&gt;value = aValue;
  return AUTORELEASE(n);
}

So it looks like the retain count of an object you get back from +numberWithInt: depends on the value you pass, and whether anyone else is trying to use a number with the same value right now.

In other words, while it's easy to get an object's retain count using the -retainCount method, the number you expect to see may well be wrong. The real value depends on so many parameters that it's reasonable to conclude that you don't know what it's expected to be right now, so don't depend on it.

By the way, the NSNumber implementation is a great place to find out more about how Foundation is built. It's a class cluster (which is why you can see NSIntNumber and NSBoolNumber classes), and you can see how to use marker pointers to encode the value of an NSNumber right into the object pointer.

#### Why you didn't like that thing that company made
There’s been a bit of a thing about software user experience going off the rails lately. Some people don’t like cross-platform software, and think that it isn’t as consistent, as well-integrated, or as empathetic as native software. Some people don’t like native software, thinking that changes in the design of the browser (Apple), the start menu (Microsoft), or everything (GNOME) herald the end of days.

So what’s going on? Why did those people make that thing that you didn’t like? Here are some possibilities.

My cheese was moved

Plenty of people have spent plenty of time using plenty of computers. Some short-sighted individual promised “a computer on every desktop”, made it happen, and this made a lot of people rather angry.

All of these people have learned a way of using these computers that works for them. Not necessarily the one that you or anybody else expects, but one that’s basically good enough. This is called satisficing: finding a good enough way to achieve your goal.

Now removing this satisficing path, or moving it a few pixels over to the left, might make something that’s supposedly better than what was there before, but is actually worse because the learned behaviour of the people trying to use the thing no longer achieves what they want.
It may even be that the original thing is really bad. But because we know how to use it, we don’t want it to change.

Consider the File menu. In About Face 3: The Essentials of Interaction Design, written in 2007, Alan Cooper described all of the problems with the File menu and its operations: New, Open, Save, Save As…. Those operations are implementation focused. They tell you what the computer will do, which is something the computer should take care of.

He described a different model, based on what people think about how their documents work. Anything you type in gets saved (that’s true of the computer I’m typing this in to, which works a lot like a Canon Cat). You can rename it if you want to give it a different name, and you can duplicate it if you want to give it a different name while keeping the version at the original name.

This should be better, because it makes the computer expose operations that people want to do, not operations that the computer needs to do. It’s like having a helicopter with an “up” control instead of a cyclic and collective controls.

Only, replacing the Open/Save/Save As… stuff with the “better” stuff is like removing the cyclic and collective controls and giving a trained helicopter pilot with years of experience the “up” button. It doesn’t work the way they expect, they have to think about it which they didn’t have to do with the cyclic/collective controls (any more), therefore it’s worse (for them).

Users are more experienced and adaptable now

But let’s look at this a different way. More people have used more computers now than at any earlier point in history, because that’s literally how history works. And while they might not like having their cheese moved, they’re probably OK with learning how a different piece of cheese works because they’ve been doing that over and over each time they visit a new website, play a new game, or use a new app.

Maybe “platform consistency” and “conform with the human interface/platform style guidelines” was a thing that made sense in 1984, when nobody who bought a computer with a GUI had ever used one before and would have to learn how literally everything worked. But now people are more sophisticated in their use of computers, regularly flit between desktop applications, mobile apps, and websites, across different platforms, and so are more flexible and adaptable in using different software with different interactions than they were in the 1980s when you first read the Amiga User Interface Style Guide.

We asked users; they don’t care

At first glance, this explanation seems related to the previous one. We’re doing the agile thing, and talking to our customers, and they’ve never mentioned that the UI framework or the slightly inconsistent controls are an issue.

But it’s actually quite different. The reason users don’t mention that there’s extra cognitive load is that these kinds of mental operations are tacit knowledge. If you’re asked about “how can we improve your experience filing taxes”, you’ll start thinking tax-related questions, before you think “I couldn’t press Ctrl-A to get to the beginning of that text field”. I mean, unless you’re a developer who goes out of their way to look for that sort of inconsistency in software.

The trick here is to stop asking, and start watching. Users may well care, even if they don't vocalise that caring. They may well suffer, even if they don't realise it hard enough to care.

We didn’t ask users

Yes, that happens. I've probably gone into enough depth on why it happens in various places, but here's the summary: the company has a customer proxy who doesn't proxy customers.

#### Why your app is not massively parallel software
That trash can Mac Pro that hasn't been updated in years? It's too hard to write software for.

Now, let's be clear, there are any number of abstractions that have been created to help programmers parallelise their thing, from the process onward. If you've got a loop and can add the words #pragma omp parallel for to your code, then your loop can be run in parallel over as many threads as you like. It's not hard.

Making sure that the loop body can run concurrently with itself is hard, but there are some rules to follow that either make it easy or tell you when to avoid trying. But you're still only using the CPU, and there's that whole dedicated GPU to look after as well.

Even with interfaces like OpenCL, it's difficult to get this business right. If you've been thinking about your problem as objects, then each object has its own little part of the data - but now you need to get that information into a layout that'll be efficient for doing the GPU work, then actually do the copy, then copy the results back from the GPU memory...is doing all of that worth it?

For almost all applications, the answer is no. For almost no applications, the answer is occasionally. For a tiny number of applications, the answer is most of the time, but if you're writing one of those then you're a scientist or a data "scientist" and probably not going to get much value out of a deskside workstation anyway.

What's needed for that middle tier of applications is the tools - by which I mostly mean the libraries - to deal with this problem when it makes sense. You don't need visualisations that say "hey, if you learned a different programming language and technique and then applied it to this little inner loop you could get a little speed boost for the couple of seconds that one percent of users will use this feature every week" - you need implementations that notice that and get on with it anyway.

The Mac Pro is, in that sense, the exact opposite of the Macintosh. Back in the 1980s, the Smalltalk software was ready well before there was any hardware that could run it well, and the Macintosh was a thing that took this environment that could be seen to have value, and made it kindof work on real hardware. Conversely, the Mac Pro was ready well before there was any software that could make use of it, and that's a harder sell. The fact that, four years later, this is still true, makes it evident that it's either difficult or not worth the effort to try to push the kind of tools and techniques necessary to efficiently use Mac Pro-style hardware into "the developer ecosystem". Yes, there are niches that make very good use of them, but everybody else doesn't and probably can't.

#### Why your security UI sucks
The principle recurring problem in user experience is creating a user interface that supports the user's mental model of how an app works, while simultaneously enabling the actions that are actually supported by the implementation's model of the problem domain. Make the interface too much like the app internals, and the user won't be able to make it fit their view of how things work.

A useful technique in this field is to present the user with a metaphor, to say "the thing you're using now is a bit like this thing you've used before". That worked much better in the old days when computers were new and people actually had used those things before. Now, metaphors like address books, landline handsets and envelopes are getting tired, and a generation of people have grown up without using the things these metaphors are based on.

So how do security metaphors stack up? Well, here the situation is even worse. In these situations even though people do use the real-world devices, the metaphors are strained beyond the point of usefulness.

Locks and Keys

You'd be hard pushed to find a security vendor or industry blog that doesn't include a picture of a padlock on it somewhere. There, mine does too! A lock, using a particular mental model, is something that keeps you from using the door to a place or container (or keeps you in and other people out). The key permits you to act almost as if the lock weren't there when you want, and almost as if the door didn't exist (i.e. that the wall is solid) when you want.

There are few places in security UIs where locks exist (ignoring Keychain, which is already a response to a security/usability fail in itself), but plenty of keys. Keys that don't interact with locks, but instead turn readable messages into unreadable messages. In the real world, that is (or was) done by an envelope, or by choosing carefully who you tell your message to.

Keys in software can be magically created from passwords. So hang on, does this door need a password or a key to get through?

Keys in software can validate the integrity of data, showing that it cannot have changed since the keyholder saw it. How does that map onto real world keys? Oh, and we call this a signature. You know, exactly unlike how you sign your name.

Finally, there are some types of key that you can give to anyone, called public keys. These can be used to lock things but not unlock them. WTF?

Certificates

While we're talking about keys, we should remember that they can be stored on keychains, keyrings or certificates. Wait, what?

In the real world, certificates that I own tell me that I am a motorcycle's registered keeper, that I have completed a variety of different academic and training courses, and related information. The certificate is usually issued by the body that's best placed to assert the truthiness of the certified statement: the DVLA, the academic institution and so on.

A digital certificate is more like a notary statement of identity than any of those certificates. Except that, far from being issued by a notary, it's issued by anybody. Some certificates are issued by organisations who pay attention to other people's notary statements about the holder's identity. Sometimes, those certificates tell you (though not in plain language, of course) about how seriously they checked the identity of the holder.

How do you find out the identity of the organisation that issued the certificate? Why, with another certificate, of course! Imagine that my degree certificate was stapled to a certificate saying that the signer worked at the tutorial office, which was stapled to another saying that the tutorial office was part of the administration department, was stapled to…

So we need better metaphors?

No, we need to acknowledge that most security behaviour is internal to the software system that's being secured, and keep it out of the user interface. A field worker in sales thinks "I've got an e-mail from my boss, I'll read that", so let her read that. Do the signature/certificate stuff internally. If that e-mail turns out not to be from her boss then rather than showing the certificate, don't let her think that it is from the certificate.

No UI is good UI, right?

Here's the security UI Apple added to Mac OS X in Lion.

So if Apple is no longer putting security user interface in their product, that must make it the universally blessed approach, right?

Not so fast! One limitation of many security systems is that they do rely on some user involvement. It'd be great to make it all entirely invisible, but some of that user involvement is necessary: software is used in an environment that is only partially technological, but is partially social. Requirements for things like confidentiality and integrity that are born of meatspace needs still must be supported by user involvement.

And that's where we meet another problem. If we completely remove security UI from our products, we lose any chance to remind people that they too have skin in this game. Where data is confidential, if it doesn't look confidential it doesn't switch the reader into "remember to keep this secret" mode.  Switching the software into that mode is almost trivial in comparison.

As an example, both Mac OS X and iOS have shipped forever without a user password and without requiring the user to set a password. Great, no security UI. But do the people using Macs and iPhones understand the difference in security posture—particularly on iOS with data protection—implied by setting a password? It doesn't fit into the password metaphor, where a password just lets me in and doesn't do anything about encryption.

Conclusion

There are two problems when it comes to security UI. Traditional UI in this arena exposes too many details of the software internals, and relies on flakey metaphors that do not translate either to the user's mental model nor the software's implementation model of the security feature. On the other hand, if we just remove this UI then users aren't reminded of their part in the security of the overall, socio-technical system.

Thankfully solving these problems is not mutually exclusive. By communicating with users on their own terms at relevant times, an application's interface can encourage users to take part in the overall security of the system without either confusing them or turning them off of engaging by using overly technical language and concepts.

#### Wikipedia == fail
On the same day that fark announce the wikipedia irony, I would like to point out a similar situation I saw just today.This is from the Susie Dent entry discussion page:IMDb relies on the contributions of the public, so isn't an overly reliable source.

#### WikityWebsite
Strange things can happen when I get bored.  There's a great little dashboard widget called WikityWidget, which lets you organise notes like a wiki.  Well, I thought, no point in only having my notes on one computer, why not scrape the wikits out of WW and put them online?You may be interested in some source code.

#### Wistfully Wonderful Den of Coders
It's the time of the year to acknowledge that yes, I am going to WWDC this year.  Left it a bit last minute to get the flights and the hotel, but everything is in place now so hopefully I'll see some of you guys/gals there.  This is the first year that there's been anything going on that isn't Mac (it isn't the first year there's been non-MacDev, though; since the first WWDC I attended in 2005 there's always been an IT track occupying around 20-25% of the sessions, though not much lab space).  There have been mixed impressions of that - a representative sample:

About TimeNew developers might screw up the experienceNew developers might realise how cool Leopard is

I think this is going to be an exciting conference, especially for the new developers.  I've never been as a newbie; in 2005 I'd already been doing GNUstep, WebObjects, Cocoa and NeXTSTEP development for varing numbers of years, though admittedly without particular expertise.  From a perfeshunal perspective I'm not amazingly excited about iPhone development, I might drop in to a few of the sessions just to see what the state of play is, what people are interested in, what apps they're creating and so on.  No, for me this is the first year that I've actually got a project in full swing over the conference week so I'll be most interested in heading down to the labs and getting mmalc to write my code finding out what I could improve.

And, of course, the networking (by which I mean the going out for beers and food every night)…

#### Withholding the Four Freedoms
Having downsized my rather over-enthusiastic computer collection (thanks, eBay!), I was down to one computer. Unfortunately, as a rather long in the tooth MacBook Air, it's no longer suited to my needs and neither is it upgradeable. I got all of the files I care about off of its disk and set out to look for a replacement, meanwhile setting the MacBook aside to one day wipe its e4fs storage and re-install Mac OS X/OS X/macOS/whatever we call it this week.

I chose the sort of spec computer I wanted, then carefully researched various vendors to see what components they used and whether there was reported driver support in the Linux kernel. Eventually, dude, I got a Dell. The Alienware 15 R3 is made of bits that are supported in Linux, with the most complex piece being that the network adapter requires binary firmware. The manufacturer includes links to the firmware blobs on their website though, so this can't be hard...can it?

I fell straight to running the Debian Jessie installer, put the firmware blobs in place, and...the wi-fi isn't detected. Oh well, it's quite an old kernel, maybe I could switch to Testing? No, that doesn't boot at all: some error about not getting valid cache information from the SSD. Neither does the Ubuntu 16.10 kernel boot, for the same reason. 16.04 boots just fine, and it detects the wireless and connects to the network...and then the installer crashes.

One thing I'm not looking forward to is the onslaught of replies to this post from people who want to help, but ultimately won't help. "You should try Arch Linux, I expect that works," or "maybe you'll have better luck with Fedora." Why do you expect that? What specific knowledge about my problem makes you think that your specific choice of distribution will work better? And why can't I just take that knowledge and apply it to Debian, or Ubuntu? They're just distributions, they're all made of the same bits.

I admit, I'm frustrated. I want to be a proud advocate for Free Software and for totally free computing environments, but being unable to even run some of the flagship software makes me reticent to recommend it to others. I was having these problems back in 2001, and I'm having these problems now. And it's not like I'm incompetent when it comes to *nix administration or to Linux driver configuration, and nor did I just buy the prettiest computer I could find and hope that it would work, I _did my research_. I spent hours making sure that there were drivers for the various components in this system, reading reviews, QA forum posts, and kernel mailing list messages. Unfortunately my willingness to screw about with configuring my computer just so that I can use it has waned over the last couple of decades, faster than the necessity to screw about with it has decreased.

For the moment, I'm running Windows 10 as a hardware abstraction layer, and have a full-screen VirtualBox VM to do my work in. It works, it makes me die a little inside but it works. But I'm worried that for all the crowing that open source is eating the world, it's still too hard to jump in, even for a diligent and experienced user. Until we can give someone a computer that they turn on and start working in, and that runs free software, this will all remain the preserve of professionals and committed hobbyists. The Four Freedoms will effectively be restricted to those in the know and with time to dedicate to obtaining them: all users are equal, but some are more equal than others.

#### Wolf!  Wolf!
In describing the Apple/UKUUG tech briefing on the Intel transition, MacWorld describe the secondary speaker as "Oxford University Unix expert Graham Lee".  Let's hope no-one finds out until I've managed to start the motor....

#### Working Effectively with Legacy Code
I gave a talk to my team at ARM today on Working Effectively with Legacy Code by Michael Feathers. Here are some notes I made in preparation, which are somewhat related to the talk I gave.

This may be the most important book a software developer can
read. Why? Because if you don't, then you're part of the problem.

It's obviously a lot easier and a lot more enjoyable to work on
greenfield projects all the time. You get to choose this week's
favourite technologies and tools, put things together in the ways that
suit you now, and make progress because, well anything is progress
when there's nothing there already. But throwing away an existing
system and starting from scratch makes it easy to throw away the
lessons learned in developing that system. It may be ugly, and patched
up all over the place, but that's because each of those patches was
needed. They each represent something we learned about the product
after we thought we were done.

The new system is much more likely to look good from the developer's
perspective, but what about the users'? Do they want to pay again
for development of a new system when they already have one that mostly
works? Do they want to learn again how to use the software? We have
this strange introspective notion that professionalism in software
development means things that make code look good to other coders:
Clean Code, "well-crafted" code. But we should also have some
responsibility to those people who depend on us and who pay our way,
and that might mean taking the decision to fix the mostly-working
thing.

A digression: Lehman's Laws

Manny Lehman identified three different categories of software system:
those that are exactly specified, those that implement
well-understood procedures, and those that are influenced by the
environment in which they run. Most software (including ours) comes
into that last category, and as the environment changes so must the
software, even if there were no (known) problems with it at an earlier
point in its evolution.

He expressed
Laws governing the evolution of software systems,
which govern how the requirements for new development are in conflict
with the forces that slow down maintenance of existing systems. I'll
not reproduce the full list here, but for example on the one hand the
functionality of the system must grow over time to provide user
satisfaction, while at the same time the complexity will increase and
perceived quality will decline unless it is actively maintained.

Legacy Code

Michael Feather's definition of legacy code is code without tests. I'm
going to be a bit picky here: rather than saying that legacy code is
code with no tests, I'm going to say that it's code with
insufficient tests. If I make a change, can I be confident that I'll
discover the ramifications of that change?

If not, then it'll slow me down. I even sometimes discard changes
entirely, because I decide the cost of working out whether my change
has broken anything outweighs the interest I have in seeing the change
make it into the codebase.

Feathers refers to the tests as a "software vice". They clamp the
software into place, so that you can have more control when you're
working on it. Tests aren't the only tools that do this: assertions
(and particularly Design by Contract) also help pin down the software.

How do I test untested code?

The apparent way forward then when dealing with legacy code is to
understand its behaviour and encapsulate that in a collection of unit
tests. Unfortunately, it's likely to be difficult to write unit tests
for legacy code, because it's all tightly coupled, has weird and
unexpected dependencies, and is hard to understand. So there's a
catch-22: I need to make tests before I make changes, but I need to
make changes before I can make tests.

Seams

Almost the entire book is about resolving that dilemma, and contains a
collection of patterns and techniques to help you make low-risk
changes to make the code more testable, so you can introduce the tests
that will help you make the high-risk changes. His algorithm is:

identify the "change points", the things that need modifying to
make the change you have to make.
find the "test points", the places around the change points where
you need to add tests.
break dependencies.
write the tests.
make the changes.

The overarching model for breaking dependencies is the "seam". It's a
place where you can change the behaviour of some code you want to
test, without having to change the code under test itself. Some examples:

you could introduce a constructor argument to inject an object
rather than using a global variable
you could add a layer of indirection between a method and a
framework class it uses, to replace that framework class with a
test double
you could use the C preprocessor to redefine a function call to use
a different function
you can break an uncohesive class into two classes that collaborate
over an interface, to replace one of the classes in your tests

Understanding the code

The important point is that whatever you, or someone else, thinks
the behaviour of the code should be, actually your customers have paid
for the behaviour that's actually there and so that (modulo bugs) is
the thing you should preserve.

The book contains techniques to help you understand the existing code
so that you can get those tests written in the first place, and even
find the change points. Scratch refactoring is one technique: look
at the code, change it, move bits out that you think represent
cohesive functions, delete code that's probably unused, make notes in
comments...then just discard all of those changes. This is like Fred
Brooks's recommendation to "plan to throw one away", you can take what
you learned from those notes and refactorings and go in again with a
more structured approach.

Sketching is another technique recommended in the book. You can draw
diagrams of how different modules or objects collaborate, and
particularly draw networks of what parts of the system will be
affected by changes in the part you're looking at.

#### Wristwatches in the Future
[Int: Moscone West convention center third floor ballroom. A presentation is taking place.]

So that was an update on our existing products, which I'm proud to say have never been stronger. Now I'd like to talk to you about our watch. We think you're gonna love it.

When thinking about recent attempts to make wristwatches by some of our competitors, you're probably thinking whatever happened to the retina display?

I'll tell you what happened: you're looking at it in the wrong size. The pixel density is fine, but the screen's smaller than it looks on this projector.

No, smaller than that.

Hmm, eye strain doesn't sound like a future thing does it? Shouldn't we come up with something a bit more ergonomic?

…actually, no, it's a reasonable compromise. Centuries of user experience research have shown that future people find it most natural to talk into their scaphoid bones. That's even true when they're plastic people who don't actually have scaphoid bones!

[Sidenote: No, I will not be apologising for the alt text on that image.]

The technology was originally introduced as part of the struggle to end the Cold War, when one key application was in the unification of Germany.

After seeing how people tried to use these devices, we came up with the breakthrough form factor: five pounds of computer-machined aluminum and an incomprehensible user interface.

So that's the watch. We can't wait to see what you do with it!

#### WTF is a Veo?
Can anyone explain what this cpu_type represents?  And yes, reasons I'm delving around in libstuff to follow within a few days...

#### WWDC - day one
The WWDC keynote is always an odd event to attend.  It's put on for the benefit of the investors and the media, with the developers being invited purely to act as braying masses expressing their adulation for His Steveness.  It's rare for any technical content to make it into the session, except in unavoidable cases such as the 2005 keynote. The focus of that was the Intel transition, so by necessity there had to be some technical justification of the switch.With this in mind, it's not hard to see that the keynote can be a somewhat dull affair.  Obviously as both an Apple customer and member of the "economic ecosystem" of the Mac, it's always good to be as informed as possible of the company's position and direction.  That said, yesterday's keynote (no wi-fi in this hotel, so a late post) contained less of interest to me than usual.As I mentioned I'm financially dependent on Apple (in an indirect sense of course; I'm paid to write Mac software for Sophos, therefore no Mac = no job at Sophos), though as I'm not an indie dev I have a bit more of a comfort buffer than many people.  The enterprise iPhone video Steve showed was basically a backslap in front of the shareholders; look, there are people who really do use this stuff! Then the laundry list of every developer who's downloaded the SDK and managed to get something to compile; interesting to see the wealth of different domains into which the iPhone is entering, but seriously. Two demos, three tops.  Not all four thousand of the known apps.  Good to see TEH CHEAP being applied to the 3G iPhone, though; I may have tohave a discussion with Orange about a PAC when that's available.Which left Mobile Me.  This is actually a pretty cool reboot of iTools^W.Mac, OK it looks like there might be no more iCards but on the other hand the Mobile Me syncing is really beneficial.  I can see that becoming more of a cash cow for Apple, though mainly because they opened it up to the PC; people who have an iPod and Windoze could buy MM to synchronise their contacts, mail and so on, as well as getting webmail access (and webmail access which doesn't suck balls as much asExchange's OWA, may I add).  That then might make them more amenable to the Halo Effect and the purchase of a Mac down the line.The rest of the day was interesting but obviously undisclosable, except for the evening I spent in a couple of bars down the financial district (the Golden%Braeburn event at 111 Minna, where I went with Steffi from BNR and a couple of Cocotron committers; then Dave's bar where I met Nigel and most of Apple UK).  Conversation ranged from Sophos feature requests to the drinkability of American IPAs; all good stuff!

#### WWDC 2022 is a WWDC watch party
Apple have shared initial timings for this year's WorldWide Developer Conference. In typical in-person years this would be the trigger for various “WWDC attendee tips” posts (don't forget to drink water! Remember to sleep sometime through the week! Don't go to the Moscone centre, they've moved the conference!) but that has not been the case through the pandemic. Instead WWDC has been fully online, so you just need to get the Developer app and watch the videos.

This year, it's sort of hybrid, in that it appears the event will be online-first with a watch party of sorts on the first day. This happened at the fully in-person events anyway, at least at the Moscone: the keynote room filled up quickly and attendees were directed to other rooms to watch a stream. Other talks would be streamed to screens around the conference venue: I remember watching Crusty's guide to protocol-oriented programming at an in-conference sports bar with a couple of good friends.

It's also a great way to run a hybrid event: it's much too easy (as those of us who worked remote in the pre-pandemic times will remember) for online attendees to be second-class citizens in the hybrid world. Making it clear that the event is an online event with the ability to engage from an on-site presence removes that distinction.

Some people will stay away, on the basis that travelling all the way to California to watch AppleTV is not a compelling use of resources. Honestly with this pandemic not being over anywhere except the minds of the politicians who need sacrifices to the line, that's not a bad thing. Except that these people will miss out on networking, which is a bad thing.

Networking is such a big part of WWDC that plenty of people who didn't have tickets to the for-realsies iterations would go anyway, maybe going to after parties, maybe going to AltConf (another opportunity to watch a stream of WWDC, alongside original talks by community members). But that was for a week of networking, not a day of watching TV.

That's OK. Hopefully online watch parties, and local watch parties, will spring up, making the networking side of WWDC more accessible. Making WWDC truly world-wide.

#### WWDC aside
None of Perl, Cocoa or some weird XML toolkit came up with the "simple things simple, complex things possible" quote.

#### WWDC dates announced
The entire of Twitter has imploded after noticing that Apple has announced the dates for WWDC, this year June 7-11. That's too short notice for me to go, and having only recently started working again after a few months concentrating solely on Professional Cocoa Application Security, I can't scrape together the few thousand pounds needed to reserve flights, hotel and ticket at a month's notice.I hope that those of you who are going have a great time. The conference looks decidedly thin on Mac content this year, and while I still class myself as more of a Mac developer than an iP* developer that shouldn't be too much of a problem. The main value in WWDC is in the social/networking side first, the labs second, and the lecture content third - so as long as you can find an engineer in the labs who remembers how a Mac works, you'll probably still have a great week and learn a lot.

#### WWDC day 00000000000000000000000000000101
Technically day five isn't over yet, but with just one session and a taxi ride remaining I doubt I'm going to get anything prematurely wrong here.  In fact, I'm no longer entirely sure that I'll remain awake through the rest of the day so even if something entirely different does happen (still haven't heard anything about IXKit this millennium) I expect I'd miss it.Planning an east-bound flight is always difficult, because an 11-hour flight through eight time zones "takes" 19 hours.  So if I sleep on the plane I'll wake up in the middle of the afternoon UK time, and going back to sleep at newly-local night-time might be complicated.  Conversely if I don't sleep on the plane then I won't be going to sleep until at least 9am currently-local time, for a total of about 28 hours uptime.  I was just talking to someone who recommended fasting, I'm not sure whether it's worth going through that hardship for an unverified theory (stuff all of that "in the name of science" crap).  The other option would be to go into work 3pm-11pm for the beginning of the week, I'm not sure everyone else will appreciate having their meetings moved to the middle of the night and I don't think the canteen's open that late either ;-).Still, of the three WWDCs I've been to this is definitely in the top ten list of interesting and exciting WWDCs.  There definitely still are fellow NeXT fans in the woodwork (actually, tap tap I think it's gypsum board) who worm their way out during these events, I guess that most have no motivation not to be working on Cocoa.  Actually, I don't recall having seen Andrew Stone this week, and I know of a couple of other guys who aren't here, but have definitely managed to gain some traction for the phrase "NeXTSTEP Mobile ;-)

#### WWDC day 2
Tuesday is typically associated with "starting to learn stuff" at WWDC, and today was definitely spent learning stuff.  Learning, for instance, that doughnuts are considered adequate breakfast material, or that if you are prepared to pay the tiny wi-fi fee at Starbucks you can get faster INTARWEBS than all the people on the steps outside the Mascarpone.  It was also a day of nice food; I've just got back from dinner with Alex at the Chevy's, having had lunch with Michael at the Ozuma sushi restaurant.  More discussing Sophos-for-Mac with customers and potential customers ensued (apparently we've saved some people from the "hell that is Norton", and I point out for legal purposes that that's a direct quote and not my own opinion or words), and I got some pretty good photos of the Bay Bridge and treasure island which I'll upload as soon as I locate the USB cable.

#### WWDC day crosspost
WWDC summary on SophosLabs blog, written by yours truly.  I wonder how many Sophos readers (I should probably avoid the term sophist, shouldn't I?) have an interest in Mac and iPhone stuff (and subequently how many comment via the e-mail address on the Labs blog).

#### WWDC day four
Not so many sessions attended today - partly because I've reached the limit of what the human physiology can achieve on a diet of coffee and doughnuts.  But also due to ducking out of sessions to meet with ex-NeXT guys, ex-Lighthouse guys and ex-colleagues for most of the afternoon.  Of course, this was followed by the beer bash, no longer the campus bash which we all know and fondly remember (or at least we all fondly remember queueing for a couple of hours at each end for the coach) but still a good event.  Speaking of ex-Lighthouse guys, Wylie and I spent a bit of time putting Sun's world to rights (essentially, if they were to stop diluting the Java brand, stop pulling themselves in every which way and try to make some money, they might do OK).So tomorrow is the NeXT meetup at the beginning of lunch, I have a lab appointment at exactly 12 which is a little annoying but I'll try to make it over.  If the G4 cube is the right shape, wrong CPU and Real Men's Objects use the NX prefix, then come over to the front of the Moscone West after the second session.

#### WWDC day three
Today was not quite a full day, so I managed to spend about an hour or so milling around Yerba Buena park, doing a little gift shopping and generally existing outside of the Mascarpone centre.  Also put in another update to the c.l.o-c FAQ (though I didn't update the date! oops&hellip;) to better discuss retain/release memory management.  There are still situations (GNUstep, Cocotron, and some particular OS X environments) where garbage collection is either unavailable or unsuitable, and it turns out that a lot of questions on either the group or cocoa-dev recently have been about the memory management system.In the evening, after dinner at Chevy's with the u.c.s.m guys (and yet more discussion of the Sophos feature set), checked in to the Apple Design Awards for the first 30 mins or so.  All of the winners (and runners-up) I saw were worthy applications, although it was interesting to note that while most of the &quot;productivity&quot; app winners were from small shops, both of the games (runner-up was Command and Conquer 3, winner was Guitar Hero 3) were from large studios, ports of Windows/console games and in existing long-running series.  They're both great games, too, of course.  Ian was, of course, rooting for Delicious Library 2, and sadly disappointed ;-).This was all followed by the AFP548 beer bash over at Thirsty Bear.  Yet more &quot;oh, you're from Sophos?  Yeah, let me say this one thing&hellip;&quot;, which I really enjoy because if something's either good or bad enough to be the subject of a beer-fuelled rant in a party, I should probably hear about it.  Also submitted a couple of entries to Peter's "Leopard bug Venn diagram" contest; I'm not sure I could do a better job of describing the situation than that but for those who were there, I came up with: Omega = "Closed/Duplicate" and Omega = "NeXTSTEP 7.0".

#### WWDC part 0
well, here it is, the pre-WWDC "I'm jetlagged so you have to put up with my wittering" post.  I'm just waiting for a softwareupdate to finish so that I can go out with my camera, taking some early-morning pictures before heading off to stand in line for the Stevenote.  I was out for beers with Ian and Neil last night, we'd all heard rumours of a 5 a.m. start to the queue.  On the two previous occasions that I've been, 9 a.m. has been sufficient; but with the sellout nature of the event it's likely that the room will fill up rather quickly so we've compromised on a 7 a.m. start.  Actually, forget the 5 a.m. nonsense, there's a line of overnight campers - I can't decide whether they're deliberately trying to re-enact a Joy of Tech cartoon, or actually have nothing to do with their lives.

#### WWDC wind-down
As everyone is getting on their respective planes and flying back to their respective homelands, it's time to look back on what happened and what the conference means.The event itself was great fun, as ever. Meeting loads of new people (a big thank-you to the #paddyinvasion for my dishonourary membership) as well as plenty of old friends is always enjoyable - especially when everyone's so excited about what they're working on, what they've discovered and what they're up to the next day. It's an infectious enthusiasm.Interestingly the sessions and labs content has more of a dual impact. On the one hand it's great to see how new things work, how I could use them, and to realise that I get what they do. The best feeling is taking some new information and being able to make use of it or see how it can be used. That's another reason why talking to everyone else is great - they all have their own perspectives on what they've seen and we can share those views, learning things from each other that we didn't get from the sessions. If you were wondering what the animated discussions and gesticulations were in the 4th Street Starbucks at 7am every morning, now you know.On the other hand, it makes me realise that OS X is such a huge platform that there are parts I understand very well, and parts that I don't really know at all. My own code spreads a wide path over a timeline between January 1, 1970 and September 2009 (not a typo). For instance, it wasn't until about 2003 that I knew enough NetInfo to be able to write a program to use it (you may wonder why I didn't just use DirectoryServices - well even in 2003 the program was for NeXTSTEP 3 which didn't supply that API). I still have a level of knowledge of Mach APIs far below "grok", and have never known even the smallest thing about HIToolbox.There are various options for dealing with that. The most time-intensive is to take time to study - I've got a huge collection of papers on the Mach design and implementation, and occasionally find time to pop one off the stack. The least is to ignore the problem - as I have done with HIToolbox, because it offers nothing I can't do with Cocoa. In-between are other strategies such as vicariously channeling the knowledge of Amit Singh or Mark Dalrymple and Aaron Hillegass. I expect that fully understanding Mac OS X is beyond the mental scope of any individual - but it's certainly fun to try :-).

#### WWDC: afp548 Venn competition
I, with some help from Ken and the guys, did a better job.

#### Yeah, we've got one of those
Title linkey (which I discovered via slashdot) goes to an interview in DDJ with Paul Jansen, the creator of the TIOBE Programmer Community Index, which ranks programming languages according to their web presence (i.e. the size of the community interested in those languages).  From the interview:

C and C++ are definitely losing ground. There is a simple explanation for this. Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection. 

So, to those people who balked at Objective-C 2.0's garbage collection, on the basis that it &quot;isn't a 4GL&quot;, I say who cares?  Seemingly, programmers don't - or at least a useful subset of Objective-C programmers don't.  I frequently meet fellow developers who believe that if you don't know which sorting algorithm to use for a particular operation, and how to implement it in C with the fewest temporary variables, you're not a programmer.  Bullshit.  If you don't know that, you're not a programmer who should work on a foundation framework, but given the existence of a foundation framework the majority of programmers in the world can call list.sort() and have done with it.

Memory management code is in the same bucket as sorting algorithms - you don't need for everybody to be good at it, you need for enough people to be good at it that everyone else can use their memory management code.  Objective-C 2.0's introduction of a garbage collector is acknowledgement of this fact - look at the number of retain/release-related problems on the cocoa-dev list today, to realise that adding a garbage collector is a much bigger enhancement to many developers' lives than would be running in a VM, which would basically go unnoticed by many people and get in the way of the others trying to use Instruments.

Of course, Objective-C and ApPLE's developer tools have a long history of moving from instrumental programming (this is what the computer must do) to declarative programming (this is what I am trying to achieve, the computer must do it).  Consider InterfaceBuilder.  While Delphi programmers could add buttons to their views, they then had to override that button's onClick() method to add some behaviour.  IB and the target-action approach allow the programmer to say &quot;when this button is clicked, that happens&quot; without having to express this in code.  This is all very well, but many controls on a view are used to both display and modify the value of some model-level property, so instead of writing lots of controller code, let's just declare that this view binds to that model, and accesses it through this controller (which we won't write either).  In fact, rather than a bunch of boilerplate storage/accessors/memory management model-level code, why don't we just say that this model has that property and let someone who's good at writing property-managing code do the work for us?  Actually, coding the model seems a bit silly, let's just say that we're modelling this domain entity and let someone who's good at entity modelling do that work, too.

&lt;

p>In fact, with only a little more analysis of the mutation of Objective-C and the developer tools, we could probably build a description of the hypothetical Cen Kase, the developer most likely to benefit from developing in Cocoa.  I would expect a couple of facts to hold; firstly that Cen is not one of the developers who believes that stuff about sorting algorithms, and secondly that the differences between my description of Cen and the description used by Apple in their domain modelling work would fit in one screen of FileMerge on my iBook.

#### Yes, you may delete tests
A frequently-presented objection to the concept of writing automated tests is that it ossifies the implementation of the system under test. "If I've got all the tests you're proposing," I hear, "then I won't be able to make any changes without breaking the tests." There are two aspects to this problem.

Tests should make assertions about the externally-observable behaviour of the system under test. If you have tests that assert that a particular private method does its thing in a particular way, then yes, you are baking in the assumption that this private method works in a particular way. And yes, that makes it harder to change things. But it isn't necessary.

Any test that is no longer useful is no longer useful: delete it. Those tests that helped you to build the thing were useful while you were building the thing: if they're not helping you now then just get rid of them.

When you're building a component, you're probably thinking quite deeply about the algorithm it will implement, and how it will communicate with its collaborators. Thinking about those will mean writing tests about them, and whenever those tests are in the suite they'll require the algorithm be implemented in a particular way, and that the module talk to its collaborators in a particular way. Do you no longer care about those details, as long as it does the right thing? Delete the tests, and just keep the ones that make sure it does the right thing.

#### You keep using that word. I do not think it means what you think it means.
In doing a little audience research for my spot at MacDev 2009, I've discovered that the word "security" to many developers has a particular meaning. It seems to be consistent with "hacker-proof", and as it could take most of my hour to set the record straight in a presentation context, here instead is my diatribe in written form. Also in condensed form; another benefit of the blog is that I tend to want to wrap things up quickly as the hour approaches midnight.Security has a much wider scope than keeping bad people out. A system (any system, assume I'm talking software but I could equally be discussing a business process or a building or something) also needs to ensure that the "good" people can use it, and it might need to respond predictably, or to demonstrate or prove that the data are unchanged aside from the known actions of the users. These are all aspects of security that don't fit the usual forbiddance definition.You may have noticed that these aspects can come into conflict, too. Imagine that with a new version of OS X, your iMac no longer merely takes a username and password to log a user in, but instead requires that an Apple-approved security guard - who, BTW, you're paying for - verifies your identity in an hour-long process before permitting you use of the computer. In the first, "hacker-proof" sense of security, this is a better system, right? We've now set a much higher bar for the bad guys to leap before they can use the computer, so it's More Secure&#x2122;. Although, actually, it's likely that for most users this behaviour would just get on one's wick really quickly as they discover that checking Twitter becomes a slow, boring and expensive process. So in fact by over-investing in one aspect of security (the access control, also sometimes known as identification and authorisation) my solution reduces the availability of the computer, and therefore the security is actually counter-productive. Whether it's worse than nothing at all is debatable, but it's certainly a suboptimal solution.And I haven't even begun to consider the extra vulnerabilities that are inherent in this new, ludicrous access control mechanism. It certainly looks to be more rigorous on the face of things, but exactly how does that guard identify the users? Can I impersonate the guard? Can I bribe her? If she's asleep or I attack her, can I use the system anyway? Come to that, if she's asleep then can the user gain access? Can I subvert the approval process at Apple to get my own agent employed as one of the guards? What looked to be a fairly simple case of a straw-man overzealous security solution actually turns out to be a nightmare of potential vulnerabilities and reduced effectiveness.Now I've clearly shown that having a heavyweight identification and authorisation process with a manned guard post is useless overkill as far as security goes. This would seem like a convincing argument for removing the passport control booths at airports and replacing them with  a simple and cheap username-and-password entry system, wouldn't it? Wouldn't it?What I hope that short discussion shows is that there is no such thing as a "most secure" applications; there are applications which are "secure enough" for the context in which they are used, and there are those which are not. But the same solution presented in different environments or for different uses will push the various trade-offs in desirable or undesirable directions, so that a system or process which is considered "secure" in one context could be entirely ineffective or unusable in another.

#### You say “cave dweller debugging”, I say debug logging
There are still many situations where it's not feasible to stop a process, attach the debugger, and start futzing with memory. We can argue over whether this is because the industry didn't learn enough from the Pharo folks later. For now, let's pretend that's axiomatic: a certain amount of debugging (and even testing) starts with asking the program to report on its state.

It's common to do this with print statements, whatever they're called in your language. The technique is even called “printf debugging” sometimes. Some teams even have lint rules to stop you merging changes that call System.out.println or console.error because they know you're all doing it. I think you should carry on doing it, and you should be encouraged to commit the changes.

Just don't call your print function. This isn't for any performance/timing/buffer flushing reason, though those are sometimes relevant and in the relevant cases sometimes problematic and in the problematic cases sometimes important. It's more because it's overwhelming. Instead, call your syslog/OSLog/logger function, with an appropriate severity level (probably DEBUG) and some easily filterable preamble. Now commit those log messages.

One benefit of doing this is that you capture the fact that you need this information to diagnose problems in this module/subsystem/class/whatever. Next time you have the problem, you already know at least some of the information that will help.

Another benefit is that you can enable this logging in the field without having to deploy a new version with different printf statements. Just change the log level or the capture filter and start getting useful information. Change it back when you're done.

There are caveats here: if the information you need to log is potentially sensitive (personal information, crypto material) you may be better off not having any way to turn it on in production.

The third benefit is that you communicate to everybody else that this part of the code is hard to understand and needed careful inspection. This can be the motivation the team needs to discuss a redesign, or it can help other people find smoking guns when trying to diagnose other failures.

#### Your build needs to be better
I've said it before, build systems are a huge annoyance. If your build is anything other than seemingly instantaneous, it's costing you severe money.

Your developers are probably off reading HN, or writing blog posts about how slow builds cost them, while the build is going. When they finish doing that, which may be some time after the build completes, they'll have forgotten some of what they were doing and need to spend some time getting back up to speed.

Your developers are probably suspicious of any build failure, thinking that "the build is flaky" rather than "I made a mistake". They'll press the button again and go back to HN. When the same error occurs twice, they might look into it.

Your developers probably know that the build is slow, but not which bit of the build is slow. And they don't have time to investigate that, where it takes so long to get any work done anyway. So everyone will agree that "there is a problem", but nothing will get done. Or maybe cargo-cult things will get done, things that speed up "builds" but are not the problem with your build.

The Joel test asks whether you can make a build in one test. Insufficient. If you notice when you're making a build, you're slowing your developers down.

...which is not always the worst thing, of course. Sometimes a lengthy translation step from some source language to some optimised form of a machine language program yields better results for your customers, because they get to use a faster program and don't need to care about the time taken to prepare that program. But let's be clear: that's part of the release, and your developers don't always need to be working from the released product (in fact, they're usually not). Releases should be asynchronous, and the latency between having something ready to be released and having released it can be fairly high, compared with the latency between having created some source and being able to investigate its utility.

Nonetheless, that should all go off in the background. So really, builds and releases should both be non-events to the developers.

#### Your reminder that "British English" and "American English" are fictional constructs
Low-stakes conspiracy theory: they were invented by word processing marketers to justify spell-check features that weren't necessary.

Evidence: the Oxford English Dictionary (Oxford being in Britain) entry for "-ise" suffix's first sense is "A frequent spelling of -ize suffix, suffix forming verbs, which see." So in a British dictionary, -ize is preferred. But in a computer, I have to change my whole hecking country to be able to write that!

#### YX problem
Software people are always all up in the XY problem: someone asks about how to do X when what they're really trying to solve is Y. I find the YX problem much more frustrating: where software people decide that they want to answer question Y even though what someone asks is question X.

I've seen a few different manifestations of this pattern:

Respondent doesn't know the answer to X, but does know the answer to Y, and hopes that answering Y demonstrates expertise/usefulness.

Respondent doesn't know the answer to X, but riffs on what the answer probably would be, and ends up answering Y.

Respondent doesn't believe that the querent should be trying X and thinks they should be trying Y instead; respondent didn't ask querent the context for X but jumped straight to answering Y.

Respondent knows of a process Y that leads up to the querent trying X and decides to enumerate the steps of that process Y; even though they know that the querent is already trying X.

Respondent misunderstood question X to be question Y.

The common advice on questions for software people is How to ask questions the smart way. The problem with this advice is that it's written from the perspective of an asymmetric relationship: the respondent is a busy expert, the querent is an idle dilettante; the querent has a responsibility to frame their question in the optimum way for the expert to impart wisdom to the idler.

Frequently the situation is more symmetric: we're both busy experts, and we both have incomplete knowledge of both the question domain and what we're trying to achieve. Have some patience with other people (whichever side of the interaction you're on), and assume good faith on the part of all involved until they present contrary evidence. That means starting from the assumption that someone asked question X because they want an answer to question X.

#### Zen and the Art of Software Maintenance
In one part of the book Zen and the Art of Motorcycle Maintenance, which is neither about Zen nor motorcycle maintenance, there are two motorcycles and two riders. John Sutherland is a romanticist who appreciates the external qualities of his motorcycle: its aesthetics, and its use as a vehicle. The narrator is a classicist who appreciates the internal qualities of his motorcycle: its workings, parts, and mechanisms. When Sutherland has a problem with his bike he takes it to a mechanic. When the narrator does, he rationalises about the problem and attempts to discover a solution.

The book, which as its subtitle gives away is "an inquiry into values", then follows the narrator's exploration of a third way of considering quality that marries the romantic and classical notions holistically.

Now we come onto software. Software doesn't exist. At some level, its abstractions and mathematics get translated into a sequence of states of an electronic machine that turns logic into procedure: but even that is a description that's a few degrees abstracted from what software and computers really do.

Nonetheless, software has external and internal qualities. It has aesthetics and utility, and can be assessed romantically. A decidedly pedestrian word to describe the romanticist view of software is "requirements", but it's a common word in software engineering that means the right thing.

Software also has workings, parts, and mechanics. Words from software engineering to describe the classical view of software include architecture, design, clean code, SOLID...

...there are many more of these words! Unsurprisingly, the people who build software and who change software tend to take a classical view of the software, and have a lot more words to describe its internal qualities than its external qualities.

Typically, the people who are paying for software are interested in the romantic view. They want it to work to achieve some goal, and want someone else (us!) to care about what makes it work. Perhaps that's why so many software teams phrase their requirements as "As a romantic, I want to task so that I can goal."

Which is to say that making software professionally involves subordinating classical interpretations of quality to romantic interpretations. Which is not to say that a purely-classical viewpoint is unvaluable. It's just a different thing from teaching a computer somersaults for a paying audience.

And maybe that subordination of our classical view to the customer/gold owner's romantic view is the source of the principles:

  Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

and:

  Working software is the primary measure of progress.

In fact, this second one is not quite true. It suggests that you could somehow "count software", and the more (working) software you've delivered, the better you're doing. In fact, romanticism shows us that people only want software in that it enables some process or business opportunity, or makes it more efficient, or reduces errors, or lets them enjoy some downtime, or helps them achieve some other goal. So really progress toward that goal is the primary measure of progress, and working software is a leading metric that we hope tells us how we're working toward that goal.

So all of those code quality and software architecture things are in support of the external view of the software, which is itself in support of some other, probably non-software-related, goal. And that's why the cleanliness, or architectural niceness, or whatever classical quality, of the code is not absolute, but depends on how those qualities support the romantic qualities of the code.

Real life comes at you fast, though. When you're working on version 1, you want to do as little work, as quickly as possible, to get to the point where you can validate that there are enough customers who derive enough value to make the product worthwhile. But by the time you come to work on version 1.0.1, you wish you'd taken the time to make version 1 maintainable and easy to change. Most subsequent versions are a little from column A and a little from column B, as you try new things and iterate on the things that worked.

As fast as possible, but no faster, I guess.

#### [objc retain] stream
Starting next week: [objc retain]; in which Steven Baker and I live-code Objective-C on a modern free software platform. Wednesday, February 10th, 1900UTC. More info at objc-retain.com.

#### [objc retain]; continues apace
I just finished recording episode 35 of [objc retain]; the stream on Objective-C programming with Free Software that I co-host with Steven Baker. It is available on Twitch and you can subscribe there to get notified about new episodes.

It will also soon be available on the replay server where you can watch all historical episodes of the show.

If you enjoy the show (or this blog, or the podcast, or…) please consider supporting my work on Patreon, thank you!

#### “Best practice” is less of a scam than claimed
Another day, another developer explaining that they don’t follow some popular practice. And their reason? Nothing more than because other people do the thing. “Best practices don’t exist,” they airily intone. “They’re really mediocre practices”.

In one sense, they’re correct. Best practices need to be evidence-based, and there’s precious little evidence in software engineering. In a regulated profession, you could avoid using accepted best practice, but if something went wrong and you ended up on the receiving end of a malpractice suit, you would lose.

So best practice as an argument in software engineering has two weaknesses: the first is that there’s no basis in evaluation of practice; and the second is that being a monetised hobby rather than a profession there’s no incentive to discover and adopt best practice anyway.

But those arguments mean that best practices are indistinguishable from alternative practices, not inherently worse. If a programmer discards a practice because they claim it’s considered best practice, they’re really just stamping their foot and shouting “I don’t wanna!”

They’re rejecting the remaining evidence in favour of the practice—that it’s survived scrutiny by a large cohort of their peers—in favour of making their monetised hobby look more like their headcanonical version of the hobby. “We are uncovering better ways of making software by doing it and by helping others to do it” be damned: I want to use this thing I read a substack post about yesterday!

Dig deeper, and you’ll find only platitudinous justification based on thought-terminating cliche: I’ve already covered “Reasoning about code”, and maybe some time I’ll cover “Right tool for the job”. This time, let’s look at “things won’t advance unless some of us try new ways of doing it”.

People tried new ways of making new steam engines all the time, during the industrial revolution. People tried new ways of making chimneys all the time, during the 15th and 16th centuries. A lot of factories and trains exploded, and a lot of buildings burnt down. If you live in a house with a chimney now, or you have ever taken a train, it’s significantly less likely to have self-immolated than at earlier times in history.

It’s not, for the most part, due to misunderstood lone geniuses rejecting what everybody else was doing, but a small amount of incremental development and a large amount of theoretical advance. It’s no coincidence that the field of thermodynamics advanced leaps and bounds during the steam age. Brad Cox makes this point about software too, in almost everything he wrote on the topic: you don’t get as much advance from random walks in the cottage industry as you do from standardisation, mass production, the division of labour, and interchangeable parts that can be evaluated on merit with reference to a strong theoretical underpinning.

Of course, the “reason about code” crowd try to stop this from happening, because if that advance happened then the code-reasoning would quickly disappear to be replaced with the problem-domain-reasoning that’s significantly harder and less of a hobby. Hence the sabotage of best practice: let’s put a stop to this before anybody realises it’s more than sufficient to the task at hand.

Alan Kay once referred to a LISP evaluator written in LISP as “the Maxwell’s Equations of software”. But what software needs before a James Clerk Maxwell are the Gibbs, Boltzmanns, Joules and Lavoisiers, the people who can stop us from blowing things up in production.

#### 客観的なc計算機言語
Just heard from someone who wants to translate the c.l.o-c FAQ into Japanese.  Sweet!

### Pages

#### About
About The Author

I'm Graham Lee. I do a lot of things, but in particular I help programmers to become software engineers. I've been in software for around two decades, working on UNIX tools, mobile apps, desktop software, micro services, supercomputing…I've seen a lot. I share what I've learned in multiple places, so here are some links.

Chiron Codex

Over at Chiron Codex I'm trying to understand how we use AI-augmented tools, including LLMs, to improve as software engineers, instead of outsourcing our abilities to language models. You can get early access to, and support this work, at the Chiron Codex Patreon.

Blog

This website's blog covers a range of topics, broadly related to software engineering and its culture and values. The name is "Structure and Interpretation of Computer Programmers", a pun on a famous computer science textbook. Check out the full archives here.

Podcast

The SICPers podcast goes into detail on the history and context of software engineering ideas, and the trade-offs we make when building software.

De Programmatica Ipsum

De Programmatica Ipsum is a monthly magazine on software engineering, co-edited with my friend Adrian Kosmaczewski. Its scope is "individuals, interactions, and the true valuation of the things on the left" in software development.

Video

Graham Lee, presenting at UIKonf 2015.

This YouTube playlist contains many of the conference talks and seminars I have given on topics of testing, software quality, and software design paradigms.

Along with my friend Steven Baker I stream on two software engineering topics. In Dos Amigans, we write software for the Amiga computer using tools native to the platform. In [objc retain]; we use free software tools to develop free software in the Objective-C language.

Books

APPropriate Behaviour is a guide to the things programmers need to know, other than the programming. It's still getting updated!

OOP: the easy way is my attempt to rehabilitate Object-Oriented Programming. It got very complicated over the 1990s and 2000s, despite being introduced as a paradigm for children to write programs easily. I show that there's a simple idea lurking at the core.

Contact

You can email me: grahamlee@acm.org or find me on Fosstodon @leeg@fosstodon.org.

#### Blog
URL: https://www.sicpers.info/blog/

#### Contact
You can contact me 

You can email me at graham at iamleeg dot com.

#### Post Archives
URL: https://www.sicpers.info/archives/

#### Privacy Statement
Structure and Interpretation of Computer Programmers has some minimal server-side tracking to enable service, and uses cookies for login sessions (but only the author can log in, so this doesn't refer to you). We do not host advertising nor monetise data about our audience in any way.

#### Software Portfolio
There's a lot of talk on this site about software engineering. So let's confirm that I have indeed done some software engineering.

Academic Software

The first time I was paid for writing software, it was an emacs major mode. This was for syntax highlighting a language called GLE, a programmatic graphics engine for charts and other illustrations. Looking back at it now it isn't the most capable of modes, but it got the job done! I went on to spend a few years administering UNIX servers in the same department, because software is nothing if it isn't deployed.

My current project is Global.health, a real-time epidemiology data platform for analysing the spread and development of infectious diseases. Developed to support scientists working on COVID-19, this open source platform will be adaptable to other outbreaks. I lead engineering on an international collaboration to keep G.h up to date.

Software Security

Early in my career I was senior Mac software engineer for Sophos. I led the development of SAV for Mac 6, the first version of the Anti-Virus application written specifically for Mac OS X and laying the ground for the company to offer Sophos Home for Macs free of charge. While I was there, I also helped the architects over at Utimaco design the Mac version of their full-disk encryption software. I've also helped plenty of app makers and service providers with information security audits, including friends in the third sector.

Mobile Software

At Facebook, my team developed testing tools for automated end-to-end testing of Android and iOS apps at scale. This let us trim a four week release cycle for mobile down to 1.5 weeks, and let manual testers concentrate on novel and exploratory tests with less re-treading old test scripts.

That's not the only test-focussed role I've had! Back at Brainstorm (now EngageHub), I designed, ran, and automated correctness and performance tests for a real-time SMS, MMS, and WAP messaging and campaigns platform. If a TV channel is running a "vote by text" campaign your platform has to count the votes correctly and quickly; I made sure that it did.

Oh, and just to namedrop: I met Terry Pratchett a couple of times, as lead developer of Discworld: the Ankh-Morpork Mapp for iPad while I was at Agant. "As full of sheer exuberant business as maggots on a dead dog."

I also trained plenty of future (well, then future, now current) app developers, as an instructor at the Big Nerd Ranch. And I helped even more software engineers out by taking a turn at writing the docs for a certain fruity mobile platform company.

Supercomputers

The first thing I did when I got to Allinea (now part of ARM HPC) was to take the lead on their integration test framework, helping the team get the regression test failures down to 0 and have confidence in the development of new features for their debugger and performance analysis tools. Then I got into writing those features myself, leading development of a NUMA memory use debugger for Intel Xeon Phi.

Fintech

They said it couldn't be done. Literally. Financial advice experts said that DB transfer advice was "impossible" to automate. So of course, that's what I did, in my first few months as head of software architecture for Wealth Wizards. By the way, that was their first product to use machine learning too. I also had a hand in designing a few of the couple of hundred microservices in the platform.

### Media

#### (No Title)
URL: https://www.sicpers.info/3237/

#### 1d6.mp4
A video of my micro:bit 1d6 app.

#### 20180429_185146
URL: https://www.sicpers.info/2018/04/rethinking-object-oriented-design-figures/20180429_185146/

#### 20180429_185242
URL: https://www.sicpers.info/2018/04/rethinking-object-oriented-design-figures/20180429_185242/

#### 20180429_185256
URL: https://www.sicpers.info/2018/04/rethinking-object-oriented-design-figures/20180429_185256/

#### 20180429_185313
URL: https://www.sicpers.info/2018/04/rethinking-object-oriented-design-figures/20180429_185313/

#### 20180429_185326
URL: https://www.sicpers.info/2018/04/rethinking-object-oriented-design-figures/20180429_185326/

#### 3mpfg-front.JPG
URL: https://www.sicpers.info/2010/06/a-solution-in-need-of-a-problem/3mpfg-front-jpg/

#### 3mpfg-side.JPG
URL: https://www.sicpers.info/2010/06/a-solution-in-need-of-a-problem/3mpfg-side-jpg/

#### 3mpfg-sun.JPG
URL: https://www.sicpers.info/2010/06/a-solution-in-need-of-a-problem/3mpfg-sun-jpg/

#### amxrci
URL: https://www.sicpers.info/2026/03/on-working-machines/amxrci/

#### AppCode.png
URL: https://www.sicpers.info/2011/09/so-you-dont-like-your-ide/appcode-png/

#### avon-watch.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/avon-watch-png/

#### bigclick.png
URL: https://www.sicpers.info/2010/07/on-fitts-law-and-security/bigclick-png/

#### Bike_with_Training_Wheels.jpg
URL: https://www.sicpers.info/2013/04/as-the-kaiser-chiefs-might-say-ruby-ruby-ruby-n00bie/bike_with_training_wheels-jpg/

#### blogstats.png
URL: https://www.sicpers.info/2012/07/on-community/blogstats-png/

#### breakpointeditor.png
URL: https://www.sicpers.info/2012/04/the-debugger-of-royalty/breakpointeditor-png/

#### Byte Magazine Smalltalk Cover
URL: https://www.sicpers.info/2019/03/the-balloon-goes-up/screenshot-2019-03-01-at-17-03-08/

#### CatchFail.PNG
URL: https://www.sicpers.info/2012/03/how-to-tdd-with-catch/catchfail-png/

#### catchpass.PNG
URL: https://www.sicpers.info/2012/03/how-to-tdd-with-catch/catchpass-png/

#### catch_default.PNG
URL: https://www.sicpers.info/2012/03/how-to-tdd-with-catch/catch_default-png/

#### cb-after.png
URL: https://www.sicpers.info/2013/12/a-sneaky-preview-of-classbrowser/cb-after-png/

#### cb-before.png
URL: https://www.sicpers.info/2013/12/a-sneaky-preview-of-classbrowser/cb-before-png/

#### cb-doit.png
URL: https://www.sicpers.info/2013/12/a-sneaky-preview-of-classbrowser/cb-doit-png/

#### Certificate.png
URL: https://www.sicpers.info/2011/10/why-your-security-ui-sucks/certificate-png/

#### clacks
URL: https://www.sicpers.info/clacks/

#### cmd-line-tool.png
URL: https://www.sicpers.info/2014/03/inside-out-apps/cmd-line-tool-png/

#### Cocoa MVC
URL: https://www.sicpers.info/2014/02/messily-veneered-c/cocoamvc-png/

#### CodeRunner.png
URL: https://www.sicpers.info/2011/09/so-you-dont-like-your-ide/coderunner-png/

#### COMPUTERS.JPG
URL: https://www.sicpers.info/2014/07/padds-not-the-ipad/computers-jpg/

#### confvenn.JPG
URL: https://www.sicpers.info/2012/05/thoughts-on-tech-conferences/confvenn-jpg/

#### connected-data-hoffed
URL: https://www.sicpers.info/connected-data-hoffed/

#### cropped-F7D14238-8296-418F-924B-5CAB0BF65AD9.jpeg
URL: https://www.sicpers.info/cropped-f7d14238-8296-418f-924b-5cab0bf65ad9-jpeg/

#### demolition-man-watch.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/demolition-man-watch-png/

#### discworld-credits.png
URL: https://www.sicpers.info/2013/07/what-graham-did-next/discworld-credits-png/

#### Early Bird Push - SD 480p
URL: https://www.sicpers.info/2026/03/chiron-codex-early-bird-ends-soon/early-bird-push-sd-480p/

#### eliminate-blake.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/eliminate-blake-png/

#### Emacs.png
URL: https://www.sicpers.info/2011/09/so-you-dont-like-your-ide/emacs-png/

#### example-2-1.jpg
URL: https://www.sicpers.info/2012/07/illuminative-c/example-2-1-jpg/

#### explosion
URL: https://www.sicpers.info/explosion/

#### explosion
URL: https://www.sicpers.info/explosion-2/

#### F7D14238-8296-418F-924B-5CAB0BF65AD9
URL: https://www.sicpers.info/f7d14238-8296-418f-924b-5cab0bf65ad9/

#### fingertap.png
URL: https://www.sicpers.info/2010/07/on-fitts-law-and-security/fingertap-png/

#### gloom
URL: https://www.sicpers.info/2018/12/figurative-programming-and-gloom-the-graphical-loom/gloom/

#### grotag.png
URL: https://www.sicpers.info/2020/08/grotag/grotag-png/

#### HelloGSW.png
URL: https://www.sicpers.info/2012/03/using-objective-c-on-the-server/hellogsw-png/

#### iamleeg _FO82FCD66EF46-01
URL: https://www.sicpers.info/about/iamleeg-_fo82fcd66ef46-01/

#### IKBCommand.JPG
URL: https://www.sicpers.info/2013/06/separating-user-interface-from-work/ikbcommand-jpg/

#### image
URL: https://www.sicpers.info/image-2/

#### imagesfolder.jpg
URL: https://www.sicpers.info/2010/02/look-what-the-feds-left-behind/imagesfolder-jpg/

#### imagesIMAG0101.jpg
URL: https://www.sicpers.info/2011/03/on-being-a-software-person/imagesimag0101-jpg/

#### imagesPicture-1.png
URL: https://www.sicpers.info/2010/03/why-do-we-annoy-our-users/imagespicture-1-png/

#### imagesssh-add.png
URL: https://www.sicpers.info/2010/03/integrating-ssh-with-the-keychain-on-snow-leopard/imagesssh-add-png/

#### imagesssh-keychain.png
URL: https://www.sicpers.info/2010/03/integrating-ssh-with-the-keychain-on-snow-leopard/imagesssh-keychain-png/

#### img_0003-1
URL: https://www.sicpers.info/2017/11/the-atoms-of-programming/img_0003-1/

#### IMG_0114.JPG
URL: https://www.sicpers.info/2012/04/culture-heritage-and-apps/img_0114-jpg/

#### IMG_0475
URL: https://www.sicpers.info/2025/02/on-nostalgia-for-physical-media/img_0475/

#### IMG_3356.jpeg
URL: https://www.sicpers.info/2021/05/graham-lee-uses-this/img_3356-jpeg/

#### IMG_3357.jpeg
URL: https://www.sicpers.info/2021/05/graham-lee-uses-this/img_3357-jpeg/

#### IMG_3943
URL: https://www.sicpers.info/2023/11/we-shall-return-one-day/img_3943/

#### Input
URL: https://www.sicpers.info/img_2883-jpg/

#### janus.jpg
URL: https://www.sicpers.info/2014/03/software-science/janus-jpg/

#### kirk-watch.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/kirk-watch-png/

#### knight-watch.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/knight-watch-png/

#### Labrary note template
URL: https://www.sicpers.info/2019/01/the-app-that-wasnt-yet/screenshot-2019-01-02-at-10-48-36/

#### Labrary Prototype
URL: https://www.sicpers.info/2019/01/the-app-that-wasnt-yet/20190102_101601/

#### Labrary prototype screenshot
URL: https://www.sicpers.info/2019/01/the-app-that-wasnt-yet/labrary_screenshot/

#### labrary-colour
URL: https://www.sicpers.info/labrary-colour/

#### layer_cake.png
URL: https://www.sicpers.info/2014/10/the-trouble-with-layers/layer_cake-png/

#### LibreLiteracy
URL: https://www.sicpers.info/libreliteracy/

#### life.png
URL: https://www.sicpers.info/2015/05/object-oriented-programming-in-objective-c/life-png/

#### MacBook Pro
URL: https://www.sicpers.info/2018/10/making-my-peace/20181004_192537/

#### msg-send.png
URL: https://www.sicpers.info/2013/12/by-your-_cmd/msg-send-png/

#### MVC-organisation.png
URL: https://www.sicpers.info/2014/03/inside-out-apps/mvc-organisation-png/

#### MVC1.png
URL: https://www.sicpers.info/2010/07/on-detecting-god-classes/mvc1-png/

#### MVC2.png
URL: https://www.sicpers.info/2010/07/on-detecting-god-classes/mvc2-png/

#### MVC3.png
URL: https://www.sicpers.info/2010/07/on-detecting-god-classes/mvc3-png/

#### ObjC Layers
URL: https://www.sicpers.info/2014/02/messily-veneered-c/objclayers-png/

#### ObjectiveBrowser.png
URL: https://www.sicpers.info/2012/05/messing-about-with-clang/objectivebrowser-png/

#### padd.jpg
URL: https://www.sicpers.info/2014/07/padds-not-the-ipad/padd-jpg/

#### padds.png
URL: https://www.sicpers.info/2014/07/padds-not-the-ipad/padds-png/

#### Path of Parsimony
URL: https://www.sicpers.info/path-of-parsimony/

#### permutations
URL: https://www.sicpers.info/2012/08/i-made-a-web/permutations/

#### Picture 1
URL: https://www.sicpers.info/picture-1/

#### plusminus.png
URL: https://www.sicpers.info/2010/07/on-fitts-law-and-security/plusminus-png/

#### Pub walks in Warwickshire
URL: https://www.sicpers.info/2019/01/grooming-the-backfog/20190129_145946/

#### sav.png
URL: https://www.sicpers.info/2010/11/on-free-mac-anti-virus-2/sav-png/

#### Scream.jpg
URL: https://www.sicpers.info/2013/02/how-to-handle-xcode-in-your-meta-build-systems-ios-or-mac-app-target/scream-jpg/

#### Screen-shot-2011-04-06-at-16.53.22.png
URL: https://www.sicpers.info/2011/04/a-first-look-at-appcode-and-the-future-of-cocoa-ides/screen-shot-2011-04-06-at-16-53-22-png/

#### Screen-shot-2011-04-06-at-17.19.32.png
URL: https://www.sicpers.info/2011/04/a-first-look-at-appcode-and-the-future-of-cocoa-ides/screen-shot-2011-04-06-at-17-19-32-png/

#### Screen-Shot-2013-07-04-at-14.22.27.png
URL: https://www.sicpers.info/2013/07/ajax-via-jquery-in-an-objective-c-webobjects-app/screen-shot-2013-07-04-at-14-22-27-png/

#### Screen-Shot-2013-07-04-at-14.37.26.png
URL: https://www.sicpers.info/2013/07/ajax-via-jquery-in-an-objective-c-webobjects-app/screen-shot-2013-07-04-at-14-37-26-png/

#### Screenshot 2026-01-12 at 09.54.12
URL: https://www.sicpers.info/2026/01/configuring-your-computer-for-local-inference-with-a-generative-ai-coding-assistant/screenshot-2026-01-12-at-09-54-12/

#### Screenshot 2026-01-12 at 10.16.04
URL: https://www.sicpers.info/2026/01/configuring-your-computer-for-local-inference-with-a-generative-ai-coding-assistant/screenshot-2026-01-12-at-10-16-04/

#### Screenshot-2020-07-09-at-19.11.12.png
URL: https://www.sicpers.info/2020/07/tiger-to-catalina-lets-port-some-code/screenshot-2020-07-09-at-19-11-12-png/

#### Screenshot-2020-07-09-at-19.11.12.png
URL: https://www.sicpers.info/2020/07/tiger-to-catalina-lets-port-some-code/screenshot-2020-07-09-at-19-11-12-png-2/

#### Screenshot-2020-07-09-at-19.11.26.png
URL: https://www.sicpers.info/2020/07/tiger-to-catalina-lets-port-some-code/screenshot-2020-07-09-at-19-11-26-png/

#### Screenshot-2020-07-09-at-19.11.26.png
URL: https://www.sicpers.info/2020/07/tiger-to-catalina-lets-port-some-code/screenshot-2020-07-09-at-19-11-26-png-2/

#### Screenshot-2020-10-12-at-17.08.30.png
URL: https://www.sicpers.info/2020/10/running-linux-gui-apps-under-macos-using-docker/screenshot-2020-10-12-at-17-08-30-png/

#### Screenshot-2022-01-28-at-12.33.43
URL: https://www.sicpers.info/about/screenshot-2022-01-28-at-12-33-43/

#### Screenshot-2022-01-28-at-12.47.23
URL: https://www.sicpers.info/about/screenshot-2022-01-28-at-12-47-23/

#### signup-1.png
URL: https://www.sicpers.info/2013/11/updating-my-objc-web-app-on-git-push/signup-1-png/

#### signup-2.png
URL: https://www.sicpers.info/2013/11/updating-my-objc-web-app-on-git-push/signup-2-png/

#### smallclick.png
URL: https://www.sicpers.info/2010/07/on-fitts-law-and-security/smallclick-png/

#### Smalltalk MVC
URL: https://www.sicpers.info/2014/02/moderately-valuable-cliche/smalltalkmvc-png/

#### Smalltalk-80 books
URL: https://www.sicpers.info/2014/02/missing-vital-content/smalltalk_books-jpg/

#### SNfoot.JPG
URL: https://www.sicpers.info/2011/08/remember-the-future/snfoot-jpg/

#### SNneck.JPG
URL: https://www.sicpers.info/2011/08/remember-the-future/snneck-jpg/

#### SNtop.JPG
URL: https://www.sicpers.info/2011/08/remember-the-future/sntop-jpg/

#### SNtrunk.JPG
URL: https://www.sicpers.info/2011/08/remember-the-future/sntrunk-jpg/

#### sophos-eicar.png
URL: https://www.sicpers.info/2010/11/on-free-mac-anti-virus-2/sophos-eicar-png/

#### subscriptions
URL: https://www.sicpers.info/2017/01/resolution-share-subscriptions/subscriptions/

#### Swift tower
URL: https://www.sicpers.info/2019/04/new-swift-hardware/d72a8cc3-68a3-4efb-b128-68e77c6dedcd/

#### table1-1.png
URL: https://www.sicpers.info/2012/09/an-apology-to-readers-of-test-driven-ios-development/table1-1-png/

#### thunderbirds-watch.png
URL: https://www.sicpers.info/2014/06/wristwatches-in-the-future/thunderbirds-watch-png/

#### TMVE
URL: https://www.sicpers.info/2014/01/meaningless-vapid-catchphrase/tvme/

#### Ubuntu
URL: https://www.sicpers.info/2018/10/making-my-peace/img_20141110_080719/

#### win8_1.jpg
URL: https://www.sicpers.info/2014/04/code-longevity/win8_1-jpg/

#### xcode-managedtest.png
URL: https://www.sicpers.info/2010/06/template-class-for-unit-testing-core-data-entities/xcode-managedtest-png/

#### zelda.jpg
URL: https://www.sicpers.info/2014/04/preparing-for-computings-big-one-oh-oh/zelda-jpg/

#### Zim-as-Labrary
URL: https://www.sicpers.info/2019/01/the-app-that-wasnt-yet/screenshot-2019-01-02-at-11-11-58/