Ultimate Programmer Super Stack: Last day!

I already wrote about the Ultimate Programmer Super Stack, a huge bundle of books and courses on a range of technologies: Python, JS, Ruby, Java, HTML, node, Aurelia… and APPropriate Behaviour, my book on everything that goes into being a programmer that isn’t programming.

Today is the last day of the bundle. Check it out here, it won’t be available for long.

Coming to terms with fewer terms

I was on a “Leadership in Architecture” panel organised by RP International recently, and was asked about problems we face using new techniques like Microservices, serverless and machine learning in the financial technology sector. The biggest blocker I see is the RFP (Request for Proposals), RFI (Request for Information), the MSA (Master Service Agreement), any document with a three-letter acronym. We would do better if they disappeared.

I’m fully paid up in my tithe to the church of “customer collaboration over contract negotiation”, and I believe that this needs to extend beyond the company boundary. If we’re going to spend a few months going back and forth over waving our certifications about, deciding who gets to contact whom within what time, and whether the question they asked constitutes a “bug report” or a “feature request”, then I don’t think it matters whether the development team use two-week sprints or not. We’ve already lost.

We’ve lost because we know that the interactions between the people involved are going to be restricted to the terms agreed during that negotiation. No longer are people concerned about whether the thing we’re making is valuable; they’re concerned with making sure their professional indemnity insurance is up to date before sending an email to the DRI (Definitely Responsibility-free Inbox).

We’ve lost because we had a team sitting on its hands during the negotiation, and used that time “productively” by designing the product, putting epics and stories in a backlog, grooming that backlog, making wireframes, and all of those other things that aren’t working software.

We’ve lost because each incompatibility between the expectation and our intention is a chance to put even more contract negotiation in place, instead of getting on with making the working software. When your RFI asks which firewall ports you need to open into your DMZ, and our answer is none because the software runs outside of your network on a cloud platform, we’re not going to get into discussions of continuous delivery and whether we both read the Phoenix Project. We’re going to get into discussions of whether I personally will warrant against Amazon outages. But here’s the thing: we don’t need the software to be 100% up yet, we don’t even know whether it’s useful yet.

Here’s an alternative.

  1. We, collectively, notice that the software we make solves the problem you have.
  2. We, collectively, agree that you can use the software we have now for a couple of weeks.
  3. We, collectively, discuss the things that would make the software better at solving the problem.
  4. We, collectively, get those things done.
  5. We, collectively, GO TO 2.

Notice that you may have to pay for steps 2-4.

Microservices for the Desktop

In OOP the Easy Way, I make the argument that microservices are a rare instance of OOP done well:

Microservice adopters are able to implement different services in different technologies, to think about changes to a given service only in terms of how they satisfy the message contract, and to independently replace individual services without disrupting the whole system. This […] sounds a lot like OOP.

Microservices are an idea from service-oriented architecture (SOA) in which each application—each microservice—represents a distinct bounded context in the problem domain. If you’re a movie theatre complex, then selling tickets to people is a very different thing from showing movies in theatres, that are coupled loosely at the point that a ticket represents the right to a given seat in a given theatre at a given showing. So you might have a microservice that can tell people what showings there are at what times and where, and another microservice that can sell people tickets.

People who want to write scalable systems like microservices, because they can scale different parts of their application separately. Maybe each franchisee in the theatre chain needs one instance of one service, but another should scale as demand grows, sharing a central resource pool.

Never mind all of that. The real benefit of microservices is that they make boundary-crossing more obvious, maybe even more costly, and as a result developers think about where the boundaries should be. The “problem” with monolithic (single-process) applications was never, really, that the deployment cost too much: one corollary of scale is that you have more customers. It was that there was no real enforcement of separate parts of the problem domain. If you’ve got a thing over here that needs that data over there, it’s easy to just change its visibility modifier and grab it. Now this thing and that thing are coupled, whoops!

When this thing and that thing are in separate services, you’re going to have to expose a new endpoint to get that data out. That’s going to make it part of that thing’s public commitment: a slightly stronger signal that you’re going down a complex path.

It’s possible to take the microservices idea and use it in other contexts than “the backend”. In one Cocoa app I’m working on, I’ve taken the model (the representation in objects of the problem I’m solving) and put it into an XPC Plugin. XPC is a lot like old-style Distributed Objects or even CORBA or DCOM, with the exception that there are more safety checks, and everything is asynchronous. In my case, the model is in Objective-C in the plugin, and the application is in Swift in the host process.

“Everything is asynchronous” is a great reminder that the application and the model are communicating in an arm’s-reach fashion. My model is a program that represents the domain problem, as mentioned before. All it can do is react to events in the problem domain and represent the changes in that domain. My application is a reification of the Cocoa framework to expose a user interface. All it can do is draw stuff to the screen, and react to events in the user interface. The app and the model have to collaborate, because the stuff that gets drawn should be related to the problem, and the UI events should be related to desired changes in the domain. But they are restricted to collaborating over the published interface of the XPC service: a single protocol.

XPC was designed for factoring applications, separating the security contexts of different components and giving the host application the chance to stay alive when parts of the system fail. Those are valid and valuable benefits: the XPC service hosting the model only needs to do computation and allocate memory. Drawing (i.e. messaging the window server) is done elsewhere. So is saving and loading. And that helps enforce the contract, because if I ever find myself wanting to put drawing in the model I’m going to cross a service boundary, and I’m going to need to think long and hard about whether that is correct.

If you want to talk more about microservices, XPC services, and how they’re different or the same, and how I can help your team get the most out of them, you’re in luck! I’ve recently launched the Labrary—the intersection of the library and the laboratory—for exactly that purpose.

Ultimate Programmer Super Stack

There’s a great bundle of polyglot learning taking place over at the Ultimate Programmer Super Stack. My book, APPropriate Behaviour – the things every programmer needs to know that aren’t programming – is featured alongside content on Python, Ruby, Java, JS, Aurelia, Node, startups, and more.

The bundle is just up for a week, but please do check it out: for not much more than you’d probably pay for APPropriate Behaviour you’ll get a whole heap of stuff that should keep you entertained for a while :).

Introducing: the Labrary

Is it that a month in the laboratory will save an hour in the library, or the other way around? A little more conversation, a little less action?

There are things to learn from both the library and the laboratory, and that’s why I’m launching the Labrary, providing consulting detective and training service to software teams who need to solve problems, and to great engineers who want to be great lead engineers, principal engineers and architects.

The Labrary is also the home to my books and other projects to come. So if you want to find out what a consulting detective can do for your team, follow the @labrarian on Mastodon or book office hours to talk things over.

On Blue Agile

Ron Jeffries has some interesting posts lately on Dark Scrum, the idea that poor programmers are being chained to the code face in the software mines, forced to unthinkingly crank out features under Agile-sequel banners like “velocity” and “embracing change”.

In one such post, he refutes the notion of a shadowy Agile Indu$trial Complex. The AIC is a hypothetical network of consultants, tool vendors and project managers who collectively profit from every story point, every Jira “story”, and every grooming session.

Here’s the thing, though. You don’t need fnord filters and Masonic handshakes to explain Dark Scrum. You need to understand that Scrum is doing exactly as intended, and that it’s orthogonal to—not opposite to—the intentions of Agile software development.

Alan Kay had this analogy of two perpendicular planes that he used to explain the difference between Object-Oriented Programming and the thing programmers do in Java. The pink plane contains existing ideas and processes. You can get better at your craft by advancing along the pink plane.

Every so often, an idea comes along (OOP, says Kay, is one such idea) that is not better, it is different. It takes you out of the pink plane altogether into the orthogonal blue plane. You can’t ask “is this idea better” because it doesn’t make sense. It’s different. It has different qualities, so your question about what makes things better no longer makes sense.

Back when we were all allegedly doing waterfall software development, we were delivering software that satisfied requirements defined for some project. We can ask “how good are we at delivering software that satisfies requirements”, and define “better” answers to that question.

Scrum is a process improvement framework for delivering products. As such it provides a “good” baseline for software delivery, and tools to help us get “better” at it. Scrum is the pink plane.

The Agile crowd, on the other hand, stopped asking how much software we are delivering, and started asking how valuable our interactions with our customers are. It’s in the blue plane. The questions we had in the pink plane are no longer relevant, so if we’re still asking them, we will get nonsensical answers.

It is possible, even likely, that the rise of Scrum is due to trying to apply pink plane thinking to the Agile idea, in the way that the rise of C++ or SOLID is due to pink plane thinking about OOP. You could imagine someone who manages a software team seeing a lot of software coming out of an XP team. They read about the XP practices and conclude “I need to make my team adopt these practices, and we’ll make more software”.

But perhaps the XP team weren’t worried about making more software, and don’t even understand why making more software would be a goal.

That doesn’t make Scrum “bad”, indeed looked at along the pink plane it’s better than its predecessors. It just makes it unexpected, and disappointing as a result.

I’m probably holding it wrong

If I wanted to do a table view data source in ObjC, it would look like this:

- tableView:aTableView objectValueForTableColumn:aColumn row:(NSInteger)row {
  return [representedObject.collection[row] valueForKey:[aColumn identifier]];

When I do it in Swift, it ends up looking like this:

func tableView(_ tableView: NSTableView, objectValueFor tableColumn: NSTableColumn?, row: Int) -> Any? {
    guard let identifier = tableColumn?.identifier else {
        assertionFailure("No table column")
        return nil
    guard let obj = (self.representedObject as? ModelType)?.collection(at:row) else {
        assertionFailure("Can't find model object at \(row)")
        return nil
    switch identifier {
    case NSUserInterfaceItemIdentifier(rawValue:"column1"):
        return obj.field1
    case NSUserInterfaceItemIdentifier(rawValue:"column2"):
        return objc.field2
        assertionFailure("Unknown table column \(tableColumn?.identifier ?? NSUserInterfaceItemIdentifier(rawValue: "unknown"))")
        return nil

I can’t help feeling I’m doing it wrong.

On the inevitability of Photoshop for iPad

Back in 2011, I was speaking at QCon London at the invitation of my friend and de Programmatica Ipsum co-conspirator akosma, and one of the conference’s community events was an iOS developer meet-up hosted in the conference centre. I think we had a speaker panel of the conference mobile track speakers: regardless, there was a panel, and I was on it.

This was when Steve Jobs’ analogy of PCs as trucks, iPads as cars was still fresh in everybody’s mind. Consensus in the room was that this made sense, that the iPad was an everyday computer where a Mac is “for pros”, and you couldn’t do a pro app, say Photoshop, for the iPad.

I was angry that a bunch of people who say that they are clever at making computers do things could so easily reject the idea that a computer could do a thing, particularly when it was a thing computers could already do. In a huff, I stomped out of the room, only to stomp back in a few minutes later carrying a flipchart. I turned to the first page and drew a big black rounded rectangle. “OK”, I said, “we’re going to design Photoshop for iPad. Go.”

Unsurprisingly, the room designed Photoshop for iPad. Nothing changed about Photoshop, or about the iPad, or about these people, except that previously they had been told by no less a person than Apple’s CEO that iPads should not be thought of as a computer for doing computer things. I had told them that it could be used for computer things and that they were the people who could make it happen, lo and behold, it happened. I don’t remember whether I had even used an iPad at that time; nonetheless, I led a team of designers who designed Photoshop for iPad.

What Apple were really saying with the trucks metaphor was “this is a new platform, please have low expectations”. “No Photoshop. No Office. Lame.” was not the review they wanted to see, and by controlling the narrative around what you should expect from an iPad, they controlled whether it lived up to expectations.

I think the point of my post is “we usually expect marketing to hype things up, beware of marketers hyping down your imagination”. I’m not quite ready to finish yet, though.

This year, of course, there are iPad Pros, for Pros, that do the kind of truck stuff we were told iPads are not for, such as Photoshop. This was inevitable. Unless Apple or Adobe went out of business, or the iPad or Photoshop really tanked, there was going to be Photoshop for iPad.

I’m wondering who wrote it, though.

I have no doubt that the developers at Adobe are capable of doing it. I also have no doubt that it’s strategically important for Apple in their new “iPad Pros are trucks” world, that there should be Pro apps for iPad Pros. I know that all of the platform vendors are happy to write ports of apps they want to see on their platforms, and give them to the app vendors to release under their own brands. To me, the story “Adobe realised this was a valuable addition for Creative Cloud customers” and the story “Apple realised this was a valuable addition for iPad Pro perception” are both convincing.

Beginner thoughts

Back story: my period of walkabout, in which I went to see the rest of the computing world beyond Apple land, started in November 2014. This was shortly after Swift’s introduction at WWDC 2014. It ended in October 2018, by which time the language had evolved considerably, its position in the community had advanced greatly, and SourceKitService had stopped crashing.

I have previously written on the learning phases I encountered on exposure to Haskell, now what about Swift? I have the opportunity to reflect on how I react as a beginner, and share that so that we all learn how we (well, I) learn, and maybe discover how we can teach.

About the project

I’m writing a tool that I want, which takes files in one format (RSS) and writes them out in another format (Maildir). You can follow along. The reason for mentioning this here are twofold:

  • I do not know what I’m doing, but I’m willing to share that.
  • To let you understand the (limited, I think) complexity of the thing I’m trying to build.

Thinks: This should not be that hard

I often feel like Swift is making me feel like an idiot. This is because my expectation is too high: I know the platform fairly well. I know the Foundation framework pretty well. I know Xcode pretty well. I understand my problem to some extent. It should just be the programming language that’s different.

And I do different programming languages all the time. What’s another one going to do?

But of course it’s not just the programming language that changed. It changed the conventions for things like naming methods or raising errors, and that means that the framework methods have changed, which means that things I used to know have changed, which means that I do not know as much as I assume. It introduced a new library, which I also don’t know.

Thinks: That unimportant thing was really frustrating

Two such convention changes are correlated: classes that used to be Foundation and are now standard library (or maybe are Foundation still but have been renamed on being bridged, I’m not sure) are renamed from NSThing to Thing. That means that the name of NSURL is now URL.

That means that if you have a variable that represents a URL, you can’t follow the Cocoa convention of leaving the abbreviation uppercased and calling it URL, because now it’s got the same name as the type URL. So the new convention is to call it url.

Objectively, that’s not a big deal. Subjectively, this stuff is baked in pretty deep, and changing it is hard.

Thinks: Even learning something is frustrating

The last event to make me get up and walk around a field was actually discovering something new about Swift, which should be the point, but nonetheless made me feel bad.

I have discovered that when it comes to working with optionals, the language syntax means that There Is More Than One Way To Do It. When I learned about if let and guard let, I was confused by the fact that the thing on the right needed to be an optional, not unwrap one: surely if my rvalue is an optional, then my lvalue should be, too?

Then, when I learned about the ?. and subsequently ?? operators, I thought “there’s no way I would ever have thought to type that, or known how to search for those things”. And even though they only make things shorter, not different, I still felt frustration at the fact that I’d gone through typing things out the long way.

Thinks: There’s More Than One Way Not To Do It

One of the Broken Expectations™ is that I know how to use Strings. Way back when, NeXT apps used char * as their string type. Then Enterprise Objects Framework came along with its Foundation library of data types, including a new-fangled Unicode string class, NSString. Then, well, that was it for absolute ages.

So, when I had a String and I wanted to take the substring to an index, I was familiar with -substringToIndex: and tried to apply that. That method is deprecated, so I didn’t want to use it. OK, well I can string[0..<N]. Apparently not, integer subscripting is not allowed, and the error message tells me to read a code comment to understand why. I wish it told me where that code comment was, or just showed it to me, instead!

Eventually I found that there’s a .prefix(N) method, again this is the sort of thing that makes me think: what’s wrong with me? I’ve been programming for years, I’ve been programming on this platform for years, I should be able to get this.

Conclusion: Read a Book

I had expected that my knowledge of the Mac, Xcode, and Cocoa would be sufficient to carry me through a four-year gap on picking up a language, particularly with the occasional observation of a conference talk about the Swift language (I’ve even given one!). I had expected that picking up a project to build a thing would give me a chance to get acquainted.

I was reflecting on my early experiences with writing NeXT and Mac applications in Objective-C. I had my copy of the NeXT Developer Documentation, or Cocoa in a Nutshell, open on the desk, looking at the methods available and thinking “I have this, I want that, can I find one of these that gets me there?” I had expected that auto-complete in Xcode would be my modern equivalent of working that way.

Evidently not. Picking up the new standard library things, and the new operators, will require deliberate learning. I’ve got some videos lined up, but I think my next action is to find a good book to read.

More on UIAutomation tests


The information below is mostly redundant. After filing a bug report with Apple, their engineers determined that the Xcode-detected set of macro actions (find a text field, double click, enter text) weren’t working because the double click action wasn’t editing the text field. It is possible to use UIAutomation Tests, you just have to carefully review the UI actions and determine that they have the effect expected, particularly after letting Xcode record UI macros.

Original Post

Unfortunately my work to organise UIAutomation tests has hit the stumbling block that the UI Automation runner doesn’t use the main thread for main-thread-only APIs.

In Xcode 9 and High Sierra, the authors of that post I just linked found that it was possible to turn off the main thread checker in the Test configuration of the build scheme and get working tests anyway. Unfortunately that doesn’t work for me in Xcode 10 and Mojave: the main thread checker isn’t killing the app: the TSM subsystem is just refusing to do its thing. So my tests can’t do straightforward things like write text into a text field. Unfortunately this is a “it’s not me, it’s you” moment, and I don’t think I can carry on using Xcode’s UI tests for my goals.

However, I still want to be able to write “end-to-end” level tests to drive my development. I have (at least) three ways to proceed:

  • I could find a third party library and discover whether it has the main thread problem. Calabash doesn’t support Mac apps, and the other examples I can find (Cucumberish and TABTestKit) both rely on UI Automation so presumably don’t address the main thread problem.
  • I could write the tests in AppleScript. That would be a good way to build up the AppleScript UI for the app, but it doesn’t represent an end-to-end test of the GUI.
  • I could write the tests using NSApplication.sendEvent(_ event:) to simulate clicks, scrolls and text entry, and use the unit test runner to host them. That could work, but I have my doubts (I would guess that the runner is synchronous and stalls the main thread).

I discovered that it is possible to write the test “at the UI level” but in the unit runner, using a combination of key events and AppKit API like sendAction( to:). The trade-offs of this approach:

  • it takes longer, as the abstractions needed to easily find and use AppKit controls don’t (currently) exist
  • it doesn’t use the Accessibility interface so isn’t an accessibility audit at the same time as a correctness test
  • you don’t hit the same problems as with the UI Automation runner
  • it’s much faster

This may be the best approach for now, though I’d welcome other views.