Introducing: the Labrary

Is it that a month in the laboratory will save an hour in the library, or the other way around? A little more conversation, a little less action?

There are things to learn from both the library and the laboratory, and that’s why I’m launching the Labrary, providing consulting detective and training service to software teams who need to solve problems, and to great engineers who want to be great lead engineers, principal engineers and architects.

The Labrary is also the home to my books and other projects to come. So if you want to find out what a consulting detective can do for your team, follow the @labrarian on Mastodon or book office hours to talk things over.

On Blue Agile

Ron Jeffries has some interesting posts lately on Dark Scrum, the idea that poor programmers are being chained to the code face in the software mines, forced to unthinkingly crank out features under Agile-sequel banners like “velocity” and “embracing change”.

In one such post, he refutes the notion of a shadowy Agile Indu$trial Complex. The AIC is a hypothetical network of consultants, tool vendors and project managers who collectively profit from every story point, every Jira “story”, and every grooming session.

Here’s the thing, though. You don’t need fnord filters and Masonic handshakes to explain Dark Scrum. You need to understand that Scrum is doing exactly as intended, and that it’s orthogonal to—not opposite to—the intentions of Agile software development.

Alan Kay had this analogy of two perpendicular planes that he used to explain the difference between Object-Oriented Programming and the thing programmers do in Java. The pink plane contains existing ideas and processes. You can get better at your craft by advancing along the pink plane.

Every so often, an idea comes along (OOP, says Kay, is one such idea) that is not better, it is different. It takes you out of the pink plane altogether into the orthogonal blue plane. You can’t ask “is this idea better” because it doesn’t make sense. It’s different. It has different qualities, so your question about what makes things better no longer makes sense.

Back when we were all allegedly doing waterfall software development, we were delivering software that satisfied requirements defined for some project. We can ask “how good are we at delivering software that satisfies requirements”, and define “better” answers to that question.

Scrum is a process improvement framework for delivering products. As such it provides a “good” baseline for software delivery, and tools to help us get “better” at it. Scrum is the pink plane.

The Agile crowd, on the other hand, stopped asking how much software we are delivering, and started asking how valuable our interactions with our customers are. It’s in the blue plane. The questions we had in the pink plane are no longer relevant, so if we’re still asking them, we will get nonsensical answers.

It is possible, even likely, that the rise of Scrum is due to trying to apply pink plane thinking to the Agile idea, in the way that the rise of C++ or SOLID is due to pink plane thinking about OOP. You could imagine someone who manages a software team seeing a lot of software coming out of an XP team. They read about the XP practices and conclude “I need to make my team adopt these practices, and we’ll make more software”.

But perhaps the XP team weren’t worried about making more software, and don’t even understand why making more software would be a goal.

That doesn’t make Scrum “bad”, indeed looked at along the pink plane it’s better than its predecessors. It just makes it unexpected, and disappointing as a result.

I’m probably holding it wrong

If I wanted to do a table view data source in ObjC, it would look like this:

- tableView:aTableView objectValueForTableColumn:aColumn row:(NSInteger)row {
  return [representedObject.collection[row] valueForKey:[aColumn identifier]];
}

When I do it in Swift, it ends up looking like this:

func tableView(_ tableView: NSTableView, objectValueFor tableColumn: NSTableColumn?, row: Int) -> Any? {
    guard let identifier = tableColumn?.identifier else {
        assertionFailure("No table column")
        return nil
    }
    guard let obj = (self.representedObject as? ModelType)?.collection(at:row) else {
        assertionFailure("Can't find model object at \(row)")
        return nil
    }
    switch identifier {
    case NSUserInterfaceItemIdentifier(rawValue:"column1"):
        return obj.field1
    case NSUserInterfaceItemIdentifier(rawValue:"column2"):
        return objc.field2
    //...
    default:
        assertionFailure("Unknown table column \(tableColumn?.identifier ?? NSUserInterfaceItemIdentifier(rawValue: "unknown"))")
        return nil
    }
}

I can’t help feeling I’m doing it wrong.

On the inevitability of Photoshop for iPad

Back in 2011, I was speaking at QCon London at the invitation of my friend and de Programmatica Ipsum co-conspirator akosma, and one of the conference’s community events was an iOS developer meet-up hosted in the conference centre. I think we had a speaker panel of the conference mobile track speakers: regardless, there was a panel, and I was on it.

This was when Steve Jobs’ analogy of PCs as trucks, iPads as cars was still fresh in everybody’s mind. Consensus in the room was that this made sense, that the iPad was an everyday computer where a Mac is “for pros”, and you couldn’t do a pro app, say Photoshop, for the iPad.

I was angry that a bunch of people who say that they are clever at making computers do things could so easily reject the idea that a computer could do a thing, particularly when it was a thing computers could already do. In a huff, I stomped out of the room, only to stomp back in a few minutes later carrying a flipchart. I turned to the first page and drew a big black rounded rectangle. “OK”, I said, “we’re going to design Photoshop for iPad. Go.”

Unsurprisingly, the room designed Photoshop for iPad. Nothing changed about Photoshop, or about the iPad, or about these people, except that previously they had been told by no less a person than Apple’s CEO that iPads should not be thought of as a computer for doing computer things. I had told them that it could be used for computer things and that they were the people who could make it happen, lo and behold, it happened. I don’t remember whether I had even used an iPad at that time; nonetheless, I led a team of designers who designed Photoshop for iPad.

What Apple were really saying with the trucks metaphor was “this is a new platform, please have low expectations”. “No Photoshop. No Office. Lame.” was not the review they wanted to see, and by controlling the narrative around what you should expect from an iPad, they controlled whether it lived up to expectations.

I think the point of my post is “we usually expect marketing to hype things up, beware of marketers hyping down your imagination”. I’m not quite ready to finish yet, though.

This year, of course, there are iPad Pros, for Pros, that do the kind of truck stuff we were told iPads are not for, such as Photoshop. This was inevitable. Unless Apple or Adobe went out of business, or the iPad or Photoshop really tanked, there was going to be Photoshop for iPad.

I’m wondering who wrote it, though.

I have no doubt that the developers at Adobe are capable of doing it. I also have no doubt that it’s strategically important for Apple in their new “iPad Pros are trucks” world, that there should be Pro apps for iPad Pros. I know that all of the platform vendors are happy to write ports of apps they want to see on their platforms, and give them to the app vendors to release under their own brands. To me, the story “Adobe realised this was a valuable addition for Creative Cloud customers” and the story “Apple realised this was a valuable addition for iPad Pro perception” are both convincing.

Beginner thoughts

Back story: my period of walkabout, in which I went to see the rest of the computing world beyond Apple land, started in November 2014. This was shortly after Swift’s introduction at WWDC 2014. It ended in October 2018, by which time the language had evolved considerably, its position in the community had advanced greatly, and SourceKitService had stopped crashing.

I have previously written on the learning phases I encountered on exposure to Haskell, now what about Swift? I have the opportunity to reflect on how I react as a beginner, and share that so that we all learn how we (well, I) learn, and maybe discover how we can teach.

About the project

I’m writing a tool that I want, which takes files in one format (RSS) and writes them out in another format (Maildir). You can follow along. The reason for mentioning this here are twofold:

  • I do not know what I’m doing, but I’m willing to share that.
  • To let you understand the (limited, I think) complexity of the thing I’m trying to build.

Thinks: This should not be that hard

I often feel like Swift is making me feel like an idiot. This is because my expectation is too high: I know the platform fairly well. I know the Foundation framework pretty well. I know Xcode pretty well. I understand my problem to some extent. It should just be the programming language that’s different.

And I do different programming languages all the time. What’s another one going to do?

But of course it’s not just the programming language that changed. It changed the conventions for things like naming methods or raising errors, and that means that the framework methods have changed, which means that things I used to know have changed, which means that I do not know as much as I assume. It introduced a new library, which I also don’t know.

Thinks: That unimportant thing was really frustrating

Two such convention changes are correlated: classes that used to be Foundation and are now standard library (or maybe are Foundation still but have been renamed on being bridged, I’m not sure) are renamed from NSThing to Thing. That means that the name of NSURL is now URL.

That means that if you have a variable that represents a URL, you can’t follow the Cocoa convention of leaving the abbreviation uppercased and calling it URL, because now it’s got the same name as the type URL. So the new convention is to call it url.

Objectively, that’s not a big deal. Subjectively, this stuff is baked in pretty deep, and changing it is hard.

Thinks: Even learning something is frustrating

The last event to make me get up and walk around a field was actually discovering something new about Swift, which should be the point, but nonetheless made me feel bad.

I have discovered that when it comes to working with optionals, the language syntax means that There Is More Than One Way To Do It. When I learned about if let and guard let, I was confused by the fact that the thing on the right needed to be an optional, not unwrap one: surely if my rvalue is an optional, then my lvalue should be, too?

Then, when I learned about the ?. and subsequently ?? operators, I thought “there’s no way I would ever have thought to type that, or known how to search for those things”. And even though they only make things shorter, not different, I still felt frustration at the fact that I’d gone through typing things out the long way.

Thinks: There’s More Than One Way Not To Do It

One of the Broken Expectations™ is that I know how to use Strings. Way back when, NeXT apps used char * as their string type. Then Enterprise Objects Framework came along with its Foundation library of data types, including a new-fangled Unicode string class, NSString. Then, well, that was it for absolute ages.

So, when I had a String and I wanted to take the substring to an index, I was familiar with -substringToIndex: and tried to apply that. That method is deprecated, so I didn’t want to use it. OK, well I can string[0..<N]. Apparently not, integer subscripting is not allowed, and the error message tells me to read a code comment to understand why. I wish it told me where that code comment was, or just showed it to me, instead!

Eventually I found that there’s a .prefix(N) method, again this is the sort of thing that makes me think: what’s wrong with me? I’ve been programming for years, I’ve been programming on this platform for years, I should be able to get this.

Conclusion: Read a Book

I had expected that my knowledge of the Mac, Xcode, and Cocoa would be sufficient to carry me through a four-year gap on picking up a language, particularly with the occasional observation of a conference talk about the Swift language (I’ve even given one!). I had expected that picking up a project to build a thing would give me a chance to get acquainted.

I was reflecting on my early experiences with writing NeXT and Mac applications in Objective-C. I had my copy of the NeXT Developer Documentation, or Cocoa in a Nutshell, open on the desk, looking at the methods available and thinking “I have this, I want that, can I find one of these that gets me there?” I had expected that auto-complete in Xcode would be my modern equivalent of working that way.

Evidently not. Picking up the new standard library things, and the new operators, will require deliberate learning. I’ve got some videos lined up, but I think my next action is to find a good book to read.

More on UIAutomation tests

Update

The information below is mostly redundant. After filing a bug report with Apple, their engineers determined that the Xcode-detected set of macro actions (find a text field, double click, enter text) weren’t working because the double click action wasn’t editing the text field. It is possible to use UIAutomation Tests, you just have to carefully review the UI actions and determine that they have the effect expected, particularly after letting Xcode record UI macros.

Original Post

Unfortunately my work to organise UIAutomation tests has hit the stumbling block that the UI Automation runner doesn’t use the main thread for main-thread-only APIs.

In Xcode 9 and High Sierra, the authors of that post I just linked found that it was possible to turn off the main thread checker in the Test configuration of the build scheme and get working tests anyway. Unfortunately that doesn’t work for me in Xcode 10 and Mojave: the main thread checker isn’t killing the app: the TSM subsystem is just refusing to do its thing. So my tests can’t do straightforward things like write text into a text field. Unfortunately this is a “it’s not me, it’s you” moment, and I don’t think I can carry on using Xcode’s UI tests for my goals.

However, I still want to be able to write “end-to-end” level tests to drive my development. I have (at least) three ways to proceed:

  • I could find a third party library and discover whether it has the main thread problem. Calabash doesn’t support Mac apps, and the other examples I can find (Cucumberish and TABTestKit) both rely on UI Automation so presumably don’t address the main thread problem.
  • I could write the tests in AppleScript. That would be a good way to build up the AppleScript UI for the app, but it doesn’t represent an end-to-end test of the GUI.
  • I could write the tests using NSApplication.sendEvent(_ event:) to simulate clicks, scrolls and text entry, and use the unit test runner to host them. That could work, but I have my doubts (I would guess that the runner is synchronous and stalls the main thread).

I discovered that it is possible to write the test “at the UI level” but in the unit runner, using a combination of key events and AppKit API like sendAction( to:). The trade-offs of this approach:

  • it takes longer, as the abstractions needed to easily find and use AppKit controls don’t (currently) exist
  • it doesn’t use the Accessibility interface so isn’t an accessibility audit at the same time as a correctness test
  • you don’t hit the same problems as with the UI Automation runner
  • it’s much faster

This may be the best approach for now, though I’d welcome other views.

given-when-then in XCTest

I started writing a new Mac app, and I started doing it by driving the implementation through Xcode UI Automation tests. But then it turned out I was driving the test infrastructure as much as the tests, and it’s that I want to talk about.

Given, When, Then

My (complete, Xcode UI Automation) test looks like this:

func testAddingANoteResultsInANoteBeingAdded() {
    given("An empty notebook")
    when("I add a note to the notebook")
    then("There is a note in the notebook")
}

The test case class has an object called a World, which holds, well, the test’s world. There are two parts to this.

The World holds regular expressions associated with blocks, where each block does some part of the test if its associated regular expression matched the description of the test. As an example, my test fixture sets up this association:

try world.then(matchingExpectation: "^There is a note in the notebook$",
                work: { _, world in
                guard let notebook:LabraryNotebook
                  = world.getFromState("TheNotebook") as? LabraryNotebook else {
                    XCTFail("No notebook to test")
                    return
                }
                XCTAssertEqual(notebook.countOfNotes(), 1,
                  "There should be one row in the notes table")
})

We’ll get back to how that block is implemented later. For the moment, I want to make it clear that this is a way to organise a UI test (or, indeed, any other functional test) using XCTest: it is not a new test framework. The test case class still subclasses XCTestCase, and assertions are still made with the XCTAssert* macros/functions. That’s just all wrapped up in this given/when/then structure.

Let’s look at the block’s two parameters: the first is an array of the regular expression’s capture groups so that you can find out information about the test specification, should you want.

The other argument is a reference to the World, which enables the second feature of the World: as state storage so that each part of the test can communicate with later parts. Notice that the when clause in my test says it adds a note to “the notebook”, and the then clause checks that there is a note in “the notebook”. How do they both use the same notebook object? The when clause stores it on the World using world.storeInState(), and the then clause retrieves it with world.getFromState().

Page Objects

Rather than putting XCUIElement goop directly in my test blocks, I use an abstraction called the Page Object pattern, popular among people writing browser tests in Selenium. This puts an adapter between my tests and my UI controls, so the test says (for example) app.newDocument() and the Application page object knows that that means finding the “File” menu, clicking it, then clicking the “New” menu item.

The way to create a new document in a Cocoa app has not changed since 1987 and may not change soon. But the details of my own UI surely will, and will change at a different rate than the goals of the people using it. While someone may want to add a note for the rest of time, there may not always be an “Add Note” button. So my test can continue to say:

when("I add a note to the notebook")

but the page object for a document can change from:

func addANote() {
    let app = XCUIApplication()
    let window = app.windows[documentName]
    let control = window.buttons["Add Note"]
    control.click()
}

to whatever will find and drive the interface in my redesigned application.

Would you like this?

I’m happy to package the given/when/then organisation up and release it under an open source licence so that you can use it in your own apps. As I’ve only just written the code, I’ve yet to do that, but it’s coming! I’m aware that there are multiple ways of getting/using Swift libraries, so if you’re interested please let me know whether you would expect to use an Xcode project that builds a framework, a Swift PM package, a CocoaPod or a Carthage…cart… so I can support you using the software in your way.

Making my peace

Nearly four years ago, in January 2015, I posted On Switching to Linux, in which my computer (in a photo from November 2014) looked like this:

Ubuntu Linux on a MacBook Air

Here’s the same photo from today:

macOS Mojave on a MacBook Pro

So what’s changed? In the intervening four years, I spent some time working with Linux desktop applications made of Qt, and some with browser and server applications made of Javascript. I used a GNU/Linux distribution, Windows 10, a Mac, iOS, and Android. I published two books. I took some time off. I did other things. Here are my relevant conclusions:

  • Free Software is important
  • Making things that are easy, or even pleasant, to use is important
  • Free Software’s Four Freedoms are only academic if usability is a barrier to being capable of using the software for any purpose
  • Apple, and the developers on their platform, are the sub-section of the developer world who care most about giving their people usable and pleasant things
  • Combining these things leads to the conclusion that bringing Free Software principles to the world of Apple makers and adopters is both important and valuable
  • Meanwhile, the people over in the web and server/backend/cloud/serverless land have done a much better job of letting makers iterate quickly and build new things
  • Conversely, the people in the Apple land have done a much better job of making it so that the thing you build works without some complicated stack of transpilers, polyfills and tree-shakers.
  • Thus there are things that the makers in Apple land should learn from the makers in web/server land before the Apple land merely becomes a window on to the web stuff.

I’m back. Watch this space.

HotSwift

A few places have linked to Apple’s use of Swift in iOS, it’s useful to put it in context.

How much of Solaris was made out of Java? Almost none. There was a web browser that you’ve never heard of called HotJava, and that shipped with Solaris, but that’s it. The rest of the OS remained resolutely C with Motif (later GTK+). While Sun wanted us to believe that Java was the developer toolkit of choice, they never chose it themselves.

OOP the Easy Way: now 100% complete

Hello readers, part 3, the final part of the “OOP the Easy Way” journey, has now been published at Leanpub! Thanks for joining me along the way! As ever, corrections, questions, and comments are welcome (you can comment here if you like), and as ever, readers who buy the book now will receive free updates for the lifetime of the book. While there’s nothing new to add, this means that corrections and expansions will be free to all readers.

If you enjoy OOP the Easy Way or found it informative (or maybe even both), please recommend it to your friends, colleagues and followers. It’d be great if they could enjoy it, be informed by it, or both, too!