More on UIAutomation tests

Unfortunately my work to organise UIAutomation tests has hit the stumbling block that the UI Automation runner doesn’t use the main thread for main-thread-only APIs.

In Xcode 9 and High Sierra, the authors of that post I just linked found that it was possible to turn off the main thread checker in the Test configuration of the build scheme and get working tests anyway. Unfortunately that doesn’t work for me in Xcode 10 and Mojave: the main thread checker isn’t killing the app: the TSM subsystem is just refusing to do its thing. So my tests can’t do straightforward things like write text into a text field. Unfortunately this is a “it’s not me, it’s you” moment, and I don’t think I can carry on using Xcode’s UI tests for my goals.

However, I still want to be able to write “end-to-end” level tests to drive my development. I have (at least) three ways to proceed:

  • I could find a third party library and discover whether it has the main thread problem. Calabash doesn’t support Mac apps, and the other examples I can find (Cucumberish and TABTestKit) both rely on UI Automation so presumably don’t address the main thread problem.
  • I could write the tests in AppleScript. That would be a good way to build up the AppleScript UI for the app, but it doesn’t represent an end-to-end test of the GUI.
  • I could write the tests using NSApplication.sendEvent(_ event:) to simulate clicks, scrolls and text entry, and use the unit test runner to host them. That could work, but I have my doubts (I would guess that the runner is synchronous and stalls the main thread).

I discovered that it is possible to write the test “at the UI level” but in the unit runner, using a combination of key events and AppKit API like sendAction( to:). The trade-offs of this approach:

  • it takes longer, as the abstractions needed to easily find and use AppKit controls don’t (currently) exist
  • it doesn’t use the Accessibility interface so isn’t an accessibility audit at the same time as a correctness test
  • you don’t hit the same problems as with the UI Automation runner
  • it’s much faster

This may be the best approach for now, though I’d welcome other views.

given-when-then in XCTest

I started writing a new Mac app, and I started doing it by driving the implementation through Xcode UI Automation tests. But then it turned out I was driving the test infrastructure as much as the tests, and it’s that I want to talk about.

Given, When, Then

My (complete, Xcode UI Automation) test looks like this:

func testAddingANoteResultsInANoteBeingAdded() {
    given("An empty notebook")
    when("I add a note to the notebook")
    then("There is a note in the notebook")
}

The test case class has an object called a World, which holds, well, the test’s world. There are two parts to this.

The World holds regular expressions associated with blocks, where each block does some part of the test if its associated regular expression matched the description of the test. As an example, my test fixture sets up this association:

try world.then(matchingExpectation: "^There is a note in the notebook$",
                work: { _, world in
                guard let notebook:LabraryNotebook
                  = world.getFromState("TheNotebook") as? LabraryNotebook else {
                    XCTFail("No notebook to test")
                    return
                }
                XCTAssertEqual(notebook.countOfNotes(), 1,
                  "There should be one row in the notes table")
})

We’ll get back to how that block is implemented later. For the moment, I want to make it clear that this is a way to organise a UI test (or, indeed, any other functional test) using XCTest: it is not a new test framework. The test case class still subclasses XCTestCase, and assertions are still made with the XCTAssert* macros/functions. That’s just all wrapped up in this given/when/then structure.

Let’s look at the block’s two parameters: the first is an array of the regular expression’s capture groups so that you can find out information about the test specification, should you want.

The other argument is a reference to the World, which enables the second feature of the World: as state storage so that each part of the test can communicate with later parts. Notice that the when clause in my test says it adds a note to “the notebook”, and the then clause checks that there is a note in “the notebook”. How do they both use the same notebook object? The when clause stores it on the World using world.storeInState(), and the then clause retrieves it with world.getFromState().

Page Objects

Rather than putting XCUIElement goop directly in my test blocks, I use an abstraction called the Page Object pattern, popular among people writing browser tests in Selenium. This puts an adapter between my tests and my UI controls, so the test says (for example) app.newDocument() and the Application page object knows that that means finding the “File” menu, clicking it, then clicking the “New” menu item.

The way to create a new document in a Cocoa app has not changed since 1987 and may not change soon. But the details of my own UI surely will, and will change at a different rate than the goals of the people using it. While someone may want to add a note for the rest of time, there may not always be an “Add Note” button. So my test can continue to say:

when("I add a note to the notebook")

but the page object for a document can change from:

func addANote() {
    let app = XCUIApplication()
    let window = app.windows[documentName]
    let control = window.buttons["Add Note"]
    control.click()
}

to whatever will find and drive the interface in my redesigned application.

Would you like this?

I’m happy to package the given/when/then organisation up and release it under an open source licence so that you can use it in your own apps. As I’ve only just written the code, I’ve yet to do that, but it’s coming! I’m aware that there are multiple ways of getting/using Swift libraries, so if you’re interested please let me know whether you would expect to use an Xcode project that builds a framework, a Swift PM package, a CocoaPod or a Carthage…cart… so I can support you using the software in your way.

Making my peace

Nearly four years ago, in January 2015, I posted On Switching to Linux, in which my computer (in a photo from November 2014) looked like this:

Ubuntu Linux on a MacBook Air

Here’s the same photo from today:

macOS Mojave on a MacBook Pro

So what’s changed? In the intervening four years, I spent some time working with Linux desktop applications made of Qt, and some with browser and server applications made of Javascript. I used a GNU/Linux distribution, Windows 10, a Mac, iOS, and Android. I published two books. I took some time off. I did other things. Here are my relevant conclusions:

  • Free Software is important
  • Making things that are easy, or even pleasant, to use is important
  • Free Software’s Four Freedoms are only academic if usability is a barrier to being capable of using the software for any purpose
  • Apple, and the developers on their platform, are the sub-section of the developer world who care most about giving their people usable and pleasant things
  • Combining these things leads to the conclusion that bringing Free Software principles to the world of Apple makers and adopters is both important and valuable
  • Meanwhile, the people over in the web and server/backend/cloud/serverless land have done a much better job of letting makers iterate quickly and build new things
  • Conversely, the people in the Apple land have done a much better job of making it so that the thing you build works without some complicated stack of transpilers, polyfills and tree-shakers.
  • Thus there are things that the makers in Apple land should learn from the makers in web/server land before the Apple land merely becomes a window on to the web stuff.

I’m back. Watch this space.

HotSwift

A few places have linked to Apple’s use of Swift in iOS, it’s useful to put it in context.

How much of Solaris was made out of Java? Almost none. There was a web browser that you’ve never heard of called HotJava, and that shipped with Solaris, but that’s it. The rest of the OS remained resolutely C with Motif (later GTK+). While Sun wanted us to believe that Java was the developer toolkit of choice, they never chose it themselves.

OOP the Easy Way: now 100% complete

Hello readers, part 3, the final part of the “OOP the Easy Way” journey, has now been published at Leanpub! Thanks for joining me along the way! As ever, corrections, questions, and comments are welcome (you can comment here if you like), and as ever, readers who buy the book now will receive free updates for the lifetime of the book. While there’s nothing new to add, this means that corrections and expansions will be free to all readers.

If you enjoy OOP the Easy Way or found it informative (or maybe even both), please recommend it to your friends, colleagues and followers. It’d be great if they could enjoy it, be informed by it, or both, too!

Two Schools

There always seem to be two schools in software, though exactly where the gates are varies. Alan Kay described how Edsger Dijkstra noticed that “the Atlantic has two sides”.

It was basically all about how different the approaches to computing science were in Europe, especially in Holland and in the United States. In the US, here, we were not mathematical enough, and gee, in Holland, if you’re a full professor, you’re actually appointed by the Queen, and there are many other uh important distinctions made between the two cultures. So, uhm, I wrote a rebuttal paper, just called On the fact that most of the software in the world is written on one side of the Atlantic.

Or maybe both schools are on the same continent.

The essence of [the MIT/Stanford approach] can be captured by the phrase the right thing. […] The worse-is-better philosophy is only slightly different […] and I will call the use of this design strategy the New Jersey approach.

Or they could be ways of thinking, school curricula, or whatever.

Maybe both schools have something to teach us. Maybe it’s the same thing.

On Sharecropping

Today I came across the site Danny Reviews, at which fellow internet Danny Yu has posted over 1400 book reviews. I realised that if I had posted book reviews of every book I have read since I became an internet, I would have more than 900 reviews online, maybe over 1000. How do I know? Because my GoodReads profile lists those 900 books.

Now actually GoodReads are quite generous in their terms: I own all of the information I’ve posted there, and I can export all my books, including my reviews such as they are. But that’s entirely up to GoodReads, they decided to be nice and provide an export feature. Other sites take their digital sharecropping more seriously.

I got lucky, but we should all think carefully about what we’re posting to where.

More on freedom and licensing

Last month, I asked whether Freedom Zero is such a great idea, whether it’s OK to limit the freedom to use the software for any purpose if you dislike the purpose to which you believe someone will put it. I gave the example of the Numerical Recipes licence as one that could be adapted to this situation.

In summary, the licence would say “you are free to read this code for the purposes of understanding it, to share the code, and to contribute changes to the code. Should you wish to use the code, contact me, we will go through some customer due diligence, and I will decide whether I want you as a customer”. So far, so unexciting: imagine the usual SaaS startup pricing page with two pricing levels. The “Basic” tier is free, and scales up to 0 uses. The “Custom” tier is $contact_us pricing, and scales up to $contact_us uses.

The Lerna project was briefly licensed under a different model, which took a different approach to solve the same problem. Their approach was “you all get to treat this as a free software project, except the named organisations, who don’t”.

Just to get this out of the way as many of us are techie people who enjoy picking nits: this approach doesn’t appear to solve the problem that the author was setting out to solve. Everybody else except @evil_corps get the software under the terms of the MIT licence, @evil_corps do not get to use the software at all. Now because I got it under the terms of the MIT licence, I am free to do whatever I want except blame the authors if it doesn’t work or remove the MIT licence. This means that I am free to give it or sell it to members of @evil_corps.

Lots of people disagreed with the idea that the Lerna project maintainers, or I, would seek to restrict freedom for moral/ethical reasons, because restricting freedom itself is a moral/ethical choice that is objectionable. Unfortunately, the argument that is often advanced is a bad one.

Richard Stallman’s Why programs must not limit the freedom to run them says numerous things:

  1. The bad actors will probably ignore your license terms anyway, so why bother? This pessimistic view could equivalently be used to argue against any commercial terms (bad actors will pirate your software anyway, so why sell it?) or even RMS’s own General Public License (bad actors will ignore the copyleft requirements anyway, so why bother?). In fact many actors voluntarily comply with the GPL without being asked, many compliance requests are resolved voluntarily on notification, GPL compliance usually starts with a polite, private request not a lawsuit or public shaming. So evidently bothering is useful.

  2. Using copyright to restrict usage is abuse of copyright. But so is copyleft, although as it’s a “good” abuse of copyright that RMS approves of, he’s OK with it.

  3. Limiting freedom is a slippery slope to not being able to build a usable combined system, because you would have to check all the licences to know whether you could do anything. RMS does not have a beard, he has a facial hair, and another facial hair, and another, and so on.

  4. You shouldn’t have the right to do that.

That last one is an interesting one that RMS doesn’t go into in depth, I would imagine because he considers it inalienable. He wrote Freedom Zero, of course he believes in Freedom Zero.

Brad Kuhn, on the other hand, has written about whether I should have that right, and his post is an interesting one. He uses the example of the ACLU’s defence of free speech to argue that having a simple platform and universally supporting a simple principle creates a powerful reputation from which to then build other principled arguments.

I think that argument is compelling, and it has brought me back round to wanting Freedom Zero again.

Book update: OOP the Easy Way

Obejct-Oriented Programming the Easy Way gets ever closer, as the first part (of three) is now substantively complete. If you have been holding off from buying the book, now would be a great opportunity to jump in, as a whole part of the book’s argument is now laid out. As ever, your feedback is welcome, and readers who buy now will get free updates throughout the development of the book.