Learning phases

I’ve been trying to learn things this week (specifically Haskell). So
far I’ve been through a lot of different moods, and I thought it’d be
handy to write them down so that next time I’m the teacher I can
remember what it’s like to be on the receiving end. I’m not sure I
(and certainly not anyone else) goes through all of these every
time, nor necessarily in the same order which I can’t remember anyway.

Frustration

These symbols all have no meaning to me, what is even going on here?

Atomic comprehension

If this symbol means that, and this symbol means that, then
combining these two must mean…

Holistic comprehension

OK, so this whole thing does this, which means that these parts must
somehow contribute to doing this. Let’s see…

Argument by analogy

Oh, hold on, what they’re saying is exactly like this other thing from
a different area of my experience. I’ll just pretend it’s the same for
the moment.

Paralysis of choice

I have a problem, and I’ve been given a bunch of tools, and I don’t
know which to apply.

All I have is a hammer

Well, I understood how to apply a lambda function to a collection with
map, so I’m going to contort this problem until it looks like I need
to apply a lambda function to a collection with map.

Progress by context

I have no idea what I’m supposed to do here, but I just read a section
on list comprehension so I bet I’m supposed to use [x | x <- xs]
somehow.

Confusion of dissimilarity

I have no idea how to turn that map into a comprehension or vice
versa, but I was able to produce each at different times on the same
day.

Joy of simplicity

You know what, it’s true when they say that if it type-checks, it
probably works.

Disillusion through broken dreams

Oh, off-by-one errors (and rotation of dimensions) still
type-check. This is bullshit.

Reinventing the wheel

So, I need a thing that applies this thing to all those things,
let me write a function applyFunction :: (a -> b) -> [a] -> [b] that
gives me one of those.

Discovering existing wheels

I have a thing and I need a different thing, let me look in the API
documentation until I find the function that does exactly that.

Rage-quitting

This error makes no sense, screw the entire city of Glasgow.

Rabbit hole tangents

This error makes no sense, but those words look sufficiently rare to
shove into Google. Ooh, three blog posts, a bug report and a PhD
thesis!

Trial and (type) error

This error makes no sense, but it’s telling me line 25 is wrong so
I’ll just rewrite line 25. Again. And again.

Stop! Hammock time

I have no idea how to solve this problem, but it’s
lunchtime. [Om nom nom] Ooh, this problem is trivial.

Minimum Viable Enlightenment

I don’t know what I got wrong, but based on the error message I think
I make this change…yes, that works. I don’t know what I got right.

Law of highfalutin words (I)

I wish someone could explain this to people who didn’t learn the word
“monoid” at school.

Law of highfalutin words (II)

Yes, I can help you with that problem, I just solved it myself. You
see, it becomes easier when you realise that these two things
constitute a monoid.

On switching to Linux

In November, I switched to GNU/Linux at home (I still use OS X at work, because I still write Objective-C in Xcode at work). Or rather, I switched back: I’d been using it around a decade ago.

The Ubuntu installer on my MacBook air

In December, I resolved to spend more time working with Free Software during 2015 and beyond. Now I find that my story is not unique, others have left OS X (now a dead link, sadly) or are concerned at the state of Apple. I have had conversations with friends about moving from OS X to Debian, to OpenBSD, to other systems.

In my case, there was no watershed moment, no Damascene conversion from the Tablet According to Jobs to the Stallman Doctrine. Rather, my experience has been one of a thousand tiny cuts, and the feeling that while GNU is not actually a better system, it’s one that I’m allowed to make changes to. When iWork was ‘upgraded’ to a less-capable version with an incompatible file format, cut. Every time I plug my Apple display into my Apple computer, then have to unplug it and connect it again before it works properly, cut. Even when the display works, I have to unplug the (Apple) keyboard and plug it back in, cut. Every time iTunes connects to the network but doesn’t let me play any of my tunes, cut. When Apple trumpets the superiority of their new map that shows I live in “Royal Spa”, cut. When iCloud showed its message that I have to upgrade to the new way and devices still on the old way can’t see my files any more, cut. Like the above-linked authors, I’ve got to a point where the only Apple-supplied software in my Dock is the Finder. In my case, it’s happened one app at a time.

The thing is, Apple’s been building some great new things. There’s the watch, Swift, improvements to Cocoa, and they still have the best hardware around. If it weren’t for both the real regressions and the fear of potential regressions on every upgrade or app update, I’d still be confident that their Unix desktop were the best. As it is, I think it could possibly be the least worst, but I’m no longer certain.

Most of the time I don’t care about my computer’s operating environment. This isn’t so much the year of Desktop Linux for me, as the year of Desktop Ambivalence. The OS is the thing that spends most of the time and RAM involved in showing me an emacs window, or a terminal emulator, or a Smalltalk machine, or a web browser. I shouldn’t need to care how it works. But if it’s going to get in my way, give me a chance to do something about it.

The standard trope response is “LOL you must have a bad idea of UX if you use Linux”, which is the sort of thing that was true at the turn of the millennium when most of us were last tyre-kicking some GNU distro but is now an outdated view of the world. But again, should there be a problem, I hope to have the ability to fix it both for myself and others.

So far this year I’ve started small. I’ve been reminding myself of the differences between OS X and BSD commands that I’m used to, and the Linux and SysV commands that I’ve not used frequently in nearly a decade. I’ve been learning about Vala and GTK+.

Fuck. This. Shit.

Enough with the subtle allusions of the previous posts. What’s going on here is not right.

It’s not right that I get to pass as a member of the group of people who can work in technology, while others have to justify their very presence in the field.

It’s not right that “looking like me” is a pass to being considered for the best-paid jobs, while “not looking like me” is not.

[that last one took me a long time to understand. To me, it seems like I worked hard to get where I am, but I needed to understand that I was given the opportunity to work at the things I worked at. That all I needed to do was to work at it, not to work at it and convince everyone else that I was eligible to work at it.]

It’s not right that while I get a free pass into various fields of endeavour, others are told that they either slept their way into the field or are groupies or are unfuckable.

Previously, I avoided writing about this stuff because I thought I might get writing about this stuff wrong. Fuck. This. Shit. I’ve got social capital to burn; it’s much easier for me to get another job around this sector than plenty of people who are as good or better than me at doing the work. I might be worried about treading the line between being supportive and getting into trouble, but that’s not as bad as the line women, trans people, non-white people, non-straight people, disabled people have to tread between asking to be considered equally and keeping their jobs have to tread. I have one job: doing my job. I do not have two jobs: doing my job, and convincing people that someone like me should be allowed to do my job. If the cost of equality is giving up my free ride, then I give up my free ride.

The pipeline is not the problem, it leads to a vat of acid. No-one wants to lean in to a vat of acid. (Thanks to Cate Huston for that metaphor, it made a lot of sense to me.)

Our industry is exclusive, and needs to be inclusive. What should you do about this? I don’t know, I’m far from knowledgable. If your position is “I agree with the straight white guy that the world is broken, I should ask the straight white guy how to fix it” then perhaps you are the problem, just as I have been and am the problem.

What should I do about this? First step for me is to listen. To not tell people who are describing their experiences what my experiences are. To avoid thinking about my reply to people, and to think about what they’ve said. To stop looking for holes in arguments and to listen for opportunities to grow. Not just to grow me, but to grow others.

Sitting on the Sidelines

Thank you, James Hague, for your article You Can’t Sit on the Sidelines and Become a Philosopher. I got a lot out of reading it, because I identified myself in it. Specifically in this paragraph:

There’s another option, too: you could give up. You can stop making things and become a commentator, letting everyone know how messed-up software development is. You can become a philosopher and talk about abstract, big picture views of perfection without ever shipping a product based on those ideals. You can become an advocate for the good and a harsh critic of the bad. But though you might think you’re providing a beacon of sanity and hope, you’re slowly losing touch with concrete thought processes and skills you need to be a developer.

I recognise in myself a lot of the above, writing long, rambling, tedious histories; describing how others are doing it wrong; and identifying inconsistencies without attempting to resolve them. Here’s what I said in that last post:

I feel like I ought to do something about some of that. I haven’t, and perhaps that makes me the guy who comes up to a bunch of developers, says “I’ve got a great idea” and expects them to make it.

Yup, I’m definitely the person James was talking about. But he gave me a way out, and some other statements that I can hope to identify with:

You have to ignore some things, because while they’re driving you mad, not everyone sees them that way; you’ve built up a sensitivity. […] You can fix things, especially specific problems you have a solid understanding of, and probably not the world of technology as a whole.

The difficulty is one of choice paralysis. Yes, all of those broken things are (perhaps literally) driving me mad, but there’s always the knowledge that trying to fix any one of them means ignoring all of the others. Like the out of control trolley, it’s easier to do nothing and pretend I’m not part of the problem than to deliberately engage with choosing some apparently small part of it. It’s easier to read Alan Kay, to watch Bret Victor and Doug Engelbart and to imagine some utopia where they had greater influence. A sort of programmerpunk fictional universe.

As long as you eventually get going again you’ll be fine.

Hopefully.

Intra-curricular activities

I’m apparently fascinated by the idea of defining curricula for learning programming. I’ve written about how we need to be careful what we try to pay forward from the way we learned in the past, and I’ve talked about how we do need to pay it forward so that the second hundred years see faster progress than the first hundred years.

I’m a fan (with reservations, as seen below) of the book series as a form of curriculum. Take something like Kent Beck’s signature series, which covers a decent subset of both technical and social approaches in software development in breadth and in depth. You could probably imagine developers who would benefit from reading some or all of the books in the series. In fact, you may be one.

Coping with people approaching the curriculum from different skill levels and areas of experience is hard. Not just for the book series, it’s hard in general. Universities take the simplifying approach of assuming that everybody wants to learn the same stuff, and teaching that stuff. And to some extent that’s easy for them, because the backgrounds of prospective students is relatively uniform. Even so, my University course organised incoming students into two groups; those who had studied complex numbers at A-level and those who had not. The difference was simply that the group who had not were given a couple of lectures on complex numbers, then it was assumed that they also knew the topic from the fourth week.

Now consider selling a programming book to the public. Part of the proposal process with all of the publishers I’ve worked with has been describing the target audience. Is this a book for people who have never programmed before? For people who have programmed a little, but never used this particular tool or technique? People who have programmed a lot but never used this tool? Is this thing similar to what they have used before, or very different? For people who are somewhat familiar with the tool? For experts (and how is that defined)? Is it for readers comfortable with maths? For readers with no maths background?

Every “no” in answer to one of those questions is an opportunity to improve the experience for a subset of the potential audience by tailoring it to that subset. It’s also an opportunity to exclude a subset of the audience by making the content less relevant to them.

[I’ll digress here to explain how I worked that out for my books: whether it’s selfishness or a failure of empathy, I wrote books that I wanted to read but that didn’t exist. Therefore the expected experience is something similar to mine, back when I filled in the proposal form.]

Clearly no single publication will cover the whole phase space of potential readers and be any good. The interesting question is how much it’s worth covering with multiple publications; whether the idea of series-as-curriculum pulls in the general direction as much as scope-limiting each book pulls in the specific. Should the curriculum take readers on a straight line from novice to master? Should it “fan in” from multiple introductions? Should it “fan out” in multiple directions of interest and enquiry? Would a non-linear curriculum be inclusive or offputtingly confusing? Should the questions really be answered by substituting the different question “how many people would buy that”?

Things I believe

The task of producing software is one of choosing and creating constraints, rules and abstractions inside a system which provides very few a priori. Typically we select a large collection of pre-existing constraints, rules and abstractions upon which to base our own: models of computation, programming and deployment environments, everything from the size of the register file to the way in which text is represented on the display is theoretically up for grabs, but we impose limitations in their freedom upon ourselves when we create a new product.

None of these limitations is essential. Many are conventional, and have become so embedded in the cultural practice of making software that it would be expensive or impractical to choose alternative options. Still others have so much rhetoric surrounding them that the emotional cost of change is too great to bear.

So what are these restrictions? Here’s a list of mine. I accept that they don’t all apply to you. I accept that many of them have alternatives. Indeed I believe that all of them have alternatives, and that enumerating them is the first thing that lets me treat them as assumptions to be challenged.

  1. Computers use the same memory for programs and data. I know the alternatives exists but wouldn’t know how to start using them.
  2. Memory is a big blob of uniform storage. Like above, except I know this one isn’t true but that I just ignore that detail.
  3. Memory and bus wires can be in one of two states.
  4. There probably is a free-form hierarchical database available.
  5. There is a thing called a stack and a thing called a heap, and the difference between the two is important.
  6. There is no point trying to do a better job at multiprocessing than the operating system.
  7. There is an operating system.
  8. The operating system, file system, indeed any first system on which my thing is a second system; those first systems are basically interchangeable.
  9. I can buy a faster thing (except in mobile, where I can’t).
  10. Whatever processor you gave me behaves correctly.
  11. Whatever compiler you gave me behaves correctly.
  12. Whatever library you gave me probably behaves correctly.
  13. Text is a poor way to represent a computer program but is the best we have.
  14. The way to write a computer program is to tell the computer what to do.
  15. The goal of the industry for last few decades has been the DynaBook.
  16. I still do not need a degree in computer science.
  17. I should know what my software does before I give it to the people who need to use it.
  18. The universal runtime environment is the C system.
  19. Processors today are basically like faster versions of the MC68000.
  20. Platform vendors no longer see lock-in as a goal, but do see it as a convenient side-effect.
  21. You will look after drawing pictures, playing videos, and making sounds for me.
  22. Types are optional.

Preparing for Computing’s Big One-Oh-Oh

However you slice the pie, we’re between two and three decades away from the centenary celebration for applied computing (which is of course significantly after theoretical or hypothetical advances made by the likes of Lovelace, Turing and others). You might count the anniversary of Colossus in 2043, the ENIAC in 2046, or maybe something earlier (and arguably not actually applied) like the Z3 or ABC (both 2041). Whichever one you pick, it’s not far off.

That means that the time to start organising the handover from the first century’s programmers to the second is now, or perhaps a little earlier. You can see the period from the 1940s to around 1980 as a time of discovery, when people invented new ways of building and applying computers because they could, and because there were no old ways yet. The next three and a half decades—a period longer than my life—has been a period of rediscovery, in which a small number of practices have become entrenched and people occasionally find existing, but forgotten, tools and techniques to add to their arsenal, and incrementally advance the entrenched ones.

My suggestion is that the next few decades be a period of uncovery, in which we purposefully seek out those things that have been tried, and tell the stories of how they are:

  • successful because they work;
  • successful because they are well-marketed;
  • successful because they were already deployed before the problems were understood;
  • abandoned because they don’t work;
  • abandoned because they are hard;
  • abandoned because they are misunderstood;
  • abandoned because something else failed while we were trying them.

I imagine a multi-volume book✽, one that is to the art of computer programming as The Art Of Computer Programming is to the mechanics of executing algorithms on a machine. Such a book✽ would be mostly a guide, partly a history, with some, all or more of the following properties:

  • not tied to any platform, technology or other fleeting artefact, though with examples where appropriate (perhaps in a platform invented for the purpose, as MIX, Smalltalk, BBC BASIC and Oberon all were)
  • informed both by academic inquiry and practical experience
  • more accessible than the Software Engineering Body of Knowledge
  • as accepting of multiple dissenting views as Ward’s Wiki
  • at least as honest about our failures as The Mythical Man-Month
  • at least as proud of our successes as The Clean Coder
  • more popular than The Celestial Homecare Omnibus

As TAOCP is a survey of algorithms, so this book✽ would be a survey of techniques, practices and modes of thought. As this century’s programmer can go to TAOCP to compare algorithms and data structures for solving small-scale problems then use selected algorithms and data structures in their own work, so next century’s applier of computing could go to this book✽ to compare techniques and ways of reasoning about problems in computing then use selected techniques and reasons in their own work. Few people would read such a thing from cover to cover. But many would have it to hand, and would be able to get on with the work of invention without having to rewrite all of Doug Engelbart’s work before they could get to the new stuff.

It's dangerous to go alone! Take this.

✽: don’t get hung up on the idea that a book is a collection of quires of some pigmented flat organic matter bound into a codex, though.

On too much and too little

In the following text, remember that words like me or I are to be construed in the broadest possible terms.

It’s easy to be comfortable with my current level of knowledge. Or perhaps it’s not the value, but the derivative of the value: the amount of investment I’m putting into learning a thing. Anyway, it’s easy to tell stories about why the way I’m doing it is the right, or at least a good, way to do it.

Take, for example, object-oriented design. We have words to describe insufficient object-oriented design. Spaghetti Code, or a Big Ball of Mud. Obviously these are things that I never succumb to, but other people do. So clearly (actually, not clearly at all, but that’s beside the point) there is some threshold level of design or analysis practice that represents an acceptable minimum. Whatever that value is, it’s less than the amount that I do.

Interestingly there are also words to describe the over-application of object-oriented design. Architecture Astronauts, for example, are clearly people who do too much architecture (in the same way that NASA astronauts got carried away with flying and overdid it, I suppose). It’s so cold up in space that you’ll catch a fever, resulting in Death by UML Fever. Clearly I am only ever responsible for tropospheric architecture, thus we conclude that there is some acceptable maximum threshold for analysis and design too.

The really convenient thing is that my current work lies between these two limits. In fact, I’m comfortable in saying that it always has.

But wait. I also know that I’m supposed to hate the code that I wrote six months ago, probably because I wasn’t doing enough of whatever it is that I’m doing enough of now. But I don’t remember thinking six months ago that I was below the threshold for doing acceptable amounts of the stuff that I’m supposed to be doing. Could it be, perhaps, that the goalposts have conveniently moved in that time?

Of course they have. What’s acceptable to me now may not be in the future, either because I’ve learned to do more of it or because I’ve learned that I was overdoing it. The trick is not so much in recognising that, but in recognising that others who are doing more or less than me are not wrong, they could in fact be me at a different point on my timeline but with the benefit that they exist now so I can share my experiences with them and work things out together. Or they could be someone with a completely different set of experiences, which is even more exciting as I’ll have more stories to swap.

When it comes to techniques and devices for writing software, I tend to prefer overdoing things and then finding out which bits I don’t really need after all, rather than under-application. That’s obviously a much larger cognitive and conceptual burden, but it stems from the fact that I don’t think we really have any clear ideas on what works and what doesn’t. Not much in making software is ever shown to be wrong, but plenty of it is shown to be out of fashion.

Let me conclude by telling my own story of object-oriented design. It took me ages to learn object-oriented thinking. I learned the technology alright, and could make tools that used the Objective-C language and Foundation and AppKit, but didn’t really work out how to split my stuff up into objects. Not just for a while, but for years. A little while after that Death by UML Fever article was written, my employer sent me to Sun to attend their Object-Oriented Analysis and Design Using UML course.

That course in itself was a huge turning point. But just as beneficial was the few months afterward in which I would architecturamalise all the things, and my then-manager wisely left me to it. The office furniture was all covered with whiteboard material, and there soon wasn’t a bookshelf or cupboard in my area of the office that wasn’t covered with sequence diagrams, package diagrams, class diagrams, or whatever other diagrams. I probably would’ve covered the external walls, too, if it wasn’t for Enterprise Architect. You probably have opinions(TM) of both of the words in that product’s name. In fact I also used OmniGraffle, and dia (my laptop at the time was an iBook G4 running some flavour of Linux).

That period of UMLphoria gave me the first few hundred hours of deliberate practice. It let me see things that had been useful, and that had either helped me understand the problem or communicate about it with my peers. It also let me see the things that hadn’t been useful, that I’d constructed but then had no further purpose for. It let me not only dial back, but work out which things to dial back on.

I can’t imagine being able to replace that experience with reading web articles and Stack Overflow questions. Sure, there are plenty of opinions on things like OOA/D and UML on the web. Some of those opinions are even by people who have tried it. But going through that volume of material and sifting the experience-led advice from the iconoclasm or marketing fluff, deciding which viewpoints were relevant to my position: that’s all really hard. Harder, perhaps, than diving in and working slowly for a few months while I over-practice a skill.

Some so-called expert

There’s a comedy sketch being frequently tweeted called The Expert. Now, all programmers will be aware that there is nothing funnier than interpreting a joke literally and telling everyone the many ways in which it’s wrong, and that there is no way to be seen as a more intelligent and empathetic person than to do this. So here we go: what are all the inexpert things this “expert” does?

Firstly, having been told how important the strategic initiative is, he makes no attempt to actually find out what it is, and how his task is connected to the objectives described. This means that he doesn’t know anything about the context of his work, which is just setting himself up for all sorts of trouble. It’s like a programmer going “yeah sure, I can add a second copy of that goto line” without checking whether they’re working on some sort of security-sensitive module.

He refuses to accept any form of creative solution to the problem, and his project manager is correct to try to tactfully defer his immediate refusal to do the work asked. Immediately saying “no, I can’t do that” is identical to saying “I have never done that, and I cannot imagine any novelty entering my life”. This is not symptomatic of expertise, but of narrow-mindedness.

A pause, and a gathering of resources, leads us to conclude that some of the tasks set are eminently achievable, making this alleged expert look like the comfort-zone-hogging risk-averse luddite that perhaps he is. Of course you can draw a red line with inks of other colours, for example. You simply rely on the relativistic Doppler effect, or on fluorescent properties of the materials. Of course you can draw seven lines all perpendicular, if your diagram can extend into seven dimensions. And that is of course assuming a Euclidean geometry for the diagram; an assumption that our “I know best” expert doesn’t even think to question. Alternatively, you can find out what the time-dependent evolution of the diagram is, as it may be that a total of seven lines that are each instantaneously perpendicular to the other lines present but that do not all simultaneously exist is a sufficient solution. Again, our unimaginative expert doesn’t think about that. In fact, he never really explores whether the perpendicularity requirement means mutually perpendicular, he just proceeds to mansplain to the client representative why he is right and she is wrong.

Assured of his expertise, he then injects sarcasm into his voice in a condescending fashion. “I’m sure your target audience doesn’t exist solely of those people.” Again, this is indicative of a lack of empathy and an unwillingness to consider other viewpoints than his own.

Although, having said that, he’s pretty quick to demur to authority, and on the few occasions that he does want to enquire about the requirements, does not pursue the matter if someone else interrupts.

This is an “expert” who is going to go away with an incomplete understanding of the problem, and will likely fail to give a satisfactory solution. Often such people will then seek to externalise any responsibility for the failure, complaining that the requirements weren’t clear or that the clients had unrealistic expectations. Maybe they weren’t and they did, but as an expert it’s his responsibility to understand those and apply his skills to solving the problem at hand, not to find ways to throw other people under the proverbial bus.

The manager in this video is clearly the sanest voice, and also manages to keep his frustration at his own mistake somewhat bottled. The extent of that mistake? He has contracted an “expert in a narrow field”, who “doesn’t see the overall picture”, and put him in a meeting with their client for which he was totally unprepared. So it’s a shame that the expert’s grandest commitment—to inflate a balloon of unknown quality and structure into the shape of a kitten—is made without the manager around to intermediate. He might have been able to intervene before the physical contact between the “expert” and the designer, which should be considered wholly inappropriate for a business meeting.

Maybe it was a mistake to put someone so junior in front of the client without some coaching. Hopefully, with appropriate mentoring and support, our “expert” can grow to be a mature, empathetic and positive contributor to his team.

Where am I going with this?

I recently asked how people would describe this Secure Mac Programming blog were they trying to tell someone else they should read it. Of all the answers, the one that most succinctly sums up the trouble with the old name is from Alan:

@secboffin Not Just Secure, Not Just Mac, Not Just Programming.

I’m probably in the midst of some existential crisis, having spent a couple of years thinking and writing about philosophy, ethics, and the social responsibility of my work and its context. It’s clear that I’m dealing with some conflict, and it doesn’t look like reconciliation is an option.

Often I write about ideas that are still knocking around my head, such that I never come to any conclusion. I’ve used multiple choice conclusions, conclusions that appear to be from a different argument, and have concluded that my entire argument may or may not be useful.

This is just something I need to work out: what do I think I do, what do other people think I do, what parts of that do I like and dislike, are there other things I would like, can I replace the disliked parts with the liked parts, and so on. I write it here as you may have related ideas, or you may be thinking about the same things yourself and benefit from knowing that other people are, too.

What I know includes a list of things that currently interest me:

With all that in mind, I’m happy to introduce the beginning of a slow rebranding of this blog. It is now called the Structure and Interpretation of Computer Programmers, and can be found at http://www.sicpers.info/ in addition to its previous home at http://blog.securemacprogramming.com.

I do not intend to remove the old domain or break existing feed subscriptions. Over time (basically, as I work out how to do it) I’ll migrate links, feed entries and so on to reference the new domain, and the age-old updated mission of the blog.