What Lenin taught me about software movements

In What is to be done?: Burning Questions of our Movement, Lenin lists four roles who contribute to fomenting revolution – the theoreticians, the propagandists, the agitators, and the organisers:

The theoreticians write research works on tariff policy, with the “call”, say, to struggle for commercial treaties and for Free Trade. The propagandist does the same thing in the periodical press, and the agitator in public speeches. At the present time [1901], the “concrete action” of the masses takes the form of signing petitions to the Reichstag against raising the corn duties. The call for this action comes indirectly from the theoreticians, the propagandists, and the agitators, and, directly, from the workers who take the petition lists to the factories and to private homes for the gathering of signatures.

Then later:

We said that a Social Democrat, if he really believes it necessary to develop comprehensively the political consciousness of the proletariat, must “go among all classes of the population”. This gives rise to the questions: how is this to be done? have we enough forces to do this? is there a basis for such work among all the other classes? will this not mean a retreat, or lead to a retreat, from the class point of view? Let us deal with these questions.

We must “go among all classes of the population” as theoreticians, as propagandists, as agitators, and as organisers.

Side note for Humpty-Dumpties: In this post I’m going to use “propaganda” in its current dictionary meaning as a collection of messages intended to influence opinions or behaviour. I do not mean the pejorative interpretation, somebody else’s propaganda that I disagree with. Some of the messages and calls below I agree with, others I do not.

Given this tool for understanding a movement, we can see it at work in the software industry. We can see, for example, that the Free Software Foundation has a core of theoreticians, a Campaigns Team that builds propaganda for distribution, and an annual conference at which agitators talk, and organisers network. In this example, we discover that a single person can take on multiple roles: that RMS is a theoretician, a some-time propagandist, and an agitator. But we also find the movement big enough to support a person taking a single role: the FSF staff roster lists people who are purely propagandists or purely theoreticians.

A corporate marketing machine is not too dissimilar from a social movement: the theory behind, say, Microsoft’s engine is that Microsoft products will be advantageous for you to use. The “call” is that you should buy into their platform. The propaganda is the MSDN, their ads, their blogs, case studies and white papers and so on. The agitators are developer relations, executives, external MVPs and partners who go on the conference, executive briefing days, tech tours and so on. The organisers are the account managers, the CTOs who convince their teams into making the switch, the developers who make proofs-of-concept to get their peers to adopt the technology, and so on. Substitute “Microsoft” for any other successful technology company and the same holds there.

We can also look to (real or perceived) dysfunction in a movement and see whether our model helps us to see what is wrong. A keen interest of mine is in identifying software movements where “as practised” differs from “as described”. We can now see that this means the action being taken (and led by the organisers) is disconnected from the actions laid out by the theorists.

I have already written that the case with OOP is that the theory changed; “thinking about your software in this way will help you model larger systems and understand your solutions” was turned by the object technologists into “buying our object technology is an easy way to achieve buzzword compliance”. We can see similar things happening now, with “machine learning” and “serverless” being hollowed out to fill with product.

On the other hand, while OOP and machine learning have mutated theories, the Agile movement seems to suffer from a theory gap. Everybody wants to be Agile or to do Agile, all of the change agents and consultants want to tell us to be Agile or to do Agile, but why does this now mean Dark Scrum? A clue from Ron Jeffries’ post:

But there is a connection between the 17 old men who had a meeting in Snowbird, and the poor devils working in the code mines of insurance companies in Ohio, suffering under the heel of the boot of the draconian sons of expletives who imposed a bastardized version of something called Scrum on them. We started this thing and we should at least feel sad that it has sometimes gone so far off the rails. And we should do what we can to keep it from going more off the rails, and to help some people get back on the rails.

Imagine if Karl Marx had written Capital: Critique of Political Economy, then waited eighty years, then said “oh hi, that thing Josef Stalin is doing with the gulags and the exterminations and silencing the opposition, that’s not what I had in mind, and I feel sad”. Well Agile has not gone so far off the rails as that, and has only had twenty years to do it, but the analogy is in the theory being “baked” at some moment, and the world continuing to change. Who are the current theorists advancing Agile “as practised” (or at least the version “as described” that a movement is taking out to change the practice)? Where are the theoreticians who are themselves Embracing Change? It seems to me that we had the formation of the theory in XP, the crystallisation (pardon the pun) of the theory and the call to action in the Agile manifesto, then the project management bit got firmed up in the Declaration of Interdependence, and now Agile is going round in circles with its tiller still set on the Project Management setting.

Well, one post-Agile more-Agile-than-thou movement for the avocado on toast generation is the Software Craft[person]ship movement, which definitely has theory and a call to action (Software Craftsmanship: the New Imperative, which is only a scratch newer than the Agile Manifesto), definitely has vocal propagandists and agitators, and yet still doesn’t seem to be sweeping the industry. Maybe it is, and I just don’t see it. Maybe there’s no clear role for organisers. Maybe the call to action isn’t one that people care about. Maybe the propaganda is not very engaging.

Anyway, Lenin gave me an interesting model.

Why your app is not massively parallel software

That trash can Mac Pro that hasn’t been updated in years? It’s too hard to write software for.

Now, let’s be clear, there are any number of abstractions that have been created to help programmers parallelise their thing, from the process onward. If you’ve got a loop and can add the words #pragma omp parallel for to your code, then your loop can be run in parallel over as many threads as you like. It’s not hard.

Making sure that the loop body can run concurrently with itself is hard, but there are some rules to follow that either make it easy or tell you when to avoid trying. But you’re still only using the CPU, and there’s that whole dedicated GPU to look after as well.

Even with interfaces like OpenCL, it’s difficult to get this business right. If you’ve been thinking about your problem as objects, then each object has its own little part of the data – but now you need to get that information into a layout that’ll be efficient for doing the GPU work, then actually do the copy, then copy the results back from the GPU memory…is doing all of that worth it?

For almost all applications, the answer is no. For almost no applications, the answer is occasionally. For a tiny number of applications, the answer is most of the time, but if you’re writing one of those then you’re a scientist or a data “scientist” and probably not going to get much value out of a deskside workstation anyway.

What’s needed for that middle tier of applications is the tools – by which I mostly mean the libraries – to deal with this problem when it makes sense. You don’t need visualisations that say “hey, if you learned a different programming language and technique and then applied it to this little inner loop you could get a little speed boost for the couple of seconds that one percent of users will use this feature every week” – you need implementations that notice that and get on with it anyway.

The Mac Pro is, in that sense, the exact opposite of the Macintosh. Back in the 1980s, the Smalltalk software was ready well before there was any hardware that could run it well, and the Macintosh was a thing that took this environment that could be seen to have value, and made it kindof work on real hardware. Conversely, the Mac Pro was ready well before there was any software that could make use of it, and that’s a harder sell. The fact that, four years later, this is still true, makes it evident that it’s either difficult or not worth the effort to try to push the kind of tools and techniques necessary to efficiently use Mac Pro-style hardware into “the developer ecosystem”. Yes, there are niches that make very good use of them, but everybody else doesn’t and probably can’t.

Resolution: Subscribe Self

I have, at least temporarily, stopped using the social media to find news. I publish an RSS feed here, and your other favourite sites probably do too, so we can all discover the things we want to read without having to wade through a morass of things we don’t.

Soon, I’ll collect some recommended feeds together. What are yours? Which are the high-quality, low-volume feeds you make sure to catch every post on?

By the river

My home stands near to a bridge over the Avon, the same river that lends its name to Stratford-on-Avon. By walking to the end of my street then through the churchyard, I could stand over the river and watch it flow beneath me. But I didn’t. I elected to walk a few kilometres upstream to the grounds of the big house, and stand on that bridge instead. Walking is something I do, apparently. My phone tells me that I walk more than the average man, an achievement I can claim as my own.

It was raining, so I took an umbrella. In theory, I was listening to the patter of the rain on the umbrella, and watching the water flow through the reeds as it washed underneath me, then past my house and the church, past Shakespeare’s birthplace and onto Tewkesbury and the Severn. In fact I did nothing of the sort. The river and its reeds must have been there, and I was pointing in their direction. I was vaguely aware at one point in my half-hour stop of a car on the driveway, approaching me, stopping just short of where I was, then carrying on over the bridge and on to the road. I did not acknowledge the driver, and if that person acknowledged me I failed to notice it.

Of things that I saw, the one that sticks in my mind is the notification light on my phone. It is my beacon to the outside world, letting me know when somebody wants my attention. Twice I remember looking at it, and twice I remember seeing the blue blink that signifies contact via email. Twice I read this link to society at large, to discover that somebody wanted to sell me the world’s most powerful flashlight and that somebody else thought I would be interested in a walk-in bathtub. Isn’t technology a marvel?

Of things that I heard, probably the car had some sort of engine and tyres that rumbled as it made its way past, and presumably the dual carriageway a few hundred metres away was contributing some white noise. The only sound that I remember hearing is my internal monologue. It’s a toxic sound, reminding me as it does that I have no reason to be pleased about anything, but it is my constant companion. I struggle to hold back tears as it reminds me that my home life, my social life, and my work life, are all in flux at the moment, all out of my control, and the common thread running through them all is the person who lost that control. It goes on to tell me that anything else I might try is doomed to failure, too.

I consider using that phone again, not to look at the notification light, but to tell somebody how I’m feeling. Why would you want to do that, asks the voice? These people have their own things to deal with, why burden them with your problems too? Why spoil somebody’s evening by telling them that you’re upset, when they’ve done nothing to upset you? Why are you so selfish? You’re a grown-up, a man who walks more than the average, you’re supposed to look after yourself now. You’re not supposed to be a mess with a state-sponsored Citalopram habit.

I turn the phone around, thinking of taking a picture of the trees silhouetting the clouds, stained Sodium-orange by the streetlights of the nearest town. The screen remains resolutely black. Of course it did, you idiot, it’s much too dark to take a photo with that phone. You’re supposed to be some kind of expert at smartphones, don’t you know how their cameras work? I vaguely consider whether to fetch my SLR camera, but of course that’s been in a box for over a year and it’s doubtful the battery is even charged. Maybe it’s time to sell it on eBay, but of course you won’t get a good price for it and then it’ll get damaged in the post and the buyer will want a refund and you won’t even have kept that money. Why bother?

It took effort to walk the twenty minutes upstream to the other bridge. My voice told me that I would be just as useful to the world if I didn’t take the walk, if I had stayed in bed and waited until it was time to go to work tomorrow. Well, voice, I took everything you told me and made a few hundred words of blog post out of it, so I have done something after all. The joke’s on you.

Turn it off and back on again

I’m now six months into what I expected to be about a year out of working in technology, and I’m starting to think about what comes next and trying to make it happen. The difficulty I have is that it’s hard to explain what I’m looking for in a way that makes sense to those that are hiring, or that I can summarise in a search term for job sites. I’m considering running a company again to do all this myself, but that doesn’t obviate the problem, I still need to be able to describe this to potential clients and explain why they would want to buy one.

The difficulty comes from being a people person. I listen to people, I talk to people, I get people to talk to other people, I learn from people, I teach people, I perform for people, I watch people, I read what people have to say, and I write for people. And I happen to want to do that for money in the software industry, but if you tell that to a hiring manager on a software team you’ll get a blank stare followed by “um, but how much have you used MongoDB from Scala in your last job?”. You don’t need to try this yourself, I have done it. This is what happens.

I don’t mind much what technology I use, as long as we’re using it because it helps to address the problems we or our customers have rather than because a developer threw a strop if they weren’t going to be allowed to rewrite everything that already works well in NodeCaml. I care about understanding and solving the problems people have, and about understanding the people who have those problems. “I think we should use this” is not fine. “I think we should use this because” is perfectly fine. “We’re a $VENDOR shop” is probably not fine.

So the problem I have is that the job I know how to apply for and get is “programmer” (these days with some highfalutin prefix that really comes down to “better paid”), but that usually comes with some expectation to focus on the programming, and leave all the gloopy soft stuff like what programming should be done and whether it’s a good idea to do it now to other people. What I want to be doing is (being paid for) the gloopy soft stuff like making programmers into better programmers, working out what programming should be done (if any) and whether it’s a good idea to do it now, helping programmers to understand the people they’re helping, and helping the people being helped by programmers to understand the programmers, with the programming itself being a context not a focus. I have no idea how to explain that succinctly to people who might want to hire one of those, nor how to find people who might want to hire one of those.

Practically, based on what I’ve experienced about my own health and its relationship with my work, I also need to be realistic about where and when I work. That’s from or within cycling distance of home (around Leamington Spa, Warwick, Kenilworth and Coventry), from usual working hours my own timezone. If your company is in a different timezone and supports remote work, that’s great, but if you need me to work from your timezone then it’s not great. In fact, you don’t support remote work, you just support local employees who don’t always come into the office.

If you are someone who wants one of those, know someone who wants one of those, or know how to describe one of those succinctly, please do help me out. Based on the last time I tried this, here’s a couple of lists:

Things I’ve never done, but would

These aren’t necessarily things my next job must have, and aren’t all even work-related, but are things that I would take the opportunity to do.

  • Work in a field on a farm. Preferably in control of a tractor.
  • Have a job title that begins with the letters ‘C’ or ‘D’ (I managed ‘Q’ a while back).
  • Spend lots of time supporting the Free Software Definition
  • Include going to lunch with each other employee in the company in my responsibilities.
  • Visit Iceland.

Things I don’t like

These are the things I would try to avoid.

  • I still seriously hate raw celery.
  • Client work, in those cases where we don’t all really think that the client is doing something important.
  • “Rock star” programmers, and people who hire them.

Clown Trousers

An indirect side effect of stopping programming is that none of my trousers fit any more.

People who like to explain things before they have all the facts (or “programmers” as we sometimes call them) will justify this observation by pointing out that I have more time for exercise now. I do, but I don’t use it. While working at Facebook I walked six miles each day as part of my commute and worked at a variable-height desk; I spent a lot of time walking and standing.

When I began my gap year, I put some effort into running every day. That didn’t last long. I still stand a lot to play musical instruments, but am significantly less active now that I’m 8kg lighter than programmer Graham.

Looking at videos of programmer me, I just see an obese, tired guy surviving on caffeine, sugar snacks and three big meals a day as he lurched between commuting trips, flights abroad, conference talks and infrequent visits to bed. Peak Graham (weight for weight) came in June, as attested by the video of my AltConf talk, I have no idea what I’m doing.

It turns out I had no idea what I was doing to myself either. But now that I’m not doing it, the historical record that is my wardrobe tells me I’m healthier than I have been in over five years.

You may not need hipster silicon valley nutritional engineering sludge. You may not need an extra hour in the day to fit in a run and a shower. You may not need to drop a few hundred quid on a watch that also reminds you to stand up. You might just need to discover what you’re doing wrong, and not do it like that.

PL personality theory

An analysis of programmer personality traits inferred from their answer to the question “which is your favourite programming language?”

Algol About to re-enact that scene in Jumanji where Robin Williams has a huge beard.

Basic Remembers a time when you could code a whole platform game with twenty levels in 6k of RAM. Probably works on some trading platform that needs the JVM heap size bumped to 4GB to add two numbers.

C Learnt programming once, what more could there be to it?

C++ Learnt programming once, it was horrible.

C# Wears Microsoft shoes, Microsoft trousers, and a Microsoft t-shirt. C# also goes by the name “Washington State Swift”.

D Probably best to ask again next week.

Elixir I used to be a Ruby programmer until I realised I hate Ruby programmers.

F# Wears the same clothes as the C# programmer but does so ironically.

Go Uses Google+ earnestly.

Java Has hobbies that aren’t programming.

Javascript Look, even Yersinia Pestis was popular once.

Lisp Mostly calm with sudden outbursts of zen.

Objective-C At the intersection of technology and liberal cash. In danger of progressing to Smalltalk.

Perl Stoic in the face of abuse. Ignores it and carries on getting loads of work done.

Python Had an argument with a Perl programmer in 2004. They each think they won; neither is correct.

Ruby Used to use Java but then learned object-oriented programming and had to move on.

Ruby on Rails Like that kid in that movie. No, not War Games, the other one. Home Alone.

Scheme Pretentious. Probably has a blog named for a pun on a classic computing textbook.

Self Slightly further along in their hatred of computing than a Smalltalker.

Smalltalk About to re-enact that scene in Planet of the Apes where Charlton Heston finds the statue.

Swift Like the person who goes into the specialist metal record store and conspiratorially asks whether they’ve got anything by Metallica.

Tcl Submits write-in answers to multiple choice questions.

GNU Terry Pratchett

(post-hoc prescript: I admit to being in two minds about sharing this post. Name-dropping can be the ultimate in reflected vanity: I have worth because I knew this worthy person. I title it about them, but we both know it’s about me. I hope this post, containing much as it does about me and my experiences, is actually about what I learned from an Old Fart in a Hat – among others.)

GNU Terry Pratchett

Nearly three years ago, The Compleat Ankh-Morpork was approaching completion, and the publishers were excited about the potential to turn it into an interactive discovery of the Circle Sea’s cleanest cityest city. At the time that I joined Agant, Dave “OMG it’s Dave Addey off the internets” Addey had already been discussing the project with them, and much was ready to go.

Indeed, Dave and I were ready to go. We took a train down to the Discworld Emporium and met Sir Terry Pratchett, his partner in crime Rob Wilkins, their able Aiders and Abettors from the Emporium and some of the most happening among those who Make Things Happen at the publishers.

In the Dolphin Hotel at Wincanton, we had a pub lunch and tried to define exactly what it meant to put Ankh-Morpork into an iPad. I’d already made a start, and here I learned Lesson Two: the creative mind knows it can be done. You don’t need to convince them it can be done. It’s already been done, it’s up there in their mind’s eye. You just need to convince them it can be done well, and in this reality.

Wait, lesson two? Yes, for here was Lesson One: when meeting your heroes, only one of the two of you is likely to try and make a big deal out of it. The other wants their ploughman’s lunch and their G&T and to get back to work. Keeping focussed on the task at hand – a difficult feat as I was dealing with an internal monologue that would not stop going “squeeeeeee” – paid off, and everyone left safe in the knowledge that there was lots to do but that we would do it because it needed to be done and we needed to be the ones to do it and we needed it to be done well.

OK, most of the rest of this story is history. Dave, the developers at Agant, and a dedicated collection of others worked very hard for a number of months and eventually Discworld: the Ankh-Morpork Map for iPad oozed its way out of the alledged waters of the Ankh (there’s no need to disinfect it, the bacteria refuse to touch the stuff) and into the App Store. It remains the project of which I am most proud, both of the contributions I made and of all that was put in by everyone involved.

All good stories, though, come to an end, and Sir Pterry’s did last week. Today I went back over to the Emporium to talk a bit about the past and a lot about the future. It’s from today’s discussions that Lesson Three originates.

Some day, you will be invited by a polite though insistent anthropomorphic personification to take a walk together. Before he arrives, do the things that need to be done. Make sure they’re the things that must be done by you, and that you will be satisfied and pleased to see completed. Otherwise the two of you won’t have a whole lot to talk about.