Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Tuesday, June 16, 2020

Forearmed

In researching my piece for the upcoming de Programmatica Ipsum issue on cloud computing, I had thoughts about Apple, arm, and any upcoming transition that didn’t fit in the context of that article. So here’s a different post, about that. I’ve worked at both companies so don’t have a neutral point of view, but I’ve also been in bits of the companies far enough from their missions that I don’t have any insider insight into this transition.

So, let’s dismiss the Mac transition part of this thread straight away: it probably will happen, for the same reasons that the PowerPC->Intel transition happened (the things Apple needed from the parts – mostly lower power consumption for similar performance – weren’t the same things that the suppliers needed, and the business Apple brought wasn’t big enough to make the suppliers change their mind), and it probably will be easier, because Apple put the groundwork in to make third-party devs aware of porting issues during the Intel transition, and encourage devs to use high-level frameworks and languages.

Whether you think the point is convergence (now your Catalyst apps are literally iPad IPAs that run on a Mac), or cost (Apple buy arm chipset licences, but have to buy whole chips from Intel, and don’t get the discount everybody else does for sticking the Intel Inside holographic sticker on the case), or just “betterer”, the arm CPUs can certainly provide. On the “betterer” argument, I don’t predict that will be a straightforward case of tomorrow’s arm Mac being faster than today’s Intel Mac. Partly because compilers: gcc certainly has better optimisations on Intel and I wouldn’t be surprised to find that llvm does too. Partly because workload, as iOS/watchOS/tvOS all keep the platform on guard rails that make the energy use/computing need expectations more predictable, and those guard rails are only slowly being added to macOS now.

On the other hand, it’s long been the case that computers have controller chips in for interfacing with the hardware, and that those chips are often things that could be considered CPUs for systems in their own rights. Your mac certainly already has arm chips in if you bought it recently: you know what’s running the OS for the touch bar? Or the T2 security chip? (Actually, if you have an off-brand PC with an Intel-compatible-but-not-Intel chip, that’s probably an arm core running the x86-64 instructions in microcode). If you beef one of those up so that it runs the OS too, then take a whole bunch of other chips and circuits off the board, you both reduce the power consumption and put more space in for batteries. And Apple do love talking battery life when they sell you a computer.

OK, so that’s the Apple transition done. But now back to arm. They’re a great business, and they’ve only been expanding of late, but it’s currently coming at a cost. We don’t have up to date financial information on Arm Holdings themselves since they went private, but that year they lost ¥31bn (I think about $300M). Since then, their corporate parent Softbank Group has been doing well, but massive losses from their Vision Fund have led to questions about their direction and particularly Masayoshi Son’s judgement and vision.

arm (that’s how they style it) have, mostly through their partner network, fingers in many computing pies. From the server and supercomputer chips from manufacturers like Marvell to smart lightbulbs powered by Nordic Semiconductor, arm have tentacles everywhere. But their current interest is squarely on the IoT side. When I worked in their HPC group in 2017, Simon Segars described their traditional chip IP business as the “legacy engine” that would fund the “disruptive unit” he was really interested in, the new Internet of Things Business Unit. Now arm’s mission is to “enable a trillion connected devices”, and you can bet there isn’t a world market for a trillion Macs or Mac-like computers.

If some random software engineer on the internet can work this out, you can bet Apple’s exec team have worked it out, too. It seems apparent that (assuming it happens) Apple are transitioning the Mac platform to arm at start of the (long, slow) exit arm make from the traditional computing market, and still chose to do it. This suggests something else in mind (after all, Apple already designs its chips in-house, so why not have them design RISC-V or MIPS chips, or something entirely different?). A quick timetable of Mac CPU instruction sets:

  • m68k 1984 – 1996, 12 years (I exclude the Lisa)
  • ppc 1994 – 2006, 12 years
  • x86 and x86-64 2006 – 2021?, 15 years?
  • arm 2020? – 203x?, 1x years?

I think it likely that the Mac will wind down with arm’s interest in traditional computing, and therefore arm will be the last ever CPU/SoC architecture for computers called Macs. That the plan for the next decade is that Apple is still at the centre of a services-based, privacy-focused consumer electronics experience, but that what they sell you is not a computer.

posted by Graham at 13:30  

Wednesday, October 16, 2019

A question of focus

The problem with The Labrary is that I offer to do so many things – because I could do them, and do them well – that it can be hard to find the one thing I could do for you that would be most helpful:

  • Artificial Intelligence
  • Agile Development
  • Continuous Delivery
  • Software Architecture
  • Technical Writing
  • Developer Experience
  • Programmer Mentoring

Each of these supports the mission of “making it faster and easier to make high-quality software that respects privacy and freedom”, but all of them is overwhelming. I have credentials/experience to back up each of them, but probably don’t have the reputation as a general expert that someone like Dan North or Liz Keogh can use to have people ask me anything.

So I want to pick one. One thing, probably from that list, and pivot to focus on that. Or at least get in through the door that way, then have the conversations about the other things once you know how much faster and easier I make it for you to make high-quality software.

And I’d really value your suggestions. Which one thing do you know me for, above all others? Which one thing is the pain that the place you work, or places you’ve worked, most need fixing?

Comment here, have a chat, send an email. Thanks for helping me find out what I want to be when I grow up.

posted by Graham at 10:03  

Thursday, October 3, 2019

Solving the underlying problem

As a software engineer, it’s easy to get work engineering software. Well, maybe not easy, but relatively so: that is the kind of work that comes along most. The kind of work that people are confident I can do. That they can’t do, so would like me to do for money.

It’s also usually the worst work available.

I don’t want to take your shopping list of features, give you a date and a cost, then make those features. Neither of us will be very happy, even if it goes well.

I want to get an understanding of your problem, and demonstrate how software can help in solving it. Maybe what we need to understand isn’t the problem you presented, but the worse problem that wasn’t on your mind. Or the opportunity that’s worth more than a solution to either problem.

Perhaps we ask a question, in solving your problem, to which the answer is that we don’t know, and now we have another problem.

You might not need me to build all of the features you thought of, just one of them. Perhaps that one works better if we don’t build it, but configure something that already exists. Or make it out of paper.

You understand your problem and its domain very well. I understand software very well. Let’s work together on combining that expertise, and both be happier in the process.

posted by Graham at 17:44  

Monday, December 17, 2018

Product teams: our products are not our products

Woah, too many products. Let me explain. No, it will take too long, let me summarise.

Sometimes, people running software organisations call their teams “product teams”, and organise them around particular “products”. I do not believe that this is a good idea. Because we typically aren’t making products, we’re solving problems.

The difference is that a product is “done”. If you have a “product team”, they probably have a “definition of done”, and then release software that has satisfied that definition. Even where that’s iterative and incremental, it leads to there being a “product”. The thing that’s live represents as much of the product as has been done.

The implications of there being a “product” that is partially done include optimising for getting more “done”. Particularly, we will prioritise adding new stuff (getting more “done”) over fixing old stuff (shuffling the deckchairs). We will target productish metrics, like number of daily actives and time spent.

Let me propose an alternative: we are not making products, we are solving problems. And, as much out of honesty as job preservation, let me assure you that the problems are very difficult to solve. They are problems in cybernetics, in other words in communication and control in a complex system. The system is composed of three identifiable, interacting subsystems:

  1. The people who had the problem;
  2. The people who are trying to solve the problem;
  3. The software created to present the current understanding of the solution.

In this formulation, we don’t want “amount of product” to be a goal, we want “sufficiency of solution” to be a goal. We accept that the software does not represent the part of the “product” that has been “done”. The software represents our best effort to date at modelling our understanding of the solution as we comprehend it to date.

We therefore accept that adding more stuff (extending the solution) is one approach we could consider, along with fixing old stuff (reflecting new understanding in our work). We accept that introducing the software can itself change the problem, and that more people using it isn’t necessarily a goal: maybe we’ve helped people to understand that they didn’t actually need that problem solved all along.

Now our goals can be more interesting than bushels of software shovelled onto the runtime furnace: they can be about sufficiency of the solution, empowerment of the people who had the problem, and improvements to their quality of life.

posted by Graham at 15:21  

Sunday, December 9, 2018

More speed, lower velocity

I frequently meet software teams who describe themselves as “high velocity”, they even have graphs coming from Jira to prove it, and yet their ability to ship great software, to delight their customers, or even to attract their customers, doesn’t meet their expectations. A little bit of sleuthing usually discovers the underlying problem.

Firstly, let’s take a look at that word, “velocity”. I, like Kevlin Henney, have a background in Physics, and therefore I agree with him that Velocity is a vector, and has a direction. But “agile” velocity only measures amount of stuff done to the system over time, not the direction in which it takes the system. That story may be “5 points” when measured in terms of heft, but is that five points of increasing existing customer satisfaction? Five points of new capability that will be demoed at next month’s trade show? Five points of attractiveness to prospects in the sales funnel?

Or is it five points of making it harder for a flagship customer to get their work done? Five points of adding thirty-five points of technical debt work later? Five points of integrating the lead engineer’s pet technology?

All of these things look the same in this model, they all look like five points. And that means that for a “high-velocity” (but really low-velocity, high-speed) team, the natural inclination is to jump on it, get it done, and get those five points under their belt and onto the burn down chart. The faster they burn everything down, the better they look.

Some of the presenting symptoms of a high-speed, low-velocity team are listed below. If you recognise these in your team, book yourself in for office hours and we’ll see if we can get you unstuck.

  • “The Business”: othering the rest of the company. The team believes that their responsibility is to build the thing that they were asked for, and “the business” needs to tell them what to build, and to sell it.
  • Work to rule: we build exactly what was asked for, no more, no less. If the tech debt is piling up it’s because “the business” (q.v.) doesn’t give us time to fix it. If we built the wrong thing it’s because “the business” put it at the top of the backlog. If we built the thing wrong it’s because the acceptance criteria weren’t made clear before we started.
  • Nearly done == done: look, we know our rolling average velocity is 20 bushels of software, and we only have 14 furlongs and two femtocandela of software to show at this demo. But look over here! These 12 lumens and 4 millitesla of software are in QA, which is nearly done, so we’ve actually been working really hard. The fact that you can’t use any of that stuff is unimportant.
  • Mini-waterfall: related to work to rule (q.v.), this is the requirement that everyone do their bit of the process in order, so that the software team can optimise for requirements in -> software out and get that sweet velocity up. We don’t want to be doing discovery in engineering, because that means uncertainty, uncertainty means rework, and rework means lower velocity.
  • Punitive estimation: we’re going to rename “ambiguity” to “risk”, and then punish our product owner for giving us risky stories by boosting their estimates to account for the “risk”. Such stories will never get scheduled, because we’ll never be asked to do that one risky thing when we can get ten straightforward things done in what we are saying is the same time.
  • Story per dev: as a team, our goal is to shovel as much software onto the runtime furnace as possible. Therefore we are going to fan out the tasks to every individual. We are each capable of wielding our own shovel, and very rarely do we accidentally hit each other in the face while shovelling.
posted by Graham at 10:37  

Friday, August 31, 2018

More on freedom and licensing

Last month, I asked whether Freedom Zero is such a great idea, whether it’s OK to limit the freedom to use the software for any purpose if you dislike the purpose to which you believe someone will put it. I gave the example of the Numerical Recipes licence as one that could be adapted to this situation.

In summary, the licence would say “you are free to read this code for the purposes of understanding it, to share the code, and to contribute changes to the code. Should you wish to use the code, contact me, we will go through some customer due diligence, and I will decide whether I want you as a customer”. So far, so unexciting: imagine the usual SaaS startup pricing page with two pricing levels. The “Basic” tier is free, and scales up to 0 uses. The “Custom” tier is $contact_us pricing, and scales up to $contact_us uses.

The Lerna project was briefly licensed under a different model, which took a different approach to solve the same problem. Their approach was “you all get to treat this as a free software project, except the named organisations, who don’t”.

Just to get this out of the way as many of us are techie people who enjoy picking nits: this approach doesn’t appear to solve the problem that the author was setting out to solve. Everybody else except @evil_corps get the software under the terms of the MIT licence, @evil_corps do not get to use the software at all. Now because I got it under the terms of the MIT licence, I am free to do whatever I want except blame the authors if it doesn’t work or remove the MIT licence. This means that I am free to give it or sell it to members of @evil_corps.

Lots of people disagreed with the idea that the Lerna project maintainers, or I, would seek to restrict freedom for moral/ethical reasons, because restricting freedom itself is a moral/ethical choice that is objectionable. Unfortunately, the argument that is often advanced is a bad one.

Richard Stallman’s Why programs must not limit the freedom to run them says numerous things:

  1. The bad actors will probably ignore your license terms anyway, so why bother? This pessimistic view could equivalently be used to argue against any commercial terms (bad actors will pirate your software anyway, so why sell it?) or even RMS’s own General Public License (bad actors will ignore the copyleft requirements anyway, so why bother?). In fact many actors voluntarily comply with the GPL without being asked, many compliance requests are resolved voluntarily on notification, GPL compliance usually starts with a polite, private request not a lawsuit or public shaming. So evidently bothering is useful.

  2. Using copyright to restrict usage is abuse of copyright. But so is copyleft, although as it’s a “good” abuse of copyright that RMS approves of, he’s OK with it.

  3. Limiting freedom is a slippery slope to not being able to build a usable combined system, because you would have to check all the licences to know whether you could do anything. RMS does not have a beard, he has a facial hair, and another facial hair, and another, and so on.

  4. You shouldn’t have the right to do that.

That last one is an interesting one that RMS doesn’t go into in depth, I would imagine because he considers it inalienable. He wrote Freedom Zero, of course he believes in Freedom Zero.

Brad Kuhn, on the other hand, has written about whether I should have that right, and his post is an interesting one. He uses the example of the ACLU’s defence of free speech to argue that having a simple platform and universally supporting a simple principle creates a powerful reputation from which to then build other principled arguments.

I think that argument is compelling, and it has brought me back round to wanting Freedom Zero again.

posted by Graham at 13:44  

Wednesday, July 25, 2018

On study-only licences

In my previous post, I mused on the value of Freedom Zero and of a non-free licence that allows for study but not for use:

I think it would have to be a licence that enabled studying, sharing and modification of the software, but that explicitly forbade any use for any purpose that isn’t studying, modifying or sharing. With a “contact me or my agent, tell us what you’re doing, and we’ll decide whether to grant you an additional licence for use” suffix. This is more open than closed proprietary software, but no more available for deployment to bad actors.

Waking up this morning I remembered that I have a copy of Numerical Recipes. This is a book, that contains code, and as such you can read the code. But not much else:

Without an additional license to use the contained software, this book is intended as a text and reference book, for reading and study purposes only. However, a restricted, limited free license for use of the software by the individual owner of a copy of this book who personally keyboards one or more routines into a single computer is granted under terms described on p.xix.

Page xix expands:

If you personally keyboard no more than 10 routines from this book into your computer, then we authorize you (and only you) to use those routines (and only those routines) on that single computer.

If you want to study, or to try things, knock yourself out. If you want to distribute things, or use things, get in touch and we’ll choose whether to sell you a licence.

This is not unexplored territory.

posted by Graham at 09:49  

Friday, September 8, 2017

Choose boring employers

Amusingly, my previous post choose boring employees was shared to hacker news under the off-by-one erroneous title choose boring employers. That seemed funny enough to run with, but what does it mean to choose boring employers?

One interpretation is that a boring employer is one where you do not live in interesting times. Where you can get on with your job, and with finding new and better ways to do your job, without constantly fighting fires.

But what if you’re happiest in an environment where you are fighting fires? In that case, you probably should surround yourself with arsonists.

Another interpretation is to invert the discussion in Choose Boring Employees: find an employer who spends their innovation tokens wisely. One who’s OK with the answer to “how do I store these tuples of known structure” being “in a relational database”, or one who doesn’t mind when the answer to “what platform should we base our whole business on” starting with “I skim-read a blog post on HN when I was riding MUNI this morning and…”.

But, let’s be clear, there’s a place for the shiny new technology. Sometimes you do need to spend your innovation tokens, so you don’t want to be somewhere that won’t let you do it at all. Working on a proof of concept, you want to get to proof quickly, so it may be time to throw caution to the wind (unless the concept you’re trying to prove involves working within some cautious boundaries). So boring need not get as far as frustrating.

posted by Graham at 19:37  

Tuesday, September 5, 2017

Choose boring employees

An idea I’ve heard from many directions recently is that “we” (whoever they are) “need to be on the latest tech stack in order to attract developers”. And yes, you do attract developers that way. Developers who want to be paid to work on the latest technology.

Next year, your company will be a year more mature. Your product will be a year more developed. You will have a year more customers. You’ll have a year more tech debt to pay off.

And your cutting-edge tech stack will be so last year. Your employees will be looking at the new startup in the office next door, and how they’re hiring to work on the latest stack while you’re still on your 2017 legacy technology.

posted by Graham at 13:12  

Thursday, July 6, 2017

The worst phrase in software marketing

“Rewritten from the ground up”.

Please. Your old version mostly worked, except for those few corner cases that I’d learned how to work around. Now I don’t know whether the stuff that did work does work now, and I don’t know that I’ll find that stuff in the same place any more.

There’s a reason that old, crufty code in the core of your application was old and crufty. It is old because it works, well enough to pay the programmers who work on maintaining it and building the new things. It is crufty because the problem you’re trying to solve was not perfectly understood, is still not perfectly understood, and is evolving.

Your old, crufty code contains everything you’ve learned about your problem and your customers. And now you’re telling me that you’ve thrown it away.

posted by Graham at 08:39  
Next Page »

Powered by WordPress