Skip to content

I make mistakes for a living

As a team lead, my job is to make all the mistakes on my team. I’m responsible for each one of them. I’m also responsible for deciding what we do about them, whether that’s to ensure that they can’t happen again or ignore them because they’re easy to recover from.

That time someone on my team built a feature that didn’t ship because of a bug? I could have made sure the requirements were clearer, or they had more access to the product owner, or a more useful design review, or a more useful code review, or a better test plan. Or all of those, or none of those, or something else. I could have – should have – got the problems out of their way.

They don’t make mistakes, I do.

Learning about software freedom

On the front page of Hacker News at the moment is a post on The Three Software Freedoms. It does away with the Free Software Foundation’s Freedom Zero:

The freedom to run the program as you wish, for any purpose.

On the basis that it “is just silly I mean of course I can use the program as I wish.” Of course you can. Except, of course, that’s not in fact true, and because software companies are licensing – not selling – you software, you’re actually bound by their terms and the limitations they impose. Picking just one example, here are some snippits from the Apple Licensed Application End-User License Agreement, relevant to apps bought (sorry, licensed) through the iTunes App Store:

The Products transacted through the Service are licensed, not sold, to You for use only under the terms of this license

The licensor (“Application Provider”) reserves all rights not expressly granted to You.

This license granted to You for the Licensed Application by Application Provider is limited to a non-transferable license to use the Licensed Application on any iPhone or iPod touch that You own or control and as permitted by the Usage Rules set forth in Section 9.b. of the App Store Terms and Conditions (the “Usage Rules”). This license does not allow You to use the Licensed Application on any iPod touch or iPhone that You do not own or control, and You may not distribute or make the Licensed Application available over a network where it could be used by multiple devices at the same time. You may not rent, lease, lend, sell, redistribute or sublicense the Licensed Application. You may not copy (except as expressly permitted by this license and the Usage Rules), decompile, reverse engineer, disassemble, attempt to derive the source code of, modify, or create derivative works of the Licensed Application, any updates, or any part thereof (except as and only to the extent any foregoing restriction is prohibited by applicable law or to the extent as may be permitted by the licensing terms governing use of any open sourced components included with the Licensed Application).

If You breach this restriction, You may be subject to prosecution and damages.

Your rights under this license will terminate automatically without notice from the Application Provider if You fail to comply with any term(s) of this license. Upon termination of the license, You shall cease all use of the Licensed Application, and destroy all copies, full or partial, of the Licensed Application.

You agree that any Services contain proprietary content, information and material that is protected by applicable intellectual property and other laws, including but not limited to copyright, and that You will not use such proprietary content, information or materials in any way whatsoever except for permitted use of the Services. No portion of the Services may be reproduced in any form or by any means. You agree not to modify, rent, lease, loan, sell, distribute, or create derivative works based on the Services, in any manner, and You shall not exploit the Services in any unauthorized way whatsoever, including but not limited to, by trespass or burdening network capacity. You further agree not to use the Services in any manner to harass, abuse, stalk, threaten, defame or otherwise infringe or violate the rights of any other party, and that the Application Provider is not in any way responsible for any such use by You, nor for any harassing, threatening, defamatory, offensive or illegal messages or transmissions that You may receive as a result of using any of the Services.

You may not use or otherwise export or re-export the Licensed Application except as authorized by United States law and the laws of the jurisdiction in which the Licensed Application was obtained. In particular, but without limitation, the Licensed Application may not be exported or re-exported (a) into any U.S. embargoed countries or (b) to anyone on the U.S. Treasury Department’s list of Specially Designated Nationals or the U.S. Department of Commerce Denied Person’s List or Entity List. By using the Licensed Application, you represent and warrant that you are not located in any such country or on any such list. You also agree that you will not use these products for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture or production of nuclear, missiles, or chemical or biological weapons.

So don’t claim, through malice or ignorance, that Freedom Zero is not worth making explicit, and doesn’t need protecting.

On the rhetorical cost of ownership

I’ve recently been talking about software engineering economics, in a very loose way, but so have other people. And now I understand that it’s annoying when people talk about it, and have decided to continue anyway. I’ve decided to continue because what I see is either inaccurate comparisons being made, or valid comparisons that have questionable applicability outside their immediate domain. The world of IT cost comparison is still run by marketing, not by operations.

Recently, I read Don’t Build Private Clouds. Subbu says that the sticker prices (up to $10k for a server that will last four years, vs. up to $1500 per month for a public cloud machine) should not be compared because there are additional costs to self-hosting:

  1. Engineering costs
  2. Network automation costs
  3. Loss of agility
  4. Opportunity costs

Fine, but what are those costs? Why do I not need to do any engineering or automation if I use a public cloud provider? If I do, what is that going to cost? What agility and opportunity do I lose by tying my infrastructure to any one cloud vendor, and what will that cost?

Subbu’s blog says that he is a “cloud helper”, and that goes a long way to explaining why we didn’t get a straight answer on the cost comparison. We’re not being told that cloud services are cheaper, instead we are being told of the Fear, Uncertainty and Doubt involved in choosing the less-favoured path.

Similarly, an IBM employee recently said that Macs are cheaper than PCs by up to $543 per user (remember that’s “up to”, not “as much as” – the lower bound given is $273). Let’s ignore the conflict of interest arising from Apple’s global partnership with IBM: IBM wouldn’t say that their partner systems are cheaper so that they could drum up interest in their partnership services, surely. Surely. I mean, this isn’t the IBM of 1984, is it?

What IBM’s VP says is that over a four-year lifetime, among employees who are given the choice of which platform and model computer they want, the Macs are cheaper. That’s of course a figure about which it’s possible to make realistic comparisons: given IBM’s approach to desktop support, IBM’s level of staffing, IBM’s applications, IBM’s approach to working, IBM’s budgeting for IT support operations, it’s cheaper for people who choose Macs to use Macs than for…well, it’s not clear, but it seems to be than for the “everybody else” bucket: not only people who chose PCs to use PCs, but people who weren’t given a choice to use PCs.

So before you make a textexpander macro for that link and insta-reply to anyone who uses the phrase “Apple Tax”, just how similar is your environment to IBM’s?

Can’t you just…

Continuing the thoughts on vexing problems, one difficulty when it comes to discussing software is talking about the size of software. I’m not really talking about productivity metrics – good or bad – like source lines of code or function points, rather the fact that the complexity of a problem looks different depending on who’s doing the looking.

Sometimes, a problem that’s very simple from a business perspective can be incredibly complex technically. One product I worked on could be summarised very quickly: let people interact with marketing campaigns by sending and receiving messages on their mobiles. The small amount of logic between send and receive – allowing the campaigns to operate as quizzes, votes, or auctions – could be detailed on an index card.

But that simplicity was backed by a huge amount of technical complexity to make it work. “Can you just send this message to everyone who got the quiz question correct?” Well, yes, but as it’s a picture message we need to work out how to make it look good on the recipient phone, change it to fit those criteria, and then send it. What makes it look good – and indeed how we can get the information to make that decision – depends on the device, but also which network it’s on, and maybe whether it’s on pre-pay or post-pay and whether they use HTTP, WAP or e-mail to send messages to that device (which might be different from how they send to other devices on the same network). And even after we’ve gathered that information, it may be wrong as some devices claim to support image formats that they can’t render, or image sizes that they’ll actually reject or fail to display.

On the other hand, sometimes the business problem is a lot more complex than the technical problem. If you’re a mobile app developer, any length of problem definition about exciting disruptive apps can be reduced to “so you want to display data from a web service in a table view”.

And then at other times, “can’t you just” gets stymied from left field. Why yes, we could simply do that, and it would be good for the business, but this regulation/patent/staff shortage means we need to do something else.

On the business case for (or against) software

In the vexing problems, I dismissed the hard problems of computer science as being incidental to another problem: we can’t say what the value of our work is. That post contained plenty of questions, precisely because the subject is so unknown.

There are plenty of ways in which the “value” of something can be discussed, but let’s stick with economic value for the moment. Imagine you have the opportunity to write some software, but only if I pay you for it. And not even then, but only if you can make me reasonably confident that I will get some return; that having the work done is worth more than the cost of you doing the work.

A corner of the software industry just cowered and stuck its head in the sand. “#NoEstimates!”, they cry. This problem is hard, so why should we solve it?

That’s a really interesting perspective. You should come and talk about #NoEstimates at my conference. It’ll be on sometime, in a place, and there’ll be a flight that can get you there. Maybe. That’s enough information to be going on for it to be clear that you should commit to it, so I’ll see you there.

At the other end are the folks who would literally write whole books about the topic (and given that books are themselves literary, that “literally” is itself meant literally). I pulled a few likely titles off the shelf to see what they would tell me, and ended up with:

We have to wonder how some people can be certain that estimates are worthlessly inaccurate, while others such as Barry Boehm (the author of Software Engineering Economics and the COnstructive COst MOdel it describes) have built careers out of explaining how to do it well.

Here’s one hypothesis: Boehm is a charlatan, and the COCOMO doesn’t work at all. Let’s see whether there’s evidence for that.

A likely source is COCOMO evaluation and tailoring by Miyazaki and Mori from Fujitsu. The authors of that paper, applying the COCOMO (’81, not the later COCOMO II) model to a corpus of projects undertaken by Fujitsu, conclude:

The original COCOMO overestimates the effort required to develop software in our environment, but its tailoring methodology is applicable. […] The resulting model fits 68% of the projects with less than 20% relative error, after the deletion of two outliers.

That’s maybe short of brilliant – the median project effort in their paper seems to be over 10 person-years, so they’re saying “there’s around two chances in three that we can tell you what this project will cost to within twice a developer’s salary” – but it’s much more precise than “#NoEstimates!” and only quite a lot less accurate.

It might be interesting to track the application of COCOMO through time, and see whether a larger corpus of projects and refinement of the model has improved its accuracy and applicability. Unfortunately, that goal is in conflict with a general industry observation, noted in the introduction of the Fujitsu paper:

We have the impression that software people prefer to start from scratch rather than improving upon the work of others, which seems to be slowing down the progress in this field.

And indeed much of what was written in the sources I’ve been talking about here is no longer applied. All three of the books I mentioned up top talk about Earned Value (EV) Analysis, recording the actual rate of delivering on a project against its Planned Value (PV). For example, Hughes and Cotterell recommend plotting earned value using the “0/100 technique”:

where a task is assigned a value of zero until such time as it is completed when it is given a value of 100% of the budgeted value

Agilistas will recognise the resulting plot: it’s a burn down chart, where a story is not delivered until it has been accepted according to the definition of done, at which point it is completely delivered.

Ultimately somebody in the Agile community must have noticed that they accidentally borrowed a software engineering concept, because the burn down chart is now unfashionable.

If the team starts with a very full, prioritized, and estimated backlog with the expectation they will burn down all tasks to zero — does this not sound like a Waterfall mentality?

Oh noes, the dreaded W word! Run away from this thought, for it is a thought that somebody in the past has already considered!

And that is why I consider these problems to be vexing: when potential solutions are proposed, they are by necessity existing ideas from the old school that must be rejected in favour of…whatever is not those solutions. As Craftsmanship is post-Agile is post-Software Engineering is post-Crisis thinking, surely it will soon be time for post-Craftsmanship thinking. And then post-whatever-comes-next thinking. And we still won’t know how to compare expected value to expected cost.

The Vexing Problems in Programming

I admit it, I’ve been on the internet for quite a while (I could tell you that my ICQ number is 95941970, but I haven’t logged in for years) and my habits haven’t changed. I still regularly get technology news from slashdot, and today was no exception. An interesting article was Here Be Dragons: The 7 most vexing problems in programming. Without wanting to spoil the article for you by giving away the punchline, there are indeed some frustratingly difficult problems mentioned: multithreading, security, encryption are among the list.

All of these problems are sideshows to what I see as one of the largest and most vexing issues in programming: the fundamental rule to business administration is that your income should be greater than your costs, but software makers still, by and large, don’t have a way to compare the expected value of their work to the expected cost of the work.

The problem in space

Different software teams – and individuals – do work in different contexts, and in different ways. The lone wolf micro-ISV is not the same as an individual contract developer. The in-house IT team does not have the same problems to solve as the shrinkwrapped software vendor, and those developing web services for public consumption have yet another context. The team with core hours all working in a single office is different from the distributed team inhabiting multiple time zones.

How much of this variety is essential, and how much is accidental? How much of it is relevant, when considering some intervention, process change, or technique? Consultants speaking at conferences (another context with its own similarities and differences from the others) don’t tend to talk about what researchers in fields such as psychology would recognise as “threats to validity” of their work, but given all of the ways in which software is made, we need to know whether some proposal applies to all of them, or to some of them, or whether it has been applied to some of them and might be applied to others, and what would assist or confound that application.

The problem in time

What are the accepted, tested and validated ways to identify who will be using and otherwise impacted by our software systems? To know whether they can use the system we propose, and whether it is the best system for their intended use? To ensure that our proposed software systems treat those people ethically? To understand the cost (to ourselves and to others) of constructing those systems? To deliver the systems to the people who will interact with them? To choose which people are or aren’t entitled to access? To build a representation of the problem to be solved, to validate that representation, to validate the solution against that representation?

Where an answer exists to those questions, what are the contexts in which it is valid and what are the threats to its validity? How has that answer been compared with other possibilities? How has it been confirmed? How has it been challenged? How can I find out about those confirmations and challenges? How can I find out about any alternatives? What techniques exist to weigh up those alternatives quantitively, rather than relying merely on the persuasiveness of the conference speakers promoting those solutions (and, by the way, the books/screencasts that describe the solutions)?

The lack of a problem

Why should I care? There’s enough money in software at the moment to mean that I don’t need to be any good at knowing what works or doesn’t work, I just need to get out there and sell some software. In the rare situation that I don’t make my money back, that’s just the market forces at work, and I can go and get a high-paying job somewhere while I lick my wounds, and pick another programming language/framework/platform/whatever it is that’s going to make my next attempt definitely succeed.

Clearly, this bottomless pit of money that arises from society’s unwavering faith in software and its ability to cure all ills is never going to run out. There’s no need to worry ever about whether we’re doing it right, because there’ll always be someone out there willing to pay for us to do it wrong. Life as a programmer is like some kind of socialist utopia where whether we’re making a valuable contribution or not, the rest of society is looking out for us.

That’s going to last forever, right?

Withholding the Four Freedoms

Having downsized my rather over-enthusiastic computer collection (thanks, eBay!), I was down to one computer. Unfortunately, as a rather long in the tooth MacBook Air, it’s no longer suited to my needs and neither is it upgradeable. I got all of the files I care about off of its disk and set out to look for a replacement, meanwhile setting the MacBook aside to one day wipe its e4fs storage and re-install Mac OS X/OS X/macOS/whatever we call it this week.

I chose the sort of spec computer I wanted, then carefully researched various vendors to see what components they used and whether there was reported driver support in the Linux kernel. Eventually, dude, I got a Dell. The Alienware 15 R3 is made of bits that are supported in Linux, with the most complex piece being that the network adapter requires binary firmware. The manufacturer includes links to the firmware blobs on their website though, so this can’t be hard…can it?

I fell straight to running the Debian Jessie installer, put the firmware blobs in place, and…the wi-fi isn’t detected. Oh well, it’s quite an old kernel, maybe I could switch to Testing? No, that doesn’t boot at all: some error about not getting valid cache information from the SSD. Neither does the Ubuntu 16.10 kernel boot, for the same reason. 16.04 boots just fine, and it detects the wireless and connects to the network…and then the installer crashes.

One thing I’m not looking forward to is the onslaught of replies to this post from people who want to help, but ultimately won’t help. “You should try Arch Linux, I expect that works,” or “maybe you’ll have better luck with Fedora.” Why do you expect that? What specific knowledge about my problem makes you think that your specific choice of distribution will work better? And why can’t I just take that knowledge and apply it to Debian, or Ubuntu? They’re just distributions, they’re all made of the same bits.

I admit, I’m frustrated. I want to be a proud advocate for Free Software and for totally free computing environments, but being unable to even run some of the flagship software makes me reticent to recommend it to others. I was having these problems back in 2001, and I’m having these problems now. And it’s not like I’m incompetent when it comes to *nix administration or to Linux driver configuration, and nor did I just buy the prettiest computer I could find and hope that it would work, I _did my research_. I spent hours making sure that there were drivers for the various components in this system, reading reviews, QA forum posts, and kernel mailing list messages. Unfortunately my willingness to screw about with configuring my computer just so that I can use it has waned over the last couple of decades, faster than the necessity to screw about with it has decreased.

For the moment, I’m running Windows 10 as a hardware abstraction layer, and have a full-screen VirtualBox VM to do my work in. It works, it makes me die a little inside but it works. But I’m worried that for all the crowing that open source is eating the world, it’s still too hard to jump in, even for a diligent and experienced user. Until we can give someone a computer that they turn on and start working in, and that runs free software, this will all remain the preserve of professionals and committed hobbyists. The Four Freedoms will effectively be restricted to those in the know and with time to dedicate to obtaining them: all users are equal, but some are more equal than others.

On the extremes of computer science

I didn’t study computer science at school or university, and still manage to work as a programmer.

That is not to say that I don’t need to know some things that are taught on computer science courses. Just this week I’ve had to build a couple of different data structures and understand their running time: very CS.

I’ve also needed to know things that aren’t on a CS degree, too. The acceptance criteria for one of my projects are written in French, and none of the CS courses I’ve seen in UK universities include that in the syllabus.

I’m neither arguing for nor against the validity of a CS background in professional software development. I’m arguing against taking either side. You need to know some CS things to write software, you need to know some other things, multiple backgrounds are appropriate and welcome.

Something old, something new

Sometimes, what you want out of a software system is something
unexciting. Something proven to work, that’s well-understood, and has
had all of the kinks smoothed out over years (or even decades) of
refinement.

Of course, what you get when that’s what you want is something like
Unix. Where the thing it was designed for…well, let’s not pretend
that
Unix was ever designed. The
thing it was used for – hosting the runoff typesetting system that is
only applied these days to formatting its own manual – is so far from
its current application as a
bootloader sitting between your other bootloader and the thing you’re using your computer for
that you can’t understand why someone would think the same thing would
be used for both purposes.

And you find that the kinks have not been ironed out so much as, well,
baked in. The limitation that processes communicate via an untyped
character stream with unspecified eight-bit encoding is not a
limitation, it’s
a design philosophy.

So what you really want to do is to throw away all of that cruft,
all of the mistakes of the past, and go somewhere new. Somewhere
exciting. Somewhere…like node.js? Well no,
JavaScript is of course far from new and exciting, even
the jokes about it are
years old now.

OK, so bad example, but maybe you have your own counter-example. The
one true environment that will be used for all future software. Well,
at least for the next nine months, before its killer comes along.

Of course, what you get when that’s what you want is another rewrite
of all of the things that were working anyway. Your new environment
probably comes with a package manager written from scratch in the new
language, when there was really nothing much wrong with CPAN.

Keeping on the programming theme, because that’s what we tend to make
computering out of, most languages that have any traction these days
almost certainly have C language bindings. So it’s no surprise that
the most popular open source projects are all written in C…

…OK, now that you’ve stopped laughing and got your breath back, you
can appreciate that actually the lingua franca of modern (ahem)
operating systems is not used to cut through the babel of other
programming languages, but just to write their runtimes. Because
that’s what C is good at: secure, bounds-checked execution
environments.

And you can also appreciate that all of the novelty of the new shiny
is used to rewrite the things that already existed and already worked,
replacing them with new implementations that work in surprising ways
(and don’t work in surprising ways, too).

There’s good in the old. There’s bad in the new. There’s bad in the
old, and good in the new, too. There are plenty of social, political,
economic, and – maybe – technical factors in whether the old survives,
or the new thrives. Arguments for the old or the new for their own
sake are facile and misguided.

Dogmatic paradigmatism

First, you put all of your faith in structured programming, and you got burned. You found it hard to associate the operations in your software with the data upon which they act, and to make sure that the expectations made on the data in one place are satisfied when that data has been modified in that other place, or over there in yet another place. Clearly structured programming is broken.

Then, you put all of your faith in object-oriented programming, and you got burned. You found it hard to follow the flow of a program when it jumps in and out of different classes, and to see which parts were coupled to what. Clearly object-oriented programming is broken.

Then, you put all of your faith in functional programming, and you got burned. You found it hard to represent real business processes in terms of immutable data structures and pure functions, and to express changes to the operating environment without using side effects. Clearly functional programming is broken.

Or maybe it’s you. Maybe, rather than relying on faith to make these conceptual thought frameworks do what you need from them, you could have thought about the concepts.