By the river

My home stands near to a bridge over the Avon, the same river that lends its name to Stratford-on-Avon. By walking to the end of my street then through the churchyard, I could stand over the river and watch it flow beneath me. But I didn’t. I elected to walk a few kilometres upstream to the grounds of the big house, and stand on that bridge instead. Walking is something I do, apparently. My phone tells me that I walk more than the average man, an achievement I can claim as my own.

It was raining, so I took an umbrella. In theory, I was listening to the patter of the rain on the umbrella, and watching the water flow through the reeds as it washed underneath me, then past my house and the church, past Shakespeare’s birthplace and onto Tewkesbury and the Severn. In fact I did nothing of the sort. The river and its reeds must have been there, and I was pointing in their direction. I was vaguely aware at one point in my half-hour stop of a car on the driveway, approaching me, stopping just short of where I was, then carrying on over the bridge and on to the road. I did not acknowledge the driver, and if that person acknowledged me I failed to notice it.

Of things that I saw, the one that sticks in my mind is the notification light on my phone. It is my beacon to the outside world, letting me know when somebody wants my attention. Twice I remember looking at it, and twice I remember seeing the blue blink that signifies contact via email. Twice I read this link to society at large, to discover that somebody wanted to sell me the world’s most powerful flashlight and that somebody else thought I would be interested in a walk-in bathtub. Isn’t technology a marvel?

Of things that I heard, probably the car had some sort of engine and tyres that rumbled as it made its way past, and presumably the dual carriageway a few hundred metres away was contributing some white noise. The only sound that I remember hearing is my internal monologue. It’s a toxic sound, reminding me as it does that I have no reason to be pleased about anything, but it is my constant companion. I struggle to hold back tears as it reminds me that my home life, my social life, and my work life, are all in flux at the moment, all out of my control, and the common thread running through them all is the person who lost that control. It goes on to tell me that anything else I might try is doomed to failure, too.

I consider using that phone again, not to look at the notification light, but to tell somebody how I’m feeling. Why would you want to do that, asks the voice? These people have their own things to deal with, why burden them with your problems too? Why spoil somebody’s evening by telling them that you’re upset, when they’ve done nothing to upset you? Why are you so selfish? You’re a grown-up, a man who walks more than the average, you’re supposed to look after yourself now. You’re not supposed to be a mess with a state-sponsored Citalopram habit.

I turn the phone around, thinking of taking a picture of the trees silhouetting the clouds, stained Sodium-orange by the streetlights of the nearest town. The screen remains resolutely black. Of course it did, you idiot, it’s much too dark to take a photo with that phone. You’re supposed to be some kind of expert at smartphones, don’t you know how their cameras work? I vaguely consider whether to fetch my SLR camera, but of course that’s been in a box for over a year and it’s doubtful the battery is even charged. Maybe it’s time to sell it on eBay, but of course you won’t get a good price for it and then it’ll get damaged in the post and the buyer will want a refund and you won’t even have kept that money. Why bother?

It took effort to walk the twenty minutes upstream to the other bridge. My voice told me that I would be just as useful to the world if I didn’t take the walk, if I had stayed in bed and waited until it was time to go to work tomorrow. Well, voice, I took everything you told me and made a few hundred words of blog post out of it, so I have done something after all. The joke’s on you.

Posted in whatevs | 2 Comments

Answer: none of them

A question programmers frequently ask when they’re considering career growth or personal learning is “which programming language should I learn next?”

Why would learning another programming language help? If you only know one programming language and it is provided by a single vendor, then learning another will decouple your success from theirs, but that might not be such a common situation. Well, a book like Seven Languages in Seven Weeks makes the point that it’s not about learning the language, but about learning the model and thought process enabled by using that language. OK, so why don’t I learn that model or thought process, using the tools that are already available to me, instead of having to add fighting unfamiliar syntax to the problem?

And if what I’m truly trying to do is to learn to think about problems in a different way, a week-long effort at dabbling in a side project isn’t going to change my way of thinking. Those years of learned processes, visualisations and analyses are going to take more than a couple of hours to dislodge. I’ve worked through Seven Languages, and the fact that I spent a couple of hours solving the Eight Queens Problem in Prolog (or in fact telling Prolog what a solution to Eight Queens looks like and letting it solve it) doesn’t mean I now think about any other software problem as if I’m using a logic programming tool, or even as if I have such a tool available. I’ve spent much longer than that studying and using the relational calculus and SQL, but don’t even think about every problem as if it should be a collection of tables in the third normal form.

It may be that it would be useful to learn something that isn’t a programming language, shock horror! It turns out that programming is an activity embedded in a socio-technical system comprising other activities, and you might need to know something about them: software security, testing (I think I can count on my noses the number of programmers I’ve met who haven’t responded to the phrase “equivalence partitioning” with a blank stare, and I wouldn’t use all of my noses), planning, business, marketing, ethics…I even wrote a whole book on the things programmers should know that aren’t programming.

And then there’s the thing that your customers, clients, colleagues, or victims are trying to do with the software. Learning something about that would make it easier to empathise with them, to evaluate your solutions in context, and to propose better ways of working and better ways for your software to enable their work. Rewriting your code in Elixir would…not do that so much.

Posted in advancement of the self | 1 Comment

I fix things for a living

Previously, on SICPers, I wrote that I make mistakes for a living. But making mistakes is no good if nobody’s cleaning up after them, so I also fix things. Whatever gets in my team’s way, it’s my responsibility as their lead to make sure that it’s no longer in their way.

Whether it’s a process we instituted that slows us down, some technical debt that gets in the way of new development, or some infrastructure not behaving itself, it needs to go, and I either need to clear it out or find somebody to do it for us. On my team I’ve been nicknamed “Mister Fix-It” as a result of my policy of getting rid of everybody else’s impediments before doing my own work, and we even have a Fix-It support queue just like our customer support queue. Except it’s not our customers I’m supporting, it’s my colleagues. Tickets in my queue range from “this script doesn’t handle this case and did the wrong thing when I tried to do a build” to “merge these three repos into a monorepo”. It doesn’t matter, it’s in our way, so it needs shifting.

Or does it? Is it, in fact, in the way, or are we solving the wrong problem? Some of the tickets in the Fix-It queue magically transmogrified from “Fix This” to “Measure This”: let’s discover where the problem really is before we solve something else.

Posted in Responsibility | Leave a comment

No, you can’t ignore politics

I wrote, a couple of years ago, about the fact that you can’t ignore ethics in software engineering. Your software is built for a reason, it’s used for a reason, you need to be aware of those reasons and whether you’re supporting or enabling them.

That goes for politics too. That hacker news declared this week to be politics-free week shows an immaturity and unprofessionalism that makes it a dangerous place to learn about making software from. Making software is the act of some people producing things for other people, it is inherently a political act. Choosing a framework or programming language is political. Attending a meet-up is political. Being paid is oh-so-political. Publishing your side project under an OSI-approved licence? The OSI is a political organisation, the courts that will determine whether the terms of your licence are binding are political, the officials of those courts are selected by a political process.

And, anyway, let’s have a look at the front page of hacker news now, roughly five days into their politics-free week.

What they probably mean is not “political stories are off-topic”, but that political stories that stray from the default politics are off-topic. Anything that doesn’t sound like I agree with it must be subjective, whereas things I agree with are objective. Just as the default narrative in society is often white, male and affluent, so it is those things in the Valley and neolibertarian. Anything else is just politics.

Posted in economics, Responsibility | Leave a comment

I make mistakes for a living

As a team lead, my job is to make all the mistakes on my team. I’m responsible for each one of them. I’m also responsible for deciding what we do about them, whether that’s to ensure that they can’t happen again or ignore them because they’re easy to recover from.

That time someone on my team built a feature that didn’t ship because of a bug? I could have made sure the requirements were clearer, or they had more access to the product owner, or a more useful design review, or a more useful code review, or a better test plan. Or all of those, or none of those, or something else. I could have – should have – got the problems out of their way.

They don’t make mistakes, I do.

Posted in Responsibility | Leave a comment

Learning about software freedom

On the front page of Hacker News at the moment is a post on The Three Software Freedoms. It does away with the Free Software Foundation’s Freedom Zero:

The freedom to run the program as you wish, for any purpose.

On the basis that it “is just silly I mean of course I can use the program as I wish.” Of course you can. Except, of course, that’s not in fact true, and because software companies are licensing – not selling – you software, you’re actually bound by their terms and the limitations they impose. Picking just one example, here are some snippits from the Apple Licensed Application End-User License Agreement, relevant to apps bought (sorry, licensed) through the iTunes App Store:

The Products transacted through the Service are licensed, not sold, to You for use only under the terms of this license

The licensor (“Application Provider”) reserves all rights not expressly granted to You.

This license granted to You for the Licensed Application by Application Provider is limited to a non-transferable license to use the Licensed Application on any iPhone or iPod touch that You own or control and as permitted by the Usage Rules set forth in Section 9.b. of the App Store Terms and Conditions (the “Usage Rules”). This license does not allow You to use the Licensed Application on any iPod touch or iPhone that You do not own or control, and You may not distribute or make the Licensed Application available over a network where it could be used by multiple devices at the same time. You may not rent, lease, lend, sell, redistribute or sublicense the Licensed Application. You may not copy (except as expressly permitted by this license and the Usage Rules), decompile, reverse engineer, disassemble, attempt to derive the source code of, modify, or create derivative works of the Licensed Application, any updates, or any part thereof (except as and only to the extent any foregoing restriction is prohibited by applicable law or to the extent as may be permitted by the licensing terms governing use of any open sourced components included with the Licensed Application).

If You breach this restriction, You may be subject to prosecution and damages.

Your rights under this license will terminate automatically without notice from the Application Provider if You fail to comply with any term(s) of this license. Upon termination of the license, You shall cease all use of the Licensed Application, and destroy all copies, full or partial, of the Licensed Application.

You agree that any Services contain proprietary content, information and material that is protected by applicable intellectual property and other laws, including but not limited to copyright, and that You will not use such proprietary content, information or materials in any way whatsoever except for permitted use of the Services. No portion of the Services may be reproduced in any form or by any means. You agree not to modify, rent, lease, loan, sell, distribute, or create derivative works based on the Services, in any manner, and You shall not exploit the Services in any unauthorized way whatsoever, including but not limited to, by trespass or burdening network capacity. You further agree not to use the Services in any manner to harass, abuse, stalk, threaten, defame or otherwise infringe or violate the rights of any other party, and that the Application Provider is not in any way responsible for any such use by You, nor for any harassing, threatening, defamatory, offensive or illegal messages or transmissions that You may receive as a result of using any of the Services.

You may not use or otherwise export or re-export the Licensed Application except as authorized by United States law and the laws of the jurisdiction in which the Licensed Application was obtained. In particular, but without limitation, the Licensed Application may not be exported or re-exported (a) into any U.S. embargoed countries or (b) to anyone on the U.S. Treasury Department’s list of Specially Designated Nationals or the U.S. Department of Commerce Denied Person’s List or Entity List. By using the Licensed Application, you represent and warrant that you are not located in any such country or on any such list. You also agree that you will not use these products for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture or production of nuclear, missiles, or chemical or biological weapons.

So don’t claim, through malice or ignorance, that Freedom Zero is not worth making explicit, and doesn’t need protecting.

Posted in freesoftware, Responsibility | 2 Comments

On the rhetorical cost of ownership

I’ve recently been talking about software engineering economics, in a very loose way, but so have other people. And now I understand that it’s annoying when people talk about it, and have decided to continue anyway. I’ve decided to continue because what I see is either inaccurate comparisons being made, or valid comparisons that have questionable applicability outside their immediate domain. The world of IT cost comparison is still run by marketing, not by operations.

Recently, I read Don’t Build Private Clouds. Subbu says that the sticker prices (up to $10k for a server that will last four years, vs. up to $1500 per month for a public cloud machine) should not be compared because there are additional costs to self-hosting:

  1. Engineering costs
  2. Network automation costs
  3. Loss of agility
  4. Opportunity costs

Fine, but what are those costs? Why do I not need to do any engineering or automation if I use a public cloud provider? If I do, what is that going to cost? What agility and opportunity do I lose by tying my infrastructure to any one cloud vendor, and what will that cost?

Subbu’s blog says that he is a “cloud helper”, and that goes a long way to explaining why we didn’t get a straight answer on the cost comparison. We’re not being told that cloud services are cheaper, instead we are being told of the Fear, Uncertainty and Doubt involved in choosing the less-favoured path.

Similarly, an IBM employee recently said that Macs are cheaper than PCs by up to $543 per user (remember that’s “up to”, not “as much as” – the lower bound given is $273). Let’s ignore the conflict of interest arising from Apple’s global partnership with IBM: IBM wouldn’t say that their partner systems are cheaper so that they could drum up interest in their partnership services, surely. Surely. I mean, this isn’t the IBM of 1984, is it?

What IBM’s VP says is that over a four-year lifetime, among employees who are given the choice of which platform and model computer they want, the Macs are cheaper. That’s of course a figure about which it’s possible to make realistic comparisons: given IBM’s approach to desktop support, IBM’s level of staffing, IBM’s applications, IBM’s approach to working, IBM’s budgeting for IT support operations, it’s cheaper for people who choose Macs to use Macs than for…well, it’s not clear, but it seems to be than for the “everybody else” bucket: not only people who chose PCs to use PCs, but people who weren’t given a choice to use PCs.

So before you make a textexpander macro for that link and insta-reply to anyone who uses the phrase “Apple Tax”, just how similar is your environment to IBM’s?

Posted in Business, economics | Leave a comment

Can’t you just…

Continuing the thoughts on vexing problems, one difficulty when it comes to discussing software is talking about the size of software. I’m not really talking about productivity metrics – good or bad – like source lines of code or function points, rather the fact that the complexity of a problem looks different depending on who’s doing the looking.

Sometimes, a problem that’s very simple from a business perspective can be incredibly complex technically. One product I worked on could be summarised very quickly: let people interact with marketing campaigns by sending and receiving messages on their mobiles. The small amount of logic between send and receive – allowing the campaigns to operate as quizzes, votes, or auctions – could be detailed on an index card.

But that simplicity was backed by a huge amount of technical complexity to make it work. “Can you just send this message to everyone who got the quiz question correct?” Well, yes, but as it’s a picture message we need to work out how to make it look good on the recipient phone, change it to fit those criteria, and then send it. What makes it look good – and indeed how we can get the information to make that decision – depends on the device, but also which network it’s on, and maybe whether it’s on pre-pay or post-pay and whether they use HTTP, WAP or e-mail to send messages to that device (which might be different from how they send to other devices on the same network). And even after we’ve gathered that information, it may be wrong as some devices claim to support image formats that they can’t render, or image sizes that they’ll actually reject or fail to display.

On the other hand, sometimes the business problem is a lot more complex than the technical problem. If you’re a mobile app developer, any length of problem definition about exciting disruptive apps can be reduced to “so you want to display data from a web service in a table view”.

And then at other times, “can’t you just” gets stymied from left field. Why yes, we could simply do that, and it would be good for the business, but this regulation/patent/staff shortage means we need to do something else.

Posted in Business | Leave a comment

On the business case for (or against) software

In the vexing problems, I dismissed the hard problems of computer science as being incidental to another problem: we can’t say what the value of our work is. That post contained plenty of questions, precisely because the subject is so unknown.

There are plenty of ways in which the “value” of something can be discussed, but let’s stick with economic value for the moment. Imagine you have the opportunity to write some software, but only if I pay you for it. And not even then, but only if you can make me reasonably confident that I will get some return; that having the work done is worth more than the cost of you doing the work.

A corner of the software industry just cowered and stuck its head in the sand. “#NoEstimates!”, they cry. This problem is hard, so why should we solve it?

That’s a really interesting perspective. You should come and talk about #NoEstimates at my conference. It’ll be on sometime, in a place, and there’ll be a flight that can get you there. Maybe. That’s enough information to be going on for it to be clear that you should commit to it, so I’ll see you there.

At the other end are the folks who would literally write whole books about the topic (and given that books are themselves literary, that “literally” is itself meant literally). I pulled a few likely titles off the shelf to see what they would tell me, and ended up with:

We have to wonder how some people can be certain that estimates are worthlessly inaccurate, while others such as Barry Boehm (the author of Software Engineering Economics and the COnstructive COst MOdel it describes) have built careers out of explaining how to do it well.

Here’s one hypothesis: Boehm is a charlatan, and the COCOMO doesn’t work at all. Let’s see whether there’s evidence for that.

A likely source is COCOMO evaluation and tailoring by Miyazaki and Mori from Fujitsu. The authors of that paper, applying the COCOMO (’81, not the later COCOMO II) model to a corpus of projects undertaken by Fujitsu, conclude:

The original COCOMO overestimates the effort required to develop software in our environment, but its tailoring methodology is applicable. […] The resulting model fits 68% of the projects with less than 20% relative error, after the deletion of two outliers.

That’s maybe short of brilliant – the median project effort in their paper seems to be over 10 person-years, so they’re saying “there’s around two chances in three that we can tell you what this project will cost to within twice a developer’s salary” – but it’s much more precise than “#NoEstimates!” and only quite a lot less accurate.

It might be interesting to track the application of COCOMO through time, and see whether a larger corpus of projects and refinement of the model has improved its accuracy and applicability. Unfortunately, that goal is in conflict with a general industry observation, noted in the introduction of the Fujitsu paper:

We have the impression that software people prefer to start from scratch rather than improving upon the work of others, which seems to be slowing down the progress in this field.

And indeed much of what was written in the sources I’ve been talking about here is no longer applied. All three of the books I mentioned up top talk about Earned Value (EV) Analysis, recording the actual rate of delivering on a project against its Planned Value (PV). For example, Hughes and Cotterell recommend plotting earned value using the “0/100 technique”:

where a task is assigned a value of zero until such time as it is completed when it is given a value of 100% of the budgeted value

Agilistas will recognise the resulting plot: it’s a burn down chart, where a story is not delivered until it has been accepted according to the definition of done, at which point it is completely delivered.

Ultimately somebody in the Agile community must have noticed that they accidentally borrowed a software engineering concept, because the burn down chart is now unfashionable.

If the team starts with a very full, prioritized, and estimated backlog with the expectation they will burn down all tasks to zero — does this not sound like a Waterfall mentality?

Oh noes, the dreaded W word! Run away from this thought, for it is a thought that somebody in the past has already considered!

And that is why I consider these problems to be vexing: when potential solutions are proposed, they are by necessity existing ideas from the old school that must be rejected in favour of…whatever is not those solutions. As Craftsmanship is post-Agile is post-Software Engineering is post-Crisis thinking, surely it will soon be time for post-Craftsmanship thinking. And then post-whatever-comes-next thinking. And we still won’t know how to compare expected value to expected cost.

Posted in Business, economics | 2 Comments

The Vexing Problems in Programming

I admit it, I’ve been on the internet for quite a while (I could tell you that my ICQ number is 95941970, but I haven’t logged in for years) and my habits haven’t changed. I still regularly get technology news from slashdot, and today was no exception. An interesting article was Here Be Dragons: The 7 most vexing problems in programming. Without wanting to spoil the article for you by giving away the punchline, there are indeed some frustratingly difficult problems mentioned: multithreading, security, encryption are among the list.

All of these problems are sideshows to what I see as one of the largest and most vexing issues in programming: the fundamental rule to business administration is that your income should be greater than your costs, but software makers still, by and large, don’t have a way to compare the expected value of their work to the expected cost of the work.

The problem in space

Different software teams – and individuals – do work in different contexts, and in different ways. The lone wolf micro-ISV is not the same as an individual contract developer. The in-house IT team does not have the same problems to solve as the shrinkwrapped software vendor, and those developing web services for public consumption have yet another context. The team with core hours all working in a single office is different from the distributed team inhabiting multiple time zones.

How much of this variety is essential, and how much is accidental? How much of it is relevant, when considering some intervention, process change, or technique? Consultants speaking at conferences (another context with its own similarities and differences from the others) don’t tend to talk about what researchers in fields such as psychology would recognise as “threats to validity” of their work, but given all of the ways in which software is made, we need to know whether some proposal applies to all of them, or to some of them, or whether it has been applied to some of them and might be applied to others, and what would assist or confound that application.

The problem in time

What are the accepted, tested and validated ways to identify who will be using and otherwise impacted by our software systems? To know whether they can use the system we propose, and whether it is the best system for their intended use? To ensure that our proposed software systems treat those people ethically? To understand the cost (to ourselves and to others) of constructing those systems? To deliver the systems to the people who will interact with them? To choose which people are or aren’t entitled to access? To build a representation of the problem to be solved, to validate that representation, to validate the solution against that representation?

Where an answer exists to those questions, what are the contexts in which it is valid and what are the threats to its validity? How has that answer been compared with other possibilities? How has it been confirmed? How has it been challenged? How can I find out about those confirmations and challenges? How can I find out about any alternatives? What techniques exist to weigh up those alternatives quantitively, rather than relying merely on the persuasiveness of the conference speakers promoting those solutions (and, by the way, the books/screencasts that describe the solutions)?

The lack of a problem

Why should I care? There’s enough money in software at the moment to mean that I don’t need to be any good at knowing what works or doesn’t work, I just need to get out there and sell some software. In the rare situation that I don’t make my money back, that’s just the market forces at work, and I can go and get a high-paying job somewhere while I lick my wounds, and pick another programming language/framework/platform/whatever it is that’s going to make my next attempt definitely succeed.

Clearly, this bottomless pit of money that arises from society’s unwavering faith in software and its ability to cure all ills is never going to run out. There’s no need to worry ever about whether we’re doing it right, because there’ll always be someone out there willing to pay for us to do it wrong. Life as a programmer is like some kind of socialist utopia where whether we’re making a valuable contribution or not, the rest of society is looking out for us.

That’s going to last forever, right?

Posted in Business, economics, Responsibility | 2 Comments