Reasoning about reasoning about software

Functional programmers like to claim that you can’t reason about mutable state programs. Some thoughts:

  • the first half of the book A Discipline of Programming by Edsger W. Dijkstra tells you how to do it. That half of the book is approximately 100 pages (the remainder of the book is worked examples).
  • object-oriented programming breaks a software system up into separate systems running miniature, message-driven programs as if on separate computers. Therefore the consideration of “mutable state” can be split in two: the state internal to the object and the state external to the object which sends messages to the object but is ignorant of its internals. If you can’t split the state that way, you have bad encapsulation.
  • The reasoning done about the external and internal behaviours had better match at the interface. Design by contract probably helps here.
  • Given a state S, an operation O can be defined as \(O(args \times S) \rightarrow (R \times S’)\), i.e. it returns a result R and updates the state to S’.
  • However, Bertrand Meyer introduced Command-Query Separation in the 1980s, so you only need to know \(O(args \times S) \rightarrow (R \times S)\) and \(O(args \times S) \rightarrow (\emptyset \times S’)\).
  • Various history “traces” can be considered equivalent and therefore a lot of knowledge about the historical state transitions elided, simplifying the reasoning. For example, given a well-designed stack, it is impossible to distinguish the history of stack.push(3); stack.pop(); stack.push(7) from stack.push(7).
  • Various operations on the state are irrelevant to the behaviour of an operation under consideration. In reasoning about the final operation in a = 3; b = 7; c = 9; stack.push(2) you do not need to consider the assignment operations (and indeed their presence may indicate a cohesion problem in your design).
  • The one remaining source of difficulty is aliasing; I do need to know about the elided operations in the sequence x = 7; *y = &x; ...; z=f(x). This is aliasing, not mutable state.
Posted in code-level, software-engineering | Tagged | Leave a comment

All the things

It’s been a long time since I had a side project, or one that didn’t get abandoned very early on. I tend to get sidetracked by other thoughts about computing, or think “while I’m doing this, I’m leaving that unsolved” so nothing gets very far.

In an attempt to address that, to clear all of the different thoughts I have about the matter of computing out of my head, organise them, identify conflicts, and prioritise what I work on, I spent this evening jotting down the big points and a brief abstract about each one. I’m hoping this will cut the Gordian knot by letting me see it all in one place and start to make choices.

The format I chose to represent this braindump is this personal Technology Radar, based on the Thoughtworks build-your-own tool. It seemed like a good place to see everything at once, and look for clusters or trends.

You’ll notice that almost everything in this radar is fairly old tech! That’s mostly a matter of taste, as I enjoy learning about things that were tried, what succeeded or failed, and what can be learnt from that to put to use today. I’m not good at novelty for novelty’s sake.

I expect to get some mileage (for my own benefit, you might like it too) out of expanding on some of the entries in this radar over a few more posts, so I’ve created a techradar category in this blog that you can filter on/out.

Posted in advancement of the self, documentation, learning, techradar | Leave a comment

Against our values as a company

I’m going to pick on Patreon, not because they’re special, but because they’re typical. Here is a quote from a blog post in which they Back-pedalled a change to their pricing structure:

We overstepped our bounds and injected ourselves into that relationship, against our core belief as a business.

What I think they want us to hear is “this was a surprising event that normally wouldn’t happen because it’s important to us that other things happen instead”. What I actually hear is that management at companies like this is at best asleep at the wheel, at worst Github/Uber levels of arrogant and ignorant. Either way, new management is needed.

It’s fine to mistakes. It’s fine for someone acting in what they believe to be the best way aligned with their company’s goals and mission to do something out of step, that gets detected and corrected. That’s good management: trust, but verify. On the other hand, this is not an intern sending out an off-tone tweet. For a whole organisation to pivot its business model, create the marketing message, release the marketing message, and schedule the implementation of the new model…either management were involved and didn’t notice or care how broken it all was, or management are not involved and the whole company works around them.

When you want to sweep bad actions under rug of misalignment with your core belief as a business, think about what it means that your business isn’t taking your core belief as a business on board.

Posted in Uncategorized | Leave a comment

Falsehoods programmers believe about programming

  • There is no ethical impact of my job; I build technological systems and it’s up to others how they use them.
  • Software is a purely technical discipline.
  • There is some innate affinity for computer programming which you must be born with, and cannot be taught.
  • Allowing people who are unlike me to program can only be achieved by “lowering the bar”.
  • Compiled languages are always faster.
  • floating point calculations will introduce non-deterministic errors into numerical results.
  • OK, they introduce some errors into numerical results.
  • alright I understand that floating point calculations are imprecise not inaccurate, Mister Pedantic Blog Author, but I cannot know what that imprecision is.
  • at least the outcome of integer maths is always defined.
  • fine, it’s not defined. But whatever it was, the result of doing arithmetic on two numbers that each fit in a data register itself fits in a data register.
  • every computer on sale today (2017) uses two’s complement notation for negative numbers.
  • every computer on sale today uses a register width that’s a multiple of eight bits.
  • the bug isn’t in my code.
  • the bug isn’t in the library.
  • the bug isn’t in the operating system.
  • the bug isn’t in the compiler.
  • the bug isn’t in the kernel.
  • the bug isn’t in the hardware.
  • bug-free computer hardware is completely deterministic.
  • the lines on the hardware bus/bridge are always either at the voltage that represents 0 or the voltage that represents 1.
  • if my tests cover 100% of my lines of code then I have complete coverage.
  • if my tests cover 100% of my statements then I have complete coverage.
  • if my tests cover 100% of my conditions then I have complete coverage.
  • if I have complete test coverage then I have no bugs.
  • if I have complete test coverage then I do not need a type system.
  • if I have a type system then I do not need complete test coverage.
  • no hacker will target my system.
  • information security is about protecting systems from hackers.
  • if the problem is SQL Injection, then the solution is to replace SQL; NoSQL Injection is impossible.
  • my project is a special snowflake; I can reject that technique I read about without considering it.
  • my project is much like that unicorn startup or public company’s project; I can adopt that technique I read about without considering it.
  • people who do not use my language, technique, framework, tool, methodology, paradigm or other practice do not get it.
  • any metaprogramming expansion will resolve in reasonable time.
  • any type annotation will resolve in reasonable time.
  • OK, well at least any regular expression will resolve in reasonable time.
  • can you at least, please, allow that regular expressions are regular?

I’m sure there must be more.

Update The following were added later; where they were supplied by others there is citing link. There are also good examples in the comments.

  • You need a computer science degree to be a good programmer.
  • A computer science course contains nothing useful to programmers.
  • Functional Programming is a silver bullet.
  • Rust is a silver bullet.
  • There is a silver bullet.
  • There can be no silver bullet.
  • Rewriting a working software system is a good idea.
  • I can write a large system in a memory unsafe language without introducing vulnerabilities.
  • I can write a large system in a memory safe language without introducing vulnerabilities.
  • Software is an engineering discipline.
  • Software is a scientific discipline.
  • Discourse on a topic is furthered by commenting on how I already knew a fact that was stated.
  • A falsehood about programming has no value unless the author of the falsehood provides supporting evidence to my satisfaction.
Posted in code-level | 14 Comments

Don’t like a new way of working? Just point out the absurdity of suggesting that the old way was broken:

Somehow, the microservices folks have failed to notice all that software that was in fact delivered as monoliths.

What the Rust Evangelism Strike Force doesn’t realise is that we’ve spent decades successfully building C programs that don’t dereference the NULL pointer.

This is a sort of “[C|Monoliths] considered harmless” statement. Yes, it’s possible to do it that way, but that doesn’t mean that there aren’t problems, or at least trade-offs. “C considered harmless” is as untrue and unhelpful as “C considered harmful”; what we want is “C considered alongside alternatives”.

Posted on by Graham | Leave a comment

An unhelpful distinction

Object-Oriented Programming is quite simple: it’s just choosing what function to run based on the parameters to the function (whether through method sending like Smalltalk, polymorphic lookup like CLOS, or table searching like C++: usually pattern-matching like Haskell would be excluded here).

Object-Oriented Analysis and Design is the thing where we represent our problem domain, and our solution, as a collection of objects, often categorised into classes, where the classes have particular relationships, properties and behaviours. And that is the thing that programmers often struggle with.

Whether it’s hard because OOA/D is hard, or because the problem domains are hard, or because OOA/D is not applicable to the problem domains, is not addressed here.

Posted in OOP | Leave a comment

GNUstep is more important now than ever

In creating a pull request for GNUstep-base, the Free Software implementation of the Foundation library from Objective-C, I realised that if there was ever a time for GNUstep, now is it.

Although GNUstep may have been envisaged as an official desktop for the GNU system – a role now fulfilled by GNOME – it has always had another position as an alternative deployment venue for OpenStep, and then Cocoa, codebases. People have done this to obtain cross-platform support (I know of a test tool that was built for Mac, Linux and Windows using GNUstep), to take advantage of better, or cheaper, server deployment on Linux, or to act as an ‘escape lane’, a place to take your code if your platform vendor changes direction.

This ability to hedge against a vendor’s whims has come in handy before: in 2001, when WebObjects 5 ditched Objective-C support in favour of Java, Objective-C WebObjects applications could be deployed on modern platforms through GNUstep Web or on legacy WebObjects 4 systems. It may come in handy again.

Even if Apple do, indefinitely, support Objective-C, the plain fact is that their community does not. Conference talks, blog posts, and community discussion now take place using Swift, which for those with an interest in Objective-C or those supporting existing code makes it harder to gain help or even to feel membership.

A vendor-independent association of Objective-C developers all interested in giving their code a comfortable, stable home is now more important than ever. May all your messages have receivers.

Posted in gnustep | Leave a comment

On the lesser presentations

Advice on presentations – including that given on this blog, is often geared toward the “showbusiness” presentation. We’re usually talking about the big conference talk or product launch, where you can afford to put in the time to make a good, slick performance: a few days of preparation for a half-hour talk is not unheard of.

Not every presentation fits that mould. There are plenty where putting so much time into the presentation would be harmful, but where is the guidance on constructing those presentations?

The minor event

If you spent a few days preparing for a sprint-end demo, or reporting back to your team on some study you did, you’d significantly harm your productivity on the rest of your job. In these contexts, you want to spend a small amount of time building your talk, and you still want to put on a good show: to make your team and your stakeholders feel happy about the work you did, or to make the case persuasively for the tool or technique you studied.

As such, in these cases I still build an outline for my talk outside of the presentation software, and construct my slides to follow the outline. I’ve used OmniOutliner in the past, I use emacs org-mode now, you could use a list of bullets in MarkDown or a pen and a sheet of paper. It doesn’t matter, what matters is that you get what you want to say structured in one place, and the slides that support the presentation done separately.

I try to keep these slides text-free, particularly if the presentation is short, so that people don’t get distracted reading them. If I’m reporting on progress, then screenshots of some progress dashboard make for quickly-constructed slides. My current team shows its burndown every sprint end, that’s a quick screencapture that tells the story of the last two weeks. If there’s some headline figure (26 stories delivered; 80% of MVP scope complete; 2 new people on the team) then a slide containing that number makes for a good backdrop to talking about the subject.

The recurrent deck

The antithesis of the conference keynote presentation style, the recurrent deck is a collection of slides you’ll use over and over again. Your approach to integrating with third-party APIs, your software architecture philosophy, your business goals for 2018…you’ll need to present these over and over again in different contexts.

More to the point, other people will want to present them, too: someone in sales will answer a question about integration by using your integration slides. Your department director will present your team’s goals in the context of her department’s goals. And this works the other way: you can use your CEO’s slide on product strategy to help situate your team goals.

So throw out everything you learned about crafting the slides to fit the story. What you’re doing here is coming up with a reusable visual that can support any story related to the same information. I try to make these slides as information-rich as possible, though still diagrammatic rather than textual to avoid the presentation failure mode of reading out the slide. My current diagram tool is Lucidchart, as it’s my company’s standard, I’ve used OmniGraffle and dia too. Whatever tool, follow the house style (e.g. colour schemes, fonts, iconography) so that when you mash up your slides and your CEO’s slides, it still looks like a coherent presentation.

I try to make each slide self-contained, because I or someone else might take one to use in a different presentation so a single idea shouldn’t need a six-slide reveal and a colleague will find it harder to reuse the slide if it isn’t self-explanatory.

A frequent anti-pattern in slide design is to include the “page number” on the slide: not only is that information useless in a presentation, the only likely outcome is a continuity error when you drag a few slides from different sources to throw a talk together and don’t renumber them. Or worse, can’t: I’ve been given slides before that are screenshots of whatever slide was originally built, so the number is part of a bitmap.

Good reusable slide libraries will also be a boon in quickly constructing the minor event presentations: “we did this because that” can be told with one novel part (we did this) and one appeal to the library (because that).

Posted in performance, Talk | Leave a comment

I frequently see posts/articles/screeds asking why people don’t contribute to open source. If it’s important that recipients of open source software contribute upstream, and you are angry when they don’t, why use licences like MIT, Apache, GPL or BSD that don’t require upstream collaboration?

Back in the day, Apple released their public source code under version 1 of the Apple Public Source Licence, which required users who changed the source to fill in a form notifying Apple of their changes. You could do the same, and not be angry.

Posted on by Graham | 1 Comment

The Atoms of Programming

In the world of physics, there are many different models that can be used, though typically each of them has different applicability to different contexts. At the small scale, quantum physics is a very useful model, Newtonian physics will yield evidently incorrect predictions so is less valuable. Where a Newtonian model gives sufficiently accurate results, it’s a lot easier to work with than quantum or relativistic mechanics.

All of these models are used to describe the same universe – the same underlying collection of observations that can systematically be categorised, modelled and predicted.

Physical science (or experimental philosophy) does not work in the same way as computational philosophy. There are physical realisations of computational systems, typically manifested as electronic systems or pencil-and-paper simulations. But the software, the abstract configurations of ideas that run on those systems, exist in entirely separate space and are merely (though the fact that this is possible is immensely powerful) translated into the electronic or paper medium.

Of course one model for the software system is to abstract the electronic: to consider the movement of electrons as the presence of voltages at terminals; to group terminals as registers or busses; to further abstract this range of voltages as 0 and that range as 1. And indeed that model frequently is useful.

Frequently, that model is not useful. And the great thing is that we get to select from a panoply of other models, at some small or large remove from the physical model. We can use these models separately, or simultaneously. You can think of a software system as a network of messages passed between independent objects, as a flow of data through transformers, as a sequence of state changes, as a graph of single-argument functions, as something else, or as a combination of these things. Each is useful, each is powerful, all are applicable.

Sometimes, I can use these models to make decisions about representing the logical structure of these systems, transforming a concept into a representation that’s valid in the model. If I have a statement in a mathematical formulation of my problem, “for any a drawn from the set of Articles there exists a p drawn from the set of People such that p is the principal author of a” then I can build a function, or a method, or a query, or a predicate, or a procedure, or a subroutine, or a spreadsheet cell, or a process, that given an article will yield exactly one person who is the principal author of that article.

Sometimes, I use the models to avoid the conceptual or logical layers at all and express my problem as if it is a software solution. Object-oriented analysis and design, data flow modelling, and other techniques can be used to represent a logical model, or they can be used to bash the problem straight into a physical model without having thought about the problem in the abstract. “Shut up and code” is an extreme example of this approach, in which the physical model is realised without any attempt to tie it to a logical or conceptual design. I’ll know correct when I see it.

I don’t see a lot of value in collecting programming languages. I can’t count the number of different programming languages I’ve used, and many of them are entirely similar. C and JavaScript both have sequences of expressions that are built into statements that are built into procedures. Both let me build aggregations of data and procedures that either let me organise sequential programs, represent objects, represent functions, or do something else.

But collecting the models, the different representations of systems conceptually that can be expressed as software, sometimes called paradigms: this is very interesting. This is what lets me think about representing problems in different ways, and come up with efficient (conceptually or physically) solutions.

More paradigms, please.

Posted in learning, philosophy after a fashion, software-engineering | 15 Comments