On my newer competence

This time last year, I evaluated myself against the programmer competency matrix. So where am I one turn around the daystar later? I have to admit that this was mainly because I was jet lagged in a hotel room in Seattle and had nothing else to do last year :-), but nonetheless it’s time to reflect on how things have gone.

Last year’s goals

I wrote:

it looks like my priorities for research goals over the upcoming year or so should be:

  1. Improve breadth and depth of API knowledge
  2. Investigate systems decomposition of a smartphone/desktop/internet system.
  3. Learn how to create and support DSLs
  4. Write and publish improved developer support tools

So how did that go?

  1. I improved here. I didn’t have a project for doing this, I just used the APIs some more. Things that I previously remembered but needed to look up I now automatically get right, and some things I didn’t know about I now do. There’ll always be room for more.
  2. My work on GNUstep web is only part of this goal.
  3. I haven’t done anything with DSLs, as I haven’t found a need for them in the last year. Maybe that’s because I don’t understand where they can be used :). Part of my GNUstep web project will include rules engines a la Direct 2 Web, so I guess I’ll have to tackle this soon.
  4. Again, in hand with the GSW project. I’ve already published a code generation tool. Some of the things I learned about at NSConference were very inspirational.

What about this year?

I’ll just dive into the matrix. If nothing’s changed from last year you won’t see any comments.

Computer Science

  • Data Structures: still level 1. I’ve learned a little more about level 2 aspects.
  • Algorithms: level 2.
  • Systems programming: level 3, if you’re willing to accept that block-diagram-level knowledge counts as knowledge on the level chart :)

Software Engineering

  • Source Control: level 3.
  • Build Automation: level 3.
  • Automated Testing: level 3.

Programming

  • Problem Decomposition: level 3.
  • Systems Decomposition: level 3. As I said earlier, I made this a specific goal of the last year, and I believe it’s paid off.
  • Communication: level 3. I’ve also done what I set out to do last year, and dialled back on the commitments to conferences and books.
  • Code Organisation Within a File: level 3. As I said last year, this is subjective.
  • Code Organisation Across Files: level 3.
  • Source Tree Organisation: level 3. For these two I’ve decided it’s what you see in the IDE (Xcode, AppCode and Eclipse are my regular haunts) that really matters, though I try to make the filesystem layout match that.
  • Code Readability: level 3. I’m not quite that bastard who rejects code because it doesn’t conform to the standard; though I used to be :-(
  • Defensive Coding: level 2. Still don’t have a “library” for doing that (I’ve written an iOS security library that’s internal to O2, but that’s not what this is about.)
  • Error Handling: level 3 (with s/xception/rror/, because we appreciate that error-handling should be sympathetic to the platform/framework style, just like other code.)
  • IDE: level 3. It was mainly working with AppCode that spurred me into investigating ways to automate IDE functions.
  • API: still going to claim level 1, because I’d say it’s for other people to decide whether they think I have sufficiently “vast and in-depth knowledge”.
  • Frameworks: level 2.
  • Requirements: level 3.
  • Scripting: level 3 (though this improvement only just scraped in this year.)
  • Database: level 1. I’m starting to see more projects where a better knowledge of databases is required, if one of those lands in front of me I’d better improve at knowing databases. However as I said last year it’s the kind of thing where I can use an ORM to get it mostly correct, then investigate whether any issues remain that require deeper inspection.
  • Experience

    I skipped this section last year, but just for kicks I’ll go through it this time. No comparisons with last year, and if you can’t work out why you’re not allowed higher than Level 0 on any entry in the matrix ;-).

    • Languages with Professional Experience: level 2. Functional programming I have worked with includes LISP, C++ and Java (yes, that Java).
    • Platforms with Professional Experience: heh, this depends as I said in a recent talk on how you define “Unix”. Conservatively I could say Unix, Mac, iOS and Java: level 1.
    • Years of Professional Experience: not that I think this matters, but level 2.
    • Domain Knowledge: wait, which domain? In the world of telco (where I currently spend my time) I’m a complete noob. In fact I’ve spent most of my career as a complete noob. Whether that’s good or bad, and something I should or would want to change, is open for debate. Maybe that’s another blog post, later.

    Yeah, having written that section I now understand why I left it out last time.

    Knowledge

    • Tool Knowledge: still level 2. I’ve written and published some codeine scripts, not sure it’s enough to qualify at level 3.
    • Languages Exposed to: still level 2. I did some concurrent programming in Objective-C, but no Erlang, Oz or Prolog. OK, I have done some Prolog tutorials, and Oz is the operating system on my Cambridge Z88 but I’m going to assume that it’s a different Oz.
    • Codebase knowledge: hard to gauge. We have about 100 repos on my team, some I’m not working on, some I collaborate on, some I’m the sole author. We’ll call this one unknowable for now.
    • Knowledge of Upcoming Technologies: level 3.
  • Platform Internals: level 2.
  • Books: still level 1, but I’ve written two now. I don’t imagine writing any more, there must be a better way to share knowledge.
  • Blogs: level 3.
  • This year’s goals

    Some of the things I planned for last year still need to be done:

    1. DSLs
    2. Developer tools

    To which I’ll add some things that, probably, don’t directly follow from the competency matrix, but which are nonetheless goals of mine.

    1. More do, less talk. A side-effect of being “the security guy” in my corner of the industry is that I do a lot of talking about security in other peoples’ products, and not much making products. It’s time to find a product I believe in, and own the crap out of it. That means investing my time in it long-term, not diving in and out like a consultant. Consultants are great and consultancy is great: but I want to build something.
    2. Mentor new developers. A recurring theme from my discussions in Appsterdam and at NSConf is that while I can certainly do a lot to become a better programmer, that won’t scale as well as making a load of other people better programmers. If I can help at least one person become a better developer than I am, I’ll improve my own abilities along the way and then there’ll be two people better than I am.

    That will, I think, do for now.

    More about the privacy pledge

    Plenty of you have seen—and indeed signed— the App Makers’ Privacy Pledge on GitHub. If you haven’t, but after reading it are interested, see the instructions in the project README.

    It’s great to see so many app makers taking an interest in this issue, and the main goal of the pledge is to raise awareness of app privacy concerns: awareness among developers that this is something to take seriously, and awareness among our customers that there are developers committed to respecting their identities and their data.

    But awareness is useless if not followed through, so we need to do more. We need materials that developers can refer to: the GSM Association have good guidelines on app practices. We need actionable tasks that developers can implement right away, like Matt Gemmell’s hashing guide for social apps. We need sample code and libraries that developers can rely on. We need data lawyers to explain what the current regulations are, and what’s coming down the pipe. We need to convince the industry and the governments that we can regulate our own actions. We need the ability to audit our apps and determine whether they’re privacy-preserving. We need to be able to demonstrate to customers what we’ve done, and explain why that’s a good thing. We need to earn customer trust.

    So there’s a lot to do, and the pledge is only the start. It’s off to a good start, but there’s still a long way to go.

    On advertising’s place in the tech industry

    Dave Winer said the way is open for a non-ad-supported tech sector:

    The tech industry has been absorbed by the ad industry, and vice versa.
    However, there is, imho, still room for a tech industry that is not merged with the ad industry.
    In fact, if we want to have a tech industry at all, we’d better invest in the “other” one, because advertising isn’t much to bet on long-term. Seriously.

    Daniel Jalkut went further, saying that ads are on the decline:

    I’ll take this a step further: advertising is on the way out. Technology loathes a middle-man, and advertising as an industry is the king of all middle-men. The purpose of advertising is to connect customers with companies, so as to facilitate a transfer of money in exchange for goods or services. As time goes by, customers and companies will be more and more capable of achieving this on their own.

    I agree with Dave, but not so much with Daniel. As I was writing this post, WPP (the world’s largest ad company) were on the radio announcing a 20% increase in profits worldwide, with an increase in the fraction of revenue they get from digital adverts up to 30%. WPP say that concerns of disintermediation (i.e. the removal of middle-men Daniel talks about) are unfounded. The internet has produced its own new middle men, particularly Google and Facebook. “Traditional” advertisers such as WPP are profiting from advertising with those companies.

    Ads can be on the way out, but if that’s a world we want to see, it’s up to us app makers to do something about it, it won’t just happen automatically. Why us? Because the future involves even more mobile internet, and that means that we are in a privileged position to control what customers of the future are looking at. If we want them to look at something other than adverts, we need to put something that isn’t an advert and is worth looking at on their screen.

    That’s a world in which many apps will need a very different business model than today’s. There are a lot—particularly on the web and on Android—of apps that are free to users and generate revenue from ads. Those would need to be funded in different ways: I don’t think that the freemium model of “pay to remove ads” will work. It doesn’t solve the goal of reducing ads (every user still sees the ad version at least once), for a start. Also, conversion rates will likely be low as people can “put up with” adverts.

    Apps that still want to rely on viral growth still need a really low barrier to entry for new users, but need to change why and how many people convert to a paid version. Companies like Evernote and Dropbox have a model that appears to work for them, where paying customers get extra features/capacity over free users, and provide enough income to support the cost of providing the free product. And then of course there’s the shareware model that’s worked for decades: where the store you’re selling in allows you to do that.

    But not all products need to be promoted virally. You can just sell something on the promise that it solves a problem, in the same way that I made a living selling solutions to app security problems, and the Omni Group make even more money selling the solution to the problem of remembering all the stuff you need to do.

    You can also sell the promise that people can come back later: that is, you make money by letting customers stick to you. Look at iTunes Match. For some amount of money, Apple manages your music library. Then, next year, they’ll ask you whether you want to carry on doing that. You get the choice between pulling all your music back down and looking after it yourself again, or you can pay Apple once more and they’ll carry on doing it for a year. Apple are betting on there being enough people who come back every year to make this worthwhile.

    So, to conclude, app makers who want to see an ad-free world are well placed to make that happen: doing so may start with an uncomfortable review of the way our apps are going to make money. In the short term, it’ll mean having an expensive fully-customer-funded app competing in a world of cheap ad-supported apps. But in the longer term it could result in better, more transparent relationships with our customers who aren’t worried about what data we’re selling to advertisers.

    On privacy, hashing, and your customers

    I’ve talked before about not being a dick when it comes to dealing with private data and personally-identifying information. It seems events have conspired to make it worth diving into some more detail.

    Only collect data you need to collect (and have asked for)

    There’s plenty of information on the iPhone ripe for the taking, as fellow iOS security boffin Nicolas Seriot discussed in his Black Hat paper. You can access a lot of this data without prompting the user: should you?

    Probably not: that would mean being a dick. Think about the following questions.

    • Have I made it clear to my customers that we need this data?
    • Have I already given my customers the choice to decline access to the data?
    • Is it obvious to my customer, from the way our product works, that the product will need this data to function?

    If the answer is “no” to any of these, then you should consider gathering the data to be a risky business, and the act of a dick. By the way, you’ll notice that I call your subscribers/licensees “your customers” not “the users”; try doing the same in your own discussions of how your product behaves. Particularly when talking to your investors.

    Should you require a long-form version of that discussion, there’s plenty more detail on appropriate handling of customer privacy in the GSMA’s privacy guidelines for mobile app developers.

    Only keep data you need to keep

    Paraphrasing Taligent: There is no data more secure than no data. If you need to perform an operation on some data but don’t need to store the inputs, just throw the data away. As an example: if you need to deliver a message, you don’t need to keep the content after it’s delivered.

    Hash things where that’s an option

    If you need to understand associations between facts, but don’t need to be able to read the facts themselves, you can store a one-way hash of the fact so that you can trace the associations anonymously.

    As an example, imagine that you direct customers to an affiliate website to buy some product. The affiliates then send the customers back to you to handle the purchase. This means you probably want to track the customer’s visit to your affiliate and back into your purchase system, so that you know who to charge for what and to get feedback on how your campaigns are going. You could just send the affiliate your customer’s email address:

    X-Customer-Identifier: iamleeg%40gmail.com

    But now everybody who can see the traffic – including the affiliate and their partners – can see your customer’s email address. That’s oversharing, or “being a dick” in the local parlance.

    So you might think to hash the email address using a function like SHA1; you can track the same hash in and out of the affiliate’s site, but the outsiders can’t see the real data.

    X-Customer-Identifier: 028271ebf0e9915b1b0af08b297d3cdbcf290e3c

    We still have a couple of problems though. Anyone who can see this hash can take some guesses at what the content might be: they don’t need to reverse the hash, just figure out what it might contain and have a go at that. For example if someone knows you have a user called ‘iamleeg’ they might try generating hashes of emails at various providers with that same username until they hit on the gmail address as a match.

    Another issue is that if multiple affiliates all partner with the same third business, that business can match the same hash across those affiliate sites and build up an aggregated view of that customer’s behaviour. For example, imagine that a few of your affiliates all use an analytics company called “Slurry” to track use of their websites. Slurry can see the same customer being passed by you to all of those sites.

    So an additional step is to append a different random value called a salt to the data before you hash it in each context. Then the same data seen in different contexts cannot be associated, and it becomes harder to precompute a table of guesses at the meaning of each hash. So, let’s say that for one site you send the hash of “sdfugyfwevojnicsjno” + email. Then the header looks like:

    X-Customer-Identifier: 22269bdc5bbe4473454ea9ac9b14554ae841fcf3

    [OK, I admit I’m cheating in this case just to demonstrate the progressive improvement: in fact in the example above you could hash the user’s current login session identifier and send that, so that you can see purchases coming from a particular session and no-one else can track the same customer on the same site over time.]

    N.B. I previously discussed why Apple are making a similar change with device identifiers.

    But we’re a startup, we can’t afford this stuff

    Startups are all about iterating quickly, finding problems and fixing them or changing strategy, right? The old pivot/persevere choice? Validated learning? OK, tell me this: why doesn’t that apply to security or privacy?

    I would say that it’s fine for a startup to release a first version that covers the following minimum requirements (something I call “Just Barely Good Enough” security):

    • Legal obligations to your customers in whatever country those countries (and your data) reside
    • Standard security practices such as mitigating the OWASP top ten or OWASP mobile top ten
    • Not being a dick

    In the O2 Labs I’ve been working with experts from various groups – legal, OFCOM compliance, IT security – to draw up checklists covering all of the above. Covering the baseline security won’t mean building the thing then throwing it at a pen tester to laugh at all the problems: it will mean going through the checklist. That can even be done while we’re planning the product.

    Now, as with everything else in both product engineering and in running a startup, it’s time to measure, optimise and iterate. Do changes to your product change its conformance with the checklist issues? Are your customers telling you that something else you didn’t think of is important? Are you detecting intrusions that existing countermeasures don’t defend against? Did the law change? Measure those things, change your security posture, iterate: use the metrics to ensure that you’re pulling in the correct direction.

    I suppose if I were willing to spend the time, I could package the above up as “Lean Security” and sell a 300-page book. But for now, this blog post will do. Try not to be a dick; check that you’re not being a dick; be less of a dick.

    On standards in free software engineering

    I have previously written on the economics of software insecurity, and I quote a couple of paragraphs from that post below:

    One option that is not fully explored in the book, but which I believe could be worth exploring, is this: development of critical infrastructure software could be taken away from the free market.

    Now the size of even the U.S. government IT budget probably isn’t sufficient to completely fund a bunch of infrastructure developers, but there are other options. Rice correctly notes the existence of not-for-profit software development organisations (particularly the Open Source Initiative and Free Software Foundation), and discusses the benefits and drawbacks of the open source model as it applies to commercial software. He does not explore the possibility that charity development organisations could withdraw from market competition, and focus on engineering practice, quality and security without feature parity or first-to-market speed.

    Today I was re-reading Free Software, Free Society by Richard Stallman, a collection of his essays and speeches on topics including copyleft, the GNU system and General Public Licence. In thinking about this book, I went wandering back to the idea of non-commercial driver for good quality software.

    I am now convinced that the Free Software Foundation should be investigating, researching and promoting standards, practices and quality in software construction.

    The principal immediate benefit the FSF would gain is in terms of visibility and support. Everywhere that software is used—public, private and academic sectors—organisations are interested in finding out ways to improve quality, reliability, deliverability: in other words, the success of their software. An entity that could offer to evaluate and report on whether particular techniques are feasible and offer improvement—in return for funding and staffing the production of their sought-after free software—would be welcomed and would be put to good use.

    The FSF is well-placed to achieve this goal, because all of its output is copyleft. A large problem with analysing the success or otherwise of development practices is that the outputs are proprietary: not just the code, but the project documentation, meeting minutes and so forth. With an FSF project everything is (or should be) freely available so inspecting how a project was run, what the developers did and—crucially—whether users are happy with the end result is much easier. Conclusions should be reproducible because everybody can see everything that went on.

    Notice that in this scheme, relationships between the FSF and other (proprietary, open source, whatever) organisations are mutually beneficial, not antagonistic as is often either actually the case or just assumed. The benefits seen by external parties are the improvements in process and technique; benefits that all developers can make use of. The discussion moves away from free vs. fettered, and becomes making the field of software engineering better for everyone.

    Incidentally, such a focus would also put free software at the forefront of discussions on software quality and deliverability. This would be something of a coup for free software, which is often associated with chaotic management, lack of road maps, and paucity of documentation and support. OK, the FSF wants people to consider freedom as a value in itself, but there’s nothing wrong with ensuring that free software is the best software too, surely?

    Want to hire iamleeg?

    Well, that was fun. For nearly a year I’ve been running Fuzzy Aliens, a consultancy for app developers to help get security and privacy requirements correct, reducing the burden on the users. This came after a year of doing the same as a contractor, and a longer period of helping out via conference talks, a book on the topic, podcasts and so on. I’ve even been helping the public to understand the computer industry at museums.

    Everything changes, and Fuzzy Aliens is no exception. While I’ve been able to help out on some really interesting projects, for everyone from indies to banks, there hasn’t been enough of this work to pay the bills. I’ve spoken with a number of people on this, and have heard a range of opinions on why there isn’t enough mobile app security work out there to keep one person employed. My own reflection on the year leads me to these items as the main culprits:

    • There isn’t a high risk to developers associated with getting security wrong;
    • Avoiding certain behaviour to improve security can mean losing out to competitors who don’t feel the need to comply;
    • The changes I want to make to the industry won’t come from a one-person company; and
    • I haven’t done a great job of communicating the benefits of app security to potential customers.

    Some people say things like Fuzzy Aliens is “too early”, or that the industry “isn’t ready” for such a service: those are actually indications that I haven’t made the industry ready: in other words, that I didn’t get the advantages across well enough. Anyway, the end results are that I can and will learn from Fuzzy Aliens, and that I still want to make the world a better place. I will be doing so through the medium of salaried employment. In other words, you can give me a job (assuming you want to). The timeline is this:

    • The next month or so will be my Time of Searching. If you think you might want to hire me, get in touch and arrange an interview during August or early September.
    • Next will come my Time of Changing. Fuzzy Aliens will still offer consultancy for a very short while, so if you have been sitting on the fence about getting some security expertise on your app, now is the time to do it. But this will be when I research the things I’ll need to know for…
    • whatever it is that comes next.

    What do I want to do?

    Well, of course my main areas of experience are in applications and tools for UNIX platforms—particularly Mac OS X and iOS—and application security, and I plan to continue in that field. A former manager of mine described me thus on LinkedIn:

    Graham is so much more than a highly competent software engineer. A restless “information scout” – finding it, making sense of it, bearing on a problem at hand or forging a compelling vision. Able to move effortlessly between “big picture” and an obscure detail. Highly capable relationships builder, engaging speaker and persuasive technology evangelist. Extremely fast learner. Able to use all those qualities very effectively to achieve ambitious goals.

    Those skills can best be applied strategically I think: so it’s time to become a senior/chief technologist, technology evangelist, technical strategy officer or developer manager. That’s the kind of thing I’ll be looking for, or for an opportunity that can lead to it. I want to spend an appreciable amount of time supporting a product or community that’s worth supporting: much as I’ve been doing for the last few years with the Cocoa community.

    Training and mentoring would also be good things for me to do, I think. My video training course on unit testing seems to have been well-received, and of course I spent a lot of my consulting time on helping developers, project managers and C*Os to understand security issues in terms relevant to their needs.

    Where do I want to do it?

    Location is somewhat important, though obviously with a couple of years’ experience at telecommuting I’m comfortable with remote working too. The roles I’ve described above, which depend as much on relationships as on sitting at a computer, may be best suited by split working.

    My first choice preference for the location of my desk is a large subset of the south of the UK, bounded by Weston-Super-Mare and Lyme Regis to the west, Gloucester and Oxford to the north, Reading and Chichester to the east and the water to the south (though not the Solent: IoW is fine). Notice that London is outside this area: having worked for both employers and clients in the big smoke, I would rather not be in that city every day for any appreciable amount of time.

    I’d be willing to entertain relocation elsewhere in Europe for a really cool opportunity. Preferably somewhere with a Germanic language because I can understand those (including, if push comes to shove, Icelandic and Faroese). Amsterdam, Stockholm and Dublin would all be cool. The States? No: I couldn’t commit to living over there for more than a short amount of time.

    Who will you do it for?

    That part is still open: it could be you. I would work for commercial, charity or government/academic sectors, but I have this restriction: you must not just be another contract app development agency/studio. You must be doing what you do because you think it’s worth doing, because that’s the standard I hold myself to. And charging marketing departments to slap their logo onto a UITableView displaying their blog’s RSS feed is not worth doing.

    That’s why I’m not just falling back on contract iOS app development: it’s not very satisfying. I’d rather be paid enough to live doing something great, than make loads of money on asinine and unimportant contracts. Also, I’d rather work with other cool and motivated people, and that’s hard to do when you change project every couple of months.

    So you’re doing something you believe in, and as long as you can convince me it’s worth believing in and will be interesting to do, and you know where to find me, then perhaps I’ll help you to do it. Look at my CV, then as I said before, e-mail me and we’ll sort out an interview.

    I expect my reward to be dependent on how successful I make the product or community I’m supporting: it’s how I’ll be measuring myself so I hope it’s how you will be too. Of course, we all know that stock and options schemes are bullshit unless the stock is actually tradeable, so I’ll be bearing that in mind.

    Some miscellaneous stuff

    Here’s some things I’m looking for, either to embrace or avoid, that don’t quite fit in to the story above but are perhaps interesting to relate.

    Things I’ve never done, but would

    These aren’t necessarily things my next job must have, and aren’t all even work-related, but are things that I would take the opportunity to do.

    • Give a talk to an audience of more than 1,000 people.
    • Work in a field on a farm. Preferably in control of a tractor.
    • Write a whole application without using any accessors.
    • Ride a Harley-Davidson along the Californian coast.
    • Move the IT security industry away from throwing completed and deployed products at vulnerability testers, and towards understanding security as an appropriately-prioritised implicit customer requirement.
    • Have direct reports.

    Things I don’t like

    These are the things I would try to avoid.

    • “Rock star” developers, and companies who hire them.
    • Development teams organised in silos.
    • Technology holy wars.
    • Celery. Seriously, I hate celery.

    OK, but first we like to Google our prospective hires.

    I’ll save you the trouble.

    On BizSpark

    You’ll remember that recently I reviewed Windows Phone 7 Mango from the perspective of an iOS guy, and actually came back pretty impressed with it.

    You’ll also remember that through my company, Fuzzy Aliens Ltd, I offer app security services to mobile app developers. So far, that basically means iOS developers: in addition to being where I have most experience, I have punted around for Android clients and got exactly zero interest.

    So I thought it would be useful to offer the same service for WP7. After all, Microsoft knows the bad press associated with having security fail on their platform, so should be welcoming of a security guy adding his biological and technological distinctiveness to their own. Not only that, but there will probably be a lot of line-of-business app developers out there who would appreciate mobile security knowledge.

    Now the thing that puts me off is basically the cost. I own exactly one copy of Windows 7, and use the free Visual Studio Express. To meaningfully research and code for Windows Phone 7 I’d need another two Windows licences (£100-£250 each roughly depending on version) and Visual Studio Pro and MSDN (roughly £700), along with at least one handset (£300) and an App Hub membership (£60). Wow. Around £1500 just to dip my toe in untested waters.

    Luckily, Microsoft have a plan designed to help. BizSpark ought to give me access to most of the above except the phone, in addition to training. It also offers that MS would put me in touch with potential clients and even investors, and could help with hosting costs for web services. The idea is that Fuzzy Aliens would get this stuff for free for a while, during which MS would help build the business. Then, once FZA “graduates” from the program, I get to keep all the software and MS have a new trusted partner.

    Seems like a low-risk way to get into Windows Phone 7, and to grow my business which – while only six months old – is already showing signs that I need to find more clients from somewhere. So I signed up at around 16:15 today.

    By 18:13 Microsoft had decided that:

    it does not appear that you meet all the eligibility requirements at this time. To enter the program, your startup must be:

    • Actively engaged in development of a software-based product or online service that will form a core piece of its current or intended business,
    • Privately held,
    • In business for less than 3 years, and
    • Less than US $1 million in annual revenue

    Well, in fact FZA meets all of those criteria. The basis of its business is secure software, and indeed I am currently (OK, I’m blogging – you see what I mean though) developing such secure software. Indeed I even help out the platform community for free by releasing some of this software here as open source.

    The business is fully held by me, and has been operating for nearly six months. I would dearly love to have more than $1M of revenue, but it hasn’t happened yet.

    So for whatever reason – though not one they care to tell me about – Microsoft has decided that they don’t want me joining their community. Given that this leaves me free to focus on making the iPhone a safer platform for its users, I don’t yet know which of us has lost out the most.

    A Cupertino Yankee in the Court of King Ballmer

    This post summarises my opinions of Windows Phone 7 from the Microsoft Tech Day I went to yesterday. There’s a new version of Windows Phone 7 (codenamed Mango) due out in the Autumn, but at the Tech Day the descriptions of the new features were mainly the sorts of things you see in the Microsoft PressPass video below (Silverlight required), the API stuff is going on in a separate event.

    I want to provide some context: I first encountered C#, J# and .NET back in around 2002, when I was given a beta of Visual Studio .NET (Rainier) and Windows .NET Server (which later became Windows Server 2003). Since then, of course most of my programming work has been on Objective-C and Java, on a variety of UNIX platforms but mainly Mac OS X and iOS. But I’ve kept the smallest edge of a toe in the .NET world, too.

    From the perspective of the phone, however, I really am coming to this as an iOS guy. Almost all of the mobile work I’ve done has been on iOS, with a small amount of Android thrown into the mix. I’ve used a WP7 phone a couple of times, but have no experience programming on WP7 or earlier Windows Mobile platforms.

    The UI

    The speakers at the Tech Day – a mix of Microsoft developer relations and third-party MVPs – brought as much focus on user experience and visual impact of WP7 apps as you’ll find at any Apple event. Windows Phone uses a very obviously distinctive UI pattern called Metro, which you can see in the demo screencasts, or the Cocktail Flow app.

    Metro is almost diametrically opposed to the user experience on iOS. Rather than try to make apps look like physical objects with leather trim and wooden panels, WP7 apps do away with almost all chrome and put the data front and centre (and, if we’re honest, sides and edges too). Many controls are implicit, encouraging the user to interact with their data and using subtle iconography to provide additional guidance. Buttons and tiles are far from photorealistic, they’re mainly understated coloured squares. Users are not interacting with apps, they’re interacting with content so if an app can provide relevant functionality on data from another app, that’s encouraged. A good example is the augmented search results demoed in the above video, where apps can inspect a user’s search terms and provide their own content to the results.

    In fact, that part of the video shows one of the most striking examples of the Metro user interface: the panorama view. While technologically this is something akin to a paginated scroll view or a navigation controller, it’s the visual execution that makes it interesting.

    Instead of showing a scroll thumb or a page indicator, the panorama just allows the title of the next page to sneak into the page you’re currently looking at, giving the impression that it’s over there, and if you swipe to it you can find it. When the user goes to the next page, a nice parallax scroll moves the data across by a page but the title by only enough to leave the edges of the previous and next titles showing.

    The Tools

    It’s neither a secret nor a surprise that Microsoft’s developer tools team is much bigger than Apple’s, and that their tools are more feature-rich as a result (give or take some ancient missteps like MSTest and Visual SourceSafe). But the phone is a comparatively new step: WP7 is under a year old, but of course Windows Mobile and Compact Editions are much older. So how have Microsoft coped with that?

    Well, just as Apple chose to use their existing Cocoa and Objective-C as the basis of the iOS SDK, Microsoft have gone with .NET Compact Framework, Silverlight and XNA. That means that they get tools that already support the platform well, because they’re the same tools that people are using to write desktop, “rich internet” and Xbox Live applications.

    From the view-construction perspective, XAML reminds me a lot more of WebObjects Builder than Interface Builder. Both offer drag-and-drop view positioning and configuration that’s backed by an XML file, but in Visual Studio it’s easy to get precise configuration by editing the XML directly, just as WebObjects developers can edit the HTML. One of the other reasons it reminds me of WebObjects is that Data Bindings (yes, Windows Phone has bindings…) seems to be much more like WebObjects bindings than Cocoa Bindings.

    Custom classes work much better in the XAML tools than in IB. IB plugins have always been a complete ‘mare to set up, poorly documented, and don’t even work in the Xcode 4 XIB editor. The XAML approach is similar to IB’s in that it actually instantiates real objects, but it’s very easy to create mock data sources or drivers for UI objects so that designers can see what the app looks like populated with data or on a simulated slow connection.

    Speaking of designers, an interesting tool that has no parallel on the iPhone side is Expression Blend, a XAML-editing tool for designers. You can have the designer working on the same files as the developer, importing photoshop files to place graphics directly into the app project.

    It’d be really nice to have something similar on iPhone. All too often I have spent loads of time on a project where the UI is specified as a photoshop wireframe or some other graphic provided by a web designer, and I’m supposed to customise all the views to get pixel-perfection with these wireframes. With Blend, the designer can waste his time doing that instead :).

    Other tools highlights include:

    • Runtime-configurable debugging output on both phone and emulator, including frame rates, graphics cache miss information, and Quartz Debug-style flashes of updated screen regions
    • The emulator supports virtual accelerometers
    • The emulator supports developer-supplied fake location information and even test drivers generating location updates <3

    The biggest missing piece seems to be a holistic debugging app like Apple’s Instruments. Instruments has proved useful for both bug fixing and performance analysis, and it’s pretty much become a necessary part of my iOS and Mac development work.

    Update: I’m told by @kellabyte that an Instruments-like tool is coming as part of the Mango SDK, and that this was announced at MIX ’11.

    The “ecosystem”

    A couple of the demos shown yesterday demonstrated phone apps talking to Azure cloud services, ASP.NET hosted web apps (mainly using the RESTful OData protocol), SOAP services etc. Because there’s .NET on both sides of the fence, it’s very easy to share model code between the phone app and the server app.

    That’s something Apple lacks. While Cocoa can indeed be used on Mac OS X Server, if you want to do anything server-side you have to either hope you can find some open-source components or frameworks, or you have to switch to some other technology like Ruby or PHP. While Apple ship that stuff, it’s hard to claim that they’re offering an integrated way to develop apps on iOS that talk to Apple servers in the same way that MS do.

    To the extent that WebObjects can still be said to exist, it doesn’t really fill this gap either. Yes, it means that Apple provide a way to do dynamic web applications: but you can’t use Apple’s tools (use Eclipse and WOLips instead), you can’t share code between iOS and WO (iOS doesn’t have Java, and WO hasn’t had ObjC in a long time), and you can just about share data if you want to use property lists as your interchange format.

    On the other hand, it’s much easier to distribute the same app on both iPhone and iPad than it would be to do so on WP7 and a Microsoft tablet/slate, because their official line is still that Windows 7 is their supported slate OS. I expect that to change once the Nokia handset thing has shaken out, but making a Silverlight tablet app is more akin to writing a Mac app than porting an iOS app.

    The market

    This is currently the weakest part, IMO, of the whole Windows Phone 7 deal, though it is clear that MS have put some thought and resources behind trying to address the problems. Given that Windows Phone 7 was such a late response to the iPhone and Android, Microsoft need to convince developers to write on the platform and users to adopt the platform. The problem is, users are driven to use apps, so without any developers there won’t be any users: without any users, there’s no profit on the platform so there are no developers.

    Well, not no developers. Apparently the 17,000 apps on the marketplace were written by 7,500 of the 42,000 registered developers (and interestingly the UK has one of the lowest ratio of submitted apps to registered developers). By comparison, there are 500,000 apps on the app store.

    Microsoft has clearly analysed the bejeesus out of the way their users interact with the marketplace. They have seen, for instance, that MO billing (essentially having your phone operator add your app purchase costs to your phone bill, rather than having a credit card account on the marketplace itself) increases purchase rates of apps by 5 times, and are working (particularly through Nokia of course who already have these arrangements) to add MO billing in as many marketplace countries as they can.

    This makes sense. People already have a payment relationship with their network operators, so it’s easier for them to add a few quid to their phone bill than it is to create a new paying account with the Windows Marketplace and give Microsoft their card details. By analogy, iPhone users already have bought stuff from Apple (an iPhone, usually…and often some music) so throwing some extra coin their way for apps is easy. Incidentally, I think this is why the Android app market isn’t very successful: people see Google as that free stuff company so setting up a Checkout account to buy apps involves an activation energy.

    Incidentally, some other stats from the app marketplace: 12 downloads per user per month (which seems high to me), 3.2% of all apps downloaded are paid, and the average price of a bought app is a shave under $3. Assuming around 3 million users worldwide (a very rough number based on MS and analyst announcements), that would mean a total of around $3.5M app sales worldwide per month. That’s nowhere near what’s going on on the iPhone, so to get any appreciable amount of cash out of it you’d better have an app that appeals to all of the platform’s users.

    The subject of appeal is a big issue, too. Microsoft aren’t really targeting the WP7 at anyone in particular, just people who want a smartphone. With Mango and Nokia handsets, they’ll be targeting people who want a cheaper smartphone than RIM/Apple/Android offers: I bet that brings down that $3 mean app price. This is, in my opinion, a mistake. Microsoft should play to their strengths, and make a generally-useful device but target it at particular groups who Microsoft can support particularly well.

    Who are those groups? Well, I think there’s Xbox 360 gamers, as WP7 has Xbox Live integration; and there’s enterprises with custom app needs, due to the integration with Azure and similarity with ASP.NET. It ought to be cheaper for game shop that’s written an Xbox game to do a WP7 tie-in than an iOS tie-in. It ought to be cheaper for an enterprise with an MS IT department to extend their line-of-business apps onto WP7 than onto Blackberry. Therefore MS should court the crap out of those developers and make the WP7 the go-to device for those people, rather than just saying “buy this instead of an iPhone or Android”.

    The reason I’d do it that way is that you bring the users and the developers together on shared themes, so you increase the chance that any one app is useful or relevant to any given customer and therefore increase the likelihood that they pay. Once you’ve got gamers and game devs together, for example, the gamers will want to do other things and so there’ll be a need for developers of other classes of app. I call using Xbox games to sell utilities the Halo Effect.

    Conclusion

    Windows Phone 7 is a well thought out mobile platform, with some interesting user experience and design. It’s got a good development environment, that’s highly consistent with the rest of the Microsoft development platform. However, no matter how easy and enjoyable you make writing apps, ultimately there needs to be someone to sell them to. Microsoft don’t have a whole lot of users on their platform, and they’re clearly banking on Nokia bringing their huge brand to beef up sales. They should be making the platform better for some people, and then getting those people on to the platform to make WP7 phones aspirational. They aren’t, so we just have to wait and see what happens with Nokia.

    Footnote: Nokisoft

    Nokia do well everywhere that Microsoft doesn’t, by which I mean Europe and China mainly. Particularly China actually, where the Ovi store is pretty big. Conversely, MS phone isn’t doing too badly in America, where Nokia traditionally are almost unheard of. So on paper, the Nokia deal should be a good thing for both companies.

    “Patently” secure

    One thing that occasionally becomes interesting about working in security is that doing security and managing business have a great deal of overlap. This makes a lot of sense: a business wants to be profitable, and profit is a reward conferred by the market for taking on some risk. But too much risk can expose your business to undesirable failures, so understanding and controlling your exposure to risk is a useful exercise.

    Well that’s fundamentally how security works too. There is some reward to be gained by performing the activity allowed by an app: that might be the enjoyment of playing a game, the cost savings of keeping track of your finances, or the health benefits of seeing what food you consume. But using the app also brings some risk, and so security people seek to quantify and reduce the risk inherent in any app.

    I’m going to compare a business risk (infringing on another’s patent) to an information security risk (leaking confidential data) to show just how similar these fields are. I choose patent infringement because it’s an apposite case: however you’ll find that I don’t name particular patents or companies for reasons that will be entered into below. Suffice it to say that I have dealt with software patent lawyers in the past and have some – but not much, by any means – experience of how the US patent system operates. If you choose to infer any advice from this blog post, please seek appropriate counsel before acting on it: I am not a lawyer, and I am certainly not your lawyer.

    Quantisation

    A risk to either a business or a user can be summed up by the expected damage caused by the event coming to pass. That is, the estimated cost (financial, emotional, intangible etc.) of the risky event multiplied by the expected probability of that event happening.

    In the leaky data case, the expected damage would be “what chance is there that an attacker will retrieve the data” × “what is the impact to the user of exposing the data”? Both of these parameters are hard to quantify: information about data security breaches is notoriously hard to get hold of because companies are reluctant to talk about problems they’ve had protecting their customer records. Combine with that the fact that in many fields even the direct costs of a breach are hard to arrive at, and you end up multiplying two very big error bars together.

    In the infringement case, things are a bit more straightforward. Legal reports are – in many jurisdictions – a matter of public record, so seeing what the damage of a case “like yours” is going to be is quite easy. That covers direct costs, anyway: indirect costs like lost custom, damaged reputation etc. are harder to arrive at. The likelihood of being caught infringing on a patentholder’s rights is harder to estimate, but I expect not beyond the realms of reason.

    Mitigation

    There are a few different approaches to reducing (mitigating) the risk involved, which either address the likelihood or expected cost of impact. Let’s look at them. You don’t have to choose any one approach: a successful strategy may combine tactics from each of these categories and even use more than one tactic from the same category.

    Withdrawal

    Remove any likelihood and impact of a risky event occurring by refusing to participate in the risky activity. In the confidentiality case, this means not storing the secrets in the first place. In the patent case, it means not using the potentially infringing invention.

    In either case withdrawing from the activity certainly mitigates any risk very reliably, but it also means no possibility of gaining the reward associated with participation.

    This is why, going back to an earlier point, I don’t comment on particular patent cases. Given that patent rights asserters are, in my opinion, more litigious than I, there’s a chance that if I talk about a particular case I’ll be considered defamatory. I’d rather avoid that risk, and choose to control it by withdrawing from talking about the cases.

    Transference

    You can opt to transfer the risk to another party, usually for a fee: this basically means taking out insurance. In either of our case studies, look for insurance that protects against the damage incurred. This doesn’t affect the probability that our risky event will come to pass, but means that someone else is liable for the damages.

    Employing Countermeasures

    Finding some technical or process approach to reduce the risk. In the patent case this is simple: the countermeasure to “sued by patent holder” is “license patent”.

    In the confidentiality case, this means technical countermeasures: access control, cryptography and the like.

    But think about deploying these countermeasures: you’ve now made your business or your application a bit more complex. Have you introduced new risks? Have you increased the potential damage from some risks by reducing others? And, of course, is your countermeasure cost-effective? The traditional security mantra is “don’t spend $1000 to save $100”: don’t license $1000 of patents to protect a $100 product, and don’t implement $1000 of crypto to hide $100 of data.

    Acceptance

    The “suck it up” approach to security: accept that the risk exists and that you may be liable for the damage if it ever comes to pass. In our information security case, this means storing the data and accepting that someone else might be able to read it. In our patent case, this means adopting the potentially-infringing invention and accepting that a licensor might come a-knocking.

    All risk mitigation strategies have a certain amount of acceptance involved, apart from withdrawal. Imagine that you pay some insurance premium, and that indemnifies you up to $10M with an excess of $1000. You have to choose whether you accept the residual risk exposed in covering the excess and any overage.

    Similarly, in the information security case, let’s say you have data assets which, if leaked, would cost $1M in damages. You implement a particular cryptography technique that reduces the likelihood of leaking the data from an estimated once per year to an estimated once per thousand years. Again, do you accept the remaining $1000?

    Conclusion

    Information security and business management are actually pretty closely related. It’s just that information security requires specialised technical knowledge: and that’s where I come in ;-).

    On internal quality

    I was asked by attendees at my VTM talk on test-driven development a small collection of questions on a similar theme, which I’ll summarise here.

    • How do I do TDD when my boss doesn’t want me to?
    • What do I do when my boss wants me to ship untested prototype code?
    • Can you give me rhetoric to convince my boss that I should be doing TDD?

    I believe that I can adequately present these questions as facets of the following root issue, and then blabber away endlessly about that one instead.

    I have externalised the obligation I have to myself to write software that makes me proud by delegating it up to my boss. Can you help me to sleep at night?

    The fundamental problem here is one of cognitive dissonance. Belief: I, as a software engineer, believe that process/technique/tool is good for writing software. Introspection: I observe that I am not using said process/technique/tool. Justifications: it must be that my boss doesn’t hold my output to the same quality standard that I do. It must be that my deadlines and ever-shifting requirements don’t give me time to do things properly. It must be that this company doesn’t want great software engineers.

    No. I have learned the hard way that such justifications are false. The only person responsible for how you write code is you. If you work on a team, or are handing your source code over to some other stakeholder (which means you’re working on a team), then you may have style guides that limit what source code you write; but ultimately how you get there is solely down to you.

    You may be thinking “but I asked my manager for time to implement TDD/CI/buzzword-compliance on my current project and she said no. Also, by the way, you’re an arsehole for being so rude and presuming to know what I think.” Let me address the latter point first: suck it. Now onto the first point.

    Your manager was correct to turn down your request for time on the project. Your customer doesn’t want unit tests, or comment documentation, or build server output. Your customer wants working features. The project exists to satisfy the user’s needs. Therefore the time allocated on the project should be dedicated to making working features.

    Now it turns out that doing TDD is one way to write code that leads to working features, so what you should have done was to agree to the version of the product plan where you make working features, and then done TDD anyway. Where this all started going wrong was not when your boss turned down your request, but when you asked your boss in the first place. It turns out that you both have the same goal—making a good product—but that you have different views on the process and different motivators. By asking your boss how to write code you gave her permission to micro-manage you, then got frustrated when she did, and decided that the problem was all her fault.

    I’ve seen this failure mode quite a few times now. As one example, I worked in a company where there was an ingrained antagonism between the product managers (who clearly just don’t get that software is a craft and a labour of love) and the programmers (who clearly just don’t get that we work in a competitive marketplace and would rewrite the product from scratch every day if they could).

    As is common (and often correct) in our industry, the product managers owned the product requirements. As should be welcomed when we care about the products we make, the programmers were invited to criticise the requirements for the projects they were working on. Due to this antagonistic culture, the programmers would typically stuff “engineering requirements” onto the project, which were things like rewriting components from scratch, epic refactorings, or setting up new developer workflows.

    Requirements were, as far as I could tell, prioritised based on how much revenue they would attract (i.e. new customers), how much revenue they would protect (retained customers) and how much they would cost. As you can probably already guess, so-called engineering requirements don’t protect any revenue, they don’t generate any revenue, and they do take time and money to implement, so they would inevitably get dropped or indefinitely postponed.

    The moral of that little story is this: it is folly to try and express development processes and methodologies as business requirements, because they fail at being business requirements. It’s the equivalent of a taxi driver asking the customer to pay a $100,000 fare for one journey because he wants to switch to automatic transmission for the next journey and that means buying a new vehicle. The customer cares about being driven somewhere, and doesn’t care about how the driver operates the vehicle as long as the journey is bug-free. He certainly doesn’t intend to pay for the driver to select a particular mode of operation: if the driver wants to use automatic transmission, the driver should just get on and do that.

    So what I’m saying is this: it’s not up to your boss, your customer or your project manager to choose how you write software. It’s up to you to choose how you write software, and as long as the approach you take leads to working software that delights your customers, the rest of the stakeholders won’t mind what you’re doing “under the hood”. My good friend and long-lost son @bmf once said “don’t let them see you making it.” I would go further: don’t let them know it was made. Let them use exciting, compelling software: the magic is your business. If you want to make the magic in one particular way, that would let you take additional pride in your creation, that’s entirely your call.