How I fail to design software
Failure is a bigger part of the design process, as you will usually encounter more failures than successes. Some ideas only encounter failure. I’ve talked about some of the experimental processes I take towards build software, so it seems appropriate to talk about some of the ways in which i’ve failed.
The most obvious mistake I make and encounter is bikeshedding, or pandering to trivial issues in lieu of progress. Bikeshedding is arguing over “num_adults” or “numAdults”, tabs or spaces, things that everyone has an opinion on, but the impact of the choice is minimal. As more people are involved, more time is wasted.
You can still pander to trivialities when you’re on your own. It’s all too easy to worry about the code you’re going to write, instead of just writing it. Sometimes you need to write code to understand the problem.
Unfortunately, writing code to understand the problem takes a lot of time, it can be quite frustrating to find our your approach doesn’t work and you have to start from scratch. I have thrown away countless prototypes, branches, and features as I find out the problem has outsmarted me.
I also fail by finding a more interesting version to solve, usually by solving an abstract or general version of the problem. Writing a web framework instead of a web site, writing a game engine instead of a game. Without research or experience, it’s hard to get these abstractions right — the most successful frameworks come from working products.
Frameworks are the most obvious symptom of over-generalizing, but it surfaces in other ways, under the guise of extensibility. Instead of writing the code I need, I write code that can be extended to solve other problems I don’t face at the moment. Extensible code is just code that it is easy to add features to, but not code that is easy to replace, change, or delete. You need to be able to remove features, as well as add them.
Instead of unnecessary abstraction, you can add unnecessary features to make the problem more interesting. I’m good at keeping bloat away from my professional work, but my hobby projects seem to suffer from it eternally. Being precise and clear about the problem will usually keep it in check, but you’ll need to add more constraints along the way.
Problems can become boring over time too. Starting an open source project is easy compared to keeping it alive. I am guilty of letting a few libraries rot, and many others never got beyond the first release. Along with abandoned libraries, there is mountains of code i’ve thought about and never written. Although my side projects are for my own entertainment, I shouldn’t be surprised if no-one else is interested, especially the ideas I haven’t written code for.
For every useful piece of software i’ve written, i’ve probably written a hundred failures.
a message from the inbox:
Apparently you can get offended by literally anything these days. Unfollowing, have a good day! — fluxcapacitate
In the same way I try not to make racist commentary to teach people about programming, I don’t think it is useful to make outdated remarks about women either. Mr Martin seems to agree with me, and he’s since updated the post.
Although he had done his best to strip the piece of sexist ideas, all he had done was to remove “he” and “she” from the piece. Many of the other ideas still were from the view of a 50’s style man’s view of the world, where women were objects to collect and conquer, along with femme fatales luring men to their doom.
I am not suggesting this is how Martin thinks, or necessarily acts, but many people (including me) are tired of these outdated stereotypes, which only serve to reinforce prejudiced ideas and systematic inequalities. These ideas are from the past, but they certainly aren’t stuck there.
Many of my smartest friends have been told that they cannot be promoted because they were women, as well as being treated like secretaries, looking after the menfolk doing hard work. This and countless other smaller incidents slowly grind down on the minorities in technology, those most vulnerable to stereotype threat.
I’ve talked about this before with Dweck’s research on gender differences. There are stereotypes that “girls can’t do math, and boys can” which are encouraged in children. Dweck’s work is really fascinating and worth a read, but in essence, if we stop telling children that mathematics ability is a gift, they will stop believing the stereotypes to be true.
Rehashing old stereotypes can be a funny joke, but we haven’t really progressed to the point where they can’t hurt anymore. These pieces little by little push people out of technology. Currently, half of the women in tech leave after a few years.
I am not asking you to put down the penis jokes. I like penis jokes. I’m just saying that I don’t always appreciate them in technical material. I also teach children as well as grown ups. People from either group don’t always appreciate unnecessary sexual commentary in their learning materials.
Meanwhile, Martin’s new version has succeeded in communicating his thoughts without making it sound like a frat-boy diatribe.
I am just trying not to be a dick, and I want Martin not to be a dick too. I also try not to make cancer jokes around people who are watching their loved ones die. There is a time and a place for self indulgent jokes about harems, concubines, and the family jewels.
Is it too much to ask to point out that talking about frameworks is neither the time, or the place?
Edit See also: http://meaganwaller.com/posts/framework-whipped
Edit #2 Martin has edited his post to remove the crass remarks, and stated that he is sorry for any offence caused
I am a 59 year old programmer who was brought up in a male dominated industry, operating in a male dominated society. To my regret, old habits and attitudes, that I thought were long dead and conquered, pop up from time to time. So, if you see me making a mistake like this, you yell out and stop me. And don’t let any of my peers get away with it either.
Dear Mr Martin, I think you’ve done it again, with your recent piece Framework Whipped.
For your future use, here is a quick guide to wether a metaphor is going to be a useful one for talking about software: Are you talking about fucking or slavery? Might as well stop, more likely than not, you’ll sound like a creepy uncle again.
Although you tell me to keep frameworks away “from the family jewels”, i’m unsure what I gain from treating business logic as being the penis of software. I’m really unsure of why I should consider frameworks as “concubines to be enjoyed on my terms”.
I’m not saying let’s not talk about fucking your secretary, owning slaves, or sub/Dom relationships. I’m saying the use of sexual imagery in explanations is usually “just a bit of fun” that alienates anyone who isn’t using penis driven development.
I can imagine this went through your head as you wrote this, so you removed any mentions of “he” or “she”, but unfortunately it’s still dripping in machismo that it reads like a bad Mad Men fanfiction. Repeat after me: using frameworks is nothing like owning humans for sexual exploitation.
The moral is “don’t tightly couple your application to your framework, especially the business logic”, not “you can stick your penis in a framework, but marriage is out!”.
From (Sir) Tim Berners-Lee’s AMA
I am not foolish enough to say that the way I write software is the only way, but I am foolish enough to believe how I approach software design might be helpful. This isn’t about project management, or algorithmic strategies, but how I fumble around until something works, guided by the scars of past mistakes.
For every good idea, there are probably three implementations out there already. If you’re lucky, one of them might work, and might even be documented. There could be some academic work too, that isn’t buried in a paywall. Search engines are your friends.
Being able to work out the right name for a problem will help you greatly in your search. This is the one advantage of a grounding in computer science — rich technical jargon to find existing work easily. This is opposed to the “which pokemon would I name this library after” school of thought.
Sometimes it’s best to start with human language. Write a readme, it should explain what problem the software solves, what sort of features it will have, and the interface planned. Avoid the trap of writing a heavy specification, skip the implementation details. The job of the readme is to set expectations.
Come up with a number of vague ideas, roughly sketched out, and list their positive and negative features. Take these positive and negative lists and distill them into a small number of constraints.
Start again, but hopefully you’ll have less options to choose from. Once your constraints are settled, it’s a good time to move onto trying things out in practice.
It is often cheaper to experiment and see what fails, rather than trying to work this out ahead of time. Prototypes are a great place to make lots of mistakes, and an opportunity to play around with rough ideas.
Beware: the only difference between production code and a prototype is deployment — it’s hard to justify rewriting things that work. Putting prototypes into production is the software equivalent of being hoisted by your own petard.
Get feedback as quickly as possible
Write the smallest program that runs to begin with, often just a skeleton or a framework stub. Get other people to write code that uses yours. Let them hack away at your library, and change it to match how they use it. Mistakes cost less to fix when you find them shortly after making them.
Don’t spend an hour writing code without checking it at least once. For the more strict among us, you may choose to test, type check, formally verify, or prove code regularly. The rest of us plebs should remember to avoid binge writing code without any feedback from the computer. It’s much easier to come back to coding a program if it still runs or compiles.
Make incremental changes
Small code changes are the easiest to recover from, but it’s often hard to break new features into small edits. People will use the term “legacy code” refer to code that is hard to change or maintain in this way (Apart from legacy code, the only other type of software is code no-one uses).
Small changes can include rewrites, if you can slowly migrate users from the old to the new component. You’ll still need to maintain the old code though.
Sometimes it’s good to hole up and get yourself up to speed on the problem. For me this involves covering my desk in scraps of paper with diagrams and scribbles. Other times it’s clawing away at an interactive session, trying things out.
Take a bath, get some sleep
Although it’s good to be immersed in a problem, it’s easy to get lost or overwhelmed in it. Go outside, stay inside, read a book, play a game, socialise, or retreat. Inspiration will come from distraction. I highly recommend naps.
Work with a friend
The code i’m most proud is the code i’ve worked on with other people. It’s easier to keep to your goals if someone else will hold you to them, and you’re not attached to your ideas.
Humans will often talk about problems in terms of the solution they have in mind, so be patient and try and extract the rationale behind it, even if the idea doesn’t sound so good (This often surfaces as the X-Y problem).
You don’t have to agree on the solutions but it’s good to agree on what makes a solution good. You can compromise on the former, but the latter usually ends up making everyone miserable.
All of the above
I approach design as a process of refining a question until the answer is obvious, and I see the above as ways of doing it. I don’t always do them in a strict order, i’ll skip some bits, repeat others, and some i’ve missed out or forgotten.
Everyone will approach design differently, which is why it’s so much fun to work with other people.