Bullet time and elephant hearts: Where to start on engineering projects with unknowns
Little has been written about more than how software engineers and teams should go about completing their projects, and yet after scanning through the unusually helpful comments on HN recently, I thought I might have something helpful to say about it.
The it here is:
How to start a project that you don’t already know how to complete.
That’s something every engineer and project manager faces, at every level, if they are doing interesting work. I hope this post might help you out with that.
The basic premise of risk reduction
At one extreme, you can plan. You could try working out everything on paper (or digital equivalent) before coding anything, traditionally called the waterfall methodology. But if you plan everything then planning itself becomes the engineering, and you’re back at asking where to start, so I don’t think this is very helpful advice on its own. At the other extreme, you can start building arbitrarily and hope for the best. Also not very helpful advice. Ultimately you need to choose a strategy that increases your likelihood of rapid success (often pessimistically called risk reduction).
If you’re a coder on the junior side, a factor that might make your rapid success more likely might be choosing a starting point where you feel the most confident because of past experience on a similar project. So if you’re not sure where to begin, you could try to answer: How does this project resemble something I’ve worked on before? How can I reuse (maybe even copy) my past experience to contribute to this project? The copy-paste might be your first commit. Other places to start might be areas where you’re most able to get help from your team if you need it or from documentation. Are there libraries with good documentation that can help? Basically, start where there is the most clarity. By the time you get to the other parts you may be better prepared to tackle them, either because you’ve learned something along the way or your team is by that point better able to help you.
But by the time you’re a senior-level coder, you will need to completely reverse how you think about this. Your choices now lock your team onto a path that can be hard to reverse, and you are now at least partly responsible for ensuring your team builds something valuable. Rapid success is not going to come from you coding faster and better but instead is going to come from avoiding your team going down too many wrong paths and having to throw time away. Most modern advice about this is that your biggest risk is that there is likely a misunderstanding about some aspect of what you should be building and that you won’t find out until a user sees it in action. These often relate to product/business goals like building a product people will want to use but also could relate to purely technical choices like frameworks and API designs. And moving rapidly has both technical advantages like avoiding merge conflicts from too many open branches and business advantages like getting to a sale sooner.
Your biggest risk is that there is likely a misunderstanding about some aspect of what you should be building and that you won’t find out until later. The sooner you can check that you have something that works and is valuable, the the more likely you can avoid wasting time on a misunderstanding.
Finding the shortest path
The post that started the HN comments (Code the Shortest Path First by Evan Lawrence-Hurt) says you should “code the shortest path thru the hardest problem” (quoting Evan’s more robust comment on HN) so that you learn as much about the problem as early as possible. Put off doing it well, put off using your senior coder skills (don’t write elegant classes and refactor code, don’t set up a CI/CD pipeline, don’t optimize or worry about performance, he says don’t write tests), put off functionality orthogonal to the hard problem, until after you’ve done something hard that works.
The sooner you can check that you have something that works and is valuable (e.g. by asking clients), the sooner you know that you haven’t wasted time on a misunderstanding. Of course, you may have wasted time on a poor implementation. But the common idea here is that it’s much faster to fix a poor implementation than it is to start over because the whole approach was wrong and that you should accept that you will misunderstand something. So to answer what to do first: Find the hardest problem in the project and focus exclusively on proving that you can solve it.
User t43562 said it really well:
Usually one understands a project poorly at the start and much better at the end. So to agree with the article I think it’s unwise to make all your decisions at the point where you know the least.
By getting something working you improve your understanding and then you can choose optimisations and abstractions in a judicious manner — no point in optimising things that end up having no impact and no point in introducing abstractions that in practice never will be used.
If you’re building an e-commerce website and have never done it before, maybe the hard problem — the crux of the project — is charging credit cards. You might start by writing a quick script that tests out a credit card merchant API, and from that you might learn something critical about the overall design of the project (like the need to handle declined charges or refunds) that could change the entire project: Maybe handling refunds is too expensive to build, so the client will accept a product catalog instead. As a senior engineer, when being assigned a new project, your gut will probably flag for you some problems that you’re not sure if they’re doable or the right thing to do— that’s probably where to start.
When you put off fancy coding, do write good code comments from the start. The point is to learn, and writing out what you learn as you figure things out is almost always a good idea.
Most of the comments on HN are about whether this is the same as building a proof-of-concept or prototype. It’s not. But let’s talk about prototypes.
Vertical versus horizontal paths
First, before a prototype, you can build a mock-up. A “concierge MVP” is my favorite form of mock-up: It looks like a working application to users, and the easiest parts you may have actually coded, but behind the screen is a person manually executing the logic or doing the work that would be the hardest to code. This is a fantastic short-cut to create a short path through a hard problem because you don’t even have to code it. You can get high-quality feedback from a small number of users on whether you are really solving their problem this way.
Building a prototype or a Minimum Viable Product is building vertically (rather than horizontally). The idea of building vertically comes from some of the things that I’ve read about Software Carpaccio/Elephant Carpaccio by Alistair Cockburn, but I’m not totally sure if what I’m saying reflects Alistair’s idea or not. Building vertically (IMO) means that from the very start you have all of the components of a prototype so that it can take real inputs and give real outputs. The Elephant Carpaccio metaphor is that from the very first iteration you have a bit of all of the elephant’s organs, they’re connected together and work together correctly, you can see that you’ve got an elephant there, and you can ask your stakeholder “is this what you meant by ‘elephant’ ?” The organs are probably supposed to be aspects of a business-driven user story. But to make this more concrete: If the organs are system components, in the first iteration they are stubs or barebones implementations. Maybe instead of a database you have an in-memory dictionary. But you have something for all of the components, just enough to string everything together. Later iterations extend those organs until finally you have a working application.
So when building vertically, the answer to where to start is stringing together stubs of components (in the first iteration) and then progressively turning those stubs into the real thing (in later iterations).
In Scrum, apparently there is something called a Tracer Bullet. I’d never heard of this before. According to Wikipedia and the HN comments, a Tracer Bullet is a prototype that helps you see that your components work together and that you’re on the right path. It’s also defined as a foundation for the production product, not a throw-away prototype. When building vertically, the first iteration and every subsequent iteration might be a Tracer Bullet.
This differs from Evan’s advice in two ways. First, stringing stubs of the components together may or may not be the hardest problem in the project or the aspect with the most to learn. (If there is a question about whether you are building the right thing at all, then it is an area with a lot to learn.) If this is the case, then building vertically or building a Tracer Bullet is a great place to start. Second, Evan’s advice is to not worry about whether the shortest path leaves you with a production-quality product, unlike a Tracer Bullet. That can be fixed later in a re-write.
Many of the comments on HN are (rightly) about how you’re likely to get told by management that your prototype has become Product 1.0, leaving you locked into your early choices. HN commenters say you shouldn’t put yourself or your organization in that position by writing high-quality code from the start. By the time your code gets to a pull request, it may need to be high quality.
But Evan’s point was not about creating a prototype. It was about diving first into the hardest problems with the most unknowns. That could be building horizontally instead. Horizontally means (IMO) focusing first on one component or one user story. Maybe you have to build a database that handles 1 million requests per second and you don’t know how to do that yet — and that’s hard! In that case, building the database first might be the best way to learn the most about the hardest part first, leaving the better understood and lower-risk aspects of the project to later (maybe that’s the UI). After building a poorly coded but correctly working prototype component, if you want to start over and code it well at least no other component will be affected (unlike when building vertically where that might affect all of the other components). But building horizontally leaves you precariously unable to show your stakeholders a prototype, cutting you off from feedback that could be vital to knowing whether you are building the right thing. Build horizontally, i.e. build components one at a time, only when there is no question about whether you are building the right thing.
Throughout a project you can switch between vertical and horizontal iterations, and that would be my own general advice for how to proceed with projects. Always first create and tie together stubs to validate the parts of the big picture that might have uncertainty, then focus on the hardest part of the hardest component, then add a bit of fidelity to all of the components to get better feedback from end users, then focus on the next hardest component, and so on.
Finally, when making decisions about where to start, use these frameworks (shortest path, hardest problem, vertical, horizontal, reducing unknowns, etc.) to explain to your teammates, supervisor, direct reports, and other stakeholders your thinking. I don’t do this enough myself and I think it would be a big help if I did more of it.
