When building large complicated things, there are two primary strategies that optimize for different things—layering and chunking.
Chunking is about taking something large, breaking it up into smaller pieces, and assembling them into one cohesive thing. This is pretty common and intuitive which pairs nicely with “divide and conquer” and can speed things up but tends to be all or nothing (either all the chunks are delivered or nothing ships).
Layering is also about breaking something large down but in a way that accumulates. The overall project is divided into smaller layers that are independently useful and shipped. This makes it more iterative, delivering value along the way and providing opportunities to adapt to new information.
Layering is much more desirable for large complicated projects because it’s incremental.
Problems arise when teams confuse chunking for layering—the team might think they are being fast and incremental, but deliver no value to the user until the end anyway.
I’ve observed many large-scale projects fail because they used the wrong strategy or because no one recognized that chunking was the only option.
To help spot these issues, a simple clarifying question to ask is: “At what point can a user actually use this?” (This can also be adapted for internal infrastructure projects, “At what point will this deliver value?").
(I read about this somewhere and apply it all the time but I can’t find the source, sorry!)
Links to this note
-
Circular Specification Problem
Writing a specification with sufficient detail to know exactly what software one should build is as much work as writing the code itself. In many cases, fully specifying the work beforehand is not possible because we don’t know enough about the problem or the domain to begin with. This is why our codebases are always in a state of flux and never complete systems.
-
Productive engineers should write one pull request per day. One pull request per day creates forward momentum. Forward momentum leads to progress. Progress is how teams overcomes their biggest challenges.
-
Physically making progress on a project makes it much harder to undo due to loss aversion and the sunk cost fallacy.
-
As software grows, patterns and practices naturally evolve. Inevitably, someone comes to the conclusion that a change should happen and code should be migrated to the new thing.