To me, the design stage is the most intellectually stimulating part of the software development cycle. After all, the design decides what the code will do, and how it will do it. The design process gives technical shape to what are till then only business requirements. We get to the deepest what’s and why’s at this stage – it is a lot of fun!
Here’s my rule of thumb about approaching software architecture – Approach designing a system/feature like you are writing an investigative story. That’s essentially what the design exercise is – we are poking around in the problem space/requirements to figure out how we might solve it. And like any good investigator, we ought not to make assumptions early on or jump back and forth between incidents. All we have is a burglary – deciding to shadow the sinister looking neighbour up front is probably not the best of course of action.
I love crafting architecture narratives because they sit “above” any technical approaches and provides a framework for applying any of them. It also like it party because it has worked very well for “most” of the problems “I have worked on” (which are mostly application engineering problems using service-based architecture), and partly because even when it doesn’t provide actual technical design (e.g. in big data related problems), it helps me understand the problem domain deeply.
Start from a broad lay-of-the-land, then identify specific facts and occurrences, then progressively focus on each of them to see how they tick – all the while keeping the thread of overall technical story in hand. The process should feel very organic and evolutionary – a breakdown of high level business requirement into somewhat lower level technical constructs into technical components into the lowest level of implementation details and technology choices. It is a little bit like doing Bog Post Driven Development for engineering teams.
You can slice this breakdown in many ways, but in most cases a few logical levels usually suffice.
High Level View
This fits our system or feature in the overall big picture and defines why we need it. It also assigns our system-specific responsibilities which no other system can fulfill. This is where we think about whether we need this thing in our organization and who might be consuming it.
Components Level Design
Now we look at our system itself and try to figure out what it is made up of. Any system of some complexity with be composed of multiple sub-parts and here we identify them, name them, and attribute behaviours to them. Every time a component is described, it feeds the narrative of the overall system by defining the unique role it plays in its functioning and how it works with the other components. This level forces decisions around domain boundaries and how different concepts of the system interact with each other.
Low Level Design
Now we look inside each component and explain how it actually works on the inside. This is usually the point at which we will be forced to make some concrete choices about technology choices and other physical details like deployment and performance.
Don’t critique the oversimplifies example taken above, but focus on the fractal like approach applies a different set of design principles at each level but in the same philosophical manner of composition and decomposition. Enterprise Design Architecture principles might apply at the highest level, Reactive design principles at the component level, and SOLID principles at the low level – but the entire exercise if about explaining the how the parts and the sum of the parts functions.
Of course, not all design will need all of these level. Don’t fit them to this pattern forcibly. Build your own story – it can be a short one too!
Why write architecture narratives
There are several advantages to using this approach for your next system.
Gut Feel Checks
The narrative centric approach comes with gut-feeling checks around jarring changes in levels of abstraction. If your technical story feels like you suddenly skipped a few chapters and need to go back and check what happened, you have either missed laying out some important details or have jumped over them by making some assumptions. If you feel this is happening like going directly to implementation details, or talking about technologies that you might use, or naming service that you will build, it is usually a good idea to step back and focus on getting the plot line in the right order. What is it -> what does it look like -> what does it do -> how does it do it.
A story that jumps all over the place is usually not very good (unless it’s a time travel story – in which case it is all right).
Build a shared vision
Another advantage of explicitly building a story comes when you have to share it/convey it to others. If you are working with a team – sticking to a narrative breakdown allows the entire team to align with and contribute to the plot as it is being shaped. Not only does this put a lot more options on the table early, it will also set a shared vision around which all further decisions can be based. And once the design is ready and to be shared with managers or stakeholders or other engineers who might not yet understand the problem domain as you now do – you already have a story to sell! Not just the final ready artefact in the shape of implementable design, but also the process of how your team got there. This will help everyone understand the design better, it will also help them the problem structure and thus give better feedback.
A steady, systematic breakdown of the problem from high level requirements to low level implementation details demonstrates that you put in the necessary diligence. The documentation will also be more self-explanatory because the processing backing it is self-explanatory. If you are selling a large idea to management, this can be invaluable. Also when you have to explain why some seemingly trivial feature will take more than 3 hours to implement.
Postpone decisions as much as possible
The last advantage of design-as-a-story is that you get to postpone decision. No good story start with lots of details. The best of them build characters over an extended period so that the characters have had to pass through many situation and evolve the version you actually fall in love with.
Likewise we don’t want to start out by saying “recommendation engine will be built using Spark and Hadoop”. That can wait. Before we reach that point, we want to talk A LOT about what the recommendation engine is, who needs it, how it recommends, how it might change over time, who will build it etc etc. All of these things are important in making that final decision about exactly what we will build it with. We want to get know the persona of the engine before we pick his favourite weapon. The narrative approach is a good way to force that discipline on the design – concrete things are always decided as late as possible.
Caveat Emptor
With all that said, understand that everyone tells stories differently. This is just my way of doing things as I have learnt from experience and from others (Thanks Shamik, Abhinav, Sanjay, Apoorva, Govind) – it is certainly not the only way. Other people certainly approach it differently. There are data-first approaches, API-first approaches, UX-first approaches, Behaviour-driven-design and so on. You can use any of them to build great design – but remember that while people can always quibble with design, everyone like a good story 🙂
Read next – Highlight from “97 things every software architect should know”
