Have you ever heard engineers in your team complain about only building “business features” and never doing any “tech work”? There are some ways in which this complain is legitimate, but I feel that there is an underlying unity to both these things. I’d like to explain this by applying a evolutionary lens to changes in software.

Let’s start by identifying the kind of change requests we typically see in software.

1. Most of the time, changes come as feature requests for enhancing whatever capabilities already exist.
2. Once in a while, changing business needs aggregate into large change requests which require new things that the system has never had before.

In the latter case, the need for ground-up thinking is obvious. New capabilities or abstractions are being explicitly requested by the external world, and we must deliver. While such large changes are tricky to deliver on, they are also the more straightforward ones (in terms of what the output is expected to be). There is an explicit directive to “evolve”
the system.

Changes add up

However, the former case often also contains the seem need, though it is embedded a little deeper. Each small feature is innocent in isolation, but applying a systemic lens to it can sometimes reveal a more fundamental gap in technical capabilities. This gives us the opportunity to devise a more holistic solution that not only addresses the current requirement but also add some new fundamental capability to the system.

This is what all the advice around “understand requirements clearly” is talking about. We have to understand the immediate requirements properly, but we also have to read between the lines a little bit to see where the customer need is coming from and try to take the system there directly instead of traversing a morass of many small, disjointed changes feature requests.

Strategy is not just the arena of business. It plays an equally important role in how technology evolves. Simon Wardley has formally adapted this relationship of strategy and evolution into his “Wardley Mapping Framework”. As a CEO looks at market trends in aggregate and builds a strategy to evolve his company to keep up with them, the engineer has to look beyond the obvious requirement today to see if there is an emergent theme underneath a set of feature requests when seen in aggregate.

Evolving software at the edge of chaos

Let’s phrase the design-for-the-future approach in evolutionary terms.

Change requests, however small, are the environmental pressure for software evolution. Teams that can identify the driving forces behind seemingly small requests and develop coherent abstractions in their systems in time will have adapted the best to this pressure. They will live to see another day. Teams that consistently fail to do this will perish, something alongside their entire organizations.

Before the rallying cry of YAGNI etc starts, there is obviously a fine line to walk here. An over-engineered system is just as bad as an under-engineered when it comes to being a fit for the business landscape. We have to find a balance where we allow the emergent themes to be manifested somewhat clearly before we solve for them. Too far to one side is the chaos of hacks and piecemeal changes, too far to the other is too many useless abstractions that slow down everything else. When done well, a system grows new layers of abstraction and complexity just in time to prevent small needs from becoming big problems.

A constant evolution mindset is best put into effect when designing anything, but we also have to deal with existing code that is getting outdated due to changes to the ecosystem. Continuous refactoring can be a good way to encounter this. Refactoring is a great opportunity to identify scattered yet recurring patterns in code and see if there is an opportunity to aggregate them into something more concrete. It also gives a good sense check of whether the perceived “theme” is real (something happening multiple times in code) or just wrong intuition.

Large change requests are top-down evolution, but this is the kind of purposeful bottom-up evolution I have written about before. If we don’t adopt this features-as-evolutionary-pressure mindset, scattered solutions to small requests will pile up and result in increasing tech debt or an eventual large change request to the system. This “stop-everything-and-rearchitect” scenario is expensive and risky (since such efforts are liable to fail or underachieve).

TL;DR

My recommendation is to use feature requests as a breeding ground for the next generation of the system’s architecture. By continuously evaluating what we are being asked to change, we can jump the gun and get to the next level faster and often more safely.

Read Next: Using Agile practices to go beyond execution excellence