Have you run across this scenario before?
- Dev team keeps getting tight deadlines for feature delivery
- It responds by shipping tech debt-laden code
- This tech debt is never paid off because there is never time to clean-up
- Goto 1 a few times
- Tech debt gets worse till it starts actually blocking new changes or breaking things in production
- Dev team campaigns for a major redesign because the current system is beyond saving
- This will take a lot of time, but the business team is promised technology indistinguishable from magic at the end
- The re-arch is executed by putting everything else on hold
- It takes far longer than imagined and yields far less impressive results,
- Goto 1 a few times
- Business teams lose all faith in the dev team and go back to using excel/Company shuts down
The above sequence of events is unfortunately far too common. While there are many causes and many corresponding courses of action to prevent this, I want to look at what the dev teams can do. The biggest problem to me is that repeated big re-arch. Like startups, big redesigns mostly fail. They fail at containing scope, they fail at delivering on their promises, they fail at levelling up the organization’s capabilities, often they fail at being complete at all. A truly agile organization should be surprised if it finds itself needs such a large one-shot change – whatever happened to discovering and implementing solutions incrementally!
However, there is something that dev teams can do to prevent this, and that is to adopt a culture of continuous refactoring. More than sprints, scrums, or daily stand-ups, continuous refactoring can make a dev team agile by improving what they work with, i.e., the code base.
Martin Fowler has written written a great book and many articles about Refactoring (including this interesting aside on the etymology of the word) so I will assume we all know what it is. Let’s jump into why I advise “continuous” refactoring.
Why refactor continuously
Consistently keeps the code base of high quality
For a software developer, a well maintained codebase is its own reward – like a personal Monalisa or a zen garden. It is a pleasure to work in, it is easy to get on-boarded to and to understand, and a team with such a unicorn is generally likely to be more productive.
Makes everyone familiar with code
Software is never “done”, so it is important to stay up to date with its “current” form to work effectively with it. If every on the team in tinkering around in the code base looking for improvement opportunities, the odds are that more than one person has seen the obscure by-lanes of it and can help in case something happens to that part. This is especially relevant for large codebases that have built up over a long time and seen multiple team members come and go. Refactoring and being on-call are the two most powerful tools I can think of for spreading broad knowledge of all aspects of the code among all. team members.
Sets up the next generation of the architecture
Everyone wants microservices right? If you are saddled with a big-ball-of-mud monolith, most likely you can’t wait to break it apart . But how to do this safely? The large, hairy beast has tentacles that you probably don’t even know of. Who knows what breaking-off a module will do? The actual splitting is simple enough, it is the unpredictable fall out that we must be wary of – we do not want to end up with a distributed big-ball-of-mud!
There is one sure shot way to set this up – start refactoring this legacy code base. You will learn things about the code that you didn’t know, previously muddled boundaries will emerge, dependencies between various parts will become clearer, bugs will be found and squashed, and low-hanging performance improvement will be made. You will develop a deeper understanding of where to go than would otherwise have been possible. Who know, may be at the end of all this, you won’t want micro-services because you will have an awesome, super-efficient monolith.
And most importantly, the business will keep running through all of this!
Even when we are working with new code, continuous refactoring reveals emerging boundaries and behaviours are in our code and this insight makes a bigger architectural change far more likely to succeed because we are not coming at it requirement from a “this-sucks-lets-rewrite-everything” mentality but from actual signs emerging in the code base from attempts to solve real business problems. Experienced developers can generally identify these patterns and set up the team towards powerful changes (e.g. split this module into a separate service, let’s put circuit breakers in the central client library). When everyone on the team is doing refactoring, more and more people start getting this sixth sense of sensing incoming change.
How to refactor continuously
There are many ways of doing this and a ton of advice on the internet. I want to focus on two aspects, one technical and one organizational.
Write tests
The oft-ignored adjunct to this oft-repeated advice is – Write tests. Refactoring doesn’t sit in some sort of technical vacuum – it is a means to an organizational end. I have read different perspectives about this, but to me, the objective is to increase shipping velocity. What we do today should make future changes faster/easier/safer. Now if we agree that refactoring today facilitates future changes, then it is a no-brainer that having testable code facilitates refactoring. We don’t have to think only in terms of changing code, we can think in terms of adding tests as well. We may not do the refactoring, but due to the extra tests, anyone else can do it safely tomorrow.
Writing test IS refactoring.
How to make time
“We”d never get time for this”.
I agree, and it is sad but true. It takes a lot of trust to get exclusive refactoring time, especially around parts of code which aren’t actually broken (but could be improved).
I prefer the widely popular “buffer your estimates” approach. Ask for an extra day to finish the task. It is easier to refactor what you were going to modify anyway, stakeholders are less likely to complain about a day or two worth of estimation as much, and we will be able to continually chip away at cruft. This is easier than getting time allocated for “tech only” changes separately.
Doing this also set bounds on how much time we will spend on refactoring – it would make things better but only in the scope of the given business feature that this is being funded via. We are far less likely to boil the ocean in these situations because there is a larger goal to be achieved – you refactor only what you are working with at the moment.
Conclusion
It is, in my opinion, unfair to expect that a team would “obviously” do continuous refactoring. It is not obvious, and it is seldom a priority in face of the pressure of delivering features. It is a culture, and like all cultures, should be explicitly talked about, discussed, and encouraged. If we can interpret our work in terms of making things a little better everyday, the gains add up really fast.
