|
Hello Everyone!
Welcome to the 32nd edition of It Depends. Hope you are all doing well and staying safe. This week we are talking about that great bugbear of software engineering - documentation. You can read the article on the blog directly if you prefer that layout.
I have been writing about documenting code of late, so of course, my Medium recommendations threw out an article about “the real reason why developers don’t write documentation”. The article claims that the lack of good tools for writing is the biggest culprit in discouraging software engineers from documenting their work and decisions.
I usually don’t pick on specific articles, but this one triggered the hell out of me. The writer makes some okay points about diagramming tools, but the overall piece is so misleading that it obfuscates this important issue. If you are going to compare two drawing tools and claim that neither being good enough is the main reason for developers not writing docs, then either you are writing for clickbait or in bad faith.
I believe that there are two main reasons software engineers don’t write documentation. Tools play their part but they are a hugely distant third.
Writing is hard
Software engineers, like everyone else, don’t write because writing clearly is very, VERY difficult.
Writing is a tough, demanding task. It requires organizing our thoughts clearly, examining them critically, and expressing them clearly. While the expressing part can be simplified to some extent (depending on the quality of writing required), all three steps are taxing when done properly.
In the world of programming, where “it depends” is often the best answer and everything is based on trade-offs, writing becomes that much harder. It needs to set the context, justify the decisions, and then power the low-level thinking leading into the code. This type of writing is only useful if done well, and since doing it well is tough, it often doesn’t get done at all. Bad code will still fly, bad documentation won’t.
This is why a lot of people argue about the value of comments in code and the merits of self-documenting code (whatever that means). Kevlin Henney says that asking for comments around complicated code is futile because we expect the same people who could not express themselves clearly in code to now turn around and express themselves clearly in English.
Not documenting doesn’t block shipping
If a developer doesn’t write documentation, their work still gets done. Not writing doesn’t block shipping (at least not right away). The damage done by not documenting technical decisions is cumulative. Much like tech debt, it doesn’t cause damage in the here and now.
Like I said above, writing is primarily a matter of thinking and analyzing. In most places, coding can be done by the seat of your pants. A disorganized pile of classes and methods in code may work – a pile of work of words and paragraphs won’t work. Writing HAS to be clear if it is to be of any use. Code will be accepted (to some extent) as long as it does its job. And since most organizations focus only on getting the product shipped, that which doesn’t block shipping gets ignored.
Unit tests face a similar problem in many teams. To test the code we need to understand it (that takes more effort than writing it), and not having tests doesn’t block shipping. Ergo, no unit tests in code.
There is also the matter of obsolescence. Even good documents go obsolete, so engineers have to keep repeating the think-analyze-express over and over again as they build out systems. So dropping off the documentation wagon is easy. So even with best intentions, documentation often happens only in spurts of writing and cleanup.
What about the tools
There is no doubt that the commonly used set of tools used for documenting software today are woefully inadequate. We don’t think in terms of documents one at a time. We think in terms of ideas and goals by pulling together multiple concepts at once. The resultant document is just one manifestation of the thought process. We need tools that can help us collate ideas across time to solve the problem at hand. Google Docs, Confluence, Markdown are all poor tools for this type of writing.
However, a new generation of tools like Notion and Roam are attacking this problem of harnessing networked thought. Hopefully, these will work as intended and help in the thinking that goes into writing.
However, the lack of a second brain cannot really be used as an excuse for not using the first one. Tools play their part, but the willingness to undertake the process is the real hurdle.
So how to do documentation
Writing software has taught me one thing. If you really want your users to do something, then doing it has to be a blocking step in their journey with your product. In the same way, tacking on documentation to written code is never going to work. Worse, it is useless. Writing is about critical thinking. It is meant to explain your thought process and intent to yourself and to your audience (e.g. your team). The thinking process is where documentation/writing adds value, not as a static record of already implemented code.
Proponents of mob/pair programming and XP often disparage documentation. But barring the adoption of those techniques, the practice of writing and reviewing technical documents is the only way teams build a collective understanding of what they are trying to build. This shared world-building is what makes this process critical to the long-term health of the team and the codebase.
I feel that the only way to make the process of writing documentation sustainable is to make it a blocker for software development. Make it lightweight but mandatory. It should become part of the process instead of being yet another thing to do. Some things that have worked for this in my experience.
- Write before you code - Unless the change is trivial, every engineer writes a note about what they are going to do and runs it by the rest of the team. At the end of the discussion, the actual coding should become trivial.
- Write simply - Don’t complicate the writing, at least until it becomes second nature. Diagrams, fancy sections etc can wait. Write very simply about what you thought, what you are doing, and why. Even if the document can serve as a basic pointer to the rest of the team now and in the future, it is superbly valuable.
- Document the decision with their alternatives – Rather than documenting the actual implementation (which may change over time) in detail, focus on documenting the choices and why they were made. This is what the code cannot ever explain and hence writing it down adds the most valuable. Details can be documented based on the time you are willing to invest.
- Make it searchable – No amount of documentation will be of any use if people cannot find it. Use tools that support text searching out of the box. This is one of the reasons I like Google Docs for documentation. It is great for writing but just horrible for collaboration and discovery.
- Track changes - Some organizations use version control to track changes to the system’s design over time. That’s great. But if you are not there yet, keep one document per feature and keep putting dated updates on it so that evolution can be tracked in one place with minimal hassle.
The hope is that as the team seems the merits of having and reviewing some documents (e.g. new members need lesser hand-holding) and writing becomes muscle memory, the practice will become self-sustaining. Till then, it should be treated like working out or dieting – painful but necessary.
From the internet this week
- If you have not yet tuned into the Linux-UMN drama, this article will get you started. Apparently, a research team from the University of Minnesota started introducing random changes into the Linux kernel, go caught, got banned, and had all their changes very publicly reverted. Popcorn required!
- Justin Jaffray explains push and pull-based query engines and the context in which they can be used.
- The folks over at Software mill have done a great visualization that explains how Apache Kafka works. Check it out.
That's it for this week folks!
Cheers!
Kislay |