I have been writing about documenting code of late, so of course, my Medium recommendations threw out an article about “the real reason why developers don’t write documentation”. The article claims that the lack of good tools for writing is the biggest culprit in discouraging software engineers from documenting their work and decisions.
I usually don’t pick on specific articles, but this one triggered the hell out of me. The writer makes some okay points about diagramming tools, but the overall piece is so misleading that it obfuscates this important issue. If you are going to compare two drawing tools and claim that neither being good enough is the main reason for developers not writing docs, then either you are writing for clickbait or in bad faith.
I believe that there are two main reasons software engineers don’t write documentation. Tools play their part but they are a hugely distant third.
Writing is hard
Software engineers, like everyone else, don’t write because writing clearly is very, VERY difficult.
Writing is a tough, demanding task. It requires organizing our thoughts clearly, examining them critically, and expressing them clearly. While the expressing part can be simplified to some extent (depending on the quality of writing required), all three steps are taxing when done properly.
In the world of programming, where “it depends” is often the best answer and everything is based on trade-offs, writing becomes that much harder. It needs to set the context, justify the decisions, and then power the low-level thinking leading into the code. This type of writing is only useful if done well, and since doing it well is tough, it often doesn’t get done at all. Bad code will still fly, bad documentation won’t.
This is why a lot of people argue about the value of comments in code and the merits of self-documenting code (whatever that means). Kevlin Henney says that asking for comments around complicated code is futile because we expect the same people who could not express themselves clearly in code to now turn around and express themselves clearly in English.
Not documenting doesn’t block shipping
If a developer doesn’t write documentation, their work still gets done. Not writing doesn’t block shipping (at least not right away). The damage done by not documenting technical decisions is cumulative. Much like tech debt, it doesn’t cause damage in the here and now.
Like I said above, writing is primarily a matter of thinking and analyzing. In most places, coding can be done by the seat of your pants. A disorganized pile of classes and methods in code may work – a pile of work of words and paragraphs won’t work. Writing HAS to be clear if it is to be of any use. Code will be accepted (to some extent) as long as it does its job. And since most organizations focus only on getting the product shipped, that which doesn’t block shipping gets ignored.
Unit tests face a similar problem in many teams. To test the code we need to understand it (that takes more effort than writing it), and not having tests doesn’t block shipping. Ergo, no unit tests in code.
There is also the matter of obsolescence. Even good documents go obsolete, so engineers have to keep repeating the think-analyze-express over and over again as they build out systems. So dropping off the documentation wagon is easy. So even with best intentions, documentation often happens only in spurts of writing and cleanup.
What about the tools
There is no doubt that the commonly used set of tools used for documenting software today are woefully inadequate. We don’t think in terms of documents one at a time. We think in terms of ideas and goals by pulling together multiple concepts at once. The resultant document is just one manifestation of the thought process. We need tools that can help us collate ideas across time to solve the problem at hand. Google Docs, Confluence, Markdown are all poor tools for this type of writing.
However, a new generation of tools like Notion and Roam are attacking this problem of harnessing networked though. Hopefully, these will work as intended and help in the thinking that goes into writing.
However, the lack of a second brain cannot really be used as an excuse for not using the first one. Tools play their part, but the willingness to undertake the process is the real hurdle.
So how to do documentation
Writing software has taught me one thing. If you really want your users to do something, then doing it has to be a blocking step in their journey with your product. In the same way, tacking on documentation to written code is never going to work. Worse, it is useless. Writing is about critical thinking. It is meant to explain your thought process and intent to yourself and to your audience (e.g. your team). The thinking process is where documentation/writing adds value, not as a static record of already implemented code.
Proponents of mob/pair programming and XP often disparage documentation. But barring the adoption of those techniques, the practice of writing and reviewing technical documents is the only way teams build a collective understanding of what they are trying to build. This shared world-building is what makes this process critical to the long-term health of the team and the codebase.
I feel that the only way to make the process of writing documentation sustainable is to make it a blocker for software development. Make it lightweight but mandatory. It should become part of the process instead of being yet another thing to do. Some things that have worked for this in my experience.
- Write before you code. Unless the change is trivial, every engineer writes a note about what they are going to do and runs it by the rest of the team. At the end of the discussion, the actual coding should become trivial.
- Write simply. Don’t complicate the writing, at least until it becomes second nature. Diagrams, fancy sections etc can wait. Write very simply about what you thought, what you are doing, and why. Even if the document can serve as a basic pointer to the rest of the team now and in the future, it is superbly valuable.
- Document the decision with their alternatives – Rather than documenting the actual implementation (which may change over time) in detail, focus on documenting the choices and why they were made. This is what the code cannot ever explain and hence writing it down adds the most valuable. Details can be documented based on the time you are willing to invest.
- Make it searchable – No amount of documentation will be of any use if people cannot find it. Use tools that support text searching out of the box. This is one of the reasons I don’t like Google Docs for documentation. It is great for writing but just horrible for collaboration and discovery.
- Track changes. Some organizations use version control to track changes to the system’s design over time. That’s great. But if you are not there yet, keep one document per feature and keep putting dated updates on it so that evolution can be tracked in one place with minimal hassle.
The hope is that as the team seems the merits of having and reviewing some documents (e.g. new members need lesser hand-holding) and writing becomes muscle memory, the practice will become self-sustaining. Till then, it should be treated like working out or dieting – painful but necessary.
Read Next: How to speed up software delivery
If you liked this, subscribe to my weekly newsletter It Depends to read about software engineering and technical leadership
I think you meant well writing this article, but as a longtime developer who has seen the horrors of bad documentation, I think the passing comment on “self-documenting code” is a key point you missed.
For context: I worked at Microsoft, where we had old, crusty documentation from devs who had long since left the company, and new, wrong documentation from devs who had just joined the company.
In both cases, the documentation could be a red herring, and in the end a good developer’s job is to figure out what the code *does*, regardless of what the documentation says. I can’t tell you how many times well-intentioned comments have cost me days, months, or weeks of my life. In my early sev days, I had a tester who used the wrong version of a file open call because there was nothing about the documentation noting the side effect of opening the file read-only. The tester just auto-completed the first sensible sounding call available, and checked the documentation to see if it did what she wanted.
Later on, the lead architect of our product, who writes thousands of lines of gorgeous code a day, would add helpful comments like /* fRefreshCache */ to his function calls, and when it turns out the parameter was actually changed to fDoNotRefreshCache, that was four days of my life debugging his code to find where a Boolean got flipped.
Today, I debug scientific software, written by computational chemists (read: great chemists, very messy programmers). Their comments regularly don’t match the code logic, because someone else came in and changed a call without understanding the upstream/downstream changes, or because they simply forgot to update the comments.
This is why “self-commenting code” is invaluable to me. My opinion is that the most important and valuable job of a programmer is not to write working code, but to write code that will save other programmers time and energy. That means code that’s easy to read, easy to understand, and easy to navigate. When variables are named intuitively, and the code’s behavior matches the way it’s written, I don’t need comments unless I’m facing an obscure algorithm or scientific requirement.
Thus, by and large, I like my code to look obvious. Better yet, if I can, I try to make it look *boring.*
To me, that’s what self-commenting code is. Code that doesn’t have any surprises, and can be easily understood.
Edit: I wish we had compilers for documentation…
I wish we had completed for documentation (we actually do have done, but they’re not fool proof tools when it comes to analyzing human level languages). Then only we would be able to make sound judgements on the quality of any documentation, and making sure it evolves as code changes.
As a lot of commenters said, maintaining documentation in an ever changing environment is a big effort, same for tests. I have spent dozen of hours doing beautifully documented and tested code that were discarded in next sprint because it lead to new discoveries. The most common mistake is to just blame coders, while usually the economical pressure does not allow for the extra time spent.
For me, the key for documenting and testing is knowing what are the key parts of your software that are more likely to stick around, grow in complexity and cause complicated bugs.
One tip on good documentation is to explain in commit messages WHY those changes were made. And if in 2021 you still don’t have version control, don’t expect to go too far in having a well documented software. I would prioritize that over anything else.
I think, you’re onto something very important, when you write about documenting the decision with their alternatives. I would add, that it is even more important to document the problem which lead to the decision, and which the software is supposed to solve. For me – it is the ultimate goal of software development: solving real life problems.
Some other reasons why people avoid writing documentation are:
– It’s not read. Thus you spend a lot of time documenting while nobody will ever read it.
– Software is changing often, so the documentation tends to go out of date all the time.
– Software should not need that much explanation, often it’s better to create documentation/inline help in the software instead of a separate file.
I think another reason documentation doesn’t get done is opportunity cost. Why use time trying to explain written code when you can write more code?
From a 30,000 foot perspective, the problem is NOT that documentation is not being written but instead that writing code and writing its documentation are separate tasks, one of which is optional, like brushing your teeth (you’ll regret it later).
Our next programming language should incorporate documentation and testing content into the code. So for example, if you are going to add a block of code, you would have to explain what it does (“Converts any ASCII characters into UTF-32”) before it is added into the module and expectations for that block of code (“Expects input String of UTF-8 characters” and “Expects to output String of UTF-32 characters”) as well as test assertions (“Byte[] -> UTF-32 String, UTF-8 String -> UTF-32 String, UTF-16 String -> UTF-32 String, UTF-32 String -> UTF-32 String”) would be required as well.
And then after compiling you can hover over that block of code and it will display a balloon with its description. And when tests are run, it automatically generates the test code to needed to verify that the assertions are met by your code. And if you right click on the block, it automatically displays a simple flowchart that was generated from your code.
As additions or corrections are needed, the developer needs to explain the need for the changes as well as update the Expectations and Assertions to match the changes. If “brushing your teeth” isn’t an option, it won’t get skipped and the documentation will always be up to date.
Any block within the module would have the same features integrated into them, which means that whole module would have those capabilities built-in, allowing you to hover over any block of code and see what it does or see a high-level flow diagram of the whole module and allow you to drill down into the individual blocks. These features would then make every module in the whole program have those “auto-documenting” and “auto-testing” capabilities that always stay current because it would be a required to make changes, the same way we now have to make sure parentheses match and statements end in semicolons.
Any new programming language that doesn’t include something similar to what was described above simply means that we will continue to be plagued by out of date or missing documentation and tests.
So I have a different perspective on all this. I’m both a software developer and a published author, so I find writing good documentation to be pretty straight forward. The biggest issue in writing documentation, I find, is the burden of maintaining it. I’ve found engineers to be pretty diligent in keeping Swagger documentation current, and it’s no surprise that this is because it does a lot of the work for you. But maintaining README and other forms of static documentation often feel like a futile effort in an agile environment where software is changing.
I think the key point here is that writing is hard. For everyone. Especially if you expect to create useful documentation in your first draft. As for tools, I think you’re right about keeping it simple and allowing collaboration. New tools like Notion and Roam do feature a strong focus on networking and collaboration. But don’t count out Google Docs, which also has great collaboration features for individuals or teams.
I’m not sure why you’re negative about Markdown, which is not really a writing tool, but a publishing tool. There are many editors (like StackEdit or Dillinger) that make creating Markdown relatively easy. I’ve created and add-on for Google Docs (Docs to Markdown: https://github.com/evbacher/gd2md-html), that lets you draft and collaborate using Google Docs, then convert to Markdown or HTML for publishing.
I was confused by these two sentences:
This is one of the reasons I like Google Docs for documentation. It is great for writing but just horrible for collaboration and discovery.
There is a contradiction here – it sounds like Google Docs does not meet your criteria for “search-ability” but you say you “like” it.
This is not contradictory at all. Google provides good tools for creating documents, but these documents just sit there. They’re not easily discoverable – tools like Confluence are built for organizing content and collaboration.
You nailed it on the reasons – especially that it is hard. Documentation (especially good documentation) requires deep work and most developers (heck anyone) has lost the ability to do deep work.
My experience is that all too often there is too much discussion about documentation getting in the way of actually doing it. Where to put it. How to organize it. Blah blah blah. Meanwhile what is asked for isn’t what is useful and thus little to gets written and even less that has any value.
Things I would like to see is an overview of the layout. Then documentation that follows said layout. Small stuff needs small documentation. Larger stuff…