Real world systems for operating a complicated process don’t start out perfectly designed complete systems. New information reveals itself only after you’ve done it a few times. Failure modes you weren’t aware of become apparent only after the system breaks.
That’s why it’s important to build a “knowledge capture loop” into systems. A knowledge capture loop is a self-reinforcing mechanism for ensuring that knowledge generated by the system ends up feeding back into the system.
Unfortunately, most systems don’t intentionally design a knowledge capture loop—it tends to be an afterthought (e.g. a knowledgebase, FAQ, sporadic bug report). This is unreliable (people won’t do it consistently) and seldom results in change to the system (e.g. updating code to fix the issue).
What does a good knowledge capture loop look like?
To start, performing the action must happen in software. The greater the distance between the software and the action the harder it becomes to capture knowledge.
Issues that arise when performing an action or step should be addressed immediately. Ideally the operator can update it in situ with no code. Making it easy to fix things as you go increases the chances odds that new knowledge makes it into the place where the work performed.
The software should have knowledge capture built in. It should be more of a tool set, where operators can update behavior safely, change copy, and share information with as little friction as possible. It should be more wiki like than a sealed system that can only be changed with code. Make it easy to report problems, add edit buttons to make changes safely, etc. The goal is to have near zero friction to capturing new knowledge.
A good way to tell if this is working is whether or not you need a separate knowledgebase.