Moving Through A Problem Space in Software Development
The Categorization of Software Problems
As a software engineer, I come across various problems. The problems can be categorized as simple, complicated, complex, and chaotic. When I meet with a new problem, the first action I take is to categorize the problem so that the best possible action to move through the problem space can be detected.
Here in this article, I would like to talk about the categorization of software problems by utilizing the Cynefin Framework terminology.
Simple Problems
The simple problems have a well-defined environment. As the problem space is well defined, there are also best practices readily available for these kinds of simple problems.
For example, when the problem is increasing the readability of a code base to make it understandable by average programmers, there are clean coding practices that work well.
The solution to the problem is quite clear. There are best practices available. The problem consists of “known knowns” in Cynefin terms.
Even though there exist best practices for simple problems, we mostly meet with harder problems that have no best practices readily available.
Complicated Problems
Here it is possible to work rationally toward a decision, but doing so requires refined judgment and expertise.
[2]
When the problems have no best practice, we mostly refer to the experts in the field to find good practices available in the problem domain. The complicated problems can be solved by getting help from domain experts and using their non-obvious techniques.
For example, in software architecture, we mostly meet with good practices rather than best practices. Almost any software architecture we meet differs from any others.
As the architecture is novel, there are no best practices for the problem space. But, there are a set of good practices that worked well for similar problems encountered by other experts in the software architecture field. In these cases, we make a trade-off analysis and apply the good practices.
In complicated software problems, we observe the anti-patterns getting detected by the experts. These anti-patterns define the practices that seem great at first but turn out to be quite damaging in the later stages.
The complicated problems consist of “known unknowns” in Cynefin terms.
This is the province of engineers, surgeons, intelligence analysts, lawyers, and other experts. Artificial intelligence copes well here: Deep Blue plays chess as if it were a complicated problem, looking at every possible sequence of moves.
[2]
Complex Problems
Sometimes, we can’t even find any good practices available for the problem. The only way to move ahead is to take the best possible action. As we tackle a complex problem, we start to see the problem space. The forces that affect the problem, and the characteristics of the problem start to appear as we carefully review the results of the actions we take.
The good practices available for the problem appear only after the problem is resolved. The complex problems consist of “unknown unknowns” in Cynefin terms.
For example, designing any new system consists of unknown unknowns. The emergent design through test-driven development leads us to take the best possible actions.
As the parts of the new system emerge, the sub-problems converge to complicated from the complex. Then, it becomes easier to solve the complicated sub-problems rather than trying to solve the complex problem.
Stewart identifies battlefields, markets, ecosystems and corporate cultures as complex systems [2]
Chaotic Problems
In the chaotic domain, cause and effect are unclear.[2]
There are other times when there is no correlation between the actions we take and the results we get. There are just too many variables moving on at the same time.
In these chaotic problems, the only way to move ahead in the problem space is to take action and try to converge the problem from chaotic to complex to further resolve it.
An example chaotic software problem is legacy software. Imagine a legacy system where there are no domain experts, no unit tests, and quite a complicated codebase combined with a tightly coupled monolithic architecture.
Events in chaotic problems too confusing to wait for a knowledge-based response.
In the chaotic domain, a leader’s immediate job is not to discover patterns but to staunch the bleeding. A leader must first act to establish order, then sense where stability is present and from where it is absent, and then respond by working to transform the situation from chaos to complexity, where the identification of emerging patterns can both help prevent future crises and discern new opportunities.
[2]
Towards The End
As software engineers, we need to be careful about the problems we face. We should be cautious before determining the type of problem. Also, we should frequently observe the problem domain for the changes and detect the changing type of the problems to better respond to them.
Even though the best practices and good practices are readily available online, they only help us solve simple and complicated problems. We are mostly left alone with complex and chaotic problems even in the highly connected online world.
Therefore, as software engineers, it is best to take a humble approach to any problem we meet as we will only solve a fraction of them by using our existing knowledge.
References
[1] J. J. Sutherland, “There are 4 main types of life and work problems we face every day. Here’s how to solve each one,” CNBC, Oct. 30, 2019. https://www.cnbc.com/2019/10/30/4-main-types-of-life-and-work-problems-we-face-everyday-how-to-solve-each-one.html (accessed Aug. 15, 2022).
[2] Wikipedia Contributors, “Cynefin framework,” Wikipedia, Aug. 15, 2022. https://en.wikipedia.org/wiki/Cynefin_framework (accessed Aug. 15, 2022).
