The Psycho-Epistemological Nature of Software Development Subject Matter Experts

Individuals within software development organizations display a great diversity of competency for different tasks. There are various aspects of these people that contribute toward their competency: experience, passion, and by far the most significant dimension - familiarity with organizational tasks. Familiarity with tasks is shown pronouncedly in a class of developers known as subject matter experts (SME). In this article, I will discuss what is essential to SMEs, how they are formed, and how managers can enable their creation through process design.

But first, why should we care to be aware of how SMEs are formed? The primary reason is the positive impact on many kinds of business strategies. Whether increasing the amount of work done, reducing costs, reducing risks, or innovation, they accelerate the pursuit of improvement. By mastering the art of creating SMEs you are making the most out of your resources. More SMEs will: 

• Increase the velocity of code implementation

• Reduce meeting time for knowledge gathering 

• Prevent burnout by distributing challenging tasks amongst more minds

• Increase the likelihood of seeing critical problems in advance

These individuals often have exceptional effectiveness with a particular part of a codebase. Their distinctive ability arises because an organization's problems are often unique. While many of an organization's problems have similarity to others in an industry, even a programmer with deep knowledge in general development may not have knowledge applicable to the choices that have been made within the organization’s codebase (architecture choices, performance choices, tech debt that accumulated to reach deadlines, etc.)

SMEs are formed through intentional effort. The harsh reality is that time alone is not sufficient. Some people exist within companies who have been working for years and not developed into one; even between two people with similar skills, passion, experience, and time at the same company, one might develop into a SME but the other might not. What’s happening here, what makes the difference between people, and why is time not enough? We will discover that crucial to understanding what makes a SME is understanding the psycho-epistemology of software developers. Ayn Rand defined psycho-epistemology as "the study of man's cognitive processes from the aspect of the interaction between the conscious mind and the automatic functions of the subconscious." What this means is, we’re going to be looking how the mind works and why it’s related to SME formation.

How to make a SME

Subject matter experts (SMEs) are formed by a very specific psychological process. Man’s consciousness has a certain nature of forming knowledge, and when action is applied ( no matter how long ) that does not adhere to that nature, it will not develop into a knowledge base with the equivalent sophistication of these effective individuals. Managers know if you want something done fast, go to a SME, and if you have the luxury of time perhaps a task can go to a non-SME to learn a new area. But what do we mean by these phrases “go fast” and “learn a new area”? A software developer is able to see the entire source code, all information is available to them. What differentiates a SME is the ability to quickly connect a task to non-explicit information laced inside the code base and to the decisions of the organization itself. How they do this, and what makes their work seem fast, is through the efficient recall of relevant information to a task and its subtasks via the subconscious.

Why the subconscious? Humans are limited in the amount of information they can maintain in their consciousness at any given time. When working on code, software developers do not have every line of code spinning actively in their brain. Certainly while many find it easy to recall details from one’s daily activity, but SMEs are recalling information from a wider structure of knowledge. Subconsciousness, is the place of that long term memory, primed for automatic recall for when the consciousness beckons for suggestions.

They are neither people with wildly intelligent brains who can parse through code quickly, nor people with unusually large working memory capacity. The subconscious is the determining factor, but how does it get there in the first place, and why does it seem like all people do not become a SME through sheer exposure to information in the office? Forming long term knowledge in the subconscious has requirements of its own and does not happen automatically. Man is a volitional creature; firstly, they must want to focus and form knowledge. Much like a student will never learn in class if they do not choose to focus on what a teacher says. Second, they cannot form knowledge for what they don’t have exposure to. Regardless of one's desire to develop into a SME, without a breadth of experience ( given opportunities, time with mentors to see different problems, etc. ) and given time to digest knowledge, it will not happen.

Rational conceptualization in particular is the essential process of what allows knowledge to form in the subconscious mind in a useful form. When we go through life, we are exposed to an innumerable number of objects. We simply cannot remember every individual object we see, and our mind has the ability to abstract these individual objects we see into concepts that we can economically retain. For instance, I do not remember every individual banana I have seen in my life, but I have a concept of a banana by which I can use and recall in a situation where fruit/yellow things/sweet things are mentioned.

The difficulty of forming conceptual knowledge in the workplace lies in the absence of explicit terms that anchor our experiences for recall. “Banana” and “fruit” are words I and everyone else have grown up with, and as soon as they are mentioned a flood of information comes to my mind for use when it does. As programmers however, we encounter many code patterns, data management techniques, and algorithms that simply do not have names. If you are lucky, a wise software engineer might have formed a jargon of an industry problem, but in the scope of your organization's unique tasks, if we are not forming concepts and labeling them mentally, we will have a mind full of individual snippets of experiences totally unconnected and thus unrecallable because they are not related to anything.

In order for the subconscious to store knowledge that is useful and recallable, we must have time to be able to see the similarity of objects in a meaningful way and intentionally recognize the most important connections. Consider if in my experiences with bananas, I only spent enough time to recognize they had a peel. I might associate a banana with an apple and orange. Outside of other things with peels I might give it no further relevancy as a concept in my mind had I not paid attention to its color, shape, or taste. All these aspects have the potential to be important, connections of the concept of a banana to other groups of things is useful (yellow things, long curved things, and sweet things). However, it’s most important we pay attention to aspects meaningful to our work. For instance, if I were a baker, it’s probably more important I know the sweetness aspect of the concept of bananas. Similar as a software developer, there are concepts we are forming with many aspects, some associations more potent than others, i.e. more connected to our wider knowledge base of our code than others. For instance it might help me to create concepts of code structures that have security concerns, CPU utilization concerns, are confusing for other programmers to read, or need thought around design patterns.

Let’s review the above quickly, before turning to consideration of how we use this knowledge in organizational process:

• Observed SME performance by outsiders is the observation of the SME’s efficient subconscious recall of information relevant to their tasks.

• Information is not stored in long term subconscious unless multiple experiences in clear conditions can be compared rationally

• Knowledge does not form automatically and must be desired by an individual to be created.

• We need to put in effort to rationally connect aspects of concepts that meaningfully tie to our actions and other knowledge.

This article isn’t a masterclass in psychology, but I hope i’ve given you a reminder of some important broad strokes we can focus on for utility.

Metrics for management

To improve organizational process with our knowledge of psycho-epistemology, effects must be measured. Given what we know now, we see that to quantify this type of improvement, we need to identify units of actions that increase rational conceptualization of knowledge or application of concepts. Some actions we might quantify include:

• Documentation - by documenting, an individual is forced to write and label concepts they’ve seen into a coherent structure. It might be architectural patterns, performance techniques, testing standards, etc. All of these solidify understanding for the writer and can help accelerate future creation of SMEs in the same domain who read it.

• Long term specialization - aligning organizations deeply with strategic initiatives rather than short term goals has the side-effect of creating repeated exposures to problems in a similar domain. This prevents the mental thrashing that occurs when someone jumps from radically different tasks that don’t allow one to see common aspects within the time of memory.

• Knowledge sharing - coders focus cannot be everywhere at once, but creating forums for people to share patterns they’ve seen can accelerate conceptualization of people who see similar patterns.

In science fiction, we might imagine a world with neuro-implants that allow us to track if people are maximizing subconscious recall of concepts. We could use this tool during the implementation of tasks, and see who and what actions are correlative. In the meantime we need our own introspective form of tracking. Individuals know their subconscious best, so to quantify this we need a system to create data points based on their own statements. Consider a survey of questions like these:

• “Do you feel like you are working on tasks you’ve seen before and are applying wisdom gained from previous work?”

• “Do you feel you have time to mentally digest tasks you’re working on?”

• “Do you feel you’re able to share knowledge created while working on your tasks with other people? How often?”

• “Do you like thinking about the problems you are working on?”

If over a long term, some process change suddenly turned all these answers into “no”, you will want to be aware of the corresponding drop in performance due to the cognitive inefficiencies created. The nature of human cognition is constant; if operations hamper it, performance will be affected.

I think it’s fair to say that this kind of tracking of conceptualization is not common in our industry, but I believe with observation, a team can see velocity shifts of an environment conducive for SME formation, hopefully in the positive. This is a science of the cognitive conditions of our organizational environment that can help managers see new dimensions of the impact to process change and interruptions. While many organizations may not have the luxury of making everyone a SME, we can try our best within financial limits to create a process that does its best by being aware of deeper effects of process policy. I hope this article has equipped you with a new tool for enhancing the efficacy of programming teams and perhaps knowledge workers more broadly.