Software Research and Development in the Era of Generative AI

The instruction guide I was never given

Welcome to my series on research and development. I am wanting to write down some details of my experience as an AI Software Architect, because I believe what I’m observing is a shift in our industry that might become more common. This trend is a shift to research and development being a primary aspect of technology organizations. The immense power of generative AI and companies evaluating its usage in applications in industries over the last years is what I suspect is the primary cause. It’s a technology that requires a non-trivial amount of discovery to confidently utilize. As someone who has historically been in 100% product development, my perspective might be useful for other product developers making the transition into these discovery roles that are effectively research and development.

The Goal

Software research at a company is not without context: your company exists to make profit and likely has a strategy on this if they are already investing in research. You must get something profitable onto the product road map as quickly as possible.  Prototypes are effectively worthless unless they sell to customers or bring feedback to improve the next prototype.  You must be someone who cares about money to increase the likelihood that you are allocating your time on viable prototypes and not mad science.

Resource allocation at companies is not random, they will be assigned where profit is proven/obvious. If you do not align people on financial value, they will never position research ideas before other product roadmap items in any significant way to make traction. And when you fail to do so, you will likely not have the help of product team members (designers, project managers, testers, etc.), relegating your work half-designed code hiding behind a feature flag never to see the light of day.  

To get something on the product roadmap at your company, you must find out the existing mechanisms for how things arrive there. Your company may have a very specific sequence of steps, but the unity among all those decision mechanisms is that someone with authority at your company will have to clearly understand the financial impact of your prototype. This is a very challenging and difficult process because it involves research into a market from various specialized perspectives that you don’t have time to assess as an engineer.

What you’ll be doing

As a software engineer, you may be used to spending a majority of your time building, but in the role of R&D a new critical need arises: educating other people on technology.  People cannot evaluate the financial impact of your prototype if they do not understand it.  If they cannot envision it in a form their customers would use, they will not be able to tell anyone of authority its value and how to prioritize it.  Even with a process of education, it will likely require a tangible pilot with a customer to evaluate. Product managers can theorize on your arguments all they like, but hearing a key customer express their judgment is worth its weight in gold. 

It’s critical in research and development that you recognize your variety of connections across a company will make or break you.  You not only ultimately need the buy-in from various perspectives to define financial impact, but you will likely need their help in the identification of non-viable prototypes.  Because research cannot be funded forever, you are forced to economize your actions, and saying “no” to things as quickly as possible is crucial. The likelihood of your first prototype being high enough in value to compete with existing product priorities is very low.  You need a system that allows you to experiment with multiple prototypes and get feedback from experienced judges with various customer/business experience (product, sales, marketing, customer success). 

Additionally, you will need conversation with people who understand your business for your own inspiration of identifying technology opportunities. As a programmer, you have awareness of countless technologies, but others in your company may not be able to tell if the capabilities are science fiction or a real opportunity.  They don’t know if integrating those advancements would take years or weeks.  You will sometimes have to be a judge of technology that only you understand, and having a mental model how your business makes money will help you in this to identify the related and unrelated pieces more efficiently. 

A concrete example of this is something like vector databases.  To many people they have no idea what this is, and to some it’s just a buzzword they know popular. They have no idea how it fits into a technological system and what it improves. In generative AI in particular - being full of new tools/libraries - you will have to judge technologies like this on your own, and must do what you can to be prepared to see the value of these opportunities from a financial lens.

Common mistakes

Promoting prototypes and educating people are a core part of the job, and it can take a variety of skills to master the art. This could be articles of their own, but for now, I’ll offer some common mistakes i’ve made I think are worth mentioning: 

• It’s important that you not be  productizing. Your goal in a prototype is to do just enough to convey the technological opportunity to the critical people of the company who can define financial impact: no more or less.  If you do less, people can’t see the opportunity, if you do more, you prevent yourself from the exploration of other prototypes.  That isn’t to say that your prototypes should be esoteric pieces of code. Keeping them in line with existing architectures can make the productization easier when it does happen, and complexity of completing the final implementation will certainly factor in weighing it against other product initiatives. You must be careful not to go overboard though.

• Avoid exaggeration. People will form mental models about your work when it’s presented, and those mental models will only flourish so much as they point to reality.  If I go around telling people that generative AI is a “thinking machine”, they will be disappointed when they go to use it in reality and realize it lacks the reasoning of many adults.  If I describe AI as a “pattern identifier”, people will find in reality that it indeed identifies patterns, and a conversation can more appropriately occur on what patterns it can confidently identify or produce. You can mitigate exaggeration where possible with testing if it’s appropriate, and in the realm of AI, it is certainly vital.

• Avoid being in a state where you cannot demo your work for too long.  Again, remembering that feedback to say “no” quicker is crucially important, and people seeing what you’ve built can convey a lot of information.  You must live in the sun with your actions. Showing your work and your ideas often will help build trust and keep people in the loop so your mental models are fresh in their mind.

Finally, let’s imagine you build a prototype that has matched with product’s priorities in your company, you’ve done the complex task of educating your leadership to identify the financial impact, and placed it onto your product roadmap when weighed against alternatives — you still have one more thing to do.  Document the impact your prototype actually has after it’s been fully productized. You must remember all facets of the company are measured in investor value, R&D is not exempt and has a challenge of being something easily forgotten because it’s a pre-production phase. The prototypes you create have to justify the costs of successful research, productization, and failed research. That can be a very large number, so take it seriously.

Ad Astra

With this strategy, I hope you have a clearer picture of the goal, how to work with your broader leadership team, and the essential role of educating your prototypes and measuring the impact they have. It’s an exciting challenge in this world, and we live in an era where the rewards can be immense for new discoveries of pattern matching applied to productivity.