Understanding and leveraging the potential of Large Language Models (LLMs) is an essential goal for many technology leaders.
However, in my discussions with our clients, I often come across a common misunderstanding of how these systems work. While it’s easy to lump LLMs with traditional computing systems, they are fundamentally different. Computers are deterministic. They follow a chain of logic (i.e., a program) to generate output from an input. In contrast, LLMs, or Machine-learning-based systems generally, are prediction machines. They predict an output based on their input, often with varying degrees of uncertainty. As a result, trying to decide whether to use LLMs can be a difficult decision.
The key to effectively leveraging LLMs is understanding this inherent uncertainty. When building systems that use LLMs, measuring your appetite for uncertainty against the model’s current capabilities is critical. Evaluating the performance of LLMs can be challenging. Still, one helpful heuristic is the Best Available Human (BAH) standard, as coined by Ethan Mollick in a recent post.
It asks whether a given AI can outperform the best available human in solving a particular problem at a specific moment and location.
For example, a GPT-4 level LLM can summarize and extract insights from a large amount of text data faster and, given the time constraints, as accurately as a human analyst. Therefore, utilizing LLMs in such a context could be helpful as a stand-alone solution or a tool to augment human capabilities.
Decisions under conditions of uncertainty are best made through empirical experimentation. The most effective way to understand the utility of LLMs for your domain is to experiment.
Using APIs from OpenAI or HuggingFace, build prototypes to identify the problems LLMs can solve in your area. At Jeavio, we understood this when we organized a two-week hackathon allowing our engineers to build anything, provided they utilized LLM capabilities. The outcome was so promising that we created an Applied AI team.
In summary, while AI is not a panacea, it holds transformative potential when its capabilities surpass or augment those of the best available human resources.
If you want to learn more about our AI hackathon, check out this post. If you are interested in how other companies use LLMs, here is a post that you might find helpful.
If you are curious about how to use LLMs to solve your problems, drop me a line!
RELATED POSTS
11-01-2023
The Blank Chat Screen: UX Challenges of Large Language Models
Large Language Models have a User Experience (UX) problem.This screenshot from ChatGPT shows the problem and OpenAI’s attempted solution. Simply put, most users need help figuring out what to do when faced with a blank chat screen. An empty chat gives us no hints about the possibilities of the technology.OpenAI tries to solve this [...]
09-27-2023
R.I.S.E. with Jeavio: Redefining Innovation in Enterprises
In the world of enterprises, unlike startups, fostering an innovation culture often takes a backseat to more immediate priorities. Business leaders find themselves struggling with the 'Innovation Paradox.' This paradox suggests that the best time to invest in strategic innovation coincides with periods when organizations feel the least [...]
07-26-2023
Using Large Language Models
I had the pleasure of moderating a stimulating discussion around the use of Large Language Models (LLMs) for Blue Cloud Ventures last week. Presenters from 10 varied B2B SaaS companies shared their current and planned use of LLM capabilities. We covered a number of use cases, challenges, as well as discussed vendors and the broad opportunities [...]
06-20-2023
Chatbots are not the only Conversational Interface
ChatGPT, Pi, Claude, Bard, BingChat, and others are all chatbots built on top of Large Language Models like GPT-4. ChatGPT is very popular because it is easy to use. It leverages the underlying large language model’s ability to parse natural language to present a compelling user experience. You ask it questions, prompt it to do[...]
