Skip to content
Let's talk

Futurum Research

Insight · Analyst Insight

AI Platforms3 min read

Large Language Model AI Needs to Be Invisible and Cheaper

Research Director, Hybrid Cloud & Infrastructure

The short answer

Alastair Cooke, CTO Advisor at The Futurum Group, shares his insights on the changes needed for organizations to overcome the complexity and cost of LLMs and finally get tremendous business value from them.

Futurum's Alastair Cooke,

Cite this

Alastair Cooke, The Futurum Group, "Large Language Model AI Needs to Be Invisible and Cheaper," May 7, 2024. https://trial.futurumgroup.com/insights/large-language-model-ai-needs-to-be-invisible-and-cheaper/

Large Language Model AI Needs to Be Invisible and Cheaper

There is no doubt that large language model (LLM) AI is revolutionizing the ability of a computer to augment or replace human effort. The challenge is taking that revolution and gaining business value from LLMs. ChatGPT’s attention-grabbing ability to write college-level essays and Sora’s ability to generate life-like videos differ from using LLMs within a business application. LLMs must integrate easily into the tools that build business applications to deliver widespread business value from LLMs. The cost of running LLM inference in business applications must be controlled to allow maximum value.

Many organizations are trying to integrate LLMs into their applications and finding that months of work are required to gain any value from an LLM, let alone transform their applications. The primary issue is that the organization’s data and business processes must be integrated with the foundation LLM. The currently available technologies are a collection of incredibly powerful science projects that require significant tuning to each use case within an organization. The mix of projects is natural in the early stages of a new application class, and more will spring up over a few years. The usual maturity curve will apply. Over time, a few projects will rise to the top as the most useful, and these will become easier to implement. In the same way, machine learning (ML)-based AI for video has become a core component of some applications, and we will see LLMs become a feature rather than a product.

The cost to run LLM-based inference is a barrier to some use cases; LLM inference is resource-intensive and often requires GPUs installed in application servers. The high cost means that LLMs are only used where there is a high return. For broader use, the cost must come down. We are already seeing the use of quantization to reduce resource requirements, and as LLM sizes increase, we will need more techniques to reduce resource use. One development is that Intel has added a matrix math accelerator to the latest Xeon Scalable CPUs, reducing the need for GPUs to deliver business value from LLM inference performance.

I hope we see another seismic shift in AI, and LLMs become more applicable because they are easier and cheaper to integrate into business applications. I doubt we will see the future of intelligent assistance robots, flying cars, and unlimited leisure. But it would be nice if LLMs could make everyday applications more straightforward to use and more insight driven.

Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.

Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.

Other Insights from The Futurum Group:

HPE Infuses GenAI LLMs to Uplift HPE Aruba Networking Central AIOps

AI Field Day: Nature Fresh Farms Profits by Machine Learning, Not LLMs

Why the Launch of LLM Gemini Will Underpin Google Revenue

Published by Futurum.

About the author

Alastair Cooke

Research Director, Hybrid Cloud & Infrastructure

Alastair has made a twenty-year career out of helping people understand complex IT infrastructure and how to build solutions that fulfill business needs. Much of his career has included teaching official training courses for vendors, including HPE, VMware, and AWS. Alastair has written hundreds of analyst articles and papers exploring products and topics around on-premises infrastructure and virtualization and getting the most out of public cloud and hybrid infrastructure. Alastair has also been involved in community-driven, practitioner-led education through the vBrownBag podcast and the vBrownBag TechTalks.

See the full picture in the platform

Vendor scoring, the benchmark data behind these findings, and the analyst team that produced them.

Open the platform (opens in a new tab)Become a client
The Weekly

Subscribe to The Weekly

The week's market signals, one email, no filler. Unsubscribe any time.