Given the diverse use cases and working conditions of AI, including data centers, clouds, and the edge, the processor market for AI-enabled systems is vast and highly diverse. No single vendor can satisfy all of these requirements today, and specialized vendors will emerge in the next year or two, further diversifying the available solutions.
Jack Gold, president of Jay Gold Associates, expects more vendors to emerge beyond the current handful of leading solution providers. The changing demand and high growth for AI-enabled systems will provide significant opportunities for specialists who focus on specific types of AI systems and processing areas.
“There’s enough breadth and scope in the market to float all boats, so to speak,” Gold said. “If you look out two to three years, the vast majority of AI workflows are not going to be on the NVIDIA H100, the high-performance machine learning system. The vast majority are going to be inference workloads. They’re going to be at the edge, but they’re going to be on PCs and mobile devices. There’s going to be some in IoT. So AI is not going to be deployed solely on the high-end NVIDIA chips; it’s going to be a diverse set of workloads.”
As we segment the market, especially starting with the cloud and hyperscalers, we expect AWS and other companies to offer performance close to NVIDIA at a lower price point through their own custom chip technology.
“We expect the hyperscaler market to offer a variety of processors to meet increasingly diverse AI training requirements and some advanced inference-based workloads,” Gold said. “In the short term, we expect Nvidia to remain dominant in this segment, but in the longer term (two years+), we expect to see significant dilution of its market share.”
In data centers, we expect to see an increase in the transition of existing data center servers to inference-based workloads, with fine-tuning of existing models and RAG optimization to run AI workloads. Inference is much less process-intensive than training and can be performed on existing CPUs instead of expensive GPUs.
This will be an opportunity for AI-as-a-service provided by major cloud service providers, allowing companies to train AI on hardware they need only once, and then perform updates or inferences on their own equipment, without having to invest large amounts of capital in expensive hardware.
He added, “As newer, more efficient modeling methods are developed, they will increasingly be run on existing servers, not only for cost-performance benefits but also to increase compute availability. This will benefit established companies with well-established data center businesses.”
Gold expects that over the next two to three years, most AI workloads will migrate to edge-based systems. These edge systems will include a variety of systems and processing capabilities, from small-scale internal processing of sensor arrays to heavy equipment, autonomous vehicles, and medical diagnostics.
Development platforms will also play a key role in responding to proprietary solutions such as NVIDIA CUDA, with open-source platforms and development environments. “Open and compatible ecosystems such as Arm and x86 offer significant advantages in providing compatibility from small compute requirements to large environments, as they can be scaled up or down depending on compute requirements, and solutions can be easily ported and reused,” Gold explained.
The IoT field overlaps a lot with edge computing, so it needs an open ecosystem to provide scalable solutions like the edge. However, while IoT devices tend to be smaller and less powerful, there are also a lot of solution providers.
With so much hype surrounding AI, there have been a number of startups in the AI processor market over the past few years, and more will emerge in the coming years. However, these are relatively new companies with little market presence and proven capabilities, making it difficult for them to effectively target specific niches. “Some of these startups will eventually succeed, but most of the rest will either disappear or be acquired in the next two to three years,” Gold said.
editor@itworld.co.kr
Source: www.itworld.co.kr
