The neocloud sector is shifting its focus from mere GPU availability to performance metrics like latency, burst capacity, and openness. CoreWeave Inc. is expanding its infrastructure offerings beyond compute to include networking, storage, and software services to support growing inference demand.
LlamaIndex Inc. has transitioned from an open-source framework provider to a specialized AI lab focused on model building for document parsing. The company processes millions of document pages daily for clients in finance, legal, and insurance sectors, with workloads consisting of 75% inference and 25% training.
CoreWeave introduced CoreWeave Forge, a development layer designed to integrate training, inference, and agent development. According to Lukas Biewald, senior vice president of AI initiatives at CoreWeave, networking and power distribution configurations for chips can impact AI latency by orders of magnitude.
Unlike major hyperscalers that rely on proprietary APIs, CoreWeave utilizes standard networking protocols recommended by Nvidia Corp. to ensure broader open-source support and compatibility with other cloud environments.
