Nvidia Preps New AI Inference Chip With Groq Inside

Nvidia is gearing up to drop a new AI inference chip at its GTC conference in March. The twist? The system will pack a chip designed by Groq, the startup known for its blazing-fast inference hardware.

OpenAI is already on board as a customer.

The move signals Nvidia feeling the heat from rivals in the inference game. While the chip giant has long dominated AI training, the industry's center of gravity is shifting toward inference — the process of actually running AI models to answer queries in real time.

That's a different beast than training, demanding rapid-fire processing rather than brute-force compute. Competitors have been circling this space aggressively, and Nvidia clearly doesn't plan to cede ground.

Partnering with Groq rather than going fully in-house is a notable strategic choice — and a tacit acknowledgment that speed-focused inference demands fresh architectural thinking.