Ferrite Labs raises $50 million Series B for AI inference efficiency as investors prioritize cost control
Sofia Reyes ·
Ferrite Labs, a San Francisco startup that optimizes large model inference, said Tuesday it raised a $50 million Series B led by Index Ventures. The company will use the capital to expand support for newer accelerators and to scale go to market, citing buyer demand to cut inference costs as models move into production.
The announcement lands amid a cluster of Tuesday funding releases, a pattern firms use to concentrate media and buyer attention. It also reflects a capital shift toward efficiency infrastructure as data center power limits, GPU availability, and unit economics constrain AI deployment.
Deal terms and investors
Index led, with participation from Redpoint, Amplify Partners, and NVentures. Existing backers Root Ventures and XYZ Capital reupped. Ferrite has raised $78 million to date, the company said, and declined to disclose valuation.
Index partner Hannah Gold said investors are underwriting efficiency platforms that can “show clear payback within a quarter” as enterprise AI budgets harden. “Training got the headlines last year. The spend now sits in inference, so the savings need to as well,” she said.
Product and traction
Ferrite sells software that compiles and serves models with quantization aware kernels, KV cache deduplication, sequence packing, and early exit routing across Nvidia H100 and B200, AMD MI300X, and AWS Inferentia 2. The company claims average cost reductions of 30 to 60 percent per million tokens and throughput gains of up to two times per GPU, depending on model and context length.
The startup was founded in 2024 by CEO Anika Desai, formerly on Nvidia’s compiler team, and CTO Luis Romero, who worked on PyTorch at Meta. Ferrite says it has dozens of paying customers across finance, e commerce, and customer support, with expansion driven by workload consolidation and higher context inference.
Investors view middleware risk as manageable if vendors integrate tightly with enterprise observability and procurement. Ferrite says it prices on a percentage of savings, which aligns incentives but requires consistent benchmarking and proof.
This deal underscores a recalibration in AI investing toward cash flow sensitive picks. With model providers compressing margins and data center operators rationing power, buyers and backers are rewarding software that lifts tokens per watt and dollars per GPU, not another general purpose model.