The days of Nvidia’s unparalleled market dominance aren’t over, but challengers and choices are arising from all directions.
ZML, a hot French AI startup endorsed by Turing Award winner Yann LeCun, has released inference-performance software that allows a variety of open-source large language models to run on a variety of chips — including Nvidia’s, AMD’s, Google’s TPU, Apple Metal and Intel Arc.
With ZML/LLMD, the newly launched LLM inference server, the company’s ambition is to break existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, ZML founder Steeve Morin told TechCrunch.
As AI becomes integrated into our work and everyday lives, optimizing inference — aka, the processing of prompts — has been outpacing model training in importance, but often feels patchy behind the scenes, with software and architecture barriers that lead to vendor lock-in, Morin said.
The promise of achieving peak performance across a variety of chips is a technological feat, but it could also be a market disruptor, amid mounting fears over AI-related costs.
ZML hopes to provide enterprises and clouds with the option to use a mix of chips, some of which might be less costly or consume less energy. “The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated,” Morin said.
Such a software assist may help novel AI chipmakers, many of which happen to be from Europe, Morin observed, citing Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. But more than their region of origin, what matters to him is that ZML can work with them on “things that haven’t been done before anywhere in the world.”
That doesn’t mean Morin is bearish on Nvidia. He’s not, in part because of its existing supply. He told TechCrunch that ZML has a good relationship with the AI chip giant, which has been gearing up for the rise of inference.
Inference has been an area of such intense investment, that the trend has been hailed the “inference gold rush.” So ZML has competition such as Baseten, recently valued at $13 billion; Inferact, from the creators of open source project vLLM; as well as RadixArk, the commercial company behind SGLang.
Both vLLM and SGLang partially compete with LLMD, but Morin’s ambitions for ZML cover a broader spectrum. “We have reached the point where we are co-designing silicon,” he said. He further credited ZML’s lean team of 20 people as the reason why the Paris-based startup has been able to move fast, with more releases in the plans.
It also helped that this small team is well funded for its size. Thanks to his track record as VP of engineering of Zenly, which Snapchat acquired for nine figures in 2017, Morin raised $20 million from venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
Unlike ZML’s first public project, the inference-focused ML framework released in 2024 and updated in March, ZML/LLMD is not open source. But it is launching as a free product with the goal of learning about usage. “I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go,” Morin said.
It is too early to tell when ZML/LLMD might become a paid product, and what its adoption will look like. But the startup’s cap table confirms that other founders are paying attention, including Dagger and Docker founder Solomon Hykes, Clément Delangue and Julien Chaumond from Hugging Face, as well LeCun, now with AMI Labs. This also builds the case that Europe’s AI startups can now build from home. “I couldn’t do ZML anywhere but in Paris,” Morin said.
Key Takeaways
- Hardware Agnosticism: ZML’s LLMD inference server enables open-source LLMs to run efficiently across a wide range of chips (Nvidia, AMD, Intel, Google TPU, Apple Metal), challenging vendor lock-in.
- Cost & Efficiency Driver: By optimizing AI inference and allowing flexibility in hardware choice, ZML aims to reduce operational costs, lower energy consumption, and democratize access to powerful AI.
- Strategic European AI Play: Backed by significant funding and prominent tech figures like Yann LeCun, ZML’s Paris-based operation highlights Europe’s growing strength in AI innovation, with ambitions to co-design future silicon.
ZML’s LLMD: Cracking the Code of AI Inference to Redefine Hardware Dominance
The artificial intelligence landscape is in constant flux, but one constant has remained: Nvidia’s formidable grip on the AI hardware market. Yet, as AI matures from ambitious research to pervasive application, the very foundation of this dominance is being scrutinized. A new wave of innovation, focusing on the often-overlooked yet critical phase of “inference,” is starting to chip away at established paradigms. Enter ZML, a burgeoning French AI startup, whose newly launched LLM inference server, LLMD, promises to be a significant disruptor, making AI accessible and efficient across an unprecedented array of hardware.
The core of ZML’s ambition, as articulated by founder Steeve Morin, is to dismantle existing silos in the AI hardware ecosystem. With LLMD, the goal is clear: to enable open-source large language models (LLMs) to perform at their peak — and sometimes even surpass it — on a diverse spectrum of chips. This isn’t merely about supporting the usual suspects like Nvidia and AMD, but also embracing Google’s TPUs, Apple Metal, and Intel Arc, offering a truly hardware-agnostic solution that could fundamentally alter how enterprises deploy AI.
The Inference Imperative: Why Optimization is Key
In the grand lifecycle of AI, model training often captures the headlines, demanding colossal computational resources and driving the GPU market. However, as AI models transition from development to deployment, the real-world demand shifts to “inference” – the process of taking a trained model and using it to make predictions or process user prompts. This phase, increasingly central to AI’s integration into our daily lives and workflows, is rapidly outpacing training in terms of importance and sheer volume.
Despite its critical role, the inference landscape has been historically fragmented and inefficient. Morin highlights the prevalent issues: software and architecture barriers that lead to frustrating vendor lock-in, suboptimal performance across different hardware configurations, and a general lack of flexibility. These bottlenecks not only hinder innovation but also inflate the operational costs associated with running AI, creating a significant barrier to wider dissemination and adoption.
ZML’s Vision: Unlocking Hardware Agnosticism
ZML’s LLMD directly addresses these challenges by offering a software layer that abstracts away the underlying hardware complexities. The technological feat here is substantial: achieving consistent, high-speed performance for LLMs regardless of the silicon they’re running on. This “break existing silos” philosophy empowers enterprises and cloud providers to move beyond a single-vendor strategy, opening up a world of possibilities.
Imagine a scenario where a company can dynamically choose between a more cost-effective AMD chip, a low-power Apple M-series processor, or a high-performance Nvidia GPU, all while maintaining optimal LLM performance. This flexibility translates into tangible benefits: reduced hardware expenditures, lower energy consumption, and the ability to leverage existing infrastructure more effectively. Morin emphasizes, “The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated.”
Beyond Performance: The Economic & Strategic Advantages
The implications of ZML’s approach extend far beyond mere technical benchmarks. In an era where “AI-related costs” are a growing concern for businesses of all sizes, LLMD offers a compelling proposition for economic relief. By diversifying hardware options, companies can optimize their CAPEX and OPEX, making powerful AI models more financially viable for a broader range of applications and industries.
Furthermore, ZML’s solution provides a crucial lifeline for a burgeoning ecosystem of novel AI chipmakers. Morin specifically points to a vibrant community of European startups – including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA – that often struggle to gain traction against established giants. By providing a universal software layer, ZML enables these innovators to demonstrate the full potential of their specialized silicon, fostering greater competition and accelerating the pace of hardware innovation globally. Morin’s aspiration is not just compatibility but co-creation: working with these new players on “things that haven’t been done before anywhere in the world.”
Navigating a Competitive Landscape: Co-existence and Differentiation
While ZML aims to disrupt, it’s not operating in a vacuum. The “inference gold rush” has attracted significant investment and talent, spawning competitors like Baseten (valued at $13 billion), Inferact (from the creators of vLLM), and RadixArk (behind SGLang). These players also focus on inference optimization, and projects like vLLM and SGLang offer partially overlapping functionalities with LLMD.
However, ZML differentiates itself through a broader vision. Morin states, “We have reached the point where we are co-designing silicon,” indicating a deeper integration and ambition that goes beyond software optimization alone. He also clarifies that ZML isn’t positioned as an adversary to Nvidia. In fact, ZML maintains a good relationship with the AI chip giant, acknowledging Nvidia’s existing supply dominance and its own significant investments in inference capabilities. ZML’s approach is more about offering choice and maximizing existing and future hardware investments, rather than outright replacing one vendor.
The ZML Story: Speed, Funding, and European Roots
Behind ZML’s rapid ascent is a lean, agile team of just 20 people, based in Paris. This small size, Morin suggests, has been key to their ability to move quickly and deliver innovative releases. The startup is also exceptionally well-funded for its size, having secured $20 million from an impressive roster of venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
This substantial backing is largely a testament to Steeve Morin’s proven track record, notably as VP of engineering for Zenly, a company acquired by Snapchat for a nine-figure sum in 2017. The investor list further solidifies ZML’s credibility, featuring influential figures like Dagger and Docker founder Solomon Hykes, Hugging Face founders Clément Delangue and Julien Chaumond, and the esteemed Turing Award winner Yann LeCun, now with AMI Labs. This concentrated talent and capital in Paris underscore a powerful narrative: Europe is not just a consumer of AI innovation, but a vibrant hub for building it from the ground up. Morin proudly declares, “I couldn’t do ZML anywhere but in Paris.”
A Strategic Rollout: Free to Learn, Built to Grow
Unlike some of ZML’s earlier public projects, such as its inference-focused ML framework released in March 2024, LLMD is not open source. However, it is launching as a free product. This strategic decision is less about immediate revenue and more about understanding user behavior and adoption patterns. Morin’s philosophy is pragmatic: “I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go.” This approach allows ZML to gather invaluable data on how enterprises utilize LLMD, informing future monetization strategies and ensuring that any eventual pricing model aligns with market value and user needs.
Bottom Line
ZML’s LLMD represents a significant leap towards a more open, efficient, and cost-effective future for artificial intelligence deployment. By empowering developers and enterprises with true hardware agnosticism for LLM inference, ZML is not just offering a new product; it’s challenging the very architecture of AI infrastructure. This French startup, backed by top-tier talent and capital, is poised to democratize access to advanced AI capabilities, reduce the industry’s reliance on single-vendor ecosystems, and foster a more diverse and innovative landscape for silicon design. The “inference gold rush” just got a powerful new contender, and its impact could reshape the AI world for years to come.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
{content}
Source:{feed_title}