ElevenLabs moves beyond voice with custom models and a work app

ElevenLabs will help businesses fine-tune AI models on data from their own customer conversations, and opened early access to ElevenAI, an app that takes meeting notes. It also launched a partner programme and marketplace, and expanded its deal with Nvidia.

ElevenLabs will help businesses fine-tune AI models on data from their own customer conversations, the voice AI company announced at its New York summit on Wednesday. It also launched a workplace app, a partner programme and a marketplace.

In the coming months, firms can work with its researchers to tune its voice models and some open-source AI models. They can pick goals such as happier customers or fewer errors. The models then keep improving as more calls come in.

“Within a few years, conversational agents will become critical infrastructure for every enterprise, as essential as cloud or payments,” said CEO Mati Staniszewski.

ElevenLabs also previewed new features for ElevenAgents, its platform for voice and chat agents. Memory will let agents remember returning customers, such as one who prefers Spanish and can only talk after 5pm. Routines will handle follow-up work, such as calling back a mortgage applicant or offering a cancelled appointment to someone on a waiting list.

For staff, ElevenAI listens to meetings and turns them into notes and to-do lists. ElevenLabs says it asks for everyone’s consent before taking notes. Free early access opened on Wednesday.

It says it has more than 550 partners in 44 countries, including Google Cloud and Deloitte. It also signed a deal with McKinsey. From next year, firms can buy apps built by partners in its new marketplace.

ElevenLabs also expanded its deal with Nvidia, adding 15 MW of new computing power for large customers. The two are building a voice agent that calls patients with landlines for Poland’s national health booking system. Nvidia CEO Jensen Huang, who had been due to speak, pulled out of the event this week, as Nvidia’s board runs an inquiry into his conduct.

The push comes as ElevenLabs, valued at $22B in September with about $600M in revenue, faces cheaper rivals such as OpenAI’s voice models. Staniszewski said last week it could start preparing for an IPO in 2028.