VideoDubber automatically dubs your videos into other languages

English may have become the default lingua franca of many facets of the Web, but the fact remains that most people online speak a language other than English as their native tongue.

For content producers, the myriad of different languages can be a headache, as it’s costly to reproduce words – spoken or written – for a multitude of markets. There are free or affordable tools available to circumvent the language barrier – Google has produced subtitling tools, for example, while third-party alternatives are available too.

But what about voiceovers and dubbing? This is where VideoDubber is setting out its stall.

Global norming

VideoDubber does exactly what its name suggests. The Israeli startup has developed an online SaaS (software as a service) platform that automatically provides voiceovers and dubbing for films, documentaries, TV shows, news broadcasts and any similar broadcasts into more than 30 languages, using synthetic voices. These are based on the voice signature of professional dubbing artists, so it all sounds perfectly natural.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Indeed, VideoDubber claims its technology is able to dub an entire feature film in less than 5 minutes. In the build up to its official launch at IBC next month, VideoDubber has been working with industry partners such as Nuance and the Acapela Group, as well as RRsat, to develop this service. It has also been working closely with Microsoft – it’s a Microsoft BizSpark Plus development partner – and has been licensing some of its speech technology for this initiative, namely Microsoft’s SAPI Text-To-Speech (TTS).

In terms of the target market here, VideoDubber is being aimed at broadcasters and content creators who need to localize masses of content in a short space of time, without breaking the bank.

There is scope for this technology to serve the movie industry in some countries, though perhaps not Hollywood for now.

“This technology is rapidly evolving, and therefore it can serve the movie studios too, depending on the market and the need,” explains Boaz Rossano, CEO of VideoDubber. “In some countries, for example Russia, Latvia, and Poland, the standard of dubbing is voice-over only, and over there automated dubbing could cover almost all needs, including movies.”

There are many facets to what could be construed as ‘dubbing’. Voiceover is actually a separate entity, and usually involves simple narration with minimal, if any, lip-syncing. Also, little attempt is made to place the voiceover artist within the context of the visuals by making their voice suit the surroundings of the on-screen actors (‘echoing’ if in a cave, for example). And all this before we even consider mimicking emotion and intonation, and using different voices for each actor. It’s fair to say that voiceovers are easier to do than full-on dubbing.

Proper dubbing scales all the way up to perfect sync, whereby attempts are made to sync the translated words to the actors’ mouth movements, and position the interpretation so it sounds as though it could be native to the original recording. VideoDubber is planning to integrate voice-intonation technology into its platform in future iterations, while text-to-speech specialists Acapela Group already offer “emotional variants” in its voices.

Moreover, VideoDubber has up to eight different voices per language, with each synthetic voice in its library built using the previous work of a professional narrator as a template.

How it works

VideoDubber doesn’t work with any old video – you will have to possess the subtitle files for it to work, as it uses text-to-speech technology to enable this. So you would upload your .AVI or .MP4 video file, and its corresponding subtitles (in .SRT, .680 or .PAC file formats), to the VideoDubber Web service, fill in a form, and the process kicks off from there. So you could have dubs for your videos in up to 30 languages within a day.

In terms of costs in relation to other localization techniques, Rossano says that VideoDubber will certainly fall below that of professional dubbing production, but will likely be more expensive than subtitling.

“This technology previously used solely in one market has now significantly impacted the broadcast market,” he says. “Text-to-speech can now enable an automated dubbing service that is available 24/7 and can dub masses of video hours to different languages simultaneously, in a small fraction of the time it traditionally took. This is something that will revolutionize the TV market.”

With patents registered, licensing agreements in place with various speech technology providers, and some private equity raised, it’s full steam ahead for VideoDubber as it looks to exit beta next month.

Feature Image Credit – Edward Lui (EdWick)/Flickr

➤ VideoDubber