DeepL has launched Deep Voice! | Real-time voice and video translations to text

We know that translation tools are based on artificial intelligence. Throughout their development, they have greatly improved, adding new features and further refining their accuracy when providing the best possible translation.

An aspect that has been little explored could be real-time voice translation. This is probably an approach that took some time to generate interest. Even at this moment, we could say that there is still much to be done before we have the necessary technology to find a product of extremely high quality. However, it is time to start working on it, at least in a much more accessible way with the current technology.

DeepL has launched a new feature called DeepL Voice. This function allows the application to listen to a person speaking and provide a real-time translation. One thing to mention is that the translation is not played back through audio, but instead is presented as text for better processing, therefore avoiding delays.

The application can also work in mirror mode, which is really great because it allows two people to have a dialogue, with each person having a part of the mobile screen to read the transcription being done in parallel. In my opinion, this is a great feature that could be very useful in meetings, or perhaps a clear example could be in a restaurant setting, where an employee needs to communicate with a customer.

Regarding the languages that DeepL Voice can support, they are diverse and more may be added in the future. At the moment, the application can listen to languages such as English, Spanish, Italian, French, Portuguese, Swedish, Dutch, Turkish, Polish, German, Russian, Japanese, and Korean. As for the subtitles that can be translated, it covers up to 33 languages, which are currently supported by DeepL Translator.

According to Jarek Kutylowski, founder and CEO of DeepL, during an interview, he mentioned that this is the first product regarding voice, but it will not be the last, as there are likely to be more products like this in the future.

In terms of competitors, the clearest example we can mention is Google, which has adopted this real-time transcription technology in its Meet conference service. There are also emerging companies using artificial intelligence to offer voice translation services, like Eleven Labs. Even the latter promotes its translation technology by relying on DeepL.

The evolution of this product has possibly taken longer to arrive, and we could even say it is maturing slowly. Kutylowski states that meetings are being held with partners and clients to integrate the service into Zoom or Google Meet, although it is unknown when this objective will be achieved. However, Teams already covers this feature.

Considering that most translation products focus on artificial intelligence powered by large language models (LLM), DeepL has taken a stance that reflects the commitment to creating the best possible service from scratch. Just in July, the company decided to launch a new LLM that is optimized for translations. It is expected that under both the context and purpose, it will surpass expectations compared to the GPT-4 model, as well as those from Google and Microsoft.

In my experience, I use this translation tool every day, and I can say that it is highly accurate compared to conventional translators. However, I still believe there is much room for improvement, although with a basic understanding of any language, it is a mistake that can be corrected by reformulating what you want to translate.

Something that may concern many is how these voice data are processed. Kutylowski ensures that while the data (voices) travel to the servers for processing, they are neither stored nor used for training the LLM. However, as more clients make agreements with DeepL, there must be agreements between both parties to ensure that data protection regulations are not compromised.