AWS and DXC Enable Real-Time Multilingual Voice Translation in Amazon Connect

Introduction

Providing multilingual customer support in global businesses comes with significant operational challenges. To address this, AWS and DXC Technology have collaborated to develop a scalable voice-to-voice (V2V) translation solution, revolutionizing how contact centers manage conversations across different languages. This innovative approach leverages Amazon Connect alongside AWS AI services to enable near real-time voice translation, enhancing customer interactions and streamlining support operations. In this blog, we explore the technology behind this solution and how it transforms multilingual communication in contact centers.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Solution overview

With the help of AWS’s advanced voice recognition and machine translation technologies, the Amazon Connect V2V translation prototype enables real-time conversation translation between agents and customers, enabling them to converse naturally in their preferred languages. It includes the following essential elements:

Speech recognition: Amazon Transcribe acts as the speech recognition engine, recording and turning the customer’s spoken words into text. The machine translation engine is then given the transcript (text).
Machine translation: In almost real-time, the customer’s transcript is converted into the agent’s preferred language by Amazon Translate, the machine translation engine. The text-to-speech engine Amazon Polly turns the translated transcript back into speech.
Bidirectional translation: In the agent’s answer, the procedure is reversed, with the client receiving the translated audio after the agent’s voice has been translated into their language.
Smooth integration: By utilizing the Amazon Connect Streams JS and Amazon Connect RTC JS libraries, the V2V translation sample project seamlessly connects with Amazon Connect, allowing agents to manage customer contacts in several languages without needing extra work or training.

DXC

An example of an agent web application is seen in the following screenshot.

DXC2

Three sections make up the user interface:

Contact Control Panel – An Amazon Connect softphone client
Customer Controls — Customer-to-agent interaction controls, such as Customer Voice Synthesis, Customer Voice Translation, and Customer Voice Transcription
Controls for interactions between agents and customers, such as the ability to translate, synthesize, and transcribe agent voice

Challenges when implementing near real-time voice translation

The goal of the Amazon Connect V2V sample project was to reduce the time the audio processing takes between the client or agent finishing their speech and the beginning of the translated audio stream. The user experience differs from that of a genuine discussion when both parties speak the same language, even the quickest audio processing time. This is because of a certain pattern where the customer only hears the speech that the agent has translated, and the agent only hears the speech that the customer has translated. That pattern may be seen in the diagram below.

The following phases make up the sample workflow:

The client begins talking for ten seconds in their native tongue.
The agent first hears ten seconds of quiet as they just hear the customer’s translated voice.
The consumer and the agent experience quiet for one to two seconds after the customer has finished speaking.
The representative receives a stream of the customer’s translated speech. The customer hears quiet during that period.
After the customer’s translated speech has finished playing, the agent talks for ten seconds.
The consumer hears ten seconds of quiet as they can only hear the agent’s translated voice.
The consumer and the agent experience quiet for one to two seconds after the agent has finished speaking.
The agent receives a stream of their translated speech. The agent hears nothing during that period.

Get started with Amazon Connect V2V

By taking the following crucial actions, you may use it as a starting point for creating creative, multilingual communication solutions in your contact center:

Git Repo (Source Link) Clone the GitHub repository
Experiment with various audio streaming add-on settings.
Read the README to review the constraints of the sample project.
Create your implementation plan:

Put in place strong security and compliance measures that satisfy the requirements of your company.
To specify your unique use case needs, work with your customer experience team.
Strike a balance between automation and the agent’s manual controls (for instance, automatically setting contact characteristics for preferred languages and audio streaming add-ons via an Amazon Connect contact flow).
Depending on your language assistance needs and your commercial, legal, and regional preferences, use the transcription, translation, and text-to-speech engines of your choice.
Create a pilot group as the first step in a phased deployment, and then incrementally improve your translation and transcription custom terminologies and vocabularies.

Conclusion

The Amazon Connect V2V translation prototype showcases how advanced AWS AI services can help contact centers overcome language barriers, improve operational efficiency, and reduce support costs. Businesses can enhance customer experiences while optimizing resources by enabling near real-time multilingual communication.

Organizations looking to implement seamless voice translation can leverage this solution as a foundation for building scalable, multilingual customer support systems.

Drop a query if you have any questions regarding Amazon Connect V2V and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

FAQs

1. How does the V2V translation solution improve multilingual support?

ANS: – It enables real-time voice translation using Amazon Transcribe, Amazon Translate, and Amazon Polly, allowing seamless customer-agent communication without multilingual staff.

2. What AWS services power the V2V(voice-to-voice) translation?

ANS: – Amazon Transcribe (speech-to-text), Amazon Translate (text translation), and Amazon Polly (text-to-speech), integrated with Amazon Connect.

WRITTEN BY Aayushi Khandelwal

Aayushi, a dedicated Research Associate pursuing a Bachelor's degree in Computer Science, is passionate about technology and cloud computing. Her fascination with cloud technology led her to a career in AWS Consulting, where she finds satisfaction in helping clients overcome challenges and optimize their cloud infrastructure. Committed to continuous learning, Aayushi stays updated with evolving AWS technologies, aiming to impact the field significantly and contribute to the success of businesses leveraging AWS services.