If you read our ‘ACL experience‘ blog, you may already know that recently the aiXplain team worked closely with the Association for Computational Linguistics (ACL) 2022 Dublin conference and implemented Translation Pipelines for the 60/60 Diversity and Inclusion Initiative.

As part of the initiative, we sampled 11,000 papers (1K this year and 10K in previous years) and created an AI-powered pipeline that helped translate the paper titles and abstracts into 60 languages.  

Additionally all conference keynote videos (hosted on ACL) were translated into ten officially recognized United Nations languages. We transcribed the videos in their original language (English), then translated and dubbed them into 10 primary languages using pipelines designed on the publicly available aiXplain Beta platform. Through this initiative, we are making ACL’s research papers and talks  accessible to much wider audiences in their native languages.

In this article, we will explain how everything came together and how you can build similar pipelines as an aiXplain member.

What is the ACL Diversity & Inclusion initiative?

This initiative aims to increase diversity and inclusion in the ACL community by trailblazing multilingual scientific communication, ensuring Globalization via localization for all ACL research. The year long Diversity & Inclusion Special Initiative 60-60, (60 languages for the 60th ACL anniversary), aims to spur research on the processing of CL scientific content leveraging our current and emerging NLP technologies. Read more about it here. 

So what exactly did aiXplain do for ACL 2022? In summary:

  • Translation of Titles and Abstracts of NLP papers
  • Translation of the most used terms in NLP papers.
    (In collaboration with Yale and the Alan Turing Institute)
  • Generate subtitles and Dubbing of recorded keynotes
  • Video series demonstrating building process

What are aiXplain pipelines, and how do they work?

https://i.postimg.cc/3r4K58Kh/functions.gif

aiXplain design/pipelines allow you to build and deploy AI applications quickly and easily. You can create a working application in minutes with our easy drag and drop pipeline features.

To build your application, simply select the AI functions you would like to use, then specify the input and output data. You can then populate the AI functions with the models of your choice, from a wide selection of vendors. Once your application is built, you have instant API access; endpoint, and API key that can be utilized in your chosen system.

How we created a papers translation pipeline

https://i.postimg.cc/wT4nfRrH/aclpipeline.png

All of this was achieved by first creating a pipeline to perform automatic speech recognition (ASR), implemented by AppTek who optimized a pre-trained ASR model using ACL’s controlled vocabulary.

Next, Yale and the Alan Turing Institute ran the text through a keyword extraction system to identify and extract keywords from the papers, and they gave us the output for translation. 

For paper translations, we used state-of-the-art Machine Translation services from the best AI suppliers (BAIDU, AWS, Azure, Google) to quickly generate high-quality translations in over 60 languages. BAIDU was used for 50 languages and AWS, ModernMT, Azure for the remaining 10 languages. 

The pipeline is composed of an input node that accepts English input and 60 output nodes, each of which produces a translation in one of the 60 different languages. All the nodes are running at the same time in parallel.

Finally, we checked the quality of the translations using our human-in-the-loop (HiTL) integration before outputting them into JSON format (output can be made in any standard format, such as csv). Post-editing was done by Meta and YaiGlobal before deployment to the ACL website.

Overall: 11k papers were translated, 1K from this year, 10K from previous years (abstracts and titles). We translated them into 60 Languages (10 main languages, 10 secondary languages, and 40 other languages).

Post-Editing

The process by which humans make changes to translation generated by machines in order to produce a readable, accurate result. The following below are the languages that were used for the translations. Post-edited and non-post-edited:

Post-edited languages:

· Primary: Arabic, Chinese, French, Hindi, Irish, Japanese, Portuguese, Russian, Spanish, Ukrainian

· Secondary: Bulgarian, Croatian, Danish, Dutch, German, Indonesian, Korean, Persian, Swahili, Turkish

Non-post edited languages/remaining:

· Remaining: Afrikaans, Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Catalan, Czech, Dari, Estonian, Finnish, Georgian, Greek, Hausa, Hebrew, Hungarian, Icelandic, Italian, Kazakh, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Mongolian,  Norwegian, Polish, Romanian, Serbian, Sinhala, Slovenian, Somali, Swedish, Tagalog, Tamil, Thai, Urdu, Uzbek, Vietnamese

Translation of Most Used Terms

Not surprisingly, most of the more common keywords and phrases were machine learning and AI-related. For context, each of the top 1,000 terms in ACL publications was translated into 10 main and secondary languages (20 total), after which they were post-edited by Meta and YaiGlobal. Associated with each of the translated terms are 5 sentences that highlight the usage of the term.  

But why were these terms translated? Certain scientific terms can’t necessarily be translated by a generic translation engine, as the meaning of words in scientific jargon may differ substantially from their meaning in regular speech. This is where post-editing comes in. 

How we created video translation

As mentioned previously, all keynote videos were transcribed via our platform using a custom-made dubbing pipeline. The final product resulted in subtitles and dubbed videos. But how was it made?

Luckily we created a series of videos explaining the process:

  1. Subtitling Pipeline Demo – aiXplain
  2. Subtitling Pipeline Code Demo – aiXplain
  3. Subtitling Pipeline Build – aiXplain

Keynote videos were recorded in English and uploaded to Underline, a streaming media and virtual events platform. After the ACL event, they shared 12 hours of videos with us. We then ran a pipeline in which we used AppTek Automated speech recognition (ASR) to convert all keynote speeches to text – all in SRT format, which conveniently time stamps speech to transcribed text. The results were then passed on to 10 machine translation nodes for translation into the target languages. For a better natural-sounding speech synthesis, we utilized Google’s premium text-to-speech models. And that’s it.

In Summary:

Keynote videos are firstly recorded and uploaded onto the Underline.io platform. We then use a custom SRT extraction pipeline built with aiXplain ‘Design’ to convert voice audio into text. After that, the text is translated, and the videos are closed-captioned and dubbed.

Feel free to follow along with the tutorial videos and build your own pipelines!

How can you get started using aiXplain pipelines for your own projects?

First, create your own unique pipeline using Designer. Once everything is finished, you will obtain your very own generated API key, which you can subsequently include in the coding of your app.

You can also utilize our new custom python SDK (software development kit) ‘aixplain-pipelines’

aiXplain Pipelines enables python programmers to add AI functions to their software.

An aiXplain pipeline is a directed graph (DAG) of AI functions built using aiXplain’s designer UI. An AI function is a data processing step that relies on a machine learning model to execute. An example of an AI function is speech recognition or machine translation. Pipelines help you process your data by calling a series of functions as defined in the DAG, abstracting the orchestration, and providing a simple python function call.

To get started use:

pip install aixplain-pipelines

For more information, please visit our GitHub repository:

https://github.com/aixplain/pipelines

– Designed by –

Senior AI/ML Engineer – Krishna Durai

Applied Scientist Thiago Castro Ferreira

Benefits of using aiXplain pipelines for rapid prototyping and deployment?

Some benefits of using aiXplain pipelines include:

  • Quick and easy setup – you can create a working prototype in minutes.
  • No need to worry about complex orchestration – our platform takes care of that for you.
  • The heavy lifting of model computation is completely remote with aiXplain pipelines, as the models are run remotely for you, meaning less work for you.
  • Access to a wide variety of high-quality AI models – you can choose the best model for your needs from a wide-range of suppliers.
  • Easy integration with your chosen system – simply use our provided API key to implement it into your system.
  • Benchmark AI models against each other – our platform makes it easy to compare the performance of different models so you can choose the best one for your needs.
  • Automated recommendations –  Use “AutoMode,” a new feature that assists you in selecting the best AI models for your needs without having to manually benchmark or compare models.

If you’re looking for a quick and easy way to add AI capabilities to your software, aiXplain pipelines are the perfect solution. 

https://aiXplain.com

Contact us today to learn more about how we can help you quickly and easily add AI to your software.

Pin It on Pinterest