Li Shiyan, director of Baidu’s Intelligent Cloud AI human-computer interaction lab, talks about the digital human sector in January. China Daily
The 2022 Winter Olympics have just come to an end, but the feast of ice and snow is still going on – Beijing Winter Paralympics kicked off on March 4. A sign language AI TV anchor that has served the Winter Olympics is now doing the broadcasting job for the Winter Paralympics, ensuring Chinese audiences who have hearing difficulties can enjoy the Games.
Created by Baidu Smart Cloud, the anchor is supported by the world’s largest sign language database with up to 200,000 pieces of data. Her mission is to provide a great service to those hearing-impaired audience, allowing them to quickly obtain event information.
The technology behind the virtual anchor ensures it can do the job. Sign language’s word order is different from spoken Mandarin. For example, people with hearing difficulties place the predicate at the end of the sentence, while we put them in the middle of the sentence, just between the subject and the object in modern Chinese. Therefore, sign language recognition is difficult, according to Yuan Tiantian, vice dean and professor at Tianjin University of Technology.
Yuan and her team conducted extensive research on action recognition. The arms have 18 points that need to be analyzed, a hand has 21 points and a face has more than 100. All of these points are challenging for the AI and algorithm to process.
Compared with human language interpreters, the AI sign language anchor has some advantages. It can help with the continuous translation for long texts, and limit the amount of information lost. Also, statistics indicated that within special scenarios, the correct rate of sign language recognition could top 97 percent while the correct rate of sign language generation could be even higher.
After the Winter Paralympics, the AI sign language TV anchor will have no shortage of applications. The research team expects that it could help those hearing-impaired people in their shopping, transportation and other aspects of daily life.
China’s virtual human industry is ready for rapid growth, with applications in areas such as broadcasting, entertainment, retail, finance and education. It is also possible that everyone eventually will have their very own avatar.