In the video, the speaker explains and demonstrates improvements made to the machine understanding of his English words, which are automatically transcribed as he speaks. Microsoft, he says, has taken in oodles of data, and can thus have that translated Mandarin spoken. And the final kicker: he has fed the system an hour’s worth of his voice, and thus the software will speak in Mandarin, using his own tones.
Amazing demonstration from Microsoft Research, I can confirm that the translated Mandarin is very accurate.
The Universal Translator is coming.