Baidu's research blog covers its work in deep learning, especially neural text-to-speech, speech recognition, and medical imaging.
HN commenters react to the Deep Voice posts: some find the text-to-speech impressive, while one calls the post misleading about the technical accomplishments.
This post on Deep Voice seems a little off-the-mark. In fact, I would say it is completely misleading about the technical accomplishments here.
From my perspective, Baidu's approach is a little embarrassing, with the use of many modeling stages in their training and production of TTS. When the rest of the community is moving towards end-to-end training, their usage of this many stages sounds excruciating. Merlin[0], which was a pretty good standard for 2016, has this painful feeling as well, with two DL stages (duration, acoustic) followed by some conditioning and then a synthesis step.
The more important technical contribution seems to be the hand-tuned synthesis code that makes their generation faster; cool but not particularly sexy (and there are few details). The details on training hyperparams are nice to have too, of course.
Contrary to the post, I would be very surprised if the voice sample included in the post was actually generated by Deep Voice -- it has none of the robotic qualities pointed out by the researchers themselves in their blog post[1]. More likely it is a demonstration of the loss in their last, WaveNet-like stage. This was also pointed out in the previous HN discussion[2]
Lastly, Andrew Ng is neither thanked in the paper nor mentioned on any webpage -- are we sure this was work he supervised?
[0] https://github.com/CSTR-Edinburgh/merlin
[1]…
It's unfortunate that there aren't any high quality projects for the other way around - text to speech.
By high quality I mean something as good as Tacotron 2 [1]. However, I have no illusions of Google ever releasing code or trained models.
Someone will likely re-implement it from the paper, but it's unlikely to be anywhere near as good as those samples, due to worse tuning and lack of a good training set[2].
This effect can be seen if you compare Baidu's samples [3] for Deep Voice 3 with those from an implementation by a third party [4].
[1]: https://google.github.io/tacotron/publications/tacotron2/index.html
[2]: Common voice, for example, will not work well, since you want lots of data from a single speaker for good results. See for example, how samples get progressively worse when increasing the number of speakers in training data[3].
[3]: http://research.baidu.com/deep-voice-3-2000-speaker-neural-text-speech/
[4]: https://r9y9.github.io/deepvoice3_pytorch/
The more interesting would be to see how feasible is to crack the in band SAS authentication string, when callers verbally verify it.
Deep learning and ability to train on a specific callers' voice [1] then mimic it might be an interesting attack vector. In practice Silent Circle's implementation does something interesting and instead of SAS numbers use dictionary words. So you end up with something like "Pink Elephant Salad". Could probably MitM that. However callers are then supposed to make some extra puns or discuss it a bit and say something like "Ha-ha! Wonder how tasty the an elephant salad would be". And if after MitM-ing, the string to the other side was "Plastic Blue Llamas" then a MitM attack becomes more obvious.
[1] http://research.baidu.com/deep-voice-production-quality-text-speech-system-constructed-entirely-deep-neural-networks/
Sounds amazing! Just to add a usecase - for many people, creating a decent voiceover is one of the big sticking points for producing youtube videos or educational courses. If I could write a script, and have software generate a decent enough voiceover, it would be amazing.
It's not even necessary to copy anyone's voice, as long as there's a selection of the most comprehensible and human-sounding ones.
Then, you could even automatically generate slideshow presentation from a few illustrations and headlines, and that would make "rendering" articles into videos very fast and easy. I'm sure a lot of people would pay for such service.
----
By the way, recently I've encountered Deep Voice 2, a similar research project by baidu:
http://research.baidu.com/deep-voice-2-multi-speaker-neural-text-speech/
Results are very impressive.
On the contrary, Graph NN's I've found Graph NNs extraordinarily useful and very easy to train!
I'm not sure what kind of problems you are trying to use them for, but generally they are really useful for node classification or recommendation type tasks.
So for example in the knowledge graph context, you can do things like give it a tiger, jaguar and panther and it will find nodes like lions and leopards.
https://arxiv.org/pdf/1901.00596.pdf is a survey paper which has a decent overview of the tasks they are useful for. http://research.baidu.com/Public/uploads/5c1c9a58317b3.pdf is using them for QA, but I'm not familiar enough with the dataset to evaluate it fairly.
"Deep Voice 2 can learn from hundreds of voices and imitate them perfectly. Unlike traditional systems, which need dozens of hours of audio from a single speaker, Deep Voice 2 can learn from hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality." - http://research.baidu.com/deep-voice-2-multi-speaker-neural-text-speech/
Now imagine that with Tacotron quality, and you'll get that "strange" effect with anyone, meeting their vocal clone.
This is still text-to-speech, so it's not live-copying your intonation, but you could easily imagine a seq2seq network designed to do so.