Matthias Niessner
19 HN mentions · mentioned in 15 months, 2016–2021
6:36 · 5M views · Mar 17, 2016
The CVPR 2016 oral demo of Face2Face, a system that captures a person's face and reenacts another video's face in real time using only RGB video. The authors frame it as research on modern vision and graphics.
HN uses it as a reference point in deepfake and fake-news discussions, showing that convincing real-time video manipulation was already possible in 2016.
19 comments from Hacker News
Founder here. AMA :) To answer a few recurring questions in the thread ---> Use case. Video is a way more effective way to communicate than text. Not for the HN crowd, but if you're a blue collar worker a 2 minute video in your native language is much preferred to a 5 page pdf for training. Anyone who has tried to record a simple corporate video know the pain of cameras, film crews, 25 takes to get one that works and post production. Cumbersome, slow and multidisciplinary. By the time the video is done the content is out of date. Synthetic video is not yet at the quality of real video. Eventually it will be. But the mistake many are making here is comparing it to real video; it should be compared with text. In X years we'll be able to make Hollywood films on a laptop without needing anything but time and imagination. Just like we can digitally compose music in Ableton, create images in Photoshop and type novels on keyboards rather than with pen and paper. My (obviously biased;)) belief is that synthetic media will eventually become foundational technology that will move media production from cameras/microphones to API's. We'll be able to do all kind of things we couldn't do before. Eg. personalized and interactive rich media, video-driven chatbots and eventually Hollywood blockbusters made by your favourite YouTuber from his or her bedroom. ---> Uncanny valley Simulating real video is incredibly hard. We're constantly improving and launching more expressive synthesis soon. Fr
OK, throw this into the mix: many of you reading this have smartphones which are voice controlled, and for which voice control is activated at all times. In the case of Google, that processing must take place on Google's centralised servers. Siri may or may not do centralised processing (and can operate in standalone modes). Microsoft's Cortana, Facebook's "M" (IIRC) and Amazon's Aero are all various stylings of "Stasi in a Glade form factor", as Maciej Czeglowski so memorably put it.[1] Voice stores distressingly cheaply in terms of space, and with the Internet of (broken) Things (that spy on you), odds of finding yourself surrounded by microphones in the most unexpected locations,[2] controlled by a wide variety of quite probably competing interests.[3] And if they cannot find what they're looking for in the surveillance tape itself, they'll simply manufacture their own evidence using your own phonemes[4] and video.[5] ________________________________ Notes: 1. [LINK] 2. [LINK] 3. [LINK] 4. [LINK] 5. this video
While the technology is amazing, I am a bit bothered by all these picture and video modifying algorithms. The issue is that we can't know what's real any more. It used to be if you saw a video or a photo depicting an event you could be pretty sure that what you're looking at actually happened. Now, if you see a video of a prominent politician saying something awful in your twitter timeline (or whatever), they may have actually never said anything remotely close. It could be a completely fictional video that looks perfectly realistic[0], made by some teen in Macedonia.[1] I realize photography and video have always been used to trick people into thinking things that aren't true, but this technology enables nuclear-grade deception. I am wondering: is there a use-case for such an algorithm that is practical and good for the world? PS: I know an eye-rolling algo is quite innocuous but I've had this thought on my mind about these in general and needed to air it out. [0] this video [1] [LINK]
> Seems from right out of the gate, they are breaking their own ethical guidelines as a cheap promotional tactic. If they care that little about themselves and a former president of the United States, what do they care about your likeness. We state in our blogpost that we make an exception for Obama/Trump in order to raise public awareness. Both of them are regularly used in Machine Learning benchmarks (for example [0] [1]). Note that we don't allow users to generate from Trump/Obama's voice. Once again, we care a lot about these issues and that's why we only allow users to copy their own voice. [0] [LINK] [1] this video These issues are challenging and suggestions about how you think the technology should be introduced/regulated are very welcome.
Photo, audio, and video evidence should already be dismissed until one is able to verify the integrity and source. All of these can already be believably faked - it's just a matter of educating people that a layperson can easily create fake things by using tools developed by research teams. Fake text is the easiest to fake if you can identify the font used - any image editor will work. HN uses 9pt Verdana, even without using dev tools I could fake your post to say anything I wanted it to say since it would just be 9pt Verdana on a solid background set to text wrap every 1050px. See: this video & [LINK]