Liu Liu writes technical posts on machine learning on devices, JavaScript face detection, Bazel builds, and fast data loading.
HN commenters link his posts on running Stable Diffusion on an iPhone and on JavaScript face detection, plus one on cryptocurrency risk.
After reading through all the comments, I'm surprised that no one has looked at this from a Siri-centric perspective [except for the comment about Clippy ;)].
To me this looks like laying the groundwork for centralizing the most relevant and important information about yourself, which so crucially includes the contexts you are in! and are so often painstaking to actually record in detail when journalling manually. For what purpose? I genuinely think this leads to a Siri that's finally useful in a way that leapfrogs everything and everyone else out there. Think GPT-4(+) with an insane amount of detail and context tailored to you, and (hopefully!) executed in a way that re-affirms Apple's stated commitment to privacy and security.
I have only the most cursory understanding of ML technology but have obviously been trying to follow along with everything pretty much since DALLE-2 made such a splash. With some of the impressive performance shown in smaller models, sometimes with different quantization, I think that Apple Neural Engine silicon might be getting more attention soon... okay, "soon" is probably over-optimistic. But – in September '22 I asked online about running Stable Diffusion[0] on M1 chips on iPad, since I was able to run it on an M1 Mac Mini. The 8-ball said "outlook not so good," and yet by November 8th liuliu had it running on _iPhone_[1]. These are truly interesting times.
[0] I know this is not a 1:1 comparison. But Llama ran on my MacBook without any…
I agree, putting one more layer on top of JS does not allow to do substantially different things. But I would argue that this is the only option. Google is pushing its PNaCl [1] clients where you can run C++ code in your browser independent of the platform. Isn't this partially solution to the chicken-and-egg problem? It may get adapted because there is a strong player behind it. It also allows substantially different things to be on the web. For instance you can run some heavy computer vision app analyzing webcam stream directly on client side. Apart from notable JS exceptions [2].
[1] http://www.chromium.org/nativeclient/pnacl
[2] http://liuliu.me/ccv/js/nss/
Back in 2009 I did some work on augmented reality in the browser[1]. Back then there was no WebRTC, so I had to shim Flash to Canvas to get webcam images, and I implemented image tracking using a simple color tracking algorithm.
If this really is doing image tracking in JS (as tomp noted the explanation isn't clear) then I'm very impressed.
The previous state of the art was some great face-detection code I'd seen, but it is much slower than this.
[1] http://nicklothian.com/blog/2009/11/18/augmented-reality-virtual-keyboard-using-html-5-video/
[2] http://liuliu.me/eyes/javascript-face-detection-explained/, http://apprtc.appspot.com/html/face.html
The blog post is a little light on details. Just go to the source and demo on the original creator's site: http://liuliu.me/detect/detect.html
Canvas allows access to individual pixels in an image, and JS is arguably faster than BASIC interpreters of the 80s. Fun things lie ahead in client-side image processing.
EDIT: Incidentally, the code is pretty hairy and has about 3 comments. It's hard to figure out what ML approach he's using from a cursory look.
EDIT: Aah, definitely an image pyramid in there, so some multi-resolution stuff happening.
It seems to on average get about half the faces.
http://liuliu.me/detect/detect.html#http%3A%2F%2Fwww.alexreisner.com%2Fimg%2Fbaseball%2Ffaces.reisner.gif
http://liuliu.me/detect/detect.html#http%3A%2F%2Fkk.org%2Fct2%2FFaceSw2.jpg
It recognizes upside down faces though, which is impressive.
http://liuliu.me/detect/detect.html#http%3A%2F%2Fwww.scifun.ed.ac.uk%2Fpages%2Fabout_us%2Fshows%2FJennifer_Lopez-upside-down-face.jpg
Overall, a great demo.
That's always the game plan though. I wrote about it a few days ago: https://liuliu.me/eyes/the-endgame-for-cryptocurrency/
Once you have big enough following, having enough systematic risk, you, average scammers can enter "too big to fail" category, especially in democratic countries.
If USDT successfully turned half of the U.S. families holding 25% of cryptocurrency-related assets, they can work out any scamming issues they have in hand.