TensorFlow Blog

Prophet has gotten a lot of attention since being released in 2017, I think because the idea of a fully automatic solution is very appealing to people. One of the original developers, Sean Taylor, recently posted a nice retrospective on the project's successes and failures:
https://medium.com/@seanjtaylor/a-personal-retrospective-on-prophet-f223c2378985 He quotes one of his earlier tweets:

If I could build it again, I’d start with automating the evaluation of forecasts. It’s silly to build models if you’re not willing to commit to an evaluation procedure. I’d also probably remove most of the automation of the modeling. People should explicitly make these choices.

Having worked on similar Bayesian time-series forecasting tools at Google, this matches my experience (though I've never used Prophet seriously, so please don't take this as any direct judgement of it as a software package). There is a lot of value in a framework that lets you easily experiment with different model structures (our version of this was the structural time series tools in TensorFlow Probability, see, e.g., https://blog.tensorflow.org/2019/03/structural-time-series-modeling-in.html). But if you're forecasting something you actually care about, it's usually worth the time to try to understand yourself what structure makes sense for your problem, and do a careful evaluation on held-out data with respect to whatever metric you're really trying to optimize. A fully automated search over model structures…

Great job.

For someone who does not know Fortran, would you agree that the conclusion that can be drawn here is that PyTorch is good enough?

> As you can see, fastGPT is slightly faster than PyTorch when doing as fair comparison as we can (both using OpenBLAS as a backend and both using caching, the default in PyTorch). You can also see that fastGPT loads the model very quickly and runs immediately, while both PyTorch and picoGPT take a long time to both load the model and to import all the Python libraries.

One one core from your benchmarks I see that the Fortran implementation is about 4% faster than the PyTorch one. On 4 cores about 13% faster.

Excluding the IO, the main advantage seems to come not from switching the language, but from using Apple's Accelerate framework. It appears to me that this framework is now available for Tensorflow [1] and PyTorch [2] too.

Do you expect that once you port the code to GPU you will see a significant improvement over the GPU version of PyTorch ?

[1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html

[2] https://towardsdatascience.com/installing-pytorch-on-apple-m1-chip-with-gpu-acceleration-3351dc44d67c

Tensorflow is accelerated just fine on AMD thanks to ROCm - everything you're praising Apple for doing, AMD has done for their hardware. You're disparaging AMD for not doing something they have actually done. There was even a tensorflow blog article announcing upstreaming AMD support, in 2018[0]

> At this point, it is basically a sealed deal - if you're even remotely dabbling in data science, you better be working on a Mac.

This is hilarious. You might have missed that the benchmark was missing Nvidia (or even AMD) graphics cards; I can't think of a lower bar for comparing ML performance than against Intel GPUs - perhaps Intel CPUs? While Apple has brilliant engineering, the M1 cannot possibly outperform the obscene number of transistors Nvidia & AMD throw at the task, even in older, mid-range cards. Not to mention power dissipation.

If you're dabbling, you're better off with Google's Colab[1] which has (free) hardware acceleration which is roughly on par with my 3-year-old RX580 for my Tensorflow projects. Colab will work on anything that can run a browser.

0. https://blog.tensorflow.org/2018/08/amd-rocm-gpu-support-for-tensorflow.html

1. https://colab.research.google.com/

You're right about human, that would explain it, but still I find it surprising that such "common item" as a human is not there.

Did you also try on items from the list ?

If there is a match (and this is not frequent), to me it's still very low confidence (like noise or luck).

It seems to be a repacking of https://blog.tensorflow.org/2020/03/higher-accuracy-on-vision-models-with-efficientnet-lite.html

So an old release from 5 years ago (like very long time in AI-world), and AFAIK it has been superseded by YOLO-NAS and other models. MediaPipe feels really old tool, except for some specific subtasks like face tracking.

And as a side-note, the OKR-system at Google is a very serious thing, there are lot of people internally gaming the system, and that could explain why it is a "new" launch, instead of a rather disappointing rebrand of the 2020-version.

I'd rather recommend building on more modern tools, such as: https://huggingface.co/spaces/HuggingFaceTB/SmolVLM-256M-Instruct-WebGPU (runs on iPhone with < 1GB of memory)

This is just patently false. Most of the folks I know that have an M1 are students that were saving up to upgrade from a much older computer and got the M1 Air. I can assure you that they don't have a decent amount of money.

Tensorflow has a branch that's optimized for metal with impressive performance. [1] It's fast enough to do transfer learning quickly on a large resnet, which is a common use-case for photo/video editing apps that have ML-powered workflows. It's best for everyone to do this locally: maintains privacy for the user and eliminates cloud costs for the developer.

Also, not everyone has an imagenet sized dataset. A lot of applied ML uses small networks where prototyping is doable on a local machine.

[1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html

Tensorflow (and by extension Keras) offload most of the actual work to C++ or C, so having those compile to WebAssembly would (I imagine) be a herculean effort.

Instead, The TF team maintains TFJS, which can run on WebAssembly[0].

There are also tractjs[1], and onnyxjs[2], both of which allow you to run (most) ONNX models (which is an open standard for specifying ML models) using WebAssembly and WebGL(only onnyxjs supports WebGL). A bunch of frameworks (caffe, pytorch, TF) support exporting to/importing from ONNX.

[0] https://blog.tensorflow.org/2020/03/introducing-webassembly-backend-for-tensorflow-js.html

[1] https://github.com/bminixhofer/tractjs

[2] https://github.com/microsoft/onnxjs