Livebook News is the official news blog for Livebook, the Elixir code notebook, covering releases and Elixir data and machine learning tools.
Commenters suggest Livebook for getting started with Elixir, and note its collaborative editing and the option to run models directly in the Elixir stack.
> I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever...
Many companies are using Distributed Erlang but not the way you described: they are using it to exchange messages between nodes running the same version of the software. Imagine that you are building a web application, you can directly exchange live messages between nodes without Redis/RabbitMQ [1]. If you are deploying a machine learning model, you can route requests through multiple nodes and GPUs without additional deps [2]. If you want to see which users are connected to your app, you can exchange this information directly between nodes [3].
In other words, there is a subset of distributed problems that Distributed Erlang solves very well out of the box: homogeneous systems working on ephemeral data. And some of the scenarios above are very common.
If you need to communicate between different systems or you need to persist data, then I 100% agree with you, I would not use Distributed Erlang (at best it would be one part of the solution). I think this distinction gets lost in many conversations and sometimes it leads to false dichotomies: "why Erlang when I have k8s/grpc/redis?" while in practice there is not a lot of overlap. I have written about Erlang/Elixir vs Redis [4] and vs k8s [5] for those interested.
[1]: https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html
[2]:…
While Livebook will mostly have an appeal for people already working with erlang/elixir, it does have a few features that are pretty nice.
- it’s collaborative (think Google docs for code) when several people are working on the same instance of a livebook;
- it’s easy to extend with so-called smart cells (which are essentially pieces of gui you can inject in your document https://news.livebook.dev/v0.6-automate-and-learn-with-smart-cells-mxJJe ). Smart cells are available for various tasks (db connection/ interaction, data frame exploration, ML tasks, maps), and building your own is relatively easy;
- you can turn a notebook into a web app ( https://news.livebook.dev/deploy-notebooks-as-apps-quality-of-life-upgrades---launch-week-1---day-1-2OTEWI )
- you can run your code an a remote elixir node by attaching to it (although this requires some knowledge of distributed elixir/erlang )
Not a joke and not everybody is jumping on "AI via API calls", luckily.
As more models are released, it becomes possible to integrate directly in some stacks (such as Elixir) without "direct" third-party reliance (except you still depend on a model, of course).
For instance, see:
- https://www.youtube.com/watch?v=HK38-HIK6NA (in "LiveBook", but the same code would go inside an app, in a way that is quite easy to adapt)
- https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY for the companion blog post
I have already seen more than a few people running SaaS app on twitter complaining about AI-downtime :-)
Of course, it will also come with a (maintenance) cost (but like external dependencies), as I described here:
https://twitter.com/thibaut_barrere/status/1722172915733430770
Also, this[1] is a game changer if you're planning on distributing your extensions but are worried that compiling Rust (especially cross compiling it) could hinder the appeal of your library.
Used it for a few projects and had one of those “How could I have lived before this?” moments.
[1] https://dashbit.co/blog/rustler-precompiled
p.s. Explorer[2], the NIF version of Polars[3] is distributed using Rustler pre-compiled, making Python less and less necessary for the Elixir developer or enthusiast
[2] https://news.livebook.dev/data-wrangling-in-elixir-with-explorer-the-power-of-rust-the-elegance-of-r---launch-week-1---day-5-1xqwCI
[3] https://www.pola.rs/
Maybe try livebook to get feet wet: https://news.livebook.dev/data-wrangling-in-elixir-with-explorer-the-power-of-rust-the-elegance-of-r---launch-week-1---day-5-1xqwCI
That exists in LiveBook now[1] as part of some visualization improvements.
[1] https://news.livebook.dev/whats-new-in-livebook-0.7-2wOPsY