The SciML open-source organization posts updates on its Julia tools for differential equations and scientific machine learning.
Commenters credit it with the best ODE/SDE support of any language, combined with machine learning, autodiff, and GPU, and cite Julia's vibrant ML ecosystem.
The unique property is the ability to just pick any code or library that is unaware of the differentiation library (unlike tf.function as it needs to specifically use tf methods) and get the gradient. In a language like Julia this is immediately useful as it has a massive ecosystem of numerical code that make sense to get gradients (like differential equations and the SciML project [1], or less conventional stuff like raytracers), but in a language like Swift (as there is no meaning to gradient of GUI libraries or frontend stuff) it is more of a "if you build they'll come" faith from Google.
But regardless unique features, it's a have cake and eat it too type of interface. You don't need to learn a second language within the language like tensorflow's tf.* making it even more natural and flexiblethan pytorch, including all debug mechanisms of the host language itself, but you still get compile time graph creation like tensorflow, including all kinds of optimizations. It makes other approaches seem primitive by comparison, but creating it is much more complex, and the main audience is already more than used to using language within language solutions (like numpy) which can provide something almost as good even if less elegantly, so it's not easy to convince people as well (when it involves changing programming languages).
[1] https://sciml.ai/
MIT Julia Lab | julia.mit.edu | Boston (remote possible) | Devops and MLops
The Julia Lab is the lab in MIT CSAIL that created Julia. Its work is on high-performance computation and scientific machine learning (https://sciml.ai/). We are looking for individuals to help the lab's development activities.
We’re looking for a new devops and MLops engineer to work with the Julia Lab. You’d be in charge of keeping all of our systems running and creating new schemes to accelerate Julia development. Julia’s Base CI, the FluxML, SciML and GPU resources are managed by this position. Automated benchmarking tools 10, automating the installation of Julia in R and Python packages, helping improve Julia’s setup for compute clusters, building tools to make it easy to manage ML training in Git repositories, etc. are the kinds of problems that are encountered. If you’re also interested in technical computing and interested in splitting time between ops and work in this domain, that can also be arranged. This is more of an entry level position, great for students who have just graduated (or are graduating soon), and great for individuals looking to have a year devoted to getting a good PhD program application together.
Can be remote, but working in Boston at MIT is preferred. Contact me via the Julia Discourse for more details.
As someone who uses TF/PyTorch/JAX and numba for their day-to-day DL work, I've been watching Julia's post-hype developments in this area intently. Julia has probably the most vibrant ML ecosystem of all the non-Python languages (remind me when someone has implemented a transformers library for Arraymancer [1]), and absolutely smokes Python in certain niches (e.g. neural ODEs and more under the SciML [2] umbrella, I'd argue Python is "lagging hard" there). Numba can work for speeding up simple operations, but is rife with limitations that prevent optimizing more idiomatic Python code and is an absolute pain to debug. Likewise, trying to decipher or even catch errors from deep in the C++ bowels of most modern DL frameworks is an exercise of frustration.
Python may still be the best language for non-cutting edge deep learning, but we ought to consider whether we've overfit our algorithms to the limitations of the current system [3].
[1] https://github.com/mratsim/Arraymancer
[2] https://sciml.ai/
[3] https://news.ycombinator.com/item?id=20301619
AoT compiled binaries and libraries are fundamental building blocks for distributing software and running it quickly without latency.
Julia itself calls AoT binaries, for example when calling Linear Algebra functions [1].
Chris Rackauckas's team had to leverage AoT compilation (i.e. system images) to reduce TTFX on DifferentialEquations.jl [2].
PyTorch and Tensorflow run without TTFX because they use AoT compiled binaries. What's more, most end users don't have to compile anything on their computer--they can simply download the binaries and libraries.
For small scripts, yes you can simply distribute the package and have the user recompile on the fly. But imagine if your operating system had to recompile every time you turned the computer off. Or if an autonomous vehicle had to recompile an unseen patch of code while in motion.
[1] https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#Standard-functions
[2] https://sciml.ai/news/2022/09/21/compile_time/
Have you explored the SciML landscape at all (?):
https://sciml.ai/
There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties.
Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy.
There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl
Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example).
Long story short -- there's quite a lot of work going on.
It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.
This is true, but even engineers see the advantages of Julia. My engineering school has went from almost pure Matlab usage to many key engineering courses switching to Julia due to its simplicity and friendliness.
It's also SOTA for many engineering applications, particularly for acausal modelling and scientific machine learning (see https://sciml.ai), which has led to big companies like Pfizer adopting it [1]. And for engineers writing novel libraries, it clearly has a strong edge. See for example the work by NASA's JPL [2, 3], the FAA [4] or the CliMa project [5].
[1]: https://juliahub.com/case-studies/pfizer/ (see also https://info.juliahub.com/case-studies) [2]: https://exoplanets.nasa.gov/news/1669/seven-rocky-trappist-1-planets-may-be-made-of-similar-stuff/ [3]: https://ntrs.nasa.gov/citations/20170008266 [4]: https://youtu.be/19zm1Fn0S9M [5]: https://clima.caltech.edu/