Uber's engineering team writes about its infrastructure, datastores, spatial indexing, and machine learning systems.
Commenters call the articles surprisingly good and recommend them for production ML and engineering topics, and cite a paper on Uber's merge queue.
Uber | Back-end, Android, iOS | Amsterdam, Netherlands | FULL-TIME ONSITE VISA
Uber's Amsterdam engineering office is looking for back-end, Android and iOS engineers for the following teams:
* Payments: building the future of payments for on-demand services. We own a very large chunk of the payments stack end-to end in this office, which opens exciting opportunities.
* Mobile platform: build tooling that will make our engineers more productive (and happier!)
Learn more about our team at http://t.uber.com/amsterdam
Interested? Shoot gergely@uber.com an email if interested.
--
About me: I'm an engineering manager started as an engineer here at Uber, in Amsterdam. My first project was rewriting the Uber app (https://eng.uber.com/new-rider-app/) and the last interesting piece of code I committed was open sourcing the architecture behind it (https://github.com/uber/RIBs).
It's a theme I like to read about (I mean "practical issues around ML in production settings"). I find company blogs and some research publications are great resources. Examples:
- https://eng.uber.com/
- https://code.fb.com/
and many more. Google also publishes papers on various engineering practices obviously, some ML-related, but I can't find a blog where they focus on that specifically.
Also it's not "to keep up to date", but there's a great paper (from Google) that's often cited:
Machine Learning: The High Interest Credit Card of Technical Debt
https://ai.google/research/pubs/pub43146
It talks about issues you face over the long run (I've experienced some of those). It
also provides interesting pointers for further reading, e.g. about "pipeline jungles".
If others have pointers, I'm curious to hear about them as well.
Hey that's great! I think everyone should do what works for them.
Using a tool "just because" it worked for others, seems to be what often drives many decisions in this industry. For example, would you go for the 2,200+ microservice architecture from Uber[1] just because they say it solved all their problems? They probably have their reasons, and they are unlikely to be the same as everyone else.
I know my stack as a solo founder is unconventional, but it's what has worked best for me. And I'm happy with my choice.
If serverless does the trick for you, then focus on that, and keep building the features that your customers actually care about.
[1] https://eng.uber.com/microservice-architecture
Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. Pyro enables flexible and expressive deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling. It was designed with these key principles:
Universal: Pyro can represent any computable probability distribution.
Scalable: Pyro scales to large data sets with little overhead.
Minimal: Pyro is implemented with a small core of powerful, composable abstractions.
Flexible: Pyro aims for automation when you want it, control when you need it.
Check out the blog post for more background or dive into the tutorials.
https://eng.uber.com/pyro/
Love them or hate them (I am in the latter category), but observe even Uber mentions some of their heavier backend stack for internal usage runs on Dropwizard.
"The highest-volume service in this area is Gurafu, which provides a set of utilities for working with road map data and improves efficiency and accuracy by providing more sophisticated routing options. Gurafu is fronted by µETA, which adds a business logic layer on top of raw ETAs (things like experiment group segmentation). Both Gurafu and µETA are web services built on the DropWizard framework."
https://eng.uber.com/tech-stack-part-one/
https://news.ycombinator.com/item?id=12152346
The lottery ticket hypothesis is IMHO the single most interesting finding for deep learning. It explains why does deep learning works (vs shallow neural nets), why is initial over-parametrization is often useful, why deeper is often better than shallow, etc.
I recommend for an overview:
- the original paper "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks", https://arxiv.org/abs/1803.03635
- "Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask" https://eng.uber.com/deconstructing-lottery-tickets/ showing that if we remove "non-winning tickets" before the training, the trained network still works well