Bzarg

https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

Kalman filters basic concept is essentially this.

1. make prediction on the next state change of some measurable n dimentional quantity, and estimate the covariance matrix across those n dimentions, which describe essentially a probability that the i-th dimention is going to increase (or decrease) with j-th dimention, where i and j are between 0 and n (indices of the vector)

2. Gather sensor data (that can be noisy), and reconcile the predicted measurement with the measured to get the best guess. The covariance matrix acts as a kind of weight for each of the elements

3. Update the covariance matrix based on the measurements in previous step.

You can do this for any vector of numbers. For example, instead of tracking individual objects, you can have a grid where each element represents a physical object that the car should not drive into, with a value representing certainty of that object being there. Then when you combine sensor reading, you still can use your vision model but that model would be enhanced by what lidar detects, both in terms of seeing things that camera doesn't pick up and rejecting things that aren't there.

And the concept is generic enough to where you can set up a system to be able to plug in any additional sensor with its own noise, and it all works out in the end. This is used all the You can even extend the concept past Gaussian noise and linearity, there are a number of other…

Congrats on the launch nextmv!

I had a chance to play with nextmv's beta when they first published it. By far the most useful aspect was the ability to bracket the decision results by calculation time. E.g., I can say, give me your best result of this choice given 100ms.

This changes the typical "train / test / deploy" ML process, to something where you can get as accurate a result as possible given some block of time. This gives you a lot more options when the value of having a super-precise decision drops off a lot after say, 80% accuracy.

For those of you familiar with rocketry, the technique is a lot similar to a Kalman runner [1]. Essentially when a rocket needs to gimbal adjust its trajectory, it has a ton of uncertainty about the nature of the environment, but it does an excellent job of making a fast educated guess for the simple purpose of "get me to this orbit and don't crash".

More generally, this gets to the core of the issues discussed in part 1 of the a16z article about AI companies [2]. Specifically that modeling to get to accurate result is a huge and hidden cost of ML, which makes it distinctly different from software startups. Decision science is an attempt to bridge that gap.

[1] https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

[2] https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/

Author here -- I'm glad you like the post! There are more like you describe out there, though they tend to be scattered around the Internet.

https://jvns.ca/ often gets mentioned on HN and has a large volume of posts that follow the format "let's assume both you and I don't know anything - how do we figure this out?". http://jamie-wong.com/, a coworker, has a few good ones on graphics/numerical stuff.

Characterizing unique IDs with emojis instead of an abstract number was inspired by Tim Babb who has a good blog post on Kalman Filters: http://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

Explaining a concept in person really helps, as you get immediate feedback if they look confused and it helps you figure out in what order you need to introduce concepts. In the case of this project, I was working on it alongside other people who were working their side projects. So I had the chance to explain what I was working on very frequently as we asked each other what we were up to, but I'm still learning to write this way intentionally.

I think a great place to start is https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

Unlike the OP article, it does make use of the math formalism for Kalman filters, but it is a relatively gentle introduction that does a very good job visualizing and explaining the intuition of each term. I have gotten positive feedback (no pun intended!) from interns or junior hires using this resource to familiarize themselves with the topic.

If you are making a deeper study and are ready to dive into a textbook that more thoroughly explores theory and application, there is a book by Gibbs[1] that I have used in the past and is well-regarded in some segments of industry that rely on these techniques for state estimation and GNC.

[1] https://onlinelibrary.wiley.com/doi/book/10.1002/9780470890042

The Kalman Filter has been one of the toughest concepts for me to grasp over the years. I have looked at many examples related to target tracking, which logically makes sense to me given physical behavior of flying targets.

But can anyone provide more applications that make learning the Kalman Filter easy?

I recommend the following two articles that helped me get a better working understanding of the Kalman Filter:

https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

https://thekalmanfilter.com/kalman-filter-explained-simply/

The maths behind this kind of fusion of different data sources is also pretty beautiful. Kalman filters (https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/) are similar to the sensor fusion community what RNNs are to machine learning; simply stick all your data sources in with their uncertainties, and put pops a corrected estimate (in this case, position). It makes it much easier to combine data sources with rapid refresh rate but high integral error (IMU, gyroscopes, ...) with GPS/celestial navigation.