Facebook Research

Hi there :)

Apologies for the super long response, but you had a lot of points.

> Am I really missing something here or this thing is a complete nonsense with no actual use cases what's so ever in practice?

Hopefully you're missing something, or we've been wasting a great deal of our time ;)

> There are a number of off-the-shelf models that would give you image/sentence embedding easily. Anyone with sufficient understanding of embedding/word2vec would have no trouble train an embedding that is catered to the specific application, with much better quality.

For images and text, it's definitely true that you can train your own embeddings with an off-the-shelf model. But I think it's more likely that we end up in a place where a small number of people train a bunch of really good models and everyone else uses them.

I think this for three reasons:

1. It's what we've seen with word2vec. The vast majority of people that use word2vec aren't training it themselves, they're downloading pretrained weights.

2. Most people don't have enough data to train a good embedding themselves. There are good public datasets for images and text, but we're planning to produce embeddings for more niche verticals too.

Keep in mind that modern deep neural nets are very data hungry, and the problem gets worse every year. In a few years I think we're going to be in a spot where getting state of the art performance requires a lot of compute, and more data than most people have access to.

3.…

Bitten by queues in every team I was at AWS:

> As a rule of thumb, target utilization below 75%

This is one good reason to rely on serverless. Simply outsource the problem to a system that knows how to handle this better.

> Steer slower workloads to paths with lower utilization

This is fraught with all sorts of perils [0], so take caution going down this valid but tricky route.

> Limit variance as much as possible when utilization is high

This is key, and one of the most elegant solutions to this problem I know of comes from a Facebook talk on CoDel + Adaptive LIFO. [1]

> Implement backpressure in systems where it is not built-in

Making downstream dependencies behave is never an option. Selectively isolating noisy neighbours, if possible, from other well behaved clients, tends to work well for multi-tenant systems [2], in addition to monitoring long work (by dropping work that takes forever) [3], or better yet, doing constant amount of work [4] (aka eliminating modes) [5].

> Use throttling and load shedding to reduce pressure on downstream queues.

One way is to impl admission control (ala Token Bucket) to minimise the impact of thundering herds. Though, clients do not take kindly to being throttled. [6]

A great article; many have been written at this point. Very many still have been burned by queues. Though, in my experience, without an exception, some component somewhere was always building up that backlog! [7][8][9]

[0] Interns with Toasters: How I taught…

Hey, Facebook VP of Integrity here (I work on this stuff).

This WSJ story cites old research and falsely suggests we aren’t invested in fighting polarization. The reality is we didn’t adopt some of the product suggestions cited because we pursued alternatives we believed are more effective. What’s undeniable is we’ve made significant changes to the way FB works to improve the integrity of our products, such as fundamentally changing News Feed ranking to favor content from friends and family over public content (even if this meant people would use our products less). We reduce distribution of posts that use divisive and polarizing tactics like clickbait, engagement bait, and we’ve become more restrictive when it comes to the types of Groups we recommend to people.

We come to these decisions through rigorous debate where we look at all angles of how our decisions will affect people in different parts of the world - from those with millions of followers to regular people who might not otherwise have a place to be heard. There’s a baseline expectation of the amount of rigor and diligence we apply to new products and it should be expected that we’d regularly evaluate to ensure that our products are as effective as they can be.

We get criticism from all sides of any decision and it motivates us to look at research, our own and external, analyze and pressure test our principles about where we do and don't draw lines on speech. We continue to do and fund research on…

Closest I can think of is Purely Functional Data Structure (Chris Okasaki) or Category Theory for Programmers (Bartosz Milewski), but they're not exactly what you're after.

I would submit that this paper from FB is more in the vein of what you're looking for: https://research.fb.com/wp-content/uploads/2020/08/Eliminating-Bugs-with-Dependent-Haskell-Experience-Report.pdf .

This interview might also be insightful: https://www.microsoft.com/en-us/research/podcast/functional-programming-languages-pursuit-laziness-dr-simon-peyton-jones/

The truth is that Haskell is still very academically focused, so you're likely to see papers of substance much more often than books or blog posts.

When you hire people to write Haskell there's two groups that always show up: young enthusiasts who are frustrated with imperative programming in their (usually first or second) day job, and academics who got transplanted into industry to work on hard problems. I've interviewed people doing formal verification of CPU circuits at Intel, people who work on compilers, people who work on verifying termination of programs (for missile guidance), and people who work on financial institution backends.

What I haven't seen is too many experienced, pragmatic engineers (rather than computer scientists) who have spent their career writing Haskell.

Facebook has a similar system, unsurprisingly. An agent runs on every host to sample 1 in N packets, accumulating counts by source/dest host/cluster/service/container/port and so on before sending the aggregate data to Scuba [0] for ad-hoc analysis. This tool was really useful — in a matter of seconds you could see traffic types and volumes broken down by almost any dimension. Did service X see a huge jump in traffic last week? From where? Which container or service? How much bandwidth did your compression changes save? And so on. It also had some really neat stuff to identify whether or not a flow was TLSed or not, which was crucial for working out what still needed to be encrypted in light of the Snowden revelations.

TCP retransmits were also sampled in detail. Being able to see at a glance if e.g. every host in a rack is the source of (or target for) retransmits made troubleshooting much faster.

These systems were really awesome and a good example of what could be built with relatively little effort (the teams involved were small) when you already have great components for reliable log transmission and flexible data analysis.

[0] https://research.fb.com/publications/scuba-diving-into-data-at-facebook/

Hmm and global shutter cameras are much more expensive. Though AFAIK a modern mirrorless camera with mechanical shutter can be used for this, yes? My GH5 can shoot at 12 frames per second with the mechanical shutter, so that may be of some use.

I did see in this paper that it seems like rolling shutter compensation is possible:
https://research.fb.com/wp-content/uploads/2019/09/An-Integrated-6DoF-Video-Camera-and-System-Design.pdf

I wonder if a library could be produced for this?

As far as my street capture I am currently using 830 frames from my 4 minute video (2 fps), images are 4992 x 3744 shot with a 12mm lens. The reconstruction is still texturing so I don't have the final result. What kind of numbers are you interested in?

Please contact me yes! My email is in my profile. See also my four wheel drive robot with four 4k cameras, and feel free to actually initiate discussion on that site where others can see it too (or emails is good). Thanks!

https://reboot.love/t/new-cameras-on-rover/