Instacart's engineering team writes about search, machine learning, Postgres, and infrastructure behind its grocery delivery service.
HN commenters cite its Postgres search move away from Elasticsearch and its zero-downtime cutover article, and Instacart often posts job listings there.
Instacart | Sr Full-stack Engineer, Sr Data Engineer, Data Scientist, Machine Learning Engineer, Sr iOS/Android Engineer | ONSITE in San Francisco/Toronto or REMOTE | https://instacart.com
Instacart (YC S12) is building the best way for people everywhere in the world to shop for groceries. Using your phone or the web, you can order groceries and have them delivered to your within an hour. You can choose from a variety of local stores, as well as being able to mix items from multiple stores into one order.
Every day, we solve incredibly hard problems to create an experience for our customers that is magical. We aim to give our customers back their invaluable time so they can spend it doing the things they love with the people they love.
The hiring process depends on the position, but will generally follow this flow: phone screen + simple challenge / take-home challenge / on-site. We're pretty flexible and can work with you on this.
iOS/Android eng: Recruiter Phone Screen - One hour technical challenge via video - ~4hrs Onsite
We care that you can ship product and enjoy taking ownership over what you're working on. We don't really care where you went to school or what companies you've worked for.
Tech: • Ruby (on Rails) • ES6+ • React / Redux • PostgreSQL • Elasticsearch • Memcached / Redis
iOS tech: Swift, Objective-C. Android tech: Kotlin, Java. We’re adopting Rx across all platforms.
Data Analysis & Data Science: SQL, R, Python
Learn more about us and apply directly…
Instacart | Sr Full-stack Engineer, Sr Data Engineer, Data Scientist, Machine Learning Engineer, Sr iOS/Android Engineer | ONSITE in San Francisco/Toronto or REMOTE | https://instacart.com
Instacart (YC S12) is building the best way for people everywhere in the world to shop for groceries. Using your phone or the web, you can order groceries and have them delivered to your door in minutes. You can choose from a variety of local stores, as well as being able to mix items from multiple stores into one order.
Every day, we solve incredibly hard problems to create an experience for our customers that is magical. We aim to give our customers back their invaluable time so they can spend it doing the things they love with the people they love.
The hiring process depends on the position, but will generally follow this flow: phone screen + simple challenge / take-home challenge / on-site. We're pretty flexible and can work with you on this.
We care that you can ship product and enjoy taking ownership over what you're working on. We don't really care where you went to school or what companies you've worked for.
Tech:
• Ruby (on Rails)
• ES6+
• React / Redux
• PostgreSQL
• Elasticsearch
• Memcached / Redis
Data Analysis & Data Science: SQL, R, Python
Learn more about us and apply directly at:
• https://careers.instacart.com/
• http://tech.instacart.com
• http://stackshare.io/posts/the-tech-behind-instacarts-grocery-delivery-service
Longtime FreshDirect user. Nothing beats having an inventory management system. And the quality of the produce is fantastic.
In 10 years of using FD, only during coronavirus have items been missing from my order. And not many.
Compare to InstaCart, which has to go the opposite direction: https://tech.instacart.com/predicting-real-time-availability-of-200-million-grocery-items-in-us-canada-stores-61f43a16eafe?gi=9ef75bacc52b
Sounds like a nightmare to me. And stores may not want to "plugin" their own systems because at that point, they're just commoditized with already razor thin margins.
The one downside to FreshDirect? It's very expensive. I'm lucky enough to be relatively price insensitive, but recently became a big fan of Trader Joe's. Obviously they're known for being economical, but I'm astounded how much cheaper it is. Even Whole Foods and the like.
Cooking 16 oz of cruciferous veggies and an ~8oz piece of meat is my standard lunch. At FD it's 10+ dollars (depending on the level of prep you want on the veggies). At TJ and grocery stores, even in NY, you can hit sub $5, easy.
Of course, the delivery itself is a huge convenience. But is it a 100% premium? I wonder.
Early on they built a Rails app backed by Postgres [0] to store, these days, 500 million items. Then in this post [1] they say they're rearchitecting the database, then mention Snowflake. It sounds to me like they need a document store like Lucene in order to get fast search performance--but they may be optimizing for other use cases like tracking orders which is probably easier in a RDMBS.
It also looks like they might first do all the work to retrieve all the results for a query. The first request is to an endpoint `search_v3/{term}?cache_key={key}` then subsequent requests are to `asyncresultset_{n}?cache_key={key}` so seems like they might have the result set cached from the first query.
The response to the autocomplete endpoint, which is fast, contains an `elastic_id` so perhaps they do partially use a document store.
[0] - https://stackshare.io/posts/the-tech-behind-instacarts-grocery-delivery-service
[1] - https://tech.instacart.com/the-story-behind-an-instacart-order-part-1-building-a-digital-catalog-46df5a8ff705
The value prop for customers vs Elasticsearch are:
- ACID w/ JOINs
- Real-time indexing under UPDATE-heavy workloads. Instacart wrote about this, they had to move away from Elasticsearch during COVID because of this problem: https://tech.instacart.com/how-instacart-built-a-modern-search-infrastructure-on-postgres-c528fa601d54
Beyond these two benefits, then the added benefits are:
- Infrastructure simplification (no need for ETL)
- Lower costs
Speaking the wire protocol is nice, but it's not worth much.
There has been a rise in romantic thought pieces lately about how Data Scientists are wizards and can solve any problem with the real superpower of teamwork. (here's an older example from Instacart: https://tech.instacart.com/data-science-at-instacart-dabbd2d3f279)
In the real world, the state of affairs in Data Science is more practical and pragmatic. And there's nothing wrong with that.