Nextdoor's engineering team writes about its infrastructure, including task workers, continuous deployment, mobile releases, and tooling.
Commenters cite its posts on dropping cron jobs and replacing Celery with a custom taskworker when discussing job scheduling and why companies build their own.
Nextdoor | San Francisco, CA; Toronto, ON, Canada | Full-time, Onsite | https://nextdoor.com/
Nextdoor is the world’s largest social network for the neighborhood, serving 247,000 neighborhoods across eleven countries. We recently closed a $170MM series F where Mary Meeker joined our board, and we officially launched in Canada. To support that, we're opening our first international engineering team in Toronto!
Here's a look at our current stack: https://engblog.nextdoor.com/what-is-the-technology-behind-nextdoor-in-2019-ccc3d2173eda
Some of our open positions in SF:
* Software Engineer: https://about.nextdoor.com/careers-list/?gh_jid=424&gh_src=f342d7fd1
* Senior Software Engineer: https://about.nextdoor.com/careers-list/?gh_jid=1228742&gh_src=f342d7fd1
* Data Scientist: https://about.nextdoor.com/careers-list/?gh_jid=1509433&gh_src=f342d7fd1
* Machine Learning Engineer: https://about.nextdoor.com/careers-list/?gh_jid=1241186&gh_src=f342d7fd1
* Engineering Manager: https://about.nextdoor.com/careers-list/?gh_jid=1215063&gh_src=f342d7fd1
* Product Manager, Community Vitality: https://about.nextdoor.com/careers-list/?gh_jid=1621033&gh_src=f342d7fd1
And in Toronto:
* Software Engineer, Android: https://about.nextdoor.com/careers-list/?gh_jid=1851532&gh_src=f342d7fd1
* Software Engineer, iOS: https://about.nextdoor.com/careers-list/?gh_jid=1851506&gh_src=f342d7fd1
* Software Engineer: https://about.nextdoor.com/careers-list/?gh_jid=1851545&gh_src=f342d7fd1
We're…
I don't really get into this argument anymore - I'm convinced opinions like this are held tighter than religion or politics.
Here are a few thoughts, written by others, that I would probably bring up if I wanted to get into this argument:
https://medium.com/eshares-blog/why-you-shouldnt-use-cron-jobs-5d76001e18d5
https://engblog.nextdoor.com/we-don-t-run-cron-jobs-at-nextdoor-6f7f9cc62040
https://medium.com/videoamp/what-we-learned-migrating-off-cron-to-airflow-b391841a0da4
https://medium.com/@rbahaguejr/airflow-a-beautiful-cron-alternative-or-replacement-for-data-pipelines-b6fb6d0cddef
As a point of reference, I run a company that brokers terabytes of mission critical data each month for our customers. I've advised dozens of companies on how to properly ensure scheduled jobs are maintainable, instrumented, logged, and reliable. The defenses I've heard for "professional" software and systems engineers using cron in production software are appalling and I find the behavior negligent and to be a terminable offense.
Looks good. Very much looking forward to this being production ready. I am currently looking for a scheduler that provides a better UI/UX than cron alone.
Currently, what do you all use for this sort of thing?
Nextdoor has a not-yet-open-sourced solution: https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-jobs-at-nextdoor/ that looks great.
Does anyone have any experience with Yelp's tron? https://github.com/Yelp/Tron
I think they have a task worker system built back in 2014[1], so they need to have something custom working with it as well. Back then I think they really didn't have many options, but if they were to do it again now, I think either AWS Lambda or AWS Batch will serve this type of scheduled job cases very well.
[1]: https://engblog.nextdoor.com/nextdoor-taskworker-simple-efficient-amp-scalable-ac4f7886957b
It seems that Nextdoor also had issues with celery[0].
"Scalability" is a great scapegoat for making dubious decisions, but my guess here would be the "task priority" requirement.
[0] https://engblog.nextdoor.com/nextdoor-taskworker-simple-efficient-amp-scalable-ac4f7886957b
Ditto, especially in combination with Python/Django, as used by Nextdoor. Ironically, they had already removed Celery from their stack a few years prior. https://engblog.nextdoor.com/nextdoor-taskworker-simple-efficient-amp-scalable-ac4f7886957b