Railway Blog

I appreciate this blogpost. I'm the one at Railway responsible for this decision, so I wanted to share some context from our side.

First off, sorry you got nailed by this. I genuinely empathize because _we_ got nailed too - the Railway.com frontend is hosted on Railway, and we had references to these vuln versions buried in old packages that weren't used in live code. We couldn't deploy for a bit until we sorted it out. It sucked.

That said, I believe this was the right call for a few reasons:

1. We have to think about our entire userbase. Our DX makes deploying easy, which attracts a lot of non-technical folks such as PMs, vibe cobers, newbies, etc. A significant chunk of them would either have no idea this was happening, no idea what an RCE even is, or no clue how to fix it.

2. We're trying to break the "I'll fix it later cycle" because that mindset is how security debt piles up. Yes, it's a heavy-handed approach. It shifts the action item left in the SDLC by blocking vuln deploys outright. We _could_ just alert people, and we did, but we've learned the hard way that people don't read emails. This was the only intervention that actually worked. Other platforms like Vercel also took the same approach.

3. This disproportionately impacted users who weren't using Next.js. We had to scramble when attackers leveraging this exploit started causing degradation across <10% of our fleet [0].

Your feedback on container and resource isolation is valid; there's stuff we could do…

Can anyone just have a job now? Or work at a company that actually does something rather than spinning out entire blog posts about their supposed amazing mission? I mean, I just slogged through this mind-numbing blog post (https://blog.railway.app/p/how-we-work-volume-iii) on "how we work," and I still don't have a bloody clue what they actually do.

Seriously, what happened to just having jobs? You know, workers and managers—simple as pie. This straightforward setup has been around since the dawn of time. But no, now we need a novel-length explanation on their work process without ever getting to the point of what they produce. They babble on with gems like, "We aspire to take the limiter off every person at this company and see where the open road takes us." Oh, really? Does the open road lead to anywhere useful, like, say, making something?

Why can't they just say, "We make biscuits. We get orders for biscuits. We have a bunch of people who make the biscuits, and we have managers to oversee them," instead of spouting, "We empower people to reach their own personal development milestones every single day" or some other such corporate drivel? I mean, who writes this stuff? Do they have a department of professional waffle writers?

And what’s with this obsession with "delivering a mission"? Are they running a company or leading a cult? I half expect them to start their meetings with a chant. Maybe throw in some motivational smoke signals for good measure. Can’t we just go…

Railway | Product Eng (full stack), Developer Relations, Infra (Platform) eng, Infra eng - Data Centers| REMOTE (International) | https://jobs.ashbyhq.com/Railway?utm_source=DayMzWzJ0P

Tired of trying to beat kube into shape? Does writing YAML to ship code fill you with utter dread? Dream of a future where deploying software is simple, and you don't need an army of infrastructure engineers to build that perfect janky bash script™ to make life easy?

We're Railway, and we think infrastructure can be better. So far we've built out a platform loved by hundreds of thousands of users who simply tell us "Give me Postgres", "Deploy this repo", and we make it happen

Fair warning! The problems are complex: home-rolled hypervisors, cut-above container orchestration, over/under/whateverlay networks, virtio device drivers, edge proxies, IAM that doesn't suck, kitchen sinks - we need to build it and we're looking for likeminded individuals who think this stuff is fun.

Four open roles (apply below!):

+Product Engineer (full-stack): -Apply here: (https://jobs.ashbyhq.com/Railway/6ddcfe47-6cce-469b-ba6d-4f0e83440c9d?utm_source=DayMzWzJ0P) -Blog post about the team: (https://blog.railway.app/p/team-spotlight-product-engineering)

+Developer Relations Engineer: -Apply here:(https://jobs.ashbyhq.com/Railway/fe33d49c-3502-4d73-80fc-c2c260576846?utm_source=DayMzWzJ0P)

+Infrastructure Engineer: -Apply…

Not long ago Railway moved from GCP to their own infrastructure since it was very expensive for them. [0] Some go for a Oxide rack [1] for a full stack solution (both hardware and software) for intense GPU workloads, instead of building it themselves.

It's very expensive and only makes sense if you really need infrastructure sovereignty. It makes more sense if you're profitable in the tens of millions after raising hundreds of millions.

It also makes sense for governments (including those in the EU) which should think about this and have the compute in house and disconnected from the internet if they are serious about infrastructure sovereignty, rather than depending on US-based providers such as AWS.

[0] https://blog.railway.com/p/data-center-build-part-one

[1] https://oxide.computer/