devtake.dev
Web · Unconfirmed

Google Cloud cut off Railway without warning, taking its platform down for eight hours

Google Cloud suspended Railway's production account by mistake on May 19, knocking the deploy platform offline for roughly eight hours across 3 million users.

Naomi Park · · 4 min read · 5 sources
Railway logo on a dark background, the deploy platform whose Google Cloud account was suspended
Image: railway.com · Source

Google Cloud suspended Railway’s production account on May 19, and the deploy platform went dark for roughly eight hours. Railway says the block was a mistake on Google’s side, part of an automated action that swept up multiple accounts. The real lesson is who holds the off switch.

Railway is a deploy platform that around 3 million developers use to ship apps without managing servers directly. When its upstream provider locked the door, the whole thing fell over. And it didn’t fall over quietly. A company that builds on a hyperscaler can be cut off in seconds, with no warning and little recourse, and that’s the part anyone running production on a single cloud should sit with.

What we know

Here’s the confirmed sequence, almost all of it from Railway’s own write-up of the May 19 incident. The failure wasn’t a Railway bug or a botched deploy. An upstream provider flipped a switch, and a platform serving millions of apps had no way to flip it back. The detail that should travel furthest is how far the damage spread past the cloud that caused it.

  • The outage ran from roughly 22:10 UTC on May 19 to 07:58 UTC on May 20, about eight hours end to end, per Railway’s incident report. Automated monitoring caught API failures at 22:10; by 22:19 Railway had traced the cause to a GCP account suspension.
  • Railway describes the trigger plainly: “Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action” that hit many accounts at once. There was no notice before the block landed.
  • The suspension took out Railway’s GCP-hosted dashboard, API, control plane, and databases, according to the status page. The control plane is the part that serves routing tables to Railway’s edge proxies.
  • It cascaded past Google. Railway’s edge proxies cache routes from that GCP control plane, so when the caches expired, workloads on AWS and Railway’s own bare metal kept running but became unreachable, InfoQ reported. Physically alive, effectively offline.
  • Google restored access fast once Railway escalated. A P0 ticket went in at 22:22 UTC; account access came back at 22:29, per the timeline. Full recovery of networking, storage, and the API took several more hours.
  • On X, Railway said: “Google Cloud has blocked our account, making some Railway services unavailable. We have escalated this directly with Google,” in a status post during the outage.

What we don’t know

The gaps here are Google’s to fill, and so far it hasn’t. As of June 25 there’s still no public root cause.

  • Why Google’s automated system flagged the account. Railway calls it incorrect; Google has not published a root-cause explanation.
  • Whether other GCP customers caught in the same automated sweep saw similar downtime, and how many.
  • What recourse a customer actually has when a hyperscaler’s automation suspends a paying production account with no human in the loop.

Who reported this

The account is Railway’s own. Founder Jake Cooper told Cybernews he was “gobsmacked” by the suspension, as InfoQ relayed. Railway also took the blame for its own design in the incident report: “We take full responsibility for the architectural decisions that allowed a single upstream provider action to cascade into a platform-wide outage.” Google, for its part, has not issued a public statement explaining what happened.

This is not the first time Google Cloud automation has nuked a major customer. In 2024 it accidentally deleted the entire private-cloud account of UniSuper, a $125 billion Australian pension fund, locking out 620,000 members for about a week, as Business Standard reported. That fund only recovered because it kept backups with a second provider. The pattern is the part worth watching.

What this means for you

If your business runs on one cloud, your uptime is partly a function of that cloud’s automated trust-and-safety systems, and you do not control those. Railway is a sophisticated infrastructure company, not a weekend project, and a single flag still took it fully offline. The fix it announced is the right read of the lesson: pull GCP out of the data plane’s hot path, keep it for failover only, spread database shards across AWS and its own metal, and build a routing mesh that survives any one provider going dark. For your own stack, the cheap version of that lesson is concrete. Know which single dependency, if suspended at 3 a.m., takes everything else down with it. Then make sure a human at the vendor can be reached, and that a cold copy of your control plane lives somewhere else. UniSuper survived because it had a backup account on a different cloud. That detail, not the apology, is the takeaway.

Share this article

Sources

Mentioned in this article