Skip to content
How do you handle interrupted AI streams during Railway deploys?
juniordias00
OP

3 hours ago

I’m building an AI app that streams responses to the browser using Server-Sent Events (SSE). One design decision I’m working through is what the user should see if a deploy interrupts a reply halfway through.

If you run a streaming app on Railway, do you let the client reconnect and resume the same reply, or show the partial answer and offer a retry? For those who support resuming, how do you avoid sending a second paid request to the model provider?

I’d be interested in what you chose for a small app and what happened during an actual deploy.

$40 Bounty

2 Replies

Railway
BOT

3 hours ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • about 3 hours ago


3 hours ago

As it is a small app, I'd recommend just showing the partial answer and offering a retry. You could set RAILWAY_DEPLOYMENT_DRAINING_SECONDS to a value higher then 0 (e.g 30s) which allows any current streams to finish, and any request will be routed to the old deployment for 30s after the new one goes up.


milo

As it is a small app, I'd recommend just showing the partial answer and offering a retry. You could set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS` to a value higher then 0 (e.g 30s) which allows any current streams to finish, and any request will be routed to the old deployment for 30s after the new one goes up.

juniordias00
OP

3 hours ago

Thanks, Milo. Keeping the partial reply and letting the user choose whether to retry sounds like a good fit for a small app. Have you seen what happens during a deploy if the browser reconnects within those 30 seconds? I’m curious which deployment handles that new connection.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...