3 hours ago
I’m building an AI app that streams responses to the browser using Server-Sent Events (SSE). One design decision I’m working through is what the user should see if a deploy interrupts a reply halfway through.
If you run a streaming app on Railway, do you let the client reconnect and resume the same reply, or show the partial answer and offer a retry? For those who support resuming, how do you avoid sending a second paid request to the model provider?
I’d be interested in what you chose for a small app and what happened during an actual deploy.
2 Replies
3 hours ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 3 hours ago
3 hours ago
As it is a small app, I'd recommend just showing the partial answer and offering a retry. You could set RAILWAY_DEPLOYMENT_DRAINING_SECONDS to a value higher then 0 (e.g 30s) which allows any current streams to finish, and any request will be routed to the old deployment for 30s after the new one goes up.
milo
As it is a small app, I'd recommend just showing the partial answer and offering a retry. You could set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS` to a value higher then 0 (e.g 30s) which allows any current streams to finish, and any request will be routed to the old deployment for 30s after the new one goes up.
3 hours ago
Thanks, Milo. Keeping the partial reply and letting the user choose whether to retry sounds like a good fit for a small app. Have you seen what happens during a deploy if the browser reconnects within those 30 seconds? I’m curious which deployment handles that new connection.