Back to overview
Resolved

Server outage affecting all services

Nov 7, 2026 at 5:55pm UTC
Affected services
Documentation
File Hosting
Short Links
Website
Spicy Lyrics Mirrors and Forks
SpotMatch

Resolved
Nov 7, 2026 at 7:41pm UTC

Resolved. All services on this page were slow or unavailable on 7 November, from about 18:22 to 19:38 UTC. The evidence points to a fault on the physical Oracle Cloud host running our server, and moving to different hardware fixed it. The server's disk stayed attached, so nothing had to be restored from a backup.

What happened and what we did (all times UTC):
- 17:55: Server CPU jumped to 100% and stayed there.
- 18:22: Our monitors began reporting services as unreachable.
- 18:20 to 18:50: Restarted the server several times, including a full stop and start. This didn't help, and each boot was extremely slow.
- 18:57 to 19:20: Investigated. Boots were slow before any of our services had started, and starting even tiny programs took around 100 times longer than normal. No recent software updates lined up with the problem, which pointed to the host rather than our software.
- 19:24: Moved the server to different hardware in another fault domain.
- 19:32: Server back online and running at normal speed.
- 19:38: All monitors recovered.

We're now reviewing backups, resource limits and alerting so we can catch and recover from this faster in future.