CommonLit - Database unavailable – Incident details

All systems operational

Database unavailable

Resolved
Operational
Started over 4 years agoLasted about 1 hour

Affected

commonlit.org

Operational from 5:11 PM to 6:24 PM

Updates
  • Resolved
    Resolved

    Thank you for your patience, we believe the issue is now fully resolved. Our team performed a major upgrade to our systems on Wednesday, November 11th. After that migration our replication system which builds data for reporting began to fall behind by a slowly increasing margin. This morning at 11:54 EST, the replication lag led to a buildup of almost 1 TB of replication logs, which filled up our database disk. We resized our database to add more capacity and stopped the faulty replication to purge the log growth. The combination of the two changes got us working again at 12:11 EST.

  • Monitoring
    Monitoring

    Our team has resolved the issue and we are continuing to monitor. commonlit.org is available for general use.

  • Identified
    Identified

    We are investigating an issue with our database that is making the application unavailable. We will continue to provide updates. We expect commonlit.org will be unavailable for for at least 30 minutes.