Database maintenance is complete.

The agent unresponsiveness was caused by a production API key being automatically deactivated when we removed a user from a provider. The issue has been resolved.

The issue has now been resolved.

This issue has now been resolved.

Problem
A backwards-incompatible production database migration renamed tables and columns, including users.autobuild_wizard_done, before the production API was updated.

Impact
Production API pods emitted repeated PostgreSQL “column does not exist” errors. Production was disrupted from approximately 9:12am to 9:51am EDT.

Cause
The migration reached the production database through Edge while production was still running code that referenced the old schema. The migration validation job allowed the rename operations through an unsafe-migration bypass because they were marked reversible.

Resolution
The team rolled production forward by deploying the latest image, restoring schema/code compatibility. Production was confirmed back at 9:51am EDT.

This incident is now resolved. If you retry your threads with errors the message will go through now.

This issue has been resolved.