Function execution has resumed normal function for 30+ minutes. The incident will be marked as resolved. A post mortem will be conducted with the team tomorrow after further investigation tonight.
Replay may be used to replay any failed functions during the incident time window: https://www.inngest.com/docs/platform/replay
Resolved
Function execution has resumed normal function for 30+ minutes. The incident will be marked as resolved. A post mortem will be conducted with the team tomorrow after further investigation tonight.
Replay may be used to replay any failed functions during the incident time window: https://www.inngest.com/docs/platform/replay
Monitoring
The issue has been pinpointed to our sharded function state database and a fix has been applied at ~ 2024-09-26T02:10:00Z UTC and errors have stopped. We are monitoring as we continue a deeper investigation.
Identified
We've narrowed the issue down to a system component and are working on a resolution.
Investigating
We're seeing a spike in functions that are failing to execute. We have been investigating and will update here accordingly.
