Resolved -
App syncs have been confirmed to be fully operational and the issue has been resolved. If you sync failed at some point today either via the dashboard or Vercel integration, you can manually sync via the dashboard now: https://www.inngest.com/docs/apps/cloud#manually
Sorry for the issue blocking the dashboard and syncs for part of today. We're running a post mortem to prevent this class of incident tomorrow.
**To re-iterate: the Event API and Function execution did not experience any interruption today.**
May 17, 00:05 UTC
Update -
We've shipped a fix for app syncs and they're fully operational.
May 17, 00:02 UTC
Update -
The Inngest GraphQL API and dashboard are now fully operational. App syncs are still failing and we're isolating that problem and working on a solution.
May 16, 20:46 UTC
Update -
The REST API is working as expected. The GraphQL API which powers the Inngest webapp is still having issues. We're working on more remediations.
May 16, 19:32 UTC
Update -
We've pushed a remediation and some additional fixes, but we're narrowing down a seconary issues with some problematic queries to our config database (this is separate from events, run history, etc.)
May 16, 19:20 UTC
Monitoring -
We've pushed some remediations to fix the bottleneck. Many app syncs during the last hour may have failed. You can manually sync your app via the UI: https://www.inngest.com/docs/apps/cloud
We are also working on a system change to syncs that should prevent this type of bottleneck from happening again.
May 16, 17:44 UTC
Identified -
We've discovered an issue that are causing new App syncs to be prevented from completing. This sync process happens via the API which is affecting it's uptime.
May 16, 17:28 UTC
Investigating -
We're encountering intermittent issues where the API that powers our UI and developer REST API is not responding. Function execution is currently still executing.
May 16, 16:51 UTC