After an extended monitoring period, function execution has returned to normal rates across all queue shards. During this issue, only a subset of users were affected on part of our infrastructure. Our infrastructure team is in the midst of rolling out additional system capacity going forward.
Resolved
After an extended monitoring period, function execution has returned to normal rates across all queue shards. During this issue, only a subset of users were affected on part of our infrastructure. Our infrastructure team is in the midst of rolling out additional system capacity going forward.
Identified
Function execution delays have returned for a subset of users. As we mitigated issues from earlier, some queue related slowness has returned affecting a subset of users running on certain shards.
Resolved
The incident is now resolved and the system is full operational.
Monitoring
We scaled up the infrastructure served the affected subset of customers, which will begin to reduce latency to normal levels.
Investigating
We are actively investigating delayed function execution.