On January 16th at 12:10 pm, we received reports from our automated monitoring tools that the Bizzabo platform was inaccessible. Our R&D team began the investigation and realized the system was not responding despite the traffic was not higher than usual and there were no issues with the application layer.
Our engineering team continued investigating and identified an issue with the infrastructure layer. At 12:44 pm the issue was narrowed down to a specific infrastructure component. We began working on a fix and a new configuration was implemented at 12:55 pm. This resolved the issue and brought full functionality back to the platform.
We performed a root cause analysis and identified a process at the infrastructure layer that consumed all available memory leading to a crash of a key component. We already implemented preventive measures to ensure that this doesn’t happen again in the future by creating new monitoring for this resource.