We found disk issues in one of the nodes in our EU cluster queues. This would have resulted in degraded performance for some of our customers for a period of 4-6 hours. The degraded performance would have resulted in a delay in loading data for the affected pipelines.
After fixing the problem, as a proactive measure, Hevo reloaded all the data that was ingested during that period to ensure no data was lost due to disk issues. The customers will not have to pay for the extra events loaded as a consequence.
We have put necessary monitoring and alerts to make sure we get to know much sooner when such issues happen again.