Skribble AG - Notice history

We have identified the root cause of the issue. It was caused by high event load. Systems are recovering, and signature protocols are becoming available again. We continue to monitor.

High-Level Summary

After moving our systems to a new data center, background data that powers some signature reports was delayed. Reports created between Monday evening and Wednesday evening may have been incomplete. We’ve fixed the issue and will automatically re-create any affected reports as soon as they are accessed.

No data has been lost. Signature reports can be re-created up to 10 years into the past at any time.

Technical Details

Following the data center migration, a backlog of audit events accumulated and was only processed with delay between Monday 23.06. 22:00 and Wednesday 25.06. 18:40.

Impact:

• Audit events in the Signature Activity tab appeared with delay.

• Signature Reports generated during this window were incomplete or not generated, depending on audit event availability at the time.

As of Wednesday 25.06. 18:40, the backlog was fully processed. All delayed audit events have been restored and are complete.

Multiple customers reported incomplete Signature Reports.

We conducted a thorough review of the audit logs and the report generation process. The issue was traced to cached reports that had been created while audit events were still delayed. Since Signature Reports are signed and cached at the time of generation, any reports generated during the affected period remained incomplete, even after the underlying audit data had been fully restored.

Mitigation 26.07. 18:20:

All Signature Reports generated between 23.06. 22:00 and 25.06. 18:40 have been removed. When accessed via UI or API, they will now be automatically re-generated using the complete and accurate audit data.

Fixed Report Cache

Signature Reports are signed and cached at the time of generation. As a result, any reports created during the incident window (23.06. 22:00 – 25.06. 18:40) may still have remained incomplete even after the audit log was fixed.

Mitigation:

We have removed all Signature Reports generated during the affected timeframe. These reports will now be automatically re-generated on demand (via UI or API) using the complete and corrected audit data

The audit event backlog has been fully processed. All delayed audit entries are now complete and available in the system.

The Signature Activity tab in the admin interface is now displaying all audit events correctly.

Incident Summary
As part of preparations for our upcoming data center migration on 23.06.2025, a critical infrastructure update introduced a version incompatibility. This led to a database schema corruption within our authentication service.

To ensure system stability and maintain data integrity, we performed a database restore, which successfully resolved the issue.