Some sub-issue descriptions lost on creation
Resolved

From April 27th at 9:40 PT to May 5th at 18:00 PT, some sub-issue descriptions were lost on creation. This would have only occurred in situations where sub-issues were created at the same time as their parent issues (when creating sub-issues through a template for example, or when creating sub-issues through the overflow menu on the issue creation modal.) We have notified users who may have been affected directly.

The problem and the fix:

During work on sync optimizations, we extracted issue descriptions from an Issue model to their own model. While doing so we inadvertently introduced a bug causing sub-issue descriptions to not save to the backend in cases where sub-issues were created with a parent issue. The PR introducing this change was larger than usual as releasing these changes in smaller pieces (or gradually) would have impaired Linear’s sync speed due to having to sync issue descriptions twice.

The root cause of this was a limitation in our client side transaction where we were unable to handle multiple levels of dependencies. We missed the bug in testing before merging the PR, and didn’t discover the issue until 17:35 PT on Friday May 5th while investigating a customer report.

On Friday at 18:00 PT we quickly added support for indirect relationships for our new issue description model. A full fix to support these kind of updates generally will be implemented as a follow-up.

Follow-up:

We know that your trust in Linear is affected by any incident, and cases where data loss occurs are particularly painful. We apologize for any work you lost because of this incident and will work to avoid similar problems in the future. We will do this by completing and implementing a long-term fix as part of our follow-up, as well as continuing to both improve our testing and keeping changes small whenever possible to avoid disruptions.

Sat, May 6, 2023, 01:00 AM
1 year ago
Affected components

No components marked as affected

Updates

Resolved

From April 27th at 9:40 PT to May 5th at 18:00 PT, some sub-issue descriptions were lost on creation. This would have only occurred in situations where sub-issues were created at the same time as their parent issues (when creating sub-issues through a template for example, or when creating sub-issues through the overflow menu on the issue creation modal.) We have notified users who may have been affected directly.

The problem and the fix:

During work on sync optimizations, we extracted issue descriptions from an Issue model to their own model. While doing so we inadvertently introduced a bug causing sub-issue descriptions to not save to the backend in cases where sub-issues were created with a parent issue. The PR introducing this change was larger than usual as releasing these changes in smaller pieces (or gradually) would have impaired Linear’s sync speed due to having to sync issue descriptions twice.

The root cause of this was a limitation in our client side transaction where we were unable to handle multiple levels of dependencies. We missed the bug in testing before merging the PR, and didn’t discover the issue until 17:35 PT on Friday May 5th while investigating a customer report.

On Friday at 18:00 PT we quickly added support for indirect relationships for our new issue description model. A full fix to support these kind of updates generally will be implemented as a follow-up.

Follow-up:

We know that your trust in Linear is affected by any incident, and cases where data loss occurs are particularly painful. We apologize for any work you lost because of this incident and will work to avoid similar problems in the future. We will do this by completing and implementing a long-term fix as part of our follow-up, as well as continuing to both improve our testing and keeping changes small whenever possible to avoid disruptions.

Sat, May 6, 2023, 01:00 AM