fix(indexer): recover from panic on corrupted issue index search#38212
Draft
bircni wants to merge 1 commit into
Draft
fix(indexer): recover from panic on corrupted issue index search#38212bircni wants to merge 1 commit into
bircni wants to merge 1 commit into
Conversation
A corrupted on-disk bleve index makes the underlying zapx search panic with "index out of range" while decoding a posting list, crashing the GET /issues request with a 500 panic page (go-gitea#38210). Wrap the bleve search in a recover() that converts the panic into an error and logs guidance to rebuild the index, mirroring the existing init-time handling. Add a regression test for the recovery wrapper. Assisted-by: Claude Code:claude-opus-4-8
GiteaBot
added
the
lgtm/need 2
puni9869
reviewed
Jun 25, 2026
|
|
||
| // searchWithPanicRecover runs a bleve search and converts any panic into an | ||
| // error. A corrupted on-disk index (e.g. a truncated zapx posting list) makes | ||
| // the underlying bleve search panic with "index out of range" (see #38210); |
Member
There was a problem hiding this comment.
When any index changed we reindex the model. I did not understood why it got paniced.
puni9869
approved these changes
Jun 25, 2026
GiteaBot
added
lgtm/need 1
Contributor
|
lunny
approved these changes
Jun 25, 2026
GiteaBot
added
lgtm/done
Contributor
|
I don't think it is good to merge. #38212 (comment) And, ideally, the bug should be fixed by upstream, but not by the Gitea side patch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #38210
Searching issues (
GET /issues?...&q=...) can panic withruntime error: index out of range [N] with length N, crashing the request with a 500 panic page. The panic originates in the Blevezapxstorage engine while decoding a term posting list (zapx/v17/memuvarint.goReadUvarint, which readsS[C]with no bounds check), indicating a corrupted on-disk issue index segment.Regression
This surfaced after #37525, which bumped
bleve v2.5.7 -> v2.6.0and introducedzapx/v17 v17.1.2— the exact version in the reported stack trace. The new v17 segment format is what panics on the corrupt posting list. Bumping the dependency does not help:memuvarint.gois byte-for-byte identical in the currentv17.1.6, so the missing bounds check is still upstream.Change
The init path already recovers from corruption panics and logs guidance to rebuild the index, but the search path had no
recover(). This wraps the Bleve search insearchWithPanicRecover, converting a panic into a returned error and logging the rebuild hint, so a single corrupted segment degrades that search instead of crashing the request. A regression test covers the wrapper.Operators still need to rebuild the index (delete
ISSUE_INDEXER_PATHand restart) to clear the corruption.Authored with assistance from Claude Code (Opus 4.8).