docs: preserve historical DEVX-162 spec before task ID reuse

This commit is contained in:
Emil Simeonov
2026-09-17 11:08:02 +02:00
parent 87d46226f6
commit e0ae1cc378
-41
View File
@@ -1,41 +0,0 @@
# DEVX-162: Fix registry push race condition: serialize uploads + retry on HTTP 500
## Problem
The Gitea container registry (v1.27.2) has a known race condition in
`BlobUploader.Append()` where concurrent blob uploads cause the file
offset and DB model to get out of sync, producing HTTP 500 "offset
mismatch between file and model" errors. This causes the build-images
workflow to fail intermittently when pushing runner images.
The `package_blob_upload` table accumulates stale entries from failed
uploads that worsen the problem over time.
## Approach
Two fixes in devx (a third fix — scheduled cleanup — is tracked
separately as OBL-INFRA-537):
1. Set `DOCKER_MAX_CONCURRENT_UPLOADS=1` in the build-images workflow
to serialize blob uploads and avoid the race condition.
2. Add HTTP 500 retry logic to `push_image` in `build_image.py`.
When a push fails with HTTP 500 (not "already exists"), retry up
to 3 times with exponential backoff (5s, 10s, 20s).
REQ-1: Build-images workflow sets DOCKER_MAX_CONCURRENT_UPLOADS=1
REQ-2: push_image retries on HTTP 500 with exponential backoff
REQ-3: All existing tests pass with 100% coverage
## Test Plan
- Unit tests for retry logic (mock subprocess)
- Manual: trigger build-images workflow and verify push succeeds
## Deploy Plan
- Merge to master
## Rollback Plan
- Revert the merge commit
## Acceptance Criteria
- [x] REQ-1: Build-images workflow sets DOCKER_MAX_CONCURRENT_UPLOADS=1
- [x] REQ-2: push_image retries on HTTP 500 with exponential backoff
- [x] REQ-3: All existing tests pass with 100% coverage