Public Access
DEVX-155: fix: use container overlay for local dockerd data root
/dev/shm is a 16G tmpfs — too small for the 505MB molecule-test-base image. Use /tmp/docker-data on the container's overlay instead, after killing the inner dockerd and cleaning up its data root to free ~2.4GB. Also use the standard DOCKER_SOCK path (/var/run/docker.sock) for the local dockerd, since the inner dockerd has been killed and the socket is free. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
parent
a2d47b7efc
commit
ae33b86ba2
@@ -214,7 +214,9 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
|
||||
# No socket with sufficient space found.
|
||||
# Don't fall back to the low-space inner dockerd — it will fail
|
||||
# on image pulls. Instead, start a local dockerd on /dev/shm.
|
||||
# on image pulls. Instead, kill the inner dockerd, clean up its
|
||||
# data root to free space, and start a new dockerd using the
|
||||
# freed space on the container's overlay.
|
||||
click.echo(_("Host Docker not available, starting local dockerd..."))
|
||||
|
||||
# Kill the inner dockerd (started by the CI image) to free its
|
||||
@@ -229,7 +231,9 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
)
|
||||
time.sleep(2)
|
||||
|
||||
# Clean up the inner dockerd's data root to free space
|
||||
# Clean up the inner dockerd's data root to free space.
|
||||
# The inner dockerd stores images, containers, and volumes here.
|
||||
# Removing them frees up ~2.4GB on the container's overlay.
|
||||
inner_data_root = "/home/grm-ci-runner-*/.local/share/docker"
|
||||
rm_paths = " ".join(f"{inner_data_root}/{d}" for d in ("overlay2", "image", "volumes", "containers"))
|
||||
with contextlib.suppress(Exception):
|
||||
@@ -239,12 +243,12 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
# The existing dockerd (started by the CI image) uses the default data
|
||||
# root on the CI container's overlay (38G, often 100% full). We can't
|
||||
# kill it (different PID namespace) or bind to the same socket. Use a
|
||||
# new socket path and data root on /dev/shm (16G tmpfs).
|
||||
local_sock = "/dev/shm/docker.sock" # nosec B108
|
||||
docker_data_root = "/dev/shm/docker" # nosec B108
|
||||
# Use the inner dockerd's socket path (now free after pkill)
|
||||
# and a fresh data root on the container's overlay.
|
||||
# /dev/shm is a small tmpfs (16G) — too small for images.
|
||||
# The container's overlay (38G) has more space after cleanup.
|
||||
local_sock = DOCKER_SOCK
|
||||
docker_data_root = "/tmp/docker-data" # nosec B108
|
||||
|
||||
# Remove stale socket if present
|
||||
with contextlib.suppress(OSError):
|
||||
@@ -253,9 +257,10 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
os.environ["DOCKER_HOST"] = f"unix://{local_sock}"
|
||||
|
||||
# Start local dockerd (requires privileged container)
|
||||
# Use /dev/shm/docker as data root — the CI container's overlay (38G)
|
||||
# is often 100% full, causing image pull failures. /dev/shm is a 16G
|
||||
# tmpfs with plenty of space for molecule test images.
|
||||
# Use /tmp/docker-data as data root on the container's overlay.
|
||||
# The inner dockerd's data root has been cleaned up, freeing ~2.4GB.
|
||||
# vfs storage driver is used since overlay2 may not work inside
|
||||
# a Docker-in-Docker container without --privileged.
|
||||
log_file = tempfile.NamedTemporaryFile( # noqa: SIM115
|
||||
mode="w", suffix="dockerd.log", delete=False
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user