fix(monitor): drain console PTY master in background mode - #247
Merged
Conversation
The monitor never read the console PTY master in background mode, only foreground did (via console_monitor_loop). Once a container's init wrote past the ~64 KiB PTY buffer, /dev/console blocked forever in n_tty_write and the whole container wedged permanently. Add the master to the monitor's existing heartbeat poll loop and drain it continuously, appending output to a per-container console log. This both fixes the deadlock and recovers previously-lost background boot logs.
ravindu644
approved these changes
Jul 23, 2026
ravindu644
pushed a commit
that referenced
this pull request
Jul 26, 2026
The monitor never read the console PTY master in background mode, only foreground did (via console_monitor_loop). Once a container's init wrote past the ~64 KiB PTY buffer, /dev/console blocked forever in n_tty_write and the whole container wedged permanently. Add the master to the monitor's existing heartbeat poll loop and drain it continuously, appending output to a per-container console log. This both fixes the deadlock and recovers previously-lost background boot logs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In background mode (
foreground=0), nothing ever reads the container's console PTY master. The container's/dev/consoleis the matching slave. Once the container's init writes more than the kernel's PTY buffer (~64 KiB) to/dev/console, the write blocks forever inn_tty_write, and because the buffer never drains, every subsequent console write blocks too. The container wedges permanently and can only be killed with SIGKILL.This is independent of guest distro and any sufficiently chatty init triggers it. It reproduces trivially with a bare
dd. All early boot/console output is also lost in background mode, since it dies in the undrained master, so there's no way to see why a container failed to come up.Root cause
console_monitor_loop(src/console.c), called fromsrc/container.c.src/container.cjust polls for the boot marker and returns.src/monitor.c) inherits the master fd but only redirects its own stdio to/dev/null, then sits in itswaitpid/virtualize heartbeat loop. It never reads the master.Fix
The monitor already runs a 500ms heartbeat
poll()loop. This adds the console master to that poll set (in background mode only) and drains it whenever readable —poll()wakes immediately on readability, so draining is effectively real-time. Drained bytes are appended to a per-container console log (Logs/<name>/console, capped at 2 MiB), which also recovers the previously-lost background boot logs. Draining continues even if the log write fails — keeping the buffer empty is the load-bearing part.Foreground mode is untouched. Change is confined to
src/monitor.c.Terminal output
Before the fix — write 1 MiB to a background container's console, it never returns:
After the fix — same write, on both test devices:
Tested environments
Same result on both — rules out this being specific to an old/vendor kernel.
No regressions
console_monitor_loopor its callers).fix itself.
Notes