Repository navigation
provisioner: cap the console logs under --log-dir - #212
Merged
Merged
Conversation
Each runner's serial console was copied to its own file with no limit, no retention and write errors deliberately swallowed, so a job writing to the console could fill the host's disk, and logs of finished runners piled up for ever. Each file now keeps at most --log-max-mib (default 64), ending with a line saying the rest was dropped, and before a new runner's log is opened the oldest are removed until all of them fit in --log-dir-max-mib (default 2048). The files are created 0600 and the directory 0700, since the console can carry what a job printed. The README also says the console reaches the daemon's own log at --log-level debug, which it already did. Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each runner's serial console was copied to its own file under
--log-dir:A job writing to the console could therefore fill the host's disk, and the logs of finished runners piled up forever.
Changes:
--log-max-mib/FR_LOG_MAX_MIB(default 64), then one line saying the rest was dropped..logfiles are removed until all of them fit in--log-dir-max-mib/FR_LOG_DIR_MAX_MIB(default 2048). Other files in the directory are never touched.--log-level debug.Tests are in
internal/provisioner/consolelog_test.goandinternal/config/config_test.go.scripts/mutate.shconfirmed that removing the cap, or removing the pruning, makes them fail.