lint-mem gate can't fail: CI analyses no file, and the gate drops every analyzer finding #2442

Version

main 80eb92a

Platform

Linux (x64): the lint / lint-mem job on ubuntu-latest; reproduced on Ubuntu 24.04 (gcc 13.3, clang-tidy 21.1.8)

Install channel

Built from source

Binary variant

standard

What happened, and what did you expect?

The lint / lint-mem job is green on every PR, but right now it can't turn red. There are two separate problems, and either one is enough on its own:

1. CI analyses nothing. The job doesn't set CC, so on ubuntu-latest make uses gcc and Makefile.cbm:39-44 puts GCC_ONLY_FLAGS into CFLAGS_COMMON. That includes -Wno-stringop-truncation and -Wno-alloc-size-larger-than, which clang rejects under -Werror; the comment on line 37 already says so. lint-mem-ci passes the same CFLAGS_COMMON to clang-tidy-21, so every file fails to compile. Makefile.cbm:1413 sends clang-tidy's stderr to /dev/null, and the gate skips any line it can't parse (scripts/lint-mem-gate.py:201-203), so the gate never sees a finding. On the latest run of fix(config): recover Codex MCP after closing marker loss #2434 the "Memory-analyzer gate" step took 2 s. The same analysis done properly takes 94 s with 16 parallel clang-tidy processes.
2. The gate's regex drops every finding. .clang-tidy:46 sets WarningsAsErrors: '*', so clang-tidy prints the check as [clang-analyzer-core.NullDereference,-warnings-as-errors]. FINDING_RE (scripts/lint-mem-gate.py:50) only allows [\w.-]+ inside the brackets, so no line matches. Fed a real finding, the gate still prints memory gate clean.

There's also a smaller third problem. function_spans() counts the braces in both branches of #ifndef _WIN32 … #else … #endif, and cbm_remove_empty_directory() opens one { in each branch. From there on it treats src/cli/cli.c lines 1576–14103 as that single function. A finding in cbm_cmd_uninstall() is therefore reported, and would be whitelisted and hashed, under the wrong name.

Expected: an analyzer finding without a whitelist entry fails the job.

With problems 1 and 2 worked around, main has 15 findings without a whitelist entry (see Logs). Those need triage before the gate is switched back on, otherwise the job goes red on its first real run.

Reproduction

Clean checkout of main 80eb92a:

# 1. as the CI job runs it
scripts/ci/lint-mem.sh clang-tidy-21; echo "exit=$?"
# -> === memory gate clean ===, exit=0, in about 2 s

# 2. one real finding, analysed with clang flags, fed to the gate
flags=$(make -s -f Makefile.cbm CC=clang --eval 'cbm-flags: ; @echo $(CFLAGS_COMMON) $(SYSROOT_FLAG)' cbm-flags)
checks=$(make -s -f Makefile.cbm --eval 'cbm-checks: ; @echo $(LINT_MEM_CHECKS)' cbm-checks)
clang-tidy-21 --quiet --checks="$checks" src/cli/cli.c -- $flags 2>/dev/null > /tmp/cli.out
python3 scripts/lint-mem-gate.py < /tmp/cli.out; echo "exit=$?"
# -> === memory gate clean ===, exit=0
sed 's/,-warnings-as-errors\]/]/' /tmp/cli.out | python3 scripts/lint-mem-gate.py; echo "exit=$?"
# -> gate FAILS, exit=1

# 3. the same file with the flags the CI job gets (default cc = gcc)
gcc_flags=$(make -s -f Makefile.cbm --eval 'cbm-flags: ; @echo $(CFLAGS_COMMON) $(SYSROOT_FLAG)' cbm-flags)
clang-tidy-21 --quiet --checks="$checks" src/cli/cli.c -- $gcc_flags

Logs

Checkout path shortened to ….

# 3. gcc flags
error: unknown warning option '-Wno-alloc-size-larger-than'; did you mean '-Wno-frame-larger-than'? [clang-diagnostic-unknown-warning-option]
error: unknown warning option '-Wno-stringop-truncation'; did you mean '-Wno-format-truncation'? [clang-diagnostic-unknown-warning-option]
Error while processing …/src/cli/cli.c.

# 2. clang flags
…/src/cli/cli.c:12848:30: error: Array access (from variable 'argv') results in a null pointer dereference [clang-analyzer-core.NullDereference,-warnings-as-errors]
$ python3 scripts/lint-mem-gate.py < /tmp/cli.out
=== memory gate clean ===
$ sed 's/,-warnings-as-errors\]/]/' /tmp/cli.out | python3 scripts/lint-mem-gate.py
=== memory-analyzer findings (gate FAILS) ===
  …/src/cli/cli.c:12848: Array access (from variable 'argv') results in a null pointer dereference [clang-analyzer-core.NullDereference]
      no whitelist entry for cbm_remove_empty_directory()

# all 129 LINT_SRCS, clang flags, suffix stripped: 15 findings, none whitelisted
src/cli/cli.c:12848            clang-analyzer-core.NullDereference
src/cli/hook_augment.c:1192    clang-analyzer-core.uninitialized.ArraySubscript
src/cypher/cypher.c:1795       clang-analyzer-unix.Malloc
src/cypher/cypher.c:5118       clang-analyzer-core.NullDereference
src/main.c:718                 clang-analyzer-core.NullDereference
src/mcp/mcp.c:196              clang-analyzer-core.NullDereference
src/mcp/mcp.c:1851             clang-analyzer-core.NullDereference
src/mcp/mcp.c:2270             clang-analyzer-core.NullDereference
src/mcp/mcp.c:10283            clang-analyzer-core.NullDereference
src/mcp/mcp.c:11165            clang-analyzer-core.NullDereference
src/mcp/mcp.c:12849            clang-analyzer-core.NullDereference
src/mcp/mcp.c:12865            clang-analyzer-core.NullDereference
src/mcp/mcp.c:13424            clang-analyzer-core.NullDereference
src/mcp/mcp.c:15367            clang-analyzer-core.NullDereference
src/pipeline/registry.c:1242   clang-analyzer-core.UndefinedBinaryOperatorResult

Possible fix, not tried in CI:

• Run the gate with clang flags (CC=clang in the job or in scripts/ci/lint-mem.sh). Also make it fail when clang-tidy can't compile a file, instead of dropping those errors.
• Accept the suffix and keep it out of the check name, for example \[(?P<check>[\w.-]+)(?:,-warnings-as-errors)?\].
• Count braces in only one branch of a preprocessor conditional in function_spans().
• Triage the 15 findings in the same change.

I found this while verifying #2434 and haven't triaged the findings themselves.

Confirmations

• I searched existing issues and this is not a duplicate. Nothing covers the lint-mem gate; Local pre-commit hook (scripts/hooks/pre-commit -> make lint) blocks every commit: ~5,100 pre-existing clang-tidy findings across 113 files #1264 is about the pre-commit hook.
• The reproduction uses this repository's own code at main 80eb92a.