You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
test-build has no real dependency edge on installcheck in base.mk -- both only
share install as a prerequisite. Under make test (serial, -j1), test-build happens
to run after the outer install/installcheck (which runs the main suite plus any test/install schedule), but this is an accident of TEST_DEPS list order interacting
with check-stale-expected's own forced edge onto installcheck (added in #79/#83) --
not a real guarantee. Under make -j, the two run concurrently, both invoking pg_regress against the same live test database at the same time.
Why this matters
Two independent investigations of this in a downstream project (a pgxntool-based
extension) reached opposite conclusions about which runs first by reading base.mk's
declared dependencies -- which is itself a symptom of there being no real, enforced order.
Empirical proof
Added a timestamped marker (\! echo "... $(date +%s%N)" >> /tmp/order_marker.log) to both
a test/install/*.sql file and a test/build/*.sql file, then ran the real suite:
stdout is now interleaved: test-build's own pg_regress invocation (installing files,
running its build test) runs while the outer schedule's remaining tests are still
executing against the same database -- a genuine race, not just a documentation gap.
Root cause
TEST_DEPS = testdeps, then (if enabled) check-stale-expected is appended, then test-build, theninstall installcheck.
check-stale-expected: installcheck (a real edge, added for check-stale-expected's installcheck dependency can run installcheck before install on a fresh tree #79) forces installcheck
(and, via installcheck: install, install) to complete before Make can consider check-stale-expected "done" -- and since check-stale-expected precedes test-build in TEST_DEPS, under -j1 this happens to force installcheck before test-build gets its
turn.
test-build's own rule is only test-build: install -- no edge to installcheck or check-stale-expected. Under -j, nothing stops Make from starting test-build (once install is satisfied) while installcheck is still running in another job slot.
Suggested fix
Add a real edge so the relationship is guaranteed rather than incidental, e.g.:
(after installcheck's own definition, so the ordering is explicit regardless of TEST_DEPS list position). Happy to open a PR if a maintainer confirms this is the
preferred shape -- open question whether test-build should run before or after the main
suite is a design decision maintainers should make explicitly rather than leave to list-order
accident; this issue is about there being no enforced order at all today, not which order
is "correct".
Update: see #108, which makes the case that test-build should gate installcheck (run before it) rather than after -- there's no point running the full suite if the build itself
is broken. That's the direction being adopted; closing this issue as a duplicate of #108.
Workaround applied downstream
That project's Makefile added:
test-build: installcheck
directly (can't edit base.mk, a synced subtree) as an initial fix, later superseded by installcheck: test-build per #108's corrected direction. Verified under both make test
and make -j4 test that either edge removes the interleaving -- the point of this issue was
that no edge existed, not which direction is correct.
Summary
test-buildhas no real dependency edge oninstallcheckinbase.mk-- both onlyshare
installas a prerequisite. Undermake test(serial,-j1),test-buildhappensto run after the outer
install/installcheck(which runs the main suite plus anytest/installschedule), but this is an accident ofTEST_DEPSlist order interactingwith
check-stale-expected's own forced edge ontoinstallcheck(added in #79/#83) --not a real guarantee. Under
make -j, the two run concurrently, both invokingpg_regressagainst the same live test database at the same time.Why this matters
Two independent investigations of this in a downstream project (a pgxntool-based
extension) reached opposite conclusions about which runs first by reading
base.mk'sdeclared dependencies -- which is itself a symptom of there being no real, enforced order.
Empirical proof
Added a timestamped marker (
\! echo "... $(date +%s%N)" >> /tmp/order_marker.log) to botha
test/install/*.sqlfile and atest/build/*.sqlfile, then ran the real suite:make test(serial, default):Non-interleaved stdout confirms the outer
installcheck(alltest/install+test/sqlentries) completes fully before
test-build's own recursive$(MAKE) ... installcheckrecipe even starts.make -j4 test(parallel):stdout is now interleaved:
test-build's ownpg_regressinvocation (installing files,running its
buildtest) runs while the outer schedule's remaining tests are stillexecuting against the same database -- a genuine race, not just a documentation gap.
Root cause
TEST_DEPS = testdeps, then (if enabled)check-stale-expectedis appended, thentest-build, theninstall installcheck.check-stale-expected: installcheck(a real edge, added for check-stale-expected's installcheck dependency can run installcheck before install on a fresh tree #79) forcesinstallcheck(and, via
installcheck: install,install) to complete before Make can considercheck-stale-expected"done" -- and sincecheck-stale-expectedprecedestest-buildinTEST_DEPS, under-j1this happens to forceinstallcheckbeforetest-buildgets itsturn.
test-build's own rule is onlytest-build: install-- no edge toinstallcheckorcheck-stale-expected. Under-j, nothing stops Make from startingtest-build(onceinstallis satisfied) whileinstallcheckis still running in another job slot.Suggested fix
Add a real edge so the relationship is guaranteed rather than incidental, e.g.:
(after
installcheck's own definition, so the ordering is explicit regardless ofTEST_DEPSlist position). Happy to open a PR if a maintainer confirms this is thepreferred shape -- open question whether
test-buildshould run before or after the mainsuite is a design decision maintainers should make explicitly rather than leave to list-order
accident; this issue is about there being no enforced order at all today, not which order
is "correct".
Update: see #108, which makes the case that
test-buildshould gateinstallcheck(runbefore it) rather than after -- there's no point running the full suite if the build itself
is broken. That's the direction being adopted; closing this issue as a duplicate of #108.
Workaround applied downstream
That project's
Makefileadded:test-build: installcheckdirectly (can't edit
base.mk, a synced subtree) as an initial fix, later superseded byinstallcheck: test-buildper #108's corrected direction. Verified under bothmake testand
make -j4 testthat either edge removes the interleaving -- the point of this issue wasthat no edge existed, not which direction is correct.