Skip to content

LT-22728: Scope local libraries to one build - #1105

Open
johnml1135 wants to merge 1 commit into
mainfrom
LT-22728-local-library-selection
Open

LT-22728: Scope local libraries to one build#1105
johnml1135 wants to merge 1 commit into
mainfrom
LT-22728-local-library-selection

Conversation

@johnml1135

@johnml1135 johnml1135 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

.\build.ps1 -LocalLibraries lcm now builds FieldWorks against your local
liblcm checkout with no machine-wide setup, and the next ordinary build puts
the published package back without redownloading it.

The unknown a reviewer starts with is why the previous version deleted cache
entries so aggressively. Because a local pack reused the published version
string, and NuGet resolves an already-extracted (id, version) before it
consults a folder feed — so a local pack could shadow the published package, or
be shadowed by it, with nothing to tell them apart. SIL.Machine showed it
plainly: no GitVersion, so it packed as a flat 3.9.2, identical in name to
nuget.org's. The eviction was a workaround for a naming problem. This branch
fixes the name, which turns the cleanup into ordinary housekeeping.

Where to look

  • Version derivation (LocalLibraries.psm1) — <core>-<branch>.<commit>. A
    published version can no longer be reused; a dirty tree gets .dirty and
    repacks every time, having no stable identity.
  • -p:DisableGitVersionTask=true — GitVersion assigns Version inside an
    MSBuild target, which outranks a command-line property. Without this the stamp
    is silently overridden; proven on libpalaso in the evidence below.
  • Build/LocalLibraries.props, generated and imported by
    SilVersions.props — the restore in PackageRestore.targets runs through
    Exec, a new MSBuild process inheriting no global properties, so a version
    passed on a command line never reached it.
  • Feed moved into the working tree (.localfeed) — a machine-wide feed let
    one working tree's build delete packages another had just produced. This is
    what makes the existing cleanup safe rather than destructive.
  • Setup-LocalLibraries.ps1 — reuses an existing worktree for a branch
    before creating one, fetches but never merges, and never switches a branch.
  • Pack reuse — a repeat build from the same commit reuses the packed package.

Deliberately not here

  • No CI guard: derived versions never reach a commit, so a local package cannot
    pass CI.
  • No per-branch package publishing from the library repositories.
  • Library paths are still environment variables, so two working trees cannot yet
    use different branches of one library.

Verification.�uild.ps1 -LocalLibraries lcm end to end: 114 projects
resolved the local package where none did before the props file existed
, and
an ordinary build then put 11.0.0-beta0178 back without redownloading it. All
five libraries pack from a clean worktree and reuse on a second run. gitlint,
comment hygiene, the PowerShell 5.1/7 compatibility check and the 70-assertion
test script are clean.

LT-22728


Reading this a year from now — start here

The reasoning behind this branch lives here rather than in the tree, on purpose.
The investigation that produced it was a one-time diagnosis, and
Docs/architecture/local-library-debugging.md deliberately carries only what a
developer needs in order to use the workflow.

The one fact worth keeping: NuGet's resolution order is global-packages folder
→ non-HTTP sources (folder feeds) → http-cache → HTTP.
If (id, version) is
already extracted in packages/, the folder feed is never consulted at all.
Every design choice below follows from that sentence.

Decisions, and why

Content-addressed, not branch-addressed. An earlier draft named packages by
branch alone. That is a stable string, so a second pack from a dirty tree
would hit the extracted cache and silently serve the previous build. The commit
hash makes a clean version identify its contents, which is also what makes pack
reuse correct rather than a gamble. A dirty tree cannot be identified this way
at all, so it is marked .dirty and always repacked.

Untracked files count as dirty. The rule is "whatever git would report",
which honours .gitignore and so excludes bin/obj. A new source file added
while prototyping changes the build without changing the commit, so treating it
as clean would serve a stale package. The cost is that a stray note beside the
source keeps the slow path, so the build now names the offending paths.

The core version comes from the library's own GitVersion, probed with
dotnet msbuild -restore -t:GetVersion, so a version bump in the library shows
up in the local package. -restore is required: the target ships inside the
GitVersion package, which a never-built checkout has not restored yet.
SIL.Machine has no GitVersion and falls back to the consumed version.

Discovery before creation. git worktree add refuses a branch already
checked out elsewhere. Rather than fight that, the setup flow reads
git worktree list --porcelain and uses whatever already holds the branch —
both the fast path and the only path git allows.

Build before pack. A package may include output from a target framework its
own project does not build, and packing first fails on the missing file.

Paths not taken

A branch name in the config filename (localLibs.<branch>.props). In a git
worktree .git is a file, not a directory, so discovering the branch at
MSBuild evaluation time — the only point at which PackageReference items can
be declared — breaks in exactly the multi-worktree case that motivates this
work. Branch names also contain /, a detached HEAD has no branch name, and
renaming a branch would silently drop the config. Go, Cargo and Gradle all use a
fixed filename in a per-checkout location for the same reason.

Publishing per-branch prereleases from the library repositories so a
FieldWorks PR could point at an unreleased library. liblcm already does this to
GitHub Packages for same-repo PRs. Dropped deliberately: it needs a
read:packages token for every developer and every CI job, and the intended
policy is that only a real release passes CI.

Nesting a dependency's worktree pinned to a live branch. No established tool
does this. Chromium's DEPS, Zephyr's west and Android's repo all
materialise to a detached HEAD or a pinned revision, and west does so
precisely to avoid the same-branch-checked-out-twice refusal.

Having the build create the worktrees. Every comparable tool keeps sync as an
explicit command. A build that mutates the source tree outside its output
directory is the failure mode, which is why Setup-LocalLibraries.ps1 is
separate and the build only verifies.

Surprising findings

GitVersion silently wins. -p:Version= is a global property, but
GitVersion.MsBuild assigns <Version> inside a target, and a target can
override a global property. This was found only because the first end-to-end
test ran against SIL.Machine — the one library with no GitVersion, and therefore
the one case that could not exhibit the bug.

A build can report using a local library while compiling against the published
one.
MSBuild global properties do not cross an Exec boundary, and
PackageRestore.targets restores through one, reached by the native build before
the managed traversal. The first end-to-end run printed "Using local libraries:
lcm", exited 0, and left every one of 120 project.assets.json files naming the
published version. Reviewer analysis predicted this before it was measured; the
earlier evidence could not have caught it, because the only library tested end to
end was SIL.Machine, whose local version equalled the pin and made the override a
no-op.

git rev-parse --git-common-dir returns a path relative to the caller, not
to the repository. A first cut used it to locate .git/info/exclude and
silently resolved against the wrong directory; --path-format=absolute is
required. The same trap applies to locating a sibling checkout from inside a
worktree, where the parent directory is .tmp/worktrees rather than the
repositories root.

L10NSharp was not broken by this change. It could not be packed at all —
NU5026, missing output/Debug/net461. The discriminating test was to pack
with the original flags, which failed identically, and the main checkout had no
such output either. Its packages include output from target frameworks its own
projects do not build, so it needs a full build first. It packs cleanly now.

Two libraries' symbol directories named paths that are never written, so the
PDB copy did nothing and said nothing. A miss is now reported together with the
directories that were searched.

What this does NOT authorize

This branch does not establish a way to reference an unreleased library from a
merged commit. Derived versions are passed to restore and MSBuild as properties
and never enter SilVersions.props; .localfeed is gitignored; the feed is
never added to nuget.config. A pushed branch therefore carries no reference to
a local package, and CI restores from nuget.org only. If a FieldWorks change
needs a new library API, it cannot go green until that library is released —
the intended constraint, not an oversight.

Deferred, and what would unblock it
  • Per-working-tree library paths. LIBLCM_PATH and its siblings are machine
    global, so two working trees cannot use two different branches of the same
    library. The fix is the pattern this repository already uses twice
    (GlobalInclude.properties, LibraryDevelopment.properties): a fixed-name,
    gitignored Build/LocalLibraries.props imported from Directory.Build.props.
    Setup-LocalLibraries.ps1 would then write the resolved path there instead of
    printing it. Build/Localize.targets still records the original move away
    from that pattern.
  • A named CI check, so a PR waiting on a library release fails legibly
    instead of as a compile error.
  • Wider PDB coverage for L10NSharp and SIL.Machine beyond the directories
    now configured.
Evidence

GitVersion override, libpalaso, same commit:

package name
with the new flags SIL.Core.18.0.0-lt22728-vp.1e46149.nupkg
without them SIL.Core.18.0.0-lt22728-vp0033.nupkg

All five libraries, packed from a clean worktree:

Library GitVersion Packages Repeat build
libpalaso 5.11.1 26 9s → 2s
liblcm 5.6.10 6 verified incl. restore and revert
chorus 5.10.3 6 5s → 2s
L10NSharp 5.11.1 4 6s → 2s
SIL.Machine none 2 7s → 0s

Full chain, liblcm. A probe class committed to a scratch branch produced
11.0.0-lt22728-e2e.08cb359. The marker was present in the packed net462,
net8.0 and netstandard2.0 assemblies; dotnet restore resolved it in 8s;
the published 11.0.0-beta0178 and the local version coexisted in packages/,
with .nupkg.metadata recording the filesystem feed as the source; an ordinary
build then cleared 9 local cache entries and 18 feed packages and left
beta0178 intact.

Symbol fix, SIL.Machine. Output/Debug went from 0 to 2 SIL.Machine PDBs,
where the previously configured path reported the directory missing.

Test assertions were mutation-tested rather than assumed: flipping
.localfeed, the reuse message, and the L10NSharp symbol directory each made
the suite fail, and pass again once restored.

End to end through build.ps1. With -LocalLibraries lcm, 114
project.assets.json files named 11.0.0-feature-grammar-json-exp.dirty; the six
still naming the published version were stale from four days earlier and untouched
by the run. An ordinary build then removed the overrides, emptied the feed, and
returned all 114 to 11.0.0-beta0178.

Not run: the full managed test suite. The diff is PowerShell, markdown,
.gitignore and nuget.config, with no compiled code.

Preflight review details

Code Review Summary

Branch: LT-22728-local-library-selection

Base: origin/main

Date: 2026-08-26

Review model: Claude Opus 5

Files changed: 10

Overview

Local library packs reused the published version string. NuGet resolves an
already-extracted (id, version) from the packages folder before it consults a
folder feed, so a local pack could shadow the published package or be shadowed
by it with nothing to distinguish them. SIL.Machine showed it plainly: with no
GitVersion it packed as a flat 3.9.2, byte-different from but identically
named to the package on nuget.org.

The branch derives each pack's version from the checkout as
<core>-<branch>.<commit>, keeps the local feed inside the working tree, builds
each library before packing it, and adds a setup command that makes a library
branch available as a worktree without disturbing uncommitted work.

Contract/API Changes

No public API change. Build-surface changes only:

  • build.ps1 -LocalLibraries no longer requires LOCAL_NUGET_REPO; the feed
    defaults to .localfeed in the working tree and the variable still overrides.
  • New Build/Setup-LocalLibraries.ps1 with -Library <name>:<branch>.
  • Build/LocalLibraries.psm1 exports eight new functions.
  • Manage-LocalLibraries.ps1 now stamps -p:Version and sets
    -p:DisableGitVersionTask=true while packing.

Findings

Critical - Must address before merge

None.

Important - Should address before merge

  • Docs/architecture/dependencies.md quick start still set
    LOCAL_NUGET_REPO
    (fixed during review: removed the line and reworded the
    surrounding text; it contradicted the updated local-library-debugging.md)
  • L10NSharp could not be packed at all (NU5026, missing net461/net48
    output)
    (fixed during review: each library is now built before it is
    packed, because a package may include output from a target framework its own
    project does not build)

Minor - Consider

  • PdbRelativeDir named directories two libraries do not write, so the
    symbol copy did nothing and said nothing (fixed during review: l10nsharp now
    uses output/Debug/net48, machine takes one directory per pack project, and
    a miss now reports the directories searched. Proven on machine: symbols in
    Output/Debug went from 0 to 2 where the old path reported nothing)
  • test.ps1 ran LocalLibraries.Tests.ps1 unconditionally, ignoring
    -TestProject and -TestFilter (fixed during review: it now runs only when
    neither is given)
  • nuget.config <clear /> dropped inherited user-level sources with no
    hint
    (fixed during review: the comment now says a private feed belongs
    here rather than in user-level configuration)

Required Validation / Evidence

Run and passing:

  • Build/LocalLibraries.Tests.ps1 - passes; 55 assertions. Mutation-tested
    twice (.localfeed to .wrongfeed, and the reuse message) to confirm the new
    assertions actually fire rather than passing vacuously.
  • Build/Agent/comment-hygiene.ps1 -BaseRef origin/main - clean.
  • gitlint --ignore body-is-missing --commits origin/main..HEAD - clean.
  • All five libraries packed from a clean worktree: every produced package
    carries the derived version, and a second run reuses it.
    • liblcm 6 packages, libpalaso 26, chorus 6, L10NSharp 4, SIL.Machine 2.
    • Reuse timings: libpalaso 9s to 2s, chorus 5s to 2s, L10NSharp 6s to 2s,
      SIL.Machine 7s to 0s.
  • GitVersion override proven real on libpalaso: with the new flags
    SIL.Core.18.0.0-lt22728-vp.1e46149.nupkg; without them
    SIL.Core.18.0.0-lt22728-vp0033.nupkg.
  • Full chain on liblcm: a committed change reached the restored assembly in
    net462, net8.0 and netstandard2.0; the published 11.0.0-beta0178 and the
    local version coexisted in the cache; an ordinary build then cleared 9 local
    cache entries and 18 feed packages and left beta0178 intact.

Not run:

  • ./build.ps1 full compile of FieldWorks against a local library. Pack and
    restore are verified; the subsequent compile is not.
  • ./test.ps1 full managed suite. The diff is PowerShell, markdown, gitignore
    and nuget.config only, with no compiled code.
  • ./Build/Agent/Setup-InstallerBuild.ps1 -ValidateOnly - no installer or WiX
    files changed.

Positive Observations

  • The aggressive cache eviction already on the branch turns out to be the
    sanctioned mitigation: NuGet documents no restore flag that bypasses the
    extraction cache for a version already present.
  • Scoping the feed to the working tree made that eviction safe, and let the
    post-pack clear be deleted; Manage-LocalLibraries.ps1 is net shorter.
  • Every git call in the new code is read-only except one worktree add, plus a
    fetch. A test asserts no switch, checkout, reset or clean appears.

Interview Notes

  • Author's boundary: nothing but a real NuGet release may pass CI. Confirmed
    satisfied - derived versions travel as MSBuild properties and never enter
    SilVersions.props, .localfeed is gitignored, and the feed is never added
    to nuget.config, so a pushed branch carries no reference to a local package.
  • -p:DisableGitVersionTask=true also disables GitVersion's DefineConstants
    and assembly-info stamping, so a local package's assembly metadata differs
    from a CI-built one. Author accepted: local versions cannot be mistaken for
    real ones and dirty detection says when to rebuild. The narrower
    -p:UpdateVersionProperties=false was offered and not taken.
  • Author asked for the L10NSharp failure to be investigated rather than
    documented, on the grounds that it was probably already solvable. That was
    correct: it needed a build before the pack, not a toolchain change.
  • Untracked files count as dirty by author's explicit choice, to keep the rule
    to what git itself would report rather than classifying file types.
  • Author asked for every minor finding in the local library area to be fixed
    rather than deferred, so all three were addressed here.

In-Review Quality Check

  • Docs/architecture/dependencies.md corrected; markdown fences verified
    balanced after a bad sed deleted the wrong line mid-edit.
  • Build-before-pack added, then L10NSharp re-verified from a fresh worktree.
  • Comment hygiene re-run after each edit; one over-budget comment shortened.
  • All five library repositories audited against snapshots taken before any
    work: branch, HEAD, worktree list, branch list, status and .git/info/exclude
    all match, including three repositories' pre-existing uncommitted files.

Suggested Review Focus

  • Whether packing should stay build-then-pack for every library, or only
    where a package spans target frameworks.
  • The dirty-tree rule: any untracked file forces a repack, which is safe but
    means a stray note beside the source keeps the slow path.

This change is Reviewable

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

NUnit Tests

    1 files  ±0      1 suites  ±0   11m 20s ⏱️ +28s
5 861 tests ±0  5 780 ✅ ±0  81 💤 ±0  0 ❌ ±0 
5 870 runs  ±0  5 789 ✅ ±0  81 💤 ±0  0 ❌ ±0 

Results for commit b267b2f. ± Comparison against base commit b8f5463.

♻️ This comment has been updated with latest results.

@codecov-commenter

codecov-commenter commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.35%. Comparing base (b8f5463) to head (b267b2f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1105   +/-   ##
=======================================
  Coverage   38.35%   38.35%           
=======================================
  Files        1507     1507           
  Lines      350617   350617           
  Branches    40298    40298           
=======================================
  Hits       134471   134471           
  Misses     186916   186916           
  Partials    29230    29230           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jasonleenaylor jasonleenaylor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnosis in LT-22728 is correct: an extracted package in packages/ survives the
deletion of its source .nupkg, because nuget.config points globalPackagesFolder at the
repository's packages directory and NuGet does not invalidate an extracted ID/version when
the feed copy disappears. That is a real trap and worth closing.

My concern is scope. Before going through the individual findings, I want to put the
proportionality question first, because it may make most of them moot.

The reported bug is really two, with very different severity

Mode 1 — the version pin was never reverted. This is what the ticket's repro actually
does: pack Machine, build, delete the .nupkg, build again. Manage-LocalLibraries.ps1 wrote
the local version into Build/SilVersions.props, so the second build resolves the local
package because the pin still names it. This affects all five libraries — and it is
visible: SilVersions.props is tracked, git status shows it dirty, and the script prints
"To revert: git checkout Build/SilVersions.props" immediately after packing. The only genuinely
surprising part is that deleting the .nupkg produces a silent success rather than a restore
failure, because the stale extraction covers for it.

Mode 2 — the local pack produced the same version string as the published pin. Then even
after a correct revert, the extracted local build masquerades as the published package. No
dirty file, no signal, nothing to notice. This is the dangerous one, and in practice it is
specific to SIL.Machine: Build/SilVersions.props:20 pins SilMachineVersion to 3.9.2,
and Machine's local pack produces 3.9.2. For palaso, lcm, chorus and l10nsharp a local
checkout produces a different version, so once the pin is reverted the cached local package is
inert.

A smaller fix for the dangerous half

Make a local pack produce a version that cannot collide with a published pin — a -local
suffix, a build stamp, anything distinguishable. Then:

  • A local package can never masquerade as the published one, for any library, including Machine.
  • Mode 2 disappears entirely.
  • Mode 1 remains, which is acceptable: it is already visible in git status and already has a
    printed revert instruction. If you want to harden it further, the cheap addition is a warning
    at pack time when the packed version equals the pin — the single case where a stale cache
    entry can quietly change a later build.

That closes the silent failure without changing how anyone works.

What the current approach costs

As written, this PR changes the local-library workflow for all five libraries in service of a
problem whose silent form belongs to one:

  • Manage-LocalLibraries.ps1 -Palaso (and the rest) now throws, redirecting to
    build.ps1 -LocalLibraries palaso.
  • The persistent user-level NuGet source is removed, and nuget.config:31 adds <clear />,
    which discards every inherited user- and machine-level package source for every restore in
    this repository
    — including for developers who never run a local-library build.
  • Every build, whether or not -LocalLibraries was passed, sweeps both the package cache and
    $env:LOCAL_NUGET_REPO, a user-owned folder outside the repository that may be shared with
    other SIL projects and is shared across worktrees.
  • Selection moves from a visible edit to a tracked file to an invisible per-invocation MSBuild
    property.

The persistent local source in particular is the intended path for painless local library
development, not a defect to be eliminated. It is also inert by construction: with exact
version pins in Directory.Packages.props, a local feed containing SIL.LibPalaso 15.0.1-local
is never consulted while the pin says 15.0.0. It becomes live only when a version override
selects it — which is exactly the per-invocation mechanism you built. Please drop <clear />
and keep the persistent source.

Worth knowing, since it is directly upstream of all of this: Build/NuGet.targets used to
carry a CleanNuGet target that did ForceDelete Files="$(fwrt)/packages/" — a standalone,
manually invoked wipe of the package cache. It was removed in 5711bf6be ("Modernize .NET
tooling and enable AI workflows", #678) when that file was deleted, and nothing replaced it.
The gap this PR is filling with automatic cleanup was previously filled by an explicit one.

Cleanup should be deliberate and visible, not automatic

Local library development is uncommon, deliberate, developer-driven work. The person doing it
knows when they have finished. That argues for a well-known post-activity cleanup rather than a
sweep on every build — and it dissolves most of the machinery in this PR:

  • Cleanup can afford to be blunt, because the developer asked for it. packages/ is repo-local
    and disposable; restore repopulates it. Prefix matching over the library families is
    sufficient — which is what Update-VersionAndClearCache already does today via
    CachePrefixes, and what this PR replaces with .nupkg.metadata inspection.
  • The metadata inspection is what creates the failure modes below. Dropping it removes them
    rather than requiring them to be fixed:
    • LocalLibraries.psm1:145 reads $metadata.source outside the try/catch at :138-144,
      under Set-StrictMode -Version Latest. A version-1 .nupkg.metadata (older NuGet) has only
      version and contentHash, so a long-lived packages/ folder containing one aborts the
      build with a property-not-found error. The fixture only ever writes version = 2.
    • A version directory with no .nupkg.metadata at all is skipped (:140-142), so a partially
      extracted package survives the cleanup that exists to remove it.
    • The cleanup at build.ps1:591-592 sits outside every guard. -LocalLibraries is correctly
      refused with -SkipRestore (:597-598), but the reverse order is not: run
      build.ps1 -LocalLibraries machine, then build.ps1 -SkipRestore, and the second run
      deletes the package and then skips the restore that would replace it.
  • Raise the visibility where the developer already is: print the cleanup command at the end
    of a -LocalLibraries build. The person who just did the uncommon thing is told, in that
    moment, how to undo it. That is the part worth engineering.

If the current approach is kept, these still need fixing

The version override does not reach the nested restore.
Build/PackageRestore.targets:100-119 Execs a fresh dotnet restore carrying only
/p:Configuration and /p:Platform. The native build reaches it every run
(mkall.targets:73 DebugProcs -> CopyDlls -> downloadDlls -> RestorePackages), and
native runs first. MSBuild global properties do not cross an Exec boundary, so that restore
resolves versions from SilVersions.props — the published values — with no local feed, and
rewrites project.assets.json before the managed traversal compiles.

I have not run this, so I am not asserting it as fact. What I am confident about is that the
existing evidence cannot settle it: the verification is a local machine 3.9.2 build, and
SilVersions.props:20 pins SilMachineVersion to 3.9.2, so the override was a no-op and the
nested restore was harmless by construction. Machine is simultaneously the only library that
was tested and the only one that could not have exposed this.

Please run it end to end with palaso or lcm, where the local version differs from the pin.
Either the binary is built against the local package and I am wrong, or it is not and the gap
is real. Both outcomes are worth having before merge. If it needs closing, a generated,
gitignored Build/LocalLibraries.props imported by Directory.Packages.props would reach every
MSBuild process including nested Execs — and matches the LibraryDevelopment.properties
precedent.

The test harness never fails properly. Build/LocalLibraries.Tests.ps1:2 sets
$ErrorActionPreference = 'Stop', so the first Write-Error at :129 is terminating: only one
failure ever prints and exit 1 at :131 is unreachable. Build/Agent/CommentHygiene.Tests.ps1
— the file this was modelled on — uses Write-Host -ForegroundColor Red in the loop and then
exit 1. The structure was copied; the detail that makes it work was not.

The new PowerShell escapes the repository's compatibility check. LocalLibraries.psm1 and
LocalLibraries.Tests.ps1 sit in Build/, not Build/Agent/, so powershell-compat.ps1:52-55
does not scan them and .github/workflows/CI.yml:34-50 does not run them under both PowerShell
5.1 and 7 — even though build.ps1 and test.ps1 load the module under 5.1 in CI.

Six of the 25 assertions are invalidated by the changes above, so the test rework is larger
than it looks. Two are incorrect: :102 ($managerText -notmatch 'dotnet nuget add source')
pins the absence of the persistent source, and :119 pins the presence of <clear />. Four are
obsolete: :63 and :65 test the filesystem-versus-HTTP source discrimination, and :110
and :114 test build.ps1 wiring that goes away.

The eleven behavioural assertions survive a design change; the twelve source-text regexes do
not — several would fail on a rename that preserved behaviour exactly, and two now make
correcting a design decision look like breaking tests. Given how much of it the adjustments
invalidate, what does the remainder need a 133-line file and a bespoke harness for?
That is a
genuine question, not a rhetorical one — if the surviving behaviour justifies it, keep it.

Smaller things:

  • build.ps1:614-617 invokes Manage-LocalLibraries.ps1 with & and then tests
    $LASTEXITCODE -ne 0. That script signals failure by throw, never by an exit code, so the
    check reads whatever native process ran last — or, if $LASTEXITCODE is unset,
    $null -ne 0 is true and the build throws "Local library packing failed." spuriously. The
    real failure path is the exception, which is already handled.
  • Pack order is now load-bearing and documented nowhere. It was
    # Pack order: libpalaso first (other libraries may depend on it) with an explicit list; it
    is now $PackOrder = @($LibraryConfig.Keys) (Manage-LocalLibraries.ps1:112), taking its
    order from the declaration order of an [ordered] hashtable in a different file. Alphabetising
    the catalogue — the obvious future tidy-up — silently breaks dependent packs, and no test
    covers ordering. Restore the comment at both sites.
  • The PR body says selected libraries "evict same-version published cache entries". The
    function's own synopsis (LocalLibraries.psm1:85-88) correctly says "every cached version",
    and the test at :83-86 evicts 3.9.3 while packing 3.9.2. The code is right; the body
    promises less than it does.
  • build.ps1:683-686 (FwBuildTasks restore/build) and :564-566 (native freshness refresh)
    run without the /p:Sil*Version properties or the local feed — same class of gap as the
    nested restore, lower risk.
  • The five library names now live in four places: LocalLibraries.psm1:4-42, the -Library
    ValidateSet, the -LocalLibraries ValidateSet at build.ps1:202-203, and the docs table.
    The test asserts the catalogue has five entries but not that the ValidateSets match it.

Comments

The three .SYNOPSIS blocks in LocalLibraries.psm1 (:85-88, :106-109, :114-117) are
one accurate sentence each, stating only their own contract — those are right, and so are the
rewritten .EXAMPLE blocks in Manage-LocalLibraries.ps1:20-27.

Four things to fix:

  • Manage-LocalLibraries.ps1:69-70.PARAMETER VersionOutputPath, "JSON output consumed by
    build.ps1 for invocation-scoped version overrides." Describe the file the parameter names, not
    who reads it; callers change silently.
  • nuget.config:33-36 — the rewritten comment keeps "See
    Docs/architecture/local-library-debugging.md for the full workflow." Editing the block was the
    moment to drop the .md pointer.
  • Two genuine WHY comments were lost in the move. # Pack only the projects FieldWorks uses (avoids native CMake deps) explained something the code cannot show and arrived bare at
    LocalLibraries.psm1:38-41; the pack-order rationale is gone from both files.
  • The one genuinely non-obvious thing is uncommented: the whole design turns on
    .nupkg.metadata's source field being a safe local-versus-published discriminator
    (:56-66, :145), and nothing says why. If the metadata approach survives, that comment is
    the one worth writing.

This review was assisted by Claude Fable 5.

@johnml1135
johnml1135 force-pushed the LT-22728-local-library-selection branch 3 times, most recently from b267b2f to beb75c3 Compare August 26, 2026 23:45
A locally packed library used to reuse the published version string, and
NuGet resolves an already-extracted (id, version) before it consults a
folder feed. A local pack could therefore be shadowed by the published
package, or shadow it, with nothing to tell them apart. SIL.Machine was
the clearest case: it has no GitVersion, so it packed as a flat 3.9.2,
identical to the package on nuget.org.

Derive each pack's version from the checkout instead, as
<core>-<branch>.<commit>, taking the core from the library's own
GitVersion where it has one. GitVersion.MsBuild assigns Version inside a
target, which outranks a command-line property, so it is switched off for
the pack. Because a clean commit identifies its contents, a second build
from the same commit reuses the package already in the feed. An
uncommitted checkout has no stable identity, so it is marked dirty,
repacked every time, and the build names the paths responsible.

Write the selected versions and the feed to a generated
Build/LocalLibraries.props that Build/SilVersions.props imports, rather
than passing them on the command line. The restore in
Build/PackageRestore.targets runs through Exec, which starts an MSBuild
process that does not inherit global properties, so a version passed that
way never reached it: the build reported using a local library while every
project resolved the published one.

Build each library before packing it. A package may include output from a
target framework its own project does not build, and pack alone does not
produce those, which left L10NSharp unpackable.

Keep the feed inside the working tree as .localfeed. A machine-wide feed
let one working tree's build delete packages another had just produced,
which is also why the existing cleanup could not be trusted; scoped to
one working tree, it can be. Skip that cleanup when the build will not
restore, so it cannot remove packages nothing will put back.

Add Setup-LocalLibraries.ps1 to make a library branch available. It finds
the checkout beside FieldWorks or through the library's path variable, and
uses an existing worktree for the branch where there is one, since git
refuses to check a branch out twice and that worktree may hold work in
progress. It fetches but never merges, never switches a branch in a
checkout that already has one, and never prompts.

Leave inherited package sources in place: a local build adds its own feed
for that build only, and the versions it packs cannot collide.

Read the cache metadata defensively. Version 1 records no source, which
under Set-StrictMode ended the build, and a version directory with no
metadata is a partial extraction rather than something to keep. Report
every failing assertion instead of stopping at the first, and cover
Build with the PowerShell compatibility check, which scanned only
Build/Agent.

Verified through build.ps1 against liblcm: 114 projects resolved the local
package where none did before, and an ordinary build then restored the
published one without redownloading it.
@johnml1135
johnml1135 force-pushed the LT-22728-local-library-selection branch from beb75c3 to 049b692 Compare August 26, 2026 23:47
@johnml1135

Copy link
Copy Markdown
Contributor Author

Thank you — this review found a bug that would have shipped, and the analysis was right on every point I could check. Head is now 049b69273; your review was against 260593ec3, so most of what follows is new.

You were right about the nested restore, and it was worse than "not settled"

I ran it end to end with lcm, as you asked. The result on the old code:

Packing as: 11.0.0-feature-grammar-json-exp.dirty
Using local libraries: lcm
BUILD EXIT=0

…and all 120 project.assets.json files named 11.0.0-beta0178. Zero named the local package. The build reported success and compiled against nuget.org's package.

Your reasoning was exactly the mechanism: PackageRestore.targets restores through Exec, global properties do not cross that boundary, and the native build reaches it first. You were also right about why the earlier evidence could not have caught it — SIL.Machine's local pack equalled the pin, so the override was a no-op, and it was simultaneously the only library tested end to end and the only one that could not expose the problem.

Fixed the way you suggested: a generated, gitignored Build/LocalLibraries.props, imported by Build/SilVersions.props after its defaults. Because it is read from disk, it reaches every MSBuild process including nested Execs. Same build now:

before after
projects on the local version 0 114
projects on the published version 120 6, all stale from four days earlier, untouched by the run

And the reverse: an ordinary build removes the overrides, empties the feed, and returns all 114 to 11.0.0-beta0178 without redownloading.

The naming fix

Adopted, and made content-addressed rather than a fixed suffix: <core>-<branch>.<commit>, with the core read from the library's own GitVersion. A stable suffix would still collide with itself — a second pack from a dirty tree would hit the extracted cache and silently serve the previous build — so a dirty tree is marked .dirty and always repacked, and the build now names the paths responsible. Your "warn when the packed version equals the pin" is now structurally impossible rather than warned about.

One thing worth flagging, since it bit me: -p:Version= alone is not enough. GitVersion.MsBuild assigns Version inside a target, which outranks a command-line global property, so the stamp was being silently overridden on the four libraries that use it. Proven on libpalaso, same commit: SIL.Core.18.0.0-lt22728-vp.1e46149.nupkg with -p:DisableGitVersionTask=true, SIL.Core.18.0.0-lt22728-vp0033.nupkg without it.

Everything else you raised

  • <clear /> — dropped, and inherited sources are left alone. Your argument holds and is stronger now: the feed is supplied per-build, so no persistent user-level source is needed either way.
  • Sweeping $env:LOCAL_NUGET_REPO — the feed now defaults to .localfeed inside the working tree. That was also what made the automatic cleanup untrustworthy; scoped to one tree, it can only affect the tree that ran it.
  • Cleanup under -SkipRestore — guarded. It no longer removes packages nothing will put back.
  • Version-1 .nupkg.metadatasource is now read via PSObject.Properties inside the try. Tested with a real v1 file under StrictMode: preserved, no throw.
  • A version directory with no metadata — now removed rather than skipped. It is a partial extraction, and restore replaces it.
  • The test harness never failing properly — correct, and it had been hiding failures from me all along. Write-Host -ForegroundColor Red in the loop, then exit 1. Two injected breaks now surface three failures and a count.
  • $LASTEXITCODE after & — check removed; the exception was already the real path.
  • The compatibility gate not covering Build/powershell-compat.ps1 now scans Build as well as Build/Agent. Clean under 5.1 and 7.
  • Pack order — rationale restored at the catalogue and at $PackOrder.
  • CommentsVersionOutputPath describes the file; the .md pointer is gone from nuget.config; the "only the projects FieldWorks uses (avoids native CMake deps)" WHY is back.
  • Print the cleanup path at the end of a local build — added. You were right that this is the part worth engineering.
  • Five names in four places — a test now asserts both ValidateSets match the catalogue exactly.

On the test file

Fair question, and the honest answer is that the source-text regexes are the weaker half — several would fail on a rename that preserved behaviour. I kept them where the thing being asserted is a property of the source (that a $LASTEXITCODE check is absent, that SilVersions.props imports the overrides after its defaults), and leaned on behavioural assertions elsewhere: it now runs the real functions against real fixtures for version derivation, feed resolution, cache discrimination, v1 metadata, partial extractions, and props generation and removal. 70 assertions, of which 25 are source-text. If you would still rather see it smaller, I would rather cut the regexes than the fixtures.

Scope

I have not shrunk the branch, and I want to be straight about why rather than quietly disagreeing. The nested-restore fix required the generated props file; the props file is what the deferred per-worktree paths needed anyway; and the worktree-local feed is what makes the cleanup you objected to defensible instead of merely convenient. What is left beyond your minimal fix is the setup command, and I would drop that if you want it separate — it is self-contained.

The one thing I did not do is make cleanup fully manual. With the feed inside the working tree and the -SkipRestore guard in place, the automatic sweep can no longer reach anything a developer owns outside the repository. If you still want it behind an explicit command, say so and I will move it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants