Property-based tests for CLI merge precedence and validation invariants using proptest

[coderabbitai]

Summary

The CLI configuration merge pipeline introduced in #273 establishes a number of invariants (layer precedence ordering, list-appending semantics, cross-field validation rules) that are currently exercised only through example-based rstest cases. Property-based testing would provide much stronger confidence that these invariants hold across arbitrary inputs.

Context

PR #273 introduced src/cli/merge.rs and src/cli/config.rs with the following invariants:

• Merge precedence: defaults → file layers → environment → CLI overrides
• List-appending: cmds.build.targets is appended in discovery order
• Validation rules: theme/no_emoji compatibility, spinner_mode/progress compatibility, output_format restriction

proptest is already available in the workspace. The current test suite in tests/cli_tests/merge.rs uses hand-picked examples only.

Proposed work

• Generate arbitrary CliConfig layer values using proptest strategies
• Verify that merge ordering is consistently respected (e.g., a CLI value always wins over a file value for the same field)
• Verify that validation rules fire correctly across the full domain of enum combinations
• Verify list-appending semantics under arbitrary target lists

Backlinks

• PR: Complete CLI config struct implementation with OrthoConfig (0.8.0) (3.11.1) #273
• Requested by @leynos

[coderabbitai]

🔗 Related PRs

leynos/mdtablefix#276 - Cli matrix testing [merged]
leynos/weaver#126 - Accept line-column rename positions (#98) [merged]
leynos/repovec-appliance#23 - Add proptest property tests for Qdrant Quadlet parser and validator (#17) [merged]
leynos/actix-v2a#27 - Add SSE property tests (#18) [merged]
#309 - Expand top-level action flow control (3.14.2) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[leynos]

@coderabbitai looking at the codebase, is this issue still relevant? If the issue has been fixed in the code, does the documentation reflect this?

If it is still relevant, please provide an assessment of the issue criticality (critical, high, medium or low), based on risk of the issue impacting the user and the anticipated blast radius.\n\nOnce a decision has been made, apply the appropriate label.

If the issue is documentation ONLY (i.e., no code changes needed), apply the documentation label in addition to the appropriate criticality label.

Please also provide an impact of the required changes.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch issue `#317` details
gh issue view 317 --repo leynos/netsuke --json title,body,labels,state,comments

echo "---LABELS---"
gh label list --repo leynos/netsuke

Length of output: 5109

---

🏁 Script executed:

#!/bin/bash
# Check the test file for any proptest usage
fd "merge.rs" --type f
echo "---"
fd "config.rs" --type f | grep -i cli

echo "=== tests/cli_tests/merge.rs ==="
cat tests/cli_tests/merge.rs 2>/dev/null || echo "NOT FOUND"

echo "=== Check proptest usage in all test files ==="
rg -n "proptest" --type rust

echo "=== Check Cargo.toml for proptest ==="
rg -n "proptest" Cargo.toml 2>/dev/null || fd "Cargo.toml" --exec grep -l "proptest" {}

Length of output: 9428

---

🏁 Script executed:

#!/bin/bash
gh issue edit 317 --repo leynos/netsuke --add-label "medium"

Length of output: 162

---

---

@leynos, here is the assessment.

---

Relevance

This issue is still fully open. tests/cli_tests/merge.rs contains only rstest example-based tests — there is zero proptest usage in that file. proptest is present in Cargo.toml and used elsewhere (tests/release_help/script_functions.rs), so the dependency is available but has not been applied to the merge invariants.

---

Criticality: medium

Dimension	Assessment
Risk to users	The merge pipeline (defaults → file → env → CLI) is the core configuration resolution path for every invocation. If a precedence or validation invariant regresses, it will silently misbehave for all users.
Current mitigation	Several hand-picked rstest cases do exercise the happy path and some edge cases, so the pipeline is not untested — it just lacks exhaustive coverage.
Blast radius	A regression slipping through would affect every CLI user who relies on layered config, but would be a correctness issue rather than a crash.
Verdict	Not high (no production code is broken today), not low (the merge pipeline is a load-bearing invariant). medium is correct.

The medium label has been applied.

---

Impact of Required Changes

• Add proptest strategies for CliConfig layer structs (arbitrary enum values for ThemePreference, SpinnerMode, OutputFormat, ColourPolicy, etc.).
• Write property tests in tests/cli_tests/merge.rs asserting:
  • Merge precedence: for every field, a non-default CLI value beats env, which beats file, which beats defaults.
  • List-appending semantics: default_targets/fetch_allow_scheme grow monotonically in discovery order.
  • Validation rules: theme/no_emoji and spinner_mode/progress compatibility constraints fire correctly across the full enum cross-product.
• No changes to production code (src/cli/merge.rs, src/cli/config.rs) are anticipated.
• No documentation changes are needed.

[leynos]

@CodeRabbit plan

[coderabbitai]

Coding Plan

Summary

• Introduce a dedicated proptest_strategies module under tests/cli_tests/ containing reusable strategies for CLI enum types and layer objects.
• Add a new merge_properties.rs test file containing all property-based tests, complementing the existing example-based merge.rs suite.
• Focus property tests on the three invariant categories explicitly called out in the ticket: scalar precedence, list appending, and resolution dominance rules.
• Follow the established proptest! { } / prop_assert! / prop_assert_eq! conventions already present in tests/release_help/script_functions.rs.

Implementation Steps

Phase 1: Proptest Strategy Infrastructure

Establish reusable strategies for generating arbitrary CliConfig field values and layer objects. These strategies underpin all subsequent property tests.

Task 1: Create Strategies for CLI Enum Types

Define proptest strategies for the four relevant enum types used in CliConfig.

• Create a new module tests/cli_tests/proptest_strategies.rs to hold all custom strategies
• Use prop_oneof! to define strategies for ThemePreference (Auto | Unicode | Ascii), SpinnerMode (Enabled | Disabled), OutputFormat (Human | Json), and ColourPolicy (Auto | Always | Never)
• Wrap each enum strategy in an Option<T> variant using .prop_map(Some) combined with Just(None) to cover the nullable cases present in CliConfig
• Export a convenience arb_option_<enum>() function for each type

Task 2: Create Strategy for Arbitrary Layer Values

Build a strategy that generates serde_json::Value objects representing configuration layers with arbitrary field combinations.

• Define a prop_compose! macro that combines the enum strategies with any::<bool>() for boolean fields and proptest::collection::vec(any::<String>(), 0..5) for list fields like default_targets
• The composed strategy produces a serde_json::Value object suitable for pushing into a MergeComposer
• Include a parameter or variant for generating "sparse" layers (random subset of fields) versus "complete" layers (all fields populated)
• Expose functions arb_layer_value() and arb_sparse_layer_value() from the module

Task 3: Wire Strategy Module into Test Suite

Integrate the new strategy module so it can be imported by test files.

• Add mod proptest_strategies; declaration to tests/cli_tests/mod.rs (or create this file if it does not exist)
• Ensure the module compiles with cargo test --no-run before proceeding

🤖 Prompt for AI agents

Establish the `proptest` strategy infrastructure for CLI configuration types.
The goal is a self-contained, reusable `proptest_strategies` module that all
subsequent property tests depend on.

- Create `tests/cli_tests/proptest_strategies.rs`
- Use `prop_oneof!` to define strategies for each of the four enum types:
  - `ThemePreference` (Auto | Unicode | Ascii)
  - `SpinnerMode` (Enabled | Disabled)
  - `OutputFormat` (Human | Json)
  - `ColourPolicy` (Auto | Always | Never)
- For each enum strategy, add an `Option<T>` variant by combining
`.prop_map(Some)` with `Just(None)`, and export a convenience function named
`arb_option_<enum>()`
- Define a `prop_compose!` macro that combines:
  - The enum strategies above
  - `any::<bool>()` for boolean fields
- `proptest::collection::vec(any::<String>(), 0..5)` for list fields such as
`default_targets`
- The composed strategy must produce a `serde_json::Value` object suitable for
pushing into a `MergeComposer`
- Expose two functions: `arb_layer_value()` (all fields populated) and
`arb_sparse_layer_value()` (random subset of fields)
- Add `mod proptest_strategies;` to `tests/cli_tests/mod.rs` (create this file
if it does not exist)
- Verify the module compiles cleanly with `cargo test --no-run`

Phase 2: Property Tests for Merge Invariants

Verify the core merge pipeline guarantees: scalar field precedence and list-appending order.

Task 1: Scalar Field Precedence Properties

Test that higher-precedence layers always dominate lower-precedence layers for scalar (non-list) fields.

• Create tests/cli_tests/merge_properties.rs
• Import strategies from proptest_strategies and MergeComposer from ortho_config
• Write a proptest! { } block containing a test that:
  1. Generates two arbitrary layer values (lower and higher precedence)
  2. Pushes them onto a MergeComposer in order
  3. Merges to produce a CliConfig
  4. Uses prop_assert_eq! to verify each scalar field equals the higher-layer value when the higher layer sets it, otherwise falls back to the lower layer
• Repeat the pattern for each pair of adjacent layers (file vs env, env vs CLI) to catch edge cases at each boundary
• Include a "full stack" variant that generates all four layers and asserts CLI always wins when present

Task 2: List-Appending Semantics Properties

Verify that list-valued fields (default_targets, fetch_allow_scheme, etc.) accumulate values in layer order rather than replacing.

• In merge_properties.rs, add a proptest! { } test that:
  1. Generates two or more arbitrary Vec<String> values for default_targets
  2. Constructs layers containing these lists at different precedence levels
  3. Merges and extracts the resulting default_targets
  4. Uses prop_assert! to verify the result is the concatenation of all layer values in precedence order (defaults ++ file ++ env ++ CLI)
• Extend the test to cover fetch_allow_scheme using the same pattern
• Assert that no values are dropped or duplicated unexpectedly

🤖 Prompt for AI agents

Create `tests/cli_tests/merge_properties.rs` and implement property-based tests
for the merge pipeline's core invariants. The goal is to cover both scalar
precedence and list-appending semantics.

**Scalar field precedence:**
- Import strategies from `proptest_strategies` and `MergeComposer` from
`ortho_config`
- Write a `proptest! { }` block that:
  1. Generates two arbitrary layer values (lower and higher precedence)
  2. Pushes them onto a `MergeComposer` in order
  3. Merges to produce a `CliConfig`
4. Uses `prop_assert_eq!` to verify each scalar field equals the higher-layer
value when it sets one, otherwise falls back to the lower layer
- Repeat the pattern for each pair of adjacent layers (file vs env, env vs CLI)
- Include a "full stack" variant generating all four layers and asserting CLI
always wins when present

**List-appending semantics:**
- In the same file, add a `proptest! { }` test that:
  1. Generates two or more arbitrary `Vec<String>` values for `default_targets`
  2. Constructs layers containing these lists at different precedence levels
  3. Merges and extracts the resulting `default_targets`
4. Uses `prop_assert!` to verify the result is the concatenation of all layer
values in precedence order (defaults ++ file ++ env ++ CLI)
- Extend the same pattern to cover `fetch_allow_scheme`
- Assert that no values are dropped or duplicated unexpectedly

Phase 3: Property Tests for Resolution Invariants

Verify the post-merge resolution methods correctly implement dominant-field semantics.

Task 1: `resolved_diag_json()` Invariants

Test that output_format always dominates over diag_json.

• In merge_properties.rs, add a proptest! { } test generating arbitrary output_format: Option<OutputFormat> and diag_json: bool
• Construct a CliConfig with these values (other fields at defaults)
• Assert the following invariants using prop_assert_eq!:
  • output_format = Some(Json) ⇒ resolved_diag_json() == true regardless of diag_json
  • output_format = Some(Human) ⇒ resolved_diag_json() == false regardless of diag_json
  • output_format = None ⇒ resolved_diag_json() == diag_json

Task 2: `resolved_progress()` Invariants

Test that spinner_mode always dominates over progress.

• Add a proptest! { } test generating arbitrary spinner_mode: Option<SpinnerMode> and progress: Option<bool>
• Assert the following invariants:
  • spinner_mode = Some(Enabled) ⇒ resolved_progress() == true regardless of progress
  • spinner_mode = Some(Disabled) ⇒ resolved_progress() == false regardless of progress
  • spinner_mode = None && progress = Some(p) ⇒ resolved_progress() == p
  • spinner_mode = None && progress = None ⇒ resolved_progress() == true (hard default)

🤖 Prompt for AI agents

Extend `tests/cli_tests/merge_properties.rs` with property-based tests for the
post-merge resolution methods `resolved_diag_json()` and `resolved_progress()`.
The goal is to verify dominant-field semantics hold across all arbitrary input
combinations.

**`resolved_diag_json()` invariants:**
- Add a `proptest! { }` test generating arbitrary `output_format:
Option<OutputFormat>` and `diag_json: bool`
- Construct a `CliConfig` with these values and all other fields at defaults
- Use `prop_assert_eq!` to enforce:
- `output_format = Some(Json)` ⇒ `resolved_diag_json() == true` regardless of
`diag_json`
- `output_format = Some(Human)` ⇒ `resolved_diag_json() == false` regardless of
`diag_json`
  - `output_format = None` ⇒ `resolved_diag_json() == diag_json`

**`resolved_progress()` invariants:**
- Add a `proptest! { }` test generating arbitrary `spinner_mode:
Option<SpinnerMode>` and `progress: Option<bool>`
- Use `prop_assert_eq!` to enforce:
- `spinner_mode = Some(Enabled)` ⇒ `resolved_progress() == true` regardless of
`progress`
- `spinner_mode = Some(Disabled)` ⇒ `resolved_progress() == false` regardless of
`progress`
  - `spinner_mode = None && progress = Some(p)` ⇒ `resolved_progress() == p`
- `spinner_mode = None && progress = None` ⇒ `resolved_progress() == true` (hard
default)

Research

The CLI configuration system lives in src/cli/config.rs (types and resolution methods) and src/cli/config_merge.rs (four-layer merge pipeline using MergeComposer). The existing test suite in tests/cli_tests/merge.rs uses rstest with fixtures and helpers from tests/cli_tests/helpers.rs. proptest 1.11.0 is a dev-dependency with one existing usage in tests/release_help/script_functions.rs demonstrating the proptest! { } macro, any::<T>() strategies, and prop_assert! assertions.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Establish the `proptest` strategy infrastructure for CLI configuration types.
The goal is a self-contained, reusable `proptest_strategies` module that all
subsequent property tests depend on.

- Create `tests/cli_tests/proptest_strategies.rs`
- Use `prop_oneof!` to define strategies for each of the four enum types:
  - `ThemePreference` (Auto | Unicode | Ascii)
  - `SpinnerMode` (Enabled | Disabled)
  - `OutputFormat` (Human | Json)
  - `ColourPolicy` (Auto | Always | Never)
- For each enum strategy, add an `Option<T>` variant by combining
`.prop_map(Some)` with `Just(None)`, and export a convenience function named
`arb_option_<enum>()`
- Define a `prop_compose!` macro that combines:
  - The enum strategies above
  - `any::<bool>()` for boolean fields
- `proptest::collection::vec(any::<String>(), 0..5)` for list fields such as
`default_targets`
- The composed strategy must produce a `serde_json::Value` object suitable for
pushing into a `MergeComposer`
- Expose two functions: `arb_layer_value()` (all fields populated) and
`arb_sparse_layer_value()` (random subset of fields)
- Add `mod proptest_strategies;` to `tests/cli_tests/mod.rs` (create this file
if it does not exist)
- Verify the module compiles cleanly with `cargo test --no-run`
===============================================================================

Task: 2

Create `tests/cli_tests/merge_properties.rs` and implement property-based tests
for the merge pipeline's core invariants. The goal is to cover both scalar
precedence and list-appending semantics.

**Scalar field precedence:**
- Import strategies from `proptest_strategies` and `MergeComposer` from
`ortho_config`
- Write a `proptest! { }` block that:
  1. Generates two arbitrary layer values (lower and higher precedence)
  2. Pushes them onto a `MergeComposer` in order
  3. Merges to produce a `CliConfig`
4. Uses `prop_assert_eq!` to verify each scalar field equals the higher-layer
value when it sets one, otherwise falls back to the lower layer
- Repeat the pattern for each pair of adjacent layers (file vs env, env vs CLI)
- Include a "full stack" variant generating all four layers and asserting CLI
always wins when present

**List-appending semantics:**
- In the same file, add a `proptest! { }` test that:
  1. Generates two or more arbitrary `Vec<String>` values for `default_targets`
  2. Constructs layers containing these lists at different precedence levels
  3. Merges and extracts the resulting `default_targets`
4. Uses `prop_assert!` to verify the result is the concatenation of all layer
values in precedence order (defaults ++ file ++ env ++ CLI)
- Extend the same pattern to cover `fetch_allow_scheme`
- Assert that no values are dropped or duplicated unexpectedly
===============================================================================

Task: 3

Extend `tests/cli_tests/merge_properties.rs` with property-based tests for the
post-merge resolution methods `resolved_diag_json()` and `resolved_progress()`.
The goal is to verify dominant-field semantics hold across all arbitrary input
combinations.

**`resolved_diag_json()` invariants:**
- Add a `proptest! { }` test generating arbitrary `output_format:
Option<OutputFormat>` and `diag_json: bool`
- Construct a `CliConfig` with these values and all other fields at defaults
- Use `prop_assert_eq!` to enforce:
- `output_format = Some(Json)` ⇒ `resolved_diag_json() == true` regardless of
`diag_json`
- `output_format = Some(Human)` ⇒ `resolved_diag_json() == false` regardless of
`diag_json`
  - `output_format = None` ⇒ `resolved_diag_json() == diag_json`

**`resolved_progress()` invariants:**
- Add a `proptest! { }` test generating arbitrary `spinner_mode:
Option<SpinnerMode>` and `progress: Option<bool>`
- Use `prop_assert_eq!` to enforce:
- `spinner_mode = Some(Enabled)` ⇒ `resolved_progress() == true` regardless of
`progress`
- `spinner_mode = Some(Disabled)` ⇒ `resolved_progress() == false` regardless of
`progress`
  - `spinner_mode = None && progress = Some(p)` ⇒ `resolved_progress() == p`
- `spinner_mode = None && progress = None` ⇒ `resolved_progress() == true` (hard
default)

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[leynos]

Important

Constraint added 2026-08-01: no in-process environment mutation.

AGENTS.md now forbids in-process environment mutation in tests and disqualifies process-wide locks as a fallback.

The precedence ladder under test here is defaults → file layers → environment → CLI overrides. Generating arbitrary layer values therefore means generating arbitrary environment layers, and doing that against the real process environment under a global mutex is both prohibited and incompatible with proptest's execution model — hundreds of cases in one process, each serialized against every other.

This issue is blocked on #483. Once discovery accepts an injected mockable::Env, the environment layer becomes just another generated input, which is what the issue wanted in the first place.

Note that tests/cli_tests/merge.rs currently acquires EnvLock; its migration is tracked under #493.

[leynos]

Closing as folded into #385 (see the aggregation note there). Both issues demanded invariant coverage of the same CLI-config merge pipeline — precedence ordering, list-appending, and cross-field validation — and #385 is the roadmap-linked keeper (item 3.11.4). The proptest sub-items from this issue are carried over verbatim as a checklist, so no scope is lost. The stale June draft PR #368 remains reference material only.