Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 40 additions & 128 deletions .rhiza/tests/README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,41 @@
# Rhiza Test Suite
# Rhiza Test Suite (`.rhiza/tests/`)

This directory contains the comprehensive test suite for the Rhiza project.
This directory is **synced from [jebel-quant/rhiza](https://github.com/jebel-quant/rhiza)**
and runs in your project with `make rhiza-test`. Its job is to validate the parts of *your*
repository that Rhiza cares about — the metadata, release config, docs and docstrings that
vary per project — using the shared fixtures below.

## Test Organization
> Tests that only exercise Rhiza's *own* template files (Makefile targets, workflow stubs,
> the project skeleton) live in Rhiza's mother-repo `tests/` suite and are **not** synced
> here — they would be identical in every consumer and can't be changed downstream. Put
> your project's own tests under your `tests/` directory, not here.

Tests are organized into purpose-driven subdirectories:
## Layout

### `structure/`
Static assertions about file and directory presence. These tests verify that the repository contains the expected files, directories, and configuration structure without executing any subprocesses.
The suite is flat — one file per concern — but **which files you get depends on the
bundles you sync**. Each is owned by whichever bundle the assertion belongs to, so a Rust
project gets the Rust manifest checks and none of the Python ones:

- `test_project_layout.py` — Validates root-level files and directories
- `test_requirements.py` — Validates `.rhiza/requirements/` structure
| file | owned by | checks |
| --- | --- | --- |
| `conftest.py` | `core` | shared fixtures (`root`, `logger`, `latest_tag`) |
| `test_release_tags.py` | `core` | the newest tag is reachable from a branch |
| `test_readme.py` | `core` | README exists; every `bash` fence parses |
| `test_pyproject.py` | `python-core` | `pyproject.toml` structure, and its `[tool.bumpversion]` block |
| `test_docstrings.py` | `python-core` | doctests across the modules in your source folder |
| `test_readme_validation.py` | `tests` | executes `python` fences and diffs them against `result` (see below) |
| `test_cargo_toml.py` | `rust-core` | `Cargo.toml` structure and the `.bumpversion.toml` wiring |
| `test_go_module.py` | `go-core` | `go.mod`, the `Version` constant, and the same wiring |

### `api/`
Makefile target validation via dry-runs. These tests verify that Makefile targets are properly defined and would execute the expected commands.
Every profile pairs `core` with exactly one language layer, so `conftest.py` is always
present alongside whichever layer's modules arrived.

- `test_makefile_targets.py` — Core Makefile targets (install, test, fmt, etc.)
- `test_makefile_api.py` — Makefile API (delegation, extension, hooks, overrides)
- `test_github_targets.py` — GitHub-specific Makefile targets
### Skipping README code blocks with `+RHIZA_SKIP`

### `integration/`
Tests requiring sandboxed git repositories or subprocess execution. These tests verify end-to-end workflows.

- `test_release.py` — Release script functionality
- `test_book_targets.py` — Documentation book build targets

### `sync/`
Template sync, workflows, versioning, and content validation tests. These tests ensure that template synchronization and content validation work correctly.

- `test_rhiza_version.py` — Version reading and workflow validation
- `test_readme_validation.py` — README code block execution and validation
- `test_docstrings.py` — Doctest validation across source modules

#### Skipping README code blocks with `+RHIZA_SKIP`

By default, every `python` and `bash` code block in `README.md` is executed or
syntax-checked by `test_readme_validation.py`. To mark a block as intentionally
non-runnable (e.g. illustrative snippets or environment-specific commands), add
`+RHIZA_SKIP` to the opening fence line:
By default, every `bash` fence in `README.md` is syntax-checked (`test_readme.py`, any
language) and every `python` fence is executed (`test_readme_validation.py`, Python
projects). To mark a block as intentionally non-runnable — an illustrative snippet, an
environment-specific command — add `+RHIZA_SKIP` to the opening fence line:

~~~markdown
```python +RHIZA_SKIP
Expand All @@ -56,114 +54,28 @@ Markdown renderers (including GitHub) ignore everything after the first word on
a fence line, so the block still renders as a normal highlighted code block.
Blocks without `+RHIZA_SKIP` continue to be validated as before.

### `utils/`
Tests for utility code and test infrastructure. These tests validate the testing framework itself and utility scripts.

- `test_git_repo_fixture.py` — Validates the `git_repo` fixture

### `deps/`
Dependency validation tests. These tests ensure that project dependencies are correctly specified and healthy.

- `test_dependency_health.py` — Validates pyproject.toml and requirements files

### `stress/`
Stress tests that verify Rhiza's stability under heavy load. These tests execute Rhiza-specific operations under concurrent load and repeated execution to detect race conditions, resource leaks, and performance degradation.

- `test_makefile_stress.py` — Makefile operations under concurrent/repeated load
- `test_git_stress.py` — Git operations under concurrent load

See [stress/README.md](stress/README.md) for detailed documentation.

## Running Tests

### Run all tests
```bash
uv run pytest .rhiza/tests/
# or
make test
```

### Run tests from a specific category
```bash
uv run pytest .rhiza/tests/structure/
uv run pytest .rhiza/tests/api/
uv run pytest .rhiza/tests/integration/
uv run pytest .rhiza/tests/sync/
uv run pytest .rhiza/tests/utils/
uv run pytest .rhiza/tests/deps/
uv run pytest .rhiza/tests/stress/
```

### Run stress tests with custom parameters
```bash
# Run all stress tests (default: 100 iterations, 10 workers)
uv run pytest .rhiza/tests/stress/ -v

# Run with fewer iterations (faster)
uv run pytest .rhiza/tests/stress/ -v --iterations=10

# Skip stress tests when running full test suite
uv run pytest .rhiza/tests/ -v -m "not stress"
```

### Run a specific test file
```bash
uv run pytest .rhiza/tests/structure/test_project_layout.py
```

### Run with verbose output
```bash
uv run pytest .rhiza/tests/ -v
```

### Run with coverage
```bash
uv run pytest .rhiza/tests/ --cov
make rhiza-test # run this suite (the usual entry point)
uv run pytest .rhiza/tests/ # equivalent, direct invocation
uv run pytest .rhiza/tests/test_pyproject.py # a single file
uv run pytest .rhiza/tests/ -v # verbose
```

## Fixtures

### Root-level fixtures (`conftest.py`)
- `root` — Repository root path (session-scoped)
- `logger` — Configured logger instance (session-scoped)
- `git_repo` — Sandboxed git repository (function-scoped)
Defined in `conftest.py` and available to every test without import:

### Category-specific fixtures
- `api/conftest.py` — `setup_tmp_makefile`, `run_make`, `setup_rhiza_git_repo`
- `sync/conftest.py` — `setup_sync_env`
- `root` — repository root path (session-scoped)
- `logger` — configured logger instance (session-scoped)

`.rhiza/tests` is on `pythonpath` (see `pytest.ini`), so intra-suite imports resolve
without any `sys.path` manipulation.
Comment on lines +73 to +74

## Writing Tests

### Conventions
- Use descriptive test names that explain what is being tested
- Group related tests in classes when appropriate
- Use appropriate fixtures for setup/teardown
- Add docstrings to test modules and complex test functions
- Use `pytest.mark.skip` for tests that depend on optional features

### Import Patterns
```python
# Import shared helpers from test_utils
from test_utils import strip_ansi, run_make, setup_rhiza_git_repo

# Import from local category conftest (for fixtures and category-specific helpers)
from api.conftest import SPLIT_MAKEFILES, setup_tmp_makefile

# Note: Fixtures defined in conftest.py are automatically available in tests
# and don't need to be explicitly imported
```

## Test Coverage

The test suite aims for high coverage across:
- Configuration validation (structure, dependencies)
- Makefile target correctness (api)
- End-to-end workflows (integration)
- Template synchronization (sync)
- Utility code (utils)

## Notes

- Benchmarks are located in `tests/benchmarks/` and run via `make benchmark`
- Integration tests use sandboxed git repositories to avoid affecting the working tree
- All Makefile tests use dry-run mode (`make -n`) to avoid side effects
98 changes: 0 additions & 98 deletions .rhiza/tests/api/conftest.py

This file was deleted.

63 changes: 0 additions & 63 deletions .rhiza/tests/api/test_github_targets.py

This file was deleted.

Loading
Loading