Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,16 @@ body:
A clear description of the unexpected behavior. Include the relevant
`@RedisCacheable` / `@RedisCacheEvict` annotation and your `resi-cache.*`
configuration.
Redact passwords, credentials, raw cache keys/values and personal data
from configuration and examples before posting publicly.
placeholder: "When I call ... I expected ... but got ..."
validations:
required: true
- type: textarea
id: repro
attributes:
label: Minimal reproduction
description: The smallest code/config that reproduces the issue. A failing test is ideal.
description: The smallest code/config that reproduces the issue. A failing test is ideal. Remove credentials, raw cache keys/values and personal data.
render: java
validations:
required: false
Expand All @@ -62,6 +64,8 @@ body:
capture the `[chain] handler=... decision=... key=... requestId=...`
trace, which correlates every handler in one cache operation by
`requestId`.
DEBUG output can contain sensitive keys or values. Redact those,
credentials and personal data; share only the relevant excerpt.
render: shell
validations:
required: false
Expand Down
28 changes: 0 additions & 28 deletions .github/workflows/_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,33 +16,5 @@ jobs:
- name: Checkout
uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5

- name: Guard against removed-feature regressions in docs
run: |
set -e
# 1. 黑名单:被移除的过度工程符号不应再出现在对外 README 中
banned="CircuitBreakerCacheWrapper RateLimiterCacheWrapper \
SpelConditionEvaluator CacheEvictedEvent CacheMetricsRecorder \
BloomFilterProvider LockProvider RedissonLockProvider"
for sym in $banned; do
hits=$(grep -rnF "\`$sym\`" README.md README.zh-CN.md 2>/dev/null || true)
if [ -n "$hits" ]; then
echo "::error::Removed symbol '$sym' is still referenced as a code identifier in README:"
echo "$hits"
exit 1
fi
done
echo "OK: No removed-symbol regressions (as code identifiers) in README."

# 2. 白名单:README 宣称的关键类必须在 src 存在
for cls in RedisCacheAutoConfiguration RedisProCacheProperties RedisProCache \
RedisProCacheManager CacheHandlerChainFactory RedisCacheInterceptor \
RedisCacheOperationSource RedissonConfiguration; do
if ! find src/main/java -name "${cls}.java" | grep -q .; then
echo "::error::Key class '$cls' referenced in docs not found in src/main/java"
exit 1
fi
done
echo "OK: All key classes referenced in docs exist in src."

- name: Validate contract documentation
run: bash ./scripts/ci/check-docs-contracts.sh
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ contract.

### CI/CD

- Validate the packaged-consumer JDK before building, resolve PATH Java shims,
and reject missing explicitly supplied candidates. Consolidate documentation
checks in one guard with canonical-owner and POM-derived version assertions.
- Document migration CLI preparation, sidecar namespace and recovery limits;
clarify root-file tracking and redact configuration/debug input in bug reports.
- Share verification across PR/main/merge-group/manual/release workflows; fail
closed on malformed job results or missing/skipped integration evidence.
- Reuse verified artifacts for packaged Boot consumers and benchmark smoke;
Expand Down
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ truly necessary, document its reader, distinct responsibility, and lifecycle in
the documentation map. Preserve changelog history, performance evidence, and
unresolved task entries.

The root `.gitignore` uses an allowlist: new top-level files or directories are
silently ignored unless explicitly included. When adding a permanent root
entry, update its allowlist rule and check `git status --short` and
`git check-ignore -v <path>` before staging. Keep temporary reports and generated
outputs ignored; do not force-add them to bypass this boundary.

## Pull requests

Use the repository PR template. Summarize the behavior and evidence, identify
Expand Down
7 changes: 5 additions & 2 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,11 @@ failures separately from test failures.
unless explicitly selected. They run the public value protocol plus a real
Boot application against Redis in minimal, Redisson and observability modes.
Use `bash scripts/ci/check-external-consumer.sh <candidate-directory>` to reuse
an existing verified candidate; without a candidate, the script packages it.
`JAVA_HOME` supplies JDK 21 (`RESICACHE_JDK21` is an optional local override).
an existing verified candidate; an explicitly missing directory fails rather
than rebuilding. Without an argument, the script reuses `target/ci-candidate`
or packages it if absent. JDK selection uses `RESICACHE_JDK21`, then
`JAVA_HOME`, then the Java installation on `PATH`; it requires JDK 21 before
packaging or installing anything.
`CONSUMER_REDIS_PORT` can point to an existing localhost Redis instead of Docker.
- **Topology smoke tests** prove Sentinel master discovery and data/lock access,
and TLS trusted/untrusted certificate behavior in both clients. They do not
Expand Down
103 changes: 100 additions & 3 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,9 +88,106 @@ JSON or JDK serializers. A safe adoption flow is:
4. retain a rollback path until the new representation is trusted.

The migration CLI and properties are operator-directed surfaces. They are not
run automatically at application startup. A serializer change without this
workflow can make existing values unreadable; a cache flush is not the only
rollback strategy and is not required by the documented migration flow.
run automatically at application startup. `DUAL_WRITE` is a one-time batch
conversion to shadow sidecars, not interception of subsequent application
writes. Provide actual application dual writes separately, or quiesce writers
and evictions for the affected keys through validation and cutover. CAS rejects
source changes during cutover; it does not establish a continuous-write protocol.

### Prepare and invoke the operator

Use JDK 21 and the checked-out core JAR with its production runtime dependencies.
The plain library JAR has no executable launcher; do not use `java -jar`.
From the repository root, reuse the existing independent consumer POM template:

```bash
(
set -euo pipefail
./mvnw install -DskipTests -B
operator_dir="$(mktemp -d)"
trap 'rm -rf "$operator_dir"' EXIT
boot_version="$(./mvnw help:evaluate -Dexpression=project.parent.version -q -DforceStdout)"
core_version="$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout)"
sed "s/@BOOT_VERSION@/$boot_version/g" scripts/ci/consumer/pom.xml > "$operator_dir/pom.xml"
./mvnw -f "$operator_dir/pom.xml" -Pminimal -Dresicache.version="$core_version" \
dependency:build-classpath -Dmdep.includeScope=runtime \
-Dmdep.outputFile="$operator_dir/classpath" -B
java -cp "$(cat "$operator_dir/classpath")${APP_VALUE_CLASSPATH:+:$APP_VALUE_CLASSPATH}" \
io.github.davidhlp.spring.cache.redis.serialization.migration.SerializationMigrationCli \
--spring.config.additional-location=file:/secure/resicache-migration.properties \
--resi-cache.serializer.migration.pattern='myapp:orders:*' \
--resi-cache.serializer.migration.phase=SHADOW_READ \
--resi-cache.serializer.migration.dry-run=true
)
```

Run these build commands in the project development environment; run the Java
invocation only in the approved operator environment. Replace the example
pattern and trusted configuration path. Set `APP_VALUE_CLASSPATH` to the JARs
containing the actual application value classes and their dependencies when
needed; the minimal consumer profile supplies library dependencies, not host
classes. Do not put a host Boot executable JAR's nested libraries directly on
this classpath. The subshell stops on failure and cleans up its temporary files.

In the protected configuration file or the deployment's existing secrets
injection, configure `spring.data.redis.*` for the intended Redis deployment,
`resi-cache.serializer.allowed-package-prefixes` for only the trusted value
packages (and required internal types), and the actual legacy format through
`resi-cache.serializer.migration.legacy-serializer`. Keep passwords and payloads
out of command arguments and public logs. Set
`resi-cache.serializer.fail-on-unknown-type=true`: permissive decoding can return
null for invalid current envelopes, which the engine can count as `envelopes`
rather than `failed`. Successful CLI validation alone does not establish a
usable cache hit: test representative converted values through the application's
actual read path, including the compatible `CachedValue` wrapper, nested values
and expiry metadata. Bare legacy DTO/String values are not automatically given
that runtime structure; regenerate incompatible entries through application
cache writes instead of cutting them over.

### Write preflight and recovery

Before any write phase, inventory the complete intended source set and reserve
collision-free `shadow-suffix` and `backup-suffix` namespaces. No source key may
end in either suffix; every derived destination must be absent or verified as
belonging to this migration. Stop on unrelated or unverifiable destinations.
The CLI skips source keys ending in these suffixes and uses UPSERT to overwrite
differing sidecar bytes; a dry run does not detect namespace conflicts. Prevent
other writers from creating reserved keys throughout the migration.

Always specify pattern, phase and dry-run explicitly. The default phase is
read-only `SHADOW_READ`, but `dry-run` itself defaults to false. After a successful
preflight, run `DUAL_WRITE` with dry-run false to produce representative sidecars,
verify actual application reads, then authorize `CUTOVER` separately. Reuse the
same source pattern and suffix settings for `ROLLBACK`; the engine appends the
backup suffix itself. Phase and budget semantics are in
[`REFERENCE.md`](REFERENCE.md#migration-phase-and-budget-semantics).

Inspect the final summary (`scanned`, `envelopes`, `decodedLegacy`, `written`,
`skippedSidecars`, `failed`) and process exit status. Rejected/failed keys make
the CLI exit nonzero, but earlier writes can already have succeeded. Resolve
the cause and reconcile source/sidecar state before retrying; an interrupted
run is incomplete. There is no persisted SCAN cursor or durable checkpoint,
and repeating a dry run or SHADOW_READ can validate the same subset again.
`max-keys` is not a scan, attempt or elapsed-time budget. Even a narrow MATCH
pattern does not bound Redis SCAN work; independently scope the key population
and apply an external execution deadline when required.

Rollback rejects changed existing source bytes, but recreates missing sources
from backups with SET_IF_ABSENT. This can resurrect intentionally evicted stale
values. Quiesce writes and evictions, reconcile prior deletions against the
backup inventory, and exclude their backups before authorizing rollback.
Backups may already have expired; rollback is not guaranteed for every key.

Sidecars copy the source's remaining TTL at creation; persistent sources produce
persistent sidecars. Neither cutover nor rollback removes them and there is no
cleanup phase. Retain backups for the agreed rollback window; after validating
the result and retiring migration/dual-write activity, review and manually
remove only the inventoried sidecars belonging to this migration in controlled
batches. Sidecars are not a durable backup service.

A serializer change without this workflow can make existing values unreadable;
a cache flush is not the only rollback strategy and is not required by the
documented migration flow.

## Release and publication boundary

Expand Down
32 changes: 32 additions & 0 deletions docs/REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,38 @@ Use the bounded shadow-read → dual-write → cutover migration described in
Do not claim that an in-place serializer swap, a cache flush, or a historical
Maven Central artifact proves compatibility with the current line.

### Migration phase and budget semantics

`SerializationMigrationCli` is an independent operator entry point. Exact
settings live in `SerializationMigrationProperties`; the runnable preparation,
namespace preflight and recovery procedure belong to
[`OPERATIONS.md`](OPERATIONS.md#prepare-and-invoke-the-operator).

| Phase | Current effect |
|---|---|
| `SHADOW_READ` | Default; decodes legacy values and checks current envelopes without writing. |
| `DUAL_WRITE` | Writes current-envelope shadow sidecars once; leaves source bytes in place and does not capture future application writes. |
| `CUTOVER` | Writes legacy backup sidecars, then compares/replaces unchanged source bytes with the current envelope using KEEPTTL. |
| `ROLLBACK` | Scans the source pattern plus backup suffix; restores the expected converted source or recreates a missing source from its backup. Changed existing bytes are rejected. |

`dry-run=true` prevents writes; its default is false. `max-keys` limits the
private selected count, not SCAN results or all attempts. In forward phases,
selection follows successful legacy decoding/serialization; current envelopes,
sidecars, completed DUAL_WRITE sidecars and failures before selection do not
consume the limit. Failures after selection do. Rollback selects before legacy
decoding, so its decode failures consume the limit. `batch-size` is a SCAN hint,
not a hard cap. `scanned` excludes forward sidecars and is not the selected
count; `written` counts successful writes and can include both backup and source
writes for one CUTOVER key. Rollback dry runs expose no planned-restoration
count: zero written does not mean no restorations are pending.

Runs have no durable cursor/checkpoint. Completed write-phase state can permit
skipping some keys on a later invocation; dry runs and SHADOW_READ persist no
completion state and may select the same keys again. Reaching max-keys or a
successful exit does not prove the entire matching keyspace was validated.
Per-key failures are counted and scanning continues, then the CLI exits nonzero;
successful earlier writes are not rolled back automatically.

## Errors and diagnostics

Binding validation reports concrete property paths. Missing distributed lock
Expand Down
63 changes: 38 additions & 25 deletions scripts/ci/check-docs-contracts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,7 @@ docs=(
docs/README.md docs/PRODUCT.md docs/ARCHITECTURE.md docs/DEVELOPMENT.md
docs/OPERATIONS.md docs/REFERENCE.md
)
required_docs=(
README.md README.zh-CN.md AGENTS.md
STABILITY.md COMPATIBILITY.md CHANGELOG.md CONTRIBUTING.md SECURITY.md PERFORMANCE.md
docs/README.md docs/PRODUCT.md docs/ARCHITECTURE.md docs/DEVELOPMENT.md
docs/OPERATIONS.md docs/REFERENCE.md
)

for doc in "${required_docs[@]}"; do
for doc in "${docs[@]}"; do
if [[ ! -f "$doc" ]]; then
printf 'Missing canonical documentation entry point: %s\n' "$doc" >&2
exit 1
Expand All @@ -26,32 +19,47 @@ done
for forbidden in \
'pr-checks.yml' \
'maven-failsafe-plugin' \
'Testcontainers | 1.20.4' \
'| Java | 21+ |' \
'| JDK | 21+ |' \
'@ComponentScan' \
'Java 21+' \
'JDK 21+'; do
if grep -nF -- "$forbidden" "${docs[@]}"; then
printf 'Forbidden stale documentation value: %s\n' "$forbidden" >&2
for doc in "${docs[@]}"; do
# History may quote superseded guidance without making it current policy.
case "$doc" in CHANGELOG.md|PERFORMANCE.md) continue ;; esac
if grep -nF -- "$forbidden" "$doc"; then
printf 'Forbidden stale documentation value: %s\n' "$forbidden" >&2
exit 1
fi
done
done

while IFS='|' read -r owner required; do
if ! grep -qF -- "$required" "$owner"; then
printf 'Missing required contract documentation value in %s: %s\n' "$owner" "$required" >&2
exit 1
fi
done <<'CONTRACTS'
COMPATIBILITY.md|PUT, PUT_IF_ABSENT, and CLEAN
COMPATIBILITY.md|Reactive
docs/REFERENCE.md|resi-cache.bloom
CONTRACTS

# README/source checks have one owner so the local guard matches CI.
for symbol in CircuitBreakerCacheWrapper RateLimiterCacheWrapper \
SpelConditionEvaluator CacheEvictedEvent CacheMetricsRecorder \
BloomFilterProvider LockProvider RedissonLockProvider; do
if grep -nF -- "\`$symbol\`" README.md README.zh-CN.md; then
printf 'Removed symbol referenced in README: %s\n' "$symbol" >&2
exit 1
fi
done

for required in \
'PUT, PUT_IF_ABSENT, and CLEAN' \
'Reactive' \
'Testcontainers | 1.20.6' \
'resi-cache.bloom'; do
found=0
for doc in "${docs[@]}"; do
if grep -qF -- "$required" "$doc"; then
found=1
break
fi
done
if [[ "$found" -ne 1 ]]; then
printf 'Missing required contract documentation value: %s\n' "$required" >&2
for class in RedisCacheAutoConfiguration RedisProCacheProperties RedisProCache \
RedisProCacheManager CacheHandlerChainFactory RedisCacheInterceptor \
RedisCacheOperationSource RedissonConfiguration; do
if [[ -z "$(find src/main/java -name "${class}.java" -print -quit)" ]]; then
printf 'Key class referenced in documentation missing from source: %s\n' "$class" >&2
exit 1
fi
done
Expand Down Expand Up @@ -94,6 +102,11 @@ if [[ -z "$pom_testcontainers_version" ]]; then
exit 1
fi

if ! grep -qF -- "| Testcontainers | $pom_testcontainers_version (test scope) |" COMPATIBILITY.md; then
printf 'Testcontainers compatibility table must match POM version: %s\n' "$pom_testcontainers_version" >&2
exit 1
fi

for resource in \
src/test/resources/testcontainers.properties \
src/test/resources/docker-java.properties; do
Expand Down
39 changes: 37 additions & 2 deletions scripts/ci/check-external-consumer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,44 @@
set -euo pipefail
root="$(cd "$(dirname "$0")/../.." && pwd)"
cd "$root"
if [[ -n "${RESICACHE_JDK21:-}" ]]; then
export JAVA_HOME="$RESICACHE_JDK21"
if (( $# > 1 )); then
echo 'Usage: check-external-consumer.sh [candidate-directory]' >&2
exit 1
fi
if (( $# == 1 )) && [[ ! -d "$1" ]]; then
echo "Candidate directory does not exist: $1" >&2
exit 1
fi
jdk_home="${RESICACHE_JDK21:-${JAVA_HOME:-}}"
if [[ -z "$jdk_home" ]]; then
if ! java_path="$(command -v java)"; then
echo 'JDK 21 is required: set RESICACHE_JDK21 or JAVA_HOME, or put it on PATH.' >&2
exit 1
fi
if ! java_settings="$("$java_path" -XshowSettings:properties -version 2>&1)"; then
echo "Cannot run Java from PATH: $java_path." >&2
exit 1
fi
jdk_home="$(awk '/^[[:space:]]*java.home[[:space:]]*=/ {sub(/^[^=]*=[[:space:]]*/, ""); print}' <<< "$java_settings")"
if [[ -z "$jdk_home" ]]; then
echo "Cannot determine java.home from PATH: $java_path." >&2
exit 1
fi
fi
if [[ ! -x "$jdk_home/bin/java" || ! -x "$jdk_home/bin/javac" ]]; then
echo "JDK 21 java and javac are required in $jdk_home/bin." >&2
exit 1
fi
if ! java_settings="$("$jdk_home/bin/java" -XshowSettings:properties -version 2>&1)"; then
echo "Cannot run Java from $jdk_home." >&2
exit 1
fi
java_version="$(awk '$1 == "java.specification.version" && $2 == "=" {print $3}' <<< "$java_settings")"
if [[ "$java_version" != 21 ]]; then
echo "JDK 21 is required; found Java ${java_version:-unknown} in $jdk_home." >&2
exit 1
fi
export JAVA_HOME="$jdk_home"
if [[ ! -d "${1:-target/ci-candidate}" ]]; then
./mvnw clean package -DskipTests -B
python3 scripts/ci/pipeline.py candidate target/ci-candidate
Expand Down
Loading
Loading