Repository navigation
Port iterative garbage collection to avoid blocking Redis during cleanup - #185
Merged
Merged
Conversation
…y script Co-authored-by: colinmollenhour <38738+colinmollenhour@users.noreply.github.com>
|
|
…n pipeline Co-authored-by: colinmollenhour <38738+colinmollenhour@users.noreply.github.com>
Co-authored-by: colinmollenhour <38738+colinmollenhour@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Port garbage collection logic from Lua-based approach
Port iterative garbage collection to avoid blocking Redis during cleanup
Dec 11, 2025
Owner
|
@copilot Update the README to document the new options |
Co-authored-by: colinmollenhour <38738+colinmollenhour@users.noreply.github.com>
Contributor
Author
Updated README with documentation for the new garbage collection options in commit 9818260. Added:
|
Owner
|
Credit to @nemphys for the GC algorithm which he provided at #181 (comment) |
Owner
|
@copilot Either run |
Co-authored-by: colinmollenhour <38738+colinmollenhour@users.noreply.github.com>
Contributor
Author
Fixed code style issues in commit ee2cd42. Ran
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
There was a problem hiding this comment.
Pull request overview
This PR optimizes Redis garbage collection by porting iterative GC logic from JavaScript to PHP, replacing a blocking Lua-based approach with an incremental scanning method that yields control to Redis between operations. The implementation uses SSCAN to process tag members in chunks, employs pipelining for batch operations, and includes a local cache to minimize redundant EXISTS calls.
Key Changes:
- Introduced new
_collectGarbageIterative()method that processes expired cache entries incrementally using SSCAN instead of blocking SMEMBERS operations - Added three configurable GC parameters (
gc_scan_count,gc_exists_cache_size,gc_remove_chunk_size) to tune performance vs. responsiveness - Created new
safeDeleteTagKeyLua script for atomic key existence checks and tag set removal
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Cm/Cache/Backend/Redis.php | Implements iterative garbage collection with SSCAN, adds new Lua script and configuration options, includes code style improvements (spacing around operators) |
| tests/RedisBackendTest.php | Updates test expectations to reflect that iterative GC uses EVAL directly (no cached scripts), includes code style fixes |
| README.md | Adds documentation for three new GC configuration options with tuning recommendations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
colinmollenhour
approved these changes
Feb 1, 2026
…e-collection-logic
- Extract all Lua scripts to class constants (LUA_SAVE_SCRIPT, LUA_CLEAN_SCRIPT, LUA_SAFE_DELETE_TAG_KEY_SCRIPT) - Add Dragonfly compatibility directive (--!df flags=allow-undeclared-keys) to safe delete script - Deduplicate script definitions in save(), _removeByMatchingAnyTags(), _loadSafeDeleteTagKeyScript(), and _evalSafeDeleteTagKeyWithEval() - Add testLuaScriptHashesAreCorrect() test that validates all SHA1 hashes match script contents - Update LUA_SAFE_DELETE_TAG_KEY_SH1 to correct hash (9af41155cf293a95fd1a39368fecfbe408362380) - Fix testScriptsCaching() to expect safeDeleteTagKey script during GC This makes it IMPOSSIBLE to pass tests with incorrect Lua script hashes. Eliminates 66 lines of duplicated code while adding comprehensive validation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Credit to @nemphys for the GC algorithm which he provided at #181 (comment)
Garbage Collection Optimization - Complete ✅
Summary
Successfully ported the iterative garbage collection logic from JavaScript to PHP, replacing the 100% Lua-based approach that blocks the Redis server for long periods.
Key Changes
1. New Lua Script
safeDeleteTagKeyscript for atomic existence checks and removal from tag setse045179d758adff8057e7bb9d15a365e86eb2ba92. Iterative GC Method
Implemented
_collectGarbageIterative()that:gc_scan_count)gc_exists_cache_sizeentries) to reduce redundant EXISTS callsgc_remove_chunk_size) using EVALusleep(20000)3. Configuration Options
Added three new configuration options:
gc_exists_cache_size(default: 300,000) - Size of local exists cachegc_scan_count(default: 500) - Number of members to scan at a timegc_remove_chunk_size(default: 100) - Number of expired IDs to remove in a batch4. phpredis Compatibility
5. Documentation
Benefits
✅ Reduces Redis blocking time during garbage collection
✅ Allows other requests to be processed between GC iterations
✅ Maintains backward compatibility with the non-Lua fallback
✅ Configurable parameters for tuning performance vs. resource usage
✅ All tests passing (50/50)
✅ Code review completed and addressed
✅ Security scan passed (no vulnerabilities)
✅ Documentation updated
✅ Code style issues fixed
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.