Add opt-in RGB lighting, HID gamepad, and KBHE 75HE support

[Fefedu973]

Why

This PR is a concrete, reviewable implementation of the two proposals discussed in #6 and #11:

• an opt-in RGB layer that does not impose a blocking LED driver on every keyboard;
• a standards-based HID gamepad alternative that reuses libhmk's existing XInput mappings, analog curves, and report state;
• a complete KBHE 75HE target for the STM32F723VET6, which is the first board exercising both features.

The intent is to keep the library concerns small and explicit. RGB policy lives in a portable core, physical signaling lives behind a backend API, and keyboards without an rgb section do not compile the core, allocate its frame buffers, or schedule its task. HID gamepad is an alternative USB presentation of the existing gamepad model, not a second input engine.

Important

This is opened as a Draft/RFC because both issue threads requested design discussion before adoption, the maintainer does not have this hardware, and the power/topology/configurator decisions below still need agreement. The branch is implemented so the proposal can be reviewed and measured concretely; Draft status is not a claim that the KBHE image is already safe to flash without bring-up.

This PR addresses #6 and #11. I am deliberately not using closing keywords because the hmkconf UI and physical KBHE bring-up should be agreed/validated before either issue is considered fully complete.

Scope at a glance

• Adds stm32f723xx hardware support and keyboards/kbhe-75he/keyboard.json.
• Adds an optional portable RGB core plus a non-blocking WS2812 backend for STM32F723 TIM2_CH1/PA15.
• Adds static, breathing, rainbow, rainbow-wave, and host-live effects.
• Adds a negotiated 64-byte RAW HID protocol for enable, brightness, color/effect, pixel access, and atomic complete frames.
• Adds disabled/XInput/HID gamepad selection while preserving one shared mapping and analog-processing path.
• Adds deterministic generated metadata and the v1.5 -> v1.6 persistent-configuration migration.
• Includes validation/hardening found while exercising the new paths: command response clearing, gamepad parameter validation, deterministic MS OS GUID generation, XInput OUT endpoint re-arming, and correct high-speed USB polling intervals.

How this answers the concerns raised in #6

Concern	Design response in this PR
RGB could slow the firmware	RGB is compile-time opt-in. The portable task runs after matrix/layout/gamepad work, renders at most every 20 ms, and only scales/copies a frame when it is dirty. The STM32F723 backend copies the submitted frame and returns; TIM2 + circular DMA performs the WS2812 transfer. There is no per-pixel bit-banging in the scan loop. Eight LEDs are encoded per DMA half-buffer, reducing the active-transfer callback rate to about 4.17 kHz with a 240 us refill window. Static frames transfer only when changed, and animation stops generating blank transfers when disabled or at zero brightness.
RGB increases current draw	Lighting is optional, can be disabled, has a global 0-255 brightness setting, and KBHE starts conservatively at 50/255. That gives the host and board configuration an immediate way to cap output. This PR does not pretend to infer an absolute milliamp limit for arbitrary LED parts or board power paths; that limit remains board-specific. If desired before merge, the schema can grow a board-level maximum-brightness/power-budget field without changing the backend contract or wire protocol.
Per-key RGB and underglow need different treatment	The library core intentionally models a logical linear pixel array and makes no key-reactive or physical-topology assumption. KBHE declares 82 per-key pixels. An underglow board can provide its own count/backend and later expose zone/topology metadata without forking the renderer or transport. In other words, this PR does not silently treat underglow LEDs as keys; semantic layout belongs in keyboard metadata/configurator UX.
The change also affects the web configurator	Firmware metadata now advertises gamepadApis and, when enabled, RGB LED count/protocol/effect capabilities. The RAW HID protocol also has explicit capability negotiation, so a host never guesses support from a VID/PID alone. I have kept the hmkconf UI out of this firmware PR so the firmware/API can be reviewed first. A separate hmkconf PR can consume the metadata and present per-key versus underglow UI appropriately. The protocol is already exercised by the KBHE configurator and a standalone Python client linked below.
No maintainer hardware is available	The KBHE target and both regression targets compile, and the portable RGB state machine has host tests. I have listed the remaining physical checks explicitly below and am not claiming hardware validation that has not happened. The implementation is isolated so architectural review can happen without needing a KBHE board.

KBHE 75HE support

keyboards/kbhe-75he/keyboard.json describes the production keyboard rather than a synthetic test target:

• STM32F723VET6 family, 16 MHz HSE, 216 MHz system clock;
• integrated USB HS PHY on PB14/PB15, with the board's no-VBUS-sense setup;
• 11 ADC inputs behind three mux select pins (8 mux positions), mapped to the 82 logical keys;
• the 82-key physical layout and default keymap;
• four profiles, four layers, 32 advanced keys, eight DKS bindings, and 128 macro nodes using libhmk's existing engines;
• non-uniform STM32F7 flash geometry for the existing wear-leveling layer;
• 82 WS2812 pixels on PA15/TIM2_CH1;
• dedicated USB VID:PID 9172:0004 so this image cannot be confused with the KBHE native application or updater.

The new STM32F723 driver provides board/clock/USB setup, ADC + DMA mux scanning, flash, CRC32, timer, bootloader jump, serial number, and the RGB backend. USB, ADC, and RGB IRQ priorities are explicit. Instruction cache is enabled; data cache is disabled by default to keep DMA coherency predictable, while the DMA buffers retain aligned cache maintenance for downstream boards that deliberately re-enable it.

This makes the target complete at the firmware/build level: it uses normal libhmk profiles, layers, calibration, advanced keys, macros, RAW HID configuration, and the selected gamepad API. It is not a wrapper around the native KBHE firmware.

RGB architecture

1. Compile-time opt-in and backend boundary

An optional rgb object in keyboard.json supplies:

• LED count (1-255);
• data pin;
• backend name;
• default brightness;
• default base color.

The generator defines RGB_ENABLE and backend constants only for such keyboards. include/hardware/rgb_api.h is intentionally small:

• initialize the physical driver;
• submit a complete logical RGB frame without retaining the caller's pointer;
• report busy state;
• perform deferred non-ISR cleanup.

The portable renderer never knows about WS2812 GRB wire order, timers, or DMA. The first backend converts logical RGB to GRB and uses TIM2 PWM + DMA. Supporting a different MCU or LED transport means implementing that four-function backend, not modifying the effect/command code.

Non-RGB builds retain only the six-byte persistent RGB slot introduced by the common v1.6 configuration layout; they do not contain the renderer, frame buffers, task, commands, or hardware backend. Keeping the slot in the common layout prevents firmware variants from interpreting the same stored configuration at different offsets.

2. Effects and persistence

Portable effect IDs are:

ID	Effect	Persistent?
0	Static base color	yes
1	Breathing base color	yes
2	Rainbow	yes
3	Rainbow wave	yes
7	Host-controlled live frame	no, RAM only

Enable state, brightness, autonomous effect, and base color use libhmk's existing configuration write path. A persistent write is completed before the runtime state is published, so a failed write does not leave RAM and storage reporting different values. Live mode remembers the previous autonomous effect for restore, but live pixels and live effect selection are never written to flash. Streaming therefore cannot create flash traffic or wear.

3. Host-controlled per-LED frames

The 64-byte RAW HID bridge uses byte 0 for the command, byte 1 for status/reserved, and payload from byte 2. Hosts must start with GET_RGB_CAPABILITIES (0x7F), which returns protocol version, LED count, bytes per pixel, maximum chunk size, live-effect ID, capability bits, and logical color order.

Commands cover:

• get/set enabled and brightness;
• get/set a logical pixel;
• get chunks of the displayed frame;
• set complete-frame chunks;
• clear/fill;
• get/set effect;
• restore the prior autonomous effect.

An 82-pixel frame is 246 bytes and travels as four 60-byte chunks plus one six-byte chunk. Upload is transactional:

1. canonical chunk 0 starts a new transaction and clears the receipt bitmap;
2. every chunk must have the canonical offset and exact expected length;
3. data is written to a staging buffer, not the displayed/DMA frame;
4. only the complete bitmap publishes the frame;
5. publication closes the transaction, so a late chunk cannot combine two frames;
6. a direct pixel/fill/effect change cancels an incomplete transaction.

This prevents dropped, duplicated, reordered, or late reports from displaying a half-old/half-new frame. The shared protocol and reference client are documented in the KBHE repository:

• protocol specification
• standalone Python client
• host protocol tests

HID gamepad architecture

The HID implementation deliberately shares libhmk's existing gamepad data rather than adding parallel mappings, as suggested in #11.

• eeconfig_get_gamepad_api() resolves one of disabled, XInput, or HID.
• XInput keeps its existing bit and wins safely if corrupt/legacy data sets both bits.
• HID reuses a formerly reserved bit. The v1.6 migration explicitly clears that historical bit before assigning its new meaning, preventing accidental HID enablement after upgrade.
• COMMAND_SET_OPTIONS rejects mutually enabled XInput/HID flags.
• layout_task() feeds the same button assignments and Hall-effect analog states for either API.
• The existing analog curve, dead zones, joystick conflict handling, triggers, and per-profile mappings remain the single source of truth.
• The final report is serialized either as the existing XInput report or TinyUSB's standard HID gamepad report.

For HID, the existing XInput model maps to four signed stick axes, two trigger axes, an eight-direction hat, and eleven buttons. Opposing D-pad directions cancel on each axis before hat conversion. HID Y axes use the conventional positive-down orientation. Duplicate physical button mappings are aggregated from scratch each matrix pass so releasing one key cannot release a still-held duplicate mapping.

USB descriptors expose exactly one optional gamepad interface in the final interface slot:

• XInput mode: vendor interface + XInput class driver + Microsoft OS 2.0/BOS descriptors;
• HID mode: standard TinyUSB HID gamepad interface, no XInput class driver, no Microsoft OS compatibility descriptor;
• disabled: neither descriptor/interface is exposed.

Changing the setting requires re-enumeration/reboot because it changes the USB descriptor set. The device advertises USB 2.1 only when the XInput BOS path is present and USB 2.0 otherwise. The XInput OUT endpoint is continuously re-armed so host rumble/output reports cannot get stuck retrying, even though rumble behavior itself remains out of scope.

Persistence, compatibility, and generation

• Configuration version moves from v1.5 to v1.6 with an explicit migration.
• Unknown/future layouts are rejected rather than reinterpreted as current data.
• Existing XInput-enabled devices remain XInput-enabled; HID cannot become enabled from the old reserved bit.
• Existing non-RGB keyboards still build through the non-RGB path. HE60/STM32F446 was used as the regression build.
• Metadata generation uses a UUIDv5 derived from USB identity and gzip mtime=0, then writes only when content changes. This removes random/generated churn and makes repeated builds reproducible.
• Firmware metadata advertises available gamepad APIs and optional RGB capabilities for future hmkconf feature detection.
• RAW HID responses are zeroed before dispatch so short responses cannot expose stale bytes from an earlier command.
• Gamepad button IDs and analog-curve monotonicity are validated on write, with a runtime division-by-zero guard for old/corrupt data.
• High-speed USB bInterval now uses the correct exponent encoding (4 for one millisecond / 1 kHz); full-speed behavior is unchanged.

Validation performed

All tests/builds below were run from commit 57074ac3e790ca5b1446aaa3aaef5b5f60afbabe:

• kbhe-75he STM32F723 release build: 44,216 bytes Flash / 26,232 bytes RAM;
• he60 STM32F446 non-RGB regression build: 38,848 bytes Flash / 14,420 bytes RAM;
• portable RGB host test compiled with -Wall -Wextra -Werror and passed;
• RGB host cases cover incomplete/late chunks, direct-write cancellation, effect rendering, persistence failure behavior, and disabled/zero-brightness idle behavior;
• 11 fake-HID Python protocol tests passed against the shared host implementation;
• git diff --check passes.

Opening this PR should also run the repository's complete keyboard build matrix on Linux.

Decisions requested before marking ready

1. Should keyboard.json gain a board-specific maximum brightness or milliamp budget which the core clamps independently of host requests?
2. Should per-key and underglow zones be first-class topology metadata now, or should the first core stay a topology-neutral strip and add zones with the hmkconf UI?
3. Should the companion hmkconf UI be required before the firmware core can merge, or reviewed as a separate follow-up once the protocol is accepted?
4. Would the project prefer this branch split into generic HID gamepad, generic RGB, and STM32F723/KBHE PRs before detailed review?

Hardware validation still required

The KBHE image is compile-validated, but I have not marked the following as verified without a physical board:

• ADC polarity, initial calibration values, and every mux-to-key mapping;
• physical LED order and TIM2/PA15 WS2812 waveform/latch timing on an oscilloscope;
• sustained scan/poll timing while animated and while receiving live frames;
• enumeration and input reports on Windows/XInput plus HID hosts on Windows, Linux, and macOS;
• power draw at representative brightness/colors;
• ROM/bootloader recovery on the KBHE PCB.

Until those checks pass, this should be treated as a reviewable hardware port rather than a claim that flashing is risk-free. A recoverable ST-Link/ROM-DFU path should be used for first bring-up.

Suggested review order

1. include/rgb.h, include/hardware/rgb_api.h, src/rgb.c — portable contract and state machine.
2. src/commands.c, include/commands.h — negotiated RAW HID surface.
3. include/eeconfig.h, src/migration.c — compatibility and v1.6 migration.
4. src/usb_descriptors.c, src/xinput.c — mutually exclusive XInput/HID presentation and shared data path.
5. scripts/schema/keyboard.py, scripts/make.py, scripts/metadata.py — opt-in generation/metadata.
6. src/hardware/stm32f723xx/, hardware/stm32f723xx/board_def.h, keyboards/kbhe-75he/keyboard.json — KBHE hardware port.

I am happy to split the hardware target, generic RGB core, and HID gamepad into separate commits/PRs if that makes upstream review easier; they are kept modular in the code even though this branch demonstrates them together.

[peppapighs]

Thank you for submitting the PR.

I'm not sure when I will have the time to review it, but can you help me split the PR into STM32F723 support -> gamepad -> RGB, so it is easier for me to review?

[Fefedu973]

Yeah sure no problem

[Fefedu973]

Split as requested:

• feat(kbhe): add KBHE 75HE target and required STM32F723 backend #25 — STM32F723 platform + KBHE 75HE target only
• feat(gamepad): add a selectable standards-based HID gamepad mode #26 — independent standards-based HID gamepad support, based directly on dev
• RGB — isolated on Fefedu973:codex/pr-rgb; it will be rebased and opened as the third PR once feat(kbhe): add KBHE 75HE target and required STM32F723 backend #25 and feat(gamepad): add a selectable standards-based HID gamepad mode #26 land, so reviewers see an RGB-only diff instead of the two prerequisite commits again

Both open replacement PRs include architecture, compatibility/migration details and Linux build measurements. I am closing this draft monolith as superseded; no work is being discarded.

[Fefedu973]

Superseded by the split review series described above.

[Fefedu973]

The requested third part is now available as draft #27. It remains explicitly stacked on #25/#26 and will be rebased before being marked ready.