Skip to content

Correlate a behavioral covariate with binned photometry - #425

Merged
pauladkisson merged 7 commits into
binned_metricsfrom
covariates
Aug 24, 2026
Merged

Correlate a behavioral covariate with binned photometry#425
pauladkisson merged 7 commits into
binned_metricsfrom
covariates

Conversation

@pauladkisson

@pauladkisson pauladkisson commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Follow-ups

Details (AI-generated)

Closes #420. Stacked on #422 — base is binned_metrics, not main.

Adds a behavioral covariate store type. A continuous variable measured outside the rig is written as an ordinary GuPPy-CSV (timestamps,data,sampling_rate), dropped in the session folder, and labeled in Step 1; Step 2 carries it into the run folder, Step 3 skips it, and Step 4 averages it onto the bins from #422 and correlates it against every per-bin metric. Outputs are binned_covariates_<site> and covariate_correlations_<site> (.h5 and .csv). Step 5 gains a Covariates tab: a scatter colored by bin number, plus the full correlations table.

No new input parameter — labeling the store is the opt-in. Combine mode and concatenate artifact removal are rejected, since covariate timestamps cannot be interpreted against either time axis.

No p-value, deliberately. Per Harris (2020), Nonsense correlations in neuroscience, per-bin photometry and a slowly varying behavioral score are both strongly autocorrelated, and the standard tests assume independent samples — false-positive rates reach 100% in simulation, and circular shifting and phase randomization do not fix it. The only defensible null here is a session permutation, which needs the group-analysis overhaul. So pearson_r, spearman_rho and n_bins ship as descriptive numbers, with the reasoning in the how-to and outputs.md.

Builds on #427, which names the is_channel_label store-label predicate. That refactor started here: with nothing but "is this a channel" to test, a covariate went down the event path — Step 3 wrote a timestamps-only artifact that dropped the scored values, and Step 4 computed a PSTH around its sample times. This PR adds is_covariate_label and is_event_label (neither a channel nor a covariate), and moves the sites that mean "skip anything that is not an event" — artifact removal, session combining, PSTH computation and group averaging, the visualization event list — onto it. That switch is the fix, so it stays visible here.

Verification: 30 new unit tests plus tests/integration/test_covariate_correlations.py (10 tests), including regression guards that no covariate_*_<site>.hdf5 is written and that the correlations table has no p-value column. Correlation literals were cross-checked against scipy.stats.pearsonr/spearmanr. test_integration_step3/4/5 and test_binned_metrics pass unchanged. pre-commit clean.

Follow-ups: session-permutation inference (with the explanation page it needs) and format relaxations from beta feedback — the sampling_rate column is required by the format but read by nothing here.

🤖 Generated with Claude Code

pauladkisson added a commit that referenced this pull request Aug 11, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.20225% with 42 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ppy/testing/scripts/create_stubbed_testing_data.py 31.57% 39 Missing ⚠️
src/guppy/testing/covariate_session.py 90.62% 1 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

@pauladkisson
pauladkisson marked this pull request as draft August 11, 2026 21:28
pauladkisson added a commit that referenced this pull request Aug 13, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pauladkisson added a commit that referenced this pull request Aug 21, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pauladkisson added a commit that referenced this pull request Aug 21, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pauladkisson
pauladkisson marked this pull request as ready for review August 21, 2026 16:16
pauladkisson added a commit that referenced this pull request Aug 21, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pauladkisson added a commit that referenced this pull request Aug 21, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pauladkisson added a commit that referenced this pull request Aug 22, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pauladkisson and others added 5 commits August 23, 2026 15:40
Adds a "behavioral covariate" store type: a continuous variable measured
outside the rig is ingested as an ordinary GuPPy-CSV, labeled in Step 1,
averaged onto the bins Step 4 already produces, and correlated against
every per-bin metric. Pearson r and Spearman rho are reported as
descriptive statistics with no p-value, since both series are
autocorrelated across bins.

Consolidates the ten copies of the "control"/"signal" substring test into
is_channel_label / is_covariate_label / is_event_label. Without a single
predicate a covariate went down the event path, which dropped its scored
values and computed a PSTH around its sample times.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging #427 left two spellings of "should this store be treated as an
event" in the tree: is_event_label at one site and is_channel_label at
eight others. Step 4 crashed as a result — execute_compute_psth skipped
the covariate while execute_compute_psth_peak_and_area did not, then
tried to read the PSTH file that was never written.

Replaces is_event_label with is_continuous_label, the union of channels
and covariates, which is what every one of those sites is really asking.
Call sites read positively, matching the shape #427 settled on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pauladkisson
pauladkisson merged commit 43a31ce into main Aug 24, 2026
12 checks passed
@pauladkisson
pauladkisson deleted the covariates branch August 24, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingest a continuous behavioral variable and correlate it with binned photometry

1 participant