Banish the ghost of codecov

[kcaisley]

I looked into these randomly appearing Codecov bot messages, and it turns out the integration has actually been active in this repository since 2020 (it was added in this commit). It has occasionally worked since then, for example, it commented on PR #251, but most pull requests never receive a report.

The integration is configured in .github/workflows/regression-tests.yml. Because we do not have a CODECOV_TOKEN configured, it uses Codecov's legacy tokenless upload method. According to the docs: "This method shares a global rate limit with other users" and is documented as unreliable, which explains why the integration only works intermittently.

So yeah... rather than re-setting up and maintaining another external account and access token for an unreliable service, this replaces Codecov with a simple coverage report produced directly by GitHub Actions.

Thoughts?

[kcaisley]

Oh also, for this to work, we may need to adjust the repo or organization settings to allow github's coverage action to run: see here

[cbespin]

Enabling code quality comes with a 10€ per month additional cost. I think there are other ways to get coverage report in the repo (but maybe not on each PR). Nevertheless, I will merge this to get rid of codecov first and we can see if we find a nice way how to do it.

[cbespin]

Approved in Draft state, sorry. Let me know if and when it is ready.

[kcaisley]

Okay, so I did a bit of searching, and it looks like the already installed coverage python package we have is already producing for us a coverage.xml file. On the cli that would already produce for us an output like this:

$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10    87%

From here, if we want to see the change in coverage, relative to another commit or branch, it seems the platform/language agnostic diff-cover package can simply be run:

diff-cover coverage.xml --compare-branch=origin/master

That's a bit wordy to run though, so to see it easily locally I've added a pre-commit hook, so you can simply do pre-commit run coverage to see if your uncommitted changes have reduced coverage.

So, for example: If a new function was added in example.py but doesn't have tests, it would show this:

Coverage................................................................Passed
- hook id: coverage
- duration: 178.2s

........................................................................ [100%]
104 passed, 6 skipped, 105 warnings in 177.5s

-------------
Diff Coverage
Diff: origin/master...HEAD, staged and unstaged changes
-------------
basil/HL/example.py (25.0%): Missing lines 84-86
-------------
Total: 25%

Here 25% means is the percentage of lines of code which are covered by tests.

And to see it in github PRs, I've added an action which displays it as a non-blocking check. This last part I'm not exactly sure if I've done right, but we'll know when we try to merge if it looks right.

[kcaisley]

Ah, btw the best part is, this is entirely platform portable, so it would be able to be trigged by any CI, since it's just based on coverage + diff-cover + pre-commit.

[kcaisley]

You can see here it creates this 'neutral check'