[EngCaioFonseca]

Description

Adds test coverage measurement, so we can see which parts of the code the tests exercise and which still need tests.

• Adds pytest-cov to the test dependency group and a [tool.coverage.*] configuration in pyproject.toml. It measures line and branch coverage of the collectoss and keyman packages.
• Coverage is opt-in locally: uv run pytest --cov. It is not in pytest's addopts, so a plain uv run pytest is as fast as before.
• The "Functional tests" workflow now measures coverage on every pull request. One matrix entry (ubuntu, Python 3.11) writes the per-file table to the job summary and uploads coverage.xml and the HTML report as a coverage-report artifact.
• Documents this in the testing page of the development guide.

Codecov now installed for the repo, and implemented on this branch.

This PR fixes #564

Notes for Reviewers

Current baseline with the tests on main (117 tests): 13% combined line and branch coverage.

Package	Statements	Line coverage	Branch coverage
collectoss/api	2,481	0%	0%
collectoss/application	5,518	41%	11%
collectoss/tasks	7,996	3%	1%
collectoss/util	316	3%	6%
keyman	296	10%	0%

Choices worth a second opinion:

• No minimum coverage is enforced. At this baseline a threshold would only add noise. It can be added later, for example as a check on changed lines only.
• Reporting uses only GitHub features (job summary and artifact), so it works for pull requests from forks with the workflow's existing contents: read permission and needs no secrets.

Tested locally: uv run pytest, uv run pytest --cov with the XML, HTML and markdown reports, uv lock --check, and the docs build with -W. The new workflow steps themselves first run on this PR.

Signed commits

• Yes, I signed my commits.

Generative AI disclosure

• This contribution was NOT assisted or created by Generative AI tools.
• This contribution was assisted or created by Generative AI tools.

If AI tools were used, please provide details below:
- What tools were used? Claude Fable 5.2
- How were these tools used? Design, implementation, local verification
- Did you review these outputs before submitting this PR? Yes.

[EngCaioFonseca]

@CodeRabbit review

[coderabbitai]

Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

• New Features
  • Functional test runs now produce terminal, XML, and HTML coverage reports. CI displays a Markdown coverage summary and stores available reports as an artifact.
• Documentation
  • Added guidance on measuring test coverage, generating reports, and finding CI results.
• Chores
  • Added coverage tooling and configured which project code is measured and how results are reported.

The changes add opt-in line and branch coverage configuration, local coverage instructions, and coverage reports in the functional test workflow.

Changes: Test Coverage Reporting

Layer / File(s)	Summary
Coverage configuration and usage pyproject.toml, docs/source/development-guide/testing/toc.rst	The test dependency group adds pytest-cov. Coverage settings track collectoss and keyman with branch coverage. The testing guide documents local report commands and where CI coverage results are available.
CI coverage generation and publication .github/workflows/functional_test.yml	The workflow generates terminal, XML, and HTML coverage reports. On Ubuntu with Python 3.11, it adds a Markdown report to the job summary and uploads the XML and HTML reports. It warns if report files are missing.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant FunctionalTestWorkflow
  participant pytest
  participant JobSummary
  participant CoverageArtifact
  FunctionalTestWorkflow->>pytest: Run tests and generate coverage reports
  FunctionalTestWorkflow->>JobSummary: Add Markdown coverage report
  FunctionalTestWorkflow->>CoverageArtifact: Upload coverage.xml and htmlcov/

Loading

Possibly related issues

• Issue 538: The coverage reports show which code the tests cover, supporting the issue’s goal of increasing meaningful unit-test coverage.

Merge Risk: Minimal · up to 404af

Coverage is opt-in locally and reported in CI. No merge-blocking risk is identified.

Security architecture risk: Minimal · up to 404af

The new reporting stays within the existing pull-request test workflow. It does not request broader repository permissions or introduce deployment credentials, and no material security risk was identified in the reporting change.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — Pull-request-controlled code can influence the new coverage reports, so those outputs are untrusted. The introduced publication destinations are the workflow summary and downloadable coverage report; the inspected changes introduce no downstream deployment or privileged consumer of those outputs.

Trust Boundaries and Controls

• observed — The workflow retains its existing pull_request trigger, GitHub-hosted runner matrix, and contents: read permission. The PR adds report publication but no explicit secret references, repository-write permissions, or deployment credentials. Pull-request test-code execution predates this PR.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly summarizes the main change: measuring test coverage in CI.
Linked Issues check		Issue #564 requests line and branch coverage data alongside test results. The PR adds pytest-cov, configures coverage for collectoss and keyman, and reports results in the Functional tests job s…
Out of Scope Changes check		The dependency, coverage configuration, CI reporting, and testing-guide updates all support issue #564. The possible coverage-service follow-up is only a comment; the PR does not add an unrelated serv…
Description check		The description includes a change summary, linked issue, reviewer notes, signed-commit confirmation, and completed Generative AI disclosure. It also reports testing. The statement that Codecov was ins…

Full details: Linked Issues check

Explanation

Issue #564 requests line and branch coverage data alongside test results. The PR adds pytest-cov, configures coverage for collectoss and keyman, and reports results in the Functional tests job summary and artifact. The testing guide documents local use. These changes meet the coding objective.

Full details: Out of Scope Changes check

Explanation

The dependency, coverage configuration, CI reporting, and testing-guide updates all support issue #564. The possible coverage-service follow-up is only a comment; the PR does not add an unrelated service integration.

Full details: Description check

Explanation

The description includes a change summary, linked issue, reviewer notes, signed-commit confirmation, and completed Generative AI disclosure. It also reports testing. The statement that Codecov was installed conflicts with the later note that reporting uses only GitHub features; clarify this point.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: chaoss/CollectOSS
• Branch: main
• File: .coderabbit.yaml
• Commit: d52912e
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4106a6bb-cea9-4938-ac10-c4cb0a424108

Commits

Reviewing files that changed from the base of the PR and between d52912e and 404af7a.

Files ignored due to path filters: 1

• uv.lock is excluded by !**/*.lock

Files selected for processing: 3

• .github/workflows/functional_test.yml
• docs/source/development-guide/testing/toc.rst
• pyproject.toml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

[MoralCode]

Bunch of small things, overall looks good, just installed codecov for this repo

[EngCaioFonseca]

Bunch of small things, overall looks good, just installed codecov for this repo

All comments addressed. Also implemented codecov as it was installed for the repo.

[codecov]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

[MoralCode]

Am a little worried about codecov "display results in all future pull requests" - its probably going to spam the PR with comments if its working in conjunction with coderabbit

[EngCaioFonseca]

The coderabbit is set as manual trigger only. We might be able to do something similar with codecov, or we can see how it goes in the next PRs, if it's too verbose, or concise enough.

[EngCaioFonseca]

Quick look into their docs, and it seems we can do just that. Trigger it manually as well. Probably also set it to a "quiet" mode like in coderabbit. But personally, I'm interested in seeing its capabilities. Seems very interesting from their demo. Being able to generate unit tests on demand by triggering it depending on the PR is quite nice.

[MoralCode]

Being able to generate unit tests on demand by triggering it depending on the PR is quite nice.

WDYM? you mean generate coverage reports?

[EngCaioFonseca]

Being able to generate unit tests on demand by triggering it depending on the PR is quite nice.

WDYM? you mean generate coverage reports?

that too. But in their docs and website, you're also able to trigger unit tests on demand, and they generate it for the specific PR. So, I think that's quite a nice feature to test as well.