Skip to content

Proper CI test runner for Linux and MacOS #75

Description

@HendrikHuebner

The CI currenly does not actually run any tests and just attempts to compile everything on Linux. I propose the following improvements:

  1. Actually run the tests on every pr-push (duh)
  • currently we're only doing ./configure ...; make; make install
  • instead, run the full test suite, report failures, number of "dashed hopes" and code coverage
  1. Validate test suite against Apple's CoreFoundation
  • Add a workflow which runs in a macos container (Triggered manually)
  • A script discovers all the tests. Test files can be marked to skip them.
  • Compile and run each test against Apple Foundation/CoreFoundation.

I ran the tests myself in a Linux docker container and noticed a couple of failures. Some of them are fixed by @DTW-Thalion's PRs, others will still need adressing. Most tests compile for MacOS (Some needed minor fixes, like missing headers). Many fail (especially things related to string parsing / formatting), but we can make most work without too much effort and guard problematic assertion with #if defined(__APPLE__).

For now I would recommend accepting more of the bug fixes, such as #57. I'll submit a set of scripts / improved workflows and workarounds to make the CI green. Once that is in place, we can continue adding tests with confidence, that we are actually testing the right behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions