Repository navigation
Detect all .test.hs or .multitest.hs test scripts #12506
Description
I expected tests in cabal-testsuite that end with .test.hs or .multitest.hs to be found but have recently discovered that tests I added and that passed individually were missed by the test detection in CI because I used . to punctuate the name rather than - like other test file names. The tests suceeded locally and CI passed but my tests weren't run.
Could we detect those tests too?
Additional Context
All test script names with counts:
$ find . -type f \( -name '*.test.hs' -o -name '*.multitest.hs' \) \
-not -path '*/dist-newstyle/*' -printf '%f\n' | sort | uniq -c | sort -rn
411 cabal.test.hs
147 setup.test.hs
4 other-arch.test.hs
4 js-arch.test.hs
3 new_freeze.test.hs
3 cabal.no-project.v2.test.hs
3 cabal.no-project.test.hs
3 cabal.ignore-project.v2.test.hs
3 cabal.ignore-project.test.hs
2 setup-per-component.test.hs
2 setup-internal.test.hs
2 setup-external-fail.test.hs
2 coverage.test.hs
2 cabal.sub-rs.v2.test.hs
2 cabal.sub-rs.test.hs
2 cabal.sub-pq.v2.test.hs
2 cabal.sub-pq.test.hs
2 cabal.project.test.hs
2 cabal.dot-uv.v2.test.hs
2 cabal.dot-uv.test.hs
2 cabal-internal.test.hs
2 cabal-external.test.hs
1 valid-and-test-suite.test.hs
1 use-local-version-of-package.test.hs
1 use-local-package-as-setup-dep.test.hs
1 use-different-versions-of-dependency-for-library-and-build-tool.test.hs
1 update-index-state.test.hs
1 target-remote-package.test.hs
1 single.test.hs
1 single-2.test.hs
1 setup-static.test.hs
1 setup-reexport.test.hs
1 setup-package-import.test.hs
1 setup-old-ar-without-at-args.test.hs
1 setup-no-tix.test.hs
1 setup-no-markup.test.hs
1 setup-internal-fail.test.hs
1 setup-gen-script.test.hs
1 setup-gen-pkg-config.test.hs
1 setup-fail.test.hs
1 setup-fail-other.test.hs
1 setup-fail-missing.test.hs
1 setup-fail-ambiguous.test.hs
1 setup-external.test.hs
1 setup-external-ok.test.hs
1 setup-external-explicit.test.hs
1 setup-explicit.test.hs
1 setup-explicit-fail.test.hs
1 setup-default-ar.test.hs
1 setup-deadlock.test.hs
1 sdist-list-sources.test.hs
1 remove-outdated.test.hs
1 quickjump.test.hs
1 overwrite-policy.test.hs
1 outdated_freeze.test.hs
1 outdated.test.hs
1 outdated-project-file.test.hs
1 ok.test.hs
1 offlineFlag.test.hs
1 multi-target.test.hs
1 multi-list-sources.test.hs
1 multi-archive-to-stdout.test.hs
1 many-data-files.test.hs
1 list-sources.test.hs
1 list-sources-output-dir.test.hs
1 install.test.hs
1 install-time-with-constraint.test.hs
1 init.test.hs
1 init-without-git.test.hs
1 init-legacy.test.hs
1 init-interactive.test.hs
1 init-interactive-legacy.test.hs
1 init-interactive-ghc2021.test.hs
1 init-interactive-empty-folder.test.hs
1 init-backup.test.hs
1 hoogle.test.hs
1 help-prints-config-file.test.hs
1 haddock.test.hs
1 haddock-project.test.hs
1 freeze.test.hs
1 fail.test.hs
1 enable-tests.test.hs
1 enable-benchmarks.test.hs
1 dry-run.test.hs
1 disable-tests.test.hs
1 disable-benchmarks.test.hs
1 dirty-install.test.hs
1 deterministic.test.hs
1 custom.test.hs
1 cov.test.hs
1 concurrent-store-init.test.hs
1 compile-fail.test.hs
1 clean-install-by-symlink.test.hs
1 clean-install-by-copy.test.hs
1 check.test.hs
1 cabal.project.v2.test.hs
1 cabal-with-hpc.multitest.hs
1 cabal-repo.test.hs
1 cabal-per-package.test.hs
1 cabal-packagedb.test.hs
1 cabal-old-build.test.hs
1 cabal-manual-packagedb.test.hs
1 cabal-list.test.hs
1 cabal-internal-target.test.hs
1 cabal-fail-no-packagedbs.test.hs
1 cabal-fail-no-p.test.hs
1 cabal-fail-no-base.test.hs
1 cabal-external-target.test.hs
1 build-package-from-repo-with-custom-setup.test.hs
1 build-local-package-with-custom-setup.test.hs
1 build-info-exe.test.hs
1 build-info-all.test.hs
1 all.test.hs
1 all-test-suite.test.hs
1 all-output-dir.test.hs
1 3-web.test.hs
1 2-web.test.hs
1 2-local.test.hs
1 1-web.test.hs
1 1-local.test.hs
1 0-local.test.hs
The missed tests:
$ find . -type f \( -name '*.*.test.hs' -o -name '*.*.multitest.hs' \) \
-not -path '*/dist-newstyle/*' -printf '%f\n' | sort | uniq -c | sort -rn
3 cabal.no-project.v2.test.hs
3 cabal.no-project.test.hs
3 cabal.ignore-project.v2.test.hs
3 cabal.ignore-project.test.hs
2 cabal.sub-rs.v2.test.hs
2 cabal.sub-rs.test.hs
2 cabal.sub-pq.v2.test.hs
2 cabal.sub-pq.test.hs
2 cabal.project.test.hs
2 cabal.dot-uv.v2.test.hs
2 cabal.dot-uv.test.hs
1 cabal.project.v2.test.hs
Activity