Bug 18238 - Test harness/suite/"test runner"
Summary: Test harness/suite/"test runner"
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Test Suite (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Chris Cormack
QA Contact: Galen Charlton
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-03-09 12:30 UTC by Olli-Antti Kivilahti
Modified: 2022-12-06 03:03 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
KD-1738 - AnsbileTorpor - Poor man's Ansible Tower - testHarness.pl (9.77 KB, patch)
2017-03-09 12:31 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Olli-Antti Kivilahti 2017-03-09 12:30:14 UTC
There is no smart interface for running different types of tests in Koha and covering code coverage metrics from those.

This bug introduces one way of running various test suites.

+my $usage = <<USAGE;
+
+Runs a ton of tests with other metrics if needed
+
+  -h --help             This friendly help!
+
+  -v --verbose          Integer, the level of verbosity
+
+  --tar                 Create a testResults.tar.gz from all tests and deliverables
+
+  --dry-run             Don't run tests or other metrics. Simply show what would happen.
+
+  --clover              Run Devel::Cover and output Clover-reports
+
+  -a --all              Run all tests.
+
+  -b --basic            Basic tests t/*.t
+
+  -x --xt               XT tests
+
+  -s --sip2             SIP2 tests
+
+  -d --db               db_dependent tests
+
+  --git                 Integer, look for this many git commits from HEAD and run
+                        all '.t'-files that they have changed.
+                        This is meaningful as a quick smoke test to verify that
+                        the latest changes haven't been broken or work as expected.
+                        Thus where the most probably reason for breakage occurs,
+                        is tested first, before executing more lengthy test suites.
+
+EXAMPLE
+
+    ##First run a smoke test for latest changes
+    ks-test-harness.pl --git 5 --tar
+    ##Then run a big test suite
+    ks-test-harness.pl --all --tar
Comment 1 Olli-Antti Kivilahti 2017-03-09 12:31:29 UTC
Created attachment 60945 [details] [review]
KD-1738 - AnsbileTorpor - Poor man's Ansible Tower - testHarness.pl

testHarness.pl for running Koha's tests in petite chunks
Comment 2 Olli-Antti Kivilahti 2017-03-09 12:34:15 UTC
I am using this script to run various kinds of tests in Koha.

Test execution starts from
    Jenkins2.0,
then passes to
    https://github.com/kivilahtio/AnsbileTorpor
which triggers a Koha build and test runs
and finally is executed via Ansible scripts
    ks-test-harness.pl --git 5
Comment 3 David Cook 2022-12-06 03:03:20 UTC
I reckon Tomas and Jonathan might have something to say on this topic... especially as there is talk about testing on the listserv at the moment...