Bug 18238

Summary: Test harness/suite/"test runner"
Product: Koha Reporter: Olli-Antti Kivilahti <olli-antti.kivilahti>
Component: Test SuiteAssignee: Chris Cormack <chris>
Status: In Discussion --- QA Contact: Galen Charlton <gmcharlt>
Severity: enhancement    
Priority: P5 - low CC: dcook, mtj
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18055
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: KD-1738 - AnsbileTorpor - Poor man's Ansible Tower - testHarness.pl

Description Olli-Antti Kivilahti 2017-03-09 12:30:14 UTC
There is no smart interface for running different types of tests in Koha and covering code coverage metrics from those.

This bug introduces one way of running various test suites.

+my $usage = <<USAGE;
+
+Runs a ton of tests with other metrics if needed
+
+  -h --help             This friendly help!
+
+  -v --verbose          Integer, the level of verbosity
+
+  --tar                 Create a testResults.tar.gz from all tests and deliverables
+
+  --dry-run             Don't run tests or other metrics. Simply show what would happen.
+
+  --clover              Run Devel::Cover and output Clover-reports
+
+  -a --all              Run all tests.
+
+  -b --basic            Basic tests t/*.t
+
+  -x --xt               XT tests
+
+  -s --sip2             SIP2 tests
+
+  -d --db               db_dependent tests
+
+  --git                 Integer, look for this many git commits from HEAD and run
+                        all '.t'-files that they have changed.
+                        This is meaningful as a quick smoke test to verify that
+                        the latest changes haven't been broken or work as expected.
+                        Thus where the most probably reason for breakage occurs,
+                        is tested first, before executing more lengthy test suites.
+
+EXAMPLE
+
+    ##First run a smoke test for latest changes
+    ks-test-harness.pl --git 5 --tar
+    ##Then run a big test suite
+    ks-test-harness.pl --all --tar
Comment 1 Olli-Antti Kivilahti 2017-03-09 12:31:29 UTC
Created attachment 60945 [details] [review]
KD-1738 - AnsbileTorpor - Poor man's Ansible Tower - testHarness.pl

testHarness.pl for running Koha's tests in petite chunks
Comment 2 Olli-Antti Kivilahti 2017-03-09 12:34:15 UTC
I am using this script to run various kinds of tests in Koha.

Test execution starts from
    Jenkins2.0,
then passes to
    https://github.com/kivilahtio/AnsbileTorpor
which triggers a Koha build and test runs
and finally is executed via Ansible scripts
    ks-test-harness.pl --git 5
Comment 3 David Cook 2022-12-06 03:03:20 UTC
I reckon Tomas and Jonathan might have something to say on this topic... especially as there is talk about testing on the listserv at the moment...