Bug 25089

Summary: Add checkout_type to circulation rules
Product: Koha Reporter: Lari Taskula <lari.taskula>
Component: CirculationAssignee: Lari Taskula <lari.taskula>
Status: BLOCKED --- QA Contact: Testopia <testopia>
Severity: new feature    
Priority: P5 - low CC: gmcharlt, kyle.m.hall, lari.taskula
Version: unspecified   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26814
Change sponsored?: Sponsored Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on: 25037, 25113, 25114    
Bug Blocks:    
Attachments: Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Change sorting order to branchcode, checkout_type, categorycode, itemtype
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Change sorting order to branchcode, checkout_type, categorycode, itemtype
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Change sorting order to branchcode, checkout_type, categorycode, itemtype
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: Change sorting order to branchcode, checkout_type, categorycode, itemtype
Bug 25089: Add a new checkout_type to circulation_rules table
Bug 25089: Schema changes
Bug 25089: Add checkout_type to Koha::CirculationRules
Bug 25089: Remove on-site specific circulation rules
Bug 25089: Sort circulation rules
Bug 25089: Add checkout_type to set_rule
Bug 25089: Add checkout_type to set_rules
Bug 25089: Add checkout_type to get_effective_rule
Bug 25089: Add checkout_type to get_effective_rules
Bug 25089: (OLD GUI) Add checkout_type to smart-rules
Bug 25089: (OLD GUI) Indent and add checkout type to all_rules hash

Description Lari Taskula 2020-04-08 21:02:59 UTC
This is a follow-up to Bug 25037.

In that Bug we enabled issues to contain data on its checkout type.

In this Bug we will add support for defining circulation rules for those types.
Comment 1 Lari Taskula 2020-04-15 23:50:04 UTC Comment hidden (obsolete)
Comment 2 Lari Taskula 2020-04-15 23:50:07 UTC Comment hidden (obsolete)
Comment 3 Lari Taskula 2020-04-15 23:50:10 UTC Comment hidden (obsolete)
Comment 4 Lari Taskula 2020-04-15 23:50:13 UTC Comment hidden (obsolete)
Comment 5 Lari Taskula 2020-04-15 23:50:16 UTC Comment hidden (obsolete)
Comment 6 Lari Taskula 2020-04-15 23:50:18 UTC Comment hidden (obsolete)
Comment 7 Lari Taskula 2020-04-15 23:50:22 UTC Comment hidden (obsolete)
Comment 8 Lari Taskula 2020-04-15 23:50:25 UTC Comment hidden (obsolete)
Comment 9 Lari Taskula 2020-04-15 23:50:28 UTC Comment hidden (obsolete)
Comment 10 Lari Taskula 2020-04-20 09:28:02 UTC Comment hidden (obsolete)
Comment 11 Lari Taskula 2020-04-20 09:28:05 UTC Comment hidden (obsolete)
Comment 12 Lari Taskula 2020-04-20 09:28:09 UTC Comment hidden (obsolete)
Comment 13 Lari Taskula 2020-04-20 09:28:13 UTC Comment hidden (obsolete)
Comment 14 Lari Taskula 2020-04-20 09:28:16 UTC Comment hidden (obsolete)
Comment 15 Lari Taskula 2020-04-20 09:28:19 UTC Comment hidden (obsolete)
Comment 16 Lari Taskula 2020-04-20 09:28:22 UTC Comment hidden (obsolete)
Comment 17 Lari Taskula 2020-04-20 09:28:26 UTC Comment hidden (obsolete)
Comment 18 Lari Taskula 2020-04-20 09:28:29 UTC Comment hidden (obsolete)
Comment 19 Lari Taskula 2020-04-20 09:28:32 UTC Comment hidden (obsolete)
Comment 20 Lari Taskula 2020-04-20 12:07:07 UTC Comment hidden (obsolete)
Comment 21 Lari Taskula 2020-04-20 12:07:11 UTC Comment hidden (obsolete)
Comment 22 Lari Taskula 2020-04-20 12:07:15 UTC Comment hidden (obsolete)
Comment 23 Lari Taskula 2020-04-20 12:07:19 UTC Comment hidden (obsolete)
Comment 24 Lari Taskula 2020-04-20 12:07:22 UTC Comment hidden (obsolete)
Comment 25 Lari Taskula 2020-04-20 12:07:26 UTC Comment hidden (obsolete)
Comment 26 Lari Taskula 2020-04-20 12:07:29 UTC Comment hidden (obsolete)
Comment 27 Lari Taskula 2020-04-20 12:07:33 UTC Comment hidden (obsolete)
Comment 28 Lari Taskula 2020-04-20 12:07:38 UTC Comment hidden (obsolete)
Comment 29 Lari Taskula 2020-04-20 12:07:42 UTC Comment hidden (obsolete)
Comment 30 Lari Taskula 2020-04-21 10:27:48 UTC Comment hidden (obsolete)
Comment 31 Lari Taskula 2020-04-21 10:27:51 UTC Comment hidden (obsolete)
Comment 32 Lari Taskula 2020-04-21 10:27:55 UTC Comment hidden (obsolete)
Comment 33 Lari Taskula 2020-04-21 10:27:58 UTC Comment hidden (obsolete)
Comment 34 Lari Taskula 2020-04-21 10:28:02 UTC Comment hidden (obsolete)
Comment 35 Lari Taskula 2020-04-21 10:28:05 UTC Comment hidden (obsolete)
Comment 36 Lari Taskula 2020-04-21 10:28:09 UTC Comment hidden (obsolete)
Comment 37 Lari Taskula 2020-04-21 10:28:12 UTC Comment hidden (obsolete)
Comment 38 Lari Taskula 2020-04-21 10:28:16 UTC Comment hidden (obsolete)
Comment 39 Lari Taskula 2020-04-21 10:28:20 UTC Comment hidden (obsolete)
Comment 40 Lari Taskula 2020-04-21 10:28:23 UTC Comment hidden (obsolete)
Comment 41 Lari Taskula 2020-04-21 10:51:29 UTC Comment hidden (obsolete)
Comment 42 Lari Taskula 2020-04-21 10:51:32 UTC Comment hidden (obsolete)
Comment 43 Lari Taskula 2020-04-21 10:51:36 UTC Comment hidden (obsolete)
Comment 44 Lari Taskula 2020-04-21 10:51:40 UTC Comment hidden (obsolete)
Comment 45 Lari Taskula 2020-04-21 10:51:43 UTC Comment hidden (obsolete)
Comment 46 Lari Taskula 2020-04-21 10:51:47 UTC Comment hidden (obsolete)
Comment 47 Lari Taskula 2020-04-21 10:51:52 UTC Comment hidden (obsolete)
Comment 48 Lari Taskula 2020-04-21 10:51:57 UTC Comment hidden (obsolete)
Comment 49 Lari Taskula 2020-04-21 10:52:03 UTC Comment hidden (obsolete)
Comment 50 Lari Taskula 2020-04-21 10:52:08 UTC Comment hidden (obsolete)
Comment 51 Lari Taskula 2020-04-21 10:52:12 UTC Comment hidden (obsolete)
Comment 52 Lari Taskula 2020-04-23 06:41:53 UTC Comment hidden (obsolete)
Comment 53 Lari Taskula 2020-04-23 06:41:57 UTC Comment hidden (obsolete)
Comment 54 Lari Taskula 2020-04-23 06:42:00 UTC Comment hidden (obsolete)
Comment 55 Lari Taskula 2020-04-23 06:42:04 UTC Comment hidden (obsolete)
Comment 56 Lari Taskula 2020-04-23 06:42:08 UTC Comment hidden (obsolete)
Comment 57 Lari Taskula 2020-04-23 06:42:11 UTC Comment hidden (obsolete)
Comment 58 Lari Taskula 2020-04-23 06:42:15 UTC Comment hidden (obsolete)
Comment 59 Lari Taskula 2020-04-23 06:42:19 UTC Comment hidden (obsolete)
Comment 60 Lari Taskula 2020-04-23 06:42:22 UTC Comment hidden (obsolete)
Comment 61 Lari Taskula 2020-04-23 06:42:26 UTC Comment hidden (obsolete)
Comment 62 Lari Taskula 2020-04-23 06:42:30 UTC Comment hidden (obsolete)
Comment 63 Lari Taskula 2020-05-04 12:53:45 UTC Comment hidden (obsolete)
Comment 64 Lari Taskula 2020-05-04 12:53:53 UTC
Created attachment 104275 [details] [review]
Bug 25089: Change sorting order to branchcode, checkout_type, categorycode, itemtype

Otherwise we need to duplicate a rule with renewalsallowed=0 every rule
if we don't want on-site checkouts to be renewalable.

To test:
1. prove t/db_dependent/Koha/CirculationRules.t
Comment 65 Lari Taskula 2020-07-29 19:54:38 UTC
Created attachment 107523 [details] [review]
Bug 25089: Add a new checkout_type to circulation_rules table

To add:
1. perl installer/data/mysql/updatedatabase.pl

Sponsored-by: The National Library of Finland
Comment 66 Lari Taskula 2020-07-29 19:54:43 UTC
Created attachment 107524 [details] [review]
Bug 25089: Schema changes

Sponsored-by: The National Library of Finland
Comment 67 Lari Taskula 2020-07-29 19:54:47 UTC
Created attachment 107525 [details] [review]
Bug 25089: Add checkout_type to Koha::CirculationRules

To test:
1. prove t/db_dependent/Koha/CirculationRules.t

Sponsored-by: The National Library of Finland
Comment 68 Lari Taskula 2020-07-29 19:54:52 UTC
Created attachment 107526 [details] [review]
Bug 25089: Remove on-site specific circulation rules

To apply:
1. perl installer/data/mysql/updatedatabase.pl

To test:
1. Search for patron_maxonsiteissueqty rule

   $ grep -rn 'patron_maxonsiteissueqty'

2. Observe no results in Koha source files
3. Search for maxonsiteissueqty rule

   $ grep -rn 'maxonsiteissueqty'

4. Observe no results (apart from .git)
5. Check modification made to C4::Circulation::GetBranchBorrowerCircRule().
   Make sure the new parameter is being used everywhere. You can search
   the usage of this subroutine with similar grep commands as above.

6. Run the following tests:

prove t/db_dependent/Circulation/Branch.t
prove t/db_dependent/Circulation/SwitchOnSiteCheckouts.t
prove t/db_dependent/Circulation/TooMany.t

Sponsored-by: The National Library of Finland
Comment 69 Lari Taskula 2020-07-29 19:54:55 UTC
Created attachment 107527 [details] [review]
Bug 25089: Sort circulation rules

Sponsored-by: The National Library of Finland
Comment 70 Lari Taskula 2020-07-29 19:54:59 UTC
Created attachment 107528 [details] [review]
Bug 25089: Add checkout_type to set_rule

To test:
1. Find all occurrences of set_rule that are missing checkout_type
   where required

grep --exclude-dir '.git' --exclude-dir 'misc/translator' \
--exclude-dir 'koha-tmpl' -Przo '(?s)(::|->)set_rule(?!s).*?\)' | \
grep -avz 'checkout_type' | grep -Pavz 'hold|reserves|article_requests' \
&& echo ""

   The only occasions this should return anything are cases where
   a HASH or HASHref is given to set_rule(), or that we are
   explicitly testing missing parameters in an unit test.

   1.2 Verify the HASH/HASHref cases. The hash should contain
             a checkout_type (unless the rule is related to holds)

2. Find all subroutines using set_rule()

git grep --no-index -n -p -P 'set_rule\s*\(' | grep -v 'sub {' \
| grep -P 'sub .*'

   2.1 The only return should be
       Koha/CirculationRules.pm=326=sub set_rules {

3. The boring step. Find all occurrences of set_rule() with
   the following command:

grep --exclude-dir='.git' -Prn 'set_rule' | grep -v set_rules

   Go through this list and make sure all neccessary locations are
   updated. A change is required when the rule scope includes
   checkout_type. See Koha/CirculationRules.pm for scopes.
   Hold/reserve related rules do not require it.

Sponsored-by: The National Library of Finland
Comment 71 Lari Taskula 2020-07-29 19:55:03 UTC
Created attachment 107529 [details] [review]
Bug 25089: Add checkout_type to set_rules

To test:
1. Find all occurrences of set_rules that are missing checkout_type
   where required

grep --exclude-dir '.git' --exclude-dir 'misc/translator' \
--exclude-dir 'koha-tmpl' -Przo '(?s)(::|->)set_rules.*?\)' | \
grep -avz 'checkout_type' && echo ""

   The only occasions this should return anything are cases where
   a HASH or HASHref is given to set_rules(), or that we are
   explicitly testing missing parameters in an unit test.

   1.2 Verify the HASH/HASHref cases. The hash should contain
       a checkout_type (unless the rule is related to holds)

2. Find all subroutines using set_rules()

git grep --no-index -n -p -P 'set_rules\s*\(' | grep -v 'sub {' \
| grep -P 'sub .*'

   2.1 The only return should be
       t/db_dependent/Holds/DisallowHoldIfItemsAvailable.t=293=
         sub set_holdallowed_rule {

3. The boring step. Find all occurrences of set_rules() with
   the following command:

grep --exclude-dir='.git' -Prn 'set_rules'

    Go through this list and make sure all neccessary locations are
    updated. A change is required when the rule scope includes
    checkout_type. See Koha/CirculationRules.pm for scopes.
    Hold/reserve related rules do not require it.

Sponsored-by: The National Library of Finland
Comment 72 Lari Taskula 2020-07-29 19:55:08 UTC
Created attachment 107530 [details] [review]
Bug 25089: Add checkout_type to get_effective_rule

To test:
1. Find all occurrences of get_effective_rule that are missing
   checkout_type where required

grep --exclude-dir '.git' --exclude-dir 'misc/translator' \
--exclude-dir 'koha-tmpl' \
-Przo '(?s)(::|->)get_effective_rule(?!s).*?\)' | \
grep -avz 'checkout_type' | \
grep -Pavz 'hold|reserves|article_requests' && echo ""

   The only occasions this should return anything are cases where
   a HASH or HASHref is given to get_effective_rule(), or that we
   are explicitly testing missing parameters in an unit test.

   1.2 Verify the HASH/HASHref cases. The hash should contain
       a checkout_type (unless the rule is related to holds)

   1.3 Verify the other cases. If you find another result than
       what is mentioned above, then this test fails.

2. Find all subroutines using get_effective_rule()

git grep --no-index -n -p -P 'get_effective_rule\s*\(' \
| grep -v 'sub {' | grep -P 'sub .*'

    This list should be returned:

    C4/Circulation.pm=376=sub TooMany {
    C4/Circulation.pm=1300=sub AddIssue {
    C4/Circulation.pm=1633=sub GetBranchBorrowerCircRule {
    C4/Circulation.pm=1689=sub GetBranchItemRule {
    C4/Circulation.pm=3030=sub GetRenewCount {
    C4/Reserves.pm=348=sub CanItemBeReserved {
    C4/Reserves.pm=2230=sub GetHoldRule {
    Koha/Biblio.pm=294=sub article_request_type_for_bib {
    Koha/Charges/Fees.pm=98=sub accumulate_rentalcharge {
    Koha/CirculationRules.pm=218=sub get_effective_rules {
    Koha/CirculationRules.pm=390=sub get_opacitemholds_policy {
    Koha/CirculationRules.pm=416=sub get_onshelfholds_policy {
    Koha/Item.pm=571=sub article_request_type {
    Koha/REST/V1/Checkouts.pm=192=sub allows_renewal {
    Koha/Template/Plugin/CirculationRules.pm=26=sub Get {

    Check these subroutines and make sure they are using
    get_effective_rule() with the new scope, unless not needed.
    When fetching a hold-related rules, checkout_type is not needed.

    These subroutines and classes need changes to sub params:

    2.1. Check modifications to Koha::Charges::Fees
    2.2. Check modification made to C4::Circulation::_debar_user_on_return().
    2.3. Check modification made to C4::Circulation::_calculate_new_debar_dt().

3. Find all occurrences of get_effective_rule with the following command:
      grep --exclude-dir='.git' -Prn 'get_effective_rule(?!s)'

    Make sure all matches (where rule scope allows checkout_type) are
    updated.

Sponsored-by: The National Library of Finland
Comment 73 Lari Taskula 2020-07-29 19:55:12 UTC
Created attachment 107531 [details] [review]
Bug 25089: Add checkout_type to get_effective_rules

To test:
1. Find all occurrences of get_effective_rules that are missing
   checkout_type where required

grep --exclude-dir '.git' --exclude-dir 'misc/translator' \
--exclude-dir 'koha-tmpl' \
-Przo '(?s)(::|->)get_effective_rules.*?\)' | \
grep -avz 'checkout_type' && echo ""

   1.1. Verify command output. Only reserve related calls to
        get_effective_rules() should be returned.

2. Find all subroutines using get_effective_rules()

git grep --no-index -n -p -P 'get_effective_rules\s*\(' \
| grep -v 'sub {' | grep -P 'sub .*'

    This list should be returned:

git grep --no-index -n -p 'get_effective_rules' | grep -P 'sub .*'

3. Check modification made to C4::Circulation::CalcDateDue()
4. Check modification made to C4::Circulation::CalcFine()
5. Check modification made to C4::Circulation::GetLoanLength()
6. Check modification made to C4::Circulation::GetHardDueDate()

    Make sure all matches (where rule scope allows checkout_type) are
    updated.

7. Run these unit tests:

grep -Prl --color=never 'get_effective_rule\(|set_rule\(|set_rules\(' \
t/db_dependent/ | grep '^.*\.t$' | xargs prove

8. Test onboarding tool
8.1. Drop&create your database (take backups first if needed)
8.2. Navigate to web installer
8.3. Continue until you reach onboarding tool step where circulation
     rules are added.
8.4. Observe new drop down selection "Checkout type".
8.5. Select any checkout type and continue the onboarding process.
8.6. Investigate your database table "circulation_rules". You should
     now have rules with checkout_type of your choice.

Sponsored-by: The National Library of Finland
Comment 74 Lari Taskula 2020-07-29 19:55:18 UTC
Created attachment 107532 [details] [review]
Bug 25089: (OLD GUI) Add checkout_type to smart-rules

To test:
1. First apply all the patches tagged with (OLD GUI) prefix in commit title
2. Go to Circulation rules page in Koha
3. Observe checkout type column in the rule matrix
4. Test circulation rules matrix by setting, modifying and deleting
   rules.
5. Test default checkout, hold and return policy settings.
6. Test default checkout, hold policy by patron category settings.

Sponsored-by: The National Library of Finland
Comment 75 Lari Taskula 2020-07-29 19:55:22 UTC
Created attachment 107533 [details] [review]
Bug 25089: (OLD GUI) Indent and add checkout type to all_rules hash

No other changes in this patch.

To test:
1. git diff --word-diff=color --ignore-all-space HEAD~1 -- koha-tmpl/intranet-tmpl/prog/en/modules/admin/smart-rules.tt
2. Observe only $ct has been added

Sponsored-by: The National Library of Finland
Comment 76 Lari Taskula 2020-07-29 19:59:39 UTC
Rebased
Comment 77 Lari Taskula 2020-09-09 13:33:39 UTC
Marked as blocked since dependent Bug 25037 does not apply.

I'm also considering of dropping it from dependencies.
Comment 78 Lari Taskula 2020-10-24 02:39:45 UTC
Taking a different approach with less code dependencies - moving these patches to Bug 26814.