Bug 21559 - Rules for automatic framework codes
Summary: Rules for automatic framework codes
Status: Failed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: master
Hardware: All All
: P5 - low new feature with 10 votes (vote)
Assignee: paxed
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-12 08:55 UTC by paxed
Modified: 2019-06-27 09:15 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 21559: Rules for automatic framework codes (16.35 KB, patch)
2018-10-12 09:01 UTC, paxed
Details | Diff | Splinter Review
Bug 21559: Rules for automatic framework codes (16.33 KB, patch)
2018-10-15 05:23 UTC, paxed
Details | Diff | Splinter Review
Bug 21559: Rules for automatic framework codes (16.92 KB, patch)
2018-10-15 08:44 UTC, paxed
Details | Diff | Splinter Review
Bug 21559: Rules for automatic framework codes (16.94 KB, patch)
2018-10-18 05:01 UTC, paxed
Details | Diff | Splinter Review
Bug 21559: Rules for automatic frameworkcodes (16.99 KB, patch)
2018-10-19 14:56 UTC, Cori Lynn Arnold
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description paxed 2018-10-12 08:55:55 UTC
Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.


Explanation of the rule format:
-------------------------------
    
The syspref is YAML, with a specific format:
    
- <field>
   <value>: <frameworkcode>
    
If the <field> in the record contains <value>, the <frameworkcode>
is used for the record.
    
<field> is one of:
"xxx" (field xxx in the MARC record)
"xxx/yy" (field xxx, character location yy)
"xxx/yy-zz" (field xxx, character locations yy-zz)
"xxx$v" (field xxx, subfield v)
    
Fields can be combined with plus-sign, then the field values have
plus-sign between them. The field-rules are checked in order from
top to bottom, and the first rule with a matching value is used.
If the (sub)field has been repeated in the MARC record, only the first
matching (sub)field is checked. If none of the rules match, the framework
code is not changed.
    
Example:
    
- 008/35-37 + 000/06:
   fin+a: BKSFIN
- 000/06:
   a: BKS
   c: MU
   e: MP

This would make record with 000/06 = "a" AND 008/35-37 = "fin" use the BKSFIN framework, and any other record with 000/06 = "a" use BKS, 000/06 = "c" would be MU, and 000/06 = "e" would give MP
Comment 1 paxed 2018-10-12 09:01:15 UTC
Created attachment 80462 [details] [review]
Bug 21559: Rules for automatic framework codes

Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.

Test plan:
----------
0) Apply patch, apply the atomicupdate, etc.
1) Go to cataloguing, and import a record without choosing a framework.
   note how the framework is set to "Default".
2) Bulkmarcimport some records. Check the framework codes for the
   imported records - they're all "default".
3) Set the following rules in MarcToFrameworkcodeAutoconvert syspref:
- 000/06:
   a: BKS
   c: SR
   m: CF
   This rule sets framework code to 'BKS' if 000/06 is 'a', etc.
4) Repeat 1 and 2, the frameworkcodes should be something else than
   default, if the record 000/06 was one of 'a', 'c', or 'm'.
5) Go to cataloguing, and import a record, but choose some specific
   framework - note how the framework you chose is kept, even if
   the record would match the syspref rules.
6) Repeat 5, but create a new record from scratch. Framework you chose
   should override the syspref rules.
7) Bulkmarcimport some record, but give the framework-parameter. The
   framework you chose should override the syspref rules.
8) prove t/Biblio/AutoFrameworkcode.t
Comment 2 paxed 2018-10-12 11:15:00 UTC
Whoops, a thinko in the patch, will fix on Monday...
Comment 3 paxed 2018-10-15 05:23:19 UTC
Created attachment 80586 [details] [review]
Bug 21559: Rules for automatic framework codes

Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.

Test plan:
----------
0) Apply patch, apply the atomicupdate, etc.
1) Go to cataloguing, and import a record without choosing a framework.
   note how the framework is set to "Default".
2) Bulkmarcimport some records. Check the framework codes for the
   imported records - they're all "default".
3) Set the following rules in MarcToFrameworkcodeAutoconvert syspref:
- 000/06:
   a: BKS
   c: SR
   m: CF
   This rule sets framework code to 'BKS' if 000/06 is 'a', etc.
4) Repeat 1 and 2, the frameworkcodes should be something else than
   default, if the record 000/06 was one of 'a', 'c', or 'm'.
5) Go to cataloguing, and import a record, but choose some specific
   framework - note how the framework you chose is kept, even if
   the record would match the syspref rules.
6) Repeat 5, but create a new record from scratch. Framework you chose
   should override the syspref rules.
7) Bulkmarcimport some record, but give the framework-parameter. The
   framework you chose should override the syspref rules.
8) prove t/Biblio/AutoFrameworkcode.t
Comment 4 paxed 2018-10-15 08:44:40 UTC
Created attachment 80587 [details] [review]
Bug 21559: Rules for automatic framework codes

Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.

Test plan:
----------
0) Apply patch, apply the atomicupdate, etc.
1) Go to cataloguing, and import a record without choosing a framework.
   note how the framework is set to "Default".
2) Bulkmarcimport some records. Check the framework codes for the
   imported records - they're all "default".
3) Set the following rules in MarcToFrameworkcodeAutoconvert syspref:
- 000/06:
   a: BKS
   c: SR
   m: CF
   This rule sets framework code to 'BKS' if 000/06 is 'a', etc.
4) Repeat 1 and 2, the frameworkcodes should be something else than
   default, if the record 000/06 was one of 'a', 'c', or 'm'.
5) Go to cataloguing, and import a record, but choose some specific
   framework - note how the framework you chose is kept, even if
   the record would match the syspref rules.
6) Repeat 5, but create a new record from scratch. Framework you chose
   should override the syspref rules.
7) Bulkmarcimport some record, but give the framework-parameter. The
   framework you chose should override the syspref rules.
8) prove t/Biblio/AutoFrameworkcode.t
Comment 5 Cori Lynn Arnold 2018-10-17 15:03:08 UTC
Failed to apply :(


Bug 21559 - Rules for automatic framework codes

80587 - Bug 21559: Rules for automatic framework codes

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 21559: Rules for automatic frameworkcodes
Using index info to reconstruct a base tree...
M       C4/Biblio.pm
M       cataloguing/addbiblio.pl
Falling back to patching base and 3-way merge...
Auto-merging cataloguing/addbiblio.pl
Auto-merging C4/Biblio.pm
CONFLICT (content): Merge conflict in C4/Biblio.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 21559: Rules for automatic frameworkcodes
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-21559-Rules-for-automatic-framework-codes-L_HoL6.patch
Comment 6 paxed 2018-10-18 05:01:29 UTC
Created attachment 80798 [details] [review]
Bug 21559: Rules for automatic framework codes

Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.

Test plan:
----------
0) Apply patch, apply the atomicupdate, etc.
1) Go to cataloguing, and import a record without choosing a framework.
   note how the framework is set to "Default".
2) Bulkmarcimport some records. Check the framework codes for the
   imported records - they're all "default".
3) Set the following rules in MarcToFrameworkcodeAutoconvert syspref:
- 000/06:
   a: BKS
   c: SR
   m: CF
   This rule sets framework code to 'BKS' if 000/06 is 'a', etc.
4) Repeat 1 and 2, the frameworkcodes should be something else than
   default, if the record 000/06 was one of 'a', 'c', or 'm'.
5) Go to cataloguing, and import a record, but choose some specific
   framework - note how the framework you chose is kept, even if
   the record would match the syspref rules.
6) Repeat 5, but create a new record from scratch. Framework you chose
   should override the syspref rules.
7) Bulkmarcimport some record, but give the framework-parameter. The
   framework you chose should override the syspref rules.
8) prove t/Biblio/AutoFrameworkcode.t
Comment 7 Cori Lynn Arnold 2018-10-19 14:56:28 UTC
Created attachment 80939 [details] [review]
Bug 21559: Rules for automatic frameworkcodes

Allow setting rules in a system preference, what framework code should
be automatically set for biblio, based on the MARC record fields.

Does not affect anything if the MarcToFrameworkcodeAutoconvert
system preference rules aren't used. The "Default" framework is
always overridden, if the rules match, but some other framework
chosen by you is always used.

Test plan:
----------
0) Apply patch, apply the atomicupdate, etc.
1) Go to cataloguing, and import a record without choosing a framework.
   note how the framework is set to "Default".
2) Bulkmarcimport some records. Check the framework codes for the
   imported records - they're all "default".
3) Set the following rules in MarcToFrameworkcodeAutoconvert syspref:
- 000/06:
   a: BKS
   c: SR
   m: CF
   This rule sets framework code to 'BKS' if 000/06 is 'a', etc.
4) Repeat 1 and 2, the frameworkcodes should be something else than
   default, if the record 000/06 was one of 'a', 'c', or 'm'.
5) Go to cataloguing, and import a record, but choose some specific
   framework - note how the framework you chose is kept, even if
   the record would match the syspref rules.
6) Repeat 5, but create a new record from scratch. Framework you chose
   should override the syspref rules.
7) Bulkmarcimport some record, but give the framework-parameter. The
   framework you chose should override the syspref rules.
8) prove t/Biblio/AutoFrameworkcode.t

Explanation of the rule format:
-------------------------------

The syspref is YAML, with a specific format:

- <field>
   <value>: <frameworkcode>

If the <field> in the record contains <value>, the <frameworkcode>
is used for the record.

<field> is one of:
"xxx" (field xxx in the MARC record)
"xxx/yy" (field xxx, character location yy)
"xxx/yy-zz" (field xxx, character locations yy-zz)
"xxx$v" (field xxx, subfield v)

Fields can be combined with plus-sign, then the field values have
plus-sign between them. The field-rules are checked in order from
top to bottom, and the first rule with a matching value is used.
If the (sub)field has been repeated in the MARC record, only the first
matching (sub)field is checked. If none of the rules match, the framework
code is not changed.

Example:

- 008/35-37 + 000/06:
   fin+a: BKSFIN
- 000/06:
   a: BKS
   c: MU
   e: MP

This would make record with 000/06 = "a" AND 008/35-37 = "fin"
use the BKSFIN framework, and any other record with 000/06 = "a" use
BKS, 000/06 = "c" would be MU, and 000/06 = "e" would give MP.

Signed-off-by: Pasi Kallinen <pasi.kallinen@joensuu.fi>
Signed-off-by: Cori Lynn Arnold <carnold@dgiinc.com>
Comment 8 Katrin Fischer 2019-01-27 23:03:31 UTC
Hi Paxed, some first QA notes:

1) There is a fail from the QA tools:

 FAIL	t/Biblio/AutoFrameworkcode.t
   FAIL	  critic
		"return" statement with explicit "undef" at line 47, column 9. See page 199 of PBP.

2) I think the system preference description would be more helpful with some a hint at documentation. I was wondering if a link to the manual would work, but maintaining the link to always link to the current manual would not be so nice. Maybe adding some more hints to the preference description for now?  

I am a bit concerned that this adds a lot of code in the "old style" to C4::Biblio. Specifically: GetAutoFrameworkCode,_matchRecordFieldspec. Not sure how much we have in place yet in Koha for dealing with records.
Comment 9 Kyle M Hall 2019-02-21 11:55:42 UTC
 
> I am a bit concerned that this adds a lot of code in the "old style" to
> C4::Biblio. Specifically: GetAutoFrameworkCode,_matchRecordFieldspec. Not
> sure how much we have in place yet in Koha for dealing with records.

Agreed. Those new subroutines can and should be moved to Koha::Biblio with minimal effort.
Comment 10 paxed 2019-06-20 10:33:54 UTC
Moving the _matchRecordFieldSpec and GetAutoFrameworkCode to Koha/Biblio.pm is the easy part. But then AddBiblio has to be rewritten. Unless there's a way to get a Koha::Biblio object without having the record in the database?
Comment 11 Tomás Cohen Arazi 2019-06-20 11:06:51 UTC
(In reply to paxed from comment #10)
> Moving the _matchRecordFieldSpec and GetAutoFrameworkCode to Koha/Biblio.pm
> is the easy part. But then AddBiblio has to be rewritten. Unless there's a
> way to get a Koha::Biblio object without having the record in the database?

Koha::Biblio->new({ attr1 => val1, ... });
Comment 12 paxed 2019-06-27 09:15:05 UTC
(In reply to Tomás Cohen Arazi from comment #11)
> (In reply to paxed from comment #10)
> > Moving the _matchRecordFieldSpec and GetAutoFrameworkCode to Koha/Biblio.pm
> > is the easy part. But then AddBiblio has to be rewritten. Unless there's a
> > way to get a Koha::Biblio object without having the record in the database?
> 
> Koha::Biblio->new({ attr1 => val1, ... });

How do I set the metadata->record using this way?