In those cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $autfrom == $autfrom). Therefore the authtype guessed from the old version of the record should be considered. To test: 1) have an authority record used in biblio; export it to file; change 1XX heading tag to different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2) make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3) repeat the test from 1).
Created attachment 69344 [details] [review] Bug 19693: authtypefrom incorrectly set when updating auth from file
Created attachment 69349 [details] [review] Bug 19693: authtypefrom incorrectly set when updating auth from file A corrected version. It is more safe to use the GuessAuthTypeCode function only if really needed, i.e. when merging to itself (especially when using some non standard authority types that for some reason are not covered in GuessAuthTypeCode).
Created attachment 69350 [details] [review] Bug 19693: authtypefrom incorrectly set when updating auth from file
Not really trivial
t/db_dependent/Authority/Merge.t triggers this code change fully. However, there is no new test that fails before this patch, and succeeds after. Marking Failed QA.
Created attachment 138121 [details] [review] Bug 19693: update of an authority record creates inconsistency when the heading tag is changed Since after a few years the problem is still unresolved here comes a new attempt to address it: In those rare cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority: Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $mergefrom == $mergeto). Therefore in case when $mergefrom == $mergeto and the heading tag changes, $authtypefrom and $authfrom calculated in merge in the ordinar way are misleading and should not be taken unto consideration. Test plan: ========== 1. run t/db_dependent/Authority/Merge.t 2. you should see problems in "Test update A with modified heading tag (changing authtype)" 3. apply the patch 4. run t/db_dependent/Authority/Merge.t again 5. the test should pass Alternatively: 1. have an authority record used in biblio; export it to file; change 1XX heading tag to a different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2. make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3. repeat the test from 1.
Created attachment 138122 [details] [review] Bug 19693: Add test sub to Merge.t
Created attachment 140915 [details] [review] Bug 19693: Add test sub to Merge.t In those rare cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority: Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $mergefrom == $mergeto). Therefore in case when $mergefrom == $mergeto and the heading tag changes, $authtypefrom and $authfrom calculated in merge in the ordinar way are misleading and should not be taken unto consideration. Test plan: ========== 1. run t/db_dependent/Authority/Merge.t 2. you should see problems in "Test update A with modified heading tag (changing authtype)" 3. apply the patch 4. run t/db_dependent/Authority/Merge.t again 5. the test should pass Alternatively: 1. have an authority record used in biblio; export it to file; change 1XX heading tag to a different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2. make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3. repeat the test from 1. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 140916 [details] [review] Bug 19693: Update of an authority record creates inconsistency when the heading tag is changed In those rare cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority: Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $mergefrom == $mergeto). Therefore in case when $mergefrom == $mergeto and the heading tag changes, $authtypefrom and $authfrom calculated in merge in the ordinar way are misleading and should not be taken unto consideration. Test plan: ========== 1. run t/db_dependent/Authority/Merge.t 2. you should see problems in "Test update A with modified heading tag (changing authtype)" 3. apply the patch 4. run t/db_dependent/Authority/Merge.t again 5. the test should pass Alternatively: 1. have an authority record used in biblio; export it to file; change 1XX heading tag to a different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2. make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3. repeat the test from 1. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Works as described and the code makes sense.. also nice looking unit tests as a quick glance. Signing off.
QAing
Great catch, Janusz. I know the code of merge very well, although time has passed :) Have the feeling that we are cheating a bit here, but I tend to agree with your solution direction. Tempted to polish it a bit further..
Created attachment 141480 [details] [review] Bug 19693: Add test sub to Merge.t In those rare cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority: Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $mergefrom == $mergeto). Therefore in case when $mergefrom == $mergeto and the heading tag changes, $authtypefrom and $authfrom calculated in merge in the ordinar way are misleading and should not be taken unto consideration. Test plan: ========== 1. run t/db_dependent/Authority/Merge.t 2. you should see problems in "Test update A with modified heading tag (changing authtype)" 3. apply the patch 4. run t/db_dependent/Authority/Merge.t again 5. the test should pass Alternatively: 1. have an authority record used in biblio; export it to file; change 1XX heading tag to a different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2. make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3. repeat the test from 1. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 141481 [details] [review] Bug 19693: Update of an authority record creates inconsistency when the heading tag is changed In those rare cases when authorities are updated by an external agency (or even internally, by reviewing and correcting an exported authority file) when the heading tag will be changed (seems odd but happens: 111 Congress ==> 110 Corporate body.Congress ; 100 Person ==> 110 Corporate body (a company named with person's name ; 151 City--object ==> 150 Object (city) etc.) and then the authority record in Koha database will be updated with bulkmarcimport or by calling directly ModAuthority from a custom script, the merge function "doesn't know" that the change to the authority type has been made and, consequently, doesn't adequately change the tag in related fields in biblio records (as it would if two different records with different authtypecode were merged with Koha interface). This is because at the moment when merge function is being called by ModAuthority: Koha::Authority::Types->find($autfrom->authtypecode) Koha::Authority::Types->find($authto->authtypecode) both have the same value (because $mergefrom == $mergeto). Therefore in case when $mergefrom == $mergeto and the heading tag changes, $authtypefrom and $authfrom calculated in merge in the ordinar way are misleading and should not be taken unto consideration. Test plan: ========== 1. run t/db_dependent/Authority/Merge.t 2. you should see problems in "Test update A with modified heading tag (changing authtype)" 3. apply the patch 4. run t/db_dependent/Authority/Merge.t again 5. the test should pass Alternatively: 1. have an authority record used in biblio; export it to file; change 1XX heading tag to a different (but reasonable) value and possibly change also the content of the heading (one can delete also 942 but it doesn't matter); make bulkmarcimport.pl -a -update -file <modified_auth_file> and see that the tag in biblio record has not been changed (whereas the type of authority record did change); 2. make orders in database (so that the authority type and the tag of the field in biblio record correspond); apply the patch; 3. repeat the test from 1. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 141482 [details] [review] Bug 19693: (QA follow-up) Replace changed tag test The test $MARCfrom->field('1..', '2..'))[0]->tag ne ($MARCto->field('1..', '2..'))[0]->tag is not completely consistent with following code in sub merge. I decided to get the authtype code from the old record that comes from Koha and should include the type. Remaining changes refer to indentation/comments. Test plan: Run t/db_dependent/Authority/Merge.t Note: The test actually could be extended a bit for mocking MARC flavor, but needs some additional framework support to work. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Pushed to master for 22.11. Nice work everyone, thanks!
Backported to 22.05.x for upcoming 22.05.08
applied to 21.11 for 21.11.15
Not backported to 21.05.x