In Germany the ISBN with hyphens is stored in 020$9. If you turn on the system preference IncludeSeeFromInSearches this can break zebra indexing process. There are potentially other fields where this problem can occour. Working on a test plan to reproduce the error consistently.
The problem is, that only sometimes the ISBN will falsely match an authority record. So the problem is a bit hard to reproduce this way, as I have not found out how to make sure that ISBN and Koha's authority number match. We had this occuring twice for a dataset of about 50.000 records. So this is a sure way to reproduce the problem: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 1) Make sure IncludeSeeFromInSearches is OFF > Reindexing should work without any problems. 2) Activate IncludeSeeFromInSearches > Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHEadings.pm line 94 So what happened: The system found the $9, found the matching authority, found the 400, wanted to add the content of 400 to the bibliographic record for indexing, constructed a field that was below >010 to store the data in - and exploded. Because controlfields have no subfields. (Naturally)
Created attachment 15382 [details] [review] Bug 9620: IncludeSeeFromInSearches and date in $9 subfields can break indexing In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this can break indexing in combination with IncludeSeeFromInSearches. To reproduce: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 3) Make sure IncludeSeeFromInSearches is OFF * Reindexing should work without any problems. 4) Activate IncludeSeeFromInSearches * Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHEadings.pm line 94 5) Apply patch and reindex. * No error should occur. * Test 4xx fields from other tags were properly indexed. What causes the problem is the fact, that Koha tries to store the date from the 400 in the bibliographic record for indexing. For this it calculates a tag in the bibliographic record from the tag it found the authority link in and the 4xx tags used in the authority record. If you have an authority linked to a 0xx field and the matching authority has a 4xx where xx < 10, the calculated tag will be a control field. The attempt to add subfields to this control field, is causing the error and breaks the indexing process. So what happened: The system found the $9, found the matching authority, found the 400, wanted to add the content of 400 to the bibliographic record for indexing, constructed a field that was below >010 to store the data in - and exploded. Because controlfields have no subfields. (Naturally)
Created attachment 15384 [details] [review] Bug 9620: IncludeSeeFromInSearches and date in $9 subfields can break indexing In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this can break indexing in combination with IncludeSeeFromInSearches. To reproduce: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 3) Make sure IncludeSeeFromInSearches is OFF * Reindexing should work without any problems. 4) Activate IncludeSeeFromInSearches * Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHEadings.pm line 94 5) Apply patch and reindex. * No error should occur. * Test 4xx fields from other tags were properly indexed. What causes the problem is the fact, that Koha tries to store the date from the 400 in the bibliographic record for indexing. For this it calculates a tag in the bibliographic record from the tag it found the authority link in and the 4xx tags used in the authority record. If you have an authority linked to a 0xx field and the matching authority has a 4xx where xx < 10, the calculated tag will be a control field. The attempt to add subfields to this control field, is causing the error and breaks the indexing process.
Created attachment 15422 [details] [review] [SIGNED-OFF] Bug 9620: IncludeSeeFromInSearches and date in $9 subfields can break indexing In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this can break indexing in combination with IncludeSeeFromInSearches. To reproduce: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 3) Make sure IncludeSeeFromInSearches is OFF * Reindexing should work without any problems. 4) Activate IncludeSeeFromInSearches * Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHEadings.pm line 94 5) Apply patch and reindex. * No error should occur. * Test 4xx fields from other tags were properly indexed. What causes the problem is the fact, that Koha tries to store the date from the 400 in the bibliographic record for indexing. For this it calculates a tag in the bibliographic record from the tag it found the authority link in and the 4xx tags used in the authority record. If you have an authority linked to a 0xx field and the matching authority has a 4xx where xx < 10, the calculated tag will be a control field. The attempt to add subfields to this control field, is causing the error and breaks the indexing process. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Comment: Work as described, and is a long description :) Test: 1) Enabled 084 (all subfields visible), added 084_9 (missing from default framework) linked to PERSO_NAME 2) Edited authority record and added some data to 400 3) Edited bib record, linked 084 to previous auth record 4) With IncludeSeeFromInSearches OFF, re-index without problem 5) With IncludeSeeFromInSearches ON, re-index problems!!! ==================== exporting biblio ==================== 1...Subfields are only for data fields (generally, just tags >= 010) at... Abort 6) Applied patch 7) With IncludeSeeFromInSearches ON, re-index without problem ... ==================== 5001.................................................................................................... Records exported: 5060 ==================== ... ==================== 2001.................................................................................................... Records exported: 2024 ==================== CLEANING ==================== Good! *) That's "works as described", don't know if other tests are needed. No errors from koha-qa
Created attachment 15924 [details] [review] Bug 9620: IncludeSeeFromInSearches and date in $9 subfields can break indexing In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this can break indexing in combination with IncludeSeeFromInSearches. To reproduce: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 3) Make sure IncludeSeeFromInSearches is OFF * Reindexing should work without any problems. 4) Activate IncludeSeeFromInSearches * Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHEadings.pm line 94 5) Apply patch and reindex. * No error should occur. * Test 4xx fields from other tags were properly indexed. What causes the problem is the fact, that Koha tries to store the date from the 400 in the bibliographic record for indexing. For this it calculates a tag in the bibliographic record from the tag it found the authority link in and the 4xx tags used in the authority record. If you have an authority linked to a 0xx field and the matching authority has a 4xx where xx < 10, the calculated tag will be a control field. The attempt to add subfields to this control field, is causing the error and breaks the indexing process. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Comment: Work as described, and is a long description :) Test: 1) Enabled 084 (all subfields visible), added 084_9 (missing from default framework) linked to PERSO_NAME 2) Edited authority record and added some data to 400 3) Edited bib record, linked 084 to previous auth record 4) With IncludeSeeFromInSearches OFF, re-index without problem 5) With IncludeSeeFromInSearches ON, re-index problems!!! ==================== exporting biblio ==================== 1...Subfields are only for data fields (generally, just tags >= 010) at... Abort 6) Applied patch 7) With IncludeSeeFromInSearches ON, re-index without problem ... ==================== 5001.................................................................................................... Records exported: 5060 ==================== ... ==================== 2001.................................................................................................... Records exported: 2024 ==================== CLEANING ==================== Good! *) That's "works as described", don't know if other tests are needed. No errors from koha-qa Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
This patch has been pushed to master.
Pushed to 3.10.x will be in 3.10.4