With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. Example: biblio: 650 _1 $a Philosophy [thesaurus: Library of Congress Children's and Young Adults' Subject Headings] will be linked to an authority 008/11 = a [Library of Congress Subject Headings] 150 __ $a Philosophy if 008/11 = b [Library of Congress Children's and Young Adults' Subject Headings] 150 __ $a Philosophy is missing in the system. This is obviously wrong--we should not link to a authority record belonging to a different thesaurus.
I put wrong example. Actually, the problem arises only with thesauri defined in with 008/11 = 'z' and 040 $f. Example: biblio: 650 _7 $a Filozofia $2 dbn [thesaurus: dbn] will be linked to an authority 008/11 = z 040 $f kaba 150 __ $a Filozofia if authority record: 008/11 = z 040 $f dbn 150 __ $a Filozofia is missing in the system. This is obviously wrong--we should not link to a authority record belonging to a different thesaurus.
This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record.
Created attachment 180082 [details] [review] Bug 39503: Bug 39503 - Linker should always respect thesaurus with LinkerConsiderThesaurus on With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all. 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.)
Created attachment 180131 [details] [review] Bug 39503: Unit tests NB, some previous test had to be adjusted to make it posible to call search_auth_compat in _search and properly interpret its results.
Created attachment 180200 [details] [review] Bug 39503: Bug 39503 - Linker should always respect thesaurus with LinkerConsiderThesaurus on With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all. 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.) Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 180201 [details] [review] Bug 39503: Unit tests NB, some previous test had to be adjusted to make it posible to call search_auth_compat in _search and properly interpret its results. Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 180844 [details] [review] Bug 39503: [alternate] Collapse thesaurus if value in 008/_11 and 040
Created attachment 180902 [details] [review] Bug 39503: Collapse thesaurus if value in 008/_11 and 040 With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all ; reindex ES for authorities with: koha-elasticsearch --rebuild -d -a kohadev 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.) Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Created attachment 180903 [details] [review] Bug 39503: Unit tests NB, some previous test had to be adjusted to make it posible to call search_auth_compat in _search and properly interpret its results. Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 180940 [details] [review] Bug 39503: Collapse thesaurus if value in 008_/11 = 'z' and 040 $f With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all ; reindex ES for authorities with: koha-elasticsearch --rebuild -d -a kohadev 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.) Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Created attachment 180941 [details] [review] Bug 39503: Unit tests NB, some previous test had to be adjusted to make it posible to call search_auth_compat in _search and properly interpret its results. Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 181205 [details] [review] Bug 39503: Collapse thesaurus if value in 008_/11 = 'z' and 040 $f With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all ; reindex ES for authorities with: koha-elasticsearch --rebuild -d -a kohadev 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.) Signed-off-by: Janusz Kaczmarek <januszop@gmail.com> Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Created attachment 182182 [details] [review] Bug 39503: Unit tests NB, some previous test had to be adjusted to make it posible to call search_auth_compat in _search and properly interpret its results. Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Created attachment 182183 [details] [review] Bug 39503: Collapse thesaurus if value in 008_/11 = 'z' and 040 $f With bug 30280 and subsequent patches we attempt to respect the thesaurus indication when linking with LinkerConsiderThesaurus on. However, there are still cases when we link wrong. This happens when thesaurus in bibliographic field 6XX is defined by subfield $2. In such a case the authority record should have 008/11 = 'z' and, according to https://www.loc.gov/marc/authority/ad008.html (cf. comment for 008/11 = 'z': "A MARC code for the conventions used to formulate the heading *may* be contained in subfield $f"), may or may have thesaurus declared in 040 $f. This is why we make a second search attempt in C4::Heading::_search, but doing so we have to make sure that the resulting authority record has not a different thesaurus declared in 040 $f. (Note that 008/11 and 040 $f are indexed with the same search field: subject-heading-thesaurus. So we search for the second time in the same search field but with 'notdefined' == 008/11='z' instead of the original thesaurus taken from subfield $2 of the bibliographic field 6XX.) This means we have to explicitly check the content of 040 $f of the resulting authority record. Test plan: ========== 1. Turn on the LinkerConsiderThesaurus systempreference. 2. Create an authority record (e.g. TOPIC_TERM) with 008/11 = 'z' and 040 $f containing a thesaurus indication (e.g. 'kaba'), and a term in field 150 $a (e.g. Early music). 3. Edit a bibliographic record: in field 650 put 2nd indicator '7', the term 'Early music' in $a, and a different thesaurus code in $2 (e.g. 'dbn'). Click the 'Link authorities automatically' button. Note that 650 'Early music' (dbn) has been linked with and authority record 'Early music' (kaba). This is wrong. 4. Apply the patch ; restart_all ; reindex ES for authorities with: koha-elasticsearch --rebuild -d -a kohadev 5. Repeat p. 3. Now the 650 'Early music' (dbn) has not been linked to an authority record (650 - No matching authority found.) Signed-off-by: Janusz Kaczmarek <januszop@gmail.com> Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
The bug description mentions LinkerConsiderThesaurus, but I don't spot any code checking for the preference in the patch set?
Pushed for 25.05! Well done everyone, thank you!
(In reply to Katrin Fischer from comment #15) > The bug description mentions LinkerConsiderThesaurus, but I don't spot any > code checking for the preference in the patch set? This is because my original version of the patch (https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=180082&action=diff) referred to C4/Heading.pm (and the snippet was executed only if LinkerConsiderThesaurus was set). Now, the bug could be named after the patch title, like "Collapse elasticsearch subject-heading-thesaurus search field if value in 008_/11 = 'z' and 040 $f" or somehow similar.