Setup: * Set AutoCreateAuthorities to generate, AutoLinkBiblios to Do, CatalogModuleRelink to Do (and EnableAdvancedCatalogingEditor to enable if you want to make your life easier) * Ensure that LinkerConsiderThesaurus is at the default Don't * Make sure you don't have a genre authority record for Boring stories Add the following headings to a bib record and save it 655 _ _ ‡aBoring stories 655 _ 0 ‡aBoring stories 655 _ 4 ‡aBoring stories 655 _ 7 ‡aBoring stories‡2gsafd 655 _ 7 ‡aBoring stories‡2lcgft Note that you now have each heading linked to a different authid, and you have five genre authority records for Boring stories, all identical. If instead you already had a Boring stories authority record (no matter the thesaurus) before saving the record, they would have all been linked to it.
Seems like it must be from the linker caching with $thesaurus in the key, and only Heading knowing that LinkerConsiderThesaurus is Don't, but it's not obvious why.
Still does, still annoying having two name authorities created for every autobiography where the author is also a subject.
The example authorities all differ in some way, so is the linker checker taking into consideration the indicators and subfields? One could argue an authority with a second indicator 4 is different from one with a second indicator 7.
They are different, and with LinkerConsiderThesaurus set to Do and AutoCreateAuthorities set to don't generate they will be treated as being different. With LinkerConsiderThesaurus set to Do and AutoCreateAuthorities set to generate, they will all be treated as being different but AutoCreateAuthorities will create identical authorities for them, which is why I filed bug 35125 to make it possible to use autocreate in a considerthesaurus world, and then that was why Nick filed bug 33557 to have a preference to shut off considerthesaurus, because otherwise without a fix for bug 35125 autocreate would be a nightmare. But with LinkerConsiderThesaurus: Don't and AutoCreateAuthorities generate, AutoCreateAuthorities will create five identical authority records for those five things, even though if you already had one authority record the linker would have linked all five to just that one. That's this bug.
Created attachment 168040 [details] [review] Bug 35125: Don't use thesaurus value in cache when LinkerConsiderThesaurus is set to don't This patch adds a new key 'notconsidered' to the authority cache when linking to avoid doubling when creating new authorities and wishing to ignore thesaurus To test: 0 - Set LinkerConsiderThesaurus to "don't" 1 - Set AutCreateAuthorities to generate 2 - Set AutLinkBiblios to Do 3 - Set CataloguingModuleRelink to Do 4 - Save a new record in Koha with the same radnomized heading repeated but from 3 different authority sources ensure source is defined by indicators only and by field $2 once ensure the heading is random so that no matches will be found 5 - Confirm each authority is linked to a new unique authority 6 - Apply patch, restart all 7 - Repeat 4 with a new subject heading 8 - Confirm all headings are linked to the same authority
I added unit tests and they pass with this, but I didn't test in the interface - if you want to flesh out the test plan more please do and I'll look again if it doesn't work
100 and 600 is the only other flavor I've seen, I'll test that when I get time to test. Thanks for the patch!
Tested in a sandbox and the patch made no difference. No reason it shouldn't work in a sandbox, so I'm not hopeful.
Alas, the patch doesn't work, AutoCreateAuthorities still creates separate authorities for each repeated subject.
Created attachment 168047 [details] Test bibs Conveniently, batch import shows the same bug. Be more convenient if you could import records in a batch individually, though. I was going to make a batch of twenty different records, but having twenty separate batches would just be a pain to keep straight. Interesting behavior (though I'm not sure it's actually A Clue): if you import the batch, then import again not looking for matches so you import duplicates, the duplicates will link exactly like the first records did: the first Subject One linked to 1710 in both records, the second to 1711 in both records, etc. I guess that just means that the cache still knows that they are different, and how they are different.
Both get_link and update_cache need to be setting 'notconsidered', but that's not enough to make it work.
Created attachment 168078 [details] [review] Bug 35125: same adjustments for update_cache
Doesn't update_cache need to be setting match_count, so we don't still have a match_count of zero after AutoCreateAuthorities created one and updated the cache?
As in, "how did that ever work?"
(In reply to Phil Ringnalda from comment #13) > Doesn't update_cache need to be setting match_count, so we don't still have > a match_count of zero after AutoCreateAuthorities created one and updated > the cache? Yes, but I think match_count only comes in to play if there is no authid. In this case, we set the authid - which means that whatever linker we are using found a positive match. The count is for when we didn't find a match and need to know if we should create or not.
Comment on attachment 168047 [details] Test bibs Oops, uploaded the pre-linked file, not the unlinked one, that leads to baffling results.
And apparently I'm confused about how batch import works, and when I see authority records created from batch imports in my production system, it's because of people or cron jobs doing other things to them, so comment 11 was just from importing a wrongly pre-linked record and not realizing it was pre-linked and not relinked or freshly linked.
Created attachment 168096 [details] [review] Bug 35125: Don't use thesaurus value in cache when LinkerConsiderThesaurus is set to don't This patch adds a new key 'notconsidered' to the authority cache when linking to avoid doubling when creating new authorities and wishing to ignore thesaurus To test: 0 - Set LinkerConsiderThesaurus to "don't" 1 - Set AutCreateAuthorities to generate 2 - Set AutLinkBiblios to Do 3 - Set CataloguingModuleRelink to Do 4 - Save a new record in Koha with the same radnomized heading repeated but from 3 different authority sources ensure source is defined by indicators only and by field $2 once ensure the heading is random so that no matches will be found 5 - Confirm each authority is linked to a new unique authority 6 - Apply patch, restart all 7 - Repeat 4 with a new subject heading 8 - Confirm all headings are linked to the same authority Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Created attachment 168097 [details] [review] Bug 35125: same adjustments for update_cache Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Created attachment 168261 [details] [review] Bug 35125: Respect LinkerConsiderThesaurus in get_link This patch adds a new key 'notconsidered' to the authority cache when linking to avoid doubling when creating new authorities and wishing to ignore thesaurus To test: 0 - Set LinkerConsiderThesaurus to "don't" 1 - Set AutCreateAuthorities to generate 2 - Set AutLinkBiblios to Do 3 - Set CataloguingModuleRelink to Do 4 - Save a new record in Koha with the same randomized heading repeated but from 3 different authority sources ensure source is defined by indicators only and by field $2 once ensure the heading is random so that no matches will be found 5 - Confirm each authority is linked to a new unique authority 6 - Apply patch, restart all 7 - Repeat 4 with a new subject heading 8 - Confirm all headings are linked to the same authority Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 168262 [details] [review] Bug 35125: Respect LinkerConsiderThesaurus in update_cache Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
All works as expected now, thanks. Passing QA
Pushed for 24.11! Well done everyone, thank you!
Backported to 24.05.x for upcoming 24.05.05
Pushed to 23.11.x for 23.11.11