Currently, the only default matching rules for MARC21 are ISBN and ISSN. However, I'm sure that many people would profit from having a 999$c "Koha biblionumber" record matching rule as well. 1) If you export your bib records to do some global/batch changes with another program (like MarcEdit), you can use this rule to replace the existing bib records with your modified records. 2) With my upcoming OAI-PMH harvesting feature, if you add a OAI-PMH repository with this rule in the configuration, you'll be able to automagically match updated records with the original records that you imported via OAI-PMH (the magic is that I check the incoming unique identifier against records in Koha, and if there is already a bib record in Koha matching that identifier, I write the biblionumber for that record back into the incoming record so that they match. It sounds convoluted but it works well). 3) People might have other ideas about this as well.
Created attachment 20752 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly
Applying: Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults Using index info to reconstruct a base tree... M installer/data/mysql/updatedatabase.pl Falling back to patching base and 3-way merge... Auto-merging installer/data/mysql/updatedatabase.pl CONFLICT (content): Merge conflict in installer/data/mysql/updatedatabase.pl Patch failed at 0001 Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults The copy of the patch that failed is found in: /home/christopher/git/koha/.git/rebase-apply/patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
Created attachment 21216 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly
Created attachment 21587 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly
Patch applied cleanly, go forth and signoff
Created attachment 23772 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN 0) Apply the patch 1) Run updatedatabase.pl 2) Export some MARC bibliographic records from Koha 3) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 4) The incoming records should match perfectly
Comment on attachment 23772 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults Review of attachment 23772 [details] [review]: ----------------------------------------------------------------- ::: installer/data/mysql/updatedatabase.pl @@ +7857,5 @@ > +if ( CheckVersion($DBversion) ) { > + if ( C4::Context->preference("marcflavour") eq 'MARC21' ) { > + $dbh->do(" > + INSERT INTO marc_matchers (code, description, record_type, threshold) > + VALUES ('KohaBiblio', '999$c', 'biblio', 1000); The $c is taken as a variable, causing this to fail the QA tool.
REVISED TEST PLAN ----------------- 1) Apply the patch 2) Run the QA test tool 3) Drop the koha database 4) Create the koha database 5) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the installer/data/mysql/en/marcflavour/marc21/optional/marc21_default_matching_rules.sql gets triggered. 6) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted. 7) restore an older database 8) Go to the staff client page to trigger an update. 9) Check the mysql tables to make sure the values were inserted. I think there may be some way of checking one could do from within Koha to test the validity of the addition, but I don't know it off the top of my head. Marked as Failed QA, because I ran up to the QA test tool step.
You can't do multiple statements in the $dbh->do(); http://search.cpan.org/~timb/DBI-1.630/DBI.pm#do
Comment on attachment 21587 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This is closer to correct.
Comment on attachment 23772 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This breaks the do() in addition to the $c issue.
Created attachment 24421 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I wasn't able to do steps 5-7 properly, because of the lack of 999$c MARC records in my data as far as I know.
(In reply to M. Tompsett from comment #13) > Created attachment 24421 [details] [review] [review] > Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults > > This patch adds a default record matching rule for biblionumber > using the 999$c and the Local-Number index. > > This is part of the optional marc21 SQL, as well as in the > updatabase.pl script. > > TEST PLAN > --------- > 1) Back up your DB, because you probably want it back. > 2) Apply the patch > 3) Run the QA test tool. > 4) Run updatedatabase.pl > 5) Export some MARC bibliographic records from Koha > 6) Import those same MARC bibliographic records using the > "KohaBiblio" record matching rule. > 7) The incoming records should match perfectly > 8) Check the mysql tables (marc_matchers, matchpoints, > matcher_matchpoints, matchpoint_components, > matchpoint_component_norms) to make sure the values were > inserted as expected. > 9) drop the koha database > 10) create the koha database > 11) Go to the staff client page, and do a "fresh" install making > sure to select lots of optional marc 21 matching rules so > the .../marc21/optional/marc21_default_matching_rules.sql > gets triggered. > 12) Check the mysql tables (marc_matchers, matchpoints, > matcher_matchpoints, matchpoint_components, > matchpoint_component_norms) to make sure the values were > inserted as expected. > 13) Restore your DB backup. :) > > I wasn't able to do steps 5-7 properly, because of the lack > of 999$c MARC records in my data as far as I know. Thanks for the update. I've been putting this one off as a trivial enhancement for a little while. You should be able to do steps 5-7. Your Koha biblionumber is stored as your 999$c. If you don't have one in your Koha records, something is seriously wrong. That said, it won't show up for you in Koha, as it's a hidden field everywhere. You'd have to verify its existence by checking the marcxml in the biblioitems table.
There are multiple ways to export. The first way is probably the best. Home -> Tools -> Export data AND Home -> Search the catalogue -> find a specific biblio -> Save -> MARC (Unicode / UTF-8) (Only the UTF-8 and MARCXML ones will save 999$c data) I also ran into a nasty save problem for a title: Identification of an ‘Unidentified’ Holle List (The): The Last Word (list) I suspect the ticks or brackets caused the issue, but that is a different bug. :)
Created attachment 24441 [details] [review] [SIGNED OFF] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I corrected any problems I found with the patch, particularly the $c issue. Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Created attachment 24528 [details] [review] Bug 10811 - Add a 999$c record matching rule to the MARC21 defaults This patch adds a default record matching rule for biblionumber using the 999$c and the Local-Number index. This is part of the optional marc21 SQL, as well as in the updatabase.pl script. TEST PLAN --------- 1) Back up your DB, because you probably want it back. 2) Apply the patch 3) Run the QA test tool. 4) Run updatedatabase.pl 5) Export some MARC bibliographic records from Koha 6) Import those same MARC bibliographic records using the "KohaBiblio" record matching rule. 7) The incoming records should match perfectly 8) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 9) drop the koha database 10) create the koha database 11) Go to the staff client page, and do a "fresh" install making sure to select lots of optional marc 21 matching rules so the .../marc21/optional/marc21_default_matching_rules.sql gets triggered. 12) Check the mysql tables (marc_matchers, matchpoints, matcher_matchpoints, matchpoint_components, matchpoint_component_norms) to make sure the values were inserted as expected. 13) Restore your DB backup. :) I corrected any problems I found with the patch, particularly the $c issue. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
I would question whether this match point should be added for upgrades, as the installer may have opted not to install the default match points. I believe that is a question for the release manager though.
(In reply to Kyle M Hall from comment #18) > I would question whether this match point should be added for upgrades, as > the installer may have opted not to install the default match points. I > believe that is a question for the release manager though. Either the database update should be skipped, or the MARC matching rules SQL should be mandatory during installation.
Pushed to master. Note that I dropped the database update and added a follow-up to ensure that matching would work regardless of the QueryParser mode. By the way, I wouldn't be opposed to making the default matching rules mandatory during installation. Thanks David!
*** Bug 8748 has been marked as a duplicate of this bug. ***