Bug 26326 - Add Koha Objects for Import Records and Import Record Matches
Summary: Add Koha Objects for Import Records and Import Record Matches
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Nick Clemens
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on: 26325
Blocks: 22785
  Show dependency treegraph
 
Reported: 2020-08-31 16:46 UTC by Nick Clemens
Modified: 2021-11-05 18:24 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Text to go in the release notes:
Version(s) released in:
21.11.00


Attachments
Bug 26326: Add Koha Objects for Record Import Matches (7.70 KB, patch)
2020-08-31 16:49 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.72 KB, patch)
2021-01-04 17:13 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.75 KB, patch)
2021-03-29 18:28 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.09 KB, patch)
2021-03-30 15:09 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.75 KB, patch)
2021-05-14 17:43 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (2.94 KB, patch)
2021-05-14 17:43 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.14 KB, patch)
2021-05-19 12:57 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.10 KB, patch)
2021-05-19 12:57 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: DO NOT PUSH Schema changes (1.29 KB, patch)
2021-05-19 12:57 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.21 KB, patch)
2021-05-19 21:31 UTC, Andrew Fuerste-Henry
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.17 KB, patch)
2021-05-19 21:31 UTC, Andrew Fuerste-Henry
Details | Diff | Splinter Review
Bug 26326: DO NOT PUSH Schema changes (1.35 KB, patch)
2021-05-19 21:31 UTC, Andrew Fuerste-Henry
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.15 KB, patch)
2021-08-27 10:44 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.45 KB, patch)
2021-08-27 10:44 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: DO NOT PUSH Schema changes (1.35 KB, patch)
2021-08-27 10:44 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: (follow-up) Address qa script failures (2.23 KB, patch)
2021-10-27 12:07 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.26 KB, patch)
2021-11-01 13:29 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.50 KB, patch)
2021-11-01 13:29 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 26326: DO NOT PUSH Schema changes (1.41 KB, patch)
2021-11-01 13:29 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 26326: (follow-up) Address qa script failures (2.28 KB, patch)
2021-11-01 13:29 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 26326: Add Koha Objects for Import Records and Import Record Matches (14.02 KB, patch)
2021-11-03 13:48 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26326: (follow-up) Add ability to check for existence of any primary key (3.77 KB, patch)
2021-11-03 13:48 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 26326: DO NOT PUSH Schema changes (1.41 KB, patch)
2021-11-03 13:48 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens 2020-08-31 16:46:25 UTC

    
Comment 1 Nick Clemens 2020-08-31 16:49:19 UTC Comment hidden (obsolete)
Comment 2 Nick Clemens 2021-01-04 17:13:16 UTC Comment hidden (obsolete)
Comment 3 Nick Clemens 2021-03-29 18:28:59 UTC
Created attachment 118974 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

https://bugs.koha-community.org/show_bug.cgi?id=22785
Comment 4 Owen Leonard 2021-03-30 14:53:01 UTC
I get an error when running updatedatabase:

DEV atomic update: bug_26326.perl
Atomic update generated errors: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'; check that column/key exists [for Statement "ALTER TABLE import_record_matches DROP PRIMARY KEY"]  at /usr/share/perl5/DBIx/Class/Schema.pm line 1118.
	DBIx::Class::Schema::throw_exception(Koha::Schema=HASH(0x55bbeee34d60), "DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'"...) called at /usr/share/perl5/DBIx/Class/Storage.pm line 113
	DBIx::Class::Storage::throw_exception(DBIx::Class::Storage::DBI::mysql=HASH(0x55bbee117670), "DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'"...) called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1501
	DBIx::Class::Storage::DBI::__ANON__("DBD::mysql::db do failed: Can't DROP 'PRIMARY'; check that co"..., DBI::db=HASH(0x55bbef809d70), undef) called at (eval 1457) line 7
	eval '$DBversion = \'XXX\';
if( CheckVersion( $DBversion ) ) {
    unless(
        primary_key_exists(\'import_record_matches\',\'import_record_id\') &&
        primary_key_exists(\'import_record_matches\',\'candidate_match_id\')
    ){
        $dbh->do( "ALTER TABLE import_record_matches DROP PRIMARY KEY" );
        $dbh->do( "ALTER TABLE import_record_matches ADD PRIMARY KEY (import_record_id,candidate_match_id)" );
    }

    NewVersion( $DBversion, 26326, "Add primary key to import_record_matches");
}
' called at /kohadevbox/koha/installer/data/mysql/updatedatabase.pl line 23755
Comment 5 Nick Clemens 2021-03-30 15:09:48 UTC
Created attachment 119017 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 6 Nick Clemens 2021-05-14 17:43:34 UTC
Created attachment 120992 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

https://bugs.koha-community.org/show_bug.cgi?id=22785
Comment 7 Nick Clemens 2021-05-14 17:43:38 UTC
Created attachment 120993 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 8 Andrew Fuerste-Henry 2021-05-19 12:34:30 UTC
I'm getting a failure on that second test:
root@kohadevbox:koha(master)$ prove -v t/db_dependent/Koha/Import/Record/Matches.t
t/db_dependent/Koha/Import/Record/Matches.t .. 
1..7
No method count found for Koha::Import::Record::Matches DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'chosen' in 'where clause' [for Statement "SELECT COUNT( * ) FROM `import_record_matches` `me` WHERE ( `chosen` = ? )" with ParamValues: 0=1] at /kohadevbox/koha/Koha/Objects.pm line 601
 at t/db_dependent/Koha/Import/Record/Matches.t line 34.
ok 1 - The 2 matches should have been added
No method count found for Koha::Import::Record::Matches DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'chosen' in 'where clause' [for Statement "SELECT COUNT( * ) FROM `import_record_matches` `me` WHERE ( `chosen` = ? )" with ParamValues: 0=1] at /kohadevbox/koha/Koha/Objects.pm line 601
 at t/db_dependent/Koha/Import/Record/Matches.t line 41.
Use of uninitialized value $nb_of_chosen_matches in addition (+) at t/db_dependent/Koha/Import/Record/Matches.t line 41.
not ok 2 - The 2 chosen matches should have been added

#   Failed test 'The 2 chosen matches should have been added'
#   at t/db_dependent/Koha/Import/Record/Matches.t line 41.
#          got: undef
#     expected: '2'
ok 3 - Find a match by import record id and candidate should return the correct match
The method Koha::Import::Record::Match->chosen is not covered by tests!

Trace begun at /kohadevbox/koha/Koha/Object.pm line 826
Koha::Object::AUTOLOAD('Koha::Import::Record::Match=HASH(0x560a165b3078)', 0) called at /kohadevbox/koha/Koha/Import/Record/Matches.pm line 49
Koha::Import::Record::Matches::unset_chosen('Koha::Import::Record::Matches=HASH(0x560a165f24b8)') called at t/db_dependent/Koha/Import/Record/Matches.t line 47
# Looks like your test exited with 255 just after 3.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 5/7 subtests 

Test Summary Report
-------------------
t/db_dependent/Koha/Import/Record/Matches.t (Wstat: 65280 Tests: 3 Failed: 1)
  Failed test:  2
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 7 tests but ran 3.
Files=1, Tests=3,  2 wallclock secs ( 0.01 usr  0.00 sys +  1.17 cusr  0.21 csys =  1.39 CPU)
Result: FAIL
Comment 9 Nick Clemens 2021-05-19 12:57:09 UTC
Created attachment 121164 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t
Comment 10 Nick Clemens 2021-05-19 12:57:12 UTC
Created attachment 121165 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 11 Nick Clemens 2021-05-19 12:57:16 UTC
Created attachment 121166 [details] [review]
Bug 26326: DO NOT PUSH Schema changes
Comment 12 Andrew Fuerste-Henry 2021-05-19 21:31:02 UTC
Created attachment 121199 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 13 Andrew Fuerste-Henry 2021-05-19 21:31:06 UTC
Created attachment 121200 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 14 Andrew Fuerste-Henry 2021-05-19 21:31:10 UTC
Created attachment 121201 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 15 Marcel de Rooy 2021-07-16 07:13:47 UTC
Looking here
Comment 16 Marcel de Rooy 2021-07-16 09:55:54 UTC
Couldnt finish this one. Resetting status
Comment 17 Nick Clemens 2021-08-27 10:44:22 UTC
Created attachment 124171 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 18 Nick Clemens 2021-08-27 10:44:25 UTC
Created attachment 124172 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 19 Nick Clemens 2021-08-27 10:44:30 UTC
Created attachment 124173 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 20 Katrin Fischer 2021-10-20 21:53:02 UTC
Please fix the QA script POD and merge marker errors.
Comment 21 Nick Clemens 2021-10-27 12:07:15 UTC
Created attachment 126979 [details] [review]
Bug 26326: (follow-up) Address qa script failures
Comment 22 Nick Clemens 2021-10-27 12:09:04 UTC
(In reply to Katrin Fischer from comment #20)
> Please fix the QA script POD and merge marker errors.

Remaining errors are spurious and can be ignored
Comment 23 Katrin Fischer 2021-11-01 13:29:20 UTC
Created attachment 127165 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 24 Katrin Fischer 2021-11-01 13:29:25 UTC
Created attachment 127166 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 25 Katrin Fischer 2021-11-01 13:29:29 UTC
Created attachment 127167 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 26 Katrin Fischer 2021-11-01 13:29:34 UTC
Created attachment 127168 [details] [review]
Bug 26326: (follow-up) Address qa script failures

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 27 Jonathan Druart 2021-11-03 13:44:19 UTC
+sub unset_chosen {
+    my ( $self ) = @_;
+    while ( my $match = $self->next ){
+        $match->chosen(0)->store;
+    }
+}

This is not used, not covered by tests, and actually could be replaced by a single $set->update call.
Comment 28 Jonathan Druart 2021-11-03 13:48:12 UTC
Created attachment 127257 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

JD Amended patch: remove unused unset_chosen method
Comment 29 Jonathan Druart 2021-11-03 13:48:17 UTC
Created attachment 127258 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 26326: (follow-up) Address qa script failures

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 30 Jonathan Druart 2021-11-03 13:48:22 UTC
Created attachment 127259 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 31 Jonathan Druart 2021-11-03 14:30:02 UTC
Pushed to master for 21.11, thanks to everybody involved!