Bug 26326

Summary: Add Koha Objects for Import Records and Import Record Matches
Product: Koha Reporter: Nick Clemens <nick>
Component: Architecture, internals, and plumbingAssignee: Nick Clemens <nick>
Status: CLOSED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: enhancement    
Priority: P5 - low CC: andrewfh, dcook, jonathan.druart, kyle, m.de.rooy
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Medium patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.11.00
Bug Depends on: 26325    
Bug Blocks: 22785    
Attachments: Bug 26326: Add Koha Objects for Record Import Matches
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: DO NOT PUSH Schema changes
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: DO NOT PUSH Schema changes
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: DO NOT PUSH Schema changes
Bug 26326: (follow-up) Address qa script failures
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: DO NOT PUSH Schema changes
Bug 26326: (follow-up) Address qa script failures
Bug 26326: Add Koha Objects for Import Records and Import Record Matches
Bug 26326: (follow-up) Add ability to check for existence of any primary key
Bug 26326: DO NOT PUSH Schema changes

Description Nick Clemens 2020-08-31 16:46:25 UTC

    
Comment 1 Nick Clemens 2020-08-31 16:49:19 UTC Comment hidden (obsolete)
Comment 2 Nick Clemens 2021-01-04 17:13:16 UTC Comment hidden (obsolete)
Comment 3 Nick Clemens 2021-03-29 18:28:59 UTC
Created attachment 118974 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

https://bugs.koha-community.org/show_bug.cgi?id=22785
Comment 4 Owen Leonard 2021-03-30 14:53:01 UTC
I get an error when running updatedatabase:

DEV atomic update: bug_26326.perl
Atomic update generated errors: {UNKNOWN}: DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'; check that column/key exists [for Statement "ALTER TABLE import_record_matches DROP PRIMARY KEY"]  at /usr/share/perl5/DBIx/Class/Schema.pm line 1118.
	DBIx::Class::Schema::throw_exception(Koha::Schema=HASH(0x55bbeee34d60), "DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'"...) called at /usr/share/perl5/DBIx/Class/Storage.pm line 113
	DBIx::Class::Storage::throw_exception(DBIx::Class::Storage::DBI::mysql=HASH(0x55bbee117670), "DBI Exception: DBD::mysql::db do failed: Can't DROP 'PRIMARY'"...) called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1501
	DBIx::Class::Storage::DBI::__ANON__("DBD::mysql::db do failed: Can't DROP 'PRIMARY'; check that co"..., DBI::db=HASH(0x55bbef809d70), undef) called at (eval 1457) line 7
	eval '$DBversion = \'XXX\';
if( CheckVersion( $DBversion ) ) {
    unless(
        primary_key_exists(\'import_record_matches\',\'import_record_id\') &&
        primary_key_exists(\'import_record_matches\',\'candidate_match_id\')
    ){
        $dbh->do( "ALTER TABLE import_record_matches DROP PRIMARY KEY" );
        $dbh->do( "ALTER TABLE import_record_matches ADD PRIMARY KEY (import_record_id,candidate_match_id)" );
    }

    NewVersion( $DBversion, 26326, "Add primary key to import_record_matches");
}
' called at /kohadevbox/koha/installer/data/mysql/updatedatabase.pl line 23755
Comment 5 Nick Clemens 2021-03-30 15:09:48 UTC
Created attachment 119017 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 6 Nick Clemens 2021-05-14 17:43:34 UTC
Created attachment 120992 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

https://bugs.koha-community.org/show_bug.cgi?id=22785
Comment 7 Nick Clemens 2021-05-14 17:43:38 UTC
Created attachment 120993 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 8 Andrew Fuerste-Henry 2021-05-19 12:34:30 UTC
I'm getting a failure on that second test:
root@kohadevbox:koha(master)$ prove -v t/db_dependent/Koha/Import/Record/Matches.t
t/db_dependent/Koha/Import/Record/Matches.t .. 
1..7
No method count found for Koha::Import::Record::Matches DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'chosen' in 'where clause' [for Statement "SELECT COUNT( * ) FROM `import_record_matches` `me` WHERE ( `chosen` = ? )" with ParamValues: 0=1] at /kohadevbox/koha/Koha/Objects.pm line 601
 at t/db_dependent/Koha/Import/Record/Matches.t line 34.
ok 1 - The 2 matches should have been added
No method count found for Koha::Import::Record::Matches DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'chosen' in 'where clause' [for Statement "SELECT COUNT( * ) FROM `import_record_matches` `me` WHERE ( `chosen` = ? )" with ParamValues: 0=1] at /kohadevbox/koha/Koha/Objects.pm line 601
 at t/db_dependent/Koha/Import/Record/Matches.t line 41.
Use of uninitialized value $nb_of_chosen_matches in addition (+) at t/db_dependent/Koha/Import/Record/Matches.t line 41.
not ok 2 - The 2 chosen matches should have been added

#   Failed test 'The 2 chosen matches should have been added'
#   at t/db_dependent/Koha/Import/Record/Matches.t line 41.
#          got: undef
#     expected: '2'
ok 3 - Find a match by import record id and candidate should return the correct match
The method Koha::Import::Record::Match->chosen is not covered by tests!

Trace begun at /kohadevbox/koha/Koha/Object.pm line 826
Koha::Object::AUTOLOAD('Koha::Import::Record::Match=HASH(0x560a165b3078)', 0) called at /kohadevbox/koha/Koha/Import/Record/Matches.pm line 49
Koha::Import::Record::Matches::unset_chosen('Koha::Import::Record::Matches=HASH(0x560a165f24b8)') called at t/db_dependent/Koha/Import/Record/Matches.t line 47
# Looks like your test exited with 255 just after 3.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 5/7 subtests 

Test Summary Report
-------------------
t/db_dependent/Koha/Import/Record/Matches.t (Wstat: 65280 Tests: 3 Failed: 1)
  Failed test:  2
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 7 tests but ran 3.
Files=1, Tests=3,  2 wallclock secs ( 0.01 usr  0.00 sys +  1.17 cusr  0.21 csys =  1.39 CPU)
Result: FAIL
Comment 9 Nick Clemens 2021-05-19 12:57:09 UTC
Created attachment 121164 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t
Comment 10 Nick Clemens 2021-05-19 12:57:12 UTC
Created attachment 121165 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Comment 11 Nick Clemens 2021-05-19 12:57:16 UTC
Created attachment 121166 [details] [review]
Bug 26326: DO NOT PUSH Schema changes
Comment 12 Andrew Fuerste-Henry 2021-05-19 21:31:02 UTC
Created attachment 121199 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 13 Andrew Fuerste-Henry 2021-05-19 21:31:06 UTC
Created attachment 121200 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 14 Andrew Fuerste-Henry 2021-05-19 21:31:10 UTC
Created attachment 121201 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 15 Marcel de Rooy 2021-07-16 07:13:47 UTC
Looking here
Comment 16 Marcel de Rooy 2021-07-16 09:55:54 UTC
Couldnt finish this one. Resetting status
Comment 17 Nick Clemens 2021-08-27 10:44:22 UTC
Created attachment 124171 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 18 Nick Clemens 2021-08-27 10:44:25 UTC
Created attachment 124172 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 19 Nick Clemens 2021-08-27 10:44:30 UTC
Created attachment 124173 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Comment 20 Katrin Fischer 2021-10-20 21:53:02 UTC
Please fix the QA script POD and merge marker errors.
Comment 21 Nick Clemens 2021-10-27 12:07:15 UTC
Created attachment 126979 [details] [review]
Bug 26326: (follow-up) Address qa script failures
Comment 22 Nick Clemens 2021-10-27 12:09:04 UTC
(In reply to Katrin Fischer from comment #20)
> Please fix the QA script POD and merge marker errors.

Remaining errors are spurious and can be ignored
Comment 23 Katrin Fischer 2021-11-01 13:29:20 UTC
Created attachment 127165 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 24 Katrin Fischer 2021-11-01 13:29:25 UTC
Created attachment 127166 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 25 Katrin Fischer 2021-11-01 13:29:29 UTC
Created attachment 127167 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 26 Katrin Fischer 2021-11-01 13:29:34 UTC
Created attachment 127168 [details] [review]
Bug 26326: (follow-up) Address qa script failures

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 27 Jonathan Druart 2021-11-03 13:44:19 UTC
+sub unset_chosen {
+    my ( $self ) = @_;
+    while ( my $match = $self->next ){
+        $match->chosen(0)->store;
+    }
+}

This is not used, not covered by tests, and actually could be replaced by a single $set->update call.
Comment 28 Jonathan Druart 2021-11-03 13:48:12 UTC
Created attachment 127257 [details] [review]
Bug 26326: Add Koha Objects for Import Records and Import Record Matches

To test:
1 - Apply patches
2 - Update database
3 - Generate schema files (dbic)
4 - prove -v t/db_dependent/Koha/Import/Records.t
5 - prove -v t/db_dependent/Koha/Import/Record/Matches.t

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

JD Amended patch: remove unused unset_chosen method
Comment 29 Jonathan Druart 2021-11-03 13:48:17 UTC
Created attachment 127258 [details] [review]
Bug 26326: (follow-up) Add ability to check for existence of any primary key

Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 26326: (follow-up) Address qa script failures

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 30 Jonathan Druart 2021-11-03 13:48:22 UTC
Created attachment 127259 [details] [review]
Bug 26326: DO NOT PUSH Schema changes

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 31 Jonathan Druart 2021-11-03 14:30:02 UTC
Pushed to master for 21.11, thanks to everybody involved!