Bug 36935

Summary: BackgroundJob/ImportKBARTFile.t generates warnings
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Test SuiteAssignee: Matt Blenkinsop <matt.blenkinsop>
Status: Pushed to main --- QA Contact: Martin Renvoize (ashimema) <martin.renvoize>
Severity: normal    
Priority: P5 - low    
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00
Circulation function:
Bug Depends on:    
Bug Blocks: 25515, 36831    
Attachments: Bug 36935: Remove warning if file parsing is successful
Bug 36935: Remove warning if file parsing is successful
Bug 36935: Remove warning if file parsing is successful
Bug 36935: Add a 'die' message to file parsing
Bug 36935: Add unit test for error handling
Bug 36935: Remove warning if file parsing is successful
Bug 36935: Add a 'die' message to file parsing
Bug 36935: Add unit test for error handling
Bug 36935: Remove warning if file parsing is successful
Bug 36935: Add a 'die' message to file parsing
Bug 36935: Add unit test for error handling

Description Jonathan Druart 2024-05-23 07:38:31 UTC
koha_1       | 2012, EOF - End of data in parsing input stream, 0 at /kohadevbox/koha/Koha/BackgroundJob/ImportKBARTFile.pm line 247, <$fh> line 3.
koha_1       | 2012, EOF - End of data in parsing input stream, 0 at /kohadevbox/koha/Koha/BackgroundJob/ImportKBARTFile.pm line 247, <$fh> line 3.
koha_1       | 2012, EOF - End of data in parsing input stream, 0 at /kohadevbox/koha/Koha/BackgroundJob/ImportKBARTFile.pm line 247, <$fh> line 3.
koha_1       | 2012, EOF - End of data in parsing input stream, 0 at /kohadevbox/koha/Koha/BackgroundJob/ImportKBARTFile.pm line 247, <$fh> line 3.
koha_1       | [15:09:28] t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t ......................... ok     3354 ms ( 0.01 usr  0.00 sys +  2.84 cusr  0.32 csys =  3.17 CPU)
Comment 1 Matt Blenkinsop 2024-05-23 08:34:39 UTC
Created attachment 167078 [details] [review]
Bug 36935: Remove warning if file parsing is successful

This patch removes the logged warning if a file parse is successful.

Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
2) The output will have a logged message - 'EOF - End of data in parsing input stream'
3) Apply patch
4) Repeat step 1
5) The warning will be gone
Comment 2 Jonathan Druart 2024-05-23 08:52:16 UTC
errno 2012 means successful?
Comment 3 Matt Blenkinsop 2024-05-23 08:55:35 UTC
Apparently so!
https://metacpan.org/pod/Text::CSV_XS#DIAGNOSTICS
Comment 4 David Nind 2024-05-23 09:00:04 UTC
Created attachment 167080 [details] [review]
Bug 36935: Remove warning if file parsing is successful

This patch removes the logged warning if a file parse is successful.

Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
2) The output will have a logged message - 'EOF - End of data in parsing input stream'
3) Apply patch
4) Repeat step 1
5) The warning will be gone

Signed-off-by: David Nind <david@davidnind.com>
Comment 5 Jonathan Druart 2024-05-23 09:17:51 UTC
(In reply to Matt Blenkinsop from comment #3)
> Apparently so!
> https://metacpan.org/pod/Text::CSV_XS#DIAGNOSTICS

I am not sure, this is not clear for me.
Comment 6 Jonathan Druart 2024-05-23 09:26:23 UTC
use Text::CSV_XS;
open(my $fh, "<", '/tmp/file') or die $!;
my $csv = Text::CSV_XS->new(
    {    
        sep_char           => ',', 
        binary             => 1,
        allow_loose_quotes => 1 
    }    
);
my @lines = @{$csv->getline_all($fh)};
close($fh);

my ( $cde, $str, $pos ) = $csv->error_diag();
my $error = $cde ? "$cde, $str, $pos" : "";

use Data::Printer colored => 1; say p @lines;


/tmp/file contains
one,two,three
1,2,3
4,5,6
7,8,9

run the script and get no error
Comment 7 Jonathan Druart 2024-05-23 09:27:10 UTC
open my $fh, "<", \$file_content or die;

This must die with the reason! The bare minimum is 'die $!'
Comment 8 Marcel de Rooy 2024-05-23 09:28:28 UTC
Or add a newline to the input file or so ?
Comment 9 Matt Blenkinsop 2024-05-23 09:40:06 UTC
Created attachment 167088 [details] [review]
Bug 36935: Remove warning if file parsing is successful

This patch removes the logged warning if a file parse is successful.

Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
2) The output will have a logged message - 'EOF - End of data in parsing input stream'
3) Apply patch
4) Repeat step 1
5) The warning will be gone
Comment 10 Matt Blenkinsop 2024-05-23 09:41:41 UTC
Created attachment 167089 [details] [review]
Bug 36935: Add a 'die' message to file parsing
Comment 11 Jonathan Druart 2024-05-23 09:42:05 UTC
Is this error handling covered by a test?
Comment 12 Matt Blenkinsop 2024-05-23 09:42:51 UTC
Not currently, I'll add one. Patch incoming
Comment 13 Matt Blenkinsop 2024-05-23 10:04:51 UTC
If I force the error handling to throw an error in a unit test we will get a warning in the output which is what we're trying to avoid. Now that we're identifying the end of file we could throw a Koha::Exception rather than a warning?
Comment 14 Jonathan Druart 2024-05-23 10:14:48 UTC
Use Test::Warn (warning_is for instance) if you want to test that a warning will be displayed in the log.
An exception would be better ofc.
Comment 15 Matt Blenkinsop 2024-05-23 11:18:07 UTC
Created attachment 167099 [details] [review]
Bug 36935: Add unit test for error handling

prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
Comment 16 Matt Blenkinsop 2024-05-23 11:19:47 UTC
Bug 36942 raised for throwing an exception, will try and have a look at that when I have some time
Comment 17 David Nind 2024-05-23 20:08:03 UTC
Created attachment 167130 [details] [review]
Bug 36935: Remove warning if file parsing is successful

This patch removes the logged warning if a file parse is successful.

Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
2) The output will have a logged message - 'EOF - End of data in parsing input stream'
3) Apply patch
4) Repeat step 1
5) The warning will be gone

Signed-off-by: David Nind <david@davidnind.com>
Comment 18 David Nind 2024-05-23 20:08:06 UTC
Created attachment 167131 [details] [review]
Bug 36935: Add a 'die' message to file parsing

Signed-off-by: David Nind <david@davidnind.com>
Comment 19 David Nind 2024-05-23 20:08:09 UTC
Created attachment 167132 [details] [review]
Bug 36935: Add unit test for error handling

prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t

Signed-off-by: David Nind <david@davidnind.com>
Comment 20 Martin Renvoize (ashimema) 2024-07-01 15:57:28 UTC
Created attachment 168326 [details] [review]
Bug 36935: Remove warning if file parsing is successful

This patch removes the logged warning if a file parse is successful.

Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t
2) The output will have a logged message - 'EOF - End of data in parsing input stream'
3) Apply patch
4) Repeat step 1
5) The warning will be gone

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 21 Martin Renvoize (ashimema) 2024-07-01 15:57:31 UTC
Created attachment 168327 [details] [review]
Bug 36935: Add a 'die' message to file parsing

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 22 Martin Renvoize (ashimema) 2024-07-01 15:57:33 UTC
Created attachment 168328 [details] [review]
Bug 36935: Add unit test for error handling

prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 23 Martin Renvoize (ashimema) 2024-07-01 15:58:13 UTC
Thanks Matt, always good to make the tests a little cleaner :)
Comment 24 Katrin Fischer 2024-07-02 08:21:19 UTC
Pushed for 24.11!

Well done everyone, thank you!