Bug 15822 - STAFF Advanced search error date utils
Summary: STAFF Advanced search error date utils
Status: Pushed to Stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: master
Hardware: All Linux
: P1 - high major (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 17821
  Show dependency treegraph
 
Reported: 2016-02-15 22:17 UTC by Humberto Blanco
Modified: 2017-02-06 07:32 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Image for one of those record width "340.11" (24.75 KB, image/png)
2016-02-16 13:31 UTC, Humberto Blanco
Details
select cn_class (33.20 KB, application/vnd.ms-excel)
2016-02-17 05:12 UTC, Humberto Blanco
Details
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00 (1.65 KB, patch)
2016-11-23 10:11 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00 (1.70 KB, patch)
2016-11-30 23:19 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00 (1.76 KB, patch)
2016-12-05 17:13 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Humberto Blanco 2016-02-15 22:17:02 UTC
When i try to search in staff advanced search "topographic signature" 340.11, appears following error:

Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /usr/share/koha/lib/Koha/DateUtils.pm line 223.

the search string is:
http://catalogo.edu.co:8080/cgi-bin/koha/catalogue/search.pl?idx=callnum&q=340.11&idx=kw&op=and&sort_by=relevance

I can probe, that when i search with "Explorar índices" option enabled the string changes(adding scan variable) and try other values to explore.
http://catalogo.edu.co:8080/cgi-bin/koha/catalogue/search.pl?idx=callnum&q=340.11&op=and&idx=kw&op=and&idx=kw&scan=1&sort_by=relevance


Really appreciate your help
Comment 1 Jonathan Druart 2016-02-16 09:29:51 UTC
Hi Humberto,
Could you give us the detail of the record matching this callnumber please?
You certainly have a date starting with '0000-0' in your DB for this record, it would be good to know why one.
Comment 2 Humberto Blanco 2016-02-16 13:29:09 UTC
(In reply to Jonathan Druart from comment #1)
> Hi Humberto,
> Could you give us the detail of the record matching this callnumber please?
> You certainly have a date starting with '0000-0' in your DB for this record,
> it would be good to know why one.

I dont know exactly what are you talking, can be a little more specific please, this was after migration to  3.22 from 3.08.006. I´m adding a image of one of those records that have "340.11" in call number.
Comment 3 Humberto Blanco 2016-02-16 13:31:27 UTC
Created attachment 48121 [details]
Image for one of those record width "340.11"
Comment 4 Jonathan Druart 2016-02-16 13:53:33 UTC
Could you export the record and attach it to the bug report.
Then execute, in your sql cli:
  SELECT * FROM biblio WHERE biblionumber=YOUR_BIBLIONUMBER
and post the result here.
Comment 5 Héctor Eduardo Castro Avalos 2016-02-16 21:19:37 UTC
082 field isn't indexed.

Line 164 to 168 in ccl.properties

#Classification-Dewey   13  A classification number from    082
#                           the Dewey Decimal
#                           Classification, developed by
#                           Melvyl Dewey.
#Dewey-classification 1=13

When you search under Callnumber in advanced search the index used is 952$o callnum or Local-classification

Like Jonathan comment maybe you have a date starting with 000-00 in another 952 field or Holding field.

I recommend you search in all bibliographic record (Leader/008) and holding data to see if you have invalid dates


Saludos Humberto
Comment 6 Humberto Blanco 2016-02-17 05:09:51 UTC
(In reply to Jonathan Druart from comment #4)
> Could you export the record and attach it to the bug report.
> Then execute, in your sql cli:
>   SELECT * FROM biblio WHERE biblionumber=YOUR_BIBLIONUMBER
> and post the result here.

Jonathan, I used:
SELECT  *  FROM biblio inner join biblioitems  using  (biblionumber) WHERE cn_class like '340.11'

I attached the output
Comment 7 Humberto Blanco 2016-02-17 05:12:42 UTC
Created attachment 48153 [details]
select cn_class
Comment 8 Humberto Blanco 2016-02-17 05:47:01 UTC
in the apache2 log found thiswhen the error exist
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 168, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package C4::Auth line 399, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 386, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 394, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 402, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 406, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 416, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 419, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:25:53 2016] -e: CGI::param called in list context from package ModPerl::ROOT::ModPerl::Registry::usr_share_koha_intranet_cgi_2dbin_catalogue_search_2epl line 420, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
[Wed Feb 17 00:26:03 2016] -e: CGI::param called in list context from package C4::Auth line 399, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436, <DATA> line 751.
Comment 9 Humberto Blanco 2016-02-17 05:58:33 UTC
(In reply to Héctor Eduardo Castro Avalos from comment #5)
> 082 field isn't indexed.
> 
> Line 164 to 168 in ccl.properties
> 
> #Classification-Dewey   13  A classification number from    082
> #                           the Dewey Decimal
> #                           Classification, developed by
> #                           Melvyl Dewey.
> #Dewey-classification 1=13
> 
> When you search under Callnumber in advanced search the index used is 952$o
> callnum or Local-classification
> 
> Like Jonathan comment maybe you have a date starting with 000-00 in another
> 952 field or Holding field.
> 
> I recommend you search in all bibliographic record (Leader/008) and holding
> data to see if you have invalid dates
> 
> 
> Saludos Humberto

those happend after migration to koha 3.22, any idea  for one more detailed log??
Comment 10 Humberto Blanco 2016-02-17 15:26:54 UTC
This morning i have the same error when do this search http://catalogo.urosario.edu.co/cgi-bin/koha/opac-search.pl?idx=kw&q=vivo&offset=80&sort_by=relevance_dsc   

and get the error 
Software error:

Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /usr/share/koha/lib/Koha/DateUtils.pm line 223.


In the QueryWeightFields KOHA preferences set to disabled.

and the query works fine.

but in the staff i have the same error in advanced search.
Comment 11 Jonathan Druart 2016-02-17 15:37:36 UTC
I am sorry but I don't have enough information to help you.
On a test server, you could try to modify Koha::DateUtils::output_pref with:

+++ b/Koha/DateUtils.pm
@@ -220,6 +220,7 @@ sub output_pref {
     return unless defined $dt;
 
     # FIXME: see bug 13242 => no TZ for dates 'infinite'
+    use Data::Dumper;warn Dumper caller;
     if ( $dt->ymd !~ /^9999/ ) {
         my $tz = $dateonly ? DateTime::TimeZone->new(name => 'floating') : C4::Context->tz;
         $dt->set_time_zone( $tz );

Looking at the logs, you will know which routines has called it just before crashing.
Comment 12 Héctor Eduardo Castro Avalos 2016-02-17 15:38:26 UTC
Your records look good, Humberto.

I could see that in my records I don't have biblioitems.cn_sort with any data populated on it. And you got 340_110000000000000 in all records

Maybe Jonathan knows more about it, He got a lot of experience in Koha

Regards
Comment 13 Humberto Blanco 2016-02-18 17:12:11 UTC
Today review and found that nothing is happend if i found "topographic signature" 340.1 or 340.12
or failed when search some terms that use spaces in the url the system puts + to do this and the problem appears like indexes
for example 
in advanced search :
(idx=kw%2Cphr, q=la+informalidad+laboral)
http://catalogo.urosario.edu.co:8080/cgi-bin/koha/catalogue/search.pl?idx=kw%2Cphr&q=la+informalidad+laboral&op=and&idx=kw&op=and&idx=kw&sort_by=relevance


In general search failed 
http://catalogo.urosario.edu.co:8080/cgi-bin/koha/catalogue/search.pl?q=la+informalidad+laboral
Comment 14 Galen Charlton 2016-02-24 18:34:01 UTC
In haste: I ran into this today and have diagnosed at least one cause of it.  Suppose that the search results contain an item where item.onloan is set to '0000-00-00', not NULL.  In that case, the following line in C4/Search.pm

$onloan_items->{$key}->{due_date} = output_pref( { dt => dt_from_string( $item->{onloan} ), dateonly => 1 } );

ends up calling output_pref with the following parameters:

{ dt => 'dateonly', 1 => undef }

Changing line 50 in Koha/DateUtils.pm from

    return if $date_string and $date_string =~ m|^0000-0|;

to

    return undef if $date_string and $date_string =~ m|^0000-0|;

works around that oddity.
Comment 15 Humberto Blanco 2016-02-25 17:04:52 UTC
(In reply to Galen Charlton from comment #14)
> In haste: I ran into this today and have diagnosed at least one cause of it.
> Suppose that the search results contain an item where item.onloan is set to
> '0000-00-00', not NULL.  In that case, the following line in C4/Search.pm
> 
> $onloan_items->{$key}->{due_date} = output_pref( { dt => dt_from_string(
> $item->{onloan} ), dateonly => 1 } );
> 
> ends up calling output_pref with the following parameters:
> 
> { dt => 'dateonly', 1 => undef }
> 
> Changing line 50 in Koha/DateUtils.pm from
> 
>     return if $date_string and $date_string =~ m|^0000-0|;
> 
> to
> 
>     return undef if $date_string and $date_string =~ m|^0000-0|;
> 
> works around that oddity.

Works for me, i probe son queries and works fine, thanks Galen
Comment 16 Jonathan Druart 2016-11-22 10:10:32 UTC
I do not recreate, but the correct fix would be:

diff --git a/C4/Search.pm b/C4/Search.pm
index 4d5c0ed..235d49b 100644
--- a/C4/Search.pm
+++ b/C4/Search.pm
@@ -2084,7 +2084,7 @@ sub searchResults {
             {
                 $onloan_count++;
                 my $key = $prefix . $item->{onloan} . $item->{barcode};
-                $onloan_items->{$key}->{due_date} = output_pref( { dt => dt_from_string( $item->{onloan} ), dateonly => 1 } );
+                $onloan_items->{$key}->{due_date} = output_pref( { str => $item->{onloan}, dateonly => 1 } );
                 $onloan_items->{$key}->{count}++ if $item->{$hbranch};
                 $onloan_items->{$key}->{branchname}     = $item->{branchname};
                 $onloan_items->{$key}->{location}       = $shelflocations->{ $item->{location} };

Lowering the severity as it does not seem to affect a lot of people.
Please confirm that this change fixes the issue.
Comment 17 Jonathan Druart 2016-11-23 10:11:11 UTC
Created attachment 57725 [details] [review]
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00

The calls
  output_pref({ dt => dt_from_string( $date ) })
are wrong and should be replaced with
  output_pref({ str => $date })
For better error handling.

Here we fix the problem of items.onloan when searching

Test plan:
- Set items.onloan=0000-00-00 (UPDATE items SET onloan='0000-00-00')
This can come from old data or bad migration
- Execute a search
=> Without this patch you get
Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /home/vagrant/kohaclone/Koha/DateUtils.pm line 225.
=> With this patch you won't get the error
Comment 18 Alex Buckley 2016-11-30 23:19:56 UTC
Created attachment 57849 [details] [review]
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00

The calls
  output_pref({ dt => dt_from_string( $date ) })
are wrong and should be replaced with
  output_pref({ str => $date })
For better error handling.

Here we fix the problem of items.onloan when searching

Test plan:
- Set items.onloan=0000-00-00 (UPDATE items SET onloan='0000-00-00')
This can come from old data or bad migration
- Execute a search
=> Without this patch you get
Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /home/vagrant/kohaclone/Koha/DateUtils.pm line 225.
=> With this patch you won't get the error

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Comment 19 Nick Clemens 2016-12-05 17:13:24 UTC
Created attachment 57985 [details] [review]
Bug 15822: Fix 500 error when searching if items.onloan=0000-00-00

The calls
  output_pref({ dt => dt_from_string( $date ) })
are wrong and should be replaced with
  output_pref({ str => $date })
For better error handling.

Here we fix the problem of items.onloan when searching

Test plan:
- Set items.onloan=0000-00-00 (UPDATE items SET onloan='0000-00-00')
This can come from old data or bad migration
- Execute a search
=> Without this patch you get
Can't locate object method "ymd" via package "dateonly" (perhaps you forgot to load "dateonly"?) at /home/vagrant/kohaclone/Koha/DateUtils.pm line 225.
=> With this patch you won't get the error

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 20 Kyle M Hall 2016-12-09 16:22:23 UTC
Pushed to master for 17.05, thanks Jonathan!
Comment 21 Katrin Fischer 2016-12-12 22:10:23 UTC
This patch has been pushed to 16.11.x, will be in 16.11.01.
Comment 22 Julian Maurice 2017-01-02 10:22:02 UTC
Pushed to 3.22.x for 3.22.14
Comment 23 Mason James 2017-01-29 11:38:25 UTC
Pushed to 16.05.x, for 16.05.06 release