Bug 39630 - Problems with search for dateaccessioned (ElasticSearch)
Summary: Problems with search for dateaccessioned (ElasticSearch)
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-04-14 16:36 UTC by trevor.diamond
Modified: 2025-10-06 14:11 UTC (History)
1 user (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 39630: WIP - Index 'Date' fields as a date type in Elasticsearch (2.00 KB, patch)
2025-10-03 14:00 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description trevor.diamond 2025-04-14 16:36:09 UTC
Starting in advanced search, if you select Acquisition date as your search qualifer and enter a date (ex: 2025-03-25) you get more results than expected.  If you "search with results" by limiting to Acquisition date, the results refresh with much more reasonable results.  The advanced search code is as follows:

523 result(s) found for '2025-03-25' with limit(s): 'date-of-acquisition:("2025-03-25")'.

Which indicates that your initial selection of Acq date didn't actually "stick".  If you click "edit this search" you are returned to the advanced search screen which shows a keyword search of 2025-03-25, again showing taht your initial selection of acq date isn't being remembered.

If you try this series of steps with a different filter (ex. Publication year), you do not see this kind of behavior (the results are correctly limited in the first pass and going back to edit the search shows the selection of Publication Year for your search qualifier).

This seems very similar to the activity in Bug 17815 ( https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17815 ) but I saw Zebra mentioned several times there and it was suggested that I open a new bug for elasticsearch. 

The solution would be for Advanced search to actual utilize and remember the search qualifer when acquisition date is selected.
Comment 1 Nick Clemens (kidclamp) 2025-10-02 14:56:05 UTC
So to search for a specific date, the syntax should be:

date-of-acquisition,st-date-normalized=2015-01-05

And that works in my testing.

Ranges don't seem to work:
date-of-acquisition,st-date-normalized=[2015-01-05 0 2016-01-05]

I think because of escaping as what is passed is:
'(date-of-acquisition:\\[2015-01-05 2016-01-05\\])'

and those slashes may be the issue
Comment 2 Nick Clemens (kidclamp) 2025-10-03 14:00:17 UTC
Created attachment 187397 [details] [review]
Bug 39630: WIP - Index 'Date' fields as a date type in Elasticsearch

This patch adds a 'date' format to the field_config and makes sure this is used for fields marked as
Date in the ES config

Currently 'Date' fields are indexed as standard and get keyword and text variations in ES - these field types cannot
be used for range searching, so, will we try to build range queries correctly, they will not return results.

TO DO:
We currently escape the brackets when building a range query, i.e.:
'(date-of-acquisition:\\[2015-01-05 2016-01-05\\])'

This does not work, we need to pass the brackets as brackets to allow ranges to work correctly
Comment 3 Nick Clemens (kidclamp) 2025-10-06 13:13:19 UTC
This is actually worse than initially expected, as the field is not being indexed as a date it is tokenizing each part of the date, year, day, month, as a separate token.

This means a search for acqdate:2025-09-28 matches a record with three items with accessiondates: 2025-01-01, 2024-09-01, 2023-01-28
Comment 4 trevor.diamond 2025-10-06 14:11:10 UTC
(In reply to Nick Clemens (kidclamp) from comment #3)
> This is actually worse than initially expected, as the field is not being
> indexed as a date it is tokenizing each part of the date, year, day, month,
> as a separate token.
> 
> This means a search for acqdate:2025-09-28 matches a record with three items
> with accessiondates: 2025-01-01, 2024-09-01, 2023-01-28

Does this default "OR" operator for the tokens apply to all the searches from the advanced search?  Just this one?