Summary: | OPACSuppress behavior with Elasticsearch | ||
---|---|---|---|
Product: | Koha | Reporter: | Fridolin Somers <fridolin.somers> |
Component: | Searching - Elasticsearch | Assignee: | Fridolin Somers <fridolin.somers> |
Status: | CLOSED INVALID | QA Contact: | |
Severity: | normal | ||
Priority: | P5 - low | CC: | caroline.cyr-la-rose, david |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
Change sponsored?: | --- | Patch complexity: | Trivial patch |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: | Version(s) released in: | ||
Circulation function: | |||
Bug Depends on: | 23676 | ||
Bug Blocks: | |||
Attachments: |
Bug 31061: Fix test suite
Bug 31061: Change type of suppress in Elasticsearch mappings Bug 31061: OPACSuppress does not work properly with Elasticsearch Bug 31061: OPACSuppress behavior with Elasticsearch |
Description
Fridolin Somers
2022-06-28 19:42:38 UTC
Created attachment 136695 [details] [review] Bug 31061: Fix test suite Created attachment 136696 [details] [review] Bug 31061: Change type of suppress in Elasticsearch mappings Created attachment 136697 [details] [review] Bug 31061: OPACSuppress does not work properly with Elasticsearch OPACSuppress is system preference to hide records in OPAC search by adding a limit in search query. With Zebra, this limit is : not Suppress=1 https://git.koha-community.org/Koha-community/Koha/src/commit/244b847a08bf61a747a1e4be4eed86cc776eabd6/Koha/SearchEngine/Zebra/QueryBuilder.pm#L83 With Elasticsearch this limit is currently quite different : suppress:false https://git.koha-community.org/Koha-community/Koha/src/commit/244b847a08bf61a747a1e4be4eed86cc776eabd6/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm#L286 Main problem is that in default framework we define Suppress search field on 942$n with authorized values YES_NO. This generates records with 1 for YES and 0 for NO. But a boolean for Elasticsearch is not a perl boolean. So only missing subfield means false : https://www.elastic.co/guide/en/elasticsearch/reference/6.8/boolean.html Records with 0 are hidden ! Another side-effect : We whould like to have several MARC fields with 'suppress' search field. To distinguish several causes of hidden at OPAC. Any of those fields having 1 must hide at OPAC. So syntax 'NOT(suppress:1)' is more correct. I propose we change to have the same behavior as Zebra. Change 'suppress' search field to not remove boolean type. Change QueryBuilder to use 'NOT(suppress:1)'. Test plan : 1) Use search engine Elasticsearch 2) Disable system preference OpacSuppression 3) Search in OPAC for all records : '*:*' => Note how many results 4) Enable system preference OpacSuppression 5) Edit a biblio record with 942$n set to YES 6) Search in OPAC for all records : '*:*' => Note how many results, you should have 1 less than 3) 7) Edit a biblio record with 942$n set to NO 8) Search in OPAC for all records : '*:*' => Note how many results, you should have same number as 3) 9) Edit a biblio record to remove 942$n 10) Search in OPAC for all records : '*:*' => Note how many results, you should have same number as 3) I must not be completely awake, but I cannot reproduce the original "problematic" behaviour. Without applying the patch, using ES, the number of records fluctuates according to 942$n being Yes (1) or No (0) and OpacSuppression being on or off. There are 3 records with 1 in 942$n With OpacSuppression = 405 Without OpacSuppression = 408 Ah indeed seems to be a code that convert 'perl boolean' value into true/false : https://git.koha-community.org/Koha-community/Koha/src/commit/6c315c164dbcf3e05fcfe4a3110b824ab3ac162c/Koha/SearchEngine/Elasticsearch.pm#L950 I need to test again Ahhhhh now that I've retested with KTD, indeed it works.
Maybe my search was too faster after unhiding record with 942$n=0.
> But a boolean for Elasticsearch is not a perl boolean.
Ouf its is thanks to a trick in the code.
I will change patches in order to only change to : NOT(suppress:true)
Created attachment 136799 [details] [review] Bug 31061: OPACSuppress behavior with Elasticsearch OPACSuppress is system preference to hide records in OPAC search by adding a limit in search query. With Zebra, this limit is : not Suppress=1 https://git.koha-community.org/Koha-community/Koha/src/commit/244b847a08bf61a747a1e4be4eed86cc776eabd6/Koha/SearchEngine/Zebra/QueryBuilder.pm#L83 With Elasticsearch this limit is currently quite different : suppress:false https://git.koha-community.org/Koha-community/Koha/src/commit/244b847a08bf61a747a1e4be4eed86cc776eabd6/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm#L286 We whould like to have several MARC fields with 'suppress' search field. To distinguish several causes of hidden at OPAC. Any of those fields having 1 must hide at OPAC. Syntax 'NOT(suppress:true)' does that, like with Zebra. Test plan : 1) Use search engine Elasticsearch 2) Disable system preference OpacSuppression 3) Search in OPAC for all records : '*:*' => Note how many results 4) Enable system preference OpacSuppression 5) Edit a biblio record with 942$n set to YES 6) Search in OPAC for all records : '*:*' => Note how many results, you should have 1 less than 3) 7) Edit a biblio record with 942$n set to NO 8) Search in OPAC for all records : '*:*' => Note how many results, you should have same number as 3) 9) Edit a biblio record to remove 942$n 10) Search in OPAC for all records : '*:*' => Note how many results, you should have same number as 3) I had a go at testing using KTD and the sample data. I couldn't replicate the issue before the patch was applied, and there were no differences in the expected results with 943$n set to Yes, No and removed. I've set out what I did below. Before the patch was applied ============================ Using Elasticsearch, index rebuilt (koha-elasticsearch --rebuild -d -b -a kohadev) OpacSuppression = Don't hide: 435 records OpacSuppression = Hide: 433 records Records that are suppressed already in the sample data = 2: biblionumber 232 and 234 (used report to find https://wiki.koha-community.org/wiki/SQL_Reports_Library#Bibs_Suppressed_in_OPAC) Recorded edited with OpacSuppression = Hide: 126 (Intermediate Perl) - 942$n = Yes: records = 432 (as expected) (435 - 2 records already hidden - 1 new record) - 942$n = No: records = 433 (as expected) - 942$n = removed: records = 433 (as expected) Tests pass: prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t After the patch is applied ========================== Using Elasticsearch, index rebuilt (koha-elasticsearch --rebuild -d -b -a kohadev) OpacSuppression = Don't hide: 435 records OpacSuppression = Hide: 433 records Records that are suppressed already in the sample data = 2: biblionumber 232 and 234 (used report to find https://wiki.koha-community.org/wiki/SQL_Reports_Library#Bibs_Suppressed_in_OPAC) Recorded edited with OpacSuppression = Hide: 126 (Intermediate Perl) - 942$n = Yes: records = 432 (as expected) (435 - 2 records already hidden - 1 new record) - 942$n = No: records = 433 (as expected) - 942$n = removed: records = 433 (as expected) Tests pass: prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t > We whould like to have several MARC fields with 'suppress' search field.
We abandonned this idea
Lets keep the actual code ;)
|