Bug 31035

Summary: Script to update OPACSuppression field depending on hidden items
Product: Koha Reporter: Fridolin Somers <fridolin.somers>
Component: Command-line UtilitiesAssignee: Fridolin Somers <fridolin.somers>
Status: ASSIGNED --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: dcook, pierre.genty, robin
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24403
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 31035: Script to update OPACSuppression field depending on hidden items
Bug 31035: Script to update OPACSuppression field depending on hidden items
Bug 31035: Script to update OPACSuppression field depending on hidden items
Bug 31035: (follow-up) handle broken MARCXML

Description Fridolin Somers 2022-06-23 23:43:25 UTC
System preference OPACSuppression enables to skip from search results biblio records with a dedicated search field (usually on 942$n).

We should have a script to update this MARC field depending if all items of the record are hidden (OpacHiddenItems and hidelostitems).

This is independant from OpacHiddenItemsHidesRecord Bug 18989 that has known issues : Bug 17787, Bug 23923 ... See Bug 24403 dependencies
Comment 1 Fridolin Somers 2022-06-24 00:13:51 UTC
Created attachment 136466 [details] [review]
Bug 31035: Script to update OPACSuppression field depending on hidden items

WIP
Comment 2 David Cook 2022-06-24 00:46:26 UTC
This script would break the functionality introduced by the syspref "OpacHiddenItemsExceptions" though, unfortunately.

I thought about this a bunch while working on bug 19704. 

In theory, you could enqueue a background job, when "OpacHiddenItems" is saved, which could process every record with items and then update a "biblio.hidden" field (so that it can be queried for SQL searches and not just Search Engine searches), if "OpacHiddenItemsHidesRecord" is set. 

--

Maybe the solution is to get rid of "OpacHiddenItemsExceptions" and then either use a cronjob or a background job to update the 942$n/biblio.hidden.

It would be great to be able to eliminate the post-processing of records/items caused by "OpacHiddenItems", so generally speaking I'm on board for what you're trying to do!
Comment 3 Fridolin Somers 2022-06-24 06:36:51 UTC
(In reply to David Cook from comment #2)
> This script would break the functionality introduced by the syspref
> "OpacHiddenItemsExceptions" though, unfortunately.
Indeed I will add this to doc.

> 
> I thought about this a bunch while working on bug 19704. 
> 
> In theory, you could enqueue a background job, when "OpacHiddenItems" is
> saved, which could process every record with items and then update a
> "biblio.hidden" field (so that it can be queried for SQL searches and not
> just Search Engine searches), if "OpacHiddenItemsHidesRecord" is set.
Sure, a data field in biblio whould be really great.
Maybe biblio.opac_hidden.
 
> 
> --
> 
> Maybe the solution is to get rid of "OpacHiddenItemsExceptions" and then
> either use a cronjob or a background job to update the 942$n/biblio.hidden.
Indeed the complexity is hard with this feature.
We should ask who uses it.

> 
> It would be great to be able to eliminate the post-processing of
> records/items caused by "OpacHiddenItems", so generally speaking I'm on
> board for what you're trying to do!

With this script actually you can not manually hide on purpose a biblio record.
I was thinking we could define a special value for the OPACSuppress field, like "2".
Value "1" would be used to manually hide biblio record.
Comment 4 Fridolin Somers 2022-06-24 07:00:18 UTC
Created attachment 136472 [details] [review]
Bug 31035: Script to update OPACSuppression field depending on hidden items
Comment 5 Fridolin Somers 2022-06-24 07:01:08 UTC
Script it still WIP but seems it works.

You may test it.
Comment 6 Fridolin Somers 2022-12-21 19:02:23 UTC
Created attachment 144793 [details] [review]
Bug 31035: Script to update OPACSuppression field depending on hidden items
Comment 7 Fridolin Somers 2023-08-25 22:04:46 UTC
Created attachment 154852 [details] [review]
Bug 31035: (follow-up) handle broken MARCXML

Plus show progress each 1000 records
Comment 8 Fridolin Somers 2023-08-25 22:07:11 UTC
Currently we use this script but not for OpacSuppress feature in Koha's OPAC.

The script is used to set special field 099$o to 1 when all items are hidden.
This allows to define an OAI set with 099$o != 1.
This set is used for portal import of records, skipping unwanted records.

May also be used to filter records export to portal.