Bug 15537

Summary: Hide records on Leader 05 = d in OPAC
Product: Koha Reporter: David Cook <dcook>
Component: SearchingAssignee: Bugs List <koha-bugs>
Status: RESOLVED WISHLIST QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: gmcharlt, hector.hecaxmmx, mtj, rickx1, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11084
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 15537 - Hide records on Leader 05 = d in OPAC (follow-up)
Bug 15537 - Hide records on Leader 05 = d in OPAC
Bug 15537 - Hide records on Leader 05 = d in OPAC
Bug 15537 - Hide records on Leader 05 = d in OPAC (follow-up)

Description David Cook 2016-01-11 01:37:23 UTC
At the moment, we hide "suppressed" records in the OPAC when they have a 942$n of 1 and when the OpacSuppression system preference is enabled.

Why don't we also hide "deleted" records with a Leader position 05 of "d"? 

In the past, "OpacSuppression" would cause problems if there wasn't at least 1 record suppressed, but I fixed that with http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15198. If we use the special attribute @attr 14=1 in the query (especially globally at the start of the query), we deactivate that problem in Zebra.

So we could hide "deleted" records from the OPAC. 

We probably want to still show them in the staff client, so that they're findable. While a cronjob for removing these records has been added in master (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11084), that cronjob might not always be run consistently by all Koha systems.

Plus, it would be good to have that record hidden immediately from OPAC users, even if the record hasn't been fully deleted from the system yet.
Comment 1 David Cook 2016-01-11 01:39:13 UTC
All we'd need to do is change biblio-zebra-indexdefs.xml/xsl so that we're indexing LDR05, add "@attr 14=1 @not record_status=d" at the start of all OPAC queries (ie where we already have OPAC Suppression code), and we'd be fine.
Comment 2 David Cook 2016-01-12 02:25:55 UTC Comment hidden (obsolete)
Comment 3 David Cook 2016-01-12 02:26:44 UTC Comment hidden (obsolete)
Comment 4 David Cook 2016-01-12 02:28:39 UTC
Created attachment 46510 [details] [review]
Bug 15537 - Hide records on Leader 05 = d in OPAC

Currently, if a bibliographic record has a record status of "deleted"
(ie LDR05=d), it shows up in the catalogue.

This patch indexes the LDR05 character into the Record-status
index (already defined in bib1.att). It creates a CCL qualifier
for that index, and it adds a condition to OPAC queries where
it only returns records that do not have a LDR05 of "d".

Test Conditions:

1) With QueryParser
2) Without QueryParser
3) With OpacSuppression
4) Without OpacSuppression

Test Plan:

Before applying patch:
1) Choose a bibliographic record and change the LDR05 to "d"
2) Try searching for that record in the OPAC
3) Note that it appears in the search results (or takes you directly to the record)

Apply patch.

After applying patch:
4) Try searching for that record in the OPAC
5) Note that the record doesn't appear in the search results
6) Re-try steps 4 & 5 with each of the 4 test conditions outlined above.
It should work the same regardless of using the QueryParser or OpacSuppression.

NOTE: When using the QueryParser, you might need to vary your syntax. For instance,
w/ QP you'd need "local-number:492 || local-number:495" whereas w/o QP you'd need
"local-number:492 or local-number:495".

NOTE: For extra points, double-check the staff client to make sure that you
can still find records that have a LDR05 of "d".
Comment 5 David Cook 2016-01-12 02:28:44 UTC
Created attachment 46511 [details] [review]
Bug 15537 - Hide records on Leader 05 = d in OPAC (follow-up)

This follow-up adds a warning message on the detail page in the staff client,
which says that the record will not appear in the OPAC.

It also includes some logic in opac-detail.pl which prevents an OPAC
user from visiting it directly.

Test Plan:

Apply the patch.

1) Visit a staff client page for a bib record with LDR05=d.
2) Note that there is a warning message saying that it will not be
visible in the OPAC
3) Visit the corresponding OPAC page for the bib record
4) Note that you get a redirect to a 404.
Comment 6 David Cook 2016-01-12 02:29:42 UTC
NOTE: I have not tested this in UNIMARC or NORMARC, although I've provided Zebra changes for both. They should be OK, but I haven't confirmed that to be the case. I have only tested MARC21.
Comment 7 David Cook 2016-01-12 02:39:25 UTC
ADDITIONAL TESTING NOTE:

You'll need to re-index Zebra before this feature works... otherwise the Record-status index won't be populated...
Comment 8 David Cook 2016-01-12 04:46:44 UTC
ADDITIONAL TESTING NOTE: 

Here's the steps you need to do to upgrade your Koha's Zebra config, if you're using a git dev install:

cd /home/dcook/git
INSTALL_LOG="/home/dcook/koha-dev/misc/koha-install-log"
cp $INSTALL_LOG backup-koha-install-log
perl Makefile.PL --prev-install-log "$INSTALL_LOG"
make
make upgrade
Comment 9 Galen Charlton 2016-01-12 13:48:56 UTC
[copying feedback I posted to koha-devel]

On Sun, Jan 10, 2016 at 8:44 PM, David Cook <dcook@prosentient.com.au> wrote:
> We can’t necessarily rely on all Koha instances running this cronjob, nor
> can we rely on the frequency. Shouldn’t we be hiding these records from the
> OPAC as soon as they’re marked as “deleted”?
>
> I’ve opened a bug for this purpose:
> http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15537

I am in mild disfavor of this proposal, particularly as implemented in current patch. Using a cronjob to delete records where Leader/05 is set to 'd' is useful when the library has arranged their workflow such that they *know* that Leader/05 = 'd' is being used consistently to signify that a record is no longer wanted. However, for a library that has not hitherto cared about the values in that position, unconditionally suppressing the display of such records could come as an unwelcome surprise.

That said, it is also a reasonable choice for a library to want to use the Leader/05 as suppression criterion.  Consequently, I suggest adding a configuration option.  For that matter, making it configurable (say, by allowing the library to specify a set of query additions for the purpose of filtering records from public display) could result in a more generally useful mechanism.

> I admit that I have a special interest in this where I might 
> be overlaying existing records using a mostly empty skeleton
>  record generated from an OAI-PMH identifier and a OAI-PMH
> deleted status (OAI-PMH doesn’t send metadata for deleted records).
>  I’d match the existing record in Koha using the identifier, and 
> then set LDR05 to “d” in accordance with the OAI-PMH deleted 
> status. Then, that record would disappear from the OPAC, so that
> end users don’t see this skeleton record.

I do not find this a compelling use case as stated.  If the goal is to allow harvesting and overlay records from an OAI-PMH provider to also delete bibs from a Koha database... coding so that the records are *actually* deleted seems more direct.
Comment 10 Héctor Eduardo Castro Avalos 2016-01-13 05:29:33 UTC
Galen is right you must add an configurable option to leave librarians to choose if they want this feature working in their libraries or not. IMHO
Comment 11 David Cook 2016-01-18 04:41:04 UTC
Hmm, I don't seem to be getting the emails about comments...

I agree that I should add a system preference in this case.

I've also followed up Galen's emails on koha-devel regarding OAI-PMH, and I think I'm going to go a different route, which makes this bug and its patches irrelevant for bug 10662. 

Resetting this bug to assigned, although I'm in no rush to finish this bug due to the change in direction of bug 10662.
Comment 12 Ricardo Ruiz 2017-07-09 00:27:48 UTC
Hello david,

In order for it to work I had to add Record-status:w:range(data,5,1) in the record.abs file (/etc/koha/zebradb/marc_defs/marc21/biblios/)

Great job, thanks
Comment 13 David Cook 2017-07-13 03:09:36 UTC
(In reply to Ricardo Ruiz from comment #12)
> Hello david,
> 
> In order for it to work I had to add Record-status:w:range(data,5,1) in the
> record.abs file (/etc/koha/zebradb/marc_defs/marc21/biblios/)
> 
> Great job, thanks

Yikes... that means that you're still using GRS1 indexing which is deprecated (or possibly even unavailable) in newer versions. What version of Koha are you running?
Comment 14 Ricardo Ruiz 2017-07-27 22:49:27 UTC
(In reply to David Cook from comment #13)
> (In reply to Ricardo Ruiz from comment #12)
> > Hello david,
> > 
> > In order for it to work I had to add Record-status:w:range(data,5,1) in the
> > record.abs file (/etc/koha/zebradb/marc_defs/marc21/biblios/)
> > 
> > Great job, thanks
> 
> Yikes... that means that you're still using GRS1 indexing which is
> deprecated (or possibly even unavailable) in newer versions. What version of
> Koha are you running?

You are right, the user uses an old version of Koha 3.18.10, with the indexing option GRS1.
I hope the user will soon update their version of Koha.
Greetings.
Comment 15 David Cook 2017-07-28 00:43:22 UTC
(In reply to Ricardo Ruiz from comment #14)
> You are right, the user uses an old version of Koha 3.18.10, with the
> indexing option GRS1.
> I hope the user will soon update their version of Koha.
> Greetings.

Sounds like a plan!

Although I wouldn't implement this patch if I were you. It hasn't been accepted into Koha yet.

You could test and sign off the patch, and then maybe the QA people will take a look at it. 

See: https://wiki.koha-community.org/wiki/Git_bz_configuration#Signing_off
Comment 16 Tomás Cohen Arazi 2018-10-09 12:52:57 UTC
David, what's the status for this one?
Comment 17 David Cook 2018-10-09 23:14:55 UTC
(In reply to Tomás Cohen Arazi from comment #16)
> David, what's the status for this one?

I'd say it's abandoned. I don't have any plans for working on it further. 

I don't recall if I had anyone specifically asking for this back in 2016 or if it was just out of my own interest.
Comment 18 David Cook 2022-12-06 06:41:56 UTC
Closing this due to age