Bug 19635 - OAI-PMH deleted records are now default and are listed first
Summary: OAI-PMH deleted records are now default and are listed first
Status: CLOSED WONTFIX
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on: 20271
Blocks:
  Show dependency tree / graph
 
Reported: 2017-11-16 12:00 UTC by Nick Clemens (kidclamp)
Modified: 2020-01-06 20:15 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2017-11-16 12:00:45 UTC 
It seems bug 15108 made some changes to which records are served up first by the OAI server.

When making a basic request with no token, i.e.:
http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords&metadataPrefix=marcxml

You get any deleted records first.

When continuing if you have more than 50 deleted records the resumptionToken also includes deleted records

We have services harvesting the entire catalog that don't expect deleted records first.

I am not seeing a way to prevent them, and wonder if they should not be served first.
Comment 1 Ere Maijala 2017-11-16 12:51:45 UTC 
As far as I know there's definitely nothing wrong in returning the deleted records first. Can you explain in a bit more detail why that is an issue? A harvester should just harvest until a resumptionToken is not returned, and there's no promise in the protocol that the records are returned in any particular order.
Comment 2 Nick Clemens (kidclamp) 2017-11-16 13:24:49 UTC 
(In reply to Ere Maijala from comment #1)
> As far as I know there's definitely nothing wrong in returning the deleted
> records first. Can you explain in a bit more detail why that is an issue? A
> harvester should just harvest until a resumptionToken is not returned, and
> there's no promise in the protocol that the records are returned in any
> particular order.

I think this came up because we have a site with more deleted than existing records, this slowed the harvest and they thought something was wrong because all records were returning deleted.

It may not be 'wrong' but I think it is a change - is there any benefit to returning deleted first? should/could it be a syspref?
Comment 3 Ere Maijala 2017-11-16 13:28:39 UTC 
(In reply to Nick Clemens from comment #2)
> I think this came up because we have a site with more deleted than existing
> records, this slowed the harvest and they thought something was wrong
> because all records were returning deleted.
> 
> It may not be 'wrong' but I think it is a change - is there any benefit to
> returning deleted first? should/could it be a syspref?

There's no real benefit but there's no downside either as far as I can see. Since the harvester needs to harvest everything, it doesn't matter if the deleted records come first or last or interleaved with non-deleted records. I'd be fine with it being a syspref, but I still can't see a real reason for that.
Comment 4 Ere Maijala 2019-03-04 08:10:39 UTC 
Note that this will change when normal and deleted records get merged into a single table. That's bug 20271. I'd like to close this one as
Comment 5 Ere Maijala 2019-03-04 08:11:18 UTC 
... no changes are needed here, but we can also keep this blocked for now.
Comment 6 Nick Clemens (kidclamp) 2019-03-04 13:20:03 UTC 
I am okay with closing, once service got used to the change things have been working fine. Cosmetic not functional issue