Bug 19635

Summary:	OAI-PMH deleted records are now default and are listed first
Product:	Koha	Reporter:	Nick Clemens (kidclamp) <nick>
Component:	Searching	Assignee:	Bugs List <koha-bugs>
Status:	CLOSED WONTFIX	QA Contact:	Testopia <testopia>
Severity:	normal
Priority:	P5 - low	CC:	ere.maijala, olli-antti.kivilahti, tomascohen
Version:	Main
Hardware:	All
OS:	All
Change sponsored?:	---	Patch complexity:	---
Documentation contact:		Documentation submission:
Text to go in the release notes:		Version(s) released in:
Circulation function:
Bug Depends on:	20271
Bug Blocks:

Description Nick Clemens (kidclamp) 2017-11-16 12:00:45 UTC

It seems bug 15108 made some changes to which records are served up first by the OAI server.

When making a basic request with no token, i.e.:
http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords&metadataPrefix=marcxml

You get any deleted records first.

When continuing if you have more than 50 deleted records the resumptionToken also includes deleted records

We have services harvesting the entire catalog that don't expect deleted records first.

I am not seeing a way to prevent them, and wonder if they should not be served first.

Comment 1 Ere Maijala 2017-11-16 12:51:45 UTC

As far as I know there's definitely nothing wrong in returning the deleted records first. Can you explain in a bit more detail why that is an issue? A harvester should just harvest until a resumptionToken is not returned, and there's no promise in the protocol that the records are returned in any particular order.

Comment 2 Nick Clemens (kidclamp) 2017-11-16 13:24:49 UTC

(In reply to Ere Maijala from comment #1)
> As far as I know there's definitely nothing wrong in returning the deleted
> records first. Can you explain in a bit more detail why that is an issue? A
> harvester should just harvest until a resumptionToken is not returned, and
> there's no promise in the protocol that the records are returned in any
> particular order.

I think this came up because we have a site with more deleted than existing records, this slowed the harvest and they thought something was wrong because all records were returning deleted.

It may not be 'wrong' but I think it is a change - is there any benefit to returning deleted first? should/could it be a syspref?

Comment 3 Ere Maijala 2017-11-16 13:28:39 UTC

(In reply to Nick Clemens from comment #2)
> I think this came up because we have a site with more deleted than existing
> records, this slowed the harvest and they thought something was wrong
> because all records were returning deleted.
> 
> It may not be 'wrong' but I think it is a change - is there any benefit to
> returning deleted first? should/could it be a syspref?

There's no real benefit but there's no downside either as far as I can see. Since the harvester needs to harvest everything, it doesn't matter if the deleted records come first or last or interleaved with non-deleted records. I'd be fine with it being a syspref, but I still can't see a real reason for that.

Comment 4 Ere Maijala 2019-03-04 08:10:39 UTC

Note that this will change when normal and deleted records get merged into a single table. That's bug 20271. I'd like to close this one as

Comment 5 Ere Maijala 2019-03-04 08:11:18 UTC

... no changes are needed here, but we can also keep this blocked for now.

Comment 6 Nick Clemens (kidclamp) 2019-03-04 13:20:03 UTC

I am okay with closing, once service got used to the change things have been working fine. Cosmetic not functional issue