Improving opac-search performances by caching the results of C4::XSLT::transformMARCXML4XSLT. This also impact the performances of C4::XSLT::XSLTParse4Display. My tests (with Devel::NYTProf) showed a gain of about 65% on C4::XSLT::transformMARCXML4XSLT and 35% on C4::XSLT::XSLTParse4Display. The gain of performances is increasing with the numbers of results. Examples: -------- 1/ searching : "sinatra", results : 2 gain : C4::XSLT::transformMARCXML4XSLT = 40% C4::XSLT::XSLTParse4Display = 19% 2/ searching : "Victor Hugo", results : 16 gain : C4::XSLT::transformMARCXML4XSLT = 71% C4::XSLT::XSLTParse4Display = 36% 3/ searching : "a", results : 2060 gain : C4::XSLT::transformMARCXML4XSLT = 82% C4::XSLT::XSLTParse4Display = 51%
Created attachment 40740 [details] [review] bug 14476 : Caching C4::XSLT::transformMARCXML4XSLT This patch is improving the opac-search performances by caching the results of C4::XSLT::transformMARCXML4XSLT My tests (with Devel::NYTProf) showed a gain of about 65% on C4::XSLT::transformMARCXML4XSLT and 35% on C4::XSLT::XSLTParse4Display. The gain of performances is increasing with the numbers of results.
Created attachment 42179 [details] [review] [SIGNED-OFF] bug 14476 : Caching C4::XSLT::transformMARCXML4XSLT This patch is improving the opac-search performances by caching the results of C4::XSLT::transformMARCXML4XSLT My tests (with Devel::NYTProf) showed a gain of about 65% on C4::XSLT::transformMARCXML4XSLT and 35% on C4::XSLT::XSLTParse4Display. The gain of performances is increasing with the numbers of results. Signed-off-by: Gaetan Boisson <gaetan.boisson@biblibre.com>
Works for me, although the performance gain was not as big as expected. I will test it again on a larger data set.
I have a concern: if we cache the record to be displayed, then the item statuses won't be up-to-date. If the tradeoff is not enough, maybe this is just not worth. Because people expect to see updated information on the records.
(In reply to Tomás Cohen Arazi from comment #4) > I have a concern: if we cache the record to be displayed, then the item > statuses won't be up-to-date. If the tradeoff is not enough, maybe this is > just not worth. Because people expect to see updated information on the > records. Indeed. I think we need to change this by moving items out of this XSLT file. I think we should use Ajax to load in realtime items infos, with and API for that. This will also avoid indexing all record on item circulation. Dreams ...
(In reply to Fridolin SOMERS from comment #5) > (In reply to Tomás Cohen Arazi from comment #4) > > I have a concern: if we cache the record to be displayed, then the item > > statuses won't be up-to-date. If the tradeoff is not enough, maybe this is > > just not worth. Because people expect to see updated information on the > > records. > > Indeed. > > I think we need to change this by moving items out of this XSLT file. > I think we should use Ajax to load in realtime items infos, with and API for > that. > This will also avoid indexing all record on item circulation. > Dreams ... I often think using Ajax to load real time item info is a good idea too, although I think we'd still have the indexing issue because of the "limit to currently available items" filter on the search. Often, I think we should think about our OPAC as if it were a totally separate system, and just link it up to the Koha instance using APIs.
(In reply to Fridolin SOMERS from comment #5) > (In reply to Tomás Cohen Arazi from comment #4) > > I have a concern: if we cache the record to be displayed, then the item > > statuses won't be up-to-date. If the tradeoff is not enough, maybe this is > > just not worth. Because people expect to see updated information on the > > records. > > Indeed. > > I think we need to change this by moving items out of this XSLT file. > I think we should use Ajax to load in realtime items infos, with and API for > that. I agree! Let's have that endpoint! > This will also avoid indexing all record on item circulation. Not sure about it, but we could certainly index items and make JOIN operations in our queries ;-) (for limits, as David also suggests).
(In reply to Tomás Cohen Arazi from comment #7) > (In reply to Fridolin SOMERS from comment #5) > > This will also avoid indexing all record on item circulation. > > Not sure about it, but we could certainly index items and make JOIN > operations in our queries ;-) (for limits, as David also suggests). I was totally thinking about that, as I was thinking about Solr query joins. I imagine ElasticSearch must be able to do the same thing, although I don't think Zebra could. Or did you have something in mind for JOIN operations?
(In reply to David Cook from comment #8) > (In reply to Tomás Cohen Arazi from comment #7) > > (In reply to Fridolin SOMERS from comment #5) > > > This will also avoid indexing all record on item circulation. > > > > Not sure about it, but we could certainly index items and make JOIN > > operations in our queries ;-) (for limits, as David also suggests). > > I was totally thinking about that, as I was thinking about Solr query joins. > I imagine ElasticSearch must be able to do the same thing, although I don't > think Zebra could. Or did you have something in mind for JOIN operations? I was thinking ES. Zebra support is in freeze mode. And I'm sure you cannot mix different databases on a query.
(In reply to Tomás Cohen Arazi from comment #9) > Zebra support is in freeze mode. That's good to hear. I've been wondering how long we'd try to support both. I look forward to a time where we're just focusing on one.
As far as I know there has been no official decision on Zebra. There are still Zebra bugs being worked on.
Since caching the XSLT result is not desirable, should we close this bug ?
(In reply to Julian Maurice from comment #12) > Since caching the XSLT result is not desirable, should we close this bug ? No answers in 3 months, so I'm changing status to WONTFIX. Feel free to reopen if you are willing to work on this.