Created attachment 135247 [details] [review] Bug 30824: Improve performance of BatchCommitItems This makes two simple changes: - Limit TransformMarcToKoha to the fields we need - Pass forward the biblioitemnumber when adding items to a new biblionumber Profiling with NYTProf I saved ~8-9 seconds importing around 400 bibs/1000 items Reducing calls in item store to use a passed biblionumber was the largest gain. To test: 1 - Import some records and items 2 - Verify values etc., revert 3 - Apply patch 4 - Import again 5 - Verify values etc. same as before
Created attachment 135307 [details] [review] Bug 30824: Improve performance of BatchCommitItems This makes two simple changes: - Limit TransformMarcToKoha to the fields we need - Pass forward the biblioitemnumber when adding items to a new biblionumber Profiling with NYTProf I saved ~8-9 seconds importing around 400 bibs/1000 items Reducing calls in item store to use a passed biblionumber was the largest gain. To test: 1 - Import some records and items 2 - Verify values etc., revert 3 - Apply patch 4 - Import again 5 - Verify values etc. same as before Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 135308 [details] [review] Bug 30824: (follow-up) POD
Strong performance improvement, signing off.
Created attachment 135369 [details] [review] Bug 30824: Improve performance of BatchCommitItems This makes two simple changes: - Limit TransformMarcToKoha to the fields we need - Pass forward the biblioitemnumber when adding items to a new biblionumber Profiling with NYTProf I saved ~8-9 seconds importing around 400 bibs/1000 items Reducing calls in item store to use a passed biblionumber was the largest gain. To test: 1 - Import some records and items 2 - Verify values etc., revert 3 - Apply patch 4 - Import again 5 - Verify values etc. same as before Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Created attachment 135370 [details] [review] Bug 30824: (follow-up) POD Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Makes the code harder to understand but I'll allow it due to the perf improvement. Passing QA.
How does the dependency work, Nick?
(In reply to Tomás Cohen Arazi from comment #8) > How does the dependency work, Nick? It comes from the updated call to TransformMarcToKoha - we could simply use 'items' in the limit_table param to remove it
Blocked by tree.
Blocking patch has been pushed now, setting back to PQA
Pushed to master for 22.11. Nice work everyone, thanks!