(cc of a mail sent to koha-devel on 2011-09-12) The loading of the template seems to cost 150ms, and I feel that those 2 options could be usefull to speed up the process. Did anyone already saw this option before ? Is it something worth investigating more ? Here is the documentation of the 1st option: COMPILE_EXT >From version 2 onwards, the Template Toolkit has the ability to compile templates to Perl code and save them to disk for subsequent use (i.e. cache persistence). The COMPILE_EXT option may be provided to specify a filename extension for compiled template files. It is undefined by default and no attempt will be made to read or write any compiled template files. my $template = Template->new({ COMPILE_EXT => '.ttc', }); If COMPILE_EXT is defined (and COMPILE_DIR isn't, see below) then compiled template files with the COMPILE_EXT extension will be written to the same directory from which the source template files were loaded. I made some tests and the results are great: NYTprof comparison: WITHOUT caching: Profile of mainpage.pl for 2.18s (of 2.79s), executing 199503 statements and 60033 subroutine calls in 279 source files and 81 string evals. WITH caching: Profile of mainpage.pl for 1.86s (of 2.27s), executing 131771 statements and 47501 subroutine calls in 281 source files and 73 string evals. The same kind of improvements can be seen with misc/load_testing/benchmark_staff.pl WITHOUT caching: Benchmarking with 20 occurences of each operation and 30 concurrent sessions Step 1: staff client main page 23465ms 0.852 pages/sec Step 2: catalog detail page 34507ms 0.579 biblios/sec Step 3: catalogue search 26909ms 0.743 biblios/sec Step 5: patron detail page 27872ms 0.717 borrowers/sec Step 5: patron search page 52577ms 0.76 borrowers/sec Step 6a circulation (checkouts) 32309ms 0.619 checkouts/sec Step 6b circulation (checkins) 32959ms 0.606 checkins/sec all transactions at once 103897ms 1.732 operations/sec WITH caching: Benchmarking with 20 occurences of each operation and 30 concurrent sessions Step 1: staff client main page 21286ms 0.939 pages/sec Step 2: catalog detail page 27213ms 0.734 biblios/sec Step 3: catalogue search 22091ms 0.905 biblios/sec Step 5: patron detail page 22015ms 0.908 borrowers/sec Step 5: patron search page 43871ms 0.911 borrowers/sec Step 6a circulation (checkouts) 27601ms 0.724 checkouts/sec Step 6b circulation (checkins) 28860ms 0.693 checkins/sec all transactions at once 87597ms 2.054 operations/sec
Yesterday, I had a long trip in a train, so I investigated more, and wrote a patch. Just a performance thing: I tried to checkout a book, with and without caching, on a copy of a real database. The result is absolutly astonishing: The command run: perl -d:NYTProf circ/circulation.pl userid=test password=test borrowernumber=7852 barcode=00519000000272 Without T::T cache = Profile of circ/circulation.pl for 3.28s (of 4.74s), executing 486760 statements and 88239 subroutine calls in 284 source files and 87 string evals. With T::T cache = Profile of circ/circulation.pl for 2.37s (of 2.83s), executing 144410 statements and 53427 subroutine calls in 293 source files and 74 string evals. The circulation.tt is one of the largest template we have, the result is ... well, use the word you want. I was so surprised that I tried to find a mistake in my test, but could not. without cache = run the command, run nytprofhtml to generate the result page with cache = * run the command once * check that /tmp now has a cached version * run the command again and run nytprofhtml to generate the result page Patch to follow, it's a 2 lines one !
Created attachment 7558 [details] [review] Bug 7511 [ENH] Template caching (T::T feature) Just add to your Koha configuration file <template_cache_dir>/path/writable/by/apache/user</template_cache_dir> and Koha will use template caching, for about 10% CPU time saving. on linux servers, /tmp is usually OK (also fixes an indenting with a TAB)
(In reply to comment #2) > Just add to your Koha configuration file > <template_cache_dir>/path/writable/by/apache/user</template_cache_dir> > and Koha will use template caching, for about 10% CPU time saving. 1- don't forget to reload apache 2- to check that cache now work: (I assume that KOHA_DIR is /home/koha/src and the cache dir is /tmp) there is now a directory /tmp/home/koha/src, with sub directories, owned by apache user I encountered a problem when trying to run NYTProf and caching at the same time: the NYTProf is run as "paul" while Koha run as "apache2". Once you've started running Koha with caching as one of those 2 users, you can't run a script as the other user. This is because of linux permissions rwxr--r-- A little sudo rm -fr /tmp/home fixes this annoying problem if you need to run a script as another user a chmod +w+x could also work, I didn't tried
Created attachment 7589 [details] [review] Bug 7511 [ENH] Template caching (T::T feature) Just add to your Koha configuration file <template_cache_dir>/path/writable/by/apache/user</template_cache_dir> in the <config> block, and Koha will use template caching, for about 10% CPU time saving. on linux servers, /tmp is usually OK (also fixes an indenting with a TAB) Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> The results of using this are very striking. Based on an insufficient sample size, it would seem that the time spent in T::T is reduced by a factor of at least 5.
Safely implements a new <config> option using the features of T:T. Marking Passed QA. Next steps are to add this <config> option to the Makefile.PL so it does not need to be manually edited into existence.
(In reply to comment #5) > Next steps are to add this <config> option to the Makefile.PL so it does not > need to be manually edited into existence. Right. I've added a bug for that, and made it dependant from this one (bug 7533)
There have been no further reports of problems so I am marking this bug resolved.