Created attachment 6355 [details] [review] git format patch adding report webservice, caching support, and public visibility setting Patch includes parts of https://github.com/ranginui/koha/commit/20cb74d1f5f86ed9fa1e6a1ead29798ff894c685 and https://github.com/ranginui/koha/commit/2a1f4172dafbfb7435e46762fcfab368d3dceb7a It also adds in caching for intranet, setting a cache expiry time for reports, and a public visibility flag to reports. I still have to add a better user input dialog and user validation (patch following soon). Currently the input is just a text field that takes seconds (not ideal for end user) and the validation is a die message (as the timeout cannot exceed 30 days or Memcached::set will take it as an absolute time). The attached patch excluded the functionality added in bug 7248 (so that bug is a dependency to this).
Created attachment 6358 [details] [review] Fixed bugs in previous patch, added better user input and feedback The previous patch had 2 (related) bugs in it, both are fixed. This patch also allows the user to select the unit to input cache_expiry in (seconds, minutes, hours and days) and will complain if the entered value is too large (greater than 30 days) using the appropriate system for doing so (error dialogs). This patch is independent to the previous patch within this bug report (it includes all the changes included in the previously attached patch plus some new changes), however it still depends on bug 7248
Hi Chris, Here's what I get Commit Body is: -------------------------- Starting work on webservices -------------------------- Apply? [y]es/[n]o/[e]dit/[v]iew patch/[a]ccept all y Applying: Starting work on webservices Commit Body is: -------------------------- Working on caching for services -------------------------- Apply? [y]es/[n]o/[e]dit/[v]iew patch/[a]ccept all y Applying: Working on caching for services Commit Body is: -------------------------- Added caching to intranet report webservices, added cache expiry and public options to reports -------------------------- Apply? [y]es/[n]o/[e]dit/[v]iew patch/[a]ccept all y Applying: Added caching to intranet report webservices, added cache expiry and public options to reports fatal: sha1 information is lacking or useless (installer/data/mysql/updatedatabase.pl). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge. Patch failed at 0003 Added caching to intranet report webservices, added cache expiry and public options to reports
Ahh my fault, this one won't apply with out 7248 Sorry, I should read better.
Signed off, but still needs a sign off outside catalyst
Created attachment 6360 [details] [review] Bug 7249 : Allow reports to be called through a restful interface Hands back JSON, and supports caching Squashed commit of the following: commit 5c93506079738b9a13139417d3a0734289a4b007 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 15:41:14 2011 +1300 Fixed bugs when creating reports, added more user friendly input and feedback commit c62430bb358ee4af6ee5331b4d3a2ed67f723032 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 12:00:27 2011 +1300 Added caching to intranet report webservices, added cache expiry and public options to reports commit 2d89f0777d95b26bf08635782070b6367d0698f3 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 20:29:26 2011 +1300 Working on caching for services commit e0511f180cebd81747858ad776433fe3a1cf6854 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 08:28:26 2011 +1300 Starting work on webservices Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Hi Chris C./Chris H. - can you please add a description of what this is supposed to do and how to test it? Thx!
Hey Katrin This allows you to visit the following urls: opac: http://localhost/cgi-bin/koha/svc/report?id=1 intranet: http://localhost:8080/cgi-bin/koha/svc/report?id=1 (replace localhost and localhost:8080 with the appropriate URLs) To fetch JSON output form previously created reports (in the above cases you are viewing report 1). The report is only visible via the opac interface if it has been explicitly set to public (you can do this when creating a report or from the use saved report -> edit page).
I was looking at the patch - I think it's missing the update to kohastructure. sql to reflect the changes made by the updatedatabase?
Created attachment 6449 [details] [review] Patch adds missing kohastructure entry Sorry about that Katrin
Hi Chris, I am sorry, the first patch does not apply: git am -is3 /tmp/Bug-7249--Allow-reports-to-be-called-through-a-res-aUdhoC.patch Apply? [y]es/[n]o/[e]dit/[v]iew patch/[a]ccept all y Applying: Bug 7249 : Allow reports to be called through a restful interface fatal: sha1 information is lacking or useless (installer/data/mysql/updatedatabase.pl). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge. Patch failed at 0001 Bug 7249 : Allow reports to be called through a restful interface Can you check what's going wrong here?
Hey Katrin This patch is dependent on 7248 being applied first. On kc/master I am able to apply the 7248 patch and then this one without any errors, does this work for you? Thanks, ~Chris
Created attachment 6773 [details] New interface for sql reports Small enhancement request: Use yes/no instead of false/true for the pull down.
Created attachment 6774 [details] [review] [SIGNED-OFF] Bug 7249 : Allow reports to be called through a restful interface Hands back JSON, and supports caching Squashed commit of the following: commit 5c93506079738b9a13139417d3a0734289a4b007 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 15:41:14 2011 +1300 Fixed bugs when creating reports, added more user friendly input and feedback commit c62430bb358ee4af6ee5331b4d3a2ed67f723032 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 12:00:27 2011 +1300 Added caching to intranet report webservices, added cache expiry and public options to reports commit 2d89f0777d95b26bf08635782070b6367d0698f3 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 20:29:26 2011 +1300 Working on caching for services commit e0511f180cebd81747858ad776433fe3a1cf6854 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 08:28:26 2011 +1300 Starting work on webservices Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz> Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Comments on follow-up.
Created attachment 6775 [details] [review] [SIGNED-OFF] Bug 7249: added saved_sql fields to kohastructure http://bugs.koha-community.org/show_bug.cgi?id=7249 Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Patches applied on top of patches for bug 7248. Patch includes another database adding 2 new fields to the saved_sql table. 1) Checked that adding/deleting/editing of sql reports still works and new fields are correctly saved to the database. 2) The saved reports page now lists the new fields correctly. 3) Checking URLs with JSON output - if report is not public http://localhost/cgi-bin/koha/svc/report?id=2 fails, but http://localhost:8080/cgi-bin/koha/svc/report?id=2 works - if report is public, both links work 4) Checking results are cached Created a report listing my borrowers, setting caching to 60 seconds and added new borrowers before doing following tests. - URL doesn't show new borrower immediately - correct - Turning off the usecache system preference updates data - correct - Restarting memcached updates data '/etc/init.d/memcached restart' - correct - Waiting until cache time runs out updates data - correct Small enhancement request: Use yes/no instead of true/false.
Created attachment 6801 [details] [review] Changes true/false to yes/no Patch attached as per request Katrin Awesome work on all those signoffs.
Hi Chris, small thing: don't forget to put the bug number in the subject line of your patches. I will fix it :)
Created attachment 6803 [details] [review] [SIGNED-OFF] Bug 7249: Small useability fix as per request http://bugs.koha-community.org/show_bug.cgi?id=7249 Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Third patch to be applied for this bug. Staff interface shows yes/no instead of true/false now.
QA question: if i'm understanding correctly, the cache duration is something meaningfull only if you have memcache set. I haven't tested the patch, but does the field disappear from the screen cache is OFF ?
Seems this patch does not apply anymore (and the message look unusual to me) fatal: sha1 information is lacking or useless (installer/data/mysql/updatedatabase.pl). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge. Patch failed at 0001 Bug 7249 : Allow reports to be called through a restful interface When you have resolved this problem run "git am --resolved". If you would prefer to skip this patch, instead run "git am --skip". To restore the original branch and stop patching run "git am --abort".
my wrong: this patch must be applied AFTER 7248, back to signed off & going to QA 7248
This patch won't apply anymore because the bug 7248 won't be applied because bug 6193 has been applied after discussion on how to handle caching in Koha: we do that at apache configuration level. So, either update this patch to use eval { if (C4::Context->ismemcached) { require Memoize::Memcached; import Memoize::Memcached qw(memoize_memcached); memoize_memcached('getTranslatedLanguages', memcached => C4::Context->memcached); } }; in your script, or rewrite patch of bug 7248 to be able to have this one valid again.
Created attachment 9350 [details] [review] Bug 7249 : Allow reports to be called through a restful interface Hands back JSON, and supports caching Squashed commit of the following: commit 5c93506079738b9a13139417d3a0734289a4b007 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 15:41:14 2011 +1300 Fixed bugs when creating reports, added more user friendly input and feedback commit c62430bb358ee4af6ee5331b4d3a2ed67f723032 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 12:00:27 2011 +1300 Added caching to intranet report webservices, added cache expiry and public options to reports commit 2d89f0777d95b26bf08635782070b6367d0698f3 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 20:29:26 2011 +1300 Working on caching for services commit e0511f180cebd81747858ad776433fe3a1cf6854 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 08:28:26 2011 +1300 Starting work on webservices Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz> Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Patch includes database update adding 2 new fields to the saved_sql table. 1) Checked that adding/deleting/editing of sql reports still works and new fields are correctly saved to the database. 2) The saved reports page now lists the new fields correctly. 3) Checking URLs with JSON output - if report is not public http://localhost/cgi-bin/koha/svc/report?id=2 fails, but http://localhost:8080/cgi-bin/koha/svc/report?id=2 works - if report is public, both links work 4) Checking results are cached Created a report listing my borrowers, setting caching to 60 seconds and added new borrowers before doing following tests. - URL doesn't show new borrower immediately - correct - Turning off the usecache system preference updates data - correct - Restarting memcached updates data '/etc/init.d/memcached restart' - correct - Waiting until cache time runs out updates data - correct Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Created attachment 9351 [details] [review] Bug 7249 : Allow reports to be called through a restful interface Hands back JSON, and supports caching Squashed commit of the following: commit 5c93506079738b9a13139417d3a0734289a4b007 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 15:41:14 2011 +1300 Fixed bugs when creating reports, added more user friendly input and feedback commit c62430bb358ee4af6ee5331b4d3a2ed67f723032 Author: Chris Hall <chrish@catalyst.net.nz> Date: Mon Nov 21 12:00:27 2011 +1300 Added caching to intranet report webservices, added cache expiry and public options to reports commit 2d89f0777d95b26bf08635782070b6367d0698f3 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 20:29:26 2011 +1300 Working on caching for services commit e0511f180cebd81747858ad776433fe3a1cf6854 Author: Chris Cormack <chris@bigballofwax.co.nz> Date: Wed Nov 9 08:28:26 2011 +1300 Starting work on webservices Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz> Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Patch includes database update adding 2 new fields to the saved_sql table. 1) Checked that adding/deleting/editing of sql reports still works and new fields are correctly saved to the database. 2) The saved reports page now lists the new fields correctly. 3) Checking URLs with JSON output - if report is not public http://localhost/cgi-bin/koha/svc/report?id=2 fails, but http://localhost:8080/cgi-bin/koha/svc/report?id=2 works - if report is public, both links work 4) Checking results are cached Created a report listing my borrowers, setting caching to 60 seconds and added new borrowers before doing following tests. - URL doesn't show new borrower immediately - correct - Turning off the usecache system preference updates data - correct - Restarting memcached updates data '/etc/init.d/memcached restart' - correct - Waiting until cache time runs out updates data - correct Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Retested the feature and renewing my sign-off. There are some things that should be noted: - currently there is a hardcoded limit of 10 results for the reports webservice - reports with runtime parameters are currently not supported
Created attachment 9354 [details] [review] Bug 7249: Syspref to control number of rows in web service results The original patches for bug 7249 had a hardcoded limit of ten rows in the results returned by the web service. This patch adds a SvcMaxReportRows syspref which allows the librarian to choose how many rows should be returned by the web service.
Created attachment 9355 [details] [review] [SIGNED-OFF] Bug 7249: Syspref to control number of rows in web service results The original patches for bug 7249 had a hardcoded limit of ten rows in the results returned by the web service. This patch adds a SvcMaxReportRows syspref which allows the librarian to choose how many rows should be returned by the web service. Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Qa comment: All seems good. Paul, could you remove backquote (kohastructure.sql) before pushing please ? Marked as Passed QA
Patch pushed, with a small follow-up on online help, to explain how to access the reports. Please check that there is no english mistake ;-)
There have been no further reports of problems so I am marking this bug resolved.