Bug 13799 - Add base for building RESTful API
Summary: Add base for building RESTful API
Status: Pushed to Master
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P5 - low new feature
Assignee: Julian Maurice
QA Contact: Kyle M Hall
URL:
Whiteboard:
Keywords: rel_3_22_candidate
Depends on:
Blocks: 13737 13920 14448 14458 14656 14746 14747 14748 14749 14843 14974 15126 17032 17041 17044 13935
  Show dependency treegraph
 
Reported: 2015-03-05 16:06 UTC by Kyle M Hall
Modified: 2016-08-03 17:32 UTC (History)
21 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Large patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.67 KB, patch)
2015-03-06 12:10 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.80 KB, patch)
2015-03-06 13:33 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add versioning to API (2.70 KB, patch)
2015-03-17 08:37 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Use Koha::Objects (2.25 KB, patch)
2015-03-17 09:47 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add Swagger UI documentation (1.01 MB, patch)
2015-03-23 13:05 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Move and fix unit test file (1.44 KB, patch)
2015-03-23 15:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Fix Koha::Objects::find when no results are found (817 bytes, patch)
2015-03-24 10:35 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Change Mojolicious minimum version required (5.54) (782 bytes, patch)
2015-04-21 13:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.80 KB, patch)
2015-05-13 19:13 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.80 KB, patch)
2015-05-13 20:26 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Add versioning to API (2.71 KB, patch)
2015-05-13 20:26 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Use Koha::Objects (2.26 KB, patch)
2015-05-13 20:26 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Add Swagger UI documentation (1.01 MB, patch)
2015-05-13 20:27 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Move and fix unit test file (1.45 KB, patch)
2015-05-13 20:27 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Fix Koha::Objects::find when no results are found (823 bytes, patch)
2015-05-13 20:27 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: Change Mojolicious minimum version required (5.54) (789 bytes, patch)
2015-05-13 20:27 UTC, Benjamin Rokseth
Details | Diff | Splinter Review
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.80 KB, patch)
2015-06-11 08:47 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon. (3.68 KB, patch)
2015-06-11 14:42 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile. (3.14 KB, patch)
2015-06-11 14:48 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Set up logging for Mojolicious. (845 bytes, patch)
2015-06-11 14:52 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Set up logging and configuration file reading for Mojolicious. (2.74 KB, patch)
2015-06-12 09:43 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon. (4.76 KB, patch)
2015-06-12 10:01 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile. (3.31 KB, patch)
2015-06-12 10:11 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Set up logging and configuration file reading for Mojolicious. (4.38 KB, patch)
2015-06-15 13:19 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon. (4.76 KB, patch)
2015-06-15 13:21 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon. (4.80 KB, patch)
2015-06-15 13:23 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (6.80 KB, patch)
2015-06-22 12:45 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (2.71 KB, patch)
2015-06-22 12:46 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (2.26 KB, patch)
2015-06-22 12:46 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (1.01 MB, patch)
2015-06-22 12:46 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (1.45 KB, patch)
2015-06-22 12:46 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (788 bytes, patch)
2015-06-22 12:47 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (4.38 KB, patch)
2015-06-22 12:47 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Rebased (4.80 KB, patch)
2015-06-22 12:47 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile. (3.34 KB, patch)
2015-06-22 12:48 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: REST API proof of concept with Mojolicious/Swagger2 (6.80 KB, patch)
2015-06-24 08:34 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: Add versioning to API (2.71 KB, patch)
2015-06-24 08:35 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: Use Koha::Objects (2.26 KB, patch)
2015-06-24 08:35 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: Add Swagger UI documentation (1.01 MB, patch)
2015-06-24 08:35 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: RESTful API with Mojolicious and Swagger2 (1.01 MB, patch)
2015-06-24 09:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Force UTF-8 charset in responses (883 bytes, patch)
2015-06-24 09:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (1.02 MB, patch)
2015-06-24 15:08 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious. (4.38 KB, patch)
2015-06-24 15:08 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (865 bytes, patch)
2015-06-24 15:09 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 4. Add base for building RESTful API - Fix missing character encoding in the index.html (779 bytes, patch)
2015-06-24 15:10 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.87 KB, patch)
2015-06-25 07:01 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: 2. Set up logging and configuration file reading for Mojolicious. (4.37 KB, patch)
2015-06-25 07:01 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (886 bytes, patch)
2015-06-25 07:01 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799 - 4. Add base for building RESTful API - Fix missing character encoding in the index.html (779 bytes, patch)
2015-06-25 08:54 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.87 KB, patch)
2015-07-24 14:04 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious. (5.27 KB, patch)
2015-07-24 14:05 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (1.21 KB, patch)
2015-07-24 14:06 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.87 KB, patch)
2015-07-31 15:10 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious. (5.27 KB, patch)
2015-07-31 15:10 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (1.21 KB, patch)
2015-07-31 15:11 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 4. API Authentication, part 1: API keys management in interface (58.88 KB, patch)
2015-07-31 15:14 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 5. API authentication system - Swagtenticator authentication (13.18 KB, patch)
2015-07-31 15:20 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 6. REST API Testing framework (20.81 KB, patch)
2015-07-31 15:25 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Schematic of the API components (169.84 KB, image/jpeg)
2015-07-31 15:26 UTC, Olli-Antti Kivilahti
Details
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain (3.37 KB, patch)
2015-08-03 15:13 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 4. API Authentication, part 1: API keys management in interface (58.90 KB, patch)
2015-08-03 16:09 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.87 KB, patch)
2015-08-04 07:56 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain (3.46 KB, patch)
2015-08-04 07:56 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious. (5.27 KB, patch)
2015-08-04 07:56 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (1.21 KB, patch)
2015-08-04 07:57 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 4. API Authentication, part 1: API keys management in interface (58.90 KB, patch)
2015-08-04 07:57 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 5. API authentication system - Swagtenticator authentication (13.18 KB, patch)
2015-08-04 07:57 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.89 KB, patch)
2015-08-17 12:02 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain (3.46 KB, patch)
2015-08-17 12:02 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious. (5.27 KB, patch)
2015-08-17 12:15 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 3. Force UTF-8 charset in responses (1.21 KB, patch)
2015-08-17 12:15 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 4. API Authentication, part 1: API keys management in interface (32.68 KB, patch)
2015-08-17 12:16 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 5. API authentication system - Swagtenticator authentication (13.18 KB, patch)
2015-08-17 12:16 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799 - 6. REST API Testing framework (21.05 KB, patch)
2015-08-17 12:17 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: 1. RESTful API with Mojolicious and Swagger2 (9.88 KB, patch)
2015-08-17 13:04 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: RESTful API with Mojolicious and Swagger2 (11.77 KB, patch)
2015-08-27 17:30 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: RESTful API with Mojolicious and Swagger2 (11.84 KB, patch)
2015-08-28 14:52 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 13799: RESTful API with Mojolicious and Swagger2 (11.85 KB, patch)
2015-08-28 14:58 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 13799: Add cookie-based authentication to REST API (3.53 KB, patch)
2015-08-31 15:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799 - Swagger2-driven Permission checking (4.25 KB, patch)
2015-09-14 12:36 UTC, Olli-Antti Kivilahti
Details | Diff | Splinter Review
Bug 13799: Add missing license notices (4.27 KB, patch)
2015-09-22 09:09 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add missing license notices (4.41 KB, patch)
2015-09-22 10:43 UTC, j.kylmala
Details | Diff | Splinter Review
Bug 13799: Fix API unit tests (2.23 KB, patch)
2015-09-23 08:07 UTC, Julian Maurice
Details | Diff | Splinter Review
[SIGNED OFF] Bug 13799: RESTful API with Mojolicious and Swagger2 (11.90 KB, patch)
2015-09-25 13:00 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED OFF] Bug 13799: Add cookie-based authentication to REST API (3.60 KB, patch)
2015-09-25 13:00 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED OFF] Bug 13799: Add missing license notices (4.48 KB, patch)
2015-09-25 13:00 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED OFF] Bug 13799: Fix API unit tests (2.30 KB, patch)
2015-09-25 13:01 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED OFF] Bug 13799: (QA followup) make tests use random data (3.82 KB, patch)
2015-09-25 13:10 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 13799: Change Mojolicious default mode to 'production' (1.14 KB, patch)
2015-10-07 09:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF (1.69 KB, patch)
2015-10-07 09:13 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Change 'required' value type from string to boolean (767 bytes, patch)
2015-10-07 12:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799 - Followup: x-mojo-controller deprecation (1.64 KB, patch)
2015-10-07 14:51 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Reword 'borrower' to 'patron' (12.00 KB, patch)
2015-10-07 15:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799 - Followup: Refactor spec file (2.94 KB, patch)
2015-10-07 15:22 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: RESTful API with Mojolicious and Swagger2 (11.95 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Add cookie-based authentication to REST API (3.65 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Add missing license notices (4.53 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Fix API unit tests (2.35 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: (QA followup) make tests use random data (3.87 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Change Mojolicious default mode to 'production' (1.21 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF (1.76 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Change 'required' value type from string to boolean (833 bytes, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Reword 'borrower' to 'patron' (12.07 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799 - Followup: x-mojo-controller deprecation (1.60 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799 - Followup: Refactor spec file (18.54 KB, patch)
2015-10-07 19:21 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 13799: Followup of "Followup: Refactor spec file" (1.03 KB, patch)
2015-10-26 10:59 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Followup of "Followup: x-mojo-controller deprecation" (10.19 KB, patch)
2015-10-26 11:00 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Followup of "Followup: x-mojo-controller deprecation" (12.44 KB, patch)
2015-10-26 12:59 UTC, Julian Maurice
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: RESTful API with Mojolicious and Swagger2 (12.04 KB, patch)
2015-10-26 13:37 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Add cookie-based authentication to REST API (3.72 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Add missing license notices (4.62 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Fix API unit tests (2.42 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: (QA followup) make tests use random data (3.94 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Change Mojolicious default mode to 'production' (1.30 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF (1.85 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Change 'required' value type from string to boolean (908 bytes, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Reword 'borrower' to 'patron' (12.13 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799 - Followup: x-mojo-controller deprecation (1.67 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799 - Followup: Refactor spec file (18.61 KB, patch)
2015-10-26 13:38 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Followup of "Followup: Refactor spec file" (1.10 KB, patch)
2015-10-26 13:39 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 13799: Followup of "Followup: x-mojo-controller deprecation" (12.50 KB, patch)
2015-10-26 13:39 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 13799: Rename t/.../v1/borrowers.t to t/.../v1/patrons.t (1.69 KB, patch)
2015-10-30 07:50 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Update dependencies (860 bytes, patch)
2015-11-02 07:11 UTC, Julian Maurice
Details | Diff | Splinter Review
Dependency tree for Swagger2 0.60 (72.77 KB, image/png)
2015-11-03 03:54 UTC, Robin Sheat
Details
Bug 13799: RESTful API with Mojolicious and Swagger2 (12.01 KB, patch)
2015-11-04 14:06 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add cookie-based authentication to REST API (3.70 KB, patch)
2015-11-04 14:07 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add missing license notices (4.59 KB, patch)
2015-11-04 14:10 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Fix API unit tests (2.41 KB, patch)
2015-11-04 14:11 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: (QA followup) make tests use random data (3.92 KB, patch)
2015-11-04 14:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Change Mojolicious default mode to 'production' (1.26 KB, patch)
2015-11-04 14:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF (1.81 KB, patch)
2015-11-04 14:12 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Change 'required' value type from string to boolean (889 bytes, patch)
2015-11-04 14:13 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Reword 'borrower' to 'patron' (12.12 KB, patch)
2015-11-04 14:13 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Rename t/.../v1/borrowers.t to t/.../v1/patrons.t (1.69 KB, patch)
2015-11-04 14:13 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 13799: Add types for patron's fields in Swagger spec (15.02 KB, patch)
2015-11-04 14:14 UTC, Julian Maurice
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle M Hall 2015-03-05 16:06:31 UTC

    
Comment 1 Julian Maurice 2015-03-06 12:10:36 UTC
Created attachment 36700 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2

1/ Install Mojolicious Perl module
2/ Launch server with:
     $ morbo ./rest

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t
Comment 2 Julian Maurice 2015-03-06 13:33:57 UTC
Created attachment 36705 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2

Fixed patch with added dependency on Swagger2 module
Comment 3 Julian Maurice 2015-03-06 13:43:09 UTC
This patch is actually testable on sandbox 16 where I installed the required dependencies. There is no special configuration, so it will run in CGI mode.

Base URL: http://pro.test16.biblibre.com/cgi-bin/koha/rest
Comment 4 Benjamin Rokseth 2015-03-10 14:35:56 UTC
Tested this patch locally on a koha dev setup, and some remarks:

* incredibly fast! ~50msec on 50 borrowers...promising!
* few deps:

apt-get install -y make cpanm
cpanm Mojolicious
cpanm Swagger2

* clean and little "glue-code" to Koha
* self-documenting, in a way, with Swagger

I propose that this POC is moved forward to a fully working example covering Borrowers, that can be part of Koha 3.20 if the New_REST_API_RFC (http://wiki.koha-community.org/wiki/New_REST_API_RFC) are fullfilled.

What's missing now:
- authentication
- C4::Borrowers vs Koha::Borrowers?
- versioning
- content negotiation
- full CRUD
Comment 5 Kyle M Hall 2015-03-13 12:13:02 UTC
(In reply to Benjamin Rokseth from comment #4)
> Tested this patch locally on a koha dev setup, and some remarks:
> 
> * incredibly fast! ~50msec on 50 borrowers...promising!
> * few deps:
> 
> apt-get install -y make cpanm
> cpanm Mojolicious
> cpanm Swagger2

Yes, the dependancies will definitely need to be added to PerlDependancies.pm
 
> * clean and little "glue-code" to Koha
> * self-documenting, in a way, with Swagger
> 
> I propose that this POC is moved forward to a fully working example covering
> Borrowers, that can be part of Koha 3.20 if the New_REST_API_RFC
> (http://wiki.koha-community.org/wiki/New_REST_API_RFC) are fullfilled.

I completely agree, this should definitely move forward!

> What's missing now:
> - authentication

Yes, there are a number of ways for handle this:
1) The way /svc works now
2) Ability to pass in a auth_username and auth_password ( simpler, stateless, easy for command line use of /rest, only safe over https or from localhost )
3) API tokens ( best idea, will require a whole separate feature )

I think one or both of the first two ways would be perfectly acceptable.

> - C4::Borrowers vs Koha::Borrowers?

We should use this as an opportunity to take full advantage of the new Koha::Object system wherever possible.

> - versioning

Versioning should be a simple matter ( as described in http://wiki.koha-community.org/wiki/New_REST_API_RFC )

> - content negotiation

I personally think supporting JSON is adequate, but that's just my opinion. Supporting other formats would be great as long as we implement it at a low enough level that future developers will not have to deal with it ; )

I believe there is already a patch floating around that adds a to_json method to Koha::Object. We could add a method 'serialize' instead that takes a format and could output json, xml, yaml or whatever.

> - full CRUD

Definitely!

This is a great start!
Comment 6 Julian Maurice 2015-03-17 08:37:39 UTC
Created attachment 36938 [details] [review]
Bug 13799: Add versioning to API
Comment 7 Julian Maurice 2015-03-17 09:47:19 UTC
Created attachment 36941 [details] [review]
Bug 13799: Use Koha::Objects

This add 'unblessed' subroutines to both Koha::Objects and Koha::Object
to be able to pass it to Mojolicious
Comment 8 Julian Maurice 2015-03-17 09:49:10 UTC
Not sure about the last patch, maybe there's a better way to transform Koha::Object(s) into JSON.
Comment 9 Julian Maurice 2015-03-23 13:05:46 UTC
Created attachment 37119 [details] [review]
Bug 13799: Add Swagger UI documentation

Move all API stuff into /api/v1 (except for Perl modules).
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification
  /api/v1/doc            Swagger UI

Add a virtual host in Apache configuration api.HOSTNAME
So we have:
  http://api.HOSTNAME/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/v1/doc            Swagger UI
  http://api.HOSTNAME/v1/{path}         API endpoint

To test, you need to:
  1/ perl Makefile.PL
  2/ make && make install
  3/ Change etc/koha-httpd.conf and copy it to the right place if needed
  4/ Reload Apache
  5/ Go to http://api.HOSTNAME/v1/doc and check everything works
  6/ Also check that http://api.HOSTNAME/v1/borrowers and
     http://api.HOSTNAME/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://api.HOSTNAME/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

For the new vhost to work, you may need to modify your /etc/hosts file
if you run Koha locally.
Comment 10 Julian Maurice 2015-03-23 15:12:51 UTC
Created attachment 37132 [details] [review]
Bug 13799: Move and fix unit test file
Comment 11 Julian Maurice 2015-03-24 10:35:15 UTC
Created attachment 37172 [details] [review]
Bug 13799: Fix Koha::Objects::find when no results are found

If we pass undef to Koha::Object::_new_from_dbic, it croaks with message
"DBIC result type  isn't of the type Borrower at Koha/Objects.pm"
(in case we're using Koha::Borrowers->find)
Comment 12 Robin Sheat 2015-04-15 05:15:20 UTC
A couple of notes before I get into it too deeply:

libmojolicious-perl in wheezy is 2.98, hopefully it'll work with that.
Swagger2 isn't in Debian at all, but is probably buildable.
Comment 13 Robin Sheat 2015-04-15 05:36:32 UTC
More comments:
* version 6.0 is mandated in the dependencies. That's in no Debian release at all right now. We need to ensure it can work on wheezy.
* the httpd conf stuff hasn't been updated in the packages.
* the code is woefully underdocumented, some new files having exactly zero comments at all, let alone POD and copyright headers.

Now to start looking into getting it actually running...
Comment 14 Robin Sheat 2015-04-16 06:25:59 UTC
So there's a bug in the version of tar that debian includes that means I can't currently build mojolicious. I'll have to find a workaround to that.
Comment 15 Julian Maurice 2015-04-21 13:16:43 UTC
It seems that Swagger2 can't work with Mojolicious versions packaged in Debian wheezy

Tested with Mojolicious 2.98 (wheezy):
  $ perl -MSwagger2 -e ''
  Can't locate encode_json.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr
  /local/lib/site_perl .) at /usr/share/perl5/Mojo/Base.pm line 32.
  BEGIN failed--compilation aborted at /usr/local/share/perl/5.14.2/Swagger2.pm line 50.
  Compilation failed in require.
  BEGIN failed--compilation aborted.

and with Mojolicious 3.97 (wheezy-backports)
  $ perl -MSwagger2 -e ''
  "encode_json" is not exported by the Mojo::JSON module
   "decode_json" is not exported by the Mojo::JSON module
  Can't continue after import errors at /usr/local/share/perl/5.14.2/Swagger2.pm line 50
  BEGIN failed--compilation aborted at /usr/local/share/perl/5.14.2/Swagger2.pm line 50.
  Compilation failed in require.
  BEGIN failed--compilation aborted.

With Mojolicious 5.54 (jessie) it works.
Comment 16 Julian Maurice 2015-04-21 13:20:08 UTC
Created attachment 38246 [details] [review]
Bug 13799: Change Mojolicious minimum version required (5.54)
Comment 17 Robin Sheat 2015-04-21 22:50:07 UTC
(In reply to Julian Maurice from comment #16)
> Created attachment 38246 [details] [review] [review]
> Bug 13799: Change Mojolicious minimum version required (5.54)

That won't help because the version we need to support is 2.98. 

I would strongly suggest separating the documentation stuff into another bug, and making the API stuff work with mojolicious 2.98, and deal with the documentation separately and more leisurely.

As it is, I can't see it getting in as no one will be able to run it without manually cpanning things, which isn't an option.
Comment 18 Julian Maurice 2015-04-22 06:45:13 UTC
Swagger2 is not only for documentation.
The API code needs Mojolicious::Plugin::Swagger2, which needs Swagger2, which needs Mojolicious >= 5.54 because Swagger2 fails to compile with a smaller version of Mojolicious.

Anyway, Jessie will become the new stable in few days. Should we just wait before integrating this ?

But what about swagger2, which is not packaged at all ?
It is not the only Perl module that needs to be "cpanned" for Koha. So why should we block for this one ? Koha will continue to work without it.

If really needed, it shouldn't be hard to make a Debian package of it.
Comment 19 Robin Sheat 2015-04-22 22:27:55 UTC
(In reply to Julian Maurice from comment #18)
> Anyway, Jessie will become the new stable in few days. Should we just wait
> before integrating this ?

We'd be waiting for a year or three, we're only dropping squeeze support as at 3.20.
 
> But what about swagger2, which is not packaged at all ?
> It is not the only Perl module that needs to be "cpanned" for Koha. So why
> should we block for this one ? Koha will continue to work without it.

There are no modules that need to be cpanned for Koha, and if there are, that's a bug (with the exception of some test case things, I think.) I think having a significantly useful feature that can't be readily used isn't good, so I'm trying to get it to a point where it can work.

> If really needed, it shouldn't be hard to make a Debian package of it.

At the moment, it's not particularly easy, as it needs a version of mojolicious that isn't packaged at all. I attempted to package that, but ran into https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=748244 which caused it to fail.

I'll have another go when I get the chance, but depending on bleeding edge modules is always going to be problematic.
Comment 20 Julian Maurice 2015-04-23 07:51:06 UTC
(In reply to Robin Sheat from comment #19)
> There are no modules that need to be cpanned for Koha, and if there are,
> that's a bug (with the exception of some test case things, I think.) I think
> having a significantly useful feature that can't be readily used isn't good,
> so I'm trying to get it to a point where it can work.

Ok, so just for pointing it out, required version of DBIx::Class::Schema::Loader (0.07039) is not in wheezy (0.07025-1), and not in debian.koha-community.org neither.

> At the moment, it's not particularly easy, as it needs a version of
> mojolicious that isn't packaged at all. I attempted to package that, but ran
> into https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=748244 which caused
> it to fail.
> 
> I'll have another go when I get the chance, but depending on bleeding edge
> modules is always going to be problematic.

How did you try to package it ? I just tried this morning and was successful packaging Mojolicious 6.08 and Swagger2 0.30 for wheezy (+ additionnal Mojo dependency IO::Socket::IP - packaged in wheezy but version too low)
In the process, I had to make several other packages, but they are only build dependencies (Test::Warnings, Test::Deep, CPAN::Meta::Check)
I only used dh-make-perl and debuild to build packages (and dpkg to install them).
Comment 21 Olli-Antti Kivilahti 2015-04-23 08:01:06 UTC
I tested this few weeks ago and I had no issues running them?
Just installed from cpan.
Maybe I missed something?

I ran the manual test cases defined.

We definetely should use the latest Mojolicious and Swagger2 implementations.
No compromise here.
Documentation and all in one bundle like it was meant to be.

Very strange that the latest Mojolicious would be unavailable via packages.
Comment 22 Robin Sheat 2015-04-23 22:46:20 UTC
(In reply to Julian Maurice from comment #20)
> Ok, so just for pointing it out, required version of
> DBIx::Class::Schema::Loader (0.07039) is not in wheezy (0.07025-1), and not
> in debian.koha-community.org neither.

That's not needed to run or build Koha. Only the release people need that. Also, IIRC it wasn't feasible to build it without upsetting half the rest of the system.

> How did you try to package it ? 

The proper way for upstreaming it. See here:

https://lists.debian.org/debian-perl/2015/04/msg00009.html

This is the proper way to build a package, as it bases it on the older existing version and hard work has generally been done for you.

> I only used dh-make-perl and debuild to build packages (and dpkg to install
> them).

That won't result in a package acceptable to Debian.

(In reply to Olli-Antti Kivilahti from comment #21)
> I tested this few weeks ago and I had no issues running them?
> Just installed from cpan.

That's fine for testing, just not acceptable for distribution.

> We definetely should use the latest Mojolicious and Swagger2 implementations.
> No compromise here.

No, that's wrong. We can't expect everyone to have it available, and we can't always build it for them. I'm still hopeful we can in this case, but unless that becomes possible, then we have problems.

> Very strange that the latest Mojolicious would be unavailable via packages.

I am a little surprised by that too, to be honest. I would have at least expected it to be in git, unless it's because everyone else is being bitten by this tar bug too.

When I get the chance, I'm going to try to downgrade tar to at least allow things to build. Of course, if someone wants to patch tar for me, that'd be great :)
Comment 23 Julian Maurice 2015-04-24 07:55:32 UTC
(In reply to Robin Sheat from comment #22)
> (In reply to Julian Maurice from comment #20)
> > I only used dh-make-perl and debuild to build packages (and dpkg to install
> > them).
> 
> That won't result in a package acceptable to Debian.

But it will be acceptable to debian.koha-community.org, won't it ?
Comment 24 Chris Cormack 2015-04-27 09:09:03 UTC
(In reply to Julian Maurice from comment #23)
> (In reply to Robin Sheat from comment #22)
> > (In reply to Julian Maurice from comment #20)
> > > I only used dh-make-perl and debuild to build packages (and dpkg to install
> > > them).
> > 
> > That won't result in a package acceptable to Debian.
> 
> But it will be acceptable to debian.koha-community.org, won't it ?

No, we build proper packages for debian.koha-community.org and upstream them also. eg https://qa.debian.org/developer.php?login=robin@catalyst.net.nz this gets them into debian proper, and from there into ubuntu.

Regardless of the packages, I think this should be split into a bunch of bugs, certainly shifting the documentation one off on to its on thing. I don't think we need to build any front end stuff for this. If we do that should be on it's on bug also. 
To get something like this in, its much better to do what was done with the acquisitions work, seperate bugs that don't mean the entire thing has to pass at once.
Comment 25 Julian Maurice 2015-04-27 09:40:18 UTC
(In reply to Chris Cormack from comment #24)
> Regardless of the packages, I think this should be split into a bunch of
> bugs, certainly shifting the documentation one off on to its on thing. I
> don't think we need to build any front end stuff for this. If we do that
> should be on it's on bug also. 
> To get something like this in, its much better to do what was done with the
> acquisitions work, seperate bugs that don't mean the entire thing has to
> pass at once.

I disagree. The "front end stuff" is the API documentation, and I don't think we can integrate the new API without a proper documentation.
Comment 26 Kyle M Hall 2015-05-06 13:32:35 UTC
(In reply to Robin Sheat from comment #19)
> (In reply to Julian Maurice from comment #18)
> > Anyway, Jessie will become the new stable in few days. Should we just wait
> > before integrating this ?
> 
> We'd be waiting for a year or three, we're only dropping squeeze support as
> at 3.20.

With 3.20 around the corner is this really an issue? It just means that it wouldn't be backportable to earlier versions of Koha, correct? Considering this is an enhancement I wouldn't expect it to be backported anyway. Are there other considerations I'm missing?
Comment 27 Kyle M Hall 2015-05-06 13:39:16 UTC
(In reply to Julian Maurice from comment #25)
> (In reply to Chris Cormack from comment #24)
> > Regardless of the packages, I think this should be split into a bunch of
> > bugs, certainly shifting the documentation one off on to its on thing. I
> > don't think we need to build any front end stuff for this. If we do that
> > should be on it's on bug also. 
> > To get something like this in, its much better to do what was done with the
> > acquisitions work, seperate bugs that don't mean the entire thing has to
> > pass at once.
> 
> I disagree. The "front end stuff" is the API documentation, and I don't
> think we can integrate the new API without a proper documentation.

I would tend to agree with Julian on this. The documentation is part and parcel of the api. I don't see any advantage to splitting it into a separate bug.
Comment 28 Kyle M Hall 2015-05-06 14:42:40 UTC
(In reply to Kyle M Hall from comment #26)
> (In reply to Robin Sheat from comment #19)
> > (In reply to Julian Maurice from comment #18)
> > > Anyway, Jessie will become the new stable in few days. Should we just wait
> > > before integrating this ?
> > 
> > We'd be waiting for a year or three, we're only dropping squeeze support as
> > at 3.20.
> 
> With 3.20 around the corner is this really an issue? It just means that it
> wouldn't be backportable to earlier versions of Koha, correct? Considering
> this is an enhancement I wouldn't expect it to be backported anyway. Are
> there other considerations I'm missing?

Ignore this comment, I had my versions crossed. Any idea what the possibility of taking the package for Jesse and using it on Wheezy is?
Comment 29 Kyle M Hall 2015-05-06 14:44:08 UTC
Another idea, can't we have a temporary package for Mojo even if it's not proper for upstreaming? Once the debian mojo package hit's the right version we could remove our own package. Would that work?
Comment 30 Chris Cormack 2015-05-06 19:01:18 UTC
This feels like making work for users and package maintainers, to make something arguably easier for developers. This is something that has made a mess for us a lot in the past. 

However, we probably should not be worrying about this, as it's missed the window for 3.20 already and be expending energy on testing and fixing bugs. We can argue about adding unnessecary dependencies again after the release.
Comment 31 Robin Sheat 2015-05-06 22:36:08 UTC
(In reply to Kyle M Hall from comment #28)
> Ignore this comment, I had my versions crossed. Any idea what the
> possibility of taking the package for Jesse and using it on Wheezy is?

Dependency packages need to be actually built for the environment, we can't just grab a .deb file and throw it in.

(In reply to Kyle M Hall from comment #29)
> Another idea, can't we have a temporary package for Mojo even if it's not
> proper for upstreaming? Once the debian mojo package hit's the right version
> we could remove our own package. Would that work?

That's exactly what I'm trying to avoid. Now it's something I have to maintain for the next few years. Hence my policy being that if I can't make a package that can go into Debian, then I'm not really willing to put it into the Koha repo. This also forces me to check the package for overall worthiness (e.g. people depending on modules that have no copyright notice, therefore we can't legally redistribute - not the case here though, this is just simply unbuildable until I find a version of tar without the bug.)

If you want to help out on this front, have a look here: http://wiki.koha-community.org/wiki/Building_Debian_Dependencies
Comment 32 Julian Maurice 2015-05-07 08:17:53 UTC
(In reply to Robin Sheat from comment #31)
> (In reply to Kyle M Hall from comment #28)
> > Ignore this comment, I had my versions crossed. Any idea what the
> > possibility of taking the package for Jesse and using it on Wheezy is?
> 
> Dependency packages need to be actually built for the environment, we can't
> just grab a .deb file and throw it in.

I tried to build Mojolicious package for Wheezy from here: git://git.debian.org/git/pkg-perl/packages/libmojolicious-perl.git
I had to install several *build* dependencies from Jessie, but the resulting binary package should work well on wheezy.
In Jessie libmojolicious-perl depends on libjs-prettify which is not in Wheezy so I had to also build this package.
Building source packages requires much more work, as there is a lot of build dependencies not packaged in wheezy, but do we need them ?

(for libjs-prettify I retrieved files from packages.debian.org and built from there)
Comment 33 Robin Sheat 2015-05-07 23:36:12 UTC
(In reply to Julian Maurice from comment #32)
> I tried to build Mojolicious package for Wheezy from here:
> git://git.debian.org/git/pkg-perl/packages/libmojolicious-perl.git

This is version 5.54 though. The problem is when you run uscan to create the DFSG-compliant source tarball of version 6.08, and swagger2 declares a dependency on Mojolicious 6.0+, so changing the minimum version that Koha requires doesn't change a whole lot.

> I had to install several *build* dependencies from Jessie, but the resulting
> binary package should work well on wheezy.
> In Jessie libmojolicious-perl depends on libjs-prettify which is not in
> Wheezy so I had to also build this package.
> Building source packages requires much more work, as there is a lot of build
> dependencies not packaged in wheezy, but do we need them ?

We will need the source packages, because we need to build them. This is because we need to modify the changelogs to specify which pockets of the repo it's going to go into.

I avoid doing it in hacky ways, or ways that are not Debian-compliant, because that ends up causing maintenance headaches and more unnecessary work in the long run, and will also interfere with my planned efforts to automate some parts of this process.

All this said, I just downgraded tar in my package building VM to the version from squeeze, and uscan succeeds. So there is hope yet.
Comment 34 Julian Maurice 2015-05-08 12:05:23 UTC
(In reply to Robin Sheat from comment #33)
> All this said, I just downgraded tar in my package building VM to the
> version from squeeze, and uscan succeeds. So there is hope yet.

Yay! \o/

About 5.54 vs 6.0, maybe we could ask the author of Swagger2 to change the required version of Mojolicious (if Swagger2 is proven to work with 5.54 of course). What do you think ? Will that make things easier ?
On the other hand, having the latest version of Mojolicious into Debian would be great if this doesn't require extra work.
Comment 35 Robin Sheat 2015-05-10 22:58:19 UTC
(In reply to Julian Maurice from comment #34)
> About 5.54 vs 6.0, maybe we could ask the author of Swagger2 to change the
> required version of Mojolicious (if Swagger2 is proven to work with 5.54 of
> course). What do you think ? Will that make things easier ?

It certainly would, but it might be the difference between 5 minutes and half and hour, or 5 minutes and 5 weeks :) it remains to be seen.
Comment 36 Julian Maurice 2015-05-12 10:09:27 UTC
Just asked: https://github.com/jhthorsen/swagger2/issues/14
Comment 37 Julian Maurice 2015-05-13 12:03:19 UTC
New version of Swagger2 on CPAN (0.31) now requires Mojolicious 5.54
Comment 38 Benjamin Rokseth 2015-05-13 19:13:22 UTC
Created attachment 39156 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2

1/ Install Mojolicious Perl module
2/ Launch server with:
     $ morbo ./rest

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t
Comment 39 Benjamin Rokseth 2015-05-13 19:15:23 UTC
Darn, messed up order when fixing resolve issue on first patch.
Will fix ASAP
Comment 40 Benjamin Rokseth 2015-05-13 20:26:02 UTC
Created attachment 39158 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2

1/ Install Mojolicious Perl module
2/ Launch server with:
     $ morbo ./rest

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t
Comment 41 Benjamin Rokseth 2015-05-13 20:26:24 UTC
Created attachment 39159 [details] [review]
Bug 13799: Add versioning to API
Comment 42 Benjamin Rokseth 2015-05-13 20:26:56 UTC
Created attachment 39160 [details] [review]
Bug 13799: Use Koha::Objects

This add 'unblessed' subroutines to both Koha::Objects and Koha::Object
to be able to pass it to Mojolicious
Comment 43 Benjamin Rokseth 2015-05-13 20:27:10 UTC
Created attachment 39161 [details] [review]
Bug 13799: Add Swagger UI documentation

Move all API stuff into /api/v1 (except for Perl modules).
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification
  /api/v1/doc            Swagger UI

Add a virtual host in Apache configuration api.HOSTNAME
So we have:
  http://api.HOSTNAME/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/v1/doc            Swagger UI
  http://api.HOSTNAME/v1/{path}         API endpoint

To test, you need to:
  1/ perl Makefile.PL
  2/ make && make install
  3/ Change etc/koha-httpd.conf and copy it to the right place if needed
  4/ Reload Apache
  5/ Go to http://api.HOSTNAME/v1/doc and check everything works
  6/ Also check that http://api.HOSTNAME/v1/borrowers and
     http://api.HOSTNAME/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://api.HOSTNAME/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

For the new vhost to work, you may need to modify your /etc/hosts file
if you run Koha locally.
Comment 44 Benjamin Rokseth 2015-05-13 20:27:20 UTC
Created attachment 39162 [details] [review]
Bug 13799: Move and fix unit test file
Comment 45 Benjamin Rokseth 2015-05-13 20:27:30 UTC
Created attachment 39163 [details] [review]
Bug 13799: Fix Koha::Objects::find when no results are found

If we pass undef to Koha::Object::_new_from_dbic, it croaks with message
"DBIC result type  isn't of the type Borrower at Koha/Objects.pm"
(in case we're using Koha::Borrowers->find)
Comment 46 Benjamin Rokseth 2015-05-13 20:27:41 UTC
Created attachment 39164 [details] [review]
Bug 13799: Change Mojolicious minimum version required (5.54)
Comment 47 Robin Sheat 2015-05-27 04:15:41 UTC
libmojolicious-perl_5.54+dfsg-1~koha1_all.deb (etc.) and libjs-prettify_2013.03.04+dfsg-4~koha1_all.deb (etc.) are now in the repo. I had to install Java on my build vm to get that last one to build. I hope you're happy! ;)

Now to look into swagger2.
Comment 48 Robin Sheat 2015-05-27 05:15:47 UTC
And, libswagger2-perl is in the repo now too.
Comment 49 Julian Maurice 2015-05-27 06:34:28 UTC
Great! Thanks Robin!
Comment 50 Olli-Antti Kivilahti 2015-05-27 10:25:36 UTC
Wow! Awesome work! Good karma!
Comment 51 Katrin Fischer 2015-05-31 12:19:01 UTC
What are the next steps here?
Comment 52 Robin Sheat 2015-06-02 00:15:46 UTC
(In reply to Katrin Fischer from comment #51)
> What are the next steps here?

Someone to do some testing and signoffing. It's on my list to look at.
Comment 53 Olli-Antti Kivilahti 2015-06-11 08:47:49 UTC
Created attachment 40088 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2

1/ Install Mojolicious Perl module
2/ Launch server with:
     $ morbo ./rest

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t
Comment 54 Olli-Antti Kivilahti 2015-06-11 08:49:17 UTC
Comment on attachment 39163 [details] [review]
Bug 13799: Fix Koha::Objects::find when no results are found

Bug 13799: Fix Koha::Objects::find when no results are found
is obsoleted now by 
Bug 13967 - System preferences need a package ??
Comment 55 Tomás Cohen Arazi 2015-06-11 11:00:12 UTC
(In reply to Olli-Antti Kivilahti from comment #54)
> Comment on attachment 39163 [details] [review] [review]
> Bug 13799: Fix Koha::Objects::find when no results are found
> 
> Bug 13799: Fix Koha::Objects::find when no results are found
> is obsoleted now by 
> Bug 13967 - System preferences need a package ??

I think so.
Comment 56 Olli-Antti Kivilahti 2015-06-11 14:42:15 UTC
Created attachment 40092 [details] [review]
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon.

After running make, the koha-api-daemon.sh -script should be in
../koha-dev/bin/koha-api-daemon.sh

Link it to the init-directory and set up upstart triggers:
..$ ln -s /home/koha/koha-dev/bin/koha-api-daemon.sh /etc/init.d/koha-api-daemon
..$ update-rc.d koha-api-daemon defaults

Start hypnotoad:
..$ service koha-api-daemon start

Hypnotoad is now listening in 127.0.0.1:8080 and is preconfigured to be production ready.

Hypnotoad config is in $KOHA_PATH/api/v1/hypnotoad.conf
Comment 57 Olli-Antti Kivilahti 2015-06-11 14:48:31 UTC
Created attachment 40093 [details] [review]
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile.

Uses a self-signed SSL-certificate to service at port 444 by default.
Reverse proxies to 127.0.0.1:8080 (expecting hypnotoad to be listening)

¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
¤¤ Koha API Nginx configuration ¤¤
¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

Deploy the Nginx configuration
------------------------------

Run make to populate the koha-nginx-kohaapi.conf

link to the /etc/nginx/sites-available/ -directory
..$ ln -s /home/koha/koha-dev/etc/koha-nginx-kohaapi.conf /etc/nginx/sites-available/kohaapi
and enable with command
..$ ln -s /etc/nginx/sites-available/kohaapi /etc/nginx/sites-enabled/kohaapi

Disable the default config
..$ rm /etc/nginx/sites-enabled/default

Create a openssl self-signed certificate or use your own.
---------------------------------------------------------

..$ cd /etc/nginx
..$ mkdir ssl
..$ chmod 400 ssl
..$ cd ssl
..$ openssl req -x509 -sha256 -newkey rsa:2048 -keyout key.pem.secure -out cert.pem -days 720
..$ openssl rsa -in key.pem.secure -out key.pem
..$ chmod 400 *

Restart nginx
..$ service nginx restart
Comment 58 Olli-Antti Kivilahti 2015-06-11 14:52:14 UTC
Created attachment 40094 [details] [review]
Bug 13799 - Set up logging for Mojolicious.

Somebody smarter can set the logging path as dynamic.
Comment 59 Olli-Antti Kivilahti 2015-06-12 09:43:30 UTC
Created attachment 40098 [details] [review]
Bug 13799 - Set up logging and configuration file reading for Mojolicious.

Use environmental values to control some aspects of Mojolicious:
This way we can have different settings for different servers running Mojolicious.

%%%% Configuration file %%%%

$ENV{MOJO_CONFIG} should be set in the system service (init) starting Mojolicious, eg:
export MOJO_CONFIG=/home/koha/kohaclone/api/v1/hypnotoad.conf

This configuration file read by the Mojolicious::Plugin::Config
http://mojolicio.us/perldoc/Mojolicious/Plugin/Config

%%%%% Logging %%%%%

Log to a filename configured in an environemnt variable $ENV{MOJO_LOGFILE} using loglevel $ENV{MOJO_LOGLEVEL}.
Defaults to '/tmp/koha-api.log' and loglevel of 'error'
Examples:
export MOJO_LOGFILE=/home/koha/koha-dev/var/log/kohaapi.mojo.log
export MOJO_LOGLEVEL=debug

Logging is done by Mojo::Log
http://www.mojolicio.us/perldoc/Mojo/Log
Comment 60 Olli-Antti Kivilahti 2015-06-12 09:43:55 UTC
Comment on attachment 40094 [details] [review]
Bug 13799 - Set up logging for Mojolicious.

>From 17fe4147396127c87be5d0270ad78a7d44576cd5 Mon Sep 17 00:00:00 2001
>From: Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi>
>Date: Thu, 11 Jun 2015 17:48:54 +0300
>Subject: [PATCH] Bug 13799 - Set up logging for Mojolicious.
>
>Somebody smarter can set the logging path as dynamic.
>---
> Koha/REST/V1.pm |    4 ++++
> 1 file changed, 4 insertions(+)
>
>diff --git a/Koha/REST/V1.pm b/Koha/REST/V1.pm
>index 3becf12..197c17b 100644
>--- a/Koha/REST/V1.pm
>+++ b/Koha/REST/V1.pm
>@@ -2,10 +2,14 @@ package Koha::REST::V1;
> 
> use Modern::Perl;
> use Mojo::Base 'Mojolicious';
>+use Mojo::Log;
> 
> sub startup {
>     my $self = shift;
> 
>+    $self->app->log( Mojo::Log->new( path => "/home/koha/koha-dev/var/log/kohaapi.mojo.log", level => 'debug' ) );
>+
>+
>     my $route = $self->routes->under->to(
>         cb => sub {
>             my $c = shift;
>-- 
>1.7.9.5
Comment 61 Olli-Antti Kivilahti 2015-06-12 10:01:55 UTC
Created attachment 40099 [details] [review]
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon.

After running make, the koha-api-daemon.sh -script should be in
../koha-dev/bin/koha-api-daemon.sh

Link it to the init-directory and set up upstart triggers:
..$ ln -s /home/koha/koha-dev/bin/koha-api-daemon.sh /etc/init.d/koha-api-daemon
..$ update-rc.d koha-api-daemon defaults

Start hypnotoad:
..$ service koha-api-daemon start

Hypnotoad is now listening in 127.0.0.1:8080 and is preconfigured to be production ready.

Hypnotoad config is in $KOHA_PATH/api/v1/hypnotoad.conf

For more info, read the koha-api-daemon.sh -service
Comment 62 Olli-Antti Kivilahti 2015-06-12 10:11:37 UTC
Created attachment 40101 [details] [review]
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile.

Uses a self-signed SSL-certificate to service at port 444 by default.
Reverse proxies to 127.0.0.1:8080 (expecting hypnotoad to be listening)

¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
¤¤ Koha API Nginx configuration ¤¤
¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

Deploy the Nginx configuration
------------------------------

Run make to populate the koha-nginx-kohaapi.conf

link to the /etc/nginx/sites-available/ -directory
..$ ln -s /home/koha/koha-dev/etc/koha-nginx-kohaapi.conf /etc/nginx/sites-available/kohaapi
and enable with command
..$ ln -s /etc/nginx/sites-available/kohaapi /etc/nginx/sites-enabled/kohaapi

Disable the default config
..$ rm /etc/nginx/sites-enabled/default

Create a openssl self-signed certificate or use your own.
---------------------------------------------------------

..$ cd /etc/nginx
..$ mkdir ssl
..$ chmod 400 ssl
..$ cd ssl
..$ openssl req -x509 -sha256 -newkey rsa:2048 -keyout key.pem.secure -out cert.pem -days 720
..$ openssl rsa -in key.pem.secure -out key.pem
..$ chmod 400 *

Restart nginx
..$ service nginx restart
Comment 63 Robin Sheat 2015-06-15 00:08:37 UTC
How will that interact with Plack? Environment variables aren't the easiest thing to pass through to that at this stage.
Comment 64 Olli-Antti Kivilahti 2015-06-15 12:25:15 UTC
(In reply to Robin Sheat from comment #63)
> How will that interact with Plack? Environment variables aren't the easiest
> thing to pass through to that at this stage.

I think this will be overwritten by plack.
I just created this system because there was no other system published in bugzilla for me to start building stuff on top of the Koha REST API.

I think the big question is how to pass variables to Mojolicious app, without using env-variables. I don't know and the solution is good enough for me.

What I would be worried of, is the redirection of STDERR and STDOUT to /dev/null in Mojo::Server::damonize().
Thus logging those streams from the service-script doesn't work, but it works when running Mojolicious directly from the terminal.
Comment 65 Katrin Fischer 2015-06-15 12:46:13 UTC
Maybe we could add some of the open questions/topics to the wiki? The next dev meeting is supposed to be mostly about the RESTful API.
Comment 66 Olli-Antti Kivilahti 2015-06-15 13:19:50 UTC
Created attachment 40140 [details] [review]
Bug 13799 - Set up logging and configuration file reading for Mojolicious.

Use environmental values to control some aspects of Mojolicious:
This way we can have different settings for different servers running Mojolicious.

%%%% Configuration file %%%%

$ENV{MOJO_CONFIG} should be set in the system service (init) starting Mojolicious, eg:
export MOJO_CONFIG=/home/koha/kohaclone/api/v1/hypnotoad.conf

This configuration file read by the Mojolicious::Plugin::Config
http://mojolicio.us/perldoc/Mojolicious/Plugin/Config

%%%%% Logging %%%%%

>NOTE!!
>There is a "feature" in Mojo::Server disabling STDOUT and STDERR, because such errors are not-suited-for-prod~
>This modification in Mojo::Server disables this and preserves the STD* handles for forked server threads
>in Mojo::Server::daemonize(), comment out the following lines
>
># Close filehandles
>#  open STDOUT, '>/dev/null';
>#  open STDERR, '>&STDOUT';

Log to a filename configured in an environemnt variable $ENV{MOJO_LOGFILE} using loglevel $ENV{MOJO_LOGLEVEL}.
Defaults to '/tmp/koha-api.log' and loglevel of 'error'
Examples:
export MOJO_LOGFILE=/home/koha/koha-dev/var/log/kohaapi.mojo.log
export MOJO_LOGLEVEL=debug

Logging is done by Mojo::Log
http://www.mojolicio.us/perldoc/Mojo/Log
Comment 67 Olli-Antti Kivilahti 2015-06-15 13:21:45 UTC
Created attachment 40141 [details] [review]
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon.

After running make, the koha-api-daemon.sh -script should be in
../koha-dev/bin/koha-api-daemon.sh

Link it to the init-directory and set up upstart triggers:
..$ ln -s /home/koha/koha-dev/bin/koha-api-daemon.sh /etc/init.d/koha-api-daemon
..$ update-rc.d koha-api-daemon defaults

Start hypnotoad:
..$ service koha-api-daemon start

Hypnotoad is now listening in 127.0.0.1:8080 and is preconfigured to be production ready.

Hypnotoad config is in $KOHA_PATH/api/v1/hypnotoad.conf

For more info, read the koha-api-daemon.sh -service
Comment 68 Olli-Antti Kivilahti 2015-06-15 13:23:01 UTC
Created attachment 40142 [details] [review]
Bug 13799 - Adding the REST API as a Hypnotoad service, koha-api-daemon.

After running make, the koha-api-daemon.sh -script should be in
../koha-dev/bin/koha-api-daemon.sh

Link it to the init-directory and set up upstart triggers:
..$ ln -s /home/koha/koha-dev/bin/koha-api-daemon.sh /etc/init.d/koha-api-daemon
..$ update-rc.d koha-api-daemon defaults

Start hypnotoad:
..$ service koha-api-daemon start

Hypnotoad is now listening in 127.0.0.1:8080 and is preconfigured to be production ready.

Hypnotoad config is in $KOHA_PATH/api/v1/hypnotoad.conf

For more info, read the koha-api-daemon.sh -service
Comment 69 Olli-Antti Kivilahti 2015-06-22 12:45:24 UTC
Created attachment 40463 [details] [review]
Rebased
Comment 70 Olli-Antti Kivilahti 2015-06-22 12:46:03 UTC
Created attachment 40464 [details] [review]
Rebased
Comment 71 Olli-Antti Kivilahti 2015-06-22 12:46:19 UTC
Created attachment 40465 [details] [review]
Rebased
Comment 72 Olli-Antti Kivilahti 2015-06-22 12:46:40 UTC
Created attachment 40466 [details] [review]
Rebased
Comment 73 Olli-Antti Kivilahti 2015-06-22 12:46:56 UTC
Created attachment 40467 [details] [review]
Rebased
Comment 74 Olli-Antti Kivilahti 2015-06-22 12:47:11 UTC
Created attachment 40468 [details] [review]
Rebased
Comment 75 Olli-Antti Kivilahti 2015-06-22 12:47:31 UTC
Created attachment 40469 [details] [review]
Rebased
Comment 76 Olli-Antti Kivilahti 2015-06-22 12:47:52 UTC
Created attachment 40470 [details] [review]
Rebased
Comment 77 Olli-Antti Kivilahti 2015-06-22 12:48:36 UTC
Created attachment 40471 [details] [review]
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile.

Uses a self-signed SSL-certificate to service at port 444 by default.
Reverse proxies to 127.0.0.1:8080 (expecting hypnotoad to be listening)

¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
¤¤ Koha API Nginx configuration ¤¤
¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

Deploy the Nginx configuration
------------------------------

Run make to populate the koha-nginx-kohaapi.conf

link to the /etc/nginx/sites-available/ -directory
..$ ln -s /home/koha/koha-dev/etc/koha-nginx-kohaapi.conf /etc/nginx/sites-available/kohaapi
and enable with command
..$ ln -s /etc/nginx/sites-available/kohaapi /etc/nginx/sites-enabled/kohaapi

Disable the default config
..$ rm /etc/nginx/sites-enabled/default

Create a openssl self-signed certificate or use your own.
---------------------------------------------------------

..$ cd /etc/nginx
..$ mkdir ssl
..$ chmod 400 ssl
..$ cd ssl
..$ openssl req -x509 -sha256 -newkey rsa:2048 -keyout key.pem.secure -out cert.pem -days 720
..$ openssl rsa -in key.pem.secure -out key.pem
..$ chmod 400 *

Restart nginx
..$ service nginx restart
Comment 78 Tomás Cohen Arazi 2015-06-22 13:34:54 UTC
Olli, while I'm not against the idea to moving into nginx and/or hypnotoad, I think putting this patches here just make more confusion.

If it is possible, could you please move your integration work into the wiki? So on this bug we discuss the REST patches only?
If you have a serious proposal of moving away from Apache, you should definitely put that on a new bug report.

AS I see it, your patch(es) aid testing the REST implementation, so they are really useful. Just move them outside this specific bug. Please :-D
Comment 79 Julian Maurice 2015-06-23 08:50:20 UTC
Also Olli, you replaced all patch titles by 'Rebased', making it harder to see which patch does what. Could you please fix that ?
Comment 80 Jonathan Druart 2015-06-23 13:51:02 UTC
Julian,
1) This needs a squash, to avoid add/del of the same file/dir in the same patch set and an up-to-date test plan.

2) /v1/doc returns "Page not found... yet!"

3) /v1/borrowers/7853
Check the encoding, I get 
    "address2":"éééصةصةصة"

With
MariaDB [koha]> select address2 from borrowers where borrowernumber=7874\G
*************************** 1. row ***************************
address2: éééصةصةصة
Comment 81 Olli-Antti Kivilahti 2015-06-24 08:34:48 UTC
Created attachment 40551 [details] [review]
Bug 13799: REST API proof of concept with Mojolicious/Swagger2
Comment 82 Olli-Antti Kivilahti 2015-06-24 08:35:06 UTC
Created attachment 40552 [details] [review]
Bug 13799: Add versioning to API
Comment 83 Julian Maurice 2015-06-24 08:35:07 UTC
Comment on attachment 40469 [details] [review]
Rebased

Patch moved to new bug 14448
Comment 84 Olli-Antti Kivilahti 2015-06-24 08:35:24 UTC
Created attachment 40553 [details] [review]
Bug 13799: Use Koha::Objects
Comment 85 Olli-Antti Kivilahti 2015-06-24 08:35:43 UTC
Created attachment 40554 [details] [review]
Bug 13799: Add Swagger UI documentation
Comment 86 Julian Maurice 2015-06-24 08:37:48 UTC
Comment on attachment 40470 [details] [review]
Rebased

Patch moved to new bug 14448
Comment 87 Julian Maurice 2015-06-24 08:38:07 UTC
Comment on attachment 40471 [details] [review]
Bug 13799 - Nginx reverse proxy config baseline for Koha REST API. Tweaking needed to work with makefile.

Patch moved to new bug 14448
Comment 88 Julian Maurice 2015-06-24 09:12:31 UTC
Created attachment 40560 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification
  /api/v1/doc            Swagger UI (documentation)

Add a virtual host in Apache configuration api.HOSTNAME
So we have:
  http://api.HOSTNAME/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/v1/doc            Swagger UI (documentation)
  http://api.HOSTNAME/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Go to http://api.HOSTNAME/v1/doc and check everything works
  7/ Also check that http://api.HOSTNAME/v1/borrowers and
     http://api.HOSTNAME/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://api.HOSTNAME/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

For the new vhost to work, you may need to modify your /etc/hosts file
if you run Koha locally.
Comment 89 Julian Maurice 2015-06-24 09:12:36 UTC
Created attachment 40561 [details] [review]
Bug 13799: Force UTF-8 charset in responses
Comment 90 Julian Maurice 2015-06-24 09:13:24 UTC
Squashed all patches + minor encoding bug fix
Comment 91 Olli-Antti Kivilahti 2015-06-24 15:08:25 UTC
Created attachment 40584 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 92 Olli-Antti Kivilahti 2015-06-24 15:08:59 UTC
Created attachment 40585 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.
Comment 93 Olli-Antti Kivilahti 2015-06-24 15:09:39 UTC
Created attachment 40586 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 94 Olli-Antti Kivilahti 2015-06-24 15:10:56 UTC
Created attachment 40587 [details] [review]
Bug 13799 - 4. Add base for building RESTful API - Fix missing character encoding in the index.html
Comment 95 Olli-Antti Kivilahti 2015-06-24 17:45:27 UTC
The order of applying these following bugs is as follow:
13799 -> 13895 -> 13903 -> 13920
Comment 96 Julian Maurice 2015-06-25 07:01:18 UTC
Created attachment 40616 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Add a virtual host in Apache configuration api.HOSTNAME
So we have:
  http://api.HOSTNAME/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://api.HOSTNAME/v1/borrowers and
     http://api.HOSTNAME/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://api.HOSTNAME/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

For the new vhost to work, you may need to modify your /etc/hosts file
if you run Koha locally.
Comment 97 Julian Maurice 2015-06-25 07:01:32 UTC
Created attachment 40617 [details] [review]
Bug 13799: 2. Set up logging and configuration file reading for Mojolicious.

Use environmental values to control some aspects of Mojolicious:
This way we can have different settings for different servers running Mojolicious.

%%%% Configuration file %%%%

$ENV{MOJO_CONFIG} should be set in the system service (init) starting Mojolicious, eg:
export MOJO_CONFIG=/home/koha/kohaclone/api/v1/hypnotoad.conf

This configuration file read by the Mojolicious::Plugin::Config
http://mojolicio.us/perldoc/Mojolicious/Plugin/Config

%%%%% Logging %%%%%

NOTE!!
There is a "feature" in Mojo::Server disabling STDOUT and STDERR, because such errors are not-suited-for-prod~
This modification in Mojo::Server disables this and preserves the STD* handles for forked server threads
in Mojo::Server::daemonize(), comment out the following lines

  # Close filehandles
  #  open STDOUT, '>/dev/null';
  #  open STDERR, '>&STDOUT';

Log to a filename configured in an environemnt variable $ENV{MOJO_LOGFILE} using loglevel $ENV{MOJO_LOGLEVEL}.
Defaults to '/tmp/koha-api.log' and loglevel of 'error'
Examples:
export MOJO_LOGFILE=/home/koha/koha-dev/var/log/kohaapi.mojo.log
export MOJO_LOGLEVEL=debug

Logging is done by Mojo::Log
http://www.mojolicio.us/perldoc/Mojo/Log
Comment 98 Julian Maurice 2015-06-25 07:01:43 UTC
Created attachment 40618 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 99 Julian Maurice 2015-06-25 07:03:06 UTC
Swagger UI removed from the first patch and moved in its own bug 14458
Comment 100 Olli-Antti Kivilahti 2015-06-25 08:54:54 UTC
Created attachment 40619 [details] [review]
Bug 13799 - 4. Add base for building RESTful API - Fix missing character encoding in the index.html
Comment 101 Olli-Antti Kivilahti 2015-07-24 14:04:22 UTC
Created attachment 41160 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 102 Olli-Antti Kivilahti 2015-07-24 14:05:11 UTC
Created attachment 41161 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.
Comment 103 Olli-Antti Kivilahti 2015-07-24 14:06:06 UTC
Created attachment 41162 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 104 Olli-Antti Kivilahti 2015-07-31 15:10:17 UTC
Created attachment 41268 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 105 Olli-Antti Kivilahti 2015-07-31 15:10:49 UTC
Created attachment 41269 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.
Comment 106 Olli-Antti Kivilahti 2015-07-31 15:11:21 UTC
Created attachment 41270 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 107 Olli-Antti Kivilahti 2015-07-31 15:14:21 UTC
Created attachment 41271 [details] [review]
Bug 13799: 4. API Authentication, part 1: API keys management in interface

Depends on Buugg 14539 and Buugg 7174.

This introduces the concept of API keys for use in the new REST API.
A key is a string of 32 alphanumerical characters (32 is purely
arbitrary, it can be changed easily).
A user can have multiple keys (unlimited at the moment)
Keys can be generated automatically, and then we have the possibility to
delete or revoke each one individually.

ApiKeys can be easily accessed using the Koha::ApiKeys-package.

Includes unit tests and selenium integration tests for Intra and OPAC.

Test plan:
1/ Go to staff interface
2/ Go to a borrower page
3/ In toolbar, click on More -> Manage API keys
4/ Click on "Generate new key" multiple times, check that they are
   correctly displayed under the button, and they are active by default
5/ Revoke some keys, check that they are not active anymore
6/ Delete some keys, check that they disappear from table
7/ Go to opac interface, log in
8/ In your user account pages, you now have a new tab to the left "your
   API keys". Click on it.
9/ Repeat steps 4-6
Comment 108 Olli-Antti Kivilahti 2015-07-31 15:20:49 UTC
Created attachment 41272 [details] [review]
Bug 13799: 5. API authentication system - Swagtenticator authentication

Reads the Swagger2 definitions and defines the API routes and controllers
for Mojolicious.

Authentiates the API consumer using Koha::Auth::Challenge::RESTV1
with all the necessary details inferred from Swagger2, like permissions.

Validates all input to match the Swagger2 definition.

Authentication is based on the permissions defined in the Swagger2 definition.
Add x-koha-permission to the Operation Object to define needed Koha permissions
to access the resource.

Eg.
  "/borrowers/{borrowernumber}": {
    "get": {
      "x-mojo-controller": "Koha::REST::V1::Borrowers",
      "x-koha-permission": {
        "borrowers": "*"
      },
      "operationId": "getBorrower",
      "tags": ["borrowers"],
Comment 109 Olli-Antti Kivilahti 2015-07-31 15:25:03 UTC
Created attachment 41273 [details] [review]
Bug 13799 - 6. REST API Testing framework

Introduces the Swagger2TestRunner which takes care of authentication and API
consumer creation behind the scenes, and makes sure all API endpoints get
test coverage.

This way the test automator can focus on testing the business logic instead of
duplicating authentication and API consumer creation in various test scripts.

Run t/db_dependent/Api/V1/testREST.pl and it will tell you what you need to do.
Comment 110 Olli-Antti Kivilahti 2015-07-31 15:26:45 UTC
Created attachment 41274 [details]
Schematic of the API components
Comment 111 Olli-Antti Kivilahti 2015-07-31 15:28:04 UTC
*** Bug 13920 has been marked as a duplicate of this bug. ***
Comment 112 Olli-Antti Kivilahti 2015-08-03 15:13:16 UTC
Created attachment 41320 [details] [review]
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain

OPTIONAL replacement to the
    Buugg 13799: 1. RESTful API with Mojolicious and Swagger2
httpd-configuration.

Instead of having to go to your dns-provider, you can run the api from your
current virtualhost.

Use these configurations to reverse proxy www.host.domain/v1/ to the REST API.

---------------------------
:INSTALLATION INSTRUCTIONS:
---------------------------

You need
    Buugg 14448 - Adding the REST API as a Hypnotoad service, koha-api-daemon.
to deploy the backend Mojolicous application as a FastCGI-daemon.

Then add the modifications from etc/koha-httpd.conf
to your /etc/apache2/sites-enabled/koha.conf
and adjust paths to fit.

..$ a2enmod proxy
..$ a2enmod proxy_http
..$ service apache2 restart

Much recommended is
    Buugg 14458: 5. Add SwaggerUI documentation for RESTful API
to browse your API capabilities.

Then just go to
    /v1/doc
to see the API autodocumentation.
Comment 113 Olli-Antti Kivilahti 2015-08-03 16:09:43 UTC
Created attachment 41321 [details] [review]
Bug 13799: 4. API Authentication, part 1: API keys management in interface
Comment 114 Olli-Antti Kivilahti 2015-08-04 07:56:18 UTC
Created attachment 41326 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 115 Olli-Antti Kivilahti 2015-08-04 07:56:38 UTC
Created attachment 41327 [details] [review]
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain
Comment 116 Olli-Antti Kivilahti 2015-08-04 07:56:55 UTC
Created attachment 41328 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.
Comment 117 Olli-Antti Kivilahti 2015-08-04 07:57:11 UTC
Created attachment 41329 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 118 Olli-Antti Kivilahti 2015-08-04 07:57:27 UTC
Created attachment 41330 [details] [review]
Bug 13799: 4. API Authentication, part 1: API keys management in interface
Comment 119 Olli-Antti Kivilahti 2015-08-04 07:57:43 UTC
Created attachment 41331 [details] [review]
Bug 13799: 5. API authentication system - Swagtenticator authentication
Comment 120 Olli-Antti Kivilahti 2015-08-17 12:02:01 UTC
Created attachment 41538 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 121 Olli-Antti Kivilahti 2015-08-17 12:02:18 UTC
Created attachment 41539 [details] [review]
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain
Comment 122 Olli-Antti Kivilahti 2015-08-17 12:15:25 UTC
Created attachment 41540 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.
Comment 123 Olli-Antti Kivilahti 2015-08-17 12:15:42 UTC
Created attachment 41541 [details] [review]
Bug 13799: 3. Force UTF-8 charset in responses
Comment 124 Olli-Antti Kivilahti 2015-08-17 12:16:00 UTC
Created attachment 41542 [details] [review]
Bug 13799: 4. API Authentication, part 1: API keys management in interface
Comment 125 Olli-Antti Kivilahti 2015-08-17 12:16:22 UTC
Created attachment 41543 [details] [review]
Bug 13799: 5. API authentication system - Swagtenticator authentication
Comment 126 Olli-Antti Kivilahti 2015-08-17 12:17:36 UTC
Created attachment 41547 [details] [review]
Bug 13799 - 6. REST API Testing framework
Comment 127 Olli-Antti Kivilahti 2015-08-17 13:04:13 UTC
Created attachment 41554 [details] [review]
Bug 13799: 1. RESTful API with Mojolicious and Swagger2
Comment 128 Julian Maurice 2015-08-27 17:01:14 UTC
Comment on attachment 41540 [details] [review]
Bug 13799 - 2. Set up logging and configuration file reading for Mojolicious.

Patch moved to bug 14746
Comment 129 Julian Maurice 2015-08-27 17:10:41 UTC
Comment on attachment 41539 [details] [review]
Bug 13799 - 1.1 RESTful API with reverse proxy configuration without api.host.domain

Patch moved to bug 14747
Comment 130 Julian Maurice 2015-08-27 17:24:23 UTC
Comment on attachment 41542 [details] [review]
Bug 13799: 4. API Authentication, part 1: API keys management in interface

Patch moved to bug 13920
Comment 131 Julian Maurice 2015-08-27 17:24:44 UTC
Comment on attachment 41543 [details] [review]
Bug 13799: 5. API authentication system - Swagtenticator authentication

Patch moved to bug 13920
Comment 132 Julian Maurice 2015-08-27 17:28:42 UTC
Comment on attachment 41547 [details] [review]
Bug 13799 - 6. REST API Testing framework

Patch moved to bug 14748
Comment 133 Julian Maurice 2015-08-27 17:30:14 UTC
Created attachment 42044 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/v1/borrowers and
     http://(OPAC|INTRANET)/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error
Comment 134 Julian Maurice 2015-08-27 17:34:41 UTC
All patches moved to their own bugs so we have a simple patch that just works (I hope...) and without a huge dependency tree.
It should make testing a lot easier.

A feature is still missing however: the cookie-based authentication. I'll try to work on that soon.
Comment 135 Julian Maurice 2015-08-27 17:39:03 UTC
(In reply to Julian Maurice from comment #134)
> A feature is still missing however: the cookie-based authentication. I'll
> try to work on that soon.
... but feel free to test the patch right now! :)
Comment 136 Alex Arnaud 2015-08-28 14:52:53 UTC
Created attachment 42085 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/v1/borrowers and
     http://(OPAC|INTRANET)/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Comment 137 Alex Arnaud 2015-08-28 14:58:34 UTC
Created attachment 42086 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/api/v1/borrowers and
     http://(OPAC|INTRANET)/api/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Comment 138 Alex Arnaud 2015-08-28 15:02:21 UTC
reattached the patch because i've changed the webservices url in the commit message (http://(OPAC|INTRANET)/api/v1/borrowers instead of http://(OPAC|INTRANET)/v1/borrowers)
Comment 139 Julian Maurice 2015-08-31 15:02:33 UTC
Created attachment 42123 [details] [review]
Bug 13799: Add cookie-based authentication to REST API
Comment 140 Julian Maurice 2015-09-01 06:24:59 UTC
The 'borrowers' permission is required for both /borrowers and /borrowers/XXXX except if XXXX is the borrowernumber of the loggedinuser

Test plan for authentication:
  1/ Log in to staff interface with a borrower that have 'borrowers' permission
  2/ Go to http://INTRANET/api/v1/borrowers and http://INTRANET/api/v1/borrowers/XXXX (where XXXX is a valid borrowernumber). You should see borrowers data
  3/ Remove the 'borrowers' permission
  4/ Go to http://INTRANET/api/v1/borrowers. You should see an error (and HTTP code 403)
  5/ Go to http://INTRANET/api/v1/borrowers/XXXX (where XXXX is a valid borrowernumber different from the logged-in user's borrowernumber). You should see an error (and HTTP code 403)
  6/ Go to http://INTRANET/api/v1/borrowers/XXXX (where XXXX is the logged-in user's borrowernumber). You should see borrower's data.
Comment 141 Zeno Tajoli 2015-09-01 08:07:17 UTC
Patch complexity is 'Large' because this change has many architectural connections
Comment 142 Olli-Antti Kivilahti 2015-09-08 12:48:57 UTC
Hi there!

We cannot push this bug to master without also pushing Bugs 13920 and Bug 14748.
Bug 14748 doesn't need to be implemented like I implemented it, but we must have a established and formal way of doing the complete authentication, permission checking and formal enforced testing, before unleashing this monster to master.

If we push just this Bug, people will start creating features on top of this one, and then we will lose the opportunity to integrate automatic documentation driven permission checking and a formal testing framework (which simplifies test writing a lot).
It will be much harder to include them later on.
Comment 143 Julian Maurice 2015-09-09 09:19:12 UTC
Yes, we can, and we should, as we agreed on that during the previous dev meeting.
It's better to have a tiny patch in master than a big, full-featured patch that will never be pushed because of its complexity.
And the API key authentication system needs some discussion, as some people disagree with it.
Comment 144 Olli-Antti Kivilahti 2015-09-14 12:36:35 UTC
Created attachment 42518 [details] [review]
Bug 13799 - Swagger2-driven Permission checking

A hasty downgrade from 13920, utilizing new features implemented by the
Mojolicious::Plugin::Swagger2-author, to make it possible to implement more
complex authentication/authorization scenarios with the Plugin.

Define 'x-koha-permission' for the Swagger2 Operation Object, to automatically
authorize against the required permissions.

This way we immediately tell the API consumer in the Swagger2-definition, which
permissions are needed to access defined resources.
Also we don't need to maintain permissions in multiple locations and we can build
a smart testing framework to help a lot in creating tests for the new REST API.
Comment 145 Olli-Antti Kivilahti 2015-09-14 12:39:50 UTC
Hi!

Had a discussion in IRC with jajm and ashimema about REST API permission checking. And since the author of Mojolicious:Plugin::Swagger2 has kindly improved the plugin per the discussions I have had with him.
I decided to utilize the new around-action -hook of the Swagger2-plugin to implement Swagger2-driven permission checking for the REST API authentication mechanism.

I really need to start migrating another legacy ILS to Koha, but it would be a real shame to let this great improvement pass.
Comment 146 j.kylmala 2015-09-14 13:53:32 UTC
Commit "RESTful API with Mojolicious and Swagger2" is missing license notices from the start of some files.
Comment 147 Julian Maurice 2015-09-15 07:08:35 UTC
I have a few concerns using x-mojo-around feature of Swagger2:
1/ It is marked as experimental (from the POD: "This feature is EXPERIMENTAL and can change without notice.")
2/ It is introduced only in Swagger2 0.45 and thus we'll need to repackage it for Debian (packaged version in debian.koha-community.org is only 0.31)

I think these two points will block integration, so I'm wondering if it's not better to include the patch in a new bug. What do you think ?
Comment 148 Julian Maurice 2015-09-22 09:01:32 UTC
No movement for a week, I'll move the last patch into its own bug.
Comment 149 Julian Maurice 2015-09-22 09:03:54 UTC
Comment on attachment 42518 [details] [review]
Bug 13799 - Swagger2-driven Permission checking

Patch moved to its own bug (bug 14868)
Comment 150 Julian Maurice 2015-09-22 09:09:32 UTC
Created attachment 42766 [details] [review]
Bug 13799: Add missing license notices
Comment 151 j.kylmala 2015-09-22 10:43:47 UTC
Created attachment 42768 [details] [review]
Bug 13799: Add missing license notices

Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>
Comment 152 Tomás Cohen Arazi 2015-09-22 20:23:47 UTC
I tested the patches successfully on a dev install. I didn't manage to have it working on a packages+Plack environment. Once I have the needed followups I'll sign this successfull implementation.
Comment 153 Julian Maurice 2015-09-23 08:07:48 UTC
Created attachment 42801 [details] [review]
Bug 13799: Fix API unit tests

They were failing because of the now required 'borrowers' permission
Comment 154 Olli-Antti Kivilahti 2015-09-24 10:51:45 UTC
Just a reminder before I scurry off again.

We also need to solve how to use Etags to prevent mid-air collissions when modifying DB-objects.

This is currently a BIG issue in Koha and causes very hard to track and understand issues.
Getting it solved using the new REST API would be super awesome and 100% inline (actually a REQUIREMENT) with the REST architecture.
Comment 155 Tomás Cohen Arazi 2015-09-25 13:00:35 UTC
Created attachment 42874 [details] [review]
[SIGNED OFF] Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/api/v1/borrowers and
     http://(OPAC|INTRANET)/api/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 156 Tomás Cohen Arazi 2015-09-25 13:00:46 UTC
Created attachment 42875 [details] [review]
[SIGNED OFF] Bug 13799: Add cookie-based authentication to REST API

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 157 Tomás Cohen Arazi 2015-09-25 13:00:58 UTC
Created attachment 42876 [details] [review]
[SIGNED OFF] Bug 13799: Add missing license notices

Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 158 Tomás Cohen Arazi 2015-09-25 13:01:17 UTC
Created attachment 42877 [details] [review]
[SIGNED OFF] Bug 13799: Fix API unit tests

They were failing because of the now required 'borrowers' permission

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 159 Tomás Cohen Arazi 2015-09-25 13:10:22 UTC
Created attachment 42878 [details] [review]
[SIGNED OFF] Bug 13799: (QA followup) make tests use random data

With the introduction of TestBuilder there's no need to
rely on existing data on the DB (the original patch relies
on categories and branches already existing).

This patch creates a random branch, category and two borrowers
with the fixed data that is needed for the tests. It adjusts
the tests to use the randomized data instead of the previously
fixed one.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The feature works as expected and all tests passes. koha-qa.pl too.
Comment 160 Kyle M Hall 2015-10-06 18:45:07 UTC
Testing so far looks good! A couple things I'd like to address before passing qa:
1) Accessing invalid paths gives us a Mojo generated page that gives data that while useful for development should probably just give us 404's in production. How do you think we should address this?

2) One of the bits of data from that invalid page leads me to this: http://mojolicio.us/perldoc/Mojolicious/Guides/FAQ#What-does-Your-secret-passphrase-needs-to-be-changed-mean
   Is this something we should be setting? Perhaps a hard coded default secret for Koha and the option to specify a different one in koha-conf.xml?

3) A wiki page for RESTful api developer guidelines would be most helpful. Nothing too fancy, just a "here are the bits you need to change and add to extend the REST api". The code itself is a good document but an actual guide would certainly be helpful.

Setting to failed qa until 1 and 2 are addressed. 3 is more of a wish list item.
Comment 161 Jonathan Druart 2015-10-07 08:38:23 UTC
I have also started QAing this one yesterday.
Back today with few comments:
1/ 404 is not documented in the swagger.json file
2/ The properties for "borrowers" are not exhaustive
3/ Wording: borrowers vs patrons? "Patron" should be used now, especially for new code.
4/ this patch set introduces swagger.json but nothing to use it, how is it useful?
5/ We need a way to split this json file, it will be a mess to edit!
And something to validate it. We need guidelines.
6/ Koha::Borrowers->find is called in Koha::REST::V1::Borrowers::get_borrower and Koha::REST::V1->startup (?)
Comment 162 Jonathan Druart 2015-10-07 08:42:13 UTC
(In reply to Jonathan Druart from comment #161)
> 6/ Koha::Borrowers->find is called in
> Koha::REST::V1::Borrowers::get_borrower and Koha::REST::V1->startup (?)

Forget that!
(I am hidden already).
Comment 163 Jonathan Druart 2015-10-07 08:53:35 UTC
(In reply to Jonathan Druart from comment #161)
> 1/ 404 is not documented in the swagger.json file

Forget that again, it is!
Comment 164 Julian Maurice 2015-10-07 09:12:58 UTC
Created attachment 43186 [details] [review]
Bug 13799: Change Mojolicious default mode to 'production'

This avoid getting a debug page when URL is wrong.
Comment 165 Julian Maurice 2015-10-07 09:13:03 UTC
Created attachment 43187 [details] [review]
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF
Comment 166 Julian Maurice 2015-10-07 09:30:19 UTC
(In reply to Jonathan Druart from comment #161)
> 2/ The properties for "borrowers" are not exhaustive
> 3/ Wording: borrowers vs patrons? "Patron" should be used now, especially
> for new code.

Ok, these points need to be fixed

> 4/ this patch set introduces swagger.json but nothing to use it, how is it
> useful?

swagger.json, apart from being the api documentation, is also used by Mojolicious' plugin Swagger2 to create the routes (map URL to controller/action) and do some basic type check on input and output. So it is useful (or at least, used) even without documentation generators like SwaggerUI

> 5/ We need a way to split this json file, it will be a mess to edit!

There is an issue on Github for splitting the JSON spec
https://github.com/jhthorsen/swagger2/issues/33
which was moved to
https://github.com/jhthorsen/json-validator/issues/4
which ends with a commit in master
https://github.com/jhthorsen/json-validator/commit/5d7d9db22b891758ac979cc87f1ab018030df2c2

It seems there is something to test here.

> And something to validate it. We need guidelines.

What kind of validation ?
Comment 167 Kyle M Hall 2015-10-07 11:40:23 UTC
(In reply to Julian Maurice from comment #166)
> (In reply to Jonathan Druart from comment #161)
> > 2/ The properties for "borrowers" are not exhaustive
> > 3/ Wording: borrowers vs patrons? "Patron" should be used now, especially
> > for new code.
> 
> Ok, these points need to be fixed

Agreed. The external api terminology must be patron / patrons at the least. At some point we need to change Koha::Borrower(s) to Koha::Patron(s) but that is entirely separate and distinct from this work.
 
> > 5/ We need a way to split this json file, it will be a mess to edit!
> 
> There is an issue on Github for splitting the JSON spec
> https://github.com/jhthorsen/swagger2/issues/33
> which was moved to
> https://github.com/jhthorsen/json-validator/issues/4
> which ends with a commit in master
> https://github.com/jhthorsen/json-validator/commit/
> 5d7d9db22b891758ac979cc87f1ab018030df2c2
> 
> It seems there is something to test here.

Agreed. It would be *much* better if we can split this file up. If the Swagger2 version we are using doesn't support it, we may have to table this issue until a newer version is available for us to use with Koha.
 
> > And something to validate it. We need guidelines.
> 
> What kind of validation ?

I think a unit test using Test::JSON::is_valid_json should be sufficient.
Comment 168 Kyle M Hall 2015-10-07 11:40:47 UTC
Also, thanks for the followups Julian!
Comment 169 Julian Maurice 2015-10-07 12:21:33 UTC
Created attachment 43189 [details] [review]
Bug 13799: Change 'required' value type from string to boolean

String is not accepted as a valid value in more recent versions of
Swagger2 module
Comment 170 Martin Renvoize 2015-10-07 12:39:27 UTC
I've been working allot lately with the developer of the Swagger2 perl tools. We have got $ref support working well now, with relative paths, full paths and url's all passing tests.  I asusme it's this piece you were refering to when talking about splitting up the spec file khall?


Regarding 'patrons' vs 'borrowers' can we not use the 'americanised' term pretty please.. to me it would be much better to generalise it.. they're not limited to borrowers/patrons at all.. they're 'users' ;)

Back on topic, I'll be looking at all this stuff today.. great work Julian in my brief look so far.
Comment 171 Kyle M Hall 2015-10-07 13:29:03 UTC
(In reply to Martin Renvoize from comment #170)
> I've been working allot lately with the developer of the Swagger2 perl
> tools. We have got $ref support working well now, with relative paths, full
> paths and url's all passing tests.  I asusme it's this piece you were
> refering to when talking about splitting up the spec file khall?

Yes, I believe so.

> Regarding 'patrons' vs 'borrowers' can we not use the 'americanised' term
> pretty please.. to me it would be much better to generalise it.. they're not
> limited to borrowers/patrons at all.. they're 'users' ;)

The general consensus has been for internal terminology to match outward terminology. Right now we use patrons, members and borrowers internally. Since we use patrons externally we should use it internally. I we want to change the nomenclature further we should put it to a dev meeting vote!

> Back on topic, I'll be looking at all this stuff today.. great work Julian
> in my brief look so far.

Excellent!
Comment 172 Martin Renvoize 2015-10-07 14:25:07 UTC
Firs comment: 

Unless we're actually going to use Mojolicious Sessions, and thus mojo cookies then we need not confuse users by making the secret configurable.

As far as I can tell, secrets are only used to sign mojo cookies (Whether we should eventually switch to mojo hmac signed cookies over csgisessid is another question all together)

As such, I would suggest we hard code a suitably silly secret just to quell warnings that will be output otherwise safe in the knowledge that we are not actually using said secret anyway.
Comment 173 Martin Renvoize 2015-10-07 14:38:13 UTC
x-mojo-controller has recently been deprecated, though not entirely removed from the Swagger2 Mojo Plugin, withe the prefernce now being that the core swagger key 'operationId' is used in preference to it.

I'm happy to submit a patch that updates both our spec and the resultant method names in the Borrowers controller.
Comment 174 Martin Renvoize 2015-10-07 14:46:15 UTC
Error responses..  I see you've added a nice simple schema for validating error responses.

However, there seems to be a general move toward standardising on the error format at described by the Swagger2 plugins error handler..

Perhaps we should follow suite and adopt that same format, which is pretty darn close to what we have already (with the exception that it allows for an array of errors with messages as a posed to just a single error message string).

I'm also happy to supply a patch if this is how we wish to proceed.
Comment 175 Martin Renvoize 2015-10-07 14:51:41 UTC
Created attachment 43190 [details] [review]
Bug 13799 - Followup: x-mojo-controller deprecation

Remove the use of soon to be deprecated x-mojo-controller from our
specification and replace with the recommended operationId format.
Comment 176 Julian Maurice 2015-10-07 15:21:44 UTC
Created attachment 43191 [details] [review]
Bug 13799: Reword 'borrower' to 'patron'

Also, complete the Swagger definition of a patron
Comment 177 Martin Renvoize 2015-10-07 15:22:01 UTC
Created attachment 43192 [details] [review]
Bug 13799 - Followup: Refactor spec file

Spreading the specification over multiple files should lead to a more
manageable specification long term
Comment 178 Martin Renvoize 2015-10-07 19:21:10 UTC
Created attachment 43204 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/api/v1/borrowers and
     http://(OPAC|INTRANET)/api/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 179 Martin Renvoize 2015-10-07 19:21:15 UTC
Created attachment 43205 [details] [review]
Bug 13799: Add cookie-based authentication to REST API

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 180 Martin Renvoize 2015-10-07 19:21:20 UTC
Created attachment 43206 [details] [review]
Bug 13799: Add missing license notices

Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 181 Martin Renvoize 2015-10-07 19:21:25 UTC
Created attachment 43207 [details] [review]
Bug 13799: Fix API unit tests

They were failing because of the now required 'borrowers' permission

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 182 Martin Renvoize 2015-10-07 19:21:30 UTC
Created attachment 43208 [details] [review]
Bug 13799: (QA followup) make tests use random data

With the introduction of TestBuilder there's no need to
rely on existing data on the DB (the original patch relies
on categories and branches already existing).

This patch creates a random branch, category and two borrowers
with the fixed data that is needed for the tests. It adjusts
the tests to use the randomized data instead of the previously
fixed one.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The feature works as expected and all tests passes. koha-qa.pl too.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 183 Martin Renvoize 2015-10-07 19:21:34 UTC
Created attachment 43209 [details] [review]
Bug 13799: Change Mojolicious default mode to 'production'

This avoid getting a debug page when URL is wrong.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 184 Martin Renvoize 2015-10-07 19:21:39 UTC
Created attachment 43210 [details] [review]
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 185 Martin Renvoize 2015-10-07 19:21:44 UTC
Created attachment 43211 [details] [review]
Bug 13799: Change 'required' value type from string to boolean

String is not accepted as a valid value in more recent versions of
Swagger2 module

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 186 Martin Renvoize 2015-10-07 19:21:49 UTC
Created attachment 43212 [details] [review]
Bug 13799: Reword 'borrower' to 'patron'

Also, complete the Swagger definition of a patron

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 187 Martin Renvoize 2015-10-07 19:21:54 UTC
Created attachment 43213 [details] [review]
Bug 13799 - Followup: x-mojo-controller deprecation

Remove the use of soon to be deprecated x-mojo-controller from our
specification and replace with the recommended operationId format.
Comment 188 Martin Renvoize 2015-10-07 19:21:59 UTC
Created attachment 43214 [details] [review]
Bug 13799 - Followup: Refactor spec file

Spreading the specification over multiple files should lead to a more
manageable specification long term
Comment 189 Martin Renvoize 2015-10-07 19:26:37 UTC
Rebased my two patches (marked with 'Followup' keyword) and signed off on prior patches.

QA Note, let's get he patrons example right and not only define all fields with descriptions but actually define types so validation may become useful.

I'm happy to do this as another followup.
Comment 190 Martin Renvoize 2015-10-07 19:31:00 UTC
My patches require the latest versions of Swagger2 and JSON::Validator modules.  I highly recommend especially whilst in development that we stick to the latest versions of these modules...  The code is moving fast and becoming more and more refined almost daily.

The latest changes include..

* Validating the swagger.json specification file against the swagger schema during application startup (This should prevent 'bad schema' making it into release).
* Correct handling of $ref json-references in schemata
* Stricter coercion functions and validation (JSON::Boolean and Integer coercion handling improvements)

and various other bugfixes.
Comment 191 Kyle M Hall 2015-10-08 11:38:01 UTC
(In reply to Martin Renvoize from comment #189)
> Rebased my two patches (marked with 'Followup' keyword) and signed off on
> prior patches.
> 
> QA Note, let's get he patrons example right and not only define all fields
> with descriptions but actually define types so validation may become useful.
> 
> I'm happy to do this as another followup.

Please do! Once you've got that dealt with I'll continue with QA
Comment 192 Julian Maurice 2015-10-26 09:10:11 UTC
I'm having problems with the latest patches

When I go to http://pro.koha-dev/api/v1/patrons/7680, error:

  Can't use string ("error.json") as a HASH ref while "strict refs" in use at /usr/local/share/perl/5.14.2/JSON/Validator.pm line 515.

If I remove patch "Followup: Refactor spec file", another error:

  {"errors":[{"path":"\/","message":"Controller could not be loaded."}]}

Then, if I remove patch "Followup: x-mojo-controller deprecation", everyting is ok.

$ pmvers JSON::Validator
0.59

$ pmvers Swagger2
0.60
Comment 193 Julian Maurice 2015-10-26 10:59:56 UTC
Created attachment 43980 [details] [review]
Bug 13799: Followup of "Followup: Refactor spec file"

Fix Swagger2 definitions file
Comment 194 Julian Maurice 2015-10-26 11:00:02 UTC
Created attachment 43981 [details] [review]
Bug 13799: Followup of "Followup: x-mojo-controller deprecation"

Rename controller Koha::REST::V1::Patrons to Koha::REST::V1::Patron so
Swagger2 can find it

Also remove most of the "type" specifications in patron schema object,
since most of the fields can be null, and Swagger2 does not support
anyOf nor oneOf
Comment 195 Martin Renvoize 2015-10-26 12:16:49 UTC
Removing 'type' from the definitions removes validation of the requests and responses.. that loses 2/3rds of the reason for using swagger with the final 3rd being the documentation of the api which by removing 'types' your also massively limiting the usefulness of.

If we're not using 'null' as a meaningful value then we should not be using it at all.  Also, the definitions still do not contain a 'requires' block which should really be present to show what fields are always required.  For a fully restful api, this would be all of them for POST, PUT and GET as rest is all about dealing with full documents.
Comment 196 Julian Maurice 2015-10-26 12:59:34 UTC
Created attachment 43987 [details] [review]
Bug 13799: Followup of "Followup: x-mojo-controller deprecation"

Rename controller Koha::REST::V1::Patrons to Koha::REST::V1::Patron so
Swagger2 can find it

Also change most of the  "type" specifications in patron schema object,
since most of the fields can be null.
Comment 197 Martin Renvoize 2015-10-26 13:06:15 UTC
Started writing some guidelines: http://wiki.koha-community.org/wiki/Coding_Guidelines_-_API

Still very much a work in progress
Comment 198 Martin Renvoize 2015-10-26 13:09:01 UTC
My current feeling now is that this is a great base and should be pushed sooner rather than later.

I expect as we refine the guidelines and documentation I will continue to update the Patrons route to conform to the evolving guidelines.

As such, I don't see any of the outstanding comments being blockers for the push.
Comment 199 Paul Poulain 2015-10-26 13:17:34 UTC
(In reply to Martin Renvoize from comment #198)
> My current feeling now is that this is a great base and should be pushed
> sooner rather than later.
+1
Comment 200 Kyle M Hall 2015-10-26 13:37:52 UTC
Created attachment 43989 [details] [review]
[PASSED QA] Bug 13799: RESTful API with Mojolicious and Swagger2

Actual routes are:
  /borrowers
    Return a list of all borrowers in Koha

  /borrowers/{borrowernumber}
    Return the borrower identified by {borrowernumber}
    (eg. /borrowers/1)

There is a test file you can run with:
  $ prove t/db_dependent/rest/borrowers.t

All API stuff is in /api/v1 (except Perl modules)
So we have:
  /api/v1/script.cgi     CGI script
  /api/v1/swagger.json   Swagger specification

Change both OPAC and Intranet VirtualHosts to access the API,
so we have:
  http://OPAC/api/v1/swagger.json   Swagger specification
  http://OPAC/api/v1/{path}         API endpoint
  http://INTRANET/api/v1/swagger.json   Swagger specification
  http://INTRANET/api/v1/{path}         API endpoint

Add a (disabled) virtual host in Apache configuration api.HOSTNAME,
so we have:
  http://api.HOSTNAME/api/v1/swagger.json   Swagger specification
  http://api.HOSTNAME/api/v1/{path}         API endpoint

Add 'unblessed' subroutines to both Koha::Objects and Koha::Object to be
able to pass it to Mojolicious

Test plan:
  1/ Install Perl modules Mojolicious and Swagger2
  2/ perl Makefile.PL
  3/ make && make install
  4/ Change etc/koha-httpd.conf and copy it to the right place if needed
  5/ Reload Apache
  6/ Check that http://(OPAC|INTRANET)/api/v1/borrowers and
     http://(OPAC|INTRANET)/api/v1/borrowers/{borrowernumber} works

Optionally, you could verify that http://(OPAC|INTRANET)/vX/borrowers
(where X is an integer greater than 1) returns a 404 error

Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 201 Kyle M Hall 2015-10-26 13:38:14 UTC
Created attachment 43990 [details] [review]
[PASSED QA] Bug 13799: Add cookie-based authentication to REST API

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 202 Kyle M Hall 2015-10-26 13:38:19 UTC
Created attachment 43991 [details] [review]
[PASSED QA] Bug 13799: Add missing license notices

Signed-off-by: Joonas Kylmälä <j.kylmala@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 203 Kyle M Hall 2015-10-26 13:38:24 UTC
Created attachment 43992 [details] [review]
[PASSED QA] Bug 13799: Fix API unit tests

They were failing because of the now required 'borrowers' permission

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 204 Kyle M Hall 2015-10-26 13:38:29 UTC
Created attachment 43993 [details] [review]
[PASSED QA] Bug 13799: (QA followup) make tests use random data

With the introduction of TestBuilder there's no need to
rely on existing data on the DB (the original patch relies
on categories and branches already existing).

This patch creates a random branch, category and two borrowers
with the fixed data that is needed for the tests. It adjusts
the tests to use the randomized data instead of the previously
fixed one.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The feature works as expected and all tests passes. koha-qa.pl too.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 205 Kyle M Hall 2015-10-26 13:38:33 UTC
Created attachment 43994 [details] [review]
[PASSED QA] Bug 13799: Change Mojolicious default mode to 'production'

This avoid getting a debug page when URL is wrong.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 206 Kyle M Hall 2015-10-26 13:38:38 UTC
Created attachment 43995 [details] [review]
[PASSED QA] Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 207 Kyle M Hall 2015-10-26 13:38:42 UTC
Created attachment 43996 [details] [review]
[PASSED QA] Bug 13799: Change 'required' value type from string to boolean

String is not accepted as a valid value in more recent versions of
Swagger2 module

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 208 Kyle M Hall 2015-10-26 13:38:47 UTC
Created attachment 43997 [details] [review]
[PASSED QA] Bug 13799: Reword 'borrower' to 'patron'

Also, complete the Swagger definition of a patron

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 209 Kyle M Hall 2015-10-26 13:38:52 UTC
Created attachment 43998 [details] [review]
[PASSED QA] Bug 13799 - Followup: x-mojo-controller deprecation

Remove the use of soon to be deprecated x-mojo-controller from our
specification and replace with the recommended operationId format.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 210 Kyle M Hall 2015-10-26 13:38:57 UTC
Created attachment 43999 [details] [review]
[PASSED QA] Bug 13799 - Followup: Refactor spec file

Spreading the specification over multiple files should lead to a more
manageable specification long term

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 211 Kyle M Hall 2015-10-26 13:39:01 UTC
Created attachment 44000 [details] [review]
[PASSED QA] Bug 13799: Followup of "Followup: Refactor spec file"

Fix Swagger2 definitions file

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 212 Kyle M Hall 2015-10-26 13:39:06 UTC
Created attachment 44001 [details] [review]
[PASSED QA] Bug 13799: Followup of "Followup: x-mojo-controller deprecation"

Rename controller Koha::REST::V1::Patrons to Koha::REST::V1::Patron so
Swagger2 can find it

Also change most of the  "type" specifications in patron schema object,
since most of the fields can be null.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 213 Julian Maurice 2015-10-30 07:50:23 UTC
Created attachment 44185 [details] [review]
Bug 13799: Rename t/.../v1/borrowers.t to t/.../v1/patrons.t
Comment 214 Julian Maurice 2015-11-02 07:06:44 UTC
Required versions of dependencies are now in Debian Sid

https://packages.debian.org/sid/libswagger2-perl
https://packages.debian.org/sid/libjson-validator-perl

Now we need to re-package them for debian.koha-community.org
Comment 215 Julian Maurice 2015-11-02 07:11:47 UTC
Created attachment 44285 [details] [review]
Bug 13799: Update dependencies
Comment 216 Robin Sheat 2015-11-02 21:53:20 UTC
(In reply to Julian Maurice from comment #214)
> Required versions of dependencies are now in Debian Sid
> 
> https://packages.debian.org/sid/libswagger2-perl
> https://packages.debian.org/sid/libjson-validator-perl
> 
> Now we need to re-package them for debian.koha-community.org

(In reply to Julian Maurice from comment #215)
> Created attachment 44285 [details] [review] [review]
> Bug 13799: Update dependencies

Is it necessary to update those versions? Because the older versions are in the repo as it is.

I haven't tried yet, but I suspect that JSON::Validator will be hard to port to wheezy, as it requires packaging a newer mojolicious, and I fear we'll end up in a bit of dependency hell. I'll give it a go though, maybe it'll be surprisingly easy.
Comment 217 Robin Sheat 2015-11-03 03:54:31 UTC
Created attachment 44313 [details]
Dependency tree for Swagger2 0.60

(In reply to Julian Maurice from comment #214)
> Required versions of dependencies are now in Debian Sid
> 
> https://packages.debian.org/sid/libswagger2-perl
> https://packages.debian.org/sid/libjson-validator-perl
> 
> Now we need to re-package them for debian.koha-community.org

I had a look, and I don't think it's going to happen. Attached is the dependency tree that I worked out I'd need to build. The red edge is where the process becomes difficult, and I'm not going to backport a new Perl version.
Comment 218 Robin Sheat 2015-11-03 04:10:11 UTC
(In reply to Robin Sheat from comment #217)
> I had a look, and I don't think it's going to happen. Attached is the
> dependency tree that I worked out I'd need to build. The red edge is where
> the process becomes difficult, and I'm not going to backport a new Perl
> version.

Actually, it's possible I spoke too soon. But it's still going to be a bit of work, so in the interests of laziness, how important are these new versions?

There's still no guarantee that it'll work overall, but I may have found a way around the perl version issue.
Comment 219 Julian Maurice 2015-11-03 10:32:43 UTC
(In reply to Robin Sheat from comment #218)
> Actually, it's possible I spoke too soon. But it's still going to be a bit
> of work, so in the interests of laziness, how important are these new
> versions?

The newer version of Swagger2 module is useful to be able to split the swagger.json specification file into multiple pieces (so we don't have a single huge unreadable file)
I don't know if there are others benefits.
Comment 220 Martin Renvoize 2015-11-03 10:43:38 UTC
We jumped on the Swagger2 Plugin way too early basically.

I've been developing an app based around it for a few months now... and have been working closely with the plugin author to write tests, fix issues and make it stable.. it's only just hitting the point where it's a stable and reliable platform for us to work atop so I wouldn't suggest using earlier versions. (Unless we wan't to fork it and maintain that fork.. which I think is a terrible idea).

For further info, I wouldn't even consider using Mojolicious in a package environment if we could avoid it.. I've voiced this before though ;).  Mojolicious is a state of the art library that's pretty fast moving.  I use it in our companies apps, but we're running in a Carton + Plenv environment and I wouldn't really advocate running a Mojo stack under much else.
Comment 221 Olli-Antti Kivilahti 2015-11-03 12:46:40 UTC
Being able to split the swagger.json to many files is really important.
The single file will become too big too soon.
Comment 222 Tomás Cohen Arazi 2015-11-03 12:50:45 UTC
(In reply to Olli-Antti Kivilahti from comment #221)
> Being able to split the swagger.json to many files is really important.
> The single file will become too big too soon.

Yeah, but we can leave that for the next release; at that point we won't have the compromise to suport Wheezy or Trusty at all.
Comment 223 Julian Maurice 2015-11-03 15:59:44 UTC
(In reply to Tomás Cohen Arazi from comment #222)
> (In reply to Olli-Antti Kivilahti from comment #221)
> > Being able to split the swagger.json to many files is really important.
> > The single file will become too big too soon.
> 
> Yeah, but we can leave that for the next release; at that point we won't
> have the compromise to suport Wheezy or Trusty at all.

So, what should we do ?
Moving the patches that need more recent versions of Perl modules into their own bug and push remaining patches into master now (so they go in 3.22) ?
Or wait for 3.22 to be released and then push the whole bug's patches for 3.24 ?

Backporting Perl to oldstable doesn't seem to be the way to go
Comment 224 Julian Maurice 2015-11-04 14:06:52 UTC
Created attachment 44375 [details] [review]
Bug 13799: RESTful API with Mojolicious and Swagger2

Rebased on master
Comment 225 Julian Maurice 2015-11-04 14:07:14 UTC
Created attachment 44376 [details] [review]
Bug 13799: Add cookie-based authentication to REST API

Rebased on master
Comment 226 Julian Maurice 2015-11-04 14:10:48 UTC
Created attachment 44377 [details] [review]
Bug 13799: Add missing license notices

Rebased on master
Comment 227 Julian Maurice 2015-11-04 14:11:42 UTC
Created attachment 44378 [details] [review]
Bug 13799: Fix API unit tests

Rebased on master
Comment 228 Julian Maurice 2015-11-04 14:12:02 UTC
Created attachment 44379 [details] [review]
Bug 13799: (QA followup) make tests use random data

Rebased on master
Comment 229 Julian Maurice 2015-11-04 14:12:23 UTC
Created attachment 44380 [details] [review]
Bug 13799: Change Mojolicious default mode to 'production'

Rebased on master
Comment 230 Julian Maurice 2015-11-04 14:12:41 UTC
Created attachment 44381 [details] [review]
Bug 13799: Allow to set Mojolicious secret in $KOHA_CONF

Rebased on master
Comment 231 Julian Maurice 2015-11-04 14:13:06 UTC
Created attachment 44382 [details] [review]
Bug 13799: Change 'required' value type from string to boolean

Rebased on master
Comment 232 Julian Maurice 2015-11-04 14:13:28 UTC
Created attachment 44383 [details] [review]
Bug 13799: Reword 'borrower' to 'patron'

Rebased on master
Comment 233 Julian Maurice 2015-11-04 14:13:54 UTC
Created attachment 44384 [details] [review]
Bug 13799: Rename t/.../v1/borrowers.t to t/.../v1/patrons.t

Rebased on master
Comment 234 Julian Maurice 2015-11-04 14:14:17 UTC
Created attachment 44385 [details] [review]
Bug 13799: Add types for patron's fields in Swagger spec
Comment 235 Julian Maurice 2015-11-04 14:15:14 UTC
Comment on attachment 43998 [details] [review]
[PASSED QA] Bug 13799 - Followup: x-mojo-controller deprecation

Patch moved to bug 15126
Comment 236 Julian Maurice 2015-11-04 14:15:18 UTC
Comment on attachment 43999 [details] [review]
[PASSED QA] Bug 13799 - Followup: Refactor spec file

Patch moved to bug 15126
Comment 237 Julian Maurice 2015-11-04 14:15:24 UTC
Comment on attachment 44000 [details] [review]
[PASSED QA] Bug 13799: Followup of "Followup: Refactor spec file"

Patch moved to bug 15126
Comment 238 Julian Maurice 2015-11-04 14:15:29 UTC
Comment on attachment 44001 [details] [review]
[PASSED QA] Bug 13799: Followup of "Followup: x-mojo-controller deprecation"

Patch moved to bug 15126
Comment 239 Julian Maurice 2015-11-04 14:15:35 UTC
Comment on attachment 44285 [details] [review]
Bug 13799: Update dependencies

Patch moved to bug 15126
Comment 240 Julian Maurice 2015-11-04 14:17:09 UTC
Comment on attachment 44313 [details]
Dependency tree for Swagger2 0.60

Moved to bug 15126
Comment 241 Julian Maurice 2015-11-04 14:18:36 UTC
Re-tested with Swagger2 0.31, Mojolicious 0.54.
Tests pass, all is working well.
Comment 242 Julian Maurice 2015-11-04 14:19:24 UTC
I meant: Mojolicious 5.54
Comment 243 Tomás Cohen Arazi 2015-11-04 16:52:08 UTC
Patches pushed to master.

Thanks everyone, specially Julian and Martin!
Things will get interesting for sure :-D
Comment 244 David Cook 2015-11-10 06:02:40 UTC
(In reply to Tomás Cohen Arazi from comment #243)
> Patches pushed to master.
> 
> Thanks everyone, specially Julian and Martin!
> Things will get interesting for sure :-D

Whoa! You can say that again!

Is there going to be some documentation for using and developing the REST APIs?

Super intrigued by this!!!