Bug 24633 - Add support for gitlab searching of plugins
Summary: Add support for gitlab searching of plugins
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Plugin architecture (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Martin Renvoize
URL:
Keywords:
Depends on: 23975 26751
Blocks:
  Show dependency treegraph
 
Reported: 2020-02-11 17:08 UTC by Martin Renvoize
Modified: 2022-04-20 10:19 UTC (History)
5 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
The enhancement allows setting Gitlab targets for retrieving plugins.
Version(s) released in:
20.11.00, 20.05.06


Attachments
Bug 24633: Add support for gitlab searching of plugins (3.65 KB, patch)
2020-10-20 18:02 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 24633: (follow-up) Tweak sample plugin targets (1.31 KB, patch)
2020-10-20 18:02 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 24633: Add support for gitlab searching of plugins (3.68 KB, patch)
2020-10-20 18:07 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 24633: (follow-up) Tweak sample plugin targets (1.33 KB, patch)
2020-10-20 18:07 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 24633: Add support for gitlab searching of plugins (3.75 KB, patch)
2020-10-20 18:27 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 24633: (follow-up) Tweak sample plugin targets (1.40 KB, patch)
2020-10-20 18:27 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 24633: Add support for gitlab searching of plugins (3.79 KB, patch)
2020-10-21 12:40 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 24633: (follow-up) Tweak sample plugin targets (1.45 KB, patch)
2020-10-21 12:40 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 24633: URI escape the search term (2.13 KB, patch)
2020-11-06 10:42 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 24633: URI escape the search term (2.10 KB, patch)
2020-11-06 10:44 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28633: DB update (5.95 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28633: Add preferred name field to patrons (21.56 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28633: (follow-up) Account for bug 29695 changes (3.52 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28633: (follow-up) Account for changes with bug 30063 (1.96 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28633: DO NOT PUSH Schema Changes (4.19 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 28633: (QA follow-up) Add preferred_name to ysearch (1.05 KB, patch)
2022-04-20 10:16 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Renvoize 2020-02-11 17:08:37 UTC
As the title says, add support for gitlab to the work started in bug 23975
Comment 1 Tomás Cohen Arazi 2020-10-20 18:02:47 UTC Comment hidden (obsolete)
Comment 2 Tomás Cohen Arazi 2020-10-20 18:02:53 UTC Comment hidden (obsolete)
Comment 3 Tomás Cohen Arazi 2020-10-20 18:07:18 UTC
Created attachment 112068 [details] [review]
Bug 24633: Add support for gitlab searching of plugins

This patch adds the option to add Gitlab repositories to look for
Koha plugins. As with Github, plugin projects need to be named
koha-plugin-* and the release file needs to be named with the .kpz
extension.

To test:
1. Apply this patchset
2. Check either koha-conf.xml or koha-conf.site.xml.in for sample
   configurations of gitlab targets.
3. Tweak your koha-conf.xml accordingly. You could have:

 <plugin_repos>
     <repo>
         <name>ByWater Solutions</name>
         <org_name>bywatersolutions</org_name>
         <service>github</service>
     </repo-
     <repo>
         <name>Theke Solutions</name>
         <org_name>thekesolutions</org_name>
         <service>gitlab</service>
     </repo>
     <repo>
         <name>PTFS Europe</name>
         <org_name>ptfs-europe</org_name>
         <service>github</service>
     </repo>
  </plugin_repos>

4. Flush memcached and restart plack:
   $ flush_memcached
   $ koha-plack --restart kohadev
5. Open the plugins page, search for anything, like 'gobi' or 'a'
=> SUCCESS: You get Theke results!
6. Pick a gitlab plugin for install
=> SUCCESS: It is installed!
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Sponsored-by: Theke Solutions
Comment 4 Tomás Cohen Arazi 2020-10-20 18:07:26 UTC
Created attachment 112069 [details] [review]
Bug 24633: (follow-up) Tweak sample plugin targets

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Sponsored-by: Theke Solutions
Comment 5 Kyle M Hall 2020-10-20 18:27:10 UTC
Created attachment 112074 [details] [review]
Bug 24633: Add support for gitlab searching of plugins

This patch adds the option to add Gitlab repositories to look for
Koha plugins. As with Github, plugin projects need to be named
koha-plugin-* and the release file needs to be named with the .kpz
extension.

To test:
1. Apply this patchset
2. Check either koha-conf.xml or koha-conf.site.xml.in for sample
   configurations of gitlab targets.
3. Tweak your koha-conf.xml accordingly. You could have:

 <plugin_repos>
     <repo>
         <name>ByWater Solutions</name>
         <org_name>bywatersolutions</org_name>
         <service>github</service>
     </repo-
     <repo>
         <name>Theke Solutions</name>
         <org_name>thekesolutions</org_name>
         <service>gitlab</service>
     </repo>
     <repo>
         <name>PTFS Europe</name>
         <org_name>ptfs-europe</org_name>
         <service>github</service>
     </repo>
  </plugin_repos>

4. Flush memcached and restart plack:
   $ flush_memcached
   $ koha-plack --restart kohadev
5. Open the plugins page, search for anything, like 'gobi' or 'a'
=> SUCCESS: You get Theke results!
6. Pick a gitlab plugin for install
=> SUCCESS: It is installed!
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Sponsored-by: Theke Solutions

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 6 Kyle M Hall 2020-10-20 18:27:59 UTC
Created attachment 112075 [details] [review]
Bug 24633: (follow-up) Tweak sample plugin targets

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Sponsored-by: Theke Solutions

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 7 Martin Renvoize 2020-10-21 12:40:24 UTC
Created attachment 112097 [details] [review]
Bug 24633: Add support for gitlab searching of plugins

This patch adds the option to add Gitlab repositories to look for
Koha plugins. As with Github, plugin projects need to be named
koha-plugin-* and the release file needs to be named with the .kpz
extension.

To test:
1. Apply this patchset
2. Check either koha-conf.xml or koha-conf.site.xml.in for sample
   configurations of gitlab targets.
3. Tweak your koha-conf.xml accordingly. You could have:

 <plugin_repos>
     <repo>
         <name>ByWater Solutions</name>
         <org_name>bywatersolutions</org_name>
         <service>github</service>
     </repo-
     <repo>
         <name>Theke Solutions</name>
         <org_name>thekesolutions</org_name>
         <service>gitlab</service>
     </repo>
     <repo>
         <name>PTFS Europe</name>
         <org_name>ptfs-europe</org_name>
         <service>github</service>
     </repo>
  </plugin_repos>

4. Flush memcached and restart plack:
   $ flush_memcached
   $ koha-plack --restart kohadev
5. Open the plugins page, search for anything, like 'gobi' or 'a'
=> SUCCESS: You get Theke results!
6. Pick a gitlab plugin for install
=> SUCCESS: It is installed!
7. Sign off :-D

Sponsored-by: Theke Solutions
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2020-10-21 12:40:28 UTC
Created attachment 112098 [details] [review]
Bug 24633: (follow-up) Tweak sample plugin targets

Sponsored-by: Theke Solutions
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize 2020-10-21 12:41:09 UTC
Thanks for taking this on Tomas.. all works well..

Passing QA
Comment 10 Jonathan Druart 2020-11-06 10:42:03 UTC
Created attachment 113224 [details] [review]
Bug 24633: URI escape the search term
Comment 11 Jonathan Druart 2020-11-06 10:43:07 UTC
It seems that we should URI escape the term before.

With this patch if you search "*" you get: 
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 190

Without the patch you can search for "*", but "&?&".
Comment 12 Jonathan Druart 2020-11-06 10:44:07 UTC
Created attachment 113225 [details] [review]
Bug 24633: URI escape the search term
Comment 13 Tomás Cohen Arazi 2020-11-06 11:09:45 UTC
(In reply to Jonathan Druart from comment #11)
> It seems that we should URI escape the term before.
> 
> With this patch if you search "*" you get: 
> malformed JSON string, neither tag, array, object, number, string or atom,
> at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm
> line 190
> 
> Without the patch you can search for "*", but "&?&".

Is it because of this patches?
Comment 14 Jonathan Druart 2020-11-06 12:39:58 UTC
Certainly not. But the problem exists for both calls I guess. Please fix separately then.
Comment 15 Jonathan Druart 2020-11-06 12:40:13 UTC
Comment on attachment 113225 [details] [review]
Bug 24633: URI escape the search term

Won't be pushed here.
Comment 16 Jonathan Druart 2020-11-06 13:45:53 UTC
(In reply to Jonathan Druart from comment #11)
> It seems that we should URI escape the term before.
> 
> With this patch if you search "*" you get: 
> malformed JSON string, neither tag, array, object, number, string or atom,
> at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm
> line 190
> 
> Without the patch you can search for "*", but "&?&".

Reported on bug 26952.
Comment 17 Jonathan Druart 2020-11-06 14:56:01 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 18 Lucas Gass 2020-11-16 22:24:29 UTC
Since the majority of the work done in Bug 23975 is already in 20.05.x I am choosing to backport the rest of this to 20.05.x

backported for 20.05.06
Comment 19 Aleisha Amohia 2020-11-17 05:38:03 UTC
missing dependencies, not backported to 19.11.x
Comment 20 Martin Renvoize 2022-04-20 10:16:19 UTC
Created attachment 133439 [details] [review]
Bug 28633: DB update

This adds the field to necessary tables and updates API and columns def

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 21 Martin Renvoize 2022-04-20 10:16:23 UTC
Created attachment 133440 [details] [review]
Bug 28633: Add preferred name field to patrons

This patch adds a new field 'preferred_name' to the patron record.

Additionally it adds a method 'effective_name' to choose the preferred name if present
and fall back to firstname

The 'firstname' is displayed as 'First name' on the details page, with id/class
patron_first_name to allow ajusting via JS/CSS if library wants to highlight/hide
the non-preferred name

PatronAutoComplete/ysearch is updated to use 'effective_name' method

To test:
 1 - Apply patches
 2 - Update database and restart all, clear browser cache
 3 - Load a patron in staff module
 4 - Confirm you see and can add a preferred name
 5 - Confirm the preferred name and first name now displays on patron details
 6 - Remove first name from patron record and confirm it no longer shows
 7 - Edit sysprefs BorrowerMandatoryFields and BorrowerUnwantedFields to confirm you can make
     new field required or hidden
 8 - Sign in as patron to opac
 9 - Confirm preferred name shows
10 - Edit account on opac and confirm field is present
11 - Verify DefaultPatronSearchFields contains 'preferredname' if your pref had firstname
12 - Perform checkout and patron search using preferred_name, confirm patron is found
13 - Enable PatronAutoComplete system preference
14 - Type patron's surname into Checkout or patron search but don't hit enter
15 - Confirm patron is displayed with 'preferred_name' in the preview

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 22 Martin Renvoize 2022-04-20 10:16:28 UTC
Created attachment 133441 [details] [review]
Bug 28633: (follow-up) Account for bug 29695 changes

During rebase of this patchset we drop the columns.def and
borrowers.json changes introduced.

We store this mapping in the module now, so this patch does
that.
Comment 23 Martin Renvoize 2022-04-20 10:16:33 UTC
Created attachment 133442 [details] [review]
Bug 28633: (follow-up) Account for changes with bug 30063

Bug 30063 removes the C4/Utils/DataTables/Members.pm module and instead
uses the API and a patron-search include to define the default search
fields. This follow-up adds preferred_name into the default fields in
that include now the the module has been removed
Comment 24 Martin Renvoize 2022-04-20 10:16:37 UTC
Created attachment 133443 [details] [review]
Bug 28633: DO NOT PUSH Schema Changes

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 25 Martin Renvoize 2022-04-20 10:16:41 UTC
Created attachment 133444 [details] [review]
Bug 28633: (QA follow-up) Add preferred_name to ysearch

This adds preferred_name to the search fields in the autocomplete search
system.