Code is available at:

Koha    - https://github.com/joubu/Koha/commits/hea_v2
hea-ws  - https://github.com/joubu/hea-ws/commits/v2
hea-app - https://github.com/joubu/hea-app/commits/v2 (contains the different from the last 6 months).

Screenshots:

Koha side:
New page to configure the info to share:
  https://pageshot.net/opGCLAZFt0s1HVzp/pro.kohadev.vm (with libraries info)
  https://pageshot.net/SKTFZgr40EKg6fge/pro.kohadev.vm (without libraries info)

Hea side:
New layout for the homepage:
  https://pageshot.net/GGKbBZlZwxpXuCoi/localhost
View "see all libraries" (existing one):
  https://pageshot.net/2XplbZXFFr9rLNJS/localhost
View "see libraries by country":
  https://pageshot.net/G5czRcqh47WFIfRD/localhost
View "see libraries on a map":
  https://pageshot.net/Xa4qW3Vc4MclrEhO/localhost
View "public view for a library" (to show libraries what is public):
  https://pageshot.net/7whCONhkPuJUUxDP/localhost

There is a popup to display the library's name on hover the markers.

Pretty! +100

Jonathan, 

I have pushed a hea repo to git.koha-community.org

You should have push rights to that, if you want to push up the hea specific code to there as well.

I will sign off 14608 so this can go forward, it is looking great

*** Bug 17802 has been marked as a duplicate of this bug. ***

*** Bug 17803 has been marked as a duplicate of this bug. ***

Created attachment 59975 [details] [review]
Bug 18066: Hea V2

This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Created attachment 59976 [details] [review]
Bug 18066: Add the leaflet library

Created attachment 60729 [details] [review]
Bug 18066: Do not use token for OSM tiles

Created attachment 60730 [details] [review]
[DO NOT PUSH] Use hea test server instead of prod

Test plan using the Hea test server:
BibLibre provided me a test server (Merci Laurent!) to test easily this patch. You will not need to set up hea-ws and hea-app.

The test server is at http://community-hea.preprod.biblibre.eu

To test Koha-side you just will need these patches (as well as those from bug 14608), it's easier to use the hea_v2 branch from my github repo.
With the "[DO NOT PUSH]" patch applied you will be able to do all what you want to stress the Hea app.

Follow the "7/ Back to Koha side" part of the original test plan.

Created attachment 60734 [details] [review]
Bug 18066: Use https for ws

Created attachment 60742 [details] [review]
Bug 18066: Hea V2

This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 60743 [details] [review]
Bug 18066: Add the leaflet library

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 60744 [details] [review]
Bug 18066: Do not use token for OSM tiles

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 60745 [details] [review]
Use hea test server instead of prod

Created attachment 60746 [details] [review]
Bug 18066: Use https for ws

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Some tests are failing:

t/db_dependent/UsageStats.t .. 1/59
#   Failed test ''$report->{library} is a HASH' isa 'HASH''
#   at t/db_dependent/UsageStats.t line 91.
#     '$report->{library} is a HASH' isn't defined

#   Failed test 'There are 6 fields in HASH(0x84c72f0)'
#   at t/db_dependent/UsageStats.t line 92.
#          got: '0'
#     expected: '6'

#   Failed test 'UsageStatsID           is good'
#   at t/db_dependent/UsageStats.t line 93.
#          got: undef
#     expected: '0'

#   Failed test 'UsageStatsLibraryName  is good'
#   at t/db_dependent/UsageStats.t line 94.
#          got: undef
#     expected: ''

#   Failed test 'UsageStatsLibraryUrl   is good'
#   at t/db_dependent/UsageStats.t line 95.
#          got: undef
#     expected: ''

#   Failed test 'UsageStatsLibraryType  is good'
#   at t/db_dependent/UsageStats.t line 96.
#          got: undef
#     expected: ''

#   Failed test 'UsageStatsCountry      is good'
#   at t/db_dependent/UsageStats.t line 97.
#          got: undef
#     expected: ''

#   Failed test ''$report->{library} is a HASH' isa 'HASH''
#   at t/db_dependent/UsageStats.t line 111.
#     '$report->{library} is a HASH' isn't defined

#   Failed test 'There are 6 fields in HASH(0x84554f8)'
#   at t/db_dependent/UsageStats.t line 112.
#          got: '0'
#     expected: '6'

#   Failed test 'UsageStatsID            is good'
#   at t/db_dependent/UsageStats.t line 113.
#          got: undef
#     expected: '1'

#   Failed test 'UsageStatsLibraryName   is good'
#   at t/db_dependent/UsageStats.t line 114.
#          got: undef
#     expected: 'NAME'

#   Failed test 'UsageStatsLibraryUrl    is good'
#   at t/db_dependent/UsageStats.t line 115.
#          got: undef
#     expected: 'URL'

#   Failed test 'UsageStatsLibraryType   is good'
#   at t/db_dependent/UsageStats.t line 116.
#          got: undef
#     expected: 'TYPE'

#   Failed test 'UsageStatsCountry       is good'
#   at t/db_dependent/UsageStats.t line 117.
#          got: undef
#     expected: 'COUNTRY'

#   Failed test 'UsageStatsLibrariesInfo is good'
#   at t/db_dependent/UsageStats.t line 119.
#          got: undef
#     expected: '33'
# Looks like you failed 15 tests of 59.
t/db_dependent/UsageStats.t .. Dubious, test returned 15 (wstat 3840, 0xf00)
Failed 15/59 subtests

Test Summary Report
-------------------
t/db_dependent/UsageStats.t (Wstat: 3840 Tests: 59 Failed: 15)
  Failed tests:  12-18, 21-28


Additionally, the map in usage_statistics.pl can be positioned outside the borders of its container. I'll attach screenshots to show what I mean

Created attachment 61355 [details]
Bad position of the map when you reload the page when not on top of the page

Created attachment 61356 [details]
Bad position of the map when you scroll down

Also, when the screen width is too small (1280px for instance) the map goes completely outside its container.

(In reply to Julian Maurice from comment #18)
> Additionally, the map in usage_statistics.pl can be positioned outside the
> borders of its container. I'll attach screenshots to show what I mean

Yes I know, I did not manage to fix it. Any ideas?

Created attachment 61358 [details] [review]
Bug 18066: Fix tests

Created attachment 61361 [details] [review]
Bug 18066: Fix map positioning in admin/usage_statistics.pl

Created attachment 61362 [details] [review]
Bug 18066: Fix map positioning in admin/usage_statistics.pl

Ok tests pass now.

But what should we do about the patches that change the URL of Hea ? They shouldn't be pushed, right ? But if we don't push them, the URL is http instead of https. I feel that something is missing here.
Should we move that URL in a syspref or in $KOHA_CONF for easier testing ?

Created attachment 61392 [details] [review]
Bug 18066: Hea V2

This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 61393 [details] [review]
Bug 18066: Add the leaflet library

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 61394 [details] [review]
Bug 18066: Do not use token for OSM tiles

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 61395 [details] [review]
Bug 18066: Use https for ws

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Created attachment 61396 [details] [review]
Bug 18066: Fix tests

Created attachment 61397 [details] [review]
Bug 18066: Fix map positioning in admin/usage_statistics.pl

Created attachment 61398 [details] [review]
[DO NOT PUSH] Bug 18066: Use hea test server instead of prod

Julian, was it a PQA?

Created attachment 61427 [details] [review]
Bug 18066: Hea V2

This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61428 [details] [review]
Bug 18066: Add the leaflet library

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61429 [details] [review]
Bug 18066: Do not use token for OSM tiles

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61430 [details] [review]
Bug 18066: Use https for ws

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61431 [details] [review]
Bug 18066: Fix tests

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61432 [details] [review]
Bug 18066: Fix map positioning in admin/usage_statistics.pl

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Created attachment 61433 [details] [review]
[DO NOT PUSH] Bug 18066: Use hea test server instead of prod

(In reply to Jonathan Druart from comment #34)
> Julian, was it a PQA?

It wasn't, but now it is ;)

Pushed to Master - Should be in the 17.05 release.  Thanks!

Created attachment 61582 [details] [review]
Bug 18066: Add the new geolocation column to kohastructure.sql

Last patch pushed to master and DBIx::Class schema regenerated.

Not pushing this to 16.11.x yet, might reconsider for the next bugfix release.

Jonathan, is it ok to push this for 16.11.07?
Will be my first Schema change patch, so want to push it early if possible.

Is this ready for stable?

(In reply to Katrin Fischer from comment #48)
> Is this ready for stable?

I did not get the confirmation yet that the test server has been moved to production.

We are in string freeze now - so will check again for 16.11.08

The production server has been updated, this is ready to be backported.

Created attachment 62646 [details] [review]
Bug 18066: Remove warnings from translate script

This patch removes the following warnings:
Use of uninitialized value $pref_name in concatenation (.) or string at
LangInstaller.pm

(In reply to Jonathan Druart from comment #52)
> Created attachment 62646 [details] [review] [review]
> Bug 18066: Remove warnings from translate script
> 
> This patch removes the following warnings:
> Use of uninitialized value $pref_name in concatenation (.) or string at
> LangInstaller.pm

This patch has been pushed to master!

Pushed to 3.22.x for 3.22.21

Pushed to 16.05.x, for 16.05.13 release