Bug 28386

Summary: Replace dev_map.yaml from release_tools with .mailmap
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Architecture, internals, and plumbingAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: fridolin.somers, martin.renvoize, victor, wainuiwitikapark
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.07,20.05.13,19.11.20
Circulation function:
Bug Depends on: 26621    
Bug Blocks: 28442    
Attachments: Bug 28386: Adjust .mailmap
Bug 28386: Some manual removals from contributors.yaml
Bug 28386: Remove unknown authors
Bug 28386: Update Nicole
Bug 28386: Update Kyle
Bug 28386: Meenakshi.R vs Meenakshi R
Bug 28386: Update Hector
Bug 28386: Add history_notes
Bug 28386: [DO NOT PUSH] Update contributors using script
Bug 28386: [DO NOT PUSH] Update history using script
Bug 28386: Disambiguate Séverine (to squash)
Bug 28386: Add sponsors to .mailmap
Bug 28386: Adjust .mailmap
Bug 28386: Some manual removals from contributors.yaml
Bug 28386: Remove unknown authors
Bug 28386: Update Nicole
Bug 28386: Update Kyle
Bug 28386: Meenakshi.R vs Meenakshi R
Bug 28386: Update Hector
Bug 28386: Add history_notes
Bug 28386: [DO NOT PUSH] Update contributors using script
Bug 28386: [DO NOT PUSH] Update history using script
Bug 28386: Disambiguate Séverine (to squash)
Bug 28386: Add sponsors to .mailmap
Bug 28386: fisheye is certainly Olwen
Bug 28386: Fix Pawel history notes
Bug 28386: Update contributors using script
Bug 28386: Update history using script
Bug 28386: Add the 3 unknown authors back into the contributors list with a note
Bug 28386: Add the 3 unknown authors back into the contributors list with a note
Bug 28386: Add more disambiguations to .mailmap
Bug 28386: Add more disambiguations to .mailmap
Bug 28386: Disambiguate Tomas and Blou
Bug 28386: Remove _others_ from the contributor list
Bug 28386: Sort authors by lastname || firstname

Description Jonathan Druart 2021-05-19 10:51:07 UTC
To generate the release notes, the history.txt and contributors.yaml files we are using a custom dev_map.yaml in the release_tools repo.

Instead we should improve our .mailmap and use it for that purpose (author's disambiguation)
Comment 1 Jonathan Druart 2021-05-19 12:27:43 UTC
Created attachment 121153 [details] [review]
Bug 28386: Adjust .mailmap

Again, we need to adjust our .mailmap
This time the goal is the use it in place of the dev_map.yaml we have in
the release tools.

This will be generated for the release notes, history.txt and
contributors.yaml file

This can be tested using:
  git check-mailmap "author <email>"

For instance:
  git check-mailmap "Wrong name <jonathan.druart@bugs.koha-community.org>"
Comment 2 Jonathan Druart 2021-05-19 12:27:47 UTC
Created attachment 121154 [details] [review]
Bug 28386: Some manual removals from contributors.yaml

Those are not real authors and should be removed from the
contributors.yaml file
Comment 3 Jonathan Druart 2021-05-19 12:27:50 UTC
Created attachment 121155 [details] [review]
Bug 28386: Remove unknown authors

Those 3 authors are not in the git history, we should remove them from
the author list.
However we could re-add them to the contributor list with a note saying
for instance they were part of the Catalyst Academy (need to double
check that first)
Comment 4 Jonathan Druart 2021-05-19 12:27:54 UTC
Created attachment 121156 [details] [review]
Bug 28386: Update Nicole

Need to match .mailmap
Comment 5 Jonathan Druart 2021-05-19 12:27:58 UTC
Created attachment 121157 [details] [review]
Bug 28386: Update Kyle

Need to match .mailmap
Comment 6 Jonathan Druart 2021-05-19 12:28:02 UTC
Created attachment 121158 [details] [review]
Bug 28386: Meenakshi.R vs Meenakshi R

Needs to match .mailmap
Comment 7 Jonathan Druart 2021-05-19 12:28:06 UTC
Created attachment 121159 [details] [review]
Bug 28386: Update Hector
Comment 8 Jonathan Druart 2021-05-19 12:28:09 UTC
Created attachment 121160 [details] [review]
Bug 28386: Add history_notes

The "developer" lines of history.txt will be regenerated using the git
history so we need to add this as a separate info.
See the release_tools changes for more info
Comment 9 Jonathan Druart 2021-05-19 12:28:14 UTC
Created attachment 121161 [details] [review]
Bug 28386: [DO NOT PUSH] Update contributors using script

This is an example of what will be regenerated.
It will need to be regenerated once it will be pushed
Comment 10 Jonathan Druart 2021-05-19 12:28:19 UTC
Created attachment 121162 [details] [review]
Bug 28386: [DO NOT PUSH] Update history using script

This is an example of what will be regenerated.
Comment 11 Jonathan Druart 2021-05-19 12:28:39 UTC
This must be tested with the merge requests linked with https://gitlab.com/koha-community/release-tools/-/issues/23
Comment 12 Jonathan Druart 2021-05-19 13:45:16 UTC
Created attachment 121177 [details] [review]
Bug 28386: Disambiguate Séverine (to squash)
Comment 13 Jonathan Druart 2021-05-19 14:38:20 UTC
Created attachment 121180 [details] [review]
Bug 28386: Add sponsors to .mailmap

They were in dev_map.yaml, we are cheating with the <sponsor> email but
it seems to work fine in my tests
Comment 14 Martin Renvoize (ashimema) 2021-05-20 15:21:33 UTC
Created attachment 121245 [details] [review]
Bug 28386: Adjust .mailmap

Again, we need to adjust our .mailmap
This time the goal is the use it in place of the dev_map.yaml we have in
the release tools.

This will be generated for the release notes, history.txt and
contributors.yaml file

This can be tested using:
  git check-mailmap "author <email>"

For instance:
  git check-mailmap "Wrong name <jonathan.druart@bugs.koha-community.org>"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 15 Martin Renvoize (ashimema) 2021-05-20 15:21:37 UTC
Created attachment 121246 [details] [review]
Bug 28386: Some manual removals from contributors.yaml

Those are not real authors and should be removed from the
contributors.yaml file

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 16 Martin Renvoize (ashimema) 2021-05-20 15:21:41 UTC
Created attachment 121247 [details] [review]
Bug 28386: Remove unknown authors

Those 3 authors are not in the git history, we should remove them from
the author list.
However we could re-add them to the contributor list with a note saying
for instance they were part of the Catalyst Academy (need to double
check that first)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 17 Martin Renvoize (ashimema) 2021-05-20 15:21:44 UTC
Created attachment 121248 [details] [review]
Bug 28386: Update Nicole

Need to match .mailmap

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 18 Martin Renvoize (ashimema) 2021-05-20 15:21:47 UTC
Created attachment 121249 [details] [review]
Bug 28386: Update Kyle

Need to match .mailmap

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 19 Martin Renvoize (ashimema) 2021-05-20 15:21:51 UTC
Created attachment 121250 [details] [review]
Bug 28386: Meenakshi.R vs Meenakshi R

Needs to match .mailmap

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 20 Martin Renvoize (ashimema) 2021-05-20 15:21:54 UTC
Created attachment 121251 [details] [review]
Bug 28386: Update Hector

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 21 Martin Renvoize (ashimema) 2021-05-20 15:21:58 UTC
Created attachment 121252 [details] [review]
Bug 28386: Add history_notes

The "developer" lines of history.txt will be regenerated using the git
history so we need to add this as a separate info.
See the release_tools changes for more info

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 22 Martin Renvoize (ashimema) 2021-05-20 15:22:02 UTC
Created attachment 121253 [details] [review]
Bug 28386: [DO NOT PUSH] Update contributors using script

This is an example of what will be regenerated.
It will need to be regenerated once it will be pushed

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 23 Martin Renvoize (ashimema) 2021-05-20 15:22:06 UTC
Created attachment 121254 [details] [review]
Bug 28386: [DO NOT PUSH] Update history using script

This is an example of what will be regenerated.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 24 Martin Renvoize (ashimema) 2021-05-20 15:22:10 UTC
Created attachment 121255 [details] [review]
Bug 28386: Disambiguate Séverine (to squash)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 25 Martin Renvoize (ashimema) 2021-05-20 15:22:14 UTC
Created attachment 121256 [details] [review]
Bug 28386: Add sponsors to .mailmap

They were in dev_map.yaml, we are cheating with the <sponsor> email but
it seems to work fine in my tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 26 Martin Renvoize (ashimema) 2021-05-20 15:24:28 UTC
Great work Jonathan, this is a massive improvement.

Signing off.
Comment 27 Victor Grousset/tuxayo 2021-05-22 23:14:41 UTC
Issue found :o

https://gitlab.com/koha-community/release-tools/-/merge_requests/20#note_582762743
Comment 28 Jonathan Druart 2021-05-23 14:22:39 UTC
Answered.
Comment 29 Victor Grousset/tuxayo 2021-05-23 15:07:13 UTC
It work! Thanks for finding the issue.
Comment 30 Jonathan Druart 2021-05-24 13:37:34 UTC
Created attachment 121329 [details] [review]
Bug 28386: fisheye is certainly Olwen

After a chat with Chris it seems that fisheye was Olwen.
Comment 31 Jonathan Druart 2021-05-24 13:37:39 UTC
Created attachment 121330 [details] [review]
Bug 28386: Fix Pawel history notes
Comment 32 Jonathan Druart 2021-05-24 13:54:23 UTC
Created attachment 121331 [details] [review]
Bug 28386: Update contributors using script

This is an example of what will be regenerated.
It will need to be regenerated once it will be pushed
Comment 33 Jonathan Druart 2021-05-24 13:54:28 UTC
Created attachment 121332 [details] [review]
Bug 28386: Update history using script

This is an example of what will be regenerated.
Comment 34 Jonathan Druart 2021-05-25 07:27:36 UTC
Created attachment 121354 [details] [review]
Bug 28386: Add the 3 unknown authors back into the contributors list with a note
Comment 35 Jonathan Druart 2021-05-25 07:30:55 UTC
Created attachment 121355 [details] [review]
Bug 28386: Add the 3 unknown authors back into the contributors list with a note
Comment 36 Jonathan Druart 2021-05-25 07:47:32 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 37 Jonathan Druart 2021-05-25 14:24:33 UTC
Created attachment 121378 [details] [review]
Bug 28386: Add more disambiguations to .mailmap
Comment 38 Jonathan Druart 2021-05-25 14:38:56 UTC
Created attachment 121384 [details] [review]
Bug 28386: Add more disambiguations to .mailmap
Comment 39 Jonathan Druart 2021-05-26 07:53:29 UTC
Last patch pushed to master.
Comment 41 Jonathan Druart 2021-05-26 10:31:32 UTC
Created attachment 121420 [details] [review]
Bug 28386: Disambiguate Tomas and Blou
Comment 42 Jonathan Druart 2021-05-26 13:11:16 UTC
Last patch pushed to master.
Comment 43 Jonathan Druart 2021-05-27 12:41:47 UTC
Created attachment 121447 [details] [review]
Bug 28386: Remove _others_ from the contributor list

We arbitrarily use "_others_" for unknown contributors (biblibre,
bywater, ptfs-e teams, etc)
Comment 44 Jonathan Druart 2021-05-27 12:41:52 UTC
Created attachment 121448 [details] [review]
Bug 28386: Sort authors by lastname || firstname

If several authors have the same lastname, they are swapped in the list between the different runs.

This is reusing the code from the release_tools, see issue #24
Comment 45 Jonathan Druart 2021-05-27 13:11:24 UTC
Last 2 patches pushed to master.
Comment 46 Fridolin Somers 2021-06-23 05:42:55 UTC
Pushed to 20.11.x for 20.11.07
Comment 47 Victor Grousset/tuxayo 2021-06-23 20:29:43 UTC
Backported: Pushed to 20.05.x branch for 20.05.13

After around 130 conflicts!
(How did Frido, Kyle and Nick dealt with this?)

Hint in case Wainui gets the same thing:
Solve the individual conflicts as usual.
When getting a 60 conflicts pack, just rapidly skim through then without solving to confirm that it will be safe to do `git checkout --theirs MY_FILE`
It was safe in my case (noticed after manually solving the 60 ones...)
There is another 60 conflicts pack, do the same as above.

The timeline in the about page isn't broken, so that seems to work ^^ and Jonathan told me that we should keep the commit numbers from upstream, not the one from our branch. And it's not used anyway.
Comment 48 Wainui Witika-Park 2021-06-24 02:31:28 UTC
Should this be backported to 19.11.x? For 19.11.20 (next release in July)?
Comment 49 Victor Grousset/tuxayo 2021-06-24 13:54:35 UTC
Yes, in bug 28442 it's mentioned that it will have to be backported to all stables. And this one it a dependency.
Comment 50 Wainui Witika-Park 2021-06-30 05:29:26 UTC
Backported to 19.11.x for 19.11.20