Bug 26007

Summary: Warning/reminder for changes to Koha to MARC mapping
Product: Koha Reporter: Cab Vinton <bibliwho>
Component: Staff interfaceAssignee: Katrin Fischer <katrin.fischer>
Status: CLOSED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: enhancement    
Priority: P5 - low CC: aleisha, david, dcook, gmcharlt, jonathan.druart, lucas, m.de.rooy
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
In current versions of Koha you can no longer change the Koha to MARC mappings from the frameworks, but only from the Koha to MARC mapping page in administration. This patch cleans up the hints on the framework page and adds a well visible note on the Koha to MARC mapping page. Any changes to the mappings require that you run the batchRebuildBiblioTables script to fully take effect.
Version(s) released in:
20.11.00, 20.05.04
Circulation function:
Attachments: Bug 26007: Add/fix warning for changes to the Koha to MARC mappings
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings
Bug 26007: (QA follow-up) Add index name to the q parameter
Bug 26007: (QA follow-up) Replace wrong link values from sql installer files
Bug 26007: (QA follow-up) Remove message on marc_subfields_structure
Bug 26007: (QA follow-up) Fix English yml file

Description Cab Vinton 2020-07-17 14:27:44 UTC
The batchRebuildBiblioTables job needs to be run whenever the mapping table is updated.

This step is not listed in the manual or on koha2marclinks.pl page.

Users may make changes without realizing that these won't take effect until the biblio tables are rebuilt, which in many cases will have to be done by a support company.

Suggest adding this step to the manual and also as a reminder at the top of koha2marclinks.pl page. Or perhaps add this as a regular job on a weekly basis?

Cab Vinton
Comment 1 David Cook 2020-07-22 05:22:15 UTC
That's a good point, although that could be dangerous to automate for libraries with large collections. You don't want to be doing that every week to 1,000,000+ records. Even doing it 1 time during the upgrade could be dangerous, as it could make an upgrade last for many hours.

This is another one of those instances where a temporary pop-up post-upgrade reminding people to run a cleanup job wouldn't be a bad idea.

I'd be anti-automation in this case, since this is a job that should be deliberately planned by a human.
Comment 2 Cab Vinton 2020-07-22 12:56:00 UTC
Thank you, David. Wasn't sure how resource-intensive that process might be. Pop-up and/or reminder would certainly do the trick.

Cheers,

Cab
Comment 3 David Cook 2020-07-23 02:22:52 UTC
(In reply to Cab Vinton from comment #2)
> Thank you, David. Wasn't sure how resource-intensive that process might be.
> Pop-up and/or reminder would certainly do the trick.

Unfortunately, I don't know the best way of doing a pop-up and/or reminder. Tomas has mentioned that Koha has done that one or two times before, but I haven't investigated yet.
Comment 4 Cab Vinton 2020-07-23 16:12:42 UTC
Above my pay grade as well, David, but easy enough to at least modify the text on that page.
Comment 5 Jonathan Druart 2020-07-30 07:58:36 UTC
Cab, which exact text would you like to be displayed?
Comment 6 Katrin Fischer 2020-08-30 20:37:24 UTC
Created attachment 109301 [details] [review]
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings

There are 2 pages where one can change the Koha to MARC mappings in Koha:

- When editing the configuration for a MARC subfield
- On administration > Koha 2 MARC mappings

This patch adds a warning that the batchRebuildBiblioTables script
must be run if the mapping is changed.

To test:
- Go to Administration > MARC bibliographic frameworks >
  Any framework > MARC structure > Any field > Edit subfield
- ÃThe hint about the batchRebuildBiblioTables will be on "Link"
  instead of "Koha Link". This is wrong as the link doesn't care
  about the mappings.
- Go to Administration > Koha to MARC mappings
- Verify there is no hint
- Apply patch
- Verify both pages now show an appropriate message
Comment 7 Katrin Fischer 2020-08-30 20:39:03 UTC
Created attachment 109302 [details] [review]
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings

There are 2 pages where one can change the Koha to MARC mappings in Koha:

- When editing the configuration for a MARC subfield
- On administration > Koha 2 MARC mappings

This patch adds a warning that the batchRebuildBiblioTables script
must be run if the mapping is changed.

To test:
- Go to Administration > MARC bibliographic frameworks >
  Any framework > MARC structure > Any field > Edit subfield
- ÃThe hint about the batchRebuildBiblioTables will be on "Link"
  instead of "Koha Link". This is wrong as the link doesn't care
  about the mappings.
- Go to Administration > Koha to MARC mappings
- Verify there is no hint
- Apply patch
- Verify both pages now show an appropriate message
Comment 8 Katrin Fischer 2020-08-30 21:12:47 UTC
I've also submitted a merge request to the manual to clean up and correct the information about Link and Koha link:
https://gitlab.com/koha-community/koha-manual/-/merge_requests/390
Comment 9 David Nind 2020-09-03 20:06:28 UTC
Created attachment 109638 [details] [review]
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings

There are 2 pages where one can change the Koha to MARC mappings in Koha:

- When editing the configuration for a MARC subfield
- On administration > Koha 2 MARC mappings

This patch adds a warning that the batchRebuildBiblioTables script
must be run if the mapping is changed.

To test:
- Go to Administration > MARC bibliographic frameworks >
  Any framework > MARC structure > Any field > Edit subfield
- The hint about the batchRebuildBiblioTables will be on "Link"
  instead of "Koha Link". This is wrong as the link doesn't care
  about the mappings.
- Go to Administration > Koha to MARC mappings
- Verify there is no hint
- Apply patch
- Verify both pages now show an appropriate message

Signed-off-by: David Nind <david@davidnind.com>
Comment 10 David Nind 2020-09-03 20:08:36 UTC
(In reply to Katrin Fischer from comment #8)
> I've also submitted a merge request to the manual to clean up and correct
> the information about Link and Koha link:
> https://gitlab.com/koha-community/koha-manual/-/merge_requests/390

Caroline has merged the excellent manual changes!
Comment 11 Katrin Fischer 2020-09-03 20:49:07 UTC
Thx David and Caroline!
Comment 12 Marcel de Rooy 2020-09-04 07:38:15 UTC
QAing
Comment 13 Marcel de Rooy 2020-09-04 09:29:01 UTC
The manual here needs some attention too !

Link

    If you enter a field/subfield here (200b), a link appears after the subfield in the MARC Detail view. This view is present only in the staff client, not the OPAC. If the librarian clicks on the link, a search is done on the database for the field/subfield with the same value. This can be used for 2 main topics :

        on a field like author (200f in UNIMARC), put 200f here, you will be able to see all bib records with the same author.

        on a field that is a link (4xx) to reach another bib record. For example, put 011a in 464$x, will find the serials that are with this ISSN.

    Warning

        This value should not change after data has been added to your catalog. If you need to change this value you must ask your system administrator to run misc/batchRebuildBiblioTables.pl.

Koha link

    Koha is multi-MARC compliant. So, it does not know what the 245$a means, neither what 200$f (those 2 fields being both the title in MARC21 and UNIMARC). So, in this list you can “map” a MARC subfield to its meaning. Koha constantly maintains consistency between a subfield and its meaning. When the user wants to search on “title”, this link is used to find what is searched (245 if you’re MARC21, 200 if you’re UNIMARC).
Comment 14 Marcel de Rooy 2020-09-04 09:30:06 UTC
The manual tells you to insert things like 200f which really should be search index.
Comment 15 Marcel de Rooy 2020-09-04 09:30:50 UTC
Should we also refer to misc/batchRebuildItemsTables.pl on the Koha to MARC mappings page? Scroll to the bottom, you can also change mappings on items.
Comment 16 Marcel de Rooy 2020-09-04 09:39:56 UTC
Created attachment 109649 [details] [review]
Bug 26007: Add/fix warning for changes to the Koha to MARC mappings

There are 2 pages where one can change the Koha to MARC mappings in Koha:

- When editing the configuration for a MARC subfield
- On administration > Koha 2 MARC mappings

This patch adds a warning that the batchRebuildBiblioTables script
must be run if the mapping is changed.

To test:
- Go to Administration > MARC bibliographic frameworks >
  Any framework > MARC structure > Any field > Edit subfield
- ÃThe hint about the batchRebuildBiblioTables will be on "Link"
  instead of "Koha Link". This is wrong as the link doesn't care
  about the mappings.
- Go to Administration > Koha to MARC mappings
- Verify there is no hint
- Apply patch
- Verify both pages now show an appropriate message

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
This commit message is not completely accurate. See the follow-ups.
Comment 17 Marcel de Rooy 2020-09-04 09:40:02 UTC
Created attachment 109650 [details] [review]
Bug 26007: (QA follow-up) Add index name to the q parameter

The link constructed in MARC-detail is not consistent. It adds
an index name but does not show in the query in the search box.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 18 Marcel de Rooy 2020-09-04 09:40:12 UTC
Created attachment 109651 [details] [review]
Bug 26007: (QA follow-up) Replace wrong link values from sql installer files

Field 943$a has link value "130". This does not work; 130 is not a search
index. It should be Title-uniform.
Note: Only used on a search link in the intranet MARC detail page.

In some cases "130" should be replaced by "Title-uniform", but sometimes
130 without quotes had to be replaced. Performed by two sed statements
on git grep results.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 19 Marcel de Rooy 2020-09-04 09:40:21 UTC
Created attachment 109652 [details] [review]
Bug 26007: (QA follow-up) Remove message on marc_subfields_structure

Why? Since the combo has been disabled since a few releases. If you
want to change this mapping, you should do it on Koha to MARC mappings.
This change is no longer per framework, but over all frameworks.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 20 Katrin Fischer 2020-09-04 17:41:17 UTC
(In reply to Marcel de Rooy from comment #13)
> The manual here needs some attention too !
> 
> Link
> 
>     If you enter a field/subfield here (200b), a link appears after the
> subfield in the MARC Detail view. This view is present only in the staff
> client, not the OPAC. If the librarian clicks on the link, a search is done
> on the database for the field/subfield with the same value. This can be used
> for 2 main topics :

Hi Marcel, I agree - that's why I patched it (see gitlab link above) ;)

There currently doesn't appear to be a preview for the master manual, but you can see the changes applied now it git:
https://gitlab.com/koha-community/koha-manual/-/commit/73660a674aaa3431ec9eac09f8fb1e38d2fafaf6

I agree with your last follow-up, but we should stop displaying this as a non-functional pull-down on the configuration page then - it fooled me.

Bug 26007: (QA follow-up) Replace wrong link values from sql installer files is missing updates to the .yml files for the en and es-ES based installers. I think this would have been better as a separate bug.
Comment 21 Marcel de Rooy 2020-09-07 06:18:41 UTC
(In reply to Katrin Fischer from comment #20)
> (In reply to Marcel de Rooy from comment #13)

> I agree with your last follow-up, but we should stop displaying this as a
> non-functional pull-down on the configuration page then - it fooled me.

Yeah, please remove it on a new report.

> Bug 26007: (QA follow-up) Replace wrong link values from sql installer files
> is missing updates to the .yml files for the en and es-ES based installers.
> I think this would have been better as a separate bug.

If you prefer, feel free to move this patch to a new report. Could you sign off?
Comment 22 Jonathan Druart 2020-09-09 13:40:38 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 23 Katrin Fischer 2020-09-10 20:34:56 UTC
We need a follow-up here still for the .yml en files.
Comment 24 Marcel de Rooy 2020-09-11 06:02:33 UTC
(In reply to Katrin Fischer from comment #23)
> We need a follow-up here still for the .yml en files.

Will add a patch. I remember that I did change it, but I may have changed my sed statements later on and it fell out of the selection due to its new formatting somehow.
Comment 25 Marcel de Rooy 2020-09-11 06:37:55 UTC
Created attachment 109881 [details] [review]
Bug 26007: (QA follow-up) Fix English yml file

git grep -l -E "link:\s*\"?[0-9]" | xargs sed -i -e "/link:/ s/130/\"Title-uniform\"/"
Only one occurrence without quotes around 130.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 26 Marcel de Rooy 2020-09-11 06:39:05 UTC
@RM: Please push last patch only.
Comment 27 Jonathan Druart 2020-09-11 07:58:41 UTC
Last patch pushed to master.
Comment 28 Lucas Gass (lukeg) 2020-09-14 22:30:48 UTC
backported to 20.05.x for 20.05.04
Comment 29 Aleisha Amohia 2020-09-15 11:55:36 UTC
enhancement, not backported to 19.11.x