I couldn't find the new location(s) for UNIMARC field help....if there is one.

It looks like the Italian version would work fine with MarcFieldDocURL, but most links, including to the English version are broken:

https://www.ifla.org/publications/unimarc-formats-and-related-documentation/

Then there is https://www.ifla.org/g/unimarc-rg/unimarc-updates/, but it looks like the full documentation is not (or no longer?) available online.

Maybe we should just remove the default URL for UNIMARC?

I also found:
- https://www.ifla.org/g/unimarc-rg/unimarc-updates/
- https://www.iflastandards.info/unimarc (but it doesn't provide as good an explanation as the narrative information)

Removing the default URL seems like a good plan!

*** Bug 38743 has been marked as a duplicate of this bug. ***

I don't think there is an official public version of the UNIMARC format in English on the IFLA website or elsewhere. So in this case we can't do the equivalent of the links made for MARC21.

I think it would be better not to try to display standard links for UNIMARC cataloguing. Maybe if we want so, the display of links to the French version of Koha could be managed by a plugin developped by a vendor, applied to French libraries (links to https://www.transition-bibliographique.fr/unimarc/manuel-unimarc-format-bibliographique/ ). But I am not sure there is a real need for that.

It's a shame there is no online version like the Library of Congress has for MARC21 (despite it being  "very vasic HTML").

I did toy with the idea of creating an HTML version of the PDFs available at https://www.ifla.org/g/unimarc-rg/unimarc-updates/

And it would be doable, as the license is CC BY 4.0.

But it would be a challenge to maintain...

Anyway a link to the english version is not a need for us French people. Not sure it would bé for portuguese, italian or russians who are using unimarc like us

(In reply to Mathieu Saby from comment #7)
> Anyway a link to the english version is not a need for us French people. Not
> sure it would bé for portuguese, italian or russians who are using unimarc
> like us

Great point!

Since I created this bug, what is a good way forward?

I've noted a few ideas, welcome feedback and some more nuances on the scenarios, audiences, and options.

Current options and set up
==========================

MarcFieldDocURL system preference: If left empty, the default hard-coded links are used, which is an issue when the links no longer work.

The display of the help links is controlled in the basic bibliographic record editor when editing a record: Settings > Show MARC tag documentation links. However, this is only remembered for the current session - if you log out, it defaults to displaying the help links the next time you log in.

The help links are only shown for bibliographic records, not authority records.

The help links are only shown on the standard bibliographic editor, there are no help links shown in the advanced editor.

Scenarios
=========

Likely scenarios for libraries:

1. Display the default help for MARC tags as an aid for catalogers:
   - No action required: 
     . links to the Library of Congress (links work) or IFLA (links don't work, as URLs have changed and no longer provide individual web pages for each tag)
     . The "Show MARC tag documentation links" is on by default

2. Display library specific help for MARC tags. This reflects local or national cataloging practices. Likely used by consortia or national libraries that use some sort of centralised or controlled cataloging. This reflects their conventions and interpretation of the various MARC tags that are specific to their collections.

3. Don't want to display any help links, documentation for MARC tags and cataloging practices is available elsewhere.

Audiences
=========

1. Experienced catalogers: 
   . likely only need to refer to MARC tags for things they are unfamiliar with, or use rarely
   . most of the time they will "know" what is required for the various tags and how their library uses them
   . very familiar with their collections, and the tags they use to describe their records and cataloging conventions
   . very familiar with MARC and cataloging terminology (indicators, headings, control fields, and so on)

2. Occasional catalogers:
   . likely to use the help frequently
   . know the basics, but anything "tricky" requires looking up documentation, and possibly local documentation for how they describe their collections (or look at existing similar records, and copy that)
   . familiar with MARC basics, but more advanced stuff is confusing

3. Novice catalogers:
   . no idea what MARC is, but probably get the basics, like records need titles and authors so that their patrons can find things
   . what are indicators?
   . likely to think "this is weird!"

Options
=======

Current settings:
- Help links shown by default in the basic editor
- Links to the Library of Congress for MARC21 work
- Links for UNIMARC no longer work - no longer provide individual web pages for tags, URLs generated no longer valid
- Can turn off help links temporarily when editing records (Show MARC tag documentation links)
- Can customise the URL for help links (MarcFieldDocURL system preference)

Option 1 - Make it configurable (ideal):
- Add an option to MarcFieldDocURL to show or not show help links (enhancement), then a field for the link (as currently), for example:
  MarcFieldDocURL: Show/Don't show links to MARC field documentation. Use ______ .... (existing text)
- If "Don't show" is selected, then:
  . no help links shown in the basic editor
  . no "Show MARC tag documentation links" menu option
- If "Show" is selected, then:
  . help links shown
  . "Show MARC tag documentation links" menu option is selected (is this even required?)
  . if no URL entered, then:
    . default for MARC21 - no change
    . if a UNIMARC instance, then display an info message "Please update the URL in the MarcFieldDocURL system preference for your UNIMARC documentation"


Option 2 - Make it configurable (UNIMARC instances):
- Same as option 1, but if no link and a UNIMARC instance, then link to a manual page (in English) that has some information about the UNIMARC standard and where to find (maybe with links to language-specific resources, if they exist).

Option 3 - Provide Koha UNIMARC docs:
- We could turn the IFLA PDFs into HTML pages, and maintain
- Long-term challenge to keep up to date!
- UNIMARC is primarily used by non-English speaking libraries (from my understanding)
- Not sure why we should do IFLA's work for them!
- Not recommended!

Option 4: No change
- If you are a UNIMARC library, you need to add a link to MarcFieldDocURL to point to your cataloging guidelines
- Update the manual and system preference description to reflect this


I think I prefer option 1, but I'm not a developer. So I'm not sure of the best or easiest approach for doing this.

Adding comment from duplicate bug by Baptiste Wojtkowski (bwoj):

In UNIMARC, links to IFLA are broken and referring to a very old version of MARC format.

 502     function _UNIMARCFieldDoc(field) {
 503         /* http://archive.ifla.org/VI/3/p1996-1/ is an outdated version of UNIMARC, but
 504            seems to be the only version available that can be linked to per tag.  More recent
 505            versions of the UNIMARC standard are available on the IFLA website only as
 506            PDFs!
 507         */
 508         var url;
 509         if (field == 0) {
 510             url = "http://archive.ifla.org/VI/3/p1996-1/uni.htm";
 511         } else {
 512             var first = field.substring(0,1);
 513             url = "http://archive.ifla.org/VI/3/p1996-1/uni" + first + ".htm#";
 514             if (first == 0) url = url + "b";
 515             url = first == 9
 516                   ? "http://archive.ifla.org/VI/3/p1996-1/uni9.htm"
 517                   : url + field;
 518         }
 519         window.open(url);
 520     }

To my knowledge, there is no good solution to this pb :
- According to the code and to my investigation, IFLA is not providing split files, field by field (except fields which have been updated after 2008, which can be accessed through the atomic updates)
- transition-bibliographique.fr is providing such a work but only in french

Note that 
- the approval status of the 2023 PDF is not clear (https://www.ifla.org/unimarc-updates/unimarc-bibliographic-format-manual-online-ed/) : "Not yet endorsed by the IFLA Professional Committee or Governing Board."
- "Documentation may change from PDF online to HTML online in the future" (https://repository.ifla.org/server/api/core/bitstreams/9a6ca461-f019-4c34-bccc-7d7175935103/content)

Some great discussion (in French) in the Mattermost chat https://chat.koha-community.org/koha-community/pl/drp65k4w3tnjzmntr4dsxiw31c

The comments (I think from using a translation tool), and as noted on other comments:

- The description of the UNIMARC bibliographic format is maintained by IFLA (600+ page PDF, which is not yet "official").

- Therefore, at the moment, the help links (?) can't be connected by to a persistent URL for each tag for the English version of UNIMARC. 

- France/French speaking libraries use https://www.transition-bibliographique.fr/unimarc/manuel-unimarc-format-bibliographique/

- For the French versions, there is no standard for URL names on transition-bibliographique.fr. Each field would need a manual unique link, and could quickly become outdated.

- Therefore, it would be better not to try to display the help links as standard for cataloging in UNIMARC.

I think the way forward would be to:

1. Add an option to MarcFieldDocURL to show or not show help links (enhancement), then a field for the link (as currently), for example:
  MarcFieldDocURL: Show/Don't show links to MARC field documentation. Use ______ .... (existing text)

2. For MARC21 instances (change, enhancement(?)):
   - Enable help links by default (set to Show), as currently - no change
   - Use a default URL pattern that works for the Library of Congress links, so that the URL is not hard coded (although these are not likely to change, you never know...)

3. For UNIMARC instances:
   - Don't show help links (set to "Don't show)
   - Don't include a URL pattern, as there is no authoritative source to link to for individual tags (currently IFLA has an unofficial 600+ page PDF in English, there is also a reference in French  - but no consistent URL pattern to link to; may change in the future, so should leave as a possibility)
   - UNIMARC libraries can add their own link if they want to link to the IFLA material, French material, or their own cataloguing reference

4. Remove the "Show MARC tag documentation links" menu option in the MARC editor settings:
   - probably no need to have this: if the setting is changed by the staff patron, it is only remembered for the current logged-in session
   - if kept, status is determined by "Show/Don't show" setting


Alternatives, other options and ideas:

1. If MarcFieldDocURL is set to "Show" but there is no URL/URL pattern:
   . display an information message, when the ? help link is clicked, such as:
    "No URL is set for the MarcFieldDocURL system preference."
   OR
  . don't show the ? help link for each tag

2. Add an HTML customization for a general MARC help message, instead of a message that needs translation in the code.

Created attachment 175967 [details]
Screenshots - Basic editor before and after the patch

Notes from working on this bug:
1. The authority record editor does not have help text.
2. For a MARC21 instance, there are some tags that use a different URL 
   pattern, such as the leader (000), and 9XX tags. There is extra code
   so that the help text link goes to the correct Library of Congress 
   pages where this is the case.
3. I originally thought that you could insert a default custom URL 
   format as a value for MarcFieldDocURL, depending on whether it 
   was a MARC21 or UNIMARC instance. Mainly as I think it is a bad idea
   to hard code URLs, unless it can be avoided. However, this doesn't 
   seem possible (to me, anyway) because of 2.
4. The help link code for the advanced editor is much easier to follow
   than the code for the basic editor (for me anyway!). 
5. I've just done a "basic" fix, so that there are no broken links for 
   UNIMARC. However, someone who knows what they are doing could 
   probably consolidate and streamline the code so both the basic and 
   advanced editor use the same code (and perhaps use the same code to
   add help links for the authority record editor in the future).
   However, doing this is beyond my skills!
6. Some links:
   - UNIMARC documentation in languages other than English - scroll 
     down to the UNIMARC Translation heading:
     https://www.ifla.org/g/unimarc-rg/unimarc-updates/
   - Canadian MARC21 documentation URL pattern: 
     http://www.marc21.ca/MaJ/BIB/B{FIELD}.pdf
7. For UNIMARC, I did think of linking to the consolidated PDF page 
   (https://www.ifla.org/unimarc-updates/unimarc-bibliographic-format-manual-online-ed/). 
   However, I think the page linked to is less likely to change in the 
   future, and it is more comprehensive.
8. If IFLA does eventually provide web pages for each tag, then another
   bug is required to update the help links.
9. In case it is useful, I've added some screenshots from before and
   after the change.

[1] The current system preference text for MarcFieldDocURL:

Use ____________ as the URL for MARC field documentation. Possible substitutions are {MARC} (marc flavour, eg. "MARC21" or "UNIMARC"), {FIELD} (field number, eg. "000" or "048"), {LANG} (user language, eg. "en" or "fi-FI"). If left empty, the format documentation on http://loc.gov (MARC21) or http://archive.ifla.org (UNIMARC) is used. For example http://fielddoc.example.com/?marc={MARC}&field={FIELD}&language={LANG}

[2] The rewritten system preference text for MarcFieldDocURL:

Use ________ as the URL for MARC field documentation.

If left empty, the default URLs are:
- MARC21: field-level pages at https://www.loc.gov/marc/bibliographic/
- UNIMARC: https://www.ifla.org/g/unimarc-rg/unimarc-updates/

Placeholders available for use in the URL:
- {MARC}: MARC flavor, "MARC21" or "UNIMARC"
- {FIELD}: field number, for example, "000" or "048"
- {LANG}: language, for example, "en" or "fi-FI"

Examples:
- https://www.example.com/marcdocs?marc={MARC}&field={FIELD}&language={LANG}
- https://www.marc21.ca/MaJ/BIB/B{FIELD}.pdf

Created attachment 175968 [details] [review]
Bug 31019: Fix broken links to UNIMARC field documentation

IFLA no longer provides field-level documentation pages for the UNIMARC
format (as at December 2024). It only provides:
- updates by year as individual PDFs (there may be multiple PDF updates
  for a field)
- a single consolidated PDF that is not yet endorsed by IFLA.

This patch updates the default UNIMARC field documentation links
to https://www.ifla.org/g/unimarc-rg/unimarc-updates/

It also:
- makes minor formating changes to improve consistency with the
  advanced editor:
  . the help link text is now [?], instead of ?
  . adds hover text for [?] - "Show help for this tag"
  . adds some additional spacing before the indicator fields
  . removes the '-' before the tag title
- updates the MarcFieldDocURL system preference description

Test plan:
1. Start up a Koha instance that uses UNIMARC (if using
   koha-testing-docker, change KOHA_MARC_FLAVOUR=unimarc in
   your .env file).
2. Set the EnableAdvancedCatalogingEditor system preference to "Enable".
3. Search for and edit any bibliographic record using the basic editor.
4. Click on the help text link (?) after any tag.
5. Note that you get an error:
   - Firefox: "The page isn’t redirecting properly"
   - Chrome-based browsers:
      "This page isn’t working
       archive.ifla.org redirected you too many times."
6. Switch to the advanced editor (Settings > Switch to advanced editor).
7. Click on a tag and then click the help link ([?]) at the bottom of
   the page.
   Result: You get the an error for the web page.
8. Switch back to the basic editor (Settings > Switch to basic editor).
9. Apply the patch.
10. Duplicate the tab and refresh the page for the record you
    were editing.
11. Click a help link for the basic editor - it now goes to a
    valid web page (https://www.ifla.org/g/unimarc-rg/unimarc-updates/).
12. Compare the basic editor page to the previously opened tab
    and note these differences:
    - the help link text is now [?], instead of ?
    - there is now hover text for [?]- "Show help for this tag"
    - there is some additional spacing before the indicator fields
    - there is no '-' before the tag title
13. Check that the page looks okay if the Settings > Show MARC
    tag document, and Show tags options are toggled off.
14. Switch to the advanced editor (Settings > Switch to advanced editor).
    If you click on the help link you now go to a valid web page.
15. View the updated description for updated MarcFieldDocURL system
    preference and check that it makes sense.
16. Enter a URL for the MarcFieldDocURL system preference, for
    example, https://example.com
17. Edit another record (using either the basic or advanced editor) and
    click the help link - it should take you to the web page you set for
    MarcFieldDocURL.
18. [Optional] Enter a custom URL pattern for the help documentation
    using {FIELD} (I don't know of any UNIMARC custom URL patterns).
19. Repeat the steps as appropriate for a MARC21 instance:
    19.1 Start up a MARC21 instance (change KOHA_MARC_FLAVOUR=unimarc
         on your .env file).
    19.2 The help links should take you to the Library of Congress MARC
         pages.
    19.3 Check that custom URL patterns continue to work, for example,
         use the Canadian custom URL pattern
         http://www.marc21.ca/MaJ/BIB/B{FIELD}.pdf
20. Sign off!

Signed-off-by: David Nind <david@davidnind.com>