Bug 32722 - Mandatory subfields from UNIMARC framework shouldn't block the creation of a bib record when field itself is optional
Summary: Mandatory subfields from UNIMARC framework shouldn't block the creation of a ...
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Bibliographic data support (show other bugs)
Version: Main
Hardware: All All
: P5 - low critical (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
: 36576 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-01-25 12:15 UTC by Thibault Keromnès
Modified: 2024-04-18 20:46 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
List of mandatory subfields (2.28 KB, application/vnd.ms-excel)
2023-01-25 12:15 UTC, Thibault Keromnès
Details
Bug 32722: Don't block record save if subfield is mandatory but field is optional (11.37 KB, patch)
2023-03-29 12:19 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Thibault Keromnès 2023-01-25 12:15:15 UTC
Created attachment 145658 [details]
List of mandatory subfields

BZ30733 added many mandatory subfields in UNIMARC default framework, following the IFLA manual updates. 
Thing is, most of those mandatory subfields are within an optional field, so they're supposed to be mandatory only if the field is used. Yet Koha blocks the record's creation with ~60 mandatory subfields.

Most practical solution would probably be to remove the mandatory flag to said subfields (the list is attached to the bug). 

Exception are 100$a and 200$a (the fields being mandatory too it makes sense)
Also have a doubt concerning 099$t and 942$c, that are not actually Unimarc, but Koha specifics, storing biblioitems.itemtype.
Comment 1 Katrin Fischer 2023-01-25 12:17:21 UTC
You could also make them important maybe instead of mandatory. That gives a warning, but still allows to save.
Comment 2 Thibault Keromnès 2023-03-27 13:56:24 UTC
Hello Katrin,
I'm not sure it's the best way, most mandatory subfields are just not relevant, depending on the document type.
For instance: 120$a asks for the scale of a map, even if you're working on a movie.
Comment 3 Jonathan Druart 2023-03-29 12:19:04 UTC
Created attachment 148909 [details] [review]
Bug 32722: Don't block record save if subfield is mandatory but field is optional

It should only be considered mandatory if at least one of the other
subfield is not empty.
Comment 4 Jonathan Druart 2023-03-29 12:22:06 UTC
This code is a nightmare!
Thibault, can you confirm the patch works as you expect? I think it also impact "important" subfields, not sure it should...
Comment 5 Thibault Keromnès 2023-03-30 07:41:50 UTC
Hey Jonathan, 
It does work as expected, but I've talked about it with Katerin & Aude, and it seems it would be a problem for Marc21 users. 
They consider a subfield to be mandatory whether the field itself is mandatory or not, so with this patch we would risk having records created without key informations.
I should have started there, sorry.
Comment 6 Thibault Keromnès 2023-03-31 12:48:56 UTC
Need for additional features on this patch: 
- an update on the database : in existing frameworks, for every mandatory subfields, make the corresponding field mandatory too (so the behaviour doesn't change by default)
- when editing a subfield : add a note to warn the user that the subfield will be mandatory only if the field is mandatory or if another subfield is not empty
- reproduce this behaviour for the authorities framework
Comment 7 Phil Ringnalda 2024-04-05 17:01:04 UTC
Since nothing involving updating existing frameworks can ever be easy:

Unlike the basic editor, the advanced editor already does exactly what you want: if 120$a is mandatory but 120 is not mandatory, then if you have no 120 in the advanced editor you are free to save, but if you have 120$b you are required to have $a.

So a MARC21 library which only uses the advanced editor, never the basic editor, could have intentionally set 100$a as mandatory but not set 100 as mandatory, intending to have the advanced editor require that if you use a main entry personal name, you actually have the name, not just an accidental "$q(Philip Anthony)$d1963-" without the name itself in $a. Automatically changing that to require that every bib record have a main entry personal name would be quite a shock.
Comment 8 Victor Grousset/tuxayo 2024-04-11 14:30:15 UTC
*** Bug 36576 has been marked as a duplicate of this bug. ***
Comment 9 Victor Grousset/tuxayo 2024-04-15 02:34:16 UTC
Found via git bisect that it comes from commit: "Bug 30373: Fix visibility, authorised values and descriptions"

----

Raised severity because UNIMARC cataloguing doesn't work out of the box.
It's a roadblock for small libraries that don't have a support company that has their own framework. Or don't know how to customize frameworks especially with such an unexpected default state.
less severe effect: In the end, it makes impossible to setup a framework that follows the standard so one must manually enforce the mandatory subfields within an optional field.

More meta issue: it's makes a pain to test UNIMARC patches.

---

Current patch makes creation and edition of biblio records usable again :D

Is it new the situation of mandatory subfields within an optional field? Or was there other occurrences before these IFLA UNIMARC updates?

Another inconsistency in handling those cases is that even if thanks to the patch, the save isn't blocked, all the 4XX subfields have the red "Required" and are colored yellow. Undistinguishable from really mandatory ones.

---


(In reply to Thibault Keromnès from comment #5)
> It does work as expected, but I've talked about it with Katerin & Aude, and
> it seems it would be a problem for Marc21 users. 

Would it work to have the change only affect UNIMARC? If MARC21 needs any change related to mandatory subfields, it could be another ticket. It's not the same impact on both standards.


(In reply to Thibault Keromnès from comment #6)
> Need for additional features on this patch: 
> - an update on the database : in existing frameworks, for every mandatory
> subfields, make the corresponding field mandatory too (so the behaviour
> doesn't change by default)

Is the need of data migration only for existing libraries that didn't got installed with the recent (22.05/bug 30373) default framework?
And those recently installed but with a different framework, like from their provider.
Comment 10 Jonathan Druart 2024-04-16 07:51:02 UTC
Just tell me what's needed. But be fast, I won't rebase this patch if there is a conflict.
Comment 11 Phil Ringnalda 2024-04-18 20:44:49 UTC
I'm curious about the size of the affected population in need of a retroactive fix. It would be new installs of 22.05 or later, who don't have anyone to fix a framework and who have continued using Koha despite having to fill in ~60 required fields which don't actually apply to the record they are saving, every single time they save a record. Would that be 1, 10, 100, or 1000?
Comment 12 Phil Ringnalda 2024-04-18 20:46:09 UTC
s/installs/UNIMARC installs/