If the record leader 18 specifies `c` or `n` the Marc record in question does not hold any punctuation chars. IOW the record would look like ``` 264 _1 |a Berlin |a Boston, Mass. |b de Gruyter |c 2011 ``` This should display like Berin ; Boston, Mass. : de Gruyter, 2011 A similar handling is required for both 260 and 264.
More samples: ``` 260 __ |a Paris : |b Impr. Vincent,$c1798 |a [i.e. Bruxelles : |b Moens, |c **1883] 260 __ |a Bethesda, Md. : |b Toxicology Information Program, National Library of Medicine [producer] ; |a Springfield, Va. : |b National Technical Information Service [distributor], |c 1974-$e(Oak Ridge, Tenn. : |f **Oak Ridge National Laboratory [generator]) ```
Start exploring Filters for this https://git.koha-community.org/Koha-community/Koha/src/branch/main/Koha/Filter/MARC A filter would add the punctuation _before_ the fields are passed on to the XSLT, so the final step would always see a (hopefully) properly punctuated record. This should make it easier to handle cases where the subfields effectively form groups of logical entries like `Paris : Impr. Vincent, 1798`.
As fields from 260 are used to build links into Koha adding the punctuation actually clutters the search terms with these chars. IOW the actual search triggered currently is not for `Paris` but for `Paris : `. This is also true for all records that hold the punctuation, so adding it automatically does not degrade the situation and Koha seems to cope with that quite well. (This issue seems to have been discussed in https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=1807 some time ago already and the current state seems _good enough_.) To improve on this end one would need to add proper functionality to the XSLT to strip (the just added or stored) punctuation. Being limited to XSLT 1 a complete solution might get a bit complex (no regexps...)
Created attachment 174325 [details] [review] Bug 37325: Add punctuation for 260 Add punctuation automatically for tag 260 (Imprint), if the record leader defines that punctuation is _omitted_ from the fields. The filter does not act on records that hold punctuation and say so in leader 18. - Hook up `Puncutation.pm` with `XSLT.pm` - Add rules for automatic punctuation of Marc21 field 260 according to https://www.loc.gov/aba/pcc/documents/isbdmarc2016.pdf - Subfields handled: a, b, c, f, g, r, t, q Test plan: 1. Select a record (A) with a properly filled and punctuated 260 field 2. Check that the punctuation display correctly in intranet and OPAC interface 3. Select a record (B) without punctuation OR remove the punctuation from the subfields of field 260. Check that leader 18 is set to `c` or `n`, IOW that this is a semantically correct record. 4. Check that _no_ punctuation chars get displayed in OPAC or intranet 5. Apply the patch 3. Edit the record (A) 4. Check that the Marc leader position 18 is _not_ set to `c` or `n` 5. Set Marc leader 18 to `c` or `n` (`c`: ISBD punctuation omitted / `n` Non-ISBD punctuation omitted) 4. Save the record 5. Check that all punctuation chars appear twice in intranet as well as OPAC interface 6. Check record (B) in intranet and OPAC interface. It should now show automatically added, proper punctuation. Note: there may be complex cases where it is not possible to properly add punctuation automatically from the subfield semantics alone. In these cases it is still possible to add punctuation to the record and specify this in leader 18. Then the filter will not change the record at all. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library
Created attachment 174326 [details] [review] Bug 37325: fix XSLT for automatic punctuation of 260 XSLT uses subfields for display _as they are_. Sometimes, these values are used to build queries against Koha based on the field content. These queries may hold punctuation chars that could potentially confuse the search. - Add simple rules to strip punctuation chars in the search terms for OPAC and intranet interfaces. Test plan: 1. Call up any record holding a 260 field in Koha 2. Check the URLs of queries generated from the subfields of 260. These show search terms including punctuation chars (`,:` etc., e.g. `Edinburg :`) 3. Apply the patch 4. Punctuation chars should vanish from the URL and only the search term should survive (e.g. `Edinburg`) This fix acts on _all_ records and is not restricted to records that got their punctuation automatically. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library
Created attachment 174467 [details] [review] Bug 37325: Add punctuation for 260 and 264 Add punctuation automatically for tag 260 (Imprint), if the record leader defines that punctuation is _omitted_ from the fields. The filter does not act on records that hold punctuation and say so in leader 18. - Hook up `Puncutation.pm` with `XSLT.pm` - Add rules for automatic punctuation of Marc21 field 260 according to https://www.loc.gov/aba/pcc/documents/isbdmarc2016.pdf - Subfields handled: a, b, c, f, g, r, t, q Note: there may be complex cases where it is not possible to properly add punctuation automatically from the subfield semantics alone. In these cases it is still possible to add punctuation to the record and specify this in leader 18. Then the filter will not change the record at all. Test plan: 1. Select a record (A) with a properly filled and punctuated 260 field 2. Check that the punctuation display correctly in intranet and OPAC interface 3. Select a record (B) without punctuation OR remove the punctuation from the subfields of field 260. Check that leader 18 is set to `c` or `n`, IOW that this is a semantically correct record. 4. Check that _no_ punctuation chars get displayed in OPAC or intranet 5. Apply the patch 6. Edit the record (A) 7. Check that the Marc leader position 18 is _not_ set to `c` or `n` 8. Set Marc leader 18 to `c` or `n` (`c`: ISBD punctuation omitted / `n` Non-ISBD punctuation omitted) 9. Save the record 10. Check that all punctuation chars appear twice in intranet as well as OPAC interface 11. Check record (B) in intranet and OPAC interface. It should now show automatically added, proper punctuation. For field 264: follow the procedure just outlined but use field 264 instead. If 264 is present, Koha will prefer it over 260. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library
Created attachment 174468 [details] [review] Bug 37325: fix XSLT for automatic punctuation of 260 and 264 XSLT uses subfields for display _as they are_. Sometimes, these values are used to build queries against Koha based on the field content. These queries may hold punctuation chars that could potentially confuse the search. - Add simple rules to strip punctuation chars in the search terms for OPAC and intranet interfaces. Test plan: 1. Call up any record holding a 260 field in Koha 2. Check the URLs of queries generated from the subfields of 260. These show search terms including punctuation chars (`,:` etc., e.g. `Edinburg :`) 3. Apply the patch 4. Punctuation chars should vanish from the URL and only the search term should survive (e.g. `Edinburg`) For field 264 follow the procedures just outlined. This fix acts on _all_ records and is not restricted to records that got their punctuation automatically. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library
Created attachment 174470 [details] [review] Bug 37325: Add punctuation for 260 and 264 Add punctuation automatically for tag 260 (Imprint), if the record leader defines that punctuation is _omitted_ from the fields. The filter does not act on records that hold punctuation and say so in leader 18. - Hook up `Puncutation.pm` with `XSLT.pm` - Add rules for automatic punctuation of Marc21 field 260 according to https://www.loc.gov/aba/pcc/documents/isbdmarc2016.pdf - Subfields handled: a, b, c, f, g, r, t, q Note: there may be complex cases where it is not possible to properly add punctuation automatically from the subfield semantics alone. In these cases it is still possible to add punctuation to the record and specify this in leader 18. Then the filter will not change the record at all. Test plan: 1. Select a record (A) with a properly filled and punctuated 260 field 2. Check that the punctuation display correctly in intranet and OPAC interface 3. Select a record (B) without punctuation OR remove the punctuation from the subfields of field 260. Check that leader 18 is set to `c` or `n`, IOW that this is a semantically correct record. 4. Check that _no_ punctuation chars get displayed in OPAC or intranet 5. Apply the patch 6. Edit the record (A) 7. Check that the Marc leader position 18 is _not_ set to `c` or `n` 8. Set Marc leader 18 to `c` or `n` (`c`: ISBD punctuation omitted / `n` Non-ISBD punctuation omitted) 9. Save the record 10. Check that all punctuation chars appear twice in intranet as well as OPAC interface 11. Check record (B) in intranet and OPAC interface. It should now show automatically added, proper punctuation. For field 264: follow the procedure just outlined but use field 264 instead. If 264 is present, Koha will prefer it over 260. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library
Created attachment 174471 [details] [review] Bug 37325: fix XSLT for automatic punctuation of 260 and 264 XSLT uses subfields for display _as they are_. Sometimes, these values are used to build queries against Koha based on the field content. These queries may hold punctuation chars that could potentially confuse the search. - Add simple rules to strip punctuation chars in the search terms for OPAC and intranet interfaces. Test plan: 1. Call up any record holding a 260 field in Koha 2. Check the URLs of queries generated from the subfields of 260. These show search terms including punctuation chars (`,:` etc., e.g. `Edinburg :`) 3. Apply the patch 4. Punctuation chars should vanish from the URL and only the search term should survive (e.g. `Edinburg`) For field 264 follow the procedures just outlined. This fix acts on _all_ records and is not restricted to records that got their punctuation automatically. Sponsored-by: Deutsches Elektronen-Synchrotron DESY, Library