Bug 10488 - New MARC21 authority tags and subfields should be propagated to non-default frameworks
Summary: New MARC21 authority tags and subfields should be propagated to non-default f...
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Bernardo Gonzalez Kriegel
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on: 9826
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-18 15:01 UTC by Galen Charlton
Modified: 2015-06-04 23:23 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Excluded tags on current auth types (35.27 KB, application/pdf)
2013-06-19 00:23 UTC, Bernardo Gonzalez Kriegel
Details
Bug 10488 - New MARC21 authority tags and subfields should be propagated to non-default frameworks (972.94 KB, patch)
2013-06-20 02:20 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
Bug 10488 - New MARC21 authority tags and subfields should be propagated to non-default frameworks (976.75 KB, patch)
2013-06-27 02:30 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
Bug 10488 - New MARC21 authority tags and subfields should be propagated to non-default frameworks (977.35 KB, patch)
2013-07-01 12:06 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 10488: Followup replacing DELETEs by more restricted INSERTs (16.79 KB, patch)
2013-07-01 12:06 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 10488: Followup replacing DELETEs by more restricted INSERTs (17.24 KB, patch)
2013-07-03 07:48 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Galen Charlton 2013-06-18 15:01:56 UTC
The patch for bug 9826 updated the default MARC21 authority frameworks with new tags and subfields, but new fields were added only to the default framework.

Appropriate tags should be added to the non-default frameworks, e.g., PERSO_NAME, TOPIC_TERM, etc.

A possible restructuring of authorities_normal_marc21.sql that might make it easier to maintain would be to have it populate each non-default framework by first making a copy of the default tags and subfields, then removing the tags that are not relevant to the authority type.
Comment 1 Bernardo Gonzalez Kriegel 2013-06-18 17:07:19 UTC
(In reply to comment #0)
> A possible restructuring of authorities_normal_marc21.sql that might make it
> easier to maintain would be to have it populate each non-default framework
> by first making a copy of the default tags and subfields, then removing the
> tags that are not relevant to the authority type.

That's easy to implement, and a great idea.
Problem is (for me), which tags are not relevant?

Current count of tags (with bug 9826) is

+-----------------+--------------+
| count(tagfield) | authtypecode |
+-----------------+--------------+
|             164 |              |
|              83 | CHRON_TERM   |
|              84 | CORPO_NAME   |
|              84 | GENRE/FORM   |
|              83 | GEOGR_NAME   |
|              85 | MEETI_NAME   |
|              84 | PERSO_NAME   |
|              83 | TOPIC_TERM   |
|              92 | UNIF_TITLE   |
+-----------------+--------------+

Is there some criteria over which tags are not relevant for a given auth type?
Comment 2 Bernardo Gonzalez Kriegel 2013-06-19 00:23:58 UTC
Created attachment 19145 [details]
Excluded tags on current auth types

This pdf show excluded tags on current auth types,
save for orange cell that ARE included.
(excluded tags are present in default framework)

Light blue are new tags from bug 9826,
and green ones are obsoleted tags.
Comment 3 Bernardo Gonzalez Kriegel 2013-06-20 02:20:56 UTC Comment hidden (obsolete)
Comment 4 Bernardo Gonzalez Kriegel 2013-06-27 02:30:07 UTC Comment hidden (obsolete)
Comment 5 Marcel de Rooy 2013-07-01 12:06:27 UTC
Created attachment 19312 [details] [review]
Bug 10488 - New MARC21 authority tags and subfields should be propagated to non-default frameworks

As proposed by the report creator, this patch
rewites authorities_normal_marc21.sql
using default authority framework to build all
current authtypes. Then proceed to delete not
relevant tags.

It also updates default authority framework
to last update (April 2013) adding new or
updated tags/subtags.

All new tags/subtags from Bug 9826 and this
one are included in all authtypes.

Deleted tags reflect current situation except
for 7XX tags that I was tell must be present
on all authtypes.

Only for new installs

To test:
1) Apply patch
2) Delete all authtypes
exec in database
    delete from auth_types;
    delete from auth_tag_structure;
    delete from auth_subfield_structure;
3) Insert new auth framework
4) Verify absent tags. This can be done running

    select tagfield as from auth_tag_structure
    where authtypecode ='' and tagfield not in
    (select tagfield from auth_tag_structure
    where authtypecode = 'AUTH_TYPE')

for each authtype ( UNIF_TITLE, TOPIC_TERM, etc.)
The attached PDF could be used as a reference,
all that is white or green for each authtype must be
in the list (except for white 7XX tags)
5) Sign :)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 6 Marcel de Rooy 2013-07-01 12:06:36 UTC Comment hidden (obsolete)
Comment 7 Marcel de Rooy 2013-07-01 12:19:04 UTC
QA Comment:
This patch makes a great contribution in removing redundancy from this file!
Looks good to me. 
I did a fresh install and [quickly] compared the list of tags that you do not insert for the other auth types with the list on loc.gov.

Note: These changes should also be propagated to the other language folders. But note that I am working on bug 10509 and its followup reports. These reports go through all folders too and will remove more redundancy.
In the light of that, it may be wise to postpone this activity a little bit and communicate with each other on the progress of the other report.
Thanks.

BTW My followup combines your insert and delete into one more restricted insert.
Just a matter of combining the two clauses.

Passed QA
Comment 8 Galen Charlton 2013-07-02 13:55:29 UTC
After applying the revised SQL, I noticed that the 162 tag (medium of performance) appears in all of the specific auth types:

| CHRON_TERM   | 148      |
| CHRON_TERM   | 162      |
| CHRON_TERM   | 182      |
| CORPO_NAME   | 110      |
| CORPO_NAME   | 162      |
| GENRE/FORM   | 155      |
| GENRE/FORM   | 162      |
| GENRE/FORM   | 185      |
| GEOGR_NAME   | 151      |
| GEOGR_NAME   | 162      |
| GEOGR_NAME   | 181      |
| MEETI_NAME   | 111      |
| MEETI_NAME   | 162      |
| PERSO_NAME   | 100      |
| PERSO_NAME   | 162      |
| TOPIC_TERM   | 150      |
| TOPIC_TERM   | 162      |
| TOPIC_TERM   | 180      |
| UNIF_TITLE   | 130      |
| UNIF_TITLE   | 162      |
+--------------+----------+

The 162 certainly doesn't belong in them all, and I'm not sure if it belongs in GENRE/FORM or if a new authority type code should be defined.

Setting to "In Discussion".  This patch is very close to being ready, but the 162 should be dealt with, either by simply excluding it from all but the default authority type or by defining a new authority type.  I'm hoping Jared can advise regarding the latter.
Comment 9 Bernardo Gonzalez Kriegel 2013-07-03 02:37:45 UTC
(In reply to Galen Charlton from comment #8)
> The 162 certainly doesn't belong in them all, and I'm not sure if it belongs
> in GENRE/FORM or if a new authority type code should be defined.
> 
> Setting to "In Discussion".  This patch is very close to being ready, but
> the 162 should be dealt with, either by simply excluding it from all but the
> default authority type or by defining a new authority type.  I'm hoping
> Jared can advise regarding the latter.

I think the fastest solution is to exclude 162 from all non default frameworks.
Any new authtype could be created later.
Comment 10 Marcel de Rooy 2013-07-03 07:27:48 UTC
In this new Bugzilla version, it happens sometimes that I cannot reply to a comment (the former comment is not pasted in to the new comment)..

> The 162 certainly doesn't belong in them all, and I'm not sure if it belongs
> in GENRE/FORM or if a new authority type code should be defined.
Good catch, Galen. Scrutiny++ :) It looks to me that this actually is a new type of authority. 
See for instance: http://www.loc.gov/marc/marbi/2012/2012-dp02.html

> ...but the 162 should be dealt with, either by simply excluding it from all
> but the default authority type or by defining a new authority type.  
I agree with Bernardo's last suggestion to exclude the 162 now.
And maybe we can let Jared(..) add the new type on a separate report.
Comment 11 Marcel de Rooy 2013-07-03 07:48:45 UTC
Created attachment 19357 [details] [review]
Bug 10488: Followup replacing DELETEs by more restricted INSERTs

Instead of inserting followed by deleting, this patch combines the two where
clauses of both sql statements, meanwhile leaving the separation intact.

As suggested by Galen, removed all 162's from the non-default types.
The 162 calls for a new authority type. We can add that later on.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Ran this .sql script after deleting all auth records.
The former version had 9 types, 1148 tags and 11621 subfields.
This one has 9 types, 1140 tags and 11597 subfields.
That is expected: 8 tags 162 less and 8x3 subfields less.
Comment 12 Marcel de Rooy 2013-07-03 07:50:39 UTC
I have updated the followup for removal of 162, and park this report into Passed QA status again to the attention of Galen.
Comment 13 Galen Charlton 2013-07-03 11:28:34 UTC
Pushed to master.  Thanks, Bernardo and Marcel!
Comment 14 Tomás Cohen Arazi 2013-07-07 00:07:22 UTC
This patch has been pushed to 3.12.x, will be in 3.12.2.

Thanks Bernardo and Marcel!