Bug 29539

This problem hapens with all fields that are validate through thesaurus and use $9 to link to authority, not only 606

I confirm that problem exist on version 20.05 and after for all unimarc installations. And the problem should be on template onesubject at UNIMARCslimUtils.xsl on opac , at intranet is working as expected.

I have tried to compare UNIMARCslimUtils release 20 and release 18 but it seems there is no difference on template onesubject.
Testing intranet UNIMARCslimUtils on OPAC it does not work. So the only way is to try to find the bug or reson for such.

As you can see this is the result on OPAC

Country: Portugal.
Publication: Lisboa : Gabinete da Inspecção-Geral do Trabalho, 1992
Subject - Topical Name: 9504 | 11070 | 9501 | 9200 | 11252 | 11305 | 8439 | 6383 | 8128
 
And similar display for Intranet

Country: Portugal.
Publication: Lisboa : Gabinete da Inspecção-Geral do Trabalho, 1992
Subject - Topical Name: Inspecção-Geral do Trabalho | Relatório | Inspecção do trabalho | Higiene do trabalho | Saúde no trabalho | Segurança no trabalho | Doença profissional | Acidente de trabalho | Dados estatisticos

Who can help?

i had a look at the XSLT code here, but it is a bit complicated. Adding Frido as he might have more UNIMARC knowledge than I.

If you compare files, please note that there are 2 versions of UNIMARCslimUtils.xsl - one on staff and one on OPAC side. So maybe comparing those 2 would give some more hints.

I am not so sure if the problem is on XSLT file or any perl function that was changed and this can be a side efect

The display is rendered from the MARCXML record in the database using XSLT. There are some replacements where authorised values are linked to fields, but they should not be to blame here I think.

For the moment I solve that problem by using the template "tag_subject" from the MARC21slimUtils.xsl and it works fine. and adding also the same codes and subdivCodes from template tag_onesubject that seems to have the problem, but my xslt knowledge is not the best at the moment to solve it or to understand what is wrong, so I decided to use the same template as MARC21.  

 <xsl:template name="tag_subject">
 <xsl:param name="tag" />
  <xsl:param name="label" />
   <xsl:param name="spanclass" />
    <xsl:if test="marc:datafield[@tag=$tag]">
     <span class="results_summary subjects {$spanclass}">
      <span class="label"><xsl:value-of select="$label"/>: </span>
       <xsl:for-each select="marc:datafield[@tag=$tag]">
<a>
 <xsl:choose>
<xsl:when test="marc:subfield[@code=9]">
        <xsl:attribute name="href">/cgi-bin/koha/opac-search.pl?q=an:<xsl:value-of select="marc:subfield[@code=9]"/></xsl:attribute>
 </xsl:when>
 <xsl:otherwise>
 <xsl:attribute name="href">/cgi-bin/koha/opac-search.pl?q=su:<xsl:value-of select="marc:subfield[@code='a']"/></xsl:attribute>
 </xsl:otherwise>
</xsl:choose>
  <xsl:call-template name="chopPunctuation">
 <xsl:with-param name="chopString">
         <xsl:call-template name="subfieldSelect">
            <xsl:with-param name="codes">abcdfijkmpnptvxyz</xsl:with-param>
          <xsl:with-param name="subdivCodes">ijknpxyz</xsl:with-param>
           <xsl:with-param name="subdivDelimiter">-- </xsl:with-param>
 </xsl:call-template>
</xsl:with-param>
 </xsl:call-template>
 </a>
   <xsl:if test="not (position()=last())">
    <xsl:text> | </xsl:text>
      </xsl:if>
      </xsl:for-each>
       </span>
</xsl:if>
  </xsl:template>

Hi,

This is a very strange feature in UNIMARC XSLT that aim to look for several $9 in authority fields.
We should remove it.

Use of $9 is very important for searching using Thesaurus (authority records).
May be we need to review XSLT but if Vearnis fix works we can wait for next release we open to be solved

Created attachment 155604 [details] [review]
Bug 29539: UNIMARC: authority number in $9 displays for thesaurus controlled fields instead of content of $a

When you try to dislpay a bibliographic record on unimarc that has subjects
linked with authorities then only the $9 is displayed as link instead of the
content of $a and it's subdivisions, if any.

To test:
1) You will need to have a bibliographic record with at least one subject
autority connected in unimarc framework.
2) View that record on OPAC on detail display. The subject will display as
a number ( $9 )  and you cannot see the text/term of the subject ( $a )
3) Apply patch
4) Repeat step 2
5) The subject display in a normal way based on content $a - or more subfields

Sponsored-by: National Library of Greece

I've changed the assignee.

I couldn't replicate the issue on the current master (using the sample data in koha-testing-docker) - the topical terms for a record displayed correctly in the OPAC.

Is this for 21.05 only?

Couldn't replicate either on a sandbox (display is ok whether it's 606$a and then 606$9 or 606$9 and then 606$a)

(In reply to David Nind from comment #11)
> I've changed the assignee.
Thanks, David.
 
> I couldn't replicate the issue on the current master (using the sample data
> in koha-testing-docker) - the topical terms for a record displayed correctly
> in the OPAC.
I just managed to replicate the issue using KTD. Since I already had a
MARC21 KTD container up and running on the current master b54da05c73
and didn't want to lose my bash history, I ran these commands to set it
up with the UNIMARC sample data and fix the Zebra index configuration:

reset_all unimarc
sed -i.safe 's/marc21/unimarc/g' \
 /etc/koha/sites/kohadev/zebra-authorities-dom.cfg \
 /etc/koha/sites/kohadev/zebra-biblios-dom.cfg
koha-rebuild-zebra -a -b -f -v kohadev

I then linked the same Topical Name subject with field 606 of a test
biblio twice, and changed the order of the subfields so that in the
first 606 the $9 would appear after the $a and in the second 606 the
$a would appear after the $9. In the OPAC details view for that biblio
I got exactly what Rafael describes in this Bug's Description.

> Is this for 21.05 only?
No, it manifests on the current master, too. I've changed the Version.

(In reply to Thibault Keromnès from comment #12)
> Couldn't replicate either on a sandbox (display is ok whether it's 606$a and
> then 606$9 or 606$9 and then 606$a)
That's interesting, I actually replicated the problem today by setting
up a UNIMARC flavour instance on ByWater Solutions' sandbox server (on
the current master b54da05c73 and without applying George's patch).

As you can see here the first Topical subject does not display properly:
https://opac-abefore9.sandbox.bywatersolutions.com/cgi-bin/koha/opac-detail.pl?biblionumber=53

The order of the relevant subfields in that biblio is as follows:
606 _a écosystèmes
    _9 6410
606 _9 6410
    _a écosystèmes

Created attachment 161262 [details] [review]
Bug 29539: UNIMARC: authority number in $9 displays for thesaurus controlled fields instead of content of $a

When you try to dislpay a bibliographic record on unimarc that has subjects
linked with authorities then only the $9 is displayed as link instead of the
content of $a and it's subdivisions, if any.

To test:
1) You will need to have a bibliographic record with at least one subject
autority connected in unimarc framework.
2) View that record on OPAC on detail display. The subject will display as
a number ( $9 )  and you cannot see the text/term of the subject ( $a )
3) Apply patch
4) Repeat step 2
5) The subject display in a normal way based on content $a - or more subfields

Sponsored-by: National Library of Greece
Signed-off-by: David Nind <david@davidnind.com>

Thanks Andreas for the steps on how to reproduce the issue!

I have now tested and signed off using KTD (koha-testing-docker).

Updated test plan:
1. KTD startup: edit your .env file and change KOHA_MARC_FLAVOUR=unimarc
2. Start up KTD as normal: ktd pull and ktd up
3. Edit any record so that it has two 606 entries - NOTE: The order displayed in the editor is important:
   - For record 560 ([Le ]Jardin des Henderson : roman [LIVR] / Hermary-Vieille Catherine), remove the existing values in the 606 field
   - Add these values:
     606 _a écosystèmes
         _9 6410
     606 _9 6410
         _a écosystèmes
   NOTE: The sample data for UNIMARC records is very incomplete - you will need to:
         1. Add data for the mandatory fields before you can save the record.
         2. Update the item(s) for the record so that any values with dropdown lists have valid values (otherwise you will have errors in the OPAC when you try to view the record).
4. View the record in the OPAC and staff interface:
   - OPAC: Subject - Topical Name: 6410 | écosystèmes 
   - Staff interface: Subject - Topical Name: écosystèmes | écosystèmes 
5. Apply the patch.
6. Restart everything.
7. View the record again in the OPAC and staff interface
   - OPAC: Subject - Topical Name: écosystèmes | écosystèmes
   - Staff interface: Subject - Topical Name: écosystèmes | écosystèmes
8. Experiment with changing the order and adding new terms - these should now display correctly, irrespective of the $a and $9 order in the record.
9. Sign off.

-            <xsl:call-template name="tag_onesubject">

This is the last reference, right?
So we are using it no longer. Please remove dead code too.

This template contains a lot of additional logic. Could you or Fridolin explain what its purpose should have been?

(In reply to Marcel de Rooy from comment #17)
> 
> This template contains a lot of additional logic. Could you or Fridolin
> explain what its purpose should have been?

This was a try to show composed authority fields, with method called Rameau.
An field like 600 may contain an authority link for $a and another authority link for $x. In this case the $9 must be before the linked field :
  $9 888
  $a History
  $9 999
  $x Antiquity

But this is not implemented in Koha record edition so it is logical to remove this behavior.

Created attachment 161622 [details] [review]
Bug 29539: (follow-up) remove tag_onesubject template

Also:
Restores the <span class="value">
Replaces "not(position()=last())" with the more used "position() != last()"
Removes unecessary change in <xsl:param name="spanclass" />

Thanks everyone for moving this dirty bug forward ;)

Created attachment 161768 [details] [review]
Bug 29539: UNIMARC: authority number in $9 displays for thesaurus controlled fields instead of content of $a

When you try to dislpay a bibliographic record on unimarc that has subjects
linked with authorities then only the $9 is displayed as link instead of the
content of $a and it's subdivisions, if any.

To test:
1) You will need to have a bibliographic record with at least one subject
autority connected in unimarc framework.
2) View that record on OPAC on detail display. The subject will display as
a number ( $9 )  and you cannot see the text/term of the subject ( $a )
3) Apply patch
4) Repeat step 2
5) The subject display in a normal way based on content $a - or more subfields

Sponsored-by: National Library of Greece
Signed-off-by: David Nind <david@davidnind.com>

Created attachment 161769 [details] [review]
Bug 29539: (follow-up) remove tag_onesubject template

Also:
Restores the <span class="value">
Replaces "not(position()=last())" with the more used "position() != last()"
Removes unecessary change in <xsl:param name="spanclass" />

Signed-off-by: David Nind <david@davidnind.com>

Created attachment 163191 [details] [review]
Bug 29539: UNIMARC: authority number in $9 displays for thesaurus controlled fields instead of content of $a

When you try to dislpay a bibliographic record on unimarc that has subjects
linked with authorities then only the $9 is displayed as link instead of the
content of $a and it's subdivisions, if any.

To test:
1) You will need to have a bibliographic record with at least one subject
autority connected in unimarc framework.
2) View that record on OPAC on detail display. The subject will display as
a number ( $9 )  and you cannot see the text/term of the subject ( $a )
3) Apply patch
4) Repeat step 2
5) The subject display in a normal way based on content $a - or more subfields

Sponsored-by: National Library of Greece
Signed-off-by: David Nind <david@davidnind.com>

Bug 29539: (follow-up) remove tag_onesubject template

Also:
Restores the <span class="value">
Replaces "not(position()=last())" with the more used "position() != last()"
Removes unecessary change in <xsl:param name="spanclass" />

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Hi Marcel, I tried to apply the patches in the attached sequence and the other way around, I always end up with: 

Apply? [(y)es, (n)o, (i)nteractive] i
Applying: Bug 29539: (follow-up) remove tag_onesubject template
error: sha1 information is lacking or useless (koha-tmpl/opac-tmpl/bootstrap/en/xslt/UNIMARCslimUtils.xsl).
error: could not build fake ancestor
Patch failed at 0001 Bug 29539: (follow-up) remove tag_onesubject template
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-29539-follow-up-remove-tagonesubject-template-daluwkeh.patch
kohadev-koha@kohadevbox:koha(master|AM 1/1)$ git log

Can you please check/rebase on latest master?

(In reply to Katrin Fischer from comment #24)
> Hi Marcel, I tried to apply the patches in the attached sequence and the
> other way around, I always end up with: 
> 
> Apply? [(y)es, (n)o, (i)nteractive] i
> Applying: Bug 29539: (follow-up) remove tag_onesubject template
> error: sha1 information is lacking or useless
> (koha-tmpl/opac-tmpl/bootstrap/en/xslt/UNIMARCslimUtils.xsl).
> error: could not build fake ancestor
> Patch failed at 0001 Bug 29539: (follow-up) remove tag_onesubject template
> hint: Use 'git am --show-current-patch=diff' to see the failed patch
> When you have resolved this problem run "git bz apply --continue".
> If you would prefer to skip this patch, instead run "git bz apply --skip".
> To restore the original branch and stop patching run "git bz apply --abort".
> Patch left in
> /tmp/Bug-29539-follow-up-remove-tagonesubject-template-daluwkeh.patch
> kohadev-koha@kohadevbox:koha(master|AM 1/1)$ git log
> 
> Can you please check/rebase on latest master?

I would be happy to leave that to the author.

That's alright, just easier for someone with the branch still around.

There's a single sign off in Comment 23 which appears to involve
two patches (instead of two separate sign offs, one per patch).
I think that's the reason the current set of patches (attachment
ids 161768 and 163191) does not apply cleanly.

Comment on attachment 161769 [details] [review]
Bug 29539: (follow-up) remove tag_onesubject template

Can confirm that the combined patch still applies.

I have obsoleted the separate follow-up patch and will change the status back to Passed QA.

Thanks for all the hard work!

Pushed to main for the next 24.11.00 release as RM Assistant

Backported to 24.05.x for upcoming 24.05.02

Pushed to 23.11.x for 23.11.07

not backporting to 23.05.x unless requested

Hi! Would it be possible to backport it to 22.11? At our library, we've been struggling with this problem for a long time and I'm glad a solution was finally found, as we have a lot of records in the situation described by Rafael António.

(In reply to Tiago Simões from comment #33)
> Hi! Would it be possible to backport it to 22.11? At our library, we've been
> struggling with this problem for a long time and I'm glad a solution was
> finally found, as we have a lot of records in the situation described by
> Rafael António.

Tiago, Wainu declined to backported this to 23.05. Although it sounds like she could be persuaded to do so? 

If it can be backported to 23.05 there is a chance it will also backported to 22.11.

(In reply to Lucas Gass from comment #34)
> (In reply to Tiago Simões from comment #33)
> > Hi! Would it be possible to backport it to 22.11? At our library, we've been
> > struggling with this problem for a long time and I'm glad a solution was
> > finally found, as we have a lot of records in the situation described by
> > Rafael António.
> 
> Tiago, Wainu declined to backported this to 23.05. Although it sounds like
> she could be persuaded to do so? 
> 
> If it can be backported to 23.05 there is a chance it will also backported
> to 22.11.

I understand. If I'm the only one wanting this, it's understandable that she doesn't backport it. I'll try and make the case to our IT department to upgrade to a more recent version sooner than planned, then.

Thanks anyway!

I can have a look at backporting this to 23.11 for the next release

Could I please have a more detailed test plan for 23.05?

(In reply to wainuiwitikapark from comment #37)
> Could I please have a more detailed test plan for 23.05?

For KTD

Not sure if the comments above were for me since I expressed interest in backporting it to previous versions than 23.11, but I can try and test in KTD. The problem is that I don't have any experience with docker so it might take a while for me to come up with a test plan as I need to familiarize myself with it.

Any kind of help in doing so is welcome, if only to speed things up. If not, as far as I'm concerned, I can wait for an upgrade to a supported version.

If the comments weren't directed at me, then please ignore.

(In reply to Tiago Simões from comment #39)
> Not sure if the comments above were for me since I expressed interest in
> backporting it to previous versions than 23.11, but I can try and test in
> KTD. The problem is that I don't have any experience with docker so it might
> take a while for me to come up with a test plan as I need to familiarize
> myself with it.
> 
> Any kind of help in doing so is welcome, if only to speed things up. If not,
> as far as I'm concerned, I can wait for an upgrade to a supported version.
> 
> If the comments weren't directed at me, then please ignore.

Hi Tiago, Thanks for your response!

My comments were just directed at anyone really :)

I am having trouble setting up the following: You will need to have a bibliographic record with at least one subject autority connected in unimarc framework.

I'm not very familiar with unimarc and unsure how to change the marcflavour on my ktd and having it persist through a reset_all.

Any help from anyone would be greatly appreciated :)

(In reply to Wainui Witika-Park from comment #40)
> (In reply to Tiago Simões from comment #39)
> > Not sure if the comments above were for me since I expressed interest in
> > backporting it to previous versions than 23.11, but I can try and test in
> > KTD. The problem is that I don't have any experience with docker so it might
> > take a while for me to come up with a test plan as I need to familiarize
> > myself with it.
> > 
> > Any kind of help in doing so is welcome, if only to speed things up. If not,
> > as far as I'm concerned, I can wait for an upgrade to a supported version.
> > 
> > If the comments weren't directed at me, then please ignore.
> 
> Hi Tiago, Thanks for your response!
> 
> My comments were just directed at anyone really :)
> 
> I am having trouble setting up the following: You will need to have a
> bibliographic record with at least one subject autority connected in unimarc
> framework.
> 
> I'm not very familiar with unimarc and unsure how to change the marcflavour
> on my ktd and having it persist through a reset_all.
> 
> Any help from anyone would be greatly appreciated :)

Hi Wainui,

I've finally set up ktd with the 23.05 version and with the UNIMARC flavour. I've initially started it with marc21 and then switched to unimarc by accessing the .env file in the koha-testing-docker directory and changing the KOHA_MARC_FLAVOUR parameter from marc21 to unimarc.

Everything seemed to work fine. I noticed the sample bibliographic records were different (all in french as far as I could tell) and the records' information was filled in the right unimarc fields (again, as far as I could tell).

I even tried reset_all and reset_all_unimarc and, again, no issues.

But I did notice that there were so little MARC bibliographic frameworks created (Default, Acquisitions and Fast Cataloguing), so the fields were kind of all over the place. Lots of mandatory unimarc fields that should not be mandatory, which makes the process of reproducing the bug difficult, since you first need to edit a record and switch the order in which the 606$9 and 606$a appear.

If you still want to go through with backporting to older versions, I can create a bibliographic framework on my ktd environment, at least for books, just to make the process less of a hassle. If not, I completely understand.

Either way, would it be worth it to ask for these frameworks to be added to ktd?

I'd also add another step between 2) and 3), which is:
Edit one or more records and switch the order of $9 and $a so the latter comes before the former. This can be tested with just the 606 field, as it is the most commonly used one in the 6 class.

I hope my reply was helpful and not too confusing :).

Hi Tiago, there are some other bugs about improving the default UNIMARC frameworks and their behavior filed and in progress already.

(In reply to Katrin Fischer from comment #42)
> Hi Tiago, there are some other bugs about improving the default UNIMARC
> frameworks and their behavior filed and in progress already.

Hi Katrin, you're right, sorry, I forgot to do a search on bugzilla for them. Thanks for the heads-up.

I might leave this out of 23.05 if that's okay?