Bug 24407

Summary: AuthDisplayHierarchies ignores new nodes and needs protection against cyclic relations
Product: Koha Reporter: Marcel de Rooy <m.de.rooy>
Component: MARC Authority data supportAssignee: Bugs List <koha-bugs>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: tomascohen, wizzyrea
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23190
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description Marcel de Rooy 2020-01-13 12:23:08 UTC
Suppose I have term A with children B and C where the children have a 550 tag with broader indication to A.
If I go to record A, the tree is built showing B and C.
If I add a child D to A, the tree for A is not rebuilt but still showing the old situation.

A related problem:
If I change term A setting it to a be child of B with a 550 (which it obviously not is), this cyclic relation will trigger deep recursion. Resulting in a gateway timeout and a process eating up resources..
Comment 1 Marcel de Rooy 2020-01-13 12:23:54 UTC
Katrin,
If I am not mistaken, you are using it? Could you comment on this?
Comment 2 Katrin Fischer 2020-01-13 13:15:42 UTC
(In reply to Marcel de Rooy from comment #1)
> Katrin,
> If I am not mistaken, you are using it? Could you comment on this?

Sadly not at the moment, see major bug 23190 - I can't get it to work at the moment.
Comment 3 Marcel de Rooy 2020-01-13 14:27:49 UTC
(In reply to Katrin Fischer from comment #2)
> (In reply to Marcel de Rooy from comment #1)
> > Katrin,
> > If I am not mistaken, you are using it? Could you comment on this?
> 
> Sadly not at the moment, see major bug 23190 - I can't get it to work at the
> moment.

Could it be related to the first thing mentioned here?
Comment 4 Katrin Fischer 2020-01-14 07:25:33 UTC
I am not sure - could you try my cataloging example on the other bug or tell me if it looks right to you? Really confused by this one.
Comment 5 Marcel de Rooy 2020-01-21 09:00:30 UTC
(In reply to Marcel de Rooy from comment #3)
> (In reply to Katrin Fischer from comment #2)
> > (In reply to Marcel de Rooy from comment #1)
> > > Katrin,
> > > If I am not mistaken, you are using it? Could you comment on this?
> > 
> > Sadly not at the moment, see major bug 23190 - I can't get it to work at the
> > moment.
> 
> Could it be related to the first thing mentioned here?

A closer look tells me that they are not the same.
Comment 6 Tomás Cohen Arazi (tcohen) 2021-12-16 12:09:18 UTC
(In reply to Marcel de Rooy from comment #0)
> Suppose I have term A with children B and C where the children have a 550
> tag with broader indication to A.
> If I go to record A, the tree is built showing B and C.
> If I add a child D to A, the tree for A is not rebuilt but still showing the
> old situation.

What you're saying is that if you sit on the parent, it doesn't have a way to know a new child showed up, and you're proposing some automatic bi-directional linking? I like this.

> A related problem:
> If I change term A setting it to a be child of B with a 550 (which it
> obviously not is), this cyclic relation will trigger deep recursion.
> Resulting in a gateway timeout and a process eating up resources..

Had to deal with this in production. I wrote a tiny script that does a DFS to find cycles from a leaf, following the $wg path (the $wh are not considered in most of the code as far as I can tell.)
Comment 7 Liz Rea 2023-11-16 21:29:41 UTC
We've seen this too! 

           authid: 25088
     authtypecode: UNIF_TITLE
      datecreated: 2023-11-15
modification_time: 2023-11-15 12:02:37
       origincode: NULL
        authtrees: 24939,25086,25088;24939,25083,25086,25088;24939,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088,25086,25088;25085,25086,25088,25086,25088,25086,25088;24939,25083,25088,25086,25088,25086,25088;2553,25088,25086,25088,25086,25088;24939,25083,5794,25086,25088,25086,25088;24939,4524,25086,25088,25086,25088;25085,25086,25088,25086,25088;24939,25083,25088,25086,25088;2553,25088,25086,25088;24939,25083,5794,25086,25088;24939,4524,25086,25088;25085,25086,25088;24939,25083,25088;2553,25088
             marc: 00815nz  a2200181o  450000100060000000300080000600500170001400800410003104000210007213000400009350000860013353000760021955000340029566700390032967001600036873000900052894200150061825088ByWater20231115115214.0231115|| aca||aabn           | a|a     d  aByWatercByWater 0aAugust 2023 wildfires in Hawaiʻi. 1 925086aLee, Erica A.c(Analyst in Emergency Management and Disaster Recovery) wg 0925083aInsight (Library of Congress. Congressional Research Service)wg  92553aEnvironmental policywg  aMachine generated authority record  aWork cat.: (ByWater)20509: 25075 Rowan, Linda R. (Analyst in Natural Resources and Earth Sciences)  author. , August 2023 wildfires in Hawaiʻi. / , c2023-  iInsight (Library of Congress. Congressional Research Service)kLegislative materials   aUNIF_TITLE
           linkid: NULL
          marcxml: <?xml version="1.0" encoding="UTF-8"?>
<record
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"
    xmlns="http://www.loc.gov/MARC21/slim">

  <leader>00815nz  a2200181o  4500</leader>
  <controlfield tag="001">25088</controlfield>
  <controlfield tag="003">ByWater</controlfield>
  <controlfield tag="005">20231115115214.0</controlfield>
  <controlfield tag="008">231115|| aca||aabn           | a|a     d</controlfield>
  <datafield tag="040" ind1=" " ind2=" ">
    <subfield code="a">ByWater</subfield>
    <subfield code="c">ByWater</subfield>
  </datafield>
  <datafield tag="130" ind1=" " ind2="0">
    <subfield code="a">August 2023 wildfires in Hawaiʻi. </subfield>
  </datafield>
  <datafield tag="500" ind1="1" ind2=" ">
    <subfield code="9">25086</subfield>
    <subfield code="a">Lee, Erica A.</subfield>
    <subfield code="c">(Analyst in Emergency Management and Disaster Recovery) </subfield>
    <subfield code="w">g</subfield>
  </datafield>
  <datafield tag="530" ind1=" " ind2="0">
    <subfield code="9">25083</subfield>
    <subfield code="a">Insight (Library of Congress. Congressional Research Service)</subfield>
    <subfield code="w">g</subfield>
  </datafield>
  <datafield tag="550" ind1=" " ind2=" ">
    <subfield code="9">2553</subfield>
    <subfield code="a">Environmental policy</subfield>
    <subfield code="w">g</subfield>
  </datafield>
  <datafield tag="667" ind1=" " ind2=" ">
    <subfield code="a">Machine generated authority record</subfield>
  </datafield>
  <datafield tag="670" ind1=" " ind2=" ">
    <subfield code="a">Work cat.: (ByWater)20509: 25075 Rowan, Linda R. (Analyst in Natural Resources and Earth Sciences)  author. , August 2023 wildfires in Hawaiʻi. / , c2023-</subfield>
  </datafield>
  <datafield tag="730" ind1=" " ind2=" ">
    <subfield code="i">Insight (Library of Congress. Congressional Research Service)</subfield>
    <subfield code="k">Legislative materials </subfield>
  </datafield>
  <datafield tag="942" ind1=" " ind2=" ">
    <subfield code="a">UNIF_TITLE</subfield>
  </datafield>
</record>


In our case it eventually segfaulted perl, then OOM, then recovery. Errors in logs were as follows: 

[WARN] Deep recursion on subroutine "C4::AuthoritiesMarc::BuildAuthHierarchies" at /usr/share/koha/lib/C4/AuthoritiesMarc.pm line 1196.
kernel: [79254.383605] /usr/share/koha[28310]: segfault at 0 ip 000055adef22011d sp 00007ffc0e9e0720 error 6 in perl[55adef21c000+15e000]