Bug 16245 - RIS export file type incorrect
Summary: RIS export file type incorrect
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: master
Hardware: All All
: P5 - low minor (vote)
Assignee: Héctor Eduardo Castro Avalos
QA Contact: Marcel de Rooy
URL:
Keywords:
: 7989 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-04-12 07:56 UTC by Hugo Agud
Modified: 2017-12-07 22:18 UTC (History)
8 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 16245: RIS export file type incorrect (4.32 KB, patch)
2016-09-28 21:43 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 16245: RIS export file type incorrect (4.53 KB, patch)
2016-09-28 21:47 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 16245: RIS export file type incorrect (4.69 KB, patch)
2016-09-29 20:31 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 16245: RIS export file type incorrect (4.74 KB, patch)
2016-10-05 02:05 UTC, Chris Cormack
Details | Diff | Splinter Review
Bug 16245: RIS export file type incorrect (4.84 KB, patch)
2016-10-14 08:25 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16245: [QA Follow-up] Replace typeofrecord2 by biblevel (2.61 KB, patch)
2016-10-14 08:26 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Hugo Agud 2016-04-12 07:56:16 UTC
Good morning

We have found one potential bug on this patch (https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12357).

The issue is that RisExportAdditionalFields doesn't recognize property the TY

This patch is not recognizing propertly the type of the record, if leader position 6 is 'a' it publishs as book even the position 7 is b=article


And it fails when try to modify it.

We have tried to add TY: [942$c] to force the patch to replace the TY value, but it keeps the default Book value

Koha 3.22.5 (debian jessy 8 with plack enabled)

Test:
Catalogue a record with LDR 6 and 7 'ab' --> this is an article
check the record type is article in koha leader

Download as RIS and check that type is book

Modify syspref  RisExportAdditionalFields 

TY: [337$a] (or whatever you wish)

Download as RIS
the ty is still book, it doesn't matter the value of 337$a or whatever
Comment 1 Héctor Eduardo Castro Avalos 2016-09-28 21:21:16 UTC
*** Bug 7989 has been marked as a duplicate of this bug. ***
Comment 2 Héctor Eduardo Castro Avalos 2016-09-28 21:43:16 UTC
Created attachment 55896 [details] [review]
Bug 16245: RIS export file type incorrect

When exporting lists to a RIS file, the TY field of the RIS file is always
'book' even if the item being exported is a journal or serial.
Since unimarc and marc21 is coded in Koha, just a few references types are
included. This needs a much better code.

This bug changes only TY BOOK result. Other resources needs a case by case
basis.

To test
-In advanced search limit by itemtype (books, music, continue resources, etc).
-Search and save result in Cart
-Export to RIS Format and notice about the new TY change with all materials
-Test with http://www.myendnoteweb.com if you have account or create one.
-Export record in bib record details page in OPAC and Intranet.
-Confirm that works as expected.

Sponsored-by: Universidad de El Salvador
Comment 3 Héctor Eduardo Castro Avalos 2016-09-28 21:47:25 UTC
Created attachment 55897 [details] [review]
Bug 16245: RIS export file type incorrect

When exporting lists to a RIS file, the TY field of the RIS file is always
'book' even if the item being exported is a journal or serial.
Since unimarc and marc21 is coded in Koha, just a few references types are
included. This needs a much better code.

This bug changes only TY BOOK result. Other resources needs a case by case
basis.

To test
-In advanced search limit by itemtype (books, music, continue resources, etc).
-Search and save result in Cart
-Export to RIS Format and notice about the new TY change with all materials
-Test with http://www.myendnoteweb.com if you have account or create one.
-Export record in bib record details page in OPAC and Intranet.
-Confirm that works as expected.

Sponsored-by: Universidad de El Salvador
Comment 4 Héctor Eduardo Castro Avalos 2016-09-29 19:55:49 UTC
Doing a little search for some other reference types, I could obtain this table:
FOR MARC 21

Book         BOOK
Leader/06 a
Leader/07 m

Serial       SER
Leader/06 a
Leader/07 s

Full Journal JFULL
Leader/06 a
Leader/07 s
008/21    p

Newspapers   NEWS
Leader/06 a
Leader/07 s
008/21    n

Book chapter  CHAP
Leader/06 a
Leader/07 a
773$7/02  a
773$7/03  m

Journal article  JOUR
Leader/06 a
Leader/07 a
773$7/02  a
773$7/03  s

*************************

Catalog  CTLG
Could be serial or simple monograph
Book
Leader/06 a
Leader/07 m
008/24-27 c

Serial
Leader/06 a
Leader/07 s
008/24    c

************************
Conference Paper  CPAPER
Serial, book
Book
Leader/06 a
Leader/07 m
008/29    1

Serial
Leader/06 a
Leader/07 s
008/29    1

*************************
Patent  PAT
Book
Leader/06 a
Leader/07 m
008/24-27 j

Serial
Leader/06 a
Leader/07 s
008/24    j

*************************
Thesis    THES
Book
Leader/06 a
Leader/07 m
008/24-27 m

Serial
Leader/06 a
Leader/07 s
008/24    m

*************************
Case    CASE
Book
Leader/06 a
Leader/07 m
008/24-27 vw

Serial
Leader/06 a
Leader/07 s
008/24    vw

*************************
Statute  STAT
Book
Leader/06 a
Leader/07 m
008/24-27 lz

Serial
Leader/06 a
Leader/07 s
008/24    lz

*************************
Encyclopedia    ENCYC
Book
Leader/06 a
Leader/07 m
008/24-27 e

Serial
Leader/06 a
Leader/07 s
008/24    e

*************************
Dictionaries    DICT
Book
Leader/06 a
Leader/07 m
008/24-27 d

Serial
Leader/06 a
Leader/07 s
008/24    d

Unpublished Work and Manuscript, the same in MARC21
MANSCPT, UNPD
Leader/06 t
Leader/07 m

########################
FOR UNIMARC

Book         BOOK
Leader/06 a
Leader/07 m

Serial       SER
Leader/06 a
Leader/07 s

Full Journal JFULL
Leader/06 a
Leader/07 s
100$a/0   a

Newspapers   NEWS
Leader/06 a
Leader/07 s
100$a/0   c

Book chapter  CHAP
Leader/06 a
Leader/07 a
463

Journal article  JOUR
Leader/06 a
Leader/07 a
461

Dictionaries, Encyclopedias, case, statutes, etc.
For book and serials
105$a/4-7
110$a/3
Comment 5 Héctor Eduardo Castro Avalos 2016-09-29 20:31:42 UTC
Created attachment 55941 [details] [review]
Bug 16245: RIS export file type incorrect

When exporting lists to a RIS file, the TY field of the RIS file is always
'book' even if the item being exported is a journal or serial.
Since unimarc and marc21 is coded in Koha, just a few references types are
included. This needs a much better code.

This bug changes only TY BOOK result. Other resources needs a case by case
basis.

To test:
-In advanced search limit by itemtype (books, music, continue resources, etc).
-Search and save result in Cart
-Export to RIS Format and notice about the new TY change with all materials
-Export record in bib record details page in OPAC and Intranet.
-Confirm that works as expected.
-Bonus test:
-Try to import the  RIS file in  http://www.myendnoteweb.com if you
have account or create one.
NOTE: According with previous comment, some values has changed.

Sponsored-by: Universidad de El Salvador
Comment 6 Chris Cormack 2016-10-05 02:05:18 UTC
Created attachment 56043 [details] [review]
Bug 16245: RIS export file type incorrect

When exporting lists to a RIS file, the TY field of the RIS file is always
'book' even if the item being exported is a journal or serial.
Since unimarc and marc21 is coded in Koha, just a few references types are
included. This needs a much better code.

This bug changes only TY BOOK result. Other resources needs a case by case
basis.

To test:
-In advanced search limit by itemtype (books, music, continue resources, etc).
-Search and save result in Cart
-Export to RIS Format and notice about the new TY change with all materials
-Export record in bib record details page in OPAC and Intranet.
-Confirm that works as expected.
-Bonus test:
-Try to import the  RIS file in  http://www.myendnoteweb.com if you
have account or create one.
NOTE: According with previous comment, some values has changed.

Sponsored-by: Universidad de El Salvador
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Comment 7 Jonathan Druart 2016-10-05 07:41:21 UTC
What is 'MANSCPT'?
Are the changes related to MANSCPT expected?
Comment 8 Héctor Eduardo Castro Avalos 2016-10-05 16:14:42 UTC
(In reply to Jonathan Druart from comment #7)
> What is 'MANSCPT'?

Hi Jonathan

MANSCPT is "manuscript" according with official documentation <http://referencemanager.com/sites/rm/files/m/direct_export_ris.pdf>

This exist in UNIMARC and MARC21. This was coded as BOOK/GEN and is a wrong reference type.

For example in UNIMARC you can find in Record label in position 6 (type of record) the value (b = language materials, manuscript) you can see more about it at <http://archive.ifla.org/VI/3/p1996-1/uni.htm>

MARC21 in Leader position 06 (type of record) is coded "t - Manuscript language material" and textually cite "Examples include marked or corrected galley and page proofs, manuscript books, legal papers, and unpublished theses and dissertations. " <http://www.loc.gov/marc/bibliographic/bdleader.html>

If librarian coded in MARC21 Leader/06 "t" or UNIMARC 000/6 "b" he or she is coding a manuscript and unplublished book rather than a published book.

> Are the changes related to MANSCPT expected?
Not was the expected but I think this is wrong coded as BOOK in UNIMARC and GEN in MARC21. So if we started comparing at this point this is wrong anyway.

The expected was as Hugo pointed out in Description
>Catalogue a record with LDR 6 and 7 'ab' --> this is an article
>check the record type is article in koha leader

But for this we need to see more subfields than leader 06 and leader 07. See "In Analytics" at <https://www.oclc.org/bibformats/en/specialcataloging.html#CHDBIIAF>

MARC21
Journal article  JOUR
Leader/06 a
Leader/07 a  not b see examples at <https://www.oclc.org/bibformats/en/fixedfield/blvl.html>
773$7/02  a
773$7/03  s

UNIMARC
Journal article  JOUR
Leader/06 a
Leader/07 a
461

But this needs more work, and if we include journal article we can include more reference types as annotated in comment 4 and I think (IMHO) this could be treat it in another bug.


Regards
Comment 9 Marcel de Rooy 2016-10-14 08:25:59 UTC
Created attachment 56476 [details] [review]
Bug 16245: RIS export file type incorrect

When exporting lists to a RIS file, the TY field of the RIS file is always
'book' even if the item being exported is a journal or serial.
Since unimarc and marc21 is coded in Koha, just a few references types are
included. This needs a much better code.

This bug changes only TY BOOK result. Other resources needs a case by case
basis.

To test:
-In advanced search limit by itemtype (books, music, continue resources, etc).
-Search and save result in Cart
-Export to RIS Format and notice about the new TY change with all materials
-Export record in bib record details page in OPAC and Intranet.
-Confirm that works as expected.
-Bonus test:
-Try to import the  RIS file in  http://www.myendnoteweb.com if you
have account or create one.
NOTE: According with previous comment, some values has changed.

Sponsored-by: Universidad de El Salvador
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 10 Marcel de Rooy 2016-10-14 08:26:04 UTC
Created attachment 56477 [details] [review]
Bug 16245: [QA Follow-up] Replace typeofrecord2 by biblevel

Position 7 of the leader is better referred to as biblevel.
(Pos 6 is record type.)
At the same time replacing some regex's by simple eq's.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
It seems that bib level in the UNIMARC leader only allows a subset of
the possible combinations for MARC21. This does not have bad side-effects
however in the new code that tests bib level to determine the type tag.
Comment 11 Marcel de Rooy 2016-10-14 08:28:51 UTC
Unfortunately, we do not have any unit tests for Ris.pm.
This patch set could have introduced one?
But I am not blocking this for that reason now.
Looking thru Ris.pm, I would love to see the STDOUT manipulation removed.

Instead of failing it for the typeofrecord2 variable, added a follow-up.

Passed QA
Comment 12 Kyle M Hall 2016-10-21 14:55:49 UTC
Pushed to master for 16.11, thanks Hector, Marcel!
Comment 13 Frédéric Demians 2016-10-22 13:07:00 UTC
Pushed in 16.05. Will be in 16.05.05.
Comment 14 Julian Maurice 2016-10-24 10:48:54 UTC
Pushed in 3.22.x, will be in 3.22.12