Bug 13642 - Adding new features for Dublin Core metadata
Summary: Adding new features for Dublin Core metadata
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: OPAC (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Héctor Eduardo Castro Avalos
QA Contact: Testopia
URL:
Keywords:
Depends on: 15162 15169 15170
Blocks:
  Show dependency treegraph
 
Reported: 2015-01-29 20:29 UTC by Héctor Eduardo Castro Avalos
Modified: 2017-12-07 22:16 UTC (History)
10 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 13642 [ENH] Adding new features for Dublin Core metadata #1/9 (33.20 KB, patch)
2015-01-30 19:47 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #2/9 (5.69 KB, patch)
2015-01-30 19:48 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #3/9 (3.17 KB, patch)
2015-01-30 19:50 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #4/9 (4.04 KB, patch)
2015-01-30 19:51 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #5/9 (5.49 KB, patch)
2015-01-30 19:52 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #6/9 (2.46 KB, patch)
2015-01-30 19:53 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #7/9 (7.68 KB, patch)
2015-01-30 19:53 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #8/9 (1.80 KB, patch)
2015-01-30 19:55 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #9/9 (2.78 KB, patch)
2015-01-30 19:56 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (1.56 MB, patch)
2015-02-04 15:22 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (1.56 MB, patch)
2015-02-16 21:43 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (1.56 MB, patch)
2015-02-17 16:43 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (1.56 MB, patch)
2015-02-17 16:56 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (8.67 KB, patch)
2015-02-17 17:19 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug-13642-ENH-Adding-new-features-for-Dublin-Core-me #1/2 (1.56 MB, patch)
2015-02-18 18:10 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug-13642-ENH-Adding-new-features-for-Dublin-Core-me #2/2 (8.68 KB, patch)
2015-02-18 18:12 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
[Signed-off] Bug 13642 [ENH] 1/2 Adding new features for Dublin Core metadata (1.56 MB, patch)
2015-02-18 20:57 UTC, Marc Véron
Details | Diff | Splinter Review
[Signed-off] Bug 13642 [ENH] 2/2 Adding new features for Dublin Core metadata (8.88 KB, patch)
2015-02-18 21:01 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (64.84 KB, patch)
2015-02-24 21:48 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
[Signed-off]Bug 13642 - Adding new features for Dublin Core metadata (65.16 KB, patch)
2015-02-25 05:33 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #2 (23.36 KB, patch)
2015-03-05 17:54 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #1 (65.15 KB, patch)
2015-03-05 18:08 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata #2 (23.36 KB, patch)
2015-03-05 18:10 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (70.90 KB, patch)
2015-03-05 21:54 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
[Signed-off] Bug 13642 [ENH] Adding new features for Dublin Core metadata (71.13 KB, patch)
2015-03-06 15:21 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (73.45 KB, patch)
2015-04-20 21:19 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (72.65 KB, patch)
2015-05-04 15:28 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (71.88 KB, patch)
2015-05-28 00:23 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
[Signed-off] Bug 13642 [ENH] Adding new features for Dublin Core metadata (72.08 KB, patch)
2015-05-29 08:02 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (71.32 KB, patch)
2015-07-03 22:10 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (157.78 KB, patch)
2015-07-03 22:35 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (159.22 KB, patch)
2015-09-01 14:42 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (159.13 KB, patch)
2015-10-09 20:57 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (159.26 KB, patch)
2015-10-09 21:41 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 13642 - Unit Test for Koha::DublinCore.pm (15.47 KB, patch)
2015-10-16 21:38 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (159.27 KB, patch)
2015-10-17 17:02 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Unit Test for Koha::Crosswalk::DublinCore.pm (15.58 KB, patch)
2015-10-17 17:11 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642: (qa follow-up) spelling + pod (1.63 KB, patch)
2015-10-19 10:27 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 13642: (following qa) Fixing according with QA comments (101.72 KB, patch)
2015-10-19 20:57 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (16.92 KB, patch)
2015-11-09 05:23 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
OPAC dialog box (14.14 KB, image/png)
2015-11-09 06:45 UTC, Frédéric Demians
Details
Bug 13642 [ENH] Adding new features for Dublin Core metadata (16.47 KB, patch)
2015-11-09 16:02 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (16.62 KB, patch)
2015-11-09 16:24 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Changes LESS opac.less > to opac.css (83.20 KB, patch)
2015-11-09 16:28 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Add 264 field to MARC21*DC.xsl (5.57 KB, patch)
2015-11-09 17:09 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 (follow-up) Adding new features for Dublin Core metadata (926 bytes, patch)
2015-11-09 17:20 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 (follow-up) Adding new features for Dublin Core metadata (5.99 KB, patch)
2015-11-09 21:02 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Add Unit test for C4::Record::marc2dcxml (4.54 KB, patch)
2015-11-09 23:12 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Add NORMARCslim2RDFDC.xsl and NORMARCslim2SRWDC.xsl (25.35 KB, patch)
2015-11-10 06:37 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 [ENH] Adding new features for Dublin Core metadata (17.69 KB, patch)
2015-11-11 09:21 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 13642 - Changes LESS opac.less > to opac.css (83.33 KB, patch)
2015-11-11 09:21 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 13642 - Add Unit test for C4::Record::marc2dcxml (4.59 KB, patch)
2015-11-11 09:21 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 13642 - Add Unit test for C4::Record::marc2dcxml (5.59 KB, patch)
2015-11-12 17:35 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Remove MARC::Crosswalk::DublinCore from Koha (4.54 KB, patch)
2015-11-12 17:45 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642 - Remove MARC::Crosswalk::DublinCore from Koha (4.64 KB, patch)
2015-11-12 20:25 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 13642: (followup) OAI-DC checked by default in OPAC modal (1.27 KB, patch)
2015-12-24 01:01 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642: (followup) OAI-DC checked by default in OPAC modal (4.26 KB, patch)
2015-12-24 01:31 UTC, Héctor Eduardo Castro Avalos
Details | Diff | Splinter Review
Bug 13642: (followup) OAI-DC checked by default in OPAC modal (4.45 KB, patch)
2016-01-12 06:39 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 13642 - Adding new features for Dublin Core metadata (17.76 KB, patch)
2016-01-13 12:29 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 13642 - Changes LESS opac.less > to opac.css (83.41 KB, patch)
2016-01-13 12:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 13642 - Add Unit test for C4::Record::marc2dcxml (5.66 KB, patch)
2016-01-13 12:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 13642 - Remove MARC::Crosswalk::DublinCore from Koha (4.71 KB, patch)
2016-01-13 12:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 13642: (followup) OAI-DC checked by default in OPAC modal (4.52 KB, patch)
2016-01-13 12:30 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Héctor Eduardo Castro Avalos 2015-01-29 20:29:57 UTC
Adding new features for Dublin Core metadata

This is an enhancement for Dublin Core metadata. When Koha export a bibliographic record to DC, makes it in XML format. This XML is not  well-formed document and do not follows the DC-XML recommendations as should be.

This enhancement gives to Koha the ability to download a file in XML or RDF format.

A new module (/C4/DublinCoreTransformer.pm) is required for this purpose. This module transform a marc record to Dublin Core following the DC Specifications: DC-XML, Simple DC-RDF, DC-RDF and as agregated OAI-DC. RDF can be exported as multiple formats like: rdfxml, rdfjson, ntriples, turtle, etc.

This patch include modification to these files:
MODULES:
/C4/Record.pm

OPAC:
/koha-tmpl/opac-tmpl/bootstrap/js/script.js
/koha-tmpl/opac-tmpl/bootstrap/en/includes/opac-datail-sidebar.inc
/koha-tmpl/opac-tmpl/bootstrap/en/includes/masthead.inc
/opac/opac-export.pl

CATALOGUE
/koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/datail.tt
/koha-tmpl/intranet-tmpl/prog/en/includes/cat-toolbar.inc
/catalogue/export.pl

NOTE: MARC::Crosswalk::DublinCore not support RDA yet.
      Not uses XSL to convert the data.
Comment 1 Héctor Eduardo Castro Avalos 2015-01-30 19:47:40 UTC Comment hidden (obsolete)
Comment 2 Héctor Eduardo Castro Avalos 2015-01-30 19:48:48 UTC Comment hidden (obsolete)
Comment 3 Héctor Eduardo Castro Avalos 2015-01-30 19:50:09 UTC Comment hidden (obsolete)
Comment 4 Héctor Eduardo Castro Avalos 2015-01-30 19:51:16 UTC Comment hidden (obsolete)
Comment 5 Héctor Eduardo Castro Avalos 2015-01-30 19:52:18 UTC Comment hidden (obsolete)
Comment 6 Héctor Eduardo Castro Avalos 2015-01-30 19:53:12 UTC Comment hidden (obsolete)
Comment 7 Héctor Eduardo Castro Avalos 2015-01-30 19:53:57 UTC Comment hidden (obsolete)
Comment 8 Héctor Eduardo Castro Avalos 2015-01-30 19:55:16 UTC Comment hidden (obsolete)
Comment 9 Héctor Eduardo Castro Avalos 2015-01-30 19:56:01 UTC Comment hidden (obsolete)
Comment 10 Héctor Eduardo Castro Avalos 2015-02-04 15:22:19 UTC Comment hidden (obsolete)
Comment 11 Marc Véron 2015-02-05 17:55:55 UTC
Exporting to Dublin Core...
Simple DC-RDF -> Can save file
DC-XML -> Can save file
OAI-DC -> Can save file

DC-RDF (Recommendend) and all of the additional selections (rdfxml or rdfjson etc) -> 
Software error:
Only alphanumerics are allowed in N-Triples bnode labels
Comment 12 Héctor Eduardo Castro Avalos 2015-02-05 22:03:19 UTC
(In reply to Marc Véron from comment #11)
> Exporting to Dublin Core...
> Simple DC-RDF -> Can save file
> DC-XML -> Can save file
> OAI-DC -> Can save file
> 
> DC-RDF (Recommendend) and all of the additional selections (rdfxml or
> rdfjson etc) -> 
> Software error:
> Only alphanumerics are allowed in N-Triples bnode labels

Hi, Marc, this error is because you need to up-to date RDF-Helper or RDF-Trine with CPAN modules.
Comment 13 Héctor Eduardo Castro Avalos 2015-02-06 17:11:00 UTC
(In reply to Héctor Eduardo Castro Avalos from comment #12)
> (In reply to Marc Véron from comment #11)
> > Exporting to Dublin Core...
> > Simple DC-RDF -> Can save file
> > DC-XML -> Can save file
> > OAI-DC -> Can save file
> > 
> > DC-RDF (Recommendend) and all of the additional selections (rdfxml or
> > rdfjson etc) -> 
> > Software error:
> > Only alphanumerics are allowed in N-Triples bnode labels
> 
> Hi, Marc, this error is because you need to up-to date RDF-Helper or
> RDF-Trine with CPAN modules.

Sorry about my previous short message, Marc. Let me ask you a question. Did you install RDF::Helper and, RDF::Trine from a .deb package?
Because the .deb packages are not up-to-date (librdf-helper-perl, librdf-trine-perl).  I forgot to explain in the commit message that is necessary to upgrade these modules. I upgraded with App::cpanminus.
Go to > More > About Koha > Perl modules to see if these modules need to be upgraded.

Regards

P.S. If another error found, please let me know
Comment 14 Marc Véron 2015-02-06 18:37:02 UTC
Hi Héctor,

Yes, I installed both from a .deb package

I was then able to update RDF::Helper to the required version using cpan, but not RDF::Trine (got a lot of dependency errors and had no luck to track them down).
Comment 15 Héctor Eduardo Castro Avalos 2015-02-06 19:19:26 UTC
(In reply to Marc Véron from comment #14)
> Hi Héctor,
> 
> Yes, I installed both from a .deb package
> 
> I was then able to update RDF::Helper to the required version using cpan,
> but not RDF::Trine (got a lot of dependency errors and had no luck to track
> them down).

Try to install libmodule-install-rdf-perl and then upgrade RDF::Trine with cpanm.

So the question is how to add this file into debian/control? Do you know something about it?
Comment 16 Katrin Fischer 2015-02-08 12:19:42 UTC
Hi Héctor,

I took a quick look at the list of files in your patch - there are a lot of permission changes on files in there, that I think were not intentional - could you check please?
Comment 17 Katrin Fischer 2015-02-08 12:21:55 UTC
Also adding Robin here, because of the new dependencies added by this patch. All dependencies needed for Koha should be available as packages later (comment#15).
Comment 18 Robin Sheat 2015-02-08 23:16:20 UTC
(In reply to Katrin Fischer from comment #17)
> Also adding Robin here, because of the new dependencies added by this patch.
> All dependencies needed for Koha should be available as packages later
> (comment#15).

Packages need to be created for:

HTML::Entities
XML::Entities

Please verify that the versions specified for:

RDF::Trine
RDF::Helper

are actually the real minimum required, as they are not in any released debian version as they are specified, and that makes life substantially harder.

Hopefully the dependency issues noted by Marc aren't too serious, or it simply might not be possible to backport these modules to squeeze.

It might be a while before I get to look into doing these (though they are added to my todo list) as I'm quite busy, if someone wants to help speed it along, try building the packages using dh-make-perl and testing them under squeeze (a pbuilder environment helps here), and noting anything special that has to be done.
Comment 19 Marc Véron 2015-02-09 08:46:57 UTC
During the weekend I finally was able to install the modules following comment #15. 
Feature seems to work OK, but I have the inpression that I messed up something in my VM during the installation of the modules. Have to go back to a previous copy of the VM.
I have not to mutch time at the moment for further testing, sorry. Maybe somebody else can try to install the modules and test?
Comment 20 Héctor Eduardo Castro Avalos 2015-02-10 20:15:15 UTC
Hi all

I Apologize, first one, because I couldn't get access to internet till today, second because I'm very newbie on Koha, git, etc., and thanks to Bernardo because he has guided me through this patch.

Katrin (comment 16). I changed the permission but was unintentionally because I couldn't install Koha and change all the permission on my kohaclone directory.
Not sure if it necessary to redo the patch with the old permissions or if the RM or QA Manager could change the permission when apply the patch.

Since long time that I'm working with the code, I forgot how to install RDF::Trine and the missed dependencies. I did a fresh installation for document all this dependencies.

List::Util      libscalar-list-utils-perl
Scalar::Util

Optional to List::Util
libscalar-util-numeric-perl
autobox::List::Util    libautobox-list-util-perl


TryCatch               libtrycatch-perl
B::Hooks::OP::Check    libb-hooks-op-check-perl
B::Hooks::OP::PPAddr   libb-hooks-op-ppaddr-perl
Scope::Upper           libscope-upper-perl
Clone                  libdata-clone-perl
Devel::Declare         libdevel-declare-perl
Parse::Method::Signatures    libparse-method-signatures-perl
MooseX::Traits         libmoosex-traits-perl
PPI                    libppi-perl

But there exist a problem with List::Util and RDF::Trine, these modules launch an error when you try to install them.

List::Util can not be installed with any method: cpanm; perl -MCPAN -e 'shell' install; dh-make-perl fail too.
So the only way to install this module is manually, i.e., download the tar.gz from search.cpan.org.

perl Makefile.PL
make
make test
make install

Then you can install cpanm RDF::Trine and the module will upgrade.

Regards

PS. I'm working with debian stable (wheezy)
Comment 21 Robin Sheat 2015-02-10 23:00:51 UTC
(In reply to Héctor Eduardo Castro Avalos from comment #20)
> Since long time that I'm working with the code, I forgot how to install
> RDF::Trine and the missed dependencies. I did a fresh installation for
> document all this dependencies.

That looks not too bad.

> But there exist a problem with List::Util and RDF::Trine, these modules
> launch an error when you try to install them.

List::Util is included in core perl, you shouldn't be installing it manually.

> So the only way to install this module is manually, i.e., download the
> tar.gz from search.cpan.org.

Why do you need to install it manually?

> Then you can install cpanm RDF::Trine and the module will upgrade.

More importantly, I need to know if the versions specified in the dependencies are actually the minimum versions. If the code can handle lower versions of those modules, then things will be a lot easier. For example, there is already a librdf-trine-perl module. Why aren't you using that?

> PS. I'm working with debian stable (wheezy)

We also need to support debian oldstable (squeeze) for the moment.

We can't have people using cpanm, if this is to go in to Koha, we need the dependencies produced too. I can do that, but I need a more information, mostly about what _really_ the minimum version numbers that are allowable are.
Comment 22 Héctor Eduardo Castro Avalos 2015-02-11 15:45:36 UTC
(In reply to Robin Sheat from comment #21)
> (In reply to Héctor Eduardo Castro Avalos from comment #20)
> > Since long time that I'm working with the code, I forgot how to install
> > RDF::Trine and the missed dependencies. I did a fresh installation for
> > document all this dependencies.
> 
> That looks not too bad.
> 
> > But there exist a problem with List::Util and RDF::Trine, these modules
> > launch an error when you try to install them.
> 
> List::Util is included in core perl, you shouldn't be installing it manually.
> 
> > So the only way to install this module is manually, i.e., download the
> > tar.gz from search.cpan.org.
> 
> Why do you need to install it manually?
> 
> > Then you can install cpanm RDF::Trine and the module will upgrade.
> 
> More importantly, I need to know if the versions specified in the
> dependencies are actually the minimum versions. If the code can handle lower
> versions of those modules, then things will be a lot easier. For example,
> there is already a librdf-trine-perl module. Why aren't you using that?
> 
> > PS. I'm working with debian stable (wheezy)
> 
> We also need to support debian oldstable (squeeze) for the moment.
> 
> We can't have people using cpanm, if this is to go in to Koha, we need the
> dependencies produced too. I can do that, but I need a more information,
> mostly about what _really_ the minimum version numbers that are allowable
> are.

Hi, Robin.

The problem is that RDF has been changing since was created. If you use librdf-trine-perl (1.0001) and librdf-helper-perl (2.01) launch an error (comment 11) when using bnode labels like 'ISO3166', the new modules from cpan didn't do it. So, for that reason I took the decision to upgrade the modules.

It is not recommended to use XML modules for RDF (like XML::LibXML). Furthermore RDF modules give the opportunity to serialize metadata in other formats.

I will follow up your advice about to change the code to accommodate with the modules provided by Debian.

I'll get in touch with you as soon as possible.

This is the list of packages on Debian:
XML::Entities, package need to be created.
HTML::Entities come in libhtml-parser-perl (3.66-1) squeeze and wheezy adhered to the debian/control
RDF::Helper come in librdf-helper-perl (2.0-1) just for wheezy.
RDF::Trine come in librdf-trine-perl (1.000-1) just for wheezy.
Comment 23 Robin Sheat 2015-02-11 21:40:52 UTC
I wouldn't go changing the code to make it work with lower versions unless that's easy to do. However, it's only one or two minor releases off of what's in debian, so if it works with that, then that'd be easier (so often, people just specify the most recent version because that's all that they tested, when it's not necessary.)

If it requires the versions you specified, that's fine too. I'll just make those packages (assuming it's possible - which it probably is.)
Comment 24 Héctor Eduardo Castro Avalos 2015-02-16 21:43:54 UTC Comment hidden (obsolete)
Comment 25 Héctor Eduardo Castro Avalos 2015-02-17 16:43:39 UTC Comment hidden (obsolete)
Comment 26 Héctor Eduardo Castro Avalos 2015-02-17 16:56:24 UTC Comment hidden (obsolete)
Comment 27 Héctor Eduardo Castro Avalos 2015-02-17 17:19:42 UTC Comment hidden (obsolete)
Comment 28 Héctor Eduardo Castro Avalos 2015-02-17 17:42:20 UTC
Hi all

I attached new patch that will work with all version of RDF::Trine and RDF::Helper especially with the version that come with debian package, even if you upgrade from CPAN.

Sorry for my lack of experience with git and git bz.

Regards.
Comment 29 Marc Véron 2015-02-18 17:16:45 UTC
Sorry, I get:

Applying: Bug 13642 [ENH] Adding new features for Dublin Core metadata
fatal: sha1 information is lacking or useless (Koha/DublinCoreTransformer.pm).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0001 Bug 13642 [ENH] Adding new features for Dublin Core metadata
Comment 30 Héctor Eduardo Castro Avalos 2015-02-18 18:00:06 UTC
Hi Marc.
I do not know why the failure. I applied the patch and I get something different:

Applying: Bug 13642 [ENH] Adding new features for Dublin Core metadata
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.

This can happen for all the mistakes I made before.

I will try to generate the patch with git format-patch origin and upload manually without using git-bz.

Regards
Comment 31 Héctor Eduardo Castro Avalos 2015-02-18 18:10:33 UTC Comment hidden (obsolete)
Comment 32 Héctor Eduardo Castro Avalos 2015-02-18 18:12:05 UTC Comment hidden (obsolete)
Comment 33 Héctor Eduardo Castro Avalos 2015-02-18 18:16:58 UTC
Hi Marc.

If this doesn't work I'll give up.

Manually test plan

Test plan
---------------
1) Apply patch[es].
2) Follow the last test plan.

Sponsored-by: Universidad de El Salvador

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Comment 34 Marc Véron 2015-02-18 20:25:41 UTC
Hi Héctor, patches apply now. Great!
Comment 35 Marc Véron 2015-02-18 20:57:38 UTC Comment hidden (obsolete)
Comment 36 Marc Véron 2015-02-18 21:01:48 UTC Comment hidden (obsolete)
Comment 37 Jonathan Druart 2015-02-19 14:28:18 UTC
Hello Héctor,
Please have a look at the patch, it adds a massive chmod on a big part (all) of the Koha code :)

 2310 files changed, 1370 insertions(+), 611 deletions(-) and 1.56Mb
Comment 38 Héctor Eduardo Castro Avalos 2015-02-20 13:39:05 UTC
Hi Jonathan

Yes, it's a big mistake.

I will fix this and send the patch again.

Regards.
Comment 39 Héctor Eduardo Castro Avalos 2015-02-24 21:48:12 UTC Comment hidden (obsolete)
Comment 40 Héctor Eduardo Castro Avalos 2015-02-24 21:55:53 UTC
Hi Marc and Jonathan

Sorry about the inconvenience.

After new Koha's cloning and installation, the patch is ready.

I fix it up the code about the chmod changes.

Please, test the patch again.

Regards.
Comment 41 Marc Véron 2015-02-25 05:33:04 UTC Comment hidden (obsolete)
Comment 42 Jonathan Druart 2015-02-25 12:49:35 UTC
(In reply to Marc Véron from comment #41)
> Created attachment 36154 [details] [review] [review]
> [Signed-off]Bug 13642 - Adding new features for Dublin Core metadata
> 
> When Koha export a bibliographic record to DC, makes it in XML format. This
> XML is not well-formed document and do not follows the DC-XML
> recommendations as should be.

How can I know if the xml is not well-formed?
I tested on http://validator.w3.org/check and got:
  This document was successfully checked as well-formed XML!
  Result: 	Passed, 2 warning(s)

> XML::Entities not found in .deb package

Maybe XML::LibXML can do the trick?

> New patch for work with librdf-helper-perl (2.0) and librdf-trine-perl
> (1.000).
> RDF::Trine and RDF::Helper not need to be upgraded. Tested in both (upgraded
> and not upgraded)

I don't have librdf-helper-perl
  E: Unable to locate package librdf-helper-perl
Comment 43 Jonathan Druart 2015-02-25 12:50:19 UTC
(In reply to Jonathan Druart from comment #42)
> > New patch for work with librdf-helper-perl (2.0) and librdf-trine-perl
> > (1.000).
> > RDF::Trine and RDF::Helper not need to be upgraded. Tested in both (upgraded
> > and not upgraded)
> 
> I don't have librdf-helper-perl
>   E: Unable to locate package librdf-helper-perl

Ha, it is in wheezy but not in jessie.
Comment 44 Robin Sheat 2015-02-26 02:36:55 UTC
(In reply to Jonathan Druart from comment #43)
> > I don't have librdf-helper-perl
> >   E: Unable to locate package librdf-helper-perl
> 
> Ha, it is in wheezy but not in jessie.

ruh roh batman, that's often a bad sign. Any idea why?
Comment 45 Jonathan Druart 2015-02-26 08:53:47 UTC
(In reply to Robin Sheat from comment #44)
> (In reply to Jonathan Druart from comment #43)
> > > I don't have librdf-helper-perl
> > >   E: Unable to locate package librdf-helper-perl
> > 
> > Ha, it is in wheezy but not in jessie.
> 
> ruh roh batman, that's often a bad sign. Any idea why?

I would say:
  https://wiki.debian.org/Teams/DebianPerlGroup/OpenTasks
=>
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=710983

"librdf-helper-perl: FTBFS with perl 5.18: test failures"
Comment 46 Héctor Eduardo Castro Avalos 2015-02-26 15:26:28 UTC
Hi Jonathan

(In reply to Jonathan Druart from comment #42)
> (In reply to Marc Véron from comment #41)
> > Created attachment 36154 [details] [review] [review] [review]
> > [Signed-off]Bug 13642 - Adding new features for Dublin Core metadata
> > 
> > When Koha export a bibliographic record to DC, makes it in XML format. This
> > XML is not well-formed document and do not follows the DC-XML
> > recommendations as should be.
> 
> How can I know if the xml is not well-formed?
> I tested on http://validator.w3.org/check and got:
>   This document was successfully checked as well-formed XML!
>   Result: 	Passed, 2 warning(s)

This is because you're not validating against the specified XSD schema. Use this one <http://www.freeformatter.com/xml-validator-xsd.html>

The old Dublin Core (XML) file is in uppercase eg. dc:Date, dc:Creator, and if you compare with the recommendation <http://dublincore.org/documents/dc-xml-guidelines/> at section 4.2 cite "Recommendation 4. The property names for the 15 DC elements should be all lower-case." eg. dc:date, dc:creator.

Also, this file use an improper xsi:schemaLocation: xsi:schemaLocation="http://example.org/myapp/ http://example.org/myapp/schema.xsd"
This XML schema does not exist. And is explained in is Section "4.3 Example - a simple DC record". The file generated with this patch deal with both.

The DC-XML file generate a not-valid file if you don't have a proper XSD schema. But if you donwload an OAI-DC this has one.

> 
> > XML::Entities not found in .deb package
> 
> Maybe XML::LibXML can do the trick?

XML::LibXML uses the library libxml. This library only uses the five basic entities (&lt, &gt, &quot, &amp, and &apos), if you want more entities, you need to declare them in a DTD Doctype file.

Record.pm in C4/ use sub _entity_encode with Unicode::Normalize (NFC function) in a foreache with a regexp for marcxml files, but got the problem of mix the five basics entities with hexadecimal entities. (eg. &gt &xFFFF)

RDF::Helper use the same five basic entities.

DublinCoreTransformer.pm in Koha/ is coded to use the entities or not.

If you think that this is a problem I can change the code to eliminate XML::Entities and use the solution of C4::Record sub _entity_encode.

> 
> > New patch for work with librdf-helper-perl (2.0) and librdf-trine-perl
> > (1.000).
> > RDF::Trine and RDF::Helper not need to be upgraded. Tested in both (upgraded
> > and not upgraded)
> 
> I don't have librdf-helper-perl
>   E: Unable to locate package librdf-helper-perl

I didn't know about isn't on Jessie Debian. I can try to fix this only using RDF::Trine this module still on Jessie <https://packages.debian.org/jessie/librdf-trine-perl>

Please, let me know what you think about my comment, and I will start to doing the changes.

Regards
Comment 47 Héctor Eduardo Castro Avalos 2015-02-26 17:43:39 UTC
This another module is a better option to use: HTML::Entities::Nubered at <https://metacpan.org/pod/HTML::Entities::Numbered>

The debian package is libhtml-entities-numbered-perl and is listed in all debian version (oldstable, stable and testing) at <https://packages.debian.org/search?searchon=names&keywords=libhtml-entities-numbered-perl>

So XML::Entities not necessary to use.

And we eliminate the cause of this headache.
Comment 48 Robin Sheat 2015-02-26 22:28:08 UTC
(In reply to Jonathan Druart from comment #45)
> I would say:
>   https://wiki.debian.org/Teams/DebianPerlGroup/OpenTasks
> =>
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=710983
> 
> "librdf-helper-perl: FTBFS with perl 5.18: test failures"

Yeah, that means a) it's probably not going to get back in any time soon, and b) I won't be able to make packages for it anyway as it's unbuildable.
Comment 49 Héctor Eduardo Castro Avalos 2015-03-05 17:54:17 UTC Comment hidden (obsolete)
Comment 50 Héctor Eduardo Castro Avalos 2015-03-05 17:57:54 UTC
Hi all

This the new patch that fix the last one.

Eliminate XML::Entities and RDF::Helper use instead HTML::Entities::Numbered and RDF::Trine.

Please, test the patch again.

Regards
Comment 51 Héctor Eduardo Castro Avalos 2015-03-05 18:08:00 UTC Comment hidden (obsolete)
Comment 52 Héctor Eduardo Castro Avalos 2015-03-05 18:10:13 UTC Comment hidden (obsolete)
Comment 53 Héctor Eduardo Castro Avalos 2015-03-05 18:16:36 UTC
I'm sorry, the applying fail. I'll try to reformat the patch as new again.
Comment 54 Héctor Eduardo Castro Avalos 2015-03-05 21:54:25 UTC Comment hidden (obsolete)
Comment 55 Marc Véron 2015-03-06 15:21:07 UTC Comment hidden (obsolete)
Comment 56 Katrin Fischer 2015-04-06 22:18:53 UTC
Bit confused about the dependencies here - can you give me some pointers?

I got libhtml-entities-numbered-perl installed.
But I am still missing RDF::Trine. 
Do I need to install from CPAN? 
Can it be packaged?
Comment 57 Héctor Eduardo Castro Avalos 2015-04-06 22:50:25 UTC
(In reply to Katrin Fischer from comment #56)
> Bit confused about the dependencies here - can you give me some pointers?
> 
> I got libhtml-entities-numbered-perl installed.
> But I am still missing RDF::Trine. 
> Do I need to install from CPAN? 
> Can it be packaged?

Hi Katrin

The package is librdf-trine-perl but this package exists only in Debian, stable, testing and unstable <https://packages.debian.org/search?searchon=names&keywords=librdf-trine-perl> not in oldstable (squeeze).

I don't know if this package can be made in oldstable, maybe Robin or some one else can help us.

Another package that is needed is libhtml-parser-perl.


Regards
Comment 58 Robin Sheat 2015-04-07 03:25:14 UTC
Just running through the dependencies:
* HTML::Entities provided by libhtml-parser-perl. 3.66 is in squeeze, the Koha dependencies file specifies 3.69. Does this patch require version 3.69, or will it work with 3.66?
* HTML::Entities::Numbered is fine.
* RDF::Trine is in as librdf-trine-perl from wheezy and up. In Ubuntu 12.04, only version 0.138 though, and we require 1.000+. I'll have to backport this anyway for squeeze...

...and this won't really work...

                                 Depends: libalgorithm-combinatorics-perl but it is not installable
                                 Depends: libmoosex-arrayref-perl but it is not installable
                                 Depends: libxml-commonns-perl but it is not installable
                                 Depends: libxml-namespace-perl but it is not installable
                                 Depends: libtrycatch-perl but it is not installable
                                 Depends: librdf-query-perl but it is not installable
                                 Depends: librdf-rdfa-parser-perl but it is not installable
                                 Depends: librdf-trine-node-literal-xml-perl but it is not installable

I think there are too many dependencies not in squeeze to make it reasonable to dive down that rabbit hole. However, I really really want to retire support for squeeze (and Ubuntu 12.04) anyway, so I'm going to use this as a chance to start that discussion.
Comment 59 Katrin Fischer 2015-04-07 06:34:58 UTC
Ok, one step further:  sudo apt-get install librdf-trine-perl :)

As usual I will start with the automated tests. In my test run, I notice that 2 tests in t/db_dependent/Records.t failing:

#   Failed test 'testing marc2dcxml'
#   at t/db_dependent/Record.t line 59.
#          got: undef
#     expected: '<?xml version="1.0" encoding="UTF-8"?>
# <metadata
#   xmlns="http://example.org/myapp/"
#   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#   xsi:schemaLocation="http://example.org/myapp/ http://example.org/myapp/schema.xsd"
#   xmlns:dc="http://purl.org/dc/elements/1.1/"
#   xmlns:dcterms="http://purl.org/dc/terms/">
# </metadata>'

#   Failed test 'testing marcQualified'
#   at t/db_dependent/Record.t line 71.
#          got: undef
#     expected: '<?xml version="1.0" encoding="UTF-8"?>
# <metadata
#   xmlns="http://example.org/myapp/"
#   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#   xsi:schemaLocation="http://example.org/myapp/ http://example.org/myapp/schema.xsd"
#   xmlns:dc="http://purl.org/dc/elements/1.1/"
#   xmlns:dcterms="http://purl.org/dc/terms/">
# </metadata>'
# Looks like you failed 2 tests of 11.

Can you please check?

The QA script points out various uses of given-when - can you please fix by rewriting those parts?

 FAIL	C4/Record.pm
   OK	  critic
   OK	  forbidden patterns
   OK	  pod
   FAIL	  valid
		when is experimental 
		given is experimental 

 FAIL	Koha/DublinCoreTransformer.pm
   OK	  critic
   OK	  forbidden patterns
   OK	  pod
   FAIL	  valid
		given is experimental 
		when is experimental 

 FAIL	catalogue/export.pl
   OK	  critic
   OK	  forbidden patterns
   OK	  pod
   FAIL	  valid
		when is experimental 
		given is experimental 

 FAIL	opac/opac-export.pl
   OK	  critic
   OK	  forbidden patterns
   OK	  pod
   FAIL	  valid
		given is experimental 
		when is experimental
Comment 60 Héctor Eduardo Castro Avalos 2015-04-07 22:18:36 UTC
(In reply to Katrin Fischer from comment #59)

Hi Katrin

> Ok, one step further:  sudo apt-get install librdf-trine-perl :)
> 
> As usual I will start with the automated tests. In my test run, I notice
> that 2 tests in t/db_dependent/Records.t failing:
> 

I didn't know about this kind of unit test. But fails because Record.t is coded for the previous output which i've changed in this new patch. As shown below in the chunk of xml tags.

Can I change this Unit Test in Record.t to fit with this patch?

Give me some positive feedback.

> #   Failed test 'testing marc2dcxml'
> #   at t/db_dependent/Record.t line 59.
> #          got: undef
> #     expected: '<?xml version="1.0" encoding="UTF-8"?>
> # <metadata
> #   xmlns="http://example.org/myapp/"
> #   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> #   xsi:schemaLocation="http://example.org/myapp/
> http://example.org/myapp/schema.xsd"
> #   xmlns:dc="http://purl.org/dc/elements/1.1/"
> #   xmlns:dcterms="http://purl.org/dc/terms/">
> # </metadata>'
> 
> #   Failed test 'testing marcQualified'
> #   at t/db_dependent/Record.t line 71.
> #          got: undef
> #     expected: '<?xml version="1.0" encoding="UTF-8"?>
> # <metadata
> #   xmlns="http://example.org/myapp/"
> #   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> #   xsi:schemaLocation="http://example.org/myapp/
> http://example.org/myapp/schema.xsd"
> #   xmlns:dc="http://purl.org/dc/elements/1.1/"
> #   xmlns:dcterms="http://purl.org/dc/terms/">
> # </metadata>'
> # Looks like you failed 2 tests of 11.
> 
> Can you please check?
> 
> The QA script points out various uses of given-when - can you please fix by
> rewriting those parts?
> 
>  FAIL	C4/Record.pm
>    OK	  critic
>    OK	  forbidden patterns
>    OK	  pod
>    FAIL	  valid
> 		when is experimental 
> 		given is experimental 
> 
>  FAIL	Koha/DublinCoreTransformer.pm
>    OK	  critic
>    OK	  forbidden patterns
>    OK	  pod
>    FAIL	  valid
> 		given is experimental 
> 		when is experimental 
> 
>  FAIL	catalogue/export.pl
>    OK	  critic
>    OK	  forbidden patterns
>    OK	  pod
>    FAIL	  valid
> 		when is experimental 
> 		given is experimental 
> 
>  FAIL	opac/opac-export.pl
>    OK	  critic
>    OK	  forbidden patterns
>    OK	  pod
>    FAIL	  valid
> 		given is experimental 
> 		when is experimental

You're right that kind of code is experimental, I will change this.

Regards
Comment 61 Katrin Fischer 2015-04-07 22:20:56 UTC
Hi Hector,
yes, changing the unit tests should be ok if the new result is the desired one.
Comment 62 Héctor Eduardo Castro Avalos 2015-04-20 21:19:31 UTC Comment hidden (obsolete)
Comment 63 Héctor Eduardo Castro Avalos 2015-04-20 21:24:49 UTC
Greetings,
Katrin and Marc

Please review the patch again.

Thank you
Comment 64 Héctor Eduardo Castro Avalos 2015-05-04 15:28:04 UTC Comment hidden (obsolete)
Comment 65 Héctor Eduardo Castro Avalos 2015-05-04 15:33:48 UTC
New patch sent since PerlDependencies.pm changed in master branch for 3.20 and merge conflict with it. I really hope this not continue changing.

Regards
Comment 66 Katrin Fischer 2015-05-24 22:13:19 UTC
Hi Hector,

can you confirm that this is the only new dependency needed here now?

'RDF::Trine' => {
+        'usage'    => 'Core',
+        'required' => '1',
+        'min_ver'  => '1.000',
+    },
Comment 67 Héctor Eduardo Castro Avalos 2015-05-27 03:33:17 UTC
(In reply to Katrin Fischer from comment #66)
> Hi Hector,
> 
> can you confirm that this is the only new dependency needed here now?
> 
> 'RDF::Trine' => {
> +        'usage'    => 'Core',
> +        'required' => '1',
> +        'min_ver'  => '1.000',
> +    },

Hi Katrin, sorry about my lateness in get in touch with you. I had an accident in a motorcycle and my clavicle is cracked. The first several days was painful. :(

Yes, is the only dependency since the problem of packaging in deb like happened in bug related to PDF::FromHTML. I have used instead C4::Record and the subroutine _entity_encode().

This eliminate HTML::Entities and, HTML::Entities::Numbered and fit very well with Koha without having a lot of problems. The last patch that I sent, fix the last change in PerlDependencies.pm due to new PDF::FromHTML was added.

Regards
Comment 68 Marc Véron 2015-05-27 05:53:36 UTC
Sorry, I got conflicts while applying:

Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/detail.tt
Auto-merging debian/control
CONFLICT (content): Merge conflict in debian/control
Auto-merging C4/Installer/PerlDependencies.pm
CONFLICT (content): Merge conflict in C4/Installer/PerlDependencies.pm
Failed to merge in the changes.
Comment 69 Héctor Eduardo Castro Avalos 2015-05-28 00:23:18 UTC Comment hidden (obsolete)
Comment 70 Héctor Eduardo Castro Avalos 2015-05-28 00:51:52 UTC
(In reply to Marc Véron from comment #68)
> Sorry, I got conflicts while applying:
> 
> Falling back to patching base and 3-way merge...
> Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/detail.tt
> Auto-merging debian/control
> CONFLICT (content): Merge conflict in debian/control
> Auto-merging C4/Installer/PerlDependencies.pm
> CONFLICT (content): Merge conflict in C4/Installer/PerlDependencies.pm
> Failed to merge in the changes.

I fixed it up. Please Marc apply patch again. If does not work, I miss the point in debian/control about libdevel-cover-perl.
Comment 71 Marc Véron 2015-05-29 08:02:38 UTC Comment hidden (obsolete)
Comment 72 Jonathan Druart 2015-06-16 15:20:19 UTC
It's much more better now!

Still some minor issues:
1/ Could you try to add some style to the modal window? And create a label for the radio buttons.
2/
+use Modern::Perl;
+use strict;
+use warnings::register;
=> You don't need to use strict and warnings if you use Modern::Perl;
3/ Use POD to document your subroutines.
4/ Since the modal is only used from the detail page, why do you set the JS code in script.js and the modal code in masthead.inc?
5/ Choose a correct id for the modal div element (something better than myModal :))
Comment 73 Héctor Eduardo Castro Avalos 2015-07-03 22:10:09 UTC Comment hidden (obsolete)
Comment 74 Héctor Eduardo Castro Avalos 2015-07-03 22:35:20 UTC Comment hidden (obsolete)
Comment 75 Héctor Eduardo Castro Avalos 2015-07-03 22:41:48 UTC
(In reply to Jonathan Druart from comment #72)
> It's much more better now!
> 
> Still some minor issues:
> 1/ Could you try to add some style to the modal window? And create a label
> for the radio buttons.
> 2/
> +use Modern::Perl;
> +use strict;
> +use warnings::register;
> => You don't need to use strict and warnings if you use Modern::Perl;
> 3/ Use POD to document your subroutines.
> 4/ Since the modal is only used from the detail page, why do you set the JS
> code in script.js and the modal code in masthead.inc?
> 5/ Choose a correct id for the modal div element (something better than
> myModal :))

Hi Jonathan

All changes done according with your comment. Review the patch to see if it's Ok!

Regards
Comment 76 Héctor Eduardo Castro Avalos 2015-09-01 14:42:20 UTC Comment hidden (obsolete)
Comment 77 Héctor Eduardo Castro Avalos 2015-09-01 14:47:16 UTC
(In reply to Héctor Eduardo Castro Avalos from comment #76)
> Created attachment 42168 [details] [review] [review]
> Bug 13642 [ENH] Adding new features for Dublin Core metadata

New attach since CONFLICT MERGE
Comment 78 Héctor Eduardo Castro Avalos 2015-10-09 20:57:24 UTC Comment hidden (obsolete)
Comment 79 Marc Véron 2015-10-09 21:41:52 UTC Comment hidden (obsolete)
Comment 80 Jonathan Druart 2015-10-15 09:13:56 UTC
Could you please provide tests for the new package?
Comment 81 Jonathan Druart 2015-10-15 09:52:24 UTC
No need to ask for a signoff for the tests.
Comment 82 Héctor Eduardo Castro Avalos 2015-10-16 21:38:55 UTC Comment hidden (obsolete)
Comment 83 Héctor Eduardo Castro Avalos 2015-10-16 21:41:00 UTC
(In reply to Jonathan Druart from comment #81)
> No need to ask for a signoff for the tests.

Hi Jonathan

Unit test added for Koha::DublinCore

Regards
Comment 84 Tomás Cohen Arazi 2015-10-16 22:21:20 UTC
I liked the Koha::Crosswalk::DublinCore package name. Can we keep it? :-D
Comment 85 Héctor Eduardo Castro Avalos 2015-10-16 23:06:34 UTC
(In reply to Tomás Cohen Arazi from comment #84)
> I liked the Koha::Crosswalk::DublinCore package name. Can we keep it? :-D

Yes, Mr. Tomás

Tomorrow morning i'll make the changes.

Regards
Comment 86 Héctor Eduardo Castro Avalos 2015-10-17 17:02:34 UTC Comment hidden (obsolete)
Comment 87 Héctor Eduardo Castro Avalos 2015-10-17 17:11:43 UTC Comment hidden (obsolete)
Comment 88 Héctor Eduardo Castro Avalos 2015-10-17 17:44:22 UTC
Hi Tomás and Jonathan

Changes Done!

I don't know if I need another sign-off or I can keep the last one by Marc.

Regards
Comment 89 Jonathan Druart 2015-10-19 09:44:27 UTC
koha-tmpl/opac-tmpl/bootstrap/css/opac.css has been updated but koha-tmpl/opac-tmpl/bootstrap/less/opac.less
Please provide the related change to this file too.
Comment 90 Jonathan Druart 2015-10-19 09:48:18 UTC
Comment on attachment 43548 [details] [review]
Bug 13642 [ENH] Adding new features for Dublin Core metadata

Review of attachment 43548 [details] [review]:
-----------------------------------------------------------------

::: debian/control
@@ -86,4 @@
>   libopenoffice-oodoc-perl,
>   libpdf-api2-perl,
>   libpdf-api2-simple-perl,
> - libpdf-fromhtml-perl,

Why did you remove it? It is used by Koha::Borrower::Discharge

@@ -99,4 @@
>   libtemplate-perl,
>   libtemplate-plugin-htmltotext-perl,
>   libtemplate-plugin-json-escape-perl,
> - libtemplate-plugin-stash-perl,

Same here.
Comment 91 Jonathan Druart 2015-10-19 09:49:39 UTC
Comment on attachment 43548 [details] [review]
Bug 13642 [ENH] Adding new features for Dublin Core metadata

Review of attachment 43548 [details] [review]:
-----------------------------------------------------------------

::: koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-detail.tt
@@ +1999,5 @@
> +        });
> +    });
> +
> +    $(document).ready(function(){
> +        $("#input-oai").click(function(){

This

@@ +2008,5 @@
> +        });
> +    });
> +
> +    $(document).ready(function(){
> +        $("#input-srw").click(function(){

and this could be merged.
Comment 92 Jonathan Druart 2015-10-19 10:19:27 UTC
Comment on attachment 43548 [details] [review]
Bug 13642 [ENH] Adding new features for Dublin Core metadata

Review of attachment 43548 [details] [review]:
-----------------------------------------------------------------

::: opac/opac-export.pl
@@ +69,5 @@
>      $format = 'bibtex';
>  }
>  elsif ($format =~ /dc/) {
> +    SWITCH:
> +        for ($recommendation) {

What are the possible values for this variable?

/^simple-dc-rdf/ => why not /^simple-dc-rdf$/ ?

I think this block could be better written with if-elsif blocks.
Comment 93 Jonathan Druart 2015-10-19 10:21:05 UTC
In the generated file I get
<dcterms:creator>158 Laslett&#xE9; St&#xE9;phanie 070</dcterms:creator>

I would be better to have the utf-8 characters.
Comment 94 Jonathan Druart 2015-10-19 10:25:36 UTC
When you click save > dublin core, "DC-RDF (Recommended)" is checked but the "Output options for RDF" block is not expanded. I think it should.

On a UNIMARC install, there is not "dcterms:title" entry, is it normal?
Comment 95 Jonathan Druart 2015-10-19 10:27:08 UTC Comment hidden (obsolete)
Comment 96 Héctor Eduardo Castro Avalos 2015-10-19 16:35:22 UTC
(In reply to Jonathan Druart from comment #94)
> When you click save > dublin core, "DC-RDF (Recommended)" is checked but the
> "Output options for RDF" block is not expanded. I think it should.
> 
> On a UNIMARC install, there is not "dcterms:title" entry, is it normal?

I Jonathan

The problem is that MARC::Crosswalk::DublinCore map only MARC21 fields. This is  because the autor of package follows the crosswalk from LC at <http://www.loc.gov/marc/marc2dc.html>, i.e., title statement is saved in different fields (MARC 21 in 245 and UNIMARC in 200).

Let me do a crosswalk for UNIMARC in another bug. This new bug must consider the MARC flauvor.
Comment 97 Héctor Eduardo Castro Avalos 2015-10-19 20:57:13 UTC Comment hidden (obsolete)
Comment 98 Héctor Eduardo Castro Avalos 2015-10-19 21:12:43 UTC
Hi Jonathan

I think the UNIMARC can be corrected of various forms. For example with XSLT Transformation, C4/Breeding.pm use something like that (C4::Breeding->_do_xslt_proc) and the xslt to transform UNIMARC2MARC21 is here <https://github.com/edsd/biblio-metadata/blob/master/UNIMARC2MARC21.xsl>

Although exist this module <https://metacpan.org/pod/MARC::Moose::Formater::UnimarcToMarc21>.

C4::Record->marc2marc deal with marc flavour, but it seems to be that it does not work or is not fully developed.

So my question is: Can I fix it in another bug or I need to fix it in this one?

Thanks Jonathan

Regards
Comment 99 Katrin Fischer 2015-10-24 23:49:12 UTC
This currently doesn't apply cleanly due to conflicts in the css files - please fix!
Comment 100 Jonathan Druart 2015-10-26 10:17:03 UTC
- You should extract the less/css changes from the mainpage.
- There is still something wrong with the radio buttons on the modal window (staff).
- export oai-dc: in opac/oai.pl we use HTTP::OAI::Metadata::OAI_DC, so now we will have 2 differents module to do the same job?


I am not familiar enough with Dublin core to know what to do with the UNIMARC export and validate it works well for other flavors.

Frédéric (cced), Is there any chance to get your opinion on this enhancement?
Comment 101 Héctor Eduardo Castro Avalos 2015-10-26 19:36:47 UTC
(In reply to Katrin Fischer from comment #99)
> This currently doesn't apply cleanly due to conflicts in the css files -
> please fix!

Okidoki. I will fix this as soon as possible.

Also I'm going to change here the status to In Discussion

Thanks Katrin
Comment 102 Héctor Eduardo Castro Avalos 2015-10-26 20:10:17 UTC
(In reply to Jonathan Druart from comment #100)

Hi Jonathan

> - You should extract the less/css changes from the mainpage.
> - There is still something wrong with the radio buttons on the modal window
> (staff).

 I'll fix this as soon as possible.

> - export oai-dc: in opac/oai.pl we use HTTP::OAI::Metadata::OAI_DC, so now
> we will have 2 differents module to do the same job?

Yep, you're right. But at least with this one I can reproduce another kind of metadata (DC-XML standard, SRW-DC, RDF serialization) starting from a marc record. Besides, this enhancement let us to work with more than simple dc (namespace for elements 1.1) otherwise works with complete dcterms namespace.

With HTTP::OAI::Metadata::OAI_DC -I think- you need to create a record from scratch.

> 
> I am not familiar enough with Dublin core to know what to do with the
> UNIMARC export and validate it works well for other flavors.

I think that this can be corrected if we do the conversion between unimarc and marc 21 flavor as i explained in comment 98.
NORMARC works like marc21, so i don't need to fix somethin here.

> Frédéric (cced), Is there any chance to get your opinion on this enhancement?


I will work with the corrections and make something to convert unimarc to marc21.

Regards
Comment 103 Frédéric Demians 2015-10-27 09:08:47 UTC
Here's what I can think of this.

There is already in Koha an inconsistency in the way DC is generated. (1) When
exporting, C4::Record::marc2dcxml() is used. (2) In OAI server, an XSLT
transformation is done using XSL file found in koha-tmpl/intranet-
tmpl/prog/en/xslt. The OAI approach seems more sensible since it allows to
have a transformation logic per MARC flavor, and even in OAI extended mode, a
transformation specific to a Koha installation.

Currently, C4::Record::marc2dcxml() delegates the semantic transformation from
MARC21 to XML DC to this module: MARC::Crosswalk::DublinCore. I don't think
this is a great idea. This is an old-unmaintained module (10 years old). And
it doesn't work for Unimarc. I haven't paid attention to that, but I can't see
how DC export could work with an Unimarc catalog. Your module
Koha::Crosswalk::DublinCore still wraps a call MARC::Crosswalk::DublinCore.

The MARC21 to Unimarc transformation on the fly isn't a good idea at all. This
would add complexity for an unreliable result. 

Why not just fixing XSL files transforming MARCXML biblios into DC and use
them in marc2dcxml()?

Side note about dependencies -- RDF::Trine have Moose as dependency. Aren't we
supposed to avoid that?
Comment 104 Katrin Fischer 2015-10-27 09:14:01 UTC
Hi Frederick, thx for your comment! I like the idea of reusing the XSLT.
Comment 105 Héctor Eduardo Castro Avalos 2015-10-27 15:35:49 UTC
(In reply to Frédéric Demians from comment #103)
> Here's what I can think of this.
> 
> There is already in Koha an inconsistency in the way DC is generated. (1)
> When
> exporting, C4::Record::marc2dcxml() is used. (2) In OAI server, an XSLT
> transformation is done using XSL file found in koha-tmpl/intranet-
> tmpl/prog/en/xslt. The OAI approach seems more sensible since it allows to
> have a transformation logic per MARC flavor, and even in OAI extended mode, a
> transformation specific to a Koha installation.
> 
> Currently, C4::Record::marc2dcxml() delegates the semantic transformation
> from
> MARC21 to XML DC to this module: MARC::Crosswalk::DublinCore. I don't think
> this is a great idea. This is an old-unmaintained module (10 years old). And
> it doesn't work for Unimarc. I haven't paid attention to that, but I can't
> see
> how DC export could work with an Unimarc catalog. Your module
> Koha::Crosswalk::DublinCore still wraps a call MARC::Crosswalk::DublinCore.
> 
> The MARC21 to Unimarc transformation on the fly isn't a good idea at all.
> This
> would add complexity for an unreliable result. 
> 
> Why not just fixing XSL files transforming MARCXML biblios into DC and use
> them in marc2dcxml()?
> 
> Side note about dependencies -- RDF::Trine have Moose as dependency. Aren't
> we
> supposed to avoid that?

Where I can find information about avoiding Moose in Koha?

Another question is: How to deal with RDF and SPARQL in Koha if we can't use Moose?

All linked data in Semantic Web point to use RDF as basis. For example, the new recommendation for DC is RDF <http://dublincore.org/documents/dc-rdf/>. BIBCO works with RDF. RDA Resgistry <http://www.rdaregistry.info/> works with RDF. RIMMF works with RDF.

With xslt transformation we can only transform to simple DC RDF (BTW, out-of-date, superseded standard <http://dublincore.org/documents/dcmes-xml/>), and what about the potential to use the complete dcterm namespace?

Regards
Comment 106 Héctor Eduardo Castro Avalos 2015-10-27 15:45:30 UTC
(In reply to Katrin Fischer from comment #104)
> Hi Frederick, thx for your comment! I like the idea of reusing the XSLT.

Let me propose a patch in this way...
Comment 107 Frédéric Demians 2015-10-27 18:13:50 UTC
> With xslt transformation we can only transform to simple DC RDF (BTW, out-
> of-date, superseded standard <http://dublincore.org/documents/dcmes-xml/>),
> and what about the potential to use the complete dcterm namespace?

Can you elaborate? I don't understand.
Comment 108 Héctor Eduardo Castro Avalos 2015-10-27 19:51:16 UTC
(In reply to Frédéric Demians from comment #107)

>Can you elaborate? I don't understand.

Have you tested the patch?

With this module (Koha::Crosswalk::DublinCore) I can support more metada than the simple 15 DC elements (Element Set Version 1.1). This Simple DC is used for OAI and SRW in xml format. But also i can reproduce this other guidelines/recommendations:

* <http://dublincore.org/documents/dc-xml-guidelines/> and example of this in <http://www.dublincoregenerator.com/generator.html>
* <http://dublincore.org/documents/dcmes-xml/>
* <http://dublincore.org/documents/dc-rdf/>

And RDF::Trine can serialize multiple rdf formats (rdf-json, rdf-xml, nquads, nquads-canonical, turtle) also give us the opportunity of storing the triples in databases; with RDF::Query we can perform queries with SPARQL
Comment 109 Frédéric Demians 2015-10-28 09:33:39 UTC
> Have you tested the patch?

Yes. My questions are not related to your patch WUI and result, but to your
implementation, your goals.

Somehwhere in the process, I take a look at the code, spend time on it, trying
to figure out how it works, how it has been done. Testing and QAing is all
about that.

I have mixed feeling, when I find duplicated code (for example). In opac/opac-
export.pl and catalog/export.pl:

SWITCH:
    for ($recommendation) {
        if (/^simple-dc-rdf/) {
            ($error,$marc) = marc2dcxml($marc, $recommendation, 0,
            1, undef, undef, $resource_url , $formats);
            $format = "dublin-core." . $formats;
            last SWITCH; }
        if (/^dc-rdf/) {
            ($error,$marc) = marc2dcxml($marc, $recommendation, 1,
            1, undef, undef, $resource_url, $formats);
            $format = "dublin-core." . $formats;
            last SWITCH; }
        if (/^dc-xml/) {
            ($error,$marc) = marc2dcxml($marc, $recommendation, $qualifier,
            1, $root_element, $xsischemalocation, undef, undef);
            $format = "dublin-core.xml";
            last SWITCH; }
        if (/^oai-dc/) {
            ($error,$marc) = marc2dcxml($marc, $recommendation);
            $format = "dublin-core.xml";
            last SWITCH; }
        if (/^srw_dc/) {
            ($error,$marc) = marc2dcxml($marc, $recommendation);
            $format = "dublin-core.xml";
            last SWITCH; }
    }


Or in Koha::Crosswalk::DublinCore.pm, line 242, you get:

if ( exists $self->{xsi_schemaLocation} ) {
    $dataxml{'xsi_schemaLocation'} = $self->{xsi_schemaLocation};
}
if ( exists $self->{xsi_schemaLocation} ) {
    $dataxml{'opt_namespace'} = $self->{opt_namespace};
}
if ( exists $self->{xsi_schemaLocation} ) {
    $dataxml{'opt_namespace_url'} = $self->{opt_namespace_url};
}

Is it valid?

> With this module (Koha::Crosswalk::DublinCore) I can support more metada
> than the simple 15 DC elements (Element Set Version 1.1). This Simple DC is
> used for OAI and SRW in xml format. But also i can reproduce this other
> guidelines/recommendations:

How will you extend the semantic of the produced metadata with
MARC::Crosswalk::Dublincore? If you want more than the 15 DC elements, doesn't
it mean that you need a mean to improve, granularize the way Koha MARC record
is transformed. The current DC exported from Koha is not that good because (1)
it doesn't work for Unimarc, (2) it doesn't allow each library to tweak the
transformation from MARC to DC (the way it's done by OAI server).

> And RDF::Trine can serialize multiple rdf formats (rdf-json, rdf-xml,
> nquads, nquads-canonical, turtle) also give us the opportunity of storing
> the triples in databases; with RDF::Query we can perform queries with SPARQL

It's far beyond the scope of this patch. Open a new bug if you want, and name
it Semantic-Koha, or Sekoa.
Comment 110 Héctor Eduardo Castro Avalos 2015-10-28 15:11:09 UTC
(In reply to Frédéric Demians from comment #109)

> Somehwhere in the process, I take a look at the code, spend time on it,
> trying
> to figure out how it works, how it has been done. Testing and QAing is all
> about that.
> 
> I have mixed feeling, when I find duplicated code (for example). In
> opac/opac-
> export.pl and catalog/export.pl:
 
> Or in Koha::Crosswalk::DublinCore.pm, line 242, you get:
> Is it valid?
 
> How will you extend the semantic of the produced metadata with
> MARC::Crosswalk::Dublincore? If you want more than the 15 DC elements,
> doesn't
> it mean that you need a mean to improve, granularize the way Koha MARC record
> is transformed. The current DC exported from Koha is not that good because
> (1)
> it doesn't work for Unimarc, (2) it doesn't allow each library to tweak the
> transformation from MARC to DC (the way it's done by OAI server).
 
> It's far beyond the scope of this patch. Open a new bug if you want, and name
> it Semantic-Koha, or Sekoa.

Yes, you're right Frédéric. I'm just a librarian trying to improve.

Let me propose another patch with your suggestion, using the xslts for transforming the marc flavors.

Regards
Comment 113 Katrin Fischer 2015-11-08 17:29:23 UTC
Hi Hector and others,

I have marked the last 2 comments 'private' as I think the links in them are spam.
Comment 114 Héctor Eduardo Castro Avalos 2015-11-08 18:33:21 UTC
(In reply to Katrin Fischer from comment #113)

> I have marked the last 2 comments 'private' as I think the links in them are
> spam.

Thanks Katrin, I did not realize it was spam
Comment 115 Héctor Eduardo Castro Avalos 2015-11-09 05:23:50 UTC Comment hidden (obsolete)
Comment 116 Héctor Eduardo Castro Avalos 2015-11-09 05:31:34 UTC
Hi Frédéric and all devs

This is the new patch using XSLTs for OAIDC, SRWDC, and RDFDC.

The bug is not complete yet! Needs Unit test, changes to opac.cs (I need to install LESS) and maybe add 264 field to XSLTs.

Frédéric, please test it and QAing and tell me how looks this wheter i'm wrong or I go in the right way.

Thanks

Regards
Comment 117 Héctor Eduardo Castro Avalos 2015-11-09 05:35:53 UTC
I forgot it, test with MARC21, UNIMARC and NORMARC, this patch takes in consideration syspref marcflavour
Comment 118 Frédéric Demians 2015-11-09 06:44:29 UTC
> Frédéric, please test it and QAing and tell me how looks this wheter i'm
> wrong or I go in the right way.

Seems good to me. I've tested: it works in intra interface, but it doesn't at OPAC (I will attach a screen capture) ; the dialog box seems mixed.
Comment 119 Frédéric Demians 2015-11-09 06:45:44 UTC Comment hidden (obsolete)
Comment 120 Frédéric Demians 2015-11-09 06:47:32 UTC
Reading the code, I'd also suggest to simplify the way you construct the XSL filename in marc2dcxml:

   my $xsl = C4::Context->config('intrahtdocs') . '/prog/en/xslt/slim2' .
             uc $format . '.xsl';
Comment 121 Héctor Eduardo Castro Avalos 2015-11-09 16:02:29 UTC Comment hidden (obsolete)
Comment 122 Héctor Eduardo Castro Avalos 2015-11-09 16:24:14 UTC Comment hidden (obsolete)
Comment 123 Héctor Eduardo Castro Avalos 2015-11-09 16:28:33 UTC Comment hidden (obsolete)
Comment 124 Frédéric Demians 2015-11-09 16:45:45 UTC
Comment on attachment 44659 [details] [review]
Bug 13642 [ENH] Adding new features for Dublin Core metadata

Review of attachment 44659 [details] [review]:
-----------------------------------------------------------------

Otherwise it works for me both pro/opac interface, with a MARC21. I will test later with an UNIMARC catalog.

::: C4/Record.pm
@@ +293,5 @@
> +            carp "\t". $warn;
> +        };
> +    } elsif ( $record =~ /^MARC::Record/ ) { # if OK makes xslt transformation
> +        my $xslt_engine = Koha::XSLT_Handler->new;
> +        if ( $format eq 'oaidc' || 'srwdc' || 'rdfdc' ) {

This test will always be true. I don't think it's what you want. You may want:

  if ( $format =~ /oaidc|srwdc|rdfdc/ ) {
Comment 125 Héctor Eduardo Castro Avalos 2015-11-09 17:09:01 UTC Comment hidden (obsolete)
Comment 126 Héctor Eduardo Castro Avalos 2015-11-09 17:20:19 UTC Comment hidden (obsolete)
Comment 127 Frédéric Demians 2015-11-09 18:04:13 UTC
It works in intra/opac. But in intra, it only works on 'Normal' detail page, it doesn't in ISBD/MARC/Item tab.
Comment 128 Frédéric Demians 2015-11-09 18:16:20 UTC
For Unimarc, it will only work for OAI-DC format. XSL for RDF-DC and SRW-DC format are missing. See bug 15162. In my opinion, it shouldn't prevent accepting this enhancement.
Comment 129 Héctor Eduardo Castro Avalos 2015-11-09 21:02:30 UTC Comment hidden (obsolete)
Comment 130 Héctor Eduardo Castro Avalos 2015-11-09 21:06:13 UTC
(In reply to Frédéric Demians from comment #128)
> For Unimarc, it will only work for OAI-DC format. XSL for RDF-DC and SRW-DC
> format are missing. See bug 15162. In my opinion, it shouldn't prevent
> accepting this enhancement.

Hi Frédéric

I will try to attach a patch for bug 15162 as soon as posible to apply on top of this one.

Regards
Comment 131 Héctor Eduardo Castro Avalos 2015-11-09 23:12:52 UTC Comment hidden (obsolete)
Comment 132 David Cook 2015-11-10 06:11:43 UTC
Not sure about the state of this patch, but I just wanted to say that I agree with Frédéric:

"It's far beyond the scope of this patch. Open a new bug if you want, and name
it Semantic-Koha, or Sekoa."

I think it would be interesting to be able to publish any record using RDF, and maybe even set up a SPARQL endpoint. I noticed that in DSpace 5.0 or 6.0 they've added support for publishing their records using RDF.

However, I think that's outside the scope of this bug. It would make sense as its own bug.
Comment 133 Héctor Eduardo Castro Avalos 2015-11-10 06:37:28 UTC Comment hidden (obsolete)
Comment 134 Frédéric Demians 2015-11-10 07:38:01 UTC
(In reply to David Cook from comment #132)

> I think it would be interesting to be able to publish any record using RDF,
> and maybe even set up a SPARQL endpoint. I noticed that in DSpace 5.0 or 6.0
> they've added support for publishing their records using RDF.

This is a huge subject. A very interesting and promising one. I'd be pleased to be included in a discussion on this matter, whatever the form it takes: bug umbrella, task force...
Comment 135 Frédéric Demians 2015-11-10 07:41:55 UTC
Héctor, would you allow me to simplify this bug? I propose to just keep 3 patches attached to this one (and sign them): the enhancement, the UT & the less to css conversion. And I would open two new patches: (1) 264 field, (2) NORMARC new XSL. It won't touch the patches themselves nor your credit.
Comment 136 Héctor Eduardo Castro Avalos 2015-11-10 21:20:35 UTC
(In reply to Frédéric Demians from comment #135)
> Héctor, would you allow me to simplify this bug? I propose to just keep 3
> patches attached to this one (and sign them): the enhancement, the UT & the
> less to css conversion. And I would open two new patches: (1) 264 field, (2)
> NORMARC new XSL. It won't touch the patches themselves nor your credit.

Sorry for my late reply

Okidoki Frédéric, do the necessary and the more adequate with this bug.

Regards
Comment 137 David Cook 2015-11-11 00:08:41 UTC
(In reply to Frédéric Demians from comment #134)
> This is a huge subject. A very interesting and promising one. I'd be pleased
> to be included in a discussion on this matter, whatever the form it takes:
> bug umbrella, task force...

Yeah, I'd like to be involved as well. I think Magnus Enger has already started doing prototype work on this with SemantiKoha, so it'll be interesting.

Certainly something to discuss with the whole group, I think.
Comment 138 Héctor Eduardo Castro Avalos 2015-11-11 02:39:58 UTC
(In reply to David Cook from comment #137)
 
> Yeah, I'd like to be involved as well. I think Magnus Enger has already
> started doing prototype work on this with SemantiKoha, so it'll be
> interesting.
> 
> Certainly something to discuss with the whole group, I think.

Hi David

Where is placed Magnus' works? I'm really interested in this kind of subjects too. Another think is -as explain me Bernardo- the plack environment needs to be complete to use modules with Moose support, this is because some packages like RDF::Trine and RDF::Query need it.
Comment 139 Frédéric Demians 2015-11-11 09:21:33 UTC Comment hidden (obsolete)
Comment 140 Frédéric Demians 2015-11-11 09:21:49 UTC Comment hidden (obsolete)
Comment 141 Frédéric Demians 2015-11-11 09:21:58 UTC Comment hidden (obsolete)
Comment 142 Héctor Eduardo Castro Avalos 2015-11-12 17:35:39 UTC Comment hidden (obsolete)
Comment 143 Héctor Eduardo Castro Avalos 2015-11-12 17:45:30 UTC Comment hidden (obsolete)
Comment 144 Héctor Eduardo Castro Avalos 2015-11-12 17:51:04 UTC
Hi Frédéric

Please sign-off the last two patches, I found another Unit Test (t/db_dependent/Record/Record.t) that affect the marc2dxml method; and finally remove the MARC::Crosswalk::DublinCore module (libmarc-crosswalk-dublincore-perl)

Thanks in advance
Comment 145 Frédéric Demians 2015-11-12 20:25:13 UTC Comment hidden (obsolete)
Comment 146 Héctor Eduardo Castro Avalos 2015-11-12 20:43:39 UTC
(In reply to Frédéric Demians from comment #145)

> Signed-off-by: Frederic Demians <f.demians@tamil.fr>
>   Ripping out any mention to MARC::Crosswalk::DublinCore can't hurt.

Getting rid really can't hurt, IMHO

git grep "marc2dcxml"
C4/Record.pm:  &marc2dcxml
C4/Record.pm:=head2 marc2dcxml - Convert from ISO-2709 to Dublin Core
C4/Record.pm:    my dcxml = marc2dcxml ($marc, $xml, $biblionumber, $format);
C4/Record.pm:    my dcxml = marc2dcxml (undef, undef, 1, "oaidc");
C4/Record.pm:sub marc2dcxml {
catalogue/export.pl:                $marc = marc2dcxml(undef, undef, $biblionum
opac/opac-export.pl:    $marc = marc2dcxml(undef, undef, $biblionumber, $format
t/db_dependent/Record.t:my $marcsrwdc=marc2dcxml( $marc, undef, undef, "srwdc" 
t/db_dependent/Record.t:my $marcoaidc=marc2dcxml( $marc, undef, undef, "oaidc" 
t/db_dependent/Record.t:my $marcrdfdc=marc2dcxml( $marc, undef, undef, "rdfdc" 
t/db_dependent/Record/Record.t:ok (eval{marc2dcxml($marc21_utf8, undef, undef,

All files affected has been changed to the new marc2dcxml method.
Comment 147 David Cook 2015-11-12 23:18:22 UTC
(In reply to Héctor Eduardo Castro Avalos from comment #138)
> (In reply to David Cook from comment #137)
>  
> > Yeah, I'd like to be involved as well. I think Magnus Enger has already
> > started doing prototype work on this with SemantiKoha, so it'll be
> > interesting.
> > 
> > Certainly something to discuss with the whole group, I think.
> 
> Hi David
> 
> Where is placed Magnus' works? I'm really interested in this kind of
> subjects too. Another think is -as explain me Bernardo- the plack
> environment needs to be complete to use modules with Moose support, this is
> because some packages like RDF::Trine and RDF::Query need it.

Hi Héctor,

The blog post about it is here http://libriotech.no/blogs/semantikoha and it includes links to the source code. It's been a few years since Magnus has worked on it, I think, but I think he might be interested in reviving it.
Comment 148 Héctor Eduardo Castro Avalos 2015-12-10 15:54:53 UTC
Still applies
Comment 149 Jonathan Druart 2015-12-23 10:46:21 UTC
Héctor, I have just found a minor issue:
At the OPAC, there is no radio buttons selected by default, so if you click on export, you get a iso2709.
Comment 150 Héctor Eduardo Castro Avalos 2015-12-24 01:01:52 UTC Comment hidden (obsolete)
Comment 151 Héctor Eduardo Castro Avalos 2015-12-24 01:18:21 UTC
(In reply to Jonathan Druart from comment #149)
> Héctor, I have just found a minor issue:
> At the OPAC, there is no radio buttons selected by default, so if you click
> on export, you get a iso2709.

I found two issues more in OPAC, the modal doesn't display in MARC view and ISBD view. I will fix this in the last follow-up.

Regards
Comment 152 Héctor Eduardo Castro Avalos 2015-12-24 01:31:07 UTC Comment hidden (obsolete)
Comment 153 Héctor Eduardo Castro Avalos 2015-12-24 01:37:18 UTC
I wish all of you a Merry Christmas and a Happy New Year

Regards
Comment 154 Frédéric Demians 2016-01-12 06:39:07 UTC Comment hidden (obsolete)
Comment 155 Jonathan Druart 2016-01-13 12:29:59 UTC
Created attachment 46584 [details] [review]
Bug 13642 - Adding new features for Dublin Core metadata

When Koha export a bibliographic record to DC, makes it in XML format.
This XML not follows the DC-XML recommendations as should be: elements
in uppercase eg. dc:Date, dc:Creator, section 4.2, recommendation 4
explain that "The property names for the 15 DC elements should be all
lower-case." eg. dc:date, dc:creator" and section "4.3 Example - a
simple DC record", xsi:schemaLocation="http://example.org/myapp/
http://example.org/myapp/schema.xsd" schema does not exist.

NOTE: This new feature implement the XSLT transformation for OAI-DC,
SRW-DC and RDF-DC

Test plan
---------------
1) Download Dublin Core file from record detail page. Open up the file,
   and make sure that the document not follows the DC-XML
   recommendations as should be.
2) Apply patch.
3) Go to whichever bib record in OPAC or staff and click on Save >
   Dublin Core.  A modal will display, prove all options.
4) Change the system preference 'Opac ExportOptions' by enabling and
   disabling Dublin Core and try to download a record.
5) Try several bibliographic records in any format (book, magazine, DVD,
   etc.) to confirm that properly exported.
6) Test with all marc flavours.

Sponsored-by: Universidad de El Salvador

Signed-off-by: Frederic Demians <f.demians@tamil.fr>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 156 Jonathan Druart 2016-01-13 12:30:07 UTC
Created attachment 46585 [details] [review]
Bug 13642 - Changes LESS opac.less > to opac.css

Changes LESS opac.less > to opac.css

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 157 Jonathan Druart 2016-01-13 12:30:14 UTC
Created attachment 46586 [details] [review]
Bug 13642 - Add Unit test for C4::Record::marc2dcxml

Add Unit test for C4::Record::marc2dcxml

To test:
prove t/db_dependent/Record.t
prove t/db_dependent/Record/Record.t

Signed-off-by: Frederic Demians <f.demians@tamil.fr>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 158 Jonathan Druart 2016-01-13 12:30:19 UTC
Created attachment 46587 [details] [review]
Bug 13642 - Remove MARC::Crosswalk::DublinCore from Koha

To test:

1) git grep "MARC::Crosswalk::DublinCore"

C4/Installer/PerlDependencies.pm:    'MARC::Crosswalk::DublinCore' => {
INSTALL.fedora7:MARC::Crosswalk::DublinCore

2) git grep "libmarc-crosswalk-dublincore-perl"

debian/control: libmarc-crosswalk-dublincore-perl,
debian/control: libmarc-crosswalk-dublincore-perl,
install_misc/debian.packages:libmarc-crosswalk-dublincore-perl install
install_misc/ubuntu.10.04.packages:libmarc-crosswalk-dublincore-perl    install
install_misc/ubuntu.12.04.packages:libmarc-crosswalk-dublincore-perl    install
install_misc/ubuntu.packages:libmarc-crosswalk-dublincore-perl  install

3) Repeat step 1 and 2

Signed-off-by: Frederic Demians <f.demians@tamil.fr>
  Ripping out any mention to MARC::Crosswalk::DublinCore can't hurt.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 159 Jonathan Druart 2016-01-13 12:30:24 UTC
Created attachment 46588 [details] [review]
Bug 13642: (followup) OAI-DC checked by default in OPAC modal

-Fixing default option checked for OAI-DC in OPAC modal
-Move OPAC modal to opac-detail-sidebar.inc to display in MARC view
 and ISBD view
This follow-up fix the QA comment in comment 149

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
  Above issue fixed & usable also in MARC/ISBD view

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 160 Jonathan Druart 2016-01-13 12:30:59 UTC
This patch set is blocked by bug 15169 which is waiting for a signoff.
Comment 161 Jonathan Druart 2016-01-13 12:31:30 UTC
Great work Héctor and Frédéric!
Comment 162 Brendan Gallagher 2016-01-27 06:24:11 UTC
Pushed to Master - Should be in the May 2016 release.  Thanks!
Comment 163 David Cook 2017-05-11 05:42:29 UTC
FYI, Magnus and I are doing a RDF project with Stockholm University Library, and we've been looking at RDF::Trine again.

I've found a bug in the most recent code (version 1.016), and sent a pull request, and the module author said that he's hoping to review it and get that merged into master in the next few days. 

But it means that we do want to be using the newest version of RDF::Trine. I'm going to send an email out in a moment and copy in Mirko as the latest package maintainer.