Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool
Summary: Add ability for plugins to convert arbitrary files to MARC from record stagin...
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: master
Hardware: All All
: P5 - low new feature (vote)
Assignee: Kyle M Hall
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
: 9964 (view as bug list)
Depends on:
Blocks: 14148
  Show dependency treegraph
 
Reported: 2014-06-12 17:35 UTC by Kyle M Hall
Modified: 2017-09-06 16:53 UTC (History)
8 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Medium patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool (12.71 KB, patch)
2014-06-12 17:42 UTC, Kyle M Hall
Details | Diff | Splinter Review
Test text file that will be transformed into MARC (65 bytes, text/plain)
2014-06-12 17:42 UTC, Kyle M Hall
Details
Kitchen-Sink Plugin v2.00 (70.99 KB, application/zip)
2014-06-12 17:43 UTC, Kyle M Hall
Details
Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool (12.73 KB, patch)
2014-09-17 13:50 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool (12.78 KB, patch)
2015-01-23 02:26 UTC, Aleisha Amohia
Details | Diff | Splinter Review
[PASSED QA] Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool (12.88 KB, patch)
2015-03-04 13:42 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle M Hall 2014-06-12 17:35:50 UTC
Many libraries would like to be able to import various types of files as MARC records ( citations, csv files, etc ). We can add a new function to the plugins system to allow that kind of behavior at a very custom level.
Comment 1 Kyle M Hall 2014-06-12 17:42:01 UTC Comment hidden (obsolete)
Comment 2 Kyle M Hall 2014-06-12 17:42:46 UTC
Created attachment 28826 [details]
Test text file that will be transformed into MARC
Comment 3 Kyle M Hall 2014-06-12 17:43:45 UTC
Created attachment 28827 [details]
Kitchen-Sink Plugin v2.00
Comment 4 Kyle M Hall 2014-09-17 13:50:30 UTC Comment hidden (obsolete)
Comment 5 verolencinas 2014-10-11 01:35:09 UTC
I have tried to test this, but in point 6) of the test plan

6) After uploading, you should see a new area "Transform file to MARC:",
   select "Example Kitchen-Sink Plugin" from the pulldown menu

there is no new area "Transform file to MARC:".

tested with Koha 3.16.03.000. (Test-Koha is bilingual english-spanish, and test was made with both, english and spanish, interfaces).
Comment 6 Kyle M Hall 2014-10-20 14:11:06 UTC
(In reply to verolencinas from comment #5)
> I have tried to test this, but in point 6) of the test plan
> 
> 6) After uploading, you should see a new area "Transform file to MARC:",
>    select "Example Kitchen-Sink Plugin" from the pulldown menu
> 
> there is no new area "Transform file to MARC:".
> 
> tested with Koha 3.16.03.000. (Test-Koha is bilingual english-spanish, and
> test was made with both, english and spanish, interfaces).

You should be testing this patch on master.

Here is a demo video as well: http://screencast.com/t/Q4xtMiQpggN
Comment 7 Aleisha Amohia 2015-01-23 02:26:21 UTC Comment hidden (obsolete)
Comment 8 Katrin Fischer 2015-03-04 13:42:11 UTC
Created attachment 36478 [details] [review]
[PASSED QA] Bug 12412 - Add ability for plugins to convert arbitrary files to MARC from record staging tool

Many libraries would like to be able to import various types of files as
MARC records ( citations, csv files, etc ). We can add a new function to
the plugins system to allow that kind of behavior at a very custom
level.

Test Plan:
1) Ensure you have plugins enabled and configured correctly
2) Installed the attached version 2.00 of the Kitchen Sink plugin
3) Download the attached text file
4) Browse to "Stage MARC records for import"
5) Select the downloaded text file for staging
6) After uploading, you should see a new area "Transform file to MARC:",
   select "Example Kitchen-Sink Plugin" from the pulldown menu
7) Click 'Stage for import"
8) Click 'Manage staged records"
9) You should now see two new MARC records!

Signed-off-by: Aleisha <aleishaamohia@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described - interesting new feature.
Comment 9 Katrin Fischer 2015-03-04 13:48:35 UTC
Wondering if we should update this to be a 'new feature'?
Comment 10 Kyle M Hall 2015-04-10 12:07:08 UTC
(In reply to Katrin Fischer from comment #9)
> Wondering if we should update this to be a 'new feature'?

I would certainly consider this to be a new feature!
Comment 11 Tomás Cohen Arazi 2015-05-04 18:58:38 UTC
Feature pushed to master.

Thanks Kyle!
Comment 12 Dobrica Pavlinusic 2015-05-05 12:17:40 UTC
This bug will break Koha installation if there is no enable_plugins in koha-conf.xml with following error:

Can't call method "GetPlugins" on an undefined value at /srv/koha_ffzg/tools/stage-marc-import.pl line 206, <DATA> line 603.

This is not ideal, since it will break old installations.

Problem is line:

Koha::Plugins->new()->GetPlugins('to_marc');

where Koha::Plugins->new() will return undef is plugins are not enabled.

It seems we also have same syntax in plugins/plugins-home.pl so it should probably be fixed also.
Comment 13 Dobrica Pavlinusic 2015-05-05 12:22:03 UTC
It's a little bit worse: plugins had to be enabled. Even adding

 <enable_plugins>0</enable_plugins>

to koha-conf.xml won't help.
Comment 14 Pete Edwards 2015-07-27 12:37:41 UTC
This is a really useful facility and I am using it import directly from Excel spreadsheets (XLS binary format). There is a problem if I try and re-import the same file. I get the impression that the import tries to reload the staged file but in this case it doesn't include the transformation step so I get an error:

stage-marc-import.pl: binmode() on closed filehandle STDOUT at /usr/share/koha/lib/C4/Templates.pm line 120., referer: http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl
stage-marc-import.pl: Filehandle STDOUT reopened as FH only for input at /usr/lib/perl5/Template/Provider.pm line 964., referer: http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl

Have I understood the issue correctly and does the staging facility need to remember the transformation too?
Comment 15 Pete Edwards 2015-07-27 14:45:03 UTC
(In reply to Pete Edwards from comment #14)
> This is a really useful facility and I am using it import directly from
> Excel spreadsheets (XLS binary format). There is a problem if I try and
> re-import the same file. I get the impression that the import tries to
> reload the staged file but in this case it doesn't include the
> transformation step so I get an error:
> 
> stage-marc-import.pl: binmode() on closed filehandle STDOUT at
> /usr/share/koha/lib/C4/Templates.pm line 120., referer:
> http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl
> stage-marc-import.pl: Filehandle STDOUT reopened as FH only for input at
> /usr/lib/perl5/Template/Provider.pm line 964., referer:
> http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl
> 
> Have I understood the issue correctly and does the staging facility need to
> remember the transformation too?

Sorry - this was a red herring. I had another error in my plugin meaning I couldn't load the file. However, I still get the following error from somewhere when I import a file:
stage-marc-import.pl: binmode() on closed filehandle STDOUT at /usr/share/koha/lib/C4/Templates.pm line 120
I don't know what causes this. Has anyone seen similar?
Comment 16 Tomás Cohen Arazi 2015-07-27 17:13:16 UTC
(In reply to Pete Edwards from comment #15)
> (In reply to Pete Edwards from comment #14)
> > This is a really useful facility and I am using it import directly from
> > Excel spreadsheets (XLS binary format). There is a problem if I try and
> > re-import the same file. I get the impression that the import tries to
> > reload the staged file but in this case it doesn't include the
> > transformation step so I get an error:
> > 
> > stage-marc-import.pl: binmode() on closed filehandle STDOUT at
> > /usr/share/koha/lib/C4/Templates.pm line 120., referer:
> > http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl
> > stage-marc-import.pl: Filehandle STDOUT reopened as FH only for input at
> > /usr/lib/perl5/Template/Provider.pm line 964., referer:
> > http://library-intra.edwards4.co.uk/cgi-bin/koha/tools/stage-marc-import.pl
> > 
> > Have I understood the issue correctly and does the staging facility need to
> > remember the transformation too?
> 
> Sorry - this was a red herring. I had another error in my plugin meaning I
> couldn't load the file. However, I still get the following error from
> somewhere when I import a file:
> stage-marc-import.pl: binmode() on closed filehandle STDOUT at
> /usr/share/koha/lib/C4/Templates.pm line 120
> I don't know what causes this. Has anyone seen similar?

I think you should either:

1) send an email to the devel list
2) fill a bug for what you are seeing, and check with the devs taking care of it.

You should probably do (1) and then (depending on the results) do (2).
Comment 17 Marcel de Rooy 2017-08-07 11:02:19 UTC
Bug 10407 made this rather invisible feature disappear again. Bug 19049 should recover that.
It would have been better (looking back) to put some more pointers in the code about this arbitrary format. You should not pass excel files in a variable called $marcrecords..
Comment 18 Jonathan Druart 2017-09-06 16:53:12 UTC
*** Bug 9964 has been marked as a duplicate of this bug. ***