Delivered-To: jcamins@cpbibliography.com Received: by 10.42.75.136 with SMTP id a8cs53796ick; Mon, 18 Apr 2011 05:33:49 -0700 (PDT) Received: by 10.216.142.85 with SMTP id h63mr4631718wej.84.1303130022766; Mon, 18 Apr 2011 05:33:42 -0700 (PDT) Return-Path: Received: from eliot.biblibre.com (eliot.biblibre.com [188.165.204.78]) by mx.google.com with ESMTPS id e7si10004230wer.61.2011.04.18.05.33.41 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 18 Apr 2011 05:33:41 -0700 (PDT) Received-SPF: neutral (google.com: 188.165.204.78 is neither permitted nor denied by best guess record for domain of koha-patches-bounces@lists.koha-community.org) client-ip=188.165.204.78; Authentication-Results: mx.google.com; spf=neutral (google.com: 188.165.204.78 is neither permitted nor denied by best guess record for domain of koha-patches-bounces@lists.koha-community.org) smtp.mail=koha-patches-bounces@lists.koha-community.org Received: from eliot.biblibre.com (localhost.localdomain [127.0.0.1]) by eliot.biblibre.com (Postfix) with ESMTP id F2AA846E44; Mon, 18 Apr 2011 14:33:40 +0200 (CEST) X-Original-To: koha-patches@lists.koha-community.org Delivered-To: koha-patches_lists.koha-community.org@eliot.biblibre.com Received: from mail-vw0-f44.google.com (mail-vw0-f44.google.com [209.85.212.44]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by eliot.biblibre.com (Postfix) with ESMTPS id 810574684B for ; Mon, 18 Apr 2011 14:33:39 +0200 (CEST) Received: by vws12 with SMTP id 12so5907901vws.17 for ; Mon, 18 Apr 2011 05:33:38 -0700 (PDT) Received: by 10.52.75.229 with SMTP id f5mr310876vdw.291.1303130017569; Mon, 18 Apr 2011 05:33:37 -0700 (PDT) Received: from localhost.localdomain (pool-98-114-30-184.phlapa.fios.verizon.net [98.114.30.184]) by mx.google.com with ESMTPS id z18sm646185vbx.2.2011.04.18.05.33.35 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 18 Apr 2011 05:33:35 -0700 (PDT) From: "Nicole C. Engard" To: koha-patches@lists.koha-community.org Date: Sun, 17 Apr 2011 18:15:51 -0400 Message-Id: <1303078551-3045-1-git-send-email-nengard@bywatersolutions.com> X-Mailer: git-send-email 1.7.2.3 Cc: Jared Camins-Esakov Subject: [Koha-patches] [PATCH] [SIGNED-OFF] Bug 5859: Allow user to set encoding for import X-BeenThere: koha-patches@lists.koha-community.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: Patches submitted to Koha List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: koha-patches-bounces@lists.koha-community.org Errors-To: koha-patches-bounces@lists.koha-community.org From: Jared Camins-Esakov Rather than having options for MARC21 and UNIMARC in the "Character encoding" dropdown, the user should be able to select the appropriate character encoding. The default retains the current behavior, which is to allow the system to guess which character encoding is in use. However, it should be noticed that this is almost always wrong for non-UTF8 records with non-ASCII characters. Specifying a character set is much more reliable if you're not using UTF-8. Rebased to use Template::Toolkit instead of HTML::Template::Pro. Signed-off-by: Jared Camins-Esakov Signed-off-by: Nicole C. Engard --- C4/ImportBatch.pm | 11 +++++++---- .../prog/en/modules/tools/stage-marc-import.tt | 9 ++------- tools/stage-marc-import.pl | 8 ++++---- 3 files changed, 13 insertions(+), 15 deletions(-) diff --git a/C4/ImportBatch.pm b/C4/ImportBatch.pm index 145e5d9..b6db406 100644 --- a/C4/ImportBatch.pm +++ b/C4/ImportBatch.pm @@ -210,7 +210,7 @@ sub ModBiblioInBatch { =head2 BatchStageMarcRecords ($batch_id, $num_records, $num_items, @invalid_records) = - BatchStageMarcRecords($marc_flavor, $marc_records, $file_name, + BatchStageMarcRecords($encoding, $marc_records, $file_name, $comments, $branch_code, $parse_items, $leave_as_staging, $progress_interval, $progress_callback); @@ -218,7 +218,7 @@ sub ModBiblioInBatch { =cut sub BatchStageMarcRecords { - my $marc_flavor = shift; + my $encoding = shift; my $marc_records = shift; my $file_name = shift; my $comments = shift; @@ -258,13 +258,16 @@ sub BatchStageMarcRecords { &$progress_callback($rec_num); } my ($marc_record, $charset_guessed, $char_errors) = - MarcToUTF8Record($marc_blob, C4::Context->preference("marcflavour")); + MarcToUTF8Record($marc_blob, C4::Context->preference("marcflavour"), $encoding); + + $encoding = $charset_guessed unless $encoding; + my $import_record_id; if (scalar($marc_record->fields()) == 0) { push @invalid_records, $marc_blob; } else { $num_valid++; - $import_record_id = AddBiblioToBatch($batch_id, $rec_num, $marc_record, $marc_flavor, int(rand(99999)), 0); + $import_record_id = AddBiblioToBatch($batch_id, $rec_num, $marc_record, $encoding, int(rand(99999)), 0); if ($parse_items) { my @import_items_ids = AddItemsToImportBiblio($batch_id, $import_record_id, $marc_record, 0); $num_items += scalar(@import_items_ids); diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt index f44181e..fc6db51 100644 --- a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt +++ b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt @@ -93,13 +93,8 @@ function CheckForm(f) {
  • - - [% IF ( UNIMARC ) %] - - [% ELSE %] - - [% END %] - + +
  • diff --git a/tools/stage-marc-import.pl b/tools/stage-marc-import.pl index b113332..745a36a 100755 --- a/tools/stage-marc-import.pl +++ b/tools/stage-marc-import.pl @@ -55,7 +55,7 @@ my $nomatch_action = $input->param('nomatch_action'); my $parse_items = $input->param('parse_items'); my $item_action = $input->param('item_action'); my $comments = $input->param('comments'); -my $syntax = $input->param('syntax'); +my $encoding = $input->param('encoding'); my ($template, $loggedinuser, $cookie) = get_template_and_user({template_name => "tools/stage-marc-import.tmpl", query => $input, @@ -130,10 +130,10 @@ if ($completedJobID) { } # FIXME branch code - my ($batch_id, $num_valid, $num_items, @import_errors) = BatchStageMarcRecords($syntax, $marcrecord, $filename, - $comments, '', $parse_items, 0, - 50, staging_progress_callback($job, $dbh)); + my ($batch_id, $num_valid, $num_items, @import_errors) = BatchStageMarcRecords($encoding, $marcrecord, $filename, $comments, '', $parse_items, 0, 50, staging_progress_callback($job, $dbh)); + $dbh->commit(); + my $num_with_matches = 0; my $checked_matches = 0; my $matcher_failed = 0; -- 1.7.2.3 _______________________________________________ Koha-patches mailing list Koha-patches@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-patches website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/