Attachment 168431 Details for Bug 36831 – Bug 36831: Add a method to determine the file delimiter

[patch] Bug 36831: Add a method to determine the file delimiter

Bug-36831-Add-a-method-to-determine-the-file-delim.patch (text/plain), 2.08 KB, created by Matt Blenkinsop on 2024-07-03 10:11:05 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Matt Blenkinsop

Created: 2024-07-03 10:11:05 UTC

Size: 2.08 KB

patch

obsolete

>From 6179b48dced81a0473f21b4c1a835a18eb52e305 Mon Sep 17 00:00:00 2001
>From: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
>Date: Wed, 3 Jul 2024 09:48:06 +0000
>Subject: [PATCH] Bug 36831: Add a method to determine the file delimiter
>
>This patch adds a method that will detect the delimiter and quote character for a file
>---
> Koha/ERM/EHoldings/Title.pm | 46 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 46 insertions(+)
>
>diff --git a/Koha/ERM/EHoldings/Title.pm b/Koha/ERM/EHoldings/Title.pm
>index ea62ecb360..33c5c289ad 100644
>--- a/Koha/ERM/EHoldings/Title.pm
>+++ b/Koha/ERM/EHoldings/Title.pm
>@@ -100,6 +100,52 @@ sub resources {
>     return Koha::ERM::EHoldings::Resources->_new_from_dbic($resources_rs);
> }
> 
>+=head3 _detect_delimiter_and_quote
>+
>+Identifies the delimiter and the quote character used in the KBART file and returns both.
>+
>+=cut
>+
>+sub _detect_delimiter_and_quote {
>+    my ($file) = @_;
>+    my $sample_lines = 5;    # Number of lines to sample for detection
>+
>+    open my $fh, '<', \$file or die "Could not open '$file': $!";
>+
>+    my @lines;
>+    while (<$fh>) {
>+        push @lines, $_;
>+        last if $. >= $sample_lines;
>+    }
>+    close $fh;
>+
>+    my %delimiter_count;
>+    my %quote_count;
>+
>+    foreach my $line (@lines) {
>+        foreach my $char ( ',', '\t', ';', '|' ) {
>+            my $count = () = $line =~ /\Q$char\E/g;
>+            $delimiter_count{$char} += $count if $count;
>+        }
>+        foreach my $char ( '"', "'" ) {
>+            my $count = () = $line =~ /\Q$char\E/g;
>+            $quote_count{$char} += $count if $count;
>+        }
>+    }
>+
>+    # Guess the delimiter with the highest count
>+    my ($delimiter) = sort { $delimiter_count{$b} <=> $delimiter_count{$a} } keys %delimiter_count;
>+
>+    # Guess the quote character with the highest count
>+    my ($quote) = sort { $quote_count{$b} <=> $quote_count{$a} } keys %quote_count;
>+
>+    # Fallback to common defaults if nothing is detected
>+    $delimiter //= ',';
>+    $quote     //= '"';
>+
>+    return ( $delimiter, $quote );
>+}
>+
> =head2 Internal methods
> 
> =head3 _type
>-- 
>2.39.3 (Apple Git-146)

Actions: View | Diff | Splinter Review

Attachments on bug 36831: 166958 | 166959 | 166960 | 166961 | 166989 | 166990 | 166991 | 168431 | 168432 | 168433 | 168434 | 170549 | 170550 | 170551 | 170552 | 170562 | 170563 | 170564 | 170565