Bug 9921 - Make it possible to force 001 = biblionumber
Summary: Make it possible to force 001 = biblionumber
Status: RESOLVED DUPLICATE of bug 27981
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Bibliographic data support (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement with 20 votes (vote)
Assignee: Nuño López Ansótegui
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-25 12:11 UTC by Magnus Enger
Modified: 2021-03-18 13:26 UTC (History)
16 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
.tar for serialized 001 field (11.73 KB, application/octet-stream)
2013-03-26 15:37 UTC, Paul A
Details
Bug 9921 - Make it possible to force 001 = biblionumber (8.23 KB, patch)
2013-06-06 10:28 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (8.24 KB, patch)
2013-06-06 10:46 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (113.51 KB, patch)
2013-06-06 13:06 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (113.34 KB, patch)
2013-06-12 08:49 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
autoControNumber preview (200.92 KB, image/jpeg)
2013-06-12 09:42 UTC, Nuño López Ansótegui
Details
Bug 9921 - Make it possible to force 001 = biblionumber (114.15 KB, patch)
2013-06-17 10:28 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (113.74 KB, patch)
2013-06-18 08:59 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921: Followup for installer languages (105.79 KB, patch)
2013-06-19 12:35 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (8.70 KB, patch)
2013-06-19 14:55 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (10.33 KB, patch)
2013-06-20 10:41 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (10.69 KB, patch)
2013-06-20 17:00 UTC, Nuño López Ansótegui
Details | Diff | Splinter Review
Bug 9921: Followup for installer languages (61.93 KB, patch)
2021-03-17 17:08 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 9921 - Make it possible to force 001 = biblionumber (11.76 KB, patch)
2021-03-17 17:08 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 9921: Remove option of 'incremental' and add tests (13.87 KB, patch)
2021-03-18 13:13 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Magnus Enger 2013-03-25 12:11:38 UTC
For some installations it would be nice to be able to force MARC field 001 to contain the biblionumber. This should obviously be governed by a syspref.
Comment 1 Paul A 2013-03-26 15:35:58 UTC
We have always used a unique 001 in conjunction with our CaOPIACS 003 (we believe that MARC structure requires unique 001/003, but that's maybe not a universal interpretation.)

I'm attaching a .tar that probably needs cleaning up, but it does trace how we got to the final code. No guarantee at all that others may find it useful, but our cataloguers can now either retain the 001/003 pair from Z39.50 imports, or "automagically" click on 001 for the next sequential CaOPIACS Control Number.
Comment 2 Paul A 2013-03-26 15:37:22 UTC Comment hidden (obsolete)
Comment 3 Nuño López Ansótegui 2013-06-06 10:28:08 UTC Comment hidden (obsolete)
Comment 4 Nuño López Ansótegui 2013-06-06 10:46:00 UTC Comment hidden (obsolete)
Comment 5 Marcel de Rooy 2013-06-06 10:52:18 UTC
Interesting patch! Few questions: 

my $incrementalCN=C4::Context->preference('incrementalControlNumber');
my $sth = $dbh->prepare(q{UPDATE systempreferences SET value = ? WHERE variable = 'incrementalControlNumber'});
Wouldn't it be more safe to ask for a new incremental number in one call (concurrent users etc.) ?
So, call a function, increment the number in the db rightaway, return the number, save it.

Copyright 2000-2002 Katipo Communications
You add a new file with a 2000 copyright?
You could strip what is not needed.

You create marc21_field_001.pl
But it looks like this plugin could be of use for UNIMARC as well (same field)? 

Thanks.
Comment 6 Nuño López Ansótegui 2013-06-06 13:06:29 UTC Comment hidden (obsolete)
Comment 7 Nuño López Ansótegui 2013-06-06 13:17:36 UTC
(In reply to comment #5)
> Interesting patch! Few questions: 
> 
> my $incrementalCN=C4::Context->preference('incrementalControlNumber');
> my $sth = $dbh->prepare(q{UPDATE systempreferences SET value = ? WHERE
> variable = 'incrementalControlNumber'});
> Wouldn't it be more safe to ask for a new incremental number in one call
> (concurrent users etc.) ?
> So, call a function, increment the number in the db rightaway, return the
> number, save it.
> 
> Copyright 2000-2002 Katipo Communications
> You add a new file with a 2000 copyright?
> You could strip what is not needed.
> 
> You create marc21_field_001.pl
> But it looks like this plugin could be of use for UNIMARC as well (same
> field)? 
> 
> Thanks.

Ok, Now the number is incremented and saved it before.

Copyright is now modified.

The plugin is only to MARC21, but the 001 is modified alike in UNIMARC when the field is empty.

If someone wants to include the plugin in UNIMARC, forward.

Thanks.
Comment 8 Marcel de Rooy 2013-06-06 13:20:52 UTC
You attached the wrong patch perhaps??
Comment 9 Marcel de Rooy 2013-06-06 13:21:30 UTC
(In reply to comment #8)
> You attached the wrong patch perhaps??

No wait..
Comment 10 Nuño López Ansótegui 2013-06-06 13:32:45 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > You attached the wrong patch perhaps??
> 
> No wait..

Does it apply?
Comment 11 Marcel de Rooy 2013-06-06 13:33:57 UTC
(In reply to comment #10)
> Does it apply?

Yes, it applies. Will try to test later, though..
Comment 12 Nuño López Ansótegui 2013-06-12 08:49:11 UTC Comment hidden (obsolete)
Comment 13 Nuño López Ansótegui 2013-06-12 09:42:31 UTC
Created attachment 18863 [details]
autoControNumber preview

Field 001 is controlled by a system preference, which will have three options:
 
If it to be "generated as biblionumber" will generate the 001 as the biblionumber of the bibliographic record (if you don´t manually enter a value in the 001).

If you select the option "generated in the form 1, 2, 3", 001 is generated from a number specified in the system preference 'incrementalControlNumber', which will automatically increase.
Comment 14 Marcel de Rooy 2013-06-14 08:26:08 UTC
Nuño:
Thanks for your patch adjustments and the effort to synchronize all language files in this regard.

But I still have some thoughts about this:
1) I think we cannot force the plugin at upgrade time (in the updatedatabase change). If someone already has another plugin there, you cannot overwrite that.
I would consider this a blocker for this patch.
You could consider the update if there is no plugin attached. You could print a upgrade warning statement if there is another plugin.

2) My personal opinion is that we should stop making changes in all the language folders. Let an active language maintainer do that. This is currently not the policy however; note that I sent a message to the dev list just now about this topic. More ideally, you would insert the plugin only in the general (English) file and just update the translations in the language (the plugin filename does not need translation at all).
What we should do in the meantime, is however a little bit gray area now.
So, this does not block your patch.

3) I feel that incrementalControlNumber is not (really) a pref. I can imagine more cases where we might need a sequential number. Would it be interesting to move it perhaps to Tools in a more general context such as a sequence number engine with its own table (id, keyname, seqnumber, date)? 
There are more fields like callnumber (with plugins) that could use such an approach..
This does again not block your patch, but we could trigger the discussion on a more general solution.
Comment 15 Galen Charlton 2013-06-14 14:21:03 UTC
(In reply to comment #14)
> 3) I feel that incrementalControlNumber is not (really) a pref. I can
> imagine more cases where we might need a sequential number. Would it be
> interesting to move it perhaps to Tools in a more general context such as a
> sequence number engine with its own table (id, keyname, seqnumber, date)? 
> There are more fields like callnumber (with plugins) that could use such an
> approach..
> This does again not block your patch, but we could trigger the discussion on
> a more general solution.

This idea (creating a sequence engine) is worth elevating to a new enhancement bug.  There's been some chatter on #koha the past few days about this in the context of improving how sequential patron cardnumbers are assigned.
Comment 16 Galen Charlton 2013-06-14 14:30:22 UTC
Comment on attachment 18862 [details] [review]
Bug 9921 - Make it possible to force 001 = biblionumber

Review of attachment 18862 [details] [review]:
-----------------------------------------------------------------

::: cataloguing/value_builder/marc21_field_001.pl
@@ +18,5 @@
> +# with Koha; if not, write to the Free Software Foundation, Inc.,
> +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> +
> +use strict;
> +#use warnings; FIXME - Bug 2505

New code *must* enable the warnings pragma; please use Modern::Perl.  More details at

http://wiki.koha-community.org/wiki/Coding_Guidelines#PERL2:_Modern::Perl
Comment 17 Marcel de Rooy 2013-06-14 15:24:19 UTC
(In reply to comment #16)
> New code *must* enable the warnings pragma; please use Modern::Perl.

This could still be a problem with the way the plugins work in the marc editor. The routine is redefined iirc..
Comment 18 Galen Charlton 2013-06-14 23:36:53 UTC
(In reply to comment #17)
> (In reply to comment #16)
> > New code *must* enable the warnings pragma; please use Modern::Perl.
> 
> This could still be a problem with the way the plugins work in the marc
> editor. The routine is redefined iirc..

All the more reason to stick to our coding guidelines, if the additional warning messages in the log annoys somebody enough to inspire them to fix that issue with the cataloging plugins, then. :)
Comment 19 Nuño López Ansótegui 2013-06-17 10:28:44 UTC Comment hidden (obsolete)
Comment 20 Marcel de Rooy 2013-06-17 11:11:33 UTC
(In reply to comment #15)
> This idea (creating a sequence engine) is worth elevating to a new
> enhancement bug.  There's been some chatter on #koha the past few days about
> this in the context of improving how sequential patron cardnumbers are
> assigned.

See bug 10478 please.
Comment 21 Marcel de Rooy 2013-06-17 11:13:55 UTC
(In reply to comment #19)
> Created attachment 19062 [details] [review] [review]
> Bug 9921 - Make it possible to force 001 = biblionumber
Thanks for doing so.

> Added 'use Modern::Perl;' in cataloguing/value_builder/marc21_field_001.pl
This will not really work. You will have lots of redefine warnings.
I will add yet another report to have another way of adding plugins without redefines. Please wait, I will submit within a few minutes.

> Update if there is no plugin attached.
Great.
Comment 22 Marcel de Rooy 2013-06-17 11:32:40 UTC
(In reply to comment #18)
> (In reply to comment #17)
> > (In reply to comment #16)
> > > New code *must* enable the warnings pragma; please use Modern::Perl.
> > 
> > This could still be a problem with the way the plugins work in the marc
> > editor. The routine is redefined iirc..
> 
> All the more reason to stick to our coding guidelines, if the additional
> warning messages in the log annoys somebody enough to inspire them to fix
> that issue with the cataloging plugins, then. :)

I actually disagree here.
We cannot block development, because we already have a bad construct in Koha and a new developer should solve that first. We need to be somewhat pragmatic.
As long as the construct is in, we are more or less obliged to support it.
We would not encourage people to submit patches, if they should start solving our old problems.

But I looked a little bit, sufficiently annoyed :) and submitted a proposal for this under bug 10480. Have a look!
BTW It will not be possible to phase out all old plugins directly, because we have to keep in mind that people may have also written custom plugins this way. 
Bug 10480 supports both ways therefore.
Comment 23 Nuño López Ansótegui 2013-06-17 12:11:17 UTC
(In reply to comment #21)
> (In reply to comment #19)
> > Created attachment 19062 [details] [review] [review] [review]
> > Bug 9921 - Make it possible to force 001 = biblionumber
> Thanks for doing so.
> 
> > Added 'use Modern::Perl;' in cataloguing/value_builder/marc21_field_001.pl
> This will not really work. You will have lots of redefine warnings.
> I will add yet another report to have another way of adding plugins without
> redefines. Please wait, I will submit within a few minutes.
> 
> > Update if there is no plugin attached.
> Great.

Ok, I wait, thanks.
Comment 24 Marcel de Rooy 2013-06-17 12:18:03 UTC
(In reply to comment #23)
> Ok, I wait, thanks.

You do no longer need to wait. Click bug 10480 :)
Comment 25 Katrin Fischer 2013-06-17 14:35:55 UTC
I think the RM's opinion on this was clearly stated and I tend to agree with him ;) We leave the Modern::Perl in, but don't require to fix all the warns immediately. Ultimately we should fix those and always keep an eye on the logs to avoid adding more noise with new features. If 10480 does achieve that - even better.
Comment 26 Nuño López Ansótegui 2013-06-18 08:59:16 UTC Comment hidden (obsolete)
Comment 27 Marcel de Rooy 2013-06-19 11:17:29 UTC
(In reply to comment #25)
> I think the RM's opinion on this was clearly stated and I tend to agree with
> him ;) We leave the Modern::Perl in, but don't require to fix all the warns
> immediately. 

I must admit that I probably misunderstood Galen somehow. So actually there is not that much disagreement after all..
At least this resulted in further work on getting the warnings pragma back :)
Comment 28 Marcel de Rooy 2013-06-19 11:20:09 UTC
(In reply to comment #26)
> Created attachment 19099 [details] [review] [review]
> Bug 9921 - Make it possible to force 001 = biblionumber
> 
> 'cataloguing/value_builder/marc21_field_001.pl' has been moved to the new
> style using $building_plugins

Cool.
Nuno, would you object if I convert your patch in two patches?
I would like to see the essential stuff in one patch, and all changes to language folders to a followup.
Comment 29 Nuño López Ansótegui 2013-06-19 11:45:47 UTC
(In reply to comment #28)
> (In reply to comment #26)
> > Created attachment 19099 [details] [review] [review] [review]
> > Bug 9921 - Make it possible to force 001 = biblionumber
> > 
> > 'cataloguing/value_builder/marc21_field_001.pl' has been moved to the new
> > style using $building_plugins
> 
> Cool.
> Nuno, would you object if I convert your patch in two patches?
> I would like to see the essential stuff in one patch, and all changes to
> language folders to a followup.

Ok, I agree with that. In this way, functionality is independent of adaptation into other languages.
Comment 30 Marcel de Rooy 2013-06-19 12:09:50 UTC
(In reply to comment #29)
> (In reply to comment #28)
> > (In reply to comment #26)
> > > Created attachment 19099 [details] [review] [review] [review] [review]
> > > Bug 9921 - Make it possible to force 001 = biblionumber
> > > 
> > > 'cataloguing/value_builder/marc21_field_001.pl' has been moved to the new
> > > style using $building_plugins
> > 
> > Cool.
> > Nuno, would you object if I convert your patch in two patches?
> > I would like to see the essential stuff in one patch, and all changes to
> > language folders to a followup.
> 
> Ok, I agree with that. In this way, functionality is independent of
> adaptation into other languages.

Thanks. I will be testing now and making little change.
Comment 31 Marcel de Rooy 2013-06-19 12:35:38 UTC
Created attachment 19153 [details] [review]
Bug 9921: Followup for installer languages
Comment 32 Marcel de Rooy 2013-06-19 14:55:45 UTC Comment hidden (obsolete)
Comment 33 Marcel de Rooy 2013-06-19 14:57:26 UTC
Still some comments, but we are coming further :) Thanks for your patience..

When editing an existing record, I would prefer not to change the biblionumber in the 001 by a string again when receiving focus. Currently, you overwrite and save the string in the record.
Since you only changed AddBiblio and not ModBiblio, this is permanent.
Same for incremental.
Please combine your code for AddBiblio and ModBiblio.

Also when I would incidentally change biblionumber by deleting a char or so, I would loose my biblionumber or incremental.
Consider adding the focus action to Blur?
Consider blocking edit in this input field via your javascript?

I am not yet fully sure if your incremental update will hold under concurrent use.
If between your fetch with preference and the next execute, someone else does an earlier execute, you could have double numbers? 
Need some way of locking or transaction?

Updatedatabase
Need a print statement if someone already has a plugin for 001? Maybe they can consider your new plugin?
I am wondering if it is dangerous to set the plugin for one framework and not for another (could be the case)?
Comment 34 Marcel de Rooy 2013-06-19 15:00:28 UTC
I made some changes in bug 10480 that I incorporated in your patch too.
Comment 35 Nuño López Ansótegui 2013-06-20 10:41:03 UTC Comment hidden (obsolete)
Comment 36 Nuño López Ansótegui 2013-06-20 10:58:57 UTC
(In reply to comment #33)
> Still some comments, but we are coming further :) Thanks for your patience..
> 
> Updatedatabase
> Need a print statement if someone already has a plugin for 001? Maybe they
> can consider your new plugin?
> I am wondering if it is dangerous to set the plugin for one framework and
> not for another (could be the case)?
Comment 37 Nuño López Ansótegui 2013-06-20 11:05:20 UTC
It's no problem to use other plugin or not using of this plugin, the field is not autofilled or will be filled with a number .
 If the field 001 is blank will be updated with incremental biblionumber or number.
Comment 38 Marcel de Rooy 2013-06-20 11:51:41 UTC
Nuno,
I came across this SQL construct. It makes the use of a exclusive row level lock not needed:

UPDATE tablename SET counter  = LAST_INSERT_ID( counter + 1) WHERE id=xxx
SELECT LAST_INSERT_ID();

The second select does not need table access anymore. So you really make it one call. Concurrent use is covered nicely.
Comment 39 Marcel de Rooy 2013-06-20 12:02:57 UTC
(In reply to comment #36)
> > I am wondering if it is dangerous to set the plugin for one framework and
> > not for another (could be the case)?

Since your pref is OFF by default, this is initially harmless. 
Suppose however that people have some frameworks, and some frameworks have a 001 plugin and other frameworks have not. (Not recommended of course..)

They upgrade and now some frameworks have your plugin and others have the old (custom) one. They are perhaps not aware of this incosistency. At some time they activate your pref and they see your plugin in some framework, making the wrong conclusion that all frameworks already have it. From that time on, we do have a problem.

Now is the question: Should we actually update a Koha db that has this inconsistency with potential for future problems, or should we only update if all frameworks do NOT have a 001 plugin?
Comment 40 Katrin Fischer 2013-06-20 12:11:57 UTC
I think we should only update the database when there are no existing plugins for any of the 001. But when we do update, it should loop over all frameworks to ensure it's a consistent change. 

Marcel - not sure, but is the LAST_INSERT_ID a MySQLism? I seem to remember a discussion in IRC about it, but of course... I can't remember enough to explain why I have it memorized as not a good idea. :(
Comment 41 Nuño López Ansótegui 2013-06-20 12:40:41 UTC
(In reply to comment #39)
> (In reply to comment #36)
> > > I am wondering if it is dangerous to set the plugin for one framework and
> > > not for another (could be the case)?
> 
> Since your pref is OFF by default, this is initially harmless. 
> Suppose however that people have some frameworks, and some frameworks have a
> 001 plugin and other frameworks have not. (Not recommended of course..)
> 
> They upgrade and now some frameworks have your plugin and others have the
> old (custom) one. They are perhaps not aware of this incosistency. At some
> time they activate your pref and they see your plugin in some framework,
> making the wrong conclusion that all frameworks already have it. From that
> time on, we do have a problem.
> 
> Now is the question: Should we actually update a Koha db that has this
> inconsistency with potential for future problems, or should we only update
> if all frameworks do NOT have a 001 plugin?

If they have their own 001 plugin, i think that they should be able to control their frameworks and differ if they have the default plugin 001 in Koha or not.

What do you think?
Comment 42 Marcel de Rooy 2013-06-20 14:11:06 UTC
(In reply to comment #41)
> If they have their own 001 plugin, i think that they should be able to
> control their frameworks and differ if they have the default plugin 001 in
> Koha or not.
> 
> What do you think?

My point only applies to several frameworks with inconsistent use of 001 plugin.
Just to be safe, we could restrict ourselves to updating only when all 001 plugins are empty and print a upgrade warning otherwise.
Comment 43 Marcel de Rooy 2013-06-20 14:18:49 UTC
(In reply to comment #40)
> Marcel - not sure, but is the LAST_INSERT_ID a MySQLism? I seem to remember
> a discussion in IRC about it, but of course... I can't remember enough to
> explain why I have it memorized as not a good idea. :(

If we go through DBI, we have actually two choices: the more generic last_insert_id and the mysql-bound mysql_insert_id.
If you do a grep on both, you will see that they are used a lot in Koha.

Actually, AddBiblio relies on:
my $biblionumber = $dbh->{'mysql_insertid'};

But note that last_insert_id may be more portable, it is considered less reliable than e.g. using MySQL with mysql_insertid.
Since AddBiblio relies on it, I would have no objection to using one of the two constructs.

One other point: I see confirmation that it is connection-safe, but I saw some claims that it could not be thread-safe (threads using the same connection).
Of course, this also holds for current code.
Comment 44 Marcel de Rooy 2013-06-20 14:28:48 UTC
Found this quote somewhere in context of using mod_perl:

As long as you get it from the same $sth, it's going to be thread safe (i.e. doesn't matter if another process inserts before you call insert_id, you still get the one you inserted).
Comment 45 Galen Charlton 2013-06-20 15:38:54 UTC
(In reply to comment #43)
> But note that last_insert_id may be more portable, it is considered less
> reliable than e.g. using MySQL with mysql_insertid.
> Since AddBiblio relies on it, I would have no objection to using one of the
> two constructs.

Actually, it looks like DBD::mysql bases its implementation of its last_insert_id() method on MySQL's mysql_insert_id function, so as far as DBI is concerned, I think $dbh->last_insert_id() does the Right Thing.

In the original context though, this bug plus recent discussion on #koha about table-locking issues with fixup_cardnumber is making me think that what we *really* need is a small module for simulating true sequences so that one can simply do something like

my $num = Koha::Database::Sequence->get_next_val($sequence_name);

The idea is that the module and the code backing it would provide the ability to create a sequence with a specified name, then fetch values from it that are guaranteed to be unique.

I'll poke at this.
Comment 46 Marcel de Rooy 2013-06-20 16:28:20 UTC
(In reply to comment #45)

> In the original context though, this bug plus recent discussion on #koha
> about table-locking issues with fixup_cardnumber is making me think that
> what we *really* need is a small module for simulating true sequences so
> that one can simply do something like
> 
> my $num = Koha::Database::Sequence->get_next_val($sequence_name);
> 
> The idea is that the module and the code backing it would provide the
> ability to create a sequence with a specified name, then fetch values from
> it that are guaranteed to be unique.

See bug 10478.
Comment 47 Nuño López Ansótegui 2013-06-20 17:00:07 UTC
Created attachment 19176 [details] [review]
Bug 9921 - Make it possible to force 001 = biblionumber

Updating only when all 001 plugins are empty and print a upgrade warning otherwise.
Comment 48 Mark Tompsett 2013-06-27 07:34:52 UTC
Koha::Sequence will be added as part of bug 10454. This may be useful in this context for incrementing something automatically.

Here's a preliminary sample test suite for it, and the output.
--- BEGIN ---
#!/usr/bin/perl

use strict;
use warnings;

use Test::More tests => 15;

BEGIN {
        use_ok('Koha::Sequence' , 'check to use Koha::Sequence');
}

my $seq = Koha::Sequence->new('blah');
ok ( defined($seq) , 'created sequence called blah');
my $rv = $seq->reset_everything;
ok ( $rv , 'triggered reset of everything' );

$seq = Koha::Sequence->new('id1');
ok ( defined($seq) , 'created sequence called id1');

$rv = $seq->del_sequence;
ok ( $rv , 'deleted sequence called id1');

my $seq1 = Koha::Sequence->new('cardnumber_katipo');
my $seq2 = Koha::Sequence->new('cardnumber_none');
ok ( $seq1->is_sequence, 'Confirmed cardnumber_katipo sequence' );
ok ( $seq2->is_sequence, 'Confirmed cardnumber_none sequence' );

my $value;
ok ( $value = $seq1->get_next_value, "Value $value" );
ok ( $value = $seq1->get_next_value, "Value $value" );
ok ( $value = $seq2->get_next_value, "Value $value" );
ok ( $value = $seq2->get_next_value, "Value $value" );

$seq = Koha::Sequence->new('blah');
ok ( defined($seq) , 'created sequence called blah');
$rv = $seq->reset_everything;
ok ( $rv , 'triggered reset of everything' );

ok ( $value = $seq1->get_next_value, "Value $value" );
ok ( $value = $seq2->get_next_value, "Value $value" );
--- END ---

t$ prove -v Sequence.t
Sequence.t ..
1..15
ok 1 - use Koha::Sequence;
ok 2 - created sequence called blah
ok 3 - triggered reset of everything
ok 4 - created sequence called id1
ok 5 - deleted sequence called id1
ok 6 - Confirmed cardnumber_katipo sequence
ok 7 - Confirmed cardnumber_none sequence
ok 8 - Value 1000000
ok 9 - Value 1000001
ok 10 - Value 162
ok 11 - Value 163
ok 12 - created sequence called blah
ok 13 - triggered reset of everything
ok 14 - Value 1000000
ok 15 - Value 162
ok
All tests successful.
Files=1, Tests=15,  1 wallclock secs ( 0.02 usr  0.00 sys +  0.10 cusr  0.00 csys =  0.12 CPU)
Result: PASS

the reset_everything rebuilds the cardnumber_katipo and cardnumber_none sequences.
Comment 49 I'm just a bot 2013-09-29 04:59:35 UTC
Patch applied cleanly, go forth and signoff
Comment 50 I'm just a bot 2013-09-29 05:00:20 UTC
Applying: Bug 9921 - Make it possible to force 001 = biblionumber
Using index info to reconstruct a base tree...
M	C4/Biblio.pm
M	installer/data/mysql/sysprefs.sql
M	installer/data/mysql/updatedatabase.pl
Falling back to patching base and 3-way merge...
Auto-merging installer/data/mysql/updatedatabase.pl
CONFLICT (content): Merge conflict in installer/data/mysql/updatedatabase.pl
Auto-merging installer/data/mysql/sysprefs.sql
CONFLICT (content): Merge conflict in installer/data/mysql/sysprefs.sql
Auto-merging C4/Biblio.pm
Patch failed at 0001 Bug 9921 - Make it possible to force 001 = biblionumber
The copy of the patch that failed is found in:
   /home/christopher/git/koha/.git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Comment 51 Zeno Tajoli 2013-10-28 11:28:35 UTC
Hi to all,

In fact the patch is blocked by discussion on Koha::Sequence (bug 10454).

Is it Ok if i rewrite the patch without the incremental option ?
I will insert two options.

1)001asBiblioNumberForce: with this option you can try to insert everything in 001, but the system force the equivalence 001 = biblionumber
You don't need the plugin

2)biblionumber: with this option you can use the plugin to insert the string 'biblionumber'. The system changes 'biblionumber' with the value of biblionumber.
You need the plugin. If you insert a different number/string, the system will insert yoyr number/string.

What do you think ?
Comment 52 Marcel de Rooy 2013-10-28 13:05:31 UTC
(In reply to Zeno Tajoli from comment #51)
> In fact the patch is blocked by discussion on Koha::Sequence (bug 10454).
> Is it Ok if i rewrite the patch without the incremental option ?
> I will insert two options.

I would favor a solution using the sequence number engine.
Comment 53 Zeno Tajoli 2013-10-28 16:14:52 UTC
The problem is that sequence number engine is blocked (for mysqlism problems).
I think we can that the low level fruit (force 001 = biblionumber and a plugin to insert it if want to choose) an wait for a futere improvement.
Comment 54 Mirko Tietgen 2016-09-09 13:41:22 UTC
This would need quite a rebase. It would still be a useful feature.

The bug that is mentioned as a blocker has not gotten any update in 3 years (besides a lowered severity).

I would consider (not promising) to rebase it if there is a consensus that it is ok to leave out incremental and have at least the biblionumber in 001 feature instead of nothing. It can be extended in another bug.

I understand that incremental was the only issue with this patch, is that correct?
Comment 55 Marcel de Rooy 2016-09-12 10:09:21 UTC
(In reply to Mirko Tietgen from comment #54)
> This would need quite a rebase. It would still be a useful feature.
> 
> The bug that is mentioned as a blocker has not gotten any update in 3 years
> (besides a lowered severity).
> 
> I would consider (not promising) to rebase it if there is a consensus that
> it is ok to leave out incremental and have at least the biblionumber in 001
> feature instead of nothing. It can be extended in another bug.
> 
> I understand that incremental was the only issue with this patch, is that
> correct?

Would be nice if you got this moving again somehow.
Comment 56 Magnus Enger 2016-09-13 06:48:29 UTC
(In reply to Mirko Tietgen from comment #54)
> This would need quite a rebase. It would still be a useful feature.
> 
> The bug that is mentioned as a blocker has not gotten any update in 3 years
> (besides a lowered severity).
> 
> I would consider (not promising) to rebase it if there is a consensus that
> it is ok to leave out incremental and have at least the biblionumber in 001
> feature instead of nothing. It can be extended in another bug.

I would be happy to have that in Koha!
Comment 57 emjhet 2018-05-30 20:33:39 UTC
Like the 2013 thread starter and initial commentator, I believe MARC designates the 001 field (Control Number) for use to uniquely identify a bibliographic record. IME the ILS does this automatically with both created and imported records. (Imported records with data in the 001 field would have that data automatically transferred into a newly created 035 field.) These unique, sequential Control Numbers were automatically generated system-wide despite multiple locations and input from multiple users.
Koha appears to be able to create a unique, auto-generated, sequential identifier for each bibliographic record despite input from multiple users through the Koha field 'biblio.biblionumber;' however, it is mapped to the MARC field 999 subfield 'c.'
It would be preferable to have this unique identifier in the MARC 001 field. Does the sequence number engine need to be fixed when the biblio.biblionumber auto-generates a unique sequential identifier already? Is it a matter of setting up the two fields alike (created upon the record being saved, and being visible after saving and during editing) so that a unique identifier could safely be generated in that field? With all the sequential auto-generated fields used currently by Koha (999$c, 999$d, 952$p), surely there is a way to auto populate the 001 field. 
If Koha requires that the 999$c field contain a unique identifier, I would find it preferable if the same identifier be used in the 001 rather than multiple unique identifiers.
Comment 58 paxed 2018-06-04 10:59:57 UTC
(In reply to emjhet from comment #57)
> Like the 2013 thread starter and initial commentator, I believe MARC
> designates the 001 field (Control Number) for use to uniquely identify a
> bibliographic record.

001 alone does not guarantee uniqueness. 001 and 003 together should be globally unique. If 001 is changed, then 003 should be changed to match the org code of the system.

You could have eg. 001 generated in your system (and 003 with your org code), and another (imported) record with 001 from another system and the 003 containing the org code of that other system.
Comment 59 emjhet 2018-06-06 17:49:38 UTC
(In reply to paxed from comment #58)
> 
> 001 alone does not guarantee uniqueness. 001 and 003 together should be
> globally unique. If 001 is changed, then 003 should be changed to match the
> org code of the system.

Indeed. That is why I referred to the original post. But as Koha has a plugin that can be used in the 003, this field does not present a hurdle for use of that MARC field as designed. 

The other Integrated Library Systems I have worked with did not overlay data in the 003 field. Usually, information in that field was also automatically transferred to the 035. Also automatically, a new number that was relevant to our system was generated for the 001 and our institutional code was put into the 003 field. 

Indeed, there is TRULY nothing globally unique to the 999$c. If a Koha system permitted another Koha system to import a record in which the 999$c overlays, there is no "003" field to designate which institution the "unique identifier" belongs. 

This came up because I am trying to create Frameworks for my co-workers that are not trained in cataloging but yet have to catalog. It didn't make sense that some of our records did not have anything in the 001 field, as I was taught that this field was essential. So I am simply trying to understand how/why Koha differs so much from the other ILS I have used in the past and from MARC 21 in general.

Thus, I am wondering what it is about the 999$c that permits an auto-generated unique identifier to be created when it can not be created in the 001, and if anything can be done to either duplicate the 999$c into the 001 or use the 001 instead of the 999$c.
Comment 60 Katrin Fischer 2018-07-14 11:51:04 UTC
> Thus, I am wondering what it is about the 999$c that permits an
> auto-generated unique identifier to be created when it can not be created in
> the 001, and if anything can be done to either duplicate the 999$c into the
> 001 or use the 001 instead of the 999$c.

The biblionumber is not a sequentially created number of the application, but it's the primary key assigned by the database. That's why it will only be set after saving a record for the first time - before it doesn't exist.

I strongly advise against trying to change the configuration to use 001 instead, because the indexing won't match and you will get a lot of hard to fix problems. Indexing relies on the biblionumber to be in 999$c and you can't map 2 fields at the time (so duplicating won't work).
Comment 61 emjhet 2018-07-18 17:19:40 UTC
(In reply to Katrin Fischer from comment #60)

> The biblionumber is not a sequentially created number of the application,
> but it's the primary key assigned by the database. That's why it will only
> be set after saving a record for the first time - before it doesn't exist.
> 

I understand the purpose of the biblionumber. It is what MARC says the field 001 is for. I am trying to understand why Koha chose to use the 999$c for this information instead of the 001, and the implications this decision has on our library.

Although it is not good cataloging practice, Koha recommends importing a "close" record if you can not find a matching record in your Z39.50 search. Then simply alter the record as needed. This has happened at the library where I work. However, the 001 field was not changed. Often these records were imported from our state union catalog. Because Koha does not create a unique 001, these records would be for different items but have the same 001, and our holdings would likely be attached to the wrong record. Thus we could receive requests for items that we do not, in fact, own.

Thus this matter is not merely academic for us.
Comment 62 Katrin Fischer 2018-07-18 17:46:16 UTC
I understand, just trying to prevent you from doing something that might break stuff (because people have gone there before). I can't tell you why for Koha 999c was chosen in the beginning, but it's one of the things that will need quite a bit of thinking if we wanted to change it now.

I think here are patches around to make a plugin to set 001 individually. I am using a variation of the stocknumberAV plugin in one of our libraries to create numbers, but you have to trigger it and it doesn't move the number to 035 as they only do original cataloguing. A catalouging plugin might be a route to explore.
Comment 63 emjhet 2018-07-18 18:41:37 UTC
(In reply to Katrin Fischer from comment #62)
> I understand, just trying to prevent you from doing something that might
> break stuff (because people have gone there before). I can't tell you why
> for Koha 999c was chosen in the beginning, but it's one of the things that
> will need quite a bit of thinking if we wanted to change it now.
> 
> I think here are patches around to make a plugin to set 001 individually. I
> am using a variation of the stocknumberAV plugin in one of our libraries to
> create numbers, but you have to trigger it and it doesn't move the number to
> 035 as they only do original cataloguing. A catalouging plugin might be a
> route to explore.

Katrin -- Thank you so much for responding. I'm a cataloger among non-catalogers and I am just trying to make things as easy as possible for my co-workers while following standard guidelines for minimum level cataloging. 

Unless I have a plug-in for the 001 and permit it to be filled in with a click of the mouse, I will not be able to convince anyone to spend the time on it. Some of our libraries are completely one-"man" operations.

Koha deviates in several significant ways from the ILS that I am accustom to, including not having a good way to run reports requesting specific MARC fields. Am I correct that I can run a report and find out how many records have a specific MARC field, but the only way to find out the content of the field in a report is to have the MARC field mapped to an authorized value?
Comment 64 Katrin Fischer 2018-07-18 19:10:09 UTC
You can query and output the content of the field without an authorised value. Did you take a look at the sql library on the wiki? Maybe a topic for a different  channel, like IRC or mailing list tho :)
Comment 65 David Cook 2018-11-28 07:26:27 UTC
What about a way of putting the biblionumber in the 001 as well as the 999$c and 999$d? That way we keep backwards compatibility as well as something for going forward?
Comment 66 Simon Hohl 2019-01-22 10:11:25 UTC
Hi Katrin,

could you elaborate on your variation of stocknumberAV? It looks like there are some plugins for setting item related fields, but none for 001.

https://github.com/Koha-Community/Koha/tree/master/cataloguing/value_builder

So basically you'd need a script that reads the maximum biblio_metadata.id instead of items.stocknumber as in https://github.com/Koha-Community/Koha/blob/master/cataloguing/value_builder/stocknumber.pl?

Is your script publicly available somewhere?

Kind regards,
Simon
Comment 67 Simon Hohl 2019-01-22 15:12:36 UTC
I just realized: biblio_metadata.id is not the system number in 001. So you would need the use the search index somehow?
Comment 68 Katrin Fischer 2019-01-25 09:29:28 UTC
The ...AV.pl catalouging plugins are based on a counter in an authorised value. The original catalouging plugin is used for generating stocknumbers, but could also be adapted to be used for generating numerus currens callnumbers.

This has the advantage that even with more than one person catalouging at the same time, the number is only generated/used once. The downside is that when you have used a number but not save the record/item, the number is lost unused.

I believe other catalouging plugins use the approach to look for the max(stocknumber or similar) with SQL and creating the next number from there. Advantage: Every number will be used, disadvantage: the number could be used multiple times. 

I can send you the file for the 001-plugin we use - but I think I really only adapted the SQL statement of stocknumberAV.pl a bit.

This might also be interesting for you (new in 18.11): bug 19263
Comment 69 Simon Hohl 2019-02-05 10:15:17 UTC
Hey Katrin,

thanks for answering.

My Perl skills are somewhat lackluster, so I'd be interested in a proof of concept (i.e. your script). ;)

Kind regards,
Simon
Comment 70 Katrin Fischer 2019-02-06 07:52:10 UTC
(In reply to Simon Hohl from comment #69)
> Hey Katrin,
> 
> thanks for answering.
> 
> My Perl skills are somewhat lackluster, so I'd be interested in a proof of
> concept (i.e. your script). ;)
> 
> Kind regards,
> Simon

Hi Simon, I've sent you an email - not much Perl magic in this case. It's only change da tiny bit from the original base plugin stocknumberAV.pl.
Comment 71 Nick Clemens 2021-03-17 17:08:13 UTC
Created attachment 118424 [details] [review]
Bug 9921: Followup for installer languages
Comment 72 Nick Clemens 2021-03-17 17:08:17 UTC
Created attachment 118425 [details] [review]
Bug 9921 - Make it possible to force 001 = biblionumber

Field 001 is controlled by a system preference, which will have three options:

The option "biblionumber" will set field 001 to the biblionumber when you save
the record.
The option "1,2,3" uses preference incrementalControlNumber to generate the
next sequential number.
The option "not generated automatically" allows for manual entry.
Comment 73 Nick Clemens 2021-03-18 13:13:21 UTC
Created attachment 118460 [details] [review]
Bug 9921: Remove option of 'incremental' and add tests
Comment 74 Nick Clemens 2021-03-18 13:26:13 UTC
There are many comments here, some about plugins as a workaround, moving to a new bug and removing the incremental option

*** This bug has been marked as a duplicate of bug 27981 ***