Bug 11529 - Add subtitle, medium and part fields to biblio table
Summary: Add subtitle, medium and part fields to biblio table
Status: Patch doesn't apply
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Target Milestone: ---
Assignee: Ere Maijala
QA Contact: Katrin Fischer
URL:
Whiteboard:
Keywords: Manual, release-notes-needed
: 7715 18408 20403 (view as bug list)
Depends on:
Blocks: 4164 4166 7074 7384 8265 10953 14911 22699
  Show dependency treegraph
 
Reported: 2014-01-13 14:46 UTC by Kyle M Hall
Modified: 2019-06-18 04:33 UTC (History)
17 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Medium patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:
Sponsored by the National Library of Finland. Keywords to MARC mapping functionality has been replaced with additional Koha fields in the bibliographic framework. The keyword mapping only offered a single field, subtitle, and the information was always retrieved from the MARC record, which made it relatively slow. The subtitle and other relevant fields are now available as normal Koha fields: - biblio.medium - Medium information (MARC 21: 245h, UNIMARC: 200b) - biblio.subtitle - Subtitle (MARC 21: 245b, UNIMARC: 200e) - biblio.part_number - Part number (MARC 21: 245n, UNIMARC: 200h) - biblio.part_name - Part name (MARC 21: 245p, UNIMARC: 200i) The subfields in the default framework are automatically updated to include these new fields unless they are already mapped to another Koha field. Note that misc/batchRebuildBiblioTables.pl needs to be run to populate the fields in the database, and it will take some time for larger databases.
Version(s) released in:


Attachments
Bug 11529: Add new fields to biblio table (69.16 KB, patch)
2018-11-05 12:23 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (27.66 KB, patch)
2018-11-05 12:23 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.01 KB, patch)
2018-11-07 14:26 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (40.52 KB, patch)
2018-11-07 14:26 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.01 KB, patch)
2018-11-07 14:30 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (32.91 KB, patch)
2018-11-07 14:30 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.01 KB, patch)
2018-11-07 14:35 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (33.12 KB, patch)
2018-11-07 14:35 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.01 KB, patch)
2018-11-08 14:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.56 KB, patch)
2018-11-08 14:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2018-11-08 14:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.00 KB, patch)
2018-11-19 08:38 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.55 KB, patch)
2018-11-19 08:38 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2018-11-19 08:38 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (55.34 KB, patch)
2018-11-19 08:38 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (71.00 KB, patch)
2018-11-20 11:24 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.55 KB, patch)
2018-11-20 11:24 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2018-11-20 11:24 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (55.34 KB, patch)
2018-11-20 11:24 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (20.84 KB, patch)
2018-11-20 11:25 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.08 KB, patch)
2018-12-12 12:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.58 KB, patch)
2018-12-12 12:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2018-12-12 12:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.87 KB, patch)
2018-12-12 12:30 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (18.97 KB, patch)
2018-12-12 12:30 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.10 KB, patch)
2018-12-12 12:30 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.08 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.92 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (18.97 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.10 KB, patch)
2019-01-15 12:12 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.90 KB, patch)
2019-02-25 08:39 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.58 KB, patch)
2019-02-25 08:40 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2019-02-25 08:40 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.88 KB, patch)
2019-02-25 08:40 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (18.97 KB, patch)
2019-02-25 08:40 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.10 KB, patch)
2019-02-25 08:40 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.90 KB, patch)
2019-03-15 14:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.58 KB, patch)
2019-03-15 14:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2019-03-15 14:48 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.88 KB, patch)
2019-03-15 14:48 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (18.97 KB, patch)
2019-03-15 14:48 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.20 KB, patch)
2019-03-15 14:48 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.90 KB, patch)
2019-04-01 08:17 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.50 KB, patch)
2019-04-01 08:17 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.01 KB, patch)
2019-04-01 08:18 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.86 KB, patch)
2019-04-01 08:18 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (18.97 KB, patch)
2019-04-01 08:18 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.20 KB, patch)
2019-04-01 08:18 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (70.08 KB, patch)
2019-04-01 08:52 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.78 KB, patch)
2019-04-01 08:52 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.11 KB, patch)
2019-04-01 08:53 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (60.33 KB, patch)
2019-04-01 08:53 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.14 KB, patch)
2019-04-01 08:53 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.30 KB, patch)
2019-04-01 08:53 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.99 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.12 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (59.52 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.07 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.29 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bgu 11529: Update kohafield in default framework only if not already set. (3.80 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.17 KB, patch)
2019-04-12 14:07 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.06 KB, patch)
2019-04-12 14:08 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.99 KB, patch)
2019-04-18 06:28 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-04-18 06:28 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.12 KB, patch)
2019-04-18 06:28 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.27 KB, patch)
2019-04-18 06:28 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.07 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.29 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bgu 11529: Update kohafield in default framework only if not already set. (3.80 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.17 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.06 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.45 KB, patch)
2019-04-18 06:29 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.99 KB, patch)
2019-04-23 08:19 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.12 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.27 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.07 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.29 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.80 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.17 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.06 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.45 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (203.87 KB, patch)
2019-04-23 08:20 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.99 KB, patch)
2019-04-23 08:44 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-04-23 08:44 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.12 KB, patch)
2019-04-23 08:44 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.27 KB, patch)
2019-04-23 08:44 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.07 KB, patch)
2019-04-23 08:44 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.29 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.80 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.17 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.06 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.45 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (108.82 KB, patch)
2019-04-23 08:45 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (69.99 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.60 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.12 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.27 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.07 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.23 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.80 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.17 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.06 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.45 KB, patch)
2019-04-25 12:10 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (108.85 KB, patch)
2019-04-25 12:11 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (70.13 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.83 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.18 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.69 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.20 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.29 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.86 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.37 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.11 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.50 KB, patch)
2019-04-26 07:36 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (109.36 KB, patch)
2019-04-26 07:37 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: (follow-up) Fix link on hold request page (1.01 KB, patch)
2019-04-26 07:37 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (70.04 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.64 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.17 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.47 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.12 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.28 KB, patch)
2019-04-30 07:46 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.85 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.22 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.11 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.50 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (111.03 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Fix link on hold request page (1.01 KB, patch)
2019-04-30 07:47 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add new fields to biblio table (70.04 KB, patch)
2019-05-02 13:04 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Use new biblio fields whenever possible (37.64 KB, patch)
2019-05-02 13:04 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl (5.17 KB, patch)
2019-05-02 13:04 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Clean up subtitle usage (58.47 KB, patch)
2019-05-02 13:04 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove field mapping table and admin page (19.12 KB, patch)
2019-05-02 13:04 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: DBIx schema changes (4.28 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Update kohafield in default framework only if not already set. (3.85 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's. (57.22 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Remove duplicate column name from select query (1.11 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission. (5.50 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: Add templates for biblio title display. Unify display. (110.70 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Fix link on hold request page (1.01 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 11529: (follow-up) Fix QA issues (12.52 KB, patch)
2019-05-02 13:05 UTC, Ere Maijala
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle M Hall 2014-01-13 14:46:58 UTC
Bug 5652 added the ability to see the record subtitle from the checkout and holds summaries, but does so by loading and parsing the entire MARC record. This seems to be causing somewhat significant slowdowns on some Koha installations and the speed could be greatly improved by just adding it to the biblio table the same way the title is stored.
Comment 1 HB-NEKLS 2014-01-13 16:28:31 UTC
I want to add some additional information to this bug report -- the checkout and holds summaries now display the entire 245 MARC field -- not just subtitle (245$b). Information like the 245$h [gmd], 245$n (number of work) and 245$p part of work, also display -- which are very useful pieces of information, especially with graphic novels. But, loading this entire field in large systems does appear to cause page loading.  

I wonder if all 4 subfields need to be added to the biblio table, not just the 245$b subfield.
Comment 2 Liz Rea 2016-07-27 01:03:12 UTC
*** Bug 7715 has been marked as a duplicate of this bug. ***
Comment 3 Owen Leonard 2016-08-19 18:14:33 UTC
Adding a single field to the biblio table seems much too limiting. Would you combine all the 245 subfields > 'a' into one string and store it that way?

Is anyone thinking about a better way to do this?
Comment 4 Katrin Fischer 2016-09-05 06:23:07 UTC
I agree, a single subfield is not enough here. I think at least having hbnp would make sense.
Comment 5 Ere Maijala 2018-10-23 12:51:07 UTC
I suppose the only problem is that this needs to be driven by the bibliographic framework, and we can't assume that the subfields include punctuation. While it's possible to map multiple subfields to a single Koha field in the framework, there's no way to define punctuation, and the rules for that might become a bit complicated too. 

One option would be to actually add several columns such as:

title_remainder = b (in MARC21)
medium = h
part_number = n
part_name = p

Then the presentation could be left to whatever module uses the data. This would require more work from the presentation code, but it would also allow way more versatile display. What do you think?
Comment 6 Katrin Fischer 2018-10-26 13:13:50 UTC
I think we can map multiple, but they are not all added to the db (I think). I like the idea of separate fields.
Comment 7 Ere Maijala 2018-10-26 13:45:41 UTC
I think these days multiple values are concatenated, but with spaces and a pipe character, so it's not very friendly for display purposes. I'll see if I can come up with a patch.
Comment 8 Katrin Fischer 2018-10-28 19:14:46 UTC
I thought this was only for repeatable fields, but might be mistaken!
Comment 9 Ere Maijala 2018-11-05 12:23:09 UTC
Created attachment 81930 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 10 Ere Maijala 2018-11-05 12:23:14 UTC
Created attachment 81931 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 11 Ere Maijala 2018-11-05 12:35:03 UTC
This is pretty much work in progress but I posted so that it's not lost in any catastrophic event. :)

In any case, the first patch is pretty much done and the second is done but completely untested. For the record, I ended up naming the field for "remainder of title" subtitle for back-compatibility, as subtitle is the predominant term in the current index.

Adding the fields increases the space requirement for the biblio table, but the increase is pretty minor in the grand scheme. The only controversial thing, as far as I can see, is that retrieval of subtitle with this mechanism doesn't use the fieldmapping table ("Keyword to MARC mapping" in the UI) anymore. I don't see it as an issue since, as far as I can see, it's a mechanism that's only been used for the subtitle. So we can probably get rid of it completely here. I'll just need to finish the transition to get rid of the GetRecordValue method or make it work with the framework data.
Comment 12 Ere Maijala 2018-11-07 14:26:22 UTC
Created attachment 82024 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 13 Ere Maijala 2018-11-07 14:26:26 UTC
Created attachment 82025 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 14 Ere Maijala 2018-11-07 14:30:27 UTC
Created attachment 82026 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 15 Ere Maijala 2018-11-07 14:30:31 UTC
Created attachment 82027 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 16 Ere Maijala 2018-11-07 14:35:06 UTC
Created attachment 82028 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 17 Ere Maijala 2018-11-07 14:35:10 UTC
Created attachment 82029 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 18 Ere Maijala 2018-11-07 14:36:25 UTC
The latest version might actually work, at least to some extent. I still need to go through all the paths that have been changed.
Comment 19 Ere Maijala 2018-11-08 14:20:18 UTC
Created attachment 82088 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 20 Ere Maijala 2018-11-08 14:20:22 UTC
Created attachment 82089 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 21 Ere Maijala 2018-11-08 14:20:27 UTC
Created attachment 82090 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 22 Ere Maijala 2018-11-08 14:21:53 UTC
With the latest patches e.g. holds reports in circulation display the information properly.
Comment 23 Ere Maijala 2018-11-19 08:38:07 UTC
Created attachment 82428 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 24 Ere Maijala 2018-11-19 08:38:11 UTC
Created attachment 82429 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 25 Ere Maijala 2018-11-19 08:38:16 UTC
Created attachment 82430 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 26 Ere Maijala 2018-11-19 08:38:20 UTC
Created attachment 82431 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 27 David Cook 2018-11-19 23:25:26 UTC
Do we really want to add "medium" considering the 245$h [gmd] has a rival with the 336 and 338 fields?
Comment 28 Katrin Fischer 2018-11-20 06:20:30 UTC
$h is obsolete with RDA, but I don't think it hurts to have the column as RDA is not used everywhere.
Comment 29 Ere Maijala 2018-11-20 11:24:37 UTC
Created attachment 82538 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 30 Ere Maijala 2018-11-20 11:24:44 UTC
Created attachment 82539 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 31 Ere Maijala 2018-11-20 11:24:49 UTC
Created attachment 82540 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 32 Ere Maijala 2018-11-20 11:24:56 UTC
Created attachment 82541 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 33 Ere Maijala 2018-11-20 11:25:03 UTC
Created attachment 82542 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 34 Ere Maijala 2018-11-20 11:34:51 UTC
I believe this is now ready to be tested. 

I Initially attempted to keep the templates unchanged as much as possible, but there were multiple issues with the old subtitle field handling, so I ended up unifying the handling a bit too. 

Part name and number have been added to the hold specific lists where I know they're needed, but they can easily be added to other screens too if deemed necessary.

The addition medium field was based on a comment on the mailing list about the old functionality being used to include that one too. I don't have a strong opinion on it, but since the content is based on the framework, it could also be configured to be taken from the RDA fields.

Test plan:

1. Apply the patches
2. Run "perl misc/batchRebuildBiblioTables.pl -c" to populate the columns in biblio table
3. Verify that at least subtitle is properly displayed in Holds queue, Holds to pull and Holds awaiting pickup list as well as patron's checkouts list.
4. Verify that tests still pass.
Comment 35 David Cook 2018-11-21 02:48:44 UTC
(In reply to Katrin Fischer from comment #28)
> $h is obsolete with RDA, but I don't think it hurts to have the column as
> RDA is not used everywhere.

Since RDA isn't used everywhere, I was thinking maybe something more generic would be good to take it into account though. But maybe that's just a premature optimization.
Comment 36 Josef Moravec 2018-12-07 14:33:33 UTC
Hi Ere, it looks almost good just few thinks:

Holds to pull does not show data in table
Hold ratios does not work (see bellow)

Note, that changes in Koha::Schema files should be in its own commit - to avoid conflicts on hash

Also this is report from qa tools:

FAIL	circ/reserveratios.pl
   FAIL	  valid
		"my" variable @reservedata masks earlier declaration in same scope 
		Global symbol "$data" requires explicit package name (did you forget to declare "my $data"?) 
		"my" variable $rd masks earlier declaration in same statement 
		circ/reserveratios.pl has too many errors.
		Global symbol "$thisratio" requires explicit package name (did you forget to declare "my $thisratio"?) 
		syntax error 


FAIL	koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/detail.tt
   FAIL	  forbidden patterns
		forbidden pattern: tab char (line 156)


FAIL	opac/opac-showreviews.pl
   FAIL	  forbidden patterns
		forbidden pattern: tab char (line 100)
		forbidden pattern: tab char (line 101)
		forbidden pattern: tab char (line 99)
		forbidden pattern: tab char (line 102)
Comment 37 Ere Maijala 2018-12-12 12:29:50 UTC
Created attachment 83103 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 38 Ere Maijala 2018-12-12 12:29:55 UTC
Created attachment 83104 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 39 Ere Maijala 2018-12-12 12:29:59 UTC
Created attachment 83105 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 40 Ere Maijala 2018-12-12 12:30:05 UTC
Created attachment 83106 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 41 Ere Maijala 2018-12-12 12:30:12 UTC
Created attachment 83107 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 42 Ere Maijala 2018-12-12 12:30:18 UTC
Created attachment 83108 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.
Comment 43 Josef Moravec 2019-01-15 10:56:34 UTC
Comment on attachment 83106 [details] [review]
Bug 11529: Clean up subtitle usage

Review of attachment 83106 [details] [review]:
-----------------------------------------------------------------

::: koha-tmpl/intranet-tmpl/prog/js/checkouts.js
@@ +244,3 @@
>                          });
>  
> +                        title += " " + oObj.part_number + " " + oObj.part_name;

If there is no part_number or part_name, there is "null" string shown in UI

@@ +601,3 @@
>                              });
>  
> +                            title += " " + oObj.part_number + " " + oObj.part_name;

If there is no part_number or part_name, there is "null" string shown in UI
Comment 44 Ere Maijala 2019-01-15 12:12:30 UTC
Created attachment 83961 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 45 Ere Maijala 2019-01-15 12:12:35 UTC
Created attachment 83962 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 46 Ere Maijala 2019-01-15 12:12:39 UTC
Created attachment 83963 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 47 Ere Maijala 2019-01-15 12:12:43 UTC
Created attachment 83964 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 48 Ere Maijala 2019-01-15 12:12:48 UTC
Created attachment 83965 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 49 Ere Maijala 2019-01-15 12:12:52 UTC
Created attachment 83966 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.
Comment 50 Ere Maijala 2019-01-15 12:13:12 UTC
Thanks Josef, fixed and rebased.
Comment 51 Ere Maijala 2019-01-16 07:51:48 UTC
*** Bug 18408 has been marked as a duplicate of this bug. ***
Comment 52 Fridolin SOMERS 2019-02-22 14:22:28 UTC
*** Bug 20403 has been marked as a duplicate of this bug. ***
Comment 53 Ere Maijala 2019-02-25 08:39:58 UTC
Created attachment 85587 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 54 Ere Maijala 2019-02-25 08:40:03 UTC
Created attachment 85588 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 55 Ere Maijala 2019-02-25 08:40:08 UTC
Created attachment 85589 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 56 Ere Maijala 2019-02-25 08:40:12 UTC
Created attachment 85590 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 57 Ere Maijala 2019-02-25 08:40:17 UTC
Created attachment 85591 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 58 Ere Maijala 2019-02-25 08:40:22 UTC
Created attachment 85592 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.
Comment 59 Ere Maijala 2019-02-25 08:40:57 UTC
Rebased and made tests more robust.
Comment 60 Michal Denar 2019-03-15 14:37:19 UTC
Hi Ere,
i tested patches i get this error:
Falling back to patching base and 3-way merge...
Removing Koha/Schema/Result/Fieldmapping.pm
Auto-merging Koha/Schema/Result/Biblio.pm
CONFLICT (content): Merge conflict in Koha/Schema/Result/Biblio.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 11529: DBIx schema changes

Can you rebase?
Comment 61 Ere Maijala 2019-03-15 14:47:49 UTC
Created attachment 86677 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 62 Ere Maijala 2019-03-15 14:47:54 UTC
Created attachment 86678 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 63 Ere Maijala 2019-03-15 14:48:00 UTC
Created attachment 86679 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 64 Ere Maijala 2019-03-15 14:48:05 UTC
Created attachment 86680 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 65 Ere Maijala 2019-03-15 14:48:10 UTC
Created attachment 86681 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 66 Ere Maijala 2019-03-15 14:48:15 UTC
Created attachment 86682 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.
Comment 67 Ere Maijala 2019-03-15 14:48:45 UTC
Thanks Michal, now rebased.
Comment 68 Michal Denar 2019-03-30 20:09:43 UTC
Hi Ere,
I'm not sure about these results:

Holds at Borrower detail: see "null"
http://prntscr.com/n56znx

Holds to Pull: 
http://prntscr.com/n56xmj

Maybe something is wrong with my catalogue template or records.

Michal
Comment 69 Ere Maijala 2019-04-01 08:17:52 UTC
Created attachment 87248 [details] [review]
Bug 11529: Add new fields to biblio table
Comment 70 Ere Maijala 2019-04-01 08:17:57 UTC
Created attachment 87249 [details] [review]
Bug 11529: Use new biblio fields whenever possible
Comment 71 Ere Maijala 2019-04-01 08:18:02 UTC
Created attachment 87250 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Comment 72 Ere Maijala 2019-04-01 08:18:07 UTC
Created attachment 87251 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.
Comment 73 Ere Maijala 2019-04-01 08:18:12 UTC
Created attachment 87252 [details] [review]
Bug 11529: Remove field mapping table and admin page
Comment 74 Ere Maijala 2019-04-01 08:18:16 UTC
Created attachment 87253 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.
Comment 75 Ere Maijala 2019-04-01 08:20:15 UTC
Thanks Michal. Both issues should now be fixed. The displaying of null happened if part_number and part_name in database were null, which is possible if there's no mapping for them or the batch rebuild has not been run. Now the service guards against that.
Comment 76 Michal Denar 2019-04-01 08:52:51 UTC
Created attachment 87256 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 77 Michal Denar 2019-04-01 08:52:57 UTC
Created attachment 87257 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 78 Michal Denar 2019-04-01 08:53:02 UTC
Created attachment 87258 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 79 Michal Denar 2019-04-01 08:53:08 UTC
Created attachment 87259 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 80 Michal Denar 2019-04-01 08:53:13 UTC
Created attachment 87260 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 81 Michal Denar 2019-04-01 08:53:18 UTC
Created attachment 87261 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 82 Katrin Fischer 2019-04-12 11:55:34 UTC
Leaving some first notes here. Ere, can you please have a look? Nothing that keeps me from testing more for now, but it will help to have follow-ups and answers :)

I Code review

Good:
- QA test tools pass
- Clean database update
- New tests pass


Bad (blocker):
- git grep bibliosubtitle.subtitle still shows several occurrences to be fixed
- Can you explain the use of $raw on the subtitle in koha-tmpl/intranet-tmpl/prog/en/modules/virtualshelves/sendshelf.tt and
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-sendshelf.tt

Non-blocker notes:
- Need to leave a note on Taiga/bug here that the Koha-to-keyword content needs to be removed from the manual
- Will need extensive release notes notes.

Pre-existing:
- GetHoldsQueueItems: biblioitems.publicationsyear is listed twice in the SQL query
- Why so many occurrences of [% subtitl %]?
- The ',' between subtitles will in opac-results.pl, opac-showreviews-rss.tt will look not work for MARC21 libraries using proper ISBD punctuation in their records.


II Questions:

- In the database update we don't check if libraries had already mapped the fields to other database columns. The remapping might leave data and reports orphaned. Should we check for this and at least print a warning?

- What is our recommendation on how to populate the new db fields? Can we imagine something that works faster than touch_all_biblios.pl? SQL updates?  
What's the difference between touch_all_biblios.pl and batchRebuildBiblioTables.pl?

- Why was the -t test option removed from batchRebuildBiblioTables.pl?

- AFAIK UNIMARC doesn't have ISBD punctuation in the records. Do we need to introduce a separator for UNIMARC between title and subtitle?
There is one place where we already seem to deal with this in opac-detail.pl:

<title>[% IF ( LibraryNameTitle ) %][% LibraryNameTitle | html %][% ELSE %]Koha online[% END %] catalog &rsaquo; Details for: [% title | html %][% FOREACH subtitl IN subtitle %][% IF Koha.Preference('marcflavour')=='UNIMARC' %],[% END %] [% subtitl | html %][% END %]</title>
Comment 83 Ere Maijala 2019-04-12 14:07:17 UTC
Created attachment 87911 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 84 Ere Maijala 2019-04-12 14:07:23 UTC
Created attachment 87912 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 85 Ere Maijala 2019-04-12 14:07:29 UTC
Created attachment 87913 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 86 Ere Maijala 2019-04-12 14:07:34 UTC
Created attachment 87914 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 87 Ere Maijala 2019-04-12 14:07:39 UTC
Created attachment 87915 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 88 Ere Maijala 2019-04-12 14:07:44 UTC
Created attachment 87916 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 89 Ere Maijala 2019-04-12 14:07:50 UTC
Created attachment 87917 [details] [review]
Bgu 11529: Update kohafield in default framework only if not already set.
Comment 90 Ere Maijala 2019-04-12 14:07:55 UTC
Created attachment 87918 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.
Comment 91 Ere Maijala 2019-04-12 14:08:01 UTC
Created attachment 87919 [details] [review]
Bug 11529: Remove duplicate column name from select query
Comment 92 Ere Maijala 2019-04-12 14:23:44 UTC
(In reply to Katrin Fischer from comment #82)
> Bad (blocker):
> - git grep bibliosubtitle.subtitle still shows several occurrences to be
> fixed

Fixed.

> - Can you explain the use of $raw on the subtitle in
> koha-tmpl/intranet-tmpl/prog/en/modules/virtualshelves/sendshelf.tt and
> koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-sendshelf.tt

I suppose it's because of the html2text filter, but I just kept what there already was. 

> Non-blocker notes:
> - Need to leave a note on Taiga/bug here that the Koha-to-keyword content
> needs to be removed from the manual

Can I access Taiga somewhere or is it better to just file a bug?

> - Will need extensive release notes notes.

I'll try to come up with something.

> Pre-existing:
> - GetHoldsQueueItems: biblioitems.publicationsyear is listed twice in the
> SQL query

Fixed here.

> - Why so many occurrences of [% subtitl %]?

I've kept as much of the code intact as possible. Since the old code used that, I've kept it intact. Also, we concatenate multiple values in the database with ' | ', so splitting and handling them one by one allows one to customize the display more. 

> - The ',' between subtitles will in opac-results.pl, opac-showreviews-rss.tt
> will look not work for MARC21 libraries using proper ISBD punctuation in
> their records.

I agree, but I believe that hasn't changed here. It could be handled as a separate issue.

> II Questions:
> 
> - In the database update we don't check if libraries had already mapped the
> fields to other database columns. The remapping might leave data and reports
> orphaned. Should we check for this and at least print a warning?

Good point, check added.

> - What is our recommendation on how to populate the new db fields? Can we
> imagine something that works faster than touch_all_biblios.pl? SQL updates?  
> What's the difference between touch_all_biblios.pl and
> batchRebuildBiblioTables.pl?

batchRebuildBiblioTables.pl is recommended. touch_all_biblios.pl also indexes all changes, which isn't necessary here.

> - Why was the -t test option removed from batchRebuildBiblioTables.pl?

It served no purpose. There was no useful output or anything that I could think of to use it for. 

> - AFAIK UNIMARC doesn't have ISBD punctuation in the records. Do we need to
> introduce a separator for UNIMARC between title and subtitle?

That'd probably be nice, but I believe that could be done in a separate bug. Since most of the existing templates didn't have that, I don't think it's vital. Also, I believe the code in opac-detail could be improved if comma is not the best separator.
Comment 93 Katrin Fischer 2019-04-18 05:46:45 UTC
Hi Ere, I am sorry, I waited to long to continue here. Could you rebase it please? I will try to come back to it as fast ass possible after (faster than last time!)
Comment 94 Katrin Fischer 2019-04-18 05:49:27 UTC
Managed the rebase myself - the change was caused by the non-XSLT view being removed.
Comment 95 Katrin Fischer 2019-04-18 05:52:50 UTC
Working here...
Comment 96 Katrin Fischer 2019-04-18 05:59:04 UTC
I'll keep testing, but we need another patch:
- Remove  manage_keywords2koha_mappings from display and patron accounts
Comment 97 Ere Maijala 2019-04-18 06:17:00 UTC
Right, I didn't notice the permission before. I'll add a follow-up.
Comment 98 Katrin Fischer 2019-04-18 06:28:13 UTC
I have another big issue here... In our installations we have mapped subtitle to 245cnp for ages now, so all those subfields show in a lot of places. With this patch set, I am afraid we will see a major regression that won't be fixable by configuration.
In a lot of places, only the subtitle c will be displayed. 

I think the thinking error here is that subtitle was only used for displaying c before, but I think this is not the case.

Examples are:
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt            |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-readingrecord.tt         |  6 +--
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-reserve.tt               |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-sendshelf.tt             |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-shelves.tt               |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-showreviews-rss.tt       |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-showreviews.tt           |  2 +-
koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-user.tt
Comment 99 Ere Maijala 2019-04-18 06:28:38 UTC
Created attachment 88247 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 100 Ere Maijala 2019-04-18 06:28:44 UTC
Created attachment 88248 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 101 Ere Maijala 2019-04-18 06:28:50 UTC
Created attachment 88249 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 102 Ere Maijala 2019-04-18 06:28:56 UTC
Created attachment 88250 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 103 Ere Maijala 2019-04-18 06:29:01 UTC
Created attachment 88251 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 104 Ere Maijala 2019-04-18 06:29:07 UTC
Created attachment 88252 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 105 Ere Maijala 2019-04-18 06:29:12 UTC
Created attachment 88253 [details] [review]
Bgu 11529: Update kohafield in default framework only if not already set.
Comment 106 Ere Maijala 2019-04-18 06:29:18 UTC
Created attachment 88254 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.
Comment 107 Ere Maijala 2019-04-18 06:29:23 UTC
Created attachment 88255 [details] [review]
Bug 11529: Remove duplicate column name from select query
Comment 108 Ere Maijala 2019-04-18 06:29:29 UTC
Created attachment 88256 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.
Comment 109 Katrin Fischer 2019-04-18 06:30:41 UTC
See comment #98 :(
Comment 110 Ere Maijala 2019-04-18 06:33:22 UTC
Rebased and added a follow-up to remove the permission.

Katrin, needing subfields n and p was anticipated, but I didn't realize they were needed in so many places. There's nothing preventing from using the new subtitle for all of them like before, but it probably makes more sense to include part_number and part_name explicitly in the templates you mentioned. I'll try to do that now.
Comment 111 Katrin Fischer 2019-04-18 06:34:34 UTC
(In reply to Ere Maijala from comment #110)
> Rebased and added a follow-up to remove the permission.
> 
> Katrin, needing subfields n and p was anticipated, but I didn't realize they
> were needed in so many places. There's nothing preventing from using the new
> subtitle for all of them like before, but it probably makes more sense to
> include part_number and part_name explicitly in the templates you mentioned.
> I'll try to do that now.

Sorry, not sure I understand the first part "using the new subtitle"?
Comment 112 Ere Maijala 2019-04-18 06:36:35 UTC
I mean you could map 245n and 245p to the biblio.subtitle field in the MARC framework, but since these changes introduce the biblio.part_name and biblio.part_number fields, it makes sense to actually bring them into the templates.
Comment 113 Katrin Fischer 2019-04-18 07:17:54 UTC
(In reply to Ere Maijala from comment #112)
> I mean you could map 245n and 245p to the biblio.subtitle field in the MARC
> framework, but since these changes introduce the biblio.part_name and
> biblio.part_number fields, it makes sense to actually bring them into the
> templates.

I am not sure that would work in the expected way - it would not make all the information go into the field, but I think only one of it. I haven't tested, but that's how I remember behaviour for mapping both 260/264.
Comment 114 Ere Maijala 2019-04-18 07:26:57 UTC
(In reply to Katrin Fischer from comment #113)
> (In reply to Ere Maijala from comment #112)
> > I mean you could map 245n and 245p to the biblio.subtitle field in the MARC
> > framework, but since these changes introduce the biblio.part_name and
> > biblio.part_number fields, it makes sense to actually bring them into the
> > templates.
> 
> I am not sure that would work in the expected way - it would not make all
> the information go into the field, but I think only one of it. I haven't
> tested, but that's how I remember behaviour for mapping both 260/264.

It used to be that way, but nowadays it will put everything into the field separated by ' | '. And since the splitting is done for the subtitle field, it's displayed one by one just like before. But that's still a not-so-great hack, so I'll make sure the part fields are displayed as well.
Comment 115 Josef Moravec 2019-04-18 07:56:10 UTC
Maybe some include template file could help to improve and standardize the way we are displaying titles and links to record in UI. We have similar one for patrons - patron-title.inc
Comment 116 Ere Maijala 2019-04-18 07:57:42 UTC
Yes, that makes sense.
Comment 117 Katrin Fischer 2019-04-18 08:02:54 UTC
(In reply to Josef Moravec from comment #115)
> Maybe some include template file could help to improve and standardize the
> way we are displaying titles and links to record in UI. We have similar one
> for patrons - patron-title.inc

+1 

Was thinking the same, it would also make our life much easier if we want to add punctuation for UNIMARC later on.
Comment 118 Ere Maijala 2019-04-18 10:13:02 UTC
Ok, I'm going for a bit more cleanup. My original approach was to minimize template changes, but since they're needed anyway, there's no point trying to hack it.
Comment 119 Ere Maijala 2019-04-23 08:19:55 UTC
Created attachment 88406 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 120 Ere Maijala 2019-04-23 08:20:00 UTC
Created attachment 88407 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 121 Ere Maijala 2019-04-23 08:20:06 UTC
Created attachment 88408 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 122 Ere Maijala 2019-04-23 08:20:11 UTC
Created attachment 88409 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 123 Ere Maijala 2019-04-23 08:20:16 UTC
Created attachment 88410 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 124 Ere Maijala 2019-04-23 08:20:21 UTC
Created attachment 88411 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 125 Ere Maijala 2019-04-23 08:20:26 UTC
Created attachment 88412 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.
Comment 126 Ere Maijala 2019-04-23 08:20:30 UTC
Created attachment 88413 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.
Comment 127 Ere Maijala 2019-04-23 08:20:35 UTC
Created attachment 88414 [details] [review]
Bug 11529: Remove duplicate column name from select query
Comment 128 Ere Maijala 2019-04-23 08:20:40 UTC
Created attachment 88415 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.
Comment 129 Ere Maijala 2019-04-23 08:20:46 UTC
Created attachment 88416 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.
Comment 130 Ere Maijala 2019-04-23 08:25:09 UTC
Okay, so this became a bit bigger set of changes and I'm not sure how to ever get this in since I can imagine the last patch rotting away like no other. Anyway, this is an attempt to use an include file for any time a biblio title is to be displayed. There are separate includes for the title field in head and any other stuff in html. There are also classes added to each part. This allows one to style them independently and to e.g. easily hide part information from breadcrumbs if that's necessary.

I've tried to find all occasions where the title is displayed, and I've also tried to test them, but there may well be something that I've missed.
Comment 131 Katrin Fischer 2019-04-23 08:36:33 UTC
Hi Ere, 
I am willing to test - even if it looks scary. Changing everywhere title shows... you mean changed everywhere subtitle was used, I hope? (just aiming for no regression right now)

One small change to make before I start:
staff-global.css should not be in there. Can you please kick it out?
Comment 132 Ere Maijala 2019-04-23 08:44:38 UTC
Created attachment 88433 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 133 Ere Maijala 2019-04-23 08:44:43 UTC
Created attachment 88434 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 134 Ere Maijala 2019-04-23 08:44:48 UTC
Created attachment 88435 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 135 Ere Maijala 2019-04-23 08:44:53 UTC
Created attachment 88436 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 136 Ere Maijala 2019-04-23 08:44:58 UTC
Created attachment 88437 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 137 Ere Maijala 2019-04-23 08:45:03 UTC
Created attachment 88438 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 138 Ere Maijala 2019-04-23 08:45:08 UTC
Created attachment 88439 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.
Comment 139 Ere Maijala 2019-04-23 08:45:14 UTC
Created attachment 88440 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.
Comment 140 Ere Maijala 2019-04-23 08:45:19 UTC
Created attachment 88441 [details] [review]
Bug 11529: Remove duplicate column name from select query
Comment 141 Ere Maijala 2019-04-23 08:45:23 UTC
Created attachment 88442 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.
Comment 142 Ere Maijala 2019-04-23 08:45:28 UTC
Created attachment 88443 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.
Comment 143 Ere Maijala 2019-04-23 08:48:15 UTC
Removed staff-global.css changes.

Katrin, I've also added the biblio-title include to places where it's logical to use it and wasn't before. E.g. it makes no sense to display subtitle for checkouts but not for article requests, so I've done my best to unify the related displays. Same with e.g. the sub-pages of catalog record (holds, article requests, checkout history etc.).
Comment 144 Michal Denar 2019-04-24 18:52:26 UTC
Hi Ere,
I get error:
Applying: Bgu 11529: Update kohafield in default framework only if not already set.
Using index info to reconstruct a base tree...
A	installer/data/mysql/atomicupdate/bug_11529.perl
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): installer/data/mysql/atomicupdate/bug_11529.perl deleted in HEAD and modified in Bgu 11529: Update kohafield in default framework only if not already set.. Version Bgu 11529: Update kohafield in default framework only if not already set. of installer/data/mysql/atomicupdate/bug_11529.perl left in tree.
error: Failed to merge in the changes.

Please, can you rebase?
Comment 145 Ere Maijala 2019-04-25 07:26:57 UTC
Oops, sorry. There was an older version of one of the patches left over because of a typo in the title. I obsoleted it so it should be better now.
Comment 146 Ere Maijala 2019-04-25 12:10:01 UTC
Created attachment 88762 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 147 Ere Maijala 2019-04-25 12:10:07 UTC
Created attachment 88763 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 148 Ere Maijala 2019-04-25 12:10:13 UTC
Created attachment 88764 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 149 Ere Maijala 2019-04-25 12:10:19 UTC
Created attachment 88765 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 150 Ere Maijala 2019-04-25 12:10:25 UTC
Created attachment 88766 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 151 Ere Maijala 2019-04-25 12:10:30 UTC
Created attachment 88767 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 152 Ere Maijala 2019-04-25 12:10:36 UTC
Created attachment 88768 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.
Comment 153 Ere Maijala 2019-04-25 12:10:44 UTC
Created attachment 88769 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.
Comment 154 Ere Maijala 2019-04-25 12:10:50 UTC
Created attachment 88770 [details] [review]
Bug 11529: Remove duplicate column name from select query
Comment 155 Ere Maijala 2019-04-25 12:10:57 UTC
Created attachment 88771 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.
Comment 156 Ere Maijala 2019-04-25 12:11:05 UTC
Created attachment 88772 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.
Comment 157 Ere Maijala 2019-04-25 12:11:33 UTC
Rebased again.
Comment 158 Josef Moravec 2019-04-26 07:36:05 UTC
Created attachment 88851 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 159 Josef Moravec 2019-04-26 07:36:12 UTC
Created attachment 88852 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 160 Josef Moravec 2019-04-26 07:36:17 UTC
Created attachment 88853 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 161 Josef Moravec 2019-04-26 07:36:23 UTC
Created attachment 88854 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 162 Josef Moravec 2019-04-26 07:36:28 UTC
Created attachment 88855 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 163 Josef Moravec 2019-04-26 07:36:33 UTC
Created attachment 88856 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 164 Josef Moravec 2019-04-26 07:36:38 UTC
Created attachment 88857 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 165 Josef Moravec 2019-04-26 07:36:44 UTC
Created attachment 88858 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 166 Josef Moravec 2019-04-26 07:36:50 UTC
Created attachment 88859 [details] [review]
Bug 11529: Remove duplicate column name from select query

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 167 Josef Moravec 2019-04-26 07:36:55 UTC
Created attachment 88860 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 168 Josef Moravec 2019-04-26 07:37:01 UTC
Created attachment 88861 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 169 Josef Moravec 2019-04-26 07:37:06 UTC
Created attachment 88862 [details] [review]
Bug 11529: (follow-up) Fix link on hold request page

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 170 Katrin Fischer 2019-04-30 06:01:18 UTC
I am back on this, but sadly it doesn't apply :( 
Please fix, I would have time tomorrow for testing.

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 11529: Add new fields to biblio table
Applying: Bug 11529: Use new biblio fields whenever possible
.git/rebase-apply/patch:390: trailing whitespace.
        count(DISTINCT reserves.borrowernumber) as reservecount, 
error: sha1 information is lacking or useless (C4/Biblio.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 11529: Use new biblio fields whenever possible
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-11529-Use-new-biblio-fields-whenever-possible-TtxGN1.patch
Comment 171 Ere Maijala 2019-04-30 07:46:28 UTC
Created attachment 89096 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 172 Ere Maijala 2019-04-30 07:46:34 UTC
Created attachment 89097 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 173 Ere Maijala 2019-04-30 07:46:39 UTC
Created attachment 89098 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 174 Ere Maijala 2019-04-30 07:46:45 UTC
Created attachment 89099 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 175 Ere Maijala 2019-04-30 07:46:50 UTC
Created attachment 89100 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 176 Ere Maijala 2019-04-30 07:46:55 UTC
Created attachment 89101 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 177 Ere Maijala 2019-04-30 07:47:01 UTC
Created attachment 89102 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 178 Ere Maijala 2019-04-30 07:47:06 UTC
Created attachment 89103 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 179 Ere Maijala 2019-04-30 07:47:12 UTC
Created attachment 89104 [details] [review]
Bug 11529: Remove duplicate column name from select query

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 180 Ere Maijala 2019-04-30 07:47:17 UTC
Created attachment 89105 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 181 Ere Maijala 2019-04-30 07:47:23 UTC
Created attachment 89106 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 182 Ere Maijala 2019-04-30 07:47:28 UTC
Created attachment 89107 [details] [review]
Bug 11529: (follow-up) Fix link on hold request page

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 183 Ere Maijala 2019-04-30 07:48:02 UTC
Rebased.
Comment 184 Katrin Fischer 2019-04-30 08:49:35 UTC
Thank you!
Comment 185 Katrin Fischer 2019-05-01 10:33:23 UTC
Embarking on my testing journey here... wish me luck!
Comment 186 Katrin Fischer 2019-05-01 10:45:23 UTC
Patches apply, QA tests pass... more later.
Comment 187 Katrin Fischer 2019-05-01 13:00:09 UTC
1) RELEASE NOTES

- Removes the Koha to Keyword mappings
- Might cause change in behaviour for people having mappign fields outside of 245bhnp or UNIMARC equivalents
- Requires you to run misc/batchRebuildBiblioTables.pl to have 245bhnp displayed

2) Template Toolkit Stash

I noticed some of the files use Stash. I haven't come across this so far. Can you explain?

koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-detail.tt
+[% USE Stash %]
Details for <i>[% INCLUDE 'biblio-title.inc' biblio=Stash.stash() %]</i>

This is a problem, because at the moment this is only used for debugging and
not marked as required in the Dependencies. Adding dependency tag to the bug.

3) SplitKohaField
 
I wonder if the use case warrants its own subroutine, it's only used in
a few places and for splitting a pipe separeted string into an array:

C4/Biblio.pm:sub SplitKohaField {
svc/checkouts:        subtitle             => C4::Biblio::SplitKohaField($c->{'subtitle'}),
svc/holds:use C4::Biblio qw(SplitKohaField);
svc/holds:        subtitle       => C4::Biblio::SplitKohaField($biblio->subtitle()),

4) Template variables

Right now the changes require a lot of new variables passed to the templates
and some older SQL methods to be extended to SELECT additional fields.
I wonder if instead of working with an INCLUDE working with a TT plugin 
taking the biblionumber and querying would hurt performance much. It would allow
us to remove quite a bit of code I think. Also I guess in the future working with
a Record object might make things easier?

5) misc/batchRebuildBiblioTables.pl

I noticed that you removed the table locking. Is this safe?

6) Unrelated changes

Small one:
koha-tmpl/intranet-tmpl/prog/en/includes/biblio-view-menu.inc

And:
t/db_dependent/Biblio.t
-    my $authorities_mod = Test::MockModule->new( 'C4::AuthoritiesMarc' );
+    my $authorities_mod = Test::MockModule->new( 'C4::Heading' );

7) Repeatable subfields np

Testing with: 245abhnpnp
Multiple np show as: 
The C programming language / Zusatz Number | Number2 Part | Part2 

Actually this is a deeper rabbit hole... as we don't keep the sequence we might lose meaning here, but that would not have worked before.

Maybe this would make things a little easier?
http://template-toolkit.org/docs/manual/VMethods.html#method_split

8) Good news

- Removal of permission complete
- Removal of Koha to keywords feature complete
- Removal of GetRecordValue and Biblio:subtitles complete
Comment 188 Katrin Fischer 2019-05-02 05:15:08 UTC
I am sorry, but after testing I feel we probably should aim for early 19.11 instead of 19.05. While the basic change is not that complicated, in the sum the changes are pretty heavy.
Comment 189 Ere Maijala 2019-05-02 13:04:32 UTC
Created attachment 89234 [details] [review]
Bug 11529: Add new fields to biblio table

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 190 Ere Maijala 2019-05-02 13:04:39 UTC
Created attachment 89235 [details] [review]
Bug 11529: Use new biblio fields whenever possible

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 191 Ere Maijala 2019-05-02 13:04:44 UTC
Created attachment 89236 [details] [review]
Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 192 Ere Maijala 2019-05-02 13:04:50 UTC
Created attachment 89237 [details] [review]
Bug 11529: Clean up subtitle usage

Unify and clean up subtitle usage so that it's always used as a simple array and not the old hash structure.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 193 Ere Maijala 2019-05-02 13:04:55 UTC
Created attachment 89238 [details] [review]
Bug 11529: Remove field mapping table and admin page

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 194 Ere Maijala 2019-05-02 13:05:01 UTC
Created attachment 89239 [details] [review]
Bug 11529: DBIx schema changes

Updates to Biblio schema and removal of FieldMapping schema.

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 195 Ere Maijala 2019-05-02 13:05:06 UTC
Created attachment 89240 [details] [review]
Bug 11529: Update kohafield in default framework only if not already set.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 196 Ere Maijala 2019-05-02 13:05:12 UTC
Created attachment 89241 [details] [review]
Bug 11529: Replace bibliosubtitle with biblio in all framework sql's.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 197 Ere Maijala 2019-05-02 13:05:17 UTC
Created attachment 89242 [details] [review]
Bug 11529: Remove duplicate column name from select query

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 198 Ere Maijala 2019-05-02 13:05:22 UTC
Created attachment 89243 [details] [review]
Bug 11529: (follow-up) Remove manage_keywords2koha_mappings permission.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 199 Ere Maijala 2019-05-02 13:05:28 UTC
Created attachment 89244 [details] [review]
Bug 11529: Add templates for biblio title display. Unify display.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 200 Ere Maijala 2019-05-02 13:05:33 UTC
Created attachment 89245 [details] [review]
Bug 11529: (follow-up) Fix link on hold request page

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 201 Ere Maijala 2019-05-02 13:05:39 UTC
Created attachment 89246 [details] [review]
Bug 11529: (follow-up) Fix QA issues

- Remove SplitKohaField
- Avoid using Stash in templates
- Improved display of part fields
Comment 202 Ere Maijala 2019-05-02 13:13:43 UTC
(In reply to Katrin Fischer from comment #187)

Thanks, Katrin. Comments below.

> 1) RELEASE NOTES
> 
> - Removes the Koha to Keyword mappings
> - Might cause change in behaviour for people having mappign fields outside
> of 245bhnp or UNIMARC equivalents
> - Requires you to run misc/batchRebuildBiblioTables.pl to have 245bhnp
> displayed

Do you want me to clarify these in the proposed release notes?

> 2) Template Toolkit Stash
> 
> I noticed some of the files use Stash. I haven't come across this so far.
> Can you explain?

Ah, I didn't notice it's only for development. I just used it to collect all the template params to a hash, but changed the code to pass biblio to the template in one case (the other actually already had it).

> 3) SplitKohaField
>  
> I wonder if the use case warrants its own subroutine, it's only used in
> a few places and for splitting a pipe separeted string into an array:

You're right. Now removed.

> 4) Template variables
> 
> Right now the changes require a lot of new variables passed to the templates
> and some older SQL methods to be extended to SELECT additional fields.
> I wonder if instead of working with an INCLUDE working with a TT plugin 
> taking the biblionumber and querying would hurt performance much. It would
> allow
> us to remove quite a bit of code I think. Also I guess in the future working
> with
> a Record object might make things easier?

It's true, but the primary aim was to improve performance, and while fetching the data for a single record is not that bad, it's not optimal either, and when there's a list of records of any kind it can be significant. In some cases we could just pass the biblio object instead of select pieces of it, but I feel that's something that could be addressed in a follow-up issue, since this is usually just extending the existing practice.

> 5) misc/batchRebuildBiblioTables.pl
> 
> I noticed that you removed the table locking. Is this safe?

It was only commented-out code that I cleaned up.

> 6) Unrelated changes

Fixed.

> 7) Repeatable subfields np
> 
> Testing with: 245abhnpnp
> Multiple np show as: 
> The C programming language / Zusatz Number | Number2 Part | Part2 
> 
> Actually this is a deeper rabbit hole... as we don't keep the sequence we
> might lose meaning here, but that would not have worked before.

I made the templates handle repeated fields a bit better. At least I think they would normally be interleaved, and that's how they're now displayed. It's perhaps still not perfect, but at least it works properly with ISBD punctuation, and further tweaks are easy to add later as necessary.

Please note that qa-test-tools is not happy about the WHILE statement in the biblio-title* templates, but that seems to be a spurious error.
Comment 203 Ere Maijala 2019-05-02 13:15:01 UTC
(In reply to Katrin Fischer from comment #188)
> I am sorry, but after testing I feel we probably should aim for early 19.11
> instead of 19.05. While the basic change is not that complicated, in the sum
> the changes are pretty heavy.

I'm fine with that. If we can get it in early in the cycle, it's of course easier and any rough edges can be found.
Comment 204 Hayley Mapley 2019-06-18 04:33:46 UTC
Patch doesn't apply:

Applying: Bug 11529: Add new fields to biblio table
Applying: Bug 11529: Use new biblio fields whenever possible
Applying: Bug 11529: Simplify and optimize batchRebuildBiblioTables.pl
Applying: Bug 11529: Clean up subtitle usage
error: sha1 information is lacking or useless (C4/Biblio.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 11529: Clean up subtitle usage