Bug 15219 - Server-side processing and pagination on checkouts tables
Summary: Server-side processing and pagination on checkouts tables
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Circulation (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Julian Maurice
QA Contact: Testopia
URL:
Keywords:
Depends on: 11703 22921
Blocks: 15235
  Show dependency treegraph
 
Reported: 2015-11-19 15:10 UTC by Julian Maurice
Modified: 2023-08-28 19:32 UTC (History)
10 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 15219: Load checkouts table incrementally (43.62 KB, patch)
2015-11-19 15:11 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix content of "Export" column (988 bytes, patch)
2015-11-20 09:36 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix column settings in table moremember/issues-table (843 bytes, patch)
2015-11-20 14:48 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (58.24 KB, patch)
2016-02-16 10:33 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (58.64 KB, patch)
2016-04-08 12:32 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (58.46 KB, patch)
2017-02-09 13:59 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (23.02 KB, patch)
2017-02-24 09:10 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (57.97 KB, patch)
2017-03-16 08:58 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (22.72 KB, patch)
2017-03-16 08:58 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (57.94 KB, patch)
2017-03-23 14:56 UTC, Biblibre Sandboxes
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (22.77 KB, patch)
2017-03-23 14:56 UTC, Biblibre Sandboxes
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (61.36 KB, patch)
2018-03-15 08:31 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (24.10 KB, patch)
2018-03-15 08:32 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (60.51 KB, patch)
2018-04-16 09:10 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (24.10 KB, patch)
2018-04-16 09:10 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (62.35 KB, patch)
2018-06-08 08:55 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (24.10 KB, patch)
2018-06-08 08:55 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix relatives issues table (1.39 KB, patch)
2018-06-08 08:55 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (65.50 KB, patch)
2018-09-11 15:19 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (27.34 KB, patch)
2018-09-11 15:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix relatives issues table (1.39 KB, patch)
2018-09-11 15:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Do not call escapeHtml on null values (2.49 KB, patch)
2018-09-11 15:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Reword "select all" links since they select only visible rows (2.03 KB, patch)
2018-09-11 15:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (65.73 KB, patch)
2018-09-21 08:38 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (27.47 KB, patch)
2018-09-21 08:38 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix relatives issues table (1.39 KB, patch)
2018-09-21 08:38 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Do not call escapeHtml on null values (2.25 KB, patch)
2018-09-21 08:38 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Reword "select all" links since they select only visible rows (2.03 KB, patch)
2018-09-21 08:39 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Server-side processing and pagination on checkouts tables (68.70 KB, patch)
2019-09-13 12:01 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix date sorting when server-side processing is disabled (28.33 KB, patch)
2019-09-13 12:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix relatives issues table (1.39 KB, patch)
2019-09-13 12:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Do not call escapeHtml on null values (2.25 KB, patch)
2019-09-13 12:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Reword "select all" links since they select only visible rows (2.03 KB, patch)
2019-09-13 12:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix relatives checkouts (2.80 KB, patch)
2019-09-13 12:02 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 15219: Fix typo 'escapeHtm' (1.02 KB, patch)
2019-10-09 13:53 UTC, Julian Maurice
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Maurice 2015-11-19 15:10:08 UTC
Some libraries have patrons with more than 1000 checkouts. Loading all of them at once can be very long, and sometimes can cause timeout errors.

Following patches will prevent that by loading checkouts in the table by batches.
Comment 1 Julian Maurice 2015-11-19 15:11:22 UTC Comment hidden (obsolete)
Comment 2 Julian Maurice 2015-11-20 09:36:16 UTC Comment hidden (obsolete)
Comment 3 Julian Maurice 2015-11-20 14:48:28 UTC Comment hidden (obsolete)
Comment 4 Julian Maurice 2015-11-24 11:06:10 UTC
Changed status to ASSIGNED, the patches still need some work.
Comment 5 Julian Maurice 2016-02-16 10:12:35 UTC
Loading checkouts table incrementally takes more time to load all lines than to load them all at once, so I'll go with a classic paginated table.
Comment 6 Julian Maurice 2016-02-16 10:33:34 UTC Comment hidden (obsolete)
Comment 7 Julian Maurice 2016-04-08 12:32:12 UTC
Created attachment 50075 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3
Comment 8 Julian Maurice 2016-04-08 12:32:57 UTC
Patch rebased on master
Comment 9 Josef Moravec 2016-11-04 10:34:57 UTC
Sorry, doesn't apply...
Comment 10 Julian Maurice 2017-02-09 13:59:17 UTC
Created attachment 60071 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3
Comment 11 Julian Maurice 2017-02-09 13:59:43 UTC
Rebased on master
Comment 12 Jonathan Druart 2017-02-23 16:53:44 UTC
Sort by due date does not work, that was the point of the "due_date_unformatted" column.
Comment 13 Julian Maurice 2017-02-24 08:28:15 UTC
(In reply to Jonathan Druart from comment #12)
> Sort by due date does not work, that was the point of the
> "due_date_unformatted" column.

You're right, I thought mRender was only used for display and sorting was done using mData, but no, mRender is used for everything...
The columns.render options from DataTables 1.10+ (https://datatables.net/reference/option/columns.render) allow to do that, so I'll rewrite this part
Comment 14 Julian Maurice 2017-02-24 09:10:00 UTC
Created attachment 60642 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)
Comment 15 Julian Maurice 2017-03-16 08:58:21 UTC
Created attachment 61158 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3
Comment 16 Julian Maurice 2017-03-16 08:58:26 UTC
Created attachment 61159 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)
Comment 17 Biblibre Sandboxes 2017-03-23 14:56:14 UTC
Patch tested with a sandbox, by Eric Gosselin <eric.gosselin@inlibro.com>
Comment 18 Biblibre Sandboxes 2017-03-23 14:56:49 UTC
Created attachment 61545 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 19 Biblibre Sandboxes 2017-03-23 14:56:53 UTC
Created attachment 61546 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 20 Jonathan Druart 2017-03-29 19:03:22 UTC
I plan to QA it, but on top of bug 15498 and bug 14224 (also modifying checkouts.js).
Comment 21 Julian Maurice 2018-03-15 08:31:55 UTC
Created attachment 72905 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 22 Julian Maurice 2018-03-15 08:32:00 UTC
Created attachment 72906 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 23 Katrin Fischer 2018-04-16 06:27:56 UTC
Please rebase!

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 15219: Server-side processing and pagination on checkouts tables
Using index info to reconstruct a base tree...
M	admin/columns_settings.yml
M	installer/data/mysql/sysprefs.sql
M	koha-tmpl/intranet-tmpl/prog/en/includes/columns_settings.inc
M	koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/circulation.pref
M	koha-tmpl/intranet-tmpl/prog/en/modules/circ/circulation.tt
M	koha-tmpl/intranet-tmpl/prog/en/modules/members/moremember.tt
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/members/moremember.tt
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/members/moremember.tt
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/circ/circulation.tt
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/circ/circulation.tt
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/circulation.pref
Auto-merging koha-tmpl/intranet-tmpl/prog/en/includes/columns_settings.inc
Auto-merging installer/data/mysql/sysprefs.sql
Auto-merging admin/columns_settings.yml
Failed to merge in the changes.
Patch failed at 0001 Bug 15219: Server-side processing and pagination on checkouts tables
The copy of the patch that failed is found in:
   /home/vagrant/kohaclone/.git/rebase-apply/patch
When you have resolved this problem run "git bz apply --continue".
If you would prefer to skip this patch, instead run "git bz apply --skip".
To restore the original branch and stop patching run "git bz apply --abort".
Patch left in /tmp/Bug-15219-Server-side-processing-and-pagination-on-niral1.patch
Comment 24 Julian Maurice 2018-04-16 09:10:38 UTC
Created attachment 74224 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 25 Julian Maurice 2018-04-16 09:10:43 UTC
Created attachment 74225 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 26 Julian Maurice 2018-04-16 09:11:13 UTC
Patches rebased on master
Comment 27 Katrin Fischer 2018-04-16 22:00:22 UTC
1) Releatives tab doesn't open. JavaScript error:
Error: Syntax error, unrecognized expression: ##relatives-issues-table

2) I think most libraries will have a mix of large accounts and smaller ones. Could it make sense to use the server side processing when the patron has a certain number of checkouts (configurable)?
Comment 28 Julian Maurice 2018-06-08 08:55:29 UTC
Created attachment 75911 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 29 Julian Maurice 2018-06-08 08:55:35 UTC
Created attachment 75912 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 30 Julian Maurice 2018-06-08 08:55:40 UTC
Created attachment 75913 [details] [review]
Bug 15219: Fix relatives issues table
Comment 31 Julian Maurice 2018-06-08 09:11:30 UTC
(In reply to Katrin Fischer from comment #27)
> 1) Releatives tab doesn't open. JavaScript error:
> Error: Syntax error, unrecognized expression: ##relatives-issues-table
Fixed in the last patch

> 2) I think most libraries will have a mix of large accounts and smaller
> ones. Could it make sense to use the server side processing when the patron
> has a certain number of checkouts (configurable)?

It makes sense, but I think than can go in a separate bug. Do you agree ?
Comment 32 Jonathan Druart 2018-06-14 21:20:16 UTC
Tiny ergonomic issue when the export column is displayed: https://screenshots.firefox.com/0gdGrGVyEBanCz8n/pro.kohadev.org
Comment 33 Jonathan Druart 2018-06-14 21:22:31 UTC
(In reply to Jonathan Druart from comment #32)
> Tiny ergonomic issue when the export column is displayed:
> https://screenshots.firefox.com/0gdGrGVyEBanCz8n/pro.kohadev.org

Hum, the export column should not be displayed actually, ExportCircHistory is off. But it is in the "column visibility" list
Comment 34 Jonathan Druart 2018-06-14 21:24:30 UTC
"Select all" only select visible rows. If it is expected it is confusing.
Comment 35 Julian Maurice 2018-09-11 15:19:57 UTC
Created attachment 78555 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 36 Julian Maurice 2018-09-11 15:20:03 UTC
Created attachment 78556 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 37 Julian Maurice 2018-09-11 15:20:08 UTC
Created attachment 78557 [details] [review]
Bug 15219: Fix relatives issues table
Comment 38 Julian Maurice 2018-09-11 15:20:14 UTC
Created attachment 78558 [details] [review]
Bug 15219: Do not call escapeHtml on null values
Comment 39 Julian Maurice 2018-09-11 15:20:20 UTC
Created attachment 78559 [details] [review]
Bug 15219: Reword "select all" links since they select only visible rows
Comment 40 Julian Maurice 2018-09-11 15:23:16 UTC
(In reply to Jonathan Druart from comment #33)
> (In reply to Jonathan Druart from comment #32)
> > Tiny ergonomic issue when the export column is displayed:
> > https://screenshots.firefox.com/0gdGrGVyEBanCz8n/pro.kohadev.org
> 
> Hum, the export column should not be displayed actually, ExportCircHistory
> is off. But it is in the "column visibility" list

It looks like this has been partially fixed in master. When ExportCircHistory is off, the column is not there. But the footer problem is still there, and it's in master too. It should be fixed in another bug.
Comment 41 Julian Maurice 2018-09-21 08:38:43 UTC
Created attachment 79195 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 42 Julian Maurice 2018-09-21 08:38:48 UTC
Created attachment 79196 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 43 Julian Maurice 2018-09-21 08:38:54 UTC
Created attachment 79197 [details] [review]
Bug 15219: Fix relatives issues table
Comment 44 Julian Maurice 2018-09-21 08:38:59 UTC
Created attachment 79198 [details] [review]
Bug 15219: Do not call escapeHtml on null values
Comment 45 Julian Maurice 2018-09-21 08:39:04 UTC
Created attachment 79199 [details] [review]
Bug 15219: Reword "select all" links since they select only visible rows
Comment 46 Fridolin Somers 2019-05-16 10:05:30 UTC
A bug as been found on relatives checkouts Bug 22921.

Thoses patches should be rebased after it.
Comment 47 Fridolin Somers 2019-05-16 10:26:13 UTC
(In reply to Fridolin SOMERS from comment #46)
> A bug as been found on relatives checkouts Bug 22921.
> 
> Thoses patches should be rebased after it.

It turns out its a bug from this feature.

== Test plan ==
1. Have a patron with two guarantees
2. Have issues for the three of them
3. Go to the relatives issues view
4. You should only see the issues of one of them
5. Apply this patch
6. Force refresh the page (eg. ctrl-F5)
7. You should see the issues of the two relatives

Needs a fix
Comment 48 Julian Maurice 2019-09-13 12:01:50 UTC
Created attachment 92776 [details] [review]
Bug 15219: Server-side processing and pagination on checkouts tables

Some libraries have patrons with more than 1000 checkouts. Loading all
of them at once can be very long, and sometimes can cause timeout
errors.
This patch prevent that by enabling server-side processing and
pagination on checkouts tables.
This affects patron's checkouts and patron's relatives' checkouts tables
on pages circ/circulation.pl and members/moremember.pl.

As server-side processing can be useless and cumbersome with small
sets of data, a new system preference is introduced to control this
behaviour (server-side processing is disabled by default).

Additionally, this patch:
- adds a switch to turn off and on row grouping (today's checkouts vs
  previous checkouts) (on by default)
- adds "column settings" for relatives' checkouts tables
- factorize some code that was duplicated across the two DataTables
  configurations (mDataProp)

Test plan:
1. Find (or create) a patron that have at least 10 checkouts and where
   the relatives checkouts table contain at least 10 checkouts.
2. Go to the patron's detail page and check everything works fine
   in both tables (sorting, pagination, the data itself, ...)
3. Do the same on circulation page (circulation.pl)
4. Enable system preference ServerSideCheckoutsTable
5. Repeat steps 2 and 3

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 49 Julian Maurice 2019-09-13 12:02:04 UTC
Created attachment 92777 [details] [review]
Bug 15219: Fix date sorting when server-side processing is disabled

Use DataTables 'columns' option instead of 'aoColumns'
This allow more control on what data is used for display, sorting,
filtering, ...

A large part of this patch is about rewording things to better match the
DataTables documentation.
The real change is that

  "mRender": renderingFunc

becomes

  "render": {
    "display": renderingFunc
  }

So the return value of renderingFunc is now only used for display (not
for filtering, sorting, ...)

Signed-off-by: Eric Gosselin <eric.gosselin@inlibro.com>
Comment 50 Julian Maurice 2019-09-13 12:02:09 UTC
Created attachment 92778 [details] [review]
Bug 15219: Fix relatives issues table
Comment 51 Julian Maurice 2019-09-13 12:02:15 UTC
Created attachment 92779 [details] [review]
Bug 15219: Do not call escapeHtml on null values
Comment 52 Julian Maurice 2019-09-13 12:02:21 UTC
Created attachment 92780 [details] [review]
Bug 15219: Reword "select all" links since they select only visible rows
Comment 53 Julian Maurice 2019-09-13 12:02:27 UTC
Created attachment 92781 [details] [review]
Bug 15219: Fix relatives checkouts

Send all borrowernumbers to /svc/checkouts instead of only the last one
Comment 54 Julian Maurice 2019-09-13 12:04:47 UTC
Patches rebased on master

(In reply to Fridolin SOMERS from comment #47)
> (In reply to Fridolin SOMERS from comment #46)
> > A bug as been found on relatives checkouts Bug 22921.
> 
> It turns out its a bug from this feature.
> 
> Needs a fix

Fix attached (attachment 92781 [details] [review])
Comment 55 Fridolin Somers 2019-10-09 13:51:03 UTC
Typo in :
Bug 15219: Fix date sorting when server-side processing is disabled :

-            title += " - <span class='" + span_class + " item-note-public'>" + oObj.itemnotes.escapeHtml() + "</span>";
+            title += " - <span class='" + span_class + " item-note-public'>" + row.itemnotes.escapeHtm() + "</span>";

escapeHtm() instead of escapeHtml()
Comment 56 Julian Maurice 2019-10-09 13:53:28 UTC
Created attachment 93921 [details] [review]
Bug 15219: Fix typo 'escapeHtm'
Comment 57 Tomás Cohen Arazi 2020-01-17 13:53:14 UTC
Julian, take a look here:

https://gitlab.com/thekesolutions/Koha/commits/bug_20212_slowness_acq

we already have an API for checkouts, so you should just use it. I'm submitting the code for acquisitions soon, and you will be able to reuse the datatables extension to use the API (native API pagination included).
Comment 58 Julian Maurice 2020-01-17 14:09:20 UTC
(In reply to Tomás Cohen Arazi from comment #57)
> Julian, take a look here:
> 
> https://gitlab.com/thekesolutions/Koha/commits/bug_20212_slowness_acq
> 
> we already have an API for checkouts, so you should just use it. I'm
> submitting the code for acquisitions soon, and you will be able to reuse the
> datatables extension to use the API (native API pagination included).

Is the REST API for checkouts also capable of handling datatable filtering (for bug 15235) ?
What about the grouping of "today's checkouts" vs "previous checkouts" ? I think that this feature would be complicated to make using the REST API.
Comment 59 Katrin Fischer 2023-08-28 19:32:46 UTC
(In reply to Julian Maurice from comment #58)
> (In reply to Tomás Cohen Arazi from comment #57)
> > Julian, take a look here:
> > 
> > https://gitlab.com/thekesolutions/Koha/commits/bug_20212_slowness_acq
> > 
> > we already have an API for checkouts, so you should just use it. I'm
> > submitting the code for acquisitions soon, and you will be able to reuse the
> > datatables extension to use the API (native API pagination included).
> 
> Is the REST API for checkouts also capable of handling datatable filtering
> (for bug 15235) ?
> What about the grouping of "today's checkouts" vs "previous checkouts" ? I
> think that this feature would be complicated to make using the REST API.

@Tomas: can you help with these questions?