Bug 21987 - Local cover 'thumbnail' size is bigger than 'imagefile' size in biblioimages table
Summary: Local cover 'thumbnail' size is bigger than 'imagefile' size in biblioimages ...
Status: Pushed to stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P3 normal (vote)
Assignee: Jonathan Druart
QA Contact: Martin Renvoize
URL:
Keywords:
Depends on:
Blocks: 22988
  Show dependency treegraph
 
Reported: 2018-12-12 14:36 UTC by Andreas Roussos
Modified: 2019-08-15 06:51 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 21987: Add tests (1.90 KB, patch)
2018-12-18 17:03 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 21987: Do not generate true color thumbnails if not needed (1.14 KB, patch)
2018-12-18 17:03 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 21987: Do not generate true color thumbnails if not needed (1.18 KB, patch)
2019-03-06 20:37 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 21987: Add tests (2.01 KB, patch)
2019-03-12 13:01 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 21987: Do not generate true color thumbnails if not needed (1.25 KB, patch)
2019-03-12 13:01 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Roussos 2018-12-12 14:36:00 UTC
We use local cover images in our setup, and also resize the covers
we scan to a width of 160px before uploading and attaching them to
a Koha bibliographic record (we upload one cover per biblio).

While using phpMyAdmin to view the contents of the 'biblioimages'
table (relevant screenshot here: https://imgur.com/a/cSEVLBI), we
noticed that the size of the BLOBs for the 'thumbnail' column was in
some cases twice as big as that of the 'imagefile' column. In fact,
this happens for more than 50% of uploaded covers as you can see
from the output of the SQL queries below:

mysql> SELECT COUNT( * ) AS count
         FROM biblioimages
        WHERE LENGTH( thumbnail ) > ( LENGTH( imagefile ) * 2 ) ;
+-------+
| count |
+-------+
|  1356 |
+-------+

mysql> SELECT COUNT( * ) AS count,
              SUM( LENGTH( imagefile ) ) AS images_size,
              SUM( LENGTH( thumbnail ) ) AS thumbnails_size
         FROM biblioimages ;
+-------+-------------+-----------------+
| count | images_size | thumbnails_size |
+-------+-------------+-----------------+
|  2347 |    68323933 |       115839686 |
+-------+-------------+-----------------+

It would appear that for each 160px-wide JPG with 24-bit depth that
we have uploaded, the 'imagefile' column has been populated with an
PNG of 8-bit depth with the same dimensions as the uploaded file,
whereas the 'thumbnail' column contains a 24-bit PNG image with a
width reduced to 140 pixels.

I've tracked down the creation of resized 24-bit PNGs to this code:
https://github.com/Koha-Community/Koha/blob/master/C4/Images.pm#L182-L183

Do thumbnails for covers need to be true colour (24-bit) images?

Also, why is it that _scale_image() returns an image of 8-bit depth
if the source image's dimensions are less than 600x800? (i.e. when
no resizing is performed)
Comment 1 Andreas Roussos 2018-12-12 14:45:56 UTC
Here's some more info from tests I conducted earlier:

Original image              -> Koha imagefile  Koha thumbnail
---------------------------    --------------  --------------
aliakmon 160x245 24-bit JPG -> 160x245  8-bit  140x214 24-bit
aliakmon 601x801 24-bit JPG -> 600x800 24-bit  140x187 24-bit

oatrotf  599x799 32-bit PNG -> 599x799  8-bit  140x187 24-bit
oatrotf  601x801 32-bit PNG -> 600x800 24-bit  140x187 24-bit

testing  200x400  8-bit GIF -> 200x400  8-bit  140x280 24-bit
testing  601x801  8-bit GIF -> 600x800 24-bit  140x187 24-bit

whitebg  599x799 24-bit PNG -> 599x799  1-bit  140x187 24-bit *
whitebg  601x801 24-bit PNG -> 600x800 24-bit  140x187 24-bit

worrying 599x799  8-bit PNG -> 599x799  8-bit  140x187 24-bit
worrying 601x801  8-bit PNG -> 600x800 24-bit  140x187 24-bit

[*] the uploaded image was empty (white background), perhaps
    that explains the resulting 1-bit imagefile?
Comment 2 Jonathan Druart 2018-12-18 17:03:38 UTC
Created attachment 83369 [details] [review]
Bug 21987: Add tests
Comment 3 Jonathan Druart 2018-12-18 17:03:42 UTC
Created attachment 83370 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.
Comment 4 Jonathan Druart 2018-12-19 18:35:45 UTC
A test plan could be:
- Attach image to a bibliographic record
- Confirm that the thumbnail is not bigger than the original image

Use 8-bit and 24-bit images.
Comment 5 Michal Denar 2019-03-06 20:37:49 UTC
Created attachment 86215 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 6 Martin Renvoize 2019-03-12 13:01:24 UTC
Created attachment 86525 [details] [review]
Bug 21987: Add tests

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 7 Martin Renvoize 2019-03-12 13:01:27 UTC
Created attachment 86526 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2019-03-12 13:02:02 UTC
All works here, Passing QA
Comment 9 Nick Clemens 2019-03-15 12:47:40 UTC
Awesome work all!

Pushed to master for 19.05
Comment 10 Martin Renvoize 2019-03-19 09:55:37 UTC
Pushed to 18.11.x for 18.11.04
Comment 11 Lucas Gass 2019-03-19 23:30:16 UTC
backported to 18.05.x for 18.05.11
Comment 12 Magnus Enger 2019-05-25 21:26:11 UTC
See bug 22988 for a possible regression caused by this bug.
Comment 13 Tomás Cohen Arazi 2019-08-14 17:00:01 UTC
(In reply to Magnus Enger from comment #12)
> See bug 22988 for a possible regression caused by this bug.

This should probably be reverted in the meantime, as it could be considered data-loss
Comment 14 Magnus Enger 2019-08-15 06:51:30 UTC
(In reply to Tomás Cohen Arazi from comment #13)
> (In reply to Magnus Enger from comment #12)
> > See bug 22988 for a possible regression caused by this bug.
> 
> This should probably be reverted in the meantime, as it could be considered
> data-loss

Our customer would tend to agree, I think.