Bug 21987

Summary: Local cover 'thumbnail' size is bigger than 'imagefile' size in biblioimages table
Product: Koha Reporter: Andreas Roussos <arouss1980>
Component: Architecture, internals, and plumbingAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Martin Renvoize <martin.renvoize>
Severity: normal    
Priority: P3 CC: jonathan.druart, lucas, magnus, martin.renvoize, nick, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22988
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on:    
Bug Blocks: 23984, 23963    
Attachments: Bug 21987: Add tests
Bug 21987: Do not generate true color thumbnails if not needed
Bug 21987: Do not generate true color thumbnails if not needed
Bug 21987: Add tests
Bug 21987: Do not generate true color thumbnails if not needed

Description Andreas Roussos 2018-12-12 14:36:00 UTC
We use local cover images in our setup, and also resize the covers
we scan to a width of 160px before uploading and attaching them to
a Koha bibliographic record (we upload one cover per biblio).

While using phpMyAdmin to view the contents of the 'biblioimages'
table (relevant screenshot here: https://imgur.com/a/cSEVLBI), we
noticed that the size of the BLOBs for the 'thumbnail' column was in
some cases twice as big as that of the 'imagefile' column. In fact,
this happens for more than 50% of uploaded covers as you can see
from the output of the SQL queries below:

mysql> SELECT COUNT( * ) AS count
         FROM biblioimages
        WHERE LENGTH( thumbnail ) > ( LENGTH( imagefile ) * 2 ) ;
+-------+
| count |
+-------+
|  1356 |
+-------+

mysql> SELECT COUNT( * ) AS count,
              SUM( LENGTH( imagefile ) ) AS images_size,
              SUM( LENGTH( thumbnail ) ) AS thumbnails_size
         FROM biblioimages ;
+-------+-------------+-----------------+
| count | images_size | thumbnails_size |
+-------+-------------+-----------------+
|  2347 |    68323933 |       115839686 |
+-------+-------------+-----------------+

It would appear that for each 160px-wide JPG with 24-bit depth that
we have uploaded, the 'imagefile' column has been populated with an
PNG of 8-bit depth with the same dimensions as the uploaded file,
whereas the 'thumbnail' column contains a 24-bit PNG image with a
width reduced to 140 pixels.

I've tracked down the creation of resized 24-bit PNGs to this code:
https://github.com/Koha-Community/Koha/blob/master/C4/Images.pm#L182-L183

Do thumbnails for covers need to be true colour (24-bit) images?

Also, why is it that _scale_image() returns an image of 8-bit depth
if the source image's dimensions are less than 600x800? (i.e. when
no resizing is performed)
Comment 1 Andreas Roussos 2018-12-12 14:45:56 UTC
Here's some more info from tests I conducted earlier:

Original image              -> Koha imagefile  Koha thumbnail
---------------------------    --------------  --------------
aliakmon 160x245 24-bit JPG -> 160x245  8-bit  140x214 24-bit
aliakmon 601x801 24-bit JPG -> 600x800 24-bit  140x187 24-bit

oatrotf  599x799 32-bit PNG -> 599x799  8-bit  140x187 24-bit
oatrotf  601x801 32-bit PNG -> 600x800 24-bit  140x187 24-bit

testing  200x400  8-bit GIF -> 200x400  8-bit  140x280 24-bit
testing  601x801  8-bit GIF -> 600x800 24-bit  140x187 24-bit

whitebg  599x799 24-bit PNG -> 599x799  1-bit  140x187 24-bit *
whitebg  601x801 24-bit PNG -> 600x800 24-bit  140x187 24-bit

worrying 599x799  8-bit PNG -> 599x799  8-bit  140x187 24-bit
worrying 601x801  8-bit PNG -> 600x800 24-bit  140x187 24-bit

[*] the uploaded image was empty (white background), perhaps
    that explains the resulting 1-bit imagefile?
Comment 2 Jonathan Druart 2018-12-18 17:03:38 UTC
Created attachment 83369 [details] [review]
Bug 21987: Add tests
Comment 3 Jonathan Druart 2018-12-18 17:03:42 UTC
Created attachment 83370 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.
Comment 4 Jonathan Druart 2018-12-19 18:35:45 UTC
A test plan could be:
- Attach image to a bibliographic record
- Confirm that the thumbnail is not bigger than the original image

Use 8-bit and 24-bit images.
Comment 5 Michal Denar 2019-03-06 20:37:49 UTC
Created attachment 86215 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 6 Martin Renvoize 2019-03-12 13:01:24 UTC
Created attachment 86525 [details] [review]
Bug 21987: Add tests

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 7 Martin Renvoize 2019-03-12 13:01:27 UTC
Created attachment 86526 [details] [review]
Bug 21987: Do not generate true color thumbnails if not needed

If the original image is not a true color image we should not generate a
true color thumbnail.

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2019-03-12 13:02:02 UTC
All works here, Passing QA
Comment 9 Nick Clemens 2019-03-15 12:47:40 UTC
Awesome work all!

Pushed to master for 19.05
Comment 10 Martin Renvoize 2019-03-19 09:55:37 UTC
Pushed to 18.11.x for 18.11.04
Comment 11 Lucas Gass 2019-03-19 23:30:16 UTC
backported to 18.05.x for 18.05.11
Comment 12 Magnus Enger 2019-05-25 21:26:11 UTC
See bug 22988 for a possible regression caused by this bug.
Comment 13 Tomás Cohen Arazi 2019-08-14 17:00:01 UTC
(In reply to Magnus Enger from comment #12)
> See bug 22988 for a possible regression caused by this bug.

This should probably be reverted in the meantime, as it could be considered data-loss
Comment 14 Magnus Enger 2019-08-15 06:51:30 UTC
(In reply to Tomás Cohen Arazi from comment #13)
> (In reply to Magnus Enger from comment #12)
> > See bug 22988 for a possible regression caused by this bug.
> 
> This should probably be reverted in the meantime, as it could be considered
> data-loss

Our customer would tend to agree, I think.
Comment 15 Martin Renvoize 2019-11-04 13:30:43 UTC
Reverted as bug 23963