Bug 7187 - Prioritize cover image sources such as Amazon, Google and add fail over i.e. if first source fails, go to next
Summary: Prioritize cover image sources such as Amazon, Google and add fail over i.e. ...
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: OPAC (show other bugs)
Version: master
Hardware: All All
: P4 enhancement (vote)
Target Milestone: ---
Assignee: Bugs List
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
: 2131 7574 11177 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-11-06 09:18 UTC by Koustubha Kale
Modified: 2019-03-21 11:35 UTC (History)
29 users (show)

See Also:
Change sponsored?: Seeking cosponsors
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Koustubha Kale 2011-11-06 09:18:49 UTC
We should be able to set priority on cover image sources such as Amazon, Google. Then if one source fails we should be able to fail over to the next.
Comment 1 Katrin Fischer 2011-11-06 11:23:39 UTC
I found bug 2131 with a very old patch attached that sounds like a duplicate to this. I am linking both together for someone interested to work on this.
Comment 2 Nicole C. Engard 2012-02-26 20:38:37 UTC
*** Bug 7574 has been marked as a duplicate of this bug. ***
Comment 3 Magnus Enger 2012-03-07 09:12:47 UTC
*** Bug 7657 has been marked as a duplicate of this bug. ***
Comment 4 Tomás Cohen Arazi 2012-08-06 14:52:11 UTC
I propose the addition of two new sysprefs for this task:

a) Choose a default cover location (Local, Google, Amazon, etc) (combo with fixed values)
b) Choose whether to traverse the list of services looking for the cover (starting from the default) or 'just use the default' (YesNo syspref.
Comment 5 Manos PETRIDIS 2012-08-13 18:12:01 UTC
(In reply to comment #4)
> I propose the addition of two new sysprefs for this task:
> 
> a) Choose a default cover location (Local, Google, Amazon, etc) (combo with
> fixed values)
> b) Choose whether to traverse the list of services looking for the cover
> (starting from the default) or 'just use the default' (YesNo syspref.


Perhaps this way AmazonLocale (Use Amazon data from its [American|British|Canadian|French|German|Japanese] website) will no longer be necessary. 

The fixed values combo would contain all Amazon locales, so that should a book not be found in the Amazon US site, koha would look it up in the other Amazon sites as well.
Comment 6 Magnus Enger 2012-09-05 13:14:10 UTC
One thing we need to decide on is whether the logic that checks multiple sources should be in JavaScript on the client side or Perl on the server side. I'm leaning towards Perl on the server, for two reasons: 

1. Displaying an image could be as simple as this:

  <img src="opac-ver.pl?biblionumber=x" />

2. We could cache the results of looking through multiple sources. (Caching might violate the terms of service for e.g. Amazon, but then we could check them last, after we have checked the ones that we can cache.)
Comment 7 Robin Sheat 2012-09-05 13:21:13 UTC
I tend to think that in Perl is a better solution too, however it's worth noting that it'll make firewalling harder.

As for the caching, I think if you cached "what was the first service that gave me a meaningful result", that wouldn't violate anything, and is also the best way to do that (short of caching the image, which really is forbidden.)
Comment 8 Katrin Fischer 2012-09-05 13:39:18 UTC
I think some sources allow it (but not Amazon) and we can store covers locally now, so those should be checked first of all. Another plus on doing it server side would be better privacy.
Comment 9 Robin Sheat 2012-09-05 13:48:20 UTC
A cover source I'd like to add (NZ Nat Lib) also doesn't allow caching of images, so relying on "only the last one doesn't allow caching" is not ideal. Caching images from those that do allow it is worthwhile however.
Comment 10 Katrin Fischer 2012-09-05 13:52:04 UTC
I think maybe the sequence should be user configurable? With an option to use the first that answered?
Comment 11 Magnus Enger 2012-09-05 14:03:38 UTC
(In reply to comment #10)
> I think maybe the sequence should be user configurable? With an option to
> use the first that answered?

That's what I'm thinking too.
Comment 12 Robin Sheat 2012-09-05 14:04:11 UTC
I'd consider that a separate enhancement. If we try to cram too much stuff in for the first go, it'll never get done. Also, I don't think our syspref system allows that kind of configuration.
Comment 13 Jared Camins-Esakov 2012-09-05 14:22:56 UTC
(In reply to comment #12)
> I'd consider that a separate enhancement. If we try to cram too much stuff
> in for the first go, it'll never get done. Also, I don't think our syspref
> system allows that kind of configuration.

I am working on a similar configuration issue on bug 8726. Once that patch is up, perhaps you can adapt^Hborrow^Hshamelessly steal the code for that. Being able to drag things around to order them is pretty cool, and once I figured out how to do it, not hard at all.
Comment 15 Zeno Tajoli 2012-09-05 14:30:21 UTC
Hi to all,

as I know Open Library, Library Thing and Google allow to cache book'covers.
I try to find Amazon license for book covers but I don't find it.
As I see in Vufind (an opac written with PHP) they cache also covers from Amazon.
They use  Amazon strictly only for covers.
Comment 16 Manos PETRIDIS 2012-09-06 07:39:09 UTC
(In reply to comment #6)
> One thing we need to decide on is whether the logic that checks multiple
> sources should be in JavaScript on the client side or Perl on the server
> side. I'm leaning towards Perl on the server, for two reasons: 
> 
> 1. Displaying an image could be as simple as this:
> 
>   <img src="opac-ver.pl?biblionumber=x" />
> 
> 2. We could cache the results of looking through multiple sources. (Caching
> might violate the terms of service for e.g. Amazon, but then we could check
> them last, after we have checked the ones that we can cache.)

Just a comment from an end-user: Could it be that Amazon returns an image, even when no cover image is available in its systems? I'm referring to the "No image available" with a hint of a smile/bent arrow below image.

Unless Amazon does provide for an indication that the image it provides is a placeholder for the cover image it does not [yet] have, caching might prove tricky, for it could hinder future [successful] look-ups in Amazon. 

Furthermore, should Amazon be at the top of the list, successfuly fetching a placeholder image from Amazon would prevent koha from retrieving a correct cover image from some other source down the list.
Comment 17 Magnus Enger 2012-09-06 07:43:51 UTC
Good points, Manos! Yeah, I am trying to figure out a way to check if the image returned is a placeholder or not, that is kind of crucial. And caching should not last forever.
Comment 18 Robin Sheat 2012-09-06 15:02:35 UTC
I've updated the RFC page with my plans for getting something done. This won't be the full solution, but I think a good-enough starting point that can be built on. It's also possible I'll have some sponsorship for this.
Comment 19 Manos PETRIDIS 2012-09-12 08:21:56 UTC
(In reply to comment #6)
> One thing we need to decide on is whether the logic that checks multiple
> sources should be in JavaScript on the client side or Perl on the server
> side. I'm leaning towards Perl on the server, for two reasons: 
> 
> 1. Displaying an image could be as simple as this:
> 
>   <img src="opac-ver.pl?biblionumber=x" />
> 
> 2. We could cache the results of looking through multiple sources. (Caching
> might violate the terms of service for e.g. Amazon, but then we could check
> them last, after we have checked the ones that we can cache.)

I know it is quite late, but it just came to me that using server-side code might simplify using/intergrating local images with the ones provided by external sources, and allow for displaying the same images at the OPAC and the administrative calatogs.
Comment 20 Robin Sheat 2012-09-12 08:25:12 UTC
While not explicitly discussed, this is covered in my plan. There will be a set of "plugins" to handle each image source, and this'll include local ones.

I've sent the details to the library, I'm just waiting for them to get back to me about whether they'll be keen to go for it.
Comment 21 Cab Vinton 2012-12-16 22:25:05 UTC
An additional thought -- I believe that at present an ISBN is required to retrieve cover images. Many movies in particular, however, lack ISBNs.

In Amazon at least, many searches for the UPC will retrieve the correct item. EG UPC 043396275140 retrieves:
http://www.amazon.com/District-9-Single-Disc-Sharlto-Copley/dp/B002SJIO4A/

So perhaps the 024 tag could be added as a potential source for cover images?

Note that in the example above Amazon provides yet another identifier, namely, the ASIN (B002SJI04A). The ASIN is always present & could be added to the 024 by staff so that in theory users would have to only rarely, if ever, upload local cover images.

That said, I have no idea how the various APIs work. Hopefully they're compatible with the above approach.
Comment 22 Owen Leonard 2012-12-17 17:48:40 UTC
(In reply to comment #21)
> In Amazon at least, many searches for the UPC will retrieve the correct
> item. EG UPC 043396275140 retrieves:
> http://www.amazon.com/District-9-Single-Disc-Sharlto-Copley/dp/B002SJIO4A/

The question is not whether you can retrieve a record on Amazon with a UPC, but whether the Amazon cover image service works with UPC. It does not. If it did, you could replace the ISBN in a link like this:

http://images.amazon.com/images/P/0596001320.01.MZZZZZZZ.jpg

...with a UPC. But this doesn't work:

http://images.amazon.com/images/P/043396275140.01.MZZZZZZZ.jpg
Comment 23 Cab Vinton 2012-12-17 19:27:02 UTC
(In reply to comment #22)
> The question is not whether you can retrieve a record on Amazon with a UPC,
> but whether the Amazon cover image service works with UPC. It does not.

Hmmm... No doubt you're correct, Owen. Amazon's Product Advertising API seems to allow the use of a number of different identifiers, including ASINs:

http://docs.amazonwebservices.com/AWSECommerceService/latest/DG/ItemLookup.html

Presumably Koha is using a different service.

Note that the LibGuides folks have added a separate option for users to specify a URL source for cover art for each record ...

http://guidefaq.com/a.php?qid=2200
Comment 24 Sebastian Hierl 2013-10-11 13:57:46 UTC
(In reply to Cab Vinton from comment #23)
As VuFind has been mentioned above, I thought I'd confirm that VuFind separates its book covers functionality from the book previews API -- and that it can search the various sources (Google Books, Amazon, Open Library, and Hathi Trust) by ISBNS, LCCNs, and OCLC numbers:

https://github.com/vufind-org/vufind/blob/master/themes/bootstrap/js/preview.js

Thus, not only the most recent publications that have ISBNs are retrieved/linked to in the OPAC, but also those that have OCLC and LCCN #s, thus numerous titles that are available in full text--which is the point of such a feature, i.e. to link to the full text for immediate access.



> (In reply to comment #22)
> > The question is not whether you can retrieve a record on Amazon with a UPC,
> > but whether the Amazon cover image service works with UPC. It does not.
> 
> Hmmm... No doubt you're correct, Owen. Amazon's Product Advertising API
> seems to allow the use of a number of different identifiers, including ASINs:
> 
> http://docs.amazonwebservices.com/AWSECommerceService/latest/DG/ItemLookup.
> html
> 
> Presumably Koha is using a different service.
> 
> Note that the LibGuides folks have added a separate option for users to
> specify a URL source for cover art for each record ...
> 
> http://guidefaq.com/a.php?qid=2200
Comment 25 Owen Leonard 2014-02-21 16:34:51 UTC
*** Bug 11177 has been marked as a duplicate of this bug. ***
Comment 26 David Cook 2014-03-13 04:05:59 UTC
(In reply to Robin Sheat from comment #20)
> While not explicitly discussed, this is covered in my plan. There will be a
> set of "plugins" to handle each image source, and this'll include local ones.
> 
> I've sent the details to the library, I'm just waiting for them to get back
> to me about whether they'll be keen to go for it.

Robin, did you ever hear back about this?
Comment 27 Robin Sheat 2014-03-13 04:14:10 UTC
(In reply to David Cook from comment #26)
> Robin, did you ever hear back about this?

Alas no.
Comment 28 Magnus Enger 2014-05-23 08:28:38 UTC
Here is a proposal to raise funds for this development:
http://lists.katipo.co.nz/pipermail/koha/2014-May/039491.html
Comment 29 Viktor Sarge 2014-08-06 07:39:30 UTC
I chose to participate in the fundraiser and can report that work is being done on this.
Comment 30 Katrin Fischer 2014-08-17 21:37:03 UTC
*** Bug 2131 has been marked as a duplicate of this bug. ***
Comment 31 Chris Cormack 2014-09-10 22:31:38 UTC
Work in progress is here

http://git.catalyst.net.nz/gw?p=koha.git;a=shortlog;h=refs/heads/wr222309

It is almost ready to be submitted as a patch set
Comment 32 LuceBarbey 2015-03-05 12:51:55 UTC
What's the news about this patch ? 
Thanks
Comment 33 Chris Cormack 2015-05-04 21:18:00 UTC
This is being tested by clients, you can (as always) see the work on the catalyst repo.
Comment 34 Brendan Gallagher 2015-08-25 17:26:52 UTC
Has this been submitted on another bug?  I just looked at the git repo and haven't seen an update in awhile.

Thanks,
Brendan 


(In reply to Chris Cormack from comment #33)
> This is being tested by clients, you can (as always) see the work on the
> catalyst repo.
Comment 35 Viktor Sarge 2016-01-20 08:17:33 UTC
(In reply to Brendan Gallagher from comment #34)
> Has this been submitted on another bug?  I just looked at the git repo and
> haven't seen an update in awhile.

As far as I know they are not happy with performance yet and will rework this patch before anything interesting starts happening. I do hope that it will get done in time for the may release since it's a feature I'd love to see, but I don't think anything is happening right now.
Comment 36 Frédéric Demians 2016-01-20 09:18:52 UTC
Bug 9580 feature is an alternative option which is functional.
Comment 37 Chris Cormack 2016-01-20 18:58:33 UTC
We are actually evaluating 9580 at the moment to see if it will meet all the requirements of this bug. Will let you know shortly
Comment 38 Katrin Fischer 2016-08-14 14:59:29 UTC
Hi Chris, any update on this maybe?
Comment 39 Christopher Brannon 2016-10-10 16:28:45 UTC
Looking forward to this feature.  :) Would love to see more consistent covers throughout Koha.
Comment 40 Magnus Enger 2016-12-13 14:22:46 UTC
(In reply to Chris Cormack from comment #37)
> We are actually evaluating 9580 at the moment to see if it will meet all the
> requirements of this bug. Will let you know shortly

Any results from the evaluation yet? ;-)
Comment 41 Katrin Fischer 2016-12-13 15:13:57 UTC
We are really interested in this as well - centralizing the code would make it easier to add new content providers.
Comment 42 Chris Cormack 2016-12-13 15:20:05 UTC
Yes, bug 9850 meets all the requirements of this. We have it in production for our sites. So we will not be doing any more work on this. 

Our recommendation is to use coce for book covers
Comment 43 David Cook 2017-01-12 06:00:45 UTC
(In reply to Chris Cormack from comment #42)
> Yes, bug 9850 meets all the requirements of this. We have it in production
> for our sites. So we will not be doing any more work on this. 
> 
> Our recommendation is to use coce for book covers

Bug 9850 is about Koha 3.12 help files. Was that a typo, Chris?
Comment 44 David Cook 2017-01-12 06:01:24 UTC
Ah, bug 9580 I'm guessing.

I really need to look into Coce...
Comment 45 Nick Clemens 2017-01-12 21:21:43 UTC
(In reply to David Cook from comment #44)
> Ah, bug 9580 I'm guessing.
> 
> I really need to look into Coce...

ByWater has a coce server that anyone is welcome to use (the more users the more images ready for use) 

https://coce.bywatersolutions.com
Comment 46 Agnes Rivers-Moore 2017-04-07 21:01:04 UTC
Hanover has started using Bywater's Coce server.  :-)

It did not work for us at first but is working this week.
It certainly does provide a wider range of jacket images than Amazon, especially for older titles and non-book media.
Considerations:

1) The first ever call for a particular jacket prompts the Coce server to search for it, but it is not returned to the Koha search until the page is refreshed or search done again or details page opened. That is easy for staff to work with but not so good on the OPAC where so many entries load with no images until Coce has retrieved them at least once.

2) Coce is only set up to work for OPAC, not staff interface. If we had the choice, two options the same as for Amazon jackets, to turn it on for either or both would be nice. Then staff could judge when Coce returns enough images to turn on for our users.

We have decided to run with both Amazon and Coce for now, so that in time the image links we need will be generated. Two jacket images looks untidy but until the server has most of the links, two jackets sometimes is better than no jacket frequently.
The more libraries use the server, the sooner it will be able to provide a comprehensive resource for jacket images.

Thank you to everyone who worked on this and Bywater for providing the server service.
Comment 47 Fridolin SOMERS 2018-11-14 08:49:01 UTC
This is still needed to prioritize local cover image over remote cover image (Coce Amazone, ...)
Comment 48 Patrick Robitaille 2019-01-14 13:33:34 UTC
Hi, 

Does anyone work on this? The possibility to prioritize cover image sources would be a usefull add for Koha. 

Thanks, Patrick.
Comment 49 Owen Leonard 2019-01-14 14:32:25 UTC
> Does anyone work on this?

I think the introduction of Coce has solved this problem well enough that there isn't much demand for more work. However, I agree with Frodolin that it would be good to make Coce and local cover images work better together.

I'm resetting the "ASSIGNED" status of this because I think it's not actively being worked on by anyone.
Comment 50 David Cook 2019-01-15 06:59:21 UTC
I'm interested in the outcome but don't have the time to work on this :(
Comment 51 George Veranis 2019-03-21 11:25:14 UTC
Someone who solved by js 

https://catalog.sll.texas.gov/opac-tmpl/bootstrap/js/localcovers_18.0510000.js
Comment 52 George Veranis 2019-03-21 11:28:12 UTC
sorry mistake, wrong reply
(In reply to George Veranis from comment #51)
> Someone who solved by js 
> 
> https://catalog.sll.texas.gov/opac-tmpl/bootstrap/js/localcovers_18.0510000.
> js