Bug 10787

Summary: Linked Data/RDF Omnibus
Product: Koha Reporter: Magnus Enger <magnus>
Component: Architecture, internals, and plumbingAssignee: Galen Charlton <gmcharlt>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: black23, caroline.cyr-la-rose, claire.hernandez, dcook, heather_hernandez, josef.moravec, mathsabypro
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18585
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18586
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18713
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 10788, 7236, 18585    
Bug Blocks:    

Description Magnus Enger 2013-08-27 10:43:59 UTC
This bug will be an omnibus for adding support for Linked Data/RDF to Koha. Actual work will be done on bugs linked to this bug. 

See http://wiki.koha-community.org/wiki/Linked_Data_RFC for some thoughts on what can/will be done.
Comment 1 David Cook 2018-08-13 01:48:01 UTC
I've been thinking a bit more about RDF in Koha and wondering if we should primarily store the RDF metadata in Koha, as Nick Clemens suggested long ago.

I suppose my reasoning is based on my experience with Fedora Commons (https://wiki.duraspace.org/display/FEDORA4x/Configuring+JDBC+Object+Store) and Apache Fuseki. The production recommendation for Fedora Commons is to use ModeShape within a relational database for storing RDF. Then people can use Fedora's ActiveMQ message queue for updating Apache Fuseki from there. 

As I understand it, Apache Fuseki seems to have one maintainer (who I've talked to using JIRA as I've reported issues and tested fixes for Fuseki). Fuseki doesn't support replication and doesn't really seem to scale that well. It can do online backups which is cool although there are several versions where the backup system is broken. If you do a disk-based backup while Fuseki is running, you'll get a corrupted backup, so you need to use its application-based backup (like with a relational database). 

I think that I like the idea of keeping the source metadata in MySQL and then having the ability to update Fuseki or any other triplestore from there. Even if you did lose all of your Fuseki data, you could always re-create it from the persisted data in MySQL. 

In other words, I think we could leverage the maturity and stability of relational databases, while also taking advantage of the SPARQL query power of a triplestore like Apache Fuseki.
Comment 2 David Cook 2020-10-21 01:44:42 UTC
It seems that WikiData use a SQL database to store their source metadata (as JSON blobs) and then use BlazeGraph to provide SPARQL access to that data, according to https://www.quora.com/Does-wikidata-store-data-as-RDF-Triples-If-yes-what-kind-of-datastore-is-used.

I reckon that they're a good example to follow.
Comment 3 David Cook 2020-10-21 01:53:06 UTC
An interesting thing to note about WikiData is that it stores all its data as "pages".

So far, I think that we've thought about RDF/Linked Data in terms of bib records, but I am thinking that we should be more broad/generic than that. 

It's tough to know because we're not 100% sure how we want to use RDF with Koha yet. 

I imagine that David Nind might have some ideas. 

WorldCat and VIAF offer bib and authority data in RDF format. Maybe we do just want to focus on bibs and authorities, but maybe we also want items... maybe information about libraries... I'm not sure.