Bug 18829 - Elasticsearch - Add ability to view the ES indexed record
Summary: Elasticsearch - Add ability to view the ES indexed record
Status: Needs documenting
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement with 10 votes (vote)
Assignee: Nick Clemens
QA Contact:
URL:
Keywords: additional_work_needed
Depends on:
Blocks:
 
Reported: 2017-06-20 14:54 UTC by Nick Clemens
Modified: 2023-04-18 12:21 UTC (History)
11 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00


Attachments
Bug 18829 - Elasticsearch - Add ability to view the ES indexed record (3.92 KB, patch)
2017-06-20 14:58 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Add ability to view the ES indexed record (4.57 KB, patch)
2019-04-29 09:29 UTC, axel
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Add ability to view the ES indexed record (4.58 KB, patch)
2019-04-29 09:36 UTC, axel
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (10.90 KB, patch)
2019-05-02 14:55 UTC, axel Amghar
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Add ability to view the ES indexed record (4.08 KB, patch)
2019-05-02 15:05 UTC, axel Amghar
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (10.90 KB, patch)
2019-05-02 15:05 UTC, axel Amghar
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (8.20 KB, patch)
2019-06-06 13:28 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (8.18 KB, patch)
2020-08-12 13:12 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (8.07 KB, patch)
2020-08-12 13:17 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (8.11 KB, patch)
2020-08-13 15:10 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (8.19 KB, patch)
2020-08-22 23:39 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title (2.89 KB, patch)
2020-08-22 23:39 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (7.94 KB, patch)
2023-03-27 12:10 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title (2.65 KB, patch)
2023-03-27 12:10 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: (follow-up) Return 404 when record cannot be loaded (2.51 KB, patch)
2023-03-27 12:10 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: (follow-up) Don't messup record encoding (1.34 KB, patch)
2023-03-27 12:10 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record (7.94 KB, patch)
2023-04-10 17:08 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title (2.65 KB, patch)
2023-04-10 17:08 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (follow-up) Return 404 when record cannot be loaded (2.51 KB, patch)
2023-04-10 17:08 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (follow-up) Don't messup record encoding (1.39 KB, patch)
2023-04-10 17:08 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (QA follow-up) I18N, test character and breadcrumbs (3.23 KB, patch)
2023-04-10 17:08 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 18829: (QA follow-up) Replace forbidden indirect syntax (1.05 KB, patch)
2023-04-17 11:26 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens 2017-06-20 14:54:40 UTC
It would be nice for the end user (or developer) to have a way to view the record as it exists in the ES index - it will help in adjusting mappings, understanding results, and identifying is a record is not being index
Comment 1 Nick Clemens 2017-06-20 14:58:50 UTC Comment hidden (obsolete)
Comment 2 axel 2019-04-29 09:29:13 UTC Comment hidden (obsolete)
Comment 3 axel 2019-04-29 09:36:44 UTC Comment hidden (obsolete)
Comment 4 axel Amghar 2019-05-02 14:55:13 UTC Comment hidden (obsolete)
Comment 5 axel Amghar 2019-05-02 15:05:04 UTC
Created attachment 89249 [details] [review]
Bug 18829: Elasticsearch - Add ability to view the ES indexed record

This is groundwork - we need to make the display prettier, but for now
it does at least dump the record into the template

We probably also need a swtich to enable/disable this - or just move it
into the left hand tabs?
Comment 6 axel Amghar 2019-05-02 15:05:08 UTC
Created attachment 89250 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElsaticSearch" as value
- Search whatever you want in Search the catalog and select a notice
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
Comment 7 Fridolin Somers 2019-05-28 14:26:53 UTC
Oups the ES connexion is hardcoded :

+                $.getJSON("http://localhost:9200/koha_kohadev_biblios/data/", biblionumber, function( data ){

You must get server+port and index name from C4::Context.
Comment 8 Nick Clemens 2019-06-06 13:28:17 UTC
Created attachment 90370 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
Comment 9 Nick Clemens 2019-06-06 13:30:55 UTC
Squashed as axel rewrote my patch :-)

Some issues:
1 - Missing filter from QA tool - this is tricky too - if I add </script> to the title I can break the view - we need to make sure the JOSN can't break
2 - We should order the view, at least for _source - this could be later enhancement, but would make it much easier to read
Comment 10 Nick Clemens 2020-08-12 13:12:42 UTC
Created attachment 108103 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record
Comment 11 Nick Clemens 2020-08-12 13:17:26 UTC
Created attachment 108104 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record
Comment 12 Fridolin Somers 2020-08-13 15:08:30 UTC
Nice work, I like it.
Comment 13 Fridolin Somers 2020-08-13 15:10:46 UTC
Created attachment 108191 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 14 Katrin Fischer 2020-08-22 23:39:13 UTC
Created attachment 108944 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 15 Katrin Fischer 2020-08-22 23:39:18 UTC
Created attachment 108945 [details] [review]
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 16 Katrin Fischer 2020-08-22 23:39:37 UTC
I think this is a nice little enhancement as is. 

I'd love to see codemirror JSON highlighting used on the modal in the future.
Comment 17 Jonathan Druart 2020-08-26 13:18:15 UTC
1. 
+catch{
+    @es_fields = ("Error fetching record: see logs for details");
+    warn $_;
+};

This is not the way to deal with error, you should return an error, here certainly a 404.

2. Incorrect unicode characters handling.
Add '❤' to the title then click the link, you get a JS error: "URIError: malformed URI sequence" and the modal is empty.
Comment 18 Nick Clemens 2023-03-27 12:10:17 UTC
Created attachment 148757 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 19 Nick Clemens 2023-03-27 12:10:20 UTC
Created attachment 148758 [details] [review]
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 20 Nick Clemens 2023-03-27 12:10:23 UTC
Created attachment 148759 [details] [review]
Bug 18829: (follow-up) Return 404 when record cannot be loaded
Comment 21 Nick Clemens 2023-03-27 12:10:26 UTC
Created attachment 148760 [details] [review]
Bug 18829: (follow-up) Don't messup record encoding
Comment 22 Katrin Fischer 2023-04-10 17:08:04 UTC
Created attachment 149400 [details] [review]
Bug 18829: Elasticsearch - Put a better display for the view of ES indexed record

To test:
- apply the patch
- go to global sysPref
- make sure that SearchEngine have "ElasticSearch" as value
- Search whatever you want in Search the catalog and select a record
- Search the link (Ctrl f) : "Elasticsearch Record :" , click on the link
- make sure that the pop-up open and you should see the elasticsearch result in JSON (the pop-up look the same as MARC preview)
- note fields are alphabetized
- from the command line delete the es record:
    curl -XDELETE es:9200/koha_kohadev_biblios/data/5
- click the preview link, it syas record not found
- check the logs - you see the error on missing record

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 23 Katrin Fischer 2023-04-10 17:08:07 UTC
Created attachment 149401 [details] [review]
Bug 18829: (QA follow-up) Fix punctuation, capitalization and page title

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 24 Katrin Fischer 2023-04-10 17:08:11 UTC
Created attachment 149402 [details] [review]
Bug 18829: (follow-up) Return 404 when record cannot be loaded
Comment 25 Katrin Fischer 2023-04-10 17:08:15 UTC
Created attachment 149403 [details] [review]
Bug 18829: (follow-up) Don't messup record encoding

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 26 Katrin Fischer 2023-04-10 17:08:18 UTC
Created attachment 149404 [details] [review]
Bug 18829: (QA follow-up) I18N, test character and breadcrumbs

* Removes a 'heart' character from the file, probably added for testing
* Corrects breadbrumbs to make QA test tool happy
* Makes JS message translatable (test by manipulating the link to an invalid record number in DOM)

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 27 Tomás Cohen Arazi 2023-04-14 18:39:08 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 29 Tomás Cohen Arazi 2023-04-17 11:26:39 UTC
Created attachment 149745 [details] [review]
Bug 18829: (QA follow-up) Replace forbidden indirect syntax

This pattern is forbidden and was overlooked by QA.

To test:
1. Run:
   $ ktd --shell
  k$ prove t/00-testcritic.t
=> FAIL: Tests fail with 'Subroutine "new" called using indirect
syntax...'
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 30 Matt Blenkinsop 2023-04-18 12:21:10 UTC
Enhancement - not backporting to 22.11.x

Nice work everyone!