Hi, when you add a new subfield in a marc bibliographic framework and that subfield is not in the marc format normally, you can not add it as a item search fields. It would be useful to be able to use these new subfields for item search for exemple. Thanks, Patrick.
Do you mean capital letter subfields?
Hi Katrin, It is exactly what I mean. Thanks, Patrick.
Created attachment 77640 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly
Created attachment 79554 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly
Created attachment 79779 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Great enhancement Katrin ! Thanks a lot !
(In reply to Séverine Queune from comment #6) > Great enhancement Katrin ! > Thanks a lot ! Thx for testing!
I may be a bad person but, ofc, I tried to find a bug. So I filled field="~" and subfield="€" The search did not like it :) I think we should keep the list of field/subfield exhaustive, and maybe add select2 to these 2 selects? Or at least add validation code to catch possible problems. Note: You will need the patch I have just submitted on bug 13618 comment 261 to test it correctly.
We don't do this in the MARC frameworks and I think some odd characters would be allowed in the standard... I've thought about it and tried to implement an HTML filter, but didn't succeed. But we are not that strict in other places and I think a list with 999+ elements is not the solution :) As I said, we even fields with letters in our database.
So maybe just a JS check for \d | \w ?
Checking for digits and word characters as per Jonathans suggestion sounds sane to me... is there really a case out there where a field may contain another character outside that range?
Created attachment 80538 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries.
Created attachment 80539 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Not sure if we should facilitate MARC deviation ?
We could also start supporting other ASCII characters like ;/% for subfield codes, that would give us more subfields for items to use - but it won't work right now and the upper case letter use is a reality not only for us but other libraries too (see Severine's comment).
Created attachment 80712 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 80713 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
(In reply to Marcel de Rooy from comment #14) > Not sure if we should facilitate MARC deviation ? I don't think this is too terrible a deviation, especially as it appears to already be widely in use.
It all works as expected for me and I can't find any regressions. Passing QA
Sorry but... the validation does not work when a new field is added, only when it is edited.
Created attachment 81137 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel.
Created attachment 81150 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Created attachment 81151 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Maybe I'm being stupid.. but this doesn't seem to be working for me. First off the validation doesn't seem to be activating on edit or new for me ( managed to enter 1~$ for example) Second, when I go to edit, I'm faced with an empty form rather than one that contains the existing data for me to edit. What am I doing wrong ?
Created attachment 83795 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Created attachment 83796 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Created attachment 83827 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Jose-Mario <jose-mario.monteiro-santos@inLibro.com>
Owen, I see pattern="[a-zA-Z0-9]+ and jQuery.validator.addMethod("marcfield", function(value, element) { return this.optional(element) || /^[0-9a-zA-Z]+$/.test(value); }, _("Please enter letters or numbers") ); Should not we have only 1 check?
(In reply to Jonathan Druart from comment #28) > Should not we have only 1 check? The pattern attribute in the markup is a non-JS fallback to the jQuery validation. It's not strictly necessary since we require JS, but it feels like good practice to me.
My concern it that they must be identical, but nothing indicate it (reading the code). In that case why the JS check when the patterns is enough?
It gives us a nicer error message in the correct GUI language I guess?
(In reply to Jonathan Druart from comment #30) > My concern it that they must be identical, but nothing indicate it (reading > the code). Similar to how the maxlength attribute on input fields should match the size of database columns and those get out of sync. I can submit a followup which adds comments if you'd like.
Created attachment 87955 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Created attachment 87956 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Jose-Mario <jose-mario.monteiro-santos@inLibro.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Created attachment 87957 [details] [review] Bug 20689 Follow - tiny typo fix
I had two issues here: 1 - When editing the Item search fields the values set previously don't populate to the form i.e. create a new fsearch fields for 520$A - edit it and you must set the field/subfield again 2 - While other fields worked, setting 520$A and searching for a unique value in that field failed, not sure why
Created attachment 88294 [details] [review] Bug 20689: (follow-up) Correct scope of template variables The removal of the <select>s should have been accompanied by a correction to the scope of tagfield and tagsubfield. It is also unnecessary to conditionally include the "value" attribute. To test, apply the patch and test the process for editing an item search field. All the correct values should appear in form fields during edit, and all values should save correctly.
My patch doesn't address this issue: > 2 - While other fields worked, setting 520$A and searching for a unique > value in that field failed, not sure why I was unable to reproduce the problem. I modified a framework to include 520$A, added a value to a record, did an item search for that value, and the correct result was returned.
The problem seems to be in ExtractValue: SELECT ExtractValue(metadata, '//record/datafield[@tag="520"]/subfield[@code="A"]') FROM biblio_metadata WHERE ExtractValue(metadata, '//record/datafield[@tag="520"]/subfield[@code="A"]') !=""; This returns both 'A' and 'a' subfields We need to figure a method to make the query case sensitive
(In reply to Nick Clemens from comment #39) > We need to figure a method to make the query case sensitive Looks like we can set COLLATE like : select city_name from cities where city_name='Paris' COLLATE utf8mb4_bin;
(In reply to Nick Clemens from comment #39) > The problem seems to be in ExtractValue: ... > This returns both 'A' and 'a' subfields Should this be considered another bug, which this bug should be made dependent on?
(In reply to Owen Leonard from comment #41) > (In reply to Nick Clemens from comment #39) > > The problem seems to be in ExtractValue: > ... > > This returns both 'A' and 'a' subfields > > Should this be considered another bug, which this bug should be made > dependent on? I think it can be done here or on another bug - it should be sorted before this is pushed though
*** Bug 13561 has been marked as a duplicate of this bug. ***
*** Bug 15384 has been marked as a duplicate of this bug. ***
(In reply to Fridolin Somers from comment #40) > (In reply to Nick Clemens from comment #39) > > We need to figure a method to make the query case sensitive > > Looks like we can set COLLATE like : > select city_name from cities where city_name='Paris' COLLATE utf8mb4_bin; I believe that is not quite the solution yet, as it would affect the strings we compare to instead of only the subfield codes, as we intend.
Created attachment 124056 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Created attachment 124057 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Jose-Mario <jose-mario.monteiro-santos@inLibro.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Created attachment 124058 [details] [review] Bug 20689: (follow-up) Tiny typo fix Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Created attachment 124059 [details] [review] Bug 20689: (follow-up) Correct scope of template variables The removal of the <select>s should have been accompanied by a correction to the scope of tagfield and tagsubfield. It is also unnecessary to conditionally include the "value" attribute. To test, apply the patch and test the process for editing an item search field. All the correct values should appear in form fields during edit, and all values should save correctly. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
I've fixed a little conflict, signed the patches off, filed bug 28892 for the search issue and... would like to throw them back into the queue. I feel like we should still push these patches, even with the search issue: * To me this patch set provides a great usability improvement to the item search fields form as it gets rid of the mighty long pull downs that were hard to handle. * The search issue already exists right now: if you are using capital subfields you will find their contents already mixed with the configurable lower case subfields. It doesn't work correctly now - that's why I made bug 28892 a "see also" instead of a "blocks" or "depends on". * At the moment you could get around the form limitation by configuring the lower case letter for your upper case subfields. But as the MARC field and subfield are part of the displayed search option description, I think it will be nicer to allow the correct thing to be entered in the first place here. * The search issue is probably only going to 'hit', when upper case subfields are used in bibliographic records. The more common use case for this kind of subfield are item fields, as all the lower case letters and numbers are already used up in MARC21. This also means, that there is a really high chance that only the capital letter subfield will be queried via more_subfields_xml, as the lower case one is very likely to be mapped to a database field. So in this case, the results will be accurate. * The new form allows entering capital letter subfields, but it doesn't advertise it. If people think it's better we could change the bug title to highlight this as a usability improvement ... or we could add a hint explaining the limitation to the configuration page.
Note: only the first patch was from me - as that had been signed off, I felt I could for the follow-ups.
(In reply to Katrin Fischer from comment #15) > We could also start supporting other ASCII characters like ;/% for subfield > codes.. Does not sound like a good plan to me.
I have the feeling here that we took the wrong road and now we are on it, lets just continue following it.
(In reply to Marcel de Rooy from comment #53) > I have the feeling here that we took the wrong road and now we are on it, > lets just continue following it. Why? Can you detail?
(In reply to Jonathan Druart from comment #54) > (In reply to Marcel de Rooy from comment #53) > > I have the feeling here that we took the wrong road and now we are on it, > > lets just continue following it. > > Why? Can you detail? Something like: We are in a hybrid situation where we actually left MARC for our items to a certain degree, but we stick to the subfields stuff. And still mimic some kind of MARC compliance. While obviously the capital letters are not MARC compliant. If we need more item columns and want to leave MARC here, we c/should do a better job than adding capitals and even other chars as subfields. We dont need them anymore. We should just use the real column names. No need for whatever character to point to a new item column, just use its name. Refactor instead of staying in between? And provide another way to export to MARC. Etc.
> Refactor instead of staying in between? The first step took 10 months to be pushed... It's not very motivating to continue.
(In reply to Marcel de Rooy from comment #52) > (In reply to Katrin Fischer from comment #15) > > We could also start supporting other ASCII characters like ;/% for subfield > > codes.. > > Does not sound like a good plan to me. I was suggesting it only because I thought I had run into this at some point in documentation - but might have been a nightmare :) I am not sure how we could support that much information easily within MARC. With Zebra, we need to stick to some conventions to have indexing work, so using a totally outside of MARC solution will probably not work. Also a lot of the code and modules we use expect a one character subfield code. The only thing I could think of is something like using some subfield with prefixes for different content... adjust indexing etc. I really believe using capital letters is a reasonable compromise to keep things easy to handle.
Created attachment 131038 [details] [review] Bug 20689: Replace marc (sub)field pull downs with input fields on item search fields config The pull down for marc subfields on the item search fields config page is long and hard to use. It also doesn't include the commonly used upper case letters for custom item subfields. Also hard to use with its 999 entries is the marc field list. It doesn't allow for tags like 01e, which exist in our MARC21 default framework. To test: - Go to Administration > Item search fields - Add different mappings with and without subfields - Verify the entered values are stored correctly - Edit mappings - Verify editing works and changes save correctly Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 131039 [details] [review] Bug 20689: (follow-up) Make it possible to add capital letter subfields to item search fields This patch adds some custom validation to the MARC tag and subfield fields so that they are limited to alphanumeric characters. Both templates (the main view and the edit view) have been modified so that item_search_fields.js can be included in both. To test, apply the patch and test the form by entering a variety of different character combinations. The "MARC field" and "MARC subfield" inputs should only accept alphanumeric entries. Test both "new" and "edit" operations. Test other operations like delete and cancel. Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Jose-Mario <jose-mario.monteiro-santos@inLibro.com> Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 131040 [details] [review] Bug 20689: (follow-up) Tiny typo fix Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 131041 [details] [review] Bug 20689: (follow-up) Correct scope of template variables The removal of the <select>s should have been accompanied by a correction to the scope of tagfield and tagsubfield. It is also unnecessary to conditionally include the "value" attribute. To test, apply the patch and test the process for editing an item search field. All the correct values should appear in form fields during edit, and all values should save correctly. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
I think the compromises here are reasonable and the timescale it's taken for this to work through makes me feel it's time to let it get in and if someone wishes to work on it further they can do in another bug. Passing QA
If it makes things a little easier, we could change description to: Improve usability of Item search fields administration page
Pushed to master for 22.05, thanks to everybody involved [U+1F984]