Unfortunately, the Zebra charmap files do not have a command for mapping a character to 'null'.  Right now, apostrophe is mapped to 'space', which serves to break up words.  So searching for "harrison's internal" searches for three words: "harrison", "s" and "internal".

We may be able to explicitly map "'s" to "s"... but then we'd need to do that for EVERY possible case where apostrophe is used....

ICU might fix this yeah?

Has anyone tried contacting Index Data about the inability to map a character to null?

Not sure if this applies.  I was doing search testing even with fuzzy search.

I tried "1000 things i love about" with no results, though the title was "1,000 things i love".  I t only found "1,000 things i love". 
So comma's or other locale specific formatting in words that are all digits might be a similiar use case.

When we make a search for titles like "Você viu meu pai por aí?", that has a "?", we have results only with a search for "Você viu meu pai por aí" (without "?"). The same problem when the search term has "!". For dots or commas this problem doesn't occurs.

(In reply to comment #4)
> When we make a search for titles like "Você viu meu pai por aí?", that has a
> "?", we have results only with a search for "Você viu meu pai por aí"
> (without "?"). The same problem when the search term has "!". For dots or
> commas this problem doesn't occurs.

You will be glad to know that the search rewrite (which is already underway) will fix this particular issue.

I'm wondering if this also includes the ampersand (&) ? I'm having an issues with searching for a title that includes it - but wasn't sure if it was a new bug.

Just to add my two cents, a user could want to search for punctuation. (So searching for "C++" actually searches for "C", which is undesirable.) Is this intended?

Having this problem with Xmen and X-men in our catalog  (3 results versus 235).  Actually X men and X-men gives same number of results so maybe that's the best we can do for now?

We're seeing this when we do searches between the word "don't" and "dont".  This is a real, ongoing problem.  I'm wondering if there is anything that can be done outside of waiting for ElasticSearch?

We should fix this taking advantage of DOM: i.e. generating indexes for the strings with punctuation removed. It could be tricky in some scenarios, but it should work.

I think also ICU could help/work here - I remember we tested both, mapping - to a space and to nothing, only thing that didn't seem possible is doing both at the same time.

Is there any update in regards to this particular issue? I'm running Koha 17.05 and are experiencing the same problem.

One example is we have an entry for "Expositor's Dictionary". The only way it will return a result is by searching for "expositor" or "expositor's", but the average person seems to type "expositors" and it returns no results. A typical search engine in 2017 is generally able to handle this, so this is the prevailing user expectation these days.

Are there any plans to resolve this sort of usability issue? We've had multiple people report this problem over the last year when searching for various items in our catalog that contain an apostrophe. If there's no fix in the works, is there a decent workaround that can be implemented in the meantime to smooth this over for our users?

Hi, I suggest filing a separate bug with your example - Zebra is able to do things like that, but there are known issues with quotes and similar. Sometimes it also depends on configuration (ICU vs. grs-1, installation choices, etc...).
Please include information about how you installed Koha and if you changed defaults etc. on installation if possible.

Sorry, I see now that this here all started with the discussion about apostrophes - just a lot of information and other problems have been added since. Maybe splitting this into several bugs would still be good - we can link them all to this one.

Is this bug specifically tied to Zebra indexing or with this also be the case with ElasticSearch as Koha users move towards that? Thanks.

I think I'd put Elasticsearch related issues on a separate bug - most of the discussion here applies to Zebra and both have different configuration files.

We originally installed Koha around May or June of 2016 and would have been on the 3.22 version that was released at that time. We used defaults for everything as far as I remember and it appears that we're using Zebra. Is it possible to switch to ElasticSearch and is it fairly involved to do so? Are there any other considerations to take into account before doing so? Is ElasticSearch production-ready?

Thanks!

(In reply to dguidry from comment #17)
> We originally installed Koha around May or June of 2016 and would have been
> on the 3.22 version that was released at that time. We used defaults for
> everything as far as I remember and it appears that we're using Zebra. Is it
> possible to switch to ElasticSearch and is it fairly involved to do so? Are
> there any other considerations to take into account before doing so? Is
> ElasticSearch production-ready?
> 
> Thanks!

Elasticsearch is still in testing phase I would say,there are a few things that need to be cleaned up to be production ready, most of them are small issues, but together they would cause some pain points. Most of them relate to indexing configuration and Koha expectations - if you don't define expected fields sortable - sorting causes a failed search for example

Setting it up should be fairly straight forward - Mirko has put together a 'koha-elasticsearch' package which will install needed dependencies for koha, and you will need to install Java and Elasticsearch itself.

I will work on some of the issues in the coming weeks and update the status page on the wiki and send something to the list.

We do need testers and are working on enabling ES on some sandboxes and will update when that work is ready.

For folk using Zebra in English, QueryStemming can help with this, although QueryStemming only works if automatic QueryTruncation isn't turned on.