API Search Results

alx wrote:
Hi,
I am using your /search API for my upcoming app. Awesome work!
In my app, users may search for TV shows. Beta testers have reported some odd behaviour when searching for TV shows.
For example, a search for "House of cards" returns the expected results, but a search for "House of car" yields 0 results.
Would it be possible to return similar results when invoking similar searches like I've depicted above?
If you need additional help/examples feel free to ask.
Again, really nice work with the API!

I think you're using the show single search, but you should instead use the show search.

MichaelDeBoey wrote:
I think you're using the show single search, but you should instead use the show search.

Hi

No, I am definitely using the version of search that supports multiple results.

The issue that I am habing is that it seems to only support full search queries such as "house of cards", but partial searches like "house of car" do not return any results.

I'd expect "house of car" to return similar results as "house of cards".

I think the Database query for searches needs to be changed to support this.

"house of cards" returns results: http://api.tvmaze.com/search/shows?q=house%20of%20...

"house of" returns results:

http://api.tvmaze.com/search/shows?q=house%20of

"house of car" does not return results:

http://api.tvmaze.com/search/shows?q=house%20of%20...

Similarly, "game" returns results: http://api.tvmaze.com/search/shows?q=game

"game o" returns 0 results: http://api.tvmaze.com/search/shows?q=game%20o

"game of" returns results: http://api.tvmaze.com/search/shows?q=game%20of

"game of th" returns 0 results: http://api.tvmaze.com/search/shows?q=game%20of%20t...

also 0 results, until you get to "game of throne"

It seems like only full words are taken into account when the search query is processed. Thus, "game of" and "game of throne" return results, but none of the search queries in between.

alx, you have to use "-" instead of " ". If you're looking for "game of thrones" you have to use http://api.tvmaze.com/search/shows?q=game-of-throne

bla0r wrote:
alx, you have to use "-" instead of " ". If you're looking for "game of thrones" you have to use http://api.tvmaze.com/search/shows?q=game-of-throne

Hm, I am getting the same results using "-". Still doesn't seem to work for "game o", i.e. ?q=game-o still returns 0 results.

I think there is something wrong with the fuzzy matching that prevents returning results for partial words.

Our fuzzy search uses automatic fuzziness: an allowed difference of 1 for words with more than 1 character; 2 for words with more than 5 characters.

So, "house of card" would return a match and indeed "house of car" won't.

What is the backend? Elasticsearch? Solr? SQL?

david wrote:
Our fuzzy search uses automatic fuzziness: an allowed difference of 1 for words with more than 1 character; 2 for words with more than 5 characters.
So, "house of card" would return a match and indeed "house of car" won't.

I understand. Is there a way to make cases like "house of car" at least return the results as if we would acually be looking for "house of"? This would make it look less like a bug in most applications I guess.

teambensalem wrote:
What is the backend? Elasticsearch? Solr? SQL?

Elasticsearch

alx wrote:
I understand. Is there a way to make cases like "house of car" at least return the results as if we would acually be looking for "house of"? This would make it look less like a bug in most applications I guess.

That's tricky, the more lenient the search function is, the more false positives.

david wrote:
That's tricky, the more lenient the search function is, the more false positives.

Yes I understand. Don't get me wrong, there is nothing inherently broken with search atm, just something that I wasn't used to in the past.

I think I can handle these cases on the front end to make the user experience a bit better. I am by far no expert in Elasticsearch but wanted to inspire a bit of discussion around this issue I was experiencing with the current implementation of the search.

Would it possible for you to expose search endpoints (maybe just a query param) to a) disable fuzzyness at all, b) change the fuzzyness to 2 or 3, just so that we can experiment with a few setups?

david wrote:
Elasticsearch

A programmer after my own heart. :) That was my guess since I saw the score.

For the other users on the site, an elasticsearch query is not as simple as a simple query. You can't just throw the query directly to the database (not like you ever want to anyway), but if you want a regex or wildcard query, it has to be built differently.

For an API that's supposed to be (and IS) speedy, in order to remain speedy, adding in parsing of your simple query with a wildcard so that we can use a different format to send to elasticsearch adds milliseconds to your return. It does not sound like much individually, but it's not about serving the individual, it's about serving the masses. Those milliseconds adds up.

I don't know what his front end is (if david said "nodejs", then please contact me, I'll code with you), but even simple functions like substr() on the front end running every time unnecessarily is not optimal. You won't care with 100 users, you might care with 10000 users.

A Levenshtein distance of more than 2 isn't even supported in Elasticsearch. It's highly unlikely that that's what you want anyway, as it would start returning a whole lot of irrelevant matches.

To match for queries like "house of ca", a whole different kind of query should be used, one that specifically looks at the beginning of the strings. We actually had such an implementation as well, but it turned out the underlying mechanism in Elasticsearch was majorly broken. Once that's fixed we might re-add it and expose it in the API as well, as a separate endpoint.

Wouldn't you be able to also use a (separate) prefix query; https://www.elastic.co/guide/en/elasticsearch/refe... which will search for those that start with the specified string - if you wish, only run it if 0 results are returned from the existing query.

I found another one of these. When searching for the Netflix show "Love", it appears third in the results after "Rove Live" and "The Lover".

srob650 wrote:
I found another one of these. When searching for the Netflix show "Love", it appears third in the results after "Rove Live" and "The Lover".

Damn, fixed now. The change has more impact; please let me know whether you think it's positive or negative.

david wrote:
Damn, fixed now. The change has more impact; please let me know whether you think it's positive or negative.

Thanks for the speedy fix! I'll keep an eye on other results and check back if things seem weird :)