API-index vs results data out of sync

peterrtv wrote 9 years ago: 1

Short Answer:
API: http://api.tvmaze.com/lookup/shows?thetvdb=304219
Correctly returns a 302 to http://api.tvmaze.com/shows/2174 BUT...
The record was updated last night so "now" the API finds it correctly (note the ID# in the URL is the correct one), however, the theTVdb# in the json data is still the old, incorrect, value. I tried it three different ways so I sure it is NOT cache error, at least not on my end.

Q. Is this something that will fix itself in the normal course of processes that run your infrastructure? Or is this something that we need to address on a record by record to identify and correct?

Background:
I figure your system likely dumps endpoints to static files to lighten the load on the database. If so, does that happen on a trigger or timed event, or is it something that needs to be pushed manually? Or worse, did some process fail and result in widespread data corruption?

I have an existing process which uses theTVdb's data/xml as a source to dynamically create a calendar of un-aired episodes. It is automated and updates nightly. Given that you guys have a fairly robust API, and one that lets me search using the theTVdb#, it makes it easy to maintain both, or so one would think... of the 721 series records, 54 failed, most because the IMDB and theTVdb IDs' were missing . I have submitted detail information in the data forum thread, and those have been already updated. However, the first test case, the record shown above, is still returning bad data, even though the API query now works.

Now, I have not yet, but can and likely will, cross ref the rest to see that the ID values line up correctly. But before I spend any time coding to that end, I thought I should share the facts so far. Also no point in coding to fix an issue that might fix itself anyway...right? ;-)

FYI: I have a background in IT, I used to do this kind of thing with millions of auto part records...this stuff it child's play by contrast. Anything I can do to help, just ask...


david wrote 9 years ago: 1

You're right, this is on our end. There's a long-standing bug where updates to a show's external ID's doesnt trigger a cache invalidation, which means you can be served stale data for a while.

I'll work on fixing it!

peterrtv wrote 9 years ago: 1

Thanks!

If there is anything I can do to help, just ask.

Q. If "stale data for a while" has a TTL, please advise, otherwise I presume that it's not until some other update triggers a cache update.

Funny, best practice would suggest clearing cache periodically, but if you do, bugs like this could go undetected indefinitely.


david wrote 9 years ago: 1

There might be a few stale entries left, but it won't happen anymore in the future. :)

Try 30 days of free premium.