Planned text-only import: third-party exceptions and deletion tracking

nayevna wrote 12 days ago: 1

Hello TVmaze team,

We run Takibi (https://takibi.pub), a small commercial cultural-content library in South Korea operated by Nobyeon Jeongdam (노변정담). We plan a text-only use of show and episode summaries and cast/character links under your CC BY-SA terms, with a per-entry link to the TVmaze page, the licence, and a notice when we translate. No images, video or user data.

Scale, so you know in advance: we would walk the show index once (about 350 pages), keep only shows that match our catalogue by external ID, and fetch cast and episodes for those, roughly 40,000 requests in total if about 20,000 shows match. We would go well below your stated limit (about one request per second, one at a time), cache responses, identify ourselves with our own User-Agent and back off on 429. After that we would follow the updates endpoint.

Two questions:
1. Are there summary fields that contain third-party text which a downstream service should leave out?
2. What is the best way to detect deleted shows and removed cast/character records, so we do not keep stale text?

If you would prefer a different route or pace for the initial import, we are happy to follow it. Thank you for the API and its documentation.

Takibi / Jam · Nobyeon Jeongdam (노변정담), South Korea
https://takibi.pub/jam · support@takibi.pub


gazza911 wrote 8 days ago: 1

All of the summaries (e.g. Shows, Networks, etc.) are written by volunteers. However, they have to adhere to the copyright policy.

The TVmaze website has an option to allow an editor to mark it as "The description is from an official source", however, that is not exposed via the API.

For detecting deleted shows, the easiest way would be to call https://api.tvmaze.com/updates/shows; if the ID is not in that list, it has been deleted.
I'd recommend calling either of the day, week, or month endpoints on the regular interval, and the full endpoint less often (e.g. every 3 months).

Unfortunately, there isn't an updates endpoint for cast/characters, so if the show is in the updates list, retrieve the cast list again and compare to your own. 

Try 30 days of free premium.