Complete dataset

RemiHin wrote a year ago: 1

Im looking to cache the entire dataset locally for my app. Using the index endpoint makes sure i dont reach the ratelimit, but it only includes the shows, there is no embedding possible here (right?).

So if i also want all the episodes and cast and crew and guestcast and guestcrew... thats a lot of requests to a lot of endpoints. Which is no fun for me, nor is it for you.

I was hoping there was a way to get the entire dataset in one go, either as a file download, or the ability to embed everything on the index (but that seems heavy).

Any chance you can help with this?

david wrote a year ago: 1

I don't think it will make sense to have an index endpoint for something like episodes (the show index is currently 267 pages long, but a similar episode index would have around 10000 pages!)

But if you stick to the rules in https://www.tvmaze.com/api#rate-limiting, feel free to manually retrieve them one-by-one, e.g. by iterating over all possible /shows/:id/episodes endpoints.

RemiHin wrote a year ago: 1

Alternatively, are nested embeds possible? If not can i request them added to the roadmap?
shows/:id?embed[]=episodes&embed[]=episodes.guestcrew&embed[]=episodes.guestcast

david wrote a year ago: 1

Ha, sorry but no, recursive embedding won't be possible.

david wrote a year ago: 1

Embedding is a compromise to make life easier in some of the more basic use cases, but if you really need a lot of different data please don't be too afraid of issuing multiple requests.

As described in the documentation:

Rate limiting takes place on the backend but not on the edge cache. So if your client is only requesting common/popular endpoints like shows or episodes (as opposed to more unique endpoints like searches or embedding), you're likely to never hit the limit.

This means that even if recursive embedding were to be implemented, the throughput would probably be significantly lower than if you'd request these endpoints separately because the latter is more likely to be served from cache. Make sure your client supports HTTP2 and there won't be any extra connection overhead either.

RemiHin wrote a year ago: 1

Awesome, thanks David