Everything should have an API: adventures in trying to automate stuff

Inspired by Robb Knight I want to build my own /now page. As a teen I used to use PHPCurrently on my personal website to list what I was listening to, thinking, feeling, even what my MSN display picture was. Here's an objectively terrible screenshot from peak Evanescence phase, circa 2004.

And just as Robb did, I want to automate as much of it as I possibly could. No matter how many apps I try for tracking books, games, TV etc., I always forget to actually update them. Everything I do requires a sign-in these days, and it's all internet based, so why shouldn't I be able to automatically generate a page based on the data these companies have on me?
In short: they don't let you access it. Netflix, Nintendo, Kobo, PSN to an extent: there's no simple way of getting your history out of these apps. And most of the third-party apps that let you track your history, like GameTrack, have no way out.

At the time of writing this I haven't cracked it, but I'll share my findings so far because I'm sure I won't be the only one who wants to do this.

If anyone has any suggestions, let me know via Mastodon or email!

Games

Steam has an API because it was historically for nerds (source: am nerd), but I also play a lot of PlayStation and Nintendo Switch.

I tried using rawg.io which connects to your PSN account, but it doesn't have a personalised API (just an IMDB-style one for game info) and scraping didn't work because the whole site is a single-page app so the content isn't there without JavaScript. It also only logs what games you have, not what you've played recently - you have to do that bit manually.

Nintendo has no API at all, and no public profile pages, so there's no way of logging what you've been playing that I can find.

Robb scrapes his latest trophies from psnprofiles.com as a workaround, but I'd really like date-based data.

Music

This bit is actually pretty easy, even being an Apple Music user. I dusted off my old Last.fm account and downloaded Marvis Pro. This is like a layer on top of Apple Music with a better interface and automatic Last.fm scrobbling. (There is an official Last.fm app for iOS which picks up Apple Music plays, but it's got limitations and you have to scrobble semi-manually.)

I can then render that data pretty nicely using Robb's script.

Books

I stopped using Goodreads in 2020 when I de-Amazonified my life as much as I could, so I haven't been tracking what I'm reading. I have a Kobo eReader, sometimes I buy books from their shop, and I borrow a lot of books via Libby/OverDrive. OverDrive does apparently have an API, but I'm not sure how easy it'll be for an individual to get access. So manually tracking it is.

Manually tracking books

Two apps were recommended to me recently: Oku and Library Thing. (Side note: I love the two completely different aesthetics of these sites: Oku looks like someone copied the CSS from Notion, and Library Thing transports me straight back to 2003.)

Oku has an RSS feed for every user collection, which means I can easily grab the feed for "currently reading"! The feed also contains metadata like cover image, so I'll be able to render the books nicely (though quite a few books appear to be missing covers in their database). Its search is pretty bad, but if you feed it an ISBN it gets the book straight away.

LibraryThing has better book data and it does have APIs, but they're disabled right now and it's unclear if they're going to turn them back on again, so that's a dead end (at least for now). So Oku it is. Let's see if I remember to track what I'm reading this time.

Podcasts

My podcast app of choice is Pocket Casts, and there appears to be no official way of getting listening data out of that. But there does seem to be an API somewhere, as some folks have reverse engineered it. Authentication looks a bit dodgy (not about to store username and password anywhere) so I'll have to figure out how long that authorization token lasts.

If anyone knows a better podcast app for gathering this kind of data, let me know!

Recipes

I'd love to log what I've been cooking this week! On the weeks when I have my shit together I plan what we're going to cook using Paprika (where I store all my recipes - about 3000 of them!) - this is ripe for automation. Sadly, the app is pretty much "done" in as much as they don't add new stuff to it any more, and there's no API or anything. I've considered switching to Pestle, but there's no MacOS app yet which is unfortunately a dealbreaker for me. So it might have to be a manual process for the time being.

Paprika does have a calendar export option, where it provides a calendar feed of planned meals. I wonder if there's any way of intercepting that.

TV & films

Again, plenty of apps here for manually tracking what I'm watching, but no way of getting the data out that I could see. However, in my search today I found Trakt which does have a public API!

Part of the fun with such data is making it available for anyone to mash up and use on their own site or app. The Trakt API was made just for this purpose. It is very easy to use, you basically call a URL and get some JSON back.

That's what I'm talking about! Why bother tracking what you're playing/reading/watching if you can't then do fun stuff with that data? Recommendations are useful enough, but this is the good stuff.

The interface is confusing at best, and it's surprisingly fiddly to mark things as watched. It thinks I watched all 9 seasons of Breaking Bad today, as I marked them as watched in one tap.

Again, the biggest downside is I have to track manually, but at least that way I don't accidentally reveal to everyone that I'm three seasons in to Emily in Paris. But ultimately I'm not sure if I'll be able to stick it out (the app, not Emily in Paris).

Conclusion

At this point I can only conclude that I've spent longer investigating how I can automate this than I would have done manually updating the /now page once a week. But it's also a matter of remembering/having the energy to update it. I can guarantee you I'd update it twice and then leave it for several years.

It frustrates me how easy this would be if all these services had APIs that you could just pull your own data from. If our lives are online now, and our data is helping them personalise their services and make more money through recommendations, the least they could do is provide that data in a consumable format. It's not exactly feasible to do a monthly DSAR to get my Netflix viewing history.