Publishers and user-agents

When an application makes a request for a web resource, it includes a user-agent string in that request to identify itself to the server. The server may use this for any number of reasons like serving up mobile vs desktop versions of websites, analytics, etc.

When Downcast first came out, it used iTunes' user-agent due to the fact that a large number of publishers' servers returned invalid responses to feed requests containing any other user-agent. The invalid responses include 404 errors, the wrong content type (e.g. an html version of the feed), a PDF, or even a valid RSS feed that lacks the important enclosure tag. Using iTunes' user-agent gave Downcast the most reliable responses across the board to feed requests.

Unfortunately, doing this results in skewed usage statistics for Downcast, iTunes, and publishers that aren't doing anything bizarre with their feeds based on user-agent.

Downcast was recently updated to use it's own user-agent¹. This has gone largely unnoticed since nearly every publisher handles user-agents correctly when it comes to serving up RSS feeds now. Sadly, a tiny minority of publishers still fail to return valid content to Downcast as of this user-agent change.

If you're experiencing an issue with a feed that suddenly doesn't refresh or shows 'No Media File' for each episode, contact support so I can check it out to see if it is affected by this problem. If confirmed I'll contact the publisher to request they correct the problem.

3/9/2016 - Update - Downcast's responsibility is to create the best experience possible. A few popular podcasts are affected by this (anything from WNYC), which means I can't just wait for publishers to correct the issue, they probably won't anyway. So, the following changes will be made in v2.9.13:

• Feeds that include episode entries with no media:content or enclosure tags.
  • These will be tracked but hidden from view. From the user's perspective, they will be ignored.
  • When a feed is found to include episodes like this, a notice will be displayed at the top of the feed's episode list letting you know there are hidden episodes with no media files.
  • You'll be able to override the behavior so these episodes will be visible if you'd like.
  • You'll be able to hide the warning notice going forward.
• For feeds that return invalid content-types, 404's, etc. only for Downcast's user-agents.
  • Once reported, if I can confirm these feeds work when using a different user-agent, like iTunes, they'll be added to a list of feeds for which an alternative user-agent should be used, which Downcast will fetch from our server.

3/20/2016 - Update - Making this a two phase change...

For Version 2.9.13

• Feeds that include episode entries with no enclosure tags.
  • These will be tracked but effectively ignored, you won't see them.
  • If an enclosure tag is included but doesn't contain a url, it will not be ignored. You'll continue to see the No Media message for these...it's more likely these are errors than intentionally excluded by the publisher.

For Version 2.9.15

• For feeds that return invalid content-types, 404's, etc. only for Downcast's user-agents¹.
  • When one of these is confirmed, Downcast will just start passing an alternative user-agent that the publisher will correctly respond to, despite the invalid analytics statistics it will generate.