Greg Reinacker’s Weblog

Last time I wrote about this (seems like yesterday, but it’s actually been a while!) I talked a bit about security for RSS feeds, and in particular authentication, authorization, and encryption.

At the end of that post, I promised to follow it with another post talking about how an online aggregator should be dealing with this stuff. So here we are. :-)

There are a number of problem areas for an online, web-based aggregator when dealing with authenticated feeds (that is, feeds that require authentication to access). I’ll take them one at a time.

Dealing with URL’s and credentials

This is the most visible problem with some online aggregators, and the one that has given authenticated feeds a bad name. Or more correctly, given online aggregators a bad name.

The fundamental problem here is that the aggregator needs to access the secured feed on behalf of the user, which means it needs the user’s credentials for the feed. So immediately we have a trust issue, but we’ll assume we’re past that.

Some aggregators in the past have used the following “URL” format to access feeds that require credentials:

http://user:pass@www.example.org/thefeed.rss

This is a shortcut way to save a URL with “embedded” credentials – but here’s a secret.  It’s not really a URL.  It’s just a shortcut way to mark it up.  When the request goes out on the wire, it doesn’t look like that – it gets split into separate credentials and URL, and the authentication handshake actually happens behind the scenes as you’d expect.

But, for aggregators that don’t support credential management, this was a cheap easy workaround to access feeds that require credentials, if your users were clever enough to figure out this shortcut format.

But here’s the problem.  Users routinely expose their OPML files to themselves, their friends, their blog, or wherever else.  And in this case, guess what would show up in the OPML file?  Yep, that whole credential-embedded-URL-thing.  Not good.  If a user puts that OPML on his blog as a blogroll, he’s basically advertising his username and password for that feed to the entire world…and the worst part?  He probably didn’t know he did it.  Users will go to some effort to protect their private information – but not if they don’t know they’re exposing it.

How to fix this?  Don’t build an aggregator this way.  Prompt for credentials in separate fields on your interface, and store them separately.

Content isolation

Big problem #2 is the fact that a secured feed may have different content for different users.

Fundamentally, web-based online aggregators work by retrieving each feed once, and distributing that content to all of their subscribers.  For unsecured content, given a particular feed, the content in the feed will be the same if you retrieve it multiple times, so you only retrieve it once.  This is, in general, very efficient for an online aggregator.

But suppose I have a feed that requires authentication, at

http://www.example.org/thefeed.rss

If I access this feed with my credentials, and you access the feed with your credentials, we may get different content.  This is key.

What does it mean?  Content from secured feeds generally needs to be isolated to the particular user it has been retrieved for – and not shared with any other user (unless they’re sharing the same credentials).

Indexing

Problem #3 is related to indexing content.  This one is easy…basically, if you retrieve content using a user’s credentials, that content may be private – and thus must not be indexed in a shared index store.

Incidentally, NewsGator Online follows the best practices outlined in this post.  It supports separate credential management, it sandboxes/isolates content, and does not index private content.

So that’s it for this chapter of the RSS security saga.  I’m not sure what the next chapter should be about – are we done?  Let me know what you think!

On the heels of our NewsGator Online API launch, NewsGator has announced an API developer competition…basically, we’re looking for the coolest applications that use the NewsGator API, in three categories:

• Windows applications
• Mac applications
• Cross platform/mobile/web applications

I’ll be at the PDC conference for most of this week, along with two of our system architects…if you’re there, and you’d like to meet up and chat, send me a note.  I’ll also be on a panel, “The Future of RSS“, on Friday morning – be sure to stop by!

As many of you know, Nick Bradbury had surgery on Tuesday 9/6 for an acoustic neuroma.  I’m happy to report that the doctor didn’t sneeze - Nick is doing very well, is in good spirits, and has started his rehab process.

Please join me and all of us at NewsGator in wishing Nick and his family well, and a speedy recovery.

I was on a call the other day with some folks in the industry, and someone made a comment to the effect of “we really need to come up with some kind of solution for securing RSS feeds – then we can really do some cool stuff.” Before I could get on my soapbox, someone else on the call concurred with the first person.  When I mentioned that this stuff has been figured out already, and started describing the existing widely-used mechanisms, they were both a bit surprised, and suggested I write something about it.  So here we go. :-)

There has been much talk about “RSS security”. The problem is, that’s such a vague term, and you can’t really make much progress until you actually break that down into what you really mean. So let’s give that a shot.

When most folks are talking about security with RSS, they tend to mean three things (or any combination of these):

authentication

authorization

encryption

Let’s take these topics one at a time.

Authentication – this basically means that you want to be able to identify the user who is requesting your feed. There are well-known solutions here – in particular, the tried-and-true HTTP authentication mechanisms, including Basic and Digest. These are the same authentication mechanisms we use on the internet for subscriber-only web sites, and they are equally applicable to RSS.  Behind the firewall, you could add NTLM/Kerberos authentication for Windows networks.

Most mature RSS aggregation tools (NewsGator Online, NewsGator Outlook edition, FeedDemon, NetNewsWire, lots of others) support these widely-accepted authentication mechanisms.

And for those using Windows servers, I’ve even written some sample code for .NET that implements these protocols on the server side, without using the built-in IIS implementation with Active Directory.

Authorization – this means that once you know who the user is, should they be allowed to access the content they are requesting?  This is the easy part…once you’ve authenticated the user requesting the feed, you know who they are, so you can now decide whether they should be allowed to access the content.  Again – just like regular web sites.

Encryption – this means that if someone is watching my network with a network sniffer of some sort, or they’ve managed to insert themselves in the network path between the client and server, then they would be able to see the content going by…so we encrypt it.  The easy way to do this is with SSL – again, exactly how we do it for regular web sites.  And this is usually as easy as adding a certificate to your web server(s), and changing your URL to https.

Piece of cake. Use the mechanisms that have been around for 10+ years securing web content, and you won’t go wrong.

“But wait,” you say. The authentication and encryption mechanisms I mention above are tied to HTTP.  What if you want to transport a feed via some other protocol?

Well then you’d have to think of something else.  You’d likely see what authentication mechanisms are supported by your new transport of choice, and leverage those.  Hmm.

My advice for now?  Don’t worry about it.  RSS today is transported via HTTP.  Sure, you could use other protocols – but almost no one does.  This same argument came up some time ago about SOAP web services…a lot of work went into making sure everything was portable enough to deliver SOAP messages through any arbitrary transport.  But in real life?  Almost no one is doing it.

We don’t need more protocols. We don’t need yet another encryption standard. We don’t need yet another authentication mechanism. Use what works today – it’s proven itself already.

I’m going to write some more about this in the next few days – first, about how an online aggregator should and shouldn’t implement this, and what to watch out for as a user.  There have been some serious problems in the past with flawed implementations of authenticated feed handling, and it’s caused some problems in the community for users…so I’ll write about that as soon as I get a chance.

A few months ago, this blog was crawling with comment spam. Yikes.

Spammers – 1.  Greg – 0.

So around that time, I implemented a CAPTCHA validator here on my blog, so to leave a comment you’d have to actually read the funny letters, and type them in. All was well for a while…this actually eliminated comment spam here for about two weeks.  Woo-hoo!

Spammers – 1.  Greg - 1.

Then the spammers came back, armed with a new resolve.  And my comment feed became useless again, filled with spam before I could delete it.

Spammers – 2.  Greg – 1.

So now, I finally broke down. Comments on posts that are older than 14 days are now moderated. I’m hoping this takes care of most of the problem, since 99% of the comment spam here is on older posts. If the spammers actually take time to read this post and start spamming the recent items, then, well, I’ll have to think of something else.  But in the meantime, there’s been no spam for 30 minutes! :-)

Spammers – 2.  Greg – 2.

Anyway, if you’ve said goodbye to my comments feed in the past due to comment spam, come back!  It should hopefully be much cleaner now. And if you’re using NewsGator Online, here’s a handy link to get it back. :-)