Category Archives: newsgator

Do More Faster

If you haven’t yet, you should take a look at Do More Faster, a book by Brad Feld and David Cohen about the experiences of early-stage investors and entrepreneurs.

I’ve even got a chapter in there myself, called “Don’t Plan, Prototype!” It’s essentially a couple of pages talking about the very early stages of NewsGator, from the sitting-around-scratching-my-head stage up to our first venture financing. I wrote it and slipped it into a stack of manuscripts David had laying around…somehow they didn’t notice, and it made it into the book. :)

Anyway, you can pre-order the book now at Amazon. Check it out!

Changes afoot

Over 7 years ago, I wrote a blog post that changed my life. It was just a screenshot of some prototype code, and a short 15 words. But it inspired the creation of NewsGator; about six weeks later, I released NewsGator 1.0 and the journey had begun in earnest.

Now, 7 years later, NewsGator is much more than I ever imagined it would become. It has found its market in enterprise social computing software, and employs over 80 people in multiple countries. The team of folks at NewsGator are the best I have ever worked with; they are truly top-notch in every way. And in 2009, the company exceeded all of its revenue goals for every quarter and for the year as a whole, and counts many of the largest companies and governments in the world as its customers.

It is with this wind at my back that I have decided to move on from NewsGator, and find the next thing for me. What that next thing is I don’t know; I may spend more time with my fashion photography, I may spend more time trading, I might do some consulting work, or I might involve myself with another startup. The future is wide open, as they say.

I’m tremendously proud of what we’ve accomplished, and happy that I had the chance to build this company from one guy in a spare bedroom, to what it is today. I will remain on NewsGator’s board, and I look forward to whatever happens next!

FeedDemon, NewsGator, and Mr. Bradbury

Way back in 2005, NewsGator acquired FeedDemon. I vividly remember sitting down with Nick Bradbury and talking about our shared vision for the future of FeedDemon and the NewsGator platform…and many a night drinking a lot of beer and talking about RSS and what we now call social computing. Through it all, Nick remained laser-focused on both the future of FeedDemon, and his customers and how they would be brought forward.

Along the way, we added NetNewsWire to the family, as well as SmartFeed (renamed NewsGator Go! for Windows Mobile). All of it was part of building out the original vision for a core online content platform, and best-of-breed applications on nearly any device to consume content. All in all, I think we delivered on this vision – we built the platform, added the best applications on the most popular platforms, and made it all work together.

Since then, as you may have gathered, our enterprise business has grown faster than we anticipated (this is a good thing!). As the company has started to focus more and more on enterprise customers, we made the difficult decision to shut down NewsGator Online, and focus our online platform in on our commercial clients. As part of this transition, most of our client applications (FeedDemon, NetNewsWire, and NetNewsWire for iPhone) were re-released to sync with Google Reader as an online store, rather than NewsGator Online.

As part of that transition, Nick has gone “back into the wild” as an independent developer. FeedDemon remains a NewsGator product, but Nick is 100% focused on it, and he has complete control over the product direction and feature set. I like to think of this not as the end of anything, but rather the beginning of the next phase of FeedDemon’s life. Just as the initial deal with NewsGator opened up new opportunities for it, so does this new direction…and FeedDemon customers will see lots of exciting things coming up.

On a personal note, I’d like to extend a big “thank you” to Nick, on behalf of myself and everyone at NewsGator. Nick has been instrumental in forming our consumer product direction, always makes sure we’re taking care of our customers in the best way we can, and has provided a huge amount of input on our other products. He’s been a very influential person at the company, and we look forward to this continuing in the future!

Enterprise RSS – the State of the Industry

There has been much talk over the last few days about Enterprise RSS, and whether it’s dead, still being born, or alive and well. Since I’ve got a pretty unique view into this particular industry, I thought I’d write some thoughts and try to give you a more clear picture of what’s really happening.

First, let me get this out of the way – RSS use in the enterprise is definitely alive and well. But it’s not in the obvious places. No one is writing articles talking about how their desktop feed readers are revolutionizing the way they do business. No one is talking about how they’re retiring their Exchange servers because so much content is delivered via RSS instead of email (and in fact, email is alive and well). No one is saying “if I only had Google Reader behind my firewall, I could save millions of dollars.” Few companies even say their users are clamoring for some sort of enterprise RSS application.

So if not all of that, then what?

My team and I, collectively, have detailed conversations with at least 50 different large companies every week, talking about the real problems they do want to solve. Many of these include 10 or more people on their side, ranging from IT folks to business owners with line-of-business responsibility. And these conversations rarely start with any mention of enterprise RSS. These include:

  • Portal enhancement. They want to pull additional content into their portal to make it more useful, or get content out of the portal to distribute in some other way. Or in many cases, they want to add personalized content capabilities to the portal. A while back, an analyst told me “this could be the technology that saves the corporate portal.” Indeed – that scenario is playing out today.
  • Alerting. There are many use cases where folks want to alert certain groups or individuals of a specific event. For example, we have a large bank that uses our system to deliver fraud alerts to teller desktops, replacing a legacy system that worked via fax and paper. Incredible efficiency gains and cost savings.
  • Competitive tracking. Or tracking any other external news for that matter – many companies wish to track media mentions and online conversations about themselves, their competitors, or anything else; once they have this information, then, they need to filter and distribute it appropriately. Increasingly, these systems are being used as a replacement for high-end premium content services. We have a large bank who reduced their premium content subscription expenses by over $1M per year by using our solutions – we’re talking about real savings here.
  • Knowledge capture. In large companies, mountains of information are created every day, in many different systems. If there is some way to collect all of this information, and surface it in appropriate places, the content’s value is multiplied.
  • Social networking. More and more, companies are investigating the idea of implementing social networking within the enterprise. The use cases for this are numerous; we have a large PR firm, for example, who uses this to manage their talent globally. A global advertising firm uses this technology to connect not only the people working on a specific account, but also others in the company who might have relevant experience or interests; it’s amazingly effective.
  • Collaboration. This is sort of a catch-all, I suppose…but it’s a popular use for this technology. For example, one of largest banks in the world is using our system to unify groups of people who are on different communication systems (due to acquisitions and such); their analysis indicates rolling this out will be much faster and cheaper than attempting to unify their email systems in 2009, and provide other benefits such as social connections.

For any of these specific problems, RSS forms the underlying plumbing to transfer content around the organization. And if you can track the user interactions with the content, using enterprise RSS infrastructure, very rich data is collected – and the combination of the content plus this interaction data forms the underlying foundation for a social computing solution. One that can even form relationships based on implicit behavior, rather than explicit actions.

An interesting tidbit – of the content recently added to NewsGator Online, 40% of it is content related to social computing (e.g. Twitter updates, Facebook feeds, etc.). In the enterprise, this percentage can be even higher in active social computing environments. But the point is, RSS (and associated enterprise infrastructure) is increasingly used as the basis for social networking and collaboration.

From a business perspective, momentum for all of this is accelerating dramatically. As Brad mentioned, NewsGator added over 30 paying enterprise customers in Q4 2008 – most of whom were large Fortune 2000 companies (there is some breakdown by industry in this press release). These enterprise installations (plus ongoing maintenance) represent about 75% of NewsGator’s revenue, the rest coming from our enterprise SaaS offerings. Enterprise RSS-related infrastructure, including social computing, is growing quickly in demand.

RSS in the enterprise is not about “reading feeds.” It’s more about enterprise integration, search, feedback, etc. It’s about extending the value of existing systems, and leveraging efficient content distribution and tracking into new applications (e.g. social computing) as well as existing applications (e.g. portal enhancement). It’s not about installing FeedDemon on every desktop to enable folks to read content.

And, thus, it’s hard to see from the outside. But it’s there, forever an enabler, making new things possible. It’s enterprise infrastructure. And it’s in more places than you probably think.

Enterprise RSS and security

Peter Verhoeven wrote an article a few days ago talking about security features he feels are missing from Enterprise RSS tools, including NewsGator Enterprise Server. I wanted to address his concerns directly, explaining exactly why NGES handles secure feeds the way it does, and talk about how specifically one could address the issue he brings up.

Public feeds

So first, let’s discuss the general case where no security is involved. If user A and user B both subscribe to publicly accessible feed X, the feed content is the same for both – so there exists only one copy of the feed in the system, and everyone sees that same copy and the contents therein. This is highly efficient for all systems involved.

Secure feeds

Now on to the slightly more complicated case. When NGES encounters a secure feed (more specifically, a feed which requires user credentials to access), it treats the content from the feed as secure. When user A retrieves feed Y using his credentials, the contents will be stored specific to user A. When user B retrieves the same feed Y but using her own credentials, the contents will be stored for user B. So now we essentially have two copies of the feed.

Why, Greg, why would you do this? It’s the same feed, so you should only store it once. Right?

As it turns out, the answer it no. The issue is that NGES has no way of knowing if feed Y will return the same content for every user that has access to it. Just because the feed URL is the same, doesn’t mean user A and user B will see the same thing when they retrieve it using their own credentials. So, the system stores the content separately.

“Shared” secure feeds

Ah, but now we get to the heart of Peter’s issue. He has a case where he _knows_ the content for a given feed will be the same for every user that has access to it. The problem is, Peter knows, but NGES has no way of knowing.

The only part of the system that could possibly know this is the publisher of the feed – because it’s the application generating the feed, and it knows whether it would generate the same feed for everyone that has access. But it’s a valid case – there are indeed situations where the publishing system knows this is the case.

So, in NGES 3.x, it will actually look for a specific header in the feed that basically says “this feed will be the same for everyone.” If it sees that header, it will still individually authorize access for every user trying to access the feed, but it will only retrieve and store the content once. So essentially, authentication and authorization works the same way – we are just making the behavioral assertion that the content in the feed may be shared among the group of people who have access.

[note: I've slightly simplified here, but this is very close to what is actually required.]

This code was initially written to optimize some of the content-retrieval that goes on with our SharePoint-integrated Social Sites Enterprise product…but there are definitely use cases for this feature beyond just SharePoint.

TopStyle acquired

I’m pleased to announce that Stefan van As has acquired TopStyle, and is already planning a major release for Q1 2009.

As many of you know, NewsGator had TopStyle as a result of the acquisition of Bradbury Software a few years back. Since then, its development has been somewhat slow; the company’s focus was on RSS and social software, and TopStyle, while a very successful product, didn’t really fit directly into that mix.

So that’s why I’m so excited that Stefan will be taking this over, and putting some serious effort into TopStyle. To all the customers that stuck with the product, a big thank you! And now, sometime in the next few months, you’ll have a new release to look forward to…and I know that Stefan is eager to hear your feedback and feature requests, so don’t be shy. :-)

Announcement

Q&A with Stefan and more TopStyle information

Nick Bradbury’s take

NewsGator Related Content

As you may have read by now, NewsGator has released the Related Content services as part of the media-targeted offerings. This is available either using a widget, or also available via API for data services customers. The widget (or API) will serve up content that is related to the content the user is reading; that content can be from across the web, or restricted to just your own site (or a set of sites you choose). In the latter cases, it can be used to drive additional traffic from existing visitors.

The system can also automatically create topics pages, whereby the user can see a list of articles matching a specific topic of the currently viewed content.

The system uses our online database of over 3 billion articles, which sees over 8 million new articles per day. At the moment, it matches content based on tags/categories, including categories derived from analyzing the content. You can also specify “fallback” tags to use in the case where you have, say, multiple unrelated articles displayed on one page, and still want to have relevant content.

Two blogs you can look at that are using the widget are the NewsGator Widget Blog, and the NewsGator Daily blog.

Give it a try, and let us know what you think!

Update: here is a tutorial on creating a related content widget!

NetNewsWire for iPhone usage

Josh Larson on NewsGator’s Media and Consumer team has a blog post up with some usage stats for NetNewsWire for iPhone in its first 9 weeks or so since release. Go take a look…over 200,000 users, and over 115,000 new-to-NewsGator users have signed up in the last 30 days.

We have some other data as well that’s more difficult to explain, so we didn’t publish it…but generally, what we’re finding is users of the iPhone app are sticking with it and using it regularly. Said another way, the abandonment rate of iPhone users is quite a bit lower than our average rate (which is already fairly low, but the difference is significant). It seems the combination of NetNewsWire and the iPhone is a winner!

NewsGator’s sync platform details

For the vast majority of NewsGator users (including folks using NetNewsWire, FeedDemon, or any of our other applications), NewsGator’s sync system works totally transparently. But there are some nuances of our implementation that are sometimes visible to users. So, in the hopes of giving people a definitive place to go to understand this, I offer the following in-depth explanation of NewsGator’s sync platform.

The Mechanics

All content is stored on NewsGator’s servers. When an application like NetNewsWire does a sync, it sends up some bookkeeping information up to the NewsGator system (a “sync token”), and the system returns a list of feeds that NNW needs to update. NNW then requests updates for each of those feeds (which is generally a subset of the list of subscribed feeds), again using a sync token, and the system returns the new (not yet seen) or updated articles for that particular feed.

This system is extremely efficient. For feeds that haven’t updated, NNW will not even have to request them. For feeds that only have a single new item, that’s the only data that will be returned to the client. For scenarios like mobile applications (like NewsGator Go! or NetNewsWire for iPhone), this is pretty close to the minimum theoretical bandwidth required to deliver the content.

It’s actually possible to reduce the number of calls even further, but at the cost of a potentially large (and expensive to process) response. Our APIs are instead optimized around the case I describe above.

[note: this is somewhat simplified; for example, metadata also travels both ways during a sync, but I'm leaving discussion of that out for purposes of this article.]

Details

NewsGator’s online platform processes about 3.5 million feeds, and stores about 9 million new articles per day, as of this writing. There are a total of about 3 billion articles in the system.

Suppose you subscribe to a feed from CNN.com, and further assume that that feed publishes 100 new articles per day (I have no idea how many it actually publishes – just go with me here). Now imagine you go on vacation for a month, and you come back, fire up FeedDemon, and sync. There are now 3000 articles you haven’t seen. Should we deliver them all to you?

Probably not.

Do we?

No, for several reasons. First and foremost, the user experience therein would totally suck; no one wants to wade through 3000 articles from a single feed. And second, it’s pretty inefficient to retrieve all of this content from the API – we could deliver it, but it’s going to take a lot of bandwidth to retrieve it, and a lot of work to process it. Using a mobile phone? This might well lock it up.

And the other reason is, our system allows you to mark individual articles as read, and that data is synchronized throughout the system and all of the clients you use. But we don’t store your read states for all time – we only store it for fairly recent data. Do you really care if you marked an article read 2 years ago? Probably not.

So what do we do? We have a limit of how many articles will have their metadata state synced through the system. Here’s the rule we currently use:

The number of articles in the current feed,

OR

14 days, up to 200 articles.

Whichever of the two conditions above yields more articles is applied.

Here are three examples of applying this rule:

Imagine a blog that publishes 5 times per month, and its feed has the most recent 10 items in it. This feed would sync 10 articles.

Now imagine the hypothetical CNN feed above, which had 100 items per day, and imagine the feed held the 20 most recent items. In this case, we would sync 200 items.

And finally, if a feed published 10 times per day, and held the most recent 20 items in the feed, we would sync 140 items.

If you really want to go back and browse through all 3000 articles you missed, you still can – they’re all stored in NewsGator Online, and you can view them from the web site. In fact, you can go back all the way to when we first discovered the feed – over 4 years ago, in many cases.

The Gotcha

For most feeds, the algorithm described above makes things completely transparent, and articles and unread counts across all NewsGator-integrated applications will match up perfectly.

The gotcha is with feeds that have a lot of articles. For example, I have a smart feed for the term “NewsGator”, and I see probably 400 new articles there per day, 200 at a time. So in this case, only 200 articles have state synchronized.

What can happen is the following:

1. NetNewsWire downloads the feed, and shows 200 items, all unread.

2. 3 hours later, you sync from FeedDemon, and you see 200 items, all unread. You read them all, and mark them read.

3. An hour after that, you sync again from NetNewsWire…it syncs state from the old articles, and downloads say 25 new ones. You see 47 unread articles. You immediately sync again with FeedDemon, and it shows 25 unread. What gives?

The problem is the 200-article limit, and the fact that some of the articles fell off that ledge while still showing as unread in one application…and thus don’t have all of their state synchronized.

This problem used to be much more acute – it’s more rare now that we’ve raised the article-sync limit up to 200. But it’s still possible to run into this, specifically with very prolific feeds.

We’ve experimented with various different limits – the current 200 articles seems to be a good compromise, perfectly syncing the vast majority of feeds while maintaining the efficiency that our client applications demand. We’re also working on some things that will make this invisible to API clients, while still working within the constraints.

NGES in 60 seconds – group clippings

Last time, we looked at how clippings work in NewsGator Enterprise Server.  At the end of the video, I mentioned that there are other ways to share clippings with other folks…I was referring to a feature called group clippings.

Group clippings are basically clippings that are shared among all the members of a group.  That group might be an Active Directory (or LDAP) group, or it might be an ad-hoc group defined in NGES.  The clippings for that group are, by default, private to that group – so you can have a confidential conversation among the members.

View QuickTime video

YouTube version:

And as always, you can try all this out for yourself by downloading the free 20-user version of NewsGator Enterprise Server.