Comments on: Aggregators that automatically download web pages

By: Stephan Hodges

Stephan Hodges — Tue, 16 Dec 2003 12:34:44 +0000

Some of the problems currently: 1) many sites stop in mid sentence (Slashdot comes to mind), 2) titles only, which is almost useless, 3) only text, whereas the blog has pictures, etc.

Newsgator should (at minimum), put an automatic link to be able to jump to the blog page. I'd appreciate a second option that would let me get all offline content, for when I only have a few minutes to collect, before going offline. That's one of the points of an aggregator.

Since you mentioned that several others do this, if Newsgator isn't ever going to provide the feature, please list these. You've provided very good support to me, but I'll use other products if I need the features.

By: David Buchan

David Buchan — Mon, 15 Dec 2003 22:17:37 +0000

Part of the problem is that people are writing blogs without knowing what an RSS feed is, let alone what they are putting into it. I and others have been making requests for this but perhaps the blog providers have to do something to assist. A check box or something.

By: Stuart Dambrot

Stuart Dambrot — Mon, 15 Dec 2003 22:16:15 +0000

Lars + Automatically = solution.

It could also work by embedding a microbrowser into the aggregator that displays the target page. An example of this is the Newsroom (read: RSS Feed) panel of Idea2’s free why-wait-for-Longhorn add-on, Desktop Sidebar (http://sidebar.tech-critic.com).

Enable “Show article text in the details window” in Newsroom’s panel properties and the target page shows up in a small (unfortunately not resizable) pop-up window simply by mousing over the article title in the sidebar.

By: RSS Usability Issue

RSS Usability Issue — Mon, 15 Dec 2003 01:04:52 +0000

Greg Reinacker discusses the issue of how RSS should be used to read posted content. (Greg is the author of the NewsGator RSS aggregator.) There are two issues: The first is should RSS aggregators also grab the referenced content. The second is what should be put into the RSS entry.

...[more]

By: Dwight Shih

Dwight Shih — Sat, 13 Dec 2003 22:49:48 +0000

I posted some thoughts on this a few months back. I think that the basic problem is that current tools are designed for always on. But I think that the reality for many of us is that we're constantly drifting between online and offline. If you think that a blend of off and on is a long term phenomena, then you also think that tools need to evolve to support it.

BTW, I believe in caching not scraping. If the author is driving you back to the site for branding purposes, then you should respect that.

By: Jorge Curioso

Jorge Curioso — Sat, 13 Dec 2003 20:56:19 +0000

Agree with most here: don’t do it. It *is* true that those sites that don’t put full post content in the RSS tend to get removed from my OPML, or at least read much less.

By: Julien CHEYSSIAL

Julien CHEYSSIAL — Sat, 13 Dec 2003 15:06:08 +0000

Well.. IMHO, there are way too many links and pages to download. What will be the percentage of downloaded pages which will be really visited ? Like 1% to 5% or 10% ? Way not enough ! RSS lets you gather a bunch of informations, maybe too much informations. If the author wanted people to really read the content he’s linking, he would have copied it into the item description.

By: Andrew Barrow

Andrew Barrow — Sat, 13 Dec 2003 01:32:56 +0000

Dont scrape – but on spittoon i only post full content but on the 18 or so feeds I subscribe to it annoys me when the posting finishes mid sentence.

By: Automatically Downloading Content from RSS

Automatically Downloading Content from RSS — Sat, 13 Dec 2003 00:13:32 +0000

Aggregators that automatically download web pages: Some people have made the request that NewsGator download the "target" page of a blog posting so disconnected users can have the content too. Greg has a good discussion of the implications. So we're...[more]

By: Bruce

Bruce — Sat, 13 Dec 2003 00:03:25 +0000

I like it the way it is, thanks! I admit, though, that I rarely read off-line, so I can always follow the link to the original source if I want to see it.