Relatively Simple RSS Aggregation
I recently posted about my need for a simpler RSS aggregator.
I worked through the massive list of existing RSS aggregators at wikipedia and couldn't reallly find something that did exactly what I wanted and worked.
I needed something that was simple, cached hourly, displayed various encodings well and worked with RSS and Atom formats. Most importantly, I wanted something I could install on my own server, and it needed to be community oriented (not designed for a single reader).
Most of the existing solutions on the web are hosted by a third party, and are limited in the number of feeds that they can aggregate.
The best hosted solution I found was Gregarius, which has a lovely community. I think that gregarious will take the cake sometime this year as aggregating catches on. For now, however, the plugins are pretty limited — you are locked into a personal reader with "Read/Unread" tagging. (Which makes Gregarious a great replacement for a newsreader like Bloglines, but it's not good for a community aggregator where lots of people visit.)
If I were a better person I'd just write a Gregarius plugin, but instead I've just reworked some of Feed2JS's code (which is in turn dependent on the Magpie RSS parser.)
You can download it here.
Otherwise edit the php file with the rss or atom feeds you want to include. It will probably need a little CSS to spice things up. You can send comments or questions about the script to chris(at)nonprofitdesign.org. I'll release another version sometime soon with better documentation and a prettier front end.
An example of the styled script in action is here (at the Piedmont Biofuels website).
UPDATE: That page now uses simplepie, which kicks ass.
Last night I was trying to do something that I thought would be pretty simple: display a bunch of recent weblog posts on one page.
There is a great online community of folks in the biofuels blogosphere, and this page would give a quick summary of their myriad, nerdy, wonderful events and research.
I'd had experience with building this type of page last year, when I just wanted to have an "aggregator" page of all my most loved online reading. I ended up just slapping things around with Magpie RSS (an excellent open source PHP class), and it worked fine. Not slick, but fine.
So forget all that, because this isn't just a wonky personal project — it will hopefully end up being part of the excellent Piedmont Biofuels website — so it needs to be quick and hosted on the server.
So last night I opened up the latest installation of MagpieRSS and installed it on my server, created all of the necessary php for each of the blogs, and I ended up with a decent document. The major problems with this first version (using just the Magpie class) is the inconsistent treatment of the posts — some appear and some don't — and the improper encoding of the blogs. (I went 'round and 'round with the encoding. It's a common problem, but I couldn't get those damn posts clean.) Probably a few days in the Magpie listserv archives at Sourceforge would clear all of this up ... but the archives are exceptionally annoying, the Magpie blog is down, and the first version was still surprisingly slow anyway, even with the cache working.
(Actually, while I'm writing all this out, I should bother to mention that there are a hell of a lot of 3rd-party RSS splicers/combiners ... but, again, they're all third party, and they seem to go extinct quickly: e.g. the defunct rollup.org. Most of these also have ads, are not free, or have limitations on the number of feeds, like feed digest. I was surprised and disappointed that I couldn't find something to install on the server that would take care of this — somebody please let me know if there's something reliable out there. This would allow me to combine all the posts and just run the JS once.)
So, from a programmer's view, this is a little inelegant, but the result is really consistent, and it still comes in at 8.5 seconds on 56K. The (minimally styled) latest version is here.
What The Hell Is RSS?
There's a good discussion going on now at TechSoup about how you can use technology to increase capacity of your nonprofit.
Their latest focus is on RSS, which is one of the more amazing technical innovations of modern communications. If you don't know what RSS is, you're not in the loop.
Most notably, you are missing two things. 1.) The ability to prrocess inhuman amounts of information on a regular basis. RSS is used by many sites to create "digests" of (a.k.a. "syndicate") their content. News sites, for example, offer these RSS "feeds" to sections of their newspaper.
It's a lot like having a TiVO for your favorite sites.
Once you subscribe to one, it appears in your RSS reader of choice. In the end, it's a lot like having a TiVO for your favorite sites — every time you sit down and you want to read, you have a list of updated articles in front of you. No wasting your life poking around the less glamorous parts of a newspaper page.
2.) If you have a website that is updated regularly, you can create a RSS feed so that others can subscribe to your page. This may seem a little superfluous if you don't even know what a RSS feed is. "Who would use it?" Plenty of folks will, especially over the next year or two. So start writing some engaging articles for your site.
Here's a bit from the Tech Soup conversation:
"If you're like most people who use the Internet, chances are you often come across new and interesting sites, but then completely forget to visit them again. Or likely you spend too much time visiting the same sites looking for new information, only to be disappointed. In addition, your e-mail inbox is probably flooded with messages you barely have time to read, including subscriptions to newsletters that tell you about new content available on still more Web sites. ... Wouldn't it be nice if there were an easy way to have all this information come to you and go to them in a way that was easy to manage, timely, and put the reader in control? "
Read the article (and check out TechSoup forums, which are among the best of their kind on the web.) TechSoup - Articles: Using the Internet - RSS for Nonprofits: