A combination of events got me starting the New Year reflecting on attention and personal creativity.
The first event occurred while listening to an episode of the Brainy Gamer Podcast. At some point during the podcast, every journalist interviewed mentioned being on Twitter. Having found their viewpoints interesting, my first instinct was “Hey! I should go follow all these folks on Twitter”. But then the thought of adding another dozen high-volume tweeters to my already over-busy timeline just made me quail.
Even as I pondered what was my S/N cut-off for Twitter, I found myself in a fairly unique situation – without any podcasts to listen to or books to read on my train ride to and from work for almost 2 weeks. At first, I fidgeted and checked Twitter obsessively on my phone. But then a fairly remarkable thing happened – a number of ideas and observations started going off in my head. It seems like my brain has been itching to think on its own for months and I’ve been drowning it in too much information instead 
Apparently, I’ve been deluding myself into thinking I could handle all the information flowing my way with no problems. Now, this is a pretty common complaint – “There’s too much to read!”. The stock answer is pretty easy as well – “That’s because you haven’t figured out how to filter the really important stuff yet”. AKA Filter Failure.
The lack of tools to help me build the right filters is something I’ll come back to later. But what I really noticed from this information overload was how it was affecting my own creativity. A constant minute-by-minute decision of “Read? Don’t Read?” that left me curiously unsettled after an hour or so of going through my feeds. As Laura Roeder put it:
After spending an hour-and-a-half going through my feeds, simultaneously twittering links and posting articles to my link blog, I’d find myself distracted and twitchy. To the extent that I would not be able to bring myself to start writing and dawdle at some website or the other until it was time to shut down the computer and go home.
It seems like I have reached an inflection point – I could continue as I had for the last year or so, adding to my reading list and twitter all the time ,while spending less and less time actually acting on all that knowledge. Or I could take a good hard look at what I was really interested in and focus on that instead.
What am I really interested in? That’s a hard, hard question – ask yourself this “What’s something I really, really like; have fun still doing; but don’t have the time anymore for?”. For me, a couple of things became obvious – one was Gaming and the other was Technology.
Gaming was my first big Geek love – I fondly remember those endless hours spent playing F-117, AoE, AoK, Caesar III… At the same time, I realized I haven’t played a game even half-way to completion in years. Gaming news tends to be a real fire hose as well- even if I limited myself to PC gaming, the updates would come thick & fast and often I’d barely glance at the headlines. So this was the first category to get cut – something made easier by the fact that I still listen to a couple of gaming podcasts.
So 13 feeds down, 289 feeds to go.
Making sense of my passion for technology was much harder. I had tried to compensate for feeds with endless updates by switching them to headlines-only mode. The thing is – once I switched to headlines mode, I’d find myself only reading articles that have “interesting” headlines. It felt like I was becoming that mouth-breathing, linkbait-sniffing Digg fanboi that I so detest. Or it could be I had just validated Sturgeon’s law – “90% of everything is crap”.
In the absence of a more scientific approach – I went with my gut instinct. I moved a few feeds into the doghouse (literally):
Once I’d done that – I forgot about them. Didn’t check that folder for days – kept hitting the “mark all as read” button. After a week or so, I realized that at no point did I feel like I was missing something by not reading these feeds. With that realization Lifehacker, NY Bits and a few others were consigned to the trash bin. A minor victory over information anxiety!
Earlier in this post, I complained about the lack of a scientific way of choosing which feeds I don’t want – and that wasn’t a joke. Buried within the FeedDemon UI is a way to check my “attention” score for every feed I subscribe to. When I did check this for my Tech feeds, I was extremely surprised – BoingBoing was first by a wide margin, with kottke and Daring Fireball in distant 2nd place. That pretty much told me the attention scores were all wrong, because here’s my “attention” gut-check:
I think what is missing is a more “semantic” attention score. What are the topics I read the most? What keywords are common across the article I’ve added to my shared folders? I think that I love reading about operating systems, design and odd facts, but do my reading habits support that belief? I feel like if I had this sort of information, I could simplify my reading list some more.
How long will this new approach last? Too early to tell yet – but certainly, being able to go through my entire feed-list in less than 1 hour last Sunday felt great. So does the Rescuetime graph that tells me I’ve spent just 30 minutes on average in my Feed reader during January. But if my notebook is any indication, I think my own brain is thanking me the most
We’ve all been there – something vitally important (!1!!) has gone belly-up and we need answers – fast. It just so happens there’s a discussion forum where we can ask for help. You’ve registered, made a gigantic post detailing every single aspect of the problem that you are aware of and now you are waiting for those replies to start rolling in.
Except well, you have no way of knowing the minute a reply actually comes. Sure, you could activate the option to have an email sent to you whenever someone posts a reply, the problem is – you used your “spam” account (the one you never bother to check remember?) or worse, you used a throwaway email account (10 minute mail, guerilla email to name just two).
If only there was a way where you could get information pushed to your desktop whenever a page was updated… oh yeah, how about RSS? Slick new Web2.0 style forums such as Get Satisfaction do offer RSS feeds for each thread and several plugins are available for phpBB (the grand-daddy of forum software) that add RSS support, but it isn’t a given.
Enter page2rss.com. page2rss is a simple idea – scrape a given URL and any time the page changes, update an RSS feed with an excerpt of that changes. So, you take the URL of your forum thread, plug it into page2rss.com and voila! instant RSS feed.
Obviously this isn’t perfect, but we’ll take about caveats later.
So, we’ve solved the big problem – we now have an RSS feed that will instantly get updated whenever our vitally important forum thread changes.
Except, well 2 things:
1. You might not leave your feed-reader running all day, so how do you find out when that RSS feed updates?
Yes I know you might be of those lucky folks who can live in their browser all day and refresh their feed reader every 30 seconds. The above applies to not so lucky folks (like me)
2. Cluttering up your feed list with one time feeds that you have to remember to unsubscribe from later can get pretty annoying.
So now what we need is a service that can read an RSS feed and alert us whenever the feed updates. Also, we should be able to quickly unsubscribe from that feed once it has served its purpose. Ladies and gentlemen, I present to you – FeedCrier.
Feedcrier provides an IM based RSS alert service. You add Feedcrier to your contacts list, give it a list of feeds to monitor and whenever a feed updates, you get an alert.
If you use the FeedCrier bot on Google Talk and you log IM conversations in your Gmail account, there’s another benefit – you could log off Google Talk and if FeedCrier updates when you are offline, it appears in your Gmail Inbox as a saved conversation – awesome! 
So there you have it – between Page2RSS and FeedCrier, you can set up a system that alerts you to changes on any page you are interested in and know whenever it happens in your email or in your IM client – what’s not to love about this?
A couple of things:
1. Page2RSS is after all a site scraper. This means it doesn’t distinguish between the main content of the page changing and a change in the footer text. A typical example – most forum threads list the number of registered users on the forum. Every time this number changes, Page2RSS will update the RSS feed for the page. On a popular forum, be prepared for lots of spurious updates.
2. FeedCrier isn’t a 100% stable – I’ve had it disappear from my IM client for a couple of days at a time and at other times, get incredibly slow at responding to any commands I type in. This could be because I use a non-standard Google Talk client but YMMV.
Talk about shooting your mouth off 
A while back I had made a long-ish post about the Recent comments feed on Flickr and how the lack of authentication could allow people to see your non-public photos.
Well I recently had posted some photos to my Flickr photostream, that were in fact marked as private (for friends & family only) and one of them recently attracted a comment.
Guess what – they never showed up in my RSS reader. That’s when I took a closer look at the description for the recent comments feed, and I realized it says “All comments posted to your public photos and/or sets.”
D-oh! So that’s how Flickr is overcoming the lack of authentication issue. Private photos and messages never appear in RSS.
I’m partly relieved and partly annoyed. Relieved – since that means your private photos are still private. Annoyed – because this means there is a chance that someone might miss these comments. Since you have to log into Flickr to see these comments, if you happen to be one of those Flickr “superstars” who attract lots of comments right off the bat, there is a good chance comments on private photos will get lost.
Anyhoo, just wanted to say “my bad, my bad!”
On a bit of a RSS trip here aren’t we?
Anyway, this is something I came across a few weeks back and it’s been bugging me ever since.
Since everyone likes a stroke (and I’m no exception) – I have subscribed to the feed for comments on my photos at Flickr. Now, the RSS feed for the recent comments takes the following format:
http://api.flickr.com/services/feeds/activity.gne?user_id=<Your Flickr ID>&format=rss_200
The Flickr ID in the above link is a 12-character alpha-numeric string that every Flickr User gets when they register for the first time.
Unfortunately, there is no authentication on this particular RSS feed – which means that if you know a user’s Flickr ID, you can easily see all the comments that any one makes on that individual’s photos.
The Flickr ID is incredibly easy to find as well – if you haven’t picked a “easy name” for your Flickr photostream – it directly appears on your photostream. Even otherwise, the Flickr API provides this information readily and there are Greasemonkey scripts out there that make this a single right-click.
So what’s the problem you ask? Anyone can see those comments anyway – sheesh! Well yes, anyone can see the comments on your public photos, but by reading your recent comments feed they can see comments on any private photos as well, i.e., those photos which you have marked as family/friends only.
Further more, the recent comments feed provides the URL for the medium size of your private photos. Downloading the original resolution images is then trivial – merely requiring replacement of a “_m” in the URL with a “_o”.
The creep factor just increased substantially didn’t it?
Now by default, using the RSS format means that no authentication mechanims are supported directly. Basic authentication can be used (which would take the format http://username : password@api.flickr.com/services/feeds/activity.gne?user_id=<Your Flickr ID>&format=rss_200), but this isn’t directly supported by the protocol.
On the other hand, the ATOM format does support authentication (see the RFC) and Flickr infact uses ATOM for all photostream RSS feeds. So why not switch the private feeds to ATOM as well?
For the oldest reason in software development: security is just not sexy – so something like this is always going to fall to the bottom of the pile.
Right upto the moment when some-one abuses this loop-hole that is. And the Flickr forums go up in flames (as they have tended to do fairly regularly these days).
Here’s hoping this gets patched soon – oh and could I get a pony with that?
Update: I spoke to soon. Flickr does in fact protect your private photos. See this follow-up post.
While I like to think I’m ahead of the curve on Internet trends and memes, in terms of actual adoption – I can be quite the luddite.
And so it is that despite the fact that I read a number of blogs and tech news sites that publish tons of RSS feeds, I generally visited these sites the old fashioned way – bookmarks in a browser.
But that all changed a while back and I’m reading most of my news via RSS now. What changed?
Social pressure is the answer – In other words, I did not care much about knowing everything that happened in the world of news, but I certainly was interested in knowing whenever my friends updated thier flickr photostreams.
As my contacts list in Flickr grew, the “Photos from my contacts” page in Flickr grew more and more unwieldy – people who posted 20+ photos a day would swamp those who posted maybe 1 photo a day.. and I wanted to keep track of both.
I’ve been on an RSS-heavy diet for about 3 weeks now and I can appreciate why it has become popular, but as always there are some gotchas:
There are quite literally “n” number of feed-readers out there and the short answer for that question typically is “Google Reader”. But what I really wanted to avoid was leaving a browser running to catch up on feeds, because that way lies the hell of “zero-productivity”. At the same time, I also appreciated having the ability not just to take my feeds with me wherever I went, but more importantly, knowing where I had left off.
Feeddemon fits the bill very nicely – an acclaimed feed reader with tight online integration (via NewsGator), but I wasn’t ready to put down some cash for a feed-reader (yet).
Some intensive googling lead me to find GreatNews – a free offline news reader. The reason I chose this is because the product road-map included integration with Google Reader. It hasn’t happened (yet) but the steady pace at which the author is pushing out updates for the reader gives me hope.
- It’s okay to close your feed-reader
Initially, I would start up GreatNews and then leave it running in my system-tray. Almost immediately, I ran into the usual distraction problem – just when I was getting into a nice groove, I would see a popup that there were New!! (exclamation points mine) news items and inevitably I would give into temptation and open up GreatNews. Instant productivity breaker.
Since then I’ve learnt to fire up GreatNews only 2-3 times a day, which helps to keep the next problem in check..
- Don’t look at that unread counter!
When I was visiting sites via the bookmarks route, I would usually hit tech sites in the afternoon as my work-load slowed. With RSS, the minute I open up GreatNews in the morning to catch up with my friends’ photostreams, I would be faced with 310 unread items in my Tech news category. At first, I would experience information anxiety if I didn’t click and read all those news items right then.. and then of course, I would have spent an hour on reading feeds which would have been better used completing my work for the day.
- Some sites are better left off the RSS list
When visiting sites via a bookmark, you might never know just how many articles are posted on that site. You just hit the first page, scan the headlines and then hit “close tab” if nothing catches your eye. When it’s RSS however, the equation changes – you know exactly how many times a post appears on a site and if your feed reader is open and running all day, that can translate into a steady stream up of “new articles” pop-ups. After a few days, I deleted those feeds and recategorized some of my bookmarks into RSS and non RSS categories so that I could catch up these sites the old-fashioned way. I have atleast a couple of other feeds that might need to be pruned this way, but for now I’m happy with the update frequency I have achieved.
On balance (and admittedly, this is a short time-frame on which I’m basing my judgement) switching to RSS has helped ease my information anxiety overall. Sometimes, technology can actually be helpful.. who knew?
I currently use Feedburner to handle my main site feed, in part because it takes care of the hassle of ensuring feed validity, providing Atom or RSS feeds and also the feed stats.
One feature on Feedburner is identifying “Uncommon Uses” or sites where the feed content appears where it usually should not.
I’ve noticed recently that some wierd porn sites keep popping up in my uncommon uses list. I have no intention of visting such NSFW sites to figure out where my site content appears on the site.
I also would like to contact the webmaster and tell them to stop doing this, but am terrified of giving away my “real” mail ID – I get enough spam as it is! Any suggestions on how to solve this problem?
