The Universal Filing System?

It struck me that the amount of content that we publish about our lives today on the web is simply staggering.

We write down our bigger ideas and thoughts in blogs, twitter the random everyday things, link to the things we like, take photos of places and things we see, converse with friends on social networks and record all the conversations that we have about these in Instant Messenger logs.

It seems to me in our own unplanned, chaotic and sometimes unreliable fashion, we have brought to life the Akashic records of Hindu philosophy – a record of all human knowledge and experiences.

Low impact Link-blogging: Design Reboot (Part 2)

I have always had mixed feelings about linkblogging – on one hand, I read a lot and it seems natural that I would come across interesting articles that other folks might enjoy. In the back of my mind however, I think of linkblogging as slagging off from the more "real" task of writing something (semi-)coherent.

A reflection of that conflict has been how my own style of linkblogging has gone through a few iterations.

One of the very first questions I was asked shortly after I twittered about the new linkblog was simply "Why not use del.icio.us?". The answer dear reader, is that I actually was using del.icio.us for linkblogging.

It’s well hidden now, with no obvious way of getting at it, but there are over 50 posts on this blog that are just links with a little commentary – here’s the first one, and this one’s the last.

del.icio.us did seem like the answer at first – as I was browsing, I hit a button whenever I came across a link; filled in some text and by some magic it appeared on the blog.

The infatuation disappeared pretty quickly though – at first, it was just the frustration of the 255 character limit for descriptions. But on reflection, it was something deeper than that – I resented the interruption that posting to del.icio.us represented.

Skimming through the never-ending stream of info that the Web can throw at you and picking out something interesting is an art form – when you are reading north of 200 feeds as I am today, it’s pretty much a survival skill. Yet every time I found something useful, I would have to drop everything and focus on writing something meaningful about why I found this link useful – after all what use is a link without context? Do this context switch 20+ times a day and you start to feel a little ragged.

Foolhardily, I decided the answer was to hand-code a daily link dump. I would open links in individual tabs as I came across them, cull them down to a few interesting links at the end of the day and then using a combination of two firefox extensions and some Autohotkey magic, kludge together the HTML required to format the post appropriately. Not unexpectedly, this experiment didn’t last very long – I made exactly 6 linkblog-style posts using this technique before I gave up all-together.

It was only when I made the switch to RSS that I began to consider how best to start linkblogging again. My requirement was simple – I wanted a way to hit a button and share a link; no comments, no tagging, nothing. I knew right away that FeedDemon supported this through "shared clippings" but I balked at paying for a feed reader and tried to postpone the inevitable by using GreatNews.

In the end however, I switched to FeedDemon and that was when the real power of using RSS feeds for linkblogging hit me – I no longer had to provide a context. The author of each individual article had taken the time to craft an introductory paragraph that explained the article better than I ever could. If you wanted to read further, you kept scrolling or you just switched to the next article – a homage to the low-impact way in which the link itself was blogged.

Is this method without flaws? From an attention perspective – I think no, there aren’t any. I can share links with minimum effort, folks who subscribe to the linkblog feed can skim through the links easily as well.

The real problems lie in usability – HTML is being converted into RSS, back into HTML (on the blog) and finally into RSS (in the blog feed). Validation is a pipe-dream, the visual layout of the blog is often broken and some functionality simply does not work.

The other problem is the possibility of ads slipping into the blog – I don’t have any ads running and don’t intend to either. I worry that someday, I might share an article that comes with advertising attached and make a mockery of that claim.

Is that risk worth the reward of sharing interesting ideas quickly and without friction? My answer is a cautious yes.

Back to the Basics – Design Reboot (part 1)

I think I’ve worked out most of the kinks in moving the blog to it’s new home, so I thought I would spend a little time talking about some of the challenges and ideas behind this shift.

But first a little bit of history – I originally started blogging on a free service provided a site called myblogsite.com. Since this was a hosted service, there was tremendous restrictions placed on what you could and could not do in terms of look and feel. A few months in, the site abruptly shut down and I lost my backup of the posts

Smarting from that loss, I did a little bit of searching and came across Weblogs. It almost was too good to be true – 2 GB storage, full wordpress hosting, FTP access; the works – and all this for free! I shamelessly begged JD for an account and he was kind enough to let me in. Obviously moving from an completely locked down service to Weblogs was like letting a kid loose in a candy store.

I wound up choosing a 3 column theme – strike 1 for visual overload. Next I installed 30-something plugins in my WordPress install – visual overload, strike 2. I then started adding all sorts of “flair” – Flickr photos, feed subscribe widgets, page ranking widgets on and on and on – visual overload, strike 3.

Now in my defence, I should say that alteast initially there was a method to this madness. I saw the blog as a “portal” to my life – a clearing-house if you will, about my online life. You could come here and find out what I was doing (hence a status page). You could look at interesting (ahem) things I had seen. Why you could even find out what the weather outside my window was!

But over time that message got a little muddied – I didn’t update the status page often enough. The blog itself wasn’t updated frequently. And my online “life” was spread far too wide to integrate in any reasonable way into this one page.

It was around this time that two things happened:

1. I actually went and bought this domain name, figuring a guy with some income and a reasonable amount of computer-savvy should really have his own website.

2. I read an article on the Coding Horror on the worst blog cliches – and realized I was guilty of more than half!

It was time to make a clean break with the past. What could I do? I knew I wouldn’t be moving away from WordPress as a platform – it was the only platform I really understood in any way. So that left the look, i.e. the theme and the feel, i.e what content was displayed on the blog.

For the look, I realized the 3 column theme simply did not make sense given the blog is not part of a network of blogs where the 3rd column might display other content. Further, 3 columns just didn’t work well with the way most people read, so it was time to switch to something simpler. A little bit of searching on WordPress Extend yielded the Web2.0 theme that I’m using now.

Reworking how content appears on the blog was two-fold: One, getting rid of the clunky HTML I had wedged into the blog as part of the “flair”. Two, KISS – no extraneous cute stuff like widgets or geo-tagging. My plugin list would be more focused on making the back-end more useful. A plugin would either help people find content in the blog (hence, Simple Tags and Popularity Contest) or help with the flow of the post (hence WP Footnotes) or provide missing functionality (Contact Forms etc.)

The final piece of this puzzle was making sure I actually built up the blog the right way instead of constantly editing and adding on a live install. This way, if I screwed up (and there were quite a few restarts) or changed my mind, a do-over was simply a question of restoring a backup. The answer to this came in the form of WAMP Server – which made the otherwise nightmarish process of getting Apache-MySQL-PHP running on Windows a simple, push-the-button affair.

There were some difficult choices along the way – for example, do I put in the Flickr photos widget or not? My stats tracking on the old blog revealed there was a decent amount of click-through from the blog to some of the photos. Why lose that traffic? The answer to that goes back to the original idea – can my blog be a portal? My own experience says no – our online lives are now too widespread, too entrenched in each of the websites we frequent to easily co-exist. It could be made to work – if you were willing to live with a page that a lot of competing visual elements. But few things are more attractive than a website that does one thing – and does it well :-) .

Don’t touch that button!

This post is aimed at the folks who subscribe to my blog via RSS (If you don’t, why not start now? :) )

If you are seeing this, then it means I have managed to flip the switches in the right order and big changes have happened!

Come take a look at the all-new, all-singing & all-dancing home of the blog.

Crunching the numbers – a look at my TwitterStats

I had seen tweetstats a while back, but Biz Stone recently tweeted about it again so I gave it a shot again and it is just as cool as I remember it :-) . So here’s my tweetstats graph and some analysis

tweetstats-20080210

1. My tweet timeline is just about a year old now (my how time flies!) and a fairly clear pattern emerges – I manage to tweet a lot during the early part of the year (93 tweets in Feb ‘07, 131 in Mar ‘07, 131 again in Jan ‘08) and then as the work crunch sets in, the pace drops off pretty rapidly. Two big slumps occur over May-Jun ‘07 and Aug-Sep ‘07. The first slump was during the time I was looking for a new apartment to move into and between that and work, twittering took a back seat. Aug-Sep ‘07 was when I was on vacation – so between clearing the decks for the holiday, being offline while on vacation and again catching up on work afterwards, twittering was pretty much forgotten about.

2. On a daily basis, I do most of tweeting on Fridays (no big surprise there) but a close 2nd place goes to Wednesdays. No easy explanation for this except for that I probably remember that I haven’t tweeted in a couple of days and try to make up. The radical slump over the weekends is because I tend to get online for only a couple of hours on weekends and that’s mostly to catch up on news and post photos on Flickr. Recently I’ve started to read tweets by others on the site over weekends and long CNY weekend has seen me twittering quite a bit actually.

3. Time-wise I have two bursts – one early in morning (around 10-11 AM) and again after lunch (2-3 PM). Both of those are because I tend to hit my feed-reader then and so the tweets naturally follow. The rest of the time is mostly D messages and @ replies. Which neatly brings me to…

4. The replies graph. The first thing to note is that I tend to do a lot more D messaging than @ replies. That’s because I got told off pretty early on by some folks who follow me, when I started using @ messages like IM :-) . But that graph is a fairly accurate representation of how my D message traffic goes.

5. Since I started off by using the site to post tweets, it still is the leader but Snitter is catching up fast and should take the top slot pretty soon. Plug for Snitter here – it is really a much better way of using Twitter and makes the site seem ridiculously clunky in comparison. Plus, Jonathan (who develops Snitter) responds to any messages you send him pretty quickly.

What we owe to the A-Bomb

As to where I first came across a recommendation for Richard Rhodes’ books on Nuclear bombs, my memory fails me. But few books I have read stretched me like “The Making of the Atomic Bomb” and “Dark Sun” have. For someone who prides himself on being able to read a 500-page novel in less than a day, 6 weeks to finish a 600 page book was a humbling experience.

I just finished Dark Sun yesterday, thanks to the long CNY weekend and am looking forward to reading the final book in this trilogy – Arsenals of Folly. A few ideas have been fermenting ever since I read the first book so here goes:

1. My mental history of the atomic bomb was largely shaped by newspaper accounts and famously Einstein’s E=MC2 equation. I had always believed that nuclear energy had been developed for peaceful purposes by the scientists and somehow the military establishment had twisted it to create weapons. It was a tremendous shock to me to discover that it was in fact quite the opposite – the scientists involved in this field early on (such as Leo Szilard) actually approached the military, fairly aggressively to convince them of the military potential of this new discovery. Civilian uses – atleast on a large scale never entered the picture, until much later.

This has to be understood in the context of when all of this research was going on – at the height of World War II, with Nazi Germany triumphant across all of Europe and Russia on the verge of collapse. Equally interesting in contrast, is the tremendous ethical conflict that the scientists heading up the atomic energy program in the US wrestled with when it came to developing H-bombs in the years following the World War.

2. The amount of everyday technology that arose from the atomic bomb and H-bomb programmes is simply staggering. I had heard of how stuff like Teflon and composite fibres were a spin-off from the space shuttle program. Here’s a partial list of what the atomic bomb programme gave us – Electro-magnetic separation (that lead to particle accelerators), Gas diffusion technology, high speed X-ray photography, Geiger counters (well duh), micro-chemistry, monte carlo simulations. Entire fields of science sprang up from this one endeavour. Also worth noting, the complex calculations required for initial H-bomb research almost directly resulted in advances in computer technology.

3. The impact of espionage at a nation-state level is simply staggering. Consider this – in 1943, a scientific paper authored by the then-head of Russia’s atomic energy programme predicted that it would take 15 years for Russia to develop and test an atomic bomb. In 1949, Russia detonated it’s first atomic bomb (Joe-1) which was in every way, identical to the Fat Man bomb dropped over Nagasaki. The only reason that Russia managed to shave 9 years off it’s original estimate was espionage – the incredibly detailed design documents passed by Klaus Fuchs. If there was an object lesson in the power of espionage, this is it.

4. Something that became very clear to me by the time I put down the first book in the series is only referred to in passing by the author at the end of Book 2 – although the basic principles are well known, the sheer effort involved in weaponizing this technology means that this will always be the preserve of nations. It’s easier and more effective for terrorists to make use of low-tech weapons like car-bombs, hijacked plans or even nerve gas. In effect, what the author is saying (and what I believe too) is that all the fear-mongering about terrorists getting their hands on nuclear weapons or atleast dirty bombs is vastly overblown.

Let’s consider the prototypical rogue nuclear weapon scenario shall we? Terrorist gets hold of portable nuclear weapon – in popular media, suitcase bombs.. more realistically, tactical bombs such as nuclear artillery shells. Let us be clear – just getting your hands on a nuclear artillery shell does not mean that it can be detonated like any other explosive device. After WW II, the bulk of Los Alamos’s efforts went towards “hardening” these weapons. In essence, what they came up with was two safeguards – one, the weapon would not arm prematurely and two, it would not detonate prematurely . In the case of a nuclear artillery shell, that would mean the warhead would not arm till a certain distance/time after launch and would only detonate provided certain criteria of speed, rate of descent and altitude were met. For a terrorist to actually overcome all of these safeguards, would require reverse-engineering the entire weapons system. Even if the terrorists managed to subvert a team of nuclear scientists to their cause, this is no trivial task.

The other scenario tossed around is of the dirty bomb – terrorists achieving a partial nuclear detonation. AFAIK, there is no such thing as partial nuclear detonation. Most nuclear bombs today are implosion weapons : a small amount of Plutonium 239, surrounded by a large mass of mildly radioactive U235 is compressed by high explosives detonated simultaneously, which causes the chain reaction to start. Now unless the terrorist actually achieves that micro-second control over the explosive detonation- what you get is a relatively standard chemical explosive detonation spraying chunks of mildly radioactive U235 around. It’s worth noting that an accident did happen at the beginning of the Korean War, when a plane carrying an early US A-bomb crashed and exploded, causing the explosives to detonate. But the lack of a simultaneous detonation stopped a chain reaction. Imagine that, a fully armed A-bomb was ignited but did not actually become an A-bomb.

Given these challenges and low chance of success, why would a terrorist bother with nuclear weapons? Indeed – the same hawks who forced tens of billions of dollars to be spent on nuclear weapons when it was clear that the USSR was nowhere near the US brought the war right to the terrorist’s doorstep – where he has the advantage of terrain, cover and supply lines. The same factors were at play in Vietnam and we all know how that ended.

A year of living comfortably – 2007 in photos

All done! It is with a sense of disappointment that I draw the curtain on my collection of photos for 2007. There were a few “a-ha” moments during that time, but more overwhelming is the sense that I didn’t do much in pursuit of this art.

A simple look at the numbers says it all – My 2006 collection contains a staggering 294 photos, posted in approximately 230 days, starting in late September ‘06 and ending in May 2007.

My 2007 collection contains an anaemic 58 photos. The kicker? I took 226 days to post that number, a glacial pace of 1.8 photos a week (the daily number is just too pitifully low to talk about).

Now, it is often said that in photography volume is no measure of quality. I could offer that statement to defend the lack of effort I’ve put into my photography.

More practically, I could tell you that in May 2007 I moved from an apartment near the Lower Seletar Reservoir to an apartment right in the concrete heart of Yishun. It is true that the loss of my muse affected me deeply. Not having a ever-changing subject close at hand every day definitely dulls the desire to take out the camera and go hunting.

I could always fall back on the old standby – work was a killer this year. That would be somewhat true as well.

Truth be told, none of those really explains why I have almost no photographic output to speak of. The reality is that I used a mixture of all those reasons to stay within a zone where I felt I had some idea of what the final result would look like.

So, yet more photographs of buildings and shots taken in the dawn hours. I worry that the ability to shoot portraits or just street-shooting has simply withered away under this onslaught.

What then does 2008 hold? A few points are worth talking about:lightroom-collections-keywords

#1: To establish my geek cred, I will begin by talking about my photographic workflow. For most of 2006 and 2007, I really didn’t have a workflow. Despite access to an extremely powerful photo management software, I spent a lot of time organizing my photos manually. Now that I’ve run of photos to actually go through, I spent some time thinking about how to speed up organizing and editing my photos and came up with a workflow that hopefully is a lot more organized.

First, I decided to switch to Adobe Lightroom – being able to organize my photos and do most of my processing in the same application should help speed up things.

I also spent a fair bit of time looking over how I organize my photos on Flickr and the keywords I set up in Iview. I then split them up into distinct sets of information and will be exclusively organizing my photos this way. Lightroom will take care of where the photos actually live.

#2: In contrast to the constant travelling that characterized 2005, I spent almost the whole of 2006 and 2007 right here in Singapore. Now I like this place a lot, but nothing quite gets the photographic eye working like something new (my Macau trip is proof of that – a fifth of the photos I posted in 2007 came from that 2 day trip). I expect to travelling again in 2008 to a couple of countries and even if it’s just buildings and hotel rooms, hey atleast it’ll be new buildings and hotel roomsCanon Powershot S5 IS.

#3: As much as I love the portability of my current camera and have been amazed by what it can do, there’s a lot missing in it – manual control of aperture and shutter speeds, optical stabilization, better high ISO performance, RAW shooting; the list could go on. I’ve been talking about buying a new camera for years now but if a few things on the personal front come together, maybe this year will be it (and I’ve probably jinxed it by saying that – oh well).

So there you have it – my very short look back on the year that was and an ever shorter look ahead. I would say watch this space, but that would just be cruel and unusual punishment :)

The lazy geek’s way to instant alerts – for anything!

We’ve all been there – something vitally important (!1!!) has gone belly-up and we need answers – fast. It just so happens there’s a discussion forum where we can ask for help. You’ve registered, made a gigantic post detailing every single aspect of the problem that you are aware of and now you are waiting for those replies to start rolling in.

Except well, you have no way of knowing the minute a reply actually comes. Sure, you could activate the option to have an email sent to you whenever someone posts a reply, the problem is – you used your “spam” account (the one you never bother to check remember?) or worse, you used a throwaway email account (10 minute mail, guerilla email to name just two).

If only there was a way where you could get information pushed to your desktop whenever a page was updated… oh yeah, how about RSS? Slick new Web2.0 style forums such as Get Satisfaction do offer RSS feeds for each thread and several plugins are available for phpBB (the grand-daddy of forum software) that add RSS support, but it isn’t a given.

Enter page2rss.com. page2rss is a simple idea – scrape a given URL and any time the page changes, update an RSS feed with an excerpt of that changes. So, you take the URL of your forum thread, plug it into page2rss.com and voila! instant RSS feed.

Page2RSS Feed

Obviously this isn’t perfect, but we’ll take about caveats later.

So, we’ve solved the big problem – we now have an RSS feed that will instantly get updated whenever our vitally important forum thread changes.

Except, well 2 things:

1. You might not leave your feed-reader running all day, so how do you find out when that RSS feed updates?

Yes I know you might be of those lucky folks who can live in their browser all day and refresh their feed reader every 30 seconds. The above applies to not so lucky folks (like me)

2. Cluttering up your feed list with one time feeds that you have to remember to unsubscribe from later can get pretty annoying.

So now what we need is a service that can read an RSS feed and alert us whenever the feed updates. Also, we should be able to quickly unsubscribe from that feed once it has served its purpose. Ladies and gentlemen, I present to you – FeedCrier.

Feedcrier provides an IM based RSS alert service. You add Feedcrier to your contacts list, give it a list of feeds to monitor and whenever a feed updates, you get an alert.

FeedCrier

If you use the FeedCrier bot on Google Talk and you log IM conversations in your Gmail account, there’s another benefit – you could log off Google Talk and if FeedCrier updates when you are offline, it appears in your Gmail Inbox as a saved conversation – awesome! :D

So there you have it – between Page2RSS and FeedCrier, you can set up a system that alerts you to changes on any page you are interested in and know whenever it happens in your email or in your IM client – what’s not to love about this?

A couple of things:

1. Page2RSS is after all a site scraper. This means it doesn’t distinguish between the main content of the page changing and a change in the footer text. A typical example – most forum threads list the number of registered users on the forum. Every time this number changes, Page2RSS will update the RSS feed for the page. On a popular forum, be prepared for lots of spurious updates.

2. FeedCrier isn’t a 100% stable – I’ve had it disappear from my IM client for a couple of days at a time and at other times, get incredibly slow at responding to any commands I type in. This could be because I use a non-standard Google Talk client but YMMV.

Now you can be racist without ever talking to your victim!

I currently live in a rented HDB apartment in Yishun. Soaring rentals forced my wife and I to move out of the (slightly-decrepit) condominium apartment that we had lived in for 2 years since moving to Singapore.

Despite the occasional sense of regret about the lack of facilities at your doorstep (swimming pools, gymnasium etc.), my wife and I settled in fairly quickly into the new HDB flat. We both agreed that atleast one thing was common between our old apartment and the new place – neighbors who stayed at the head-nodding stage when we passed by each other in the corridor.

That all changed this past Sunday evening. At around 4 PM, a series of knocks and insistent doorbells rings bring me to the door – to find 2 policemen standing outside, eyeing me suspiciously. They inform me that they have received complaints about a clothes-stand outside the apartment “blocking the corridor”. I apologize and tell them I will move it closer to the corridor walls. The policeman shuffles his feet and says something about “That’s upto you sir but… “. When I push him about what I should do – he says “Actually, they are saying because of your clothes-stand, they cannot open the windows in their apartment”.

I blank for a few seconds here – They? They who? And then suddenly it dawns – my neighbors! My head-nodding, never-disturbing, back-stabbing sons of bitches neighbors had gone to the police! Apparently, we were getting in the way of their million-dollar view of the identical windows of the HDB apartment block next-door. So instead of coming over, knocking on our door and asking us to remove the offending pieces of clothing – they got the Singapore police to their dirty work for them.

Just when the confusion and shock had passed and anger was beginning to set in, I see that one of the neighbors was talking to the police – “Why are you talking to them? He is a tenant! Call the landlord! Ask him for the landlord’s number!”. The anger disappeared and fear gripped me. It’s an open secret that most HDB flats rented out are not legally allowed to do so – hence the phenomenon of the “locked room“. My neighbor knew this and by trying to drag the dubious legality of my rental agreement into this, had just upped the stakes. I either acquiesced or faced scrutiny from 2 or even 3 other government departments.

I man-handled the offending clothes-stand into the apartment, then handed over my identity card to the police-officer for his records.

As I closed the apartment door, a bitter taste flooded my mouth – the acrid taste of racism. Sure, all of us have had chinese folks (typically ladies between the ages of 25 and oh, 100) remain standing on the MRT than sit next to us or even stand up and dash to the opposite side when a seat became available. We have heard the stories of cab-drivers refusing to stop for Indians waiting near popular watering-holes, as we don’t tip like the rich ang-mohs do. But these were stories of strangers – surely people who knew us, who saw us everyday could look past our skin colour?

I now know better – the chinese in Singapore, rather than have their sensibilities offended by having to talk to the “native” can get the Singapore police to do it for them. Even better, it appears the Singapore police have nothing better to do than resolve complaints of underwear fouling someone’s feng-shui.

The final bonus? I now have a police record for the first time in my life – Woo fucking hoo.

I fear my Government, anywhere

At this moment, there are two petitions sitting in my feed-reader/email asking me to do the right thing – Get the Singapore government to repeal a law against gays and another asking for action against Myanmar.

In my heart I know the right thing to do is to sign these petitions. I also know I should be brave and put my name where my opinions are. Another part of me “knows” that this not really going to change the Singapore government’s opinions or actions.

Yet, a tiny voice in me stops me from clicking that final submit button. The voice that reminds me of the horrors that have visited so many of my country-men in India when they have stood up against corrupt or unjust officials. Lives broken, careers ruined, families torn apart.

The voice reminds me that I am living in a strange country – my only right to living and working here a small plastic card. That card keeps my family fed, puts a roof over our heads and allows us to fulfill a dream of owning a house.

It asks “Are you really sure you are safe? Confident that this will not get your visa canceled and have you sent back to India in disgrace?”.

I listen to the voice and I am afraid. I am embarrassed to be afraid, but I cannot click that Submit button.

This is the real price of living under an unjust government – you are too afraid to protest anytime, anywhere.