I’ve had this idea bouncing around in my head for a few days, and with your patience in mind, I’ll jump right in: what if we could combine the strengths of Twitter and RSS into a standard that could form the new backbone of the open Internet?
What Unites Them
I’m amazed at how “streams” are already serving us as a kind of meta-Internet. A person can explore unlimited content from a huge swath of news, blogs, forums, pictures and video, all from one consolidated page, while never looking at a single one of those sources’ own websites. And by “a person” I mean “me.” And by “can,” I mean “do on a daily basis.” And by “one consolidated page,” I mean Google Reader.
For those who aren’t in the club yet, Google Reader is one of dozens of “feed readers”: applications which gather new content in real-time from any number of scattered websites, and deliver it to you. It’s unusual to find a website that doesn’t publish a feed these days. And if you couldn’t tell, I’m a big fan. I’ve been using feeds for about four years – first through Thunderbird, now Google – and they’re one of the pillars of my Internet experience.
Meanwhile, though, another little world has quietly snuck up on me: Twitter. What I once saw as another social network for me to ignore, the Great Flying Whale is suddenly permeating the fabric of the Internet better than anything short of email itself. More and more sites are letting me use a Twitter account as my primary identity: on Gawker sites, for example, I can log in with it, and even have my comments syndicated as tweets. At the moment, I’m using Twitter via Trillian, where it looks like this:
More interestingly, though, I soon realized that I rely on Twitter for many of the same things as Google Reader. One of the most common uses of a tweet is to pass on a link to some interesting gem, sometimes with a dash of commentary. Further, many websites have dedicated Twitter accounts where they post links to their own content throughout the day. In other words, if you just want to “subscribe” to new content from one of your favorite sites, you can choose to use RSS or Twitter – whichever you prefer. All of a sudden, they’re both doing the same job.
Tweets and feeds have another thing in common, which sharply distinguishes them from other communication services like email and instant messaging: they’re not inboxes. Unless you plan to drive yourself crazy trying to keep up with the deluge of new content, you don’t actually read every tweet or feed item that comes your way. Instead, in the words of Gina Trapani, they’re streams: constant, flowing torrents of information that you can dip in and out of.
And as I thought about RSS and Twitter, I kept coming to the conclusion that there’s more that unites them than divides them. Somewhere in the middle of that Venn diagram, I believe, is the tool we’re all waiting for.
What Divides Them
I think this is the best way to demonstrate how easily these two standards could converge. Naturally, RSS does some things much better than Twitter, and vice versa; but when you identify the key differences between them, you get a list of features that one of them has, and the other not only could implement, but would benefit from doing so in very clear ways.
First, look at where Twitter excels:
- Identity. Everyone has an account that is intimately associated with them as a person. You are in control of your own feed, and those who know you, care about you or value your opinion can directly subscribe to your thoughts. Most people don’t have a “personal” RSS feed. The closest equivalent would be the personal blog: Blogger, WordPress, LiveJournal and others all served us well for much of the last decade, and RSS is a big part of keeping them relevant in the modern web. But the focus is still on the source, not the medium.
- Common input-output tool. All Twitter clients let you send and receive tweets from the same interface. This seems obvious, but RSS has no equivalent to it. Your feed reader does not let you post to a feed, and your feed-producing website has no way of receiving posts. In short, RSS publishers do not function as members of an integrated network. They’re meant for one-shot broadcasts, not a common medium of exchange.
- Conversation. Tweets can be published independently, or as replies to other tweets, allowing users and sources to engage with each other. While Twitter’s implementation of this leaves much to be desired – you still can’t have it list replies to a given tweet, much less view them in a thread – it’s still an improvement over RSS, which has no mechanism for engagement between items. Traditional websites do have the trackback method, but it’s totally independent of the feed and generally limited to blog software.
- Discrimination (the good kind). A tweet’s visibility can be limited to one’s followers, or to a specific recipient. Again, Twitter’s method is primitive; you can’t choose your own groups, for example, and (for no good reason that I’ve ever heard explained) you can’t send a direct message to someone who isn’t following you. But a feed, like a radio broadcaster, is blind. It has no way of tracking its subscribers or discriminating between them.
- Independence. Every tweet actually has its own dedicated page, complete with a permalink URL. An RSS item usually links to a permanent page on the source site, but it has no form of its own, except as a bunch of XML metadata in the feed file. This is also true of the content: while a tweet may link back to a source, the tweeter usually has some room to add a summary, or a few of his own thoughts. An RSS item, on the other hand, is generally a carbon copy of whatever triggered it. Again, this goes back to the issue of creating stream content: where an RSS feed is a zombie, merely translating a single source into something a machine can read, a Twitter feed is dynamic, connecting subscribers to any number of integrated services – or directly to its owner.
On the RSS side:
- Sorting. Feed readers are much better at sorting subscriptions by user-defined groups and reacting to them in different ways. For example, if one of my best friends posts to his blog, I might want to be notified right away, whereas a new webcomic strip can probably wait until the weekend. I think this would be even more valuable for Twitter. A sorting system would make it much more feasible to subscribe to dozens or hundreds of streams and still be able to control your own system of consumption.
- Archives. Because a feed reader actively downloads and stores new items to wherever it’s hosted, you’re not at the mercy of external forces: people who prune their old posts, or servers losing scores of data when they crash and burn. It’s also easier for a locally-run reader to mark items as read/unread or “favorites,” especially in the absence of a central server.
- Decentralization. RSS items are downloaded directly from the host to the reader – there’s no middleman whatsoever. WordPress is a perfect example of the greater freedom this gives us: without being dependent on a central server, you can run exactly the same software on a free WordPress.com account, or on your own server; it makes zero difference to your RSS subscribers.
The only advantage of Twitter’s central server that I can think of is in search. Using conventions like the hash tag, all of the tweets about a given topic – or a person – can be indexed, searched, displayed on one page and updated in real-time. This would be difficult on a decentralized network. In addition, having every Twitter profile on the same site, in the same format, is probably an advantage to discovering new feeds and acquiring new followers. But there is a solution to this problem, and it’s already been done by none other than Google.
Google Reader keeps a syndicated archive of literally every public feed that any Google user has ever subscribed to. Naturally, this archive is completely searchable, and Reader has a tool for recommending both items and whole subscriptions based on your reading habits. Twitter, if it became available for self-hosting, could do the same thing, either by “following” as many of its far-flung users as it finds, or by having the software (with its user’s permission) actively talk back to the central server. Thus, Twitter.com would act not as a host, but as an index. And anyone else would be free to set up a parallel index that does exactly the same thing.
So. What would this look like?
The Proposal
Let’s call it a “spring” for now, in keeping with the waterworks theme. In my digital dreamland, a spring is a piece of web-based software, probably running on PHP and MySQL. The spring has a standard protocol for talking to springs on other servers. It keeps a local database of all the other springs its owner is following, and all the springs following him in turn.
The owner can publish items using nothing but the spring’s web interface; or he can publish through a synchronized desktop or mobile app; or he can connect it to his blog, his forum accounts, his Flickr stream, his YouTube channel, or any other service with content that he may wish to syndicate, probably with OAuth.
The owner is able to sort all of his followers into groups, and new items (which are usually publicly visible, in a browser or from any spring-compatible software) can be restricted to groups or individuals. He can add springs that aren’t following him as recipients of his items, though their own springs will have options for filtering out unsolicited content. He can also sort his followed springs into groups, and choose to receive notification – by email, IM, wave, SMS – for new items in particular groups.
He can choose to follow another spring anonymously. He can choose to block another spring from receiving his public items. He can add the URLs of index sites, much like BitTorrent trackers, inviting them to index his items; or he can block them from indexing him. The spring can also search any connected index from within the interface.
Items can be posted in response to other items. The owner can decide whether a reply also appears in his public stream, or is sent only to other recipients of the original item. Reply IDs can be traced to display the complete thread, unless one of the links in the chain was a private posting; the thread is not hosted on any spring, but locally generated when accessed. Users can track a particular item (without following its owner), or ignore an item from a followed spring.
How close are we?
Google Buzz got a lot of things right. The issue with Buzz – much like Wave, sad to say – is that it’s hosted only by Google, and tied to a Gmail account. But they got a lot of things right – integration, threading, grouping, notifications, and especially services integration. If I were able to manually add both RSS feeds and Twitter accounts to Buzz, and maybe even associate them with my Contacts, I might be a satisfied customer.
Diaspora is being released on September 15th, and I’m very interested to see how much of this is already in there. I predict that its ability to integrate and synchronize with other services, particularly Facebook, will either make or break it.
Speaking of Facebook, I do think they deserve credit for creating and maintaining a complete, fully-integrated miniature Internet within a single site. If it weren’t inextricably tied to facebook.com, it honestly wouldn’t be hard for me to imagine the Facebook “protocol” becoming the de facto standard. But without decentralization, Facebook lives and dies with the reputation of Mark Zuckerberg, and that, as we’ve all observed, is not a stable equilibrium.
Finally, there’s the mythical Google Me. “Me” was supposedly coming any day now – back in July – and I’m not sure what to expect at this point. But Google has a lot of puzzle pieces on its table: Buzz, Talk, Voice, Contacts, Wave. If they have some ingenious plan to weave them all together, I’ll be first in line for beta invites.
Bonus prediction: Apple already thought of everything in this article six months ago and will make me look like an idiot at tomorrow’s press event.
