James Whittaker - Interactive developer, designer and consultant

How I built my lifestream

comments closed posted on 2 February 2008

Yes, I’m a bit late to the party with the whole let’s aggregate some data together thing. However new site so I thought wouldn’t it be great to build my own lifestream, a chronological listing of all my online activities. Well, web 2.0 is all about freedom and interoperability of data so a lifestream it is.

Step 1 – Identify your feeds

What do you use online frequently? Photos to Flickr?, bookmarks to Delicious or Magnolia? And I bet your a massive Twitter fan too right? Yes see you have a list already. Basically any web based service that you currently use could be used as an input to your lifestream. Most sites provide public RSS or JSON feeds for you to use those mentioned above do.

So go and list the URL’s for all of your feeds.

Step 2 – Mashing it up

Now there are so many ways in which you could combine your feeds together. You could do this server side using PHP like Jeremy Keith, or Django like Jeff Croft anything you are comfortable with. I decided to try a different route. Yahoo Pipes has been around for a while now and the toolset is getting better. Yahoo pipes is built for mashing up information into new feeds using a great drag and drop graphical user interface.

What I wanted to achieve with Yahoo! Pipes is to combine my RSS feeds together. Then tweak a few things like:

• Add a feed name to each item
• Add a foreword to each title element
• Sort all feeds by date

Better that explaining in detail what I have done in Pipes it’s better if you head on over and have a look for yourself.
You can view my lifestream pipe here.

Also be aware that Yahoo! Pipes caches it’s feed output for obvious performance reasons. I’m not sure what the actual time for the cache is.

Step 3 – Consuming your new mashed up feed

There are a number of different output formats that Pipes supports. Of most interest for us is going to be either RSS or JSON. The choice depending on how you are going to be consuming your data. If you plan on using Flash or Flex then it’s better to opt for an XML based feed like RSS. This time I decided that I was going to use client side JavaScript to inject my feed items into the DOM of my page, going to use JavaScript then it’s easier to use the native JSON format. It is worth pointing out that Pipes supports different formats from the same pipe so you don’t have to create new pipes for each format you wish to use, it’s just a parameter in the URL query string.

Using JSON can be pose a security risk especially if you don’t know or trust the source, this is because you need to eval the response on the client to turn it into JavaScript objects. Yahoo makes using JSON easier and safer (well they did come up with the idea) by passing a callback function name along in the query string. This eliminates the need to use Ajax for fetching the data then having to eval the response. The process is as follows:

• Dynamically add a new script tag into the page DOM setting the source as the URL of your pipe plus you callback function name
• Page loads this script as if it were native JavaScript in your page – the response back from Pipes is of type application/json
• Your callback function is called passing in the JSON object
• Your function then manipulates the data

Read more about Yahoo! and JSON within their web services here. It’s a good article that will really help you understand JSON better.

Step 4 – Injecting the results into the DOM

You now have a JavaScript object full of data from your feed, time to add it to your XHTML by injecting new elements into the DOM. Sounds scary well not really. This can be achieved using plain JavaScript but I felt the need to play with the MooTools JavaScript library, having previously used and loved the now very overweight PrototypeJS library. The great thing with MooTools is that you can pick the elements of the suite that you need reducing the file size dramatically if you don’t require all of the extras.

Firstly inject the new script tag containing your Pipes URL into the head of your page.

	
		var headId = document.getElementsByTagName("head")[0];
		var newScript = document.createElement('script');
		newScript.type = 'text/javascript';
		newScript.src = your pipes URL goes here as a string;
		headId.appendChild(newScript);
	

Using MooTools I have taken a class based approach to the JavaScript, doesn’t really make a difference here but I think it looks neater and prepares you for any bigger stuff.

To get the code to run when your page loads MooTools has a new event that you can attach to the window which detects when the DOM is ready. This is better in many sense because the DOM can be ready before your page is fully visually rendered by the browser:

	
		window.addEvent('domready', function(){init()});
	

Now create a function to handle the response callback from your Pipe. This must be the same as what you passed in on the query string URL earlier to Yahoo!. My callback is called ProcessJSON(obj). Inside this function you will need to loop over each item in the JSON object:

	function ProcessJSON(obj)
	{
		var items = obj.value.items;
		
		//Go through all of the items
		for(var i = 0; i <  items.length; i++)
		{
			//build new DOM elements for each item here
		}
	}	

To build a new DOM element using MooTools is really easy, see the example below for creating a new definition description DOM element then adding some text to that element:

	// create element
	var myDD = new Element('dd', { id:'feedItem', 'class':'vevent'});
	
	// add text node to above element
	myDD.appendText("hello i'm a text node");
	
	// add an element inside of the dd - example 
	var myAbbr = new Element('abbr', {'class':'dstart published dtend', title:currentItem.pubDate}).injectInside(myDD);
	myAbbr.appendText(postedTime);
	
	// now add inside the dd created above
	myHref.injectInside(myDD);
	
	// once you have all of your elements add them into the DOM
	$('lifeStream').appendChild(myDD);

For my lifestream I used a definition list to hold all of my items. I think this is a really semantic way of marking up this type of data. For the titles I have the date of the posts then underneath I then have a series of definition descriptions that describe the item for that date. My Javascript contains a little more date sorting to get the nice headings out of the numeric dates sent from Pipes, view the javascript and take a look. It’s all fairly harmless, I promise.

Step 5 – Feed icons

To get the icons I made my own and placed them into an images directory, naming them the same as the feedname that I added during my Pipe creation. That way in my code when I’m building the elements I can just place the feedname for the image file name:

	var myImgIcon = new Element('img', {'class':'postType', src:'images/' + currentItem.feedName + '.gif', width:'32', height:'32', alt:currentItem.feedName});
	myImgIcon.injectInside(myHref);

Step 6 – Showing your flickr thumbnails (if you have flickr images)

I wanted to show a thumbnail image of the photo that I uploaded to my flickr account.

This is really easy as the feed from flickr has the url for the thumbnail already set in the response. You will have to be a little careful as it is namespaced to ‘media’. So just add a new image DOM element setting the source to be the url from your flickr feed and the size is 75px square. Don’t forget your alt tags I use the image title again as that makes semantic sense.

	//if flickr
	if(currentItem.feedName == "flickr")
	{
		var myImgFlickr = new Element('img', {'class':'flickrImg', src:currentItem["media:thumbnail"].url, width:'75', height:'75', alt:currentItem["media.title"]});
		myImgFlickr.injectInside(myHref);
	}

Step 7 – Styling

Well I shall leave that up to you!

Finish – grab a nice cool beer!

I hope that has given you a few pointers and ideas that will help you build a lifestream for your site. All the techniques used here can be done with any JavaScript library or none at all. You could go crazy with effects etc if you like, I have just opted for a nice fade in when all of the elements are built.

The good thing is that you now have a feed that you could re-use in Flash or Flex and display your lifestream in new and fantastic ways. Go crazy!

add your comment permanent link to article

posted in: Blogging  | CSS  | Javascript  | UI