A Twitter – Google Maps mashup to view your friends network …

So it all started off with me wanting to see the geographical distribution of my Twitter friends (the Twitter-ers that I am following) and since I don’t host a website (and WordPress doesn’t allow JavaScript in blog posts) I had to do all this in Java.

In the end it worked out pretty well – it was actually quite simple using the Google Static Maps API. All you need to know is some scripting language and you need a Google Maps API Key.

So you start off with the Twitter REST API and get all your friends (the people that you are following) . I used a Java URL object but you can use cURL or any other scripting language that has HTTP URL support. The output looks something like this (the Twitter ids of all your friends) :

<?xml version="1.0" encoding="UTF-8"?>
<ids>
<id>14112660</id>
<id>798731</id>
<id>15256245</id>
<id>16824408</id>
<id>17191554</id>
<id>18743188</id>
<id>26459315</id>
<id>21276419</id>
<id>18702371</id>
<id>34632001</id>
<id>14576471</id>
<id>13954772</id>
<id>16038405</id>
<id>15880443</id>
</ids>

You have to now parse out the IDs from the above output and for each ID (and for yourself also), you have to once again use the Twitter API to get each user’s information. Now the output looks something like this :

<?xml version="1.0" encoding="UTF-8"?>
<user>
  <id>12683672</id>
  <name>Kartick Suriamoorthy</name>
  <screen_name>karticks</screen_name>
  <location>Munich, Germany</location>
  <description>Software professional (JAVA, Network Management, Open Source, Application Servers, etc.)</description>
  <profile_image_url>http://s3.amazonaws.com/twitter_production/profile_images/57456567/got-beer_normal.png</profile_image_url>
  <url>https://karticks.wordpress.com</url>
  <protected>false</protected>
  <followers_count>29</followers_count>
  <profile_background_color>9ae4e8</profile_background_color>
  <profile_text_color>000000</profile_text_color>
  <profile_link_color>0000ff</profile_link_color>
  <profile_sidebar_fill_color>e0ff92</profile_sidebar_fill_color>
  <profile_sidebar_border_color>87bc44</profile_sidebar_border_color>
  <friends_count>14</friends_count>
  <created_at>Fri Jan 25 14:36:13 +0000 2008</created_at>
  <favourites_count>0</favourites_count>
  <utc_offset>3600</utc_offset>
  <time_zone>Berlin</time_zone>
  <profile_background_image_url>http://static.twitter.com/images/themes/theme1/bg.gif</profile_background_image_url>
  <profile_background_tile>false</profile_background_tile>
  <statuses_count>109</statuses_count>
  <notifications></notifications>
  <following></following>
  <status>
    <created_at>Mon May 25 05:25:11 +0000 2009</created_at>
    <id>1909795818</id>
    <text>End of a long weekend (and a long vacation). Now trudging back to stuttgart with my morning kaffee and breze.</text>
    <source>&lt;a href=&quot;http://help.twitter.com/index.php?pg=kb.page&amp;id=75&quot;&gt;txt&lt;/a&gt;</source>
    <truncated>false</truncated>
    <in_reply_to_status_id></in_reply_to_status_id>
    <in_reply_to_user_id></in_reply_to_user_id>
    <favorited>false</favorited>
    <in_reply_to_screen_name></in_reply_to_screen_name>
  </status>
</user>

Now you have to parse out the location (4th element) and use the Google Geocode API to geocode the location. This is a little bit tricky – as sometimes the location attribute can be empty (you have to skip those users) or they can be iPhone lat-long coordinates (i.e. they are already geocoded – see below)

  <location>iPhone: 30.417343,-97.710770</location>
  <description>I do not fear you, gypsy. I only want your tears.</description>

Once you have all the geocodes, you construct a Google Static Maps URL by using your geocode as the center, and all the geocodes (including your location) as markers. I used a black marker to mark my location, and red markers to mark my friends. The result looks something like this …

Kartick's Twitter Network

Ok not very impressive, but still fun …

Caveats :
– Google Static Maps API allows only 50 markers, so if your friends list is more than 50, you will have to prune out the unwanted ones 😉
– The Google Geocode API returns coordinates of a point as a longitude-latitude pair but the markers in the Google Static Maps API expect it in latitude-longitude format and it took me some time to figure out this (I was wondering why my markers were located in Antarctica and sometimes way outside any conceivable point in the globe !!). I wonder why Google did that (iPhone actually got it right and it was trivial to take the iPhone Geocode and insert it into the Google Maps URL) – it would have been just so easy to pass the lat-long pair from the geocode output into the Google Maps URL (to construct the marker’s coordinates) but now I had to add parsing logic to swap the lat-long locations (ugly).

Anyway, I hope this was fun and enjoyable, and as always, the code (TwitterGoogleMashup.java) can be found in the Google Code project, DalalStreet (in the FunStuff project). You can run this class with your own Twitter username (and Google Maps API key) by checking out the project into Eclipse. It will print out a URL when it finishes running and all you need to do is copy and paste this URL into your browser.

Until next time …

What is missing in Twitter Search ? Twitter Find !!

It all started with me trying to find what kind of a dog one of my friends had.
He had posted the breed on a Tweet and I tried to search for it.

And then I realized how difficult it can be to use Twitter Search to find something that is not necessarily happening in real-time. Unfortunately (from my perspective) Twitter Search focuses on giving matches (i.e. Tweets that contain the search string) that have been posted recently (i.e. real-time) a higher ranking. Which is fine for real-time search (however one defines real-time search) but misses out on one important aspect of social networking – the NETWORK. I do realize that Twitter is more than just a social networking tool, but one of the most important reasons I use Twitter is to stay connected with my friends and find out what they (i.e. my network) are doing. And I am sure there are people (not just a few) who want to go back and lookup some interesting tidbit of information that somebody in their network had posted, and if that Tweet was posted a few weeks back, then it is like looking up the 455th link on the 17th page of a Google Search.

The image below shows the search results for “Camtasia” and as one can see, all the results are sorted by time – so if I was looking for a Tweet from somebody in my network who had posted a URL a few weeks back on how to convert a .camrec file into a .mp3 file, I would be totally lost trying to search through these results.

Twitter Search Results

And so I started playing around with Twitter Search and its advanced options, and a host of other Twitter search tools like TweetScan and Flaptor but none of them take into account the notion of the NETWORK (and somehow the whole idea of using another tool to search Twitter is just so un-user-friendly). Twitter Search has an advanced feature to specify the “user” whose update you are interested in but once you are following more than 30-50 users and you are posting and receiving daily updates, one tends to forget who posted what. So basically what Twitter needs is a separate mini-search (I call it Twitter Find) that just looks in the Tweets of persons that you are following (and your own Tweets) to handle a host of use-cases like

– A dish that somebody had recommended at a restaurant (e.g. the Al Pastor at Tacos y Tequila and you remember just “Tacos” and “Tequila” as the search terms).
– A URL that talks about solving a specific problem (e.g. one-to-one Hibernate mapping and you remember just “one-to-one” and/or “Hibernate” as the search terms).
– The name of a dentist that was recommended by somebody to somebody else and you just remember “dentist” as the search term.
– And so on and so forth.

But as of now, to use Twitter search to find something in your network is like using Google to find something on your computer’s hard drive. Sorry, bad example, Google Desktop actually does a pretty good job of doing that !!