Karthik Murugan's Blog

Released new version of "Daily Cartoons" App

Sun, 19 Dec 2010 12:57:00 +0530

Published next version of my Android app - Daily Cartoons. New features are,

Ability to browse and view cartoons from various channels like AnderToons, Cricinfo, Times Of India, Deccan Chronicle, Amul.
More channels will be added on the fly, without needing to upgrade the app
Cartoons are now grouped by publishing date viz. Today, Yesterday, 2 days ago and older.

Download by clicking this link from an Android phone. Or scan this QR code

Daily Cartoons - Android app

Tue, 19 Oct 2010 16:42:00 +0530

My first Android app is in market now. Its a simple app to browse cartoons from CricInfo.

Download by clicking this link from an Android phone

Search for “Daily Cartoons” in market

Scan this QR code

Future releases will add Cartoons from magazines like Times of India, Indian Express, Deccan Chronicle.

I also hope to add features like notifications, unread items, custom subscriptions in future releases.

Twitter OAuth PHP library

Tue, 07 Sep 2010 23:23:00 +0530

Twitter API shut down Basic Auth on August 31. I was a big fan of the Arc90 Twitter library, but its no longer usable because of the shutdown. Even before the shutdown, I’d been using Abraham’s twitterOAuth for Twitter Signin integration with my websites. Its a nice library that hides the OAuth security aspects from developers. But after having used Arc90 library in all my projects, migrating the code to another library was costly and time consuming.

So, I merged both the libraries. I enabled OAuth in Arc90 library by making some internal calls to twitterOAuth library. This way, I could continue using Arc90 library in my projects just by porting the new library and adding few lines of code to pass access tokens to the library object.

I’ve published the library in GitHub at http://github.com/yemkay/arc90-twitteroauth, which is also my first Open Source contribution.

Here is a code snippet that retrieves Twitter home timeline using the new library:

require_once("lib/Arc90/Service/Twitter.php");

$twitter = new Arc90_Service_Twitter();
//Pass consumer key, secret of Oauth app and user's access token, secret
$twitter->useOAuth('OAUTH_CONSUMER_KEY', 'OAUTH_CONSUMER_SECRET', 'USER_TOKEN', 'USER_SECRET');

//Get tweets from user's timeline
$response = $this->twitter->getFriendsTimeline('json', array('count' => 200, 'page' => $page));

echo ('HTTP code: '.$response->getHttpCode());

if (!$response->isError())
{
	$messages = $response->getJsonData();
	echo 'Found '.count($messages).' new tweets';	
	/* GO AHEAD AND PROCESS TWEETS */
}
else
{
	echo 'Error description: '.$response->getData();
}


?>

Combining MySQL FULLTEXT match with other WHERE conditions

Sat, 21 Aug 2010 23:26:38 +0530

I was too late to realize that a query like the following, which combines FULLTEXT match and another where condition is not scalable.

SELECT * FROM mytable

WHERE MATCH(message) AGAINST (‘inception’) AND is_member = 1

(In this table message is FULLTEXT indexed and the boolean type column is_member is B-Tree indexed)

The query is not scalable, because MySql can’t use both the indices (It can use only one index per table). And, we can’t index both these columns together too.

The solution is pretty simple. While inserting message into the table, append an unique value to identify the record as member’s one. I’d do something like this,

INSERT INTO mytable(message, is_member)

VALUES(‘Just saw inception ismember’, 1)

Now, MySql indexes the word “ismember” along with other texts in the record, and we can use a query like the following, without the extra WHERE condition,

SELECT * FROM mytable

WHERE MATCH(message) AGAINST (‘+inception +ismember’)

This query will return only the member records that matched “inception”. This query uses just one index and runs pretty faster than the previous one. The difference can be seen, when you have millions of records in your table.

This example showed a boolean column. But this can be extended to columns of any type. We just have to make sure to append unique identifiers to the FULLTEXT indexed column.

Associated Parsers: FootTweet and Twitter Streaming API

Tue, 06 Jul 2010 15:24:00 +0530

Associated Parsers: FootTweet and Twitter Streaming API:

My post on AP blog about how we used Streaming API for gathering Twitter posts to FootTweet

FootTweet is a virtual FootBall game played on the platform of Twitter. We use the powerful Twitter Streaming API to get the tweets to FootTweet in a few seconds after they were posted to Twitter.

When you have number of keywords to track and you want the information in real time,…

Associated Parsers: Foottweet !

Sat, 03 Jul 2010 15:22:00 +0530

Associated Parsers: Foottweet !:

Our first launch from Associated Parsers. Go check out!!!

Introducing FootTweet, a very advanced web app that displays and categorizes tweets according to their content !

We’ve decided to show our technology applied to the World Cup, ‘cause there’s about 3 billion ppl watching, and we thought some of them could be tweeting as well ;)

Twitter.com…

Quick hack on Twitter Follow bug

Tue, 11 May 2010 15:25:00 +0530

We made a quick hack to find out the most demanded followers during the Twitter follow bug drama.

Associated parsers:

Seemingly Twitter had a small bug tonight. Here are the top “requests” for acceptance made by users : http://www.associated-parsers.com/accept/

CricTwits

Sat, 18 Apr 2009 12:19:00 +0530

Check out my biggest launch till date: CricTwits - an aggregator of Twitter conversations revolving around Cricket matches. This project was inspired from Title Tweets, a similar initiative for a basketball tournament in USA.

It took 2 weeks to build this website after getting an inspiration from Title Tweets. CricTwits collects twitter posts containing references to cricket matches, team names, player names. The match page shows tweets favoring the two teams and also allows you to post to your twitter timeline from within the Website. There is also a live scoreboard on the match page. So, you have one reason less to leave the website.

I timed the launch to coincide with IPL. CricTwits is a great platform for IPL fans to stay close to thier favorite club. Go Tweet!!

Postal Address Locator (PAL) - an innovation by India Post

Sat, 16 Aug 2008 17:50:00 +0530

Those who think Postal Services are not meant for this generation, rethink!!!. India post has proposed to replace the age old Postal Index (PIN) code with 8 digit Postal Address Locator (PAL) code. In the past we have seen such revolutions in Telecom industry, when the number of digits in our telephone number increased from five to six and then to eight. Now we will soon see our pin codes also changing this way, but your new code is not going to be your old code prefixed with the state/city code.

As per this proposal, the new PAL code will have 8 digits. First two digits denote the state, next two digits denote either the metro or the district. For a metro locality, the fifth digit denotes its zone and the last three digits denote the locality. For a non-metro locality, the 5th digit denotes a town in the district and the last 3 digits denote the village.

For metro localities

22015248

{State}{Metro City}{Metro Zone}{Locality}

For non-metro localities

22106178

{State}{District}{Town}{village}

In third and fourth digits, the range 01 to 09 is used to denote a metro in the state and values 10 to 99 are used for districts. So, this code can accommodate 9 metros and 90 districts in a state, which is good enough even for the largest state U.P, which has 70 districts. It’s imminent that the code for state capital will be 01. Metro, in this context, doesn’t mean metropolitan cities but municipal cities. So second tier cities like Coimbatore, Pune, Surat, Mysore will be identified from each state and metro codes are assigned for them. There can be 9 such cities from a state.

If the fifth digit is between one and five, it denotes a zone in metro and other values denote a town in the district. As this system accommodates just 5 towns in a district, it’s not clear on what basis such towns are identified from districts.

At maximum, the system supports 9999 villages in a district or same number of localities in a metro. So the smallest entity identified by this code will be a village or a metro locality, regardless of whether it has a post office or not. A question arises here is that a small district in the state of Pondichery and the Mumbai city, which has a population of 2 crore, will get the same number of slots for their localities according to this design. I think, for postal department geography is more important than the demography. So Mumbai with an area comparable with that of a district will get the same number of slots for its localities. Rather I feel, by allotting single digit metro code value (01 to 09) in the third and fourth positions, they are wasting a precious digit in the PAL code. We could increase the number of locality slots for a metro by 10 fold, if just the third digit is used for the metro code and the last 5 digits are used for the localities. India post encourages citizens to send their views on the new PAL system. Note the email address - deputydgpoi at gmail.com. I’ve just sent the above suggestion to this email.

The proposal document claims that the first half of the code will be used to segregate the outgoing mails and second half will be used in delivering the incoming mails.

Another interesting aspect I noted from this document is about the increasing passion in Government organizations like India Post and BSNL. See this excerpt from a slide for popularizing the PAL system,

Launch a bold and innovative publicity campaign for 2 months before introduction that should catch the imagination of the nation like wild fire - Something like KBC or IPL

Would you have imagined this passion in a Govt. organization? Though they are far behind the private organizations in terms of liberal decision making, there are already promising signs of improvement. One should note that unlike growth in the private sector, the growth in Government sector triggers growth of the Nation by multifold.

A RSS feed for your country’s medal hunt in Olympics 2008

Sat, 09 Aug 2008 21:18:00 +0530

I’ve created a Yahoo Pipe to find the list the medals won by a country in the ongoing Olympics 2008. Visit the pipe, enter the country code and click on “Run”. Find the list of Country codes here.

The results are displayed as shown in this picture,

And how to create a feed for the medals won by your country’s Olympians from this Pipe? Here you go!!. This is the format of the feed url,

http://pipes.yahoo.com/yemkay/olympics2008medalfeed?_render=rss&country=XXX

Replace XXX with the country code. Open your feed reader, subscribe to this URL and be the first to know when Olympians from your country win medals. Feed for my country is still empty. Dont miss the hot ones - China and United States.

Read the related entry in Pipe Feeds

Pipe Feeds - a website for showcasing my Yahoo Pipes

Sat, 09 Aug 2008 12:37:00 +0530

Pipe Feeds is the directory of my Yahoo Pipes. I will publish the Yahoo pipes created for my personal use on this website. Most of my pipes attempt to simplify the way you read your favorite content across the websites. My objective of creating Pipes is that, if you like a content from a particular section of a website, let the content come to you next time without you having to visit the website again. Widgetizing and embedding are some other benefits from Yahoo Pipes. This diagram depicts it more clearly.

To start with, I’ve published pipes for Cricket Quotes (from cricinfo), ThiruKural Widget and History of Tamil Cinema (from DinaThanthi).

The pipes Cartoons from Indian dailies, Cartoons from Tamil dailies and Adade Cartoons feed which were created in the past and later graduated to auto publishing sites are also listed.

The site is hosted on blogger. Each post will represent a pipe and has various links for embedding, gadget creation, feed subscription and Pipes source.

Read more about Yahoo Pipes and Pipe Feeds site

View cartoons from Indian magazines

Thu, 17 Jul 2008 08:32:00 +0530

In continuation to the Adade Cartoons site and feed, I’ve created two similar sites for aggregating cartoons from English and Tamil magazines.

The Toons of India site pulls cartoons from Times of India, Economic Times, Indian Express, Deccan Chronicle and Hindu, while the Tamil cartoons website aggregates cartoons from the popular Tamil magazines of Dinamani, Dinamalar and Dinathanthi.

View Toons of India
View Tamil cartoons

As explained here, these custom aggregators were created using Yahoo pipes, a free tool to generate custom feeds.

Common problems faced in Yahoo Pipes

Mon, 30 Jun 2008 07:41:00 +0530

Why my Yahoo Pipe shows “No Results” on running it?

Beginners in Yahoo Pipes may end up getting this error message while running their Yahoo Pipes. There are two reasons for a Yahoo Pipe not showing any results.

The Pipe Output is not wired.

No description element
Make sure your pipe has an element called “description”, which is required for any HTML based feed output. Output the HTML code of your pipe only to the “description” element. Having your HTML output in any other element will make your pipe output go unrecognized by feed readers.

If your pipe still shows no results, ensure that your pipe sources, i.e. modules like Fetch Page, Fetch Data and Item Builder, pull data needed for the pipe. If you find nothing fishy, debug the pipe starting from “Pipe Output” module and navigating upwards.

Finding it difficult to Regex multi line text?

Simple. Convert it into single line and Regex it. Use a Regex command to replace the new line characters with empty text and apply your Regex commands on the single line text. Do like this,

Noted the option boxes checked in both the commands? Its ‘g’ and ‘m’ for the new line replacement and ‘g’ for remaining commands.

How to add more elements to the feed?

Use the Rename module and copy an existing element. You can remove the source element’s contents by using a Regex module as below.

Connecting feed output to a string module

Yahoo Pipes doesn’t allow you to wire a port of type “items” to a port of type “string”. But in many cases we require this wiring to process the strings in the feed items. Here is how you should do this…

Add a loop module and place the string module within the loop module. Then connect a module’s “items” output with the loop module. Now the fields in the string module, which is now hosted on the loop module, will be populated with the elements available in the input feed. In the loop module’s field ‘For each’, select the element which has to be processed by the string module. This element in each item of the input feed will be processed by the string module and the string output can either add on to or replace the elements in the input feed, depending on your selection between “assign results” or “emit results”. Here is a sample Pipe that demonstrates Loop module.

Chart added to Poorman’s Stock watchlist

Fri, 18 Jan 2008 14:22:00 +0530

Today is a testing day for naive investors in the Indian stock market. Index advanced and declined at 1:14 and it touched 700 negative in late trading. I wanted to make some quick bugs by investing in a loser and selling it when the market bounces back. So I shortlisted five such stocks using the BSE trends worksheet. As I also wanted to track few other stocks that are not in the index, I added all of them in my watch list.

But that didn’t help me tracking my favorite losers. Charts in Google Spreadsheets came handy at that moment and I created a new worksheet for the Gain-Loss chart which shows only the items in the watch list.

BSE and NSE entries are grouped in the chart. This is a published version of the chart which keeps on updating in every 5 minutes (Refresh the page to see the latest).

The chart may not be useful in the published version of the spreadsheet, but in the editable version, it’s interactive and shows a tool tip upon clicking the bar.

World Wide Spreadsheet - WWS

Thu, 29 Nov 2007 16:03:00 +0530

The title doesn’t mean that a spreadsheet can be world wide, but can be a powerful www contributor. Believe it or not, I’ve just constructed a stock watch list in a spreadsheet in 2 hours.

I’m a great fan of Microsoft Excel and have used it effectively for many of my projects. It allows you to manage large data effectively. You can do calculations, define formulas, conditional formatting, lookup in the sheet and even you can create interactive forms out of excel. It also has a nice feature, by which you can import XML data from web and map it to a range of cells. Data becomes dynamic in this case.

Well, if Google is coming to this market then there must be lot of unexplored opportunities in this field. It is actually, and having worked on a Google spreadsheet for sometime, I’ve realized its potential. It allows you to import XML, HTML and CSV data from external sources into the spreadsheet. There are functions like importHTML, importXML and importData that help you to do so. Interesting one is the importHTML function. Use this function to import a table from a web page and use the CONTINUE function to refer to a particular cell in the imported table. ImportHTML needs the URL and the index of the table in the HTML page. CONTINUE function takes the address of the cell, which has the importHTML command, the row number and the column number of the table cell to be referred.

For example, ImportHtml(“http://en.wikipedia.org/wiki/Demographics_of_India”, “table”,4) imports the demographic information for the population of India from a wiki page.

For more info on the above said functions visit this help page.

Here is a small example to illustrate the potential of Google Spreadsheets. This sheet imports data from ICICIDirect.com. I’ve created sheets for Indices, NSE & BSE exchanges and watchlist. First three are open to public as no input is needed for them. Watchlist, the last one is for my personal use where I can add quotes of my favorite stocks to track them.

View this spreadsheet in a new window.

If you never seen a Google spreadsheet I would strongly advice you to visit and have a glance of it.

Update: Added a worksheet named “GET STOCK CODE” for finding stock codes against company names.

Reasoning for why bodies fall at same rate

Wed, 14 Nov 2007 16:19:00 +0530

We all are aware that bodies fall at same rate irrespective of their weights. Well, let’s see how our geniuses have arrived at this in the days where there was no definition for velocity or acceleration. While reading Stephen Hawking’s A Briefer History of Time, I stumbled upon the explanation given for bodies falling at same rate. This snippet taken from Google Book Search shows,

(continuing…) forces cancel out and a body will always fall at the same rate irrespective of its weight.

How a layman would understand this? There comes the giant Galileo, who gave a simpler explanation long back in 16th Century. Here is the excerpt taken from the book “On Giants’ shoulders”. Here is the excerpt,

He (Galileo) imagined a light and a heavy body tied together and asked the question ‘Does the presence of the light body help or hinder the fall of the heavy body?’. Now if, as Aristotle had maintained and, as intuition may tell us, light body would impede the fall of the heavy body, because it would lag behind and restrain it. But Galileo reasoned that if we consider the entire assemblage of heavy plus light body together, well, that is heavier than the heavy body on its own, so the assemblage should fall faster, and so we arrive at the contradiction that the presence of the light body should both speed up and slow down the fall of the heavy body. That is obviously nonsense, and so the only way we can reconcile it is to suppose that both heavy and the light body fall at the same speed.

What a wonderful example of Galileo’s reasoning? The former explanation using Newton’s laws can be commonly found in our text books. But Galileo’s reasoning makes more sense and our education system doesn’t induce such a reasoning power into us.

This video below shows the Hammer-Feather drop experiment performed by the astronaut David Scott on moon during Apollo 15 mission. Since there was no atmosphere, a feather and a hammer fell at the same rate and the entire world witnessed this historic proof for Galileo’s findings.

Crystal Xcelsius

Fri, 10 Aug 2007 10:30:00 +0530

I got an opportunity to learn about a tool called Xcelsius. It’s been a wonderful experience so far. This tool lets you create handy dashboards in less time. The tool is very useful for those who work in Data Ware Housing and would like to create a presentation out of their data. Designing a visualization is very simple and requires no analytical skills. It generates a interactive Flash movie of the presentation which can be easily shared by mail.

One should possess good knowledge on spreadsheets and cell formulae before using this tool. If you want to display dynamic data, there is XML and Web services for you.

Flash knowledge is not required as the tool generates the flash movie and Actionscript based on our design using the underlying data model (spreadsheet). Excel is used only in the design time for defining the data model.

For a demonstration, I created a visualization which compares popularity of the two or more terms. The logic is implemented in a ASP.Net web service, which makes use of the Google SOAP Search API to find the number of result pages for a search query.

The visualization invokes the web service and gets back the XML data containing number of result pages for each of the inputted terms.

Number of result pages is not a perfect measure for popularity. But by phrasing a search query for each of those terms, we can get an approximate picture of their popularity.