Though it’s really hard to maintain “real-time” search, Twitter specifies it as real-time itself, despite using humans to power it. So, for anyone’s expectation, real time search has to scroll out the data as quick as possible. Definitely, Twitter would be using enhanced algorithm to keep up this real-time search– to bring up results quickly, rank them, and give out that mass of data one is looking out for. With this in mind, the social network’s updated search engine will now use crowdsourced human feedback (Mechanical Turk) to boom its results.
However, data scientist Hilary Mason described the efforts toiled behind this new real-time search engine. It uses click rates to track spontaneous activity, and then it reassembles all of these links into coherent stories, while still tracking location and other metadata. Twitter is not relying on primary computational solutions but rely on CAPTCHA. Twitter real-time engine identifies spike in search by a real-time computation system called storm, which then is taken over by Mechanical Turk to pass out annotating that particular information to users all over the world. These annotations basically take these search spikes and tags it for most relevance and intent. A human annotator, called as “judge” by Twitter can specify the Twitter Search to get more accurate result, Example: for “India” refer to a country or to its Hockey team, this can actually help Twitter make its search more accurate.
Why new search engine for Twitter? The older “search” couldn’t respond fast enough. When Obama’s re-election photo was first tweeted, or when the breaking story about Gangnam style song first hit, its algorithm wasn’t able to recognize that trending hashtags relevant to these incidents in their initial occurrence on the site. Another example described by Twitter data scientist Edwin Chen and Senior Software Engineer Alpa Jain, the moment that Mitt Romney infamously said “binders full of women” the Internet-connected world flocked to Twitter to find out more about the phrase. Searching for anything relevant to “binders full of women” on Twitter before Romney’s remark would have displayed results about “binders” and “women.” But immediately after Romney’s flub, the meaning of “binders full of women” had instantly changed. Unfortunately Twitter’s engine wasn’t able to recognize this change in context quickly enough when it should originally happen. Now that the search is said to be more advanced if anyone search for “binders full of women” or any variation of that, you’ll see tweets about politics, Obama, Romney, and of course the phrase itself.
Well the human powered topic identification search system will help in better targeting of ads. Though the search system using human beings to identify trends might sound like a old wood which has been polished, Twitter having high end engineers in its pocket would have surely made its way finding something really unique. Will Twitter real-time search engine proves itself as the most fastest search in terms of breaking news and trends ?? Only time can tell us. Also, the search engine will hoard cash as Search-driven ads are smart way to make money on the web. But it has to definitely be patient and do not rush to make money, also it should not overload ads by which frustrates users.