Phil Leggetter - Real-Time Web Software and Technology Evangelist » web 2.0

Wikipedia now has a definition for Real-Time Web

Phil Leggetter — Thu, 09 Jul 2009 14:43:23 +0000

A while ago a wrote a blog article called What is the real-time Web?. At this time (19/04/2009) even Wikipedia did not have a definition for what “real-time web” was. It does now.

The definition is short and sweet:

Real-time web is the concept of searching for and finding information online as it is produced. Advancements in web search technology coupled with growing use of social media enable online activities to be queried as they occur.

A traditional web search crawls and indexes web pages periodically, returning results based on relevance to the search query. The real time web delivers the most popular topics recently discussed or posted by users. The content is often “soft” in that it is based on the social web – people’s opinions, attitudes, thoughts and interests – as opposed to hard news or facts.

At present there is no mention of the technology behind the real-time web. Nothing about Twitter and FriendFeed helping to form the definition.

Do you agree with this definition? What web 2.0 platforms do you think have helped define what “real-time web” is? What technologies do you believe have pushed the “real-time web” into such prominence?

Microsoft Bing.com round table thoughts

Phil Leggetter — Wed, 01 Jul 2009 19:07:02 +0000

A couple of days ago I blogged that I was going to attend a Microsoft bing round table. I’ve now attended and here are the key points and interesting facts that I took from the event. You can also search for #meetbing on twitter for relevant tweets.

Bing.com is a re-brand of Live Search

And Microsoft openly admitted this. Basically Live Search wasn’t getting any traction so Microsoft had to do something (see my next point).

“Microsoft believe they will provide as relevant search results as Google in the UK in a number of months #meetbing” @leggetter

What they’ve done is go with a “web 2.0 name” of “Bing” to try and jazz things up a little. They’ve also gone back to basics and want to concentrate on the quality and relevance of search results and believe they will provide as relevant results as Google in a few months and are judging the quality of their results on something called NDCG. I certainly didn’t use Live Search because I found the results pretty useless. Bing definitely returns better results than Live did. An example on this focus on quality is that Bing has only presently indexed around 10 Billion pages which in the scheme of things isn’t very much.

“Only 1 in 4 searches deliver a successful result. Data from MSN search, tool bar in IE. Could be searching Google, yahoo etc.” @karenblakeman

Microsoft also believe they’ve improved the UX a bit to allow users to refine their search results with the introduction of the quick tabs/suggested links in a menu on the left hand side of the user interface. They’ve done this because they believe that only 1 in 4 searches deliver a successful result so this allows users to more easily find things faster.

Microsoft seem to think the re-branding and basic search tweaks have worked as they showed us data that they believe proves Bing has overtaken Yahoo! as No.2 in the search market. Their source of data to determine where users were going to perform their web search was quite interesting. Their source was the MSN toolbar which is only available to those with Internet Explorer. I guess since IE still has a large share of the market (W3Schools Stats | Wikipedia Stats) that the figures are more or less correct but some people at the event did question this as a suitable source.

The other thing that Microsoft are doing is to integrate Bing as a “search brand” into all their products in the same way that Google integrate Google search into Gmail, Google Docs etc. So, you will now see “Bing” used to power Hotmail search and most probably all other Microsoft sites or web applications over time.

Microsoft tried to buy Yahoo! to become No.2 in search

“Microsoft tried to buy Yahoo to become No.2 in search. Fact admitted here” @leggetter

This may not be a big surprise to some people but I didn’t know that this was Microsoft’s main motivation to buy Yahoo! and I did read a few articles about the attempted purchase. Microsoft admitted that Yahoo! had the No.2 slot and if they bought them they would gain this position in addition to a well-known brand. They also admitted that they may have kept the Yahoo! brand for their search and ditched Live altogether.

Bing UK is in Beta. Bing US is not

Bing UK is lacking a lot of the functionality of Bing US since it’s still in beta. Microsoft wanted to launch Bing and felt it was better to do it world wide so stuck with the beta. Bing UK will move out of Beta when more of the functionality you can see in the US is available in the UK.

Only the standard verticals are being targeted

Local, travel, maps, videos, shopping and news will be the verticals that Microsoft initially target. This is a little disappointing as it would be great to see Microsoft break the mould and do something innovative. The only unique vertical that was mentioned is that a music search function is being worked on by the Bing China team.

All in all I thought it was a worthwhile exercise by Microsoft to tell people what they are doing and gather a number of opinions. The consensus of the people at the event is that Bing is going to have to something different to take a significant portion of people away from Google and their current offering doesn’t offer too much to do this.

I’m simply happy that Microsoft have upped their game and will at least have Google looking over their shoulders albiet way back into the distance. Hopefully this competition will provoke innovation in search that has been generally stagnent for the past few years up until the introduction of the real-time web. Maybe this could be something that Microsoft could excel at?

Click here to view all the #meetbing tweets

Bing.com round table

Phil Leggetter — Sat, 27 Jun 2009 09:56:53 +0000

On Monday between 19:00 and 21:00 I’m going to a Bing roundtable (round table) in London. The event is being organise by Colin Mercer and is described as:

a small round-table event to discuss Bing and since we saw your recent tweet regarding comparative search results between Bing and Google, we thought that you might be interested in coming along. The idea would be to get your initial thoughts and you’ll get an update on the future steps for Bing which are being developed specifically for the UK. During the session you’ll have the chance to talk with Microsoft representatives and other people from social media like yourself, who have expressed some early opinion.

It would appear the event is being attended by a number of web 2.0, social media, marketing and internet types.

Phil Bradley: Phil is interested in Internet searching, Web 2.0 resources, search engines and their development. Phil has confirmed. I recommend following Phil Bradley on Twitter.
James Barlow: James Barlow is an independent consultant specialising in information technology and project/programme management.
Tom Warren: I don’t have any additional information on Tom.
Chris Hambly: Chris lists himself as a successful Digital Media Entrepreneur
Josh Russell: Josh has tweeted that he’s going. He provides micro-consulting on web application strategy
David Stuart: David is going. David blog provides thoughts of a Web 2.0 research fellow on all things in the technological sphere that capture his interest.
Karen Blakeman: Karen’s blogs about search tools, Internet technologies and resources for business information. Karen has confirmed.
Neil Robinson: Neil describes himself as passionate about using the right technology to drive business forward.
Kevin Harrington: Looks like Kevin had some initial trouble confirming his attendance but I think he’s coming now.
Mark Sharp: Mark designs and runs a number of websites for myself and for other people and organisations
Andrea Giammarchi: Andrea’s profile says JavaScript Ninja and both PHP/ECMAScript Certified Engineer

We are being encouraged to “live blog” the event.

Please come armed with all of your thoughts, opinions and questions for the Bing team. This meet-up is about everyone getting the chance to have their say. We’ll make sure you’re able to live blog during the event

So, I’ll be armed with my laptop (for tweeting) and HTC Touch Pro (for video, if appropriate) and will try and ask as many questions as possible and get as much information out to the real-time web as I can. My personal angle will be one of technology. For example:

How Bing are pushing use of the Bing API and Bing developer centre.
What are they doing to ensure information is available to the real-time web as quickly as possible.

Please let me know if you have any questions that you’d like asked and I’ll do my best to ask them. You can do this in a number of ways. During the event the hashtag in use will be #meetbing. Prior to the event I’d suggest people maybe use #askbing. Failing that you can contact me directly or leave a comment on this blog post.

Caplin Systems Blog Post – What is the real-time web?

Phil Leggetter — Tue, 28 Apr 2009 15:27:40 +0000

About a week ago I wrote a blog article at work entitled “What is the real-time web?“. I’m seeing even more tweets and posts about the real-time web and I’d love to hear if other people agree with my opinion on what the real-time web is.

View the What is the real-time web? article here.

What is the “real-time web”?

Phil Leggetter — Mon, 20 Apr 2009 11:00:54 +0000

Are we really seeing the real-time web?

The phrase “real-time web” has been streaming its way around the Internet for a while now. It’s presently being used to describe information being available in search results as soon as it has been published by its author. Examples of this are Twitter or FriendFeed search.

As far as I’m aware it was Robert Scoble who made this phrase mainstream. Here’s an example from the start of February, but by this time Robert had been using the term for a while and a quick Google finds the term used as early as June 2008:

Let’s do a search for anyone who has written about the Canon 5D MK II but lets constrain that to posts that have at least one like and at least four comments. Here’s the search. Note that the post I wrote just one minute ago is already in the results page. This is the real-time web. – Robert Scoble (http://scobleizer.com/2009/02/09/is-the-real-time-web-a-threat-to-google-search/)

Whilst this is an example of something being almost instantly available from search does it really qualify as being “real-time”?

The first thing to do, as others have done, is understand what the term “real-time” means. Rather than going into this I recommend you have a read of the wikipedia definition of real-time and watch a Kaazing presentation on What is the Real-Time Web. There have been fewer attempts at defining “real-time web”. Even Wikipedia presently (19/04/2009) lacks a definition.

For me for something to be real-time web it should meet the following criteria:

Get up to the second information on a chosen subject.
Make an initial request for information to be delivered across the Internet. Subsequent updates to that information, or information stream, should be delivered as soon as possible without the need to make a additional requests.
Initial information and subsequent updates should be delivered in an acceptable amount of time.

The real-time web experience

If I go to my twitter homepage I don’t want to have to refresh the page, by clicking the refresh button, or hitting the F5 key, in order to see new messages from the people that I’m following. Twitter doesn’t really want users doing this either as it puts load on their server – but let’s leave this for another time.

Twitter

This lack of real-time updates on the Twitter site has lead to a surge in Twitter client applications such as TweetDeck and Thwirl. These clients simulate real-time by polling the Twitter API at intervals. I’d like to highlight the word “simulate” again and clarify what polling is. What the Twitter client applications are actually doing is making requests to Twitter at periodic intervals to see if the user has had any messages and if they have it displays them, if there are no new messages then it’s a wasted request.

TweetDeck

Twhirl

A much better model would be for Twitter to notify the client that new messages are available by passing the additional messages straight to the Twitter client.

Real-time web delivery time

What determines the amount of time that is acceptable between a message being published and it being received by the consumer (user) depends on the context of its use. For something such as instant messaging or a Twitter conversation the time between publication and consumption can be reasonably large, maybe 10 seconds. However, if the publisher is a trading system sending a price and the consumer is a trader 10 seconds is far too long. By the time the price reaches the trader it may be out of date.

Real-time technology

XMPP and SUP have been touted as the technology to deliver the real-time web but they appear to be server side solutions or require clients with knowledge of the specific technologies. For a web solution people start to turn to Comet server solutions. Caplin Systems – the real-time web company – have had the technology available to deliver real-time data for many years now so I wanted to understand why, only now, has the idea of a “real-time web” really taken off.

Caplin Systems

Caplin Systems have a high performance scalable real-time comet server called Liberator that can push updates to subscribing clients as soon as it receives information. The clients are:

Java clients: standalone application or applet
.NET clients such a windows desktop application or web page embedded SilverLight widget
JavaScript clients: any website such as http://twitter.com or http://friendfeed.com

Twitter and FriendFeed don’t use Liberator but they could easily do so. All that needs to be done to integrate Real-time Rich Internet Application (RTRIA ?) functionality to an existing web site is include a JavaScript library called SL4B (StreamLink for Browsers). The RTRIA functionality can then be added to an existing site in the same way that RIA functionality can be introduced to a website using other JavaScript libraries such as JQuery. However, in this case the RIA functionality that can be added is a true real-time web experience of data being pushed to the client as soon as it becomes available, without any need for the user to check to see if any new information is available.

Conclusion

My opinion is that the instant availability of information via search straight after it’s published doesn’t really justify the term “real-time web”. To be truely real-time the information needs to be pushed to the user. The true real-time web experience and the technology to achieve this is already here but at the moment few people are aware of it and therefore it’s not in mainstream use. The ideas behind Comet, the Liberator server and Liberator clients provide technology and infrastructure to push data to clients without the need for polling or a user driven information request. Scalability and side by side integration with existing systems are already possible which means there’s no reason why existing clients such as websites can’t start taking advantage of it.

Some relevant links: