Da BOSS Gets Social

Posted by Ofer, Product Architect on Nov 4, 2008

We’ve just announced our partnership with Yahoo! to augment Delver Social Search with results from Yahoo! Search BOSS. The new service is now live on http://www.delver.com/ so go ahead and experience it, and let us know what you think.

Unlike some other products where BOSS serves as the product’s entire search backbone, Delver uses BOSS to complement our core search service, and we’d like to share some of our experience and insights in this post.

1. The concept

Delver is a Social Search engine, and more specifically a socially-connected type of Social Search, with each user seeing results prioritized by that specific user’s social networks and graph. This subjective type of ranking is quite different from the authoritative approach that characterizes all major search engines, including Yahoo!. We see our approach as one that is complemented by authoritative ranking, depending on the query - if our user is searching for advice from friends on how to best spend the time in an upcoming vacation, Delver will have the best answers. But when the user needs information on Napoleon’s battles for a history homework, authoritative ranking may become more useful.

Yahoo! BOSS allowed us to answer all of our users’ needs without developing a traditional search engine from scratch, on top of our core agenda. When we serve a query with results, we retrieve BOSS results and embed them complementary results, positioned according to its relevance to the query.

2. Seamless experience

In any major search engine, a user seeking authoritative results will usually be satisfied with the first 1-2 results. Rather than merging a large portion of results and confusing the user with which result is from where (a-la meta-search), we chose to display 1-2 BOSS results in the standard Delver metaphor, with our “General Web” as the source. If the user would like to see more, the “More results” link leads to a full results page, again in the standard Delver metaphor that groups search results by user (rather than domain). All these customizations, plus keeping in line with the exact Delver look and feel, could only be supported by a flexible search partnership and TOS, such as BOSS.

3. Merging result sets

Even when not operating as a full meta-search, some merge questions arise. Questions such as how to produce a combined ranking formula, or how to filter out duplicate results. With disparate search indices, such questions can become a major performance issue, so consider them carefully.

4. Search services

BOSS offers a lot more than just search results. In particular, some search services can be very difficult for a young search company to create without access to search essentials, such as massive query logs. One example where BOSS comes in handy is the Spelling Suggestion API, which is offered also by other providers, but the BOSS TOS truly allow us to put this concern aside and use the Yahoo! infrastructure. We have implemented spell suggestions into Delver and are looking into other similar services which Yahoo! exposed, or will expose soon.

5. Search infrastructure community

Yahoo’s BOSS Developers Community is growing rapidly, and a lot of it is thanks to the excellent responsiveness and cooperation of Yahoo! BOSS staff. When we encountered some issues with results parsing, the BOSS team responded within a very short time with a fix. Thanks guys!

Gmailizing blogs

Posted by Ofer, Product Architect on Oct 30, 2008

When I first started using gmail, I was shocked: “What? no folders??…” I couldn’t figure out those funny labels, and searching my emails instead seemed a strange idea. Nowadays, when I have to locate an old email, I pray that it’s on gmail and not in my Outlook (even with Vista’s improved search).

The dilemma between search and browse paradigms runs through many software user interfaces, and was especially emphasized with Google’s focus on search in their products. In some areas, such as finding web sites, the search paradigm has undisputably won and the once-king Yahoo! Directory barely has a stub article in Wikipedia. In others, such as news, search is a rarely used service, and a portal-like browse interface rules.

But in reality these are complementary paradigms, rather than competing. Browsing is excellent when the data fits a clear and sufficiently granular taxonomy, shared by the author and reader, and unstructured searching fits into all the other cases (and in some cases, like web search, that’s all there is). Oh, and one more difference: search is A LOT easier. Just stuff all the text into strong index machines, and give the user the ubiquitous search box.

With gmail I wouldn’t think twice before moving an email to the archive, I have no doubt I’ll find it when needed, and all the hassle of managing folders is gone. Blogs should be no different. You have an author communicating a heap of knowledge to readers, and instead of sorting it for future reference in tags and categories (the complete opposite of “…a clear and sufficiently granular taxonomy…“) they should be gmailized - stuff them in an index and search.

Ah, you say, just embed a blog search box. Sure, but I have dozens of blogs I want to search in. So use some blogs search aggregator, you suggest. But I don’t want to get results from all the blogs out there, just from those I care about. Well, then, guess you’ll need to build yourself a custom search… or just use Delver. Knowing that in a few years every major search engine will integrate social features, I can carelessly blog about anything my social circle could find useful, without bothering about categorizing with the perfect keywords (hint: there aren’t any) - social search will find them!

“Identifying Your User in 30 Seconds” or “A Balancing Act”

Posted by Pasha, Application Development Team Lead on Aug 4, 2008

One of the design challenges we had in Delver was one of delicate balancing. On the one hand, our service is a search engine that delivers results based on exact identity of the user. On the other hand, we wanted to give users the full experience without having to register to the service. The reason for this is mainly because users expect their internet search engine to “just work”, without registration (even though today, most people are logged in to Yahoo or Google anyway).

As a startup, registered users are very important to us (registered users are the currency of the web 2.0 economy), but we were willing to give up maximizing that particular metric, in order to provide the smoothest user experience.

The solution we found was a “temporary user” entity. This solution allows the user that arrives at www.delver.com to “tell us who they are” by finding themselves in the Delver people database that we have created by crawling the internet.

The downside of this solution is that users created in this manner do not have a username or password. This means that when their web cookie is gone or they switch computers - we no longer know who they are and the process has to be re-done. To overcome this, we show a suggestion tip to the user, recommending that they register. By this time the user is familiar with our service and can better decide whether registering is something they’re interested in. Also, the registration at this point is quicker as we have already collected most of the data we need.

So, what’s the best way to let the user identify themselves? The design goal was that the user should be able to start using the search engine in 30 seconds or less from the time they landed on our homepage.

In an early prototype we asked the user to enter any piece of information about themselves:

After playing around with this version, we decided that this was confusing to the user and it was hard for us to show the best matching people, because of the possible ambiguity (for example: “George Washington” can mean a person who’s last name is “Washington” or a George that lives in Washington). We wanted the user to enter as little information as possible in order to start using Delver as quickly as possible.

We decided that it’s most natural for users to type in their name or email, so we changed the behavior to:

Now we had to deal with two situations:
1. Your name brings up too many people
2. We can’t find your name

For the first case, we offer narrowing the results by using other criteria:

Now the user only needs to enter additional information if their name is too common.

For the second case, we added the “social circles” step:

Using the “social circles” feature, we can build your search network not only using connections we’ve found on the internet (social networks, photo sharing sites etc.), but also using information on your workplace, location and so on. For example: two people working at “ACME chemicals” will be connected in Delver.

In short, the currently implemented user identification process answers our goals of:
1. Being quick
2. Not requiring the user to register
3. Identifying the user in a way that allows us to provide custom search results based on the user’s social network

As always, we’d love to hear your feedback on the result.