Come meet us at the TechCrunch50

Posted by Noa, Marketing Manager at Delver on Aug 25, 2008

We’d like to invite you to visit our booth at the Techcrunch 50 conference in San Francisco, September 8 – 10, and learn more about Delver.

We will be introducing our partner program and demonstrating how Delver partners get more page views. Delver has a unique search solution for your social networks, widgets and social applications, one that will make your users have a great search experience and come back for more!

Also, don’t forget to ask for our T-shirt! (Delver is not only changing Search, we also do it in style)

Delver Launch Party!

Posted by Liad Agmon on Aug 24, 2008

“Identifying Your User in 30 Seconds” or “A Balancing Act”

Posted by Pasha, Application Development Team Lead on Aug 4, 2008

One of the design challenges we had in Delver was one of delicate balancing. On the one hand, our service is a search engine that delivers results based on exact identity of the user. On the other hand, we wanted to give users the full experience without having to register to the service. The reason for this is mainly because users expect their internet search engine to “just work”, without registration (even though today, most people are logged in to Yahoo or Google anyway).

As a startup, registered users are very important to us (registered users are the currency of the web 2.0 economy), but we were willing to give up maximizing that particular metric, in order to provide the smoothest user experience.

The solution we found was a “temporary user” entity. This solution allows the user that arrives at www.delver.com to “tell us who they are” by finding themselves in the Delver people database that we have created by crawling the internet.

The downside of this solution is that users created in this manner do not have a username or password. This means that when their web cookie is gone or they switch computers – we no longer know who they are and the process has to be re-done. To overcome this, we show a suggestion tip to the user, recommending that they register. By this time the user is familiar with our service and can better decide whether registering is something they’re interested in. Also, the registration at this point is quicker as we have already collected most of the data we need.

So, what’s the best way to let the user identify themselves? The design goal was that the user should be able to start using the search engine in 30 seconds or less from the time they landed on our homepage.

In an early prototype we asked the user to enter any piece of information about themselves:

After playing around with this version, we decided that this was confusing to the user and it was hard for us to show the best matching people, because of the possible ambiguity (for example: “George Washington” can mean a person who’s last name is “Washington” or a George that lives in Washington). We wanted the user to enter as little information as possible in order to start using Delver as quickly as possible.

We decided that it’s most natural for users to type in their name or email, so we changed the behavior to:

Now we had to deal with two situations:
1.    Your name brings up too many people
2.    We can’t find your name

For the first case, we offer narrowing the results by using other criteria:

Now the user only needs to enter additional information if their name is too common.

For the second case, we added the “social circles” step:

Using the “social circles” feature, we can build your search network not only using connections we’ve found on the internet (social networks, photo sharing sites etc.), but also using information on your workplace, location and so on. For example: two people working at “ACME chemicals” will be connected in Delver.

In short, the currently implemented user identification process answers our goals of:
1.    Being quick
2.    Not requiring the user to register
3.    Identifying the user in a way that allows us to provide custom search results based on the user’s social network

As always, we’d love to hear your feedback on the result.

A Taxonomy of Social Search Approaches

Posted by Ofer, Product Architect on Jul 31, 2008

Delver has recently launched an alpha of its Social Search service. I mean, social-powered search. Um, make that Socially Connected Search. Wait – perhaps Social Graph Search? Or even social media search?…

Why is the terminology so difficult? The almighty Wikipedia (heck, even WP itself was branded as type of social search back in 2006) defines Social search as

“a type of web search method that determines the relevance of search results by considering the interactions or contributions of users… Social search takes many forms, ranging from simple shared bookmarks or tagging of content with descriptive labels to more sophisticated approaches that combine human intelligence with computer algorithms.”

Aha, so that means stuff like del.icio.us and Flickr and Mahalo and Wikia search are really all the same type of service? Hmm.

Yes, a lot of services are titled “social search” these days, and a lot of them indeed are such. It’s just that search itself is a pretty complex process, and “…considering the interactions or contributions of users” can take many forms. Let’s try and put some order into social search approaches, and while at it, we’ll also pinpoint what it is about Delver’s approach that we think will shoot it to infinity and beyond.

So - web search is about: crawling, indexing, ranking, querying. Now, let’s see how we can put those humans into the loop:

• Crawling: products such as Mahalo employ humans to discover and add new content into its index. This approach goes back to the days of Web directories (Yahoo!, dmoz). Others, such as Lijit or Eurekster use humans to define relevant subsets on a machine-crawled index.
• Indexing: a search index maps keywords to documents, so when humans tag content items they do exactly that, describing the document by tags, as in del.icio.us or Flickr.
• Querying: most search engines analyze user query logs to suggest “query reformulations” (or even employ them automatically behind the scenes). Products such as ChaCha go even further to have a human analyze each and every query. I even met a person who positioned himself a “human search interface”, selling a service of building queries for difficult needs…
• Ranking: well, that’s the Holy Grail. Let’s break that one up further now.

Attributes of Social Ranking

We all know that not only pigeons can rank documents in web search, humans are fully capable of doing that too. And the approaches vary from describing Google’s link-based PageRank as kind of social search, to manual building of search indices per query. Sources for the human input range from direct explicit manipulation as in Wikia Search, through interpreting users’ indirect actions such as on Digg, and on to implicit inference from behavior patterns of large numbers of users, inspired by recommender systems.

To properly compare social search approaches, let us define two major attributes that best differentiate and cluster social ranking approaches, and through this prism we’ll look at existing products: Personalized vs. Aggregated, and Structure-based vs. Behavior-based.

Personalized methods tailor results to each individual user’s social footprint, whereas Aggregated methods have all of the users’ footprints contribute to a central ranking value. Structure-based approaches take social context from explicit social graph structure, as opposed to behavior-based using implicit social hints, such as like-minded clicks and votes. The chart below shows how some social search players fit into this taxonomy (remember – the attributes refer to the social aspect of ranking only):

The “Link Analysis” and “Recommender” quadrants represent approaches (employed mainly by the large search players) that have prevailed in the previous decade. Link Analysis, taking inspiration from scientific citation, used human social input in the form of hyperlinks to determine the “global” importance of a given document. Query log analysis, borrowing ideas from recommender systems, was employed by search engines to improve ranking, recent example being Microsoft’s “BrowseRank” paper. Labeling itself as “community-powered search”, Collarity attempted to use collaborative filtering for direct ranking of web search results, but recently the company abandoned this direction to focus on advertising. Several other companies provide recommendation engines, with the recent example of Baynote powering search at Expedia, but all these offer site-search functionality only, and do not even attempt to scale the solution to the entire web.

The two other quadrants form the new directions social search has been heading in recent years. Most of the activity we’ve seen so far has been in the quadrant we call “Crowdsourced ranking”, where users are asked to rank search results, directly or indirectly, and the input is used to produce a global ranking scheme, with all the scalability questions.

At Delver, we believe that information encoded in a well-structured social graph is one of the most valuable resources, that are yet untapped in the quest for better relevance. So far, the most similar value has been in link analysis, as demonstrated by the strength of Google’s PageRank, but PageRank is an aggregated approach built on links between websites. To make results relevant for a specific searcher, and help that user clearly understand exactly why each result is relevant, ranking must take into account the graph describing that user, and this is exactly what we’re doing. Stay tuned to find out more on the challenges in doing that, and how we tackle them!

Launching Delver

Posted by Pasha, Application Development Team Lead on Jul 20, 2008

It’s been two days since we’ve publically launched Delver, but with all the excitement around it feels like it’s been a year.So, in my first post here I’d like to share my perspective of what it takes to launch a product and how important it is (hint: it’s *important*).In fact, there’s nothing more important in a software project than actually getting the first version out of the door. This may sound trivial, but it’s anything but.Interestingly enough - most software projects never actually make it to this stage. If you think there are too many new services announced every day on Tech Crunch – you’re right. But this is just the tip of the iceberg – there are ten times more projects that started but never delivered their first release to the public.

The motivations to get the service out of the door are huge. Some are obvious and some are not.

First reason to release early is that you need to validate your idea. One of the hardest things about developing a new product (“New” as in different and innovative. Not a new toothpaste, which is “just the same, but we need to keep the cash flowing in”) is not knowing whether it’s actually going to be useful to people. Before launching you’re just basically throwing stones at a small target on an imaginary wall in total darkness. You don’t know what you’re doing. The best products (think the PC, the iPod, sliced bread) make perfect sense to us now, but that’s just hindsight. People who developed these products had really strong belief in what they were doing, but that belief was anything but based on facts.

What features should you develop and which not? What should the UI behave like? It’s not just that you don’t know the answers, there’s no way to even argue about them.

You need to get real users to use the actual product. A lot.

The second thing is being ahead of the competition.

Once you have an idea for a service, realize that there are only two options: a) your idea sucks or b)there is already more than one group of people somewhere out there working on the same idea.

Hopefully your idea is of the second type. Now you’re in trouble. An excellent service will fail, just because one or two other services were delivered faster. Interestingly, the other services don’t have to be better and they don’t even have to be the same as yours. It’s enough that they appear similar. First impressions count a lot. It’s ten times more difficult to succeed with a new product in a field with existing competitors. The iPhone comes to mind, but that is, as my father says, “the exception to the rule, that proves the rule”.

This is especially true of internet services. Ideas in this field are moving around so fast that things change beyond recognition within a few months. Many times an internet startup company will start with one idea and will end up changing it drastically because the outside world changed before you delivered. Paypal started out building a service for small payments between Palm Pilot devices. A similar change happened to our product here at Delver – the service we just launched is not what we envisioned when starting the company just a year ago.

Last but not least are your own motivation and focus.

Having a release milestone is probably the best thing you can do to get your team aligned and focused on what’s important and on getting things done. Without everybody having a clear idea about the release target – you will suffer from feature creep, people will be pulling in different directions and not necessarily working on the most important things. Once everybody is aligned on the goal to release the service early – motivation will go up.

It’s the same as when you’re riding your bike up a really tough climb. When you see the summit – breathing becomes effortless.

And once you have the service out of the door and actual people are using it – your behavior changes. The programmers think more about committing a change in code. Bugs are fixed faster because you know that actual users feel the pain.

More than anything – you can listen to you real users instead to voices inside your head. What features they like? What is useful and what is useless?

But just like “riding a llama” or “sex without becoming attached” – it’s so much easier said than done.

There are many things you need to fight off to actually make it to the launch of your first version.

Feature creep: people who are developing new products are very good at coming up with ideas for stuff to put in the product. And if you’re lucky, you’ve probably hired the smartest and most creative types. They’re the worse. They’ll shout: “We must support all the search operators Google supports. And we simply must enable drag and drop for users to save interesting items they found in Delver.”

You need to take a long and hard look at your feature list. And then you need to throw out 80%. Seriously.

This poses a hard dilemma: in order to release the product you will need to make painful sacrifices, especially on the usability and stability front. You’ll need to be ready to deal with some of your users getting reduced service and sometimes no service. Someone will write in their blog that you’re half baked. Be ready for it. I believe strongly that you’re almost always better off releasing early than keep polishing the product. You’ll end up releasing the most robust and polished service but have no users. Because all your users now live in glass capsules filled with jello  on a spacecraft driven by robots. Or something.

One of the things we did in Delver to deal with potential overload on our servers in the first days after launch is count the number of active users and if that amount exceeds a predefined value – navigate them to a “we’re too busy” page. We’re going to have to deal with some disappointed users, but at least we can provide service to some users earlier.

On a related note – you need to actually have a very well defined feature list for your version one. Trivial? Many software projects don’t really have one.

Another issue is the “hidden tasks”. These are the tasks you’ll have to do in order to launch but you forget about them when you make your plans. You still end up doing them because they’re so obvious once you’re close to the release but by then they’ll take ten times more work.

Typical examples of this are:

• Production servers setup and configuration
• Deployment procedures and scripts
• Mail delivery mechanisms

Another challenge in a startup company is that you’re building the service *and* the company at the same time. And building a company takes a lot of energy. Think hiring people, buying equipment, handling legal issues. Just getting the right coffee machine can take multiple attempts, many man-weeks and some programmers with a grudge and a bad stomach. It’s the coffee.

So how did we fare with our first release?

First of all – we made it. We’re out and that’s a big thing.

It took us exactly one year from the creation of the company to the launch. I would say that nowadays, an optimal time for an internet service to launch is between 6 months and a year, with some exceptions depending on the scale and complexity of what you’re doing.

We had to deal with all the challenges I mentioned and now we have to work twice as hard. But it definitely feels great to have something to show for the effort.

Delver is now open to all

Posted by Noa, Marketing Manager at Delver on Jul 15, 2008

We’re happy to share that after over a year of development and hard work, Delver is finally up and open to the public (in an alpha version).

Many exciting features and additions are on their way so be patient, it will be worth your wait!

 

I have 449 facebook friends and all I got is a lousy vampire

Posted by Liad Agmon on Jul 13, 2008

When I started using the Internet in 1989 (pretty shameful but I was only 13 years old), my social graph was very simple – I had about 10 online friends with whom I shared insightful information on the Holy Grail of the geek Eighties – blue boxing.

Alas, those days are long over now, together with the awful haircuts. My recent count of real-world connections mapped online showed 449 Facebook friends, 305 Linkedin connections and 500 Outlook contacts. Omitting some overlap, I have about 1000 direct connections: friends, colleagues and acquaintances. Now, assuming that each of my connections has about a 100 connections of his own, my implicit network (friends-of-friends) is of a significant size: about 100,000 members.

I have such an amazing network, spread over the entire globe, and all I end up getting is this?

Something is awfully wrong with the way we benefit from our social network today. Our friends and friends-of-friends are continuously creating online content in dozens of social platforms: Myspace, Flickr, Youtube, blogs, etc. However, the high quality and valuable stuff is getting lost in the web 2.0 noise.

Desktop of Techcrunch writer Erick Schonfeld shortly after installing a Twitter client (Web 3.0 Will Be About Reducing the Noise)

That said, even if miraculously you do manage to follow your online connections, what is the true value of most of the content they create? Do you really have the time and attention span to care about what I am doing right now?

On the other hand, when you plan your next spring break in Mexico, I’m sure you’d love to find out that a friend of yours just returned from two months of beach-bumming. Not only he can give you excellent advice – he might also connect you with the right people that will take you to the coolest bars and parties.

This brings us back to the social discovery challenge: How do I filter out the noise and find information that is relevant to me? In order for the wealth of user-generated-content to become valuable, there needs to be a paradigm shift. In the late ‘90s, Google’s search engine replaced the inefficient navigation of Yahoo-like indexes; in the ‘social web’ era, we need to move away from the cluttered ‘social activity feed’ concept to Social Search - the ability to efficiently find information that was created or referenced by our social circles.

Social search is very different from web search, since it allows me to find subjective information created or qualified by my network. When I social-search “Sex and the City”, I want to find the reviews of my friends, not the official website of the movie. Result quality is measured by its relevance to me; hence, one of the key ranking factors is the relation of the content author to me.

So, should you watch “Sex and the city”? Google directs us to IMDB, which displays an average user ranking of 5.3/10: 6000 people gave it a rank of 10/10 (not counting my girlfriend, my sister and all their shopping-loving friends) and 6000 people gave ranked it 1/10 (wild speculation: these are the poor boyfriends of the previous group).

The contradicting opinions are pretty confusing, yet a one-line Twitter review from a trusted friend solved the conflict for me:

That’s the great promise of social search: quickly finding relevant information created or referenced by your network. This is Delver’s vision and we are here to make it happen.