The recent release of Linux
Mint 12 surprised many by using the little-known DuckDuckGo (DDG) as its default search
engine. Through a confidential agreement, Linux Mint will "share the
revenue generated by the sponsored links" when users click on them
using DDG. That arrangement is a creative way to help fund the distribution.
On the other side, though, readers may wonder why, when Google, Yahoo!, and Bing so thoroughly dominate search engines, is DDG developing another one?
DDG founder Gabriel Weinberg has a ready answer:
"[There are] things we think the bigger search engines don't do well for a
reasons (generally not technical, but legal, business, and cultural). These
areas of focus have been: way more instant answers, way less spam, real
privacy, and a less cluttered user interface.
Add an emphasis on using free software and being a good citizen within
the community, and perhaps DDG has a chance to prosper, even though its 12
million searches per month
are next to nothing compared
to the December 2009 figures for some of the larger players: 88 billion for
Google and 9.4 billion for Yahoo!.
An MIT graduate with a master's degree from the Technology and Policy Program, Weinberg is a small-time angel investor with a strong interest in companies powered by open source technologies. Four years ago, he founded DDG, which has grown slowly to three employees, a number of part-time-contributors, and what he calls the "growing open source wing" of about twenty collaborators.
Anticipating market directions
According to one of his blog entries in which he discusses his various investment strategies, Weinberg's approach with DDG is "to work within a big market and concentrate on where you think it is headed."
Part of what this approach means is that DDG tries to improve the results
returned on a search. Lacking the resources to do all its own web crawling,
on fifty other sites for its results, including Yahoo! and Bing, as well as
more specialized sites. To a limited extent, users can control which sites
are used through the combo box to the right of the search field, choosing,
for example, to use Bing or Google for image searches. Users can also
choose whether to order results by date or alphabetical order. Weinberg
We believe that for any given search, there is usually a vertical search
engine out there (or API or data set) that does a better job at answering
that query than a general search engine. Our
long term goal is to get you information from that best source, ideally in
instant answer form.
From the initial results, DDG filters ad-heavy portal sites and presents results without bubbling — that is, ordering results in light of your previous searches. In fact, DDG claims that, unlike larger search engines, it doesn't collect information about user's searches at all. Instead, it attempts to order results by crowdsourcing, just as YaCy, another new and small search engine does.
In fact, DDG makes some efforts to protect user privacy and to educate users about why they
should care about privacy. Although no details are given, DDG claims to
redirect a search request "in such a way so that it does not send
your search terms to other sites. The other sites will still know that you
visited them, but they will not know what search you entered
beforehand." In addition, DDG uses an HTTPS version of a site when
Weinberg described these features as offering users "real
Previously, there haven't been many real choices when it comes to
protecting privacy online. You could either disengage completely (not a
great option) or decide to give up significant privacy (another not great
option). We've taken it over as part of our mission to both a) help educate
people on issues and b) give people real control over their privacy, and
thus a real choice when it comes to search privacy.
Searches support the syntax users may know from other search engines, such
as the use of quotations mark to search for an exact phrase, or a minus
sign to exclude results that contain a specific word or phrase. Users can
also filter results by toggling "safe search" or "meanings" (provides
choices for ambiguous terms) settings, or using the region setting to filter results for increased relevance.
By default, search results are topped with a red box, the so-called
"zero click" or "instant answer" feature which
tries to place the most important result first. When searching for a
concept, the red box result might be a link to the basic definition; for a
person, to their blog or web page. To the right of the red box, a list of
suggestions for refining the search appears. Should the query have more
than one meaning, suggestions similar to the disambiguation pages on
Wikipedia are given.
Another feature of DDG is !bang
searches: automatic searches, somewhat similar to Google's "site:"
searches, which are available for common sites like Amazon or YouTube. For
instance, if you enter "!youtube pogues," DDG shows you the results on
YouTube for The Pogues, saving you several additional clicks. Similarly,
you could specify !openstreetmap at the start of your search to get
results from OpenStreetMap or !monster to search Monster for a job
description. These !bang searches include a wide variety of different
categories, such as major corporations, domains, programming languages,
shopping sites, tech domains, research topics, news, and online services. The only problem is that, if you haven't memorized a supported !bang search, you either have to take a chance that your site of choice is supported, or else look it up on the DDG site.
DDG also includes the ability to do calculations, measurement and currency conversions, and to answer direct questions on weather, food, geography, and time-related topics such as tides or sunrises in a specific location.
Also included on DDG is a small number of apps, as well as popup how-tos about adding DDG to common browsers.
Many of these features, such as the ability to do calculations, are paralleled in major search engines. Others are unique to DDG. However, what stands out is not any particular feature so much as the total number of them. For some reason, DDG lacks a summary of total results, which is often used as a rough indication of a topic's importance (and for ego surfing), but, otherwise, the main drawback is that taking advantage of DDG's features requires a willingness to learn — a willingness that might be lacking in many who are only mildly curious about such a niche service.
Interacting with the open source community
DDG has always relied on free software, as it is written in Perl and
These new pieces include the upcoming community platform, which Weinberg describes as a collection of "tools for communities to use to help participate." So far, the tools include a translation interface and a server for Extensible Messaging and Presence Protocol (XMPP), with "a data store to store settings and URL shortener" expected to follow. DDG is also encouraging contributions to expand and improve DDG's zero-click info repos, the source of the material in the red boxes at the top of search results.
"As these contributions increase, the percentage of DDG that is open
source is going up. I'm not sure about becoming completely open source for
a variety of reasons (spam paramount among them), but we are certainly
thinking about [the possibility]," Weinberg said.
Even if DDG does not become completely free, Weinberg emphasized his determination to use free software "as much as possible." In particular, referring to DDG's multiple sources for results, he suggests that the use of free software is directly related to the availability of expert results: "If you know an esoteric piece of the query space really well, you should be able to develop for it." In other words, using free software not only produces more specialized sources, but also indirectly increases the accuracy and usefulness of DDG's results.
Similarly, asked to comment on Katherine Noyes's suggestion that taking results from Microsoft's Bing might deliver results with an anti-open source bias to Linux Mint users, Weinberg pointed out that Bing is only one of over fifty sources.
"I haven't seen any compelling evidence that we're biased against open source," Weinberg said. "And in fact, we're working on ways to essentially do the opposite." For example, DDG already uses Ask Ubuntu as a source for technical results, and is currently working on tighter integration with alternativeTo in order to increase the accuracy of free-software related queries.
In return for bootstrapping off free software, Weinberg said, he would "very much like to help start a movement where companies that use open source give back in systematic ways to those communities." As a preliminary effort, he has established Foss tithe, a site on which corporate owners can pledge to donate a percentage of their net income to the community. The suggested tithe is ten percent.
So far, only one other company (search [co.de]) has pledged to tithe, and Weinberg himself has not done much to develop the idea. However, he has made his own tithe, with half the donation decided upon by him and half by the DDG community. In 2010, for the corporate portion, he chose to give $482 to nginx and $475 to FreeBSD, two projects that he described as "an integral part of our architecture." Choosing security and privacy as a donation theme, the DDG community chose to donate $238 to each of Tor, Clamwin, Tahoe-LAFS, and OpenSSH.
Whether DDG will ever be a major contender among search engines is
doubtful. A buyout by a larger competitor is an obvious possibility,
though it is unclear whether DDG's privacy policies and options would
survive such an event. However, by seeking out closer ties with free and
open source software, DuckDuckGo might just find itself the search engine
of choice among a small, dedicated group of users with enough knowledge to
appreciate its philosophy and features. That could be a path to success
and financial sustainability for a smaller search engine like DDG.
to post comments)