Please Do Not Read The Text Below.

Download the original file at

We copied the scrambled text from pdf file (that's how it gets copied, sorry we had no control) so the keywords would be searchable.

SEARCHING FOR DISCLOSURE
How Search Engines Alert Consumers to the Presence of Advertising in Search Results
By Jørgen J. Wouters
November 8, 2004
Consumer Reports WebWatch
101 Truman Avenue
Yonkers, NY 10703-1057
Tel: 914.378.2600
www.ConsumerWebWatch.org
Part 1 of 3
PART 1
Executive Summary .................................................................... 4
About Consumer Reports WebWatch .......................................... 9
A Brief History of Paid Placement & Inclusion ............................10
Introduction ........................................................................ 12
Examining Paid Placement .................................................... 14
Examining Paid Inclusion ...................................................... 15
Renewed Debate ................................................................ 16
Study Methodology .............................................................. 17
How These Sites Were Selected ............................................ 17
Designing the Questionnaire ................................................ 18
Testing Procedures................................................................ 19
Evaluating the Results .......................................................... 19
Major Findings ........................................................................ 20
Results by Search Engine ........................................................ 23
1st Blaze ............................................................................ 25
Alta Vista ............................................................................ 25
AOL Search ........................................................................ 28
Ask Jeeves .......................................................................... 30
PART 2
Results by Search Engine (Continued)........................................ 34
CNET’s Search.com ............................................................ 34
Google .............................................................................. 37
InfoSpace Web Search ........................................................ 37
Lycos Network Search .......................................................... 39
MSN Search ...................................................................... 43
My Search .......................................................................... 45
TABLE OF CONTENTS
CONSUMER REPORTS WEBWATCH 3
PART 3
Results by Search Engine (Continued)........................................ 47
My Way Search .................................................................. 47
Netscape Search ................................................................ 47
Overture ............................................................................ 49
Web Search........................................................................ 51
Yahoo! Search .................................................................... 54
Recommendations ................................................................ 57
Consumer Tips.......................................................................... 59
Appendices .............................................................................. 61
Appendix A: Testing Background .......................................... 61
Appendix B: Tester Profiles.................................................... 62
Appendix C: Disclosure Questionaire .................................... 63
Appendix D: Search Engines’ Disclosure Statements ................ 66
Appendix E: Search Engine Interrelationships.......................... 73
Appendix F: Guidelines for Search Engines ............................ 75
Reference ................................................................................ 77
Endnotes .................................................................................. 79
TABLE OF CONTENTS
CONTINUED
CONSUMER REPORTS WEBWATCH 4
When Consumer Reports WebWatch launched its site in
April 2002, it published a comprehensive survey of
1,500 U.S. adult Internet users who had six months experience
or more online.i The purpose of the survey, which
used traditional social science methods, was to find out
how much (or how little) experienced Web users trusted
the content of sites they visited. One of the survey’s most
surprising results was that more than 60 percent of its
respondents said they were unaware that search engines
accept fees to list some sites more prominently than
others in search results, a practice commonly known
as paid placement.
Intrigued, Consumer Reports WebWatch then commissioned
a study using a less common ethnographic
research methodology to try to understand why the
practice of paid placement was not transparent to
consumers.ii The study indeed demonstrated among its
respondents a lack of awareness of the influence of
advertising on search results, but perhaps more significantly,
many of them reacted negatively when told details
of how search engines integrate advertising into content.
In this report, its second on the credibility of the Web’s
most significant information guides, WebWatch chose to
examine how search engines explain business relationships
with advertisers to their users. To create an evaluation
tool, WebWatch employed disclosure guidelines
issued by the U.S. Federal Trade Commission (FTC)
in 2002.
EXECUTIVE SUMMARY
Consumer Reports WebWatch examined how
the top 15 most-trafficked search engines
explain their business relationships with
advertisers and discovered:
�� Paid inclusion was not satisfactorily
disclosed or explained by any of the 15
search engines tested.
�� Disclosures by all three meta-search
engines (CNET’s Search.com, InfoSpace,
Web Search) were lacking.
�� Eight of the 15 engines labeled paid
search listings with disclosure headings
that were difficult to spot (e.g., small type,
faint color).
KEY FINDINGS:
CONSUMER REPORTS WEBWATCH 5
The FTC formulated these recommendations for more
transparent disclosure among search engines after
Commercial Alert, a Portland, Oregon-based consumerwatchdog
group, filed a complaint with the commission
in 2001, criticizing several major search engines for
deceptive advertising practices in their treatment of
search results.iii
In response, the FTC issued a letter calling on the entire
search engine industry to provide “clear and conspicuous
disclosure” of paid placement and paid inclusion, the two
primary methods of incorporating advertising into search
results. iv Paid placement programs charge advertisers a
fee in exchange for higher rankings within search results.
Paid inclusion programs also charge a fee, but only to
assure a site’s listing within a search engine’s full index of
possible results—without a guarantee of ranking.
WebWatch’s body of research demonstrates consumers’
strong interest in knowing whether search engines sell
rankings to advertisers, as well as users’ considerable frustration
in trying to locate and understand these disclosures.
This study, “Searching for Disclosure,” evaluated the
compliance of 15 major search engines with the FTC’s
guidelines. WebWatch hopes this report will illuminate
the disclosure practices of search engines, as well as
help educate consumers about the financial forces at
work every time they click a “search” button.
The sites evaluated in the study were: 1st Blaze, Alta
Vista, AOL Search, Ask Jeeves, CNET’s Search.com,
Google, InfoSpace Web Search, Lycos Network Search,
MSN Search, My Search, My Way Search, Netscape
Search, Overture, Web Search and Yahoo! Search.
(See Study Methodology for information on how the sites
were chosen.)
Because of the complexity of the subject matter and
the well-documented lack of consumer awareness,
WebWatch recruited information retrieval experts
in library science to test each search engine. (See
Study Methodology for information on how the
questionnaire was designed, and how and the sites
were evaluated.)
Overall, the study revealed most major search engines
have made some efforts to satisfy the FTC’s recommendations,
but compliance varied widely, leaving ample room
for improvement throughout the industry. While some
sites diligently disclose and explain their business
relationships, others appear to obscure the presence of
advertising within search results.
DEFINITION OF TERMS
�� Paid Placement
When Web sites pay a fee to be ranked
prominently in search results.
�� Paid Inclusion
When Web sites pay a fee to increase
the likelihood they will appear somewhere
within search results, without a
guarantee of a high ranking.
FIGURE 1
WEBWATCH’S
BODY OF RESEARCH
DEMONSTRATES
CONSUMERS’ STRONG
INTEREST IN KNOWING
WHETHER SEARCH
ENGINES SELL RANKINGS
TO ADVERTISERS.
CONSUMER REPORTS WEBWATCH 6
Because of the demonstrable importance consumers place
on the integrity and transparency of search results, the
industry needs to enhance the effectiveness of disclosures
to ensure they are noticed and understood by users. If not,
search engines risk losing their credibility with consumers.
KEY FINDINGS
�� Paid inclusion was not satisfactorily disclosed or
explained by any of the search engines tested.
The credibility of this practice is of such concern
to the industry itself that, after Consumer Reports
WebWatch testing had been completed, two of the top
five search engines announced plans to terminate paid
inclusion programs. MORE ON PAGE 16 >>
�� Meta-engines, which present results from several
search engines simultaneously, repeatedly failed to
adequately disclose the presence of paid placement and
paid inclusion within search results.
MORE ON PAGE 21 >>
�� Disclosures are generally hard to find, accessible by
headings and hyperlinks that often blend in with the
THE CREDIBILITY OF
PAID INCLUSION IS OF
SUCH CONCERN TO THE
INDUSTRY ITSELF THAT,
AFTER TESTING, TWO OF
THE TOP FIVE SEARCH
ENGINES ANNOUNCED
PLANS TO TERMINATE
THESE PROGRAMS.
SEARCH ENGINES SELECTED
FOR THIS STUDY
1ST BLAZE
HTTP://WWW.1STBLAZE.COM
ALTA VISTA
HTTP://WWW.ALTAVISTA.COM
AOL SEARCH
HTTP://SEARCH.AOL.COM/AOLCOM/INDEX.JSP
ASK JEEVES
HTTP://WWW.ASK.COM
CNET'S SEARCH.COM
HTTP://WWW.SEARCH.COM
GOOGLE
HTTP://WWW.GOOGLE.COM
INFOSPACE WEB SEARCH
HTTP://WWW.INFOSPACE.COM/HOME/SEARCH
LYCOS NETWORK SEARCH/DIRECTORIES
HTTP://WWW.LYCOS.COM
MSN SEARCH
HTTP://WWW.SEARCH.MSN.COM
MY SEARCH
HTTP://WWW.MYSEARCH.COM/JSP/HOME.JSP
MY WAY SEARCH
HTTP://WWW.MYWAY.COM
NETSCAPE SEARCH
HTTP://NETSCAPE.COM/SEARCH
OVERTURE
HTTP://WWW.CONTENT.OVERTURE.COM
WEB SEARCH
HTTP://WWW.WEBSEARCH.COM
YAHOO! SEARCH
HTTP://SEARCH.YAHOO.COM
FIGURE 2
CONSUMER REPORTS WEBWATCH 7
page, making them easy for consumers to overlook.
MORE ON PAGE 21 >>
�� Information disclosed by the sites on business practices
with advertisers—and how these practices may affect
search results—was often confusing and jargon-laden.
MORE ON PAGE 21 >>
�� Some engines, like Google—one of the few majors
not named in the original FTC complaint—took pains to
visually segregate paid results from non-paid results.
Consumers may want to avoid others, like 1st Blaze,
because of inadequate or absent disclosures that undermine
the integrity of search results.
MORE ON PAGE 21 >>
CREDITS
This project was written and directed by Jørgen J.
Wouters, a consultant to Consumer Reports WebWatch
who has been writing about the Internet since 1993. In
SEARCH ENGINE STATISTICS
NIELSEN//NETRATING’S TOP SEARCH DESTINATIONS FOR DECEMBER 2003*
BRAND OR CHANNEL UNIQUE AUDIENCE ACTIVE REACH (%) TIME PER PERSON
1. GOOGLE 53,058,000 37.41 0:26:40
2. MSN SEARCH 42,297,000 29.82 0:06:53
3. YAHOO! SEARCH 41,250,000 29.08 0:09:56
4. AOL SEARCH 21,953,000 15.48 0:29:55
5. ASK JEEVES 11,481,000 8.1 0:11:50
6. OVERTURE 7,163,000 5.05 0:04:54
7. WEB SEARCH 6,225,000 4.39 0:03:59
8. LYCOS NETWORK SEARCH 5,884,000 4.15 0:04:44
9. NETSCAPE SEARCH 5,563,000 3.92 0:12:40
10. MY WAY SEARCH 5,137,000 3.62 0:05:27
11. INFOSPACE WEB SEARCH 5,031,000 3.55 0:11:47
12. ALTA VISTA 4,041,000 2.85 0:08:40
13. MICROSOFT SEARCH * 3,833,000 2.7 0:01:57
14. 1ST BLAZE 3,485,000 2.46 0:00:47
15. MY SEARCH 3,336,000 2.35 0:06:00
16. CNET SEARCH 3,291,000 2.32 0:01:32
Source: Nielsen//NetRatings Top Search Destinations Month of December 2003 US, Home and Work
*Note: Number 13, Microsoft Search, was omitted from the list of reviewed sites because it is an internal search engine, and thus unsuitable for this
report. WebWatch therefore used Number 16 on the list, CNET Search, in its place.
FIGURE 3:
CONSUMER REPORTS WEBWATCH 8
addition to reporting on the nascent Internet industry
for the Washington Post Company’s Washington
Technology, he also edited the Information & Interactive
Services Report, a weekly newsletter for information
industry executives. Wouters also has written for the
international consulting firm McKinsey & Company. He
participated in testing for this study.
This project was funded by Consumer Reports WebWatch.
The research report was edited by Tracy L. Ziemer,
WebWatch’s researcher and site producer. Neither
ConsumerWebWatch.org nor ConsumersUnion.org participate
in paid placement or paid inclusion programs with
any search engine. ConsumerReports.org does participate
in both paid placement and paid inclusion programs.
CONSUMER REPORTS WEBWATCH 9
Consumer Reports WebWatch is a project of Consumers
Union, the non-profit publisher of Consumer Reports
magazine and ConsumerReports.org. The project is
supported by The Pew Charitable Trusts, which invests in
ideas that fuel timely action and results; the John S. and
James L. Knight Foundation, which promotes excellence
in journalism worldwide and invests in the vitality of 26
U.S. communities; and the Open Society Institute, which
encourages debate in areas in which one view of
an issue dominates all others. WebWatch’s Web site
launched April 16, 2002.
http://www.consumerwebwatch.org
ABOUT CONSUMER REPORTS
WEBWATCH
CONSUMER REPORTS WEBWATCH 10
For many consumers, search engines serve literally
dozens of important functions, helping to make sense of
a cluttered universe of information, promising speed,
relevance and convenience. As such, they are looked on
as trusted guides.
For the first few years of their existence, search
results remained essentially advertising-free. A software
program known as a “spider” searched or “crawled”
the Web’s linked pages and created indexes from
which results, based on algorithmic relevance, were
then delivered to the user who had entered a search
term. The results were typically known as “pure”
results–based on relevancy to the user, not marketing
or advertising.
Consumer Reports WebWatch learned in its 2003 study,
“False Oracles: Consumer Reaction to Learning the Truth
About How Search Engines Work,” respondents in the
study generally believed most, if not all, search results
are “pure,” or untainted by advertising dollars. The
reverse, however, has been true for several years, and
“pure” results are now the exception, rather than the rule.
In 1998, Overture became the first major search engine
to sell rankings within search results to the highest
bidder, ushering in a form of advertising now commonly
known as “paid placement.” Advertisers didn’t need
much convincing, since paid placement represented
the most targeted form of advertising imaginable, one
far less obvious and more effective than banner ads
or pop-ups.
In 2000, Inktomi introduced the practice of “paid inclusion,”
charging Web sites to be spidered more frequently
for inclusion within its index—without a guarantee of
higher ranking within its search results. These results are
typically presented as the so-called “main” results,
making it all too easy for consumers to mistake them for
“pure” results, when, in fact, they are not.
That same year, Overture and Inktomi ushered in a new
era when they began selling their paid placement
and paid inclusion results to other search engines.
Newcomers like Google quickly followed as well, which
led to the establishment of an industry business model,
closely followed by a bewildering and ever-changing
A BRIEF HISTORY OF PAID
PLACEMENT & INCLUSION
CONSUMER REPORTS WEBWATCH 11
array of corporate relationships. (See Appendix E:
Search Engine Interrelationships.)
By the end of 2001, the results pages of virtually every
major search engine contained either paid placement or
paid inclusion listings—and usually both. The dot-com
implosion only fueled the trend, as cash-strapped advertisers
turned to these more targeted and cost-effective
methods, creating one of the fastest-growing advertising
mediums in history.
According to the Interactive Advertising Bureau, paid
placement listings displaced banner ads in 2003 to
become the Internet’s single biggest source of ad
revenue, helping fuel an almost 21 percent jump in overall
sales. Paid placement more than doubled its market
share from the previous year to account for 35 percent of
the almost $7.3 billion spent on Internet advertising in
2003.v Jupiter Research estimates spending on paid
placement will increase by a compound annual rate of
more than 20 percent over the next five years, hitting
$4.3 billion by 2008.vi
Spending on paid inclusion sales in comparison, and is
only expected to grow from $167 million in 2003 to
$293 million by 2006.vii But this projection will probably
be revised downwards now that two major search
engines have terminated their paid inclusion programs.
Paid placement is where the real money lies, as shown
by the explosive growth of Google. The world’s most
popular search engine—one whose brand name has
already entered the vernacular as a verb—earns
the lion’s share of its revenues from selling paid
placement listings.
Sales are skyrocketing. According to figures released in
preparation for its long-awaited initial public offering
(which valued the company at more than $23 billion),
Google lost $15 million in 2000 on sales of $19 million.
By 2003, Google earned $106 million on sales of $962
million. Based on its performance during the first quarter
of 2004, Google is on track to earn some $250 million
in profits on sales of more than $1 billion for the year.
This transformation of search results from impartial and
algorithmic to commercial and advertising-driven went
largely unnoticed by consumers, since there was little
attempt—with Google as a notable exception—to
distinguish between paid and non-paid placement
listings. Disclosure of paid inclusion was essentially
non-existent.
Some industry watchers grew alarmed at the trend. In
July 2001, the consumer watchdog group Commercial
Alert, based in Portland, Ore., filed a complaint with the
U.S. Federal Trade Commission (FTC), charging eight
major search engines with deceptive advertising
practices.1 The FTC’s response to this complaint cast a
spotlight on the problem of inadequate search engine
disclosure. That FTC letter forms the basis for this study.
1 The July 2001 complaint specifically named Alta Vista Co., AOL Time Warner, Inc., Direct Hit Technologies, iWon, Inc., LookSmart Ltd.,
Microsoft Corp. and Terra Lycos S.A. Available online at:
http://www.commercialalert.org/index.php/article_id/index.php/category_id/1/subcategory_id/24/article_id/33
BY THE END OF 2001,
VIRTUALLY EVERY
MAJOR SEARCH ENGINE
USED EITHER PAID
PLACEMENT OF PAID
INCLUSION—AND
USUALLY BOTH.
CONSUMER REPORTS WEBWATCH 12
Research shows a surprisingly large percentage of
Web users still fail to comprehend just how extensively
advertising dollars influence their search results.
In less than a decade, millions of consumers who grew
up relying on encyclopedias, telephones or the Yellow
Pages for information now turn instinctively to search
engines. In January 2004, Nielsen //NetRatings
estimated 151 million active U.S. Internet users spend
an average of almost 40 minutes searching. More
than 75 percent of Internet users conduct at least
one search a month, in ever-increasing volume.2 In
May 2004, Nielsen recorded 1.2 billion searches
among American Internet users, a 30-percent jump over
the previous year.3
As consumers click away on their favorite search engine
in search of airfares, life insurance or love, most do so in
the belief that they’re getting impartial or “pure” results
in return. In fact, depending on the search engine,
many—if not most—results are influenced to some
degree or other by commercial considerations, and the
burgeoning search engine advertising market only underscores
the crucial need for more transparent disclosure of
these practices.
Of course, a search for “digital cameras” that generates
advertising-driven links from Nikon or Canon is certainly
no cause for alarm, and may well be what the consumer
is looking for. But the issue of paid search can become
troubling when consumers looking for factual, unbiased
information about health care or investments, for
example, are steered toward sites selling particular
drugs or financial services—where a bad decision
can result in implications far more serious than buying
the wrong camera.
INTRODUCTION
2 Sullivan, D. “ Nielsen NetRatings Search Engine Ratings.” Search Engine Watch.com 23 February 2004.
Available online at: http://searchenginewatch.com/reports/article.php/2156451
3 “High Demand for Search Advertising Outpaces Supply, Underscoring Nedd for Innovative by Search Engines.”
Nielsen//Netratings July 19, 2004 Available online at: http://www.nielsen-netratings.com/pr/pr_040719.pdf
CONSUMER REPORTS WEBWATCH 13
WebWatch’s first commissioned study in 2002, “A
Matter of Trust: What Users Want From Web Sites”—a
national telephone survey of 1,500 Web-savvy U.S.
adults conducted by Princeton Survey Research
Associates—showed more than 60 percent of respondents
had no idea search engines are paid to list some
sites more prominently than others. However, 80 percent
of those surveyed believed it was important to disclose
such practices. viii
Consumer watchdog groups agreed. In July 2001,
Commercial Alert wrote to the FTC, charging the failure
of search engines to disclose either paid placement
or paid inclusion within search results amounted to a
deliberate deception—one that took advantage of
consumers’ conditioned expectation for advertisingfree
search results. The crux of the Commercial Alert
complaint read as follows:
Because of the earlier editorial integrity in search
engine results, there is an implied representation to
search engine users that listings are not skewed by
marketing or commercialism. Consumers are accustomed
to search engine protocols based on editorial
integrity, and have not been told of the departure
from these protocols. In effect, this is a high-tech case
of “bait and switch.” ix
The FTC’s Bureau of Consumer Protection responded one
year later, and agreed many search engines were not
clearly and openly explaining to consumers the presence
of advertising within search results. Although many
search engines were attempting some form of paid-placement
disclosure, the FTC said these explanations were
not clearly worded, and no search engines were adequately
disclosing paid inclusion.x
The FTC’s reply to Commercial Alert also echoed
language in the complaint, noting paid-placement listings
represented a departure from standard industry practice—
one that had the potential to deceive consumers
if not adequately disclosed. The reply also said intermingling
search results that paid to be included with
those that did not might mislead consumers without
a clear explanation.
While the FTC refrained from taking any formal action
against the companies listed in the complaint, it did send
them a letter outlining recommendations sites needed to
implement to avoid possible future action by the commission.
xi In that letter, the FTC also cited WebWatch
research showing four in five Internet users want search
engines to disclose their paid search practices.xii
The FTC’s recommendations apply to all search
engines—regardless of whether they were named in the
complaint—as well as any search engine using another
site’s results. The FTC advised companies to consult
the commission publication, “Dot Com Disclosures:
Information About Online Advertising,” upon which the
questionnaire used to test search engines for this study
was based. (See Study Methodology for more on how
the questionnaire was designed.)
One year after the FTC’s letter was sent to search engines,
a 2003 WebWatch study, “False Oracles: Consumer
Reaction to Learning the Truth About How Search Engines
Work,” employed an ethnographic approach to examine
attitudes of Web-savvy consumers to paid placement
programs. Among other things, the study found:
THE FTC’S REPLY
SAID INTERMINGLING
SEARCH RESULTS THAT
PAID TO BE INCLUDED
WITH THOSE THAT DID
NOT MIGHT MISLEAD
CONSUMERS.
CONSUMER REPORTS WEBWATCH 14
�� Most participants had little understanding of how
search engines retrieve Web pages or rank links on a
results page.
�� The majority of participants never clicked beyond
the first page of search results, so some 40% of the
links they selected were in fact paid placement listings,
i.e. advertising.
�� All participants said paid placement links were often
too hard to recognize or find, and the available disclosure
information was clearly written for the advertiser, not
the consumer. xiii
Consumers aren’t alone in their confusion. Even members
of the business community appear to have trouble distinguishing
between paid and “pure” search results.
Research by Internet marketing firm WebAdvantage.net
showed slightly less than half of 450 small business
owners it surveyed were able to recognize paid-placement
listings, a situation the firm attributed to lackluster
disclosure efforts. xiv
EXAMINING PAID PLACEMENT
Of the two kinds of search engine advertising addressed
by the FTC, paid placement is easily the more obvious—
both in terms of its relative visibility and its impact on
search results.
In its response to Commercial Alert, the FTC defined
“paid placement” as follows:
Any program in which individual Web sites or URLs
can pay for a higher ranking in a search results list,
with the result that relevancy measures alone do not
dictate their rank. xv
The FTC advises search engines to distinguish paid placement
listings from “main” results with disclosures that
are easy to spot and understand. Paid placement listings
are typically grouped under headings such as “sponsored
links” or “sponsored sites,” and are usually placed
either above, below, or to the right of the main results—
and sometimes all three. Some search engines also highlight
these listings by use of graphics such as shaded
boxes or borders. Certain search engines, however, mix
paid-placement and non-paid placement listings together,
and identify them on an individual basis.
Although the FTC did not suggest any further paid
placement disclosure beyond a clear and conspicuous
heading on the results page, many search engines go
above and beyond the commission’s recommendations
with hyperlinks to separate disclosure pages explaining
their paid placement programs. Most of the engines that
do this tend to operate paid inclusion programs as well,
which require additional disclosures.
Most search engines feature paid placement listings
supplied by other engines—most often Google and
Yahoo-owned Overture, which dominate the industry.
Typically, advertisers bid for placement within search
results based upon certain search terms, with the highest
rankings going to those willing to pay the highest “costper-
click,” which can range from a few pennies to a
few dollars.
Because these links are typically segregated from main
listings, and the influence of payment upon rankings is at
least somewhat obvious, one might expect little trouble
on the part of consumers to identify and understand paid
placement listings for what they are: Advertising.
Nevertheless, as noted earlier, WebWatch research
indicates many consumers remain unaware about the
existence of paid placement listings—underscoring the
need for greater transparency among search engines.
And advertisers seem to be betting on this ignorance, as
the following quote by an online marketer in a tradepress
column about search engines results pages (SERP)
clearly illustrates:
Overture listings perform slightly better than Google
AdWords listings for our clients. We suspect the
reason is the SERP structure. Google’s SERPs display
clearly labeled ads in text boxes on the right side of
the page. Overture ads on Yahoo! aren’t so obvious.
Ads populate prime real estate, in the center of the
CONSUMER REPORTS WEBWATCH 15
page two inches from the top. Less-savvy users might
not realize the Yahoo! sponsor results are ads. xvi
The last sentence says it all: Unsuspecting or “less-savvy”
Web users are more likely to click on links they don’t
recognize as advertising.
EXAMINING PAID INCLUSION
Paid inclusion, on the other hand, represents a far
more subtle and pervasive practice. Indeed, with a
few notable exceptions (Google, AOL and Netscape),
virtually every search engine uses paid inclusion for its
main results, which usually outnumber paid placement
listings. As such, the main results of most search engines
contain a mixture of sites that paid and did not pay to
be “crawled.”
In its letter to Commercial Alert, the FTC defined “paid
inclusion” as follows:
Any program in which individual Web sites or
URLs are included in a search engine’s index, or
pool, of sites available for display as search results,
when that Web site or URL might not otherwise have
been included, or might not have been included at a
particular point in time, but for participation in the
paid program. xvii
The FTC says search engines must clearly and conspicuously
disclose their use of paid inclusion, explain how
Web sites are chosen, and allow users to discern
what, if any, impact it may have on rankings. This
requires a disclosure heading and hyperlink to an easily
locatable page or pop-up window disclosing and
explaining the paid inclusion program. Paid inclusion
results in the main listings are typically grouped
under headings such as “Web results,” “Web pages” or
other generic headings that do not sufficiently manage to
indicate paid inclusion.
In fact, the FTC noted in 2002 that none of the sites in
question adequately disclosed or explained their paid
inclusion programs. Although the situation has certainly
improved, the very nature of paid inclusion makes it
more difficult to disclose and explain—let alone describe
via a heading. If consumers and business people have
difficulty identifying paid placement, one can imagine
how few are even aware of the more subtle practice of
paid inclusion.
As with paid placement results, most search engines
receive their paid inclusion results from other
search engines. (See Appendix E: Search Engine
Interrelationships.) Web sites typically pay a flat, annual
fee per URL for inclusion within a search engine’s index,
although some sites also charge a “cost-per-click.” Even
with “per-click” pricing, payment is only supposed to
guarantee inclusion in a search engine’s index—not
a boost in rankings.
But despite disclaimers by every search engine that paid
inclusion has absolutely no impact upon rankings, there
is no way to confirm these claims, since search engines
are not required to distinguish between sites that paid to
be crawled and those that did not.
IF CONSUMERS AND
BUSINESS PEOPLE HAVE
DIFFICULTY IDENTIFYING
PAID PLACEMENT, ONE
CAN IMAGINE HOW
FEW ARE EVEN AWARE
OF THE MORE SUBTLE
PRACTICE OF PAID
INCLUSION.
CONSUMER REPORTS WEBWATCH 16
One of the industry’s leading experts,
SearchEngineWatch.com editor Danny Sullivan, has
repeatedly written about the need to better disclose
paid inclusion, which he suggests may in fact sometimes
affect rankings: “In contrast, paid inclusion interacts
directly with editorial results, and we have only the
search engines to take at face value when they say it
doesn’t impact rankings. After seeing ranking boosts
explained away for technical reasons, I’ve become
more convinced that paid inclusion content may ultimately
have to be labeled or segregated just like paidplacement
content.” xviii
This does not, however, suggest paid inclusion is an
inherently harmful or underhanded practice. In fact,
properly disclosed, paid inclusion can provide benefits to
the consumer, as the FTC itself points out in its letter to
Commercial Alert:
To the extent that paid inclusion does not distort the
ranking of a Web site or URL, many of these
programs provide benefits to consumers, by incorporating
more Web sites—or content—into an individual
search engine’s database than might otherwise be
the case. This can give consumers a greater number
of choices in search results lists. xix
RENEWED DEBATE
The debate over paid inclusion was reignited in March
2004, when search engine Ask Jeeves dropped one
of its paid inclusion programs. Although company
executives said the program was discontinued for inadvertently
skewing search results, they also acknowledged
concerns over the perception of using a “cost-per-click”
model for paid inclusion.
Almost simultaneously, Yahoo announced two new paid
inclusion programs for its stable of search engines, both
of which feature cost-per-click pricing. Yahoo’s announcement
fueled a heated debate in the trade press over
cost-per-click for paid inclusion and the practice of paid
inclusion in general.
In an interview with DM News and to the larger trade
press, a Google executive reiterated his company’s deepseated
aversion to paid inclusion, which he denounced
as a violation of consumer trust. “The only way to keep
user trust is to keep a church-and-state divide between
what is paid and non-paid,” said Tim Armstrong,
Google’s vice president of advertising.xx
Intensifying the debate, a December 2003 report by
analysts Jupiter Research called for abolishing cost-perclick
pricing for paid inclusion programs, which it
condemned as completely lacking in credibility. “Paid
inclusion programs that currently have a cost-per-click
pricing component must eliminate it, as the model has a
fundamental and inherent conflict of interest,” the report
noted. “Under a cost-per-click model, claims that the
rankings of paid inclusion customers are unaffected
cannot be believed, as the search index has a clear
financial incentive to promote rankings.” xxi
The controversy over paid inclusion continues. In late
June 2004, Ask Jeeves announced it would terminate its
remaining paid inclusion program, although contractual
obligations mean these listings will linger on its site until
2005. One week later, MSN Search followed suit as
well, leaving Yahoo as the last paid inclusion hold-out
among the top five search engines.
CONSUMER REPORTS WEBWATCH 17
Consumer Reports WebWatch evaluated 15 search
engines for this study, culling the sites from the list of the
top-most trafficked search engines in December 2003,
according to Nielsen//NetRatings:
1.1st Blaze
2. Alta Vista
3. AOL Search
4. Ask Jeeves
5. CNET’s Search.com
6. Google
7. InfoSpace Web Search
8. Lycos Network Search/Directories
9. MSN Search
10. My Search
11. My Way Search
12. Netscape Search
13. Overture
14. Web Search
15. Yahoo! Search
(See Figure 2: Search Engine Selected For This Study for
a list of the site names and URLs.)
HOW THESE SITES WERE SELECTED
Commercial Alert’s letter of complaint to the FTC specifically
named seven companies that owned and operated
at least one search engine each.4 The FTC replied one
year later with a letter to companies included in the
original complaint, as well as seven others.5
But as the FTC made clear in its 2002 letter to
STUDY METHODOLOGY
4 The July 2001 complaint specifically named Alta Vista Co., AOL Time Warner, Inc., Direct Hit Technologies, iWon, Inc., LookSmart Ltd.,
Microsoft Corp. and Terra Lycos S.A. Available online at:
http://www.commercialalert.org/index.php/article_id/index.php/category_id/1/subcategory_id/24/article_id/33
5 According to the FTC, 14 parent companies were sent action letters in June 2002: About, Ask Jeeves, Alta Vista, AOL Time Warner,
Teoma (DirectHit), Walt Disney, Google, InfoSpace, iWon, LookSmart, Microsoft, Overture, Terra Lycos and Yahoo.
Available online at: http://www.ftc.gov/os/closings/staff/commercialalertattatch.htm
CONSUMER REPORTS WEBWATCH 18
Commercial Alert, its recommendations apply to all
search engines, regardless of whether they were
named in the original complaint. These guidelines
also apply to any third-party Web sites using another
site’s search results, be they paid placement or
paid inclusion.
Two years is an eternity in the Internet business, one in
which the competitive landscape remains in constant flux.
Some of the companies named in the Commercial Alert
complaint have since been absorbed by competitors,
while other once-dominant search engines have since
been reduced to bit players, particularly as Google’s
popularity grew.
Faced with such a fluid market, Consumer Reports
WebWatch decided to let the market decide and
surveyed the most-trafficked search engines, regardless
of whether they were named in the original complaint—
and regardless of whether one or more sites were
owned by the same parent company. As it turned
out, nearly all engines appearing on the FTC’s list—with
the exception of Go.com, About and iWon, which
is now owned by Ask Jeeves—were evaluated in
this study.
Nielsen//NetRatings provided WebWatch with its list of
the 15 top -trafficked search destinations by U.S. users at
home and work as of December 2003. No.13 on that
list, “Microsoft Search,” was in fact an internal search
engine—designed to search Microsoft’s own site—and
thus was unsuitable for evaluation. That site was removed
from the list and No. 16, “CNET Search,” was added
to maintain the number of sites evaluated at 15.
(See Figure 3: Search Engine Statistics for the
Nielsen//NetRatings list.)
DESIGNING THE QUESTIONNAIRE
In order to evaluate compliance among major search
engines with the FTC’s recommendations, Consumer
Reports WebWatch created a questionnaire designed to
measure clear and conspicuous disclosure of paid placement
and paid inclusion programs.
In its letter of recommendation to the search
engine industry, the FTC encouraged companies
to review and implement the guidelines laid out in
the commission publication, “Dot Com Disclosures:
Information About Online Advertising.” xxii Although
this publication acknowledges the need for flexibility
among the methods search engines use to disclose
their advertising practices, it does provide a detailed
list of criteria the FTC deems necessary to protect
consumers from deceptive advertising practices.
WebWatch relied exclusively on this publication
and the FTC’s letters to inform its questionnaire,
which it believes provides an objective tool as close
as possible to the spirit, letter and intent of the
FTC recommendations. (See Appendix C: Disclosure
Questionnaire.)
These guidelines outlined in the FTC publication revolve
around a few basic concepts necessary for clear and
conspicuous disclosure, and include “placement,”
“proximity” and “prominence,” as well as the need of the
disclosure language to be simple, straightforward and
understandable. These concepts apply to headings,
hyperlinks and the actual disclosures themselves. (See
Appendix D: Search Engines’ Disclosure Statements for
an annotated results page.)
While the FTC does not call for the use of hyperlinks to
disclosure pages for paid placement programs, most
major search engines do, in fact, disclose their paid
placement programs in this manner, as they do for paid
inclusion programs.
Because most search engines tested exceed the FTC
recommendations regarding the disclosure of paid
placement by providing an explanation page and not
just a heading, questions regarding hyperlinks and disclosure
pages were included in the “paid placement”
section of the questionnaire as well. Although some
testers found fault with search engines that disclose
paid placement with only a heading, it must be stressed
these sites are technically in compliance with FTC
guidelines. However, the obviousness and effectiveness
of these headings to users, in terms of visibility and
clarity, is open to scrutiny.
CONSUMER REPORTS WEBWATCH 19
TESTING PROCEDURES
Previous Consumer Reports WebWatch research demonstrates
a lack of awareness among consumers regarding
the impact of advertising dollars on search engine rank.
Consumers—indeed, even savvy information retrieval
experts, as this report will show—have even greater
difficulty identifying and understanding paid inclusion.
WebWatch’s desire to evaluate search engine compliance
with FTC guidelines for both paid placement and
paid inclusion resulted in the decision to use information
retrieval experts with a thorough understanding of the
issues. Four information specialists in library science
were recruited to participate in the study and asked to
test 15 search engines each, for which they were each
paid $500. (See Appendix B: Testing Profiles.)
Because of this study’s focus on disclosures rather than
results, queries were not chosen in any scientific manner.
Search terms were instead chosen for timeliness,
perceived popularity and for a balance between commercial
and informational topics. (See Figure 20 in
Appendix A: Testing Background for query terms.)
A search engine’s disclosure of paid placement or paid
inclusion doesn’t vary from query to query, although noncommercial
searches may sometimes fail to generate paid
placement listings. As such, each tester was assigned two
queries per search engine from a rotating list.
The actual testing took place between April 28 and May
4, 2004, during which time testers were free to evaluate
the 15 search engines in any order—or at any time—
they chose. The testers were not required to click beyond
the first page of results. They completed one questionnaire
per search engine, submitting a total of 15
questionnaires each. Besides noting specifics such as
whether a given search engine followed a given guideline,
the testers were also asked to record any comments
and observations, which were noted where appropriate
throughout the engine-by-engine findings.
Several search engines sometimes practiced “content
promotion,” a confusing gray area used to highlight a
site’s related internal content, other worthwhile links,
sponsored results or a combination of all three. Content
promotion results (which often include links and images)
are typically inserted at the top of the page, above paid
placement listings, but are sometimes found above, or
within, the main listings.
Since the FTC did not address content promotion, it was
not specifically addressed in this study, except when
identified by testers or the author for one or other reason.
EVALUATING THE RESULTS
This study, it must be stressed, focused on the manner in
which search engines disclose their results rather than the
actual nature of those results. This study did not attempt
to compare search results on an engine-by-engine basis,
or to determine whether the search results returned were
relevant to a given query.
The purpose of these evaluations were not to assign
grades, rankings or even a pass/fail to individual search
engines based on their perceived compliance with FTC
recommendations. The focus was on singling out practices,
rather than engines. That said, the study did note
instances in which certain search engines appeared to
be meeting—or falling short—of the FTC guidelines.
Consumer Reports WebWatch hopes, with this study, to
alert the industry to any current practices that may result
in long-term credibility problems, as well as to call
attention to potential inadequacies in the FTC recommendations
and alert consumers to the presence of advertising
within search results.
The search engine industry is nothing if not dynamic, with
business models and relationships subject to frequent and
abrupt change. Some of the engines reviewed may have
altered or discontinued some paid placement and paid
inclusion programs in place during testing. Further, their
disclosure of paid placement and /or paid inclusion
programs may also have changed—for better or worse—
between the time testing was conducted and the report
was published.
CONSUMER REPORTS WEBWATCH 20
Although most major search engines are making some
effort to comply with the FTC’s recommendations to clearly
reveal the presence of advertising within search results,
compliance varied widely, our study revealed. All sites
tested could improve the visibility and/or clarity of their
disclosures. Most troubling, some search engines seem
to be doing as little as possible to explain their relationship
with advertisers—and as much as possible
to obfuscate both their use and disclosure of paid
placement and paid inclusion.
Of the 15 search engines tested, only one—1st Blaze—
was found to disregard disclosures and each of the FTC’s
guidelines. Although the site appears to engage in at
least some paid placement—the results it provides under
the heading “Featured Sites” were identical for every
tester, regardless of the search term—the site makes no
attempt to disclose this fact with obvious headings, hyperlinks
or disclosure pages. Whether 1st Blaze uses paid
inclusion remains uncertain, since testers found no
information or explanation on the site about how search
results are generated.
Disclosures on other sites could improve as well. Despite
the expertise of the testers, nearly all of them experienced
some difficulties with certain search engines, whether
identifying headings or hyperlinks; differentiating
between content promotion, paid placement and paid
MAJOR FINDINGS
IF EXPERTS
IN INFORMATION
RETRIEVAL HAVE
PROBLEMS FINDING
AND UNDERSTANDING
DISCLOSURE
INFORMATION, WHAT
DOES THIS MEAN
FOR THE AVERAGE
CONSUMER?
CONSUMER REPORTS WEBWATCH 21
inclusion; or making sense of disclosure statements.
These problems tended to occur with greater frequency
when evaluating paid inclusion. If experts in information
retrieval, on the lookout for disclosures, have problems
finding and understanding this information, what does
this mean for the average consumer?
AMONG THE MAJOR FINDINGS
�� Majority of Disclosure Headings Were Difficult to Spot
Eight of the 15 search engines tested labeled their paid
search listings with headings that were both smaller and
duller in color than the search results themselves. The use
of inconspicuous fonts and colors increases the likelihood
consumers may not notice disclosures and fail to grasp
the true nature of their results.
�� Most Hyperlinks to Disclosures Were Imperceptible
With only three exceptions (Yahoo, AOL, Lycos), all
search engines tested used tiny and faint fonts, such as
light gray, for hyperlinks to disclosure pages. Some of
these hyperlinks blended in so well with the page that
some testers missed them completely. Engines’ pervasive
use of eye-straining hyperlinks, which almost seemdesigned
to be overlooked, greatly reduces the chances
consumers will ever see their disclosures.
�� Many Disclosures Were Incomprehensible
Disclosure statements should be both simple and straightforward.
But testers found many of them—for both paid
placement and paid inclusion—were anything but, and
almost seemed written to discourage reading. Many
disclosures also seemed geared more toward advertisers
than consumers and were peppered with jargon and
trademarked names for various programs—particularly
for paid inclusion—leaving some testers baffled and
uncertain about what they had read.
�� Meta-Search Engines Were Mega-Offenders
Every tester found fault with the disclosure, or lack thereof,
provided by the three meta-search engines tested:
CNET’s Search.com, InfoSpace and Web Search. Metasearch
engines present results from several other engines
simultaneously. But they also tend to strip away all existing
disclosures search engines have provided for their
sites and often do a poor job providing any of their own.
Although CNET and InfoSpace disclosed paid placement,
none of the three meta-engines tested adequately
disclosed paid inclusion—despite relying on results that
may have contained paid inclusion. In fact, CNET and
InfoSpace omitted any mention of paid inclusion anywhere
on their respective sites, while Web Search mentioned
the practice might occur without indicating where
or when. Several months after testing concluded, Web
Search began identifying paid placement listings on a
link-by-link basis.
�� Most Search Engines Exceeded Paid Placement
Guidelines
Although the FTC only requires a visible and clearly
worded heading to indicate paid placement results,
most major search engines tested went a step further by
providing a hyperlink to a separate disclosure page.
Those sites that did not, however, were criticized by
testers for not voluntarily offering additional information
for consumers. Google, which supplies many sites with
paid results, was singled out for failing to provide an
explanation of its paid placement programs beyond a
heading on its own site. Most of the sites Google
supplies with paid placement listings provide fuller
and easier-to-locate disclosures than Google does
itself, testers noted.
EVERY TESTER FOUND
FAULT WITH THE
LACK OF DISCLOSURE
PROVIDED BY THE
THREE META-SEARCH
ENGINES TESTED.
CONSUMER REPORTS WEBWATCH 22
�� Some Disclosures Were Buried Too Deep
Search engines should make full disclosures available in
a prominent position after the first click, according to the
FTC. Many of the engines tested took users directly to the
appropriate disclosure whether it stood alone (Ask
Jeeves) or shared the page with other disclosures (AOL).
But some search engines forced users to scroll or click
again—or both—in order to locate disclosures (MSN,
My Way Search), making them unnecessarily difficult
and frustrating for testers to find.
�� Several Engines Divorced Hyperlinks from Headings
Hyperlinks should be placed near headings so consumers
can easily find the disclosure once they’ve
noticed the heading, says the FTC. Whenever certain
search engines separated hyperlinks from headings,
testers complained of the difficulty in finding them—and
sometimes failed to spot them at all. Some engines
placed hyperlinks across the page from the heading (Ask
Jeeves), at the top of the page (InfoSpace), or buried
them at the bottom of the page (Web Search, My
Search), undermining the disclosure.
�� Paid Inclusion Headings Confusing
Almost all of the search engines tested labeled their paid
placement listings with headings that most testers
believed clearly conveyed the use of advertiser-driven
results (“Sponsored Links,” “Sponsored Results,” etc.).
However, none of the search engines tested used headings
(“Web Results,” “Web Pages,” etc.) that any tester
said clearly indicated paid inclusion. These shortcomings
may soon become moot, due to the growing controversy
over the credibility of paid inclusion. Some two
months after the conclusion of testing, both Ask Jeeves
and MSN Search announced plans to phase out
their programs, leaving Yahoo as the last major
paid inclusion player.
�� Lack of Clarity Can Lead to False Assumptions About
Paid Inclusion
At the time of testing, all but a few of the sites tested—
Google and the sites it supplies with main listings,
namely, AOL and Netscape—used paid inclusion.
Because the use of paid inclusion was almost universal,
and certain sites used it without openly stating so,
Google, AOL and Netscape inadvertently misled some
testers by not explicitly stating their main results were
advertising-free. The lack of any such statement by these
sites led several testers to express uncertainty over their
use of paid inclusion; or to erroneously conclude they
used paid inclusion without bothering to disclose it—as
certain other sites did.
�� Content Promotion Creates Confusion
Some of the search engines tested (AOL, MSN, Netscape,
Lycos and Yahoo) sometimes use “content promotion,”
placing internal and/or sponsored content either above
their main results or under separate headings. Regardless
of where they were placed, or how they were disclosed,
these content promotion listings confused testers. Testers
sometimes mistook them for paid placement and /or paid
inclusion. Some content promotion disclosures only muddied
the waters by mentioning the use of “sponsored”
content. Some two months after testing, MSN Search,
which made frequent use of content promotion under a
“Featured Sites” heading, dropped the practice.
�� Cross-Ownership is Not Always Acknowledged
Some testers were misled by the failure of certain search
engines to acknowledge their business relationships with
other sites mentioned in disclosures. Yahoo, for instance,
acknowledges its ownership of Overture in its disclosure,
but does not mention its ownership of Alta Vista in the
latter’s disclosure. In its disclosure pages, Ask Jeeves
does not readily acknowledge its ownership of Teoma.
This led one tester to wonder whether Ask Jeeves
intended to obfuscate that fact.
CONSUMER REPORTS WEBWATCH 23
Below are detailed engine-by-engine assessments
distilled from questionnaires completed by each tester. In
addition to noting specifics—such whether or how a
certain search engine satisfied a given FTC disclosure
recommendation—the results also make liberal use of
observations by the testers.
It should be noted that one of the testers repeatedly
questioned whether the use of the word “Sponsored” in
headings (e.g., “Sponsored Links”) adequately conveyed
advertising-driven results. This observation supports a key
finding of an earlier WebWatch report, “False Oracles:
Consumer Reaction to Learning the Truth About How
Search Engines Work.” Nearly all participants in that
study, who were informed about the influence of advertising
on search results and asked to identify indicators of
paid placement, said the term “sponsored” was either
vague, misleading or confusing. Furthermore, participants
likened the term to something positive or virtuous,
such as giving money to charity, and not to advertising.
Every search engine tested for this study featured paid
placement, and all but a handful used paid inclusion.
Both practices were examined separately.
Using Yahoo as an example, the terminology for the
purposes of this study is as follows (See Figure 4:
Disclosure Elements):
Heading: The label used to indicate a group of paid
placement or paid inclusion listings. Using Yahoo as an
example, “Sponsor Results” and “Top 20 Web Results”
represent paid placement and paid inclusion headings,
respectively.
Hyperlink: The link that takes users to a disclosure page.
Hyperlinks should appear next to the heading. In the
example below, hyperlinks are represented by the
“What’s This?” links.
Disclosure Page: A separate Web page or pop-up
window that explains paid placement and paid inclusion.
Because most sites use identical fonts and colors for both
their paid placement and paid inclusion headings (as
well as duplicate wording of hyperlinks), fuller descriptions
were given for the former than the latter to avoid
needless repetition. Exceptions were noted. Finally, many
of the comments make reference to language used in disclosure
pages, some of which are partially quoted below.
The complete disclosure statements offered by each
search engine are available in Appendix D.
RESULTS BY SEARCH ENGINE
CONSUMER REPORTS WEBWATCH 24
FIGURE 4: DISCLOSURE ELEMENTS
HEADINGS HYPERLINKS
CONSUMER REPORTS WEBWATCH 25
1ST BLAZE
http://www.1stblaze.com
Dates tested: April 28, 29 & May 4
See Figure 5
Paid Placement
1st Blaze appeared to use paid placement for at least
some results, but did not explain how search results are
generated, either on the results page or anywhere else
on the site, a practice that was criticized by every tester.
Visually, 1st Blaze returned results in three batches. The
first group appeared after the statement: “Your search for
“X” returned the following results.” The second group
appeared after the heading “Featured Sites,” while the
third group appeared under the statement: “Results from
around the world related to X.” While the first and third
groups returned results related to the search terms, the
“Featured Sites” group did not. None of the headings
offered hyperlinks to disclosure pages.
Strangely, the “Featured Sites” results remained identical
—and irrelevant—for each and every query. Whether
searching for information on breast cancer, the Atkins
Diet or digital cameras, 1st Blaze returned the same
“Featured Sites” results: Links to a contest, an offer of
easy credit, a dating service, and a solicitation to earn a
college degree. The results led some testers to suspect
paid placement, although the listings’ irrelevancy caused
confusion. “I can’t tell if ‘Featured Sites’ are paid placement
ads or not,” said one tester. “They’re ads, but
they’re totally unrelated to the search.”
Several testers said it was impossible to tell whether the site
used paid placement, although most suspected it did.
Every tester found fault with the site’s lack of transparency.
“This search engine does not have any disclosure information
whatsoever,” and “very secretive search engine” said
one. “There is really no indication as to the basis on which
a search result is generated,” remarked another.
Paid Inclusion
1st Blaze offered no obvious headings, hyperlinks or
disclosure pages. Testers were highly critical of the site
as a result.
“1st Blaze does not provide enough information on its site
to make any determinations regarding its paid placement
and/or paid inclusion programs,” said one, echoing
comments of the entire group.
An attempt to reach 1st Blaze on September 16, 2004 by
e-mail—the only means of contact available anywhere on
the site—bounced, returning the message, “I’m sorry to
have to inform you that the message returned below
could not be delivered to one or more destinations.”
ALTA VISTA
http://www.altavista.com
Dates tested: April 28, 29 & May 4
See Figure 6
Paid Placement
Alta Vista’s paid placement listings appeared in groups
at the top and bottom of the results page and were distinguished
from other results by a heading and hyperlink
to a disclosure page. Results were provided by Overture,
a subsidiary of Yahoo, which also owns Alta Vista—
although this fact was not mentioned in the site’s
disclosure pages.
Most testers believed the “Sponsored Matches” heading
clearly conveyed the engine’s use of paid placement,
and all praised the easy-to-spot red font. Although
most said the “About” hyperlink label indicated
the availability of a disclosure statement, the hyperlink’s
faint, gray font blended in with the page, making it
hard to spot.
“Light gray on white background, poor contrast—[it’s]
not clear that this is an active link since it reminds one of
normal Windows menu items that are ‘grayed out,’ i.e
inaccessible,” complained one tester.
Clicking on the “About” hyperlink opened a separate
page containing all disclosures. This page, several testers
noted, was incorrectly labeled “Alta Vista—Types of
Audio Results” at the top of the browser window. The
paid placement disclosure, which was explained under
the heading “Sponsored Matches,” required users to
CONSUMER REPORTS WEBWATCH 26
FIGURE 5: 1ST BLAZE
CONSUMER REPORTS WEBWATCH 27
FIGURE 6: ALTA VISTA
CONSUMER REPORTS WEBWATCH 28
scroll down the page to find since the hyperlink incorrectly
directed users to the site’s paid inclusion disclosure.
One tester complained the disclosure page was set at a
fixed line-length, which meant the text did not “wrap”
when viewed in minimized form, and ran off the page.
“The line length is longer than acceptable typographic
standards for the font size,” the tester said. “Thus I would
say the layout is made intentionally difficult to read.”
Despite these technical quibbles, most testers deemed
Alta Vista’s paid placement disclosure clear and accessible
to the average consumer. “Alta Vista is up-front about
its paid placement program, which makes it easy for the
alert searcher to follow-up on what a sponsored match
means,” said one.
Paid Inclusion
Alta Vista uses paid inclusion. Results were supplied by
Yahoo, appeared in between two sets of paid placement
results, and were disclosed with a heading and a hyperlink
to a disclosure page.
All testers said the “Alta Vista found X results” heading
failed to adequately convey the site’s use of paid
inclusion, but they praised its appearance. “Good
contrast—dark red on white background,” said
one. Comments on the “About” hyperlink were similar
to those for paid placement, i.e. good language,
poor visibility.
Clicking on the hyperlink opened a separate page
containing all disclosures and took users directly to the
paid inclusion statement, which was explained under the
heading “Alta Vista Results.” Apart from the inexplicable
page labeling (“Audio Results”) and the line-length
issues, some testers found the disclosure somewhat
jargon-laden and confusing.
“The text is littered with proprietary names like Inktomi
SearchSubmit TM and Content Acquisition Program
(CAP),” noted one.
“While Alta Vista is to be praised for explaining paid
inclusion, the explanation is lengthy and heavy with
acronyms—likely confusing to most readers,” said
another. “Alta Vista does a better job explaining its paid
placement programs.”
AOL SEARCH
http://search.aol.com/aolcom/index.jsp
Dates tested: April 28, 29, 30 & May 4
See Figure 7
Paid Placement
AOL Search’s paid placement listings appeared at the
top and bottom of the results page, and were distinguished
from other results by a heading and hyperlink to
a separate disclosure page. Results were provided by
Google, a fact disclosed directly on the results page.
The site sometimes uses “content promotion,” which can
include both AOL and advertiser results selected by
editors. These “Recommended Sites” were typically
placed between paid placement and paid inclusion
results, and were disclosed by a heading and hyperlink
to a disclosure.
All testers noted the large, red, eye-catching “Sponsored
Links” heading, which most agreed indicated the use of
advertising-driven results. One tester, however, mistook
“Recommended Sites” for paid placement listings, underscoring
a recurring problem with content promotion,
which occupies a gray area straddling paid and nonpaid
results.
Testers liked the unusual disclaimer following the heading:
“Provided by a third party and not endorsed by
AOL.” They also praised the clear and unambiguous
language of the “What is a Sponsored Link?” hyperlink,
as well as its bright blue font, which made it one of the
few truly noticeable links among all engines tested.
Testers were generally less enamored of the actual disclosures,
which were contained on a single page titled
“About AOL Search.” Paid placement was explained
under two separate headings: “Sponsored Links—
Search” and “Sponsored Links—Content,” which was
unnecessarily confusing, testers said.
CONSUMER REPORTS WEBWATCH 29
FIGURE 7: AOL SEARCH
CONSUMER REPORTS WEBWATCH 30
“While AOL does a good job at explaining the concept
of paid placement in a concise manner, the explanation
gets lost among the other information on the page,” said
one tester. “The difference between ‘Sponsored Links—
Search’ and ‘Sponsored Links—Content’ may not be
clear to all searchers.”
Another tester noted with disapproval the disclosure
offers both a link to Google disclosure pages aimed at
advertisers rather than consumers, and the opportunity to
sign up for Google’s ADWords program.
Paid Inclusion
AOL’s main results, provided by Google, were grouped
under the heading “Matching Sites” and a link (labeled
“What is a Matching Site”) to a disclosure page.
AOL did not use paid inclusion, making it one of the few
sites that doesn’t. This fact wasn’t imminently clear to
testers, however, underscoring a potential communication
problem for consumers as well.
When asked to determine whether AOL used paid inclusion,
one tester responded, “I believe that it does, but it
is hard to tell.” Said another: “They say they get their
matching sites from Google, but Google does not say
whether they include paid inclusion in their index.”
ASK JEEVES
http://www.ask.com
Dates tested: April 29, 30 & May 4
See Figure 8
Paid Placement
Ask Jeeves uses paid placement and relies on results from
Google. These listings appeared at the top of the page,
and were distinguished from other results by a heading,
a hyperlink to a disclosure page, and a thin line framing
the section. Ask Jeeves sometimes placed a “featured
sponsor” listing at the very top of the page in a separate
box with its own heading and hyperlink.
All testers praised the “Sponsored Web Results” heading
for its good visibility. “Red print on yellow background—
clearly stands out,” said one. Most believed the wording
clearly conveyed the site’s use of paid placement, while
one disapproved of the potential misinterpretation of the
word “sponsored.”
While the “about” hyperlink adequately alerted testers to
the availability of a disclosure, most testers criticized the
small, faint hyperlink for poor visibility and its inconspicuous
location across the page from the heading. Because
of the link’s size, color and position, one tester failed to
notice it at all.
Clicking on the hyperlink took users to a page titled
“Sponsored Web Results” that contained only the paid
placement disclosure, which most testers found both
simple and straightforward.
“Very brief and easy to understand,” remarked a tester.
“In fact, the disclosure about Google-provided links is
much clearer than the Google pages themselves.”
Paid Inclusion
Ask Jeeves used paid inclusion at the time of testing but
announced in early July that it was phasing out this
program. Results were supplied by Teoma, which is
owned by Ask Jeeves, a fact not readily acknowledged
without some additional digging through the help pages.
Paid inclusion results appeared below the paid placement
results, and were disclosed with a heading and
hyperlink to a separate disclosure page.
None of the testers thought the “Web Results” heading
adequately conveyed the use of paid inclusion, but most
found the bold, red font easy to spot. Comments on the
“About” hyperlink were identical to those for the paid
placement link, which was also overlooked by one of the
testers due to its poor visibility and placement.
Clicking on the “About” hyperlink took users to a page
called “Web Search Results” containing the paid inclusion
disclosure. Although most testers found the disclosure
simple and straightforward, one tester did not.
In fact, the failure of Ask Jeeves to acknowledge its ownership
of Teoma in the disclosure led one tester to
CONSUMER REPORTS WEBWATCH 31
FIGURE 8: ASK JEEVES
CONSUMER REPORTS WEBWATCH 32
mistakenly suspect deception in a sentence explaining
how some companies pay to have their Web sites included
in Teoma’s index: “This is a subtle use of paid inclusion
where the search engine uses the fact they are using
another company’s search technology to obfuscate paid
inclusion,” said the tester. This misplaced suspicion might
have been avoided had Ask Jeeves been more up-front
about its ownership of Teoma.
Continued in Part 2


	© 2002-2004 Home Page ; Iconocast offers eMarketing, Internet Advertising, Online Advertising, Internet Marketing, Online Branding, and eMarketing News Services.