 |
Internet Marketing · eMarketing · Internet Advertising · Online Branding |
|
Internet Marketing · eMarketing · Internet Advertising · Online Branding |
|
|
|
||
| ADVERTISING BY ICONOCAST |
||
FALSE ORACLES:
Consumer Reaction to Learning the Truth
About How Search Engines Work
Results of an Ethnographic Study
FALSE ORACLES:
Consumer Reaction to Learning the Truth
About How Search Engines Work
Results of an Ethnographic Study
A report by
Leslie Marable
Researcher/Writer
Consumer WebWatch
Based on field research conducted by
Context-Based Research Group
Released: June 30, 2003
2
AKNOWLEDGEMENTS:
Consumer WebWatch would like to thank the following people for their assistance in helping to
shape the early phases of this project.
Sambhavi Cheemalapati, Research Librarian, Information Center, Consumers Union
Denise I. Garcia, Principal Analyst, Media & Advertising, Gartner G2
Sandy Schlosser, New Media Project Manager, ConsumerReports.org
Beverly J. Thomas, Senior Attorney, Internet Advertising Program, U.S. Federal Trade
Commission
3
TABLE OF CONTENTS
EXECUTIVE SUMMARY........................................................................................................................................4
MAJOR FINDINGS:.......................................................................................................................................5
FIGURE 1: PARTICIPANT DEMOGRAPHIC PROFILES.......................................................................5
INTRODUCTION......................................................................................................................................................7
THE 15 SITES ASSESSED IN THIS STUDY: ..............................................................................................7
WHY FOCUS ON SEARCH ENGINES? .....................................................................................................8
WHAT CONSUMER WEBWATCH HOPED TO ACCOMPLISH: ..........................................................8
WHAT THE STUDY DOES NOT COVER:..................................................................................................9
STUDY RESULTS AND DISCUSSION.................................................................................................................10
PRE-ENLIGHTENMENT ATTITUDES AND BEHAVIORS...................................................................10
POST-ENLIGHTENMENT ATTITUDES AND BEHAVIORS ................................................................17
SUMMARY AND CONCLUSIONS.......................................................................................................................38
CONSUMER TIPS ..................................................................................................................................................40
RECOMMENDED GUIDELINES FOR SEARCH AND NAVIGATION SITES..............................................43
APPENDICES..........................................................................................................................................................45
APPENDIX A: BACKGROUND AND STUDY METHODOLOGY ........................................................45
APPENDIX B: THE MERITS OF ETHNOGRAPHIC RESEARCH.......................................................50
APPENDIX C: SEARCH ENGINE AND NAVIGATIONAL SITES EVALUATED, BY
PARTICIPANT..............................................................................................................................................52
APPENDIX D: “ABOUT SEARCH” AND DISCLOSURE LINKS REVIEWED BY PARTICIPANTS,
BY SEARCH ENGINE .................................................................................................................................53
APPENDIX E: INDIVIDUAL DEMOGRAPHIC PROFILES OF 17 PARTICIPANTS........................57
REFERENCES: .......................................................................................................................................................62
4
EXECUTIVE SUMMARY
A year after the U.S. Federal Trade Commission (FTC) determined there was a need for “clear
and conspicuous disclosures of paid placement” on search engines1, Consumer WebWatch
released data from a new study that shows consumers can’t always discern paid search from pure
search results.2 Key findings were released April 24, 2003, in New York City at Consumer
WebWatch’s First National Summit on Web Credibility.
This study, which tested 15 major search and navigation sites, used an ethnographic approach
that allowed researchers to observe experienced Web searchers in their natural surroundings.
Search companies have done their own usability-related studies, but the results are usually
proprietary. Context-Based Research Group, a cultural anthropology research firm based in
Baltimore, conducted the field research.
Four anthropologists recruited 17 Web consumers from four U.S. metropolitan areas (Kansas
City, Mo., Phoenix, Ariz., Providence, R.I., and Raleigh-Durham, N.C.). None of the
participants knew about pay-for-placement on search engines prior to joining the study. Each
candidate answered a pre-screener open-ended question that asked for his or her understanding
of how a typical search engine ranked its results. (See FIGURE 1: Participant demographic
profiles.)
Once admitted to the study, each consumer performed two online searches (one e-commerce, the
other informational) on five pre-assigned search sites while being observed and interviewed by
the anthropologist. A total of 163 completed searches were performed in which a participant
selected at least one link from the delivered results pages. (The 15 sites assessed in this study
were About.com, AlltheWeb.com, AltaVista.com, AOL.com, Ask.com, Go.com, Google.com,
InfoSpace.com, iWon.com, Kanoodle.com, LookSmart.com, Lycos.com, MSN.com,
Overture.com and Yahoo.com.) See APPENDIX A: Background and study methodology for
information on how we selected these sites.
After the planned searches, participants were told how pay-for-placement works on each search
site under assessment. To lead into this discussion, the ethnographer pointed out each of the
“disclosure” pages and “About Search” links posted on each site under evaluation and asked the
participant to read through them. (Many of these disclosure pages were located in areas geared
toward advertisers, not consumers.) One-on-one field interviews with participants, which each
averaged two hours in duration, were conducted pre- and post-enlightenment. After being
enlightened, participants were asked to do “homework” assignments, 10 additional searches on
sites and topics of their choosing, before a second field interview with the ethnographer.
(Participants were asked to record in a search journal their thoughts and behaviors recalled
1 Federal Trade Commission letter to Commercial Alert, “Complaint Requesting Investigation of Various Internet
Search Engine Companies for Paid Placement and Paid Inclusion Programs.” Available online at
http://www.ftc.gov/os/closings/staff/commercialalertletter.htm
2 Paid search (a.k.a., pay-for-placement, pay-for-performance, pay-per-click, paid listings): Guaranteed prominent
placement of a Web page link or ad in the search results of a search engine, triggered by keyword terms an advertiser
has purchased.
5
during these “homework” sessions.) The participants’ pre- and post-enlightenment online
searching was observed to monitor changes in attitudes and behaviors over time.
Major findings:
• Most participants had little understanding of how search engines retrieve Web pages or how
they rank or prioritize links on a results page.
• The majority of participants never clicked beyond the first page of search results. They
trusted search engines to present only the best or most accurate, unbiased results on the first
page. As a result, two-in-five links (or 41%) selected by our participants during the assigned
search sessions were paid search results.
• Once enlightened about pay-for-placement, each participant expressed surprise about this
search engine marketing practice. Some had negative, emotional reactions.
• All participants said paid search links on search and navigation sites were often too difficult
to recognize or find on many sites, and the disclosure information available was clearly
written for the advertiser, not the consumer. Search engine sites that were perceived to be
less transparent about these related disclosures lost credibility amongst the group.
FIGURE 1: Participant demographic profiles
Name* Age Occupation HH Income Family Type Web Use Web Years
KANSAS CITY
Lucy 24 Child Daycare Coor. $70,000 Married with child office 7
Jack 28 Graphic Artist $15,000 Single, no children home 8
Kareem 37 Restaurant Owner $30,000 Married with child home/office 5
Geena 48 Legal Asst. $35,000 Married with children office 10
PHOENIX
Yvonne 32 Physician's Asst. $76,000 Single, no children home/office 12
Penny 34 Stay-at-Home Mom $100,000 Married with child home 12
Gary 40 Environmental Compliance $50,000 Married with children office 8
Bob 46 Consultant, Geographic Info. $45,000 Single with child office 6
PROVIDENCE
Vivian 19 Full-time College Student $5,200 Single, no children home/school 7
Maria 27 Office Mgr., Non-Profit $30,000 Single, no children office 9
Dennis 40 V.P., Jewelry Comp. $160,000 Married with child home/office 9
Larry 62 Consultant, Marina/Boating $120,000 Married with grown children home/office 17
RALEIGH-DURHAM
Diane 20 Full-time College Student $10,000 Single, no children home/school 5
Mike 25 Electric Meter Reader $18,000 Single, no children home 5
Sara 30 Elementary School Teacher $40,000 Single, no children office 6
Claude 36 Construction Worker $35,000 Single with children home 6
Anna 55 Retired Banker $200,000 Married with grown children home 5
*Pseudonyms assigned to participants to protect their privacy. See also APPENDIX E: Individual demographic profiles
of 17 participants.
6
About Consumer WebWatch:
Consumer WebWatch is a project of Consumers Union, the non-profit publisher of Consumer
Reports magazine and ConsumerReports.org. The project is supported by The Pew Charitable
Trusts, which invests in ideas that fuel timely action and results; the John S. and James L.
Knight Foundation, which promotes excellence in journalism worldwide and invests in the
vitality of 26 U.S. communities; and the Open Society Institute, which encourages debate in
areas in which one view of an issue dominates all others. Consumer WebWatch’s Web site
launched April 16, 2002.
http://www.consumerwebwatch.org
Note: ConsumerWebWatch.org does not participate in any pay-for-placement or paid
inclusion/submission program on any search engine or navigation Web site.
About Context-Based Research Group:
Context-Based Research Group is an ethnographic research and consulting firm located in
Baltimore, M.D. with a global network of cultural anthropologists located around the world.
Professional anthropologists Robbie Blinkoff, Ph.D. and Belinda Blinkoff, M.A., A.B.D. created
Context in partnership with Chuck Donofrio, the President and C.E.O. of Carton Donofrio
Partners, Inc., an international brand experience design firm also located in Baltimore.
http://www.contextresearch.com
7
INTRODUCTION
Consumer WebWatch learned in its first commissioned study - a national telephone survey of
1,500 Web-savvy U.S. adults conducted by Princeton Survey Research Associates - that 60
percent did not know some search engines accept fees in exchange for giving prominent
placement to advertiser Web pages within the results pages.i In addition, four-in-five
respondents (or 80%) said it was important search engines disclose their paid search policies in
the search results or an easy-to-find page on the site.
A more recent Consumer WebWatch credibility-related joint study, lead by Stanford
University’s Persuasive Technology Lab, found half the consumers (or 46.1%) judged the
credibility of a Web site based, in part, on superficial aspects such as Web design or overall
visual appeal.ii These Stanford participants were more likely to assess a search engine site’s
credibility based on visual cues (52.6% vs. 46.1% overall).3 In addition, less than four percent of
consumers who assessed the 10 search sites in this Web-based study commented about
information bias. In comparison, 11.6 percent of information bias-related comments were made
about the study sites overall, and 30.2 percent of comments were derived while assessing newsspecific
sites, where the issue of information bias and sponsorship influence is presumably more
understood (and expected) amongst the public.
In this most recent study on search engines and navigation sites, Consumer WebWatch decided
to use an ethnographic or anthropological approach in order to dive deeper into how consumers
use these popular Web navigational points-of-entry. We commissioned Context-Based Research
Group, an ethnographic research and consulting firm based in Baltimore, to conduct this project.
Robbie Blinkoff, Ph.D., a managing partner and principal anthropologist, directed the project.
(See also APPENDIX B: The merits of ethnographic research.)
The 15 sites assessed in this study:
About.com
AlltheWeb.com
AltaVista.com
AOL Search (AOL.com)
Ask.com
Go.com
Google.com
InfoSpace.com
iWon.com
Kanoodle.com
LookSmart.com
Lycos.com
MSN Search (MSN.com)
3 The Stanford study asked participants to assess the credibility of 10 search sites, randomly assigned in pairs, that
were pre-selected by researchers for a variety of reasons, including Web design, user interface, brand name, and the
presence or absence of disclosure information. The search sites assessed between June 15 – August 15, 2002 were
About.com, AlltheWeb.com, Ask.com, Google.com, Insider.com, iWon.com, LookSmart.com, Overture.com,
Vivisimo.com and Yahoo.com.
8
Overture.com
Yahoo.com
Why focus on search engines? 4
Switching stations: Searchers use these navigational portals to move from one site to another,
particularly when the user is unsure of where to surf next. Findings from various research reports
show at least 85 percent of Web users query search engines with great frequency.iii Other
research indicates the percentage of search engine referrals worldwide has increased
significantly over the past year: up to 13.5 percent of global referrals from 7.2 percent the
previous year, according to online marketing analytics firm WebSideStory.com.iv (Search sites
in the U.S. account for 15 percent of all referrals, up from 8 percent a year ago.) Clearly, online
consumers rely on search engine and portal sites to find and to navigate Web pages more quickly
and efficiently.
Paid search factor: These advertiser-based results can nudge online consumers to click on Web
pages, links or ads listed prominently in the results, yet not necessarily the most relevant to their
search query. Even well informed or Web-savvy users may not know whether a listing is a paid
ad. In addition, in the wake of the dot-com advertising bust in 2001, and the decreasing
effectiveness of banner ads, online advertisers and marketers are anxious to find tactics that will
boost visibility with consumers. Keyword and other forms of paid search can make a significant
difference to a company’s bottom line. A recent Jupiter Research poll found that 76 percent of
marketing executives who have used search engine marketing rate it more effective than bannerstyle
online advertising.v About two-thirds (or 64%) said they intended to increase their search
engine marketing budgets in the next 12 months. (Indeed, a June 2003 report released by the
Interactive Advertising Bureau said keyword search made up 15 percent of last year’s total
Internet advertising revenues, up from 4 percent in year 2001.) According to industry analysts
U.S. Bancorp Piper Jaffray, total paid search revenues in the U.S. will approach $2 billion by the
end of 2003, then jump to nearly $5 billion by 2007.vi
Consumer detriment: There’s a higher risk to the Web consumer of making flawed buying and
other decisions as a result of reliance on information from a prominently ranked site that doesn’t
necessarily have the most accurate or most authoritative information. This risk factor is highest
when consumers search for information on health or financial issues in which actions based on
faulty, misleading or incomplete information could adversely affect their health or money
matters.vii For example, a November 2002 Consumers International study of 460 Web sites
found half the sites giving advice on medical and financial matters failed to provide full
information about the authority and credentials of the people behind that advice.5
What Consumer WebWatch hoped to accomplish:
4 Consumer WebWatch’s definition of a search engine: Any major site with the ability to point or navigate hundreds of
thousands of Internet users to different points of entry online, including search services, directories, portals and
community sites.
5 Consumers International (2002), Credibility on the Web: An international study of the credibility of consumer
information on the Internet.
9
• First, to elaborate on the Princeton survey statistics about search engines. Consumer
WebWatch wanted more confirmation Web consumers are still largely unaware of paid
search, and that such paid results might influence the ranking of their search results.
• Second, to assess whether consumers could tell the difference between paid search results
and pure or algorithmic search results, or not.
• Third, to assess whether consumers noticed and understood the “About Search” links or
related disclosure information posted on such sites a year after the FTC determined there
was “the need for clear and conspicuous disclosures of paid placement…so that businesses
may avoid possible future Commission action.”viii
• Fourth, to understand whether consumers’ online behavior would change once they learned
more about search engine pay-for-placement marketing and how related results rankings
might be biased toward advertisers.
• Fifth, to alert the search engine community that current marketing and advertising practices
could lead to long-term credibility problems for the industry.
• Sixth, to call attention to potential concerns or inadequacies in the FTC recommendations to
search engines to make their practices more transparent.
What the study does not cover:
The study did not focus on the concept of paid submission (a.k.a., paid inclusion)6 because such
techniques do not typically taint search engine result relevancy or rankings. In addition,
Consumer WebWatch thought that adding the concept of paid inclusion to the concept of payfor-
placement would only confuse and possibly frustrate participants.
Consumer WebWatch decided to focus on consumer understanding of paid placement, as its
influence can adversely affect search results positioning, thus potentially denying consumers the
ability to make the best buying and related household decisions while online. Further, basing a
health- or finance-related decision on inaccurate information can be literally dangerous to a
consumer’s well-being.
Secondly, the study did not attempt to determine whether the search results the participants
received or clicked on were in fact accurate or relevant to their query. However, we took notice
of occurrences in which the participant remarked to the ethnographer about the perceived
accuracy or inaccuracy of search results received during search sessions.
6 Paid submission: Submitting a Web site or pages for guaranteed inclusion into a search engine's database or listings.
This does not guarantee prominent listings or a rankings boost over those pages retrieved from the engine’s indexing
(crawlers), but it does guarantee a site is “crawled” more deeply or with greater frequently.
10
STUDY RESULTS AND DISCUSSION
Please note:
The names of participants have been changed. Participants were assigned pseudonyms to
protect their privacy and to encourage candidness with the ethnographer.
Excerpted ethnographer field notes and long quotations from participants appear in block
quote format. Direct quotes from participants are set off in double quotation marks.
Some participant quotes or ethnographer field notes were edited for grammar or clarity.
PRE-ENLIGHTENMENT ATTITUDES AND BEHAVIORS
1. All participants used one search engine as their sole or primary point of Web search
entry.
Thirteen participants reported using either Yahoo or Google for all or nearly all of their online
searching. (Google garnered eight loyalty votes, while Yahoo had five. AltaVista, Dogpile, MSN
Search and Search.com each received one vote apiece.)
Users were loyal to Google because they perceived it to deliver “relevant” and “satisfactory
search results.” Participants also mentioned its ease-of-use due to an “uncluttered” Web page
with “no distractions” like pop-up ads, images and a lot of text. Yahoo’s popularity amongst
participants was credited to its perceived delivery of “the best results,” “appearance and
accessibility” and “brand name and overall usability.” Three participants admitted heavy usage
of one particular search engine simply because it was the default Web home page setting on their
PC’s Internet browser.
For example, the Kansas City ethnographer reported Geena, 48, a legal assistant who does a lot
of work-related searches, has used Yahoo ever since she first went online 10 years ago:
When she first began using the Internet [Yahoo] was set as her default start page.
Since then she has not felt a need to change the way in which she searches on the
Web. It is also set as her default page at her workplace, however, she did not set
it as her start page. She explained that, “it was just like that so I use it.”
MSN.com is the default choice for fellow Kansas City participant Kareem, 37, a restaurant
owner and home/office Web user for five years.
MSN serves as the homepage on his computer and he therefore uses it the most
frequently…When he first used the Internet, and first got a computer, the
homepage was set to MSN, which is the reason he has used it as his primary
search engine all along.
11
2. Most participants did not understand how search engines retrieve information from the
Web or how they rank or prioritize links on a results page.
About half the participants (8 of 17) had a vague understanding that search engine results are
influenced by keyword or a closest match relevancy. In other words, they had a fuzzy idea
search engines relied on the keywords or phrases typed into the search boxes to find Web pages,
but didn’t understand much else.
For example, Penny, 34, a stay-at-home mom in Phoenix said “the search engine goes out and
looks at the keywords in the codes of Web pages and if the words match it pulls it down for the
results page.” Similarly, Vivian, 19, a full-time college student in Providence, suggested search
sites “pull up things that have the closest matches to what you typed in and then maybe it ranks
them in order of what page or where on the page the words show up.”
Two people characterized search and navigation sites as “something you can give a word to and
it goes out and finds matches that most closely match your word” (Sara, 30, elementary school
teacher, Raleigh-Durham) or “software that goes out on the Internet using search words to find
Web sites.” (Yvonne, 32, physician’s assistant, Phoenix)
Bob, a 46-year old Phoenix-based geographic consultant with six years online, gave this
response:
“A search engine is an interface with a Web page you go to in order to query
where to find information. You type in keywords and you get 100,000’s of
returns [that are] prioritized by how well your keywords match the Web page’s
keywords.”
Likewise, Dennis of Providence, 40, a jewelry-accessories company vice president with a
background in trademark law, explained:
“I would say [search engines] allow you to find information about a topic that is
located elsewhere on the Internet by clicking in key phrases related to what you
are interested in.”
Not surprisingly, some participants were more advanced in their understanding than others. Two
in particular, Gary of Phoenix (a 40-year-old environmental compliance coordinator) and Larry
of Providence (a 62-year-old marina and boating consultant) had the most sophisticated
understanding of how search engines work:
Here’s how Gary, who manages a personal home page, explains how search engines work to the
Phoenix ethnographer:
“[E]ach search engine is set up differently to do Web searches. Each engine has
an algorithm that determines how the engine’s ‘spider’ or Web ‘crawler’ (the
information retrieval thing) will chose the Web sites for the results page. Some
spiders are sent out to seek ‘meta-tags,’ but some reject ‘meta-tags.’ Some accept
hidden text, while others reject hidden text. It depends on the search engine’s
goals, which are determined by whoever runs/owns the search engine.”
12
Larry of Providence, who has experience compiling a database of Web pages related to
environmental issues, conveyed a higher level of understanding to the researcher:
“[It] operates on keyword searches, which might be words embedded in text. It
scans through and reads names of individual places and products and with some
systems, they will even list whatever keyword they think you want to have…That
is what gets registered with the search engine.”
Three participants (Jack and Kareem of Kansas City and Mike of Raleigh-Durham) had naïve
perceptions of how search engine result links are ranked, believing priority listing was based, in
part, on some kind of popularity- or seniority-related measurement.
According to the ethnographer’s field notes, Jack, a 28-year-old graphic artist with eight years of
online experience, assumed the results were selected based on the “most frequently visited Web
sites.” He also stated that Web sites ended up in the results based on their relevancy to keywords
in a query.
Fellow Missouri statesman Kareem, who also maintains a Web page for his small business,
“assumes” certain links are ranked higher than others because they are logged on more
frequently.
Mike of Raleigh-Durham, 25, an electric meter reader and part-time student with five years’
Web experience, suggested results were ranked based on seniority:
He believes the sites that are ranked higher are there “because they are the most
established or oldest. Perhaps because they have the most hits.”
Four participants (Geena of Kansas City, and Anna, Claude, and Diane of Raleigh-Durham)
admitted they had little understanding of how search engines prioritize results.
Anna, a 55-year-old retired banker, guessed search engines rank results based on the “more
words I entered are on that site.” Claude, 36, a construction worker with six years online,
believed search results are ranked, “listed by the best site.” And Diane, 20, an undergraduate
university student with five years of Web experience, thought search engines “scanned [the
Web]…and the first ones are just like you want it.”
While no participant knew about pay-for-placement on search sites, two participants suspected
results rankings weren’t based entirely on a pure search or algorithmic method.
The Kansas City ethnographer noted this about Lucy, 24, a child daycare coordinator with seven
years of Web experience:
She alluded to pay-for-placement a tiny bit when asked about the reason certain
items are ranked higher than others did in a search result, but then backed off. “I
don’t know why, I guess because of pay or something, or actually it’s because of
keyword I think.”
Likewise, Providence resident Maria, 27, an office manager for a non-profit art organization
with nine years online, suggested there were factors at hand that drive search results other than
unbiased or algorithmic rankings.
13
“I always [thought]…it was depend[ent] on what the search engine wanted to
present to people and also what the places [Web sites] kind of give to the search
engine. I don't know if it is a money deal or whatever, I don't know if they
exchange [money], but it has something to do with helping each other out.”
3. The majority of participants never clicked beyond the first page of search results as they
had trust in the search engine to present only the best or most accurate results on the first
page, making it unnecessary to review later results pages.
Each of the 15 search engines under evaluation was assessed by a minimum of five participants.
A total of 170 unique search sessions were performed during the planned browse phases of Field
Interview #1, an equal number of commercial searches and informational searches for health,
finance or travel information (i.e., 85 searches of each type.) Seven searches were aborted before
a result link was clicked because the participant reported he or she didn’t see a link of interest to
them, or because the results were redundant to those seen on other search engines tested. (There
were two aborted searches on MSN.com and one apiece on About.com, Go.com, InfoSpace.com,
iWon.com, and Kanoodle.com.)
(See APPENDIX C: Search engine and navigational sites evaluated, by participant.)
Not one participant explored search results beyond the fifth delivered page. Of the 163 searches
in which a participant selected a link to explore, the majority or 88 percent of the result links
selected were located on the first page. The number of pages explored by participants decreased
considerably after page one. There were only 16 cases in which a participant viewed the second
page of results, two cases where the third page was reviewed, and one case in which the fifth
page was scanned.
There was a slight difference between commercial and informational searches. Of the 144 cases
in which a link was selected from the first results page, 75 were picked during commercial
searches, while 69 were chosen during informational searches.
Prior to the planned browse searches, in which the ethnographer probed a participant on his or
her searching behavior, pre-enlightenment, most participants had a fairly accurate depiction of
their online behavior. The majority (or 12 of 17 participants) stated they only clicked on links
listed on the first page of search results, which later proved to be accurate once these statements
were compared to their actual behavior displayed during respective planned searches.
Two participants, Anna and Diane of Raleigh-Durham, were overly conservative in what they
reported to the ethnographer. While each stated she typically stopped at the first page of results,
in three of 10 cases during the planned search each reviewed two or three pages of results,
certainly far more than the group on average. (Anna assessed About.com, AltaVista.com,
Google.com, MSN.com, and Overture.com. Diane assessed Go.com, InfoSpace.com, iWon.com,
Kanoodle.com and LookSmart.com.)
Another participant not only reported he was diligent about reviewing a number of pages before
making a link selection, but his actual behavior during the planned search bore this out. Kareem
14
of Kansas City’s exchange with the anthropologist indicates how thorough he is when reviewing
the results he receives:
He typically scrolls through several pages, “at least three,” before choosing a
result, however, he said it ranges from one to six [pages]. He explained, “because
when I’ve found my appropriate search phrase, everything will be in those results
pages. I click on lots of pages because there’s lots of good information.” He
scrolls through anywhere from “one to twenty to fifty” actual results before
selecting a link.
True to form, an analysis of his 10 planned searches revealed Kareem had clicked on a first-page
result only three times. He reviewed through the second page of results half the time (5 of 10
times), reviewed through the third page of results one time, and through the fifth page one time.
(Kareem performed his 10 planned searches on AOL.com, AlltheWeb.com, Ask.com,
Lycos.com, and Yahoo.com.)
There were, however, two participants who said one thing, but actually did another. (i.e., there
was a disconnect in reported behavior versus actual behavior.) The first is Lucy of Kansas City.
According to the ethnographer’s field notes, Lucy reported the following:
She always scrolls through all the results, glancing at the descriptions quickly,
before making any decisions. Then, she zeroes in on the one she wants and reads
the details and sometimes the Web address, if it is offered. She explained that,
“the top one isn’t always the one I want so I look through them all.” I noticed that
although she looks through “them all,” she only looked on the first page. When I
asked her about that, she concurred, stating that she “almost never goes beyond
the first page because it would take too much time.”
It seems the answer extracted from Lucy, triggered by probing from the anthropologist, proved
to be most accurate. A review of Lucy’s planned searches, in which she assessed AOL.com,
AlltheWeb.com, Ask.com, Lycos.com and Yahoo.com, showed she only selected links from the
first page of results.
Similarly, the Raleigh-Durham researcher recorded that Mike said:
He typically will “either pick a result on the first two pages, or by the third I may
skip to the farthest one I can find, just to see what is there.” He will search
through three pages of 25 results for a more directed search.
His actual planned searching behavior, however, indicated he never went beyond the first page
of results served by assigned sites Go.com, InfoSpace.com, iWon.com, Kanoodle.com and
LookSmart.com.
Perhaps more importantly, four participants justified their behavior of stopping at the first page
of results because they assumed a search engine would present only the most trustworthy Web
sites first, therefore there was no need for them to read beyond the first page.
According to the Providence anthropologist, Larry generally doesn't scroll through any search
pages:
15
Larry chooses one of the first five or so links he sees on his first search results
page…If Larry doesn't see anything “in the ballpark” of what he is looking for,
he will restructure his search because he believes results are ranked by relevancy
and the frequency of the occurrence of his search term in the results that are
generated. If he doesn't find what he wants in the first five, he believes that he
won't find them further down the screen or on subsequent screens.
Yvonne justified her behavior to the Phoenix ethnographer in this way:
Often she doesn’t even scroll down to view all of the results on that page and
makes her link choice from the links at the top of the page. It’s okay to do this
because the search engine has prioritized the links to have the best matches at the
top.
Two Kansas City participants expressed similar blind trust of search engines:
Jack claimed, based on his assumption that the first results are the Web sites most
frequently visited, that “the most trustworthy sites are the first to come up.”
When using Google.com, he chose a sponsored link at the top of the results page
because he said it looked “trustworthy.”
The same ethnographer wrote these notes in reference to Geena:
She does not think there is any difference between the results provided by
different search engines.
4. Nearly half the links selected by participants were paid search results.
Of the 163 searches in which a link was selected, 67 results (or 41%) could be classified as paid
search. These included links euphemistically labeled “sponsored” link, listing, match, or result,
and “featured site” or “featured sponsor.” Most results generated by “pay-for-performance” site
Kanoodle were coded as paid search results7, as well as all results generated by Overture,
whether these Overture results came directly from its home page or picked during a search of
partner sites Go.com or InfoSpace.com. Similarly, links that could be attributed to the Google
AdWords or Sponsored Links programs were coded as paid search, including some results that
appeared on partner sites such as iWon and InfoSpace.com.
Slightly more paid links were selected during the commercial searches compared to the
informational searches. A total of 36 links were selected during pre-enlightenment e-commerce
searches. Fewer paid links (31 to be exact) were selected during informational searches. On the
other hand, there were slightly more pure search links selected during informational searches.
(50 pure links selected during informational searches vs. 46 pure links selected during
commercial searches.)
(See the POST-ENLIGHTENMENT ATTITUDES AND BEHAVORS section for
information on consumer perceptions of pay-for-placement results received during
commercial vs. informational searches.)
7 Other results “powered by Google” were coded as “pure” search results.
16
There were some cases in which it was difficult for the Consumer WebWatch researcher to
determine whether a selected link should be coded as paid versus non-paid. This is because the
disclosure and “About Search” page language for a particular engine was so vaguely worded it
was impossible to determine what the site classified as a paid search result. That said, the
researcher coded 96 links (or, 59%) as derived from pure or algorithmic generated results,
including those Web pages in which there was reason to believe they were present due to paid
inclusion or paid submission programs.
5. Very few participants noticed the paid search labeled “sponsored” links or listings, but
only when they were easy to see (i.e., at the top of the search page or clearly marked in a
box).
These same participants reported skipping over links labeled “sponsored” because they assumed
these results didn’t have what they wanted, even though they couldn’t always articulate why.
For example, the Phoenix ethnographer reported this about Gary:
When he did the search on About.com (a site he had never used before), he
quickly skipped the top section of results. On Google, he skipped over the
highlighted results and behaved similarly on the other sites. He said he skipped
those results because they seemed less (or not at all) relevant. That is when he
started noticing the sponsored link notes and other clues to pay-for-placement
and recognized those links had paid to be included in searches.
The same researcher noted:
Penny skips over sponsored links on any results page, not because they are payfor-
placement, but because they don’t look right. Because they are highlighted or
separated or made to stand out somehow, they don’t fit into the format that she
expects on the results page, so she ignores them.
In a different city, the Providence ethnographer recorded similar online behavior for Vivian:
She says she never looks at sponsored links, though she didn't know they were
paid for. When we talk about premium sponsorship on Google, she says she
never looks at the information at the top. “I never actually look at them very
closely, but I know they are trying to sell me something. I think that these are
advertising and that these (points to sponsored links on side) are the same, so I
just never look at them.”
The Providence researcher noted Larry also ignored sponsored links, such as those labeled
“sponsor matches”:
“I don't look at those as a general rule if it is in a separate box as I know it is a
sponsored thing. Well, you know, I'll give them a quick glance, but I know that
those aren't what I want.”
6. Paid search disclosures were overlooked in every case.
17
The four ethnographers noted none of the participants in his or her city noticed the “About
Search” or other disclosure information pages on the 15 search engines under assessment during
the 10 planned searches conducted pre-enlightenment.
POST-ENLIGHTENMENT ATTITUDES AND BEHAVIORS
1. Each participant expressed surprise after learning about this search engine marketing
practice. Some had negative, emotional reactions.
Sara of Raleigh-Durham reacted emotionally when told about pay-for-placement. The ethnographer
recorded this exchange:
“Well, that figures!” I believe she may get a little depressed about this
information. She remained quite melancholy throughout the rest of the visit. I
could tell her mind was rolling with the repercussions of pay-for-placement. She
seems concerned her students [will have to grow up and have] to learn how to see
bias while using a computer.
Similarly, fellow North Carolina resident Anna said she was concerned about those more
vulnerable to information bias, like children:
“I expect information bias and, therefore, take it into consideration when making
decisions because I’m old and jaded.” She later added, “I feel like there are
people, like family, that are more gullible and [will] accept anything they read on
the Internet as gospel.” When I asked, whom, she responded, “My mother, my
daughter, and other people’s families.”
Four participants reacted angrily or expressed a sense of betrayal, including Kareem of Kansas
City:
Upon enlightenment, his face contorted a bit. He expressed immediate dismay,
stating, “Well, that explains it. No wonder I run into so much [expletive].” Yet,
he tempered his displeasure quickly, stating, “I expect it though, I guess I kind
of suspected something like this. I just never thought too much about it. It just
annoyed me.”
Mike of Raleigh-Durham also had an angry reaction:
I found he is not happy he has to “be careful” when searching “so I don't get sites
I don't want,” the ethnographer recorded in the field notes.
Geena of Kansas City was taken aback at first, and then she blamed herself for not noticing
disclosure information sooner:
When we visited the disclosure links she was somewhat stunned. She seems to
be a very, very trusting person that thinks the best of everyone and everything,
18
so she was shocked. She felt extremely betrayed. She expressed a slight bit of
disappointment with herself because she said she should have visited the
disclosure link sooner, but then brought the site to task for not disclosing it in a
more explicit manner.
Similarly, Claude of Raleigh-Durham expressed a sense of betrayal and then became disturbed
when he suspected this practice might have influenced his more serious informational searching.
The participant insisted informational searches “are no place for advertising.”
The participant made an excellent point when reflecting on pay-for-placement.
“People like me may only get to learn stuff online, and if we can’t then that’s just
as biased as having to pay lots of money to go to some college to learn.”
The ethnographers recorded three exchanges with participants that indicated they were confused or
disappointed.
Vivian of Providence expressed disappointment, but like Geena, blamed herself:
“Oh, I'm such a sucker!” (She laughs.) I tell her she isn't [a sucker], but that she
might want to know this when she is shopping or looking for information. I also
point out the different headings for search results as they are listed on sites like
iWon, LookSmart and Kanoodle. She hadn't noticed these before. “My little
fantasy world is wrecked!”
Diane of Raleigh-Durham felt somewhat defenseless, as if unsure what to do next:
“I feel helpless and must tolerate it. It’s annoying everywhere it happens, but
what can you do about it.”
Lucy of Kansas City was confused at first, until she recalled a previous experience:
When Lucy and I visited the disclosure pages on the search engines, she was a
bit puzzled, at least the look on her face seemed to be a mix of bewilderment,
confusion, and displeasure. She expressed a sort of 'ah-ha' and continued, “So
that is why I kept getting sent to eBay and other annoying product sites I didn't
want.” She then expressed her appreciation for being informed of this fact, but
then went on to chastise herself for not ever looking at the disclosure links on
the search engine homepages.
Two participants questioned the trustworthiness of advertisers that choose to participate in search
engine marketing programs.
Gary of Phoenix called into question the ethics of advertisers that pay-for-placement:
“They are being placed there because of money, not content. I am after content.”
He summed it up by saying, “the bottom line is, if you’ve got a good Web site
don’t pay. Let the search engines do their thing.”
Providence participant Dennis questioned the ethics of some Web companies. He expressed
confusion about what is paid and what is not paid, in this case on a Google results page:
19
Dennis doesn't click on sponsored links on Google because “they are biased.
They are paying Google to be there.” He appears to still be confused. “The
advertisements are over there (motions to right side of an imaginary screen), and
the results are over here. Whether they mix them in there, I'm not sure. To get
on the first page maybe they do? Because sometimes I have seen, like, why is
this on the first page when what I really need is on the third page?”
The last six participants, while initially surprised about pay-for-placement, eventually came to a
more philosophical or realistic outlook. They characterized search engine marketing as a sort of
price to pay, like other forms of Internet advertising that keep online information free.
The Phoenix anthropologist recorded this response from Bob:
Being as skeptical as he is, he said he “figured” the results were influenced that
way, but he had never bothered to investigate. He was very interested in the
‘about the search’ pages. “[I]nformation bias is tolerable because any venture has
to be profitable and if the Internet wasn’t profitable it wouldn’t exist.” Anyway
sometimes he’s looking for commercial sites so it’s good that they pay to get to
the top. He said, “Even if the sites didn’t pay-for-placement, the listings would
still be biased somehow.”
Maria of Providence responded in a similar fashion:
Maria seemed surprised by the information, but not outraged. As she said, “I
understand how it works.” She added, “I kinda had an idea a little bit that that was
how it works, but I didn't understand how money was exchanged or how exactly
everything is going together. It is interesting, I didn't realize how much of the
beginning pages [were] money pages, like people paying. To find the meat of
what it is that I want, what I really want, I have to look farther [down the search
results.]”
Jack of Kansas City, said he expects information bias to occur in the commercial realm:
Jack seemed unfazed. He said, “Well, I guess I should have known.” He went on
to express his displeasure with pay-for-placement, stating he “needs to look more
into it because my art [stored on an online gallery site] will never get seen
because big sites will always get visited first.” Jack seemed a bit unsure, but
threw out the idea that some corporations are [more] forceful with their money,
which leads to some bias [on search engines.]
Yvonne of Phoenix had this reaction:
Yvonne’s searches were a great lead in to the pay-for-placement enlightenment.
Even though she had noticed the different [pages] of Amazon were coming up in
different order, she still hadn’t noticed some were listed as “sponsored” or
“recommended” sites. She said learning about pay-for-placement is “kind of a
big bummer.” She was not really surprised, but a little disappointed. She
elaborated, “It’s a little bit disturbing, especially when I’m searching for
information, I don’t know. I guess I expect to be duped when I’m searching for
things [to buy], but then again when I put in a specific string I don’t want that
string to get dummied down or dulled by the fact that somebody paid money to
20
get to the top of the pile.” She compares [her post-enlightenment knowledge of]
pay-for-placement on search engines to commercials on television. “It’s like
commercials on television. I don’t particularly care [that they are there], but it’s
good to know the information you are getting is being filtered through that lens.
Then you can adjust your perception of that information. You are buyer beware
because you are aware.”
The same Phoenix researcher noted a similar kind of exchange with Penny:
She was very realistic about pay-for-placement saying, “It’s just like going to the
mailbox, you just have to filter through the junk—there might be a check for
$1,000 in there, but there will also be at least four pieces of junk mail.” Penny
defines information bias as the presentation of material in a manner to achieve a
desired outcome. “It’s what people want. They have a particular outcome in mind
so they present information in a particular way.” Though Penny isn’t fond of all
the advertising on the Internet, especially the flickering ads, she believes that it is
worth tolerating because it allows access to the Web sites to remain free of cost.
Providence resident Larry was the sole participant to express an interest in the disclosure
information on the search site so he could use it for personal business purposes.
Larry's reaction to enlightenment about pay-for-placement was that of a
businessman…Once made aware, he wanted to use it himself. He hadn't
understood how Google made money. We explored this (how they sell their
search services to AOL, to Cox Cable, etc.) and… pay-for-placement. As we read
the descriptions of different pay-for-placement programs on [Google], I could see
the gears turning and hear his comments, mostly about how he could use this
method to get people to look at his not-for-profit Web site and Web-based
database for information about marinas he writes for the Federal government.
2. No participant stopped using a favored search engine after learning about pay-forplacement
practices, although they reviewed search results with a more critical eye.
For example, Sara of Raleigh-Durham thinks, post-enlightenment, the results across search
engines are generally “all the same, but I’m just more in tune to it, or mindful.”
Some participants reported they scrutinized the results pages more carefully. Jack of Kansas City
tends to scroll through more options and go to more pages to look for a site to choose, according
to a verbal report to the anthropologist.
Some participants used more than one search engine before making a final decision to click a
link, as was the case of Lucy of Kansas City:
Since her enlightenment [Lucy] has begun to use more search engines than
before because she was previously unaware of all the different engines, but also
to compare their results. Prior to enlightenment, Lucy never used a search engine
other than Yahoo, and never “went past the first page of results.”
21
Some participants used a combination of methods to compare results across engines before making
a decision to click on a link. For example, Geena of Kansas City reported viewing more results
pages across multiple search sites.
[Geena] has become a much more astute and observant searcher and Web user.
She went on to explain, “I have begun to go through more pages of results. When
I went to look for cruises the other day, I noticed the same results came up
multiple times at different spots in the results section.”
Similarly, Yvonne of Phoenix reviews more results pages and checks non-paid links before
making a decision to click:
After learning about pay-for-placement she intends to now look at the second
half of the results page. “I would never go beyond the Top Ten so I’m [most
likely] only getting biased information.” She looks forward to her future Web
searches and knows she will look at the search results with a more critical eye.
She is sure to check the unsponsored links before she makes her first click.
Another participant sought and read the disclosure information more carefully. After reviewing a
homework search conducted by Maria of Providence at work, the ethnographer made this
notation:
It was clear Maria was looking at some of the disclosure pages on different
search engines, and thinking more critically about where the information that was
generated was coming from. Maria now has a better understanding of why the
“mom and pop stores” are listed lower in search results than large corporations -
because of pay-for-placement.
As previously reported, Jack of Kansas City shared the same concern about smaller businesses
being shut out of paid search, or at least segregated to the last, and mostly unseen, results pages.
3. The participants were more tolerant of pay-for-placement during commercial or ecommerce
searches (yet not informational searches), but only if paid links were clearly
marked as such.
Twelve of 17 participants viewed paid search listings as a helpful tool during e-commerce
searches, as it helped them quickly find vendors who sold the products or services they were
looking for.
Larry of Providence told his ethnographer the following:
“If I am looking for a product, then that is fine and I will use [sponsored links],
but often I am just looking for information and understanding, and I’m not
looking for products so [sponsored links] become extraneous garbage I’ve got to
skip over or through. But when I’m doing Christmas shopping, however, I pay a
lot of attention to the top first hits, and the ones that are on boxes because they
may lead me to a [Navy] Peacoat or whatever it is I’m looking for.”
The Raleigh-Durham researcher made this notation about Diane:
22
The participant did not respond well to any prompt for money while performing
her informational searches. She would end her search if this happened instead of
returning to her results to look for another site. When performing her commercial
searches, the participant showed no amount of frustration related to advertising.
She seemed to accept it and often chose sites that were categorized as sponsored.
Vivian of Providence exhibited similar online behavior, according to the anthropologist:
She says she generally ignores sponsored links, when they are labeled
“sponsored” unless she is looking to buy something specific, and the sponsored
link is exactly what she needs.
Geena of Kansas City reported this to the field researcher:
She expects bias in advertising and therefore tolerates more information bias in
the commercial realm. She said she is not really bothered by bias on the Web
because she is “interested in finding the best deals and now it’s more competitive
for airlines.” If she encounters biased, unreliable information, she goes
elsewhere.
The Kansas City researcher also noted this comment from Jack:
He continues to go to sponsored links if he is searching for a product. He
qualifies that thought, stating, “at least now I know they’re pay-for-placement. I
know they’re paid for and I still go to them.” If he is searching for information,
rather than a product, he does not go to sponsored links because he does not
expect to find unbiased and useful information on sponsored links.
Dennis of Providence and Claude and Anna of Raleigh-Durham each expressed tolerance of paid
search during commercial searching, according to the respective field anthropologists:
Dennis ignores sponsored links if he is looking for “general information” about a
topic, and isn’t interested in buying a product. He will click on a sponsored link
if he thinks it is “exactly on target” and he is interested in buying something.
Claude said he “does not want to be given incorrect information when searching
for anything [non-commercial because he] won’t be able to find out the truth.”
Anna said, “If I’m researching an issue, these activities are [negatively]
influenced more than if I’m looking for a product.”
Two participants became irritated when a health-related information search delivered paid results
when they had no intention to make a purchase.
Sara of Raleigh-Durham was one such participant:
While she will “moderately tolerate” bias “in a commercial sense” she found bias
“intolerable in the informational sphere.” The ethnographer noted he observed
some very hostile body posture and language during one of Sara’s informational
23
searches. The participant had a personal investment in her informational search.
If she received the wrong information, then she felt it could affect her health.
Maria of Providence had a similar reaction when looking for information about diabetes:
The results Maria found were mostly from drug companies, which she
recognized as paid placement. I asked her how she felt about finding this much
paid placement. “I felt kinda weird about it. I just wanted to get the facts, and
that’s not what I got. It’s that same feeling you get in a store when someone just
keeps asking you if you need help and you don’t, you just want to look. It’s the
same feeling. I don’t want to look at advertisements. I just want to find out a little
information about [the topic.]”
Two study participants reacted similarly when it came to finding travel planning-related
information searches.
Lucy of Kansas City expressed the following to her researcher:
She said when she does certain searches that invite commercial responses she has
to be more critical of the results. She also explained when doing a travel search
she feels she has to be careful about putting in too general of a search query
because it will result in too many pay-for-placement results.
Bob of Phoenix made this entry in his search journal, the result of a homework search for
information about Hawaiian vacations:
“I was looking for long-range planning info not immediate booking info. The
links all wanted you to spend money by credit card now. They’re looking out for
opportunity with short-term time horizon.”
One Kansas City participant, Kareem, held a contrarian view:
“It’s like selling underwear in an auto parts store.” This comment, in addition to
the rest of his statements during our discussion and his search behavior, leads
me to believe he sees [paid search] advertisements as non-threatening or
irrelevant during his information/non-commercial searches because they are so
out-of-place they do not tempt him or even get his attention.
4. The participants unanimously believed paid search links were too tough to recognize on
many sites, and the related disclosure information was clearly written for the advertiser, not
the consumer.
Paid search disclosures not easy to find:
Dennis of Providence provided the most poignant response, which he jotted down in his search
journal after homework on Overture, even though it could easily apply to most sites evaluated for
this study:
24
“User should not have to dig into layers of information to find disclosures on
whether someone is paying to be listed on the search results page.”
Dennis continued by suggesting the search engine industry could provide more transparent paid
search disclosures if it modeled them on what already appears in print media.
“…I think advertising is O.K., anywhere in any form, as long as it is disclosed
that it is advertising. For instance, if you look in print media there are obvious
advertisements. [If] there is a product, and when there is an article that is buried
in the newspaper that looks like an article, like part of the newspaper, it will say
‘advertising’ all across the bottom so you know. And above it, it will say ‘paid
for by advertising.’ That’s what they do in newspapers.”
Lucy of Kansas City also found it very difficult to see disclosure links during her online
experiences.
“I had never even noticed it before [the ethnographer] showed it to me.
Beforehand, it wasn’t clear to me what sponsored links were or what it meant,
but now, even after you showed me, it’s still vague because I don’t know who’s
sending it.”
Visual aids make paid listings easier to see:
Dennis also suggested most search engines could use more visual aids to help consumers discern
which parts of results pages are paid versus unpaid. During his follow-up interview with the
Providence ethnographer, he suggested visual aids such as “boxes” and “bright colors” and “a
separate, distinct entity that is conspicuously an advertisement.” He added, “I’ve seen recently
search engines make them a little insidious and not so conspicuous.”
Many participants echoed Dennis’ recommendation to make paid search far more conspicuous to
the online consumer eye. Suggestions included the use of symbols or icons, placed next to a link or
above an ad, to make it clear an advertiser paid for its prominent ranking on the results page.
Larry of Providence also thought “a little logo” with a “this is a paid advertisement” label would
convey paid search to the typical consumer.
Sara of Raleigh-Durham suggested placing a dollar sign ($) beside each paid search site on a results
page. “That way if I would like to look at those sites I can and if I want to skip I can, but at least I
have the knowledge to make an educated decision. I want complete disclosure no matter what. It
would help me save time so I don’t go to that site and get disgusted.” She also suggested the
following scheme, based on a search engine’s editorial review, which would help consumers figure
out which links are skewed more commercial:
“‘C’ for commercial and ‘I’ for informational next to each site. Next to the ‘C’
sites it would say ‘We do not endorse this site in any way. Use your own
discretion.’ It would be for educators and students to use because of the
information slant to it. It would be very clear and it would be quicker and easier
for those looking for information.”
25
As with Sara, Vivian of Providence believed the presence or absence of a price tag symbol next to a
link could best connote to consumers that it is the equivalent of paid advertising.
Bob of Phoenix also supported what Sara suggested, that is, the clear labeling of purely
informational pages “so they are easier to find, rather than paid ads to be labeled.” Likewise,
Claude of Raleigh-Durham suggested “an icon or symbol helping you to know it’s an Informational
or Commercial site.”
Other participants echoed the need for color to distinguish paid search:
“Orange lettering or some different color [to let me know to avoid paid search].”
(Kareem of Kansas City)
“Highlight all ‘pay-for-placement’ and label as such.” (Sara of Raleigh-Durham)
Some consumers underscored the idea that physically separating paid search from non-paid search,
if on the same results page, was the best way to go.
According to Diane of Raleigh-Durham, search engines “shouldn’t intertwine [paid search] with
your [algorithmic] results. They should put it separate.”
Fellow state resident Anna also said she would prefer a more clear delineation between paid and
unpaid search results:
The participant [Anna] claims to be able to recognize paid placement information
“if it’s labeled in a section of your search results as sponsored sites. If it’s
blended into my results, I don’t know it. I would like for them to indicate
sponsored sites by grouping up first, or putting an asterisk next to it. It probably
would be easier to do it that way. Keep it simple. It would simplify my search if
it were consistent [across search engines], like a symbol or a graphic.”
Explicit disclosures needed to explain how search engines rank results:
Gary of Phoenix asked search engines to “make it easier to find the ‘about the search’ information.”
Geena of Kansas City demanded:
…More explicit and clear explanations of the order of results disclosed, which
explains why “number one is one, two is two, and fifty is fifty.” She feels there
should be explicit disclosure on any sites that use any variation of pay-forplacement
and that there should be some type of indicator to show purchased
keywords.
Sara of Raleigh-Durham expressed a similar sentiment, although she wanted search engines to
go a step further. She believes consumers should receive “a notice [that] should pop-up before
[the search] results to say the first 20 results [for example] are pay-for-placement results and are
not necessarily the most appropriate links to your desired info.”
Bob of Phoenix was in favor of a seal-of-approval approach:
26
He suggested some sort of disclaimer for each link or page showing who put the
page together, the source of any information, and its business practices. Maybe
there should be a [third-party] network that ethical companies could join and
mark the links to those sites.
Disreputable advertisers should share the blame with search engines:
Vivian of Providence was not sure of how best to convey paid search to consumers, in part,
because some of the companies that participate in search engine marketing may not be honest
when it comes to the keywords they buy.
“I think part [of the problem] is that people’s ads are really sneaky, so it’s not
like a commercial where you can clearly tell it is different from the show you are
watching, or a billboard, clearly it is on the side of a building.”
As with fellow Rhode Island resident Dennis, Vivian made reference to advertising disclosures
used in offline situations, according to the researcher notations:
[Vivian] makes comparisons to “special advertising sections” in magazines that
are sometimes difficult to tell are ads, even when they are labeled.
5. Nearly all participants said the commonly used paid search term “sponsored” was
confusing and potentially misleading to the consumer. They also expressed frustration a
better label had not already been adopted as an industry standard, and used consistently
across all search and navigation sites.
“Sponsored” label is too vague:
The majority of participants thought the commonly used labels “sponsored link,” “sponsored
listing,” “sponsored results,” “sponsored search listing,” or “sponsored match” were too vague and
easily misinterpreted. Other labels such as “featured sites,” “partner sites,” and “additional listings”
were universally panned as being confusing.
For example, Kareem of Kansas City said he considered the term sponsored to mean “something
that someone has paid for,” but he admitted, “until now, pay-for-placement wasn’t in my lingo, so
before now I didn’t think sponsored meant pay-for-placement.”
Fellow Kansas City resident Geena told the same ethnographer she believes the word sponsored
“should be changed to ‘paid’ because it sounds too benign.”
Paid search label euphemisms were also problematic, in this case for Maria of Providence:
The word “featured” was confusing to Maria. She wasn't sure if it meant paid for
or not. She would not like this term to be used.
Not surprisingly, the participants were vocal about recommending better paid search labeling. The
most commonly suggested terms were “paid advertisement” or “advertisement.” One person
27
(Yvonne of Phoenix) thought “advertised link” most clearly stated the motivation of the company
behind the paid search ad.
In addition, Jack of Kansas City recommended that such label terminology appear “at the bottom of
the search engine’s main page, not on another link. It should be no longer than a paragraph and it
should be at the bottom of the page no matter the search.”
“Sponsored” label is misleading:
A number of participants thought the term “sponsored” suggested to consumers the company
behind the paid search ad had philanthropic motivations, rather than those more commercial in
nature.
According to Yvonne of Phoenix, “the term sponsored probably isn’t the best word to be used to
denote pay-for-placement links. If I were naïve, sponsored would mean they were donating to [a
company’s] charitable arm or something.”
Fellow Phoenix resident Gary and Raleigh-Durham resident Diane agreed:
“Using the word sponsored sounds almost charitable.”(Gary)
“Sponsored sounds like charity, so I don’t like that as a possible label.” She
prefers the term “‘Paid Links’ because it reminds me more of businesses and
money, which is what they’re trying to get from you.” (Diane)
The responses of other participants suggested they assumed a positive intent on the part of the
company behind the paid search ad.
For example, the Providence ethnographer recorded this exchange with Maria:
Maria recognized the term sponsored as conveying “some sort of money deal.”
She based this on how [her non-profit employer] accepts sponsorship from
Dunkin’ Donuts and other large companies. She did, however, see sponsorship as
a kinder term than paid. This ties into Maria's explanations during the first
interview of Web sites “helping each other out.” [Maria later explained]: “I think
sponsored is a pretty good word. I don't know if it's really a ‘helping word’ but it
kinda shows [companies that participate in paid search] are helping [search
engines] keep [their] Web site going. So that is fine. It’s again part of our society.
People need money to keep going.”
Fellow Providence resident Larry associated the word sponsor with charitable organizations.
“I get sponsors for my programs (Boy and Girl scouting), so I know what they
are. Sponsor to me is someone who gives money to help support a program that
is going to happen. A community play has sponsors who donate $50 to help pay
the costs. And, in a commercial program there are sponsors on TV like Ford
Motor Company or a soap company or whatever.”
28
The Providence ethnographer recorded a verbal exchange, this time with Vivian, that suggests
she, too, assigns positive values to search links from companies bearing a sponsored paid search
label.
Anthropologist: What does sponsored or similar terms convey to you?
Vivian: It is a Web site that contributes money to the engine, not necessarily.
Well it’s less harsh than saying they pay money to them to be Number One, but
there is a reason why this site did come up in the search results.
Anthropologist: So it’s a less harsh term than paid for placement?
Vivian: Yeah.
Anthropologist: So sponsored – what other ways do you use that term or hear it
used?
Vivian: What do you mean?
Anthropologist: I’m just trying to figure out how you think that sponsored sounds
less harsh.
Vivian: Well, when I think of sponsored, I think of PBS [TV’s Public
Broadcasting Service] and or like, you know, you sponsor someone in a race. It’s
more of a friendly term if I think about it, I guess.
Anthropologist: And that is the term that you think would be good?
Vivian: Yeah, I think so.
(We discuss a little more why she likes this term.)
Vivian: I would understand what it is…it would make more sense for the actual
engine to put [sponsored] up rather than “paid for by.” It might be more honest if
it was rigorously truthful about where it was coming from, that might be more
honest, but I don’t think that anyone would do that. I mean if InfoSpace put up
links that said, “these are sponsored” I might still look at it, but if [InfoSpace]
said everything was ranked based on money, then I probably wouldn’t use the
site at all.
Anthropologist: Does the word sponsor convey to you pay-for-placement?
Vivian: It conveys to me there is money involved and that the search engine isn’t
out – its sole purpose isn’t to help me.
Industry-approved, standard labeling for paid search ads:
Two Providence residents said search engines should be more editorially consistent.
According to Maria:
“There should be a uniform category for all sites that are paid for. It shouldn't be
all these different categories like ‘featured’ and ‘sponsored’ and ‘topics.’ It
should just be cut and dry, one thing - like ‘sponsored’ or ‘paid’ that would be
ideal.”
Larry echoed the same belief:
“Whatever system is used should be consistent across the board, no matter what
it is. Let’s not keep changing it. And I’d like to see it consistent across the
different search engines. It could be a standardized industry-wide logo that means
the same thing.”
29
6. Some sites were better than others at making clear which results were paid for and at
highlighting related disclosure information.
Even though participants were asked to review a limited number of search engines, several
participants noted situations in which a particular navigation site did well or poorly in this area.
Participant comments are broken out by search engine in alphabetical order. Some comments
comparing a number of search engines appear at the end of this section.
About.com:
Dennis of Providence (in reference to the “On the About Network” labeled links):
“I assume these search results aren’t true search results, that there is a keyword I
typed in that relates to a list they have of people who paid to be on the About
network that may or may not be relevant to my search. I’m assuming if the search
engine works properly this stuff is on the Web (pointing at listings under the
heading ‘On the Web.’) is going to find me truer results.” (He looks at screen.
Scrolls to bottom of the screen to see if he can find disclosure information.) “Oh,
it doesn’t really tell you. I guess I could go and spend some time here to visit
some of these links” (small links to ‘about us’ and ‘User agreement’). “I don’t’
know what the hell is going on.”
AlltheWeb.com:
(verbal exchange between Larry and the Providence ethnographer):
Anthropologist: Did you see the word sponsor?
Larry: No.
Anthropologist: In tiny letters here (pointed at word sponsor next to each link
title)
Larry: Oh, I didn’t see that. O.K. There we go. O.K. I didn’t see that. I didn’t see
that. I mean clearly – I’m looking right at it now and I didn’t see that until you
pointed it out.
AltaVista.com:
Maria of Providence:
“The way this search engine places ads on every page. …A lot of the sites seem
the same.”
Maria of Providence:
She noted AltaVista was really unclear about its results because there were so
many different headings. She found this misleading and confusing. She seemed
30
outraged by the blatancy of the advertising (for example, the ads at the bottom of
each result page on AltaVista.com).
Dennis of Providence:
“I noticed that there is some small type that says ‘preferred vendors’ or they use
some language and then they give you 4 or 5 results and then it says ‘Web
results.’ I recall that, but if you are not paying attention, mixed into those Web
results, and that is a little devious I think, are paid results.”
Dennis of Providence on a different search:
“But the interesting thing is the sponsored matches give me four choices and then
it says AltaVista found 25,000 some results, so I am assuming that those 25,000
results would be unsponsored until you get further down the page and then you
see another sponsored site (at the bottom of the listings on the page)…. It’s very
vague. That could basically mean anything. I have no way of knowing what I’m
actually looking at.”
AOL.com:
Sara of Raleigh-Durham:
After a post-enlightenment homework search on AOL.com she wrote in her
journal: “I like the way AOL shows ‘sponsored links’ with very brief
descriptions – that way you know what you’re getting into before you waste your
time on those sites.”
Ask.com:
Larry of Providence after a homework search:
“Don’t like obviously paid sponsors at top [Sponsored Web Results] – but at
least they are in [a] box so [they] can be seen easily.”
Dennis of Providence:
Dennis seemed to know there was advertising on the results page on Google, but
not that the paid placements are sometimes integrated into search results
‘seamlessly’ (as AskJeeves pitches it to potential advertisers). He also seemed to
classify the ‘advertisements’ as different from the search results. (They were on a
different part of the page; they were labeled).
Google.com:
Gary of Phoenix:
31
Before starting this research he was a loyal Google user and now he has even
more respect for Google because “it was easy to find the advertising information
and the paid links are clearly marked.” He added, “The sponsored links are
colored, which is very helpful.”
Maria of Providence:
During a price comparison travel-related search, Maria wrote in her search log:
“Chose a green square to the right [Google AdWords ad] for Priceline.” She
recognized this as a [sponsored link] advertisement.
Maria of Providence:
“I like that it's so clear on Google. It actually highlights [paid links] so I can
ignore them completely.”
Jack of Kansas City:
After a homework informational search on Google, he noted: “Recommended
guidelines: Better disclosure notice.”
Larry of Providence:
“Google lists [paid links] to the side and they are in a little box and they are
clearly labeled ‘sponsor’ and that is fine.”
Dennis of Providence (on building his own consumer-friendly search engine):
“I would disclose it clearly and conspicuously, people would know. Separated
from the normal search like Google does. I think what they do is honorable.
Don’t bury it in the middle, keep it to the side, or put it at the top in different
colors.” (He is talking about sponsored links, not disclosure).
InfoSpace.com:
Geena of Kansas City:
She explained her first impression of InfoSpace.com was that it was not as biased
as other engines, however, after a little reflection, she said, “Or maybe it did
seem biased and I just couldn’t tell. On most search engines it seems the results
could be broken up into categories, such as, first ones – very expensive ads,
second ones – middle range, and the third ones – not paid for at all. But before I
learned this pay-for-placement stuff, I didn’t notice these levels. Oh yeah, also
there are the sponsored links, but it seems that all the engines did not have these
exact levels.”
Kanoodle.com:
Penny of Phoenix:
32
She does think some of the sites are more “cheesy” than others in the way that
they handle pay-for-placement. Kanoodle.com shows the cost to the advertiser
right under the paid link and she said it made her feel kind of yucky, so she has
no plans to ever use that site again.
LookSmart.com:
(recorded verbal exchange between Vivian and the Providence ethnographer):
Vivian: I think LookSmart isn’t good because they don’t offer an immediate way
to find out what is sponsored.
Anthropologist: Oh, but they do. (Pointed out the link that says “sponsored
listing” at the top of the section of sponsored links – off to the far right.) You
didn’t see that, did you?
Vivian: No, it’s so small. I don’t think they would make it bigger. It’s like the
Surgeon General’s warning they have on cigarette and alcohol ads, but they don’t
make it the biggest part of their ad…They are not totally clear what they are.
MSN.com:
Lucy of Kansas City (a search journal entry after a homework search):
“I noticed more of the sales-end of the searches, such as [after] the word that is
put in some items for sale come up. I think it would be nice to have a separate
location for items that are being sold.”
Maria of Providence:
She did seem to like MSN.com, and explained that their headings for results
were clearer. (But then we did the final planned search, and this threw this
[previous thought] into doubt.) She did, however, have difficulty figuring out if
the heading ‘featured site’ connoted pay-for-placement, or simply an editor at
the Web site “choosing to highlight” the listing. The more she thought about it
the more she realized that this act of choosing was probably due to financial
compensation, but she wasn't entirely sure.
Overture.com:
Dennis of Providence:
Dennis didn't notice the inclusion of text such as “sponsored link” at the end of
search results descriptions on Overture…. He said he found a listing that was
labeled “additional listing” [supplied by Inktomi] and clicked on the “additional
listing” link to see what this meant. “I wanted to know if it was a sponsored site.”
He explained to me that the explanation he found was not satisfactory. It
basically said – maybe someone paid for this or maybe they didn’t.
33
Dennis of Providence (ethnographer note about a revisited homework search on Overture):
He looked at the disclosure pages on Overture after noticing an ad on the search
page about “Get listed in these results” [box at top of the results page.] Once he
read the disclosure, he was so disgusted with Overture’s lack of clear disclosure
about its business practices he doesn’t plan on using the search engine again.
Maria of Providence (ethnographer note):
She noticed Overture was…all sponsored sites.
Anna of Raleigh-Durham:
“Some [search engines] will have the same Web site for each result, but on a
different page. Overture.com was really bad about that.”
Yahoo.com:
Larry of Providence:
During the second interview, we looked through Yahoo’s search result print outs
from the First Interview, and I asked if he could tell what was sponsored. He
said, “Yes, the ones at the top under ‘sponsored matches.’” I reminded him of
how I explained exactly how Yahoo generates Web results, some of them come
from Inktomi and are paid for. “That’s not clear at all! And in that regard, you
know, if Yahoo is getting things from Inktomi and from Overture, why not
acknowledge it [with] ‘from Overture’, or ‘from Inktomi?’ Why not have a little
box?”
Two or more comparisons of search sites:
Larry of Providence (ethnographer notation):
Larry thinks [his preferred site] Google has a lot less bias than the other search
engines we used [AOL.com, Ask.com, AlltheWeb.com, Lycos.com, and
Yahoo.com] because of the way it displays sponsored results. Also he talks
about Yahoo.com as being “very biased” because it has a cluttered search page,
full of advertising.
Larry of Providence (ethnographer notation):
During my observation of Larry during the planned browses, he recognized
sponsored links as advertisements on some engines (AOL.com, Yahoo.com,
Lycos.com, and Ask.com), but not on others (AlltheWeb.com).
Dennis of Providence (ethnographer notation):
34
He will use Google because they have the clearest disclosure about what is paid
and what isn’t paid. He won’t use Overture. He doesn’t like AltaVista either.
Vivian of Providence (ethnographer notation):
What is interesting about her description of how she intuits that some things on
the Google result pages are advertisements is that it clearly has to do with graphic
layout including lack of numbers on sponsored results and use of colored
backgrounds. She doesn't recognize advertising or pay-for-placement on other
search engines [Go.com, InfoSpace.com, iWon.com, Kanoodle.com,
LookSmart.com], when they are listed in the center of the screen and numbered,
under headings such as “featured sites” etc….
7. Once they were aware of pay-for-placement, the majority of participants had lowered trust
in search engines in general. Some even distrusted the accuracy or the credibility of links on
the first page of results.
Once enlightened, Vivian of Providence assumed all links on the first page were biased:
(ethnographer notation):
She took my detailed explanation of pay-for-placement on the five different
search engines we used last time [Go.com, InfoSpace.com, iWon.com,
Kanoodle.com and LookSmart.com], and came to the conclusion all search
engines use pay-for-placement…all of the first links listed on any search engine
are paid for, including search results in [preferred search engine] Google. (Which
are not, only the sponsored links are). She took what she thought was my word –
“It’s all paid for” and applied it to everything she saw on the Web.
Dennis of Providence, who searched on About.com, AltaVista.com, MSN.com, Overture.com, and
preferred site Google.com, conveyed how a lack of paid search disclosure could hurt a search
engine’s credibility:
If a search engine “were up front, I would probably use it more because I know
where the information is coming from. They’ve got to be honest in telling me
what the scoop is, so maybe [when I] look at their search engine I won’t feel so
used.”
Lucy of Kansas City said search engines should “definitely disclose [paid search] that is much more
trustworthy.” She specifically commented “pay-for-placement [as it now stands] makes the Internet
and search engines less trustworthy.” (She assessed search sites AOL.com, AlltheWeb.com,
Ask.com, Lycos.com, and her preferred site, Yahoo.com.)
Likewise, Yvonne of Phoenix was disturbed by the idea the highest bidders got the best placement
on search sites. “The idea of a search engine is to find the best matches, not the best matches that
paid 50 bucks.” (She visited the same sites as Lucy during the planned search sessions.)
Yvonne went a step further, hypothesizing − correctly − that search engines began using paid search
labels and disclosures only after warned to do so by U.S. government regulators.
35
[Yvonne] hypothesizes the search engines must be required to mark paid links
due to some FCC (or some government agency) rule because “if they could get
away without doing that they would.” She notices some engines put sponsors on
the results page even though they aren’t good matches for the keywords. This
makes Yvonne less likely to use that engine.”
Kareem of Kansas City, who reviewed AOL.com, AlltheWeb.com, Ask.com, Lycos.com and
Yahoo.com, took a different tack. He worried small businesses such as his would suffer from paid
placement, as consumers are being nudged toward companies that can afford to participate in search
engine marketing programs. “My small business shouldn’t have to compete for informational
listings.”
Bob of Phoenix, who searched on About.com, Go.com, Kanoodle.com, MSN.com and preferred
site Yahoo.com, said:
Pay-for-placement and online advertising, in general, bothered him because
“[search engines] are trying to disguise taking your money as something else—
if there is nothing wrong with it, why are you trying to hide it?”
Mike of Raleigh-Durham reported he would be “pushed away” from search engines that did not
separate “objective search results” from “advertisements.” (Mike performed planned searches on
Go.com, InfoSpace.com, iWon.com, Kanoodle.com, LookSmart.com.)
Several participants suggested their perception had changed about the companies that participate in
paid search marketing.
Geena of Kansas City, who assessed the same sites as Mike of Raleigh-Durham, expressed mixed
sentiments:
“In some ways it may be they have enough money to pay for this ad because they
are a stable company, so I may think they are more reliable or safer to buy from
online. But, on the other side, maybe I should give a smaller company a chance.”
Diane of Raleigh-Durham expressed frustration with search engine advertisers who “force me to
look at [their ad] or trick me into reading them” when the sites do not match what she was
searching for. She reported her irritation is present whether or not she “falls for it, because
somebody out there will, and that’s not right.” (She assessed Go.com, InfoSpace.com, iWon.com,
Kanoodle.com, and LookSmart.com.)
8. Most participants generally did not comprehend the inter-company deals within the
industry and didn’t understand how these deals might influence their search results. This
ignorance was due to a tendency to use only one or two engines for their online searching.
Among participants who searched on Overture.com, no one noticed or seemed to understand why
Overture paid search listings also appeared on partner sites such as Go.com or InfoSpace.com, until
the planned search phase, or in some cases, not until it was pointed out by the anthropologist.
Similarly, participants did not always realize Google sponsored links also appeared on partner sites
such as AOL Search (AOL.com) and Kanoodle.com, whether Google.com was their preferred
search engine or one site evaluated for the purpose of this study.
36
As they completed the planned searches during the first field visit, during which they were
instructed to use search engines they had never heard of or used before, they began to pick up clues
about how business deals affect the search results.
Dennis of Providence demonstrated, in trying to explain Google’s business model, how consumers
don’t truly understand search engine business models.
He held up Google as an example of an “independent” search engine, and
couldn't understand how the site was profitable. He mentioned almost
immediately Google had ads, but they were “subtle.” He couldn't figure out how
Google was such a big company. Again, he had “read somewhere it's one of the
largest search engines out there.” During the enlightenment phase of the
interview, Dennis was surprised to see how Google financed itself. (We explored
both the advertising on Google itself, and how Google sells its services to other
portals and search engines, like AOL.com. This information was found under the
“search solutions” link on the Google search page.)
On a related note, several participants were surprised to see Google results “powered” or
“enhanced” the listings of other search and navigation sites. Larry of Providence noticed Google
results on the AOL.com and Cox Cable (home) page, but he didn't understand how it worked.
Larry pointed out at the very beginning of the interview Cox Cable used Google
as their search engine (something he had just noticed and found confusing).
When we visited AOL.com (which he thought he would not be able to use
because he was no longer a member of AOL), he similarly noticed its search
engine was Google. He was skeptical, however, it truly was Google, as he felt the
search results were not as specific as [what] he usually gets from the Google Web
site.
Vivian of Providence didn’t connect the financial dots between Google.com, iWon.com and
Kanoodle.com until the field researcher drew the line.
She was surprised to see iWon.com was “powered by Google” and when I
pointed out to her Kanoodle.com was also “powered by Google,” she was a little
surprised, but immediately recognized stylistic similarities between the two
engines (lack of extra content on the search page, etc.)
In other words, Vivian did not recognize the financial relationships between the three navigational
sites pre-enlightenment. But once it was pointed out to her, she figured the relationship between
Kanoodle and Google made sense, as Kanoodle not only shared Google’s results, but the site also
shared what she perceived to be Google’s sparse and simple user interface appearance.
Similarly, Gary of Phoenix demonstrated how consumers are not aware of how pervasive paid
search is on the metasearch engines. During a homework assignment search on MetaCrawler.com,
he saw a “very lengthy” list of paid sponsorship links at the top of the search page. He skipped
down and “looked at unpaid links.” He noticed the “descriptions were very short and appear to be
the same ones from the previous search on www.dogpile.com.” This is an indication Gary had no
understanding that MetaCrawler.com and Dogpile.com are owned and operated by the same parent
company, InfoSpace, Inc.
37
9. The role of the U.S. government or third parties in regulating search engine marketing
practices.
Participants were mixed about what role, if any, U.S. federal regulatory agencies should play in
monitoring search engine marketing practices.
Four consumers strongly recommended regulatory agencies increase their scrutiny.
For example, Maria of Providence supported the notion of government intervention, particularly
when search engine marketing might affect more vulnerable Web users, like children.
It would be good for the Federal government to set up guidelines for how
advertisements are labeled on the Web, but she doesn't think this is possible
because of the very nature of the Internet. The one area where she really thought
there should be Federal regulation was for Web sites and search engines targeted
at children. (We had discussed AJKids.com.) “There should be areas, like with
children. They shouldn't be targeted. Kids should be searching online for
[information], and not to find things to buy.”
Geena of Kansas City suggested government regulation might be warranted:
“Most people don’t think about [paid search] and everyone I talked to said they
didn’t know. A friend said, ‘you’re kind of conditioned to think number one is
the best and on down the line.’ When you search for airfares, the cheapest is first
and the most expensive is last in the list. There should probably be a disclaimer
that shows Number One isn’t necessarily the best.”
Fellow Missouri resident and daycare coordinator Lucy echoed, not surprisingly, a similar concern
about younger online users:
“[Paid search on search engines] should be regulated because anything can be on
there. I’m personally thinking of the kids because frequently inappropriate things
are slipped in and the advertisers or whoever are not concerned about the effect
on kids.”
Mike of Raleigh-Durham expressed a similar idea:
“I think it would be cool if someone made a rule everyone had to obey, that
restricted pop-ups and advertisements that trick people. Who ever can do that
should do it. I don’t know if the free-marketplace would be capable of doing
that.”
The bulk of participants, however, suggested the onus of search engine site transparency should fall
mainly on the industry itself.
Two participants, Larry of Providence and Jack of Kansas City, best articulated the thoughts of a
number of participants:
38
Larry doesn’t think the government should get involved. “Let the free market do
it. I don’t favor government inclusion into everything…I think they should be
doing it voluntarily, the industry itself. I’m sure there is an industry organization
and they get together and discuss issues of conduct. I would recommend that they
do something like, something about truth in advertising. I mean there are truth in
advertising laws, and I would think those laws would also apply to [search
engines] as they do to publishing and television and other things. I think those
[laws] should apply, and if a search engine isn’t doing it, there may already be
laws on the books that somebody could challenge them.”
Likewise, the researcher reported Jack said:
…Businesses are, or should be, “trustworthy” and that government intervention
may be necessary if they are not trustworthy. He believes search engines that use
pay-for-placement or paid advertisements should have to disclose those facts. He
feels that regardless of the type of search you are engaged in, the site should have
to disclose their business practices.
Two participants suggested monitoring by an unbiased third party might be a better solution.
Claude of Raleigh-Durham suggested the Better Business Bureau. Bob of Phoenix would like to
see a scenario in which there is “some regulation that requires all the advertisers to contribute to
a public information source. This would be a kind of [no-advertising accepted National Public
Radio/Public Broadcasting Service]-ish search engine where people could go to find quality,
non-commercial information online.”
SUMMARY AND CONCLUSIONS
Consumer WebWatch researchers probed a diverse group of online searchers for this project,
which included, coincidentally, four people who have experience managing their own Web sites or
online databases. Although each person had a minimum of five years’ of online experience, they
did not understand how search engines prioritize results or how the inclusion of paid search
listings might influence the types of Web pages they see first. For example, several participants
didn’t realize search engines are moneymaking businesses. No one realized the ranking of results
they received could possibly be influenced by the inter-industry business deals between two or
more search companies, until pointed out by a researcher.
These misconceptions about search sites suggest the less-experienced Internet user is even more in
the dark about pay-for-placement on search engines than our panel of 17 consumers with
broadband access.
Our findings show Web searchers in this ethnographic study chose links from the first page of
results nearly 50 percent of the time because they trusted search engines to present only the best or
most accurate results first. This trust led them to believe it was unnecessary for them to review
later results pages.
39
This “first equals best” mentality among Web consumers, along with the quick-fire nature of the
Internet, makes them more likely to click prominent paid search listings, particularly if such
listings are not clearly demarcated from pure search results. This confusion among consumers
about what is paid search and what is pure search is exacerbated by unclear or vague labels search
sites use, such as “sponsored,” which almost every participant thought was confusing, possibly
even misleading.
This tendency to click on paid search results makes consumers vulnerable as they are more likely
to encounter faulty or biased information on Web pages of companies that can afford to be listed
on the first results page, yet do not necessarily have the most accurate or unbiased information. If
the information received is acted upon during, say, an informational search, the consumer’s
decision could adversely affect his or her health. In these situations, it’s possible a lower-ranked
search listing could have been better suited for what the consumer was seeking.
In addition, the participants did not notice the disclosure links or “About Search” pages on the 15
sites under review until pointed out by the ethnographer, including such disclosures that appeared
above the search results or next to individual listings. Most participants said these disclosures were
too small for consumers to see or were hidden in areas the typical consumer would not think to
look. The issue is even more elusive when applied to meta-search engines, in which disclosures
that appear on the originating search site, even if flawed, are in some cases stripped away once
these results are fed through a new meta-search filter and ranking mechanism.
Each participant was surprised once told about how pay-for-placement works on these popular
search engines. Several participants had negative reactions, ranging from anger and betrayal to a
sense of helplessness and disappointment. Most participants expressed a lowered trust in search
engines, while some were more suspicious about advertisers whose prominent listings they
perceived to be irrelevant.
The consumers we talked to at length became more critical of the search results they received and
began to use different methods of screening the delivered search results before making a final
decision to click on a listing. These strategies included reviewing more results pages and making
an effort to first review pure search listings before making a decision to click on a link.
Participants were even more adamant about this practice while seeking information online,
particularly when they had no intention of making a purchase and did not want to see paid ads.
When asked how the search engine industry could make paid search listings easier to spot, the
participants overwhelmingly suggested clear and conspicuous labeling, such as the term “paid
advertisement.” They also supported the use of visual aids on search sites that would help Web
consumers to quickly see what areas of the results page, or individual listings, were paid for,
including the use of boxed advertisements, and advertiser-only segregated areas.
The consumers in our study expressed frustration the search engine industry had not already come
to a consensus about what disclosure labeling to use, which would promote consistency from
search engine to search engine. The participants demanded the search engine industry, or a trade
group representing industry concerns, make some decisions in this area. In addition, a sizeable
number of consumers in our study felt the U.S. government regulatory agencies should increase
scrutiny in this area.
The research of other groups supports Consumer WebWatch’s ethnographic findings. Last May,
SearchEngineWatch.com, an online marketing resource, assigned estimated “disclosure ratings” to
11 search sites based on how well each complied with FTC guidelines on paid placement and paid
40
inclusion disclosures.ix (The sites evaluated were AlltheWeb.com, AltaVista.com, AOL Search,
Ask.com, Google.com, HotBot.com, Lycos.com, MSN Search, Netscape.com, Teoma.com and
Yahoo.com.) Seven of the 11 sites were issued “FAIL” or “QUALIFIED PASS” ratings, an
indication that better paid search disclosure is needed in the interest of the Web consumer. (Three
of these seven sites made it hard for searchers to see disclosure links specifically about paid
placement, according to SearchEngineWatch’s evaluation.)
Other researchers have produced data that supports our consumer behavior findings. For example,
an analysis of 1 million search queries made by 200,000 users on Excite.com found more than 70
percent looked at two pages of search results or fewer.x A study released by iProspect.com, a
search engine marketing company, reported that half of all the Web users make their selection
after reviewing the first page of results because “they expect that the Web sites with the highest
search engine listings are the top in their field.”xi In a health-information-specific focus-group
study, conducted in March 2001, the authors reported that in 97 percent of cases consumers chose
a search result ranked among the top 10 search results.xii (In 206 of 289 cases, the participants
selected a link from the top five results.)
Many of the participants in this ethnographic study said they do not believe paid search on search
engines is “evil.” In fact, a majority of participants reported using paid search listings as tools
during commercial searches, which helped them hone in on vendors selling the products or
services they desired, when ready to make a purchase.
But even so, all consumers have the right to know in advance that a prominent search result listing
may be the equivalent of a paid advertisement before they make a decision to click or to buy.
Education about such online practices improves consumer choice. Consumers also have the right
to know that information better suited to them, such as a company Web page from a small business
site or a non-profit organization that cannot afford to participate in paid search, might appear
several results pages from the top. Poor disclosure limits consumer choices online, as expressed by
Dennis and recorded by the Providence ethnographer:
Dennis compared pay-for-placement to another type of advertising on another
medium: TV commercials. “It’s like shutting the television or changing the
channel when you are watching the news. If I knew [that a link was a paid
listing], it would change my perspective and I would be more likely to move on
to another choice. I would probably choose to use another Web site.”
Based on the voices and experiences of our 17 consumers, it’s clear search engine companies need
to work hard to maintain the trust of their users, rather than focusing on their bottom lines.
CONSUMER TIPS
Search engines provide a valuable service to millions of Web consumers each day, including many
sites not evaluated as part of this ethnographic study. Before embarking on your next search, keep
these tips in mind:
You need to understand:
The major search engines, like any business, need to make money:
These companies are businesses, not independent, philanthropic or research-based in nature. The
cost of indexing billions of Web pages each month can be very expensive, so search companies that
41
provide algorithmic, keyword relevancy-based search services, like Google.com, AlltheWeb.com
and Teoma.com, need advertising revenue to support their daily operations.
Paid search is not evil:
Don’t assume just because a listing has been paid for that it is not relevant to your search query.
That site may well have the information you’re looking for, however, you’ll be more likely to spot
any bias in the site’s information before you make a final decision. Also keep in mind that just
because a search engine offers paid search listings doesn’t mean every listing it delivers or every
result on the first page is tantamount to a paid ad.
Follow the dollars:
Some companies have different business models than others. The major companies sign deals that
may affect the types of results you receive. Sites that provide algorithmic search results often
provide these, and sometimes paid search results, to other companies in exchange for a fee. Some
sites like Kanoodle.com, LookSmart.com and Overture.com primarily list the Web pages of
companies that have paid. This is also why the listings of companies that have paid
LookSmart.com to be listed in its database may also appear on partner sites About.com,
InfoSpace.com and MSN.com.
Some companies own more than one search site, each of which carries its own, consumer-friendly
name. For example, InfoSpace, Inc. owns InfoSpace.com, Dogpile.com, Excite.com,
MetaCrawler.com, and WebCrawler.com. AskJeeves, Inc. also owns Teoma.com and
DirectHit.com. And, as reported earlier, Overture recently acquired AlltheWeb.com and
AltaVista.com. This may explain why you may see the same search results or Web pages appear
on each search property under the same corporate umbrella.
Searching tips:
1. Make choices: Before you decide which search engine to use, consider whether the use of paid
search is important to you, or, if paid search would matter to you during an information search
versus one in which you are seeking to buy something.
2. Use multiple search engines: It’s okay to use a favorite search engine time and time again,
particularly if you’re looking for less serious information such as the latest sports scores. But once a
decision is made to do a serious search, say, for health or financial information, consider searching
for the same query on several engines before making a final decision. Compare the results listings
you receive from the sites you’ve queried and take note of any listings that appear at the top of the
rankings, along the sides, or those that pop up on more than one site. This may be an indication that
that search result is a paid advertisement.
Caveat: Some searchers like to use meta-search engines such as Dogpile.com, Excite.com or
MetaCrawler.com because these sites search and rank listings from several search engines
simultaneously. Although these sites can be great time-savers, you should realize the results would
be skewed. Meta-search sites, which tend to provide the first few links retrieved from each engine,
are more likely to present sponsored links. And in some cases, the meta-search engine removes the
valuable disclosure links information that appears on the originating search engine.
3. Read further: Be sure to review several pages of results before you decide to click on a result.
You might find the product or information you’re looking for, on, say, page 10, provided by a
smaller company or a non-profit organization that cannot afford to pay for a page-one search
position.
42
4. Look for telltale labels:
Look for signs a result has been paid by an advertiser. If a search engine separates some search
results from others, either in a special section at the top, at bottom or to one side, advertisers most
likely support these segregated listings. You should also look for labels such as “Sponsored
Listing,” “Sponsored Match,” “Sponsored Site” or “Sponsored Link.” Don’t be confused by other
labels