OVERALL DISCUSSION ON COMMENTS

While this report could discuss the findings about the overall comments at length, we will focus on two areas in this concluding discussion: (1) why Design Look is so prominent, and (2) what topics may have been overlooked in this analysis.

Why is Design Look so prominent?

The results of this research show that the Design Look of Web Sites was clearly the most prominent issue when people evaluated Web Site credibility in this study. Almost 50 percent of comments about Web credibility contained something about the look of the site, either in general (e.g., "looks professional") or in specifics (the layout, the colors, etc.). The dominance of Design Look may be surprising at first. One might ask, 'Are people really so influenced by design look and not by more substantial issues?' The answer appears to be yes — at least in this setting.

It's important to note that looking good is often interpreted as being good-and being credible. Since at least the 1940s,social psychology research has shown that physically attractive sources (usually people) have been perceived to be credible sources (Benoy, 1982; Berscheid, 1981; Berscheid & Walster, 1974; Dion, Berscheid, & Walster, 1972; Eagly, Ashmore, Makhijani, & Longo, 1991). This basic human processing bias — "looking good is being good" — also seems to hold true for evaluating the credibility of Web Sites, especially since Design Look is highly noticeable.

The research context is another factor that likely contributed to the overwhelming prominence of Design Look as a rationale for determining site credibility. Because people participated in this study to earn a donation for a nonprofit organization — not because of a deep personal interest or need — they did not likely have the motivation to process the Web Sites deeply. According to the Elaboration Likelihood Model (ELM) (Petty & Cacioppo, 1986), without deep motivation, people will rely on peripheral cues, such as appearance, for making assessments. The ELM would predict that if the participants had both the ability and the motivation to scrutinize these sites carefully, the percentages in this study would change, with peripheral cues playing a less significant role.

Although people in this study were probably not deeply involved in the evaluation task, this is not a fatal flaw in the research. Our results are still valid. One could argue that people typically process Web information in superficial ways, that using peripheral cues is the rule of Web use, not the exception (for empirical research supporting this point, see Cockburn & McKenzie, 2001). From a user perspective there are many sites available, with the next piece of information just one click away. Even the words people use to describe Web use — "visiting sites" and "surfing the Web" — suggest lightweight engagement, not deep content processing. Research has yet to examine the relationship between engagement level and credibility assessments of Web Sites.

An important follow-up study would be to manipulate the engagement level of the participants (e.g., finding health information for a loved one in dire need) and see how the comments about credibility change. Studies along these lines could show how involvement level affects what people notice when evaluating a Web Site Credibility. Our hypothesis is this: Even for highly involved Web surfers, Design Look would still play a role in credibility, though it would be less dominant in overall evaluations.

The high value for Design Look is also due to the coding categories themselves. Design Look may be the broadest category, causing many Web Site elements to be coded as Design Look. In a future analysis of the comments, dividing the Design Look category into more focused categories could be illuminating. We suspect that some interesting findings are still concealed in the data because of the breadth of this category.

Areas that did not emerge in this research
We would be misleading readers if we implied that our analysis of the comments has brought to light all the interesting issues in our data. In all likelihood, our analysis has missed some key findings. For example, as the previous paragraph notes, our analysis categorized many issues into the design look category. A subsequent analysis could further refine this category to give additional insight into different aspects of Design Look.

Although we suspect other categories are also hiding key insights, it's difficult to pinpoint where the hidden insights are. We certainly have some suspicions about where to reanalyze the comments. Shown to be important in other research (Fogg et al., 2001; Fogg et al., 2002), the timeliness of a site's information did not surface as an important issue in this study. It's possible that comments about timeliness were missed because we initially had no category for coding this issue, or that these comments were coded as information accuracy (current information = accurate information). Another area that likely exists in the data but that did not surface in our analysis is that of information source — providing citations and references to show that the site content came from an expert source. The issue of information source proved to be important in the expert study performed in tandem with our study (vi); the analysis for the tandem research showed that experts are much more tuned in to the source of information than are consumers.

Although our analysis probably did not reveal all issues related to the credibility of the sites in this study, there were topics we looked for in our analysis but did not find. For example, we coded the data for comments about privacy policy, and we found that people mentioned privacy policy in less than 1 percent of the comments. We also looked for comments about correcting false or misleading information and found no comments along these lines. These two issues apparently did not affect how our participants evaluated the credibility of Web Sites in this study.

A Focus on Individual Web Site Categories

In this part of our Results & Discussion, we change our focus away from the overall comments in order to focus specifically on the site categories (health, news, etc.). Some of the information for this analysis may be gleaned by examining various charts in Part 1, but to do this readers would need to flip back through the report and compare charts on various pages. We hope to make this task easier by presenting the results one category at a time.

In the paragraphs that follow, we present data about how comments in one particular site category differed significantly from the comments from all the categories. For example, we point out how people's comments on the e-commerce sites differed from people's comments on all 100 sites. After we present what we see as notable differences in the comments (those that are more than 5%), we then present a new type of data: the credibility rankings of the sites.

A WORD ABOUT THE CREDIBILITY-RANKING RESULTS

This study was designed to generate a credibility ranking for Web Sites within each category. As discussed in the Methods section, these rankings may be interesting and provocative, but they are not definitive credibility assessments for an individual Web Site. A credible Web Site could end up with a low ranking if other sites in the same category were extremely credible sites. In other words, the ranking for a site depended entirely on the competition; if the competition were different, then the ranking would be different. Also, because some content categories were undoubtedly more competitive than others, the ranking score for Web Sites cannot be compared across categories. And of course, these rankings reflect the views of those who happened to participate in this study.

As the tables below show, each site was evaluated at least 35 times and some sites as many as 89 times (total number of rankings: 5,242, from 2,684 completions of the study). Sites in the health and finance categories were ranked more often, since a study parallel to this one (4) focused on these two categories and we wanted to provide sufficient data for that project.

For each of the 10 tables, the first column represents the credibility ranking for the sites, with the site listed #1 as being the most credible. The value in the "average score" column is the mean score that site received over the course of the study, as described in the Methods section. It was this score that determined a site's ranking.

The discussion below does not use the credibility comments to explain the ranking results (or vice versa), because we do not want people to read more into these rankings than is warranted. Our intent in the coming pages is to focus on a single Web Site category and share two types of results that relate to that category. At times, links between the comments and rankings seem obvious; at other times the connection is not immediately apparent.

That said, we'll begin presenting and discussing the results for each Web Site category one by one, in the following order:

• E-Commerce

• Entertainment

• Finance

• Health

• News

• Nonprofit

• Opinion or Review

• Search Engines

• Sports

• Travel

E-COMMERCE Web SiteS: CREDIBILITY RESULTS

This section presents what we learned about the credibility of e-commerce Web Sites, starting with presenting our results about how the credibility comments for e-commerce sites differed from the overall averages, and then presenting the final credibility rankings of the 10 e-commerce sites.

Credibility comments about e-commerce Web Sites: What stands out?

The data in our study show that e-commerce sites stand out from other types of sites in four ways. First, issues of name recognition and reputation were mentioned more often by people evaluating e-commerce sites, compared to the overall mean for this type of comment (25.9% compared to 14.1% overall). Next, comments about customer service were also relatively more frequent (16.7% compared to 6.4%). On the other side of the scale, compared to the overall mean, e-commerce sites received far fewer comments about information bias (2.6% vs. 11.6%). One final difference, not discussed earlier in this report, has to do with comments that our coders classified as "general suspicion." These comments would include things like "It just makes me suspicious," and "I'm always wary of these type of sites." These types of comments appeared in 9.4% of all the comments in this study. The "general suspicion" comments appeared more often when people evaluated e-commerce sites, in 17.2% of the comments people made.

Taken together, the data show that when people evaluated the credibility of e-commerce sites, they more frequently mentioned issues of customer service and name recognition and reputation as indicators of whether or not a site was credible. In addition, people approached e-commerce sites with more suspicion than other sites in this study.

Credibility rankings of e-commerce Web Sites

Table 2 shows the final ranking of e-commerce sites in this study.

Table 2: Ranking and scores for e-commerce Web Sites.

What's notable about e-commerce site rankings?

The scores and rankings for the e-commerce sites led to three observations from our team. First, big-name players Amazon and Barnes & Noble led the credibility rankings, in nearly a dead heat for the top spot (the difference in average scores is less than .01). The top placement of these two sites is not surprising, given the previous experience many people have in dealing successfully with these companies (what has been called "earned credibility"). The surprise comes in seeing who landed in third place: McMaster-Carr — a relative unknown entity compared to others who ranked high in this category. The McMaster-Carr site has relied on its own merits — not on a public relations campaign or a big reputation — in conveying high levels of credibility. This should be good news for people who want to set up an e-commerce site, or any other kind of site, but are faced with competitors who own big brand names and wield significant marketing budgets. At least when it comes to evaluations of Web credibility, the high ranking of McMaster.com shows that a site can win credibility on its own merits.

The second notable result is the mediocre ranking of eBay, a bit of a puzzling outcome given that eBay is widely known and used.

Our third observation is that two e-commerce sites did quite poorly in the credibility rankings: MTE Nutrition and ThymuSkin. When it comes to establishing Web credibility, these sites might serve as models about what not to do.

Back to start of the article: A Good Web Site Design Brings Trust and Success

Continue with:

• ENTERTAINMENT Web Sites: CREDIBILITY RESULTS
  
• HEALTH Web SiteS: CREDIBILITY RESULTS
  
• NONPROFIT Web SiteS: CREDIBILITY RESULTS
  
• TRAVEL Web SiteS: CREDIBILITY RESULTS