Internet Marketing · eMarketing · Internet Advertising · Online Branding |
| |
|
|
||
| ADVERTISING BY ICONOCAST |
||
MAJOR TRAVEL SITES
FACE CREDIBILITY CRUNCH
An Examination of Booking First-Class Tickets Online
MAJOR TRAVEL SITES
FACE CREDIBILITY CRUNCH
An Examination of Booking First-Class Tickets Online
By William J. McGee
March 1, 2005
Consumer Reports WebWatch
101 Truman Avenue
Yonkers, NY 10703-1057
Tel: 914.378.2600
www.ConsumerWebWatch.org
Executive Summary ........................................................................3
Consumer Reports WebWatch Mission Statement ................................6
Consumer Reports WebWatch Travel Projects, 2002-2005 ..................7
Testing Methodology and Parameters ................................................9
Selection of Travel Web Sites ..........................................................9
Testing Methodology......................................................................10
Testing Parameters and Specific Testing Criteria................................10
Selection of Results ........................................................................12
Currency Issues ............................................................................13
Rounding of Fares ........................................................................13
Conversion Rates ..........................................................................13
Availability of Airfares ..................................................................13
Valid Tests ....................................................................................13
Invalid Data..................................................................................13
Technical or System Failures ..........................................................14
Results, Rankings, and Conclusions..................................................15
Lowest First-Class Fares for All Tests ................................................16
Closest First-Class Fares for All Tests ................................................17
Lowest First-Class Fares for Each of the Four Individual Tests ..............19
Lowest First-Class Fares for Round-Trip Nonstop Flights ......................22
Findings and Concerns ..................................................................25
Pricing and Display Inaccuracies (Including “Fare-Jumping”) ..............25
Availability of Valid Fares ..............................................................28
Invalid First-Class Listings................................................................28
Potentially Deceptive Labeling of First-Class Listings ..........................29
Rounding Down of Fares................................................................32
Omissions of Viable Airlines ..........................................................32
Integrated Travel Sites vs. Branded Travel Sites ................................33
The Orbitz “Low Fare Promise” ......................................................34
Tips for Booking Premium-Class Airline Seats....................................35
TABLE OF CONTENTS
CONSUMER REPORTS WEBWATCH 3
While travel remains the largest sector of online
commerce, more Americans have moved from “looking”
to “booking” on travel Web sites. The complexities of
what’s on offer, as well as choices among search tools,
continue to increase as well.
Since 2002, Consumer Reports WebWatch has examined
a variety of travel Web sites in the United States
and abroad. Airline ticket purchases continue to be the
biggest slice of the pie, but even so, certain aspects of
searching for and booking seats continue to be daunting.
Previously, Consumer Reports WebWatch compared
flight and airfare data only for economy-class airline
tickets. This project addresses the more complicated firstclass
airfare market.
It seems apparent that the Internet is having an effect on
the sale of first-class airline tickets. According to the U.S.
Department of Transportation’s Bureau of Transportation
Statistics (BTS), 7,390,453 first-class tickets were sold in
the U.S. in 2004. This marked a 9.5% increase in this
class of service over 2003. In 2004, a number of news
reports said the average price of a first-class airline
ticket had decreased. This may have spurred more
travelers to shop for them online.
This examination of lowest first-class fares on a variety
of domestic routes uncovered many complexities and
challenges, as well as several concerns. In fact, buying
first-class tickets online can be much more difficult than
buying economy class tickets.
EXECUTIVE SUMMARY
In Consumer Reports WebWatch’s first
analysis of booking first-class airfares
online, we discovered:
Fare-jumping continues to be a problem
on Expedia, Orbitz and Travelocity—one
fare on Expedia increased by $748.
Expedia, Orbitz and Travelocity all had
problems with missing and/or incorrect
labeling information regarding class of
service.
Expedia led all sites by providing the
highest percentage of lowest fares, while
Orbitz led by providing the most lowest
fares within $10.
KEY FINDINGS:
CONSUMER REPORTS WEBWATCH 4
Among key topics addressed in this report:
“Fare-jumping” (in which a rate suddenly increases
or decreases during the booking process, sometimes
without a notification to the user) continues to occur on
Expedia, Orbitz, and Travelocity.
The single largest fare-jumping increase occurred
testing Expedia, totaling $748.00.
Searching for first-class fares can be harder than it
seems; during this testing project, sites often were unable
to deliver a first-class fare.
Although testers queried for first-class fares, several
sites continually returned itineraries with business and
even economy-class flights.
Testers found multiple problems on all three integrated
sites with missing and/or incorrect labeling information
for the airlines’ classes of service. In some cases, business
class was incorrectly described as first class.
Among the top three third-party sites, competition has
increased, with testers finding a growing number of
razor-thin differences among fares.
Expedia led all six sites, returning the lowest first-class
fare four out of every 10 queries.
Orbitz was equally impressive when results were
weighted to include “closest” first-class fares. Orbitz
returned fares within $10 of the lowest 82% of the time.
Travelocity returned the highest percentage of nonstop
first-class flights among all six sites, indicating the quality
of its lowest-fare first-class offerings was better.
Orbitz recorded the highest savings for a single
itinerary for this project, when it returned a first-class fare
$1,348.70 (242%) less than the lowest fare its closest
rival offered.
PREMIUM “FARE-JUMPING”
FIGURE 2
WEB SITE % OF FARES THAT NUMBER OF FARES AVERAGE AMOUNT
INCREASED OR THAT INCREASED THAT FARE INCREASED
OR DECREASED OR DECREASED OR DECREASED
MID-BOOKING MID-BOOKING MID-BOOKING
TRAVELOCITY 4% 5 +$5.32
EXPEDIA 3% 3 +$303.23
ORBITZ 1% 1 -$610.80
INTEGRATED
TRAVEL SITES
AMERICAN 0% 0 0
CONTINENTAL 0% 0 0
DELTA 0% 0 0
AIRLINE SITES
WEB SITES TESTED FOR THIS REPORT
INTEGRATED SITES AIRLINE SITES
EXPEDIA AMERICAN
ORBITZ CONTINENTAL
TRAVELOCITY DELTA
FIGURE 1
CONSUMER REPORTS WEBWATCH 5
LOWEST PREMIUM AIRLINE FARES
(INCLUDING TIES)
40%
31% 31%
0%
FIGURE 3
28%
9%
EXPEDIA
TRAVELOCITY
ORBITZ
CONTINENTAL
DELTA
AMERICAN
Expedia, Orbitz and Travelocity often returned lower
fares on American and Delta than on those airlines’
own sites. Only Continental’s site offered the best fares
on Continental.
CREDITS
The project was directed by William J. McGee, a travel
journalist and consultant to Consumer Reports WebWatch
and the Editor of Consumer Reports Travel Letter from
2000 to 2002. The project was completely funded by
Consumer Reports WebWatch. Employees of Consumer
reports WebWatch assisted in drafting the methodology,
participated in the testing, contributed to this research
report, and provided fact-checking. The research report
was edited by Beau Brendler, the director of Consumer
Reports Web Watch.
CONSUMER REPORTS WEBWATCH 6
Consumer Reports WebWatch is a project of Consumers
Union, the non-profit publisher of Consumer Reports
magazine. The project is supported by grants from The
Pew Charitable Trusts, the John S. and James L. Knight
Foundation, and the Open Society Institute. Consumer
Reports WebWatch in turn funded the research and
production of this report, as it has done with similar
reports on Web site credibility, travel ticket-booking Web
sites (performed and published independently and in
conjunction with Consumer Reports Travel Letter, which
ceased publication in December 2002), search engines,
and health Web sites.
Consumer Reports WebWatch’s mission is to improve
the credibility of Web sites, through research, through
articulation of best practices guidelines in specific
sectors of Web publishing, and by working with
ConsumerReports.org to produce ratings of Web sites
using those guidelines. Ratings of these sites were
released in 2004, using research conclusions from
this report. Consumer Reports WebWatch’s research,
guidelines, and e-Ratings are available for free at
http://www.consumerwebwatch.org.
CONSUMER REPORTS
WEBWATCH MISSION STATEMENT
CONSUMER REPORTS WEBWATCH 7
Over three years, Consumer Reports WebWatch has
created an unprecedented database of original and
unbiased research on the online travel market, the single
largest sector of Internet commerce. Here is a summary
of the travel Web site projects undertaken to date:
“Travel Web Sites: You Still Need to Compare,” was
released in June 2002. Consumer Reports WebWatch
teamed with Consumer Reports Travel Letter (which
ceased publication in December 2002) to extensively test
travel Web sites providing domestic airfares. This report
is available at :
www.consumerreports.org/main/detailv2.jsp?CONTEN
T%3C%3Ecnt_id=158287&FOLDER%3C%3Efolder_id=
158259&bmUID=1033759487281
“Booking Hotels Online: An In-Depth Examination of
Leading Hotel Web Sites,” was a research report that
was released in April 2003. This report is available at :
http://www.consumerwebwatch.org/dynamic/travelreport-
booking-hotels-online-abstract.cfm
“An Analysis of the Potential Benefits and Dangers
of Booking Through a Car Rental Web Site,” was a
research report that was released in October 2003. This
report is available at:
http://www.consumerwebwatch.org/dynamic/travelreport-
renting-cars-online-abstract.cfm
“Booking and Bidding Site Unseen: A Consumer’s
Guide to Opaque Travel Web Sites,” was a research
report that examined alternative “opaque” travel booking
Web sites and was released in December, 2003.
This report is available at:
http://www.consumerwebwatch.org/dynamic/travelreport-
booking-bidding-abstract.cfm
“Global Concerns: An In-Depth Examination of Travel
Web Sites Selling International Airline Tickets,” was a
research report released Sept. 22, 2004, that examined
U.S.-based travel sites selling airline tickets on international
routes originating in the United States. This report
is available at:
http://www.consumerwebwatch.org/dynamic/travelreport-
global-abstract.cfm
“Spanning the Atlantic: A Cross-Border Examination
of 20 Travel Web Sites in the U.S. and Europe,” was a
CONSUMER REPORTS WEBWATCH
TRAVEL PROJECTS, 2002-2005
CONSUMER REPORTS WEBWATCH 8
research report based on a project directed by Consumer
Reports WebWatch and involving six other non-profit
consumer advocacy organizations in Europe. The report
was released Nov. 12, 2004 and is available at :
http://www.consumerwebwatch.org/dynamic/travelreport-
cross-border-abstract.cfm
CONSUMER REPORTS WEBWATCH 9
Consumer Reports WebWatch continues to hone, refine,
and improve its methodology to conduct side-by-side
tests of leading travel Web sites. Many modifications and
improvements are the direct result of feedback from the
travel sites. Such criticism is always carefully considered,
but not always acted upon.
Although Consumer Reports WebWatch has extensively
tested U.S. and foreign travel sites offering domestic
and international airline fares, this project marked the
first evaluation of premium airline products (other than
economy-class or coach-class seats). The specific testing
methods are described in detail below.
SELECTION OF TRAVEL WEB SITES
Consumer Reports WebWatch analyzed market share
data, airline traffic statistics, and other factors to determine
which sites to include in this testing project. Data
were employed from a variety of independent sources,
and all of this information is in the public domain. For this
project, Consumer Reports WebWatch did not contract
with independent consultants to obtain information.
The three largest domestic integrated travel sites were
selected: Expedia, Orbitz, and Travelocity. Here is
further information about them:
Expedia (www.expedia.com) is based in Bellevue,
Wash. and is owned by IAC/InterActiveCorp
(USA Interactive). Expedia was formerly owned by
Microsoft. Expedia is a sister company of Hotels.com
and Hotwire.
Orbitz (www.orbitz.com) is based in Chicago
and was founded by the nation’s five largest
airlines: American Airlines, Continental Airlines, Delta
Air Lines, Northwest Airlines, and United Airlines.
In September 2004, Orbitz was acquired by Cendant
Corporation, a New York City-based travel conglomerate
that owns several travel Web sites (including
CheapTickets.com and Lodging.com), a leading travel
technology company (Galileo), and car rental firms
(Avis and Budget).
Travelocity (www.travelocity.com) is based in Fort
Worth, Texas and is owned by Sabre Holdings.
Travelocity was formerly owned by AMR, the parent
TESTING METHODOLOGY AND
PARAMETERS
CONSUMER REPORTS WEBWATCH 10
company of American Airlines. Travelocity is a sister
company of Sabre Travel Network, a global distribution
system (GDS) used by travel agencies.
In addition, three of the largest domestic branded airline
sites were selected: American, Continental, and Delta.
Here is further information about them:
American Airlines (www.aa.com), is based in Fort
Worth, Texas.
Continental Airlines (www.continental.com), is based
in Houston.
Delta Air Lines (www.delta.com), is based in Atlanta.
TESTING METHODOLOGY
The testing methodologies employed and improved upon
by Consumer Reports WebWatch since 2000 were
developed in concert with key departmental personnel at
Consumers Union, including Research, Statistics, Survey,
and Fact-checking, as well as the staff of the now-defunct
Consumer Reports Travel Letter.
All testing was conducted by testers trained by Consumer
Reports WebWatch. As always, dry-run testing was conducted
in advance to help ensure accuracy. Consumer
Reports WebWatch then conducted the comparative
analysis and provided fact checking of the final findings
and this report.
Statistical analysis provided by Consumers Union led
to the creation of four separate tests, grouped into four
separate testing days in early and mid-December 2004.
All testing was scheduled in advance and completed
simultaneously in real-time. All testers queried airfares
from previously distributed itineraries. In all cases, airfares
were available for booking, but in no cases were airline
seats purchased.
Each of the four separate tests consisted of 36 queries or
trials. In total, this project consisted of 144 separate trials.
Each of the four integrated travel sites (Expedia, Orbitz,
and Travelocity) were included in all 144 trials. Each of
the four branded airline sites (American, Continental,
and Delta) were included in one separate test consisting
of 36 trials.
However, no site produced valid flight and fare information
for all trials. Further explanation is provided below.
TESTING PARAMETERS AND SPECIFIC
TESTING CRITERIA
Each test consisted of searching for specific airfares on
high-volume domestic routes, within the continental
United States and Hawaii. Consumer Reports WebWatch
examined passenger traffic and airfare data provided
by the Bureau of Transportation Statistics of the U.S.
Department of Transportation.
The challenge was to provide geographical balance in
the routes selected. On the one hand, the goal was to
examine a wide variety of routes from major cities across
the country. At the same time, these routes were distributed
so that each of the three branded airline sites would
be fairly represented by flights departing from their
key hub and gateway airports. In this way, they could
EACH OF THE
FOUR SEPARATE TESTS
CONSISTED OF 36
QUERIES, OR TRIALS.
IN TOTAL, THIS PROJECT
CONSISTED OF 144
SEPARATE TRIALS.
CONSUMER REPORTS WEBWATCH 11
compete fairly on price and on providing nonstop flights.
In addition, most of the 36 routes selected included
competition from at least one low-fare carrier, so that richer
results would be provided. The sampling was weighted
to include many routes with large percentages of leisure
traffic, although routes with heavy business traffic were
included as well. Many of these routes offered nonstop
service from one or more carriers, and many did not.
The itineraries for all four tests remained the same
throughout the testing period. For TEST #1 through TEST
#4, the routes were:
1) Atlanta/ATL—New York-Newark/ANY
2) Atlanta/ATL—Boston/BOS
3) Atlanta/ATL—St. Louis/STL
4) Atlanta/ATL—Phoenix/PHX
5) Atlanta/ATL—Los Angeles/LAX
6) Atlanta/ATL—Seattle/SEA
7) New York-Newark/ANY—Miami/MIA
8) New York-Newark/ANY—Orlando/MCO
9) New York-Newark/ANY—Ft. Lauderdale/FLL
10) New York-Newark/ANY—Chicago/ANY
11) New York-Newark/ANY—Los Angeles/LAX
12) New York-Newark/ANY—San Francisco/SFO
13) Chicago/ANY—Philadelphia/PHL
14) Chicago/ANY—Orlando/MCO
15) Chicago/ANY—Phoenix/PHX
16) Chicago/ANY—Las Vegas/LAS
17) Chicago/ANY—Los Angeles/LAX
18) Chicago/ANY—Honolulu/HNL
19) Dallas/ANY—Houston/ANY
20) Dallas/ANY—New York-Newark/ANY
21) Dallas/ANY—Milwaukee/MKE
22) Dallas/ANY—Denver/DEN
23) Dallas/ANY—Oakland/OAK
24) Dallas/ANY—Honolulu/HNL
25) Houston/ANY—Boston/BOS
26) Houston/ANY—Baltimore/BWI
27) Houston/ANY—Miami/MIA
28) Houston/ANY—St. Louis/STL
29) Houston/ANY—San Francisco/SFO
30) Houston/ANY—Honolulu/HNL
31) Los Angeles/LAX—Boston/BOS
32) Los Angeles/LAX—New York-Newark/ANY
33) Los Angeles/LAX—Washington, D.C./IAD
34) Los Angeles/LAX—Cincinnati/CVG
35) Los Angeles/LAX—Las Vegas/LAS
36) Los Angeles/LAX—Oakland/OAK
For those cities designated with a specific three-letter
airport code, searches were conducted only for that
airport. Cities designated with “ANY” were metropolitan
areas with more than one major airport. Tertiary,
suburban, or exurban airports were not included. The
acceptable airports for those cities were:
New York/Newark: LaGuardia; JFK International;
Newark Liberty
Chicago: O’Hare International; Midway
Dallas: Dallas-Fort Worth International; Love Field
Houston: Bush Intercontinental; Hobby
Each test consisted of searching for the lowest
first-class airfares on these routes. Booking criteria were
established in advance.
For all four tests, these criteria included:
one adult
round-trip
no frequent-flyer memberships or discounts apply
(senior, AARP, AAA, government, military, student,
child, etc.)
connecting flights acceptable
multiple-airline itineraries acceptable
flight dates as specifically noted
first class as specifically noted
NOTE: Not all Web sites allowed such specificity for
each test, but these parameters were established in
advance to ensure consistency.
In all cases, testers searched for first-class routes. Prior
research indicated that searching for both business-class
and first-class fares would not be fruitful since many
carriers offer first-class service on domestic routes and
business-class service on international routes. Since this
project focused on domestic flights, the search was
limited to first-class fares.
CONSUMER REPORTS WEBWATCH 12
As stated, the nature of searching for first-class airfares
required changes to established Consumer Reports
WebWatch airline booking criteria. Specifically, this
meant accepting 1) connecting flights in addition to
non-stop and direct flights; 2) multiple-airline itineraries
in addition to single-airline itineraries; and 3) departure
and arrival times throughout the day.
These changes reflect the challenge of obtaining first-class
service on every leg of a domestic journey, since availability
could be a key factor, either on a given carrier or
on that carrier’s marketing and code-sharing partner(s).
Because the emphasis for this project was on providing
first-class fares, the latitude of the search criteria was
expanded so that nonstop, direct, and connecting flights
were allowed, as well as departure times throughout the
day. At the same time, this report provides a competitive
analysis of how each of the sites performed at providing
nonstop flights.
In order to simulate a variety of trips, the advance
booking windows varied. The booking times were:
TEST #1 (LONGER ADVANCE BOOKING):
21 days in advance
TEST #2 (LONGER ADVANCE BOOKING):
21 days in advance
TEST #3 (LONG ADVANCE BOOKING):
28 days in advance
TEST #4 (MODERATE ADVANCE BOOKING):
14 days in advance
In order to be fair to all sites tested, Consumer Reports
WebWatch departed from earlier methodology and did
not simulate a variety of itineraries by varying the length
of the stays with 2-day trips, 3-day trips, etc. Instead, all
itineraries were the same, so availability problems would
not give one site an edge over a rival. However, all trips
included a Saturday-night stay, which often reduces
airfare cost. For this project, the stays were:
TEST #1: 7 days
TEST #2: 7 days
TEST #3: 7 days
TEST #4: 7 days
SELECTION OF RESULTS
Consumer Reports WebWatch analyzed flight and
fare data by selecting either the first five returns or
the first full printed page of returns, whichever was
greater. Previously, only the first return was analyzed, but
this methodology was amended to adapt to new presentation
tools employed by travel Web sites, since vertical
presentations have become obsolete in many cases. Most
integrated travel sites (including Expedia, Orbitz, and
Travelocity) employ horizontal and vertical display tools
that show a variety of fares from competing airlines.
Note that all rankings included ties. For every fare query,
it is possible every Web site tested could have provided
the lowest airfare for that query.
It’s critical to note that for this project all valid itineraries
included first-class service on each leg of the journey. In
numerous cases, sites returned itineraries that included a
combination of first-class service coupled with economy-
MANY TIMES, SITES
RETURNED ITINERARIES
THAT INCLUDED
FIRST-CLASS SERVICE
COUPLED WITH
ECONOMY- AND/OR
BUSINESS-CLASS LEGS;
THESE RETURNS WERE
DEEMED INVALID.
CONSUMER REPORTS WEBWATCH 13
class and/or business-class legs; these returns were
deemed invalid. If a pure first-class itinerary was not
included in the first five returns or the first full page of
returns (whichever was greater), then “not available”
was recorded for that itinerary for that site.
It’s important to note some consumers might opt to book
a first-class itinerary that included another class on one
or more legs of the journey. In addition, some consumers
might opt to select a close fare outside the
parameters of the query, such as a flight departing on
a different date, or from another airport. Consumer
Reports WebWatch recognizes this, but in all cases the
testers only selected flights applicable to that specific
itinerary. Selecting other flights that did not adhere to
the specific criteria would provide that Web site with an
unfair advantage.
CURRENCY ISSUES
Consumer Reports WebWatch now tabulates total airfares,
meaning fares inclusive of all applicable taxes,
fees, and booking costs.
There were two reasons for this change. 1) Some travel
sites only provide total airfares, so using total fares offers
a better apples-to-apples comparison. 2) In many cases,
total airfares include booking fees levied by the Web
site. Therefore, using total fares offers a more accurate
tabulation of a consumer’s “bottom-line” cost. The difference
in fees, in fact, can sometimes constitute the entire
difference in cost.
For this project, all six travel sites returned fares with all
taxes and fees included in the total.
ROUNDING OF FARES
Although Consumer Reports WebWatch sometimes
rounds amounts of fares and rates, for this project exact
prices were used. This allowed more detailed analysis
of “closest” fares, since in several cases the difference
was 1¢.
CONVERSION RATES
For this project, all fares were provided in U.S. dollars.
AVAILABILITY OF AIRFARES
For this project, Consumer Reports WebWatch did not
book any fares. In all cases, the Web sites stated that the
seats requested were available. When the airfare
requested was not available, the next lowest first-class
fare that was available was used. In some cases, this
meant that multiple fares repeatedly were not available
for the same query. If no first-class fare was available for
that query, then this was recorded as well.
VALID TESTS
All sites were evaluated solely on valid tests. If there was
an error on the part of Consumer Reports WebWatch
testers, this was deemed an invalid test. Invalid tests
were eliminated and did not affect final rankings. These
errors could include incorrect data entries such as travel
dates or airport codes, insufficient returns of data,
and/or printing errors.
For this project, there were five errors in total. All five
were due to insufficient returns of data; i.e., the tester
apparently input information correctly but did not
completely print the results. Three of the errors occurred
with the American Airlines site and two of the errors
occurred with the Delta Air Lines site.
Thus the completion rate for Consumer Reports
WebWatch testers for this project was 99.1%.
INVALID DATA
Through no fault of the testers, in some cases all six sites
failed to return valid data, even after all entries were
completed properly.
In all cases, these failures affected the Web site’s final
CONSUMER REPORTS WEBWATCH 14
rankings. These failures were due to a variety of factors,
including:
First-class service was not available for one or
more legs of the itinerary.
The flight displayed was not available for booking
(with no further explanation).
The flight displayed contained “schedules or fares
not published” for booking.
The flight displayed was not available on that date.
Other failures that Consumer Reports WebWatch has
encountered in the past but did not experience with this
project include:
The flight displayed was not available at the time
specified (not applicable to this testing project).
The flight’s origin and/or departure city could not
be processed.
The airfare did not include all applicable taxes
and fees.
The booking window was too short for the site to
process.
The site stated: “Call Customer Service to process
this itinerary.”
The site experienced technical or system failures.
Specific problems with lack of availability are discussed
on page 28.
For this project, the overwhelming number of invalid
fares was due to the sites’ inability to provide first-class
fares. Specific problems with the availability of valid
first-class fares are discussed on page 28.
TECHNICAL OR SYSTEM FAILURES
During this project, testers experienced technical or
system failures several times. However, these incidents
were temporary and did not prevent the tester from
processing the queries.
It should be noted that during the course of this testing
project, Travelocity completely revamped its Web
site, including many of its display and shopping tools.
While this did not prevent the tester from obtaining flight
and fare information, this “construction phase” may
or may not have affected the quality of content
Travelocity provided.
CONSUMER REPORTS WEBWATCH 15
This project involved months of extensive research,
testing, and analysis. Much of that analysis focused
on the competitive performance rankings of each of
the sites. Besides other critical factors uncovered by
Consumer Reports WebWatch, the sites’ ability to
compete in providing flights and fares remains a critical
component of their effectiveness.
The findings of Consumer Reports WebWatch’s premium
airline Web site testing are presented in the attached
figures and are described below. There are seven major
categories of rankings:
1) Lowest first-class fares for all tests;
2) Closest first-class fares for all tests;
3) Lowest first-class fares for each of the four
individual tests;
4) Lowest first-class fares for round-trip nonstop flights.
This testing project uncovered a number of remarkable
performance results when these six sites were pitted
against each other, including:
A dramatic rise in competition among the top
three integrated travel sites (Expedia, Orbitz,
and Travelocity), with a marked increase in
razor-thin differences in fares;
A strong showing from Expedia, which led all six
sites by providing the lowest fare for four out of
every 10 queries.
An equally strong showing from Orbitz, when
the results were weighted to include “closest”
fares within $10 of the lowest fare; Orbitz
notched the lowest and closest fare for 82% of
all queries;
Despite failing to beat its two primary rivals
in these key performance categories, Travelocity
provided the highest percentage of nonstop
flights among all six sites, indicating the quality
of its lowest fare offerings was better.
The results are detailed in Figures 4 through 10.
RESULTS, RANKINGS, AND
CONCLUSIONS
CONSUMER REPORTS WEBWATCH 16
LOWEST FIRST-CLASS FARES FOR
ALL TESTS
From a business perspective, it’s impossible to underevaluate
a travel Web site’s ability to provide the lowest
fare or rate for an airline seat, hotel room, rental car,
cruise cabin, or vacation package. It is critical to a thirdparty
integrated site’s business plan. Indeed, last year
Orbitz initiated a “Low Fare Promise” guarantee, which
despite its restrictions makes clear the importance of
providing lowest rates (the guarantee is discussed in
further detail on page 34).
Travel pricing, particularly within the airline sector, reflects
one of the most competitive industries in any market.
Furthermore, it has been demonstrated historically that
most airline passengers are extremely price-sensitive and
willing to switch carriers for a very small difference in
fares, despite other compelling factors. In fact, this lack
of brand loyalty is what spurred the launch of airline
frequent-flyer programs in the 1980s.
Therefore it’s interesting to note that Expedia has dramatically
improved its performance from past Consumer
Reports WebWatch travel testing projects. As shown in
Figure 4, the site ranked first in this project by providing
the highest number and highest percentage of lowest
fares. Among 144 valid queries, Expedia provided 58
lowest fares, for an overall percentage of 40%.
Expedia’s showing was clearly better than its two
integrated Web site rivals: Travelocity at 31% (45 lowest
fares with 144 valid queries) and Orbitz at 28% (40
lowest fares with 144 valid queries).
The Orbitz showing was particularly weak in that even
one of the three airline sites was more adept at providing
lowest fares. The Continental site provided lowest
fares 31% of the time, followed by the Delta site 9%
of the time. Unfortunately, the American site failed to
provide any lowest fares.
The Continental Airlines site’s strong performance is
particularly noteworthy since it was included in only one
of the four tests. Further, the site offers only one brand –
its own –rather than a number, like Orbitz, which is
owned in part by Continental.
Expedia’s first-place finish requires historical perspective.
In head-to-head competition with Orbitz and Travelocity
LOWEST PREMIUM AIRLINE FARES (INCLUDES TIES)
FIGURE 4:
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
FARES PROVIDED FARES PROVIDED VALID QUERIES
PERFORMED
EXPEDIA 40% 58 144
TRAVELOCITY 31% 45 144
ORBITZ 28% 40 144
INTEGRATED
TRAVEL SITES
CONTINENTAL 31% 11 36
DELTA 9% 3 34
AMERICAN 0% 0 33
AIRLINE SITES
CONSUMER REPORTS WEBWATCH 17
in the last five consecutive Consumer Reports WebWatch
travel testing projects, Expedia never ranked ahead of
both of its two primary rivals in providing lowest fares
and rates. In one of these five projects, it ranked behind
one of the two sites and in the other four projects, ranked
behind both.
This project ended that streak, with Expedia ranking
first by a comfortable margin of nine percentage points.
Furthermore, Expedia’s strong showing in each of the
four individual tests made it clear the first-place overall
ranking was no fluke.
One point must be made about the rankings for lowest
fares. As discussed on page 28, Expedia failed to
provide a valid lowest first-class fare at all in a large
percentage of cases, usually because the site provided
fares for other classes of service that were deemed
invalid. Orbitz and especially Travelocity proved much
better at returning valid fares for all queries.
Therefore Expedia’s performance is a positive and a
negative. The site failed to meet basic testing criteria
for about one-third of all queries, yet still managed to
out-perform its rivals by providing a higher percentage of
lowest fares in fewer trials.
The following section on “closest” fares reveals interesting
aspects of these rankings for lowest fares. Of particular
note is that Expedia was particularly adept at providing
the lowest fare but not adept at providing a low fare
when a rival beat its price. Conversely, Orbitz became a
much more formidable competitor and a boon for consumers
when close fares were factored into the equation.
CLOSEST FIRST-CLASS FARES FOR
ALL TESTS
Consumer Reports WebWatch’s research makes clear
that competition remains fierce in the online travel market.
Ostensibly this is a good thing for consumers, provided
of course that the competition drives prices down.
The evidence seems to indicate consumers are benefiting.
Third-party sites—including but by no means limited to
Expedia, Orbitz, and Travelocity—have transformed travel
distribution, in many ways for the better for shoppers.
This is particularly true for the airline industry, which in
recent years has seen record passenger load factors
not reached since World War II. Industry analysts have
concluded that third-party sites have been a key factor in
allowing airlines to sell “distressed inventory” that otherwise
would have remained on the carriers’ financial
ledgers as empty seats.
This competition, coupled with a young industry’s maturation
process, have allowed third-party travel sites to
harness technology in new ways to refine their pricing
policies. This testing project makes it clear that travel sites
are monitoring their rivals.
That’s why Consumer Reports WebWatch continues to
provide rankings not only for those travel sites that
provide the lowest fares, but also for sites that provide
“closest” fares that are slightly higher but still quite
competitive. The amount of the closest fare varies from
project to project, depending on the travel product
being tested (such as airline seats, hotel rooms, rental
cars, etc.) and the parameters of the specific queries.
However, this amount is always determined in advance
of the testing.
For this project, it was determined $10 was an appropriate
amount for the closest fare, since first-class airline
THIRD-PARTY SITES HAVE
TRANSFORMED TRAVEL
DISTRIBUTION, IN MANY
WAYS FOR THE BETTER
FOR SHOPPERS.
CONSUMER REPORTS WEBWATCH 18
seats are often priced at double or even triple the price
of economy or coach.
As Figure 5 shows, five of the six travel sites tested
improved their performance when lowest fares were combined
with closest fares of $10 or less. Only the American
Airlines site failed to provide either lowest or closest fares.
Orbitz, however, was greatly aided by the inclusion of
closest fares. The site provided just 41 lowest fares, but
77 closest fares. This combined total of 118 meant a
whopping 82% of the 144 valid queries performed with
Orbitz resulted in a lowest or closest fare.
Travelocity also improved, as the site provided lowest
fares 31% of the time, but lowest and closest fares 57%
of the time. Interestingly, among the three integrated
travel sites, Expedia gained the least. Even though it led
all sites by providing lowest fares 40% of the time, the
inclusion of closest fares nudged that figure up to 53%.
The rankings for the Continental and Delta sites also
were assisted by closest fares, but not enough to remain
competitive with the three integrated sites. The performance
of the American Airlines site was dismal.
The Orbitz performance was quite impressive. Simply
put, when the site provided a valid first class fare, in
more than four out of five cases the Orbitz fare was
either the lowest available or no more than $10 higher
than the lowest available. Whether or not this was due
even in part to the Orbitz Low Fare Promise is not known
(see page 34).
Historically, this was unprecedented. The mark of providing
lowest and closest fares 82% of the time was the
highest posted by any integrated travel site since
Consumer Reports WebWatch began recording such
findings. And the spread of 25 percentage points over
second-place finisher Travelocity at 57% was also the
largest gap between sites ranked first and second for
providing lowest and closest fares in any Consumer
Reports WebWatch travel testing project. Furthermore, in
TEST #3, Orbitz returned a lowest or closest fare for all
33 of its valid returns, an achievement that had never
been accomplished before for a single test.
Evidence would also seem to indicate that Orbitz has
refined its search technology to increase its competitive
advantage in this arena. This is because in the most
recent Consumer Reports WebWatch travel testing
LOWEST AND CLOSEST PREMIUM AIRLINE FARES
(WITHIN $10 PER BOOKING, INCLUDES TIES)
FIGURE 5:
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
AND CLOSEST AND CLOSEST VALID QUERIES
FARES PROVIDED FARES PROVIDED PERFORMED
ORBITZ 82% 118 144
TRAVELOCITY 57% 82 144
EXPEDIA 53% 77 144
INTEGRATED
TRAVEL SITES
CONTINENTAL 36% 13 36
DELTA 15% 5 34
AMERICAN 0% 0 33
AIRLINE SITES
CONSUMER REPORTS WEBWATCH 19
project focusing on domestic sites (“Global Concerns: An
In-Depth Examination of Travel Web Sites Selling
International Airline Tickets,” released in Sept. 2004),
Orbitz also set new marks. Its ability to provide lowest
and closest fares rose to 79% (a new high at the time)
and the spread over second-place finisher Expedia was
24 percentage points (another new high at the time).
Orbitz also scored another significant achievement
during this project: It recorded the single highest savings
for any itinerary. In TEST #1, for a route from Dallas
to New York/Newark with 21 days advance booking,
Orbitz provided a fare of just $950.00 for a combined
American-Continental itinerary. This represented a 242%
savings of $1,348.70 over the $2,298.70 fare provided
by the American Airlines site. It also represented a
savings of $1,353.70 over the $2,303.70 fare on
American provided by Travelocity. Expedia was unable
to provide a first class fare for this itinerary.
For that particular route from Dallas to New York/
Newark, such savings were not an anomaly. In TEST #2,
Orbitz recorded a savings of $1,188.40 over its nearest
competitor. In TEST #3, the lowest Orbitz fare was
$598.99 less than the lowest fare from a competing site.
And in TEST #4, Orbitz generated a savings of $812.49
over its closest rival.
Upon closer examination of the test results, two discoveries
emerged: 1) There were more close fares than ever and
2) the close fares were closer than ever.
More than in any other Consumer Reports WebWatch
online travel testing project, there were many lowest
fares that were just barely lower than rival sites’ lowest
fares. In fact, it was a contributing factor in Expedia’s
ranking as the leading travel site in providing lowest
fares. In numerous cases, Expedia’s lowest fare was just
pennies less than the lowest fare provided by Orbitz
and/or Travelocity. The rivalry between Expedia and
Travelocity was particularly heightened, in that Expedia
beat Travelocity by 1¢ in several cases.
LOWEST FIRST-CLASS FARES FOR EACH
OF THE FOUR INDIVIDUAL TESTS
Analyzing the individual results for the four separate tests
that comprised this project is important for two reasons:
Although the three integrated sites were included in
all four tests, each of the three branded airline sites
were included in only one test apiece.
A key difference among all four tests was the date
of departure, so therefore the advance booking
window influenced the results provided by the sites.
As noted previously, this project consisted of four separate
tests, conducted on four days, with identical routes.
However, the travel dates for these itineraries changed
as follows:
TEST #1: 21 days in advance
TEST #2: 21 days in advance
TEST #3: 28 days in advance
TEST #4: 14 days in advance
For the most part, the individual test results were closely
aligned with the overall test results for the entire project,
when rankings for providing the highest percentages
of lowest fares were compared. Just as Expedia ranked
first overall, it ranked first in all four individual tests. But
some differences did emerge.
Figures 6 through 9 detail each of the results in order for
TEST #1 through TEST #4.
IN TEST #2, ORBITZ
RECORDED A SAVINGS
OF $1,188.40 OVER ITS
NEAREST COMPETITOR.
CONSUMER REPORTS WEBWATCH 20
Results for TEST #1 (booking 21 days in advance)
Expedia ranked first in TEST #1, which was indicative
of its performance for this project in that it provided lowest
fares 42% of the time for TEST #1 and 40% of the time
overall. In fact, Expedia’s strong showing clearly was due
to consistency, since it ranked first in all four individual tests.
TEST #1 also enabled Orbitz to post its strongest shoing,
ranking second behind Expedia by providing lowest fares
36% of the time. The American Airlines site was not a factor.
Although TEST #1 and TEST #2 both featured 21-day
advance booking windows, Travelocity did not perform
particularly well in TEST #1.
Results for TEST #2 (booking 21 days in advance)
This was the only one of the four individual tests which
did not include a branded airline site, and thus the
competition was limited to the three integrated travel
sites. Once again, Expedia ranked first, although this
time it was tied with Travelocity.
This test was the strongest showing for Travelocity. Orbitz
trailed at 25%.
LOWEST PREMIUM FARES FOR TEST #2
(21-DAY ADVANCE BOOKING, INCLUDES TIES)
FIGURE 7:
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
FARES PROVIDED FARES PROVIDED VALID QUERIES
PERFORMED
EXPEDIA 42% 15 36
TRAVELOCITY 42% 15 36
ORBITZ 25% 9 36
LOWEST PREMIUM FARES FOR TEST #1
(21-DAY ADVANCE BOOKING, INCLUDES TIES)
FIGURE 6
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
FARES PROVIDED FARES PROVIDED VALID QUERIES
PERFORMED
EXPEDIA 42% 15 36
ORBITZ 36% 13 36
TRAVELOCITY 28% 10 36
INTEGRATED
TRAVEL SITES
AIRLINE SITES AMERICAN 0% 0 33
CONSUMER REPORTS WEBWATCH 21
Results for TEST #3 (booking 28 days in advance)
TEST #3 featured the longest advance booking window
of this project, with travel dates four weeks from the date
the queries were conducted.
Expedia again led the pack, by providing lowest
fares 42% of the time. Travelocity and Expedia were
grouped a fair distance behind, at 33% and 31%
respectively. The Delta Air Lines was not much of a competitive
factor at 9%.
Results for TEST #4 (booking 14 days in advance)
This test consisted of the shortest time between reservations
and travel, with just a two-week window. Yet while all three
integrated sites did not perform up to their overall standard
for this project, they ranked in the same order.
Expedia led by providing lowest fares 36% of the time,
followed by Travelocity at 22% and Orbitz at 19%. The
Continental Airlines site far outshone its two rival airline
sites at 31%.
LOWEST PREMIUM FARES FOR TEST #3
(28-DAY ADVANCE BOOKING, INCLUDES TIES)
FIGURE 8:
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
FARES PROVIDED FARES PROVIDED VALID QUERIES
PERFORMED
EXPEDIA 42% 15 36
TRAVELOCITY 33% 12 36
ORBITZ 31% 11 36
INTEGRATED
TRAVEL SITES
AIRLINE SITES DELTA 9% 3 34
LOWEST PREMIUM FARES FOR TEST #4
(14-DAY ADVANCED BOOKING, INCLUDES TIES)
FIGURE 9
WEB SITE % OF LOWEST NUMBER OF LOWEST NUMBER OF
FARES PROVIDED FARES PROVIDED VALID QUERIES
PERFORMED
EXPEDIA 36% 13 36
TRAVELOCITY 22% 8 36
ORBITZ 19% 7 36
INTEGRATED
TRAVEL SITES
AIRLINE SITES CONTINENTAL 31% 11 36
CONSUMER REPORTS WEBWATCH 22
Results for All Four Tests
Thus the individual performances shown in TEST #1
through TEST #4 shed additional light on key issues. But
as shown above, the cumulative results of these four indi
vidual tests provide greater statistical reliability and therefore
more indelible findings.
LOWEST FIRST-CLASS FARES
FOR ROUND-TRIP NONSTOP FLIGHTS
For the first time in an online travel project, Consumer
Reports WebWatch quantified the lowest fares provided
by the sites being tested. This was done by calculating
the total number of flights that operated as nonstops in
conjunction with those that required stops enroute. The
results are found in Figure 10.
It’s important to note that these statistics reflect round-trip
itineraries that included nonstops in both directions, and
did not include stops on either the outbound or inbound
journeys. Therefore, stops were calculated in either direction
as well as both directions. This included direct flights
that stopped enroute but did not require a change of aircraft,
as well as stops that required connecting flight(s),
with a change of aircraft and in some cases a change of
aircraft and airline.
As Figure 10 shows, Travelocity returned both the highest
number and the highest percentage of nonstop flights
among its lowest fares. Of a total of 132 lowest-fare
flights provided, 83 were nonstop itineraries, or 63%.
Travelocity’s performance outpaced Expedia’s mark of
54% (52 nonstops among 97 lowest-fare flights) and
Orbitz’s mark of 46% (58 nonstops among 125 lowestfare
flights).
Among the three airline branded sites, American was
first at 48% (11 nonstops among 23 lowest-fare flights),
followed by Continental at 47% (15 nonstops among 32
lowest-fare flights) and Delta at 19% (6 nonstops among
32 lowest-fare flights).
Furthermore, some sites offered a higher number of multiple
stops within an itinerary. Therefore the total number
of stops for the total number of lowest-fare flights provided
by that site were statistically higher.
What follows is the breakdown of total stops per total
number of lowest-fare one-way flights, in both percentages
and whole numbers. In simple terms, these statistics
reflect the frequency a traveler was required to make
a stop enroute for every lowest-fare flight booked on
that site:
PREMIUM NONSTOP FLIGHTS
FIGURE 10:
WEB SITE % OF ROUND-TRIP NUMBER OF NUMBER OF
NONSTOP FLIGHTS ROUND-TRIP NONSTOP FLIGHTS
PROVIDED FLIGHTS PROVIDED PROVIDED
TRAVELOCITY 63% 83 132
EXPEDIA 54% 52 97
ORBITZ 46% 58 125
INTEGRATED
TRAVEL SITES
AMERICAN 48% 11 23
CONTINENTAL 47% 15 32
DELTA 19% 6 32
AIRLINE SITES
-CONSUMER REPORTS WEBWATCH 23
Integrated Travel Web Sites
Travelocity (34%): 90 stops per 264 one-way flights
Expedia (46%): 90 stops per 196 one-way flights
Orbitz (51%): 128 stops per 250 one-way flights
Branded Airline Web Sites
American (48%): 22 stops per 46 one-way flights
Continental (61%): 39 stops per 64 one-way flights
Delta (78%): 50 stops per 64 one-way flights
Once again, Travelocity led both Expedia and Orbitz.
One way of looking at these numbers is that—on
average—Orbitz requires a stop for more than half (51%)
of every 100 lowest-fare one-way flights it provides.
As expected, the three airline sites posted higher percentages.
But even American required fewer stops on average
than Orbitz did when providing lowest-fare flights.
In this analysis, the three integrated sites would seem to
have a built-in advantage over the three airline branded
sites. At this time, no domestic airline of any size operates
a high percentage of nonstop flights from airports
not designated as their primary hubs and gateways. For
better or worse, this is the nature of the “hub-and-spoke”
system adopted by seven of the eight largest U.S.
carriers (America West, American, Continental, Delta,
Northwest, United, and US Airways). Among the eight
largest domestic airlines, only Southwest has grown to
major carrier status (with revenues in excess of $1 billion
annually) while still maintaining a route map comprised
almost entirely of point-to-point flights rather than huband-
spoke connections.
That said, Consumer Reports WebWatch recognized
these inherent restrictions when this project’s methodology
was crafted. That’s why the routes were carefully chosen
to provide fairness to the three airlines, by including a
balanced number of their largest hubs and gateways.
Conversely, the three integrated sites—by their very
nature—would appear to be free of such restrictions.
Since they offer a variety of competing airline choices on
any given route, consumers would seem to have access
to a combination of both low fares and better service
(such as nonstop itineraries and a large selection of flight
frequencies).
That’s why it’s interesting to note that Orbitz’s ability to
provide lowest-fare flights that operated nonstop just
46% of the time trailed two of the three airline sites,
American (48%) and Continental (47%). This is quite
significant, and underscores a competitive advantage for
both Expedia and Travelocity over Orbitz.
An objective comparison of the test results for all three
integrated travel sites clearly shows that Orbitz offered
customers both good news and bad news. The advantage
is that Orbitz often provided a greater selection
of flight times. The disadvantage is that Orbitz often
overwhelmed a shopper with redundant choices, usually
from a single airline or combination of airlines on a
given route.
Consider that the single most complex lowest-fare itinerary
provided by any of the six Web sites for this project
occurred in TEST #4, when Orbitz produced a seven-leg
routing for a first-class flight from Houston to Honolulu.
Although the itinerary involved just one airline
(American), it required an outbound journey from
Houston to Austin to Dallas to Los Angeles to Honolulu
(four legs), and a return journey from Honolulu to Dallas
to Austin to Houston (three legs).
However, the price for the Orbitz flight with 14 days
advance booking was just $1,855.58. This airfare represented
a considerable savings since it was $211.71 less
than the fare of $2,067.29 on American and Northwest
provided by Expedia, and $595.52 less than the
$2,451.10 fare provided by the Continental site;
Travelocity was unable to provide a first class fare for
this itinerary.
In the case of Orbitz, the combination of a very high
percentage of lowest and closest fares overall (82%)
is the competitive advantage that seems inherently wedded
to the competitive disadvantage of a low percentage
of nonstop flights (46%) and hobnobbed itineraries
when shopping using the “lowest fare” search criterion.
Of course, users could always choose other shopping
CONSUMER REPORTS WEBWATCH 24
options on Orbitz (as well as Expedia and Travelocity).
Ultimately, this issue reflects the individual tastes of
the buyer.
CONSUMER REPORTS WEBWATCH 25
Like its predecessors, this online travel project uncovered
a variety of issues and problems that could not have been
foreseen prior to testing. In-depth analysis shed new light
on old issues and uncovered new ones. Unfortunately,
serious concerns arose as well.
There are eight important categories of findings and
concerns:
Pricing and display inaccuracies (including “farejumping”)
Availability of valid fares
Invalid first-class listings
Potentially deceptive labeling of first-class listings
Rounding down of fares
Omissions of viable airlines
Integrated travel sites vs. branded travel sites
Orbitz “Low Fare Promise”
PRICING AND DISPLAY INACCURACIES
(INCLUDING “FARE-JUMPING”)
In recent reports, Consumer Reports WebWatch has
focused attention on a topic of great concern: the propensity
for fares or rates displayed on travel Web sites to
suddenly change in real time—usually by increasing—
during the course of shopping. This problem has been
chronic among the three largest integrated travel Web
sites, particularly Orbitz.
Unfortunately, this testing project made clear the problem
has not gone away. All three integrated sites—Expedia,
Orbitz, and Travelocity—were hampered by it when
queried for first-class airfares.
The encouraging news is that Orbitz appeared to have
greatly improved in this area, with just one occurrence
recorded for this project. And not only was the fare
change a decrease, but it was accompanied by a notification
to the user.
However, a potential problem associated with fare-jumping
arose as well. With Orbitz, it became apparent that
fares often jumped or did not jump based on how the
tester tallied the cost of the trip, an arcane method that
undoubtedly many consumers would not grasp. Worse,
these jumps were not accompanied by notifications so
the user would know that the price had changed.
Consumer Reports WebWatch has published statements
from Expedia, Orbitz, and Travelocity directly
FINDINGS AND CONCERNS
CONSUMER REPORTS WEBWATCH 26
addressing the fare-jumping issue. For the most part,
the Web sites have noted that real-time travel search
engines are prone to millions of fare and rate changes
in a single day. This is duly noted. However, the issue
of failing to notify users that fares have changed is a
separate one.
As shown in Figure 11, the fare-jumping phenomenon
occurred with all three integrated travel sites: Expedia,
Orbitz, and Travelocity. It did not occur at all with
the three branded airline sites: American, Continental,
and Delta.
The problem was most chronic for Travelocity (5 occurrences
in 132 flights provided, or 4%) and Expedia
(3 occurrences in 97 flights provided, or 3%). It was
less of a problem for Orbitz (1 occurrence in 125 flights
provided, or less than 1%).
Figure 12 provides a detailed breakdown of all 9 occurrences,
including: integrated Web site; test; route; number
of increases or decreases within a single booking;
original fare; revised fare; amount of increase or
decrease; and whether or not the user was provided a
notification that the fare had changed mid-booking.
Key findings
The single largest increase occurred with Expedia, for
$748.00. The single largest decrease occurred with
Orbitz, for $610.80—but this included a service change
from first class to economy class (with a notification), so
in effect these were two different fares rather than a
single fare that had decreased.
The difference in average fare changes was significant,
with a wide spread among the three sites:
Expedia, average increase of $303.23
Travelocity, average increase of $5.32
Orbitz, actual decrease of $610.80
Both Expedia and Travelocity provided occurrences with
multiple fare jumps within the same query. This occurred
when 1) different flights with the same fare all jumped in
price; and/or 2) different flights with different fares all
jumped in price; and/or 3) the same flight jumped in
price more than once.
Notifications to users
All three of the Expedia fare jumps and all five of
the Travelocity fare jumps were accompanied with
PREMIUM “FARE-JUMPING”
FIGURE 11
WEB SITE % OF FARES THAT NUMBER OF FARES NUMBER OF
INCREASED OR THAT INCREASED FLIGHTS
DECREASED OR DECREASED PROVIDED
MID-BOOKING MID-BOOKING
TRAVELOCITY 4% 5 132
EXPEDIA 3% 3 97
ORBITZ 1% 1 125
INTEGRATED
TRAVEL SITES
CONTINENTAL 0% 0 32
DELTA 0% 0 32
AMERICAN 0% 0 23
AIRLINE SITES
CONSUMER REPORTS WEBWATCH 27
clear warnings to the user that a change in price had
occurred. Consumer Reports WebWatch has often noted
the need for such notifications and applauds sites for
providing them.
With Orbitz, a clear warning appeared saying the fare
decreased. However, that decrease was accompanied
by a change of service: although the flights remained the
same, the seats changed from first class to economy
class, rendering the fare “not available” for the rankings.
Orbitz: Potential confusion
In effect, this project uncovered a potential problem
not discovered by Consumer Reports WebWatch testers
during previous projects. This was due to a new interactive
feature on Orbitz that allows consumers to separately
select the outbound and inbound legs of a journey.
The site featured the Orbitz Matrix Display, a vertical and
horizontal grid showing a variety of fares; airlines and
airline combinations; and nonstop, one-stop, and multistop
flight choices. Below the Orbitz Matrix Display were
more traditional vertical listings of the flight choices, in
descending order from lowest-priced to highest-priced,
based on the display method chosen.
The vertical listings of flight choices included a new
option: Icons declaring “choose this flight” for both
outbound and inbound legs of the journey. When the
tester selected the outbound legs, the fare sometimes
jumped. When the tester selected the inbound legs and
built the itinerary in reverse, the fare never jumped.
EXAMPLES OF FARE-JUMPING*
FIGURE 12
TEST AND FARE NEW FARE FARE USER ALERTED
ROUTE QUOTED CHANGE TO CHANGE?
TEST #1 $641.19 $1,389.19 +$748.00 YES
ATL-LAX
TEST #2 $549.79 $640.39 +$90.60 YES
ATL-LAX
TEST #2 $952.80 $1,023.90 +$71.10 YES
ATL-SEA
TEST #4 $1,022.00 $411.20 -$610.80 YES
NYC-SFO
TEST #3 $998.00 $1,003.30 +$5.30 YES
ATL-STL
TEST #3 $692.00 $697.20 +$5.20 YES
ATL-LAX
TEST #3 $759.00 $764.20 +$5.20 YES
NYC-CHI
TEST #3 $1,016.00 $1,021.70 +$5.70 YES
NYC-LAX
TEST #3 $318.00 $323.20 +$5.20 YES
LAX-LAS
WEB SITE
EXPEDIA
ORBITZ
TRAVELOCITY
* Sites are listed alphabetically and are not ranked in any way.
CONSUMER REPORTS WEBWATCH 28
However, consumers should select a third choice—the
"Book it" icon, which thankfully is larger.
It seems apparent that some consumers would a) select
the first (outbound) legs and 2) assume they were selecting
the entire itinerary by selecting the first legs. In fact,
they would be wrong. This strange permutation of the
shopping process was not explained, and when the fare
did increase or decrease mid-booking no explanation
was provided. Furthermore, the sheer amount of flight
and fare data generated by these Orbitz searches was
staggering: Securing and selecting the correct, lowest,
first-class airfare often required printing more than 30
pages of results.
Addendum
It may be of interest to note that all five of the Travelocity
fare-jumping occurrences took place during a single day
of testing. Whether or not this was related to the revamping
of the site’s interface is unknown.
AVAILABILITY OF VALID FARES
This testing project produced more invalid airline
results than any other undertaken by Consumer Reports
WebWatch. Clearly this was due to the restrictions
inherent in obtaining first-class fares. All six Web sites
were unable to provide valid flight and fare data for at
least 2 of the queries (Delta) and as many as 47 of the
queries (Expedia).
As shown in Figure 13, the percentages of invalid results
were quite high in several cases. Among the three integrated
sites, Expedia was unable to provide a valid firstclass
flight 32% of the time, or 46 times in 144 valid
queries. Thus the site that led all other sites in providing
lowest fares (40% of the time) was unable to produce
any fare in one out of every three cases.
The percentages for Orbitz (13%, 19 times in 144 valid
queries) and Travelocity (8%, 12 times in 144 valid
queries) were much lower. Based on their business
models, it seems apparent that integrated sites would
have a clear advantage over branded sites, since the
integrated sites have a much larger pool of airlines from
which to derive valid flights.
Among the three airline branded sites, American was
unable to provide a valid first-class fare 30% of the time
(10 times in 33 valid queries), followed by Continental
11% of the time (4 times in 36 valid queries) and Delta
6% of the time (2 times in 34 valid queries).
The overwhelming percentage (96%) of invalid returns
was due to the sites’ inability to provide a valid first-class
fare. Of the 94 cases in which a flight was not available,
only 4 were due to causes other than this (i.e., flight not
available; schedule or fare not published; flight not available
for specified date).
The issue of invalid listings due to class of service is discussed
at length in the next section.
INVALID FIRST-CLASS LISTINGS
All six travel Web sites included in this testing project
were unable to process at least two queries because firstclass
service was unavailable. In each of these cases, this
was calculated into the site’s performance rankings.
When applicable, all three branded airline sites
EXPEDIA WAS
UNABLE TO PROVIDE
A VALID FIRST-CLASS
FLIGHT 32% OF THE
TIME, OR 46 TIMES IN
144 VALID QUERIES.
CONSUMER REPORTS WEBWATCH 29
(American, Continental, and Delta) were straightforward
in stating that first-class fares were not available.
Unfortunately, all three integrated sites (Expedia, Orbitz,
and Travelocity) repeatedly and continually displayed
invalid flight and fare data at the top of the integrated
listings for seats that were not first class.
Consumer Reports WebWatch recognizes that some
consumers searching for first-class fares may find value
in being presented with other options, including itineraries
comprised of business-class or even economy-class
service. But this raises two key issues:
If the user selected a search option that specifically
requested first-class service, why were other
options returned, particularly when it was not clear
these were not first-class options?
More important, why were options that did not
include first-class service returned in the integrated
listings ahead of options that were “pure” firstclass
flights?
Each of the three integrated travel sites offered functionality
that allowed the consumer to search by specific
criteria, including class of service. Muddying the results
by providing invalid flights and fares not only makes for
a more cumbersome shopping experience, but also
raises questions about potentially deceptive labeling,
which is discussed below.
POTENTIALLY DECEPTIVE LABELING OF
FIRST-CLASS LISTINGS
The difference between first class and business class was
a critical component of this testing project. In pre-test
trials, when Consumer Reports WebWatch testers queried
business-class and first-class fares, they were repeatedly
offered itineraries that combined both. Eventually the
determination was made to search for first-class flights
only, since this project was confined to domestic routes.
(Business class would have been included had the project
been expanded to include international routes.)
There may well be many consumers who would not
differentiate between business class and first class when
purchasing a domestic airline ticket. However, the price
differences were crucial when competitively ranking
multiple travel sites, as Consumer Reports WebWatch
did. Mixing the two classes would not have produced a
true “apples-to-apples” testing environment.
PREMIUM FLIGHTS NOT AVAILABLE
FIGURE 13:
WEB SITE % OF PREMIUM NUMBER OF NUMBER OF
FLIGHTS NOT PREMIUM FLIGHTS VALID QUERIES
AVAILABLE NOT AVAILABLE PERFORMED
EXPEDIA 32% 46 144
ORBITZ 13% 19 144
TRAVELOCITY 8% 12 144
INTEGRATED
TRAVEL SITES
AMERICAN 30% 10 33
CONTINENTAL 11% 4 36
DELTA 6% 2 34
AIRLINE SITES
CONSUMER REPORTS WEBWATCH 30
It’s important to note the actual differences inherent in
airlines’ premium classes can be considerable. Premium
airline products are sold by a variety of incentives,
including but not limited to additional seat pitch, more
comfortable seats, advanced onboard entertainment and
electronic amenities, upgraded meal service, complimentary
alcoholic beverages, additional onboard storage
space, airport lounge access, expedited check-in and
boarding access, and other perks.
Comparing premium classes from airline to airline
can be subjective. For many travelers, one carrier’s
economy class or economy-plus class may be equal or
even superior to another carrier’s business class.
Similarly, other travelers may consider a given airline’s
business class better than a rival airline’s first class. But
such differences and subjective analyses were not
germane to this report.
For this project, Consumer Reports WebWatch was not
concerned with the relative merits or shortcomings of
any airline’s class of service. The emphasis was not
on the product itself, but rather on how that product
was marketed and sold. In this regard, there were no
ambiguities. Individual airlines designate their classes of
service in terms understood by other carriers throughout
the world. This is one key reason airline classes are
labeled with pricing codes used by global distribution
systems, travel agents, and other airlines, so that nomenclature
is consistent and compensation for interline
tickets is equitable.
That said, all three integrated sites (Expedia, Orbitz, and
Travelocity) presented multiple problems with missing
and/or incorrect labeling information for the airlines’
classes of service.
There were no such problems with the three airline branded
sites (American, Continental, and Delta).
In some cases, integrated sites provided airline fares that
were labeled as first class even though that carrier did
not provide first class. These listings were deemed invalid
and in some cases caused the site to be ranked as “not
available” for that query.
In particular, problems with four low-fare carriers
emerged repeatedly: AirTran Airways, ATA, Spirit
Airlines, and USA 3000 Airlines. All four of these
carriers offer premium products that are not marketed as
first class, nor are they sold using first-class pricing
codes. In fact, in the cases of AirTran and ATA, these
products are actually referred to as “Business Class” on
both of the carriers’ own branded Web sites.
This is a separate issue from travel sites providing itineraries
that included legs that were clearly labeled as
economy class or business class. In those instances, the
sites failed to respond properly to the queries. But mislabeling
the products sold by individual airlines is a more
serious concern.
There were also cases where one integrated site referred
to a specific airline product correctly, while a rival integrated
site did not. Spirit, for example, offers Spirit Plus,
a premium class described on the carrier’s own Web site
in terms that would suggest an economy-plus or businessclass
product. Expedia correctly labeled it as business
class, while Orbitz labeled it first class. Because Orbitz
repeatedly provided lowest fares on Spirit in response to
queries for first-class fares, several Orbitz trials were
deemed “not available” under the conditions outlined in
the methodology section of this report.
At the same time, many of Expedia’s queries were
deemed “not available” because the itineraries included
Spirit’s business-class flights. The point was that in neither
case was Spirit acceptable, but at least Expedia was not
potentially deceptive, as Orbitz may have been.
What follows is a breakdown of the labeling problems
found on each of the three integrated sites.
Expedia
In response to a query for a first-class flight, Expedia
provided flight and fare information from multiple
carriers but did not label the airline’s class of service in
these integrated listings. Only after the user selected a
given flight, and the “total price for this trip” page
appeared, was specific information provided on the
class of service for each leg of the journey.
CONSUMER REPORTS WEBWATCH 31
This practice made for a confusing shopping process. It
was of particular concern when flight and fare information
was provided for classes of service other than the
class requested (first class), as occurred repeatedly
during this testing project. In effect, the tester had to
repeatedly process an additional step by tallying the
“total price for this trip” page, before finally discovering
many of the listed flights were invalid due to the wrong
class of service.
In multiple instances, airline products were included at the
top of the integrated listings even though these products
were not first class as queried. Here are some examples:
AirTran: listed with first-class offerings, labeled
business class
Spirit: listed with first-class offerings, labeled
business class
ATA: listed with first-class offerings, labeled
economy class
United: listed with first-class offerings, labeled
economy class
To its credit, Expedia properly listed the carriers’
classes of service, once the additional pricing step was
completed. But it still made for a cumbersome search.
Orbitz
The good news was that Orbitz was the best among the
three integrated sites at clearly labeling each leg of an
itinerary by the airline’s class of service. This provided
instant confirmation (unlike Travelocity) and eliminated
an additional shopping step (unlike Expedia).
The bad news was that Orbitz repeatedly mislabeled
airline products as first class when they were not. Here
are some examples:
ATA: labeled first class, actually business class
Spirit: labeled first class, actually business class
USA 3000: labeled first class, actually business class
Orbitz did correctly label AirTran as business class.
However, it’s unclear why the premium products offered
by these other three carriers were mislabeled. It’s
certain that many consumers booked these flights
believing that they offered first class service when in
fact they did not.
Travelocity
Travelocity did not label the classes of service for flights
when they were initially provided in the integrated listings
in response to queries for first-class service.
However, the site did label the class of service only when
a flight included a leg or legs that were not first class.
Travelocity offered a helpful flag that stated “First class
not available; Business class [or economy class] operated
by [airline name].” The problem with this warning
was that a user could reasonably expect that the absence
of such a flag meant that the listed flight contained firstclass
service only. Unfortunately, this was not always the
case. In one instance, a business-class seat on AirTran
was provided without indicating that it was not a firstclass
seat.
Travelocity did offer an additional click-through function
that provided “fare basis” codes for each leg of a journey.
However, these complex codes used by industry
IN SOME CASES,
INTEGRATED SITES
PROVIDED AIRLINE
FARES THAT WERE
LABELED AS FIRST CLASS
EVEN THOUGH THAT
CARRIER DID NOT
PROVIDE FIRST CLASS.
CONSUMER REPORTS WEBWATCH 32
professionals were offered to consumers with no explanation
or translation. In fact, in several cases, Consumer
Reports WebWatch translated these codes and discovered
that AirTran was listed as business class, not first class.
This mislabeling seemed to be an anomaly, however. In
several other cases, Travelocity properly flagged AirTran
flights as business class. In addition, Travelocity also
properly flagged Spirit flights as business class and in
some cases flagged ATA flights as economy class.
Note: As stated earlier, Travelocity completely revamped
its graphic interface and search tools after this testing
project was launched. These specific criticisms refer to
the search tools that were in place during this testing
period in December 2004.
Summary
The awkward display tools and incorrect data provided
by the three integrated sites when searching for first-class
fares was a disturbing discovery. Consumer Reports
WebWatch had not encountered such basic interface
problems while searching for economy-class fares over
the past several years. In many ways, these fundamental
functionality problems associated with first-class bookings
harkened back to the cruder tools used in the
infancy of online travel.
Clearly, all three major integrated sites need to revamp the
shopping tools provided to consumers searching for premium
fares. In some cases, the tools were seriously flawed.
In the worst cases, they may have been deceptive.
ROUNDING DOWN OF FARES
Both Orbitz and Travelocity presented fares in the integrated
listings that were rounded down to the nearest
dollar, even when the amount should have been rounded
up (i.e., 50¢ to 99¢). Consumer Reports WebWatch has
noted this issue before in conjunction with both sites.
When tabulating fares provided by Orbitz and
Travelocity, Consumer Reports WebWatch corrected the
amounts in accordance with Internal Revenue Service
policy. That is, if the actual fare was $99.99, and Orbitz
or Travelocity displayed the fare as $99.00, Consumer
WebWatch tabulated the fare as $100.00.
Travelocity and the three branded airline sites returned
actual amounts in all cases.
OMISSIONS OF VIABLE AIRLINES
A critical component in understanding the online travel
market is awareness of the product “universe” a given
travel site offers the consumer. That is, no single integrated
site offers all available travel products, whether
the sector is airlines, hotels, car rentals, cruises,
or vacation packages. And it’s understood nearly all
branded airline sites offer specific products, usually
from a single airline or combination of that airline’s
marketing partners.
In the domestic airline sector, there are two key
omissions among nearly all integrated travel sites:
Southwest Airlines and JetBlue Airways. In an effort to
contain their costs, both of these carriers have developed
distribution strategies that for the most part bypass thirdparty
travel sites as well as global distribution systems
used by travel agencies. This is one significant reason no
integrated travel site can truly claim to offer the lowest
fares on hundreds of routes across the U.S.
However, neither Southwest nor JetBlue offer premiumclass
service, so for this testing project their omissions
had no effect. In fact, by focusing on first-class domestic
fares, this testing project may have provided the best
head-to-head competition among the leading U.S.
integrated travel sites ever conducted by an unbiased
testing organization.
That said, Consumer Reports WebWatch urges all travel
sites to offer consumers a complete listing of all travel
suppliers available on each site. This provides the user
with a much better sense of the “universe” of products
being queried, as well as the products that fall outside
that scope. Happily, this request has been fulfilled by
some travel Web sites, though not all.
CONSUMER REPORTS WEBWATCH 33
INTEGRATED TRAVEL SITES VS.
BRANDED TRAVEL SITES
Whenever Consumer Reports WebWatch conducts a
testing project that includes both integrated travel sites
and branded travel sites, one of the most intriguing
questions that arises is: Did any of the integrated sites
offer a lower fare or rate than a branded site did for its
own product? Once again, the answer is yes.
For this project, however, Consumer Reports WebWatch
conducted some additional analysis to provide a
clearer view.
Overall, past projects revealed an interesting dichotomy:
In the broadest of terms, integrated travel sites provided
lower prices for airline seats and rental cars, but branded
sites provided lower prices for hotel rooms. For this
project, that general axiom held true for two of the
airlines, but not for the third.
As shown in Figure 14, there was a clear delineation
between Continental Airlines on one side and both
American Airlines and Delta Air Lines on the other.
During 36 valid trials, none of the three integrated sites
provided a single first-class fare on Continental lower
than the lowest first-class fare provided by the carrier’s
own Web site.
Conversely, one or more of the three integrated sites
provided a lowest fare on Delta that was lower than
Delta’s lowest fare 21% of the time (7 fares for 34 valid
trials). And one or more of the three integrated sites
provided a lowest fare on American that was lower
than American’s lowest fare 30% of the time (10 fares in
33 trials).
Continental’s performance would indicate it is more
aggressive than its two airline rivals in providing competitive
fares under its own online banner. Full-service
airlines such as these three currently face intense marketing
pressure to sell their products through a variety of
distribution channels, particularly third-party integrated
sites that can effectively promote last-minute “distressed”
inventory. At the same time, these airlines have
expressed concerns about diluting their own pricing
models by conditioning consumers to expect lower fares
on their own branded sites.
Despite or because of these complex sales strategies,
it’s apparent that Consumer Reports WebWatch found
PREMIUM FARES: INTEGRATED SITES VS. BRANDED AIRLINE SITES
FIGURE 14
CONTINENTAL 0% 0 36
DELTA 21% 7 34
AMERICAN 30% 10 33
%OFLOWERFARESFOR
THAT AIRLINE PROVIDED BY
ONEORMOREOFTHE
INTEGRATED TRAVEL SITES
NUMBER OF LOWER FARES
FOR THAT AIRLINE PROVIDED
BY ONE OR MORE OF THE
INTEGRATED TRAVEL SITES
NUMBER OF VALID
QUERIES PERFORMED
AIRLINE SITES
CONSUMER REPORTS WEBWATCH 34
two separate methodologies, as the fares provided
by Continental and American/Delta made clear. For
consumers, it’s equally apparent that Continental’s own
Web site deserves a close look.
THE ORBITZ “LOW FARE PROMISE”
On Oct. 26, 2004, Orbitz began to advertise a “Low
Fare Promise” for consumers purchasing airline seats that
provided a coupon redeemable only on Orbitz for $50
in future travel purchases. Earlier in 2004, the Web site
launched the OrbitzSaver Low Rate Guarantee for hotel
rooms, but offering a price guarantee was not common.
However, even within the short history of Internet
commerce the move was not without precedent, due to
Hotels.com. In early 2003, Consumer Reports WebWatch
conducted comparative testing of online hotel products,
which was summarized in “Booking Hotels Online: An
In-Depth Examination of Leading Hotel Web Sites”
(http://www.consumerwebwatch.org/dynamic/travelreport-
booking-hotels-online-abstract.cfm. The report
specifically examined the Hotels.com rate guarantee
and found that, “Overall, among the 130 queries in
which Hotels.com provided a rate, that rate was beaten
by its competitors a total of 99 times (including multiple
times for the same query). In all cases, of course, the
rates were queried for the same dates simultaneously.”
The Orbitz promise carries a lengthy list of caveats
and restrictions, which are available to consumers at
www.orbitz.com/global/legal/terms.jsp?type=lfp.
Among the key provisions of the promise:
it applies only to domestic flights;
it applies only when a rival site offers a fare that is
at least $5 lower;
it applies only for identical dates, airlines, flight
numbers, classes of service, etc.;
it applies only when a lower fare is found on a
“full-service travel Web site or airline site.”
This testing project took place in December 2004, so the
Orbitz Low Fare Promise was in effect. Since Orbitz
provided the lowest first-class fare just 28% of the time,
clearly at least one rival site provided a lower first-class
fare 72% of the time. But as noted earlier, this project has
shown that in terms of overall competitiveness, Orbitz
handily led all six sites by providing the lowest or closest
fare 82% of the time.
The specific restrictions of the Low Fare Promise severely
reduced the number of cases that would have provided
legitimate evidence for a valid claim under the conditions
of the policy.
Among all 144 trials for this testing project, there were
only four cases in which one or more rival sites provided
a lowest fare that was more than $10 lower than the
lowest fare provided by Orbitz for that same airline. This
in itself was a remarkable achievement.
Of those four cases, in only one instance would the
Orbitz Low Fare Promise have been invoked legitimately.
In TEST #1, from Los Angeles to Oakland, Orbitz provided
a lowest first-class fare on United for $466.00, while
Expedia provided a lowest first-class fare on the identical
United flights for $455.20, or $10.80 less. Considering
the high number of valid queries performed (144), this
seems to be a strong showing for Orbitz.
Whether or not Orbitz’s performance in providing lowest
and closest fares was due even in part to the implementation
of the Orbitz Low Fare Promise is unknown. What
is known is that the existence of the new policy is
undoubtedly forcing the Web site to remain competitive
with its largest rivals. Therefore even those consumers
who do not directly benefit from the Low Fare Promise by
invoking it may still be well served by it.
CONSUMER REPORTS WEBWATCH 35
Buying travel via the Internet can be fairly challenging,
but buying first-class flights can present a whole new set
of issues. Take time to review the following advice before
you book your next trip online.
Unfortunately, the “fare-jumping” issue requires that
you remain vigilant throughout the shopping process,
because prices and flights sometimes can suddenly
change mid-booking.
Always ensure the class of service you are booking is
the class of service you requested. Consumer Reports
WebWatch found that travel sites sometimes offer economy
class or business class when first class is requested.
Furthermore, the class of service isn’t always defined.
Always confirm that the airline you’re booking offers
the class of service you requested. Consumer Reports
WebWatch found that travel sites sometimes label the
class of service incorrectly, so that business class is
offered as first class, for example.
Always confirm that the class of service you are booking
is available for every leg of your journey. Consumer
Reports WebWatch found that travel sites sometimes offer
itineraries that include a mixture of two or more classes,
despite requests for a single class.
Yes, you still have to shop around. There is no one
travel site that completely dominates its competitors.
Furthermore, the differences in price for any given itinerary
can be considerable.
TIPS FOR BOOKING
PREMIUM-CLASS AIRLINE SEATS
ALWAYS ENSURE THE
CLASS OF SERVICE YOU
ARE BOOKING IS THE
CLASS OF SERVICE YOU
REQUESTED.
CONSUMER REPORTS WEBWATCH 36
Always make sure the “bottom-line” price includes
all taxes, fees, surcharges, and booking costs levied by
the Web site itself. These required add-ons can change
the fare.
Remain flexible whenever possible. A relatively minor
change (say, moving your departure date back by one
day, or departing from a secondary airport) can have a
dramatic effect on the total airfare.