A new advertising program rolled out this month by Treadmill Doctor's equipment review site -- as well as the ratings, words and prices on some reviews -- has unleashed a furious debate in the fitness industry about the legitimacy of review sites in general and about the revered -- and sometimes berated -- Treadmill Doctor in particular.
On the heels of a SNEWS® investigation into the Treadmill Sensei review site that showed the "Sensei" had lied about identities and experience while linking his income and seemingly his ratings to affiliate earnings (click here to see that Sept. 24 story), the rash of emails and phone calls avalanching into the SNEWS® offices escalated. The industry knew that the Treadmill Doctor website was going to launch its new flotilla of many hundreds of reviews in October while also for the first time unveiling a sponsorship program.
Of all the reviews sites and magazines, Treadmill Doctor (www.treadmilldoctor.com) is one of three that causes the most industry debate. The others are Runner's World and Consumer Reports. Runner's World, part of the larger Rodale Press group, carries the clout of huge verified readership numbers of devout runners; Consumer Reports is a non-profit, advertising-free group with engineers and biomechanists on staff. SNEWS® has covered the three sites, their reviews and logic in numerous past stories. Treadmill Doctor, based in Memphis, Tenn., was founded by brothers, Jon and Clark Stevenson nearly 10 years ago as a way to sell equipment parts on the Internet. Both had worked during high school and college in fitness stores doing sales, repairs and service, according to the site. Jon Stevenson said the business added reviews as a way to drive traffic to the website.
"We're a parts company," Stevenson told SNEWS®. "The reviews are the biggest headache we have."
In mid-2006, the two brought on former Spirit and True sales manager Woody Fisher and, together, the three canvassed the floor at the 2007 Health & Fitness Business show in August selling a new pay-per-click advertising program. Only one other time had the company used ads on its reviews -- about five years ago and that endeavor was short-lived. This round, however, Stevenson told SNEWS®, the company like any business needed to get a reasonable return-on-investment from the time and money it spends doing reviews otherwise, "I will drop reviews."
"It has to support itself," he said, "or it has to go."
That new Treadmill Doctor program's premise was to allow links on each manufacturer's brand home page and the product page of each piece of equipment reviewed (for example, "click here for more information on XXX treadmills") that would, in theory, link to that manufacturer's website -- but only if the manufacturer ante'd up to pay a hefty fee to Treadmill Doctor every time someone clicked on the link. Links declined by one manufacturer on its own pages could be bought by another (for example, on brand BBBB page, the link would read "click here for more information on YYYY ellipticals").
Manufacturers opting in to the program, which went live with the reviews the first week of October, are paying $2.50 per click -- significantly higher than most pay-per-click (PPC) programs which can be 25 cents to 50 cents, although adult and gambling sites can commonly get $2 or more, according to PPC experts.
Does Stevenson see that rate as high? "Absolutely," Jon Stevenson said. "Because I want to make money.
"People who are looking at Treadmill Doctor reviews are interested in treadmills or ellipticals for some reason. They've found their way to Treadmill Doctor. They're highly filtered. That demands a premium," Stevenson said.
"Do you like us or not, or are you getting a good review or not, the people who are buying treadmills are coming to our website," he added.
The Treadmill Doctor team claims 3.4 million visitors each year. Other traffic rating websites have statistics for the site -- noted in some cases as estimates since sites with low traffic are difficult for the ratings services to quantify -- that are a fraction of that or as low as 360,000 a year or 30,000 unique visitors a month.
"They're way off," Stevenson said about conflicting estimates. When asked why he didn't have his rates independently verified to substantiate his hefty PPC rates (like publishers do with magazines or trade shows do with attendance figures) he said, "Why should I? That costs me money."
Not as if verification is needed for word-of-mouth to work its wonders. A recent study published in early October by eMarketer.com found that recommendations from consumers were trusted by Internet users 78 percent of the time, but that consumer opinions posted online, brand websites and brand sponsorships are still trusted upward of 49 percent of the time. (Click here to see that study.)
Advertisers, top brands and reviews
Of 52 treadmill brands listed on the treadmill review page, eight brands and all nine Icon -- or 17 total brands -- have opted into the PPC program for about a third of the brands on the site.
Every single "Best Buy" award winner of 32 except just one (one Best Buy and three runners-up in eight price categories under $3,999) is an advertiser. In the top price category ($4,000 and up) and two optional categories (Best Fold Up and Best New), of 12 awarded pieces, six are non-advertisers, although all three No. 1 pieces are from participating PPC advertisers.
"All I can say is, it falls the way it falls," Stevenson said, when asked about the statistics. "If somebody wants to knock us for that, they should."
Of 37 elliptical brands listed, the same 17 have chosen to take part or less than half (45 percent). Still, in the elliptical category -- only the second year the brothers have undertaken that category -- only three award winners of 24 (one Best Buy and three runners-up in six price categories under $2,499) are not participating in the program. In the three higher-end price categories, none are participating advertisers, although all four winners in the "Best New" category are advertisers.
"If you have to say, 'It's fishy,' I think that's fair," Stevenson said. Does it look fishy to him? "Not for me. I never looked at it that way."
He added, "A lot of these people didn't commit to advertising until the reviews were finalized."
PPC advertisers are Bodyguard, Diamondback, Horizon, Landice, Lifespan, PaceMaster, Spirit and True, as well as the Icon brands Epic, FreeMotion, Gold's Gym, Healthrider, Image, NordicTrack, Proform, Reebok and Weslo.
"Is there anything objective?" he asked. "Is there any objective review?"
Treadmill Doctor also only allows its PPC advertisers to use the term "Best Buy" award (which it calls "intellectual property") in advertising or promotional efforts either online, in print or in any other materials. Stevenson said the company can't trademark or copyright the term, but its "look and feel" can't be used in a way that would be confusing, such as the way he said they have always used the terminology ("2005-2006 Best Buys"), although this year it diverted from that standard ("2008 Treadmill Best Buy Awards").
To the site's credit, no advertisers or any part of the program are veiled or hidden, as it is on sites that make money off affiliate fees, such as Treadmill Sensei. Without exception, every PPC link has the tag "sponsored link," following the Google model. All participants also pay the same fee per click and can put a cap on how much they will allow each month after the minimum $100.
For the vice president of marketing at Icon, the program doesn't raise eyebrows.
"It makes the reviews actionable," said Colleen Logan of Icon. "We want to sell product and people research big ticket items online. And it says 'sponsored link.' It's very transparent.
"The proof's going to be in the pudding," she said, "when it comes to sales. It's a qualified audience, theoretically, and that could justify the higher costs. If it converts to sales, great."
Not every manufacturer is so convinced. Many wouldn't talk on-the-record for fear of reprisal since the reviews are brought into stores by customers. And some simply don't advertise with review websites.
"We don't believe in advertising on independent review sites because it would appear we're trying to buy votes," said Chris Cox, director of marketing at Vision Fitness, which is not a participant and, for the record, racked up no treadmill awards and one elliptical award (No. 2 in the $2,500-$2,999 category where no winners were PPC participants).
Aside from allegations of bias, what rankles a number of both manufacturers and retailers in the industry is the attitude both by the team on its own blog and the choice of words on some reviews.
SNEWS® heard both retailers and manufacturers call the Treadmill Doctor blog entries "pompous," "arrogant" or "nonsensical." This is an entry from September 2007:
"…we will be launching the reviews for the 2008 season very soon. We have, unwittingly, been thrust into a position of some authority with regards to our reviews, and authority begets responsibility. To that end I want all people to understand that we have a single test for our reviews when they are written before we post them to the website. Are they reasonable? Will others be able to see where we are coming from when they read the reviews? Yes we are sarcastic, and yes we serve a fair amount of ridicule as well, but frankly much of the ridicule is well deserved. So if you don't like what we are saying … too bad…."
And from an entry in August: "…before you buy that treadmill or elliptical make sure that you look at our upcoming reviews. We are taking the staying power of the company in to account a little more than usual this year."
Others have called a few of the review comments endeavor everything from "slanderous" and "a piece of crap" to "crass" and "locker room stuff." Still, the comments are equal opportunity, meaning they are aimed at both advertisers and not.
A few examples and links (there are more to be found on the site):
>> Proform Crosswalk 415: "Your pants will be around your ankles and you'll never know your belt was undone." And, "this one is essentially a tread where the consumer gets screwed."
>> Octane Q35: "They are just trying to do to you (the consumer), what the Bull does to the Cow. If you need further explanation go talk to your mother."
>> Bowflex TreadClimber TC5000: "If you consider this one, you might want to go to your doctor and get a saliva test because we think you may have been bit by a rabid animal."
Said Stevenson about the no-punches-pulled tone, "I ridicule some people. I've wondered if it's necessary,… but we've done it for 10 years…. Some of these guys earn these comments."
But retailers, who hear from consumers about the site, have said they aren't convinced:
"It's unprofessional and childish," said Bryan Dorksen, co-owner of retailer AtHome Fitness in Phoenix, Ariz. "If they were legitimate, they'd not make personal comments."
And from John Carter, owner of retailer Home Fit Equipment in Kitchener, Ontario in Canada: "I find it very irritating."
How they are done
The reviews are done, the brothers have always said, based only on a piece of equipment's durability and reliability -- black-and-white stuff -- since they come from a background of parts, repair and service. "That's always been our hook," Stevenson said.
Yet in a conversation with SNEWS®, Stevenson said the "ergonomics" rating in the elliptical category is "strictly my opinion." Same with the "value" rating in both sets of reviews, which he called "a combination thing" with some of it his opinion.
"It comes down to, 'What's my opinion of the value?'" he said. "A lot of that is subjective."
And Stevenson, despite the claim about the nuts-and-bolts of motors, belts and repair records, re-iterated what's on the company FAQ, "At the end of the day, it's about our opinion. We have never said it's not subjective."
But with some 500 individual pieces of equipment reviewed, some question how the Stevenson brothers can afford to buy many of them, as they say they do, or even get them all done in just a few months as they are entering the country or rolling to market.
"In person, they say, 'We know nothing about feel. Yet they are judging feel with the ergonomics rating," said Tim Porth, co-founder of Octane Fitness.
"They also did some reviews on machines that haven't been out very long," Porth said, "and I question their ability to have gotten the pieces and done them…. How could they go out and do their due diligence on all these machines? No way."
With one brand, the prices on several pieces cited on its pages on the Treadmill Doctor website have come into question:
>> The Landice L7 Sport Trainer suggested retail is, per the company, $3,200. The price on Treadmill Doctor is $2,495. It won a Best Buy award in the $2,000 to $2,495 category that it eked into.
>> The Landice L8 Sport Trainer has a suggested retail of $4,100, but is listed as $3,995 on the site, slipping down a category -- and earning it a Best Buy.
>> The Landice L7 Pro Sports Trainer's retail is $3,600 and is listed on Treadmill Doctor site as $2,995 -- dropping it dramatically and earning it a No. 2 rating.
Landice declined to comment about the prices differences. Stevenson said he got the prices from the company: "I have to go by what they give me." A current price list shows the higher retails.
Retailers have told us they have lost sales or had to deal with angry customers claiming the retailer was trying to pull the wool over them -- sometimes unsuccessful in their calming attempts.
Same goes with the Treadmill Doctor's comments about the high-end Octane Q47ce elliptical, telling readers it's not worth the money and to consider a Life Fitness X7 or Precor EFX 5.23 instead. That has Octane up in arms, since the Life Fitness model and the Precor model cost the same or more. Precor, of course, has a name in the category, being the founding supplier, but some would say that Octane's elliptical offers more features than either. Porth said he knows of competitive dealers passing out negative comments to sell other brands. Usually, he said, his dealers can combat it by showing other reviews, including the SNEWS® independent survey of retailers (which only asks for opinions about companies, tallies and reports them).
"How many people don't look at our product because of it?" Porth asked. "We don't know."
For some companies, like Vision or Precor, the best offense seemingly is just not to partake in sparring or even communication:
"Clark (Stevenson) did ask if, basically, we wanted to protect ourselves by buying up all of the advertising on pages our product reviews appeared," said Precor spokesman Jim Zahniser. "We chose not to participate. And he has reiterated that the site is unbiased, and advertisers don't get special consideration.
"But, in our opinion, the more directly a third-party information source integrates commerce with content, the more directly it undermines its own credibility. Consumers are quick to sniff out the shift in the balance, and take this into account," he said in an email.
"Right now the site is relying on advertising support from the very companies being reviewed and that isn't a model that we'd choose to participate in as an advertiser…. But the fact is that we've taken issue with product reviews in mainstream media as well, most often due to lack of diligence or even a basic familiarity with the product category. So we're fairly circumspect about product reviews as a whole, not just this site."
As the reach of the Internet has increased and consumers have learned to rely on the Internet for pre-purchase research, review sites have proliferated.
"They're a part of our industry," said Bob Lachniet, owner of retailer Fitness4Home Superstore in Phoenix. "If you get a rating in your favor, of course, you're going to use it to your advantage, and you find a way to get around a bad one."
SNEWS® View: Treadmill Doctor has retained some clout in the online review category and it has been very clear in print -- despite initial fears floated in the industry -- about what is a sponsorship link on its site. That's all good. The site founders have also been pretty forthright about much of it being their opinion and being subjective, as of course any review will inevitably be in some regard since none of us can totally eliminate personal biases. We at SNEWS® have always chosen not to rate gear against each other but simply against itself as one way to combat the debates that can erupt. (Click here to read more about how we conduct reviews.)
But the clout of Treadmill Doctor perceived by the industry carries with it a burden of responsibility -- to act professionally. We think using some of the crass, insulting, pre-teen terminology in reviews is uncalled-for and should be eliminated. We also believe it is pretty much impossible for the company's staff to indeed have gotten under the hood of every single piece of about 500 pieces of equipment reviewed. Certainly not in the time span that it would have needed to occur. Certainly, some of the pieces are nearly the same as the previous year's and the reviews only require a touch-up, but they still require a look and at least a basic test drive to be sure that's all that's needed. We are skeptical that they can really know every piece that intimately when they are posted in one fell swoop in October and, seemingly, all written in just a few weeks. Heck, we're writers here and have been reviewing product professionally for over 20 years and we'd be hard-pressed to review, analyze and write that many stories in such a short time.
Not to say that a really good review site isn't possible and needed. The Treadmill Doctor, if it brought in independent experts and perhaps did reviews under a separate non-profit umbrella, could be the group to do that. Noise levels can be measured, ergonomics can be analyzed, and value ratings can be laid out in black-and-white. The industry, like many others, needs and deserves better. The Treadmill Doctor is a business and, of course, needs to be operated as one. Heck, none of us have businesses just to do a good deed, even if we love what we do. We are there to make a living. Treadmill Doctor has that right too. But when as a part of making its living it allegedly damages others, publishes what is called incorrect material (the Landice prices, for example) and uses unprofessional language, the industry needs to stop hiding for fear of a bad review and collectively speak up to effect change. We suspect the folks at Treadmill Doctor would listen.
We have heard from Stevenson that he is thinking of allowing others on his blog from the industry. We think that's a great idea -- like a community or industry forum. Of course, that too carries a great deal of responsibility for the industry that may participate. Tossing mud, slinging shots or hiding behind fake identities isn't appropriate and, we're sure, wouldn't be tolerated. We'll be the first to call that out too. Meanwhile, if the Doctor is going to try to stand above it all in a swath of angelic white -- all the while standing behind the over-used shield of "it's only our opinion" -- it will need to take a more proactive and supportive stance toward ensuring editorial and review integrity. The industry deserves at least that.