m$ slammed for badmouthing benchmark
 

Manages to make Vista look crap


A VOLE has whinged about recent benchmark testing that compared the performance of Windows XP SP3 with that of Windows Vista SP1.

Benchmark test results released by Devil Mountain Software two weeks ago concluded that Microsoft Office 2007 ran a suite of productivity tasks twice as fast under XP SP3 RC than under Vista SP1 RC.

Subsequent testing with 2GB of RAM using Microsoft Office 2003 still showed XP SP3 RC to be 1.8 times faster than Vista SP1 RC. Posts describing those benchmark tests are available here.

Nick White, a Vista launch team Product Manager, wrote in his bog Friday that Microsoft hasn't released any benchmark testing results for Vista SP1 because " the code is still in development and... remains a moving target."

He also denigrated script-based benchmark testing because he claimed it " isn't representative of real-world user behavior and hence isn't an accurate guage of the actual end-user experience."

Embedded in his bog post is fast-motion video clip of a Windows Vista desktop doing something, we're not even vaguely sure what.

Research staff at Devil Mountain Software fired back, writing: "Mr. White blabbers on a bit about unrealistic benchmarking, then proceeds to badmouth our OfficeBench test script by calling it nothing more than a 'window-open, window-close' routine. He even includes a ridiculously accelerated video of our test script in action, using the 'speed' at which it executes as an excuse for discrediting its validity."

They go on to defend the development, version independence and sophistication of their benchmarking software. "OfficeBench scares the hell of out Microsoft." they claim. "For the first time ever the industry has the tools necessary to call the company to the mat for its bloated, CPU cycle-sucking ways."

"Microsoft's response? Slam the benchmark! Try to discredit the author/source! And crank-up the FUD machine! Sorry, guys! You can run, but you can't hide, from OfficeBench."
Great fun.

The INQuirer