yagmark - yet another graphics benchmark
Yagmark is a script to automatically install and run several Windows graphics benchmarks (currently 3dmark2000, 3dmark2001, 3dmark06, and Unigine Heaven 2) on either Windows or Wine.
The script itself is at http://code.google.com/p/winezeug/source/browse/trunk/yagmark.
It assumes that winetricks and wisotool are in the same directory, so it's best to grab the whole svn tree with the command
svn checkout http://winezeug.googlecode.com/svn/trunk/ winezeug
Then you can run one or more benchmarks, e.g. 3dmark2000 and 3dmark06, on Wine with commands like
cd winezeug sh yagmark 3dmark2000 3dmark06
Result Format
The script saves its results in (surprise!) the 'results' subdirectory, in a directory whose name describes the current system (the format is cputype-gputype-os-hostname-wineversion). Each time you run the script, it saves the output of all the tests you selected in a file named systemid-date-time.txt, and it computes the median of all runs so far and saves it in a file named stats.dat (which is a text file, named .dat to avoid confusing the script).
In stats.dat, each line is of the form benchmark_variable \t value. For instance, if you run "sh yagmark 3dmark06 3dmark2000", it will contain lines like this:
3dmark06_3DMark_Score 5807.000000000 3dmark06_CPU1_Red_Valley 1.523867607 3dmark06_CPU2_Red_Valley 2.057754278 3dmark06_CPU_Score 4425.000000000 3dmark06_GT1_Return_To_Proxycon 21.619014740 3dmark06_GT2_Firefly_Forest 16.834365845 3dmark06_HDR1_Canyon_Flight 18.595914841 3dmark06_HDR2_Deep_Freeze 21.234073639 3dmark06_HDR_SM3_0_Score 1991.000000000 3dmark06_SM2_0_Score 2311.000000000 3dmark2000_3DMark_Result 21530 3dmark2000_CPU_Speed 1081 3dmark2000_Game_1_Helicopter_High_Detail 210.6 3dmark2000_Game_1_Helicopter_Low_Detail 343.1 3dmark2000_Game_1_Helicopter_Medium_Detail 261.6 3dmark2000_Game_2_Adventure_High_Detail 175.0 3dmark2000_Game_2_Adventure_Low_Detail 531.5 3dmark2000_Game_2_Adventure_Medium_Detail 275.5
Results Archive
Dan Kegel maintains results at http://kegel.com/wine/yagmarkdata along with various reports.
In particular, http://kegel.com/wine/yagmarkdata/vista-vs-wine-1.1.44-19.txt shows how Vista and Wine-1.1.44-19 compare in performance on one system.
Running on Windows
First, disable UAC and reboot (else Autohotkey won't be able to install or control the apps, sorry).
Then install cygwin, and use its setup app to install cabextract, perl, subversion, unzip, wget, and xmllib2.
Finally, follow the directions to check out and run the script.
You'll probably have to babysit the first run. For instance, 3dmark2000 and 3dmark2001 will fail until you mark their executable as needing Windows XP compatibility mode.
Issues
On really big screens, at least under wine, 3dmark2000 seems to lose its mind and put up a warning dialog behind the main window (thus hanging the test). I had to set the screen size down to something closer to 1024x768 for it to calm down and run the benchmark.
3dmark2000 seems to crash my X server reliably these days, so I don't run it at the moment. (Probably I should try the latest beta nvidia drivers.)
Once in a while, an install may hang at the end, waiting for the Finish button to be clicked. To recover, click the button by hand, then if needed, also kill autohotkey.
Bug Tracking
To report a bug in yagmark, please use http://code.google.com/p/winezeug/issues/list
Getting reproducible results
Here's a checklist of things to do before running yagmark:
- Switch from Gnome or KDE to something lighter weight, e.g. Xubuntu's Xfce
- Close any web browsers or other large apps
- Close Workrave or any other little app that sits in the tray
- Disable the screensaver
- Run nvidia-settings and set powermizer to "Maximum Performance"
- Run the governor applet and select "Performance"
- In Software Sources, turn off automatic updates
- Run 'top' and make sure the load average is under 0.05
Work around 3dmark06 bug http://bugs.winehq.org/show_bug.cgi?id=9210 by applying patch http://bugs2.winehq.org/attachment.cgi?id=27642, else yagmark will hang at a 3dmark06 dialog