Jump to navigation

Poupou's Corner of the Web

Looking for perfect security? Try a wireless brick.
Otherwise you may find some unperfect stuff here...

Weblog

Gendarme 2.2 Performance

Like the previous 2.0 release I wanted to invest some time to track where time and memory goes - and make sure Gendarme stays fast. My (personal) performance goal for Gendarme 2.2 was that time/memory increases (percentages) should be lower than the added new features (i.e. rules).

Now be warned that this benchmark (like most of them ;-) is not fair. Why ? because I want to compare performance between Gendarme 2.0 and 2.2. This is not the same as comparing Gendarme 2.0 on Mono 2.0 versus Gendarme 2.2 on Mono 2.2 - even if this is what most people will end up doing. E.g. The newer JIT in Mono 2.2 is a lot better, thanks to some pretty cool hackers - but I'll let them (or others) show you how much better it is ;-). So in the following tests both versions of Gendarme are being executed on the Mono 2.2 JIT. Also the older 2.0 was modified to report the initialization time (between a tenth and half of a second) to be similar to what 2.2 does. This is important because the new engines, present in 2.2, spend their time in initialization (from a few to several seconds). So timing this step became critical in 2.2. In other words you should see even better performance than the current numbers shows.

Gendarmeversion2.02.2 preview 1change (%)
Rules#151183121.19%[1]
Applications# assembliestime (sec)time (sec)change (%)
Banshee 1.2 14 6.55 7.58 115.70%[2]
Beagle 20 10.52 11.32 107.59%
F-Spot 22 676.01 586.33 86.73%[3]
Giver 1 1.44 1.41 98.15%
Gnome-Do 0.6.1 3 1.82 1.87 102.45%
Monodoc 12 2.15 2.44 113.73%
Monsoon 4 5.93 6.75 113.83%
Tasque 2 2.12 2.33 109.54%
Tomboy 4 3.65 4.05 110.84%
Mono 2.2 (2.0 profile)
default [4] 85 70.74 154.15 217.92%[5]
all rules 85 697.14 819.39 117.54%[6]
Total1,478.081,597.62108.09%

Notes:

  1. The "real" percentage should be a bit higher since we merged two rules and other got more features/checks.
  2. Don't worry I updated to Banshee 1.4, but that was after I got those numbers :-)
  3. Most of the time still comes from Tao.OpenGl.dll, the net gain too. Nestor is working on the main culprit so it will be even faster in 2.4
  4. The "default" rule set includes every rules except the Smells
  5. That was quite unexpected. Even more since 2.1 was more performent than 2.0 up to 175 rules.
  6. Fixing #5 will help for the final 2.2 but even more important will be the fix for #3 (but that 2.4 stuff ;-)

The final index (108%) is under the rules increase (121%) but the time required to fully analyze Mono 2.2 (2.0 profile) assemblies shows a (or some) bottleneck(s) that needs further work. Analysis (and fixes) have already started this weekend. Expect an updated table for the next preview (or RC).


11/16/2008 21:10:51 | Comments

The views expressed on this website/weblog are mine alone and do not necessarily reflect the views of my employer.