Weblog
Gendarme 2.2 Performance
Like the previous 2.0 release I wanted to invest some time to track where time and memory goes - and make sure Gendarme stays fast. My (personal) performance goal for Gendarme 2.2 was that time/memory increases (percentages) should be lower than the added new features (i.e. rules).
Now be warned that this benchmark (like most of them ;-) is not fair. Why ? because I want to compare performance between Gendarme 2.0 and 2.2. This is not the same as comparing Gendarme 2.0 on Mono 2.0 versus Gendarme 2.2 on Mono 2.2 - even if this is what most people will end up doing. E.g. The newer JIT in Mono 2.2 is a lot better, thanks to some pretty cool hackers - but I'll let them (or others) show you how much better it is ;-). So in the following tests both versions of Gendarme are being executed on the Mono 2.2 JIT. Also the older 2.0 was modified to report the initialization time (between a tenth and half of a second) to be similar to what 2.2 does. This is important because the new engines, present in 2.2, spend their time in initialization (from a few to several seconds). So timing this step became critical in 2.2. In other words you should see even better performance than the current numbers shows.
| Gendarme | version | 2.0 | 2.2 preview 1 | change (%) | ||
| Rules | # | 151 | 183 | 121.19% | [1] | |
| Applications | # assemblies | time (sec) | time (sec) | change (%) | ||
| Banshee 1.2 | 14 | 6.55 | 7.58 | 115.70% | [2] | |
| Beagle | 20 | 10.52 | 11.32 | 107.59% | ||
| F-Spot | 22 | 676.01 | 586.33 | 86.73% | [3] | |
| Giver | 1 | 1.44 | 1.41 | 98.15% | ||
| Gnome-Do 0.6.1 | 3 | 1.82 | 1.87 | 102.45% | ||
| Monodoc | 12 | 2.15 | 2.44 | 113.73% | ||
| Monsoon | 4 | 5.93 | 6.75 | 113.83% | ||
| Tasque | 2 | 2.12 | 2.33 | 109.54% | ||
| Tomboy | 4 | 3.65 | 4.05 | 110.84% | ||
| Mono 2.2 (2.0 profile) | ||||||
| default [4] | 85 | 70.74 | 154.15 | 217.92% | [5] | |
| all rules | 85 | 697.14 | 819.39 | 117.54% | [6] | |
| Total | 1,478.08 | 1,597.62 | 108.09% | |||
Notes:
- The "real" percentage should be a bit higher since we merged two rules and other got more features/checks.
- Don't worry I updated to Banshee 1.4, but that was after I got those numbers :-)
- Most of the time still comes from Tao.OpenGl.dll, the net gain too. Nestor is working on the main culprit so it will be even faster in 2.4
- The "default" rule set includes every rules except the Smells
- That was quite unexpected. Even more since 2.1 was more performent than 2.0 up to 175 rules.
- Fixing #5 will help for the final 2.2 but even more important will be the fix for #3 (but that 2.4 stuff ;-)
The final index (108%) is under the rules increase (121%) but the time required to fully analyze Mono 2.2 (2.0 profile) assemblies shows a (or some) bottleneck(s) that needs further work. Analysis (and fixes) have already started this weekend. Expect an updated table for the next preview (or RC).
11/16/2008 21:10:51 | Comments
The views expressed on this website/weblog are mine alone and do not necessarily reflect the views of my employer.
