Poupou's blog

Gendarme 0.2

Wed, 16 Jul 2008 23:15:34 GMT

Mono has branched for 2.0! Quite a good news for everyone. It's also a great moment for all gendarmes because it means we branched too!

The Gendarme 0.2 release includes the results of the last six months of hacking, including some major events like the Novell Hack Week (Winter 2008 edition), which changed much of the Gendarme framework, the first "Gendarme Rule Day" and the first part of GSoC 2008 work by Néstor Salceda.

General highlights:

Performance enhancements, memory usage reduction (mostly inside Cecil but the gains are very visible in Gendarme)
MDB/PDB support: source files and line numbers are now provided with defects
New Windows installer: You can now easily enjoy Gendarme, and it's wizard, from Windows computers
.desktop file for the Wizard runner
Console runner now uses color when displaying on the console
Albeit a bit invisible to end users, we started using the new unit test helpers made during GHOP
Bug fixes: works continue, like previous releases, to remove false positives from existing rules
and a lot of new rules including...

Gendarme.Rules.BadPractice

AvoidCallingProblematicMethodsRule
AvoidVisibleConstantFieldRule
DoNotForgetNotImplementedMethodsRule
DisableDebuggingCodeRule
ObsoleteMessagesShouldNotBeEmptyRule
ReplaceIncompleteOddnessCheckRule

Gendarme.Rules.Concurrency

DoNotLockOnThisOrTypesRule
DoNotLockOnWeakIdentityObjectsRule
DoNotUseLockedRegionOutsideMethodRule

Gendarme.Rules.Correctness

DoNotRoundIntegersRule
ReviewDoubleAssignmentRule
ReviewSelfAssignmentRule
ReviewUselessControlFlow
ReviewUseOfInt64BitsToDoubleRule
ReviewUseOfModuloOneOnIntegersRule

Gendarme.Rules.Design

AvoidMultidimensionalIndexerRule
AvoidRefAndOutParametersRule
AvoidVisibleNestedTypesRule
ConsiderConvertingFieldToNullableRule
ConsiderUsingStaticTypeRule
ImplementGenericCollectionInterfacesRule
ImplementIComparableCorreclyRule
PreferIntegerOrStringForIndexersRule

Gendarme.Rules.Exception

AvoidArgumentExceptionDefaultConstructorRule
AvoidThrowingBasicExceptionsRule
DoNotThrowReservedExceptionRule
ExceptionShouldBeVisibleRule
MissingExceptionConstructorsRule

Gendarme.Rules.Interoperability

DoNotCastIntPtrToInt32Rule

Gendarme.Rules.Maintainability (new)

AvoidComplexMethodsRule
AvoidDeepInheritanceTreeRule
AvoidLackOfCohesionOfMethodRule
ConsiderUsingStopwatchRule
PreferStringIsNullOrEmptyRule

Gendarme.Rules.Naming

DoNotPrefixEventsWithAfterOrBeforeRule

Gendarme.Rules.Performance

AvoidLargeNumberOfLocalVariablesRule
AvoidLargeStructureRule
AvoidRepetitiveCastsRule
AvoidTypeGetTypeWhenPossibleRule
AvoidUnusedPrivateFieldsRule
AvoidUnneededUnboxingRule
AvoidUnsealedUninheritedInternalClassesRule
ImplementEqualsTypeRule
OverrideValueTypeDefaultsRule
RemoveUnusedLocalVariablesRule
UseIsOperatorRule
UseTypeEmptyTypesRule

Gendarme.Rules.Portability

DoNotHardcodePathsRule

Gendarme.Rules.Serialization (new)

CallBaseMethodsOnISerializableTypesRule
DeserializeOptionalFieldRule
ImplementISerializableCorrectlyRule
MarkAllNonSerializableFieldsRule
MissingSerializableAttributeOnISerializableTypeRule
MissingSerializationConstructorRule
UseCorrectSignatureForSerializationMethodsRule

Contributors for this release are:
Andres G. Aragoneses, Cedric Vivier, Daniel Abramov, JB Evain, Nestor Salceda, Scott Peterson, Sebastien Pouliot, Seo Sanghyeon.

Availability: Preview packages should be available soon (Gendarme resides inside mono-tools). I'll update the Win32 installer asap (probably tomorrow night), check the Files section of the group to get it. Now is the time for testing and reporting :-) Thanks to everyone!

Gendarme Speed Up

Thu, 17 Jul 2008 18:55:11 GMT

In the last six months a lot of attention was given to Gendarme's performance. First we updated it's framework to make it easier to skip rules when they are not applicable, e.g. checking for 2.0 features inside a 1.x assembly. Then I spent some time reducing memory allocations in Cecil. Since Gendarme is a big consumer (it reads everything) even small gains show up quickly.

This was an important step to move forward because the some of the new features, like debugging symbols (mdb and pdb on windows) and automatic assembly resolving were sure to require a lot more time (and memory) to process rules (not to mention that the number of rules is itself growing fast). Also to keep its value Gendarme, or at least its default rule set, needs to execute quickly enough that there's no good reason not to execute it at each build. And finally it's also because we're not much more patient than our users - we want results now :-)

So last night I took the time to measure how the new Gendarme 0.2, to be released with Mono 2.0, compares the the old Gendarme 0.0.5.1 released with Mono 1.9. The next table shows the time (in seconds) required to analyze the 72 assemblies that Mono ships for it's 2.0 profile.

	Gendarme 0.0.5.1		Gendarme 0.2.0.0		Change		Mean time per rule delta
	# rules	time	# rules	time	rules factor	time factor	Mean time per rule delta
BadPractice	7	9.39688	13	21.933973	186%	233.42%	0.34
Concurrency	3	9.703159	6	14.05999	200%	144.90%	-0.89
Correctness	7	11.732152	13	15.228963	186%	129.81%	-0.50
Design	25	4.235242	33	4.29234	132%	101.35%	-0.04
Exceptions	2	9.467328	7	16.499359	350%	174.28%	-2.38
Interoperability	4	8.841298	5	14.582129	125%	164.93%	0.71
Maintainability	0	0	6	18.499207	N/A	N/A	N/A
Naming	11	8.352653	12	2.133514	109%	25.54%	-0.58
Performance	13	157.135513	25	39.716426	192%	25.28%	-10.50
Portability	4	11.556642	5	42.844831	125%	370.74%	5.68
Security	10	6.434193	10	8.709008	100%	135.36%	0.23
Serialization	0	0	7	1.539076	N/A	N/A	N/A
Smells	6	652.999564	6	685.303299	100%	104.95%	5.38
UI	3	3.163413	3	0.106041	100%	3.35%	-1.02

TOTAL	95	893.018037	151	885.448156	159%	99.15%	-3.54

default	89	172.60422	145	99.322118	163%	57.54%	-1.25

The number of rules and execution times are given for each rule assembly/category. We can see the impact of the new features when looking at Security, it has the same number of rules but it now requires 35% more time to execute them. We can also see how skipping rules can help in UI where it nows takes only 3% of the previous time to execute the rules (since UI rules don't applies to the class librairies).

The good news is that, globally, executing every rules takes just a bit less time than before. Considering that we added 56 rules in this release this was quite a nice surprise (even if a bit planned I did not anticipate a status quo). But the results are even better if we consider Gendarme's default rule set (i.e. everything but the Smell rules) as it requires less than 60% of the time than the previous version did, even with the extra 56 rules.

Results are still preliminary since Mono 2.0 won't ship until September. Some existing rules don't yet take advantages of the new API and I intend to update them (or at least a few of them) before then and provide an updated table. And who knows maybe I'll even dive into Cecil a littl'bit more ;-)

There is no end in performance (so expect more blog entries on the subject ;-). We need to stay vigilant because even small things, like memory allocations, can make big differences in rules executed thousands of times. We already have ideas to speed up a lot of rules using a new feature. This should happen in the next Novell Hack Week and the results will be visible in Mono 2.1. With a bit of luck we'll have more rules and results in even less time (or at least worked upon that goal).

Big Bump

Thu, 14 Aug 2008 19:33:21 GMT

Earlier this week the Mono 2.0 branch was tagged for preview #2. Included with it, inside mono-tools, is the updated Gendarme.

Most of the changes between now and the first preview are not very visible. Like promised a lots of rules were tweaked to reduce false positives. Some of them were also optimized to be (quite a bit) faster. A big thanks to everyone for the feedback (and the patches too ;-). Hopefully I'll have time to get more fixes into the branch before the release candidate in early September.

So what's the most visible change ? Well from now on Gendarme version will match Mono version numbers. This should avoid any confusion (e.g. bug reports) concerning the versions and it also answers the feedback I received that 0.2 was not very representative of Gendarme's status.

This second preview is also a good time to update the performance table of my previous blog entry. Again I compared the time needed for Gendarme 0.0.5.1, shipped with Mono 1.9.1 (confusing heh?), versus the 2.0p2 release (less confusing :-) by executing each rule category (and default one) on all (72) Mono 2.0 assemblies.

	Gendarme 0.0.5.1		Gendarme 2.0 p2		Change		Mean time per rule delta
	# rules	time	# rules	time	rules factor	time factor	Mean time per rule delta
BadPractice	7	9.839234	13	20.940601	186%	212.83%	0.21
Concurrency	3	10.03967	6	14.167014	200%	141.11%	-0.99
Correctness	7	11.229753	13	15.350205	186%	136.69%	-0.42
Design	25	4.437044	33	3.738977	132%	84.27%	-0.06
Exceptions	2	9.743716	7	16.194831	350%	166.21%	-2.56
Interoperability	4	9.044328	5	14.150847	125%	156.46%	0.57
Maintainability	0	0	6	17.895726	N/A	N/A	N/A
Naming	11	8.482943	12	2.145394	109%	25.29%	-0.59
Performance	13	155.027838	25	33.18588	192%	21.41%	-10.60
Portability	4	11.598787	5	21.198526	125%	182.77%	1.34
Security	10	6.734685	10	8.593943	100%	127.61%	0.19
Serialization	0	0	7	1.349845	N/A	N/A	N/A
Smells	6	652.531303	6	686.105971	100%	105.15%	5.60
UI	3	3.272789	3	0.114405	100%	3.50%	-1.05

TOTAL	95	891.98209	151	855.132165	159%	95.87%	-3.73

default	89	174.698965	145	67.466905	163%	38.62%	-1.50

So the new total percentage of 95.87% is a bit better than the previous one (99.15%). Not surprising since the smells are taking a huge amount of time, compared to all other rules, and have not been much optimized since the first preview (but I got ideas for them too ;-).

However there's a bigger difference for the default rule set (what the console runner execute unless instructed otherwise). It nows takes 38.62% of the time (versus 57.54% for the first preview). In both cases the times are compared to the version 0.0.5.1 and includes the 56 additional rules.

The win32 installer for Gendarme 2.0p2 will be available shortly on the google group file section. Linux packages (i.e. mono-tools) should be available early next week.