Sunday, February 25, 2007

Performance, Overhead, and csUnit

Over the last several months we have been working on improving the performance of csUnit. Not that it was really bad. However, we thought that it is about time to look at whether we can improve it. Generally we improved two areas, one of which is of interest to users, while the second area is more of interest for the csUnit development team. Area one is basically the overhead that csUnit adds to the code that you execute. As a user you would like this to be as low as possible as you would like your resources (computer, but also your time) is mostly spent on what you really want to achieve. For improving the performance we are using JetBrains' dotTrace. It's an easy to use tool, and provided us with just the information we needed to reduce the overhead. For a recipe with about 4 assemblies containing tests you can expect an overhead of about 2 to 3 seconds on a decent notebook. Faster machines lead to better results. Currently we have a few options left, which we consider for future improvements, but right now we believe that we played the major cards. The second area was improving csUnit's own test suite. Again with the help of the profile, but also with csUnit's own "Statistics" tab, we identified the most expensive tests, and worked through them one at a time. We increased the use of mocks, but we also improved the testability of csUnit by introducing more internal interfaces. Overall we are now able to run the test suite with a speed of approx. 6,000 tests per minute on a decent notebook. We are currently shooting for a new version in March. We have a short list of known issues we want to fix. Stay tuned!