Creating Fake Data For Unit Tests With RimuTec.Faker

Setting up data for tests can be tedious. For example sometimes you may end up using first names like “First 1”, “First 2”, etc. Data like this may result in duplicates, so you then go to the next step using first names like “First “+ Guid.NewGuid() which may be a problem, too. Using such a name in a user interface may no longer fit the available space, where real-life names like “John” or “Anna” would fit easily.

Another aspect can be that you are required to have a most recent version of your system or product for demo purposes, let’s say a pre-sales situation or an internal product review. Trying having a conversation about a screen that says “First123456 Last645861”. Wouldn’t it be nice if it would display names such as “John Doe” or “Anna Karenina”? It most likely would also create a much better impression with prospects, customers or other stakeholders. Your software looks more “complete”.

Ruby developers have a gem named Faker which is available on github. Faker is based on a Perl library and has a large library of fake data generators.

There have been a few efforts to port the Ruby gem to .NET. Most of these have a limited set of generators and haven’t been maintained for years.

Therefore we decided to have another look. Where other projects use regular resource files, we decided to use the yaml files from Ruby’s Faker gem verbatim. By taking this approach we would automatically have some guardrails that would allow leveraging as many of the great ideas from the Faker gem as possible.

The result is RimuTec.Faker which offers the following classes of generators:

• Address
• Company
• Educator
• Finance
• IDNumber
• Internet
• Job
• Lorem
• Name
• PhoneNumber
• RandomNumber

Obviously this is not the complete list yet. We release frequently and are happy to receive suggestions for priorities in terms of what should be added next. Just log an issue on github.

Using RimuTec.Faker the example from the beginning be written as follows:

You can get both, the source code as well as the NuGet package under the MIT license. This means you can use it for free and also use it in your own projects as you see fit, including commercial and closed-source projects.

Happy coding!

csUnit Interim Release 2.8.4723

We have made available a new release of csUnit. This is an interim release with the version number 2.8.4723 and can be found on GitHub.

This release requires .NET 4.5. It doesn’t have an installer and no Visual Studio plugin. The download is a zip file that you need to extract into a folder then run csUnitRunner from there. If you encounter an issue please log the issue here with detailed info about your environment, the version of csUnit you are using and steps to reproduce.

We intend to create a WiX based installer next.

For more info about csUnit check out csUnit’s homepage.

Behavior of DirectoryInfo.Delete and Directory.Exists: Directories reappear!?

Please note that the following may be a corner case or an isolated case. If not then something is not quite right with running NUnit based test suites from within Visual Studio 2010 using ReSharper 6. Unfortunately I don’t have proof either way but I still wanted to share my observation just in case you have encountered a similar issue.

A couple of weeks ago I wrote some test code that cleaned up folders after tests were executed. The code looked as follows:

public static void CleanUpFolders(
                    List<DirectoryInfo> directoryInfos) {
   foreach (var directoryInfo in directoryInfos) {
      foreach (var file in directoryInfo.GetFiles("*", 
                            SearchOption.AllDirectories)) {
         file.IsReadOnly = false;
      }
      directoryInfo.Delete(true);
   }

   foreach (var directoryInfo in directoryInfos) {
      Assert.IsFalse(Directory.Exists(directoryInfo.FullName));
   }

   directoryInfos.Clear();
}

For each directoryInfo in the collection the code iterates through all files and sets the read-only attribute to false for each of them. Then the directory is deleted with all its contents.

Then I executed the test suite from within Visual Studio 2010 using ReSharper 6. It turned out that in some cases the assertion in the above code would fail. Despite the directories having been deleted the call to Directory.Exists would return true! So for some reasons the directory was deleted but then it wasn’t. When I reran the tests sometimes this assertion would fail and sometimes it wouldn’t. There were days where it was fine and there were days when the assertion would fail in 90% of the cases.

In addition to this the assertions would only fail on two particular directories but not on any other. I couldn’t identify the commonality between the directories on which the assertion would fail and the directories that were successfully deleted.

Initially I thought that maybe a different process or a different thread would have a handle to the directory. In that case my test would just ask the operation system (Windows 7, 64 bit) to mark the directory as deleted and as soon as the last handle was closed it would be removed. However, the tests would generate names and then create directories with that name. No other thread or process knew about the names. I didn’t have any thread in my system under test that would scan the parent directory thus ‘learning’ about the generated directory names.

To diagnose this issue I tried various things including restarting of Visual Studio and rebooting the workstation. I also tried some of Sysinternals’ tools.

The biggest surprise was an observation that I made when running the tests under the debugger. I set a breakpoint just after the loop that deleted all the directories. At that point Windows Explorer would report those directories as gone. Also when continuing the execution the assertions would not fail. However, once the entire suites was complete, two directories would reappear in Windows Explorer! So despite Directory.Exists stating the Directory to be non-existent it reappeared! This is repeatable.

To take elements out of the equation I got the latest revision of csUnit’s source code and upgraded it to VS2010 and .NET 4.0 (these changes are available at Sourceforge). Then I executed the same test suite without any modification using csUnit. In this case the directories were properly deleted and did not reappear.

Where does that leave me? I don’t know. All I can say is this: My test suite creates a number of directories with generated names. When I execute this suite from within VS2010 and using ReSharper 6, two folders reappear despite DirectoryInfo.Delete() being executed successfully and Directory.Exists() confirming that. However when I execute the same test suite from csUnitRunner, the code behaves as expected and the folders remain deleted. Despite searching for a long time I have not been able to find the reason for this difference in behavior.

The only reasonable conclusion at this stage seem to be that as developers we always need to be suspicious about the tools we use. While they may work in almost all the cases, sometimes they may be the cause for something that doesn’t work.

ReSharper Not Executing Your Tests?

Today I ran into a small issue as ReSharper was not willing to execute my tests. Generally it does but today it was different. Let me explain.

I have two assemblies, one that contains the unit tests and one that contains the code under test. In ReSharper’s session window the unit tests were properly listed. I was even able to “execute” them. However it would always show that them as not executed with a gray bullet in front of them. This was the behavior for both “Run Test” and “Debug Test”. The Output window was empty and didn’t show anything that would indicate what was wrong.

I also check my unit tests but all attributes were properly in place and both the class and the methods were public and the methods had the correct signature. So what was going on?

To diagnose the issue I launched the unit test tool as a stand-alone application. When I tried to load the assembly with the unit tests it was immediately clear what was wrong when I saw the word BadImageFormatException displayed in a message box.

It turned out that the assembly with the unit tests was set to build for “Any CPU” while the assembly under test was build for “x86”. Since I am using a 64 bit machine the two assemblies were compiled as per those targets. The unit tests as 64 bit while the assembly under test as 32 bit. No wonder it didn’t work.

This is easy to fix, in my case by setting the target for the assembly with the unit tests to “x86” as well. It would, however, be nice if ReSharper had reported the exception in some form as it would have saved time. Well, maybe in the next version? Maybe this post helps you to save some time.

(Note: I’m using ReSharper version 6.0.2202.688 with Visual Studio 2010 SP1 on a 64 bit Windows 7 Enterprise Edition)

Selenium RC and ASP.NET MVC 2: Controller Invoked Twice

Admittedly MVC (as of writing I use ASP.NET MVC 2) has been designed from the ground up for automated testability (tutorial video about adding unit testing to an MVC application). For example you can test a controller without even launching the ASP.NET development web server. After all a controller is just another class in a .NET assembly.

However, at some point you may want to ensure that all the bits and pieces work together to provide the planned user experience. That is where acceptance tests enter the stage. I use Selenium for this, and a few days ago I hit an issue that turned out to be caused by Selenium server version 1.0.3. Here are the details.

The symptom that I observed was that a controller action was hit twice for a single Selenium.Open(…) command. First I thought that my test was wrong, so I stepped through it line by line. But no, there was only one open command for the URL in question. Next I checked my implementation, whether maybe accidentally I had created some code that implicitly would call or redirect to the same action. Again, this wasn’t the case as each time when I hit the break point on the action controller there was nothing in the call stack.

Then I used Fiddler (a web debugging proxy) for a while and yes, there were indeed a HEAD request and a GET request triggered by the Selenium.Open(…) command. And even more interesting, when I ran my complete test suite I found several cases where the GET request was preceded by a HEAD request for the same URL.

The concerning bit, however, was that I couldn’t find a way how to reproduce this with a browser that I operated manually. Only the automated acceptance tests through Selenium RC created this behavior.

For a moment I considered trying to use caching on the server side to avoid executing the action more than once. But then I decided to drill down to get more details. In global.asax.cs I added the following code (Of course you can use loggers other than log4net):

protected void Application_BeginRequest() {
   Log.InfoFormat("Request type is {0}.", Request.RequestType);
   Log.InfoFormat("Request URL is {0}.", Request.Url);
}

private static readonly log4net.ILog Log =
   log4net.LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);

As a result I was able to track all requests. Of course you wouldn’t want to do this for production purposes. In this case I just wanted to have more information about what was going on. It turned out that Fiddler was right as I found several HEAD requests followed by a GET request.

After some research I came across a discussion about Selenium RC head requests and it turned out that this was a known issue in Selenium server version 1.0.3. As of writing this was fixed in trunk and I thought for a moment about building from trunk but then decided on a different path. And that solution worked as well: Instead of using version 1.0.3 I am now using Selenium Server version 2.0a5 plus the .NET driver from the 1.0.3 package.

So here is what you need to download:

1. Selenium Remote Control 1.0.3 which includes the .NET driver. Don’t use the server from this download.
2. Selenium Server Standalone 2.0a5. Use this jar file as your server. The command line at the Windows command prompt changes from “java –jar selenium-server.jar” to “java -jar selenium-server-standalone-2.0a5.jar”.

Then start the 2.0a5 server and run your tests. The HEAD/GET issue should be gone. In my case it was and I’m now back to extending my test suite finally making progress again.

My configuration: Visual Studio 2010, .NET 4.0, ASP.NET MVC 2, Vista Ultimate 32, various browsers (IE, Firefox, Chrome, Opera, Safari). The issue I describe here may be different than the one you observe. Consequentially it is possible that this suggested solution doesn’t work for you.

How To Test Sending an Email in .NET?

Sending an email can be tested in many different ways. One option could be setting up an account with an online email provider (Yahoo, Hotmail, Google etc.) and then use that account for sending email.

To save time, however, it might be valuable to look at "SMTP Server for Developers” (SSFD on Codeplex). This simple tool – developed by Antix – gives you a local SMTP server which looks like a standard server from your application’s perspective but on the back side simply writes all emails to a folder. The emails are stored in a text file with the extension EML and with a predefined format (headers + empty line + body).

By using SSFD the round trip will be faster and for retrieving the email you simply read a file.

To configure email in .NET (I am using version 4.0) add the following to your web.config/app.config file (the highlighted line is needed so SmtpClient.Dispose() doesn’t throw an exception, see comments at the end of this post):

<configuration>
  <system.net>
    <mailSettings>
      <!-- Setting for release (need to update account settings): -->
      <!--<smtp deliveryMethod="Network">
            <network host="..." port="25" userName="..." password="..." />
         </smtp>-->
      <!--Setting for development (uses ssfd.codeplex.com):-->
      <smtp deliveryMethod="SpecifiedPickupDirectory">
        <network host="localhost" />
        <specifiedPickupDirectory pickupDirectoryLocation="C:\data\Temp" />
      </smtp>
    </mailSettings>
  </system.net>
</configuration>

Then in your code write the following:

using System;
using System.Net.Mail;
using System.Reflection;

namespace WebPortal.Controllers {
   internal class EmailHelper {
      public static void SendEmail(string from, string to, string subject, string body) {
         try {
            using(var smtpClient = new SmtpClient()) {
               smtpClient.Send(new MailMessage(from, to, subject, body));
            } 
            // SmtpClient.Dispose() may throw exception despite Microsoft's own guide.
            // See blog post for further details.
         }
         catch (Exception ex) {
            Log.Error(ex);
         }
      }

      private static readonly log4net.ILog Log =
         log4net.LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
   }
}

Of course this is only the simplest version. If you need to send to multiple recipients or want to use a specific encoding or HTML instead of text format then this code would needs a bit more meat. Furthermore in a production system you may want to add error handling that allows some feedback to the user or the system administrator.

One observation I made while working on this: Microsoft recommends that “a Dispose method should be callable multiple times without throwing an exception”. Unfortunately SmptClient.Dispose() throws an exception when no host has been specified, thus contradicting their own recommendation.

SSFD doesn’t require the hostname for operation but when you implement your client side code you may want to use “using(var smtpClient = new SmtpClient()) {…}” to ensure that all resources used by your code (e.g. server connections) are properly cleaned up. Without a hostname in the web.config file (or specified through some other means, e.g. programmatically) SmtpClient.Dispose() will throw an exception. Therefore even though SSFD doesn’t need it, add “<network host="localhost" />“ as shown in the above web.config example.

IE8 and Selenium RC: Tests won't execute

Just noticed that running tests through Selenium RC in Internet Explorer 8 (IE8) displays a message containing an info to disable pop-up blocker. IE reports a script error and also says that it cannot load "...htmlutils.js". The solution was simple: Run the Selenium RC server in admin mode. Launch the command line as administrator then start selenium RC using "java -jar selenium-server.jar". The scripts should then run. (Of course you would put all of this into a short script or batch file, which in turn is set to run "As Administrator". I observed this behavior on Windows Vista Ultimate 32 bit with IE8 and all of that updated to the latest patch level. The Selenium version was 1.0.1. The behavior might also be present in other combinations. Also the solution described above may or may not help on other tech stacks.

Implicit Typing Can Make TDD Harder

Starting with C# 3.0 the language supports the concept of implicit typing at the method level using 'var'. While on one hand this new keyword has its benefits such the possibility to change return types of methods without having to change the type of all local variables that the return value is assign to, there is also a draw back of this new language feature. If you are using strict TDD, you write your tests to drive your design. That means it is not uncommon that a new method on a class doesn't exist yet, though you can write your test including assertions on the return type. Example:

public class Result {
   public long Status { get; set; }
}

public class Foo {
}

Given this starting point you may want to write the following unit test to ensure that class Foo requires a method 'Result NewMethod()':

[TestMethod]
public void TestNewMethod() {
   var foo = new Foo();
   var result = foo.NewMethod();
   Assert.AreEqual(0, result.Status);
}

As you type you will notice that when you have typed the dot after 'result' you will not be offered the member list of Result. It's not possible to implicitly type the variable 'result' since the method 'NewMethod()' doesn't exist yet. As a result writing tests in a TDD approach is slowed down when using 'var' instead of explicit types. Here is another view you may take: Writing tests for 'NewMethod()' should include all specifications, including the type of the return value. If you agree with that view you may want to avoid using 'var' in your unit tests. This certainly doesn't apply to people who create their unit tests after the method has been added to the class. I personally wouldn't call this test-first development, let alone test-driven development (or test-driven design as some people argue). Bottom line it depends on where you are coming from. 'var' might not always be the best choice even if it is a new 'cool' feature in C# 3.0.

Executing MS Unit Tests in csUnit

I have made a little progress on the MS Unit Test support in csUnit. So far I managed to create support for:

• TestClass
• TestMethod
• ExpectedException

Admittedly, these are just the most basic ones but it is a start. Next, I'll have to do some refactoring to clean up the code. As of now, csUnit can execute tests implemented using the unit testing frameworks of either csUnit, NUnit, or MS Unit Test. (Note: This is in trunk as of now. No release is available yet.)

Another Tool for Silverlight Unit Testing

Just came across another unit testing tool for Silverlight. It's called SilverUnit. I haven't tried it out but I certainly will have a closer look. It will be interesting to see how this compares to Jeff Wilcox's approach and how it integrates with established unit testing tools. I'll keep you posted.

csUnit migrated to .NET 3.5 and VS 2008

Finally I have found some time again to do a few things on csUnit. Actually the main driver was that I tried out the unit testing features that come out of the box with Visual Studio 2008 and I found them a little bit too cumbersome for my taste. I'm sure there are scenarios, teams, and people who are looking exactly for what VS's unit testing provides, including the ability to look at old test runs. But overall it felt a little bit too heavy. One example. A test fails. The result's view lists all tests and you can click on the one that failed. But it doesn't bring you straight to the failed test. That was my expectation. Instead it brings you to a page with the result details of that test. And only there you find a link to the actual implementation of the test. Conceptually that's probably what MSFT wanted. For me it felt like being slowed down. So now I've moved csUnit to .NET 3.5 and migrated the solution and all projects within it to VS 2008. And I'm looking into making it possible for csUnit to run tests implemented using MSFT's unit testing framework. Let's see how that goes. One difficulty I already discovered: Counting assertions. I don't have a good solution for that yet but if you do, please let me know!

Unit Testing for Silverlight

Looking for instructions for Unit Testing for Silverlight 2? Jeff Wilcox posted an excellent tutorial in March 2008 here. Since then he's also updated his testing framework. In addition he has posted required changes to make the tutorial work with the final release of Silverlight 2. Although the latter post refers to RC0 of the testing framework the information still applies to the December 2008 binaries. The December 2008 release of the binaries of Jeff's unit testing framework can be downloaded from here. The project's homepage is here.