Yesterday I was implementing some NUnit tests for a C# database access object (or DAO). If you have ever implemented such tests before you’ll probably agree with me that it’s important for your test data to be in a known and consistent state before every test run.
For example, let’s say you are testing a method that inserts a record into a Person table, which uses the person’s full name as the primary key (probably not a great choice for a primary key but this is a contrived example so don’t flame me). In your test, you might insert a record for “John Smith” then do a select for “John Smith” to assert that the record was actually inserted.
The first time you run the test, it might work but what happens on the second test run? If you did not revert the data back to the state it was in before the first test run, you’ll get a primary key violation because one “John Smith” already exists and you are trying to insert a second “John Smith”. In other words, your test will fail, even though there is really nothing wrong with the method you are testing.
Another vital property of data access tests is that there should be no dependencies between your tests. That is, one test should not rely on side effects caused by another test. Furthermore, test results should not depend upon the order in which the tests are run.
Continuing from the previous example, let’s say you have a method that loads (i.e. selects) people by last name. In the corresponding test method, you might load all people with the last name “Smith” and then do an assertion such as making sure that the returned row count is say 5 (assuming your test database has 5 “Smiths” in it). The problem is, if you run the above insert test before the select test and you don’t rollback the insert, you’ll have have 6 rows returned and your test will fail. Naively, you could make your select test assert for 6 rows but what if your automated testing tool decides to run the select test before the insert test? You’ll have failure again.
The trick to solving these problems of data consistency and test dependency is to rollback the effects of each test immediately after each test is run. As far as I know, neither JUnit nor NUnit have out-of-the-box support for automatic rollback (at least at the time of writing this article). However, there are a few third party extensions for each tool that can do this rollback.
For example, in the JUnit and Java world, I have successfully used Spring Framework 2′s AbstractTransactionalSpringContextTests to automatically rollback the effect of each test. In the NUnit and C# world, you could use XtUnit, which I discovered yesterday. XtUnit is very easy to use. Essentially, this is all you are need to do:
- Download the compiled XtUnit assembly.
- Add a reference to the XtUnit assembly in your test project.
- Add a
usingstatement to your test class. For example:
- Derive your test class from ExtensibleFixture. For example:
public class MyDaoTest : ExtensibleFixture
- Add the DataRollback attribute to each test. For example:
[Test, DataRollBack] public void TestSavePerson()
The DataRollBack attribute does all of the heavy lifting here. Essentially, it creates an aspect that intercepts calls to your test method, wrapping it in a new transaction, which is then rolled back when the test method completes.
Initially, I had some problems getting XtUnit to work. When I loaded my test project in the NUnit GUI test runner, I got this error:
Assembly Not Loaded. System.IO.FileNotFoundException: Could not load file or assembly ‘mytests’ or one of its dependencies. The system cannot find the file specified. For further information, use the Exception Details menu item.
As the error message suggested, I clicked the Exception Details menu item in the NUnit GUI, which displayed the following unhelpful message:
Server stack trace:
at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
at System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.Load(String assemblyString)
at System.UnitySerializationHoldeaspectr.GetRealObject(StreamingContext context)
at System.Runtime.Serialization.ObjectManager.ResolveObjectReference(ObjectHolder holder)
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject(MemoryStream stm)
at System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeMessageParts(MemoryStream stm)
at System.Runtime.Remoting.Channels.CrossAppDomainSink.SyncProcessMessage(IMessage reqMsg)
Exception rethrown at :
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at NUnit.Core.TestRunner.Load(TestPackage package)
at NUnit.Util.TestDomain.Load(TestPackage package)
at NUnit.Util.TestLoader.LoadTest(String testName)
What gives Reggie? According to the NUnit wiki, XtUnit works with NUnit 2.4.x but may need changes for 2.5. I was using NUnit 2.5 so maybe that was the problem?
To fix the error, I simply loaded the XtUnit source code project into Visual Studio 2008 and recompiled the assembly
TeamAgile.UnitTestExtensions.dll. After recompiling the assembly I was able to load my test project in the NUnit GUI. I think this guy might have had a similar problem. See this post for more information and an updated binary that might work for you.
Next, I tried to run a test with the DataRollBack attribute applied and got this error, after a timeout of several seconds:
System.Transactions.TransactionManagerCommunicationException : Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for network access in the security configuration for MSDTC using the Component Services Administrative tool.
—-> System.Runtime.InteropServices.COMException : The transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D024)
Enable DTC? Um, okay, how do I do that? After a bit of googling, I found the answer. See Microsoft’s article Enable Network DTC Access.
Although the aforementioned article doesn’t mention it, you’ll need to enable DTC on both the client side and the server side. If you enable DTC only on the client side, you will probably get this error when you run your test:
System.Transactions.TransactionException : The partner transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D025)
—-> System.Runtime.InteropServices.COMException : The partner transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D025)
Note that after enabling DTC on both the client and server, you’ll need to restart the NUnit GUI before running your tests.
When it’s all of this is working, you’ll see something like this in NUnit’s Text Output tab:
ENTERING transaction context on method: TestSaveFoo
ENTRED transaction context on method: TestSaveFoo
LEAVING transaction context on method: TestSaveFoo
I’m sure there are other ways of doing this besides using XtUnit. If you know of any alternative tool or technique, feel free to share your experiences in the comments.
I hope this helps someone else!
For the record, I was using the following tools in my setup:
- Visual Studio 2008 version 9.0.21022.8
- .NET Framework 3.5 SP1
- NUnit 220.127.116.1122
- XtUnit version 18.104.22.168, recompiled by me in Visual Studio 2008
- SQL Server 2005
- Windows Server 2003 R2 SP2 on the server side.
- Windows XP SP3 on the client side.