Unit Test - MSTest

Created:2019-06-26  Last modified:2019-07-03


  1. Introduction

    Unittest: testing a specific class and its function.
    1). create its dependencies.
    2). using its dependencies to initialize the target object.
    3). verify expections.
    Testing code does not have a main program, it is compiled into a library, and then invoked by Testing tools. e.g. MSTest.exe

    1. MSTest (Visual Studio Unit testing framework)

      Microsoft's test framework that supported by Visual Studio. (Inside visual studio "Test Explore", you can directly run test). MSTest provides attributes to mark test classes and test methods, it also provides a family of Assert functions to verify results.

    2. Moq

      Moq (pronounced "Mock-you" or just "Mock") is a .NET Framework to help create fake objects. 1). It can create objects based on interface, and designate the behavior when calling a method (with specific arguments). 2). It can also inspect the behavior of fake objects when running the target object's methods. e.g. (# of calls)

  2. Example

    1. Example code

                                  // target
                                  public class Target
                                  {
                                      private readonly IFormatter _formatter;
                                      public Target(IFormatter formatter){
                                          _formatter = formatter;
                                      }
                                      public string Print(params int arrNumber){
                                          var sb = new StringBuilder();
                                          foreach(var number in arrNumber){
                                              sb.Append(_formatter.Format(number));
                                          }
                                          return sb.ToString();
                                      }
                                  }
      
                                  // dependency
                                  public interface IFormatter
                                  {
                                      string Format(int i);
                                  }
                          

      Test Target.append method. To isolate the potential bugs in ISimpleServices's implementation, we need to create a Mock ISimpleServices.

    2. Structure

      A unittest should be created as a separate project under the same solution, and it should include the reference of the target project.

                              dotnet new sln; # create a solution
                              mkdir project_name; cd project_name;
                              dotnet new console; # or dotnet new classlib;
                              cd ..;
                              mkdir project_name.tests;
                              cd project_name.tests;
                              dotnet new mstest; # initialize a test project;
                              dotnet add reference ../project_name/project_name.csproj; # add reference.
                          

      The dotnet new mstest command includes MSTest library.

                                  <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.9.0" />
                                  <PackageReference Include="MSTest.TestAdapter" Version="1.3.2" />
                                  <PackageReference Include="MSTest.TestFramework" Version="1.3.2" />
                          

      Naming convention

      1). the test file and test class name is {{target-classname}}Tests
      2). the test method is {{target-method}}_With[out]{{Input-description}}_Should{{Output-expectation}}

                                  using Microsoft.VisualStudio.TestTools.UnitTesting;
      
                                  namespace Targets.Tests
                                  {
                                      [TestClass]
                                      public class TargetTests
                                      {
                                          // properties...
      
                                          [TestInitialize]
                                          public void Init(){
                                              // Init() is optional
                                          }
      
                                          [TestMethod]
                                          public void Append_WithIncThree_ShouldReturnThree()
                                          {
      
                                          }
                                      }
                                  }
                          

      These attributes help MSTest.exe to invoke methods, also help Visual Studio to present UI.

    3. Moq (without Autofac)

      Install: dotnet add package Moq --version 4.12.0

      Moq helps initialize dependencies based on interface (service), it can also help inject dependencies into the target object by using 'Autofac.Extras.Moq.AutoMock'

                              [TestMethod]
                              public void Append_WithIncThree_ShouldReturnThree()
                              {
                                  var formatterMock = new Mock<IFormatter>();
                                  // formatterMock is a wrapper of IFormatter, it has method to configure the dependence.
                                  formatterMock.Setup(x => x.Format(It.IsAny<int>())).Returns("10");
                                  // setup the method with any int input, return "10"
                                  var target = new Target(formatterMock.Object);
                                  var result = target.Print(1, 2, 3);
                                  // create the target with the mock dependence, and invoke method.
                                  Assert.AreEqual("101010", result);
                                  // MSTest provided Assert.
                                  formatterMock.Verify(x => x.Format(It.IsAny<int>()), Times.Exactly(3));
                                  // Moq inspect dependence.
                              }
                          
  3. MSTest & Moq APIs

    Moq APIs

    1. Configure Mock Objects' method

      1). the behavior of a method is deterministic, then its result is determined by its inputs
      2). the behavior of a method is non-deterministic, then its result is random.
      3). the behavior of a method is stateful, meaning also related to the number of calls made on it.

      A result can be a value or an exceptions. Moreover, in C#, result can also be async.

      ** Because it is mock object, usually just need hardcode. No if-else logic required"

                              MockObject.Setup takes an input of expression of  TResult Fun(T)
                                  returns a ISetup<T, TResult>
                              ISetup has methods:
                                  1). a family of Returns: IReturnsResult
                              ISetup has extension methods:
                                  1). a family of ReturnsAsync, ThrowsAsync, which can also have time-delay argument: IReturnsResult
      
                              Mock<T> {.Setup<T, TResult>(Expression<Func<T, TResult>> )}
                              |      |----------|                             |
                              |                 |                             |
                              Mock            IMock<T> {T Object}          |
                                                                            returns
                                                                              |
                                                                              \/
                                                                          ISetup<T, TResult>
                                                                              |
                                                                              |
                                                                          IReturns<T, TResult> { Returns(TResult ), ReturnsAsync(TResult )}
                                                                                                              |               |
                                                                                                              ----------------    
                                                                                                                      |
                                                                                                                  IReturnsResult<T>
      
                              MockObject.SetupSequential takes an input of expression of  TResult Fun(T)
                                  returns a ISetupSequentialResult
                              ISetupSequentialResult has extension methods:
                                  1). ReturnsAsync, 2). ThrowsAsync, both of them also return a ISetupSequentialResult, so it's chainable.
                              
      
      
                              
                          
                              # deterministic stateless.
                              mock.Setup(foo => foo.DoSomething(It.IsAny<string>())).Returns(true);
                              mock.Setup(foo => foo.DoSomething("hello")).Returns(true);
                              // other input condition can be specified by a range, by a lambda.
      
                              // non-deterministic
                              mock.Setup(foo => foo.DoSomething("hello")).Returns((New Random()).Next(1000));
      
                              // throws exceptions
                              mock.Setup(foo => foo.DoSomething("hello"))
                                  .Throws<InvalidOperationException>();
                              mock.Setup(foo => foo.DoSomething("hi"))
                                  .Throws(new ArgumentException("command"));
      
                              // stateful
                              // 1st call returns "10", 2nd call returns "11", 3rd call throws error.
                              formatterMock.SetupSequence(x => x.Format(It.IsAny<int>()))
                                      .Returns("10")
                                      .Returns("11");
      
                              // 1st call returns "10", 2nd call returns "11", 3rd call throws exception.
                              formatterMock.SetupSequence(x => x.Format(It.IsAny<int>()))
                                      .Returns("10")
                                      .Returns("11")
                                      .Throws(new ArgumentException("command"));
                              
                          

      sequential asycn

    2. Initialize target objects with Autofac

      The simple way is to create Mock object and initialize a target object by using its constructor. Another approach is to use Autofac Dependency Injection to inject Mock Object.

      Strict mocking: If the test method invokes that method or property and the Mock object does not set them, then it will throw exceptions.
      Loose mocking: If the test method invokes that method or property and the Mock object does not set them, then you get the default value for that operation (which mostly will be a zero or a null). For example, calling a method "string Format(int i)" will get null.

                                  using Autofac.Extras.Moq;
                                  using Moq;
                                  using Microsoft.VisualStudio.TestTools.UnitTesting;
                                  using Service;
                                  using Targets;
                                  namespace targets.tests
                                  {
                                      [TestClass]
                                      public class AnotherTargetTests
                                      {
                                          private AutoMock _mock; // a container
                                          private Target _target; // target object.
                                          [TestInitialize]
                                          public void Init()
                                          {
                                              _mock = AutoMock.GetLoose();
                                              _target = _mock.Create<Target>();
                                          }
                                  
                                          [TestMethod]
                                          public void Print_WithNumberArray_ShouldReturnString()
                                          {
                                              /*
                                               * var formatterMock = new Mock<IFormatter>();
                                               * formatterMock.Setup(x => x.Format(It.IsAny<int>()))
                                                  .Returns("11");
                                              */
                                              // register and configure dependencies
                                              _mock.Mock<IFormatter>()
                                                  .Setup(x => x.Format(It.IsAny<int>()))
                                                  .Returns("11");
                                  
                                              // var target = new Target(formatterMock.Object);
                                  
                                              var result = _target.Print(0, 2, 3);
                                              Assert.AreEqual("111111", result);
                                              _mock.Mock<IFormatter>().Verify(x => x.Format(It.IsAny<int>()), Times.Exactly(3));
                                              Assert.AreEqual(0, It.IsAny<int>());
                                          }
      
                                          [TestMethod]
                                          public void Print_WithNumberArray_ShouldReturnString2()
                                          {
                                              // even though we still use the same container, but dependencies are reset!!!!
                                              var result = _target.Print(0, 2, 3);
                                              Assert.AreEqual("", result);
                                              _mock.Mock<IFormatter>().Verify(x => x.Format(It.IsAny<int>()), Times.Exactly(3));
                                              Assert.AreEqual(0, It.IsAny<int>());
                                          }
      
                                      }
                                  }
                          

      Because container is reset for each test method, it is better to use the following manner.

                                  [TestMethod]
                                  public void Print_WithNumberArray_ShouldReturnString()
                                  {
                                      using (var mock = AutoMock.GetLoose())
                                      {
                                          var formatterMock = mock.Mock<IFormatter>();
                                          formatterMock.Setup(x => x.Format(It.IsAny<int>()))
                                              .Returns("13");
                                          var target = mock.Create<Target>(new NamedParameter("formatter", formatterMock.Object)); // with a designated parameter.
                                          var result = target.Print(0, 2, 3);
                                          Assert.AreEqual("131313", result);
                                          mock.Mock<IFormatter>().Verify(x => x.Format(It.IsAny<int>()), Times.Exactly(3));
                                      }
                                  }
                          

      ** Target objects can be .Create before registering dependencies. When dependencies is registered, the behavior of target will be modified. ***

    3. Moq Verification

    MSTest APIs

    1. Assert family