private override

Autotest… in .NET

The first time I saw autotest (presented by Anthony), the idea of Continuous Testing captured me.

I live in a .NET world most of the time, and I know of no similar solution for .NET.  It’s been awhile since that first time, and I’ve tinkered here and there trying to get something comparable, but usually come up short.  That is until I found watchr.

Watchr gave me the file change detection capabilities I needed, and the extensibility to do whatever I want when a file has been detected as changed.  This made it incredibly easy to hook up some autotest goodness in my .NET world.

You'll have to have ruby installed, and gems.  Then, the very first thing you'll have to do is

gem install watchr --source=http://gemcutter.org

Here is my watchr script:

require 'autotest.rb'

watch( '^.*UnitTest.*.cs$' ) do |match|
  run_test(match.to_s)
end

This is basically just a regex that says to watch any *.cs files that also contain the string “UnitTest”, and when it finds a change in a file matching that description, call run_test with the matched file name.

So all the magic is in autotest.rb… lets check it out:

require 'rexml/document'

def build(test_project)
  `msbuild /nologo #{test_project}`
end

def mstest(test_container, test_results_file, tests_to_run)
  tests_to_run = ([""] << tests_to_run).flatten

  File.delete(test_results_file) if File.exists?(test_results_file)
  `mstest /nologo /resultsfile:#{test_results_file} /testcontainer:#{test_container} #{tests_to_run.join(" /test:")}`
  test_results = process_mstest_results(test_results_file)
  File.delete(test_results_file) if File.exists?(test_results_file)

  return test_results
end

def process_mstest_results(results_file)
  results = {}
  File.open(results_file) do |file|
    xml = REXML::Document.new(file)

    results[:num_tests] = xml.get_elements("//UnitTestResult").length
    failures = []
    xml.elements.each("//UnitTestResult[@outcome='Failed']") do |e|
      failure = {}
      failure[:message] = e.elements["Output/ErrorInfo/Message"].get_text

      stack = e.elements["Output/ErrorInfo/StackTrace"].get_text.value
      stack_match = /^.*at (.*) in(.*):line (\d+)$/.match(stack)

      failure[:stack] = stack_match[1] if stack_match
      failure[:location] = stack_match[2] if stack_match
      failure[:line] = stack_match[3] if stack_match

      failure[:stack] = stack if !stack_match

      failures << failure
    end
    results[:failures] = failures
  end

  return results
end

def show_results(results)
  puts "#{results[:num_tests]} tests run (#{results[:failures].length} failures)"
  results[:failures].each do |failure|
      puts "---------------------------------------"
      puts "Message: #{failure[:message]}"
      puts "Location: #{failure[:location]}"
      puts "Line: #{failure[:line]}"
      puts "Stack Trace: #{failure[:stack]}"
  end
end

def run_test(file_name)
  test_container = ""
  test_results_file = "result.trx"
  test_project = ""

  system("cls")
  system("echo Detected change in:")
  system("echo   #{file_name}")
  system("echo Building and Testing")

  test_namespace = ''
  test_class = ''
  test_names = []

  File.open(file_name, "r") do |f|
    f.each do |line|
      ns_match = /^namespace (.*)$/.match(line)
      test_namespace = ns_match[1] if ns_match

      class_match = /^\s*public class (.\w*).*$/.match(line)
      test_class = class_match[1] if class_match

      test_name_match = /^\s*public void (\w*).*$/.match(line)
      test_names << test_name_match[1] if test_name_match
    end
  end

  test_names = test_names.map { |n| "#{test_namespace}.#{test_class}.#{n}" }

  build(test_project)
  results = mstest(test_container, test_results_file, test_names)
  show_results(results)
end

The key parts (I think) are the fact that I’m using MSTest to run my tests (this can easily be modified to run your framework of choice… note MSTest is not my choice ).  The result parsing is also specific to the MSTest output format, but should be simple enough for any framework that can output XML. Also, I'm making some assumptions based on my project... we've got one unit test project, so I know I can run tests in a single DLL, and rebuilding only that project, I don't have to worry about choosing the correct project and output dll to build and run tests in.

To get the thing up and running, just run

watchr <path to watchr script>

Please, use/adapt/give feedback/whatever at will.

Go forth and autotest, .NET comrades!