Automated Testing Methods and Frameworks by way of Microsoft

Through an email exchange with a friend, we went entered a high-level discussion with respect to Microsoft automated testing methods and frameworks. The following is a result of that discussion.

It’s always interesting sharing with other developers to gain additional insight as to what their development environments are like.

I present below three primary discussion questions and a summary question:

  • Is it going to take more time to do this Test-Driven Development (TDD) process?
  • Who, what, where, when, and how do I start with TDD?
  • How do I convince my stakeholders, my customers and teams, that TDD is worth the effort?
  • What were some of the key discussion points?

Q: How long does TDD take?

A: This is a loaded question. It all depends on various factors. Time for who and to do what exactly? Time for the developer? The project team? QA? Business analysts? The end-customer? How are you approaching, implementing, and verifying TDD? Also, is quality a measurable output of the product? If you are not going to be around when the bugs show up, what is your incentive to deliver quantifiable quality? And, no, writing tests alone does not guarantee quality. Over time, quality TDD will reduce the total cost of software. It will also reduce the time to fix software bugs. Good TDD makes certain bugs NEVER reappear (I make it an explicit point to write a test for every bug at a minimum). Wrapping TDD around business requirements minimizes the impact of developer assumptions on the product. Mistakes will still happen and the business requirements can also be flawed from the start or through translation. Writing tests as a specification before the implementation further reduces impacts by forcing the developer to understand and codify requirements up-front before fulfilling them through implementation.

Q: Who, what, where, when, and how do I start with TDD?

A: Hopefully the answer is business requirements or technical requirements tied to business requirements.

For stakeholders who are not technical, evangelize TDD and enable developers in whatever way you can.

For the little piggies who bacon is on the line (the developer), start with a requirement that can be solved in minutes (break down the task), open up a code file, write a few lines of code that will call a non-existent code API that could do what is expected (you are codifying the actual design of your API explicitly as a test client of the API). Now, code the implementation. Compile and run the test, tweaking code until the initial test expectations of the API are met (and nothing extra!). Next requirement please. Of course, supporting software will influence the actual approach taken.

*For bug fixes, write a test to verify the bug exists and then modify the software to fix the bug.

Q: What is the best approach in convincing my stakeholders, my customers and teams, TDD is worth the effort?

A: Do I need to deal with a lot of variables?

This is the simple, primary Bug Argument: “How does your customer / team ensure that a bug never reappears in the software after it has been fixed?”

This is the Reverse-Maintenance Argument: Developers will utilize and modify code that wasn’t authored by them, or that they wrote, but have become unfamiliar with over time. When they use or modify the code, they will do so with certain and often necessary assumptions (conscious or subconscious). Assumptions, especially under pressure, are ripe with potential for creating bugs. If the code being modified has associated tests, the developer can test and verify their assumptions against existing expectations of the software; expectations as business requirements, technical requirements, as well as the explicit and codified assumptions of previous authors.

This is the Forward-Maintenance Argument: Developers are forced to consider future maintenance of their solutions and document expectations and assumptions up-front, because writing test clients forces developers to use the implementation API from a different perspective – the perspective of consumers and other developers.

Q: What were some of the key discussion points?

A: Let us look at the core of how this evolved from Microsoft’s vision.

Microsoft “Code Contracts” was emphasized for two reasons: 1) It allows developers to state expectations and assumptions as embedded code specifications that can be optionally verified under specified run-time configurations and optionally at compile-time through static analysis. 2) It has been included in .NET 4.0 and Visual Studio 2010.

Fitnesse.NET opened eyes toward better testing integration throughout the development lifecycle and greater testing visibility for QA and optionally the business analysts and/or stakeholders.

More information: Microsoft Automated Testing Methods and Frameworks


I am a technologist with a strong background in software engineering. I have many interests. My current distractions are 70s-80s-90s music [it's a very eclectic collection], ontology, information architecture, mobile device technology, medical bioinformatics, artificial intelligence, and nanorobotics.

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: