Automated Testing Basics
A lot of people have asked Greg and me about automated testing. This is an important subject for me since I feel like it’s the most important part of my job. I’m a smart guy and I know a lot about software development, so it’s clearly not the best use of my time to click on the same button looking for the same dialog box every single day. Part of smart testing is delegating those kinds of tasks away so I can spend my time on harder problems. And computers are a great place to delegate repetitive work.
That’s really what automated testing is about. I try to get computers to do my job for me. One of the ways I describe my goal as a tester is to put myself out of a job – meaning that I automate my entire job. This is, of course, unachievable so I don’t worry about losing my job. But it’s a good vision. My short term goal is always to automate the parts of my job I find most annoying with the selfish idea of not having to do annoying stuff any more.
With people new to automated testing, that’s always how I frame it. Start small, pick an easy menial task that you have to do all the time. Then figure out how to have a computer do it for you. This has a great effect on your work since after you get rid of the first one that will free up more of your time to automate more and more annoying, repetitive tasks. Now with all this time you can go an focus on testing more interesting parts of your software.
That last paragraph makes it sound like writing automated tests is easy, when in fact it’s typically quite hard. There are some fundamentally hard problems in this space, and those are what I’m going to talk about most here. Lots of test tools try to help out with these problems in different ways, and I highly recommend finding a tool that works for you. Unfortunately all the tools I use here at Microsoft are custom built internal tools (it’s one of the advantages of a huge development team, we have people who’s full time jobs are developing test tools for us.) So I can’t really recommend specific tools. But I will talk about are the two major parts of an automated test, the major problems involved, and some of the general ways to solve those problems. Hopefully it will be valuable as a way to better understand automated testing and as a way to help choose your own test tools. As a side note, implementing automated tests for a text based or API based system is really pretty easy, I’m going to focus on a full UI application – which is where the interesting issues are.
Automated test can be broken into two big pieces: Driving your program and Validating the results.
