Testing Win32 GUIs with Perl and C


Ernesto Guisado
This work was originally published in the February 2002 issue of Windows Developer Magazine, copyright 2002.

Testing methods vary widely depending on the code being tested. Programs designed to interact with other programs can be easily tested. If your end-product is a C++ library, for example, you can easily generate a test program that makes use of its functions. For GUI-based programs, Dynamic Testing is a common solution, where you simply fire up the program and use it in the way a typical user would. If your organization has an effective software development process, you might have a test case document detailing all the things that you should be trying out and the expected outcome. For a program of any complexity, this means a lot of time. Testing GUI programs is a challenging task. So challenging in fact that the first rule of GUI testing is: Don’t!

I don’t mean you shouldn’t test at all, but you should design your programs in a way that allows them to be tested without relying on costly GUI testing. If you use a sound architecture separating your program into a GUI part and an engine or server part that does all the dirty work, you can test all the functionality of the engine without actually using the GUI. What remains to be tested is actually checking that the buttons are "hooked" to the right functions (of course, I’m simplifying grossly here).

The GuiTest Perl module

Even if you’ve tried hard to separate user interface from engine code, you may still want to test your GUI. I’ve written the Win32::GuiTest Perl module (see Resources) in a combination of Perl and C. It bundles together the facilities for GUI testing offered by the Win32 API with some powerful general-purpose Perl functionality. The combination helps automate the GUI testing process.

The first step is starting the application that you want to test. Perl offers the system function:

system("start notepad.exe");

The system function has one drawback: it doesn’t return until the application you’ve invoked is closed. The windows command shell (cmd.exe) supports the start command allowing you to start an application in a new window, thereby avoiding this limitation. The start command also allows a certain degree of control over the application you’re starting. The following code starts an application but in a maximized window:

system("start /max notepad.exe");

If you’re invoking a console application, you might want to process the text output somehow. In that case, Perl’s qx// operator is more useful. If you need some extra control you can also use the Win32::Process Perl module. This allows suspending, killing, or waiting until the application finishes.

Once you’ve started the application, you want to make sure that you’re "talking" with the right window. You don’t want to send the keystroke commands to your e-mail program inadvertently and end up sending some random API calls to your mom. The usual approach is to traverse the window hierarchy looking for a window with the right name or attributes. To that effect, the Win32 API provides the EnumChildWindows function. I’ve created a Perl wrapper called FindWindowLike around this function. FindWindowLike returns all the children of a given window that match a particular title, window class, or control id. FindWindowLike also allows limiting the search to a particular depth and makes full use of Perl’s powerful regular expression engine.

my @w = FindWindowLike(0,
  "^Microsoft Excel",
  "^XLMAIN\$");

This example looks for Microsoft Excel using pattern matching on the window title and the window class. The first argument indicates that the search should start with the desktop window. If you’re looking for a button, you might prefer starting with the dialog box window instead. Once you’ve found the window, you might need to bring it to the foreground to be able to interact with it. The SetForegroundWindow function is just a wrapper around the Win32 function of same name. On Windows 2000, using this function might be problematic because only the process that created the window is allowed to bring it to the foreground. A workaround is to use the ALT+TAB key sequence to change the foreground window.

Now we’re ready to tell the window what to do. The SendKeys function is a clone of the function with same name that you have in Visual Basic and is built upon the keybd Win32 API function. SendKeys uses a string to represent the characters that you want to send to the active window. The window effectively behaves as if the end user had typed those characters into the keyboard. SendKeys also allows certain meta-characters in the string specifying things like CTRL, ALT, or BACKSPACE. Consult the Win32::Guitest documentation for a full description of the supported format. The following example sends the key sequence ALT+f, n, ENTER, a, TAB, b, TAB, BACKSPACE:

SendKeys("%fn~a{TAB}b{TAB}{BS}");

For completeness sake, GuiTest also includes functions that allow simulating user input using a mouse. The following code opens a well-known paint program and draws a triangle in it:

system("start /max pbrush.exe");
sleep 2;
MouseMoveAbsPix(100,100);
SendLButtonDown();
MouseMoveAbsPix(500,500);
MouseMoveAbsPix(100,500);
MouseMoveAbsPix(100,100);
SendLButtonUp();

After you’ve made your application jump through some hoops using SendKeys and the mouse functions, you actually have to verify that the application worked as expected. As we’ll see in the next section, a whole range of errors (namely navigation errors) are detected simply by running the script: if the script is able to finish, the navigation has to be working.

In order to test your application more thoroughly, you’ll have to apply some ingenuity. No silver bullets for this part. However, I’ll give you some ideas on how to do it using the functions in GuiTest.

  1. Test the title of the window you’re on using GetWindowText.
  2. Test if a particular check-box is checked or not using IsCheckedButton.
  3. Test the contents of list and combo-boxes using GetListContents and GetComboContents.
  4. Test the class name of the current window using GetClassName.
  5. Extract the text from an edit control using WMGetText.
  6. Get the program to generate an output file and compare with some predefined template.
  7. Test the structure of your dialog box with FindWindowLike or GetChildWindows.

See the fonts.pl script adapted from the GuiTest distribution for an example on how to extract information from a dialog box using a combination of system, SendKeys, FindWindowLike, and GetComboContents. This script needs to be adapted to the language of the machine you plan to use it on. I’ve marked those dependencies with the text l10n.

Make your GUI easy to test

Using the mouse handling functions to navigate through the menus and dialog boxes in an application is difficult. You have to rely on knowing the exact layout of the controls, because you have to specify exact coordinates for each mouse movement. It is much easier to navigate using the keyboard. These are the main tools to help you in that area:

  1. Short-cut keys to invoke functionality without navigating at all (CTRL+C for copy is a good example).
  2. Hotkeys like ALT+f to open the File menu.
  3. Relying on TAB and correct tab order to help you navigate through window controls.
  4. ENTER or SPACE to accept a dialog box.

If you get your tabs wrong or forget some hotkeys, the user might still be able to make his way using the mouse, but your test script will break. GUI testing your application actually forces you to follow certain minimum usability standards when building the application.

L10N Woes

The example script (download here) also serves to illustrate some problems related to localization (l10n). The fonts.pl script uses the caption of the font dialog box to verify that the menu navigation worked. "Font" in Spanish is Fuente, so this needs to be changed. The script also uses the ALT+E,F key sequence to navigate to the font dialog box. This depends on the particular menu names and hotkeys assigned in your language version of Notepad. Some GUI specific localization dependencies you need to be aware of are as follows:

In the end, this means quite simply that you’ll have to adapt your test script for the different languages that you plan to support. In that case, it will pay-off nicely if you’ve used sound programming techniques even in your test scripts. If you’ve done that, you’ll have just one file shared by all scripts that defines the key sequences necessary to reach any GUI element. This is the second law of GUI testing: Test-script development is software development. Don’t treat it as a second-class activity. All the nice things that you learned to help make programs more maintainable apply to your test scripts as well. I strongly believe this to be true and as a consequence, I don’t plan to implement any type of keystroke recording program because I believe that this encourages sloppy programming and treating test scripts as a throw-away side product.

Written for Openness

Being written in Perl, GuiTest plays well with other Perl modules. Some popular ones are Win32::API, Win32::OLE, and Win32::Clipboard. A pretty common scenario is some GuiTest user finding a creative use of the module by combining it with Win32::API, after which I normally try to put the corresponding Win32 API function into GuiTest, to make it more convenient to use. This is what happened with the PostMessage function.

Not surprisingly, a lot of people use GuiTest for GUI testing (stress testing or monkey testing being particularly popular), but others use it for a wide range of ad-hoc tasks, from transferring mail between incompatible systems to automating a GUI program to perform a tedious nightly task. A common Win32 technique that has proved useful in a wide variety of contexts is using the SendKeys function to send the ATL+TAB key combination. This allows switching between open windows just as if a user had done it:

SendKeys("%{TAB}");

GuiTest has its limitations. As I mentioned, it lacks a keystroke recording utility. Another often requested feature is being able to take screen-shots and comparing them with saved images. This would be useful for regression testing because you could take a screen-shot of a dialog box and later on detect any changes to it.

Part of my background is in software localization, so I’m a bit wary of this functionality. Localizing the GUI changes the layout a lot — different languages have different lengths for strings (Spanish and German strings are at least 20 percent longer than English strings), and this means changing the sizes of dialogs and controls. A screen-shot-based test suite is useless to test the localized version of your product. Also, maintaining this kind of test suite is a pain during development because dialog box layout changes a lot during prototyping and usability testing. Even so, if demand is high for this functionality, I might eventually include it; in fact, some users have already sent me patches to implement parts of it. If you think this would be a worthwhile feature, drop me a line and let me know.

Resources

The Win32::GuiTest Perl module is updated regularly. The latest version is available at the following URLs:
http://www.cpan.org/authors/id/E/ER/ERNGUI/
http://search.cpan.org/search?dist=Win32-GuiTest
http://groups.yahoo.com/group/perlguitest/files/

I’d like to thank all the people who have helped me with GuiTest over the last few years. You’ll find some of them mentioned in the README file that comes with the distribution. There is a Yahoo! Group dedicated to the use of GuiTest. You can join it at: http://groups.yahoo.com/group/perlguitest/join.