| Ernesto Guisado's Website » Articles » Testing Win32 GUIs with Perl and C | Articles | Miscellanea | |
Testing methods vary widely depending on the code being tested. Programs designed to interact with other programs can be easily tested. If your end-product is a C++ library, for example, you can easily generate a test program that makes use of its functions. For GUI-based programs, Dynamic Testing is a common solution, where you simply fire up the program and use it in the way a typical user would. If your organization has an effective software development process, you might have a test case document detailing all the things that you should be trying out and the expected outcome. For a program of any complexity, this means a lot of time. Testing GUI programs is a challenging task. So challenging in fact that the first rule of GUI testing is: Don’t!
I don’t mean you shouldn’t test at all, but you should design your programs in a way that allows them to be tested without relying on costly GUI testing. If you use a sound architecture separating your program into a GUI part and an engine or server part that does all the dirty work, you can test all the functionality of the engine without actually using the GUI. What remains to be tested is actually checking that the buttons are "hooked" to the right functions (of course, I’m simplifying grossly here).
Even if you’ve tried hard to separate user interface from
engine code, you may still want to test your GUI. I’ve
written the Win32::GuiTest Perl module (see Resources)
in a combination of Perl and C. It bundles together the facilities
for GUI testing offered by the Win32 API with some powerful
general-purpose Perl functionality. The combination helps automate
the GUI testing process.
The first step is starting the application that you want to test. Perl offers the system function:
system("start notepad.exe");
The system function has one drawback: it doesn’t return until the application you’ve invoked is closed. The windows command shell (cmd.exe) supports the start command allowing you to start an application in a new window, thereby avoiding this limitation. The start command also allows a certain degree of control over the application you’re starting. The following code starts an application but in a maximized window:
system("start /max notepad.exe");
If you’re invoking a console application, you might want
to process the text output somehow. In that case, Perl’s
qx// operator is more useful. If you need some extra
control you can also use the Win32::Process Perl
module. This allows suspending, killing, or waiting until the
application finishes.
Once you’ve started the application, you want to make sure
that you’re "talking" with the right window. You don’t
want to send the keystroke commands to your e-mail program
inadvertently and end up sending some random API calls to your mom.
The usual approach is to traverse the window hierarchy looking for
a window with the right name or attributes. To that effect, the
Win32 API provides the EnumChildWindows function.
I’ve created a Perl wrapper called
FindWindowLike around this function.
FindWindowLike returns all the children of a given
window that match a particular title, window class, or control id.
FindWindowLike also allows limiting the search to a
particular depth and makes full use of Perl’s powerful
regular expression engine.
my @w = FindWindowLike(0, "^Microsoft Excel", "^XLMAIN\$");
This example looks for Microsoft Excel using pattern matching on
the window title and the window class. The first argument indicates
that the search should start with the desktop window. If
you’re looking for a button, you might prefer starting with
the dialog box window instead. Once you’ve found the window,
you might need to bring it to the foreground to be able to interact
with it. The SetForegroundWindow function is just a
wrapper around the Win32 function of same name. On Windows 2000,
using this function might be problematic because only the process
that created the window is allowed to bring it to the foreground. A
workaround is to use the ALT+TAB key sequence to
change the foreground window.
Now we’re ready to tell the window what to do. The
SendKeys function is a clone of the function with same
name that you have in Visual Basic and is built upon the
keybd Win32 API function. SendKeys uses a
string to represent the characters that you want to send to the
active window. The window effectively behaves as if the end user
had typed those characters into the keyboard. SendKeys
also allows certain meta-characters in the string specifying things
like CTRL, ALT, or
BACKSPACE. Consult the Win32::Guitest
documentation for a full description of the supported format. The
following example sends the key sequence ALT+f,
n, ENTER, a,
TAB, b, TAB,
BACKSPACE:
SendKeys("%fn~a{TAB}b{TAB}{BS}");
For completeness sake, GuiTest also includes functions that allow simulating user input using a mouse. The following code opens a well-known paint program and draws a triangle in it:
system("start /max pbrush.exe");
sleep 2;
MouseMoveAbsPix(100,100);
SendLButtonDown();
MouseMoveAbsPix(500,500);
MouseMoveAbsPix(100,500);
MouseMoveAbsPix(100,100);
SendLButtonUp();
After you’ve made your application jump through some hoops
using SendKeys and the mouse functions, you actually
have to verify that the application worked as expected. As
we’ll see in the next section, a whole range of errors
(namely navigation errors) are detected simply by running the
script: if the script is able to finish, the navigation has to be
working.
In order to test your application more thoroughly, you’ll have to apply some ingenuity. No silver bullets for this part. However, I’ll give you some ideas on how to do it using the functions in GuiTest.
GetWindowText.IsCheckedButton.GetListContents and
GetComboContents.GetClassName.WMGetText.FindWindowLike or GetChildWindows.See the fonts.pl script adapted from
the GuiTest distribution for an example on how to extract
information from a dialog box using a combination of
system, SendKeys,
FindWindowLike, and GetComboContents.
This script needs to be adapted to the language of the machine you
plan to use it on. I’ve marked those dependencies with the
text l10n.
Using the mouse handling functions to navigate through the menus and dialog boxes in an application is difficult. You have to rely on knowing the exact layout of the controls, because you have to specify exact coordinates for each mouse movement. It is much easier to navigate using the keyboard. These are the main tools to help you in that area:
CTRL+C for copy is a good example).ALT+f to open the File menu.TAB and correct tab order to help you
navigate through window controls.ENTER or SPACE to accept a dialog
box.If you get your tabs wrong or forget some hotkeys, the user might still be able to make his way using the mouse, but your test script will break. GUI testing your application actually forces you to follow certain minimum usability standards when building the application.
The example script (download here) also serves to illustrate some
problems related to localization (l10n). The fonts.pl script uses
the caption of the font dialog box to verify that the menu
navigation worked. "Font" in Spanish is Fuente, so this
needs to be changed. The script also uses the ALT+E,F
key sequence to navigate to the font dialog box. This depends on
the particular menu names and hotkeys assigned in your language
version of Notepad. Some GUI specific localization dependencies you
need to be aware of are as follows:
CTRL-N (for New) might
be CTRL+U in Spanish. Some shortcut keys are widely
used (like CTRL+C, X, V for
copy/cut/paste), but in most cases shortcut keys are considered
mnemonics and therefore different from one language to the
next.In the end, this means quite simply that you’ll have to adapt your test script for the different languages that you plan to support. In that case, it will pay-off nicely if you’ve used sound programming techniques even in your test scripts. If you’ve done that, you’ll have just one file shared by all scripts that defines the key sequences necessary to reach any GUI element. This is the second law of GUI testing: Test-script development is software development. Don’t treat it as a second-class activity. All the nice things that you learned to help make programs more maintainable apply to your test scripts as well. I strongly believe this to be true and as a consequence, I don’t plan to implement any type of keystroke recording program because I believe that this encourages sloppy programming and treating test scripts as a throw-away side product.
Being written in Perl, GuiTest plays well with other Perl
modules. Some popular ones are Win32::API,
Win32::OLE, and Win32::Clipboard. A
pretty common scenario is some GuiTest user finding a creative use
of the module by combining it with Win32::API, after
which I normally try to put the corresponding Win32 API function
into GuiTest, to make it more convenient to use. This is what
happened with the PostMessage function.
Not surprisingly, a lot of people use GuiTest for GUI testing
(stress testing or monkey testing being particularly popular), but
others use it for a wide range of ad-hoc tasks, from transferring
mail between incompatible systems to automating a GUI program to
perform a tedious nightly task. A common Win32 technique that has
proved useful in a wide variety of contexts is using the
SendKeys function to send the ATL+TAB key
combination. This allows switching between open windows just as if
a user had done it:
SendKeys("%{TAB}");
GuiTest has its limitations. As I mentioned, it lacks a keystroke recording utility. Another often requested feature is being able to take screen-shots and comparing them with saved images. This would be useful for regression testing because you could take a screen-shot of a dialog box and later on detect any changes to it.
Part of my background is in software localization, so I’m a bit wary of this functionality. Localizing the GUI changes the layout a lot — different languages have different lengths for strings (Spanish and German strings are at least 20 percent longer than English strings), and this means changing the sizes of dialogs and controls. A screen-shot-based test suite is useless to test the localized version of your product. Also, maintaining this kind of test suite is a pain during development because dialog box layout changes a lot during prototyping and usability testing. Even so, if demand is high for this functionality, I might eventually include it; in fact, some users have already sent me patches to implement parts of it. If you think this would be a worthwhile feature, drop me a line and let me know.
Win32::GuiTest Perl module is updated regularly.
The latest version is available at the following URLs:I’d like to thank all the people who have helped me with GuiTest over the last few years. You’ll find some of them mentioned in the README file that comes with the distribution. There is a Yahoo! Group dedicated to the use of GuiTest. You can join it at: http://groups.yahoo.com/group/perlguitest/join.