Improving GUI Test Maintenance with a Data Driven Framework

One the problems with automated GUI testing is maintenance. Dialog titles change, buttons move, tabs change, etc. These changes generally break automated tests. To make tests maintenance easier I have been working on a test framework that includes JSON files which define UI elements. This almost certainly will not be the final definition, but I thought I'd throw it out there and see if I get comments back on the structure.

Here is an example of a definition for the Page Setup dialog in Notepad.

"Notepad Page Setup": {
   Name: "Page Setup",
   Keystrokes: "%(fu)",
   SelectionLists: {
      "Sizes": {
         AutomationId: "1137",
         SelectionItems: {
            "Ledger": {
               Name: "11x17",
            },
            "A5": {
               Name: "A5",
            },
            "Letter": {
               Name: "Letter",
            }
         }
      }
   },
   Fields: {
      "Header": {
         AutomationId: "30"
      },
      "Footer": {
         AutomationId: "31"
      }
   }
   Buttons: {
      "Ok": {
         AutomationId: "1"
      }
   }
}

You will need UISpy to find the Name and/or AutomationId for UI elements.

You can think of this structure as a hash table with key => value pairs. The main element is "Notepad Page Setup". This is the key we will use to reference the Page Setup dialog in our test code. The value, in this case, is a bunch more key => value pairs that define the GUI element. BTW this is not a complete definition of the Page Setup dialog.

As an example I'll explain the paper size drop down on the Page Setup dialog (launch Notepad and UISpy if you want to follow along). The paper size drop down is a Selection List and in our definition we have called it "Sizes". It's AutomationId is "1137" (from UISpy).

The "Sizes" list is made of many Selection Items. I have only include a few here: "Ledger", "A5", and "Letter". These are the names that will be used in the test code to refer to these elements. Notice the hash key ("Ledger") does not have to match the Name of the element name ("11x17").

Here are some general definitions:

Name:

The name of the UI element (from UISpy).  The Name or the AutomationId can be used to find UI elements.

AutomationId:

The AutomationId for the UI element (from UISpy)

Keystroke:

Key stroke pattern to open the UI element

SelectionLists:

Array of Selection List defintions. Selection Lists are UI element that allow the user to chose one or more items from a list, e.g. ComboBox, ListBox, and even a DataGridView.

SelectionItem:

The definition for an item in a SelectionList

Fields:

Test fields

Buttons:

Generally buttons, but could be other things the user might click

The code that uses this JSON definition might look something like this:

[Test]
public void SetPageSize()
{
   // Assume Notepad has already been opened
   ...
   // Open the Page Setup Dialog
   UiItem ui = UiItem.Open("Notepad Page Setup");
   ui.SelectionLists["Sizes"].SelectionItems["Ledger"].Select();
   ui.Buttons["Ok"].Click();
   ...
}

Now if the title of the dialog changes or the keystrokes it launch it or the name of a paper size changes we don't have to modify the test code.  Instead we just update the JSON definition and any tests using that definition will automatically get the new definition.  All the test code remains unchanged 8]