Improving GUI Test Maintenance with a Data Driven Framework

One the problems with automated GUI testing is maintenance. Dialog titles change, buttons move, tabs change, etc. These changes generally break automated tests. To make tests maintenance easier I have been working on a test framework that includes JSON files which define UI elements. This almost certainly will not be the final definition, but I thought I'd throw it out there and see if I get comments back on the structure.

Here is an example of a definition for the Page Setup dialog in Notepad.

"Notepad Page Setup": {
   Name: "Page Setup",
   Keystrokes: "%(fu)",
   SelectionLists: {
      "Sizes": {
         AutomationId: "1137",
         SelectionItems: {
            "Ledger": {
               Name: "11x17",
            },
            "A5": {
               Name: "A5",
            },
            "Letter": {
               Name: "Letter",
            }
         }
      }
   },
   Fields: {
      "Header": {
         AutomationId: "30"
      },
      "Footer": {
         AutomationId: "31"
      }
   }
   Buttons: {
      "Ok": {
         AutomationId: "1"
      }
   }
}

You will need UISpy to find the Name and/or AutomationId for UI elements.

You can think of this structure as a hash table with key => value pairs. The main element is "Notepad Page Setup". This is the key we will use to reference the Page Setup dialog in our test code. The value, in this case, is a bunch more key => value pairs that define the GUI element. BTW this is not a complete definition of the Page Setup dialog.

As an example I'll explain the paper size drop down on the Page Setup dialog (launch Notepad and UISpy if you want to follow along). The paper size drop down is a Selection List and in our definition we have called it "Sizes". It's AutomationId is "1137" (from UISpy).

The "Sizes" list is made of many Selection Items. I have only include a few here: "Ledger", "A5", and "Letter". These are the names that will be used in the test code to refer to these elements. Notice the hash key ("Ledger") does not have to match the Name of the element name ("11x17").

Here are some general definitions:

Name:

The name of the UI element (from UISpy).  The Name or the AutomationId can be used to find UI elements.

AutomationId:

The AutomationId for the UI element (from UISpy)

Keystroke:

Key stroke pattern to open the UI element

SelectionLists:

Array of Selection List defintions. Selection Lists are UI element that allow the user to chose one or more items from a list, e.g. ComboBox, ListBox, and even a DataGridView.

SelectionItem:

The definition for an item in a SelectionList

Fields:

Test fields

Buttons:

Generally buttons, but could be other things the user might click

The code that uses this JSON definition might look something like this:

[Test]
public void SetPageSize()
{
   // Assume Notepad has already been opened
   ...
   // Open the Page Setup Dialog
   UiItem ui = UiItem.Open("Notepad Page Setup");
   ui.SelectionLists["Sizes"].SelectionItems["Ledger"].Select();
   ui.Buttons["Ok"].Click();
   ...
}

Now if the title of the dialog changes or the keystrokes it launch it or the name of a paper size changes we don't have to modify the test code.  Instead we just update the JSON definition and any tests using that definition will automatically get the new definition.  All the test code remains unchanged 8]

Sitemap for Blogger When Using FeedBurner

So I signed up with FeedBuner for CoderDoWhat.com and it broke (sort of) the sitemap used by Web Master Tools. Normally in WMT I would use http://blog.coderdowhat.com/atom.xml for the sitemap, but when you signup for and configure FeedBurner, requests for atom.xml get redirected to http://feeds.feedburner.com/coderdowhat which is not what I want to use as my sitemap.

As an alternative to atom.xml you can use http://blog.yourdomain.com/feeds/posts/full as your sitemap URL in WMT. This is not redirected and works perfect.

Implementing Join() for Arrays in C# Using Extension Methods

Microsoft seems pretty good at borrowing good ideas from others, so why is it that .NET still doesn't include a Join() method for IEnumerables? Luckily it's easy to add this functionality using extension methods. Here is a fairly simple implementation.

using System;
using System.Collections;
using System.Text;
using System.Collections.Generic;

namespace Helpers
{
    public static class CollectionHelper
    {
        public static string Join(this IEnumerable list, string separator)
        {
            string text = "";

            if (null == list)
                return text;

            foreach (object item in list)
            {
                text += String.Format(
                    "{0}{1}", item.ToString(), separator);
            }

            int idx = text.LastIndexOf(separator);
            if (-1 < idx)
            {
                text = text.Remove(text.LastIndexOf(separator));
            }

            return text;
        }
    }
}

You can include the Join() method by adding a using Helpers.CollectionHelpers statement to your source file. It looks something like this.

using System;
using System.Collections;

namespace MyAppSpace
{
    using Helpers.CollectionHelper;

    public partial class SomeClass
    {
        ...

        ...

        public string ProcessData(IList<double> data)
        {
            List<double> new_data = new List<double>();

            double value = 0;
            foreach (double d in data)
            {
                value += d
                new_data.Add(value);
            }

            return new_data.Join(','); // <= Extension method
        }
    }
}

AutomationElement.SetFocus() is Unreliable

Microsoft's UiAutomation framework has been fairly useful in my quest to automate all our GUI testing.  My previous solution involved Ruby, WinUser32Ruby, and RSpec (which was a fun solution), but I got tired of try to wrap Win32 with Ruby.

UiAutomation seemed to solve all my problems right up until I tried to bring an AutomationElement to the foreground with a call to AutomationElement.SetFocus().  It would randomly fail.  About 90% of the time the window in question would pop up and the test would pass with flying colors.  But every once in a while the test would fail.  Some quick debugging lead me to question the reliability of SetFocus().

Rather than monkey around with SetFocus(), Full Trust, and the .NET Security Framework I decide to rely on good old Win32.  For everyone who's being driven crazy by SetFocus(), all three of us, here is my win32 based SetForegroundWindow() method.

public bool SetForegroundWindow (AutomationElement elm, uint retries)
{
    Util.ValidateNotNull ("elm", elm);

    try
    {
        if (retries < RETRY_LIMIT)
        {
            // Using Win32 to set foreground window because
            // AutomationElement.SetFocus() is unreliable

            // Get handle to the element
            IntPtr other = FindWindow (null, elm.Current.Name);

            // Get the Process ID for the element we are trying to
            // set as the foreground element
            int other_id = GetWindowThreadProcessId (
                other, IntPtr.Zero);

            // Get the Process ID for the current process
            int this_id = GetWindowThreadProcessId (
                Process.GetCurrentProcess ().Handle, IntPtr.Zero);

            // Attach the current process's input to that of the 
            // given element. We have to do this otherwise the
            // WM_SETFOCUS message will be ignored by the element.
            bool success = 
                AttachThreadInput (this_id, other_id, true);

            // Make the Win32 call
            IntPtr previous = SetForegroundWindow (other);

            if (IntPtr.Zero.Equals(previous))
            {
                // Trigger re-try
                throw new Exception(
                    "SetForegroundWindow failed");
            }
            else
            {
                Log ("  focus set");
            }

            return true;
        }

        // Exceeded retry limit, failed!
        return false;
    }
    catch
    {
        retries++;
        uint time = retries * RETRY_FACTOR;
        Log ("  Could not SetFocus(), retry in {0} ms", time);

        Thread.Sleep ((int)time);

        return SetForegroundWindow (elm, retries);
    }
}

This method relies on Win32 calls, so you must import the requisite Win32 functions.

using System.Runtime.InteropServices;

...

[DllImport ("user32.dll", CharSet = CharSet.Auto)]
static extern IntPtr FindWindow (
  string lpClassName, string lpWindowName);

[DllImport ("user32.dll", CharSet = CharSet.Auto)]
static extern bool AttachThreadInput (
    int idAttach, int idAttachTo, bool fAttach);

[DllImport ("user32.dll", CharSet = CharSet.Auto)]
static extern int GetWindowThreadProcessId (
    IntPtr hWnd, IntPtr lpdwProcessId);

[DllImport ("user32.dll", CharSet = CharSet.Auto)]
static extern IntPtr SetForegroundWindow (IntPtr hWnd);