Start speaking your own language (GUI automation III)

Introduction

At this point we KNOW HOW TO AUTOMATE and we know HOW TO AUTOMATE TESTING. Still although the way the tests are created seems to be very flexible, their content remains unclear especially for non developers. They look more like a code and they may require extra documentation to clear them out. What should we do to make it easy to understand for every party in the project, and speed up writing the tests at the same time? The advantage is at this point we have a custom framework so we can improve it any way we wish. Let’s move towards behaviour driven development then.

Behaviour driven development

You can read more about it HERE. In general it means focusing on behaviour of the system by describing it with help of domain specific language (dsl) that is a language which is on one hand easy to understand for everyone because of its similiarity to English and on the other hand is tightly related to the application itself. The main advantages in practice are: improved overall communication (we don’t have to ask what business analyst had on his mind, nor what is the specific test doing), automation speed up (the behaviour described in domain specific language can be directly executable, which means just after it is created it can be run by QA engineer or tester), documentation improvement (we do not have to create aditional documentation as the described behaviour of the system becomes documentation itself).
As you can read in Wikipedia the popular template for the dsl are GIVEN, WHEN, THEN words. Thinking about it from a perspective of automation engineer one can say GIVEN may be considered as a setup, WHEN is a body of a test and THEN is a assertion. Let’s use it in practice!

Notepad specific language

For interacting with Notepad we need specific language. There is no such a language yet so let’s create Notepad Specific Language – NSL.
We need words which are related to Notepad GUI entities like document, tab, bookmark and actions like open, write, close. Let’s look at the example below:

import com.passfailerror.notepadplusplus.DSL.DSLTestCase;
import com.passfailerror.notepadplusplus.TestDriver;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

import java.io.IOException;
import java.util.logging.LogManager;
import java.util.logging.Logger;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.CoreMatchers.*;

public class NotepadTestCases extends DSLTestCase{

    static TestDriver testDriver;


    @BeforeClass
    public static void commonSetUp(){
        START_APPLICATION();
    }

    @AfterClass
    public static void commonTearDown(){
        CLOSE_APPLICATION();
    }

    @Test
    public void testTyping(){
        String text = "Sikuli can automate every kind of GUI.";

        try {

            GIVEN().tab().isOpened().
            WHEN().tab().isWritten(text).
                    AND().tab().contentIsCopied().
            THEN().clipboardContent().shouldBe(equalTo(text));

        }
       finally {

            TEARDOWN().tab().isClosed();

        }
    }


}

So if we follow the pattern from Wikipedia, we can say:

Story: Multiple documents

IN ORDER to work with multiple documents
AS a user
I WANT TO have separate tabs available

Scenario 1: Tab content may be copied to clipboard
 GIVEN().tab().isOpened().
            WHEN().tab().isWritten("Sikuli can automate every kind of GUI.").
                    AND().tab().contentIsCopied().
            THEN().clipboardContent().shouldBe(equalTo(text));

Test report can also speak NSL

After the test is run we need to review results. Often happens we need to know what actions were taken at specific point in time. Let’s look at the test ouput we get when appropriate DSL logging is present.

[INFO ] 2015-11-13 23:58:44.078 GIVEN
[INFO ] 2015-11-13 23:58:44.081 tab
[INFO ] 2015-11-13 23:58:44.085 is opened
[INFO ] 2015-11-13 23:58:50.725 WHEN
[INFO ] 2015-11-13 23:58:50.725 tab
[INFO ] 2015-11-13 23:58:50.726 is written: "Sikuli can automate every kind of GUI."
[INFO ] 2015-11-13 23:58:53.690 AND
[INFO ] 2015-11-13 23:58:53.690 tab
[INFO ] 2015-11-13 23:58:53.691 content is copied
[INFO ] 2015-11-13 23:58:53.822 THEN
[INFO ] 2015-11-13 23:58:53.824 clipboard content
[INFO ] 2015-11-13 23:58:53.841 should be "Sikuli can automate every kind of GUI."
[INFO ] 2015-11-13 23:58:53.881 assertion, actual: ["Sikuli can automate every kind of GUI."] expected: ["Sikuli can automate every kind of GUI."]
[INFO ] 2015-11-13 23:58:53.883 TEARDOWN
[INFO ] 2015-11-13 23:58:53.883 tab
[INFO ] 2015-11-13 23:58:53.883 is closed
[INFO ] 2015-11-13 23:58:55.058 window
[INFO ] 2015-11-13 23:58:55.059 is closed

As we can see not only the testcase itself becomes very clear but also the logging done in NSL level when analyzed with application logs allows mapping each action done on system test level to the sequence of events which take place on single service and class levels (component level) at the given point in time. Of course this is easy now to do the reverse thing and tweak logging a little bit to allow copy paste of log output to allow instant replay of the logged actions. So if we just change the logging messages a bit so that all of them are Java methods, we will be able to copy paste them and execute instantly. Consider this:

[INFO ] 2015-11-13 23:58:44.078 GIVEN().
[INFO ] 2015-11-13 23:58:44.081 tab().
[INFO ] 2015-11-13 23:58:44.085 isOpened().
[INFO ] 2015-11-13 23:58:50.725 WHEN().
[INFO ] 2015-11-13 23:58:50.725 tab().
[INFO ] 2015-11-13 23:58:50.726 isWritten("Sikuli can automate every kind of GUI.").
[INFO ] 2015-11-13 23:58:53.690 AND().
[INFO ] 2015-11-13 23:58:53.690 tab().
[INFO ] 2015-11-13 23:58:53.691 contentIsCopied().
[INFO ] 2015-11-13 23:58:53.822 THEN().
[INFO ] 2015-11-13 23:58:53.824 clipboardContent().
[INFO ] 2015-11-13 23:58:53.841 shouldBe(equalTo("Sikuli can automate every kind of GUI."))
[INFO ] 2015-11-13 23:58:53.881 assertion, actual: ["Sikuli can automate every kind of GUI."] expected: ["Sikuli can automate every kind of GUI."]
[INFO ] 2015-11-13 23:58:53.883 TEARDOWN().
[INFO ] 2015-11-13 23:58:53.883 tab().
[INFO ] 2015-11-13 23:58:53.883 isClosed().
[INFO ] 2015-11-13 23:58:55.058 window().
[INFO ] 2015-11-13 23:58:55.059 isClosed().

More about test report

This is nice thing we can see the test run log in NSL, but this is even better to have the screenshots. This is good test proof and very good way to see what went wrong in case test produce error instead of pass or fail, especially when it is hard to tell from application logs what is the reason. We are using custom framework so we can have screenshots anyway we like. Let’s do it then.

Summary

There are new nice things I presented you here. BDD aproach constitutes efficient solution for quality control. However, we can do more, we can do better. As usual… I will cover this in my next part about GUI automation.