Software Testing

Retry Failed Tests using TestNG

Retry Failed Tests using TestNG

In this tutorial, you will learn to retry failed tests using TestNG framework.

There might be many reasons for a Test case to fail, may be due to element not found or time out exception or stale element exception etc.

Normally in automation after executing scripts/tests, you will check for the results and if the test fails, you will re-run them again. TestNG allows you to execute the failed test cases again for n number of times. To achieve this we need to implement TestNG IRetryAnalyzer.

TestNG IRetryAnalyzer

package com.testingdocs.test;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class TestsRetry implements IRetryAnalyzer {
    private int retryCount = 0;
    private int maxRetryCount = 1;

// Below method returns 'true' if the test method has to be retried else 'false' 
//and it takes the 'Result' as parameter of the test method that just ran
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            System.out.println("Retrying test " + result.getName() + 
" with status "
                    + getResultStatusName(result.getStatus()) + 
" for the " + (retryCount+1) + " time(s).");
            retryCount++;
            return true;
        }
        return false;
    }
    
    public String getResultStatusName(int status) {
    	String resultName = null;
    	if(status==1)
    		resultName = "SUCCESS";
    	if(status==2)
    		resultName = "FAILURE";
    	if(status==3)
    		resultName = "SKIP";
		return resultName;
    }
}

 

Create an other class ‘RetryListener‘ by implementing ‘IAnnotationTransformer‘. We need to setRetryAnalyzer for ITestAnnotation.

 

package com.testingdocs.test;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;

public class RetryListener implements IAnnotationTransformer {

	@Override
	public void transform(ITestAnnotation testannotation, Class testClass,
			Constructor testConstructor, Method testMethod)	{
		IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

		if (retry == null)	{
			testannotation.setRetryAnalyzer(TestsRetry.class);
		}

	}
}

 

Let us see the example by executing simple tests. Create a Test that fails. The test will be executed for the value that we defined for ‘maxRetryCount’ in Retry Class.

 

Sample Test

package com.testingdocs.test;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class ExampleTest {
	WebDriver driver;
	String baseURL = "https://www.linkedin.com/";

	@BeforeClass
	public void setup() {
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
	}

	@Test(priority=1)
	public void verifyLoginPageText() {
		driver.navigate().to(baseURL);
		System.out.println("Verify login page test started");
		WebElement element = driver.findElement(By.cssSelector(".header>h2"));
		String headerText = element.getText();
		Assert.assertEquals(headerText, "Get started – it’s free.");
	}

	@Test(priority=2)
	public void verifyForgotPasswordPage() {
		driver.navigate().to(baseURL);
		System.out.println("Verify Forgot password page test started");
		WebElement element = driver.findElement(By.linkText("Forgot 
your password?"));
		element.click();
		WebElement pageTextElement = driver.findElement(
By.cssSelector(".flow-login-content>fieldset>h1"));
		String pageText = pageTextElement.getText();
		Assert.assertEquals(pageText, "Bogus text");
	}
}

 

 

You need to add the Listener to testng.xml file. Below is syntax to add listener for the RetryListener.

<listeners>
        <listener class-name="com.testingdocs.test.RetryListener"/>
  </listeners>

 

The test will be executed for two times as we have defined ‘maxRetryCount = 1’, hence when the test fails, it will excute again for one time.