This Manager is at Release level. You can view the Javadoc documentation for the Manager here.

Overview
Configuring
Provided annotation
Code snippets and examples

Overview

If there is an open issue in a GitHub repository that tracks a known problem which is expected to cause a test class or test method to fail, you can add the @GitHubIssue annotation to a test class or test method so that the expected failure is recorded as Failed with Defects  in the test report.

Using this annotation on a test class or test method means that the expected failure is clearly identifiable in the test report, making it easier to spot any unexpected test failures and saving time by avoiding the investigation of a known issue. 

Including the Manager in a test

To use the GitHub Manager in a test you must import the @GitHubIssue annotation into the test, as shown in the following example:

import dev.galasa.githubissue.GitHubIssue;
...

You also need to add the Manager dependency into the pom.xml file if you are using Maven, or into the build.gradle file if you are using Gradle.

If you are using Maven, add the following dependencies into the pom.xml in the dependencies section:

<dependency>
<groupId>dev.galasa</groupId>
<artifactId>dev.galasa.githubissue.manager</artifactId>
</dependency>

If you are using Gradle, add the following dependencies into build.gradle in the dependencies closure:

dependencies {
compileOnly 'dev.galasa:dev.galasa.githubissue.manager'
}

Finally, use the annotation that is provided by the Manager in the test. The annotation calls the Manager at runtime. In this example, the provided annotation is @GitHubIssue.

Using the Manager

The GitHub Manager is activated at runtime if there is a @GitHubIssue annotation on the test class or any of its test methods. The presence of the @GitHubIssue annotation means that there is a known problem which is expected to affect the result of the test.

If the issue is expected to affect one or more specific test methods, add the annotation on the test method. If the scope of the issue is larger and might affect the whole test class, add the annotation at the class level.

Use the following examples, which are based on BatchAccountsOpenTest to understand how to add the @GitHubIssue annotation for GitHub issue number 1178 to a test method and to a test class.

Examples

GitHub issue 1178 has a status of open on GitHub in the galasa-dev/projectmanagement repository. Issue 1178 tracks a known problem with the behaviour of the Z/OS Batch Manager. The issue causes the test class BatchAccountsOpenTest to fail.

The following example adds the @GitHubIssue annotation on the test method:

@Test
@GitHubIssue(issue = "1178", repo = "galasa-dev/projectmanagement")
public void batchOpenAccountsTest() throws TestBundleResourceException, IOException, ZosBatchException {
    	// Create a list of accounts to create
    	List<String> accountList = new LinkedList<>();
    	accountList.add("901000001,20-40-60,1000");
...

The following example adds the @GitHubIssue annotation on the test class:

@Test
@GitHubIssue(issue = "1178", repo = "galasa-dev/projectmanagement")
public class BatchAccountsOpenTest {
...

Once the GitHub issue is closed, remove the @GitHubIssue annotation from the test class or test method.

Using regex

To ensure that the failing exception is due to the known problem that is identified in the GitHub issue, use regex in the annotation to check that the recorded test failure matches the string that is specified in regex.

For example, GitHub issue 1179 describes a problem in source code that results in a HttpServerErrorException message being returned when a test is run. You can add the following annotation to your test to ensure that it is failing because of this exception, and not for a different reason:

@GitHubIssue(issue = "1179", repo = "galasa-dev/projectmanagement",
regex = "HttpServerErrorException")

If the failing exception does not match the regex, the test has failed for a different or unknown reason. If no regex is provided in the annotation, any failing exception is accepted.

Setting overrides

You can override the result of a test method or a test class by using the @GitHubIssue annotation in conjunction with the @ContinueOnTestFailure annotation to return a test result for all test methods within a test class.

Overriding a test method result:

If a test method is annotated with @GitHubIssue, and the initial result of the method is Failed, the GitHub Manager uses the GitHub API to retrieve the current state of the GitHub issue from the annotation. If the issue is in open status, the result of the test method is overridden from Failed to Failed With Defects. If the issue is in closed status, the result is not overridden, as that issue is supposedly fixed and is not expected to affect the test method anymore. In the latter scenario, further investigation into the cause of the failure is required.

Overriding a test class result:

At the end of a test class run, the results of each individual test method are combined to determine whether the result of the test class should be overridden.

Use the following scenarios and examples to understand the expected test class result and what to do next. In each scenario, the issue is in an open state.

Scenario 1: 

Scenario 1 uses the @GitHubIssue annotation on the test method testNotNull with the @ContinueOnTestFailure annotation:

@Test
@ContinueOnTestFailure
Test class SimBankIVT
@GitHubIssue( issue = "1178")
Test method testNotNull() result: Failed with defects
Test method checkBankIsAvailable() result: Failed

Test class result: Failed

Next steps:
Test method testNotNull returned a result of Failed With Defects. Test method checkBankIsAvailable returned a result of Failed for an unknown reason. Further investigation is required to understand why test method checkBankIsAvailable failed.

Scenario 2: 

Scenario 2 uses the @GitHubIssue annotation on the test method testNotNull with the @ContinueOnTestFailure annotation:

@Test
@ContinueOnTestFailure
Test class SimBankIVT 
@GitHubIssue( issue = "1178")
Test method testNotNull() result: Failed with defects
Test method checkBankIsAvailable() result: Passed

Test class result: Passed with defects

Next steps:
The first method, testNotNull, returned a result of Failed With Defects. As the ContinueOnTestFailure annotation is used, the test continued to run and the following method checkBankIsAvailable returned a result of Passed. All methods that are expected to pass returned a result of Passed, so the overall result of the test is Passed With Defects. No further investigation is required.

Scenario 3: 

Scenario 3 uses the @GitHubIssue annotation on the test method checkBankIsAvailable without the @ContinueOnTestFailure annotation:

@Test
Test class SimBankIVT
Test method testNotNull() result: Passed 
@GitHubIssue( issue = "1178")
Test method checkBankIsAvailable() result: Failed with defects

Test class result: Passed with defects  

Next steps:
The method checkBankIsAvailable returned a result of Failed With Defects. As the test did not use the ContinueOnTestFailure annotation, the test would have ended prematurely. However, as the checkBankIsAvailable method was the last test method, the results of all test methods in the class are known. All methods that are expected to pass returned a result of Passed, so the overall result of the test is Passed With Defects. No further investigation required.

Scenario 4:

Scenario 4 uses the @GitHubIssue annotation on the test method testNotNull without the @ContinueOnTestFailure annotation:

@Test
Test class SimBankIVT
@GitHubIssue( issue = "1178")
Test method testNotNull() result: Failed with defects
Test method checkBankIsAvailable() result: Unknown

Test class result: Failed with defects

Next steps:
The testNotNull method returned a status of Failed With Defects. As the ContinueOnTestFailure annotation is not used, the results of the following methods - in this example checkBankIsAvailable are not known. The overall result returned is Failed With Defects. To check the result of all methods, add the @ContinueOnTestFailure annotation for future test runs.

Configuring

You must configure your CPS properties so that the GitHub Manager knows the GitHub instance in which to look for the specified issue. The GitHub Manager converts the GitHub instance URL that is provided in the CPS to the correct URL for the GitHub API, removing the need to manually supply a URL for the GitHub API. Alternatively, you can set the GitHub repository in the @GitHubIssue annotation. The value in the annotation takes precedence over the value in the CPS.

Configuration Properties

The following are properties used to configure the GitHub Manager:

githubissue.instance.url=https://github.com

secure.credentials.GITHUB.username=COMPANYUSER
secure.credentials.GITHUB.password=Password123

Annotations provided by the Manager

The following annotation is provided by the GitHub Manager:

Code snippets and examples

@Test
@GitHubIssue( issue = "1178", repo = "galasa-dev/projectmanagement" )
@ContinueOnTestFailure
public class ProvisionedAccountCreditTest {
...

@Test
@GitHubIssue( issue = "1178", repo = "galasa-dev/projectmanagement" )
public void updateAccountWebServiceTest() throws Exception {
    // Obtain the initial balance
    BigDecimal userBalance = account.getBalance();
...