A SDET approach for implementing SonarQube

Quality Engineering is a big topic that is constantly evolving. The scope and methods of implementing quality engineering practices and tools will keep changing and evolving as technology advances. However, we should not lose focus on the purpose of implementing quality engineering as a part of the software development processes. We must avoid getting lost trying to implement some of the latest trends that may not add any real value to our process. 

In today’s Agile development world, it is necessary to include automation engineers as a part of the software development team, from the early sprints. It is the responsibility of these engineers to not just test the product when ready and ensure quality of the software, but also understand the product as it is being built, collaborating to ideate and analyze potential gaps, as early bug detection is always desired in any project. As development cycles advance, we need to ensure that our automated testing coverage increases so we’re not left behind. Every sprint must have new test automations that are added on to the previous ones, and the latest code changes must be updated as well.

So, what are the usual asks from a Software Development Engineer in a Test (SDET) kind of role?

It is not just testing or a test automation, but someone that can bring a more integrated vision to ensure quality in development. Beyond just expertise with tools, someone who knows when and how to implement them the right way, and ensure all software requirements are covered by asking the right questions at the right time.
It is not a simple task, and turns all the more challenging when we need to replicate this across clients or organizations. The need to ensure that quality services are consistently delivered, all guidelines and best practices are being met, ensuring success based on both processes, having the right person can make sure all the processes fall in place. Failing to ensure consistency in success and processes could affect the performance between different areas of an organization, resulting in the inability to meet expectations.

In order to be able to accomplish this, SDETs could rely on tools that help them ensure all codes meet the defined guidelines and best practices. For the purpose of this article, we are going to be focusing on a tool called SonarQube.

SonarQube is a tool that executes static code analysis and supports most common languages like Java, Javascript, Python and C#. SonarQube comes with a set of rules and best practices defined for each of those languages. It is a great tool, as it provides additional metrics that could help identify duplicated code, potential issues, code flaws, and even some security breaches.

By abstracting some of our development standards and guidelines into SonarQube rules, we can take SonarQube to the next level, and use it to analyze our development or testing code. This could help ensure that our codes meet all the minimum requirements. 

In order to implement our own rules, we need to download the custom rules project template from the SonarQube site. This will be the starting point for creating the custom rules library that we would have to include in our SonarQube installation as an extension library.

Here’s how you can include custom rules in SonarQube analysis:

1. First, you will need to have a clear definition as to what your rule is intended to do. This basically means that you need to abstract the desired checks or validations into some logical instructions, based on how SonarQube analyzes the code.
In SonarQube, we define the kind of objects that should be analyzed, and then define what the steps for the analysis will be. For example, if you want to ensure that all methods from a project are public, you could define that SonarQube only looks for elements that are ‘methods’, and for all elements found, you would look for the access identifiers associated with those methods and validate that all of them have been defined as public.
In our current example, we want to ensure that our test methods include an attribute called testName defined in TestNG framework.

2. Get the project template mentioned first, then create files associated with your rule. A good practice when creating rules is to have 3 files for every rule you create:

• A file containing the actual rule implementation.

package org.sonar.samples.java.checks;

import java.util.List;
import org.sonar.check.Priority;
import org.sonar.check.Rule;
import org.sonar.plugins.java.api.IssuableSubscriptionVisitor;
import org.sonar.plugins.java.api.tree.AnnotationTree;
import org.sonar.plugins.java.api.tree.ExpressionTree;
import org.sonar.plugins.java.api.tree.MethodTree;
import org.sonar.plugins.java.api.tree.Tree;
import org.sonar.plugins.java.api.tree.Tree.Kind;
import com.google.common.collect.ImmutableList;

@Rule(
	key = "IncludeTestNameAttribute",
	name = "Test Name attribute specified in every test method",
	description = "Attribute testName of Test annotation should be specified in order to appear properly in execution report",
	priority = Priority.MAJOR,
	tags = {"convention"})
public class IncludeTestNameAttribute extends IssuableSubscriptionVisitor {	

	private static final String TEST = "Test";

	private static final String TEST_NAME = "testName";	@Override
	
	public List nodesToVisit() {
		return ImmutableList.of(Kind.METHOD);
	}	@Override
	
	public void visitNode(Tree tree) {		
		MethodTree methodTree = (MethodTree) tree;		
		for (AnnotationTree annotationTree : methodTree.modifiers().annotations()) {
			if (annotationTree.annotationType().firstToken().text().equals(TEST)) {
				boolean blankTestName = true;
				for (ExpressionTree attributes : annotationTree.arguments()) {
					String attribute = attributes.firstToken().text();
					if (attribute.equals(TEST_NAME)) {
						String attributeValue = attributes.lastToken().text();
						if (attributeValue.length() > 2) {
							blankTestName = false;
						}
					}
				}
				if (blankTestName) {
					reportIssue(annotationTree, "Test method doesn't have a test name to be used in final reports");
				}
			}
		}
	}
}

• An example test file that will be used for analysis in our unit test. If this file covers different potential cases and combinations, we will have more certainty that our rule will behave properly in the real world.

import org.testng.annotations.Test;

class Tests {	

	@Test // Noncompliant
	public void testMethodWithoutAttributes() {
		privateStringMethod();
	}	

	public void randomVoidMethod() {	}	

	public String randomStringMethod() {
		return "";
	}	

	private String privateStringMethod() {
		return "";
	}	

	@Test(testName = "Test name to be included")
	public void testMethodWithTestName() {
		privateStringMethod();
	}	

	@Test(testName = "") // Noncompliant
	public void testMethodWithBlankTestName() {
		privateStringMethod();
	}
}

A unit test file that will refer to both, the example file and to the rule that we will use for analyzing it.

package org.sonar.samples.java.checks;import org.junit.Test;
import org.sonar.java.checks.verifier.JavaCheckVerifier;
public class IncludeTestNameAttributeTest {	
	@Test
	public void check() {
		// Verifies that the check will raise the adequate issues with the expected
		// message.
		// In the test file, lines which should raise an issue have been commented out
		// by using the following syntax: "// Noncompliant
		JavaCheckVerifier.verify("src/test/files/IncludeTestNameAttribute.java", new IncludeTestNameAttribute());
	}
}

3. Once we have these files in place, we need to include our rules to the central configuration. There is a file in our project called RulesList.java, where we need to include our rule to the list of rules to be added to the checks.

public static List<Class<? extends JavaCheck>> getJavaTestChecks() {
    return ImmutableList.<Class<? extends JavaCheck>>builder()
    		.add(IncludeTestNameAttribute.class)
    		.build();
  }
 

4. Once done, we need to compile the project in order to generate the jar file containing all the rules. If the unit test code fails, that could mean that our rule is not doing what it is intended to do, or that our test file is having wrong assertions.
If everything works fine, we can take the generated jar and include it to our SonarQube installation in the /extensions/plugin folder.
Restart the SonarQube instance in case it was running, else just get it started.

5. Login to SonarQube and go to Rules menu. You can search for the rule there. If everything went fine, the rule should appear here:

6. Now, in order to be able to include our rule in the SonarQube analysis, we need to define a quality profile. Go to Quality Profile menu and you can see the current ones. An easy and usually safe way to do this is to extend the default profile for the language (in this case the Java profile defined by Sonar) and include the additional rules that we added. Set this profile as the default profile.

7. Once the quality profile is defined, we should trigger a SonarQube analysis from the code we want to analyze. After the analysis is done, when we open it, we should verify that it was done using the quality profile we just configured. It would look like this:

8. Now, go to details by clicking on any of the values for Bugs or Code Smells. Here, we can see a summary of them. So, in case our analysis detected that some of the rules were not met by our code all those findings will appear here. The last result from next image belongs to a finding detected by the custom rule we just added:

9. Clicking it will point to the exact code portion that requires some work as shown below:

Here, we just covered the full cycle from defining our guidelines to actual analysis implementation. That’s the power of taking advantage of existing tools and applying your knowledge for specific domains. You can build great stuff that will keep your customers happy by adding value to their processes.