Audit JUnit Export

Kiuwan supports exporting your Audit summary into a JUnit XML report via the Kiuwan API. 

This report allows you to import your passing and failing audits and checkpoints into test case management tools, such as TestRail and Xray, or any other tool that supports a JUnit import. This functionality gives greater visibility for your test managers and other team members into the results of your Security scans.

What Does the JUnit Report Contain?

The JUnit report aligns with the Audit overview page for a particular delivery. The report contains the following information:

• Overall Audit result
• Results for each checkpoint in the audit (Pass or Fail)
  • Additional data for failures
  • No additional data for Passing checkpoints
• Links to the Audit in Kiuwan

As mentioned above, additional data will be included in the failure message for failed checkpoints and audits. No additional data is included for passing elements, and the report does not support other status types.

For failures:

• The overall Audit will include the number of passing and failing checkpoints, overall audit score, and threshold value.
• For all individual checkpoints except Insights filter components by group, name, and version, the failure message will contain data from the Mandatory, Value, and Threshold columns for failed checkpoints.
• For the Checkpoint type Insights filter components by group, name, and version, information about the applied filter will be included, such as the component and version the checkpoint is analyzing.

Additionally, each item in the report will have a property containing a link to the Kiuwan Audit report. See TestRail and Xray sections for additional information.

Example XML Report

Below is a sample of the JUnit output, containing the overall audit score, a failing checkpoint, and a passing checkpoint:

<?xml version='1.0' encoding='utf-8'?>
<testsuite name="JUnit Audit">
   <properties>
       <property name="Details" value="https://www.kiuwan.com/saas/web/dashboard/delivery?h=SF11QDAJDA13SFBP6D142UA35EHJJ78IDRX6PYMFVBTGVT" />
   </properties>
   <testcase classname="Overall Result" name="Audit Score">
       <properties>
           <property name="testrun_customfield:link" value="[Details in Kiuwan|https://www.kiuwan.com/saas/web/dashboard/delivery?h=SF11QDAJDA13SFBP6D142UA35EHJJ78IDRX6PYMFVBTGVT]"/>
       </properties>
       <failure message="Threshold not reached.">
           <![CDATA[
               Threshold: 100.0
               Value: 41
               Checkpoints: 2
               OK: 1
               Failed: 1
           ]]>
       </failure>
   </testcase>
   <testcase classname="Selected Very High Priority and Very Hard Effort Java Rules" name="Defects - Maximum defects for specific rules">
       <properties>
           <property name="testrun_customfield:link" value="[Details in Kiuwan|https://www.kiuwan.com/saas/web/dashboard/delivery?h=EXG50XDHIYTIWA8C8PSF11QDAJDA13SFBP6D142UA35EHJJ78IDRX6PYMFVBTGVT]"/>
       </properties>
       <failure message="Threshold not reached.">
       <![CDATA[
           Mandatory: No
           Threshold: 0
           Value: 307
       ]]>
       </failure>
   </testcase>
   <testcase classname="Threshold for maximum new defects of specific rules" name="Defects - Maximum NEW defects for specific rules">
       <properties>
           <property name="testrun_customfield:link" value="[Details in Kiuwan|https://www.kiuwan.com/saas/web/dashboard/delivery?h=SF11QDAJDA13SFBP6D142UA35EHJJ78IDRX6PYMFVBTGVT]"/>
       </properties>
   </testcase>
</testsuite>

Importing JUnit Results Into TestRail

TestRail supports importing JUnit reports using the TestRail CLI tool. When importing a JUnit report, TestRail will:

• Parse the file
• Determine if any test cases exist or need to be created
• Create a new test run based on the cases
• Add results for each testcase in the file
• Add the link contained in the testsuite properties element to the Test Run description.

TestRail will NOT process the links within each test case, as test case property elements are not supported.

Note: If importing a JUnit report into a multi-suite project, you must use the --suite argument with the TestRail CLI. The export from Kiuwan does not contain multiple suites, or a testsuites element.

Importing JUnit Results Into Xray

Xray supports importing JUnit reports using the Xray API. When importing a JUnit report, Xrayl will:

• Parse the file
• Determine if any test cases exist or need to be created
• Import Test Cases to a new (or user-specified) Test Execution in the context of a project
• Add execution results
• Add the link contained in each testcase’s property element to the corresponding custom field in the test case.

Xray will NOT process the link for the testsuite element, as test suite property elements are not supported