Audits Overview

This guide shows you how to manage Audits in Kiuwan Life Cycle.

What is a Kiuwan Audit?

The main purpose of an Audit is to evaluate if the results of an application delivery analysis satisfy a pre-defined set of conditions (checkpoints). Based on the results of that evaluation, the Audit will pass or fail.

In Kiuwan, you can pre-define as many Audits as you want as a set of checkpoints that are evaluated when the Audit is applied. These pre-defined Audits will be available in your account to be assigned to applications. The specific Audit assigned to an application is automatically applied to the results of any delivery analysis of that application.

For example, we have an application with a baseline analysis describing the actual state of the application (current defects and indicator level). And we want to define a corporate policy stating that any delivery (total or partial) of that application must not contain any new defect. In such a case, the delivery should not be accepted.

In this case, we can define an Audit to check that any delivery does not contain any new defects. In the case of a new defect, the Audit will FAIL, otherwise, it will be OK. This case is exactly what Kiuwan's Default Audit does, and the delivery will be marked as OK or FAIL depending on the analysis results.

Similarly, we might define any other policies. Some examples might be:

• Enforce security: there should not be any defect in Security rules (or very high priority security rules)
• Enforce maintainability and efficiency: there should not be any new defects related to high and very high maintainability and efficiency rules

All these Audits (and any others you might consider) can be defined in Kiuwan (without any programming) and will be applied automatically to every delivery analysis.

Kiuwan not only marks a delivery as OK or FAIL, but it also specifies:

• WHY (specific reasons for Audit failure),
• HOW to remediate it (i.e., the specific actions to be done to pass the Audit), and even
• HOW MUCH effort will it take to pass the Audit

For every delivery analysis, Kiuwan provides a full Audit Report with all this useful information.

Audit Checkpoints

Kiuwan Audits are based on Checkpoints.

A checkpoint is a specific (atomic) condition to be met by the analysis. An Audit may contain as many checkpoints as validations you want to check.

Every checkpoint has two possible results:

• OK (condition is met)
• FAIL (condition is not met)

Furthermore, depending on its level of compliance, a checkpoint can be classified as:

• Mandatory: something the delivery  must meet to accept it or deploy in production
• Optional: something to check for, but not essential for accepting the delivery

An Audit will FAIL if any of its mandatory checkpoints fail. 

Further reading: Please see Checkpoint Management for details on how to create and manage them.

Checkpoint types

Kiuwan provides a library of checkpoint types you can use to define your specific checkpoints when creating an Audit.

Code Security and Code Analysis checkpoints

When defining your checkpoints, you will be able to define thresholds for:

• The maximum number of Total defects:
  • The total number of defects in the delivery must not be higher than the defined threshold.
• The maximum number of New defects:
  • The number of New defects (i.e., defects introduced by the delivery that don't exist in the baseline) must not be higher than the defined threshold.
• Some other metrics such as:
  • Global Indicator improvement, Global Indicator defined threshold, Duplicated Code threshold, Percentage of Very High defects, etc.

In addition, Kiuwan not only allows you to define the number of defects and Insights components, but it also allows you to define the nature or type of those defects.

When selecting the nature or types of defects considered in a checkpoint, you can specify the following criteria:

• By nature: Languages, Characteristics, and/or Priorities of found defects
  • For example, defects of high priority in Security and Maintainability in Java and COBOL

• By type: Defects of specific types
  • For example, defects found by a specific set of rules

Insights components checkpoints

When you define your checkpoints, you can also define thresholds for:

• The maximum number of total Insights components that have a defined security risk(s):
  • The total number of components in the delivery that meet the defined security risks must not be higher than the defined threshold.

You can also check if the discovered components meet specific criteria based on:

• Group: the group of the component to be found (e.g., com.fasterxml.jackson.core).
• Name: the name of the component to be found (e.g., jackson-databind).
• Version: the version number of the component to be found (e.g., 2.8.10).
• Comparator: the comparison operator to be used when finding components (allowed operators are: >, >=, =, <=, <).

Other Audit and Checkpoints parameters

Before explaining the logic applied during Audit evaluation, we need to define a couple of concepts and parameters you can control in the definition of Kiuwan Checkpoints and Audits.

Checkpoint Weight

Every checkpoint has an associated Weight that represents the relative weight of the checkpoint in the Audit.

The weights you specify translate (automatically) into a percentage contribution to the overall Audit.

For example, if your Audit has 2 checkpoints of equal importance, you should set this value to 1 for both, translating into a 50% contribution for each checkpoint.

Now, if you consider that one is 2 times more important than the other, you should set them as 2 and 1, respectively, translating into a 66% and a 33% contribution.

Audit Approval Threshold

In a Kiuwan Audit, you can specify an Approval Threshold.

This threshold will represent the minimum percentage of checkpoint contribution to consider the Audit as OK. Independent of whether they are mandatory or not, only the contribution percentage of each checkpoint is taken into account to evaluate this threshold.

Learn how the Audit evaluation logic works in the next section.
 

Audit evaluation logic

Note: Any checkpoint related to a baseline analysis, when such baseline does not exist, will automatically be OK with 100% of its contribution to the audit.

Examples of checkpoints related to a baseline analysis:

• New defects
• Global Indicator improvement