ISO and COSO haven’t got a clue. You can and should quantify compliance risks

Every organisation is required to comply with laws within the countries it operates in, the legal and regulatory requirements vary between different regions adding to the need to have understanding and confidence in the risk management processes in place. Organisations face considerable uncertainty when making decisions and taking actions that may have significant compliance consequences. The management of compliance risks helps organisations protect and increase its value.

This series of publications will provide guidance on the activities to be undertaken to support decision makers to assess and treat compliance risks efficiently and cost effectively to meet the expectations of a wide range of stakeholders. Failure to meet legal requirements and stakeholder expectations can have considerable and immediate negative consequences that could affect performance, reputation and might lead to criminal prosecution of top management.

Compliance risk within this series of publications is broadly defined and is not limited to, for example, risk related to compliance or contractual matters, including risks from or to third parties where there may be no contractual relationship but where there may be a possibility of litigation or other action depending on that third parties’ contractual requirements with their stakeholders.

This methodology is developed in line with the requirements of ISO 31022:2020 Risk management — Guidelines for the management of legal risk and Compliance Risk Management: Applying the COSO ERM Framework. Just kidding, it’s light years ahead of the nonsense written in the Compliance Risk Management: Applying the COSO ERM Framework. See my page by page review to understand why you should never apply COSO to compliance risks. 

For the purposes of this article, compliance risk management includes:

  • Timely identification and recording of compliance risks
  • Risk assessment and prioritization of compliance risk for further analysis
  • Detailed risk analysis for most significant compliance risks and identification of suitable risk mitigation measures
  • Monitoring and reporting.

Risk identification

The purpose of identifying compliance risks is to find, recognize and describe the risks that can help or prevent an organization to achieve or from achieving its objectives.

To have a comprehensive understanding of compliance risks, organisations may do the following:

  • Review relevant laws and regulations across all of the countries of operation.
  • Review claims and incident statistics captured across the organization.
  • Review claims against industry peers and other relevant organizations in the countries of operation.
  • Consult with relevant legal and compliance advisors and service providers.
  • Review information and guidelines from regulators and government authorities.

Identified compliance risks have to be mapped against the legal entities to make sure no significant risks are missed:

Licensed activities and subsoil use Environmental management (ecology) Sanctions compliance Anti-monopoly compliance Tax compliance Fire supervision, emergency protection Labor and industrial safety Covenant compliance Economic and information security, state secret Land and property relations Construction and reconstruction of hazardous facilities Physical security of production facilities and vehicles
Group of companies
Legal entity 1 X X X X
Legal entity 2 X X X X X X X X
Legal entity 3 X X X X X

Compliance risks can be documented in a manual or online risk register for further analysis.

Risk Analysis

Wherever possible companies should apply quantitative risk analysis to measure and prioritize compliance risks. Wait what? We can do better than a compliance heatmap? Apparently :))

The following information should be collected and recorded for each identified risk:

  • Possible consequence scenarios as described in the legislation or other regulatory requirements (usually includes fines, 3rd party claims, criminal prosecution, temporary production closure, sanctions and so on)
  • Range of possible values for each of the consequence scenario (for example, according to the legislation fines may vary from 100K to 1M, production closure can be for a period between 0 and 90 days, etc.)
  • The logical relationship between each consequence scenario (for example, large fines are much more likely once the small fines have been already received or for some risks it could be the opposite, if small fines haven’t been issued over the last 2+ years this could mean that the large fine is imminent and so on)
  • Historical incident and claims data, known court cases or other relevant information.
  • Risk owner and key stakeholders.
  • Current controls and assessment of their effectiveness, if available.

Step 1. Represent each risk as a bow-tie diagram

Each risk can be graphically represented as a bow-tie diagram. A bow tie is a graphical depiction of pathways from the causes of an event or risk to its consequences in a simple cause-consequence diagram. It is a simplified combination of a fault tree that analyses the cause of an event or risk, the left hand side of the diagram, and an event tree that analyses the consequences, the right hand side. I borrowed some diagrams and generic words from a wonderful article by Broadleaf https://broadleaf.com.au/resource-material/bow-tie-analysis/

The focus of bow tie analysis is on the barriers or controls depicted to the left-hand side of the knot that can change the likelihood of the event or circumstance, or on those on the right-hand side that can change its consequences. It is used when assessing the completeness of controls, to check that each pathway from cause to event and event to consequence has effective controls, and that factors that could cause controls to fail (including management systems failures) are recognized:

  • The most effective controls usually address causes, generally to stop them arising or leading to the risk (preventive controls). They should match the causes, in extent and nature.
  • On the right of the bow tie, controls should provide appropriate responses to consequences being felt or create barriers to the consequences developing. They might either influence the consequences on business objectives directly (corrective or reactive controls), or detect changes quickly and provide triggers for contingency plans (detective controls).

Actuarial science

Any compliance risk can be depicted as a bow-tie diagram by following these steps:

  1. Select the risk to be examined in the bow tie analysis.
  2. Describe the risk, in the form [something happens] and leads to [a consequence for our objectives], and note the main risk analysis outcomes from the risk register.
  3. List the causes of the risk on the left and the consequences of the risk on the right, using the information from the regulations as well as through consultation with risk owners and subject matter experts.
  4. List the existing controls on the causes (preventive controls) below the causes on the left, and the controls on the consequences (corrective controls) below the consequences on the right. If a control acts on both causes and consequences, then show it twice, on each side of the template.
  5. Identify options for enhancing existing controls, to improve their effectiveness or to fill gaps. This may include enhanced monitoring and more frequent review, for example using control self-assessment.

Step 2. Identify causes and consequence scenarios

Causes and consequences for the bow-tie diagram are normally derived from the regulations as well as through consultation with risk owners and subject matter experts.

Common consequence scenarios for compliance risks (just a quick example, there is more) include:

Risk area Examples of consequence scenarios
Licensed activities and subsoil use
  • The need to re-obtain a license
  • Redemption of rights from other owners of the object
  • Fines for operating without the license
Environmental management (ecology)
  • Administrative fines
  • 3rd party claims
  • Production halt or stop
  • Criminal prosecution or management disqualification
Sanctions compliance
  • Fines as a proportion of revenue
  • Restrictions on existing or potential markets
  • Restrictions on capital markets and ability to refinance existing loans
  • Restrictions on the use of foreign technology or equipment
  • Losing control over overseas assets
Anti-monopoly compliance
  • Fines up to 2% of revenue
Tax compliance
  • Administrative fines
  • Additional taxes to be paid
Fire supervision, emergency protection
  • Administrative fines
  • 3rd party claims
  • Production halt or stop
  • Criminal prosecution or management disqualification
Labor and industrial safety
  • Administrative fines
  • 3rd party claims
  • Production halt or stop
  • Criminal prosecution or management disqualification
Covenant compliance
  • Repayment of existing loans
  • Increase in financing costs
  • Difficulty in refinancing
Economic and information security, state secret
  • Administrative fines
  • Criminal prosecution or management disqualification
Land and property relations
  • Administrative fines
  • 3rd party claims
  • Production halt or stop
  • Criminal prosecution or management disqualification
Construction and reconstruction of hazardous facilities
  • Administrative fines
  • 3rd party claims
  • Production halt or stop
  • Criminal prosecution or management disqualification
Physical security of production facilities and vehicles
  • Administrative fines
  • Criminal prosecution or management disqualification

An example for a bow-tie for a typical compliance risk is presented below:

Academic disciplines

Where, V – means several events can occur at the same time, and XOR means the variability of either one event or the other. For example, fines can be either for three days of water pollution (small), or for a year (moderate) or three years (large), and criminal prosecution and termination of business can occur simultaneously.

Step 3. Determine the range of consequences for each scenario

In order to quantitatively assess compliance risks the next step involves defining the possible range of values for each consequence scenario. Typical consequences can involve the following factors:

Consequence scenario Range of consequences
A.     Small fine for violation, for example a fine for three days of water pollution
  • Analysis of legislation in terms of violation of the quality of the spillway
  • Analysis of the structure of the drainage system of the entity
  • Analysis of the volatility of the discharge indicators for supervisory and internal inspections of water quality.
  • Statistics of court decisions (sanctions) in similar cases

 

B.     Moderate fine calculated cumulatively for the year using extrapolation of supervisory audit results
C.    Large fine calculated cumulatively for the three years using extrapolation of supervisory audit results
D.    Suspension of business
  • Statistics of business suspensions adjusted for our company
  • Calculate the cost of a plant’s downtime per day multiplied by the range of days
E.     Criminal prosecution of company management
  • Expert legal assessment of the cost of legal defense and possible reputational losses due to criminal prosecution or disqualification of management.

Depending on the availability and reliability of the data various severity distributions can be used (only examples, relax, could be others):

  • Lognormal distribution – where the range of consequences is not bounded and there is a small probability of catastrophic losses.
  • PERT distribution – for simulating consequences based on expert opinions where historical data may not be available or the range of consequences is bounded by regulation.
  • Discrete distribution – for simulating a select number of well defined scenarios.
  • Fitted distributions – wherever historical data is available it can be used to fit a distribution suitable for the specific loss profile.

For each consequence scenario a distribution is selected and the range of possible values are determined, for example minimum, expected loss and maximum loss.

Academic disciplines

Step 4. Allocate weights to each scenario

In order to determine the weight allocated to each consequence scenario of events triggered by compliance risk, historical data, modelling, as well as expert opinions, can all be used, individually or in combination.

Weight of each scenario can involve the following factors:

  • the range of laws, along with enforcement practices and conventions by the relevant regulatory authorities;
  • the improvement of, and compliance with, the existing framework for the management of legal risk, including strategies, governance, internal rules and policies;
  • employees’ and contractors’ demonstrated compliance with laws, and the rules and policies of the organization;
  • the frequency and number of activities related to legal risk occurring within a certain period;
  • failure to record, analyse and learn from previous events;
  • benchmarking the frequency and number of activities related to legal risk occurring within a certain period against other organizations.

Wherever possible historical data on each of the consequence scenarios is collected. When no historical data is available or no claims have been made against the company in the past, we use Bayesian statistics to estimate the weights for the scenario. Depending on the availability and reliability of the data various distributions can be used to estimate the weight of each of the consequence scenarios:

  • Bernoulli or discrete distribution – where there limited historical data and the probability of a single or multiple consequences needs to be estimated.
  • Poison distribution – where we have historical data to estimate the frequency of each of the consequence scenarios.

Academic disciplines

Current controls, their effectiveness and other factors affecting the probability of claims against the company have to be accounted for when allocating weights to each of the scenarios.

Step 5. Measure the effect of risks on decisions

In order to account for the uncertainty both in the consequences of each scenario and its weight, consequence distributions are multiplied by weight distributions using the Monte-Carlo simulation method. Normally 10000 simulation runs should be sufficient for most compliance risks, however more simulation runs may be required for highly unlikely and catastrophic events.

The output of risk analysis can be represented as a distribution or box plot as shown below:

Disqualification

The distribution of the possible outcomes shows:

  • Reasonable optimistic scenario (usually minimal or no financial consequences)
  • Expected scenario (50th percentile)
  • Reasonable pessimistic scenario (financial consequences which would not be exceed 95% of the time, 5% probability that impact may be even greater).

An integral part of the risk analysis is a tornado diagram showing which of the consequence scenarios is having the most effect on the overall risk exposure level. An example is shown below:

Law

In the situation where the risk exposure is deemed significant, risk mitigation measures need to be discussed and agreed upon.

Often it may be insufficient to just estimate the compliance risk exposure, instead it may be required to measure how compliance risks would affect an investment decision, a performance target or business plan or budget. In such cases it may be necessary to estimate how compliance risks change the project NPV / other decision making metric or how compliance risks change the probability of successfully finishing the project on time and budget.

Risk mitigation and trade-off

The treatment of compliance risks refers to the corresponding strategies implemented by an organization to deal with its risks. A risk treatment plan should consider a range of treatment options, which may include legal remedies as well as financial, operational and reputational remedies for each prioritized risk.

The following factors should be considered when choosing an appropriate option for the treatment of compliance risks:

  • the organizational risk management policy, strategic objectives, core values and legal responsibility of the organization;
  • a cost benefit analysis of responding to compliance risk;
  • the stakeholders’ perception and their values, attitude to risk and tolerance levels, as well as their preferences on certain compliance risk treatment strategies;
  • the availability and allocation of resources needed to manage the risk;
  • a legal review (including scope and depth) of laws, contractual commitments and limiting risk contractually;
  • legal opinions;
  • the extent to which the compliance risk can under law be transferred, delegated or insured against;
  • the level of risk awareness and maturity level within the organization.

Different mitigation and treatment strategies can be tested to determine which option provides the best value in risk reduction for the cost involved. Different mitigation strategies can be graphically represented as on the diagram below:

Academic disciplines

Reporting and monitoring

The monitoring and review of the management of compliance risks includes the following:

  • staying abreast of changes in the environment, such as the introduction of new laws and the enforcement of such laws, in order to adjust the organization’s strategy accordingly;
  • monitoring events triggered by compliance risk, analysing their frequency and patterns, and drawing conclusions from them (including potential correlation with and amplification of other risks);
  • considering an early warning system with key stakeholders to identify warning signals for significant compliance risks that could arise;
  • monitoring and reviewing:
    • outcomes following risk treatment;
    • changes in the environment;
    • the building of integrated risk treatment plans;
    • the designation of the responsible and accountable parties;
  • comparing progress with the risk treatment plan, reviewing and updating the risk treatment plan periodically and in a timely manner to seek assurance on its adequacy, suitability and effectiveness in relation to the management of compliance risk.

An organization should consider the following issues in relation to record-keeping and reporting:

  • legal professional privilege, attorney–client privilege and work product (or their equivalent concepts and terms under the relevant national law);
  • destruction, retention and privacy policies, in accordance with data protection laws;
  • the availability and accessibility of documentation for stakeholders to improve decision-making and for internal or external audit purposes.
  • whether the relevant documentation needs to be maintained securely, with a chain of evidence process documenting that no alterations have been made to the documents, information or evidence;
  • confidentiality and security measures in relation to documentation of a confidential nature, such as setting up limited and authorized access to such documentation.

An organization should report on the progress of changes in implementing the management of compliance risks and adherence to the measures.

RISK-ACADEMY offers online courses

sample85
+ Buy now

Informed Risk Taking

Learn 15 practical steps on integrating risk management into decision making, business processes, organizational culture and other activities!

19,999,99
sample85
+ Buy now

ISO31000 Integrating Risk Management

Alex Sidorenko, known for his risk management blog http://www.riskacademy.blog, has created a 25-step program to integrate risk management into decision making, core business processes and the overall culture of the organization.

199,999,99
sample85
+ Buy now

Управление рисками

В этом коротком и очень увлекательном курсе, Алексей Сидоренко расскажет о причинах внедрения риск менеджмента, об особенностях принятия управленческих решений в ситуации неопределенности и изменениях в новом стандарте ИСО 31000:2018.

19,999,99

2 thoughts on “ISO and COSO haven’t got a clue. You can and should quantify compliance risks

  1. I think you skipped something in Step 3. How did translate the qualitative consequence ranges into quantitative distributions?

    1. Not sure what qualitative consequence ranges are, the ranges we use in calculations are very quantitative and either come from the legislation or from statistics or are estimated by business owners

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.