Moving Beyond the Qualitative vs. Quantitative Risk Debate

By Prof. Hernan Huwyler, MBA CPA CAIO · Academic Director, IE Law School
Published 4 November 2026 · Last reviewed 8 November 2026

In short

• The debate between qualitative and quantitative risk assessment misses a fundamental challenge: autonomous systems make probabilistic decisions at runtime. Static heat maps alone cannot capture how their behaviour and risks change in operation.
• Risk reporting should support real decisions. Measure the business impact of verified failure modes and demonstrate whether technical safeguards, human oversight and runtime controls effectively limit those risks.

Organizations establish risk committees, draft policies, and populate registers with qualitative heat maps, yet these measures frequently create a false sense of security. When systems operate autonomously and make probabilistic decisions, static compliance frameworks break down. Moving past this paper-pushing illusion requires a fundamental shift in how teams measure exposure, evaluate runtime controls, and govern agentic behavior across production environments.

Governance must become an operating system for everyday decisions rather than a graveyard of static documents. International frameworks like the NIST AI Risk Management Framework frame governance as a continuous, cross-cutting function that embeds risk management across the entire technology lifecycle. That means pulling policies, accountability structures, third-party integrations, technical design, and corporate culture into a unified operational pattern. The EU AI Act takes a similar stance by treating high-risk systems as an ongoing lifecycle obligation. Risk management, data governance, continuous logging, technical transparency, human oversight, system robustness, cybersecurity, and post-market monitoring are living operations, not one-time compliance artifacts to file away before an audit.

 

Oversight in AI Systems From Governance Illusion to Operational Control

By Prof. Hernan Huwyler, MBA CPA CAIO · Academic Director, IE Law School
Published 8 November 2026 · Last reviewed 8 November 2026

In short

• A policy, a committee, or a risk register entry is not evidence that an AI model is safe to approve. Evidence is a dated record that a quantified assessment was done, that a named executive accepted a stated loss tolerance, and that the model was tested against acceptance criteria set before the test.
• Acceptance criteria come from the loss the organization can absorb, not from a vendor benchmark. In the worked example below, a tolerance of 5 percent probability of exceeding 1.5 million in annual fraud loss translates into a minimum recall of about 0.79, so the approval threshold is set at 0.80 and tested with a confidence interval.
• Test the controls the criteria rely on. Measure reviewer catch rates in drills, red-team the agent permissions, and rerun the quantification when drift, a new tool, or a new model version changes the assumptions.

 

An approval board reviews a payment fraud screening model on a Tuesday afternoon. The vendor slide reports 94 percent accuracy, the project team is pleased, and the chair asks who decided that 94 percent was enough. Nobody answers, because the figure came from the vendor's test set and the bar was set after the result was known. The board approves the model anyway, and the minutes record that it met expectations.

AI governance needs evidence of that decision, and evidence has a specific form. It starts with a quantified assessment of what the model could cost if it fails: how often, how much, and how likely a bad year is. An accountable executive then states a tolerance, such as the largest annual loss they accept at a stated probability. Acceptance criteria are derived from that tolerance before anyone tests the model, and the test results, the approval, and the later monitoring are stored in records a reviewer could rerun. Without that chain, a governance program produces documents that describe intent and nothing that proves control.

This guide is for Chief AI Officers, risk and compliance leaders, internal auditors, model owners, and anyone who signs an approval. It explains what counts as evidence, what a quantified AI risk assessment contains, and how to turn a loss tolerance into numerical acceptance criteria for predictive and agentic systems. It then covers testing against those criteria, measuring human oversight as a control, how the EU AI Act and the main standards point the same way, and the evidence register that ties it together.

 

 

Risk Heat Maps to Monte Carlo: Practical Guide for Risk Owners

By Prof. Hernan Huwyler, MBA CPA CAIO · Academic Director, IE Law School
Published 4 November 2026 · Last reviewed 4 November 2026

In short

• A risk heat map is a vage way to display risk, not a measure of financial exposure. It never tells decision-makers how much they could lose, how often, or which assumptions drive the result.
• Range compression and rank reversal can conceal important differences between risks, therefore, it will induce wrong decision-making. Simply multiplying frequency by impact with qualitative scales and scores is professional malpractice.
• Loss exceedance curves provide a more useful alternative. When historical data is limited or unavailable, model uncertainty explicitly instead of relying on arbitrary colour categories.

 

A risk workshop ends at four in the afternoon. Twenty people have argued for two hours about whether a vendor outage is a three or a four on likelihood, and the facilitator closes by moving a dot into a yellow cell. Everyone leaves with the feeling that something was decided. Nothing was. Nobody can say how much money the company might lose, how often, or which assumption would change the answer.

A risk heat map is a display format. It is not a risk analysis, and it cannot replace one. The replacement is a quantified model: frequency and severity estimated as ranges, combined by simulation, and reported as a loss exceedance curve that tells an executive the probability of losing more than a given amount in a year. This guide is for risk owners, fraud and cybersecurity leads, internal auditors, and AI product owners who need to defend a number in front of a board and want a method that survives questions.

The path runs in a fixed order. First you see why the matrix fails, with the research behind it. Then you replace frequency times impact with the correct combination of the two distributions, build the curve, handle missing data, add machine learning where it earns its place, and connect the output to a decision. The last section covers the politics of retiring a tool that management likes.

 

Compliance Theater Is Hiding Your Biggest Security Risks

By Prof. Hernan Huwyler, MBA CPA CAIO · Academic Director, IE Law School
Published 17 October 2026 · Last reviewed 17 October 2026

In short

• Compliance should be the outcome of an effective security programme, not a substitute for one. Reversing that logic can produce certificates and completed checklists while leaving critical security gaps undiscovered.
• Replace activity-based metrics with evidence of results: verified remediation, actual asset and vulnerability scan coverage, and audit evidence generated through routine operations.
• Attackers often exploit assets that the organisation does not know exist. An incomplete asset inventory can therefore undermine even a well-documented compliance programme.

 

Consider an illustrative scenario, a composite of patterns that repeat across security programs. A payments processor presents its quarterly security report to the audit committee. Every scheduled vulnerability scan ran on time. Nearly all employees finished security training, the certification renewed without a gap, and every audit finding from the prior year is closed. Three weeks later an attacker enters through a server that never appeared on a scan target list, because the team built it after the list was last updated. The report was accurate on every line, and it still told the committee nothing about whether the company was getting safer.

Compliance is a byproduct of security. When an organization knows its threats, protects the assets that matter, tests its controls, and fixes weaknesses until a retest confirms the fix, most of its compliance evidence becomes a record of work already done to reduce risk. When the sequence runs the other way, the organization collects evidence first and hopes protection follows. That gap is where false confidence grows, and decision-makers stop asking hard questions at the moment they most need to ask them. This article is for CISOs and compliance officers who want to report outcomes instead of activity, and it ends with seven metrics you can start swapping in this quarter.

The argument covers the obligations most teams already carry. GDPR, NIS2, HIPAA, SOX, PCI DSS, ISO/IEC 27001, and SOC 2 each ask, in different words, whether controls work and whether you can show it. Compliance stays a legal and contractual duty with its own deadlines, and nothing here argues for skipping it. The point concerns order and evidence. Security work comes first, and the records that work produces earn the compliance claim. The sections below cover what each obligation requires, how checkbox reporting goes wrong, which metrics replace it, and how to make the evidence trail a normal output of operations.

 

Guide for Risk Managers for DORA Compliance

By Prof. Hernan Huwyler, MBA CPA CAIO · Academic Director, IE Law School
Published 16 October 2026 · Last reviewed 16 October 2026

In short

• DORA has applied since 17 January 2025. The priority is now demonstrating that data, accountable owners and operational processes can support compliance and respond to supervisory scrutiny.
• Map critical or important functions to the ICT assets and services that support them before preparing reporting templates. This register provides the foundation for the wider DORA program.
• Build the programme around day-to-day operations, with particular attention to incident reporting deadlines and ICT third-party oversight, two areas where implementation can become difficult.

 

A quarterly operational risk report that takes four months to produce is already a DORA problem, and nobody has opened the regulation yet. By the time the report lands, two newer quarters of incidents, supplier changes and control failures have happened. The Digital Operational Resilience Act, Regulation (EU) 2022/2554, has applied since 17 January 2025, so the open question for most financial entities is no longer when to start. The question is whether their data, owners and processes can answer a supervisor within hours instead of months.

The short answer is to build one maintained dependency model that links each critical or important function to its information, ICT assets, providers, contracts, incidents and test results, and then run the five DORA chapters on top of it. The chapters cover ICT risk management, incident management and reporting, resilience testing, ICT third-party risk, and information sharing. Most of the effort sits in the links between those chapters. A firm that treats them as five projects ends up with five registers that disagree with each other.

This guide is for risk owners, compliance officers, internal auditors and security leads who have to turn the regulation into working procedures. It follows the order in which a program usually runs, and it separates what the regulation requires from what is a sensible implementation choice. It also corrects several statements that circulate in conference slides and vendor material, such as the idea that DORA requires threat-led penetration testing from every firm.

 

How to Plan, Govern, and Build Enterprise AI Agents and AI Architecture

By Prof. Hernan Huwyler, MBA, CPA, CAIO · Academic Director, IE Law School
Published 14 October 2026 · Last reviewed 14 October 2026

In short

• An AI pilot can perform impressively in a controlled demonstration yet fail in production when real users, operational costs, data access, permissions and unpredictable outputs enter the picture.
• Enterprise AI governance must extend beyond model selection and prompt design. It must cover the complete operating environment: data, integrations, identity, access rights, monitoring, escalation and accountability.
• AI agents need clearly defined permissions and boundaries. Their ability to call tools, retrieve information or initiate transactions should be limited to approved tasks and risk levels.
• Deterministic software controls should enforce critical business rules, security requirements and approval thresholds. Prompts and model instructions alone are not reliable enforcement mechanisms.
• Successful AI deployment requires named business and technical owners, measurable performance indicators, human intervention procedures and continuous monitoring after go-live.
• The central lesson for executives, AI architects and risk professionals: production readiness is an architecture and governance challenge, not simply a question of choosing a more powerful AI model.

 

AI governance professionals need more than policies. They need to understand how AI systems are designed, where controls belong, and what happens when agents act in the real world.

An AI pilot can look impressive in a demonstration and still fail when a company tries to put it into production. Costs grow, permissions become difficult to manage, outputs vary, and nobody knows exactly who should intervene when something goes wrong.

For risk managers, internal auditors, compliance officers, cybersecurity experts, and IT professionals, these are not abstract technology problems. They are questions of accountability, control design, operational resilience, and business value.

That is why I believe AI architecture deserves a place at the center of enterprise AI governance.