A risk workshop ends at four in the afternoon. Twenty people have argued for two hours about whether a vendor outage is a three or a four on likelihood, and the facilitator closes by moving a dot into a yellow cell. Everyone leaves with the feeling that something was decided. Nothing was. Nobody can say how much money the company might lose, how often, or which assumption would change the answer.
A risk heat map is a display format. It is not a risk analysis, and it cannot replace one. The replacement is a quantified model: frequency and severity estimated as ranges, combined by simulation, and reported as a loss exceedance curve that tells an executive the probability of losing more than a given amount in a year. This guide is for risk owners, fraud and cybersecurity leads, internal auditors, and AI product owners who need to defend a number in front of a board and want a method that survives questions.
The path runs in a fixed order. First you see why the matrix fails, with the research behind it. Then you replace frequency times impact with the correct combination of the two distributions, build the curve, handle missing data, add machine learning where it earns its place, and connect the output to a decision. The last section covers the politics of retiring a tool that management likes.
Why Risk Heat Maps Fail as Risk Analysis
A heat map takes a judgment about likelihood, a judgment about impact, and places the pair in a grid cell with a color. The grid is useful for one thing: sorting a long list of issues into rough piles so a meeting can move on. It fails the moment someone treats the color as a measurement. A risk is an uncertain future loss, and an uncertain loss has a shape: a most likely range, a tail, and a probability of nothing happening at all. A single cell erases that shape before anyone has looked at it.
The first failure is mathematical. Likelihood and impact ratings are ordinal labels. A four is bigger than a three, but nobody has defined by how much, so multiplying a three by a four to get a twelve produces a number with no unit. A twelve of what? Dollars, hours, customers? The product also behaves badly: many different pairs land on the same score, and the scores bunch in the middle of the scale. Peer-reviewed work on the mathematics of risk matrices, published in the journal Risk Analysis, shows that matrices can give identical ratings to risks that differ enormously in quantitative terms, and can even rate a smaller risk above a larger one.
The second failure is that the grid hides judgment instead of removing it. Every dot in a cell is the end of a chain of assumptions about time horizon, scenario, control effectiveness, and consequence. Once the dot appears, the chain is invisible and the output looks more objective than the opinions that produced it. Two people who disagree by a factor of ten on probability can both agree the risk is yellow, so the workshop records consensus where the real disagreement still sits. A method that reaches agreement quickly is not always a method that found the truth. Sometimes it only found a label both sides could live with.
The third failure is aggregation. Boards ask a reasonable question: what is our total exposure, and does it fit inside our appetite? Colors cannot be added. Portfolio effects depend on distributions and on how risks move together, so adding red and yellow dots says nothing about enterprise exposure. A company with fifty yellow risks might face a combined loss that is trivial or one that threatens solvency, and the heat map gives the same picture for both.
The fourth failure is direction. The standard matrix looks only at downside. Management decisions weigh downside against upside: a product launch, an acquisition, a new market. A tool that cannot show a range of outcomes on both sides cannot support the choice that management actually has to make. This is why risk functions that rely on matrices often find themselves invited to meetings after the decision is made, to record the risks of a plan nobody can change.
None of this means every organization must abandon colors tomorrow. Ordinal categories are fine for triage, for sorting incoming issues into a queue, or for a first screen before deeper work. The problem starts when the sorted list becomes the analysis and the board paper. A practical rule for risk owners: if the output of your method cannot be expressed in a unit the finance team recognizes, such as currency, hours of downtime, or number of records, then it is a classification and should be labeled as one.
How Range Compression and Rank Reversal Hide Real Exposure
Range compression happens when a coarse scale forces very different risks into the same cell. Suppose a five-point impact scale puts everything above one million in the top band. A fraud scheme that could cost 1.2 million and a data breach that could cost 80 million both score a five. The likelihood axis has the same problem. A risk with a ten percent annual probability and one with a thirty percent annual probability can both be labeled possible. The matrix then reports each of those pairs as equal, even though one fraud scheme can be more than sixty times smaller than the breach, and one risk can be three times more likely than its neighbor.
Rank reversal is worse because it produces the wrong order, not just a tie. Take two risks. Risk A has a likelihood in band three and an impact in band two. Risk B has a likelihood in band two and an impact in band three, but its real numbers are far larger on both axes than the band edges suggest. Depending on where inside each band the true values sit, the risk that receives the lower color can carry the higher expected loss. The research on risk matrices describes this directly: the qualitative rating can rank a quantitatively smaller risk above a larger one. If you use the matrix to decide where to spend the next control budget, you will sometimes spend it on the smaller problem.
Here is the practical way to test your own matrix. Pick ten risks from your register. For each one, ask the owner for a low, most likely, and high value for annual frequency, and a low, most likely, and high value for loss per event. Calculate the expected annual loss for each, then rank the ten by that number and compare the ranking with the colors. Look for pairs of risks whose order flips against the colors, and for single cells that hold risks whose expected losses differ by a factor of five or more. The exercise costs an afternoon and gives you evidence from your own data, which persuades more than any citation.
A second test concerns the number of cells. Count how many distinct risk scores your scale can produce, and compare it with the number of risks in your register. A five by five grid has twenty-five cells and, after multiplication, only fourteen distinct products. A register of two hundred risks therefore contains many items that the method calls equal by construction. Ask yourself whether any prioritization decision in the past year depended on the difference between a twelve and a fifteen. If none did, the extra resolution is decoration.
Ordering errors have a human cost beyond the budget. When a function reports a risk as amber and the same exposure turns out to be larger than a red one, the person who rated it carries the blame for a flaw in the instrument. The peer-reviewed literature on this failure is more than a decade old, and it reports little rigorous validation that matrices improve risk decisions. The burden of proof therefore sits with the tool, not with the people asking for a better one.
What Probability Words Really Communicate
Every heat map rests on words: rare, unlikely, possible, probable, almost certain. People attach very different numbers to these words, and the difference is not small. A journal published by an intelligence agency printed an article in 1964 describing officers who read the same estimate phrase and assigned probabilities with a wide spread. The article proposed fixed numeric ranges for each term, and the proposal was not widely adopted. Surveys of other groups since then have reported the same spread for words such as probably and likely.
In a risk workshop this means that a rating of high carries no stable meaning. For one participant high means once a year. For the person sitting beside them it means once in a decade. Both write high on the sticky note, and the facilitator averages the notes into a number that matches neither opinion. The disagreement survives the exercise, hidden inside a label. If you later ask why a risk is rated high, you will get a story about how the room felt that day, and the story will change if you hold the workshop again in six months.
The fix is cheap: use numbers. Replace the word possible with a stated probability range over a stated period, for example between five and fifteen percent over the next twelve months. Ranges are honest about uncertainty and still easy to read. If the participants resist because they do not know the exact value, tell them that nobody needs the exact value. A range of five to fifteen percent says the event is unlikely but not remote, and says how unlikely. It also gives later evidence something to disprove.
A second fix applies to the scale labels themselves. If a group insists on keeping a qualitative scale for communication, anchor every label to a numeric band and print the bands on every page of the report. Then require that the bands be written in the same units as the loss model, so the label and the quantified analysis agree. This keeps the language shared across fraud, cybersecurity, operational risk, and compliance teams, which otherwise describe the same exposure in four dialects. It also prevents the quiet drift in which a team recalibrates its words each quarter to keep the colors stable.
One more consequence is worth naming. Verbal scales also hide where the uncertainty is greatest. A risk rated medium with strong loss history and a risk rated medium with almost no data look identical on the grid. A numeric range makes the difference visible: a narrow range signals evidence, a wide range signals a guess. Executives can then decide whether to fund more analysis for the wide ones, which is an actual decision, instead of arguing over colors.
Why Multiplying Frequency by Impact Gives the Wrong Answer
The formula most teams learn is risk equals likelihood times impact. It looks like arithmetic, and it is the root of the matrix. The trouble is that frequency and severity are different kinds of quantities. Frequency counts events: zero, one, two, three. You never have one and a half ransomware incidents in a year. Severity measures money or time, and it is continuous: a loss can be 178.74 dollars or 199 million dollars and four cents. A count and a continuous amount cannot be multiplied as if both were single numbers when what you want is the distribution of the total.
The correct combination is a compound model. In a given year, the number of events is a random count. Each event draws a random loss from the severity distribution. The annual loss is the sum of those draws, and the distribution of that sum is the convolution of the frequency and severity distributions. In operational risk the usual starting point for frequency is the Poisson distribution, a count model with one parameter, the average number of events per year. That is a convention rather than a law: it fits many event processes well, and you should change it when the data show clustering or a different pattern. Severity is typically right-skewed, so a lognormal or a similar heavy-tailed distribution is a common first choice.
What does multiplication give you? If you multiply the average number of events by the average loss per event, you get the mean of the annual loss distribution, and only the mean. That number has legitimate uses, for example in budgeting expected losses. It tells you nothing about the tail, which is where the board's real question lives: what is the chance we lose more than we can absorb? If you multiply conservative inputs together, say a high frequency and a high severity, you get a figure that corresponds to no percentile of anything. Sometimes it overstates the exposure, and sometimes it understates it. The next section shows an example where multiplying a high-percentile severity by the average frequency lands below the true ninety-fifth percentile of annual loss.
The same logic explains why averages mislead even when the method is right. Compare two exposures that both have an expected annual loss of 4 million. One can plausibly cost between 3 and 5 million. The other can cost anywhere from 2 to 11 million. A decision maker who sees only the 4 million treats them as the same. They are not the same, because the second one carries a tail that could exceed the organization's tolerance, and the first one does not. In risk, the distribution is the information. The average is one summary of it, and the weakest one for protective decisions.
There is a practical test for whether your team is combining frequency and impact correctly. Ask for the probability that annual losses from the risk exceed a stated amount. If the method cannot answer that question, it is not modeling the loss distribution. A matrix cannot answer it, and a product of two averages cannot answer it. A compound model answers it directly, and it also answers the follow-up questions that executives ask: what happens if the frequency doubles, and what happens if the control halves the severity.
Finally, keep the vocabulary straight so reviewers can follow. Expected loss is the mean. Unexpected loss is the amount above the mean that you want to be protected against at a stated confidence level. A loss exceedance curve plots the probability that the annual loss is greater than each possible amount. Value at risk and similar measures are single points read from that curve. A reader who knows these four terms can audit your analysis, and one who does not will fall back on colors.
How to Build a Loss Exceedance Curve Step by Step
Start with the decision, not with the risk register. Write one sentence that says what the model will help someone decide: whether to approve a product launch, whether to buy more insurance, whether to fund a control, or whether an AI system can go live. Then name the owner of that decision and the tolerance they already work with, such as a minimum operating profit or a maximum annual loss. A model without a decision is an exercise, and it tends to grow until nobody can finish it. A useful test of a good decision statement is that you can name an assumption whose change would flip the answer.
Next, define the scenario in terms of the objective at risk. A scenario names the asset or process, the cause, the event, and the consequence: for example, a ransomware event that locks a payment platform for several days and triggers customer compensation, regulatory notification, and recovery costs. Keep each scenario narrow enough that one frequency and one severity distribution make sense. Related scenarios that share causes, such as a data leak and the privacy penalty it may trigger, should be modeled with their dependency stated, not in separate workshops in different months. Breaking the analysis into silos is how organizations end up with four teams producing four unreconciled numbers for the same event.
Third, estimate frequency and severity as distributions. For frequency, gather internal incident counts, industry loss data, control test results, and expert judgment, then fit or choose a count distribution with an annual rate. For severity, collect loss amounts for similar events, split them into cost components such as response, downtime, legal, and customer compensation, and fit a distribution that allows for a long right tail. Record the source of every parameter and who approved it. The evidence you retain at this step is what an auditor will ask for first, so keep the data extract, the fitting notes, and the approval record together.
Fourth, run the simulation. For each of many iterations, which can be one million in a laptop-sized run, draw a number of events from the frequency distribution, draw a loss for each event from the severity distribution, and sum them to get one simulated year. Repeat. The simulation method itself dates back to the mid-1940s, when researchers at a national laboratory in the United States developed it to study how neutrons move through materials. It was first published openly in 1949. The method is old and well understood. The failure in risk management has been adoption, not novelty.
Fifth, read the output as a curve and a few percentiles, then test it. Plot the probability of exceeding each loss amount, mark the tolerance, and read the probability of breaching it. Run sensitivity analysis by changing one input at a time and recording the movement in the percentile you care about; a tornado diagram is a horizontal bar chart that ranks the inputs by how much they move the result. Compare the model with history: if the model says a loss above a given amount happens once in a hundred years and your sector has seen it twice in a decade, challenge the model. A model that cannot be wrong is not a model.
Consider an illustrative scenario with assumptions stated, not a real engagement. A mid-sized company models one ransomware-type event on a core platform. The frequency is a Poisson count with an average of 1.2 events a year. Severity is lognormal with a median of 250,000 dollars per event and a shape parameter of 1.1, which gives a mean of about 458,000 per event. The simulation runs one million years. The table shows the result.
| Metric | Definition | Value from the simulation |
|---|---|---|
| Probability of no event in a year | Years with zero events divided by all years | About 30 percent |
| Expected annual loss | Mean of simulated annual losses | About 550,000 dollars |
| Median annual loss | Middle simulated year | About 235,000 dollars |
| 90th percentile annual loss | Loss exceeded in 10 percent of years | About 1.47 million dollars |
| 95th percentile annual loss | Loss exceeded in 5 percent of years | About 2.14 million dollars |
| 99th percentile annual loss | Loss exceeded in 1 percent of years | About 4.15 million dollars |
| Probability of loss above 1 million | Years above 1 million divided by all years | About 17 percent |
| Probability of loss above 2 million | Years above 2 million divided by all years | About 6 percent |
| Probability of loss above 5 million | Years above 5 million divided by all years | About 0.6 percent |
The arithmetic reconciles with the simple formula for the mean: 1.2 events times about 458,000 per event gives about 549,000, which matches the simulated mean. Now suppose an analyst multiplies the same frequency by the ninety-fifth percentile of severity, about 1.52 million, to get a worst-case figure of roughly 1.83 million. That number sits below the true ninety-fifth percentile of annual loss, which is about 2.14 million, because it ignores years with two or three events. If the board's stated tolerance is a 2 million annual loss and its appetite is a breach probability below 5 percent, the model says the company sits at about 6 percent and a decision conversation is required. The matrix would have shown a single orange cell. All numbers here are assumptions chosen for illustration, and in your own company the owner would replace each one with sourced data.
What to Do When You Have No Data
The most common objection to quantification is that the data does not exist. Treat it as a claim to test. If a risk matters to the business, someone measures something related to it: incident tickets, near misses, losses booked in the ledger, downtime logs, insurance claims, audit findings, vendor outage records, industry reports. The first move is a data inventory. List the questions the model needs answered, then list where partial evidence lives, including systems owned by other departments. Many teams discover that the data exists and that no one had asked for it in the form the model needs.
When the evidence is thin, the correct tool is structured expert judgment with calibration. Ask each expert for a low and a high bound that they believe contain the true value with ninety percent confidence, and also ask for a most likely value. Then train them. Calibration training presents trivia questions with known answers, asks for ranges, and shows how often the true value fell outside. Most people start overconfident, with ranges too narrow, and improve quickly after feedback. Combine the calibrated ranges from several experts by averaging the distributions or by using a documented weighting, and keep the individual estimates so the disagreement stays visible.
A second source is the structure of the problem. Some risks have no history because the event has not happened yet. A cryptographic break that threatens how bank accounts are protected is one example, and a new AI system with no operating record is another. Break such a scenario into components that you can estimate: how long until the capability exists, how many systems depend on the affected method, how quickly you could migrate, and what a failure would cost per day. Each component has some evidence even if the whole has none. This decomposition is a standard technique, and it often turns an unanswerable question into five answerable ones.
Bayesian updating gives you a way to start with expert priors and revise them as evidence arrives. Begin with a calibrated range for the event rate, record each quarter's observed count, and update the distribution. After a few periods, the data begins to dominate and the prior fades. This is a good fit for new products, new vendors, and newly deployed AI systems, where every month of operation is new evidence. It also supports a policy that the board can understand: estimates are provisional, and they tighten as the organization learns.
Synthetic data is a further option, and it needs honest labeling. Simulation can generate plausible event histories from a stated model, which helps test whether your pipeline works and how sensitive the result is to assumptions. It does not create evidence about the real world. A synthetic dataset built from the same expert beliefs you started with cannot confirm those beliefs. Use it to stress the model, and keep real observations as the only input that can lower the uncertainty.
There is a limit that deserves the same honesty. Under deep uncertainty, when the possible futures are not well enough understood to defend a probability structure, adding another distribution can make the model look more sophisticated while making it less defensible. In those cases, scenarios, stress tests, robust options that perform acceptably in many futures, and monitoring with triggers are better tools. A risk report should say which exposures were quantified and which were not, and it should include a short section on what the organization does not know. Executives tend to value that section, because it tells them where to be careful.
Where Machine Learning and AI Agents Fit in Risk Quantification
Machine learning belongs in risk quantification where there is a stream of labeled events and a decision made on each one. Fraud screening, payment monitoring, access anomalies, claims triage, and vendor behavior all fit. A model such as a random forest or gradient boosting classifier learns from past transactions which patterns preceded a loss and outputs a probability for each new transaction. Those scores can feed the frequency side of the loss model, or they can drive an operational rule: hold the payment above one threshold, ask for a second approval above another. The model supplies a prediction. The compound loss model still supplies the exposure.
Validation is where most of the risk sits, so define it before the model is built. Split the data by time, train on the earlier period, and test on the later one, which is called backtesting. Report precision and recall at the operating threshold, the false-positive rate and what each false positive costs in analyst hours or customer friction, and the loss prevented per month at that threshold. Set a drift trigger: if the share of flagged transactions or the score distribution moves beyond a stated band, the model goes back to review. A model approved on last year's fraud pattern can degrade quietly when attackers change behavior, and monitoring is the control that catches it.
Agents add automation on top of the prediction. An agent here is software that takes an action when a condition is met: it freezes a card, opens a case, requests documents, or pushes a vendor into enhanced monitoring. The design rule is that authority is granted in writing and in code. Define which actions the agent may take alone, which need human approval, the loss or customer impact above which it must stop, and how each action is logged and reversed. Test the agent's rules against the same backtest data you used for the model. An agent that applies a treatment protocol automatically is a control, so it needs an owner, evidence, and periodic testing like any other control.
The AI system is itself a risk to assess, and standards exist for that. ISO/IEC 23894 provides guidance on risk management for AI and is meant to be used with ISO 31000, which is guidance and not a certifiable standard. The NIST AI Risk Management Framework is voluntary and organizes the work into governing, mapping, measuring, and managing AI risks. Use them for the questions a model approval must answer: who owns the system, what it is allowed to decide, what metrics define acceptable performance, who approves changes to thresholds, and what evidence shows that monitoring happened. These questions also feed commercial terms, because pricing, insurance, and warranties for AI software depend on the same measured error rates and the same owner accountability.
A separate warning concerns chatbots used as analysts. A large language model will produce a fluent risk report from almost any input, much as a person with no statistical training can produce a report with three colored pens. The report is not the problem. The problem is that nobody can explain how a number was derived, which inputs drove it, or what changes if an assumption moves. Prompts are not a methodology. If a team uses a language model in risk work, restrict it to drafting, summarizing, and code assistance around a model whose inputs, parameters, and outputs are documented and reproducible. The numbers in the report should come from the model run, and a reviewer should be able to rerun it and get the same percentiles.
How to Make Quantified Risk Change Decisions
A technically excellent distribution can still change nothing. Management is often impressed by sophisticated analysis and then proceeds as planned, because the output was information and not a decision input. The gap between analytical quality and decision readiness is the main reason risk functions lose influence. A distribution becomes decision-ready when management can connect it to something it owns: a tolerance, an alternative, a consequence, or a choice. The model informs the decision. Judgment stays with management, and the model should say so.
Start with the question that decides whether more analysis is worth doing: which assumption would make us change the decision? If no plausible assumption would change the choice, further analysis adds cost and no value, and this is often what people call analysis paralysis. Paralysis usually starts when the decision question was never stated. Once the question is written down, the sensitivity analysis shows which two or three inputs matter, and the team can spend its effort on those. The tornado diagram is the right chart for this, because it displays each input as a range and ranks the ranges by their effect on the outcome.
Risk appetite turns into something useful only when it becomes a boundary with a consequence. A statement that the organization has a low appetite for cyber risk tells a manager nothing about tomorrow. Compare it with a rule that says if the model shows a probability above five percent that annual operating profit falls below a stated minimum, management must hold a decision meeting and consider cost measures, a change of timing, or additional insurance. The number does not choose the action. It decides when the conversation must happen. Write boundaries like this for the two or three exposures that matter most, and test whether anyone used them in the last year.
Board reporting should be built as an attention allocation tool. A board has a limited number of minutes, and a list of twenty-five top risks does not set priorities, because twenty-five items compete for the same attention. Lead the paper with what changed since the last meeting, what the change means for a decision, and what action is requested. Put the loss exceedance curve for the three or four exposures near the tolerance on the first page, with a sentence in business language beside each. The detailed register can sit in an appendix. An easy signal of whether the function influences decisions is where risk sits on the meeting agenda: an item placed at the end, after the strategic choices were made, is a governance record and not decision support.
Communication matters as much as calculation. A table of dollar figures with no context rarely triggers action, while the same figures with a short narrative about what the event would look like, which customers or processes it would hit, and what the company would do on day one tend to move people. Show the same distribution to a finance lead, a chief executive, and an audit chair, and expect three different questions: what is the number, what is the downside, and who made the assumptions. Prepare an answer to each. It also helps to replay an event that happened at a peer organization and ask the group to run it against your own numbers, since that is a safe way to challenge scenarios nobody had on the register.
How to Retire the Risk Matrix Without a Fight
Management often likes the matrix for reasons that have nothing to do with its accuracy. It is familiar, quick to produce, and it reaches agreement in an afternoon. A directive to drop it will meet resistance, and the resistance is rational: the replacement has to deliver something management values at least as much. The workable path is a parallel run. Keep the matrix for one or two reporting cycles and add the quantified view beside it for a small set of exposures. Over time, executives start asking questions that only the quantified view can answer, and the matrix loses its role without anyone declaring it dead.
Choose the first exposures carefully. Pick two or three where a decision is coming within six months, such as a vendor consolidation, a cloud migration, an insurance renewal, or a fraud control investment. Build the model for them with a named business owner who will use the result. A model that supports a real decision creates demand, and a model built for its own sake does not. Report ranges and a tornado chart in the same paper as the matrix. Executives will notice that the chart shows which assumptions drive the result and the matrix does not.
Prepare for three predictable objections. The first is that there is no data, which the calibration process addresses. The second is that numbers imply false precision, and the reply is that ranges reveal the uncertainty that the colors conceal. The third is that the board will not understand simulation. In practice boards do not need the mechanics. They need the probability of breaching a tolerance, the main drivers, and the options with their costs. Teach the three or four terms from earlier in this guide and keep the method in an appendix for reviewers.
Set measures of effectiveness for the change itself, because risk functions are good at counting activity and weak at counting impact. Useful measures include the number of decisions in the past quarter that referenced a risk model, the number of tolerance breaches that triggered a documented decision conversation, the share of top exposures with a quantified range and an owner, and the forecast error of the models when compared with realized losses. Review the last measure honestly. If model estimates repeatedly miss, recalibrate, and tell the board that you did. The goal of the transition is better decisions, and the evidence is in what management did differently.
Expect the change to take years in some organizations. The parallel run can last up to three years before the matrix retires, and a Big Bang replacement rarely survives the first reporting cycle. Keep the quantified methods consistent across cybersecurity, fraud, third-party risk, projects, and compliance, because a single method lets you aggregate exposures and compare them in one currency. Different business lines may have different risk cultures and appetites, and a common distribution-based method makes those differences explicit instead of burying them in color choices. That is the real argument for the change: the method lets disagreement happen about probabilities, ranges, and assumptions, which are places where evidence can settle it.
Final Perspective
The next time a workshop ends with a dot in a cell, ask for the range behind it and the decision it supports. If the owner cannot give either, the dot is a placeholder, and your report should label it as one. The strongest change you can make this quarter is small: choose one exposure with a pending decision, state the tolerance, estimate frequency and severity as calibrated ranges, run the simulation, and bring the probability of breach to the people who own the choice. That single paper will teach your organization more about the value of quantification than another round of workshops.
Machine learning and agents raise the stakes on the same question. They make it cheap to produce predictions and automated actions, and they make it easy to produce a report nobody can explain. So the open question for the next year is which of your risk numbers a reviewer could rerun from raw inputs and defend line by line. Start a register of those numbers today, mark the ones that fail the test, and fix the highest-value ones first.
References
- Cox, L. A., What's Wrong with Risk Matrices?, Risk Analysis, vol. 28, issue 2, 2008, pages 497 to 512, https://econpapers.repec.org/RePEc:wly:riskan:v:28:y:2008:i:2:p:497-512
- Studies in Intelligence, Words of Estimative Probability, Fall 1964, https://webharvest.gov/peth04/20041016062723/http://www.cia.gov/csi/books/shermankent/6words.html
- Los Alamos National Laboratory, Hitting the Jackpot: The Birth of the Monte Carlo Method, Actinide Research Quarterly, first quarter 2023, https://discover.lanl.gov/publications/actinide-research-quarterly/first-quarter-2023/hitting-the-jackpot-the-birth-of-the-monte-carlo-method/
- ISO, ISO 31000:2018, Risk management, Guidelines, https://www.iso.org/standard/65694.html (link to verify)
- ISO/IEC, ISO/IEC 23894:2023, Information technology, Artificial intelligence, Guidance on risk management, https://www.iso.org/standard/77304.html (link to verify)
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, https://www.nist.gov/itl/ai-risk-management-framework (link to verify)