Communicating Confidence and Uncertainty in OSINT

An often misunderstood aspect of intelligence analysis is the difference between what we know and what we believe. Investigators are often asked to answer difficult questions with incomplete information. Does this social media account belong to the subject? Is this company linked to organised crime? Is this individual likely to attend a planned event? In many cases, the available information does not allow absolute certainty, yet a decision still needs to be made.

This uncertainty is the reality of working with incomplete, evolving and sometimes contradictory information. Part of the role of an intelligence analyst is to reduce and communicate uncertainty honestly. This is why intelligence products should communicate the confidence that can reasonably be placed in an analyst’s conclusions.

Intelligence rarely provides certainty

OSINT investigations are built upon fragments of information collected from many different sources. Some sources are official and highly reliable, while others may be anonymous, incomplete or impossible to verify. New information may appear tomorrow that completely changes today's assessment. This means intelligence is often a judgement rather than a statement of fact.

Professional analysts should therefore avoid presenting assessments with greater certainty than the information available supports. Decision-makers are far better served by an honest explanation of uncertainty than by false confidence.

Confidence is not the same as probability

One of the most common mistakes made by new analysts is confusing confidence with probability. These are related concepts, but they describe completely different things.

Confidence reflects the strength of the evidence supporting an assessment.

Probability reflects the likelihood that an event will occur.

For example, an analyst may have high confidence that a subject has travelled to London because multiple independent sources support that conclusion. Yet, they may assess only a realistic possibility that the subject intends to attend a particular event. The evidence may clearly demonstrate where the individual is, while providing only limited insight into what they intend to do next.

Likewise, an analyst may assess that an attack is highly likely, but only with moderate confidence because the available intelligence contains important gaps or has not yet been fully corroborated.

Understanding this distinction is fundamental to communicating intelligence accurately.

Confidence begins with the information

Confidence should never reflect how strongly an analyst believes something to be true. Instead, it should reflect the quality of the underlying intelligence.

Several factors influence confidence, including:

• The reliability of the sources.
• The credibility of the information.
• The amount of corroboration available.
• The completeness of the intelligence picture.
• The presence of conflicting information.
• The number of remaining intelligence gaps.

High confidence does not mean an assessment is guaranteed to be correct. It simply means that the available evidence provides strong support for the conclusions reached.

Similarly, low confidence does not necessarily mean the assessment is wrong. It indicates that the evidence is limited, incomplete or subject to significant uncertainty.

Concept	Question it answers	Example
Source Reliability	Can I trust the source?	A government register is generally more reliable than an anonymous forum post.
Information Credibility	Can I trust this information?	A claim corroborated by several independent sources is more credible than an unverified rumour.
Analytical Confidence	How strong is the evidence supporting my assessment?	Moderate confidence that the subject owns the account.
Probability	How likely is a future event?	It is likely that the subject will attend the event.

Using confidence levels

Many intelligence organisations communicate confidence using a simple three-tier model. The wording varies slightly between organisations, but the underlying principle remains the same. 

• High confidence - The assessment is supported by reliable sources, corroborated information and relatively few significant intelligence gaps. While future information could still change the assessment, the available evidence provides a strong basis for the conclusions reached.
• Moderate confidence -The assessment is supported by credible evidence, but there are limitations. Some information may remain unverified, alternative explanations may still exist or important intelligence gaps have yet to be resolved.
• Low confidence -The available evidence is limited, contradictory or difficult to verify. The assessment represents the analyst's best judgement based on the current intelligence picture, but significant uncertainty remains.

Using confidence levels helps decision-makers understand not only what the analyst believes, but how strongly those conclusions are supported by the available evidence.

Communicating probability

Confidence describes the quality of the evidence. Probability describes the likelihood of an event. This distinction becomes particularly important when assessing future behaviour.

Questions such as whether an individual will commit fraud, attend an event or target a particular victim can rarely be answered with certainty. Instead, analysts assess probability based upon the available intelligence.

The challenge is that words such as likely, possible and unlikely mean different things to different people. One manager may interpret likely as meaning a 60% chance, while another assumes it means closer to 90%.

To reduce this ambiguity, many intelligence organisations use standardised probability language.

The probability yardstick

The Probability Yardstick was developed to encourage consistency when communicating likelihood. Rather than allowing analysts to use any wording they choose, it promotes a common set of expressions that can be understood consistently across organisations.

Typical expressions include:

Probability Language	Meaning	Probability Percentage
Almost certain	The event is almost certainly expected to occur.	>95%
Highly likely	The event is expected to occur in most circumstances based on the available intelligence.	80% - 90%
Likely	The event is more likely to occur than not, although uncertainty remains.	55% - 75%
Realistic possibility	The event could reasonably occur, but there is insufficient evidence to conclude it is more likely than not.	40% - 50%
Unlikely	The event is considered less likely to occur based on the available intelligence.	25% - 35%
Highly unlikely	The event is not expected to occur, although it cannot be completely ruled out.	10% - 20%
Remote chance	The event is almost certainly not going to occur.	<5%

The objective is to reduce the risk that different readers interpret the same assessment in different ways. Standardised language is particularly valuable when intelligence is shared between organisations, ensuring that probability is communicated consistently regardless of who produced the assessment.

Confidence in OSINT

OSINT can present particular challenges when communicating confidence. A single piece of information found online may appear highly convincing while remaining impossible to verify. Equally, several independent public sources may provide sufficient corroboration to support a high-confidence assessment.

Analysts should resist the temptation to either trust or dismiss information simply because it was found on the internet. For example, an address obtained from a government register may suggest high confidence. A social media profile claiming to belong to the same individual may initially justify only moderate confidence until it can be corroborated through additional evidence such as photographs, known associates or historical activity.

Confidence should always be based on the evidence rather than the platform from which it was collected.

Avoid false precision

One of the biggest dangers in intelligence analysis is presenting subjective assessments with unwarranted precision. It is rarely appropriate to state that there is a 73% chance that an individual will commit an offence or an 84% chance that a company is engaged in fraud unless those figures are supported by a validated statistical model.

For most intelligence assessments, qualitative language communicates uncertainty more honestly than arbitrary numerical probabilities. Analysts should be wary of giving the impression of scientific accuracy where none exists.

Good intelligence acknowledges uncertainty

Some analysts may worry that expressing uncertainty makes their work appear less credible. In reality, the opposite is true.

Decision-makers should understand that intelligence rarely provides complete certainty. What they need is an honest assessment of what is known, what remains uncertain and how strongly the available evidence supports the conclusions presented.

The most effective analysts are those who communicate confidence clearly, acknowledge uncertainty openly and ensure that their assessments accurately reflect the strength of the underlying intelligence. That is what allows intelligence to support better decisions, even when the full picture is not yet known.