Beyond the Black-Box—GenAI and Bayesian Networks for Intelligence Analysis

Large language models (LLMs) are increasingly used by analysts for drafting, summarizing, and exploring courses of action. However, the reasoning they appear to perform is not grounded in operational reality but in the statistical structure of language. LLMs work with linguistic representations that have already passed through layers of human interpretation, which means they operate on abstractions that are several steps removed from the causal mechanisms that matter in the field. Their internal processes involve completing patterns in text, inferring associations from co-occurrences, and synthesizing analogies across documents. This form of meta-reasoning can support hypothesis generation and conceptual exploration, but it cannot provide the traceable, auditable, and reproducible logic required for defense and intelligence decisions, particularly when judgments must withstand detailed review or adversarial scrutiny.

This talk introduces a hybrid analytic approach that integrates LLMs with Bayesian networks to produce explicit and reproducible models of uncertainty and causality. Unlike traditional practice, where AI-generated insights and quantitative models remain separate, this method incorporates LLM-derived hypotheses directly into a formal probabilistic structure that can be inspected, validated, and audited. The goal is to preserve the generative strengths of LLMs while grounding conclusions in a transparent, mathematically coherent model.

A central component of this framework is the explicit representation of time and information value. Many defense problems involve shrinking decision windows in which information improves even as operational options deteriorate. Bayesian updating is required to reason correctly about this evolving uncertainty, and the Value of Information (VoI) provides the formal mechanism to determine whether waiting for additional intelligence is likely to improve expected outcomes or simply reduce the range of feasible actions. Heuristic rules or intuitive probability adjustments cannot accurately capture how beliefs, risks, and utilities shift over time.

Examples from joint operations, diplomatic negotiations, and search-and-rescue illustrate how this integrated approach unifies human expertise, empirical data, and LLM-generated insights within a single decision model. The result is a practical and rigorous method for improving analytic effectiveness in national security contexts, especially when decisions are time-critical, assumptions must be transparent, and the payoff of additional information must be computed rather than guessed.

Also check out other Workshops in Arlington.