Research
Where the field has been, and where the evidence points. History and projection are separated deliberately below - the first is a matter of record, the second is expectation, and conflating them is how technology writing goes wrong.
Established history
Matters of record, datable to specific work.
Expert Systems
Hand-coded rules and decision trees, limited to narrow domains.
Statistical Machine Learning
Models learned from data: SVMs, random forests, gradient boosting.
Deep Learning
AlexNet in 2012 made GPU-trained deep networks the default for vision; CNNs and RNNs followed.
Transformers
Attention Is All You Need (Vaswani et al., 2017) removed sequential processing. BERT, GPT-2, T5.
Foundation Models
Instruction-following, multimodal, tool-using general models reached wide deployment.
Agentic Systems
Models given tools, memory and autonomy. OWASP moved Excessive Agency to third in its 2026 Top 10 on the strength of incident evidence. Covered in the AI engineering path, stages 5 and 7.
Projections
Not established. Each states what supports it and what argues against it.
Supports Tool-using single agents are already deployed and the failure modes are documented. Coordination is the obvious next step.
Against Each additional agent multiplies the failure surface, and cascading failure between agents is already a named risk category rather than a theoretical one.
Supports Test-time compute scaling shifts spend from training to serving, where it recurs per request rather than once.
Against Hardware and quantisation improvements have repeatedly cut unit costs faster than expected.
Supports The EU AI Act imposes technical documentation and logging duties on high-risk systems from 2 August 2026, with penalties attached.
Against Enforcement capacity is unproven and the obligations are widely read as satisfiable with minimal change.
Paper explainers
What a paper actually said, what it demonstrated versus suggested, and what is commonly misattributed to it.
The Transformer paper was a machine translation result, not a language model paper. What it proposed, what it measured, and what is misattributed to it.
"Retrieval-Augmented Generation" - what it actually saidThe RAG paper proposed a fine-tuning recipe with a trained retriever and marginalisation over latent documents, not prompt-stuffing a frozen model.
Open threads
Scaling laws and test-time compute
How performance scales with parameters, data and compute, and why spending compute at inference is now a distinct lever from spending it at training.
Retrieval and memory architectures
Durable memory beyond a single context window: retrieval, summarisation, and structured stores. Covered in part by the RAG page.
Interpretability
Probing, feature visualisation and mechanistic interpretability - understanding what a model is doing rather than only what it outputs.
Alignment and adversarial robustness
RLHF, constitutional methods, and red-teaming under adaptive attack. The OWASP prompt injection entry covers the security-facing half.