Learn AI Series (#140) - Scientific AI
Learn AI Series (#140) - Scientific AI

What will I learn
- How AlphaFold cracked the 50-year protein folding problem, and which pieces of THIS series it quietly reused to do it;
- AI for drug discovery -- target hunting, molecular generation, virtual screening -- and where the real bottleneck actually sits (spoiler: it is not the chemistry);
- AI for mathematics: theorem proving, conjecture generation, and whether a statistical pattern-matcher can honestly be said to "do maths";
- weather forecasting with neural nets (GraphCast, Pangu-Weather) and why they out-predict physics simulations that run on supercomputers;
- materials science -- predicting crystal structures and inventing materials that never existed;
- the line between AI-as-a-tool and AI-as-a-scientist, and an HONEST take on which side of that line we are actually standing on in 2026.
Requirements
- A working modern computer running macOS, Windows or Ubuntu;
- Python 3.10+ with PyTorch installed (
pip install torch) -- every snippet here is illustrative, nothing needs a GPU or a lab; - You've been through graph neural networks (#131), foundation models (#137), and the transformer arc (#52-53). We lean hard on all three, plus callbacks to self-supervised learning (#90) and RL search (#112).
Difficulty
- Beginner
Curriculum (of the Learn AI Series):
- Learn AI Series (#1) - What Machine Learning Actually Is
- Learn AI Series (#2) - Setting Up Your AI Workbench - Python and NumPy
- Learn AI Series (#3) - Your Data Is Just Numbers - How Machines See the World
- Learn AI Series (#4) - Your First Prediction - No Math, Just Intuition
- Learn AI Series (#5) - Patterns in Data - What "Learning" Actually Looks Like
- Learn AI Series (#6) - From Intuition to Math - Why We Need Formulas
- Learn AI Series (#7) - The Training Loop - See It Work Step by Step
- Learn AI Series (#8) - The Math You Actually Need (Part 1) - Linear Algebra
- Learn AI Series (#9) - The Math You Actually Need (Part 2) - Calculus and Probability
- Learn AI Series (#10) - Your First ML Model - Linear Regression From Scratch
- Learn AI Series (#11) - Making Linear Regression Real
- Learn AI Series (#12) - Classification - Logistic Regression From Scratch
- Learn AI Series (#13) - Evaluation - How to Know If Your Model Actually Works
- Learn AI Series (#14) - Data Preparation - The 80% Nobody Talks About
- Learn AI Series (#15) - Feature Engineering and Selection
- Learn AI Series (#16) - Scikit-Learn - The Standard Library of ML
- Learn AI Series (#17) - Decision Trees - How Machines Make Decisions
- Learn AI Series (#18) - Random Forests - Wisdom of Crowds
- Learn AI Series (#19) - Gradient Boosting - The Kaggle Champion
- Learn AI Series (#20) - Support Vector Machines - Drawing the Perfect Boundary
- Learn AI Series (#21) - Mini Project - Predicting Crypto Market Regimes
- Learn AI Series (#22) - K-Means Clustering - Finding Groups
- Learn AI Series (#23) - Advanced Clustering - Beyond K-Means
- Learn AI Series (#24) - Dimensionality Reduction - PCA
- Learn AI Series (#25) - Advanced Dimensionality Reduction - t-SNE and UMAP
- Learn AI Series (#26) - Anomaly Detection - Finding What Doesn't Belong
- Learn AI Series (#27) - Recommendation Systems - "Users Like You Also Liked..."
- Learn AI Series (#28) - Time Series Fundamentals - When Order Matters
- Learn AI Series (#29) - Time Series Forecasting - Predicting What Comes Next
- Learn AI Series (#30) - Natural Language Processing - Text as Data
- Learn AI Series (#31) - Word Embeddings - Meaning in Numbers
- Learn AI Series (#32) - Bayesian Methods - Thinking in Probabilities
- Learn AI Series (#33) - Ensemble Methods Deep Dive - Stacking and Blending
- Learn AI Series (#34) - ML Engineering - From Notebook to Production
- Learn AI Series (#35) - Data Ethics and Bias in ML
- Learn AI Series (#36) - Mini Project - Complete ML Pipeline
- Learn AI Series (#37) - The Perceptron - Where It All Started
- Learn AI Series (#38) - Neural Networks From Scratch - Forward Pass
- Learn AI Series (#39) - Neural Networks From Scratch - Backpropagation
- Learn AI Series (#40) - Training Neural Networks - Practical Challenges
- Learn AI Series (#41) - Optimization Algorithms - SGD, Momentum, Adam
- Learn AI Series (#42) - PyTorch Fundamentals - Tensors and Autograd
- Learn AI Series (#43) - PyTorch Data and Training
- Learn AI Series (#44) - PyTorch nn.Module - Building Real Networks
- Learn AI Series (#45) - Convolutional Neural Networks - Theory
- Learn AI Series (#46) - CNNs in Practice - Classic to Modern Architectures
- Learn AI Series (#47) - CNN Applications - Detection, Segmentation, Style Transfer
- Learn AI Series (#48) - Recurrent Neural Networks - Sequences
- Learn AI Series (#49) - LSTM and GRU - Solving the Memory Problem
- Learn AI Series (#50) - Sequence-to-Sequence Models
- Learn AI Series (#51) - Attention Mechanisms
- Learn AI Series (#52) - The Transformer Architecture (Part 1)
- Learn AI Series (#53) - The Transformer Architecture (Part 2)
- Learn AI Series (#54) - Vision Transformers
- Learn AI Series (#55) - Generative Adversarial Networks
- Learn AI Series (#56) - Mini Project - Building a Transformer From Scratch
- Learn AI Series (#57) - Language Modeling - Predicting the Next Word
- Learn AI Series (#58) - GPT Architecture - Decoder-Only Transformers
- Learn AI Series (#59) - BERT and Encoder Models
- Learn AI Series (#60) - Training Large Language Models
- Learn AI Series (#61) - Instruction Tuning and Alignment
- Learn AI Series (#62) - Prompt Engineering - Getting the Most from LLMs
- Learn AI Series (#63) - Embeddings and Vector Search
- Learn AI Series (#64) - Retrieval-Augmented Generation (RAG) - Basics
- Learn AI Series (#65) - RAG - Advanced Techniques
- Learn AI Series (#66) - Working with LLM APIs
- Learn AI Series (#67) - Building AI Agents (Part 1) - Foundations
- Learn AI Series (#68) - Building AI Agents (Part 2) - Advanced Patterns
- Learn AI Series (#69) - Fine-Tuning Language Models
- Learn AI Series (#70) - Running Local Models
- Learn AI Series (#71) - Text Generation Techniques
- Learn AI Series (#72) - Tokenization Deep Dive
- Learn AI Series (#73) - LLM Evaluation
- Learn AI Series (#74) - The Hugging Face Ecosystem
- Learn AI Series (#75) - Multimodal Models - Text Meets Vision
- Learn AI Series (#76) - Mini Project - Your Own AI Assistant
- Learn AI Series (#77) - Image Processing Fundamentals
- Learn AI Series (#78) - Object Detection (Part 1) - Foundations
- Learn AI Series (#79) - Object Detection (Part 2) - Modern Approaches
- Learn AI Series (#80) - Image Segmentation
- Learn AI Series (#81) - Pose Estimation and Tracking
- Learn AI Series (#82) - Optical Character Recognition
- Learn AI Series (#83) - Video Understanding
- Learn AI Series (#84) - Generative Images - Diffusion Models (Part 1)
- Learn AI Series (#85) - Generative Images - Diffusion Models (Part 2)
- Learn AI Series (#86) - Image-to-Image and Editing
- Learn AI Series (#87) - 3D Vision
- Learn AI Series (#88) - Face Analysis
- Learn AI Series (#89) - Medical and Scientific Imaging
- Learn AI Series (#90) - Self-Supervised Learning for Vision
- Learn AI Series (#91) - Mini Project - Building a Visual AI System
- Learn AI Series (#92) - Audio Fundamentals for AI
- Learn AI Series (#93) - Speech Recognition
- Learn AI Series (#94) - Text-to-Speech (TTS)
- Learn AI Series (#95) - Audio Classification
- Learn AI Series (#96) - Music Generation
- Learn AI Series (#97) - Speaker Recognition and Diarization
- Learn AI Series (#98) - Natural Language Understanding for Voice
- Learn AI Series (#99) - Audio Enhancement
- Learn AI Series (#100) - Multimodal Audio-Visual Models
- Learn AI Series (#101) - Mini Project: Voice-Controlled AI Assistant
- Learn AI Series (#102) - What Is Reinforcement Learning?
- Learn AI Series (#103) - Multi-Armed Bandits
- Learn AI Series (#104) - Dynamic Programming
- Learn AI Series (#105) - Monte Carlo Methods
- Learn AI Series (#106) - Temporal Difference Learning
- Learn AI Series (#107) - Deep Q-Networks (DQN)
- Learn AI Series (#108) - Policy Gradient Methods
- Learn AI Series (#109) - Advanced Policy Optimization
- Learn AI Series (#110) - Model-Based Reinforcement Learning
- Learn AI Series (#111) - Multi-Agent Reinforcement Learning
- Learn AI Series (#112) - RL for Games
- Learn AI Series (#113) - RL for Real-World Applications
- Learn AI Series (#114) - Inverse Reinforcement Learning
- Learn AI Series (#115) - Offline Reinforcement Learning
- Learn AI Series (#116) - Mini Project: Training a Game-Playing AI
- Learn AI Series (#117) - ML System Design
- Learn AI Series (#118) - Data Engineering for AI
- Learn AI Series (#119) - Experiment Tracking and Reproducibility
- Learn AI Series (#120) - Model Optimization: Making Models Fast
- Learn AI Series (#121) - Model Serving Architecture
- Learn AI Series (#122) - Edge AI: Running Models on Devices
- Learn AI Series (#123) - Monitoring ML in Production
- Learn AI Series (#124) - CI/CD for Machine Learning
- Learn AI Series (#125) - GPU Programming Basics
- Learn AI Series (#126) - Distributed Training
- Learn AI Series (#127) - AI Security
- Learn AI Series (#128) - Privacy-Preserving AI
- Learn AI Series (#129) - AutoML and Neural Architecture Search
- Learn AI Series (#130) - Causal Inference and ML
- Learn AI Series (#131) - Graph Neural Networks
- Learn AI Series (#132) - AI for Structured Data
- Learn AI Series (#133) - Synthetic Data Generation
- Learn AI Series (#134) - AI Infrastructure Economics
- Learn AI Series (#135) - Building AI Teams and Processes
- Learn AI Series (#136) - Mini Project: Production AI Platform
- Learn AI Series (#137) - Foundation Models
- Learn AI Series (#138) - Multimodal AI
- Learn AI Series (#139) - AI for Code
- Learn AI Series (#140) - Scientific AI (this post)
Learn AI Series (#140) - Scientific AI
Last episode I left you with a deliberate cliffhanger. We had spent #139 watching AI learn to write the language of machines -- a domain with instant, binary feedback (it runs or it does not). And I asked: what happens when you point that exact same machinery at domains where "correct" is not a passing test but a physical law, a chemical structure, a mathematical theorem? Where being right means matching the universe itself, and being wrong wastes a real laboratory's real month?
That is today. This is the episode where AI stops being a clever autocomplete and starts sitting alongside the telescope, the microscope and the particle accelerator as an instrument of discovery. And I mean that literally -- not as marketing. Let's dive right in ;-)
Solutions to episode #139's exercises
As always, we settle last time's homework before anything new. Episode #139 was AI for code, so the exercises were all about the machinery that makes code special: objective feedback.
Exercise 1 -- Break the evaluator. The point was to prove to yourself that "it parses" is a MUCH weaker guarantee than "it passes". Here I feed the evaluator three deliberately broken strings and watch each of the three failure branches fire.
# Solution 1: exercising all three failure branches of evaluate_generated_code (#139).
import ast
def evaluate_generated_code(code, test_cases):
results = {"syntactically_valid": False, "tests_passed": 0,
"tests_total": len(test_cases), "errors": []}
try:
ast.parse(code)
results["syntactically_valid"] = True
except SyntaxError as e:
results["errors"].append(f"SyntaxError: {e}")
return results
ns = {}
try:
exec(code, ns) # illustrative only -- sandbox in real life
except Exception as e:
results["errors"].append(f"Execution error: {e}")
return results
func = next((o for n, o in ns.items()
if callable(o) and not n.startswith("_")), None)
for inputs, expected in test_cases:
try:
if func(*inputs) == expected:
results["tests_passed"] += 1
else:
results["errors"].append(f"{inputs}: expected {expected}, got {func(*inputs)}")
except Exception as e:
results["errors"].append(f"{inputs}: {e}")
return results
tests = [((5,), 5), ((10,), 55)]
# (a) parses fine, but WRONG -- off-by-one in the range
wrong = ("def fib(n):\n a, b = 0, 1\n"
" for _ in range(1, n):\n a, b = b, a + b\n return a")
# (b) parses fine, but blows up at runtime
crashes = "def fib(n):\n return undefined_name + n"
# (c) does not even parse
broken = "def fib(n)\n return n"
for label, c in [("wrong", wrong), ("crashes", crashes), ("broken", broken)]:
r = evaluate_generated_code(c, tests)
print(label, "-> valid:", r["syntactically_valid"],
"passed:", r["tests_passed"], "| first error:", r["errors"][0])
The key insight in one sentence: a parser only checks GRAMMAR, so wrong sails through ast.parse and still fails the tests, crashes reaches exec and dies there, and only broken trips the syntax branch -- three different guarantees, three different failure points, and only the test run tells you the code is actually correct.
Exercise 2 -- Ground a review in an AST. Extend the analyzer to flag high-complexity functions and record their line numbers, then hand those precise facts to a model. The grounding is the whole point (this should ring a bell from RAG, #64).
# Solution 2: complexity-flagging analyzer that produces facts, not guesses.
import ast
class ReviewAnalyzer(ast.NodeVisitor):
def __init__(self, threshold=5):
self.threshold = threshold
self.functions = []
def visit_FunctionDef(self, node):
branches = sum(1 for n in ast.walk(node)
if isinstance(n, (ast.If, ast.For, ast.While,
ast.ExceptHandler, ast.With)))
complexity = branches + 1
self.functions.append({
"name": node.name, "line": node.lineno,
"complexity": complexity, "hot": complexity > self.threshold,
})
self.generic_visit(node)
def review_prompt(facts):
hot = [f for f in facts if f["hot"]]
listing = "\n".join(f"- {f['name']} (line {f['line']}, complexity {f['complexity']})"
for f in hot)
return (f"Review ONLY these high-complexity functions for bugs and simplification:\n"
f"{listing}\nGround your review in the fact that each exceeds our "
f"complexity budget of 5. Do not comment on anything else.")
src = ("def tiny(x):\n return x + 1\n"
"def gnarly(x):\n"
" if x < 0:\n return 0\n"
" if x > 0:\n for i in range(x):\n"
" while i > 0:\n if i % 2: i -= 1\n"
" else: i -= 2\n return x\n")
a = ReviewAnalyzer()
a.visit(ast.parse(src))
for f in a.functions:
print(f"{f['name']}: line {f['line']}, complexity {f['complexity']}, hot={f['hot']}")
print("---\n" + review_prompt(a.functions))
Notice what happened: the AST supplies facts that are TRUE by construction (it cannot invent a function that isn't there), and the model only ever sees the functions worth its attention. Facts from the parser, judgement from the model -- the pattern I told you to remember.
Exercise 3 -- Design a self-repair budget. The naive loop retries a fixed number of times. A deployable one knows when to give up, when to change strategy instead of tweaking, and where the hard ceiling lives so an agent can't burn tokens till the heat death of the universe.
# Solution 3: a self-repair loop with a real stopping policy.
def smart_self_repair(spec, tests, generate_fn, max_attempts=5, hard_token_cap=8000):
feedback, tokens_spent, last_pass = None, 0, -1
for attempt in range(1, max_attempts + 1):
code, cost = generate_fn(spec, feedback) # real call returns (code, tokens)
tokens_spent += cost
result = evaluate_generated_code(code, tests)
passed = result["tests_passed"]
if passed == result["tests_total"]:
return {"code": code, "attempt": attempt, "tokens": tokens_spent}
# Give up early if we are making ZERO progress (same or fewer passes).
if passed <= last_pass:
feedback = {"errors": result["errors"], "hint": "previous approach stalled; "
"try a DIFFERENT algorithm, not a patch"}
else:
feedback = {"errors": result["errors"]} # progress -> keep refining
last_pass = max(last_pass, passed)
# Hard ceiling so an agent cannot loop forever.
if tokens_spent > hard_token_cap:
return {"code": None, "reason": "token budget exhausted", "tokens": tokens_spent}
return {"code": None, "reason": "max attempts", "tokens": tokens_spent}
My reasoning: refining works when you are CLOSER than last time (more tests green), so I keep the feedback narrow then. When passes stall or regress, tweaking the same broken idea is a trap -- so I switch the hint to "try a different approach entirely". And regardless of cleverness, a hard token cap sits underneath everything, because an autonomous agent with no ceiling is a bill with no ceiling. That gap between a toy loop and this one IS the gap between a demo and something you would actually deploy.
Right -- homework settled. On to the universe ;-)
AlphaFold: the breakthrough that changed the mood
In December 2020, DeepMind announced that AlphaFold had essentially solved protein folding -- a grand challenge biologists had chewed on for over 50 years. Within two years it had predicted the structure of essentially every known protein: over 200 MILLION structures, released free to the world. For context, all of experimental science, over half a century of X-ray crystallography and cryo-EM, had produced roughly 170,000.
Here is the problem in one breath. A protein arrives as a sequence of amino acids -- a string over a 20-letter alphabet. That string folds itself into a precise 3D shape, and the SHAPE determines the FUNCTION (enzyme, receptor, structural strut). Get the structure and you unlock drug design, disease understanding, bioengineering. Before AlphaFold, getting one structure took months to years and cost thousands to millions.
What makes this so satisfying to teach is that AlphaFold is not alien technology. It is OUR technology, the exact machinery we've been assembling since episode #1:
Input as evolutionary context. The target sequence is bundled with its evolutionary relatives into a Multiple Sequence Alignment (MSA). Amino acids that mutated together across millions of years of evolution tend to be physically close in the folded shape -- co-evolution leaks structural information, and the MSA is how you feed it in.
Attention, but biological. AlphaFold's "Evoformer" runs attention across TWO axes: along the MSA (different relatives) and across residue pairs (relationships between positions). It is cross-attention (#51, #138) wearing a lab coat.
Structure plus its own uncertainty. The final module predicts 3D coordinates AND a per-residue confidence that correlates startlingly well with real accuracy -- the model knows when it does not know.
import torch
import torch.nn as nn
class SimplifiedEvoformer(nn.Module):
"""A toy slice of AlphaFold's Evoformer.
The real thing has row attention, column attention, triangular
updates and outer-product means. This keeps only the core idea:
attention over sequence relationships, feeding a pair representation.
"""
def __init__(self, seq_dim=256, pair_dim=128, n_heads=8):
super().__init__()
self.row_attn = nn.MultiheadAttention(seq_dim, n_heads, batch_first=True)
self.pair_proj = nn.Sequential(
nn.Linear(pair_dim, pair_dim), nn.ReLU(),
nn.Linear(pair_dim, pair_dim),
)
self.norm1 = nn.LayerNorm(seq_dim)
self.norm2 = nn.LayerNorm(pair_dim)
def forward(self, msa_repr, pair_repr):
batch, n_seq, seq_len, dim = msa_repr.shape
flat = msa_repr.reshape(batch * n_seq, seq_len, dim)
attn_out, _ = self.row_attn(flat, flat, flat)
msa_repr = self.norm1(msa_repr + attn_out.reshape(batch, n_seq, seq_len, dim))
pair_repr = self.norm2(pair_repr + self.pair_proj(pair_repr))
return msa_repr, pair_repr
evoformer = SimplifiedEvoformer()
msa = torch.randn(1, 4, 50, 256) # 1 batch, 4 relatives, 50 residues, 256-dim
pair = torch.randn(1, 50, 50, 128) # pairwise features over 50x50 residues
msa_out, pair_out = evoformer(msa, pair)
print(f"MSA: {tuple(msa_out.shape)}, Pair: {tuple(pair_out.shape)}")
The follow-up, AlphaFold 3, pushed into protein-ligand, protein-DNA and protein-RNA complexes -- the interactions that underlie basically all of biology. Nota bene: this is the single result that shifted the mood of an entire field. After AlphaFold, "AI for science" stopped sounding like a pitch deck.
AI for drug discovery: compressing a decade
Structure prediction is one rung. The full drug pipeline traditionally runs 10-15 years and north of a billion dollars per approved drug. AI is squeezing several stages of that -- though, as you'll see, not the stage you might hope.
Target identification. Which protein, nudged, would treat the disease? Models chew through gene-expression data, protein-interaction networks (#131, graph neural networks) and mountains of literature to nominate candidates.
Molecular generation. Given a target, design a molecule that binds it. A molecule IS a graph -- atoms are nodes, bonds are edges -- so this is GNN territory. Predict properties (binding affinity, toxicity, solubility) from the graph, then generate novel graphs optimised for the properties you want.
# Molecular property prediction with a bare-bones message-passing GNN.
# In production you'd reach for PyTorch Geometric or DGL and real chemistry.
class MoleculeGNN(nn.Module):
"""Predict a molecular property from atoms + bond connectivity."""
def __init__(self, atom_features=32, hidden=64, output=1):
super().__init__()
self.atom_embed = nn.Linear(atom_features, hidden)
self.conv1 = nn.Linear(hidden, hidden)
self.conv2 = nn.Linear(hidden, hidden)
self.readout = nn.Linear(hidden, output)
def forward(self, atom_feats, adj):
h = torch.relu(self.atom_embed(atom_feats))
h = torch.relu(self.conv1(adj @ h)) # pull in neighbour info
h = torch.relu(self.conv2(adj @ h)) # two hops of message passing
graph_repr = h.mean(dim=0, keepdim=True) # readout over the whole molecule
return self.readout(graph_repr)
n_atoms = 9 # ethanol-ish: 2C + 6H + 1O
atom_feats = torch.randn(n_atoms, 32)
adj = torch.zeros(n_atoms, n_atoms)
adj[0, 1] = adj[1, 0] = 1 # C-C
adj[1, 8] = adj[8, 1] = 1 # C-O
pred = MoleculeGNN()(atom_feats, adj)
print(f"predicted property: {pred.item():.4f}")
Virtual screening. Score millions of candidate molecules and rank the promising ones BEFORE a single test tube gets touched. Classical screening simulates the physics of docking; learned representations do it roughly 1000x faster at comparable accuracy, shrinking the search space by orders of magnitude.
ADMET prediction. Absorption, Distribution, Metabolism, Excretion, Toxicity -- the unglamorous properties that kill most candidates. Models trained on historical ADMET data filter the doomed molecules out before expensive animal work.
Now the honest part, because I promised you honesty. AI has genuinely accelerated EARLY drug discovery. But as of early 2026, no AI-designed drug has completed clinical trials and reached patients. The bottleneck did not vanish -- it MOVED. It slid from molecular design (fast now) to clinical validation (still slow, because you are testing on actual humans and biology refuses to be rushed). AI compresses years of lab work into months. It cannot compress the years of trials that follow. Keep that distinction sharp -- it is exactly where the hype gets sloppy.
AI for mathematics: can a pattern-matcher prove things?
Maths feels like the LAST place a statistical model should thrive -- it is the summit of formal reasoning, and our models are, at heart, glorified next-token predictors. And yet.
Theorem proving. Formal provers (Lean 4, Isabelle, Coq) verify proofs with absolute certainty. The hard bit is WRITING them, which needs creativity and taste. Models are learning to generate proof steps -- DeepMind's AlphaProof solved several International Mathematical Olympiad problems by pairing a language model with an RL loop that searched proof strategies (a search that rhymes with the game-playing back in #112).
Conjecture generation. Before you prove something you must first GUESS it. Models have surfaced genuinely new relationships by spotting patterns in mathematical data -- Google's collaborations with mathematicians turned up publishable conjectures in knot theory and representation theory.
Symbolic computation. Transformers trained on expressions can integrate, solve differential equations and simplify -- treating manipulation as sequence-to-sequence translation, the same seq2seq idea from #50.
# Symbolic integration as seq2seq translation -- the conceptual shape.
integration_examples = [
("x^2", "x^3/3"),
("sin(x)", "-cos(x)"),
("e^x", "e^x"),
("1/x", "ln(x)"),
("x*e^x", "e^x*(x - 1)"),
]
# The model learns integration as a token transformation:
# input tokens: ['x', '^', '2']
# output tokens: ['x', '^', '3', '/', '3']
# Lample & Charton (2019) trained a transformer on millions of such pairs and
# beat Mathematica on some classes of symbolic integration. Yes, really.
print("integration as translation:")
for expr, integral in integration_examples:
print(f" integral({expr}) dx = {integral} + C")
The philosophical needle to thread: is the model "doing maths"? It does not understand a proof the way a human mathematician does. But its output is VERIFIABLE -- once a formal prover checks a generated proof, that proof is as valid as any human's, full stop. The process differs; the result is identical. I find that a weirdly comforting kind of honesty -- the machine does not need to understand, because the proof checker does the trusting for us.
Weather forecasting: beating physics with data
For decades, forecasting meant Numerical Weather Prediction (NWP) -- gigantic simulations solving the equations of atmospheric physics on a 3D grid of the planet, running for hours on some of the largest supercomputers we own. In 2022-2023 that quietly got upended. GraphCast (DeepMind) and Pangu-Weather (Huawei) posted LOWER forecast error than the ECMWF gold standard, while running roughly 1000x faster -- on a single GPU.
GraphCast treats the atmosphere as a graph (of course it does). Grid points are nodes, nearby points share edges, and each node carries temperature, humidity, pressure and wind at many altitudes. The task: given the current state (plus the previous one, for momentum), predict the state 6 hours out. Chain those steps and you get a ten-day forecast.
# The conceptual skeleton of a neural weather model.
# Real GraphCast uses ~37 atmospheric levels and about a million grid points.
class SimpleWeatherPredictor(nn.Module):
"""Predict the next atmospheric state from current + previous."""
def __init__(self, n_features=6, hidden=128, n_layers=4):
super().__init__()
self.encoder = nn.Linear(n_features * 2, hidden) # current ++ previous
self.layers = nn.ModuleList([
nn.Sequential(nn.Linear(hidden, hidden), nn.ReLU(), nn.LayerNorm(hidden))
for _ in range(n_layers)
])
self.decoder = nn.Linear(hidden, n_features)
def forward(self, current, previous):
x = torch.cat([current, previous], dim=-1)
h = torch.relu(self.encoder(x))
for layer in self.layers:
h = h + layer(h) # residual stacks (#52-53 habit)
delta = self.decoder(h) # predict the CHANGE, not the absolute
return current + delta # next = current + predicted change
n_points, n_features = 1000, 6 # real: ~1 million points
current = torch.randn(1, n_points, n_features)
previous = torch.randn(1, n_points, n_features)
next_state = SimpleWeatherPredictor()(current, previous)
print(f"predicted next state: {tuple(next_state.shape)}")
WHY does this even work? The physics models solve the actual equations from first principles -- they KNOW Navier-Stokes. The neural models know nothing of the sort. They have simply seen so many "this state led to that state six hours later" examples that they learned the dynamics empirically. Decades of historical weather, distilled into a network that never once read a physics textbook.
The upside is speed, and speed changes what is POSSIBLE: a ten-day global forecast in under a minute means you can run hundreds of them with jittered initial conditions (ensemble forecasting) to estimate uncertainty. The downside is the classic one -- these models learned from the PAST. Unprecedented, climate-driven events sit outside their training distribution, and there the physics models, which encode the real laws, may prove more robust. The grown-up future is almost certainly HYBRID: neural nets for fast routine forecasting, physics for the extreme and the never-before-seen.
Materials science: inventing what never existed
Discovering materials with desired properties -- stronger, lighter, more conductive, cheaper -- has forever meant expensive trial-and-error. AI is making it systematic.
Crystal structure prediction. Given a chemical composition, predict the most stable crystal arrangement (protein folding's inorganic cousin). GNoME (Google DeepMind, 2023) predicted 2.2 MILLION new stable crystals -- an order of magnitude more than humanity had found in its entire history. Hundreds have since been synthesised for real, confirming the calls.
Property prediction. Structure in, properties out: band gap for semiconductors, thermal conductivity, mechanical strength, melting point. GNNs on crystal graphs (atoms as nodes, bonds as edges) are the workhorse -- the SAME architecture you just saw predict a drug molecule's binding, pointed at a lattice instead.
Inverse design. Run it backwards -- desired properties in, a structure that achieves them out. This is generative modelling (#55, #84-85) applied to matter, the direct analogue of generating a drug molecule to hit a target.
# One architecture, two sciences: a crystal graph feeds the SAME GNN as a molecule.
# atoms -> nodes, bonds -> edges, readout -> a bulk material property.
crystal_atoms = torch.randn(12, 32) # 12 atoms in the unit cell
crystal_adj = torch.eye(12) # placeholder lattice connectivity
crystal_adj[0, 1] = crystal_adj[1, 0] = 1
band_gap = MoleculeGNN(atom_features=32)(crystal_atoms, crystal_adj)
print(f"predicted bulk property (e.g. band gap): {band_gap.item():.4f}")
That reuse is not a coincidence, and it is the whole lesson of this episode -- but let me make it explicit in a second.
AI as tool versus AI as scientist
There is a distinction here worth nailing down, because sloppy thinking about it produces both breathless hype and lazy dismissal.
AI as tool. The scientist frames the problem, designs the experiment, interprets the result. AI accelerates specific steps -- predict a structure, screen candidates, crunch data -- but the scientific JUDGEMENT stays human. This is where we live today, and it is already transformative.
AI as scientist. The system forms hypotheses, designs experiments to test them, reads the results and iterates -- the full autonomous-discovery loop. Pieces exist (automated labs running AI-suggested experiments), but the complete loop, especially the interpretation, is still beyond us.
The gap between those two is ENORMOUS, and here is the crux. Current AI can predict that a molecule binds a protein. It cannot tell you WHY in a way that advances fundamental understanding. AlphaFold predicts structures with astonishing accuracy -- yet it has not handed us new physics of WHY proteins fold. The prediction is superhuman; the understanding remains stubbornly human. That is not a criticism (a tool this powerful needs no apology), but be precise about what "AI for science" means right now: these are magnificent prediction machines, not yet discovery machines in the deepest sense.
What actually connects it all
Step back and squint across the whole landscape -- folding, drugs, maths, weather, materials -- and the same handful of techniques from THIS series keeps reappearing:
- Transformers (#52-53) for sequence processing and attention over tangled relationships;
- Graph neural networks (#131) for molecules, proteins, crystals AND atmospheric grids;
- Self-supervised pre-training (#90, #137) on oceans of unlabelled scientific data;
- Reinforcement learning (#102-116) for searching over proof strategies and molecular designs;
- Generative models (#55, #84-85) for proposing new molecules, materials and structures.
The architectures do not change. The training principles do not change. What changes is the DATA and the domain expertise needed to frame the problem correctly. This is precisely why understanding the fundamentals -- the thing this series has been drilling since episode #1 -- matters so much. The techniques transfer. The domain is the variable. Master the toolkit and every science becomes a place you can point it.
Exercises
Before the next episode, get your hands into it. Three tasks, climbing:
Watch confidence and shape interact. Extend
SimplifiedEvoformersoforwardalso returns a per-residue confidence vector of lengthseq_len(a smallnn.Linear(seq_dim, 1)head on the MSA output, averaged over the relatives, squashed withsigmoid). Print its shape and confirm it matches the number of residues. Then write one sentence on why a model reporting its OWN uncertainty is more useful to a scientist than one that only reports an answer.One GNN, two sciences. Take
MoleculeGNNand feed it (a) a small molecule graph and (b) a crystal-cell graph of a different atom count, WITHOUT changing the model. Confirm both produce a single scalar. Then, in a comment, explain what has to be true about the readout step for the same architecture to swallow graphs of different sizes -- and connect it back to why "atoms as nodes" generalises across chemistry and materials.Draw the tool/scientist line yourself. Pick any ONE system from this episode (AlphaFold, a drug-screening model, AlphaProof, GraphCast, GNoME). In a short paragraph, argue exactly where it sits on the tool-versus-scientist spectrum: what human judgement does it still depend on, and what would have to change for it to cross into autonomous discovery? Be specific -- "it would need to understand" is not an argument until you say what understanding would let it DO.
We'll open the next episode with full solutions, as always.
Quick recap
- AlphaFold cracked protein structure prediction using attention over evolutionary sequences and pairwise residue relationships -- 200M+ structures that experiments would take centuries to determine -- and it did it with OUR toolkit (attention, cross-attention, uncertainty);
- AI drug discovery accelerates target ID, molecular generation, virtual screening and ADMET, but no AI-designed drug has cleared clinical trials -- the bottleneck moved from chemistry to human biology, which will not be rushed;
- AI for maths generates proofs and conjectures (AlphaProof solving olympiad problems via RL-guided search), and the results are trustworthy precisely because a formal prover checks them;
- neural weather models (GraphCast, Pangu-Weather) out-predict physics simulations at 1000x the speed, though they wobble on unprecedented, out-of-distribution conditions -- the future is hybrid;
- materials science uses GNNs on crystal graphs to predict structures and properties, with GNoME turning up 2.2 million new stable crystals -- the SAME GNN that scores drug molecules;
- the honest frame is AI as tool (accelerating human science) not yet AI as scientist (autonomous discovery) -- and the deep lesson is that the same five techniques from this series power every one of these domains. The toolkit transfers; the science is the variable.
And here is the thread for next time. We've watched AI reason about molecules, proofs and weather -- but always as a mind in a box, reading numbers and emitting numbers. What happens when you rip it OUT of the box and bolt it into something with motors, sensors and a body that has to survive contact with the messy physical world -- where a wrong prediction does not waste a lab month but tips over a very expensive robot? That is where we head next ;-)