Frontier agents are the visible top of a stack. Under them: machine learning. Under machine learning: classical AI. Under everything: estimation theory, probability, stochastic optimization and physics. Teams that only rent the top layer cannot explain, certify or fix what they ship. We build critical systems for governments, banks and portfolios - so we own every layer down to the mathematics.
Classical AIsearch, logic, constraints, planning - the algorithms that never stopped working
Mathematics & physicsestimation, probability 2 & 3, stochastic approximation, entropy, energy
Layer 0 · Estimation theory
Every model is an estimator
Training is not magic: it is the estimation of parameters from noisy data. Three rules carry the whole field.
θ̂MLE = argmaxθ ∑i log p(xi | θ)
Maximum likelihood: choose the parameters that make the observed data most probable. Least squares is exactly MLE under Gaussian noise - regression was probability all along.
θ̂MAP = argmaxθ [ log p(x|θ) + log p(θ) ]
Maximum a posteriori: add a prior. An L2 penalty is a Gaussian prior, an L1 penalty is a Laplace prior - "regularization" is Bayesian estimation in disguise.
E[(ŷ − y)²] = Bias² + Variance + σ²
Bias-variance decomposition: the irreducible law of every estimator. Model selection, ensembling and early stopping are all ways of trading these three terms.
Layer 0 · Probability 2 & 3
The probability that runs underneath
P(θ|x) = P(x|θ) P(θ) / P(x)
Bayes. Every posterior, every filter, every spam classifier, every belief update an agent makes.
X̄n → μ (a.s.) · √n (X̄n − μ) → N(0, σ²)
Law of large numbers + central limit theorem: why minibatch gradients work, why evaluation on a sample tells you anything about production.
P(|X̄n − μ| ≥ t) ≤ 2 exp(−2nt²)
Hoeffding concentration: the guarantee behind "n samples are enough". This is how you size a test set honestly.
At the measure-theoretic level (probabilité 3): a random variable is a measurable function, expectation is an integral, and "convergence" splits into almost-sure, in-probability and in-distribution. That precision is not academic - it is why stochastic gradient descent provably converges, and under which step-size conditions. The engineer who knows the convergence modes knows when to trust the training curve and when not to.
Cross-entropy loss is not a "choice": minimizing it is maximizing likelihood of a categorical model. Softmax is not a "trick": it is the Gibbs distribution of statistical physics. The deeper you go, the fewer arbitrary decisions remain.
Layer 0 · Stochastic optimization
How every modern model is actually trained
θt+1 = θt − ηt ∇̂L(θt) with ∑ηt = ∞, ∑ηt² < ∞
Robbins-Monro stochastic approximation (1951). SGD is a 70-year-old theorem, not a 2015 invention. The step-size conditions are the reason learning-rate schedules exist.
mt = β₁mt-1 + (1−β₁)gt · vt = β₂vt-1 + (1−β₂)gt²
Adam is literally estimation theory applied to optimization: exponential-moving-average estimators of the first and second moments of the gradient, bias-corrected. Estimation inside optimization inside learning.
Drawn, not decorated: contour lines of a quadratic loss bowl and a stochastic gradient path computed in your browser right now (noisy steps, decaying step size). Momentum turns this walk into a heavy ball rolling with inertia - a physics equation reused as an optimizer.
Layer 0 · Physics
Entropy and energy are load-bearing
S = −kB ∑ pi ln pi ↔ H = −∑ pi log₂ pi
Boltzmann's entropy and Shannon's entropy are the same functional. Information theory is thermodynamics with bits.
p(i) = e−Ei/T / Z
The Gibbs distribution. Set E = -logit and T = 1: you get softmax. Raise T: exploration. Anneal T toward 0: simulated annealing, the classical-AI search that escapes local optima.
Diffusion image generators - including the open model behind the DarwinX Studio - are a stochastic differential equation run backwards: noise is added by a known physical process, and a network is trained to estimate the reverse drift. Generation is literally estimation of a reverse-time SDE. When you know that, "AI art" stops being mysterious and becomes engineerable.
Langevin dynamics, heavy-ball momentum, annealing schedules: the optimizer zoo is applied physics. That is why physical intuition transfers so well to training behaviour.
Worked examples · curves computed in-page
Where the mathematics surfaces in ML
1 · Activation functions are probability made differentiable
Sigmoid is the logistic CDF - its output IS a Bernoulli probability, and its inverse is the log-odds. GELU gates by the Gaussian CDF Φ(x). Softmax is Gibbs. None of these shapes is arbitrary: each is a distribution function doing its job.
A tree chooses the split that maximizes information gain: the drop in entropy H(p) = −p log p − (1−p) log(1−p), or in Gini impurity 2p(1−p). Both curves peak at p = 0.5 (maximum disorder) and vanish at purity - the tree is doing thermodynamics on labels.
Entropy (scaled) and Gini impurity as functions of class probability p. The split criterion of CART and C4.5, drawn from their formulas.
3 · PCA is an eigendecomposition, and the scree curve is a spectrum
PCA diagonalizes the covariance matrix Σ = QΛQT and keeps the eigenvectors with the largest eigenvalues - the directions of maximal variance. Choosing "how many components" is reading a spectrum, exactly like a physicist reads energy levels.
A typical eigenvalue spectrum (bars) and cumulative explained variance (line). The elbow is where signal ends and noise begins.
4 · CNNs are a probabilistic prior about images
Convolution = weight sharing = the assumption that image statistics are translation-invariant. That is a prior, stated in architecture instead of in a formula - and it is why CNNs need orders of magnitude less data than dense networks. Training them is cross-entropy (negative log-likelihood) pushed through backpropagation (the chain rule), stepped by SGD (Robbins-Monro). Every piece is a named theorem.
Same mathematics · critical systems
We apply this where failure is not an option
The same estimation discipline runs our engineering for governments and finance: fiscal integrity chains, e-invoicing, core banking. Below, the real integration flow of a fiscal control capsule (NACEF) as homologated on 5,000+ points of sale in Tunisia.
Security & DevSecOpsLicensing, secure SDLC, secrets discipline, signed artefacts, homologation dossiers - the parts auditors actually read
GovernanceDelivery at three altitudes: project, program, government portfolio · sustainability/RSE as a pivot role inside teams
This is the point of the layer stack: the mathematics is not for papers. It is what lets one team ship a probabilistic model AND a fiscally-signed transaction chain AND defend both in front of an auditor.