The engine and its architectures

The core engine

The foundation lives in four folders: Core/ holds the tensor primitives (DenseTensor<T> and SIMD helpers built on System.Numerics.Tensors), Model/ the Layer and its FASTER-KV store, Networks/ the Network engine, Training/ the epoch loop. Each layer's state is a handful of tensors — the latent Mu, the generative weights and bias, the prediction and its error — persisted in a FASTER-KV store keyed by layer index, with lazy writes flushed once per epoch for CIFAR-scale runs.

The layer layout is fixed by convention: layer 0 is the sensory layer, clamped to the input; the top layer is the label, clamped only during training and prediction; everything between is free hidden state. The engine is thread-safe via a reader–writer lock: inference takes a read lock, learning and energy-based prediction take a write lock.

var cfg = new NetworkConfig { LayerSizes = [784, 256, 10], Path = "./data" };
using var net = new Network(cfg, new Random(42));
net.Learn(image, oneHotLabel, inferenceSteps: 20);   // per sample
int cls = net.Predict(image, 20);                    // argmin-energy class

That is the whole API surface for a classifier. Everything else in this page is built from it.

The advanced architectures

Each module below is a small IDisposable wrapper with a paired immutable config record, composed around the core Network:

Module	Idea
HierarchicalNetwork	A deep modular hierarchy: a stack of autonomous generative PCNs where each module encodes the latent of the one below — greedy layer-wise learning, like a hierarchical VAE without backprop.
ActiveNetwork	Active inference: a world-model PCN predicts the next observation from [obs ⊕ action]; a policy picks the action that minimises expected free energy toward a goal (Rao 2024; Friston 2017).
DeepRecurrentNetwork	A cascade of temporal PC stages; each stage carries its own μ^{t-1} and feeds its latent up the stack.
GatedStateNetwork	LSTM-style per-unit write gates over a working-memory cell, injected as the top-layer prior — surprise-modulated retention.
HopfieldNetwork	A generative PCN plus a modern (continuous) Hopfield attractor over the latent (Ramsauer 2021): pattern completion from partial cues.
GraphPredictiveNetwork	Predictive coding over arbitrary graphs: a shared edge-predictor learns node → neighbour; inference is message passing by error minimisation.
HierarchicalMixtureOfExperts	A two-level energy-gated MoE (Jordan & Jacobs 1994): within-group and across-group softmax over free energies, with emergent specialisation and no supervision.
SpikingNetwork	A self-contained LIF spiking PCN: rate-coded states, errors as firing-rate differences, local Hebbian learning — the neuromorphic reading of the same rule.

Memory, modality and routing

Module	Idea
GenerativeNetwork	A VAE-like PCN used in both directions: Encode (recognition), Decode (generation), Sample; a Gaussian top-layer prior plays the KL role.
MemoryNetwork + MemoryBank	A hippocampal episodic prior: a cosine-similarity key-value store that biases the top-layer prior before inference.
MultiModalNetwork	N modality encoders into a shared associative PCN; cross-modal completion (hear the bell, see the bell).
RoutedNetwork	A learned router PCN blended with per-sample free-energy gating — amortised mixture of experts.

A few networks that started in the core were deliberately moved out to keep the core minimal: TemporalNetwork (Friston 2008 temporal continuity), MixtureOfExperts, TemporalMixtureOfExperts, EnsembleNetwork (Bayesian model averaging over PCN members), AttentiveNetwork (per-unit learnable attention gates on the errors) and ResidualNetwork (two-hop skip connections in the PC hierarchy). They now live in PredictiveCoding.Expanded and the core stays dependency-free.

Energy as the universal currency

What makes the composition work is that every module speaks the same language: settled free energy. The mixture-of-experts gate is a softmax over negative energies, so the expert that explains the input best takes the update:

g_i = softmax(−β · F_i)          i* = argmin_i  min_c  F_i(x, one-hot(c))

Experts specialise purely through differential free-energy minimisation — no learned router, no supervision. The same gating helper is reused by the cortex's Thalamus (precision allocation across areas) and BasalGanglia (go/no-go action selection), which is the kind of reuse that only works because the primitive — a number that means "how surprised is this model" — is shared everywhere.

Two worked examples

Modern Hopfield pattern completion — store prototypes, recall from a noisy cue:

using var net = new HopfieldNetwork(new HopfieldNetworkConfig {
    NetworkConfig = new NetworkConfig { LayerSizes = [144, 96, 48] },
    RetrievalBeta = 8f, RetrievalIterations = 5, BasePath = "./hopfield",
}, new Random(0));

foreach (var p in prototypes) { net.Learn(p, 20); net.Store(p, 20); }
var clean = net.Recall(noisyCue, 20);   // snap to nearest attractor, then decode

Active inference — learn a world model, then act to reach a goal:

var cfg = new ActiveNetworkConfig {
    ObservationDim = 2, ActionCount = 4,
    WorldModelConfig = new NetworkConfig { LayerSizes = [6, 16, 2] },
};
using var agent = new ActiveNetwork(cfg, new Random(0));
agent.Observe(state, action, nextState, 12);          // learn a transition
int a = agent.Act(state, goal, 12);                   // minimise expected free energy

What holds it together

• Locality — core learning is Hebbian; nothing propagates gradients through the PC graph.
• Composition — higher modules wrap lower ones; the cortical areas in the next page are mostly these networks wired together.
• Dependency-free core — the engine needs no framework and falls back to plain CPU SIMD everywhere.