Model architecture
Every DendriNet model has three parts: an encoder that prepares the input, a
core network that does the computation, and a decoder that maps the core
output to the task output. This page describes the three parts, the model
wrapper that is built around them for each task, the families of core
networks, the model.core.biological_neuron declaration and the output scale
of classifiers. Each core family has its own page, linked from
Core families.
Encoder, core and decoder
The model section of a configuration names the three parts:
Part |
Option (default) |
Types |
|---|---|---|
Encoder |
|
|
Core |
|
A dendritic E/I network or a baseline network; see Core families. |
Decoder |
|
|
Encoder and decoder types lists the parameters each encoder and decoder type reads. The sizes of the three parts are connected automatically:
The training command sets
model.encoder.params.input_dimandinput_shapefrom the first training sample:input_dimis the number of input elements for a feedforward core and the number of features per time step for a recurrent core. Values written in the configuration are replaced.The core is built for the output size of the encoder.
The decoder’s
params.input_dimis set to the core output size. An identity decoder has no parameters; aparams.output_dimwritten for it must equal the core output size.For classification, the number of outputs must cover the training labels: a label beyond the last output raises, and outputs that no label uses only produce a warning.
A complete model section with a population-network core and a linear
readout:
model:
task: classification
encoder:
type: identity
core:
type: population_network
population_network:
input_transform: nonnegative
layers:
- name: hidden
readout_population: pyr
population_defaults:
ff_excitatory_synapses: 16
ff_inhibitory_synapses: 4
populations:
- {name: pyr, polarity: excitatory, n_neurons: 32, branch_factors: [2, 2]}
- {name: pv, polarity: inhibitory, n_neurons: 8, branch_factors: [1]}
connections:
- {source: input, target: pyr}
- {source: input, target: pv}
- {source: pv, target: pyr}
decoder:
type: MLP
params:
hidden_dims: [] # linear readout
output_dim: 10
Tasks and model wrappers
model.task is classification (the default) or regression. The
model wrapper, from dendritic_modeling.models, depends on the task and on
whether the core is recurrent:
Core |
|
|
|---|---|---|
Feedforward |
|
|
Recurrent |
|
|
Recurrent cores read input sequences shaped [batch, time, features]; an
encoder other than identity is applied at every time step. A core is
recurrent when:
its type is a recurrent baseline:
gru,lstm,vanilla_rnn(aliasrnn),heterogeneous_leak_ctrnn(aliasheterogeneous_ctrnn) orlegendre_memory(Baselines and comparison models);it is a population network with a recurrent layer, a delayed connection, a population with temporal state (spiking soma, receptor synapse types, dendritic spikes or soma feedback) or
output_mode: meanorall(Population networks, Temporal dynamics);it is a unified E/I core with a recurrent layer or recurrent synapses (Unified E/I core);
it is a structured E/I core with recurrent synapse counts or a stateful feature (Recurrent structured cores).
The training loss is training.main.common.loss_function (default ce,
cross-entropy); see Loss functions.
Starting from saved weights
model.initial_checkpoint names a checkpoint file whose weights replace the
initial weights of the built model before training starts:
model:
initial_checkpoint: results/previous_run/final_model.pt
initial_checkpoint_sha256: null # optional: the file's SHA-256, checked before loading
initial_checkpoint_strict: true # missing or unexpected keys raise
The file is read with PyTorch’s weights-only loader. It may hold the model’s
state dictionary itself or a mapping with that dictionary under
model_state_dict, state_dict or model, such as the checkpoints that
training writes; a module. prefix from DDP is removed. The model must be
built from a configuration that matches the saved weights. With
initial_checkpoint_strict: false missing and unexpected keys are recorded
instead of raising. The run writes initial_checkpoint_provenance.json with
the resolved path, the file’s SHA-256 and the keys that did not match
(Run directory layout). Only the weights are loaded: the optimizer state and the
epoch counter start fresh (Resuming from an epoch boundary continues a run instead).
Core families
The dendritic cores are built from the same parts: dendritic neurons (Dendritic neurons) that receive sparse excitatory and inhibitory synapses (Synapses and connectivity), optionally with receptor kinetics, spiking somas, dendritic plateaus and soma-to-dendrite feedback (Temporal dynamics). They differ in how populations of these neurons are declared and wired:
Family |
Description |
|---|---|
Population network |
|
Structured E/I core |
|
Unified E/I core |
|
Spatial cores |
Dendritic cores applied to image feature maps, as a flattened map or as a shared core on local patches, and an AlexNet-shaped dendritic network. See Embedded and spatial cores. |
Baselines and controls |
Point-neuron MLPs, dense or sparse affine MLPs, parameter-matched controls and recurrent baselines. See Baselines and comparison models. |
Core types lists every model.core.type with its aliases.
A population network reads only model.core.population_network, besides
type, initialization_seed and biological_neuron; the same holds for
unified_ei and the recurrent baselines with their own sections. Writing
another model.core section with a non-default value for such a core, for
example model.core.morphology, raises when the configuration loads.
Population networks and unified E/I cores set their neuron and synapse options
as population fields (for population networks in population_defaults or a
population’s population block), where structured E/I cores use the
model.core sections. The tables in Dendritic neurons and Synapses and connectivity give both
spellings of each option.
The figure shows a feedforward structured E/I core inside the model.
A feedforward structured E/I core. TransferLayer turns the encoder
output into an excitatory stream x_E and an inhibitory stream x_I.
In each ExcitationInhibitionLayer the inhibitory cells run first, on
both streams; the excitatory cells then read the excitatory stream and the
output of the inhibitory cells (or x_I itself when the layer has no
inhibitory cells). The core output is the output of the last excitatory
population.
Biological constraints
model.core.biological_neuron: true declares that a core is a biologically
constrained E/I network. The declaration is checked on the built model, so
type aliases, defaults and per-population settings are all taken into account,
and building fails with a list of every broken rule. With configuration format
4, this is the default for standard dendritic E/I cores. The explicitly signed
and MLP aliases, and other model families, keep their conventional contracts.
Set biological_neuron: false explicitly for non-biological exploration;
this disables the constraint check without changing the integration equations.
The declaration is an operational constraint set for a rate or spike model,
not a claim of complete biological realism. A core satisfies it when:
Synaptic weights are nonnegative. Every synapse bank uses a nonnegative weight transform (
exp,softplusorrelu; see Weight transforms), and the core has no layer with unconstrained weights: no signed input projection, noinhibitory_network_type: mlpand no soma feedback. The*_signedand*_mlpcore types use signed weights and cannot satisfy the set.Excitation and inhibition stay separate (Dale’s law). No synapse bank merges excitatory and inhibitory inputs, and every source acts with one sign. An inhibitory pathway may read the network input only when the transfer layer splits it into separate excitatory and inhibitory channels (
independent_pathways: true), or explicitly declares distinct external afferents (input_source_identity: paired_afferents). The latter supplies the same feature-dependent rates to two separate afferent populations, one excitatory and one inhibitory. Dale’s law concerns source identity, not statistical independence. This assumption does not model the upstream inhibitory circuit and must be reported; there are twice as many external source identities as input features, with no added trainable parameters. The defaultinput_source_identity: sharedmakes no such declaration: duplicated channels cannot both excite and inhibit. Feedforward inhibition through inhibitory cells that read the excitatory stream needs no split. In a population network, each connection must use the pathway of its source’s polarity.The input is nonnegative. The core checks or rectifies its input before the first synapses:
transfer.output_activationisrelu,softplusorsigmoid; or a population network setsinput_transform: reluorsigned_split, or uses a transfer layer with one of these output activations. For already nonnegative data, useoutput_activation: nonnegativein the transfer layer orinput_transform: nonnegativein a population network. This checks finite, nonnegative values on every batch and returns the input unchanged, including its derivative at zero. ReLU also preserves nonnegative values, but changes the derivative at zero. Runtime checks synchronize the device once per batch, or once per sequence in a recurrent unroll, which reads the checks of all its steps together; they are not free. For signed features,signed_splitpreserves both polarities in separate nonnegative channels and doubles the input width. Neither encoding may be introduced silently into a historical experiment.Populations emit nonnegative activity. Each population outputs spikes, a spike rate or the output of its soma gate (Activation functions).
relu,softplus,sigmoid,param_relu,param_tanhandparam_tanh_only_mare nonnegative for every voltage.none(reactivate: false),endpoint_tanh,linear_sigmoid,linear_tanhand theirparam_forms are nonnegative only for a nonnegative voltage; they qualify when every level of the population integrates by shunting with such gates and no synapse type has a negative reversal potential, because shunting then keeps every voltage nonnegative. A spiking soma that reads out its membrane potential does not qualify.
The declaration validates a specified circuit; it does not choose an input
encoding or create inhibitory afferents. A bare ModelConfig() therefore
needs these choices before it can build. The reference configurations
configs/config_exp.yaml and configs/examples/population_network_feedforward.yaml
show biological cores with nonnegative image input and modeled inhibitory cells.
Only dendritic E/I cores can declare true: structured E/I, unified E/I and
population-network types. Other core types accept false and reject
true when the configuration loads. The check lives in
dendritic_modeling.networks.architectures.biological_constraints;
biological_violations(module) returns the rules that a built module breaks.
A training run with a dendritic E/I core records the declaration and that list
in the biological_neuron entry of operator_card.json. The check and the
record concern the neuronal core. A task encoder or signed classification
decoder outside it is not certified; report these components separately.
The constraint set concerns the forward neuronal model, not the biological
plausibility of a training algorithm or its signed error signals.
The operator card also records the external-afferent declaration.
Population and unified E/I networks can use
input_projection_weight_transform: softplus with input_projection_dims
to build a positive input projection followed by ReLU. Its equation is
y = relu(softplus(W_raw) x + b): x must be nonnegative, the weights are
positive, and b is an intrinsic offset, not a synaptic input. Weights start
uniformly between zero and inverse fan-in, and offsets start at zero. This
option changes the model and initialization; signed projection checkpoints
cannot be relabeled as positive projections. The historical identity
option still builds ordinary signed Linear layers and requires an explicit
non-biological declaration. The positive projection is supported through the
population-network and unified E/I builders with standard backpropagation.
Both accept input_transform: signed_split to retain both polarities of a
signed task observation as separate nonnegative channels before that projection.
The default input transform remains identity; changing to a split doubles
the input width and requires retraining. Unified E/I options live under
model.core.unified_ei and population options under population_network.
Analytic local updates reject trainable positive projections; a frozen projection
can instead supply fixed input features to those rules.
Before using an older configuration, run dendritic-upgrade-config. It writes
biological_neuron: false where an earlier format omitted the declaration,
preserving the historical validation policy and computation. Adoption is a
separate step: identify external sources, establish the input domain, then
enable the check and compare states, outputs and gradients. If that requires
changing an encoding, synaptic signs or circuitry, retrain and retain the old
result and its source pin. The flag alone does not convert a trained network.
Classifier output scale
A feedforward classifier multiplies its decoder output by
model.fixed_output_scale (default 1.0, finite and positive) whatever
the decoder type. With an identity decoder it can also learn a positive scale,
selected by model.output_scale_mode:
per_class: one learned scale per output; the default for an identity decoder;global: one learned scale for all outputs;fixed: no learned scale.
The learned scale is stored as a logarithm (log_output_scale or
log_global_output_scale) and trained with decoder_lr. Other decoders
learn no scale, so writing per_class or global for them raises when the
model is built. Recurrent classifiers and regression models apply no output
scale: they accept only output_scale_mode: fixed (or no value) with
fixed_output_scale: 1.0, and raise otherwise.
The retired key model.learned_output_scale still loads with a warning:
false becomes output_scale_mode: fixed, true keeps the default, and
a value that contradicts a written output_scale_mode raises.