Engineering architecture

This page is for developers who change the library. It maps the package, follows a training run from the YAML file to the run directory, and names the modules to read when tracing a bug or adding a component.

System overview. Configuration (example YAML, typed config, alias resolution, registries, factory) feeds the model modules (encoder: identity, pathway router or autoencoders; population core with biophysical options; decoder: MLP or identity) and the training, sweep and analysis stack (data loaders, trainers, sweeps, analysis manager, artifacts).

The configuration is loaded into typed settings, the factory builds the encoder, core and decoder through the registries, and the trainers, sweeps and analysis manager produce the run’s artifacts.

For the structure of the dendritic neurons themselves, see Model architecture and Dendritic neurons.

Package layout

The Python package dendritic_modeling lives in src/dendritic_modeling:

Subpackage

Contents

config

Typed configuration dataclasses, the strict loader (load_config, config_from_mapping), retired and renamed keys (keys.py), the rules for which model.core sections and encoder/decoder options each type reads (core_sections.py, io_params.py) and core-type alias semantics (model_aliases.py).

datasets

Vision, synthetic and sequence datasets, the dataset registry and the data-root resolution (paths.py).

networks.architectures

The architecture factory and registry, and the cores: excitation_inhibition (structured E/I networks, dendritic trees, sparse synapses and their kernels), recurrent (population networks, unified E/I networks, baseline recurrent networks), classical (MLP, CNN, autoencoder and pathway-router building blocks and the matched controls), vision (dendritic AlexNet), replacement (input and output adapters that place a dendritic core inside another network) and transformer (dendritic replacements of transformer feed-forward blocks).

networks (other)

Activation and reactivation-gate builders (activations), weight transforms, sparse-maintenance and readout helpers (utils), the parameter-budget model families (benchmarks) and checkpoint helpers (checkpoints.py).

models

The wrappers that combine encoder, core and decoder: Classifier, RecurrentClassifier and Regressor, with parameter groups for the optimizer.

training

The trainer factory and strategies (strategies; the local-learning implementation is split into mixins under strategies/local_learning_parts), optimizers, losses, regularization and pruning, data loaders and FSDP helpers.

analysis

The analysis manager (managers), the analyzer registry, and the analyzers (core, metrics, tools) with shared helpers (utils).

deployment

Compact export of trained models with frozen sparse topology (compression.py) and the optional torchao backends.

kernels

Triton kernels for indexed synapses (triton_ell.py).

plotting

Plotting helpers used by the analyzers.

scripts

The command-line programs: training, sweeps, analysis, deployment, tools (Slurm job writers, GPU memory estimates), benchmarks, verification and docs, with shared helpers in script_utils.

utils

Logging setup, seeds and reproducibility records, the runtime environment record, information-estimation utilities (utils/information) and general helpers.

Outside the package, the repository holds configs/ (example and reference YAML files; see Example configurations), docs/ (these pages), tests/, setup/ (environment file, constraints files and freeze_constraints.py) and scripts/train_fsdp.sh (a Slurm launcher for FSDP training).

How a training run executes

dendritic-train calls dendritic_modeling.scripts.training.train_experiments.main, which runs these steps:

  1. Load the configuration. dendritic_modeling.config.load_config applies the command-line options as dot-path overrides and builds the typed configuration. Unknown keys raise; retired and renamed keys produce a warning or an error that names the replacement (see How configs are checked). utils.training_stability_validator.validate_training_stability then rejects configurations with known critical problems.

  2. Start distributed training, when launched by torchrun, and check distributed.mode against the launcher.

  3. Set up the run (scripts.script_utils.setup_utils.setup_environment): resolve the seed streams and numerics settings, create the run directory and its log files, write resolved_seeds.json and runtime_environment.json, load the datasets (datasets.get_unified_datasets: registered datasets first, then the built-in loaders) and build the model.

  4. Build the model (setup_utils.initialize_model): the encoder, the core through networks.architectures.factory.get_architecture and the decoder, wrapped in a Classifier, RecurrentClassifier or Regressor. Structured E/I cores resolve their type aliases (config.model_aliases) while they are built.

  5. Record the start state: wrap the model for DDP when there are several processes, write init_gate_stats.json and config.json.

  6. Prepare training: create the analysis manager (analysis.AnalysisManager), calibrate data-driven reactivation gates (reactivation_calibration.json), create the optimizer from the model’s parameter groups and the trainer from training.factory.get_trainer. --validate-only stops here, after writing nparams, model_resources.json and operator_card.json.

  7. Train, then apply pruning when it is configured.

  8. Write the results: nparams, model_resources.json, operator_card.json, training_summary.json, final_model.pt, and the outputs of the final analysis.

Run directory layout describes every file. The FSDP entry point (train_experiments_fsdp) follows the same order with an FSDP-wrapped model and its own trainer.

Experiment lifecycle. Build and initialization: load YAML, resolve config, build model, gate calibration, data loaders. Training paths: standard backpropagation, local rules, recurrent backpropagation through time, distributed training, metrics. Analysis, sweeps and reproducibility: analysis runtime, analyzers, sweeps, run artifacts, comparisons of paired seeds and checkpoint summaries.

The resolved configuration drives model construction, gate initialization, training, analysis and the saved artifacts.

Feedforward and recurrent E/I layers

The feedforward and recurrent E/I layers use the same branch-level building blocks (DendriticBranchLayer and the initialization in dendritic/initialize*.py). The recurrent layers add temporal state and recurrent synapses; they do not change how a branch integrates its inputs.

One time step t of a recurrent unified E/I layer. The inhibitory population updates first from the excitatory stream x_ff, the inhibitory stream x_I and the previous-step outputs, and produces h_I(t). The excitatory population then reads x_ff, the same-step h_I(t) and the previous-step outputs, and produces s_E(t). Only s_E(t) goes to the next layer or the decoder. The layer state holds s_E(t-1) and h_I(t-1) and is replaced by s_E(t) and h_I(t).

One time step of a recurrent unified E/I layer (EILayerConfig with an excitatory and an inhibitory PopulationConfig).

At each time step the inhibitory population updates first, from the feedforward excitatory and inhibitory streams and the previous-step outputs of both populations. The excitatory population then reads the feedforward stream, the same-step inhibitory output and the previous-step outputs. Only the excitatory output goes to the next layer or to the decoder. allow_self_recurrence: false removes the previous-step input of each population to itself and keeps the recurrence between the two populations. See Unified E/I core.

Recurrent module boundaries

The recurrent dendritic implementation in networks/architectures/recurrent is split by responsibility:

  • network orchestration: population_network.py builds the ordered population-network layers, manages the sequence state and reduces the readout;

  • layer routing: population_layer.py maps named populations and directed connections onto StatefulDendriNet modules;

  • configuration and state: population_configs.py, population_common.py, population_sources.py, population_state.py and population_constants.py hold the dataclasses, source-name normalization and runtime state;

  • population options: ei_config.py defines PopulationConfig (synapse counts, morphology, time constants, synapse types, soma dynamics, dendritic spikes, soma feedback);

  • temporal execution: stateful_dendrinet.py owns the branch layers and the per-step update, with helpers in stateful_layers.py, stateful_setup.py, stateful_step.py and stateful_feedback.py;

  • typed synapse dynamics: stateful_typed_dynamics.py integrates receptor traces, short-term plasticity, voltage gates and typed shunting or additive currents;

  • soma and branch events: spiking_soma.py (leaky integrate-and-fire somas) and dendritic_spikes.py (branch spikes and plateaus);

  • morphology: dendritic_geometry.py (level sizes and synapse budgets);

  • structured recurrent cores: the structured_* modules translate the structured E/I sections into recurrent networks;

  • unified E/I networks: ei_layer.py, ei_network.py and ei_state.py; baselines: baseline_rnn.py, heterogeneous_leak_ctrnn.py and legendre_memory.py.

A new recurrent feature puts its standalone dynamics in a focused helper module and leaves the branch construction and the ordered step in StatefulDendriNet.

Key modules

  • scripts/training/train_experiments.py: the training entry point.

  • scripts/script_utils/setup_utils.py: run setup and initialize_model.

  • config/config.py: load_config, config_from_mapping and load_analysis_config.

  • networks/architectures/factory.py: get_architecture and the built-in core builders (with factory_recurrent.py, factory_spatial.py and factory_unified.py); networks/architectures/registry.py: registered architectures.

  • networks/architectures/excitation_inhibition/ei_network.py: ConfigurableEINetwork, the structured E/I core.

  • networks/architectures/excitation_inhibition/dendritic/: DendriNet, DendriticBranchLayer, BlockLinear and the initialization methods (initialize.py re-exports the initialize_* modules).

  • networks/architectures/excitation_inhibition/synapse/: the sparse synapse classes, their registry (factory.py), the indexed synapses (indexed.py re-exports the fixed, rewiring and candidate-pool variants) and the projection backends (kernels/backend.py).

  • models/base.py: parameter groups and the training hooks shared by the model wrappers.

  • training/factory.py: get_trainer and the strategy registry; training/strategies/: the trainers; training/optimizers/ and training/loss/: optimizer and loss registries.

  • datasets/standard_datasets.py: get_unified_datasets; datasets/registry.py: registered datasets; datasets/sequence/: sequence tasks, with the builder registry in sequence/builders.

  • analysis/managers/analysis_manager.py: runs the analyzers; analysis/registry.py: analyzer metadata (configuration field, class, output directory) from which the manager builds the built-in analyzers.

  • analysis/core/information.py: InformationAnalyzer; its implementation mixins live in analysis/core/information_parts/.

  • networks/architectures/classical/pretrained.py: a torchvision backbone splitter (list_backbone_layers), a Python API that the training entry points do not use.

  • utils/runtime_environment.py and utils/logging_config.py: the environment record and the logging setup.

Adding a component

  1. Add or extend the typed configuration field in src/dendritic_modeling/config, with validation, so the loader accepts and checks it.

  2. Implement the component and register it where a registry exists (see Extension registries), or add it to the built-in dispatch.

  3. Keep the default behaviour unchanged: a new option is off by default, and a test shows that the default output does not change.

  4. Add focused tests next to the changed subsystem (see Contributing).

  5. Document the option on the page that explains its topic, and add an example configuration when it helps.

Debugging a run

When a result changes unexpectedly, compare the run directories in this order: config.json, resolved_seeds.json, runtime_environment.json (library commit, dependency versions, devices), init_gate_stats.json, the first epochs in training_summary.json and the logs, then the outputs of the analyzers for the affected subsystem. See Run directory layout and Reproducibility.