Annotated reference configurations

Two complete configurations in configs/ carry a comment on every option: its type, the values it accepts and what it does. The values in the files are a working example, not the library defaults. Where a value differs from its default and the difference matters, the comment gives the default; an option that a file leaves out takes its default. Configuration guide explains how configs are loaded and checked and links the pages that document each option in full.

Single-process configuration

configs/config_exp.yaml trains a one-layer population network on CIFAR-10 in one process: an inhibitory population feeds an excitatory population whose neurons have a three-level dendritic tree. Besides the options this run uses, the file lists the options of the other training strategies, the local learning rules and most analyzers, so it can serve as a starting point. Check it without training:

dendritic-train configs/config_exp.yaml --validate-only
   1# ============================================================================
   2# DENDRINET ANNOTATED REFERENCE CONFIG (single process)
   3# ============================================================================
   4# A complete configuration: a one-layer population network (an inhibitory
   5# population feeding an excitatory population with a three-level dendritic
   6# tree) trained on CIFAR-10. Each option carries a comment with its type, its
   7# accepted values and what it does. The values are an example, not the library
   8# defaults; a comment gives the default where the difference matters. Options
   9# left out of this file take their defaults.
  10#
  11# The core is model.core.type "population_network". Sections of other core
  12# families (architecture, connectivity, morphology, sparsity, spatial, ...) are
  13# left out: a population_network core does not read them, and the loader
  14# rejects them when they differ from their defaults.
  15#
  16# Run from the repository root:
  17#   dendritic-train configs/config_exp.yaml
  18#   dendritic-train configs/config_exp.yaml --validate-only   # build everything, then exit
  19#   dendritic-train configs/config_exp.yaml --output_dir runs --run_name test --epochs 5
  20# docs/configuration_guide.rst lists every command-line override.
  21#
  22# Multi-GPU FSDP training: config_exp_fsdp.yaml. Parameter sweeps: sweep_config.yaml.
  23
  24config_version: 4                       # int: configuration format; dendritic-upgrade-config rewrites earlier ones
  25
  26# ============================================================================
  27# EXPERIMENT SETUP
  28# ============================================================================
  29experiment:
  30  seed: 42                              # int: Root seed; every stream below that is null uses it
  31  # Independent random streams (null = seed). Fix one to hold it while varying
  32  # the others, for example topology_seed for a fixed connectivity.
  33  dataset_seed: null                    # int | null: Generated datasets, data noise and label noise
  34  split_seed: null                      # int | null: Train/validation split
  35  model_seed: null                      # int | null: Parameter initialization and training-time noise
  36  topology_seed: null                   # int | null: Connectivity seeds the model config leaves unset
  37  loader_seed: null                     # int | null: Order of the training batches
  38  evaluation_seed: null                 # int | null: Sampling of the analysis runtime
  39  probe_seed: null                      # int | null: Analysis probes, decoders, nulls and other analyzer draws
  40  strict_deterministic: false           # bool: Exact math and PyTorch deterministic algorithms; a contradicting flag below raises
  41  train_valid_split: 0.8                # float: Fraction of the training data used for training; the rest is the validation split (datasets with their own splits ignore it)
  42  fast_mode: false                      # bool: Flags below that are null follow it (true: nondeterministic cuDNN, autotuning, TF32)
  43  # The four flags below are written out, so fast_mode does not change them
  44  # here. Set them to null (their default) to let fast_mode decide.
  45  deterministic: true                   # bool | null: Deterministic cuDNN kernels (null = not fast_mode); PyTorch deterministic algorithms need strict_deterministic
  46  cudnn_benchmark: false                # bool | null: cuDNN autotuner, off while deterministic is true (null = fast_mode and not deterministic)
  47  allow_tf32: false                     # bool | null: TF32 matmuls and convolutions (null = fast_mode); must agree with float32_matmul_precision
  48  float32_matmul_precision: "highest"   # str | null: "highest" (no TF32) | "high" | "medium" (null = "high" if fast_mode else "highest")
  49  # Not shown: sealed_test, allow_test_data, max_test_samples (test-split
  50  # sealing, docs/datasets.rst) and record_dataset_fingerprints
  51  # (docs/runs_and_outputs.rst).
  52
  53  checkpointing:
  54    enabled: false                      # bool: Save the weights every save_every_n_epochs epochs (main_network/<prefix>checkpoints/)
  55    save_every_n_epochs: 10             # int: Checkpoint interval in epochs (>= 1)
  56
  57# ============================================================================
  58# DATA
  59# ============================================================================
  60data:
  61  # dataset_name: a loadable dataset, for example the vision datasets "cifar10",
  62  # "cifar100", "mnist", "fashion_mnist" and "imagenet", derived tasks such as
  63  # "poisson_generator" and "mnist_modulo10", sequence tasks such as
  64  # "sequential_mnist" and "copy_task", and generated datasets such as
  65  # "correlated_gaussian". docs/datasets.rst lists every name with its
  66  # dataset_params; an unknown name raises with the full list when the
  67  # datasets are built.
  68  dataset_name: "cifar10"
  69  base_dir: ""                          # str: Data root (empty = $DENDRITIC_DATA_DIR, else ~/.cache/dendritic_modeling)
  70
  71  # ---------- Data processing ----------
  72  processing:
  73    flatten: null                       # bool | null: Flatten each sample to a vector (null = loader default: true for vision datasets)
  74    normalize: null                     # bool | null: Dataset-specific input normalization (null = loader default: false, except sequential_mnist and stringer_v1)
  75    label_noise_rate: 0.0               # float: Fraction of training labels replaced by random classes (mnist, fashion_mnist, cifar10 and cifar100 only)
  76
  77  # ---------- Dataset-specific params ----------
  78  # Only the block named by dataset_name is read; its keys are checked when the
  79  # datasets are built. The blocks below show the options of other datasets and
  80  # are ignored while dataset_name is "cifar10".
  81  dataset_params:
  82    mnist_modulo10:
  83      shuffle_iterations: 2             # int: Rounds of random image pairing; each adds one pair per two images
  84    poisson_generator:
  85      base_dataset: "mnist"             # str: "mnist" | "cifar10"
  86      multiplicative_gain: true         # bool: Multiply each input by a gain before Poisson sampling
  87      fixed_gain_factor: null           # float | null: One gain for every input (null = draw gains up to max_gain_factor)
  88      max_gain_factor: null             # float | null: Draw gains from [1, max_gain_factor] (null with uniform_gain: gain 1)
  89      max_gain_tau_ratio: null          # float | null: Sets max_gain_factor = ratio * stimulus_duration when max_gain_factor is null
  90      uniform_gain: true                # bool: One gain per sample (false: one per input element)
  91      gain_sampling: "log_uniform"      # str: "log_uniform" | "linear_uniform"
  92      poisson_sampling: true            # bool: Replace each input by a Poisson count rate over stimulus_duration
  93      stimulus_duration: 1.0            # float: Counting window of the Poisson draw
  94    imagenet:
  95      transform_preset: "legacy_resize"      # str: "legacy_resize" (resize to 224x224) | "pretrained" (train/eval transforms of pretrained torchvision backbones)
  96      loader_backend: "auto"            # str: "auto" | "torchvision" | "imagefolder" (root with train/<class> and val/<class> directories)
  97      allow_flatten_materialize: false       # bool: Allow processing.flatten: true for ImageNet (memory-heavy)
  98      train_samples_per_class: null     # int | null: Training images per class (null = all)
  99      val_samples_per_class: null       # int | null: Validation and test images per class (null = all)
 100
 101    # ---------- Sequence dataset params ----------
 102    # n_samples is the size of the training pool, which train_valid_split
 103    # divides; the test split has max(n_samples / 5, a dataset minimum) samples.
 104    sequential_mnist:
 105      mode: "pixel"                     # str: "pixel" (784 steps of 1 value) | "row" (28 steps of 28 values)
 106      normalize: true                   # bool: Scale pixels to [0, 1] (false: raw 0-255)
 107    copy_task:
 108      n_samples: 10000                  # int: Training-pool sequences
 109      seq_len: 10                       # int: Length of the sequence to copy
 110      delay: 10                         # int: Steps between the sequence and its recall (default 50)
 111      n_symbols: 8                      # int: Number of symbols
 112    adding_problem:
 113      n_samples: 10000                  # int: Training-pool sequences
 114      seq_len: 100                      # int: Sequence length (default 200)
 115    variable_delay_dms:
 116      n_samples: 10000                  # int: Training-pool sequences
 117      stimulus_dim: 16                  # int: Stimulus dimensionality
 118      stimulus_duration: 10             # int: Stimulus presentation steps
 119      min_delay: 5                      # int: Minimum delay steps (default 10)
 120      max_delay: 50                     # int: Maximum delay steps (default 100)
 121      probe_duration: 10                # int: Probe presentation steps
 122    context_dependent_decision:
 123      n_samples: 10000                  # int: Training-pool sequences (default 20000)
 124      signal_dim: 16                    # int: Signal dimensionality (default 8)
 125      context_duration: 10              # int: Context cue steps
 126      integration_duration: 30          # int: Integration window steps, used when variable_integration is false (default 50)
 127      coherence_levels: [0.1, 0.2, 0.4, 0.8]  # list[float]: Coherence levels (default [0.05, 0.1, 0.2, 0.4])
 128      noise_std: 0.1                    # float: Noise standard deviation (default 1.0)
 129      variable_integration: false       # bool: Sample the integration length from [min_integration, max_integration] (default true); false uses integration_duration
 130    timescale_generalization:
 131      stimulus_dim: 16                  # int: Stimulus dimensionality
 132      stimulus_duration: 10             # int: Stimulus presentation steps
 133      min_delay: 5                      # int: Minimum delay steps (default 10)
 134      max_delay: 50                     # int: Maximum delay steps (default 100)
 135      probe_duration: 10                # int: Probe presentation steps
 136
 137# ============================================================================
 138# MODEL
 139# ============================================================================
 140model:
 141  task: "classification"                # str: "classification" | "regression"
 142  # Optional warm start: a state-dict file loaded into the built model before
 143  # training.
 144  initial_checkpoint: null              # str | null: State-dict checkpoint path
 145  initial_checkpoint_sha256: null       # str | null: Expected SHA-256 of the file (a mismatch raises)
 146  initial_checkpoint_strict: true       # bool: Reject missing or unexpected keys
 147  # Not shown: fixed_output_scale and output_scale_mode (scaling of the
 148  # classifier output; docs/network_architectures.rst).
 149
 150  # ---------- Encoder ----------
 151  encoder:
 152    type: "identity"                    # str: "identity" | "none" | "pathway_router" (alias "router") | "mlpautoencoder" | "mlpvariationalautoencoder" | "cnnautoencoder" | "cnnvariationalautoencoder"
 153    # Each encoder type reads only some params; the identity encoder reads
 154    # input_dim and input_shape, which training sets from the data. The other
 155    # fields belong to the autoencoder and router types. Because this block
 156    # lists every field (as a saved config.json does), the loader only warns
 157    # when the selected type ignores a field that differs from its default; a
 158    # shorter block raises instead.
 159    params:
 160      input_dim: null                   # int | null: Input size; set from the data at training time
 161      input_shape: [3, 32, 32]          # list[int]: Input shape [C, H, W]; set from the data at training time
 162      encoder_hidden_dims: [128, 64]    # list[int]: (autoencoders) Encoder hidden sizes
 163      latent_dim: null                  # int | null: (MLP autoencoders) Latent size
 164      latent_shape: null                # list[int] | null: (CNN autoencoders) Latent shape
 165      decoder_hidden_dims: [64, 128]    # list[int]: (autoencoders) Decoder hidden sizes
 166      activation: "relu"               # str: (autoencoders) Any activation type, for example "relu" | "gelu" | "silu" | "tanh" | "sigmoid"
 167      router_mode: "fixed"             # str: (pathway_router) "fixed" | "learned"
 168      pathway_groups: []               # list[list[int]]: (pathway_router) Input indices of each pathway (empty = split the inputs into n_pathways groups)
 169      shared_indices: []               # list[int]: (pathway_router) Input indices given to every pathway
 170      n_pathways: 2                    # int: (pathway_router) Number of pathways when pathway_groups is empty
 171      pathway_dim: null                # int | null: (pathway_router) Output size of every pathway (null = its input count)
 172      pathway_dims: []                 # list[int]: (pathway_router) Output size of each pathway (overrides pathway_dim)
 173      router_activation: "none"        # str: (pathway_router) Any activation type, for example "none" | "sigmoid" | "relu"
 174      learned_router_temperature: 1.0  # float: (pathway_router, learned) Temperature of the learned assignment
 175      learned_router_init_scale: 4.0   # float: (pathway_router, learned) Initial logit scale of the learned assignment
 176
 177  # ---------- Core network ----------
 178  core:
 179    biological_neuron: true   # Enforce the neuronal-core constraint set.
 180    # Named-population model. configs/examples/ holds focused variants
 181    # (spiking, receptor kinetics, recurrent timing, additive normalization,
 182    # sparse rewiring, local learning); docs/population_networks.rst documents
 183    # the options.
 184    type: "population_network"
 185
 186    # ---- model.core.population_network ----
 187    # Keys inside this block are checked when the model is built, not when the
 188    # config loads; dendritic-train --validate-only checks them without
 189    # training. Network options not shown: readout_layer, readout_population,
 190    # compile_step. input_transform also accepts identity, relu and signed_split.
 191    population_network:
 192      input_transform: nonnegative     # Check nonnegative data without changing values or gradients.
 193      use_transfer: false               # bool: Split the input into excitatory and inhibitory streams with a TransferLayer
 194      # transfer_params: TransferLayer options (independent_pathways, excitatory_dim,
 195      #   inhibitory_dim, split_strategy, output_activation, ...); allowed only with
 196      #   use_transfer: true.
 197      transfer_params: {}
 198      output_mode: "last"               # str: Sequence output of recurrent networks: "last" | "mean" | "all"
 199      input_projection_dims: []         # list[int]: Linear + ReLU layers applied to the input before the first layer
 200      store_routing: false              # bool: Record per-step routing information on every forward pass (sets it on every layer)
 201      layers:                           # list[dict]: Layer keys: name, populations, connections, population_defaults, readout_population, recurrent, dt, connection_seed, store_routing
 202        - name: "layer0"
 203          readout_population: "e"       # str: Population whose somas form the layer output
 204          # population_defaults: options shared by every population of the
 205          # layer; a population's own `population` block overrides them.
 206          # n_neurons and branch_factors belong to each population definition
 207          # and are rejected here.
 208          population_defaults:
 209            ff_excitatory_synapses: 40  # int: Feedforward excitatory synapses per dendritic compartment (default 8)
 210            ff_inhibitory_synapses: 10  # int: Feedforward inhibitory synapses per compartment (default 4)
 211            rec_excitatory_synapses: 0  # int: Recurrent excitatory synapses per compartment (delayed connections)
 212            rec_inhibitory_synapses: 0  # int: Recurrent inhibitory synapses per compartment (delayed connections)
 213            somatic_synapses: true      # bool: The soma level also receives synapses
 214            use_shunting: true          # bool: Shunting (conductance) integration; false = additive
 215            reactivate: true            # bool: Apply the reactivation gate to each compartment output (false: no gate)
 216            reactivation_type: "param_tanh"   # str: Gate activation; any activation type (docs/dendritic_neurons.rst)
 217            reactivation_memory_efficient: true  # bool: Save fewer tensors for backward in param_tanh and param_tanh_only_m gates (default false)
 218            # reactivation_init_policy: "analytical" | "fixed" | "empirical" |
 219            #   "occupancy_quantile" | "analytical_slope_occupancy_center" |
 220            #   "occupancy_slope_analytical_center". Omitted, it is
 221            #   "analytical" for shunting populations and "occupancy_quantile"
 222            #   for additive ones (use_shunting: false).
 223            reactivation_init_policy: "analytical"
 224            # reactivation_init_m: 1.0      # float: Start slope (default 1.0), read only by init_policy "fixed"
 225            # reactivation_init_b: 0.5      # float: Start midpoint (default 0.5), read only by init_policy "fixed"
 226            dbl_init_method: "analytical_expectation"  # str: Initial synaptic weights: "analytical_expectation" | "ei_equivalence" | "naive" | "default" | "fan_in_lognormal" | "mechanism_neutral"
 227            weight_transform: "softplus"       # str: "exp" (default) | "softplus" | "relu" | "identity"
 228            # topk_type: "standard" | "stochastic" | "variance" | "deepst" |
 229            #   "credit_deepst" | "dense_to_sparse" (alias "annealed_topk") |
 230            #   "indexed" | "indexed_dynamic" | "indexed_rewire" | "indexed_release".
 231            # "standard" keeps dense candidate weights and selects the active
 232            # contacts; the indexed types store only the active indices and
 233            # suit large sparse models. The indexed_* options below are read
 234            # only by the indexed types (indexed_rewire_* only by
 235            # "indexed_rewire"); "standard" ignores them.
 236            topk_type: "standard"
 237            topk_init_method: "xavier_normal"    # str: Initializer of contacts that "indexed_rewire" creates; dbl_init_method sets the initial weights
 238            indexed_rewire_frequency: 100        # int: (indexed_rewire) Steps between rewiring passes
 239            indexed_rewire_quantile: 0.05        # float: (indexed_rewire) Fraction of each output's weakest contacts replaced per pass
 240            indexed_rewire_until_step: null      # int | null: (indexed_rewire) Last rewiring step (null = no limit)
 241            indexed_output_chunk_size: 8192      # int: (indexed) Output rows per gather chunk (default 2048)
 242            indexed_index_dtype: "auto"          # str: (indexed) "auto" | "int32" | "int64" (default); auto uses int32 when it fits
 243            indexed_workspace_mb: 128            # float | null: (indexed) Cap of the temporary gather workspace in MB (default null)
 244            indexed_cache_transformed_weights: false # bool: (indexed) Reuse fixed transformed weights within an unroll
 245            indexed_projection_backend: "auto"   # str: (indexed) Projection backend (default "eager"); auto picks a Triton kernel on CUDA when available, else "recompute"
 246            indexed_persistent_indices: true     # bool: (indexed) Store the topology in checkpoints
 247            indexed_init_mode: "per_rank"       # str: (indexed) "per_rank" | "rank0_broadcast"
 248            efficient_blocklinear: true          # bool: BlockLinear branch couplings without the dense block-diagonal matrix (default false)
 249          populations:
 250            - name: "i"
 251              polarity: "inhibitory"    # str: "excitatory" | "inhibitory"
 252              n_neurons: 32             # int: Number of neurons (somas)
 253              branch_factors: [1]       # list[int]: Branches per compartment from the soma outward (default [1])
 254              population:               # PopulationConfig options of this population only
 255                ff_inhibitory_synapses: 0
 256            - name: "e"
 257              polarity: "excitatory"
 258              n_neurons: 128
 259              branch_factors: [3, 3, 3]
 260          connections:                  # Connection keys: source, target, pathway, timing ("same_step" | "delayed"), enabled, probability, seed, target_levels
 261            - source: "input"
 262              target: "i"
 263            - source: "input"
 264              target: "e"
 265            - source: "i"
 266              target: "e"
 267
 268  # ---------- Decoder ----------
 269  decoder:
 270    type: "MLP"                         # str: "MLP" | "identity" | "none" (identity and none pass the core output through; a linear readout is MLP with hidden_dims: [])
 271    params:
 272      input_dim: 20                     # int: Replaced by the core output size when the model is built
 273      hidden_dims: [16]                 # list[int]: Hidden layer sizes
 274      activation: "relu"               # str: Any activation type, for example "relu" | "gelu" | "silu" | "tanh"
 275      output_dim: 10                    # int: Number of classes or outputs (identity/none: omit it, or give the core output size)
 276
 277# ============================================================================
 278# TRAINING
 279# ============================================================================
 280training:
 281
 282  # ---------- Main training ----------
 283  main:
 284    # strategy: "standard" | "fa" | "dfa" | "soma_dfa" | "local_ca" | "multi_stage" |
 285    #   "recurrent" | "presentation_stream" | "voltage_stabilization" |
 286    #   "homeostatic_control" | "ei_equilibrium" | "freeze_layers" |
 287    #   "freeze_branches" | "freeze_branch_kl" (docs/training_strategies.rst).
 288    # A strategy reads its own block under strategies; the other blocks are ignored.
 289    strategy: "standard"
 290
 291    common:
 292      epochs: 1000                      # int: Training epochs
 293      batch_size: 512                   # int: Training batch size
 294      num_workers: null                 # int | null: DataLoader workers (null = chosen from the dataset type)
 295      pin_memory: null                  # bool | null: Pin host batches (null = when training on a GPU)
 296      persistent_workers: null          # bool | null: Keep workers alive between epochs (needs num_workers > 0; null = automatic)
 297      prefetch_factor: null             # int | null: Batches prefetched per worker (needs num_workers > 0; null = automatic)
 298      in_order: null                    # bool | null: Deliver batches in sampler order (null: false for image datasets with workers; strict_deterministic requires true)
 299      profile_dataloader: false         # bool: Log the time each epoch waits for data
 300      shuffle: true                     # bool: Shuffle the training data
 301      grad_clip_value: 2.0              # float | null: Gradient clipping threshold (0 or null = off)
 302      grad_clip_mode: null              # str | null: "value" (clamp each element) | "norm" (rescale the global norm); null = "norm" for strategy recurrent and FSDP, else "value"
 303      # loss_function: "ce" | "bce" | "mse" | "log_mse" | "cat_nll" | "neg_cos_sim" |
 304      #   "log_adj_neg_cos_sim" | "temporal_event_ce" | "masked_dual_task" |
 305      #   "masked_dual_task_temporal" | "vae_elbo" (long names such as "cross_entropy" also work)
 306      loss_function: "ce"
 307      early_stopping: true              # bool: Stop when the validation loss has not improved for patience epochs
 308      patience: 500                     # int: Early-stopping patience in epochs
 309      load_best_state_dict: true        # bool: End training with the weights of the best validation epoch
 310      plot_losses: true                 # bool: Save loss curves (main_network/<prefix>loss_curves.png)
 311      suppress_prints: false            # bool: Suppress per-epoch console output
 312      print_every: 10                   # int: Epochs between progress prints
 313      use_amp: false                    # bool: Automatic mixed precision
 314      lr_schedule: "none"               # str: "none" | "cosine" (strategies standard, recurrent, fa, dfa, soma_dfa and presentation_stream apply it; the others raise)
 315      lr_warmup_epochs: 0               # int: Linear warm-up epochs of the cosine schedule (needs lr_schedule: "cosine")
 316      enable_nan_checking: false        # bool: Zero NaN/inf gradient entries before each step
 317      synapse_decay_rate: 0.01          # float: Sparse-synapse maintenance: active Top-K and indexed weights decay by lr * rate before each step (not optimizer.weight_decay)
 318      weight_boosting: true             # bool: Also boost unselected candidate weights by that factor (default false; needs synapse_decay_rate > 0)
 319      param_groups:                     # Learning rates of the optimizer's parameter groups
 320        lr: 0.001                       # float: Rate of parameters outside the groups below, or of all parameters with split_params: false (default 0.01)
 321        split_params: true              # bool: Give each component group its own rate
 322        topk_lr: 0.001                  # float: Sparse synapses (Top-K and indexed banks)
 323        blocklinear_lr: 0.001           # float: BlockLinear branch couplings (default 0.0001)
 324        reactivation_lr: 0.001          # float: Reactivation gates (default 0.0001)
 325        decoder_lr: 0.001               # float: Decoder and learned output scales
 326        encoder_lr: null                # float | null: Trainable encoder parameters (null = lr)
 327
 328    optimizer:
 329      name: "adam"                      # str: "adam" | "adamw" | "sgd" | "rmsprop" | "muonh" (alias "muon_hyperball"; the muon_* options apply only to it)
 330      lr: 0.001                         # float: Not used for training: every parameter group takes its rate from common.param_groups (a non-default value must equal param_groups.lr)
 331      weight_decay: 0.0                 # float: Optimizer weight decay of every group (decoupled for adamw)
 332      momentum: 0.9                     # float: Momentum (sgd, rmsprop)
 333      betas: [0.9, 0.999]              # list[float]: Betas (adam, adamw, and the AdamW part of muonh)
 334      eps: 0.00000001                   # float: Epsilon (adam, adamw, rmsprop, muonh)
 335
 336    regularization:                     # Auxiliary loss terms (local_ca, freeze_branches, freeze_branch_kl and FSDP reject them)
 337      l1_weight: 0.0                    # float: L1 penalty strength (0 = off)
 338      l2_weight: 0.0                    # float: L2 penalty strength (0 = off; separate from optimizer.weight_decay)
 339      selective:                        # Which weights the L1/L2 terms cover (synapse, BlockLinear and dense weights)
 340        layers: []                      # list[int]: Layer indices, matched with the first integer in a parameter name (empty = all)
 341        branches: []                    # list[str]: Parameter-name substrings (empty = all)
 342        weight_types: ["all"]           # list[str]: Parameter-name substrings, or "all"
 343      split_params: false               # bool: Use the per-group strengths below (groups not listed use l1_weight/l2_weight)
 344      param_group_weights:
 345        topk:
 346          l1_weight: 0.0               # float: Sparse-synapse L1 strength
 347          l2_weight: 0.0               # float: Sparse-synapse L2 strength
 348        blocklinear:
 349          l1_weight: 0.0               # float: BlockLinear L1 strength
 350          l2_weight: 0.0               # float: BlockLinear L2 strength
 351        reactivation:
 352          l1_weight: 0.0               # float: Reactivation L1 strength
 353          l2_weight: 0.0               # float: Reactivation L2 strength
 354        decoder:
 355          l1_weight: 0.0               # float: Decoder L1 strength
 356          l2_weight: 0.0               # float: Decoder L2 strength
 357      enforce_ei_weight_ratio: false    # bool: Add a penalty on the E/I weight ratio
 358      target_ei_weight_ratio: 0.5       # float: Target excitatory/inhibitory weight ratio
 359      ei_ratio_loss_weight: 0.5         # float: Strength of the E/I ratio penalty
 360      ei_ratio_scope: "per_branch"      # str: "per_branch" | "per_layer" | "global" ("global" averages the per-layer ratios)
 361      ei_ratio_metric: "mean"           # str: "mean" | "median" | "rms"
 362
 363    pruning:                            # Zeroes stored weights after training
 364      enabled: false                    # bool: Enable pruning. It rejects exp/softplus-transformed synapses, so it cannot prune this model's synapses
 365      threshold: 2                      # float: Prune stored weights with abs(value) < threshold
 366      selective:
 367        layers: []                      # list[int]: Layer indices, matched with the first integer in a parameter name (empty = all)
 368        branches: []                    # list[str]: Parameter-name substrings (empty = all)
 369        weight_types: ["excitatory", "inhibitory"]  # list[str]: Parameter-name substrings, or "all"; these two match parameter names of structured E/I cores, not of population networks
 370      re_evaluate_after: false          # bool: Record training and validation loss before and after pruning
 371      report_non_pruned: true           # bool: Print the pruning report
 372      save_pruning_stats: false         # bool: Write pruning_stats.json
 373      detailed_branch_report: true      # bool: Include per-branch statistics in the report
 374      redo_analysis_after_pruning: false  # bool: Not implemented; true raises when pruning is enabled
 375
 376    strategies:
 377      standard: {}                      # The standard strategy uses the common options only
 378
 379      multi_stage:
 380        analyze_between_stages: true    # bool: Run the analysis after each stage
 381        final_state: "last_stage"      # str: Weights the run ends with: "last_stage" | "best_stage" (lowest validation loss)
 382        continue_on_failure: false      # bool: Continue with the next stage when a stage fails
 383        stages:                         # list[dict]: Stages in order; their learning_strategy is checked when the stage starts
 384          - learning_strategy: "standard"
 385            epochs: 100                 # int: Stage epochs (default common.epochs)
 386            batch_size: 512             # int: Stage batch size (default common.batch_size)
 387            save_checkpoint: false      # bool: Write <stage directory>/stage_checkpoint.pt (default true)
 388            param_groups:               # Stage learning rates (default common.param_groups)
 389              lr: 0.001
 390              split_params: true
 391              topk_lr: 0.001
 392              blocklinear_lr: 0.001
 393              reactivation_lr: 0.001
 394              decoder_lr: 0.001
 395            trainer_config:             # Common trainer options overridden for this stage
 396              loss_function: "ce"
 397              early_stopping: true
 398              patience: 500
 399          - learning_strategy: "local_ca"   # Uses training.main.learning_strategy_config unless local_rule_config is set
 400            epochs: 100
 401            batch_size: 512
 402            reset_optimizer: false      # bool: Rebuild the optimizer before this stage
 403            save_checkpoint: false
 404            param_groups:
 405              lr: 0.0001
 406              split_params: true
 407              topk_lr: 0.0001
 408              blocklinear_lr: 0.00001
 409              reactivation_lr: 0.00001
 410              decoder_lr: 0.001
 411            trainer_config:
 412              loss_function: "ce"
 413              early_stopping: false
 414
 415      voltage_stabilization:
 416        pretrain_epochs: 100            # int: Warm-up epochs before standard training, >= 1 (default 1)
 417        stabilize_mode: "vout"          # str: Activations the warm-up drives: "vinf" (default, gate input) | "vout" (gate output) | "both" | "gate_occupancy"
 418        freeze_reactivation: true       # bool: Freeze the gates during the warm-up (default false)
 419        target_voltage: 0.5             # float: Target of the warm-up loss
 420        loss_metric: "mse"              # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
 421
 422      homeostatic_control:
 423        pretrain_epochs: 100            # int: Warm-up epochs before standard training, >= 1 (default 1)
 424        stabilize_mode: "vinf"          # str: "vinf" | "vout" | "both" | "gate_occupancy"
 425        freeze_reactivation: false      # bool: Freeze the gates during the warm-up
 426        target_voltage: 0.5             # float: Target of the warm-up loss
 427        loss_metric: "mse"              # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
 428
 429      ei_equilibrium:                   # Warm-up that drives each branch's excitatory conductance toward its inhibitory one; needs inhibitory synapses
 430        pretrain_epochs: 1              # int: Warm-up epochs before standard training, >= 1
 431        freeze_reactivation: false      # bool: Freeze the gates during the warm-up
 432        loss_metric: "mse"              # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
 433
 434      recurrent:                        # null inherits the training.main.common option of the same name
 435        grad_clip_value: null           # float | null: Gradient clipping threshold (0 = off)
 436        use_amp: null                   # bool | null: Mixed precision
 437        lr_schedule: null               # str | null: "none" | "cosine"
 438        lr_warmup_epochs: null          # int | null: Warm-up epochs of the cosine schedule
 439
 440      freeze_layers:
 441        pretrain_epochs: 10             # int: Full-network epochs before the layer-by-layer phase
 442        reverse_training: false         # bool: Train the layers from last to first
 443        epochs_per_layer: 50            # int: Epochs per layer while the others are frozen (default 5)
 444        final_tune_epochs: 10           # int: Full-network epochs after the layer-by-layer phase
 445
 446      freeze_branches:
 447        pretrain_epochs: 10             # int: Stage-1 epochs, each training every core layer in turn with a local head (default 5); stage 2 trains end to end for common.epochs
 448        reverse_training: false         # bool: Not implemented; a value other than false raises
 449        epochs_per_branch: 50           # int: Not implemented; a value other than 50 raises
 450        final_tune_epochs: 10           # int: Not implemented; a value other than 10 raises
 451
 452      freeze_branch_kl:
 453        pretrain_epochs: 10             # int: Stage-1 epochs as in freeze_branches (default 5)
 454        reverse_training: false         # bool: Not implemented; a value other than false raises
 455        epochs_per_branch: 50           # int: Not implemented; a value other than 50 raises
 456        final_tune_epochs: 10           # int: Not implemented; a value other than 10 raises
 457        kl_weight: 0.1                  # float: Weight of the KL term pulling each local head toward the model's predictions
 458
 459    # ---- Local learning rules (training.main.learning_strategy_config) ----
 460    # Read by strategy local_ca and by the local_ca stages of multi_stage; the
 461    # other strategies ignore it (a block that differs from the defaults then
 462    # warns).
 463    learning_strategy_config:
 464      rule_variant: "3f"                # str: "3f" | "4f" | "5f" | "stdp", or "3f_vh" | "4f_vh" | "5f_vh" (adds voltage_homeostasis)
 465      error_mode: "auto"               # str: "auto" (from loss_function) | "mse" | "ce" | "bce"
 466      error_broadcast_mode: "scalar"    # str: "scalar" | "per_soma" | "per_soma_shared" | "per_soma_tree" | "per_soma_shuffled" | "local_mismatch" | "low_rank" | "path_transport" | "pathway_vector"
 467      broadcast_seed: null              # int | null: Seed of the randomized feedback (per_soma_shuffled, low_rank); null = derived from the run seed
 468      error_noise_sigma: 0.0           # float: Additive Gaussian noise on the broadcast error (0 = off)
 469      broadcast_rank: 4                 # int: Rank of the low_rank broadcast
 470      broadcast_init_scale: 1.0         # float: Scale of the low_rank random broadcast
 471      pathway_broadcast_residual: 0.25  # float: Blend with the direct local error
 472      pathway_activity_gate_strength: 1.0  # float: Modulation by local pathway activity
 473      broadcast_bandwidth: "full"       # str: "full" | "sign_only" | "quantized" | "sparse_topk"
 474      broadcast_bits: 8                 # int: Quantization bits, >= 1 (2^bits levels); for broadcast_bandwidth "quantized"
 475      broadcast_topk_fraction: 0.3      # float: Fraction in (0, 1] kept by broadcast_bandwidth "sparse_topk"
 476
 477      three_factor:
 478        dynamics_mode: "auto"           # str: "auto" (each layer's own integration) | "conductance" | "additive" (must match every layer unless allow_mismatched_dynamics_mode)
 479        allow_mismatched_dynamics_mode: false  # bool: Apply a forced dynamics_mode to every layer (a rule variant)
 480        use_conductance_scaling: true   # bool: Scale by R_tot = 1/g_tot
 481        use_driving_force: true         # bool: (E_rev - V_n) driving force; false uses (V_n - theta) / (theta - V_n)
 482        theta: 0.0                      # float: Threshold of the use_driving_force=false variant and of additive "input_dependent" mode
 483        e_rev_exc: 1.0                  # float: Fixed at 1.0, the model's excitatory reversal
 484        e_rev_inh: 0.0                  # float: Fixed at 0.0, the model's inhibitory reversal
 485        additive_gain_mode: "none"      # str: "none" | "input_dependent" | "running_stats" (dynamics_mode "additive" only)
 486        additive_stats_ema_alpha: 0.01  # float: EMA rate of the running_stats mode
 487
 488      four_factor:
 489        rho_mode: "pearson"             # str: "pearson" | "dot" | "none"
 490        rho_estimator: "ema"            # str: "ema" | "augment" | "ema+augment"
 491        ema_alpha: 0.05                 # float: EMA smoothing rate
 492        layer_wise_rho_scale: 1.0       # float: rho / sqrt(layer_depth)
 493        augment_k: 8                    # int: Number of augmented samples
 494        augment_noise_sigma: 0.01       # float: Augmentation noise level
 495
 496      five_factor:
 497        phi_mode: "conditional"         # str: "conditional" | "variance"
 498        phi_estimator: "conditional_ema"  # str: "conditional_ema" | "variance_ema"
 499        phi_ridge_lambda: 0.001         # float: Ridge regularization
 500        layer_wise_phi_scale: 1.0       # float: phi / sqrt(layer_depth)
 501        phi_clamp_min: 0.25             # float: Stability clamp min
 502        phi_clamp_max: 4.0              # float: Stability clamp max
 503
 504      morphology_aware:
 505        use_path_propagation: false     # bool: Path-integrated propagation
 506        path_factor_mode: "per_branch"  # str: "per_branch" | "scalar_mean"
 507        morphology_modulator_mode: "none"  # str: "none" | "depth" | "centrality" (inverse branch depth)
 508        morphology_depth_offset: 1.0    # float: Offset alpha of the depth modulation
 509        use_dendritic_normalization: false  # bool: Normalize by the sum of conductances
 510        use_branch_type_rules: false    # bool: Apical vs basal differentiation
 511        apical_branch_scale: 1.0        # float: Scale for apical branches
 512        basal_branch_scale: 1.0         # float: Scale for basal branches
 513        use_branch_length_modulation: false  # bool: Branch length modulation
 514        use_branch_role_rules: false    # bool: Router-derived branch roles
 515        branch_role_source: "router_pathways"  # str: "router_pathways"
 516        specialized_branch_scale: 1.0   # float: Specialized branch scale
 517        mixed_branch_scale: 1.0         # float: Mixed branch scale
 518        branch_role_power: 1.0          # float: Branch role power
 519        branch_role_alignment_weight: 0.0  # float: Alignment weight
 520
 521      hsic:
 522        enabled: false                  # bool: Enable the HSIC auxiliary loss
 523        weight: 0.0                     # float: HSIC loss weight
 524        self_weight: 1.0                # float: Self-decorrelation weight
 525        target_weight: 1.0              # float: Target-correlation weight
 526        target_source: "labels"         # str: "labels" | "logits"
 527        kernel: "linear"                # str: "linear" | "rbf" | "polynomial"
 528        sigma: 1.0                      # float: RBF bandwidth
 529        degree: 2                       # int: Polynomial degree
 530        coef0: 1.0                      # float: Polynomial coefficient
 531        grad_clip_value: 0.0            # float: HSIC gradient clip (0 = off)
 532        warmup_epochs: 0                # int: HSIC warm-up epochs
 533        apply_last_layer_only: false    # bool: Apply HSIC to the last layer only
 534
 535      stdp:
 536        enabled: false                  # bool: Trace-based STDP as an auxiliary rule; rule_variant "stdp" enables it
 537        apply_to: ["exc", "rec_exc"]    # list[str]: "exc" | "inh" | "rec_exc" | "rec_inh" | "all"
 538        activity_mode: "relu"           # str: "relu" | "identity" | "binary" | "abs"
 539        pre_threshold: 0.0              # float: Presynaptic activity threshold
 540        post_threshold: 0.0             # float: Postsynaptic activity threshold
 541        tau_pre: 20.0                   # float: Presynaptic trace tau
 542        tau_post: 20.0                  # float: Postsynaptic trace tau
 543        a_plus: 1.0                     # float: Potentiation scale
 544        a_minus: 0.5                    # float: Depression scale
 545        learning_rate_scale: 1.0        # float: Multiplier on the learning rate of the STDP update
 546        inhibitory_update_sign: -1.0    # float: Sign multiplier for inhibitory pathways
 547        use_error_modulation: false     # bool: Modulate STDP by the broadcast task error
 548        error_modulation_mode: "scalar_abs"  # str: "scalar_abs" | "scalar_signed"
 549        clamp_update: 0.0               # float: Absolute update clamp (0 = off)
 550        detach_traces: true             # bool: Detach local traces from the autograd graph
 551
 552      inhibitory_homeostasis:
 553        enabled: false                  # bool: Enable inhibitory homeostasis
 554        mode: "r_tot"                   # str: "r_tot" | "voltage"
 555        weight: 0.0                     # float: Homeostasis loss weight
 556        target_r_tot: 0.35              # float: Target total resistance
 557        target_voltage: 0.15            # float: Target voltage
 558
 559      voltage_homeostasis:
 560        enabled: false                  # bool: Enable strictly local voltage centering
 561        weight: 0.0                     # float: Local voltage-homeostasis weight
 562        target_voltage: 0.5             # float: Midpoint target voltage
 563        # The *_vh rule variants enable this block; a weight of 0 then becomes 0.02.
 564
 565      gate_homeostasis:
 566        enabled: false                  # bool: Enable the local gate-occupancy update
 567        center_weight: 0.0              # float: Weight of the mean gate-occupancy target
 568        saturation_weight: 0.0          # float: Weight of the saturation target
 569        target_mean: 0.5                # float: Desired mean gate value
 570        target_saturation: 0.2          # float: Desired fraction of saturated gates
 571        low_threshold: 0.1              # float: Low gate threshold of the saturation metric
 572        high_threshold: 0.9             # float: High gate threshold of the saturation metric
 573        saturation_slope: 20.0          # float: Slope of the smooth saturation indicator
 574
 575      update_inactive_weights: false    # bool: Also update inactive candidate weights
 576      update_reactivation: true         # bool: Update the gate parameters with the local rule
 577      explicit_inhibitory_update_mode: "local_ca"  # str: "local_ca" | "freeze" for the dendrites of explicit inhibitory populations
 578      encoder_update_mode: "none"       # str: "backprop" | "none"
 579      decoder_update_mode: "backprop"   # str: "backprop" | "local" | "none"
 580      freeze_encoder_epochs: 0          # int: Freeze the encoder for N epochs
 581      freeze_reactivation_epochs: 0     # int: Freeze the gates for N epochs
 582      freeze_decoder_epochs: 0          # int: Freeze the decoder for N epochs
 583      training_schedule: null           # list[dict] | null: Epoch ranges (start_epoch, end_epoch) that set update_reactivation, encoder_update_mode or decoder_update_mode
 584      clip_grad_value: 5.0              # float: Element-wise clip of the local-rule gradients (local_ca uses it instead of common.grad_clip_value)
 585      normalize_by_batch: true          # bool: Normalize updates by the batch size
 586
 587# ============================================================================
 588# ANALYSIS
 589# ============================================================================
 590# Every enabled analyzer runs once after training; training: true also runs it
 591# during training (some analyzers are final-only). docs/analysis.rst describes
 592# every analyzer, including those not shown here (they are off when omitted);
 593# configs/templates/analysis_full_template.yaml shows most of them.
 594analysis:
 595  # Dataset iteration shared by all analyzers. mode:
 596  #   "auto"        materialize a split when its estimated size is at most
 597  #                 materialize_threshold_mb, otherwise stream it
 598  #   "materialize" load each split as one batch (hook-based analyses then see
 599  #                 the whole split at once when the dataset supports slicing)
 600  #   "stream"      iterate in batches through a DataLoader (for large datasets)
 601  # Keys left unset take their defaults, except mode: an unset mode is
 602  # "materialize" in single-process runs and "auto" under DDP/FSDP.
 603  runtime:
 604    training:
 605      mode: "auto"                      # str: "auto" | "materialize" | "stream"
 606      batch_size: 256                   # int: Analysis batch size
 607      num_workers: 4                    # int: DataLoader workers
 608      pin_memory: true                  # bool: Pin host memory for GPU analysis (off for CPU analysis)
 609      persistent_workers: true          # bool: Keep workers alive between streamed passes (default false)
 610      prefetch_factor: 4                # int | null: Batches prefetched per worker (default 2)
 611      in_order: null                    # bool | null: false lets batches arrive as workers finish them (null = PyTorch default, in order)
 612      max_batches: null                 # int | null: Cap on batches per split (null = all)
 613      max_samples: null                 # int | null: Cap on the examples given to each analysis (null = all)
 614      materialize_threshold_mb: 1024    # int: Size limit (MB) up to which "auto" materializes a split
 615    final:
 616      mode: "auto"                      # str: "auto" | "materialize" | "stream"
 617      batch_size: 256                   # int: Analysis batch size
 618      num_workers: 4                    # int: DataLoader workers
 619      pin_memory: true                  # bool: Pin host memory for GPU analysis (off for CPU analysis)
 620      persistent_workers: true          # bool: Keep workers alive between streamed passes (default false)
 621      prefetch_factor: 4                # int | null: Batches prefetched per worker (default 2)
 622      in_order: null                    # bool | null: false lets batches arrive as workers finish them (null = PyTorch default, in order)
 623      max_batches: null                 # int | null: Cap on batches per split (usually null for the final analysis)
 624      max_samples: null                 # int | null: Cap for heavy analyses that should inspect only a subset
 625      materialize_threshold_mb: 1024    # int: Size limit (MB) up to which "auto" materializes a split
 626
 627  # ---------- Performance ----------
 628  performance:
 629    enabled: true                       # bool: Enable performance analysis (default false)
 630    training: false                     # bool: Also run during training
 631    params:
 632      accuracy: true                    # bool: Compute accuracy
 633      auc: true                         # bool: Compute AUC
 634      categorical_loglikelihood: true   # bool: Compute categorical log-likelihood
 635      mse: false                        # bool: Compute MSE
 636      cosine_similarity: false          # bool: Compute cosine similarity
 637    training_splits: ["train", "validation", "test"]  # list[str]: Splits evaluated during training
 638    final_splits: ["train", "validation", "test"]     # list[str]: Splits evaluated after training
 639
 640  # ---------- Information ----------
 641  information:
 642    enabled: true                       # bool: Enable information analysis (default false)
 643    training: false                     # bool: Also run during training
 644    params:
 645      # -- Grouped structure --
 646      selection:
 647        # components: "basic" | "pairwise" | "conditional" | "gaussian_fisher" | "pid" |
 648        #   "ablation_aligned" | "layer_proxies" | "upstream_unique_cmi" | "soma_coupling".
 649        #   Empty computes basic, pairwise, conditional, ablation_aligned and layer_proxies.
 650        components: []
 651        scopes: []                      # list[str]: "layer" | "einet" | "neuron"
 652        signal_variants: []             # list[str]: "network" | "lda" | "random" (random needs baselines.random_weights.n_shuffles > 0)
 653      compute:
 654        max_samples: null               # int | null: Limit samples (null = all)
 655        include_vinf: true              # bool: Include the soma activation
 656        normalize_mi: false             # bool: Normalize by H(C)
 657        output_units: "bits"            # str: "bits" | "nats"
 658        analysis_split: "test"          # str: "validation" | "test"
 659        verbose: false                  # bool: Verbose output
 660        computation_level: "single_branch"  # str: "single_branch" | "layer_branch" | "all_branch"
 661        branch_aggregation: "mean"      # str: "mean" | "multivariate" | "sample"
 662        branch_sample_count_per_layer: null  # int | null: Deterministic per-layer branch subset for single_branch (null = all)
 663        branch_sample_seed: null         # int | null: null = derived from experiment.probe_seed
 664        layer_total_topk: 10            # int: K of the topK_sum metrics
 665        gaussian_fisher:
 666          aggregation: "sum"            # str: "sum" | "mean" | "max"
 667          mi_transform: "half_log1p"    # str: "half_log1p" | "log1p" | "none"
 668          eps: 0.00000001               # float: Numerical epsilon
 669          ridge: null                   # float | null: Ridge parameter (null = use eps)
 670      estimator:
 671        method: "kraskov"               # str: "kraskov" | "decoder" | "binned" | "sklearn" | "copula" | "auto"
 672        kraskov:
 673          n_neighbors: 10               # int: Number of KSG neighbors
 674        binned:
 675          n_bins: 20                    # int: Number of bins (binned estimator; the sklearn estimator uses it too)
 676          binning_strategy: "quantile"  # str: "quantile" | "uniform"
 677          bias_correction: "none"       # str: "none" | "miller_madow"
 678        decoder:
 679          cv_folds: 5                   # int: Cross-validation folds
 680          C: 1.0                        # float: Regularization parameter
 681          standardize: true             # bool: Standardize features
 682          seed: null                    # int | null: null = derived from experiment.probe_seed
 683          continuous:
 684            strategy: "none"            # str: "none" | "binned" | "gaussian"
 685            max_total_dim: 2            # int: Max total dim for continuous MI/CMI
 686            gaussian_ridge: 0.000001    # float: Covariance ridge
 687        copula:
 688          type: "gaussian"              # str: "gaussian" (scalars, vectors, labels) | "kernel" (two continuous scalars)
 689        sklearn:                        # Not read: the sklearn estimator takes its bins from estimator.binned.n_bins
 690          n_bins: 20                    # int: Not read
 691          bandwidth: null               # float | null: Not read
 692      baselines:
 693        random_weights:
 694          n_shuffles: 0                 # int: Number of random-weight shuffles (0 = off)
 695          seed: null                    # int | null: null = derived from experiment.probe_seed
 696        label_shuffle_null:
 697          n_shuffles: 0                 # int: Number of label permutations (0 = off)
 698          seed: null                    # int | null: null = derived from experiment.probe_seed
 699          components: ["basic"]         # list[str]: "basic" | "pairwise" | "conditional" | "ablation_aligned" | "upstream_unique_cmi" | "soma_coupling" | "all"
 700      pid:
 701        binarize: false                 # bool: Threshold E, I and Vout (never the labels) before PID
 702        binarize_method: "median"       # str: "median" | "mean" | "quantile"
 703        binarize_threshold: 0.5         # float: Quantile in [0, 1]; used only with binarize_method "quantile"
 704
 705  # ---------- Correlation ----------
 706  correlation:
 707    enabled: false                      # bool: Enable correlation analysis
 708    training: false                     # bool: Also run during training
 709    params:
 710      # -- Grouped structure --
 711      selection:
 712        components: []                  # list[str]: "total" | "noise" | "signal" | "tuning"
 713        pairs: []                       # list[str]: "EE" | "II" | "EI" | "E_output" | "I_output" | "output_output"
 714        scopes: []                      # list[str]: "layer" | "einet" | "neuron"
 715      compute:
 716        computation_level: "layer_branch"  # str: "synaptic" | "single_branch" | "layer_branch" | "all_branch"
 717        max_samples: 5000               # int | null: Limit samples
 718        per_layer_analysis: true        # bool: Per-layer breakdown
 719        per_einet_analysis: true        # bool: Separate E and I pathway summaries
 720        per_neuron_analysis: false      # bool: Per-neuron breakdown
 721      sampling:
 722        n_synapse_pairs: 10000          # int: Synapse pairs to sample (synaptic level)
 723        sampling_strategy: "stratified" # str: "stratified" | "random" (other values raise)
 724        stratified_categories: ["within_branch", "within_layer"]  # list[str]: "within_branch" | "within_layer"
 725        max_pairs_per_category: 1000    # int: Max pairs per stratified category
 726
 727  # ---------- Ablation ----------
 728  ablation:
 729    enabled: false                      # bool: Enable ablation analysis
 730    training: false                     # bool: Also run during training
 731    params:
 732      selection:                        # Empty lists select nothing: an ablation needs at least one level, target and metric
 733        levels: []                      # list[str]: "layer" | "compartment"
 734        targets: []                     # list[str]: "all_synapses" | "excitation" | "inhibition" | "upstream"
 735        metrics: []                     # list[str]: "accuracy" | "auc" | "categorical_loglikelihood" | "mse" | "cosine_similarity" | "pred_label_mi_bits"
 736      methods:
 737        ablation_methods: ["lesion"]    # list[str]: "lesion" | "shuffle" | "mean_clamp"
 738        shuffle_seed: null              # int | null: null = derived from experiment.probe_seed
 739        clamp_statistic: "mean"         # str: "mean" (the only supported value)
 740
 741  # ---------- Synapse turnover ----------
 742  synapse_turnover:
 743    enabled: false                      # bool: Enable synapse turnover analysis
 744    training: true                      # bool: Must be true; turnover compares training snapshots
 745    params:
 746      snapshot_interval: 1              # int: Epochs between snapshots (first = before training)
 747
 748  # ---------- Synapse pruning (final-only census) ----------
 749  synapse_pruning:
 750    enabled: false                      # bool: Census of realized contacts; before/after pruning when the run prunes
 751    training: false                     # bool: Must remain false
 752    params:
 753      save_figures: true                # bool: Per-branch and per-depth contact plots
 754
 755  # ---------- Noise perturbation ----------
 756  noise_perturbation:
 757    enabled: false                      # bool: Enable noise perturbation analysis
 758    training: false                     # bool: Also run during training
 759    params:
 760      uniform_noise: true               # bool: Uniform noise sweeps
 761      uniform_magnitudes: [0.1, 0.2]    # list[float]: Uniform noise magnitudes
 762      gaussian_noise: true              # bool: Gaussian noise sweeps
 763      gaussian_sdevs: [0.1, 0.2]       # list[float]: Gaussian noise standard deviations
 764      n_samples: 10                     # int: Noisy replicates per setting
 765      accuracy: true                    # bool: Compute accuracy
 766      auc: true                         # bool: Compute AUC
 767      categorical_loglikelihood: true   # bool: Compute categorical log-likelihood
 768      mse: true                         # bool: Compute MSE
 769      cosine_similarity: true           # bool: Compute cosine similarity
 770
 771  # ---------- Adversarial robustness ----------
 772  adversarial_robustness:
 773    enabled: false                      # bool: Enable adversarial robustness (FGSM)
 774    training: false                     # bool: Also run during training
 775    params:
 776      epsilons: [0.0, 0.05, 0.1, 0.15, 0.2, 0.3]  # list[float]: FGSM epsilon values
 777      batch_size: 256                   # int: Not used in training runs: the analysis runtime batch_size replaces it
 778
 779  # ---------- Weight analysis ----------
 780  weight:
 781    enabled: false                      # bool: Enable weight analysis
 782    training: false                     # bool: Also run during training
 783    params:
 784      synapse_weights: true             # bool: Analyze synapse weights
 785      branch_weights: true              # bool: Analyze branch-to-output weights
 786      computation_level: "single_branch"  # str: "single_branch" | "layer_branch" | "all_branch"
 787      branch_aggregation: "mean"        # str: "mean" | "multivariate" | "sample"
 788      per_layer_analysis: true          # bool: Per-layer breakdown
 789      per_einet_analysis: true          # bool: Separate E and I network summaries
 790      per_neuron_analysis: false        # bool: Per-neuron breakdown (expensive)
 791      analyze_excitatory: true          # bool: Analyze E weights
 792      analyze_inhibitory: true          # bool: Analyze I weights
 793      analyze_branch_output: true       # bool: Analyze branch-to-output weights
 794      compute_mean: true                # bool: Compute mean weights
 795      compute_variance: true            # bool: Compute weight variance
 796      compute_percentiles: true         # bool: Compute percentiles
 797      compute_min_max: true             # bool: Compute min/max
 798      compute_dendritic_strength: true  # bool: Compute total dendritic strength
 799      weight_threshold: 0.000001        # float: Minimum active weight
 800
 801  # ---------- Compartment statistics ----------
 802  compartment_statistics:
 803    enabled: false                      # bool: Enable compartment statistics
 804    training: false                     # bool: Also run during training
 805    params:
 806      synapse_weights: true             # bool: Analyze synapse weights
 807      branch_weights: true              # bool: Analyze branch weights
 808      inputs: true                      # bool: Analyze inputs
 809      activations: true                 # bool: Analyze activations
 810      global_analysis: true             # bool: Global-level statistics
 811      layer_analysis: true              # bool: Layer-level statistics
 812      branch_analysis: true             # bool: Branch-level statistics
 813      compute_mean: true                # bool: Compute mean
 814      compute_variance: true            # bool: Compute variance
 815      compute_percentiles: true         # bool: Compute percentiles
 816      compute_min_max: true             # bool: Compute min/max
 817      compute_entropy: true             # bool: Compute entropy
 818      per_class_analysis: false         # bool: Per-class statistics
 819      n_samples: null                   # int | null: Limit samples (null = all)
 820
 821  # ---------- Compartment SNR ----------
 822  # This is a discriminability analysis, not a simple signal/noise power ratio:
 823  # branch metrics report d'^2, layer/input metrics report d_max, d_total,
 824  # n_sig_dirs, and ratio_sig_dirs from between-class vs total covariance.
 825  compartment_snr:
 826    enabled: false                      # bool: Enable compartment SNR analysis
 827    training: false                     # bool: Also run during training
 828    params:
 829      input_analysis: true              # bool: Input-level statistics
 830      input_capture_mode: "branch"      # str: "branch" = dendritic branch inputs; "network" = network-level E/I input-output diagnostic
 831      layer_analysis: true              # bool: Layer statistics
 832      branch_analysis: true             # bool: Branch statistics
 833      global_aggregation: true          # bool: Aggregate global statistics
 834      layer_aggregation: true           # bool: Aggregate layer statistics
 835      branch_aggregation: true          # bool: Aggregate branch statistics
 836      epsilon: 0.000001                 # float: Numerical epsilon
 837      null_permutations: 100            # int: Label shuffles for n_sig_dirs and d_total_null_mean (0: off)
 838      null_quantile: 0.95               # float: Null quantile a significant direction must exceed
 839      null_seed: null                   # int | null: Shuffle seed (null: from the run's probe seed)
 840
 841  # ---------- Multiplicative gain ----------
 842  multiplicative_gain:
 843    enabled: false                      # bool: Enable multiplicative gain analysis
 844    training: false                     # bool: Also run during training
 845    params:
 846      uniform_gain: true                # bool: One gain per input (false: per-element gains in [1, g])
 847      gain_min: 0.0                     # float: Minimum gain
 848      gain_max: 10.0                    # float: Maximum gain
 849      n_steps: 10                       # int: Number of gain steps
 850      logspace: false                   # bool: Log-spaced gains ([1/gain_max, gain_max] when gain_min <= 0)
 851      noise_seed: null                  # int | null: Seed of random gains and noise (null: from the run's probe seed)
 852      accuracy: true                    # bool: Compute accuracy (default false; enabling the analyzer needs at least one score)
 853      auc: false                        # bool: Compute AUC
 854      categorical_loglikelihood: false  # bool: Compute categorical log-likelihood
 855      mse: false                        # bool: Compute MSE
 856      cosine_similarity: false          # bool: Compute cosine similarity
 857
 858  # ---------- Shared gain crossed with independent load ----------
 859  gain_load_perturbation:
 860    enabled: false                      # bool: Enable final-only gain/load grid
 861    training: false                     # bool: Must remain false
 862    params:
 863      gain_log_sds: [0.0, 0.2, 0.5, 0.8]  # list[float]: Mean-one lognormal gain s.d.
 864      load_levels: [0.0, 0.1, 0.3, 0.5]   # list[float]: Declared load levels
 865      load_mode: gaussian_input         # str: gaussian_input | positive_inhibitory
 866      load_log_sd: 0.2                  # float: Log-s.d. for positive I load
 867      inhibitory_start: null            # int | null: Start of ordered I stream
 868      draws_per_condition: 5            # int: Technical perturbation draws
 869      base_draw_seed: null              # int | null: null = derived from experiment.probe_seed
 870      max_samples: null                 # int | null: Evaluation sample cap
 871      clamp_min: 0.0                    # float | null: Lower input clamp
 872      clamp_max: null                   # float | null: Upper input clamp
 873      accuracy: true                    # bool: Record classification accuracy
 874      categorical_loglikelihood: true   # bool: Record categorical log-likelihood
 875      record_input_statistics: false    # bool: Record realized input moments per cell
 876      record_branch_local_sensitivity: false  # bool: Evaluate exact fitted branch derivatives
 877      branch_sensitivity_module_name_prefix: null  # str | null: One tree prefix
 878      branch_sensitivity_expected_n_somas: null     # int | null: Tree validation
 879      branch_sensitivity_include_soma: false        # bool: Include soma-relative d=0
 880      branch_sensitivity_max_samples: null           # int | null: Sensitivity sample cap
 881      branch_sensitivity_draws: [0]                 # list[int]: Technical draws analyzed
 882
 883  # ---------- Receptive field ----------
 884  # Default mode collects raw dendritic RF maps. Set compute_activation_tuning for
 885  # class-tuned E/I/Vout summaries, compute_component_rfs for compact soma-level
 886  # E/I mean and conditioned component RF maps, and compute_rf_tuning for
 887  # aggregate RF tuning, entropy, discriminability, and E-vs-I RF plots.
 888  receptive_field:
 889    enabled: false                      # bool: Enable receptive field visualization
 890    training: false                     # bool: Also run during training
 891    params:
 892      n_rows: 2                         # int: Grid rows for plotting
 893      n_cols: 2                         # int: Grid columns for plotting
 894      rf_shape: null                    # list[int] | null: Shared RF shape [H, W]
 895      exc_rf_shape: null                # list[int] | null: Excitatory RF shape [H, W]
 896      inh_rf_shape: null                # list[int] | null: Inhibitory RF shape [H, W]
 897      fig_save_format: png              # str: Figure extension, e.g. png or pdf
 898      compute_activation_tuning: false  # bool: Class-tuning summaries from branch E/I ratio and Vout activations
 899      activation_sample_signals: []     # list[str]: Optional "exc" | "inh" | "upstream" | "depolarizing_drive" | "vout" | "ei_ratio" sample records
 900      activation_sample_depths: []      # list[int]: Soma-relative depths retained in activation sample records
 901      activation_samples_per_class: 0   # int: Label-balanced records per class; 0 disables records
 902      activation_sample_seed: null      # int | null: null = derived from experiment.probe_seed
 903      compute_component_rfs: false      # bool: Compact soma-level E/I mean and conditioned component RF maps
 904      plot_component_rfs: true          # bool: Save compact component RF figure on final analysis
 905      compute_rf_tuning: false          # bool: Rich aggregate RF tuning, entropy, and discriminability analysis
 906      plot_rf_tuning: true              # bool: Save rich RF tuning plots on final analysis when compute_rf_tuning is true
 907      rf_tuning_layer_index: 0          # int: E/I layer index used for rich RF tuning
 908      n_class_examples: 5               # int: Examples per class in RF input-example plots
 909      save_raw_data: true               # bool: Save raw RF tensor checkpoint
 910      load_raw_data: false              # bool: Reuse raw RF checkpoint if present
 911      raw_filename: null                # str | null: Override raw RF checkpoint filename
 912      epsilon: 0.00000001               # float: Numerical epsilon for tuning ratios
 913
 914  # ---------- Single-layer contribution ----------
 915  single_layer_contribution:
 916    enabled: false                      # bool: Enable single-layer contribution analysis
 917    training: false                     # bool: Also run during training
 918    params:
 919      excitation_contribution: true     # bool: Excitation-only contribution
 920      inhibition_contribution: true     # bool: Inhibition-only contribution
 921      both_contribution: true           # bool: Combined E+I contribution
 922      accuracy: true                    # bool: Compute accuracy
 923      auc: true                         # bool: Compute AUC
 924      categorical_loglikelihood: true   # bool: Compute categorical log-likelihood
 925      mse: true                         # bool: Compute MSE
 926      cosine_similarity: true           # bool: Compute cosine similarity
 927
 928  # ---------- Synaptic activation ----------
 929  synaptic_activation:
 930    enabled: false                      # bool: Enable synaptic activation analysis
 931    training: false                     # bool: Also run during training
 932    params:
 933      network_summary: true             # bool: Network-level summary
 934      layer_summary: true               # bool: Layer-level summary
 935      branch_summary: true              # bool: Branch-level summary
 936      raw_summary: true                 # bool: Raw activation summary
 937      logspace: false                   # bool: Log-space reporting
 938      downsample: true                  # bool: Downsample activations
 939      samples: 5                        # int: Samples to keep
 940
 941  # ---------- Branch activation ----------
 942  branch_activation:
 943    enabled: false                      # bool: Enable branch activation analysis
 944    training: false                     # bool: Also run during training
 945    params:
 946      synapse_activations: true         # bool: Log synapse activations
 947      branch_activations: true          # bool: Log branch activations
 948      network_summary: true             # bool: Network-level summary
 949      layer_summary: true               # bool: Layer-level summary
 950      branch_summary: true              # bool: Branch-level summary
 951      raw_summary: true                 # bool: Raw activation summary
 952      logspace: false                   # bool: Log-space reporting
 953      downsample: true                  # bool: Downsample activations
 954      samples: 5                        # int: Samples to keep
 955
 956  # ---------- Local rule components ----------
 957  local_rule_components:
 958    enabled: false                      # bool: Enable LocalCA component-factor logging
 959    training: false                     # bool: Also run during training, for epoch trajectories
 960    params:
 961      max_samples: 1024                 # int | null: Examples per snapshot
 962      include_dendritic: true           # bool: Include branch-to-parent conductance terms
 963      include_broadcast: true           # bool: Log scalar/per-soma broadcast-error proxy
 964      broadcast_mode: "scalar"          # str: "scalar" | "per_soma"
 965      error_mode: "auto"                # str: "auto" | "ce" | "bce" | "mse" | "none"
 966      e_rev_exc: 1.0                    # float: Fixed at 1.0, the model's excitatory reversal
 967      e_rev_inh: 0.0                    # float: Fixed at 0.0, the model's inhibitory reversal
 968      save_json: true                   # bool: Save per-snapshot JSON
 969      save_csv: true                    # bool: Save per-snapshot CSV
 970      append_training_csv: true         # bool: Append training snapshots to trajectory.csv
 971
 972  # ---------- Spike train ----------
 973  spike_train:
 974    enabled: false                      # bool: Enable LIF spike-train analysis
 975    training: false                     # bool: Also run during training, for epoch trajectories
 976    params:
 977      dt: 1.0                           # float: Time per recurrent step for rate reporting
 978      threshold: 0.5                    # float: Binarization threshold for recorded spike outputs
 979      max_samples: 512                  # int | null: Cap examples used for analysis
 980      include_raster: true              # bool: Include compact raster event lists
 981      max_raster_samples: 8             # int: Number of examples to include in raster events
 982      include_per_timestep_rate: true   # bool: Include population firing-rate traces over time
 983
 984  # ---------- Routing ----------
 985  routing:
 986    enabled: false                      # bool: Enable routing analysis (recurrent models only)
 987    training: false                     # bool: Also run during training
 988
 989# ============================================================================
 990# OUTPUTS
 991# ============================================================================
 992outputs:
 993  run_name: "ei_sweep"                  # str: Run name; a run writes to <results_dir>/<run_name>_<YYYYMMDD_HHMMSS>/
 994  results_dir: "results"                # str: Parent directory of the run directories
 995  # Not shown: exact_run_dir (true: write into results_dir itself;
 996  # docs/runs_and_outputs.rst).
 997
 998# ============================================================================
 999# DISTRIBUTED TRAINING
1000# ============================================================================
1001distributed:
1002  # "none": one process (dendritic-train). "ddp": DistributedDataParallel,
1003  # launched with torchrun -m dendritic_modeling.scripts.training.train_experiments.
1004  # "fsdp": FullyShardedDataParallel, launched with torchrun -m
1005  # dendritic_modeling.scripts.training.train_experiments_fsdp (see
1006  # config_exp_fsdp.yaml). The launcher sets the number of nodes and
1007  # processes, not this config.
1008  mode: "none"                          # str: "none" | "ddp" | "fsdp"
1009  # Not shown: timeout_seconds (int | null), the collective timeout of ddp and
1010  # fsdp runs (null = PyTorch's default of 600 s under DDP, 1800 s under
1011  # FSDP); raise it when rank-0 analyses during training take long.
1012
1013  # FSDP settings (read only with mode: "fsdp"; docs/distributed_training.rst)
1014  fsdp:
1015    # FULL_SHARD: shard parameters, gradients and optimizer state (lowest memory)
1016    # SHARD_GRAD_OP: shard gradients and optimizer state only
1017    # NO_SHARD: no sharding (like DDP)
1018    # HYBRID_SHARD: shard within a node, replicate across nodes
1019    sharding_strategy: "FULL_SHARD"       # str: "FULL_SHARD" | "SHARD_GRAD_OP" | "NO_SHARD" | "HYBRID_SHARD"
1020    mixed_precision: true                 # bool: FSDP mixed precision; dtypes from mixed_precision_dtype (default "bf16")
1021    cpu_offload: false                    # bool: Offload parameters to CPU memory (saves GPU memory, slower)
1022    auto_wrap_policy: "none"              # str: "none" (whole model is one unit) | "indexed_synapses" (one unit per indexed synapse module) | "size"
1023    min_num_params: 1000000               # int: Size threshold of auto_wrap_policy "size" (other policies reject a non-default value)
1024    limit_all_gathers: true               # bool: Limit concurrent all-gathers
1025    forward_prefetch: true                # bool: Prefetch the next FSDP unit in forward
1026    sync_module_states: true              # bool: Broadcast rank 0's module states when wrapping
1027    use_orig_params: true                 # bool: Keep the original parameters (needed for the per-component parameter groups)
1028    backward_prefetch: "BACKWARD_PRE"     # str: "BACKWARD_PRE" | "BACKWARD_POST"
1029    gradient_checkpointing: false         # bool: Activation checkpointing (recompute activations in backward to save memory)

FSDP configuration

configs/config_exp_fsdp.yaml trains a population network with indexed synapses with FullyShardedDataParallel (distributed.mode: fsdp). Its sections are in alphabetical order, as in a saved config.json, and its comments explain the settings that matter under FSDP. Launch one process per GPU, here on a node with four GPUs:

torchrun --nproc_per_node=4 \
    -m dendritic_modeling.scripts.training.train_experiments_fsdp \
    configs/config_exp_fsdp.yaml

Distributed training describes multi-node launches and the FSDP options.

  1# ============================================================================
  2# DENDRINET ANNOTATED REFERENCE CONFIG (FSDP)
  3# ============================================================================
  4# A population network with indexed synapses on CIFAR-10, trained with
  5# FullyShardedDataParallel (distributed.mode: fsdp). Launch one process per GPU
  6# on one node from the repository root:
  7#   torchrun --nproc_per_node=<gpus> \
  8#     -m dendritic_modeling.scripts.training.train_experiments_fsdp \
  9#     configs/config_exp_fsdp.yaml
 10# Add --validate-only to build the datasets and the wrapped model and
 11# optimizer without training. docs/distributed_training.rst covers multi-node
 12# launches and every FSDP option.
 13#
 14# The sections are in alphabetical order, as in a saved config.json.
 15# config_exp.yaml describes every option; the comments here explain the
 16# settings that matter under FSDP. The FSDP trainer trains with
 17# backpropagation and a plain optimizer step. It rejects LR schedules,
 18# regularization, pruning, sparse-synapse maintenance (synapse_decay_rate must
 19# be 0 and weight_boosting false for Top-K and indexed synapses), rewiring
 20# synapses, and strategies other than "standard" and "recurrent".
 21
 22config_version: 4
 23
 24analysis:
 25  runtime:
 26    # Same keys as in config_exp.yaml. Under DDP and FSDP an omitted mode is
 27    # "auto"; "stream" never materializes a split, which suits large datasets.
 28    training:
 29      mode: "auto"                      # "auto" materializes splits up to materialize_threshold_mb and streams larger ones
 30      batch_size: 256
 31      num_workers: 4
 32      pin_memory: true
 33      persistent_workers: true
 34      prefetch_factor: 4
 35      in_order: false                   # false: batches arrive as workers finish them
 36      max_batches: null
 37      max_samples: null
 38      materialize_threshold_mb: 1024
 39    final:
 40      mode: "stream"                    # Stream every split in batches
 41      batch_size: 256
 42      num_workers: 4
 43      pin_memory: true
 44      persistent_workers: true
 45      prefetch_factor: 4
 46      in_order: false
 47      max_batches: null
 48      max_samples: null
 49      materialize_threshold_mb: 1024
 50  ablation:
 51    enabled: false
 52    params:
 53      selection:
 54        levels: ["layer"]              # Levels: "layer", "compartment" (compartment is lesion-only)
 55        targets: ["excitation", "inhibition"]  # Targets: "all_synapses", "excitation", "inhibition", "upstream"
 56        metrics: ["accuracy", "auc", "categorical_loglikelihood"]  # Metrics: "accuracy", "auc", "categorical_loglikelihood", "mse", "cosine_similarity", "pred_label_mi_bits"
 57
 58      methods:
 59        ablation_methods: ["lesion"]  # Methods: "lesion", "shuffle", "mean_clamp"
 60        shuffle_seed: 0               # Shuffle seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
 61        clamp_statistic: "mean"       # mean_clamp statistic: "mean" (the only supported value)
 62    training: false
 63  branch_activation:
 64    enabled: false
 65    params:
 66      branch_activations: true
 67      branch_summary: true
 68      downsample: true
 69      layer_summary: true
 70      logspace: false
 71      network_summary: true
 72      raw_summary: true
 73      samples: 5
 74      synapse_activations: true
 75    training: false
 76  correlation:
 77    enabled: false
 78    params:
 79      selection:
 80        components: ["total", "noise", "signal", "tuning"]  # Components: "total", "noise", "signal", "tuning"
 81        pairs: ["EE", "II", "EI", "E_output", "I_output", "output_output"]  # Pairs: "EE", "II", "EI", "E_output", "I_output", "output_output"
 82        scopes: ["layer", "einet"]  # Scopes: "layer", "einet", "neuron"
 83
 84      compute:
 85        computation_level: "layer_branch"  # Level: "synaptic", "single_branch", "layer_branch", "all_branch"
 86        max_samples: 5000                 # Max samples (null = all)
 87        per_layer_analysis: true          # Extra breakdown/plots: per-layer (overridden by selection.scopes if non-empty)
 88        per_einet_analysis: true          # Extra breakdown/plots: per-E/I-net (overridden by selection.scopes if non-empty)
 89        per_neuron_analysis: false        # Extra breakdown/plots: per-neuron (expensive; overridden by selection.scopes if non-empty)
 90
 91      sampling:
 92        n_synapse_pairs: 10000             # Synaptic-only: number of synapse pairs (int >= 1)
 93        sampling_strategy: "stratified"    # Sampling: "stratified", "random" (other values raise)
 94        stratified_categories: ["within_branch", "within_layer"]  # Stratified categories: "within_branch", "within_layer"
 95        max_pairs_per_category: 1000       # Stratified-only: max pairs per category (int >= 1)
 96    training: false
 97  information:
 98    enabled: false
 99    params:
100      selection:
101        components: ["basic", "pairwise", "conditional", "ablation_aligned", "layer_proxies"]  # Components: "basic", "pairwise", "conditional", "gaussian_fisher", "pid", "ablation_aligned", "layer_proxies", "upstream_unique_cmi", "soma_coupling"
102        scopes: []                        # Scopes: "layer", "einet", "neuron" (alias: views)
103        signal_variants: ["network", "lda"]  # Variants: "network", "lda", "random" (random requires baselines.random_weights.n_shuffles > 0)
104
105      compute:
106        max_samples: null                 # Max samples (null = all)
107        include_vinf: true                # Include soma activation: true|false
108        normalize_mi: false               # Normalize by entropy: true|false
109        verbose: false                    # Verbose: true|false
110        computation_level: layer_branch   # Level: "single_branch", "layer_branch", "all_branch"
111        branch_aggregation: multivariate  # Aggregation: "mean", "multivariate", "sample"
112        per_layer_analysis: false         # Extra breakdown/plots: per-layer (overridden by selection.scopes if non-empty)
113        per_einet_analysis: false         # Extra breakdown/plots: per-E/I-net (overridden by selection.scopes if non-empty)
114        per_neuron_analysis: false        # Extra breakdown/plots: per-neuron (expensive; overridden by selection.scopes if non-empty)
115        layer_total_topk: 10              # K for *_topK_sum proxies (int >= 1)
116        gaussian_fisher:
117          aggregation: "sum"  # Aggregate per-feature F: "sum"|"mean"|"max"
118          mi_transform: "half_log1p"  # Transform F→MI proxy: "half_log1p"|"log1p"|"none"
119          eps: 1.0e-8  # Denominator stabilizer (alias: ridge) (float > 0)
120
121      pid:
122        binarize: false  # Threshold E, I and Vout (never the labels) before PID: true|false
123        binarize_method: median  # Binarization: "median"|"mean"|"quantile"
124        binarize_threshold: 0.5  # Used only if binarize_method="quantile": q∈[0,1]
125
126      estimator:
127        method: kraskov                   # MI estimation: "kraskov", "decoder", "binned", "sklearn", "copula", "auto"
128        kraskov:
129          n_neighbors: 10                 # Kraskov k-NN (int >= 1)
130        binned:
131          n_bins: 20                      # Histogram bins of the binned estimator; the sklearn estimator uses it too (int >= 2)
132          binning_strategy: "quantile"    # binned: "quantile" or "uniform"
133          bias_correction: "none"         # binned: "none" or "miller_madow"
134        decoder:
135          cv_folds: 5                     # decoder: CV folds (int >= 2)
136          C: 1.0                          # decoder: LogisticRegression C (float > 0)
137          standardize: true               # decoder: standardize X (true|false)
138          seed: 0                         # decoder: CV shuffle seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
139          continuous:
140            strategy: "none"              # decoder continuous: "none", "binned", "gaussian"
141            max_total_dim: 2              # decoder continuous: max total dim (x+y [+z]); used only if strategy != "none"
142            gaussian_ridge: 1e-6          # decoder continuous: gaussian ridge (float > 0)
143        copula:
144          type: gaussian                  # Copula: "gaussian" (scalars, vectors, labels), "kernel" (two continuous scalars)
145        sklearn:                          # Not read: the sklearn estimator takes its bins from estimator.binned.n_bins
146          n_bins: 20                      # Not read
147          bandwidth: null                 # Not read
148
149      baselines:
150        random_weights:
151          n_shuffles: 0                   # Random-weight baseline shuffles (int >= 0; produces *_sh metrics)
152          seed: 42                        # Random-weight seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
153        label_shuffle_null:
154          n_shuffles: 0                   # Null (label-shuffle) baseline shuffles (int >= 0; produces *_null metrics)
155          seed: 0                         # Null baseline seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
156          components: ["basic"]           # Null components: "basic", "pairwise", "conditional", "ablation_aligned", "upstream_unique_cmi", "soma_coupling", "all"
157    training: false
158  noise_perturbation:
159    enabled: false
160    params:
161      accuracy: true
162      auc: true
163      categorical_loglikelihood: true
164      cosine_similarity: false
165      gaussian_noise: true
166      gaussian_sdevs:
167      - 0.1
168      - 0.2
169      mse: false
170      n_samples: 10
171      uniform_magnitudes:
172      - 0.1
173      - 0.2
174      uniform_noise: true
175    training: false
176  performance:
177    enabled: true
178    params:
179      accuracy: true
180      auc: true
181      categorical_loglikelihood: true
182      cosine_similarity: false
183      mse: false
184    # training: true runs the performance analysis after every epoch on rank 0
185    # while the other ranks wait in a collective (distributed.timeout_seconds,
186    # default 1800 s under FSDP, bounds the wait).
187    training: true
188    training_splits: ["train", "validation", "test"]   # use ["validation"] for cheaper training-time checks
189    final_splits: ["train", "validation", "test"]
190  single_layer_contribution:
191    enabled: false
192    params:
193      accuracy: true
194      auc: true
195      both_contribution: true
196      categorical_loglikelihood: true
197      cosine_similarity: false
198      excitation_contribution: true
199      inhibition_contribution: true
200      mse: false
201    training: false
202  synapse_turnover:
203    enabled: false
204    training: false                     # Set to true before enabling: turnover compares training snapshots, and training: false raises
205    params:
206      snapshot_interval: 10
207  synaptic_activation:
208    enabled: false
209    params:
210      branch_summary: true
211      downsample: true
212      layer_summary: true
213      logspace: false
214      network_summary: true
215      raw_summary: true
216      samples: 5
217    training: false
218  weight:
219    enabled: false
220    params:
221      analyze_branch_output: false
222      analyze_excitatory: true
223      analyze_inhibitory: true
224      branch_aggregation: mean
225      branch_weights: true
226      computation_level: single_branch
227      compute_dendritic_strength: true
228      compute_mean: true
229      compute_min_max: true
230      compute_percentiles: false
231      compute_variance: true
232      per_einet_analysis: true
233      per_layer_analysis: true
234      per_neuron_analysis: false
235      synapse_weights: true
236      weight_threshold: 1e-6
237    training: false
238  compartment_statistics:
239    enabled: true                       # Enabled in this example (default false)
240    training: false
241    params:
242      synapse_weights: true
243      branch_weights: true
244      inputs: true
245      activations: true
246      global_analysis: true
247      layer_analysis: true
248      branch_analysis: true
249      compute_mean: true
250      compute_variance: true
251      compute_percentiles: true
252      compute_min_max: true
253      compute_entropy: true
254      per_class_analysis: false
255      n_samples: 1000
256data:
257  base_dir: ''                          # Data root (empty = $DENDRITIC_DATA_DIR, else ~/.cache/dendritic_modeling)
258  dataset_name: cifar10
259  dataset_params:                       # Only the block named by dataset_name is read; these two are ignored with cifar10
260    mnist_modulo10:
261      shuffle_iterations: 2
262    poisson_generator:
263      base_dataset: mnist
264  processing:
265    flatten: true
266    normalize: false
267experiment:
268  checkpointing:
269    enabled: false                      # true: the FSDP trainer saves checkpoints (with optimizer state) under checkpoints/ in the run directory
270    save_every_n_epochs: 10
271  seed: 42
272  train_valid_split: 0.8
273distributed:
274  mode: fsdp                            # Train with FSDP: launch train_experiments_fsdp under torchrun (see the header)
275  fsdp:
276    backward_prefetch: BACKWARD_POST    # "BACKWARD_PRE" (default) | "BACKWARD_POST"
277    cpu_offload: false
278    forward_prefetch: true
279    gradient_checkpointing: false
280    limit_all_gathers: true
281    mixed_precision: false              # Full-precision parameters, reductions and buffers (default true: bf16)
282    sharding_strategy: FULL_SHARD       # "FULL_SHARD" | "SHARD_GRAD_OP" | "NO_SHARD" | "HYBRID_SHARD"
283    sync_module_states: true
284    use_orig_params: true               # Needed for the per-component learning rates (training.main.common.param_groups)
285  # Not set here: auto_wrap_policy ("none" makes the whole model one FSDP unit;
286  # "indexed_synapses" wraps each indexed synapse module, which bounds the
287  # parameter all-gathers of contact-heavy models), mixed_precision_dtype and
288  # the per-tensor dtypes, freeze_core, cache_frozen_core_outputs,
289  # cached_core_batch_size, and distributed.timeout_seconds.
290model:
291  core:
292    biological_neuron: true   # Enforce the neuronal-core constraint set.
293    type: population_network
294    population_network:
295      input_transform: nonnegative     # Check nonnegative data without changing values or gradients.
296      use_transfer: false
297      transfer_params: {}
298      output_mode: "last"
299      input_projection_dims: []
300      store_routing: false
301      layers:
302      - name: layer0
303        readout_population: e
304        population_defaults:
305          ff_excitatory_synapses: 40
306          ff_inhibitory_synapses: 10
307          rec_excitatory_synapses: 0
308          rec_inhibitory_synapses: 0
309          somatic_synapses: true
310          use_shunting: true
311          reactivate: true
312          reactivation_type: param_tanh
313          reactivation_init_policy: occupancy_quantile  # Data-driven gate placement (the default for shunting populations is "analytical")
314          dbl_init_method: analytical_expectation
315          weight_transform: softplus
316          # FSDP has no rewiring step, so rewiring synapses (indexed_rewire,
317          # deepst, credit_deepst, dense_to_sparse with advance_on_forward:
318          # false) raise ValueError.
319          topk_type: indexed                # Indexed synapses store only the active contacts
320          topk_init_method: xavier_normal
321          indexed_output_chunk_size: 2048
322          efficient_blocklinear: true
323        populations:
324        - name: i
325          polarity: inhibitory
326          n_neurons: 32
327          branch_factors: [1]
328          population:
329            ff_inhibitory_synapses: 0
330        - name: e
331          polarity: excitatory
332          n_neurons: 128
333          branch_factors: [3, 3, 3]
334        connections:
335        - source: input
336          target: i
337        - source: input
338          target: e
339        - source: i
340          target: e
341  decoder:
342    params: {}                          # MLP defaults: hidden_dims [16], output_dim 10; input_dim comes from the core
343    type: MLP
344  encoder:
345    params: {}
346    type: identity
347  task: classification
348outputs:
349  results_dir: results
350  run_name: fsdp_large_exp
351training:
352  main:
353    common:
354      num_workers: 8
355      pin_memory: true
356      persistent_workers: true
357      prefetch_factor: 4
358      param_groups: {}
359      synapse_decay_rate: 0.0           # Must be 0: FSDP runs no sparse-synapse maintenance (weight_boosting keeps its default, false)
360    pruning: {}
361    regularization: {}
362    strategies:                         # Only the block of the selected strategy is read
363      freeze_branch_kl: {}
364      freeze_branches: {}
365      freeze_layers: {}
366      homeostatic_control: {}
367      multi_stage: {}
368      standard: {}
369      voltage_stabilization: {}
370    strategy: standard                  # The FSDP trainer runs "standard" or "recurrent"

Sweep template

configs/sweep_config.yaml is the annotated template of a parameter sweep: a base_config and the options to vary (see Parameter sweeps). Generate its configs without submitting jobs:

dendritic-sweep --config configs/sweep_config.yaml --generate-only --output-dir sweeps
  1# ============================================================================
  2# SWEEP CONFIGURATION TEMPLATE
  3# ============================================================================
  4# Annotated template for parameter sweeps with DendriNet. docs/sweeps.rst
  5# describes every section, option and command.
  6#
  7# USAGE:
  8#   Generate the configs and job scripts without submitting (an empty account
  9#   or partition becomes PLACEHOLDER):
 10#     dendritic-sweep --config configs/sweep_config.yaml --generate-only
 11#
 12#   Generate the configs and submit the Slurm jobs (account and partition are
 13#   required, here or in slurm_config):
 14#     dendritic-sweep --config configs/sweep_config.yaml \
 15#       --slurm-account ACCOUNT --slurm-partition PARTITION
 16#
 17#   Train one generated config without Slurm:
 18#     dendritic-train <sweep_dir>/configs/unified_config_0.yaml
 19#
 20#   Tabulate and plot the finished runs (pass the sweep directory, not its
 21#   results/ subdirectory):
 22#     dendritic-analyze <sweep_dir>
 23#
 24# WHAT CAN BE SWEPT:
 25#   - Any option of base_config, through its dot path (sweep_config)
 26#   - E/I synapse-count pairs that satisfy a constraint (filter_config, A)
 27#   - Dendritic tree depths and branching patterns (filter_config, B)
 28#   - Named conditions that change several options together (sweep_variants)
 29#   All of these combine in one grid.
 30#
 31# HOW IT WORKS:
 32#   The sweep manager takes the Cartesian product of the sweep_config axes and
 33#   the sweep_variants, applies filter_config, and writes one training config
 34#   per condition and replicate seed to <sweep_dir>/configs. Every generated
 35#   config is loaded with the strict config loader before any job script is
 36#   written; the models are not built. The sections of this file are checked
 37#   when it loads: an unknown key in slurm_config, experiment_settings,
 38#   filter_config or outputs raises with the closest valid key, and so does a
 39#   top-level key that nearly matches a section name. Other top-level keys are
 40#   reported in a warning and ignored; put annotations under metadata:.
 41#
 42# TOP-LEVEL SECTIONS:
 43#   slurm_config          - Slurm resources, environment and launcher
 44#   experiment_settings   - Replicate seeds (seeds_per_condition, base_seed, seed_paths)
 45#   sweep_config          - Axes: dot paths into base_config -> values or ranges
 46#   sweep_variants        - Optional named conditions that change several paths together
 47#   filter_config         - E/I pair filter and branch sweeps
 48#   base_config           - Complete training config shared by all conditions
 49#   execution_results_dir - Optional separate root for the run outputs
 50#   sweep_contract        - Optional expected_config_count, checked at generation
 51#   output_dir, outputs   - Optional base directory (output_dir or outputs.dir;
 52#                           default base_config.outputs.results_dir) and name
 53#                           prefix (outputs.run_name; default
 54#                           base_config.outputs.run_name) of the sweep directory
 55#   extends               - Optional parent sweep file, relative to this file
 56#                           (mappings are merged, lists are replaced)
 57# ============================================================================
 58
 59
 60# ============================================================================
 61# 1. SLURM CONFIGURATION
 62# ============================================================================
 63# The #SBATCH header, environment and launch command of every job script of
 64# the sweep. By default all configs run as one array job, one config per array
 65# task (--split-jobs N writes N sequential jobs instead). The job name is the
 66# sweep's prefix (base_config.outputs.run_name here).
 67#
 68# account, partition: required to submit; set them here or pass
 69#   --slurm-account and --slurm-partition. They become PLACEHOLDER only with
 70#   --generate-only or --dry-run. A site policy file
 71#   (DENDRITIC_SCHEDULER_PROFILES, or
 72#   $XDG_CONFIG_HOME/dendritic_modeling/scheduler_profiles.yaml) may restrict
 73#   the allowed account/partition pairs.
 74# launcher: "torchrun" runs torch.distributed.run --standalone with
 75#   gpus_per_node processes, one node per config. Use it with
 76#   base_config.distributed.mode: ddp, or with
 77#   training_script: src/dendritic_modeling/scripts/training/train_experiments_fsdp.py
 78#   and distributed.mode: fsdp.
 79# Every job runs "module purge" and then "module load" for modules_to_load.
 80# Two more optional keys, qos and constraint, add --qos and --constraint.
 81# An unknown slurm_config key raises when the sweep loads, naming the closest
 82# valid key.
 83
 84slurm_config:
 85  account: ""                                # Slurm account (required to submit)
 86  partition: ""                              # Slurm partition (required to submit)
 87  time: "12:00:00"                           # Per job or array task: quoted HH:MM:SS or D-HH:MM:SS (default 02:00:00)
 88  nodes: 1                                   # Nodes per job; keep 1
 89  ntasks_per_node: 1                         # Tasks per node
 90  gpus_per_node: 1                           # --gres=gpu:N; also the torchrun process count
 91  launcher: "python"                         # "python" or "torchrun" (see above)
 92  cpus_per_task: 12                          # CPU cores per array task (default 4)
 93  mem: 200GB                                 # Memory per node, passed to Slurm as written (default 32G)
 94  max_concurrent_jobs: 32                    # Max array tasks running at once (default 10)
 95  exclude_nodes: []                          # Nodes to exclude (e.g., ["node001"]); --exclude-nodes replaces it
 96  modules_to_load: []                        # Environment modules loaded after "module purge"
 97  conda_env_path: ""                         # Environment root whose bin/python trains; empty = the "python" on PATH after "module purge"
 98  training_script: "src/dendritic_modeling/scripts/training/train_experiments.py"  # Relative to the library checkout
 99
100
101# ============================================================================
102# 2. EXPERIMENT SETTINGS
103# ============================================================================
104# Replicate seeds. Replicate r of every condition gets
105# experiment.seed = base_seed + r, so all conditions use the same seeds.
106# Total configs = conditions x seeds_per_condition (here 8 x 5 = 40); they run
107# as one array job.
108
109experiment_settings:
110  seeds_per_condition: 5                     # Replicates per condition (default 1)
111  base_seed: 0                               # Seeds 0, 1, ..., 4 (default base_seed 42)
112  # Random streams the base config leaves unset (data split, initialization,
113  # topology, data order) follow experiment.seed, so replicates also differ in
114  # connectivity. Set base_config.experiment.topology_seed to hold the
115  # connection masks and indexed-synapse indices fixed. seed_paths writes the
116  # replicate seed into further paths, e.g. a component seed the base config
117  # pins:
118  # seed_paths:
119  #   - model.core.population_network.layers.0.population_defaults.indexed_seed
120
121# Optional: keep the sweep files (configs, job scripts, logs) in the sweep
122# directory and write the run outputs under this separate root. Each generated
123# config writes to <execution_results_dir>/config_<index>, and
124# <sweep_dir>/results links to the root. Generation does not create the root.
125# execution_results_dir: "/project/results/my_sweep"
126
127# Optional named conditions. Each variant is one condition whose overrides are
128# applied together; every variant is crossed with the sweep_config axes. A
129# variant may change only paths the base core type reads: a
130# population_network base takes overrides under model.core.population_network.
131# This pair doubles the grid (2 x 8 = 16 conditions):
132# sweep_variants:
133#   - name: shunting
134#     overrides:
135#       model.core.population_network.layers.0.population_defaults.use_shunting: true
136#       model.core.population_network.layers.0.population_defaults.use_additive_normalization: false
137#   - name: normalized_additive
138#     overrides:
139#       model.core.population_network.layers.0.population_defaults.use_shunting: false
140#       model.core.population_network.layers.0.population_defaults.use_additive_normalization: true
141
142
143# ============================================================================
144# 3. SWEEP CONFIG -- Parameter Grid
145# ============================================================================
146# Each key is a dot path into base_config. Each value is a list of literal
147# values, a scalar (one value), or an explicit range:
148#   {range: {start, stop, step}}     start, start + step, ..., up to and including stop
149#   {linspace: {start, stop, num}}   num evenly spaced values, both ends included
150#   {geomspace: {start, stop, num}}  num geometrically spaced values, both ends included
151#   {values: [...]}                  the literal values
152# A plain three-number list that reads as [start, stop, step], such as
153# [1, 10, 2], is rejected as ambiguous. The sweep manager computes the full
154# Cartesian product of the axes.
155#
156# SYNTAX:
157#   dot.path.to.param: [val1, val2, val3]
158#
159# The dot path mirrors the YAML hierarchy of base_config; numeric components
160# index lists. For example:
161#   training.main.common.param_groups.topk_lr -> base_config.training.main.common.param_groups.topk_lr
162#   data.processing.flatten                   -> base_config.data.processing.flatten
163#   model.core.population_network.layers.0.populations.1.n_neurons
164#                                             -> n_neurons of population "e" in layer "layer0"
165#
166# VALUE TYPES (each line works with this file's base_config):
167#   Numbers:    training.main.common.param_groups.topk_lr: [0.001, 0.01, 0.1]
168#   Strings:    model.core.population_network.layers.0.population_defaults.weight_transform: ["softplus", "exp"]
169#   Booleans:   model.core.population_network.layers.0.population_defaults.somatic_synapses: [true, false]
170#   Lists:      model.core.population_network.layers.0.populations.1.branch_factors: [[3, 3], [4, 4]]
171#   Ranges:     training.main.common.batch_size: {range: {start: 128, stop: 512, step: 128}}
172#
173# LEARNING RATES: with split_params: true (see base_config), the sparse
174# synapses, BlockLinear couplings, reactivation gates and decoder train at
175# topk_lr, blocklinear_lr, reactivation_lr and decoder_lr. param_groups.lr
176# sets no parameter of this model, so sweep the group rates instead.
177#
178# EXAMPLES (each replaces the axes below; counts are conditions):
179#   # Sweep the sparse-synapse learning rate and batch size (2 x 3 = 6 conditions):
180#   # training.main.common.param_groups.topk_lr: [0.001, 0.01]
181#   # training.main.common.batch_size: [128, 256, 512]
182#
183#   # Sweep population size and morphology (2 x 2 = 4 conditions):
184#   # model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
185#   # model.core.population_network.layers.0.populations.1.branch_factors: [[3, 3], [4, 4]]
186#
187#   # Sweep E/I synapse counts in population-network defaults (3 x 3 = 9 conditions):
188#   # model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 30, 40]
189#   # model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 15, 20]
190
191sweep_config:
192  # Population-network synapse counts and morphology (2 x 2 x 2 = 8
193  # conditions). Numeric path components index YAML lists, so layers.0 is the
194  # first population-network layer and populations.1 is the excitatory
195  # readout population "e" of the base config.
196  model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses:
197    - 20
198    - 30
199
200  model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses:
201    - 10
202    - 15
203
204  model.core.population_network.layers.0.populations.1.branch_factors:
205    - [3, 3]
206    - [4, 4]
207
208
209# ============================================================================
210# 4. FILTER CONFIG -- E/I Filtering and Branch Sweeps
211# ============================================================================
212# Two independent mechanisms that can be used alone or combined:
213#
214# A) E/I FILTERING -- keep only some excitatory/inhibitory synapse-count pairs
215# B) BRANCH SWEEP  -- add a dimension of dendritic trees to the grid
216#
217# --------------------------------------------------------------------------
218# A) E/I FILTERING
219# --------------------------------------------------------------------------
220# sweep_config must contain both E/I axes: the population_defaults
221# ff_excitatory_synapses and ff_inhibitory_synapses paths (population
222# networks) or model.core.connectivity.ee_synapses_per_branch_per_layer and
223# ie_synapses_per_branch_per_layer (structured E/I cores). The other axes are
224# crossed with the pairs that remain.
225#
226#   mode: "all"          - No filtering (the default; "default" is the same)
227#   mode: "ei-sum"       - Keep pairs with |E + I - value| <= tolerance
228#   mode: "ei-ratio"     - Keep pairs with I > 0 and |E / I - value| <= tolerance
229#   mode: "e-const"      - Keep E = value with every I (value must be on the E axis)
230#   mode: "i-const"      - Keep I = value with every E (value must be on the I axis)
231#   mode: "branch_sweep" - No filtering; required for a branch sweep without
232#                          sweep_config axes or sweep_variants
233#
234#   value: <number>      - Target value; required by the four E/I modes
235#   tolerance: <number>  - Required by ei-sum and ei-ratio (0 = exact match)
236#   A filter that keeps no pair raises.
237#
238# EXAMPLES (with E and I axes of [10, 20, 30, 40] each):
239#   # Keep the pairs with E + I = 50 (4 pairs):
240#   # mode: "ei-sum"
241#   # value: 50
242#   # tolerance: 0
243#
244#   # Keep the E/I ratios within 0.5 of 2.0 (3 pairs):
245#   # mode: "ei-ratio"
246#   # value: 2.0
247#   # tolerance: 0.5
248#
249#   # Hold inhibition at 10 (4 pairs):
250#   # mode: "i-const"
251#   # value: 10
252#
253# --------------------------------------------------------------------------
254# B) BRANCH SWEEP
255# --------------------------------------------------------------------------
256# Each generated tree replaces branch_factors of every excitatory population
257# (polarity "excitatory" or no polarity) in every population_network layer
258# (model.core.architecture.excitatory_branch_factors for structured E/I
259# cores). Remove any sweep_config axis on those branch_factors first: the tree
260# overwrites it, and conditions that differ only in that axis become
261# identical runs. A factor b at depth d gives d levels of b branches. Depths
262# and factors are axis values: literal lists or explicit ranges (section 3).
263#
264# branch_sweep_mode options:
265#
266#   "uniform_sweep"   - Every depth with every branch factor (default mode)
267#     dendrite_tree_depth: {range: {start: 1, stop: 4, step: 1}}
268#     dendrite_branches_per_layer: {values: [2, 3]}
269#     Example: depths 1-4 x factors 2, 3 -> 8 trees ([2], [3], [2, 2], ...)
270#
271#   "depth_sweep"     - Vary the depth with one branch factor at every level
272#     dendrite_tree_depth: {range: {start: 2, stop: 5, step: 1}}
273#     uniform_branches_per_layer: 4
274#     Example: [4, 4], [4, 4, 4], [4, 4, 4, 4], [4, 4, 4, 4, 4] -> 4 trees
275#
276#   "width_sweep"     - Vary the branch factor at a fixed depth
277#     fixed_dendrite_tree_depth: 3
278#     dendrite_branches_per_layer: {range: {start: 2, stop: 8, step: 2}}
279#     Example: [2, 2, 2], [4, 4, 4], [6, 6, 6], [8, 8, 8] -> 4 trees
280#
281#   "pattern_sweep"   - Named branching patterns across depths
282#     dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
283#     base_branches: 3
284#     branch_patterns:
285#       - "uniform"      # base_branches at every level, e.g. [3, 3, 3]
286#       - "decreasing"   # base_branches, base_branches - 1, ..., at least 1, e.g. [3, 2, 1]
287#     Example: 3 depths x 2 patterns -> 6 trees
288#     Two more patterns ignore base_branches: "increasing" (1, 2, ..., depth)
289#     and "pyramid" (1, 2, ... up to level depth // 2, then depth - level:
290#     [1, 2, 1] at depth 3, [1, 2, 3, 1] at depth 4).
291#
292#   "custom_configs"  - Explicit trees, each a mapping with depth and branches
293#     custom_branch_configs:
294#       - {depth: 3, branches: [4, 4, 4]}
295#       - {depth: 3, branches: [8, 4, 2]}
296#       - {depth: 3, branches: [2, 4, 8]}
297#     Example: 3 trees, each used as written
298#
299# COMBINING E/I FILTER + BRANCH SWEEP:
300#   Set the E/I mode and branch_sweep_mode together. The total grid is:
301#     (sweep_config Cartesian product, with the E/I axes reduced to the
302#      pairs the filter keeps)
303#       x (branch sweep trees)
304#       x (sweep_variants)
305#       x seeds_per_condition
306#
307# --------------------------------------------------------------------------
308
309filter_config:
310  # E/I filtering (disabled by default)
311  mode: "all"                                # "all" | "ei-sum" | "ei-ratio" | "e-const" | "i-const" | "branch_sweep"
312  # value: 50                                # Target value (required by the E/I modes)
313  # tolerance: 0                             # Allowed deviation (required by ei-sum and ei-ratio; 0 = exact)
314
315  # Branch sweep (disabled by default). Before uncommenting these lines,
316  # delete the populations.1.branch_factors axis in sweep_config, which the
317  # trees replace (2 x 2 E/I conditions x 4 trees = 16 conditions):
318  # branch_sweep_mode: "depth_sweep"
319  # dendrite_tree_depth: {range: {start: 1, stop: 4, step: 1}}
320  # uniform_branches_per_layer: 3
321
322
323# ============================================================================
324# 5. BASE CONFIG -- Training Configuration Template
325# ============================================================================
326# The complete training config shared by all sweep conditions, in the schema
327# of config_exp.yaml. The sweep axes overwrite the values at their paths in
328# each generated config.
329
330base_config:
331  config_version: 4
332
333  # ---------- Experiment ----------
334  experiment:
335    seed: 42                                 # Replaced in each generated config by base_seed + replicate index
336    train_valid_split: 0.8
337
338    checkpointing:
339      enabled: false
340      save_every_n_epochs: 10
341
342  # ---------- Data ----------
343  data:
344    dataset_name: "mnist"
345    base_dir: ""
346
347    processing:
348      flatten: true
349      normalize: true
350
351  # ---------- Model ----------
352  model:
353    task: "classification"
354
355    encoder:
356      type: "identity"
357      params:
358        input_dim: 784
359        input_shape: [1, 28, 28]
360        latent_dim: null
361
362    core:
363      biological_neuron: false  # Preserve this example's historical model contract.
364      type: "population_network"
365      population_network:
366        use_transfer: false
367        transfer_params: {}
368        layers:
369          - name: "layer0"
370            readout_population: "e"
371            population_defaults:
372              ff_excitatory_synapses: 30
373              ff_inhibitory_synapses: 15
374              rec_excitatory_synapses: 0
375              rec_inhibitory_synapses: 0
376              somatic_synapses: true
377              use_shunting: true
378              use_additive_normalization: false
379              reactivate: true
380              reactivation_type: "param_tanh"
381              reactivation_init_policy: "occupancy_quantile"
382              dbl_init_method: "analytical_expectation"
383              weight_transform: "softplus"
384              topk_type: "indexed_rewire"
385              topk_init_method: "xavier_normal"
386              indexed_rewire_frequency: 100
387              indexed_rewire_quantile: 0.05
388              indexed_rewire_until_step: null
389              indexed_output_chunk_size: 2048
390              blocklinear_gradient_scaling: "none"
391              reactivation_gradient_scaling: "none"
392              efficient_blocklinear: true
393            populations:
394              - name: "i"
395                polarity: "inhibitory"
396                n_neurons: 32
397                branch_factors: [1]
398                population:
399                  ff_inhibitory_synapses: 0
400              - name: "e"
401                polarity: "excitatory"
402                n_neurons: 128
403                branch_factors: [4, 4]
404            connections:
405              - source: "input"
406                target: "i"
407              - source: "input"
408                target: "e"
409              - source: "i"
410                target: "e"
411
412    decoder:
413      type: "MLP"
414      params:
415        input_dim: null
416        hidden_dims: [32, 16]
417        activation: "relu"
418        output_dim: 10
419
420  # ---------- Training ----------
421  training:
422
423    main:
424      strategy: "standard"
425
426      common:
427        epochs: 200
428        batch_size: 512
429        shuffle: true
430        grad_clip_value: 5.0
431        loss_function: "ce"
432        early_stopping: false
433        patience: 500
434        load_best_state_dict: true
435        plot_losses: false
436        suppress_prints: false
437        print_every: 10
438        use_amp: false
439        enable_nan_checking: false
440        synapse_decay_rate: 0.01
441        weight_boosting: true
442
443        # With split_params: true the optimizer trains the sparse synapses,
444        # BlockLinear couplings, reactivation gates and decoder at their group
445        # rates; lr covers only the parameters outside those groups (none in
446        # this model).
447        param_groups:
448          lr: 0.001
449          split_params: true
450          topk_lr: 0.001
451          blocklinear_lr: 0.0001
452          reactivation_lr: 0.0001
453          decoder_lr: 0.001
454
455      regularization:
456        l1_weight: 0.0
457        l2_weight: 0.0
458        enforce_ei_weight_ratio: false
459
460      pruning:
461        enabled: false
462
463      strategies:
464        standard: {}
465
466  # ---------- Analysis ----------
467  # Analysis sections use the analyzer names, as in config_exp.yaml. The
468  # performance analyzer writes performance/final.json, which
469  # dendritic-analyze and the rerun tool read; keep it enabled.
470  analysis:
471    performance:
472      enabled: true
473      training: true
474      params:
475        accuracy: true
476        auc: true
477        categorical_loglikelihood: true
478        mse: false
479        cosine_similarity: false
480
481    information:
482      enabled: false
483      training: false
484
485    correlation:
486      enabled: false
487      training: false
488
489    ablation:
490      enabled: false
491      training: false
492
493    synapse_turnover:                        # Enabling it requires training: true (it compares snapshots taken during training)
494      enabled: false
495      training: false
496
497    noise_perturbation:
498      enabled: false
499      training: false
500
501    weight:
502      enabled: false
503      training: false
504
505    single_layer_contribution:
506      enabled: false
507      training: false
508
509    synaptic_activation:
510      enabled: false
511      training: false
512
513    branch_activation:
514      enabled: false
515      training: false
516
517    compartment_statistics:
518      enabled: false
519      training: false
520
521    compartment_snr:
522      enabled: false
523      training: false
524
525    multiplicative_gain:
526      enabled: false
527      training: false
528
529  # ---------- Outputs ----------
530  outputs:
531    run_name: "sweep_run"                    # Prefix of the sweep directory and the Slurm job name
532    results_dir: "outputs"                   # Base directory of the sweep directory; each run writes to <sweep_dir>/results/config_<index>
533
534  # ---------- Distributed ----------
535  distributed:
536    mode: none                               # ddp with launcher "torchrun"; fsdp with the FSDP training_script
537
538
539# ============================================================================
540# EXAMPLES -- Common Sweep Configurations
541# ============================================================================
542# Each example replaces the sweep_config and filter_config sections above;
543# keep base_config, experiment_settings and slurm_config. Results count
544# conditions; the generated configs are conditions x seeds_per_condition.
545# Examples 5, 6 and 7 need the base_config of the example config they name.
546#
547# --------------------------------------------------------------------------
548# Example 1: Sparse-synapse learning rate x batch size
549# --------------------------------------------------------------------------
550# sweep_config:
551#   training.main.common.param_groups.topk_lr: [0.0001, 0.001, 0.01]
552#   training.main.common.batch_size: [128, 256, 512]
553# filter_config:
554#   mode: "all"
555# Result: 3 x 3 = 9 conditions
556# (param_groups.lr sets no parameter of this model while split_params is
557#  true; set split_params: false to train every parameter at one rate.)
558#
559# --------------------------------------------------------------------------
560# Example 2: Population-network E/I synapse sweep with sum constraint
561# --------------------------------------------------------------------------
562# sweep_config:
563#   model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [10, 20, 30, 40]
564#   model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20, 30, 40]
565# filter_config:
566#   mode: "ei-sum"
567#   value: 50
568#   tolerance: 0
569# Result: 4 E/I pairs with E + I = 50: (10, 40), (20, 30), (30, 20), (40, 10)
570#
571# --------------------------------------------------------------------------
572# Example 3: Population size and morphology comparison
573# --------------------------------------------------------------------------
574# sweep_config:
575#   model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
576#   model.core.population_network.layers.0.populations.1.branch_factors:
577#     - [3, 3]
578#     - [4, 4]
579# filter_config:
580#   mode: "all"
581# Result: 2 x 2 = 4 conditions
582#
583# --------------------------------------------------------------------------
584# Example 4: Population-network branch depth sweep
585# --------------------------------------------------------------------------
586# sweep_config:
587#   {}
588# filter_config:
589#   mode: "branch_sweep"
590#   branch_sweep_mode: "depth_sweep"
591#   dendrite_tree_depth: {range: {start: 1, stop: 6, step: 1}}
592#   uniform_branches_per_layer: 3
593# Result: excitatory branch_factors [3], [3, 3], ..., [3, 3, 3, 3, 3, 3]
594# (6 conditions)
595#
596# --------------------------------------------------------------------------
597# Example 5: Recurrent tau and delay sweep
598# --------------------------------------------------------------------------
599# sweep_config:
600#   model.core.population_network.layers.0.dt: [0.5, 1.0]
601#   model.core.population_network.layers.0.population_defaults.tau_base: [10.0, 20.0, 40.0]
602#   model.core.population_network.layers.0.connections.3.probability: [0.3, 0.6, 0.9]
603# filter_config:
604#   mode: "all"
605# Result: 2 x 3 x 3 = 18 conditions. Use the base_config of
606# configs/examples/population_network_taus_and_delays.yaml: a recurrent layer
607# whose fourth connection (connections.3) is the delayed pyr -> pyr
608# connection. tau_base has no effect in a feedforward layer such as the one
609# above, and connections.3 does not exist there.
610#
611# --------------------------------------------------------------------------
612# Example 6: Spiking population sweep
613# --------------------------------------------------------------------------
614# sweep_config:
615#   model.core.population_network.layers.0.population_defaults.spike_threshold: [0.25, 0.5]
616#   model.core.population_network.layers.0.population_defaults.spike_tau: [12.0, 24.0]
617# filter_config:
618#   mode: "all"
619# Result: 2 x 2 = 4 conditions. Use the base_config of
620# configs/examples/population_network_spiking.yaml, whose populations spike
621# (population_defaults.dynamics_mode: "spike").
622#
623# --------------------------------------------------------------------------
624# Example 7: Synapse-type kinetics sweep
625# --------------------------------------------------------------------------
626# sweep_config:
627#   model.core.population_network.layers.0.population_defaults.synapse_types.excitatory.1.fraction: [0.1, 0.25, 0.4]
628#   model.core.population_network.layers.0.population_defaults.synapse_types.excitatory.1.tau_decay: [40.0, 70.0, 100.0]
629# filter_config:
630#   mode: "all"
631# Result: 3 x 3 = 9 conditions. Use the base_config of
632# configs/examples/population_network_synapse_types_ampa_nmda_gaba.yaml, where
633# excitatory.1 is the NMDA type. The fractions of one polarity are normalized
634# to sum to 1, so the NMDA fraction axis also changes the AMPA share.
635#
636# --------------------------------------------------------------------------
637# Example 8: Combined population E/I filter + branch sweep
638# --------------------------------------------------------------------------
639# sweep_config:
640#   model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [10, 20, 30, 40]
641#   model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20, 30, 40]
642# filter_config:
643#   mode: "ei-sum"
644#   value: 50
645#   tolerance: 0
646#   branch_sweep_mode: "depth_sweep"
647#   dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
648#   uniform_branches_per_layer: 3
649# Result: 4 E/I pairs that sum to 50 x 3 branch depths = 12 conditions
650#
651# --------------------------------------------------------------------------
652# Example 9: Dataset comparison sweep
653# --------------------------------------------------------------------------
654# sweep_config:
655#   data.dataset_name: ["mnist", "fashion_mnist"]
656#   model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 30]
657# filter_config:
658#   mode: "all"
659# Result: 2 datasets x 2 synapse counts = 4 conditions
660#
661# --------------------------------------------------------------------------
662# Example 10: Full mixed population-network sweep
663# --------------------------------------------------------------------------
664# sweep_config:
665#   model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 40]
666#   model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20]
667#   model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
668#   training.main.common.param_groups.topk_lr: [0.001, 0.01]
669# filter_config:
670#   mode: "all"
671#   branch_sweep_mode: "depth_sweep"
672#   dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
673#   uniform_branches_per_layer: 3
674# Result: E x I x n_neurons x topk_lr x branch depth = 2 x 2 x 2 x 2 x 3 =
675# 48 conditions

Other templates

  • configs/templates/analysis_full_template.yaml lists the analyzer sections with their options, all disabled. Copy sections into an experiment config, or pass a copy of the file to dendritic-analyze-checkpoint --analysis-config (see Analyzer catalogue).

  • configs/templates/indexed_runtime_template.yaml shows the indexed-synapse runtime settings of a structured E/I core and their names in population networks (see Projection backends).

  • Example map lists the focused example configs in configs/examples/.