Annotated reference configurations
Two complete configurations in configs/ carry a comment on every option:
its type, the values it accepts and what it does. The values in the files are
a working example, not the library defaults. Where a value differs from its
default and the difference matters, the comment gives the default; an option
that a file leaves out takes its default. Configuration guide explains
how configs are loaded and checked and links the pages that document each
option in full.
Single-process configuration
configs/config_exp.yaml trains a one-layer population network on CIFAR-10
in one process: an inhibitory population feeds an excitatory population whose
neurons have a three-level dendritic tree. Besides the options this run uses,
the file lists the options of the other training strategies, the local
learning rules and most analyzers, so it can serve as a starting point. Check
it without training:
dendritic-train configs/config_exp.yaml --validate-only
1# ============================================================================
2# DENDRINET ANNOTATED REFERENCE CONFIG (single process)
3# ============================================================================
4# A complete configuration: a one-layer population network (an inhibitory
5# population feeding an excitatory population with a three-level dendritic
6# tree) trained on CIFAR-10. Each option carries a comment with its type, its
7# accepted values and what it does. The values are an example, not the library
8# defaults; a comment gives the default where the difference matters. Options
9# left out of this file take their defaults.
10#
11# The core is model.core.type "population_network". Sections of other core
12# families (architecture, connectivity, morphology, sparsity, spatial, ...) are
13# left out: a population_network core does not read them, and the loader
14# rejects them when they differ from their defaults.
15#
16# Run from the repository root:
17# dendritic-train configs/config_exp.yaml
18# dendritic-train configs/config_exp.yaml --validate-only # build everything, then exit
19# dendritic-train configs/config_exp.yaml --output_dir runs --run_name test --epochs 5
20# docs/configuration_guide.rst lists every command-line override.
21#
22# Multi-GPU FSDP training: config_exp_fsdp.yaml. Parameter sweeps: sweep_config.yaml.
23
24config_version: 4 # int: configuration format; dendritic-upgrade-config rewrites earlier ones
25
26# ============================================================================
27# EXPERIMENT SETUP
28# ============================================================================
29experiment:
30 seed: 42 # int: Root seed; every stream below that is null uses it
31 # Independent random streams (null = seed). Fix one to hold it while varying
32 # the others, for example topology_seed for a fixed connectivity.
33 dataset_seed: null # int | null: Generated datasets, data noise and label noise
34 split_seed: null # int | null: Train/validation split
35 model_seed: null # int | null: Parameter initialization and training-time noise
36 topology_seed: null # int | null: Connectivity seeds the model config leaves unset
37 loader_seed: null # int | null: Order of the training batches
38 evaluation_seed: null # int | null: Sampling of the analysis runtime
39 probe_seed: null # int | null: Analysis probes, decoders, nulls and other analyzer draws
40 strict_deterministic: false # bool: Exact math and PyTorch deterministic algorithms; a contradicting flag below raises
41 train_valid_split: 0.8 # float: Fraction of the training data used for training; the rest is the validation split (datasets with their own splits ignore it)
42 fast_mode: false # bool: Flags below that are null follow it (true: nondeterministic cuDNN, autotuning, TF32)
43 # The four flags below are written out, so fast_mode does not change them
44 # here. Set them to null (their default) to let fast_mode decide.
45 deterministic: true # bool | null: Deterministic cuDNN kernels (null = not fast_mode); PyTorch deterministic algorithms need strict_deterministic
46 cudnn_benchmark: false # bool | null: cuDNN autotuner, off while deterministic is true (null = fast_mode and not deterministic)
47 allow_tf32: false # bool | null: TF32 matmuls and convolutions (null = fast_mode); must agree with float32_matmul_precision
48 float32_matmul_precision: "highest" # str | null: "highest" (no TF32) | "high" | "medium" (null = "high" if fast_mode else "highest")
49 # Not shown: sealed_test, allow_test_data, max_test_samples (test-split
50 # sealing, docs/datasets.rst) and record_dataset_fingerprints
51 # (docs/runs_and_outputs.rst).
52
53 checkpointing:
54 enabled: false # bool: Save the weights every save_every_n_epochs epochs (main_network/<prefix>checkpoints/)
55 save_every_n_epochs: 10 # int: Checkpoint interval in epochs (>= 1)
56
57# ============================================================================
58# DATA
59# ============================================================================
60data:
61 # dataset_name: a loadable dataset, for example the vision datasets "cifar10",
62 # "cifar100", "mnist", "fashion_mnist" and "imagenet", derived tasks such as
63 # "poisson_generator" and "mnist_modulo10", sequence tasks such as
64 # "sequential_mnist" and "copy_task", and generated datasets such as
65 # "correlated_gaussian". docs/datasets.rst lists every name with its
66 # dataset_params; an unknown name raises with the full list when the
67 # datasets are built.
68 dataset_name: "cifar10"
69 base_dir: "" # str: Data root (empty = $DENDRITIC_DATA_DIR, else ~/.cache/dendritic_modeling)
70
71 # ---------- Data processing ----------
72 processing:
73 flatten: null # bool | null: Flatten each sample to a vector (null = loader default: true for vision datasets)
74 normalize: null # bool | null: Dataset-specific input normalization (null = loader default: false, except sequential_mnist and stringer_v1)
75 label_noise_rate: 0.0 # float: Fraction of training labels replaced by random classes (mnist, fashion_mnist, cifar10 and cifar100 only)
76
77 # ---------- Dataset-specific params ----------
78 # Only the block named by dataset_name is read; its keys are checked when the
79 # datasets are built. The blocks below show the options of other datasets and
80 # are ignored while dataset_name is "cifar10".
81 dataset_params:
82 mnist_modulo10:
83 shuffle_iterations: 2 # int: Rounds of random image pairing; each adds one pair per two images
84 poisson_generator:
85 base_dataset: "mnist" # str: "mnist" | "cifar10"
86 multiplicative_gain: true # bool: Multiply each input by a gain before Poisson sampling
87 fixed_gain_factor: null # float | null: One gain for every input (null = draw gains up to max_gain_factor)
88 max_gain_factor: null # float | null: Draw gains from [1, max_gain_factor] (null with uniform_gain: gain 1)
89 max_gain_tau_ratio: null # float | null: Sets max_gain_factor = ratio * stimulus_duration when max_gain_factor is null
90 uniform_gain: true # bool: One gain per sample (false: one per input element)
91 gain_sampling: "log_uniform" # str: "log_uniform" | "linear_uniform"
92 poisson_sampling: true # bool: Replace each input by a Poisson count rate over stimulus_duration
93 stimulus_duration: 1.0 # float: Counting window of the Poisson draw
94 imagenet:
95 transform_preset: "legacy_resize" # str: "legacy_resize" (resize to 224x224) | "pretrained" (train/eval transforms of pretrained torchvision backbones)
96 loader_backend: "auto" # str: "auto" | "torchvision" | "imagefolder" (root with train/<class> and val/<class> directories)
97 allow_flatten_materialize: false # bool: Allow processing.flatten: true for ImageNet (memory-heavy)
98 train_samples_per_class: null # int | null: Training images per class (null = all)
99 val_samples_per_class: null # int | null: Validation and test images per class (null = all)
100
101 # ---------- Sequence dataset params ----------
102 # n_samples is the size of the training pool, which train_valid_split
103 # divides; the test split has max(n_samples / 5, a dataset minimum) samples.
104 sequential_mnist:
105 mode: "pixel" # str: "pixel" (784 steps of 1 value) | "row" (28 steps of 28 values)
106 normalize: true # bool: Scale pixels to [0, 1] (false: raw 0-255)
107 copy_task:
108 n_samples: 10000 # int: Training-pool sequences
109 seq_len: 10 # int: Length of the sequence to copy
110 delay: 10 # int: Steps between the sequence and its recall (default 50)
111 n_symbols: 8 # int: Number of symbols
112 adding_problem:
113 n_samples: 10000 # int: Training-pool sequences
114 seq_len: 100 # int: Sequence length (default 200)
115 variable_delay_dms:
116 n_samples: 10000 # int: Training-pool sequences
117 stimulus_dim: 16 # int: Stimulus dimensionality
118 stimulus_duration: 10 # int: Stimulus presentation steps
119 min_delay: 5 # int: Minimum delay steps (default 10)
120 max_delay: 50 # int: Maximum delay steps (default 100)
121 probe_duration: 10 # int: Probe presentation steps
122 context_dependent_decision:
123 n_samples: 10000 # int: Training-pool sequences (default 20000)
124 signal_dim: 16 # int: Signal dimensionality (default 8)
125 context_duration: 10 # int: Context cue steps
126 integration_duration: 30 # int: Integration window steps, used when variable_integration is false (default 50)
127 coherence_levels: [0.1, 0.2, 0.4, 0.8] # list[float]: Coherence levels (default [0.05, 0.1, 0.2, 0.4])
128 noise_std: 0.1 # float: Noise standard deviation (default 1.0)
129 variable_integration: false # bool: Sample the integration length from [min_integration, max_integration] (default true); false uses integration_duration
130 timescale_generalization:
131 stimulus_dim: 16 # int: Stimulus dimensionality
132 stimulus_duration: 10 # int: Stimulus presentation steps
133 min_delay: 5 # int: Minimum delay steps (default 10)
134 max_delay: 50 # int: Maximum delay steps (default 100)
135 probe_duration: 10 # int: Probe presentation steps
136
137# ============================================================================
138# MODEL
139# ============================================================================
140model:
141 task: "classification" # str: "classification" | "regression"
142 # Optional warm start: a state-dict file loaded into the built model before
143 # training.
144 initial_checkpoint: null # str | null: State-dict checkpoint path
145 initial_checkpoint_sha256: null # str | null: Expected SHA-256 of the file (a mismatch raises)
146 initial_checkpoint_strict: true # bool: Reject missing or unexpected keys
147 # Not shown: fixed_output_scale and output_scale_mode (scaling of the
148 # classifier output; docs/network_architectures.rst).
149
150 # ---------- Encoder ----------
151 encoder:
152 type: "identity" # str: "identity" | "none" | "pathway_router" (alias "router") | "mlpautoencoder" | "mlpvariationalautoencoder" | "cnnautoencoder" | "cnnvariationalautoencoder"
153 # Each encoder type reads only some params; the identity encoder reads
154 # input_dim and input_shape, which training sets from the data. The other
155 # fields belong to the autoencoder and router types. Because this block
156 # lists every field (as a saved config.json does), the loader only warns
157 # when the selected type ignores a field that differs from its default; a
158 # shorter block raises instead.
159 params:
160 input_dim: null # int | null: Input size; set from the data at training time
161 input_shape: [3, 32, 32] # list[int]: Input shape [C, H, W]; set from the data at training time
162 encoder_hidden_dims: [128, 64] # list[int]: (autoencoders) Encoder hidden sizes
163 latent_dim: null # int | null: (MLP autoencoders) Latent size
164 latent_shape: null # list[int] | null: (CNN autoencoders) Latent shape
165 decoder_hidden_dims: [64, 128] # list[int]: (autoencoders) Decoder hidden sizes
166 activation: "relu" # str: (autoencoders) Any activation type, for example "relu" | "gelu" | "silu" | "tanh" | "sigmoid"
167 router_mode: "fixed" # str: (pathway_router) "fixed" | "learned"
168 pathway_groups: [] # list[list[int]]: (pathway_router) Input indices of each pathway (empty = split the inputs into n_pathways groups)
169 shared_indices: [] # list[int]: (pathway_router) Input indices given to every pathway
170 n_pathways: 2 # int: (pathway_router) Number of pathways when pathway_groups is empty
171 pathway_dim: null # int | null: (pathway_router) Output size of every pathway (null = its input count)
172 pathway_dims: [] # list[int]: (pathway_router) Output size of each pathway (overrides pathway_dim)
173 router_activation: "none" # str: (pathway_router) Any activation type, for example "none" | "sigmoid" | "relu"
174 learned_router_temperature: 1.0 # float: (pathway_router, learned) Temperature of the learned assignment
175 learned_router_init_scale: 4.0 # float: (pathway_router, learned) Initial logit scale of the learned assignment
176
177 # ---------- Core network ----------
178 core:
179 biological_neuron: true # Enforce the neuronal-core constraint set.
180 # Named-population model. configs/examples/ holds focused variants
181 # (spiking, receptor kinetics, recurrent timing, additive normalization,
182 # sparse rewiring, local learning); docs/population_networks.rst documents
183 # the options.
184 type: "population_network"
185
186 # ---- model.core.population_network ----
187 # Keys inside this block are checked when the model is built, not when the
188 # config loads; dendritic-train --validate-only checks them without
189 # training. Network options not shown: readout_layer, readout_population,
190 # compile_step. input_transform also accepts identity, relu and signed_split.
191 population_network:
192 input_transform: nonnegative # Check nonnegative data without changing values or gradients.
193 use_transfer: false # bool: Split the input into excitatory and inhibitory streams with a TransferLayer
194 # transfer_params: TransferLayer options (independent_pathways, excitatory_dim,
195 # inhibitory_dim, split_strategy, output_activation, ...); allowed only with
196 # use_transfer: true.
197 transfer_params: {}
198 output_mode: "last" # str: Sequence output of recurrent networks: "last" | "mean" | "all"
199 input_projection_dims: [] # list[int]: Linear + ReLU layers applied to the input before the first layer
200 store_routing: false # bool: Record per-step routing information on every forward pass (sets it on every layer)
201 layers: # list[dict]: Layer keys: name, populations, connections, population_defaults, readout_population, recurrent, dt, connection_seed, store_routing
202 - name: "layer0"
203 readout_population: "e" # str: Population whose somas form the layer output
204 # population_defaults: options shared by every population of the
205 # layer; a population's own `population` block overrides them.
206 # n_neurons and branch_factors belong to each population definition
207 # and are rejected here.
208 population_defaults:
209 ff_excitatory_synapses: 40 # int: Feedforward excitatory synapses per dendritic compartment (default 8)
210 ff_inhibitory_synapses: 10 # int: Feedforward inhibitory synapses per compartment (default 4)
211 rec_excitatory_synapses: 0 # int: Recurrent excitatory synapses per compartment (delayed connections)
212 rec_inhibitory_synapses: 0 # int: Recurrent inhibitory synapses per compartment (delayed connections)
213 somatic_synapses: true # bool: The soma level also receives synapses
214 use_shunting: true # bool: Shunting (conductance) integration; false = additive
215 reactivate: true # bool: Apply the reactivation gate to each compartment output (false: no gate)
216 reactivation_type: "param_tanh" # str: Gate activation; any activation type (docs/dendritic_neurons.rst)
217 reactivation_memory_efficient: true # bool: Save fewer tensors for backward in param_tanh and param_tanh_only_m gates (default false)
218 # reactivation_init_policy: "analytical" | "fixed" | "empirical" |
219 # "occupancy_quantile" | "analytical_slope_occupancy_center" |
220 # "occupancy_slope_analytical_center". Omitted, it is
221 # "analytical" for shunting populations and "occupancy_quantile"
222 # for additive ones (use_shunting: false).
223 reactivation_init_policy: "analytical"
224 # reactivation_init_m: 1.0 # float: Start slope (default 1.0), read only by init_policy "fixed"
225 # reactivation_init_b: 0.5 # float: Start midpoint (default 0.5), read only by init_policy "fixed"
226 dbl_init_method: "analytical_expectation" # str: Initial synaptic weights: "analytical_expectation" | "ei_equivalence" | "naive" | "default" | "fan_in_lognormal" | "mechanism_neutral"
227 weight_transform: "softplus" # str: "exp" (default) | "softplus" | "relu" | "identity"
228 # topk_type: "standard" | "stochastic" | "variance" | "deepst" |
229 # "credit_deepst" | "dense_to_sparse" (alias "annealed_topk") |
230 # "indexed" | "indexed_dynamic" | "indexed_rewire" | "indexed_release".
231 # "standard" keeps dense candidate weights and selects the active
232 # contacts; the indexed types store only the active indices and
233 # suit large sparse models. The indexed_* options below are read
234 # only by the indexed types (indexed_rewire_* only by
235 # "indexed_rewire"); "standard" ignores them.
236 topk_type: "standard"
237 topk_init_method: "xavier_normal" # str: Initializer of contacts that "indexed_rewire" creates; dbl_init_method sets the initial weights
238 indexed_rewire_frequency: 100 # int: (indexed_rewire) Steps between rewiring passes
239 indexed_rewire_quantile: 0.05 # float: (indexed_rewire) Fraction of each output's weakest contacts replaced per pass
240 indexed_rewire_until_step: null # int | null: (indexed_rewire) Last rewiring step (null = no limit)
241 indexed_output_chunk_size: 8192 # int: (indexed) Output rows per gather chunk (default 2048)
242 indexed_index_dtype: "auto" # str: (indexed) "auto" | "int32" | "int64" (default); auto uses int32 when it fits
243 indexed_workspace_mb: 128 # float | null: (indexed) Cap of the temporary gather workspace in MB (default null)
244 indexed_cache_transformed_weights: false # bool: (indexed) Reuse fixed transformed weights within an unroll
245 indexed_projection_backend: "auto" # str: (indexed) Projection backend (default "eager"); auto picks a Triton kernel on CUDA when available, else "recompute"
246 indexed_persistent_indices: true # bool: (indexed) Store the topology in checkpoints
247 indexed_init_mode: "per_rank" # str: (indexed) "per_rank" | "rank0_broadcast"
248 efficient_blocklinear: true # bool: BlockLinear branch couplings without the dense block-diagonal matrix (default false)
249 populations:
250 - name: "i"
251 polarity: "inhibitory" # str: "excitatory" | "inhibitory"
252 n_neurons: 32 # int: Number of neurons (somas)
253 branch_factors: [1] # list[int]: Branches per compartment from the soma outward (default [1])
254 population: # PopulationConfig options of this population only
255 ff_inhibitory_synapses: 0
256 - name: "e"
257 polarity: "excitatory"
258 n_neurons: 128
259 branch_factors: [3, 3, 3]
260 connections: # Connection keys: source, target, pathway, timing ("same_step" | "delayed"), enabled, probability, seed, target_levels
261 - source: "input"
262 target: "i"
263 - source: "input"
264 target: "e"
265 - source: "i"
266 target: "e"
267
268 # ---------- Decoder ----------
269 decoder:
270 type: "MLP" # str: "MLP" | "identity" | "none" (identity and none pass the core output through; a linear readout is MLP with hidden_dims: [])
271 params:
272 input_dim: 20 # int: Replaced by the core output size when the model is built
273 hidden_dims: [16] # list[int]: Hidden layer sizes
274 activation: "relu" # str: Any activation type, for example "relu" | "gelu" | "silu" | "tanh"
275 output_dim: 10 # int: Number of classes or outputs (identity/none: omit it, or give the core output size)
276
277# ============================================================================
278# TRAINING
279# ============================================================================
280training:
281
282 # ---------- Main training ----------
283 main:
284 # strategy: "standard" | "fa" | "dfa" | "soma_dfa" | "local_ca" | "multi_stage" |
285 # "recurrent" | "presentation_stream" | "voltage_stabilization" |
286 # "homeostatic_control" | "ei_equilibrium" | "freeze_layers" |
287 # "freeze_branches" | "freeze_branch_kl" (docs/training_strategies.rst).
288 # A strategy reads its own block under strategies; the other blocks are ignored.
289 strategy: "standard"
290
291 common:
292 epochs: 1000 # int: Training epochs
293 batch_size: 512 # int: Training batch size
294 num_workers: null # int | null: DataLoader workers (null = chosen from the dataset type)
295 pin_memory: null # bool | null: Pin host batches (null = when training on a GPU)
296 persistent_workers: null # bool | null: Keep workers alive between epochs (needs num_workers > 0; null = automatic)
297 prefetch_factor: null # int | null: Batches prefetched per worker (needs num_workers > 0; null = automatic)
298 in_order: null # bool | null: Deliver batches in sampler order (null: false for image datasets with workers; strict_deterministic requires true)
299 profile_dataloader: false # bool: Log the time each epoch waits for data
300 shuffle: true # bool: Shuffle the training data
301 grad_clip_value: 2.0 # float | null: Gradient clipping threshold (0 or null = off)
302 grad_clip_mode: null # str | null: "value" (clamp each element) | "norm" (rescale the global norm); null = "norm" for strategy recurrent and FSDP, else "value"
303 # loss_function: "ce" | "bce" | "mse" | "log_mse" | "cat_nll" | "neg_cos_sim" |
304 # "log_adj_neg_cos_sim" | "temporal_event_ce" | "masked_dual_task" |
305 # "masked_dual_task_temporal" | "vae_elbo" (long names such as "cross_entropy" also work)
306 loss_function: "ce"
307 early_stopping: true # bool: Stop when the validation loss has not improved for patience epochs
308 patience: 500 # int: Early-stopping patience in epochs
309 load_best_state_dict: true # bool: End training with the weights of the best validation epoch
310 plot_losses: true # bool: Save loss curves (main_network/<prefix>loss_curves.png)
311 suppress_prints: false # bool: Suppress per-epoch console output
312 print_every: 10 # int: Epochs between progress prints
313 use_amp: false # bool: Automatic mixed precision
314 lr_schedule: "none" # str: "none" | "cosine" (strategies standard, recurrent, fa, dfa, soma_dfa and presentation_stream apply it; the others raise)
315 lr_warmup_epochs: 0 # int: Linear warm-up epochs of the cosine schedule (needs lr_schedule: "cosine")
316 enable_nan_checking: false # bool: Zero NaN/inf gradient entries before each step
317 synapse_decay_rate: 0.01 # float: Sparse-synapse maintenance: active Top-K and indexed weights decay by lr * rate before each step (not optimizer.weight_decay)
318 weight_boosting: true # bool: Also boost unselected candidate weights by that factor (default false; needs synapse_decay_rate > 0)
319 param_groups: # Learning rates of the optimizer's parameter groups
320 lr: 0.001 # float: Rate of parameters outside the groups below, or of all parameters with split_params: false (default 0.01)
321 split_params: true # bool: Give each component group its own rate
322 topk_lr: 0.001 # float: Sparse synapses (Top-K and indexed banks)
323 blocklinear_lr: 0.001 # float: BlockLinear branch couplings (default 0.0001)
324 reactivation_lr: 0.001 # float: Reactivation gates (default 0.0001)
325 decoder_lr: 0.001 # float: Decoder and learned output scales
326 encoder_lr: null # float | null: Trainable encoder parameters (null = lr)
327
328 optimizer:
329 name: "adam" # str: "adam" | "adamw" | "sgd" | "rmsprop" | "muonh" (alias "muon_hyperball"; the muon_* options apply only to it)
330 lr: 0.001 # float: Not used for training: every parameter group takes its rate from common.param_groups (a non-default value must equal param_groups.lr)
331 weight_decay: 0.0 # float: Optimizer weight decay of every group (decoupled for adamw)
332 momentum: 0.9 # float: Momentum (sgd, rmsprop)
333 betas: [0.9, 0.999] # list[float]: Betas (adam, adamw, and the AdamW part of muonh)
334 eps: 0.00000001 # float: Epsilon (adam, adamw, rmsprop, muonh)
335
336 regularization: # Auxiliary loss terms (local_ca, freeze_branches, freeze_branch_kl and FSDP reject them)
337 l1_weight: 0.0 # float: L1 penalty strength (0 = off)
338 l2_weight: 0.0 # float: L2 penalty strength (0 = off; separate from optimizer.weight_decay)
339 selective: # Which weights the L1/L2 terms cover (synapse, BlockLinear and dense weights)
340 layers: [] # list[int]: Layer indices, matched with the first integer in a parameter name (empty = all)
341 branches: [] # list[str]: Parameter-name substrings (empty = all)
342 weight_types: ["all"] # list[str]: Parameter-name substrings, or "all"
343 split_params: false # bool: Use the per-group strengths below (groups not listed use l1_weight/l2_weight)
344 param_group_weights:
345 topk:
346 l1_weight: 0.0 # float: Sparse-synapse L1 strength
347 l2_weight: 0.0 # float: Sparse-synapse L2 strength
348 blocklinear:
349 l1_weight: 0.0 # float: BlockLinear L1 strength
350 l2_weight: 0.0 # float: BlockLinear L2 strength
351 reactivation:
352 l1_weight: 0.0 # float: Reactivation L1 strength
353 l2_weight: 0.0 # float: Reactivation L2 strength
354 decoder:
355 l1_weight: 0.0 # float: Decoder L1 strength
356 l2_weight: 0.0 # float: Decoder L2 strength
357 enforce_ei_weight_ratio: false # bool: Add a penalty on the E/I weight ratio
358 target_ei_weight_ratio: 0.5 # float: Target excitatory/inhibitory weight ratio
359 ei_ratio_loss_weight: 0.5 # float: Strength of the E/I ratio penalty
360 ei_ratio_scope: "per_branch" # str: "per_branch" | "per_layer" | "global" ("global" averages the per-layer ratios)
361 ei_ratio_metric: "mean" # str: "mean" | "median" | "rms"
362
363 pruning: # Zeroes stored weights after training
364 enabled: false # bool: Enable pruning. It rejects exp/softplus-transformed synapses, so it cannot prune this model's synapses
365 threshold: 2 # float: Prune stored weights with abs(value) < threshold
366 selective:
367 layers: [] # list[int]: Layer indices, matched with the first integer in a parameter name (empty = all)
368 branches: [] # list[str]: Parameter-name substrings (empty = all)
369 weight_types: ["excitatory", "inhibitory"] # list[str]: Parameter-name substrings, or "all"; these two match parameter names of structured E/I cores, not of population networks
370 re_evaluate_after: false # bool: Record training and validation loss before and after pruning
371 report_non_pruned: true # bool: Print the pruning report
372 save_pruning_stats: false # bool: Write pruning_stats.json
373 detailed_branch_report: true # bool: Include per-branch statistics in the report
374 redo_analysis_after_pruning: false # bool: Not implemented; true raises when pruning is enabled
375
376 strategies:
377 standard: {} # The standard strategy uses the common options only
378
379 multi_stage:
380 analyze_between_stages: true # bool: Run the analysis after each stage
381 final_state: "last_stage" # str: Weights the run ends with: "last_stage" | "best_stage" (lowest validation loss)
382 continue_on_failure: false # bool: Continue with the next stage when a stage fails
383 stages: # list[dict]: Stages in order; their learning_strategy is checked when the stage starts
384 - learning_strategy: "standard"
385 epochs: 100 # int: Stage epochs (default common.epochs)
386 batch_size: 512 # int: Stage batch size (default common.batch_size)
387 save_checkpoint: false # bool: Write <stage directory>/stage_checkpoint.pt (default true)
388 param_groups: # Stage learning rates (default common.param_groups)
389 lr: 0.001
390 split_params: true
391 topk_lr: 0.001
392 blocklinear_lr: 0.001
393 reactivation_lr: 0.001
394 decoder_lr: 0.001
395 trainer_config: # Common trainer options overridden for this stage
396 loss_function: "ce"
397 early_stopping: true
398 patience: 500
399 - learning_strategy: "local_ca" # Uses training.main.learning_strategy_config unless local_rule_config is set
400 epochs: 100
401 batch_size: 512
402 reset_optimizer: false # bool: Rebuild the optimizer before this stage
403 save_checkpoint: false
404 param_groups:
405 lr: 0.0001
406 split_params: true
407 topk_lr: 0.0001
408 blocklinear_lr: 0.00001
409 reactivation_lr: 0.00001
410 decoder_lr: 0.001
411 trainer_config:
412 loss_function: "ce"
413 early_stopping: false
414
415 voltage_stabilization:
416 pretrain_epochs: 100 # int: Warm-up epochs before standard training, >= 1 (default 1)
417 stabilize_mode: "vout" # str: Activations the warm-up drives: "vinf" (default, gate input) | "vout" (gate output) | "both" | "gate_occupancy"
418 freeze_reactivation: true # bool: Freeze the gates during the warm-up (default false)
419 target_voltage: 0.5 # float: Target of the warm-up loss
420 loss_metric: "mse" # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
421
422 homeostatic_control:
423 pretrain_epochs: 100 # int: Warm-up epochs before standard training, >= 1 (default 1)
424 stabilize_mode: "vinf" # str: "vinf" | "vout" | "both" | "gate_occupancy"
425 freeze_reactivation: false # bool: Freeze the gates during the warm-up
426 target_voltage: 0.5 # float: Target of the warm-up loss
427 loss_metric: "mse" # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
428
429 ei_equilibrium: # Warm-up that drives each branch's excitatory conductance toward its inhibitory one; needs inhibitory synapses
430 pretrain_epochs: 1 # int: Warm-up epochs before standard training, >= 1
431 freeze_reactivation: false # bool: Freeze the gates during the warm-up
432 loss_metric: "mse" # str: "mse" | "mae" | "log_mse" | "log_mae" (also the squared/abs_mean_error forms)
433
434 recurrent: # null inherits the training.main.common option of the same name
435 grad_clip_value: null # float | null: Gradient clipping threshold (0 = off)
436 use_amp: null # bool | null: Mixed precision
437 lr_schedule: null # str | null: "none" | "cosine"
438 lr_warmup_epochs: null # int | null: Warm-up epochs of the cosine schedule
439
440 freeze_layers:
441 pretrain_epochs: 10 # int: Full-network epochs before the layer-by-layer phase
442 reverse_training: false # bool: Train the layers from last to first
443 epochs_per_layer: 50 # int: Epochs per layer while the others are frozen (default 5)
444 final_tune_epochs: 10 # int: Full-network epochs after the layer-by-layer phase
445
446 freeze_branches:
447 pretrain_epochs: 10 # int: Stage-1 epochs, each training every core layer in turn with a local head (default 5); stage 2 trains end to end for common.epochs
448 reverse_training: false # bool: Not implemented; a value other than false raises
449 epochs_per_branch: 50 # int: Not implemented; a value other than 50 raises
450 final_tune_epochs: 10 # int: Not implemented; a value other than 10 raises
451
452 freeze_branch_kl:
453 pretrain_epochs: 10 # int: Stage-1 epochs as in freeze_branches (default 5)
454 reverse_training: false # bool: Not implemented; a value other than false raises
455 epochs_per_branch: 50 # int: Not implemented; a value other than 50 raises
456 final_tune_epochs: 10 # int: Not implemented; a value other than 10 raises
457 kl_weight: 0.1 # float: Weight of the KL term pulling each local head toward the model's predictions
458
459 # ---- Local learning rules (training.main.learning_strategy_config) ----
460 # Read by strategy local_ca and by the local_ca stages of multi_stage; the
461 # other strategies ignore it (a block that differs from the defaults then
462 # warns).
463 learning_strategy_config:
464 rule_variant: "3f" # str: "3f" | "4f" | "5f" | "stdp", or "3f_vh" | "4f_vh" | "5f_vh" (adds voltage_homeostasis)
465 error_mode: "auto" # str: "auto" (from loss_function) | "mse" | "ce" | "bce"
466 error_broadcast_mode: "scalar" # str: "scalar" | "per_soma" | "per_soma_shared" | "per_soma_tree" | "per_soma_shuffled" | "local_mismatch" | "low_rank" | "path_transport" | "pathway_vector"
467 broadcast_seed: null # int | null: Seed of the randomized feedback (per_soma_shuffled, low_rank); null = derived from the run seed
468 error_noise_sigma: 0.0 # float: Additive Gaussian noise on the broadcast error (0 = off)
469 broadcast_rank: 4 # int: Rank of the low_rank broadcast
470 broadcast_init_scale: 1.0 # float: Scale of the low_rank random broadcast
471 pathway_broadcast_residual: 0.25 # float: Blend with the direct local error
472 pathway_activity_gate_strength: 1.0 # float: Modulation by local pathway activity
473 broadcast_bandwidth: "full" # str: "full" | "sign_only" | "quantized" | "sparse_topk"
474 broadcast_bits: 8 # int: Quantization bits, >= 1 (2^bits levels); for broadcast_bandwidth "quantized"
475 broadcast_topk_fraction: 0.3 # float: Fraction in (0, 1] kept by broadcast_bandwidth "sparse_topk"
476
477 three_factor:
478 dynamics_mode: "auto" # str: "auto" (each layer's own integration) | "conductance" | "additive" (must match every layer unless allow_mismatched_dynamics_mode)
479 allow_mismatched_dynamics_mode: false # bool: Apply a forced dynamics_mode to every layer (a rule variant)
480 use_conductance_scaling: true # bool: Scale by R_tot = 1/g_tot
481 use_driving_force: true # bool: (E_rev - V_n) driving force; false uses (V_n - theta) / (theta - V_n)
482 theta: 0.0 # float: Threshold of the use_driving_force=false variant and of additive "input_dependent" mode
483 e_rev_exc: 1.0 # float: Fixed at 1.0, the model's excitatory reversal
484 e_rev_inh: 0.0 # float: Fixed at 0.0, the model's inhibitory reversal
485 additive_gain_mode: "none" # str: "none" | "input_dependent" | "running_stats" (dynamics_mode "additive" only)
486 additive_stats_ema_alpha: 0.01 # float: EMA rate of the running_stats mode
487
488 four_factor:
489 rho_mode: "pearson" # str: "pearson" | "dot" | "none"
490 rho_estimator: "ema" # str: "ema" | "augment" | "ema+augment"
491 ema_alpha: 0.05 # float: EMA smoothing rate
492 layer_wise_rho_scale: 1.0 # float: rho / sqrt(layer_depth)
493 augment_k: 8 # int: Number of augmented samples
494 augment_noise_sigma: 0.01 # float: Augmentation noise level
495
496 five_factor:
497 phi_mode: "conditional" # str: "conditional" | "variance"
498 phi_estimator: "conditional_ema" # str: "conditional_ema" | "variance_ema"
499 phi_ridge_lambda: 0.001 # float: Ridge regularization
500 layer_wise_phi_scale: 1.0 # float: phi / sqrt(layer_depth)
501 phi_clamp_min: 0.25 # float: Stability clamp min
502 phi_clamp_max: 4.0 # float: Stability clamp max
503
504 morphology_aware:
505 use_path_propagation: false # bool: Path-integrated propagation
506 path_factor_mode: "per_branch" # str: "per_branch" | "scalar_mean"
507 morphology_modulator_mode: "none" # str: "none" | "depth" | "centrality" (inverse branch depth)
508 morphology_depth_offset: 1.0 # float: Offset alpha of the depth modulation
509 use_dendritic_normalization: false # bool: Normalize by the sum of conductances
510 use_branch_type_rules: false # bool: Apical vs basal differentiation
511 apical_branch_scale: 1.0 # float: Scale for apical branches
512 basal_branch_scale: 1.0 # float: Scale for basal branches
513 use_branch_length_modulation: false # bool: Branch length modulation
514 use_branch_role_rules: false # bool: Router-derived branch roles
515 branch_role_source: "router_pathways" # str: "router_pathways"
516 specialized_branch_scale: 1.0 # float: Specialized branch scale
517 mixed_branch_scale: 1.0 # float: Mixed branch scale
518 branch_role_power: 1.0 # float: Branch role power
519 branch_role_alignment_weight: 0.0 # float: Alignment weight
520
521 hsic:
522 enabled: false # bool: Enable the HSIC auxiliary loss
523 weight: 0.0 # float: HSIC loss weight
524 self_weight: 1.0 # float: Self-decorrelation weight
525 target_weight: 1.0 # float: Target-correlation weight
526 target_source: "labels" # str: "labels" | "logits"
527 kernel: "linear" # str: "linear" | "rbf" | "polynomial"
528 sigma: 1.0 # float: RBF bandwidth
529 degree: 2 # int: Polynomial degree
530 coef0: 1.0 # float: Polynomial coefficient
531 grad_clip_value: 0.0 # float: HSIC gradient clip (0 = off)
532 warmup_epochs: 0 # int: HSIC warm-up epochs
533 apply_last_layer_only: false # bool: Apply HSIC to the last layer only
534
535 stdp:
536 enabled: false # bool: Trace-based STDP as an auxiliary rule; rule_variant "stdp" enables it
537 apply_to: ["exc", "rec_exc"] # list[str]: "exc" | "inh" | "rec_exc" | "rec_inh" | "all"
538 activity_mode: "relu" # str: "relu" | "identity" | "binary" | "abs"
539 pre_threshold: 0.0 # float: Presynaptic activity threshold
540 post_threshold: 0.0 # float: Postsynaptic activity threshold
541 tau_pre: 20.0 # float: Presynaptic trace tau
542 tau_post: 20.0 # float: Postsynaptic trace tau
543 a_plus: 1.0 # float: Potentiation scale
544 a_minus: 0.5 # float: Depression scale
545 learning_rate_scale: 1.0 # float: Multiplier on the learning rate of the STDP update
546 inhibitory_update_sign: -1.0 # float: Sign multiplier for inhibitory pathways
547 use_error_modulation: false # bool: Modulate STDP by the broadcast task error
548 error_modulation_mode: "scalar_abs" # str: "scalar_abs" | "scalar_signed"
549 clamp_update: 0.0 # float: Absolute update clamp (0 = off)
550 detach_traces: true # bool: Detach local traces from the autograd graph
551
552 inhibitory_homeostasis:
553 enabled: false # bool: Enable inhibitory homeostasis
554 mode: "r_tot" # str: "r_tot" | "voltage"
555 weight: 0.0 # float: Homeostasis loss weight
556 target_r_tot: 0.35 # float: Target total resistance
557 target_voltage: 0.15 # float: Target voltage
558
559 voltage_homeostasis:
560 enabled: false # bool: Enable strictly local voltage centering
561 weight: 0.0 # float: Local voltage-homeostasis weight
562 target_voltage: 0.5 # float: Midpoint target voltage
563 # The *_vh rule variants enable this block; a weight of 0 then becomes 0.02.
564
565 gate_homeostasis:
566 enabled: false # bool: Enable the local gate-occupancy update
567 center_weight: 0.0 # float: Weight of the mean gate-occupancy target
568 saturation_weight: 0.0 # float: Weight of the saturation target
569 target_mean: 0.5 # float: Desired mean gate value
570 target_saturation: 0.2 # float: Desired fraction of saturated gates
571 low_threshold: 0.1 # float: Low gate threshold of the saturation metric
572 high_threshold: 0.9 # float: High gate threshold of the saturation metric
573 saturation_slope: 20.0 # float: Slope of the smooth saturation indicator
574
575 update_inactive_weights: false # bool: Also update inactive candidate weights
576 update_reactivation: true # bool: Update the gate parameters with the local rule
577 explicit_inhibitory_update_mode: "local_ca" # str: "local_ca" | "freeze" for the dendrites of explicit inhibitory populations
578 encoder_update_mode: "none" # str: "backprop" | "none"
579 decoder_update_mode: "backprop" # str: "backprop" | "local" | "none"
580 freeze_encoder_epochs: 0 # int: Freeze the encoder for N epochs
581 freeze_reactivation_epochs: 0 # int: Freeze the gates for N epochs
582 freeze_decoder_epochs: 0 # int: Freeze the decoder for N epochs
583 training_schedule: null # list[dict] | null: Epoch ranges (start_epoch, end_epoch) that set update_reactivation, encoder_update_mode or decoder_update_mode
584 clip_grad_value: 5.0 # float: Element-wise clip of the local-rule gradients (local_ca uses it instead of common.grad_clip_value)
585 normalize_by_batch: true # bool: Normalize updates by the batch size
586
587# ============================================================================
588# ANALYSIS
589# ============================================================================
590# Every enabled analyzer runs once after training; training: true also runs it
591# during training (some analyzers are final-only). docs/analysis.rst describes
592# every analyzer, including those not shown here (they are off when omitted);
593# configs/templates/analysis_full_template.yaml shows most of them.
594analysis:
595 # Dataset iteration shared by all analyzers. mode:
596 # "auto" materialize a split when its estimated size is at most
597 # materialize_threshold_mb, otherwise stream it
598 # "materialize" load each split as one batch (hook-based analyses then see
599 # the whole split at once when the dataset supports slicing)
600 # "stream" iterate in batches through a DataLoader (for large datasets)
601 # Keys left unset take their defaults, except mode: an unset mode is
602 # "materialize" in single-process runs and "auto" under DDP/FSDP.
603 runtime:
604 training:
605 mode: "auto" # str: "auto" | "materialize" | "stream"
606 batch_size: 256 # int: Analysis batch size
607 num_workers: 4 # int: DataLoader workers
608 pin_memory: true # bool: Pin host memory for GPU analysis (off for CPU analysis)
609 persistent_workers: true # bool: Keep workers alive between streamed passes (default false)
610 prefetch_factor: 4 # int | null: Batches prefetched per worker (default 2)
611 in_order: null # bool | null: false lets batches arrive as workers finish them (null = PyTorch default, in order)
612 max_batches: null # int | null: Cap on batches per split (null = all)
613 max_samples: null # int | null: Cap on the examples given to each analysis (null = all)
614 materialize_threshold_mb: 1024 # int: Size limit (MB) up to which "auto" materializes a split
615 final:
616 mode: "auto" # str: "auto" | "materialize" | "stream"
617 batch_size: 256 # int: Analysis batch size
618 num_workers: 4 # int: DataLoader workers
619 pin_memory: true # bool: Pin host memory for GPU analysis (off for CPU analysis)
620 persistent_workers: true # bool: Keep workers alive between streamed passes (default false)
621 prefetch_factor: 4 # int | null: Batches prefetched per worker (default 2)
622 in_order: null # bool | null: false lets batches arrive as workers finish them (null = PyTorch default, in order)
623 max_batches: null # int | null: Cap on batches per split (usually null for the final analysis)
624 max_samples: null # int | null: Cap for heavy analyses that should inspect only a subset
625 materialize_threshold_mb: 1024 # int: Size limit (MB) up to which "auto" materializes a split
626
627 # ---------- Performance ----------
628 performance:
629 enabled: true # bool: Enable performance analysis (default false)
630 training: false # bool: Also run during training
631 params:
632 accuracy: true # bool: Compute accuracy
633 auc: true # bool: Compute AUC
634 categorical_loglikelihood: true # bool: Compute categorical log-likelihood
635 mse: false # bool: Compute MSE
636 cosine_similarity: false # bool: Compute cosine similarity
637 training_splits: ["train", "validation", "test"] # list[str]: Splits evaluated during training
638 final_splits: ["train", "validation", "test"] # list[str]: Splits evaluated after training
639
640 # ---------- Information ----------
641 information:
642 enabled: true # bool: Enable information analysis (default false)
643 training: false # bool: Also run during training
644 params:
645 # -- Grouped structure --
646 selection:
647 # components: "basic" | "pairwise" | "conditional" | "gaussian_fisher" | "pid" |
648 # "ablation_aligned" | "layer_proxies" | "upstream_unique_cmi" | "soma_coupling".
649 # Empty computes basic, pairwise, conditional, ablation_aligned and layer_proxies.
650 components: []
651 scopes: [] # list[str]: "layer" | "einet" | "neuron"
652 signal_variants: [] # list[str]: "network" | "lda" | "random" (random needs baselines.random_weights.n_shuffles > 0)
653 compute:
654 max_samples: null # int | null: Limit samples (null = all)
655 include_vinf: true # bool: Include the soma activation
656 normalize_mi: false # bool: Normalize by H(C)
657 output_units: "bits" # str: "bits" | "nats"
658 analysis_split: "test" # str: "validation" | "test"
659 verbose: false # bool: Verbose output
660 computation_level: "single_branch" # str: "single_branch" | "layer_branch" | "all_branch"
661 branch_aggregation: "mean" # str: "mean" | "multivariate" | "sample"
662 branch_sample_count_per_layer: null # int | null: Deterministic per-layer branch subset for single_branch (null = all)
663 branch_sample_seed: null # int | null: null = derived from experiment.probe_seed
664 layer_total_topk: 10 # int: K of the topK_sum metrics
665 gaussian_fisher:
666 aggregation: "sum" # str: "sum" | "mean" | "max"
667 mi_transform: "half_log1p" # str: "half_log1p" | "log1p" | "none"
668 eps: 0.00000001 # float: Numerical epsilon
669 ridge: null # float | null: Ridge parameter (null = use eps)
670 estimator:
671 method: "kraskov" # str: "kraskov" | "decoder" | "binned" | "sklearn" | "copula" | "auto"
672 kraskov:
673 n_neighbors: 10 # int: Number of KSG neighbors
674 binned:
675 n_bins: 20 # int: Number of bins (binned estimator; the sklearn estimator uses it too)
676 binning_strategy: "quantile" # str: "quantile" | "uniform"
677 bias_correction: "none" # str: "none" | "miller_madow"
678 decoder:
679 cv_folds: 5 # int: Cross-validation folds
680 C: 1.0 # float: Regularization parameter
681 standardize: true # bool: Standardize features
682 seed: null # int | null: null = derived from experiment.probe_seed
683 continuous:
684 strategy: "none" # str: "none" | "binned" | "gaussian"
685 max_total_dim: 2 # int: Max total dim for continuous MI/CMI
686 gaussian_ridge: 0.000001 # float: Covariance ridge
687 copula:
688 type: "gaussian" # str: "gaussian" (scalars, vectors, labels) | "kernel" (two continuous scalars)
689 sklearn: # Not read: the sklearn estimator takes its bins from estimator.binned.n_bins
690 n_bins: 20 # int: Not read
691 bandwidth: null # float | null: Not read
692 baselines:
693 random_weights:
694 n_shuffles: 0 # int: Number of random-weight shuffles (0 = off)
695 seed: null # int | null: null = derived from experiment.probe_seed
696 label_shuffle_null:
697 n_shuffles: 0 # int: Number of label permutations (0 = off)
698 seed: null # int | null: null = derived from experiment.probe_seed
699 components: ["basic"] # list[str]: "basic" | "pairwise" | "conditional" | "ablation_aligned" | "upstream_unique_cmi" | "soma_coupling" | "all"
700 pid:
701 binarize: false # bool: Threshold E, I and Vout (never the labels) before PID
702 binarize_method: "median" # str: "median" | "mean" | "quantile"
703 binarize_threshold: 0.5 # float: Quantile in [0, 1]; used only with binarize_method "quantile"
704
705 # ---------- Correlation ----------
706 correlation:
707 enabled: false # bool: Enable correlation analysis
708 training: false # bool: Also run during training
709 params:
710 # -- Grouped structure --
711 selection:
712 components: [] # list[str]: "total" | "noise" | "signal" | "tuning"
713 pairs: [] # list[str]: "EE" | "II" | "EI" | "E_output" | "I_output" | "output_output"
714 scopes: [] # list[str]: "layer" | "einet" | "neuron"
715 compute:
716 computation_level: "layer_branch" # str: "synaptic" | "single_branch" | "layer_branch" | "all_branch"
717 max_samples: 5000 # int | null: Limit samples
718 per_layer_analysis: true # bool: Per-layer breakdown
719 per_einet_analysis: true # bool: Separate E and I pathway summaries
720 per_neuron_analysis: false # bool: Per-neuron breakdown
721 sampling:
722 n_synapse_pairs: 10000 # int: Synapse pairs to sample (synaptic level)
723 sampling_strategy: "stratified" # str: "stratified" | "random" (other values raise)
724 stratified_categories: ["within_branch", "within_layer"] # list[str]: "within_branch" | "within_layer"
725 max_pairs_per_category: 1000 # int: Max pairs per stratified category
726
727 # ---------- Ablation ----------
728 ablation:
729 enabled: false # bool: Enable ablation analysis
730 training: false # bool: Also run during training
731 params:
732 selection: # Empty lists select nothing: an ablation needs at least one level, target and metric
733 levels: [] # list[str]: "layer" | "compartment"
734 targets: [] # list[str]: "all_synapses" | "excitation" | "inhibition" | "upstream"
735 metrics: [] # list[str]: "accuracy" | "auc" | "categorical_loglikelihood" | "mse" | "cosine_similarity" | "pred_label_mi_bits"
736 methods:
737 ablation_methods: ["lesion"] # list[str]: "lesion" | "shuffle" | "mean_clamp"
738 shuffle_seed: null # int | null: null = derived from experiment.probe_seed
739 clamp_statistic: "mean" # str: "mean" (the only supported value)
740
741 # ---------- Synapse turnover ----------
742 synapse_turnover:
743 enabled: false # bool: Enable synapse turnover analysis
744 training: true # bool: Must be true; turnover compares training snapshots
745 params:
746 snapshot_interval: 1 # int: Epochs between snapshots (first = before training)
747
748 # ---------- Synapse pruning (final-only census) ----------
749 synapse_pruning:
750 enabled: false # bool: Census of realized contacts; before/after pruning when the run prunes
751 training: false # bool: Must remain false
752 params:
753 save_figures: true # bool: Per-branch and per-depth contact plots
754
755 # ---------- Noise perturbation ----------
756 noise_perturbation:
757 enabled: false # bool: Enable noise perturbation analysis
758 training: false # bool: Also run during training
759 params:
760 uniform_noise: true # bool: Uniform noise sweeps
761 uniform_magnitudes: [0.1, 0.2] # list[float]: Uniform noise magnitudes
762 gaussian_noise: true # bool: Gaussian noise sweeps
763 gaussian_sdevs: [0.1, 0.2] # list[float]: Gaussian noise standard deviations
764 n_samples: 10 # int: Noisy replicates per setting
765 accuracy: true # bool: Compute accuracy
766 auc: true # bool: Compute AUC
767 categorical_loglikelihood: true # bool: Compute categorical log-likelihood
768 mse: true # bool: Compute MSE
769 cosine_similarity: true # bool: Compute cosine similarity
770
771 # ---------- Adversarial robustness ----------
772 adversarial_robustness:
773 enabled: false # bool: Enable adversarial robustness (FGSM)
774 training: false # bool: Also run during training
775 params:
776 epsilons: [0.0, 0.05, 0.1, 0.15, 0.2, 0.3] # list[float]: FGSM epsilon values
777 batch_size: 256 # int: Not used in training runs: the analysis runtime batch_size replaces it
778
779 # ---------- Weight analysis ----------
780 weight:
781 enabled: false # bool: Enable weight analysis
782 training: false # bool: Also run during training
783 params:
784 synapse_weights: true # bool: Analyze synapse weights
785 branch_weights: true # bool: Analyze branch-to-output weights
786 computation_level: "single_branch" # str: "single_branch" | "layer_branch" | "all_branch"
787 branch_aggregation: "mean" # str: "mean" | "multivariate" | "sample"
788 per_layer_analysis: true # bool: Per-layer breakdown
789 per_einet_analysis: true # bool: Separate E and I network summaries
790 per_neuron_analysis: false # bool: Per-neuron breakdown (expensive)
791 analyze_excitatory: true # bool: Analyze E weights
792 analyze_inhibitory: true # bool: Analyze I weights
793 analyze_branch_output: true # bool: Analyze branch-to-output weights
794 compute_mean: true # bool: Compute mean weights
795 compute_variance: true # bool: Compute weight variance
796 compute_percentiles: true # bool: Compute percentiles
797 compute_min_max: true # bool: Compute min/max
798 compute_dendritic_strength: true # bool: Compute total dendritic strength
799 weight_threshold: 0.000001 # float: Minimum active weight
800
801 # ---------- Compartment statistics ----------
802 compartment_statistics:
803 enabled: false # bool: Enable compartment statistics
804 training: false # bool: Also run during training
805 params:
806 synapse_weights: true # bool: Analyze synapse weights
807 branch_weights: true # bool: Analyze branch weights
808 inputs: true # bool: Analyze inputs
809 activations: true # bool: Analyze activations
810 global_analysis: true # bool: Global-level statistics
811 layer_analysis: true # bool: Layer-level statistics
812 branch_analysis: true # bool: Branch-level statistics
813 compute_mean: true # bool: Compute mean
814 compute_variance: true # bool: Compute variance
815 compute_percentiles: true # bool: Compute percentiles
816 compute_min_max: true # bool: Compute min/max
817 compute_entropy: true # bool: Compute entropy
818 per_class_analysis: false # bool: Per-class statistics
819 n_samples: null # int | null: Limit samples (null = all)
820
821 # ---------- Compartment SNR ----------
822 # This is a discriminability analysis, not a simple signal/noise power ratio:
823 # branch metrics report d'^2, layer/input metrics report d_max, d_total,
824 # n_sig_dirs, and ratio_sig_dirs from between-class vs total covariance.
825 compartment_snr:
826 enabled: false # bool: Enable compartment SNR analysis
827 training: false # bool: Also run during training
828 params:
829 input_analysis: true # bool: Input-level statistics
830 input_capture_mode: "branch" # str: "branch" = dendritic branch inputs; "network" = network-level E/I input-output diagnostic
831 layer_analysis: true # bool: Layer statistics
832 branch_analysis: true # bool: Branch statistics
833 global_aggregation: true # bool: Aggregate global statistics
834 layer_aggregation: true # bool: Aggregate layer statistics
835 branch_aggregation: true # bool: Aggregate branch statistics
836 epsilon: 0.000001 # float: Numerical epsilon
837 null_permutations: 100 # int: Label shuffles for n_sig_dirs and d_total_null_mean (0: off)
838 null_quantile: 0.95 # float: Null quantile a significant direction must exceed
839 null_seed: null # int | null: Shuffle seed (null: from the run's probe seed)
840
841 # ---------- Multiplicative gain ----------
842 multiplicative_gain:
843 enabled: false # bool: Enable multiplicative gain analysis
844 training: false # bool: Also run during training
845 params:
846 uniform_gain: true # bool: One gain per input (false: per-element gains in [1, g])
847 gain_min: 0.0 # float: Minimum gain
848 gain_max: 10.0 # float: Maximum gain
849 n_steps: 10 # int: Number of gain steps
850 logspace: false # bool: Log-spaced gains ([1/gain_max, gain_max] when gain_min <= 0)
851 noise_seed: null # int | null: Seed of random gains and noise (null: from the run's probe seed)
852 accuracy: true # bool: Compute accuracy (default false; enabling the analyzer needs at least one score)
853 auc: false # bool: Compute AUC
854 categorical_loglikelihood: false # bool: Compute categorical log-likelihood
855 mse: false # bool: Compute MSE
856 cosine_similarity: false # bool: Compute cosine similarity
857
858 # ---------- Shared gain crossed with independent load ----------
859 gain_load_perturbation:
860 enabled: false # bool: Enable final-only gain/load grid
861 training: false # bool: Must remain false
862 params:
863 gain_log_sds: [0.0, 0.2, 0.5, 0.8] # list[float]: Mean-one lognormal gain s.d.
864 load_levels: [0.0, 0.1, 0.3, 0.5] # list[float]: Declared load levels
865 load_mode: gaussian_input # str: gaussian_input | positive_inhibitory
866 load_log_sd: 0.2 # float: Log-s.d. for positive I load
867 inhibitory_start: null # int | null: Start of ordered I stream
868 draws_per_condition: 5 # int: Technical perturbation draws
869 base_draw_seed: null # int | null: null = derived from experiment.probe_seed
870 max_samples: null # int | null: Evaluation sample cap
871 clamp_min: 0.0 # float | null: Lower input clamp
872 clamp_max: null # float | null: Upper input clamp
873 accuracy: true # bool: Record classification accuracy
874 categorical_loglikelihood: true # bool: Record categorical log-likelihood
875 record_input_statistics: false # bool: Record realized input moments per cell
876 record_branch_local_sensitivity: false # bool: Evaluate exact fitted branch derivatives
877 branch_sensitivity_module_name_prefix: null # str | null: One tree prefix
878 branch_sensitivity_expected_n_somas: null # int | null: Tree validation
879 branch_sensitivity_include_soma: false # bool: Include soma-relative d=0
880 branch_sensitivity_max_samples: null # int | null: Sensitivity sample cap
881 branch_sensitivity_draws: [0] # list[int]: Technical draws analyzed
882
883 # ---------- Receptive field ----------
884 # Default mode collects raw dendritic RF maps. Set compute_activation_tuning for
885 # class-tuned E/I/Vout summaries, compute_component_rfs for compact soma-level
886 # E/I mean and conditioned component RF maps, and compute_rf_tuning for
887 # aggregate RF tuning, entropy, discriminability, and E-vs-I RF plots.
888 receptive_field:
889 enabled: false # bool: Enable receptive field visualization
890 training: false # bool: Also run during training
891 params:
892 n_rows: 2 # int: Grid rows for plotting
893 n_cols: 2 # int: Grid columns for plotting
894 rf_shape: null # list[int] | null: Shared RF shape [H, W]
895 exc_rf_shape: null # list[int] | null: Excitatory RF shape [H, W]
896 inh_rf_shape: null # list[int] | null: Inhibitory RF shape [H, W]
897 fig_save_format: png # str: Figure extension, e.g. png or pdf
898 compute_activation_tuning: false # bool: Class-tuning summaries from branch E/I ratio and Vout activations
899 activation_sample_signals: [] # list[str]: Optional "exc" | "inh" | "upstream" | "depolarizing_drive" | "vout" | "ei_ratio" sample records
900 activation_sample_depths: [] # list[int]: Soma-relative depths retained in activation sample records
901 activation_samples_per_class: 0 # int: Label-balanced records per class; 0 disables records
902 activation_sample_seed: null # int | null: null = derived from experiment.probe_seed
903 compute_component_rfs: false # bool: Compact soma-level E/I mean and conditioned component RF maps
904 plot_component_rfs: true # bool: Save compact component RF figure on final analysis
905 compute_rf_tuning: false # bool: Rich aggregate RF tuning, entropy, and discriminability analysis
906 plot_rf_tuning: true # bool: Save rich RF tuning plots on final analysis when compute_rf_tuning is true
907 rf_tuning_layer_index: 0 # int: E/I layer index used for rich RF tuning
908 n_class_examples: 5 # int: Examples per class in RF input-example plots
909 save_raw_data: true # bool: Save raw RF tensor checkpoint
910 load_raw_data: false # bool: Reuse raw RF checkpoint if present
911 raw_filename: null # str | null: Override raw RF checkpoint filename
912 epsilon: 0.00000001 # float: Numerical epsilon for tuning ratios
913
914 # ---------- Single-layer contribution ----------
915 single_layer_contribution:
916 enabled: false # bool: Enable single-layer contribution analysis
917 training: false # bool: Also run during training
918 params:
919 excitation_contribution: true # bool: Excitation-only contribution
920 inhibition_contribution: true # bool: Inhibition-only contribution
921 both_contribution: true # bool: Combined E+I contribution
922 accuracy: true # bool: Compute accuracy
923 auc: true # bool: Compute AUC
924 categorical_loglikelihood: true # bool: Compute categorical log-likelihood
925 mse: true # bool: Compute MSE
926 cosine_similarity: true # bool: Compute cosine similarity
927
928 # ---------- Synaptic activation ----------
929 synaptic_activation:
930 enabled: false # bool: Enable synaptic activation analysis
931 training: false # bool: Also run during training
932 params:
933 network_summary: true # bool: Network-level summary
934 layer_summary: true # bool: Layer-level summary
935 branch_summary: true # bool: Branch-level summary
936 raw_summary: true # bool: Raw activation summary
937 logspace: false # bool: Log-space reporting
938 downsample: true # bool: Downsample activations
939 samples: 5 # int: Samples to keep
940
941 # ---------- Branch activation ----------
942 branch_activation:
943 enabled: false # bool: Enable branch activation analysis
944 training: false # bool: Also run during training
945 params:
946 synapse_activations: true # bool: Log synapse activations
947 branch_activations: true # bool: Log branch activations
948 network_summary: true # bool: Network-level summary
949 layer_summary: true # bool: Layer-level summary
950 branch_summary: true # bool: Branch-level summary
951 raw_summary: true # bool: Raw activation summary
952 logspace: false # bool: Log-space reporting
953 downsample: true # bool: Downsample activations
954 samples: 5 # int: Samples to keep
955
956 # ---------- Local rule components ----------
957 local_rule_components:
958 enabled: false # bool: Enable LocalCA component-factor logging
959 training: false # bool: Also run during training, for epoch trajectories
960 params:
961 max_samples: 1024 # int | null: Examples per snapshot
962 include_dendritic: true # bool: Include branch-to-parent conductance terms
963 include_broadcast: true # bool: Log scalar/per-soma broadcast-error proxy
964 broadcast_mode: "scalar" # str: "scalar" | "per_soma"
965 error_mode: "auto" # str: "auto" | "ce" | "bce" | "mse" | "none"
966 e_rev_exc: 1.0 # float: Fixed at 1.0, the model's excitatory reversal
967 e_rev_inh: 0.0 # float: Fixed at 0.0, the model's inhibitory reversal
968 save_json: true # bool: Save per-snapshot JSON
969 save_csv: true # bool: Save per-snapshot CSV
970 append_training_csv: true # bool: Append training snapshots to trajectory.csv
971
972 # ---------- Spike train ----------
973 spike_train:
974 enabled: false # bool: Enable LIF spike-train analysis
975 training: false # bool: Also run during training, for epoch trajectories
976 params:
977 dt: 1.0 # float: Time per recurrent step for rate reporting
978 threshold: 0.5 # float: Binarization threshold for recorded spike outputs
979 max_samples: 512 # int | null: Cap examples used for analysis
980 include_raster: true # bool: Include compact raster event lists
981 max_raster_samples: 8 # int: Number of examples to include in raster events
982 include_per_timestep_rate: true # bool: Include population firing-rate traces over time
983
984 # ---------- Routing ----------
985 routing:
986 enabled: false # bool: Enable routing analysis (recurrent models only)
987 training: false # bool: Also run during training
988
989# ============================================================================
990# OUTPUTS
991# ============================================================================
992outputs:
993 run_name: "ei_sweep" # str: Run name; a run writes to <results_dir>/<run_name>_<YYYYMMDD_HHMMSS>/
994 results_dir: "results" # str: Parent directory of the run directories
995 # Not shown: exact_run_dir (true: write into results_dir itself;
996 # docs/runs_and_outputs.rst).
997
998# ============================================================================
999# DISTRIBUTED TRAINING
1000# ============================================================================
1001distributed:
1002 # "none": one process (dendritic-train). "ddp": DistributedDataParallel,
1003 # launched with torchrun -m dendritic_modeling.scripts.training.train_experiments.
1004 # "fsdp": FullyShardedDataParallel, launched with torchrun -m
1005 # dendritic_modeling.scripts.training.train_experiments_fsdp (see
1006 # config_exp_fsdp.yaml). The launcher sets the number of nodes and
1007 # processes, not this config.
1008 mode: "none" # str: "none" | "ddp" | "fsdp"
1009 # Not shown: timeout_seconds (int | null), the collective timeout of ddp and
1010 # fsdp runs (null = PyTorch's default of 600 s under DDP, 1800 s under
1011 # FSDP); raise it when rank-0 analyses during training take long.
1012
1013 # FSDP settings (read only with mode: "fsdp"; docs/distributed_training.rst)
1014 fsdp:
1015 # FULL_SHARD: shard parameters, gradients and optimizer state (lowest memory)
1016 # SHARD_GRAD_OP: shard gradients and optimizer state only
1017 # NO_SHARD: no sharding (like DDP)
1018 # HYBRID_SHARD: shard within a node, replicate across nodes
1019 sharding_strategy: "FULL_SHARD" # str: "FULL_SHARD" | "SHARD_GRAD_OP" | "NO_SHARD" | "HYBRID_SHARD"
1020 mixed_precision: true # bool: FSDP mixed precision; dtypes from mixed_precision_dtype (default "bf16")
1021 cpu_offload: false # bool: Offload parameters to CPU memory (saves GPU memory, slower)
1022 auto_wrap_policy: "none" # str: "none" (whole model is one unit) | "indexed_synapses" (one unit per indexed synapse module) | "size"
1023 min_num_params: 1000000 # int: Size threshold of auto_wrap_policy "size" (other policies reject a non-default value)
1024 limit_all_gathers: true # bool: Limit concurrent all-gathers
1025 forward_prefetch: true # bool: Prefetch the next FSDP unit in forward
1026 sync_module_states: true # bool: Broadcast rank 0's module states when wrapping
1027 use_orig_params: true # bool: Keep the original parameters (needed for the per-component parameter groups)
1028 backward_prefetch: "BACKWARD_PRE" # str: "BACKWARD_PRE" | "BACKWARD_POST"
1029 gradient_checkpointing: false # bool: Activation checkpointing (recompute activations in backward to save memory)
FSDP configuration
configs/config_exp_fsdp.yaml trains a population network with indexed
synapses with FullyShardedDataParallel (distributed.mode: fsdp). Its
sections are in alphabetical order, as in a saved config.json, and its
comments explain the settings that matter under FSDP. Launch one process per
GPU, here on a node with four GPUs:
torchrun --nproc_per_node=4 \
-m dendritic_modeling.scripts.training.train_experiments_fsdp \
configs/config_exp_fsdp.yaml
Distributed training describes multi-node launches and the FSDP options.
1# ============================================================================
2# DENDRINET ANNOTATED REFERENCE CONFIG (FSDP)
3# ============================================================================
4# A population network with indexed synapses on CIFAR-10, trained with
5# FullyShardedDataParallel (distributed.mode: fsdp). Launch one process per GPU
6# on one node from the repository root:
7# torchrun --nproc_per_node=<gpus> \
8# -m dendritic_modeling.scripts.training.train_experiments_fsdp \
9# configs/config_exp_fsdp.yaml
10# Add --validate-only to build the datasets and the wrapped model and
11# optimizer without training. docs/distributed_training.rst covers multi-node
12# launches and every FSDP option.
13#
14# The sections are in alphabetical order, as in a saved config.json.
15# config_exp.yaml describes every option; the comments here explain the
16# settings that matter under FSDP. The FSDP trainer trains with
17# backpropagation and a plain optimizer step. It rejects LR schedules,
18# regularization, pruning, sparse-synapse maintenance (synapse_decay_rate must
19# be 0 and weight_boosting false for Top-K and indexed synapses), rewiring
20# synapses, and strategies other than "standard" and "recurrent".
21
22config_version: 4
23
24analysis:
25 runtime:
26 # Same keys as in config_exp.yaml. Under DDP and FSDP an omitted mode is
27 # "auto"; "stream" never materializes a split, which suits large datasets.
28 training:
29 mode: "auto" # "auto" materializes splits up to materialize_threshold_mb and streams larger ones
30 batch_size: 256
31 num_workers: 4
32 pin_memory: true
33 persistent_workers: true
34 prefetch_factor: 4
35 in_order: false # false: batches arrive as workers finish them
36 max_batches: null
37 max_samples: null
38 materialize_threshold_mb: 1024
39 final:
40 mode: "stream" # Stream every split in batches
41 batch_size: 256
42 num_workers: 4
43 pin_memory: true
44 persistent_workers: true
45 prefetch_factor: 4
46 in_order: false
47 max_batches: null
48 max_samples: null
49 materialize_threshold_mb: 1024
50 ablation:
51 enabled: false
52 params:
53 selection:
54 levels: ["layer"] # Levels: "layer", "compartment" (compartment is lesion-only)
55 targets: ["excitation", "inhibition"] # Targets: "all_synapses", "excitation", "inhibition", "upstream"
56 metrics: ["accuracy", "auc", "categorical_loglikelihood"] # Metrics: "accuracy", "auc", "categorical_loglikelihood", "mse", "cosine_similarity", "pred_label_mi_bits"
57
58 methods:
59 ablation_methods: ["lesion"] # Methods: "lesion", "shuffle", "mean_clamp"
60 shuffle_seed: 0 # Shuffle seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
61 clamp_statistic: "mean" # mean_clamp statistic: "mean" (the only supported value)
62 training: false
63 branch_activation:
64 enabled: false
65 params:
66 branch_activations: true
67 branch_summary: true
68 downsample: true
69 layer_summary: true
70 logspace: false
71 network_summary: true
72 raw_summary: true
73 samples: 5
74 synapse_activations: true
75 training: false
76 correlation:
77 enabled: false
78 params:
79 selection:
80 components: ["total", "noise", "signal", "tuning"] # Components: "total", "noise", "signal", "tuning"
81 pairs: ["EE", "II", "EI", "E_output", "I_output", "output_output"] # Pairs: "EE", "II", "EI", "E_output", "I_output", "output_output"
82 scopes: ["layer", "einet"] # Scopes: "layer", "einet", "neuron"
83
84 compute:
85 computation_level: "layer_branch" # Level: "synaptic", "single_branch", "layer_branch", "all_branch"
86 max_samples: 5000 # Max samples (null = all)
87 per_layer_analysis: true # Extra breakdown/plots: per-layer (overridden by selection.scopes if non-empty)
88 per_einet_analysis: true # Extra breakdown/plots: per-E/I-net (overridden by selection.scopes if non-empty)
89 per_neuron_analysis: false # Extra breakdown/plots: per-neuron (expensive; overridden by selection.scopes if non-empty)
90
91 sampling:
92 n_synapse_pairs: 10000 # Synaptic-only: number of synapse pairs (int >= 1)
93 sampling_strategy: "stratified" # Sampling: "stratified", "random" (other values raise)
94 stratified_categories: ["within_branch", "within_layer"] # Stratified categories: "within_branch", "within_layer"
95 max_pairs_per_category: 1000 # Stratified-only: max pairs per category (int >= 1)
96 training: false
97 information:
98 enabled: false
99 params:
100 selection:
101 components: ["basic", "pairwise", "conditional", "ablation_aligned", "layer_proxies"] # Components: "basic", "pairwise", "conditional", "gaussian_fisher", "pid", "ablation_aligned", "layer_proxies", "upstream_unique_cmi", "soma_coupling"
102 scopes: [] # Scopes: "layer", "einet", "neuron" (alias: views)
103 signal_variants: ["network", "lda"] # Variants: "network", "lda", "random" (random requires baselines.random_weights.n_shuffles > 0)
104
105 compute:
106 max_samples: null # Max samples (null = all)
107 include_vinf: true # Include soma activation: true|false
108 normalize_mi: false # Normalize by entropy: true|false
109 verbose: false # Verbose: true|false
110 computation_level: layer_branch # Level: "single_branch", "layer_branch", "all_branch"
111 branch_aggregation: multivariate # Aggregation: "mean", "multivariate", "sample"
112 per_layer_analysis: false # Extra breakdown/plots: per-layer (overridden by selection.scopes if non-empty)
113 per_einet_analysis: false # Extra breakdown/plots: per-E/I-net (overridden by selection.scopes if non-empty)
114 per_neuron_analysis: false # Extra breakdown/plots: per-neuron (expensive; overridden by selection.scopes if non-empty)
115 layer_total_topk: 10 # K for *_topK_sum proxies (int >= 1)
116 gaussian_fisher:
117 aggregation: "sum" # Aggregate per-feature F: "sum"|"mean"|"max"
118 mi_transform: "half_log1p" # Transform F→MI proxy: "half_log1p"|"log1p"|"none"
119 eps: 1.0e-8 # Denominator stabilizer (alias: ridge) (float > 0)
120
121 pid:
122 binarize: false # Threshold E, I and Vout (never the labels) before PID: true|false
123 binarize_method: median # Binarization: "median"|"mean"|"quantile"
124 binarize_threshold: 0.5 # Used only if binarize_method="quantile": q∈[0,1]
125
126 estimator:
127 method: kraskov # MI estimation: "kraskov", "decoder", "binned", "sklearn", "copula", "auto"
128 kraskov:
129 n_neighbors: 10 # Kraskov k-NN (int >= 1)
130 binned:
131 n_bins: 20 # Histogram bins of the binned estimator; the sklearn estimator uses it too (int >= 2)
132 binning_strategy: "quantile" # binned: "quantile" or "uniform"
133 bias_correction: "none" # binned: "none" or "miller_madow"
134 decoder:
135 cv_folds: 5 # decoder: CV folds (int >= 2)
136 C: 1.0 # decoder: LogisticRegression C (float > 0)
137 standardize: true # decoder: standardize X (true|false)
138 seed: 0 # decoder: CV shuffle seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
139 continuous:
140 strategy: "none" # decoder continuous: "none", "binned", "gaussian"
141 max_total_dim: 2 # decoder continuous: max total dim (x+y [+z]); used only if strategy != "none"
142 gaussian_ridge: 1e-6 # decoder continuous: gaussian ridge (float > 0)
143 copula:
144 type: gaussian # Copula: "gaussian" (scalars, vectors, labels), "kernel" (two continuous scalars)
145 sklearn: # Not read: the sklearn estimator takes its bins from estimator.binned.n_bins
146 n_bins: 20 # Not read
147 bandwidth: null # Not read
148
149 baselines:
150 random_weights:
151 n_shuffles: 0 # Random-weight baseline shuffles (int >= 0; produces *_sh metrics)
152 seed: 42 # Random-weight seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
153 label_shuffle_null:
154 n_shuffles: 0 # Null (label-shuffle) baseline shuffles (int >= 0; produces *_null metrics)
155 seed: 0 # Null baseline seed; a written seed is used as is in every run (null = derived from experiment.probe_seed)
156 components: ["basic"] # Null components: "basic", "pairwise", "conditional", "ablation_aligned", "upstream_unique_cmi", "soma_coupling", "all"
157 training: false
158 noise_perturbation:
159 enabled: false
160 params:
161 accuracy: true
162 auc: true
163 categorical_loglikelihood: true
164 cosine_similarity: false
165 gaussian_noise: true
166 gaussian_sdevs:
167 - 0.1
168 - 0.2
169 mse: false
170 n_samples: 10
171 uniform_magnitudes:
172 - 0.1
173 - 0.2
174 uniform_noise: true
175 training: false
176 performance:
177 enabled: true
178 params:
179 accuracy: true
180 auc: true
181 categorical_loglikelihood: true
182 cosine_similarity: false
183 mse: false
184 # training: true runs the performance analysis after every epoch on rank 0
185 # while the other ranks wait in a collective (distributed.timeout_seconds,
186 # default 1800 s under FSDP, bounds the wait).
187 training: true
188 training_splits: ["train", "validation", "test"] # use ["validation"] for cheaper training-time checks
189 final_splits: ["train", "validation", "test"]
190 single_layer_contribution:
191 enabled: false
192 params:
193 accuracy: true
194 auc: true
195 both_contribution: true
196 categorical_loglikelihood: true
197 cosine_similarity: false
198 excitation_contribution: true
199 inhibition_contribution: true
200 mse: false
201 training: false
202 synapse_turnover:
203 enabled: false
204 training: false # Set to true before enabling: turnover compares training snapshots, and training: false raises
205 params:
206 snapshot_interval: 10
207 synaptic_activation:
208 enabled: false
209 params:
210 branch_summary: true
211 downsample: true
212 layer_summary: true
213 logspace: false
214 network_summary: true
215 raw_summary: true
216 samples: 5
217 training: false
218 weight:
219 enabled: false
220 params:
221 analyze_branch_output: false
222 analyze_excitatory: true
223 analyze_inhibitory: true
224 branch_aggregation: mean
225 branch_weights: true
226 computation_level: single_branch
227 compute_dendritic_strength: true
228 compute_mean: true
229 compute_min_max: true
230 compute_percentiles: false
231 compute_variance: true
232 per_einet_analysis: true
233 per_layer_analysis: true
234 per_neuron_analysis: false
235 synapse_weights: true
236 weight_threshold: 1e-6
237 training: false
238 compartment_statistics:
239 enabled: true # Enabled in this example (default false)
240 training: false
241 params:
242 synapse_weights: true
243 branch_weights: true
244 inputs: true
245 activations: true
246 global_analysis: true
247 layer_analysis: true
248 branch_analysis: true
249 compute_mean: true
250 compute_variance: true
251 compute_percentiles: true
252 compute_min_max: true
253 compute_entropy: true
254 per_class_analysis: false
255 n_samples: 1000
256data:
257 base_dir: '' # Data root (empty = $DENDRITIC_DATA_DIR, else ~/.cache/dendritic_modeling)
258 dataset_name: cifar10
259 dataset_params: # Only the block named by dataset_name is read; these two are ignored with cifar10
260 mnist_modulo10:
261 shuffle_iterations: 2
262 poisson_generator:
263 base_dataset: mnist
264 processing:
265 flatten: true
266 normalize: false
267experiment:
268 checkpointing:
269 enabled: false # true: the FSDP trainer saves checkpoints (with optimizer state) under checkpoints/ in the run directory
270 save_every_n_epochs: 10
271 seed: 42
272 train_valid_split: 0.8
273distributed:
274 mode: fsdp # Train with FSDP: launch train_experiments_fsdp under torchrun (see the header)
275 fsdp:
276 backward_prefetch: BACKWARD_POST # "BACKWARD_PRE" (default) | "BACKWARD_POST"
277 cpu_offload: false
278 forward_prefetch: true
279 gradient_checkpointing: false
280 limit_all_gathers: true
281 mixed_precision: false # Full-precision parameters, reductions and buffers (default true: bf16)
282 sharding_strategy: FULL_SHARD # "FULL_SHARD" | "SHARD_GRAD_OP" | "NO_SHARD" | "HYBRID_SHARD"
283 sync_module_states: true
284 use_orig_params: true # Needed for the per-component learning rates (training.main.common.param_groups)
285 # Not set here: auto_wrap_policy ("none" makes the whole model one FSDP unit;
286 # "indexed_synapses" wraps each indexed synapse module, which bounds the
287 # parameter all-gathers of contact-heavy models), mixed_precision_dtype and
288 # the per-tensor dtypes, freeze_core, cache_frozen_core_outputs,
289 # cached_core_batch_size, and distributed.timeout_seconds.
290model:
291 core:
292 biological_neuron: true # Enforce the neuronal-core constraint set.
293 type: population_network
294 population_network:
295 input_transform: nonnegative # Check nonnegative data without changing values or gradients.
296 use_transfer: false
297 transfer_params: {}
298 output_mode: "last"
299 input_projection_dims: []
300 store_routing: false
301 layers:
302 - name: layer0
303 readout_population: e
304 population_defaults:
305 ff_excitatory_synapses: 40
306 ff_inhibitory_synapses: 10
307 rec_excitatory_synapses: 0
308 rec_inhibitory_synapses: 0
309 somatic_synapses: true
310 use_shunting: true
311 reactivate: true
312 reactivation_type: param_tanh
313 reactivation_init_policy: occupancy_quantile # Data-driven gate placement (the default for shunting populations is "analytical")
314 dbl_init_method: analytical_expectation
315 weight_transform: softplus
316 # FSDP has no rewiring step, so rewiring synapses (indexed_rewire,
317 # deepst, credit_deepst, dense_to_sparse with advance_on_forward:
318 # false) raise ValueError.
319 topk_type: indexed # Indexed synapses store only the active contacts
320 topk_init_method: xavier_normal
321 indexed_output_chunk_size: 2048
322 efficient_blocklinear: true
323 populations:
324 - name: i
325 polarity: inhibitory
326 n_neurons: 32
327 branch_factors: [1]
328 population:
329 ff_inhibitory_synapses: 0
330 - name: e
331 polarity: excitatory
332 n_neurons: 128
333 branch_factors: [3, 3, 3]
334 connections:
335 - source: input
336 target: i
337 - source: input
338 target: e
339 - source: i
340 target: e
341 decoder:
342 params: {} # MLP defaults: hidden_dims [16], output_dim 10; input_dim comes from the core
343 type: MLP
344 encoder:
345 params: {}
346 type: identity
347 task: classification
348outputs:
349 results_dir: results
350 run_name: fsdp_large_exp
351training:
352 main:
353 common:
354 num_workers: 8
355 pin_memory: true
356 persistent_workers: true
357 prefetch_factor: 4
358 param_groups: {}
359 synapse_decay_rate: 0.0 # Must be 0: FSDP runs no sparse-synapse maintenance (weight_boosting keeps its default, false)
360 pruning: {}
361 regularization: {}
362 strategies: # Only the block of the selected strategy is read
363 freeze_branch_kl: {}
364 freeze_branches: {}
365 freeze_layers: {}
366 homeostatic_control: {}
367 multi_stage: {}
368 standard: {}
369 voltage_stabilization: {}
370 strategy: standard # The FSDP trainer runs "standard" or "recurrent"
Sweep template
configs/sweep_config.yaml is the annotated template of a parameter sweep:
a base_config and the options to vary (see Parameter sweeps). Generate its
configs without submitting jobs:
dendritic-sweep --config configs/sweep_config.yaml --generate-only --output-dir sweeps
1# ============================================================================
2# SWEEP CONFIGURATION TEMPLATE
3# ============================================================================
4# Annotated template for parameter sweeps with DendriNet. docs/sweeps.rst
5# describes every section, option and command.
6#
7# USAGE:
8# Generate the configs and job scripts without submitting (an empty account
9# or partition becomes PLACEHOLDER):
10# dendritic-sweep --config configs/sweep_config.yaml --generate-only
11#
12# Generate the configs and submit the Slurm jobs (account and partition are
13# required, here or in slurm_config):
14# dendritic-sweep --config configs/sweep_config.yaml \
15# --slurm-account ACCOUNT --slurm-partition PARTITION
16#
17# Train one generated config without Slurm:
18# dendritic-train <sweep_dir>/configs/unified_config_0.yaml
19#
20# Tabulate and plot the finished runs (pass the sweep directory, not its
21# results/ subdirectory):
22# dendritic-analyze <sweep_dir>
23#
24# WHAT CAN BE SWEPT:
25# - Any option of base_config, through its dot path (sweep_config)
26# - E/I synapse-count pairs that satisfy a constraint (filter_config, A)
27# - Dendritic tree depths and branching patterns (filter_config, B)
28# - Named conditions that change several options together (sweep_variants)
29# All of these combine in one grid.
30#
31# HOW IT WORKS:
32# The sweep manager takes the Cartesian product of the sweep_config axes and
33# the sweep_variants, applies filter_config, and writes one training config
34# per condition and replicate seed to <sweep_dir>/configs. Every generated
35# config is loaded with the strict config loader before any job script is
36# written; the models are not built. The sections of this file are checked
37# when it loads: an unknown key in slurm_config, experiment_settings,
38# filter_config or outputs raises with the closest valid key, and so does a
39# top-level key that nearly matches a section name. Other top-level keys are
40# reported in a warning and ignored; put annotations under metadata:.
41#
42# TOP-LEVEL SECTIONS:
43# slurm_config - Slurm resources, environment and launcher
44# experiment_settings - Replicate seeds (seeds_per_condition, base_seed, seed_paths)
45# sweep_config - Axes: dot paths into base_config -> values or ranges
46# sweep_variants - Optional named conditions that change several paths together
47# filter_config - E/I pair filter and branch sweeps
48# base_config - Complete training config shared by all conditions
49# execution_results_dir - Optional separate root for the run outputs
50# sweep_contract - Optional expected_config_count, checked at generation
51# output_dir, outputs - Optional base directory (output_dir or outputs.dir;
52# default base_config.outputs.results_dir) and name
53# prefix (outputs.run_name; default
54# base_config.outputs.run_name) of the sweep directory
55# extends - Optional parent sweep file, relative to this file
56# (mappings are merged, lists are replaced)
57# ============================================================================
58
59
60# ============================================================================
61# 1. SLURM CONFIGURATION
62# ============================================================================
63# The #SBATCH header, environment and launch command of every job script of
64# the sweep. By default all configs run as one array job, one config per array
65# task (--split-jobs N writes N sequential jobs instead). The job name is the
66# sweep's prefix (base_config.outputs.run_name here).
67#
68# account, partition: required to submit; set them here or pass
69# --slurm-account and --slurm-partition. They become PLACEHOLDER only with
70# --generate-only or --dry-run. A site policy file
71# (DENDRITIC_SCHEDULER_PROFILES, or
72# $XDG_CONFIG_HOME/dendritic_modeling/scheduler_profiles.yaml) may restrict
73# the allowed account/partition pairs.
74# launcher: "torchrun" runs torch.distributed.run --standalone with
75# gpus_per_node processes, one node per config. Use it with
76# base_config.distributed.mode: ddp, or with
77# training_script: src/dendritic_modeling/scripts/training/train_experiments_fsdp.py
78# and distributed.mode: fsdp.
79# Every job runs "module purge" and then "module load" for modules_to_load.
80# Two more optional keys, qos and constraint, add --qos and --constraint.
81# An unknown slurm_config key raises when the sweep loads, naming the closest
82# valid key.
83
84slurm_config:
85 account: "" # Slurm account (required to submit)
86 partition: "" # Slurm partition (required to submit)
87 time: "12:00:00" # Per job or array task: quoted HH:MM:SS or D-HH:MM:SS (default 02:00:00)
88 nodes: 1 # Nodes per job; keep 1
89 ntasks_per_node: 1 # Tasks per node
90 gpus_per_node: 1 # --gres=gpu:N; also the torchrun process count
91 launcher: "python" # "python" or "torchrun" (see above)
92 cpus_per_task: 12 # CPU cores per array task (default 4)
93 mem: 200GB # Memory per node, passed to Slurm as written (default 32G)
94 max_concurrent_jobs: 32 # Max array tasks running at once (default 10)
95 exclude_nodes: [] # Nodes to exclude (e.g., ["node001"]); --exclude-nodes replaces it
96 modules_to_load: [] # Environment modules loaded after "module purge"
97 conda_env_path: "" # Environment root whose bin/python trains; empty = the "python" on PATH after "module purge"
98 training_script: "src/dendritic_modeling/scripts/training/train_experiments.py" # Relative to the library checkout
99
100
101# ============================================================================
102# 2. EXPERIMENT SETTINGS
103# ============================================================================
104# Replicate seeds. Replicate r of every condition gets
105# experiment.seed = base_seed + r, so all conditions use the same seeds.
106# Total configs = conditions x seeds_per_condition (here 8 x 5 = 40); they run
107# as one array job.
108
109experiment_settings:
110 seeds_per_condition: 5 # Replicates per condition (default 1)
111 base_seed: 0 # Seeds 0, 1, ..., 4 (default base_seed 42)
112 # Random streams the base config leaves unset (data split, initialization,
113 # topology, data order) follow experiment.seed, so replicates also differ in
114 # connectivity. Set base_config.experiment.topology_seed to hold the
115 # connection masks and indexed-synapse indices fixed. seed_paths writes the
116 # replicate seed into further paths, e.g. a component seed the base config
117 # pins:
118 # seed_paths:
119 # - model.core.population_network.layers.0.population_defaults.indexed_seed
120
121# Optional: keep the sweep files (configs, job scripts, logs) in the sweep
122# directory and write the run outputs under this separate root. Each generated
123# config writes to <execution_results_dir>/config_<index>, and
124# <sweep_dir>/results links to the root. Generation does not create the root.
125# execution_results_dir: "/project/results/my_sweep"
126
127# Optional named conditions. Each variant is one condition whose overrides are
128# applied together; every variant is crossed with the sweep_config axes. A
129# variant may change only paths the base core type reads: a
130# population_network base takes overrides under model.core.population_network.
131# This pair doubles the grid (2 x 8 = 16 conditions):
132# sweep_variants:
133# - name: shunting
134# overrides:
135# model.core.population_network.layers.0.population_defaults.use_shunting: true
136# model.core.population_network.layers.0.population_defaults.use_additive_normalization: false
137# - name: normalized_additive
138# overrides:
139# model.core.population_network.layers.0.population_defaults.use_shunting: false
140# model.core.population_network.layers.0.population_defaults.use_additive_normalization: true
141
142
143# ============================================================================
144# 3. SWEEP CONFIG -- Parameter Grid
145# ============================================================================
146# Each key is a dot path into base_config. Each value is a list of literal
147# values, a scalar (one value), or an explicit range:
148# {range: {start, stop, step}} start, start + step, ..., up to and including stop
149# {linspace: {start, stop, num}} num evenly spaced values, both ends included
150# {geomspace: {start, stop, num}} num geometrically spaced values, both ends included
151# {values: [...]} the literal values
152# A plain three-number list that reads as [start, stop, step], such as
153# [1, 10, 2], is rejected as ambiguous. The sweep manager computes the full
154# Cartesian product of the axes.
155#
156# SYNTAX:
157# dot.path.to.param: [val1, val2, val3]
158#
159# The dot path mirrors the YAML hierarchy of base_config; numeric components
160# index lists. For example:
161# training.main.common.param_groups.topk_lr -> base_config.training.main.common.param_groups.topk_lr
162# data.processing.flatten -> base_config.data.processing.flatten
163# model.core.population_network.layers.0.populations.1.n_neurons
164# -> n_neurons of population "e" in layer "layer0"
165#
166# VALUE TYPES (each line works with this file's base_config):
167# Numbers: training.main.common.param_groups.topk_lr: [0.001, 0.01, 0.1]
168# Strings: model.core.population_network.layers.0.population_defaults.weight_transform: ["softplus", "exp"]
169# Booleans: model.core.population_network.layers.0.population_defaults.somatic_synapses: [true, false]
170# Lists: model.core.population_network.layers.0.populations.1.branch_factors: [[3, 3], [4, 4]]
171# Ranges: training.main.common.batch_size: {range: {start: 128, stop: 512, step: 128}}
172#
173# LEARNING RATES: with split_params: true (see base_config), the sparse
174# synapses, BlockLinear couplings, reactivation gates and decoder train at
175# topk_lr, blocklinear_lr, reactivation_lr and decoder_lr. param_groups.lr
176# sets no parameter of this model, so sweep the group rates instead.
177#
178# EXAMPLES (each replaces the axes below; counts are conditions):
179# # Sweep the sparse-synapse learning rate and batch size (2 x 3 = 6 conditions):
180# # training.main.common.param_groups.topk_lr: [0.001, 0.01]
181# # training.main.common.batch_size: [128, 256, 512]
182#
183# # Sweep population size and morphology (2 x 2 = 4 conditions):
184# # model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
185# # model.core.population_network.layers.0.populations.1.branch_factors: [[3, 3], [4, 4]]
186#
187# # Sweep E/I synapse counts in population-network defaults (3 x 3 = 9 conditions):
188# # model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 30, 40]
189# # model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 15, 20]
190
191sweep_config:
192 # Population-network synapse counts and morphology (2 x 2 x 2 = 8
193 # conditions). Numeric path components index YAML lists, so layers.0 is the
194 # first population-network layer and populations.1 is the excitatory
195 # readout population "e" of the base config.
196 model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses:
197 - 20
198 - 30
199
200 model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses:
201 - 10
202 - 15
203
204 model.core.population_network.layers.0.populations.1.branch_factors:
205 - [3, 3]
206 - [4, 4]
207
208
209# ============================================================================
210# 4. FILTER CONFIG -- E/I Filtering and Branch Sweeps
211# ============================================================================
212# Two independent mechanisms that can be used alone or combined:
213#
214# A) E/I FILTERING -- keep only some excitatory/inhibitory synapse-count pairs
215# B) BRANCH SWEEP -- add a dimension of dendritic trees to the grid
216#
217# --------------------------------------------------------------------------
218# A) E/I FILTERING
219# --------------------------------------------------------------------------
220# sweep_config must contain both E/I axes: the population_defaults
221# ff_excitatory_synapses and ff_inhibitory_synapses paths (population
222# networks) or model.core.connectivity.ee_synapses_per_branch_per_layer and
223# ie_synapses_per_branch_per_layer (structured E/I cores). The other axes are
224# crossed with the pairs that remain.
225#
226# mode: "all" - No filtering (the default; "default" is the same)
227# mode: "ei-sum" - Keep pairs with |E + I - value| <= tolerance
228# mode: "ei-ratio" - Keep pairs with I > 0 and |E / I - value| <= tolerance
229# mode: "e-const" - Keep E = value with every I (value must be on the E axis)
230# mode: "i-const" - Keep I = value with every E (value must be on the I axis)
231# mode: "branch_sweep" - No filtering; required for a branch sweep without
232# sweep_config axes or sweep_variants
233#
234# value: <number> - Target value; required by the four E/I modes
235# tolerance: <number> - Required by ei-sum and ei-ratio (0 = exact match)
236# A filter that keeps no pair raises.
237#
238# EXAMPLES (with E and I axes of [10, 20, 30, 40] each):
239# # Keep the pairs with E + I = 50 (4 pairs):
240# # mode: "ei-sum"
241# # value: 50
242# # tolerance: 0
243#
244# # Keep the E/I ratios within 0.5 of 2.0 (3 pairs):
245# # mode: "ei-ratio"
246# # value: 2.0
247# # tolerance: 0.5
248#
249# # Hold inhibition at 10 (4 pairs):
250# # mode: "i-const"
251# # value: 10
252#
253# --------------------------------------------------------------------------
254# B) BRANCH SWEEP
255# --------------------------------------------------------------------------
256# Each generated tree replaces branch_factors of every excitatory population
257# (polarity "excitatory" or no polarity) in every population_network layer
258# (model.core.architecture.excitatory_branch_factors for structured E/I
259# cores). Remove any sweep_config axis on those branch_factors first: the tree
260# overwrites it, and conditions that differ only in that axis become
261# identical runs. A factor b at depth d gives d levels of b branches. Depths
262# and factors are axis values: literal lists or explicit ranges (section 3).
263#
264# branch_sweep_mode options:
265#
266# "uniform_sweep" - Every depth with every branch factor (default mode)
267# dendrite_tree_depth: {range: {start: 1, stop: 4, step: 1}}
268# dendrite_branches_per_layer: {values: [2, 3]}
269# Example: depths 1-4 x factors 2, 3 -> 8 trees ([2], [3], [2, 2], ...)
270#
271# "depth_sweep" - Vary the depth with one branch factor at every level
272# dendrite_tree_depth: {range: {start: 2, stop: 5, step: 1}}
273# uniform_branches_per_layer: 4
274# Example: [4, 4], [4, 4, 4], [4, 4, 4, 4], [4, 4, 4, 4, 4] -> 4 trees
275#
276# "width_sweep" - Vary the branch factor at a fixed depth
277# fixed_dendrite_tree_depth: 3
278# dendrite_branches_per_layer: {range: {start: 2, stop: 8, step: 2}}
279# Example: [2, 2, 2], [4, 4, 4], [6, 6, 6], [8, 8, 8] -> 4 trees
280#
281# "pattern_sweep" - Named branching patterns across depths
282# dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
283# base_branches: 3
284# branch_patterns:
285# - "uniform" # base_branches at every level, e.g. [3, 3, 3]
286# - "decreasing" # base_branches, base_branches - 1, ..., at least 1, e.g. [3, 2, 1]
287# Example: 3 depths x 2 patterns -> 6 trees
288# Two more patterns ignore base_branches: "increasing" (1, 2, ..., depth)
289# and "pyramid" (1, 2, ... up to level depth // 2, then depth - level:
290# [1, 2, 1] at depth 3, [1, 2, 3, 1] at depth 4).
291#
292# "custom_configs" - Explicit trees, each a mapping with depth and branches
293# custom_branch_configs:
294# - {depth: 3, branches: [4, 4, 4]}
295# - {depth: 3, branches: [8, 4, 2]}
296# - {depth: 3, branches: [2, 4, 8]}
297# Example: 3 trees, each used as written
298#
299# COMBINING E/I FILTER + BRANCH SWEEP:
300# Set the E/I mode and branch_sweep_mode together. The total grid is:
301# (sweep_config Cartesian product, with the E/I axes reduced to the
302# pairs the filter keeps)
303# x (branch sweep trees)
304# x (sweep_variants)
305# x seeds_per_condition
306#
307# --------------------------------------------------------------------------
308
309filter_config:
310 # E/I filtering (disabled by default)
311 mode: "all" # "all" | "ei-sum" | "ei-ratio" | "e-const" | "i-const" | "branch_sweep"
312 # value: 50 # Target value (required by the E/I modes)
313 # tolerance: 0 # Allowed deviation (required by ei-sum and ei-ratio; 0 = exact)
314
315 # Branch sweep (disabled by default). Before uncommenting these lines,
316 # delete the populations.1.branch_factors axis in sweep_config, which the
317 # trees replace (2 x 2 E/I conditions x 4 trees = 16 conditions):
318 # branch_sweep_mode: "depth_sweep"
319 # dendrite_tree_depth: {range: {start: 1, stop: 4, step: 1}}
320 # uniform_branches_per_layer: 3
321
322
323# ============================================================================
324# 5. BASE CONFIG -- Training Configuration Template
325# ============================================================================
326# The complete training config shared by all sweep conditions, in the schema
327# of config_exp.yaml. The sweep axes overwrite the values at their paths in
328# each generated config.
329
330base_config:
331 config_version: 4
332
333 # ---------- Experiment ----------
334 experiment:
335 seed: 42 # Replaced in each generated config by base_seed + replicate index
336 train_valid_split: 0.8
337
338 checkpointing:
339 enabled: false
340 save_every_n_epochs: 10
341
342 # ---------- Data ----------
343 data:
344 dataset_name: "mnist"
345 base_dir: ""
346
347 processing:
348 flatten: true
349 normalize: true
350
351 # ---------- Model ----------
352 model:
353 task: "classification"
354
355 encoder:
356 type: "identity"
357 params:
358 input_dim: 784
359 input_shape: [1, 28, 28]
360 latent_dim: null
361
362 core:
363 biological_neuron: false # Preserve this example's historical model contract.
364 type: "population_network"
365 population_network:
366 use_transfer: false
367 transfer_params: {}
368 layers:
369 - name: "layer0"
370 readout_population: "e"
371 population_defaults:
372 ff_excitatory_synapses: 30
373 ff_inhibitory_synapses: 15
374 rec_excitatory_synapses: 0
375 rec_inhibitory_synapses: 0
376 somatic_synapses: true
377 use_shunting: true
378 use_additive_normalization: false
379 reactivate: true
380 reactivation_type: "param_tanh"
381 reactivation_init_policy: "occupancy_quantile"
382 dbl_init_method: "analytical_expectation"
383 weight_transform: "softplus"
384 topk_type: "indexed_rewire"
385 topk_init_method: "xavier_normal"
386 indexed_rewire_frequency: 100
387 indexed_rewire_quantile: 0.05
388 indexed_rewire_until_step: null
389 indexed_output_chunk_size: 2048
390 blocklinear_gradient_scaling: "none"
391 reactivation_gradient_scaling: "none"
392 efficient_blocklinear: true
393 populations:
394 - name: "i"
395 polarity: "inhibitory"
396 n_neurons: 32
397 branch_factors: [1]
398 population:
399 ff_inhibitory_synapses: 0
400 - name: "e"
401 polarity: "excitatory"
402 n_neurons: 128
403 branch_factors: [4, 4]
404 connections:
405 - source: "input"
406 target: "i"
407 - source: "input"
408 target: "e"
409 - source: "i"
410 target: "e"
411
412 decoder:
413 type: "MLP"
414 params:
415 input_dim: null
416 hidden_dims: [32, 16]
417 activation: "relu"
418 output_dim: 10
419
420 # ---------- Training ----------
421 training:
422
423 main:
424 strategy: "standard"
425
426 common:
427 epochs: 200
428 batch_size: 512
429 shuffle: true
430 grad_clip_value: 5.0
431 loss_function: "ce"
432 early_stopping: false
433 patience: 500
434 load_best_state_dict: true
435 plot_losses: false
436 suppress_prints: false
437 print_every: 10
438 use_amp: false
439 enable_nan_checking: false
440 synapse_decay_rate: 0.01
441 weight_boosting: true
442
443 # With split_params: true the optimizer trains the sparse synapses,
444 # BlockLinear couplings, reactivation gates and decoder at their group
445 # rates; lr covers only the parameters outside those groups (none in
446 # this model).
447 param_groups:
448 lr: 0.001
449 split_params: true
450 topk_lr: 0.001
451 blocklinear_lr: 0.0001
452 reactivation_lr: 0.0001
453 decoder_lr: 0.001
454
455 regularization:
456 l1_weight: 0.0
457 l2_weight: 0.0
458 enforce_ei_weight_ratio: false
459
460 pruning:
461 enabled: false
462
463 strategies:
464 standard: {}
465
466 # ---------- Analysis ----------
467 # Analysis sections use the analyzer names, as in config_exp.yaml. The
468 # performance analyzer writes performance/final.json, which
469 # dendritic-analyze and the rerun tool read; keep it enabled.
470 analysis:
471 performance:
472 enabled: true
473 training: true
474 params:
475 accuracy: true
476 auc: true
477 categorical_loglikelihood: true
478 mse: false
479 cosine_similarity: false
480
481 information:
482 enabled: false
483 training: false
484
485 correlation:
486 enabled: false
487 training: false
488
489 ablation:
490 enabled: false
491 training: false
492
493 synapse_turnover: # Enabling it requires training: true (it compares snapshots taken during training)
494 enabled: false
495 training: false
496
497 noise_perturbation:
498 enabled: false
499 training: false
500
501 weight:
502 enabled: false
503 training: false
504
505 single_layer_contribution:
506 enabled: false
507 training: false
508
509 synaptic_activation:
510 enabled: false
511 training: false
512
513 branch_activation:
514 enabled: false
515 training: false
516
517 compartment_statistics:
518 enabled: false
519 training: false
520
521 compartment_snr:
522 enabled: false
523 training: false
524
525 multiplicative_gain:
526 enabled: false
527 training: false
528
529 # ---------- Outputs ----------
530 outputs:
531 run_name: "sweep_run" # Prefix of the sweep directory and the Slurm job name
532 results_dir: "outputs" # Base directory of the sweep directory; each run writes to <sweep_dir>/results/config_<index>
533
534 # ---------- Distributed ----------
535 distributed:
536 mode: none # ddp with launcher "torchrun"; fsdp with the FSDP training_script
537
538
539# ============================================================================
540# EXAMPLES -- Common Sweep Configurations
541# ============================================================================
542# Each example replaces the sweep_config and filter_config sections above;
543# keep base_config, experiment_settings and slurm_config. Results count
544# conditions; the generated configs are conditions x seeds_per_condition.
545# Examples 5, 6 and 7 need the base_config of the example config they name.
546#
547# --------------------------------------------------------------------------
548# Example 1: Sparse-synapse learning rate x batch size
549# --------------------------------------------------------------------------
550# sweep_config:
551# training.main.common.param_groups.topk_lr: [0.0001, 0.001, 0.01]
552# training.main.common.batch_size: [128, 256, 512]
553# filter_config:
554# mode: "all"
555# Result: 3 x 3 = 9 conditions
556# (param_groups.lr sets no parameter of this model while split_params is
557# true; set split_params: false to train every parameter at one rate.)
558#
559# --------------------------------------------------------------------------
560# Example 2: Population-network E/I synapse sweep with sum constraint
561# --------------------------------------------------------------------------
562# sweep_config:
563# model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [10, 20, 30, 40]
564# model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20, 30, 40]
565# filter_config:
566# mode: "ei-sum"
567# value: 50
568# tolerance: 0
569# Result: 4 E/I pairs with E + I = 50: (10, 40), (20, 30), (30, 20), (40, 10)
570#
571# --------------------------------------------------------------------------
572# Example 3: Population size and morphology comparison
573# --------------------------------------------------------------------------
574# sweep_config:
575# model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
576# model.core.population_network.layers.0.populations.1.branch_factors:
577# - [3, 3]
578# - [4, 4]
579# filter_config:
580# mode: "all"
581# Result: 2 x 2 = 4 conditions
582#
583# --------------------------------------------------------------------------
584# Example 4: Population-network branch depth sweep
585# --------------------------------------------------------------------------
586# sweep_config:
587# {}
588# filter_config:
589# mode: "branch_sweep"
590# branch_sweep_mode: "depth_sweep"
591# dendrite_tree_depth: {range: {start: 1, stop: 6, step: 1}}
592# uniform_branches_per_layer: 3
593# Result: excitatory branch_factors [3], [3, 3], ..., [3, 3, 3, 3, 3, 3]
594# (6 conditions)
595#
596# --------------------------------------------------------------------------
597# Example 5: Recurrent tau and delay sweep
598# --------------------------------------------------------------------------
599# sweep_config:
600# model.core.population_network.layers.0.dt: [0.5, 1.0]
601# model.core.population_network.layers.0.population_defaults.tau_base: [10.0, 20.0, 40.0]
602# model.core.population_network.layers.0.connections.3.probability: [0.3, 0.6, 0.9]
603# filter_config:
604# mode: "all"
605# Result: 2 x 3 x 3 = 18 conditions. Use the base_config of
606# configs/examples/population_network_taus_and_delays.yaml: a recurrent layer
607# whose fourth connection (connections.3) is the delayed pyr -> pyr
608# connection. tau_base has no effect in a feedforward layer such as the one
609# above, and connections.3 does not exist there.
610#
611# --------------------------------------------------------------------------
612# Example 6: Spiking population sweep
613# --------------------------------------------------------------------------
614# sweep_config:
615# model.core.population_network.layers.0.population_defaults.spike_threshold: [0.25, 0.5]
616# model.core.population_network.layers.0.population_defaults.spike_tau: [12.0, 24.0]
617# filter_config:
618# mode: "all"
619# Result: 2 x 2 = 4 conditions. Use the base_config of
620# configs/examples/population_network_spiking.yaml, whose populations spike
621# (population_defaults.dynamics_mode: "spike").
622#
623# --------------------------------------------------------------------------
624# Example 7: Synapse-type kinetics sweep
625# --------------------------------------------------------------------------
626# sweep_config:
627# model.core.population_network.layers.0.population_defaults.synapse_types.excitatory.1.fraction: [0.1, 0.25, 0.4]
628# model.core.population_network.layers.0.population_defaults.synapse_types.excitatory.1.tau_decay: [40.0, 70.0, 100.0]
629# filter_config:
630# mode: "all"
631# Result: 3 x 3 = 9 conditions. Use the base_config of
632# configs/examples/population_network_synapse_types_ampa_nmda_gaba.yaml, where
633# excitatory.1 is the NMDA type. The fractions of one polarity are normalized
634# to sum to 1, so the NMDA fraction axis also changes the AMPA share.
635#
636# --------------------------------------------------------------------------
637# Example 8: Combined population E/I filter + branch sweep
638# --------------------------------------------------------------------------
639# sweep_config:
640# model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [10, 20, 30, 40]
641# model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20, 30, 40]
642# filter_config:
643# mode: "ei-sum"
644# value: 50
645# tolerance: 0
646# branch_sweep_mode: "depth_sweep"
647# dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
648# uniform_branches_per_layer: 3
649# Result: 4 E/I pairs that sum to 50 x 3 branch depths = 12 conditions
650#
651# --------------------------------------------------------------------------
652# Example 9: Dataset comparison sweep
653# --------------------------------------------------------------------------
654# sweep_config:
655# data.dataset_name: ["mnist", "fashion_mnist"]
656# model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 30]
657# filter_config:
658# mode: "all"
659# Result: 2 datasets x 2 synapse counts = 4 conditions
660#
661# --------------------------------------------------------------------------
662# Example 10: Full mixed population-network sweep
663# --------------------------------------------------------------------------
664# sweep_config:
665# model.core.population_network.layers.0.population_defaults.ff_excitatory_synapses: [20, 40]
666# model.core.population_network.layers.0.population_defaults.ff_inhibitory_synapses: [10, 20]
667# model.core.population_network.layers.0.populations.1.n_neurons: [64, 128]
668# training.main.common.param_groups.topk_lr: [0.001, 0.01]
669# filter_config:
670# mode: "all"
671# branch_sweep_mode: "depth_sweep"
672# dendrite_tree_depth: {range: {start: 2, stop: 4, step: 1}}
673# uniform_branches_per_layer: 3
674# Result: E x I x n_neurons x topk_lr x branch depth = 2 x 2 x 2 x 2 x 3 =
675# 48 conditions
Other templates
configs/templates/analysis_full_template.yamllists the analyzer sections with their options, all disabled. Copy sections into an experiment config, or pass a copy of the file todendritic-analyze-checkpoint --analysis-config(see Analyzer catalogue).configs/templates/indexed_runtime_template.yamlshows the indexed-synapse runtime settings of a structured E/I core and their names in population networks (see Projection backends).Example map lists the focused example configs in
configs/examples/.