Conductance options

These options treat the parameters of a shunting dendritic layer as conductances: a learnable compartment leak, an activation normalized to the voltage range, stochastic synaptic release with cellwise rewiring (indexed_release), a fan-in-scaled log-normal initialization and reduced-precision signal rounding. They are opt-in fields of the standard builders, and their defaults leave a model unchanged. Use them in feedforward rate networks trained with backpropagation; the end of the page describes the shipped context-cued EMNIST examples and the connectivity analysis for these models.

Where the options are set

In a population network the options are fields of model.core.population_network.layers[].population_defaults, and a population’s population block overrides them (Options reference). A unified E/I core takes the same names in its population blocks (Unified E/I core). A structured E/I core sets them in its sections (Structured E/I cores):

Option

Population field

Structured E/I path (under model.core)

Shunting integration

use_shunting: true

type: dendritic_shunting, or einet with morphology.use_shunting: true

Leak

leak_conductance (default 1.0)

morphology.leak_conductance

Learnable leak

learnable_leak (default false)

morphology.learnable_leak

Endpoint-normalized gate

reactivate: true, reactivation_type: endpoint_tanh

reactivation.enabled: true, reactivation.type: endpoint_tanh

Gate start values

reactivation_init_policy: fixed, reactivation_init_m, reactivation_init_b

reactivation.init_policy: fixed, reactivation.init_m, reactivation.init_b

Positive conductance weights

weight_transform: exp or softplus

morphology.weight_transform: exp or softplus

Stochastic release

topk_type: indexed_release

sparsity.type: indexed_release

Release options

release_p_min, release_p_max, release_weight_scale, release_impact_decay, release_impact_metric, release_max_conductance, release_track_source_activity

the same names under sparsity.indexed

Fan-in-scaled log-normal initialization

dbl_init_method: fan_in_lognormal

morphology.dbl_init_method: fan_in_lognormal

Mean child-to-parent coupling

initial_child_conductance (default 1.0)

implementation.initial_child_conductance

Spread of the child couplings

initial_child_conductance_log_sd (default 0.0)

implementation.initial_child_conductance_log_sd

Initial conductance multiplier

initial_conductance_scale (default 1.0)

implementation.initial_conductance_scale

Signal rounding

signal_rounding (default none)

morphology.signal_rounding

Shunting denominator constant

epsilon (default 1e-8)

fixed at 1e-8 in feedforward cores; recurrent_ei.epsilon in recurrent cores

The population defaults of one layer of the shipped example configs/examples/population_network_contextual_remapping.yaml, reduced to these options:

population_defaults:
  use_shunting: true
  weight_transform: exp
  leak_conductance: 0.03
  learnable_leak: true
  reactivate: true
  reactivation_type: endpoint_tanh
  reactivation_init_policy: fixed
  reactivation_init_m: 3.0
  reactivation_init_b: 1.5
  dbl_init_method: fan_in_lognormal
  initial_child_conductance: 0.3
  topk_type: indexed_release
  release_p_min: 0.5
  release_p_max: 1.0

The same options in a structured E/I core:

model:
  encoder:
    type: identity
    params: {input_dim: 784}
  core:
    type: dendritic_shunting
    transfer:
      output_activation: nonnegative
    architecture:
      excitatory_layer_sizes: [64]
      inhibitory_layer_sizes: [16]
      excitatory_branch_factors: [3, 2]
      inhibitory_branch_factors: [3, 2]
    connectivity:
      ee_synapses_per_branch_per_layer: [40]
      ei_synapses_per_branch_per_layer: [40]
      ie_synapses_per_branch_per_layer: [4]
      ii_synapses_per_branch_per_layer: [0]
    morphology:
      somatic_synapses: true
      weight_transform: exp
      leak_conductance: 0.03
      learnable_leak: true
      dbl_init_method: fan_in_lognormal
    sparsity:
      type: indexed_release
      indexed:
        release_p_min: 0.5
        release_p_max: 1.0
    reactivation:
      type: endpoint_tanh
      init_policy: fixed
      init_m: 3.0
      init_b: 1.5
    implementation:
      initial_child_conductance: 0.3
  decoder:
    type: MLP
    params: {hidden_dims: [], output_dim: 10}

Requirements

Each option raises when it is combined with a setting it does not support, instead of being ignored:

  • Shunting integration. A leak other than 1, a learnable leak, endpoint_tanh and signal rounding need use_shunting: true; the leak is a term of the shunting denominator, and the gate expects voltages in [0, 1].

  • Feedforward rate populations. Configurable leak, endpoint_tanh, indexed_release and signal rounding raise in populations with recurrent synapses, a spiking soma, synapse_types, dendritic spikes, soma feedback or a cross_level_mode other than hierarchical.

  • Backpropagation. The analytic local rules (Local credit assignment) reject indexed_release, a configurable leak and endpoint_tanh. endpoint_tanh also rejects reactivation_gradient_scaling: straight_through.

  • Gate initialization. endpoint_tanh and fan_in_lognormal need reactivation_init_policy: fixed (reactivation.init_policy: fixed), which takes the gate’s start values from reactivation_init_m and reactivation_init_b. Data-driven calibration would write a voltage midpoint, which is not the offset of endpoint_tanh.

  • Release synapses. indexed_release needs exp or softplus weights without weight_norm_order, the eager projection backend, persistent indices and no cached transformed weights.

  • Initialization. initial_conductance_scale other than 1 and a positive initial_child_conductance_log_sd need fan_in_lognormal; initial_child_conductance other than 1 needs fan_in_lognormal or mechanism_neutral with a positive weight transform.

  • Signal rounding. signal_rounding other than none needs indexed_release synapses.

  • Unit-leak analyses. The analysis-only branch-rule overrides and the calibration of tangent_matched anchors from a shunting reference raise for a learnable leak or a leak other than 1 (Integration modes).

Learnable leak

A shunting compartment’s voltage is

\[V = \frac{E + C}{g_L + E + I + G + \epsilon}\]

where \(E\) and \(I\) are the excitatory and inhibitory synaptic drives, \(C\) the weighted input from child branches, \(G\) the sum of the child couplings, \(g_L\) the leak and \(\epsilon\) a small constant (Integration modes). leak_conductance sets \(g_L\) (1 by default). With learnable_leak: true every compartment learns its own leak, stored as one log-leak parameter per output of each branch level and starting at leak_conductance. Changing the leak alone moves the operating point of every compartment; Conductance units describes how to rescale all conductances together.

Endpoint-normalized gate

reactivation_type: endpoint_tanh applies

\[f(V) = \frac{\tanh(mV - b) - \tanh(-b)}{\tanh(m - b) - \tanh(-b)}\]

with a trainable slope \(m > 0\) (stored as its logarithm) and offset \(b\) per compartment. It maps \(f(0) = 0\) and \(f(1) = 1\) for any \(m, b\), and its midpoint is the voltage \(b/m\). The function is evaluated through an exponential identity that avoids subtracting saturated tanh values, in float32 for half-precision inputs. reactivation_init_m and reactivation_init_b (reactivation.init_m and init_b) give the starting \(m\) and \(b\); both remain trainable. param_tanh and param_relu are other parametric gates (Activation functions), and reactivate: false removes the gate.

Stochastic release

topk_type: indexed_release (sparsity.type: indexed_release) stores K contacts per output, their source indices and conductances, without dense candidates. Each contact of conductance \(w\) transmits with probability

\[p(w) = p_\mathrm{min} + (p_\mathrm{max} - p_\mathrm{min}) \frac{w}{w + 0.25\, s}\]

where \(s\) is release_weight_scale. In training every contact and example draws an independent Bernoulli release; evaluation transmits the mean \(w\,p(w)\). Outputs are not rescaled by the release probability. The Bernoulli draw is not differentiated; the evaluation path is differentiable through \(p(w)\).

Field

Default

Meaning

release_p_min, release_p_max

0.5, 1.0

Bounds of the release probability; 0 <= p_min <= p_max <= 1.

release_weight_scale

null

\(s\) in final conductance units. null selects \(1/K\); with fan_in_lognormal it selects initial_conductance_scale divided by the compartment’s total contact count.

release_impact_decay

0.99

Decay of the exponential average of each contact’s impact, in [0, 1).

release_impact_metric

drive

Impact statistic used by rewiring: drive (absolute realized drive), active_drive (absolute drive averaged only over examples whose source is nonzero, so rarely active sources such as one-hot cues are not marked low-impact for being rare), weight (the conductance) or saliency (|w * dL/dw| of the last backward pass). weight and saliency are updated after every optimizer step, the others in training forward passes.

release_max_conductance

null

Cap on every effective conductance, including the one used by the release law; capped contacts pass no gradient. It cannot be combined with saliency.

release_track_source_activity

false

Keep a running average of each source’s absolute activity (buffer source_activity), required by regrowth_sources: active.

weight_transform: softplus makes conductance growth linear in the stored parameter at large conductance, where exp grows exponentially. Other sparse synapse types do not read the release fields. With release_p_min: 1 and release_p_max: 1 every contact transmits in training and evaluation, which separates the effect of release from that of the connectivity.

Cellwise impact rewiring

training.main.common.impact_rewiring replaces low-impact indexed_release contacts during training; an empty mapping (the default) disables it. The contacts of one population and sign (excitatory or inhibitory) at all dendritic levels form one group. Every frequency optimizer steps, a contact is eligible when its impact is below threshold times its cell’s mean impact and it is at least maturity_steps optimizer steps old. Each cell replaces at most floor(n_eligible * eligible_fraction) eligible contacts, and at most floor(K * segment_fraction) per dendritic segment, so a cell with fewer than 1 / eligible_fraction eligible contacts replaces none at that step. A segment is the K contacts of one compartment in one bank. A new contact takes a source that its compartment does not use yet and that the connection mask allows; its impact, age and optimizer moments are reset.

Field

Default

Meaning

frequency

100

Optimizer steps between replacement events.

maturity_steps

100

Minimum age, in optimizer steps, of a replaceable contact.

threshold

0.1

Eligibility: impact below this fraction of the cell’s mean impact.

eligible_fraction

0.1

Largest fraction of a cell’s eligible contacts replaced per event.

segment_fraction

0.25

Largest fraction of a segment’s K contacts replaced per event.

freeze_last_epochs

0

Stop replacing contacts for the final epochs; weights keep training.

regrowth_sources

uniform

uniform samples any legal unused source; active only sources whose tracked activity exceeds min_source_activity (requires release_track_source_activity). A compartment without such sources keeps its contacts.

min_source_activity

0.0

Activity threshold of active regrowth.

regrowth_weight

cell_median

Conductance of a new contact: the median of the cell’s conductances of that sign, or initial, a draw from the bank’s recorded fan_in_lognormal law (other initializers raise).

The trainer derives the last replacement step from epochs and freeze_last_epochs and seeds the replacement sampling from the experiment’s topology seed; writing until_step or seed raises. Impact rewiring needs a population network with indexed_release synapses, the standard or presentation_stream strategy and a single process. Checkpoints hold the model state, not the controller’s step count or random generator, so a run started from a checkpoint restarts the replacement schedule (Best state and checkpoints). Code that builds the controller itself (CellwiseImpactRewiring in dendritic_modeling.networks.architectures.excitation_inhibition.synapse.impact_rewiring) can save and restore it with its state_dict and load_state_dict.

Comparing sparse synapse types

The conductance, leak and gate options compose with these sparse synapse types (Sparse synapse types describes all of them):

Type

Stored parameters and selection

Transmission and maintenance

indexed_release

K stored contacts per output with their source indices.

Bernoulli release in training, mean transmission in evaluation; optional cellwise impact rewiring.

indexed

K stored contacts with fixed source indices.

Deterministic; contacts never change source.

indexed_rewire

K stored contacts; each bank periodically replaces its lowest-weight contacts.

Deterministic; replacement by weight, separate from impact rewiring.

standard

A dense candidate matrix; every forward pass selects the K largest transformed weights per output.

Deterministic; only selected weights receive gradients; maintenance by synapse_decay_rate and weight_boosting (Sparse synapse maintenance and rewiring).

variance

Dense candidates; selects the K largest values of transformed weight times an input score.

As standard; the input statistics update in training mode only and are saved in checkpoints.

A Top-K bank stores in_features candidates per output but transmits through K of them, while the indexed types store only their K contacts, so stored and active parameter counts differ for Top-K. Top-K starts from the upper tail of its candidate initialization, so equal fan-in and initializer settings do not give the same initial function as uniformly sampled indexed contacts. stochastic Top-K samples its mask by weight rank; it does not model release.

variance tracks the mean and second moment of every input with an exponential moving average (momentum 0.9). With variance_score: absolute (the default) the input score is the variance, which depends on how each input is scaled: a one-hot context cue of amplitude 1 has a much smaller variance than pixels under a wide gain range, so its contacts are rarely selected. topk_variance_score: relative (sparsity.variance_score) uses the scale-free \(\mathrm{Var}[x] / E[x^2]\) in [0, 1], which is 0 for an input that has been constantly 0; before any statistics exist the selection is by weight alone. Other synapse types reject a non-default score.

Fan-in-scaled initialization

dbl_init_method: fan_in_lognormal draws every synaptic conductance of a compartment from one log-normal law with mean 1 / n and log standard deviation 0.5, where n is the compartment’s total number of contacts over its excitatory, inhibitory and recurrent banks. Excitatory and inhibitory contacts use the same law, so it does not balance their drives. Child couplings start at initial_child_conductance. softplus weights draw the same conductances from the same random stream as exp weights and store their inverse-softplus values. The law is recorded on each bank for regrowth_weight: initial. Gates without a slope and midpoint (tanh, relu, sigmoid) have nothing to initialize; reactivation_init_m and reactivation_init_b apply to gates that have them.

  • initial_conductance_scale multiplies the initialized synaptic and coupling conductances and the automatic release scale. An explicit release_weight_scale is in final conductance units and is not multiplied, and the leak and epsilon stay as written.

  • initial_child_conductance_log_sd draws the child couplings from a mean-preserving log-normal law with mean initial_child_conductance times initial_conductance_scale. Zero keeps constant couplings and draws no random numbers. With an initialization seed the draws use the keyed stream fan_in_lognormal.child_coupling (Random seeds and reproducibility flags).

The defaults (scale 1, spread 0) leave outputs, state-dict keys and random streams unchanged.

Signal rounding

signal_rounding: float16 or bfloat16 emulates reduced-precision arithmetic in a layer. The layer rounds its presynaptic signals, synaptic and coupling conductances, leak, mean-transmission probabilities, summed synaptic drives, voltages and outputs to that format; the terms inside each sum, the parameters and the optimizer state keep float32. Rounding is a cast to the format and back, so a gradient through a rounded value is rounded too, as in half-precision autocast without loss scaling. A conductance above the float16 maximum (65504) becomes infinite; release_max_conductance can bound it. Rounding emulates the precision of a reduced-precision implementation while the computation itself stays in float32; it is not mixed-precision training.

Conductance units

The voltage above is unchanged when every conductance and the constants of its denominator are multiplied by the same positive factor. To compare conductance units, multiply leak_conductance, initial_conductance_scale and epsilon by the same factor and keep the automatic release scale. The initial voltages, release probabilities and gate outputs then stay the same, and so do the gradients with respect to the log-parameterized conductances, up to floating-point roundoff. Training can still diverge later, for example through a learned leak or contact replacement. Feedforward structured cores fix epsilon at 1e-8, which is negligible at these scales.

Context-cued EMNIST examples

Three shipped configurations train on the contextual_remapping dataset: EMNIST Balanced images whose label is shifted by one of num_contexts contexts, with a one-hot context cue appended to the image and per-image gain and noise corruption (Registered datasets). The inputs have 784 + num_contexts features (816 by default). The module dendritic_modeling.datasets.contextual_remapping also provides the batch operations for other code: contextual_remapping_batch (corruption, cue and label shift), brightness_dependent_noise (Gaussian noise with one standard deviation per image, proportional to the square root of its mean gained intensity; only negative values are clipped) and load_emnist_balanced.

Config in configs/examples/

Model

population_network_contextual_remapping.yaml

Two layers, each with one excitatory and one inhibitory population; indexed_release contacts, endpoint_tanh gates, learnable leak, fan_in_lognormal initialization and impact rewiring that stops for the final 20 epochs.

population_network_target_levels.yaml

Two inhibitory populations per layer: p inhibits the excitatory somas and primary branches, t the distal branches (target_levels, Connections); variance Top-K with topk_variance_score: relative and active-weight maintenance.

contextual_remapping_affine_mlp.yaml

A dense MLP baseline (affine_mlp, Baselines and comparison models) with the context cue scaled by 8.

dendritic-train configs/examples/population_network_contextual_remapping.yaml \
  --data_dir <root>

<root>/raw is the torchvision EMNIST directory; set data.dataset_params.contextual_remapping.download: true to download it there. The examples set generator_device: cuda, which draws the fixed validation and test corruptions on a GPU and raises without one; cpu draws them on the CPU, which gives different evaluation sets.

The dataset requires training.main.strategy: presentation_stream (Presentation streams): every epoch samples presentations_per_epoch fresh presentations, evaluates the noisy and the clean validation sets, appends a row to metrics.jsonl and selects the best epoch by noisy-validation accuracy. This strategy rejects data-driven gate initialization and periodic gate recalibration, because the training set’s items are clean context-0 images rather than the sampled presentations; the examples use reactivation_init_policy: fixed.

Connectivity and tuning analysis

analysis.source_tuning_support asks whether the sources that a population’s contacts select are tuned like their target somas (the analyzer and its outputs are described in Analyzer catalogue). It applies to feedforward population networks with indexed_release, indexed, standard or variance synapses, including same-step interneuron pathways and branched morphologies, and runs the model’s forward pass in evaluation mode without changing weights, indices or buffers. Each record reports three weightings of the sources: uniform over the eligible sources, by contact count, and by effective transmission coefficient. For release synapses the coefficient is the mean transmission \(w\,p(w)\), while contact counts count stored contacts; with signal rounding the coefficients use the rounded conductances and probabilities, and the rounding of inputs and outputs remains part of the forward pass. Variance Top-K masks use the training statistics saved in the checkpoint. Rows record selection_policy, transmission_mode and signal_rounding.

configs/examples/conductance_source_tuning_analysis.yaml is an analysis profile for a trained checkpoint of a contextual example:

dendritic-analyze-checkpoint \
  --config <run-dir>/config.json \
  --checkpoint <checkpoint.pt> \
  --analysis-config configs/examples/conductance_source_tuning_analysis.yaml \
  --output-dir <analysis-dir> --device cpu

Preferred classes are selected on reference_split and source responses measured on an independent evaluation_split (require_independent_splits). With a contextual task the labels that define tuning are the shifted outputs by default; other labels must be provided by the analysis dataset and selected with label_batch_index. An input of pixels followed by context cues is not a 28 x 28 image, so the profile leaves image_shape unset and keeps the native source coordinates. Connectivity alone does not show functional tuning or causal importance: report excitatory and inhibitory populations separately, compare independently trained seeds, and use perturbation analyses for causal questions.