kempnerforge.config.optimizer

Optimizer configuration.

Classes

OptimizerConfig

Optimizer settings.

class kempnerforge.config.optimizer.OptimizerConfig

Bases: object

Optimizer settings.

name: str = 'adamw'

lr: float = 0.0003

weight_decay: float = 0.1

betas: tuple[float, float] = (0.9, 0.95)

eps: float = 1e-08

fused: bool = True

muon_momentum: float = 0.95

muon_ns_steps: int = 5

muon_adam_lr: float | None = None

schedule_free_warmup_steps: int = 0

__init__(name='adamw', lr=0.0003, weight_decay=0.1, betas=(0.9, 0.95), eps=1e-08, fused=True, muon_momentum=0.95, muon_ns_steps=5, muon_adam_lr=None, schedule_free_warmup_steps=0)

Parameters:

• name (str)

• lr (float)

• weight_decay (float)

• betas (tuple[float, float])

• eps (float)

• fused (bool)

• muon_momentum (float)

• muon_ns_steps (int)

• muon_adam_lr (float | None)

• schedule_free_warmup_steps (int)

Return type:

None