kempnerforge.config.scheduler

LR scheduler configuration.

Classes

SchedulerConfig	Learning rate schedule settings.
SchedulerType

class kempnerforge.config.scheduler.SchedulerType

Bases: StrEnum

cosine = 'cosine'

linear = 'linear'

wsd = 'wsd'

constant = 'constant'

rex = 'rex'

none = 'none'

__new__(value)

class kempnerforge.config.scheduler.SchedulerConfig

Bases: object

Learning rate schedule settings.

name: SchedulerType = 'cosine'

warmup_steps: int = 2000

decay_steps: int | None = None

min_lr_ratio: float = 0.1

stable_steps: int | None = None

wsd_decay_type: str = 'cosine'

rex_alpha: float = 1.0

__init__(name=SchedulerType.cosine, warmup_steps=2000, decay_steps=None, min_lr_ratio=0.1, stable_steps=None, wsd_decay_type='cosine', rex_alpha=1.0)

Parameters:

• name (SchedulerType)

• warmup_steps (int)

• decay_steps (int | None)

• min_lr_ratio (float)

• stable_steps (int | None)

• wsd_decay_type (str)

• rex_alpha (float)

Return type:

None