Optimizer — Abstract Base Optimizer Class¶
The foundational abstract base class for all MAGNET optimization algorithms.
Overview¶
Optimizer is an abstract base class (ABC) that defines the interface for all optimization strategies in MAGNET. It provides common utilities for parameter updates, state management, and learning rate control, with specialized handling for gnostic parameters (s, z0, *_s, *_z0).
Key Features¶
- Abstract Interface: Enforces consistent
step(param, grad)signature across all optimizers - State Management:
state_dict()andload_state_dict()for serialization - Learning Rate Control:
get_lr()andset_lr()for dynamic adjustment - Gnostic Parameter Support: Specialized scaling for
s,z0,*_s,*_z0parameters
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
lr |
float |
0.001 |
Base learning rate for parameter updates |
Methods¶
step(param, grad, **kwargs) (abstract)¶
Update parameter param using gradient grad. Must be implemented by all subclasses. For MAGNET parameters (s, z0, *_s, *_z0), applies specialized scaling.
state_dict()¶
Return optimizer state (momentum vectors, step counts, etc.) for serialization.
load_state_dict(state)¶
Restore optimizer state from a saved dictionary.
zero_grad()¶
Zero out gradients for all parameters being optimized.
get_lr()¶
Return current learning rate(s) for the optimizer.
set_lr(lr)¶
Update the learning rate(s) for the optimizer.
Example Usage¶
from machinegnostics.magnet import Optimizer
class MyOptimizer(Optimizer):
def __init__(self, lr=0.01):
super().__init__(lr=lr)
def step(self, param, grad, **kwargs):
# Simple gradient descent
param -= self.lr * grad
return param
opt = MyOptimizer(lr=0.01)
Author: Nirmal Parmar