ReLU — Rectified Linear Unit Activation¶
The ReLU activation applies the rectified linear unit (max(0, x)) elementwise.
Overview¶
ReLU is the default hidden-layer activation in many feed-forward networks because it is simple, inexpensive, and avoids heavy saturation behavior. It returns zero for negative inputs and preserves positive values unchanged.
Key Features¶
- Simple, fast elementwise operation
- No saturation for positive inputs (gradients flow freely)
- Compatible with MAGNET
Tensorfacade - Standard drop-in replacement for conventional networks
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
name |
str | None |
None |
Optional layer name |
verbose |
bool |
False |
Enable debug logging |
Methods¶
forward(x, training=True)¶
Return max(0, x) elementwise.
- x:
Tensoror array-like — Input values of any shape - Returns:
Tensorwith same shape asx
Example Usage¶
import numpy as np
from machinegnostics.magnet import ReLU
activation = ReLU()
result = activation(np.array([-2.0, 0.0, 3.0]))
print(result.data.tolist()) # [0.0, 0.0, 3.0]
Author: Nirmal Parmar