Skip to content

Activations (Machine Gnostics MAGNET)

The machinegnostics.magnet module provides activation layers for the MAGNET neural network framework. It includes standard nonlinearities (ReLU, Sigmoid, Tanh, Softmax, Step, ELU, LeakyReLU, Softplus, Swish, Square) and dedicated gnostic activation functions (Fi, Fj, Hi, Hj, Ei) that implement fidelity and information characteristics.


Key Activation Classes

  • ReLU — Rectified linear unit (max(0, x))
  • Sigmoid — Logistic sigmoid (1 / (1 + exp(-x)))
  • Tanh — Hyperbolic tangent (zero-centered, [-1, 1])
  • Softmax — Normalized probability distribution across last axis
  • Step — Hard threshold activation (threshold=0.0)
  • LeakyReLU — Leaky rectified linear (alpha=0.01)
  • ELU — Exponential linear unit (alpha=1.0)
  • Softplus — Smooth ReLU approximation (log(1 + exp(x)))
  • Swish — Self-gated nonlinearity (x * sigmoid(x))
  • Square — Elementwise square (x ** 2)
  • Fi — Gnostic fidelity activation (sech(2θ) with learnable z0, S)
  • Fj — Gnostic infidelity / complement activation
  • Hi — Gnostic information / hyperbolic tangent activation
  • Hj — Gnostic information complement
  • Ei — Gnostic entropy / error activation

Helper Functions

  • get_activation(name_or_obj, verbose=False) — Resolve a string, callable, or layer instance into an activation layer.
  • ActivationFunctions — Container with staticmethod access to Fi, Fj, Hi, Hj, Ei.

Why Magnet Activations?

  • Standard layers provide familiar drop-in replacements for conventional networks.
  • Gnostic activations (Fi, Hi, etc.) learn concept centers (z0) and scales (S) to align with data geometry rather than fixed functions.
  • All implementations preserve autograd through MAGNET's Tensor facade backed by PyTorch.

Author: Nirmal Parmar