Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks

Why on-sensor deployment is hard

Spiking neural networks are appealing for event-camera pipelines because the computation is sparse, binary, and naturally aligned with low-power neuromorphic hardware. The catch is that training usually happens with a smooth surrogate nonlinearity, while deployment has to use a hard threshold. That mismatch can look small in the loss function and still be painful at inference time, especially when many membrane potentials sit close to threshold.

This paper reframes that gap as an on-sensor systems problem rather than just a learning problem. The deployment target is not only a hard-spike model, but one that also tolerates quantized weights, fixed-point membrane potentials, discrete leak factors, and tight energy budgets. The question is whether the same flat-minima intuition from SAST survives those constraints.

What this paper adds

The starting point is the original SAST formulation: train a surrogate-forward SNN so the objective is genuinely smooth, then apply sharpness-aware minimization to prefer flatter regions of that surrogate loss surface. At deployment, only the spike nonlinearity is swapped from the surrogate to the hard Heaviside step; weights, thresholds, leak, and reset behavior stay fixed.

Relative to the earlier preprint, this version makes the on-sensor story much more explicit. It adds hardware-aware inference simulations, reports SynOps as an energy proxy, includes corruption and compute-matched controls, and extends the analysis with state stability, input-Lipschitz, smoothness, and nonconvex convergence results under explicit contraction assumptions.

Results

The main pattern is simple: SAST keeps surrogate-forward accuracy high while making the eventual hard-spike model much less brittle. That shows up both in the direct surrogate to hard swap and again when the evaluation stack is pushed closer to realistic hardware.

Swap-only transfer

In the cleanest test, the paper just replaces the surrogate activation with hard spikes and leaves everything else unchanged. Under that setup, the transfer gap shrinks sharply on both benchmarks at the best tested perturbation radius.

The paper also reports compute-matched controls. Even when the baseline gets more epochs to compensate for SAM's extra cost, it still does not close the gap: on N-MNIST the compute-matched baseline reaches 65.7% hard-spike accuracy versus 93.9% for SAST, and on DVS Gesture it reaches 28.0% versus 57.8%.

Hardware-aware inference

The stronger result is that the benefit survives a more deployment-like evaluation with quantized weights, fixed-point membrane state, and discrete leak. Under the tested INT8 and INT4 profiles, SAST still wins by a large margin and often reduces SynOps at the same time.

The paper also reports better robustness under random event-drop corruption and notes the expected training-time cost: roughly 2.1x wall-clock on N-MNIST and 1.8x on DVS Gesture, with no increase in peak memory when the two SAM minibatches are loaded sequentially.

Why it matters

A lot of SNN work can look good in surrogate mode and disappoint once the model is actually forced into binary hardware behavior. This paper is useful because it evaluates the method at that boundary directly. The headline is not just that flat minima help generalization in the abstract; it is that they seem to push membrane dynamics away from ambiguous threshold regions, making the eventual hard-spike implementation more reliable.

Just as importantly, the gain is not purely an accuracy story. The reduction in SynOps on some hardware-aware settings suggests that the same training signal can also produce sparser, cheaper spike activity, which is exactly the kind of tradeoff on-sensor systems care about.

Limits

The paper is careful not to oversell the result. The experiments are on two event-camera benchmarks and small SNN architectures, and the author frames SAST as one promising gap-reduction strategy under the tested settings rather than a universal answer for all neuromorphic deployments.

That is the right reading. The evidence here is strong for the specific swap-only and hardware-aware settings that were evaluated, but broader claims would still need more architectures, more sensors, and direct measurement on real hardware.

References