Advancing implicit neural representations for efficiency and robustness on hardware

Abstract

Deep neural networks (DNNs) have achieved significant advancements in fields such as computer vision, natural language processing, and signal processing. Unlike conventional DNNs that rely on discrete structures, Implicit Neural Representations (INRs) encode continuous signals through neural network parameters, offering advantages such as memory efficiency, resolution independence, and the ability to model complex continuous functions. However, deploying INRs on resource-constrained hardware platforms presents significant challenges in efficiency and robustness. This thesis addresses these challenges across three critical areas: inference efficiency on Field-Programmable Gate Arrays (FPGAs), robustness on Compute-in-Memory (CIM) systems, and training efficiency on Graphics Processing Units (GPUs).

Inference efficiency in INRs is constrained by their reliance on high-precision weights and computationally expensive activation functions. To address these limitations, this thesis introduces two key techniques: DHQ, which reshapes weight and activation distributions to facilitate effective quantization, and QuadINR, a framework that leverages hardware-efficient quadratic activation functions as alternatives to costly nonlinear operations. These methods, validated on FPGA platforms, enable efficient INR deployment by balancing representation quality and resource constraints. For CIM platforms, which accelerate matrix computations by integrating memory and computation, robustness is a critical challenge due to hardware-induced weight perturbations and variability. To mitigate this, the thesis proposes a gradient-based regularization technique to minimize discrepancies between models with and without weight perturbations, alongside a low-rank constraint to selectively preserve critical singular values. These approaches ensure that INRs maintain high-quality reconstructions despite hardware imperfections. Lastly, the prolonged training time of INRs on GPU platforms, caused by the per-pixel fitting paradigm and the one-signal-one-model paradigm, is addressed through two novel strategies. A frequency-aware incremental training scheme is introduced to accelerate the training process for images, while a meta-learning strategy is developed to encode multiple images into a single model. Together, these contributions advance the efficiency and robustness of INR training and deployment across diverse hardware platforms.

In summary, this thesis provides a comprehensive framework for deploying INRs efficiently and robustly across diverse hardware platforms. By addressing challenges in efficiency and robustness, it enables practical INR applications in real-world scenarios, ranging from resource-constrained edge devices to energy-efficient computing systems.