Efficient and accurate neural-field reconstruction using resistive memory

Abstract Applications such as medical imaging, augmented and virtual reality, and embodied artificial intelligence (AI) depend on the ability to reconstruct complex signals from sparse observations. These applications are characterized by incomplete measurements and limited computational resources. Traditional approaches to digital hardware face the following challenges: explicit signal representations require heavy sampling and storage, data movement across the von Neumann bottleneck dominates energy and latency, and CMOS (complementary metal–oxide–semiconductor)-based circuits offer limited parallel efficiency. Here we present a software–hardware co-optimization framework for sparse-input signal reconstruction. At the software level, we use neural fields1 to implicitly represent signals using neural networks, which are further compressed by low-rank decomposition and structured pruning. At the hardware level, we design a resistive-memory-based computing-in-memory platform, featuring a Gaussian encoder and a multi-layer perceptron processing engine. The Gaussian encoder leverages the intrinsic stochasticity of resistive memory for efficient encoding, whereas the processing engine enables precise weight mapping through a hardware-aware quantization circuit. On a 40-nm 256 Kb resistive-memory macro, the system delivers 23.5×, 21.0× and 32.3× gains in projected energy efficiency, together with 10.8×, 38.8× and 6.2× gains in projected parallelism, for three-dimensional computed tomography sparse reconstruction, novel view synthesis and dynamic-scene novel view synthesis, without compromising on reconstruction quality. This work advances AI-driven signal reconstruction technology and paves the way for future efficient and robust medical AI and three-dimensional vision applications. This is a preview of subscription content, access via your institution Access options Access Nature and 54 other Nature Portfolio journals Get Nature+, our best-value online-access subscription £17.99 / 30 days cancel any time Subscribe to this journal Receive 52 print issues and online access £199.00 per year only £3.83 per issue Buy this article - Purchase on SpringerLink - Instant access to the full article PDF. £ 29.95 Prices may be subject to local taxes which are calculated during checkout Data availability The pancreas four-dimensional CT data50, NeRF synthetic dataset4 and D-NeRF dataset53 are publicly available. All other measured data are freely available upon reasonable request. Source data are provided with this paper. Code availability The code that supports the simulation in this paper and other findings of this study is available at GitHub56 (https://github.com/SuperFrankyy/Memristive_Neural_Field). References Sitzmann, V., Martel, J., Bergman, A., Lindell, D. & Wetzstein, G. Implicit neural representations with periodic activation functions. In Proc. 34th International Conference on Neural Information Processing Systems 7462–7473 (ACM, 2020). Liu, R., Sun, Y., Zhu, J., Tian, L. & Kamilov, U. S. Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields. Nat. Mach. Intell. 4, 781–791 (2022). Shen, H. et al. Missing information reconstruction of remote sensing data: a technical review. IEEE Geosci. Remote Sens. Mag. 3, 61–85 (2015). Mildenhall, B. et al. NeRF: representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 99–106 (2021). Bartolozzi, C., Indiveri, G. & Donati, E. Embodied neuromorphic intelligence. Nat. Commun. 13, 1024 (2022). Santos, J. E. et al. Development of the Senseiver for efficient field reconstruction from sparse observations. Nat. Mach. Intell. 5, 1317–1325 (2023). Tononi, G., Edelman, G. M. & Sporns, O. Complexity and coherency: integrating information in the brain. Trends Cogn. Sci. 2, 474–484 (1998). Schafer, R. W. & Rabiner, L. R. Digital representations of speech signals. Proc. IEEE 63, 662–677 (1975). Rabbani, M. & Jones, P. W. Digital Image Compression Techniques Vol. TT7 (SPIE Optical Engineering Press, 1991). Wu, Z. et al. 3D shapenets: a deep representation for volumetric shapes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1912–1920 (IEEE, 2015). Karni, Z. & Gotsman, C. Spectral compression of mesh geometry. In Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques 279–286 (IEEE, 2000). Qi, C. R., Su, H., Mo, K. & Guibas, L. J. Pointnet: deep learning on point sets for 3D classification and segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 652–660 (IEEE, 2017). Lin, L., Liao, X., Jin, H. & Li, P. Computation offloading toward edge computing. Proc. IEEE 107, 1584–1607 (2019). Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In Proc. 4th International Conference on Learning Representations (ICLR, 2016). Horowitz, M. 1.1 computing’s energy problem (and what we can do about it). In Proc. 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) 10–14 (IEEE, 2014). Zidan, M. A., Strachan, J. P. & Lu, W. D. The future of electronics based on memristive systems. Nat. Electron. 1, 22–29 (2018). Wong, H.-S. P. & Salahuddin, S. Memory leads the way to better computing. Nat. Nanotechnol. 10, 191–194 (2015). Chen, Y., Xie, Y., Song, L., Chen, F. & Tang, T. A survey of accelerator architectures for deep neural networks. Engineering 6, 264–274 (2020). Hinton, G. How to represent part-whole hierarchies in a neural network. Neural Comput. 35, 413–452 (2023). Jaderberg, M., Vedaldi, A. & Zisserman, A. Speeding up convolutional neural networks with low rank expansions. In Proc. British Machine Vision Conference (BMVA Press, 2014). Denil, M., Shakibi, B., Dinh, L., Ranzato, M. & de Freitas, N. Predicting parameters in deep learning. In Proc. Advances in Neural Information Processing Systems (NeurIPS, 2013). Fang, G., Ma, X., Song, M., Mi, M. B. & Wang, X. Depgraph: towards any structural pruning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 16091–16101 (IEEE, 2023). Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018). Ambrogio, S. et al. An analog-AI chip for energy-efficient speech recognition and transcription. Nature 620, 768–775 (2023). Wan, W. et al. A compute-in-memory chip based on resistive random-access memory. Nature 608, 504–512 (2022). Wang, Z. et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat. Mater. 16, 101–108 (2017). Zhang, W. et al. Edge learning using a fully integrated neuro-inspired memristor chip. Science 381, 1205–1211 (2023). Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577, 641–646 (2020). Xia, Q. & Yang, J. J. Memristive crossbar arrays for brain-inspired computing. Nat. Mater. 18, 309–323 (2019). Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. & Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol. 15, 529–544 (2020). Song, L., Qian, X., Li, H. & Chen, Y. Pipelayer: a pipelined ReRam-based accelerator for deep learning. In Proc. 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) 541–552 (IEEE, 2017). Ielmini, D. & Wong, H.-S. P. In-memory computing with resistive switching devices. Nat. Electron. 1, 333–343 (2018). Rao, M. et al. Thousands of conductance levels in memristors integrated on CMOS. Nature 615, 823–829 (2023). Yi, S.-in, Kendall, J. D., Williams, R. S. & Kumar, S. Activity-difference training of deep neural networks using memristor crossbars. Nat. Electron. 6, 45–51 (2023). Kumar, S., Wang, X., Strachan, J. P., Yang, Y. & Lu, W. D. Dynamical memristors for higher-complexity neuromorphic computing. Nat. Rev. Mater. 7, 575–591 (2022). Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015). Sun, Z., Pedretti, G., Bricalli, A. & Ielmini, D. One-step regression and classification with cross-point resistive memory arrays. Sci. Adv. 6, eaay2378 (2020). Yuan, R. et al. A neuromorphic physiological signal processing system based on VO2 memristor for next-generation human-machine interface. Nat. Commun. 14, 3695 (2023). Cai, F. et al. Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks. Nat. Electron. 3, 409–418 (2020). Wang, S. et al. Echo state graph neural networks with analogue random resistive memory arrays. Nat. Mach. Intell. 5, 104–113 (2023). Yang, Y. et al. Observation of conducting filament growth in nanoscale resistive memories. Nat. Commun. 3, 732 (2012). Banner, R., Nahshan, Y. & Soudry, D. Post training 4-bit quantization of convolutional networks for rapid-deployment. In Proc. Advances in Neural Information Processing Systems 7950–7958 (NeurIPS, 2019). Jacob, B. et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2704–2713 (IEEE, 2018). Jacot, A., Gabriel, F. & Hongler, C. Neural tangent kernel: convergence and generalization in neural networks. In Proc. Advances in Neural Information Processing Systems 8580–8589 (NeurIPS, 2018). Vaswani, A. et al. Attention is all you need. In Proc. Advances in Neural Information Processing Systems 6000–6010 (NeurIPS, 2017). Tancik, M. et al. Fourier features let networks learn high frequency functions in low dimensional domains. In Proc. Advances in Neural Information Processing Systems 7537–7547 (NeurIPS, 2020). Volder, J. E. The CORDIC trigonometric computing technique. IRE Trans. Electron. Comput. EC-8, 330–334 (1959). Chen, G.-H., Tang, J. & Leng, S. Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. Med. Phys. 35, 660–663 (2008). Sidky, E. Y., Kao, C.-M. & Pan, X. Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT. J. Xray Sci. Technol. 14, 119–139 (2006). Shen, L., Pauly, J. & Xing, L. NeRP: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction. In Proc. IEEE Transactions on Neural Networks and Learning Systems 770–782 (IEEE, 2022). Eslami, S. M. A. et al. Neural scene representation and rendering. Science 360, 1204–1210 (2018). Kerbl, B., Kopanas, G., Leimkühler, T. & Drettakis, G. 3D Gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. 42, 139 (2023). Pumarola, A., Corona, E., Pons-Moll, G. & Moreno-Noguer, F. D-NeRF: neural radiance fields for dynamic scenes. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10318–10327 (IEEE, 2021). Horé, A. & Ziou, D. Image quality metrics: PSNR vs. SSIM. In Proc. 2010 20th International Conference on Pattern Recognition 2366–2369 (IEEE, 2010). Zhang, R., Isola, P., Efros, A. A., Shechtman, E. & Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 586–595 (IEEE, 2018). Yu, Y. Memristive neural field. GitHub https://github.com/SuperFrankyy/Memristive_Neural_Field (2026). Funding M.L., Q.L., D.S., and Xumeng Zhang disclose support for the research of this work from the National Natural Science Foundation of China (grant nos. 62488101, T2293732, 62374181 and 62374040, respectively). D.S. discloses support for the research of this work from the Joint Laboratory of Microelectronics (JLFS/E-601/24). X.Q. discloses support for the research of this work from the Hong Kong Research Grants Council General Research Fund (grant nos. 17202422, 17212923 and 17215025). Z.W. discloses support for the research of this work from ACCESS (AI Chip Center for Emerging Smart Systems), supported by the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government. Author information Authors and Affiliations Contributions Z.W. and Y.Y. conceived the work. Y.Y. contributed to the model, software and software–hardware co-design. Y.Y. and Xinyuan Zhang contributed to the hardware development. Xinyuan Zhang, Woyu Zhang, Wenkui Zhang and J.C. conducted the hardware experiments. Y.Y., Z.W., Xinyuan Zhang, Shaocong Wang, Woyu Zhang, Xiuzhe Wu, Xumeng Zhang, X.Q., D.S. and Q.L. interpreted, analysed and presented the experimental results. Y.Y. and Z.W. wrote the paper. Y.H., J.Y., Y.Z., N.L., B.W., X.C., Songqi Wang, Xiaoshan Wu, S.H., Y.L., M.X., H.C., K.-T.C. and M.L. discussed the results and implications. Corresponding authors Ethics declarations Competing interests The authors declare no competing interests. Peer review Peer review information Nature thanks Guangming Wang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Additional information Extended data figures and tables Extended Data Fig. 1 Proposed structured pruning approach. a,b, Comparison of unstructured pruning and our structured pruning. c, Our progressive structured pruning approach during training. d, Weight evolution of a single linear layer throughout training in the novel view synthesis task. Extended Data Fig. 2 Comparison between HAQ and standard write-verify. a, The flowchart of the HAQ method. b, The flowchart of the conventional write-verify method. c, Decrease in mean error and standard deviation of weights as quantization bits increase. d, Gradual convergence of the conventional write-verify method. e, Programming cycles per cell. f, Total programming cycles across cells per weight. Extended Data Fig. 3 System architecture and block circuit details. a, Overview of the system. b, Resistive memory macro fabricated using 40 nm CMOS technology. c, Analogue readout macro fabricated using 180 nm CMOS technology. d, Digital core including volume rendering, activation, accumulation, and buffer. Extended Data Fig. 4 Unified hardware-aware optimization framework for resistive memory-based in-memory computing systems. The framework employs hardware-aware population-based training for software optimization and evolutionary algorithms with hardware simulation for hardware parameter optimization. Supplementary information Rights and permissions About this article Cite this article Yu, Y., Zhang, X., Wang, S. et al. Efficient and accurate neural-field reconstruction using resistive memory. Nature (2026). https://doi.org/10.1038/s41586-026-10646-w Received: Accepted: Published: Version of record: DOI: https://doi.org/10.1038/s41586-026-10646-w