Research Interests

My research interests mainly include topics in Machine Learning, especially Intelligence-Inspired Computing and Time Series Analysis, mainly involving the topics of neural computation, learning theory, data ming, and logical reasoning.

My Curriculum Vitae: Link

Correspondings

• zhangsq{at}lamda{dot}nju{dot}edu{dot}cn or zhangsqhndn{at}gmail{dot}com for research-related matters (paper, code, review, etc.)
• zhangsq{at}nju.edu.cn for teaching-related matters (teaching, admission, hiring, etc.)

• National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China
• School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China

• Nanjing University Suzhou Campus, No. 1520 Taihu Avenue, Huqiu District, Suzhou 215163, China
• 江苏省苏州市虎丘区太湖大道1520号南京大学苏州校区（东校区）, 邮编: 215163

Topic 1: Intelligence-Inspired Computing

• (A) Overview: Biological system colocates the operations with the physical substrate it is processed on. The fundamental computational unit of artificial neural networks is the neuron, corresponding to the cell in biological (nervous) systems. An artificial neuron receives signals from connected neurons, then processes the received signals and generates a signal to other connected neurons. Neurons and edges typically have a weight that adjusts as learning proceeds; the weight increases or decreases the strength of the signal at a connection. Typically, neurons are aggregated into layers, corresponding to the neural circuit. Different layers may perform different transformations on their inputs. Neural operations are the adaptation of the biological system to interact with the environment. The representative operation includes Hebbian rules. Correspondingly, learning algorithms contribute to the neural network to better handle a task by considering sample observations. Learning algorithms involve adjusting the weights (and optional thresholds) of the network to improve the accuracy of the result, which procedure usually is implemented by minimizing the observed errors.
  
  
• (B) Units: Paradigm of neural network learning and Types of artificial neuron models. The neural network learning comprises the neuron model, network architecture, and learning algorithm. Though neural networks have been studied for more than half a century, and various learning algorithms and network architectures have been developed, the modeling of neurons is relatively less considered. The most famous and commonly used formulation of a neuron is the MP neuron model [Mc-Culloch and Pitts, 1943], which formulates the neuron as executing an activation function on the weighted aggregation of signals received from other neurons compared with a threshold. The MP model is very successful though the formulated cell behavior is quite simple. Actual nervous cells are much more complicated, and thus, exploring other bio-plausible formulations with neuronal plasticity is a fundamental and significant problem.
  • (B1) Comprehensive Investigations on Spiking Neural Networks. The spiking neuron model, the computational unit of spiking neural networks (SNNs), takes into account the time of spike firing rather than simply relying on the accumulated signal strength in conventional artificial neural networks, thus offering the potential of temporal and sparse computing. Here, we provide a theoretical framework for investigating the intrinsic structure of spiking neuron models from the perspective of dynamical systems, which exposes the effects of intrinsic structure on approximation power, computational efficiency, and generalization.
  
  
  • (B2) Exploring Time-varying Neuron Models. Recently, we proposed a novel bio-plausible neuron model, the Flexible Transmitter (FT) model. The FT model is inspired by the one-way communication neurotransmitter mechanism in nervous systems and mimics long-term synaptic plasticity. In contrast to the MP neuron model (at macroscopic scale) and the spiking neuron model (at microscopic scale), the FT model builds upon the mesoscopic scale and has the formation of a two-variable two-valued function, thus taking the commonly-used MP neuron model as its special case. Besides, the FT model employs an exclusive variable that leads to a local recurrent system, thus having the potential to handle spatio-temporal data. We empirically show its effectiveness on handling spatio-temporal data and present theoretical understandings of the advantages of the FT model.
  
  
  • (B3) Lightweight Computations of Deep Neural Networks, even Large-Language Models. The expanding scale of deep neural networks usually requires higher computational power and larger memory size, thus causing a hardware threshold for developers and efficient manufacturing. Recently, we focus on the cost-effective computing of deep neural networks and resort to the low-bit quantization. The proposed method eliminates numerous multiplication operations, alleviating the computational consumption over those of full-precision formats. The quantized model maintains highly sparse weight matrices and activation rates, thus significantly reducing memory size and computational complexity. The experiments conducted on deep learning models and large-language models demonstrate the effectiveness of our work, which reduces to at most 8.1‰ inference complexity and 7.8 ‰ memory size while maintaining competitive accuracy.


• (C) Circuits: Neural selection and competition during memory formation. Neural selection promotes the generation of pseudo circuits from primary circuits, leading to pseudo-groundings; the pseudo-groundings are revised via logical abduction based on minimizing the inconsistency with the Biological Sytem (knowledge base); the abduced circuit competes with the pseudo circuit, which is used to update the selection strategy in the next iteration. Notice the selection and competition here can be extended to the AI tasks such as selective integration and incomplete-information games (ethnic competition, gambling, chess and card games, etc.) During this procedure, machine learning takes powerful prediction computing from supervised instances, whereas logical reasoning provides credible support for machine learning, which is related to the Abductive Learning paradigm.


• (D) Systems: Biological perception system with modeling and optimization on hardwares. A high-level overview of how conventional von Neumann processing isolates the various layers, and how in-memory computing aims to converge these, is depicted in Figure (D). Modern computing, based on the von Neumann architecture, optimizes for generality such that learning algorithms are treated somewhat independently of the hardware they are processed on. We focus on high-performance computing relative to neuromorphic computing.
• (D1)Biological Perception System. Here, we focus on the intricate network of sensory organs, neural pathways, and cognitive processes that enable living organisms to interpret and respond to their environment. It integrates multimodal sensory inputs, such as vision, hearing, touch, taste, and smell, through complex neural encoding and processing mechanisms. Governed by principles of adaptation, learning, and feedback regulation, this system allows organisms to recognize patterns, predict outcomes, and make real-time decisions. Its efficiency, robustness, and energy optimization inspire artificial intelligence and neuromorphic computing, bridging neuroscience with technology to develop perception models that mimic human-like cognition, sensory integration, and intelligent decision-making capabilities.


• (D2) Neuromorphic Computing and Acceleration. Low-power biocompatible memristors may enable the construction of artificial neurons that function at voltages of biological action potentials and could be used to directly process bio-sensing signals for neuromorphic computing and/or direct communication with biological neurons. Most artificial neural network accelerators address frequent data movement between the processor and memory by distributing memory arrays across processing units. Recently, we pointed out that the acceleration of neural network learning relies on the discretization of four types of variables: input, weight, neural state, and output. We present early empirical evidence of how artificial neural networks can be discretized to facilitate learning convergence and how this reduces the burden of mixed-signal processing in memristive accelerators. We also aim to overcome several challenges that face the development of memristive accelerators while reducing the adverse impact of limited-precision computation.




Topic 2: Time Series Analysis

My focus on this topic are summarized in the figure below.



Especially, I care about the following issues:
• about Forecasting and Imputation for Industrial and Neurologic Scenarios. I am interested in time series forecasting, including Accurate Forecasting and Quantitative Analysis, is on how to impute and forecast the discretized, unstructured, and non-stationary sequence data.


• about Uncertainty Estimation. Recently, we concern about the uncertainty estimation of generative models and deep neural networks led by stochastic configurations. This investigation involves two steps: uncertainty quantification and optimization, such as using the variance of deep neural kernel for uncertainty estimation and make decision with confidence.
  
  
• about Predictable PAC Learning theory. We also focus on the generalization of machine learning and neural network learning. Generalization is the most fundamental issue in machine learning, which refers to a model's ability to apply what it has learned from the training data to unseen data. A model that generalizes well performs accurately on new, previously unseen examples, which is crucial for its practical applicability.

Journals

• Shao-Qun Zhang, Jia-Yi Chen, Jin-Hui Wu, Gao Zhang, Huan Xiong, Bin Gu, and Zhi-Hua Zhou. On the Intrinsic Structures of Spiking Neural Networks. Journal of Machine Learning Research (JMLR), 25(194):1-74. 2024. [paper] [bib]
• Jin-Hui Wu, Shao-Qun Zhang, Yuan Jiang, and Zhi-Hua Zhou. Theoretical Exploration of Flexible Transmitter Model. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 35(3): 3674-3688. 2024. [paper] [bib]
• Shao-Qun Zhang, Fei Wang, and Feng-Lei Fan. Neural Network Gaussian Processes by Increasing Depth. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 35(2): 2881-2886. 2024. [paper] [code] [bib]
• Gao Zhang and Shao-Qun Zhang^#. Lax Extensions of Conical I-Semifilter Monads. Axioms, 12(11):1034. 2023. [paper] [bib]
• Gao Zhang and Shao-Qun Zhang^#. On Discrete Presheaf Monads. Axioms, 12(6):610. 2023. [paper] [bib]
• 张绍群, 张钊钰, 姜远, 周志华. 基于误差截尾假设的时间序列预测可学习性理论与算法. 计算机学报, 45(11):2279-2289. 2022. [paper] [bib]
  CORR: Shao-Qun Zhang, Zhao-Yu Zhang, Yuan Jiang, and Zhi-Hua Zhou. Time Series Theory and Algorithm of Predictable Learnability Based on Error Truncation Assumption [in Chinese]. Chinese Journal of Computers, 45(11):2279-2289. 2022.
• Shao-Qun Zhang, Wei Gao, and Zhi-Hua Zhou. Towards Understanding Theoretical Advantages of Complex-Reaction Networks. Neural Networks (Neural Netw.), 151:80-93. 2022. [paper] [bib]
• Shao-Qun Zhang, Zhao-Yu Zhang, and Zhi-Hua Zhou. Bifurcation Spiking Neural Network. Journal of Machine Learning Research (JMLR), 22(253):1-21. 2021. [paper] [poster] [code] [bib]
• Shao-Qun Zhang and Zhi-Hua Zhou. Flexible Transmitter Network. Neural Computation (Neural Comput.), 33(11): 2951–2970. 2021. [paper] [poster] [code] [bib]
• 张绍群. 基于紧集子覆盖的流形学习算法. 计算机科学, 44(Z6), 88-91. 2017. [paper] [bib]
  CORR: Shao-Qun Zhang. Manifold learning algorithm based on compact set sub-coverage [in Chinese]. Chinese Journal of Computer Science, 44(Z6):88-91. 2017.

Conferences

• Xiao-Dong Bi, Shao-Qun Zhang^#, and Yuan Jiang. MEPSI: An MDL-based Ensemble Pruning Approach with Structural Information. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI'24), pp. 11078-11086. 2024. [paper] [poster] [code] [bib]
• Jin-Hui Wu, Shao-Qun Zhang, Yuan Jiang, and Zhi-Hua Zhou. Complex-valued Neurons Can Learn More but Slower than Real-valued Neurons via Gradient Descent. In: Advances in Neural Information Processing Systems 36 (NeurIPS'23), pp. 23714-23747. 2023. [paper] [poster] [bib]
• Qin-Cheng Zheng, Shen-Huan Lve, Shao-Qun Zhang, Yuan Jiang, and Zhi-Hua Zhou. On the Consistency Rate of Decision Tree Learning Algorithms. In: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS'23), pp. 7824-7848. 2023. [paper] [poster] [code] [bib]
• Shao-Qun Zhang and Zhi-Hua Zhou. Theoretically Provable Spiking Neural Networks. In: Advances in Neural Information Processing Systems 35 (NeurIPS'22), pp. 19345-19356. 2022. [paper] [poster] [code] [bib]
• Zhao-Yu Zhang, Shao-Qun Zhang, Yuan Jiang, and Zhi-Hua Zhou. LIFE: Learning Individual Features for Multivariate Time Series Prediction with Missing Values. In: Proceedings of the 21st International Conference on Data Mining (ICDM'21), pp. 1511-1516. 2021. [paper] [code] [bib]
• Shao-Qun Zhang and Zhi-Hua Zhou. Harmonic recurrent process for time series forecasting. In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI'20), pp.1714-1721. 2020. [paper] [code] [bib]

Manuscripts

• Qi-Jie Li, Qian Sun, and Shao-Qun Zhang#. Horizon-wise Learning Paradigm Promotes Gene Splicing Identification. 2024. [arXiv:2406.11900] [bib]
• Shao-Qun Zhang, Zong-Yi Chen*, Yong-Ming Tian*, and Xun Lu*. A Unified Kernel for Neural Network Learning. 2024. [arXiv:2403.17467] [bib]
• Shao-Qun Zhang and Zhi-Hua Zhou. ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions. 2021. [arXiv:2111.06222] [bib]
• Gao Zhang, Jin-Hui Wu, and Shao-Qun Zhang^#. On the Approximation and Complexity of Deep Neural Networks to Invariant Functions. 2022. [arXiv:2210.15279] [paper] [bib]
• Shao-Qun Zhang, Jin-Hui Wu, Gao Zhang, Huan Xiong, Bin Gu, and Zhi-Hua Zhou. On the Generalization of Spiking Neural Networks via Minimum Description Length and Structural Stability. 2023. [arXiv:2207.04876] [paper] [bib]

Current

• Probability and Statistics (概率论与数理统计) for Undergraduate Students at Nanjing University (NJU); Fall, 2025.

Previous

• Frontier Theories of Artificial Intelligence (人工智能前沿理论) for Postgraduate Students at Nanjing University (NJU); Fall, 2024.
• Probability and Statistics (概率论与数理统计) for Undergraduate Students at Nanjing University (NJU); Fall, 2024.
• Neural Networks: Theory and Applications (神经网络及其应用) for Postgraduate Students at Nan University (NJU); Spring, 2024.
• Probability and Statistics (概率论与数理统计) for Undergraduate Students at Nan University (NJU); Fall, 2023.
• Time Series Analysis (时间序列分析) for Undergraduate Students at Sichuan University (SCU); Spring, 2017.
• Linear Algebra (线性代数) for Undergraduate Students at Sichuan University (SCU); Full, 2016 and Full, 2017.
• Calculus (微积分) for Undergraduate Students at Sichuan University (SCU); Spring, 2015 - Spring, 2018, per semester.

Teaching Assistants

• Game Theory (博弈论) with Assoc. Prof. Wei Gao; for Graduate and Undergraduate Students at Nanjing University (NJU); Spring, 2020.
• Mathematical Analysis (数学分析) with Assoc. Prof. Hong-Jun Fan; for Undergraduate Students at Nanjing University (NJU); Full, 2019.