Jun-Peng Jiang @ LAMDA, NJU-AI

蒋俊鹏
Jun-Peng Jiang
Ph.D. candidate
LAMDA Group
School of Artificial Intelligence
Nanjing University, Nanjing 210023, China

Email: jiangjp [at] lamda.nju.edu.cn

• Jun-Peng Jiang, Yu Xia, Hai-Long Sun, Shiyin Lu, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye. Multimodal Tabular Reasoning with Privileged Structured Information. [Paper] [Code]

Tabular reasoning involves multi-step information extraction and logical inference over tabular data. While recent advances have leveraged large language models (LLMs) for reasoning over structured tables, such high-quality textual representations are often unavailable in real-world settings, where tables typically appear as images. In this paper, we tackle the task of tabular reasoning from table images, leveraging privileged structured information available during training to enhance multimodal large language models (MLLMs). Experimental results demonstrate that, with limited (9k) data, TURBO achieves state-of-the-art performance (+7.2% vs. previous SOTA) across multiple datasets.

• Jun-Peng Jiang, Si-Yang Liu, Hao-Run Cai, Qile Zhou, Han-Jia Ye. Representation Learning for Tabular Data: A Comprehensive Survey. [Paper] [Code]

Tabular data, structured as rows and columns, is among the most prevalent data types in machine learning classification and regression applications. Models for learning from tabular data have continuously evolved, with Deep Neural Networks (DNNs) recently demonstrating promising results through their capability of representation learning. In this survey, we systematically introduce the field of tabular representation learning, covering the background, challenges, and benchmarks, along with the pros and cons of using DNNs. We organize existing methods into three main categories according to their generalization capabilities: specialized, transferable, and general models.

• Jun-Peng Jiang, Si-Yang Liu, De-Chuan Zhan, Han-Jia Ye. Unlocking the Power of Text in Deep Tabular Data Learning. [Paper] [Code]

We propose a simple yet effective approach called Tabular TExt Contrastive fusiOn (TABTECO), which aims at maximizing the mutual information within two modalities in a joint space, unlocking the power of text and fusing the discriminative information from two modalities. TABTECO employs contrastive learning to align table and text modalities within a shared latent space, maximizing their mutual information. Additionally, it incorporates the Information Bottleneck principle to strengthen the relationship between this shared space and the label space, enhancing the model’s ability to focus on taskrelevant features.

• Jun-Peng Jiang, Tao Zhou, De-Chuan Zhan, Han-Jia Ye. Compositional Condition Question Answering in Tabular Understanding. In Proceedings of the 42nd International Conference on Machine Learning (ICML-2025), Vienna, Austria, 2024. [Paper] [Code]

We introduce a new Compositional Condition Tabular Understanding method, called CoCoTab. Specifically, to capture the structural relationships within tables, we enhance the visual encoder with additional row and column patches. Moreover, we introduce the conditional tokens between the visual patches and query embeddings, ensuring the model focuses on relevant parts of the table according to the conditions specified in the query. Additionally, we also introduce the Massive Multimodal Tabular Understanding (MMTU) benchmark, which comprehensively assesses the full capabilities of MLLMs in tabular understanding.

• Jun-Peng Jiang, Han-Jia Ye, Leye Wang, Yang Yang, Yuan Jiang, De-Chuan Zhan. Tabular Insights, Visual Impacts: Transferring Expertise from Tables to Images. In Proceedings of the 41st International Conference on Machine Learning (ICML-2024 Spotlight), Vienna, Austria, 2024. [Paper] [Code]

We propose CHannel tAbulaR alignment with optiMal tranSport(CHARMS), which establishes an alignment between image channels and tabular attributes, enabling selective knowledge transfer that is pertinent to visual features. By maximizing the mutual information between image channels and tabular features, knowledge from both numerical and categorical tabular attributes are extracted. Experimental results demonstrate that CHARMS not only enhances the performance of image classifiers but also improves their interpretability by effectively utilizing tabular knowledge.