Research overview
From useful tables to reusable intelligence.
We study how to learn from heterogeneous tables: build reliable foundations, predict new outcomes, discover causal structure, and turn complex requests into executable data workflows.
Historical examples
| Week | Season | Promotion | Demand |
|---|---|---|---|
| 01 | Winter | No | 84 |
| 02 | Winter | Yes | 112 |
| 03 | Spring | No | 96 |
| Next | Spring | Yes | ? |
Different tables bring different schemas, meanings, and targets.
Demand in week 02.
Look up an observed value.
Expected demand next week.
Learn from the labeled context.
Infer a graph under causal assumptions.
Association alone is insufficient.
A pretrained predictor uses labeled rows as context to estimate an outcome for a new row.
The table and graph are teaching examples. No model is run here; causal direction requires additional assumptions or evidence.
Build the foundations. Predict outcomes. Discover structure. Understand and control the table.
Map the field. Test the ideas. Make them usable.
Surveys organize the questions; benchmarks establish the evidence; baselines test what is necessary; toolboxes make the work reproducible. Together, these resources provide a starting point for learning with tabular data.
Survey · IEEE TPAMI 2026
A map of the field
Organize tabular representation learning by how models generalize: learning within one dataset, transferring through adaptation, and applying pretrained models directly to new tables.
Choose a learning setting before comparing methods.
Representation Learning for Tabular Data: A Comprehensive Survey
Survey · Frontiers of Computer Science 2026
A view across tables, time series, and workflows
A unified view of input encoding, architectures, pretraining, and adaptation for tabular and time-series foundation models. The survey connects reusable prediction with agents that carry out complete data workflows.
Follow the shift from predicting a target to executing a workflow.
From Universal Prediction to Universal Workflows: A Survey of Foundation Models for Tabular and Time Series Data
Benchmark · arXiv 2024, revised 2025
Establish evidence across diverse datasets
A comparison across more than 300 datasets tests which conclusions survive changes in scale, feature types, and prediction targets. TALENT-tiny provides a 45-dataset core for faster evaluation, with extensions for harder settings.
Broad evaluation helps distinguish robust gains from dataset-specific wins.
A Closer Look at Deep Learning Methods on Tabular Datasets
Toolbox · JMLR 2025
Make experiments reproducible and extensible
A common interface for training, analyzing, and comparing tabular models, with reusable feature encoding and normalization. TALENT brings benchmarks and strong baselines into experiments that others can run and extend.
Start from a shared implementation and evaluation pipeline.
TALENT: A Tabular Analytics and Learning Toolbox
ModernNCA · the core idea
A few neighbors are shown for clarity. Positions illustrate the idea, rather than a measured embedding.
Observed features
Learned representation
Prediction from neighbors
A learned representation makes task-relevant neighborhoods useful for a new query.
ICLR 2025
Learn a representation with useful neighbors
ModernNCA revisits differentiable nearest-neighbor learning with deep representations and stochastic training. It learns a task-relevant distance so that neighboring examples provide useful evidence for classification and regression.
Reported result Across the paper's 300-dataset evaluation, this simple baseline is competitive with CatBoost on classification and regression.
A carefully designed baseline helps separate useful modeling choices from unnecessary complexity.
Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later
These resources establish the evidence. The next question is what a pretrained model can reuse on a new table.
Learn across tables. Predict on a new one.
A predictive tabular foundation model (TFM) learns a reusable prediction procedure across many tables. For a new task, labeled rows provide context for predicting a class or numerical target on unseen rows. We first explain this mechanism, then analyze TabPFN v2 and develop efficient, expressive predictors with TabSwift and Retro.
In-context prediction · illustrated with TabSwift
A tabular foundation model learns across many prediction tasks before deployment. On a new table, labeled rows provide context: the model uses those examples to predict the target of a new row, without fitting its parameters again.
| Season | Promo | Demand |
|---|---|---|
| Winter | No | 84 |
| Winter | Yes | 112 |
| Spring | No | 96 |
| Spring | Yes | ? |
| Temp. | Vibration | Quality |
|---|---|---|
| 68 °C | Low | Pass |
| 82 °C | High | Fail |
| 73 °C | Low | Pass |
| 79 °C | Medium | ? |
The target is known in context rows and hidden in the new row.
TabSwift backbone
θParameters stay fixedThe context changes with the table.Regression
Regression head
TabSwift reuses its pretrained backbone with a regression head to predict the new row's demand from this table's context.
Illustrative data, not model outputs. This shared-backbone setup is specific to TabSwift. Other TFMs can use different checkpoints for different task types; Retro trains classification and regression models separately.
Analysis · NeurIPS 2025
Understand the predictor, then extend its use
Our analysis examines how TabPFN v2 handles heterogeneous columns and builds predictive representations. It shows how to extract useful features and use a divide-and-conquer strategy for high-dimensional, many-class, and larger datasets without retraining the pretrained model.
Understanding an existing foundation model can reveal new ways to use it.
A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities
Two design questions
Query-adaptive inference depth
A predicted reliability score reaches the exit threshold; this illustrative query stops early.
Retrospective information reuse
Earlier contributions remain available
Later stages read the initial encoding and saved groups of residual updates.
TabSwift adjusts how much computation a query uses. Retro changes which intermediate information remains available.
Layer counts and exit positions are schematic. TabSwift estimates reliability; early exit does not guarantee a correct prediction. Retro reuses saved contributions in forward computation, without rerunning earlier layers.
ICML 2026 · Spotlight
Make in-context prediction economical
TabSwift shares a compact row-wise attention backbone across classification and regression, with task-specific output heads. Gated attention and register tokens support efficient inference; adaptive early exit lets query rows use different amounts of depth.
Reported result On the paper's TALENT evaluation of 188 classification and 100 regression datasets, TabSwift achieves predictive performance competitive with TabPFN v2 and TabICL while reducing inference cost under the reported configurations.
Reuse a pretrained predictor across tables, while matching computation to each query.
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
arXiv 2026
Revisit useful intermediate information
Later stages explicitly reuse the initial encoding and saved groups of residual updates. Attention Residuals select earlier contributions, while query-conditioned gating shapes each new contextual update.
Reported result In the paper's evaluated candidate pools, Retro ranks among the top three by overall Elo on TabArena, TALENT, and RelArena, and lies on the predictive-performance–runtime Pareto frontier.
Depth can support complementary refinements when useful earlier information remains accessible.
Thinking in Depth: Retrospective Inference for Tabular Foundation Models
Predicting a target is one task. Discovering the relationships among all variables calls for another kind of foundation model.
From a table of observations to candidate causal structure.
Causal discovery asks which variables directly influence others. A causal discovery foundation model (CDFM) learns a reusable dataset-to-graph mapping across environments. TabCausal studies this modeling problem; CausalArena tests how well discovery methods recover structure across different settings.
TabCausal · from a table to a graph
| Temp. | Pressure | Yield |
|---|---|---|
| 68 °C | 1.8 bar | 74% |
| 75 °C | 2.2 bar | 81% |
| 82 °C | 2.5 bar | 86% |
| Season | Promo | Demand |
|---|---|---|
| Winter | No | 84 |
| Winter | Yes | 112 |
| Spring | No | 96 |
Observations, plus intervention indicators when available.
One forward pass
θSame TabCausalNo per-dataset parameter fittingDirected-edge predictions
The manufacturing table maps to a process graph. Switch to retail: the variables and graph change, while the pretrained parameters remain fixed.
Tables and graphs are invented teaching examples, not measured predictions or causal claims about real systems. Graph identification depends on causal assumptions and available evidence. CausalArena separately evaluates transfer across different benchmark families.
NeurIPS 2026
Learn reusable dataset-to-graph inference
TabCausal pretrains across varied graphs, mechanisms, noise distributions, dimensions, sample sizes, and intervention regimes. Given a new dataset, it predicts directed-edge probabilities in a forward pass through shared pretrained parameters.
Different datasets can produce different graphs through the same pretrained parameters.
TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery
CausalArena · four evaluation families
Controlled graph structures, mechanisms, and noise.
Executable models grounded in operational scenarios.
Scientific equations and interpretable mechanisms.
Public datasets with published reference graphs.
TabCausal pretrains on generated causal environments. CausalArena evaluates discovery methods across these four families.
arXiv 2026
Reveal where causal discovery succeeds and fails
CausalArena evaluates recovery of causal graphs under a common protocol. It combines controlled synthetic environments, auditable operational scenarios, scientific formula-based mechanisms, and public real-world checks. Method rankings can change substantially across environments.
A model's ranking in one benchmark family does not establish that it generalizes everywhere.
CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
Before prediction or discovery can help, information must become a usable table and the user's request must become a well-defined task.
From raw information and user intent to executed analysis.
Tabularization gives information an explicit structure: entities become rows, attributes become columns, and values retain their meaning and source. We connect this data preparation with language, visual table understanding, and tool use so that a complex request can become a task we can execute and check.
Tabularization · from evidence to a learning problem
Recover information from different sources, align it around the same entity and time period, then specify what is known and what is to be predicted.
Tea promotion: W02 and W04.
No promotion in W01 or W03.
Store A and tea stay fixed here.
| Weekkey | SeasonI | PromoD | Salesunits P |
|---|---|---|---|
| W01 | Winter | No | 84 |
| W02 | Winter | Yes | 112 |
| W03 | Spring | No | 96 |
| W04 | Spring | Yes | ? |
Keep the source with the value. P, D, and I identify where each column came from. Sales use one unit; unknown stays unknown.
Before W04 begins: estimate tea sales for Store A in W04.
Season, promotion, and observed sales.
84 · 112 · 96 unitsSeason: Spring
Promotion: Yes
A number of units.
Not yet observed.
The table defines the learning problem: observed weeks are context, W04 is the query, and its sales are the unknown target.
All records and sources are fictional teaching examples. The calendar and promotion plan are assumed to be available before W04; no future sales enter the context. Tabularization preserves evidence and task boundaries before a predictor is applied.
ICML 2026
From language to a predictive table task
TopBench tests whether a model recognizes predictive intent in a natural-language question and infers unobserved answers from historical tables. Its tasks cover single-point prediction, decision making, treatment-effect analysis, and ranking or filtering, with answers that can include structured tables.
Understanding a request means identifying the computation it needs, beyond extracting or looking up a table.
TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering
ICML 2026
From document images to usable structure
OvisOCR converts full-page document images into structured Markdown, preserving text, tables, formulas, and reading order. It provides a perception step for making visually embedded information available to downstream analysis.
Recover structured content before asking a learning system to reason over it.
OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning
NeurIPS 2025
Understand a table through its visual form
Tabular reasoning combines cell lookup with comparison, aggregation, and calculation. TURBO uses clean structured tables during training to teach multimodal models how to reason; at inference, the model answers from a table image and a question.
Use structured supervision to improve reasoning when the table is available as an image.
Multimodal Tabular Reasoning with Privileged Structured Information
TabAgent · a workshop exploration
“Use our sales and product records to build a table, forecast next month's demand, and explain the results in a planning report.”
Execution records connect the answer to the data and tools used.
Deliver an answer together with tables, assumptions, and a report that the user can inspect.
Conceptual workflow inspired by the TabAgent workshop study. The example explains how data construction, task understanding, tool execution, and result validation connect; no live agent is run here.
ICML 2026 · FMSD Workshop
Connect table construction, task understanding, and tool execution
This workshop study explores an agent that coordinates data access, table construction, processing, modeling, and visualization. Task understanding guides tool selection; inspection of intermediate results connects execution with the final answer.
Useful table intelligence spans both the right computation and the workflow that carries it out.
Agentic Data Intelligence for General Tabular Modeling
Together, these steps motivate a broader question: can reusable models understand and predict the data environment itself?
Future work · broader environments and prediction tasks
We want to extend reusable learning from individual tables to richer data environments. Two directions connect our work on prediction, causal discovery, and executable workflows. These are research goals for future work.
Future work
Learn how a data environment evolves and how its observations may change under actions or interventions. Connect predictive modeling and causal structure with simulation, planning, and feedback.
How can a model anticipate the consequences of a decision in a data environment?
Future work
Extend reusable prediction across tabular data, time series, relational structures, and multimodal evidence. Study how a shared prediction capability can adapt to new schemas, tasks, and domains.
What can transfer when the format, context, and prediction task all change?
The next step is to connect these capabilities and test where their generalization holds.
Continue with the work
Papers, implementations, models, and benchmarks for the work introduced above.
This content was generated with a large language model and reviewed by a human.