Tabular Foundation Models for Data Science Agents

Transcript

[00:00] Large language models have changed how
[00:01] we work with unstructured data like text
[00:04] and code. But structured data, that is
[00:07] data living in tables, is still
[00:09] important [music]
[00:09] for many analysis and prediction
[00:11] problems. Say you’re predicting whether
[00:13] customer would churn, or if you’re
[00:15] forecasting demand, or assessing a
[00:18] patient’s risk. LLMs aren’t built to
[00:20] answer these questions from structured
[00:22] data. I’m thrilled [music] to introduce
[00:24] Tabular Foundation Models for data
[00:26] science agents, built in partnership
[00:28] with Pry Labs, led by [music] Frank
[00:30] Hutter, Taha Chekh, and Elliot
[00:33] Caulfield. Traditional machine learning
[00:35] often means training and tuning a model
[00:38] for every data set.
[00:40] >> [music]
[00:40] >> Tabular Foundation Models work
[00:41] differently. They are pre-trained across
[00:44] millions of different tabular prediction
[00:46] tasks. And when you give them a new data
[00:48] set, they can learn from the examples in
[00:50] that table through in-context learning,
[00:53] >> [music]
[00:53] >> meaning without having to train a new
[00:55] model.
[00:56] >> Thanks, Andrew.
[00:57] You’ll see in detail why LLMs are
[00:59] ill-suited for these problems, and how
[01:01] much simpler the data science workflow
[01:03] gets when you go from traditional
[01:05] machine learning to tabular foundation
[01:07] models. You’ll then use TabPFN yourself
[01:10] for both the classification and a
[01:11] regression task.
[01:12] >> When you use TabPFN in practice, you may
[01:15] [music] need predictions in real time.
[01:17] You learn how a KV cache gets you the
[01:20] sub-second latency you need for that.
[01:22] You may also need to interpret TabPFN’s
[01:25] predictions. For this, you’ll examine
[01:27] which features influence the model
[01:29] globally
[01:30] >> [music]
[01:30] >> and what drove an individual prediction.
[01:32] >> Finally, you’ll connect TabPFN to an AI
[01:35] agent. The LLM can reason about whether
[01:38] the task [music] at hand is a prediction
[01:39] task and hand it over to TabPFN when it
[01:42] is. You’ll see how TabPFN MCP gives
[01:45] [music] the agent access to the model
[01:47] and prediction tools, while a skill
[01:49] gives it guidance for when and how to
[01:50] use them.
[01:51] >> Tabular Foundation Models offer a new
[01:53] way to approach many of the
[01:55] classification, regression, [music]
[01:57] and forecasting tasks that appear
[01:59] throughout science industry.
[02:01] I hope you enjoy the course.