Our Cayley-Encoder for improving LLMs' final representations was accepted to NeurIPS 2026, together with 3 new workshop papers! 🎉

🎥 Main track: Improving LLM Final Representations with Inter-Layer Geometry.
Most LLM-based predictors use only the final layer, but the ry useful, complementary signal. We introduce the Cayley-Encoder, a lightweight GNN that treats the LLM's layers as nodes in a Cayley graph over SL(2, ℤₙ), vand produces improved final representation for the LLM.
Across 13 tasks and 9 LLMs, the Cayley-Encoder:
📈 improves accuracy by up to 40 points over the last layer, adding at most 0.1% extra parameters,
🔬 lets a 14M-parameter LLM beat the last layer of a 200× larger one,
🎯 beats full-data baselines with just 8 labeled examples per class,
⚡ outperforms LoRA fine-tuning in 32 of 39 settings, with 6–12× fewer trainable parameters.

📄 Topology-Injected Masked Fine-Tuning for Graph-Conditioned Language Models, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
TIM-FT is a new method for aligning GFM embeddings into LLM sequences. It places graph embeddings directly at the token positions where an entity is mentioned, keeps the sequence length the same, needs no vocabulary expansion, and improves performance when only 15% of entities have a graph embedding. (Available soon 👀 )

📄 Graph-Conditioned Language Models for Security Systems, NeurIPS Workshop on ML Systems.
SecBFF brings TIM-FT to security monitoring, where graph builds always lag behind the live event stream. It adds a single 530K-parameter projector and leaves the serving path unchanged. (Available soon 👀 )

📄 Predicting Cable Dynamics with Physical Attention Bias, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
A cable has two distances between any two points: along its length and through space. Adding both as attention biases cuts prediction error by 15% and more than halves length drift on cables the model has never seen.

Congrats and thanks to all my collaborators!
#NeurIPS2026

Ill just say that my slogan since 2015 is ”I believe P=NP”!😉

Another Thursday at Meta Superintelligence Labs 🪄

Another Monday at Meta Superintelligence Labs 🪄

Accepted to NeurIPS 2026!💃 Go improve your LLM's final representation with Cayley-Encoders!! 🤩
@NeurIPSConf

I recently joined Fundamental AI Research (FAIR) at Meta Superintelligence Labs, and after two fun years at Meta TLV, I will soon be moving to @Meta New York🗽.

I love (1) math and (2) building things from scratch, so the opportunity to join FAIR’s new AI Verification team as a under the leadership of Gödel Prize winner Prof. Peter O’Hearn, was a no-brainer.

I’m honored to be working with this amazing team to push the frontier of safe, secure, and correct code-generation LLMs and agents 💃 🙆‍♀️ .

From the board at NYU to a paper! 👩‍🔬
A recent collaboration with Prof. Joan Bruna, Gilad Yehudai, Gil Harari, Clayton Sanford and Prof. Amir Globerson:
“Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers”

There are many ways to turn a graph sformer can process: we can encode each node using its neighbors, spectral information, random-walk statistics, and more.
In this work, we show that the tokenization can fundamentally determine what the Transformer is able to compute.
Using the wrong tokenization can mean that:
😱 the Transformer needs many more layers to solve the same task,
😱 learning the task becomes badly conditioned, or
😱 the task becomes completely impossible, regardless of how deep or wide the Transformer is.
The main takeaway is that how we tokenize a graph is not separate from the model’s computational power. Tokenization is part of the architecture!!!
Paper: arxiv.org/pdf/2605.22471

GraphBench just won the Best Paper Runner-Up Award at the Graph Foundation Models Workshop at ICML 2026!!! 🥳

We developed GraphBench as a response to the challenges raised in our "Position: Graph Learning Will Lose Relevance Due to Poor Benchmarks" ICML 2025 paper. It exciting tasks, such as algorithmic reasoning and hardware design, with the goal of pushing the community toward evaluating models on real, impactful problems rather than toy datasets that often fail to generalize to the real world.

GraphBench also comes with predefined data splits and evaluation protocols, along with seamless PyTorch-Geometric integration, making it easy to get started.

Check it out: graphbench.io

Happy that GraphBench has been accepted as an oral at the GFM workshop @icml. 🚀

We also updated GraphBench🆕:
- Extensive documentation📔,
- more baselines📊,
- bugfixes🪲,
- extended 80+ page preprint 📰(arxiv.org/abs/2512.04475).

Check it out: graphbench.io.

Now you can have importance scores for nodes and subgraphs in your LLM GraphRAG pipeline!
“Ex-GraphRAG” at the ACL 2026 Towards Knowledgeable Foundation Models Workshop. 😎

In this work, we study interpretable evidence routing for graph-augmented LLMs. We show that standard re which retrieved entities actually influence the LLM, and replace them with M-GNAN, an intrinsically decomposable encoder that provides exact node-level importance during the forward pass.

M-GNAN is now available in PyTorch Geometric and can be used directly in the G-Retriever framework by simply choosing it as the G-Retriever GNN encoder:

pytorch-geometric.readthedocs.io/en/latest/modu…

Congrats to all authors! @arvindh__a Yoav kor-Sade, Rishi Puri

Gave a talk today at IMVC on our Billion-Scale Graph Foundation Models! 🤩
Thanks to the many ppl who attended!!

Today I received the Deutsch Institution Computer Science Excellence Prize for my PhD 🤩

🤔 Why do we still rely on the final layer of an LLM, when different layers encode different information? 🤔

In our new work, “Improving LLM Final Representations with Inter-Layer Geometry” (ICLR 2026 Workshop on Geometry-grounded Representation Learning and Generative actually, LLMs do not have one “best” layer.
We introduce the Cayley-Encoder: an efficient and effective geometric encoder that learns one strong representation from all layer representations of the LLM, without biasing the representation toward any specific layer.
While adding at most 0.1% learned parameters to the LLM, the Cayley-Encoder achieves large empirical gains over LoRA fine-tuning, final-layer representations, expensive attention-based aggregation, and methods that optimize specific layers for the task.

Improving LLM Final Representations with Inter-Layer Geometry: arxiv.org/pdf/2603.22665

The Cayley-Encoder builds on two previous works work:
(1) “Graph Neural Networks Use Graphs When They Shouldn’t,” where we showed that GNNs tend to overfit graph structures and that regular graphs are robust to this overfitting; and
(2) “Cayley-Graph Propagation,” with @PetarV_93 and @sonjj74 , where we showed how to leverage Cayley Graphs to break computational bottlenecks in GNNs.
Together, these make Cayley Graphs a natural candidate for sparse, efficient, and robust communication between layer representations.

Congrats to Tom Ulanovski and Eyal Blyachman for leading this great work!

The core idea of the Cayley-Encoder is to let all layers communicate through an efficient sparse symmetric structure that is robust to structural overfitting. We do so by introducing a Cayley graph of the Special Linear Group over the LLM’s layer representations and learning over it with a small Graph Neural Network. This enables efficient information flow across the full depth of the model, without overfitting to arbitrary layer-interaction patterns or biasing the representation toward any single layer.

My MSc was in discrete geometry, and it was by far the hardest thing I have done in my career. The open problems are extremely deep and difficult, and the proofs almost never fit into fewer than 20 pages.
What an amazing time to be alive 🥹
Thank you @OpenAI 🙌🏼

ICLR was fun 🇧🇷🤩
We had an awesome Meta event and Meta booth. 🍷
I gave a talk at the Workshop on Geometry-grounded Representation Learning and Generative Modeling, on billion-scale Graph Foundation Models.🙋🏼‍♀️
We presented “SuperMAN: Interpretable and Expressive Networks over ogeneous Data” 🦹🏼‍♀️
and also “Improving LLM Predictions via Inter-Layer Structural Encoders”. 🍇

Links in the first comment! 💃🏼
And this is my first Instagram trend 🌝.

SuperMAN: arxiv.org/abs/2505.19193
ILSE: arxiv.org/pdf/2603.22665

Had a fascinating deep conversation with @ylecun on Geometry and World Models at @iclr_conf 🤩