Shikhar Bharadwaj, Kwanghee Choi, Stephen McIntosh, Chin-Jou Li, Eunjung Yeo, Daisuke Saito, Nobuaki Minematsu, Shinji Watanabe, Jian Zhu, David Harwath, David R. Mortensen, "Phone Segmentation and Recognition through Phonological Activation Mapping" arxiv.org/abs/2607.09020

6 papers (4 main and 2 findings) were accepted at #ACL2026! All are speech papers :)

All the American AI companies talk about sharing the wealth, but all the top open source models are Chinese.

Thinking about model generalization is quite painful.

We observe empirically that models trained with SGD on cross-entropy generalize, instead of just memorize the train data. Even when they have sufficient capacity to memorize.

We do not---i repeat--- we. do. not. have a mathematical proof for why this happens. Don't let anyone convince you otherwise :D

A common current hypothesis involves the 'implicit bias' of SGD, i.e., that gradient descent somehow favors "simple" or "generalizing" solutions. That sounds pleasing, because it's plausible, but we can't precisely define what "simple" means in a manner that is precise. We also can't quite prove why this "bias" leads to the generalization we observe.

This is very deep open problem in machine learning.

Why does SGD find generalizing solutions when memorizing solutions also exist?

The fact that our models become SIMULATORS rather than MEMORIZERS isn't explained by the loss function, algorithm, model arch, data alone.

It emerges from the INTERACTION between the loss function, the optimization algorithm, the model architecture, and the structure of language itself.

We do not understand this, it's an empirical gift that SGD on cross entropy against a model trained on human data produces simulators...

P.S. Please don't just share the arxiv link to your paper that explains generalization :)

Beware of FDs via @StableMoney_ to smaller banks. We did an FD with Shivalik Bank via Stable and when it matured they accidentally sent maturity amount twice. We followed up on email with them and ICICI Bank to reverse one transaction.

Meanwhile Shivalik bank has sent Vasuli Agents who have been calling my wife @smitades to come and sign bunch of documents! They kept calling it recovery of loan!

She called them at nearby ICICI Bank and it's a mess. These Vasuli Agents are not sure how to get that money back, to what account etc. My wife had to spend 2 hours explaining them that trust the icici bank that they will reverse the extra credit but they don't understand how as they are just Vasuli Agents.

Avoid @StableMoney_ and such apps, directly invest via banks 🙏🏻

Highest road tax in the country. This isn’t some random bylane in a nondescript neighbor btw. MG Road - one of the most crucial roads in Bangalore.

🚀 Excited to share our new paper, STAB: Speech Tokenizer Assessment Benchmark! 🔥

STAB provides multiple axes to compare speech tokenizers and gain deeper insights into speech tokens.
#SpeechProc #SpeechRecognition #SpeechTokenization

Transformers can learn arithmetic with the right embeddings. Models trained on 20-digit addition can generalize to 100-digit addition. The same tricks can do 15 digital multiplication, sorting, etc. 🧮

Unlike the previous SOTA, we don't need fancy hardware. We do training (or should I say cramming?) for just 1 day on a 16GB GPU. 🌲🌎

This will be a completely normal workshop proposal

People who envision AI models, in particular LLMs, "doing maths" as some kind of rule-based formal system, you realize that an LLM is a big cascade of spaghetti mess of density modeling with a sampling at the end?

I gave an RLHF lecture at Stanford today, here are the slides. The newer figures from other talks I've given:
* visuals on history of RLHF / related fields
* figures on advanced RL methods (CAI / DPO / rejection sampling) docs.google.com/presentation/d…

One thing we know is that if future AI systems are built on the same blueprint as current Auto-Regressive LLMs, they may become highly knowledgeable but they will still be dumb.
They will still hallucinate, they will still be difficult to control, and they will still merely regurgitate stuff they've been trained on.
MORE IMPORTANTLY, they will still be unable to reason, unable to invent new things, or to plan actions to fulfill objectives.
And unless they can be trained from video, they still won't understand the physical world.

Future systems will *have* to use a different architecture capable of understanding the world, capable of reasoning, and capable of planning so as to satisfy a set of objectives and guardrails.

These objective-driven architectures will be safe and will remain under our control because *we* set their objectives and guardrails and they can't deviate from them.

They won't want to dominate us because they won't have any objective that drives them to dominate (unlike many living species, particularly social species like humans). In fact, guardrail objectives will prevent that.

They will be smarter than us but will remain under our control.
They will make *us* smarter.
The idea that smart AI systems will necessarily dominate humans is just wrong.

Instead of scaling current systems 100x, which will go nowhere, we need to make these Objective-Driven AI architectures work.

if you value intelligence above all other human qualities, you’re gonna have a bad time

A LaTeX embed on 𝕏 would be great for writing mathematical equations, tables, figures, etc. @elonmusk

"India has 140 crore people, and if X% can become customers because of Y% in a developed country," is a dangerous line of thinking for founders and investors.

While building for up to 3% of English-speaking affluent Indians is easier, it's much harder and complex beyond that. ing at Zerodha as well.

I guess the right way to think of this is, India is many countries in one. What works in one state or region may not work in another.

Meta revenue. US versus India.

India is a MAU farm for global companies.

Indian PhD students from @iiscbangalore, who have first-authored papers at prestigious conferences like @CVPR, are facing unjust denials of Canadian visas. With Shocking reasons "limited employment possibilities in India" and "purpose of visit not consistent with a temp. stay."

It is widely thought that neural networks generalize because of implicit regularization of gradient descent. Today at #ICLR2023 we show new evidence to the contrary. We train with gradient-free optimizers and observe generalization competitive with SGD.
openreview.net/forum?id=QC10R…

This curve needs to be there in every ML book/courses now! Implicit Regularization!