We plugged Beam into @opencode, asked it to look at @UnslothAI, and create a fine-tuning notebook for the latest and smallest Gemma-4 model on a Text2SQL task – an out of distribution domain for Beam.
The model successfully researches the model cards, Unsloth documentation, and implements the fine-tuning logic and surrounding machinery. As a result, Beam is able to successfully finetune Gemma, increasing Gemma’s accuracy on its heldout test set by 66.5%.
The model successfully researches the model cards, Unsloth documentation, and implements the fine-tuning logic and surrounding machinery. As a result, Beam is able to successfully finetune Gemma, increasing Gemma’s accuracy on its heldout test set by 66.5%.
Oct 22, 2026 · 9:51 PM UTC
11
19
187
11,600
@reflection_ai @opencode Thanks for sharing, this is a super cool demo of automated fine-tuning! 🔥
0
0
15
804
@reflection_ai @opencode @UnslothAI Thank you for sharing; the community surely appreciates these transparent breakthroughs.
0
0
1
123
@reflection_ai @opencode @UnslothAI Is the 66.5% points or relative? Without Gemma's starting accuracy on the held-out set I can't tell how big it is.
0
0
1
99
@reflection_ai Is the held-out split by query or by database schema? I'd interpret the gain differently depending on whether those schemas appeared in the fine-tuning data.
0
0
0
46
