This isn’t as severe as the latest PHP and Node exploits, but if you’re still using a frontier LLM model for classification and not a small language model (like the ones from @scaledown_team) you might be vulnerable to overspending on basic tasks.

Does classifying a customer request need the same model as holding a conversation?

In this episode of the Convo AI World Podcast, @hermes_f speaks with @neal_k_patel, co-founder of @scaledown_team, about choosing models for the specific jobs inside an AI application.

@hermes_fi@AgoraIOused for the AI friend/enemey.I'll share what that looks like later but it's actually really cool. Just think: an AI companion that actually plays a game with you so you can say, "Hey go protect that side" or "Send some reinforcements here," and it actually responds.

I built a fighting game where you create your own ultimate moves by voice.
@AgoraIO powers voice chat. @reactorworld FastH3 Omni turns your words into a cinematic attack
Early demo. Going open source soon

We talk a lot about Voice AI, but not nearly enough about the infrastructure making it possible.

Real-time voice AI keeps evolving, but one thing remains constant: the network.

Back at @ISTatPENNSTATE talking about networking tech.

We talk a lot about Voice AI, but not nearly enough about the infrastructure making it possible.

Real-time voice AI keeps evolving, but one thing remains constant: the network.

Back at @ISTatPENNSTATE talking about networking tech.

Some more pics from today.

🗣️ User: "STOP!"
🤖 AI Agent: "Sure! But first, let me explain..."

Why is interrupting a Voice AI agent so hard? 👀

Because stopping audio playback ≠ cancelling the entire pipeline.

VAD → ASR → LLM → TTS → Playback

Barge-in is a real-time coordination problem, not just a ere Agora Conversational AI comes in. ⚡

#VoiceAI #Agora #AIEngineering

You already gave the LLM the answer. So why is it writing it out from scratch?

Introducing a new AI model architecture: Constrained Decoding Models (CDMs)

LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.

CDMs are k each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.

Not just a decision model, but a family of 3 CDMs live today:

• Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
• Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
• Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.

Built for production:

• 1M tokens of multimodal (text + image) input, 31x larger than Jev
• 77.2% on DecisionBench vs Jev's 72%
• 200x faster, 467x cheaper than frontier LLM APIs
• $0.05/M input tokens. Output is free (yes, even for summarize)

Live on your favorite platforms today:
@OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus

Get 50M input tokens for free (for the first 50 hours) at link below 👀

Kicking off Day 3 of SF @Techweek_ by ordering a coffee with my voice at the @cartesia event.

Your voice agent responds instantly.

Cool. But did it actually understand the person?

I attended Agora’s From Prototype to Production: Scaling Voice AI at #SFTechWeek. The panel got into what happens AFTER the demo.

Speed isn’t everything. Customer service and coding agents ions. Acknowledge quickly, but take the time to do the work properly.

More context isn’t always better. Give the model what it needs, when it needs it. Keep conversations responsive while heavier work happens in the backend.

Stop asking one model to do every job. Route simple tasks to faster models. Bring in deeper reasoning when it matters.

Test beyond the happy path. Right tool? Right information? Correct permissions? Trace failures and evaluate audio, not just transcripts. Your tests need to reflect your business.

The panel discussed smart glasses combining voice, gaze and spatial context. “Remember that door design” hits differently when the agent knows which door you’re looking at.

My takeaway, working at @Speechmatics getting to the wrong answer faster is still getting it wrong.

Huge thanks to @AgoraIO for an awesome event!

Next, will be at Speechmatics × Aqua Voice × LiveKit for Solving Voice as an Interface.

partiful.com/e/Qfb44oJOo4cY…
#VoiceAI #AIAgents

Thanks to everyone that came out to the Agora event tonight!

@AgoraIO @RiseLink_X @OpenAI @Speechmatics @getbluejay_ai @autodesk Side note: was great to see @BinaryCoder tonight!

Day 2 at SF @Techweek_ … kicking it off at the @googledevs event

I’m in SF this week with the @AgoraIO team for @Techweek_!

Looking forward to seeing what people are shipping, and talking a lot of voice AI, real-time AI, and developer tools.

If you’re around, come say hi.

I join me and the team tomorrow night partiful.com/e/IL3sKweGcfoh…

Built an AI Teacher powered by @AgoraIO Conversational AI.

Ask naturally—it explains, draws on a live board, and adapts to follow-up questions.
In this demo, it visually teaches how an AI agent works, including feedback and failure paths.

#VoiceAI #EdTech

"Where should we go?" shouldn't take 100 messages.
I built Agora Decision Rooms: group voice with Kabir, an AI facilitator, everyone gets a vote.
Built for the @AgoraIO × #AIMobileCoders hackathon.
Talk together. Decide together.
commudle.com/builds/roundta…
@anandgaur22 @akshay81844

🎙️ Announcing the Agora Conversational AI LinkedIn Live series, starting this October.

Grab your seat: linkedin.com/events/7510448…

𝗙𝗶𝗿𝘀𝘁 𝘀𝗲𝘀𝘀𝗶𝗼𝗻: 𝗚𝗲𝘁𝘁𝗶𝗻𝗴 𝗦𝘁𝗮𝗿𝘁𝗲𝗱 𝘄𝗶𝘁𝗵 𝗔𝗴𝗼𝗿𝗮 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗔𝗜

We'll go from "What exactly does stack?" to getting something up and running.

📅 October 1
⏱️ 30–40 mins
📍 LinkedIn Live

No slides-only webinar. We'll actually open the tools and build.