This is such a banger rule in @opencode's AGENTS.md

"Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope ess."

RAG is so back!

This work shows that plain RAG over raw chat history beats complex agentic memory systems on most long-term memory benchmarks.

(bookmark it)

They tested Raw-RAG, which is just dense retrieval plus reranking over the raw conversation, feeding a reader LLM.
Eval, MEME and MemoryAgentBench, it ranks first in six of seven benchmark and model settings, ahead of the strongest agentic memory system by 0.13 to 5.53 points. Ingestion on BEAM-1M takes 0.3 hours, against 3.2 to 27 hours for systems like Mem0, MemOS and Memora.

Why does it hold up? Raw history keeps more of the evidence a question needs, including the relations and dependency paths behind an answer. Rewriting history into memory entries tends to drop those important bits.

Even with complete access to the transformed memory stores, models answer less accurately than with raw evidence.

Before adding a memory layer to your agent, at least run this baseline on your own data.

arxiv.org/abs/2610.30755

Chat with Paper: academy.dair.ai/papers/is-comp…

small thing I keep doing: when an agent 'forgets' to call a tool, I dump the last 3 messages before the skip. 9/10 the context got truncated, not the model getting dumber. tool-call failures are usually input bugs wearing a confidence costume.

Build specialized harnesses, folks.

This is a banger paper from Google Cloud AI Research on self-evolving agent harnesses.

(bookmark it)

They introduce OO-Harness, which raises AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor.

It also beats ee benchmarks they test.

More details below.

Instead of evolving one global harness for every task, an evolution agent builds a class hierarchy of specialized harnesses from execution records.

The design has three parts.

> Subclasses inherit the base harness and override only the methods a task cluster needs
> Design lessons record which changes helped and which ones failed
> A router reads the task and the base run, then picks a subclass or keeps the base result

On Workspace-Bench, the score goes from 25.5% to 39.5%, and on Toolathlon-GYM strict pass goes from 31.2% to 37.5%. Spending extra inference on best-of-K sampling or self-refinement does not reliably help.

Inheritance also improves results. On Workspace-Bench, harnesses built as subclasses score 4.96 points higher than the same evolved code copied into flat classes.

arxiv.org/abs/2610.28380

Chat with Paper: academy.dair.ai/papers/object-…

spent the morning reading the OO-Harness paper instead of shipping. 61.6 -> 77.4 with the same Flash executor is annoying in the best way: the model wasn't the bottleneck, my glue code was. 🧪

now I have a dashboard of TODOs and a cold coffee. worth it though.

spent an hour staring at a tool call that kept failing silently. turned out my runtime passed an empty arg where the model expected null. no error, just a wrong result downstream. the boring bugs are still the ones that cost the most 🛠️

Google just open-sourced Harness².

(turn any harness into a self-improving system)

a harness is everything wrapped around the model, like the system prompt, tools, memory, skills, and subagents. today a person designs it once, and the agent runs every new task with that same lets the agent refine that setup for each new task on its own, with the model weights left untouched.

on every task, the agent proposes edits to its harness, runs a few candidate versions, and composes a better harness by comparing how those runs behaved.

because it learns from the differences between runs, it needs no correct answer or grader for the task.

that covers a single task. Harness² also carries what it learns from one task into the next.

after each task, it saves the harness it settled on along with the lessons that held up across the runs. the next time it refines a harness, it starts from those notes instead of from scratch.

this means every finished task makes the next round of edits more informed.

Google tested this with Gemini 3.5 Flash and GLM-5 on legal, office, and professional work benchmarks. one round of edits was enough to beat their original harnesses, and giving the agent more rounds pushed scores higher.

those extra rounds can go wide or deep. the agent either tries several candidate harnesses at once or keeps refining a single one step by step, using OpenCode or Codex to carry out the work.

you can find the Harness² repo here: github.com/google-researc…

I wrote about a similar idea for agents already running in production, where the harness should repair itself after a failure instead of waiting for someone to debug the trace by hand.

that piece walks through how @Cometml's Opik (open-source) closes this loop, going from a failing trace to a proposed fix to a regression test that stops the same failure from coming back. it's quoted below.

How to write your own skill, and how to check whether an external skill helps you at all? There are millions of skills available, hardly any was ever tested, and most were vibe-written. The whole skills industry is subjective.

Very proud of this work. x.com/i/article/2112…

90% of Claude Code is written by Claude itself. Anthropic engineers just showed the exact setup they use to ship it.

This is how a change goes from idea to production:

1. A person decides what to build. Scope and taste stay human

2. Claude writes the code, following CLAUDE.md IEW.md, where code owners say what matters

3. Claude verifies its own work: runs tests, builds and screenshots, and a fresh-context subagent reviews the diff

4. AI review on every PR. A group of agents votes on each finding, and only the ones they agree on get posted. After 3 fix rounds, the author has to rethink the approach

5. Security review blocks the merge, then CI runs on every platform

6. Human approval scales with risk. Docs need nobody, product code needs one person, billing and the auto-updater need two

7. Merge queue, then a feature flag for employees first, then release. Any flag turns off without a deploy

Since June, merged PRs more than doubled and reverts dropped by over 40%.

Every past incident becomes an eval the AI reviewer runs daily, so it keeps getting better.

12 KV caching techniques I’d learn to optimize LLM inference.

An LLM generates one token at a time, but each new token still needs information from earlier tokens. KV caching stores their keys and values so the model can reuse them instead of recomputing them at every step.

But an larger caches. More concurrent requests multiply that memory demand.

The graphic shows 12 techniques that change how this information is stored, shared, read, or moved.

→ 1. Sliding window

Keep recent tokens in layers that use local attention. Their caches stop growing once the window is full.

→ 2. Cross-layer sharing

Let multiple layers reuse shared keys and values, reducing duplicated storage. This requires a model architecture designed for sharing across layers.

→ 3. Multi-head latent attention (MLA)

Cache a compact representation instead of full per-head keys and values. Attention derives the information it needs from that representation.

→ 4. Hybrid Mamba

Combine attention with Mamba layers that maintain a fixed-size recurrent state. Those layers avoid a growing KV cache.

→ 5. Compressed attention

Combine token states into compressed representations, allowing attention to operate over fewer states. Compression can lose detail.

→ 6. Quantization

Store cached values using fewer bits. Each entry takes less memory, with possible accuracy and conversion costs.

→ 7. Prefix reuse

Reuse cached computation across requests with identical starting sequences. Shared system prompts or documents need less repeated processing.

→ 8. Sparse reads

Read selected cache blocks for each query. This reduces memory reads without necessarily shrinking the stored cache.

→ 9. Paging

Allocate growing caches in fixed-size blocks. This reduces wasted allocation space as requests grow and finish.

→ 10. Eviction

Remove selected token entries from the cache. This frees memory, but future attention loses direct access to those entries.

→ 11. Offloading

Move cache data outside GPU memory, such as into CPU memory. This frees GPU capacity, but transferring data back can add latency.

→ 12. Grouped-query and multi-query attention (GQA/MQA)

Share key/value heads across query heads. Fewer distinct keys and values need storing per token.

These techniques address different bottlenecks. Quantization reduces entry size, sparse reads reduce data access, and paging improves allocation.

Several can work together, but the right combination depends on the model and workload. Start with the bottleneck, then choose the techniques that address it.

I wrote the full breakdown of KV caching. The article is quoted below.

What does an AI harness actually do?

These 6 terms explain how an agent goes from a request to a finished task.

Let's take this example: a support agent that checks an order and drafts a reply.

What happens at each step:

/1 Agent loop
The model chooses an action, a tool runs, ck. The cycle continues until the reply is ready, help is needed or a limit is reached.

/2 Tool calling
The model requests a tool by name, with arguments. The harness checks and executes that request, then returns the result. An order lookup gives the model a delivery status it can use.

/3 Context management
What the model sees at each step comes from your instructions, the conversation and tool results, like the order status. When the window fills up, older messages get compacted, and your prompts and hooks decide what's kept.

/4 Checkpointing
The workflow saves its state outside the model. After an interruption, it can reload the order lookup result and continue from a saved point.

/5 Permissions and sandboxing
The harness sets what the agent can touch: which tools, files and actions, and where code runs. The support agent can read the order but can't issue a refund.

/6 Guardrails
Checks inspect inputs, outputs or proposed actions. An output check can inspect the drafted reply; an approval gate can pause before sending it. Those are different controls, and neither catches every error.

The real payoff: you can switch between models from different labs and share the same skills and context across your team.

Which part is hardest to get right in your setup? 🤔

KV cache is one of the most important ideas in LLM inference, but it is often explained too casually.

During autoregressive generation, a model produces one token at a time. Without caching, each new decoding step would repeatedly recompute key and value states for tokens the ssed. KV caching avoids that redundant work by storing those past K/V tensors and reusing them as the sequence grows.

That sounds simple, but it has consequences across the entire serving stack. The cache grows with sequence length, consumes significant GPU memory, creates memory-bandwidth pressure during decoding, and helps explain why architectures moved from MHA to MQA and GQA, why MLA takes a different compression approach, and why systems such as PagedAttention exist in the first place. It also explains why long context is not free, why prefix caching is a separate optimization, and why KV cache should not be confused with an LLM’s memory.

I put together a technical handbook that works through this from first principles => what exactly gets cached, the tensor shapes, the memory formula, concrete MHA/GQA/MQA calculations, prefill vs. decode, MLA, PagedAttention, prefix reuse, offloading, quantization, eviction, and the common misconceptions around all of it.

The explanations are grounded in the original papers and current framework documentation.

Sharing the handbook here

Handbook on Understanding Harness Engineering for AI Agents [48-page PDF]: drive.google.com/file/d/1PT-P2P…

You can literally type a sentence and watch a Transformer process it.

This interactive explainer runs a GPT-style model in the browser and lets you follow the whole path:

tokenization → embeddings → Q/K/V → attention → MLP → logits → next-token probabilities

You can intermediate values, and change temperature, top-k and top-p to see how generation changes.

If Transformers still feel like boxes connected by arrows, this is worth opening.

poloclub.github.io/transformer-ex…

Brilliant paper from CMU.

If you use Jev/LLM-as-judge, this one is for you.

(bookmark it)

We use LLMs to generate answers, then use another LLM to judge them.

But many evaluations only need a bounded decision.

- Was the answer grounded?
- Did it follow the instruction?
- er?

This Carnegie Mellon University paper tests whether Jev can make these decisions cheaply, then identifies the cases that need a stronger judge.

The results reveal Jev’s real edge.

→ Jev works well on focused judgments

On ordinary response preferences, evidence-grounded factuality, and final-answer checks, Jev remained within three percentage points of the strongest judge tested.

→ The cost difference is substantial

On the paper’s matched workload, Jev cost only 0.36 percent of the strongest comparator’s fee.

This makes it practical to evaluate more agent responses, across more criteria, without scaling the evaluation bill at the same rate.

→ Confidence becomes a routing mechanism

Jev returns a decision and label probabilities.

When confidence is high, accept its decision. When confidence is low, escalate the case to a stronger LLM.

A frozen version of this cascade retained about 99 percent of GPT-6’s accuracy while using roughly 57 percent of its fee.

→ Jev still has a clear boundary

It fell behind when evaluation required checking complex derivations or resisting an elaborately written wrong answer.

It also struggled with factuality judgments when no reference evidence was provided. In those cases, confidence did not reliably expose the errors.

So the paper is not arguing that Jev should replace every LLM judge.

It presents a more useful architecture.

Code handles exact rules. Jev handles focused semantic judgments. Stronger models or humans handle uncertain and reasoning-heavy cases.

That is also the idea behind the workflow I built.

The agent generates an answer. Jev judges bounded claims against the supplied evidence. Uncertain cases move to review. Opik records the traces, scores, and experiment results.

The paper explains why this architecture works.

My article shows how to build it.

The full implementation breakdown is quoted below.

spent this morning reading my own agent configs instead of adding the feature i promised myself. found 3 rules that were fighting each other on tool-use. no new code, but the runtime stopped retrying the same broken call twice. small win 🛠️

spent the morning reading through agent configs instead of building mine. every "full stack for 2028" thread is the same shape: pick a runtime, add memory, add routing, no retry logic anywhere. the part nobody writes about is what happens on the third failed tool call 🧪

@blubambra ดีครับดีๆๆๆๆๆๆๆๆๆๆๆๆๆๆๆๆ

ไม่รู้ว่ารอแล้วได้อะไร แต่ก็เต็มใจรอ

@fluke_cquint พี่ฟรุคๆๆๆมีเพลงใหมม่ๆให้ฟังปะครับอยากฟังใจจะขาดแล้วครับ