Agents debugging their own harnesses sounds like an absolute rabbit hole and I am 100% going down it.
๐ ๏ธ [๐๐ซ๐ข๐ฆ๐๐ซ] ๐๐๐ซ๐ง๐๐ฌ๐ฌ ๐๐ง๐ ๐ข๐ง๐๐๐ซ๐ข๐ง๐ โข harness.aman.ai
- Iโve increasingly found that improving an AI agent isnโt only about improving the underlying model. The harness around it, including tools, memory, context, verification, and control flow, can the system can accomplish.
- I find self-improving harnesses especially exciting: agents can inspect execution traces, diagnose recurring failures, modify their surrounding system, and evaluate whether those changes actually improve performance.
This primer covers harness design and self-improvement end-to-end, from autoresearch and meta-harness to memory, evaluation, safety, and scaling.
๐น ๐๐๐ซ๐ง๐๐ฌ๐ฌ ๐๐ง๐ ๐ข๐ง๐๐๐ซ๐ข๐ง๐
โข Why the Harness is a First-Class Capability Variable
โข Autoresearch: Agent-Run Optimization Loops by @karpathy
โข Meta-Harness: Optimizing the System Around a Fixed Model
๐น ๐๐๐ฅ๐-๐๐ฆ๐ฉ๐ซ๐จ๐ฏ๐ข๐ง๐ ๐๐๐ซ๐ง๐๐ฌ๐ฌ๐๐ฌ
โข Trace-Driven Harness Self-Modification ("Self-Harness: Harnesses That Improve Themselves")
โข Open-Ended Agent Evolution ("Darwin Gรถdel Machine: Open-Ended Evolution of Self-Improving Agents" by @jennyzhangzt et al.)
โข Learning How to Improve the Harness ("Harness Learning Enables Generalizable Test-Time Adaptation" by @ZAlvin39105 et al.)
๐น ๐๐ซ๐๐ก๐ข๐ญ๐๐๐ญ๐ฎ๐ซ๐, ๐๐๐ฆ๐จ๐ซ๐ฒ, ๐๐ง๐ ๐๐จ๐ง๐ญ๐๐ฑ๐ญ
โข Proposer Agents, Evaluators, and Validation Gates
โข Durable Memory and Execution Traces
โข Context Compaction and Retrieval
๐น ๐๐ฏ๐๐ฅ๐ฎ๐๐ญ๐ข๐จ๐ง, ๐๐๐๐๐ญ๐ฒ, ๐๐ง๐ ๐๐๐๐ฅ๐ข๐ง๐
โข Evaluate the Model + Harness As a Compound System
โข Regression Testing and Trace-Driven Failure Diagnosis
โข Sandboxed Self-Modification and Immutable Evaluation
โข Scaling Search While Preserving Attribution
Primer written in collaboration with @VinijaJain.
#ArtificialIntelligence #GenAI #Harness
0
0
1
1
Anyone else notice review comments always come in right after you switch to a different ticket?
0
0
0
3
Jared D. Richardson retweeted
Concept art of Valeriaโs office has been revealed and it looks like originally the plan with Pandoraโs Box was going to be DRASTICALLY different! ๐
In the concept art we can see a smaller version of Pandoraโs Box sitting on a bookshelf.
Pretty major difference in comparison to ow the story of Pandoraโs Box wouldโve played out in Chapter 5 had it remained smallerโฆ ๐ค
Either way, Iโm glad they went with the larger version of the box. I think it was the better decision.
In the concept art we can see a smaller version of Pandoraโs Box sitting on a bookshelf.
Pretty major difference in comparison to ow the story of Pandoraโs Box wouldโve played out in Chapter 5 had it remained smallerโฆ ๐ค
Either way, Iโm glad they went with the larger version of the box. I think it was the better decision.
7
53
800
20,187
Orville Peck as Vega is such a wild casting choice, I need to see how this plays out
My ranking of Street Fighter (2026) character portrayals. This is just my opinion:
17. Joe (Played by Alexander Volkanovski) - His one scene was hilarious, but didn't really get to showcase the character much.
16. Vega (Played by Orville Peck) - He did a decent job with the ly fleshed out. Definitely a step above Taboo's portrayal, but Jay Tavare did this role best back in 1994.
15. Guile (Played by Cody Rhodes) - Cody did much better than I originally thought he was going to. Although his portrayal of Guile wasn't totally accurate to his video game counterpart, he was actually entertaining. What hurts his ranking is the lack of a definitive Guile fight scene, The multi fighter brawl doesn't count.
14. Juli (Played by Rayna Vallandingham) - She had the martial arts skills down, as shown in her fight scene, which was great! But the competition from the other strong character portrayals places her at 14.
0
0
0
30
I need life to start handing out XP again, preferably for doing literally anything useful.
EA co-founder @bingfish thinks people are drawn to games because they give players something life stops providing in adulthood, which is levels.
"Humans mostly stop levels somewhere between age 12 and their college degree. After that, everything is wide open and disorganized."
levels on top of it: I want to be a podcast level A, and I'm only a podcast B+. How do I do that?"
"We like to build systems of ranking and leveling and improving, because it makes us feel better."
0
0
0
1
AI flagging duplicate invoices before payment might actually be worth the hassle of checking every alert.
0
0
0
7
I Still Want A Shelf Full Of Games I Can Actually Keep. Keep Discs Alive
RUMOR: XBOX could be betting on bringing back physical media for next generation!๐
โ Microsoft is actively hiring an XBOX Games Supply Strategist
โ This position manages the entire lifecycle of physical media, including manufacturing, distribution, inventory, and retail sales.
t Microsoft still plans to produce physical game copies for future releases.
โ This against recent rumors about Xbox moving to an all-digital format and aligns with their โDisc to Digitalโ program, which allows users to convert physical discs into digital licenses.
Source: @eXtas1stv
____________
Rooting for Asha Sharma to put XBOX on the right side of history and force the continued production of physical media in gaming.
Set an example @asha_shar and our competition back into gaming, let XBOX be the system that forces others to maintain physical media production.
I, as a huge PlayStation fan since PS1, have been MASSIVELY disappointed with their strategy this generation and their predatory practices especially in the last few months.
Physical media must be protected at all costs because:
โ Represents ownership
โ Maintains competitive pricing
โ Used on any account/region
โ Not reliant on networks
โ Allows collectorโs market, second hand market and trading to continue
NO DISC ๐ฟ NO BUY
0
0
0
21
Joshua Graham is one of the coolest characters Fallout ever gave us. New Vegas goes so hard.
Salt Lake City, October 2077.
The dispatcher had stopped asking officers to disperse the crowds. Now she was begging anyone still on the frequency to respond.
Behind his patrol car, an SLCPD SWAT officer watched another city block disappear into flames. He had been on duty for unition was running low, reinforcements weren't coming, and nobody seemed to know who was still in charge.
He had joined the force to protect the city. Tonight, he was just trying to survive it.
His name would be lost to history. But somewhere among the ruins of Salt Lake City, a battered police vest would survive.
One day, it would find its way into the hands of a man who knew a thing or two about surviving the flames.
Joshua Graham. The Burned Man. #FalloutFriday โข๏ธ
0
0
1
7
Missed the train and now the escalator's out too. Do I even bother going upstairs or just live here now?
0
0
0
3
Jared D. Richardson retweeted
GitHub Actions is the industry standard for learning CI/CD Pipeline.
So if you are into application security engineering or want to get in on that niche, understand/learn to write CI/CD pipeline through GitHub actions first and know how to write YAML before trying to know GitLabs CI/CD.
So if you are into application security engineering or want to get in on that niche, understand/learn to write CI/CD pipeline through GitHub actions first and know how to write YAML before trying to know GitLabs CI/CD.
1
4
22
1,231
I need to replay Zero and actually read every single record this time
cool thing i noticed while examining the Assault Records of Ace Combat Zero
one of the Aces you can splash would go on to be heavily involved with the R&D of UAV and AI tech with North Grunder that would become a key aspect of the Lighthouse War 24 years after the Belkan War
0
0
0
3
I would absolutely find out how much React I retained the hard way with this
Today, I worked on final assessments, adding a way for developers to test their understanding after completing a course's lessons and exercises.
Today is Day 283 of 365 Days of Code.
Developers can now unlock a course's final assessment after completing its content and take a ngle-sitting assessment with multiple-choice, true/false, and short-answer questions.
I implemented server-side grading, pass marks, attempt history, corrections after submission, and results pages that show how they performed and where to go next.
I also added an assessment editor to Course Studio, making a ready final assessment a requirement before a course can be submitted for review.
Learn, practice, and prove your understanding.
Take your first assessment here: learn.coddle.dev
0
0
1
79
Aegis in a Sonic racing game is ridiculous and I need to play this immediately
NEW REVEALS FOR SONIC RACING: CROSSWORLDS
๐ McLaren Racing (Sonic)
๐ Aegis from Persona 3
#SonicNews
0
0
0
Xbox making its publishing tools less painful for developers is the part I care about
Inside Microsoft's plan to boost Xbox and Battle .net on PC, with more support for third-party publishers, eased certification rules for games, more polish on the back end, and much more.
โข Xbox plans to invest more heavily in Battle net as a PC platform, with Asha Sharma on is โnearly as good as Steamโs retention.โ
โข Sharma said Microsoft wants to support Battle net โthe right wayโ for its existing community while bringing more first-party and select third-party games to the platform. The Witcher 3 is currently the first major third-party addition.
โข Microsoft also plans to simplify Xboxโs publishing tools and APIs to make it easier for developers to release and manage games across the Xbox ecosystem.
โข Sharma stressed that Xbox will continue investing in other PC storefronts, including Steam, as long as they reach distinct audiences.
Asha's Quotes:
โOn PC historically, we've taken a one-shot approach. We've had one app, and that's what we've focused on. As we have dug in, we're leaving a lot on the table," Sharma said. "It turns out that Battlenet supports an audience that we're not reaching outside of it, and it is awesome at it. The retention is nearly as good as Steam's retention, and we've never really put any work behind it as a platform. And so, we are doing that. We are doing it the right way for that community. It'll be great for our first-party games, and great for select third-party games too.โ
โWe will continue to invest in other PC apps as long as they reach distinct audiences,โ Sharma emphasized. โThe Windows app [Microsoft Store] reaches a lot of people around the world. In fact, in China, it is the number one store,โ
โWe're collaborating across the board,โ Sharma said, โwe're doing a lot this fall [in terms of offers] through OEMs. We have an offer going out to 14 million OEMs. I saw some great pre-install work that's happening for [World of Warcraft], and some great conversion numbers. We will continue to succeed on Windows. There are a billion Windows PCs. There's no reason we shouldn't be serving our audiences there.โ
โRight now, it is simply very hard to work with Xbox,โ Sharma admitted. โSo, we are simplifying the targets, we're moving to a headless partner center, we're taking the cruft out of our APIs, and we're making them machine-readable. There's a lot of work here. We need developers to be able to work with us, understand how they are performing, and have tools that actually grow their business. There's a lot we can do here. I'm encouraged by the work happening.โ
via: Jez Corden
windowscentral.com/gaming/xbox/inโฆ
0
0
0
6
Jared D. Richardson retweeted
The next generation of AI engineers will need to understand:
LLMs + distributed systems + databases + networking + product thinking.
Not just Python.
LLMs + distributed systems + databases + networking + product thinking.
Not just Python.
11
5
96
3,180
Man, I wanted Gears to have a huge comeback moment
Gears of War E-Gay's wide release is today, and things are not looking good.
The game lost player population over the weekend, peaking at only 20k players on the first day.
It has reportedly cost over 400 million dollars to develop and market, and these numbers aren't gonna cut nsive. Based on previous games, you need at least 100k concurrent players to approach breaking even for a game this expensive.
So, we shall see... how high will it go? Will it actually go higher than 100k? Or will it flop like the garbage game that it is?
0
0
0
3
I have absolutely been sleeping on how much useful signal is sitting in bug reports and PR comments
No production traces? You can still build an eval dataset.
Look for places where humans already judged your AI, such as GitHub issues, edited PRs, support tickets, corrected answers, thumbs-down.
Your eval data might already exist.
0
0
0
4