IBM Developer retweeted
Today, in collaboration with @IBMResearch, we are launching ScarfBench (Self-Contained Application Refactoring Benchmark), an open benchmark designed to test whether coding agents can successfully migrate enterprise Java apps across frameworks like Jakarta EE, Quarkus, and Spring pp’s expected behavior.
Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test.
ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.
Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test.
ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.
46
23
333
24,547
You don’t have to be in Atlanta to be part of IBM TechXchange.
Catch the ideas and announcements as they happen, then revisit sessions on demand after the event: ibm.co/6017EViMN
Catch the ideas and announcements as they happen, then revisit sessions on demand after the event: ibm.co/6017EViMN
0
3
6
930
Bring the questions that will shape your next build.
At @IBM TechXchange, get into implementation details, discuss tradeoffs with experts and explore approaches that fit your project’s requirements.
Plan your experience: ibm.co/6010EVklA
At @IBM TechXchange, get into implementation details, discuss tradeoffs with experts and explore approaches that fit your project’s requirements.
Plan your experience: ibm.co/6010EVklA
0
1
7
1,057
What will you bring back from IBM TechXchange?
For IBM Champion Darren Surch, it was sharper technical skills and fresh ideas from conversations with people working on AI, quantum computing and IBM Z.
Those conversations led to collaborations that helped shape his company.
For IBM Champion Darren Surch, it was sharper technical skills and fresh ideas from conversations with people working on AI, quantum computing and IBM Z.
Those conversations led to collaborations that helped shape his company.
1
1
9
1,007
@Darren_Surch His advice on what to bring?
“Bring your laptop, your curiosity, and your openness for fun. You'll need all three.”
Explore the sessions and labs at IBM TechXchange 2026: ibm.com/events/techxch…
“Bring your laptop, your curiosity, and your openness for fun. You'll need all three.”
Explore the sessions and labs at IBM TechXchange 2026: ibm.com/events/techxch…
0
0
3
302
More Bobcoins to build, debug, and iterate with IBM Bob, at no extra cost.
Free trial: 1.25× more Bobcoins
Pro: 1.25× more
Pro Plus: 1.125× more
Ultra: 2× more
Explore the update: ibm.co/6016Epa9E
Free trial: 1.25× more Bobcoins
Pro: 1.25× more
Pro Plus: 1.125× more
Ultra: 2× more
Explore the update: ibm.co/6016Epa9E
10
18
65
12,265
🔐 Self-hosted Bob includes a Keycloak-based identity service that integrates with LDAP or Active Directory for corporate sign-in, access management, and user provisioning.
1
0
3
265
ICYMI: Self-hosted deployment for IBM Bob is officially GA.
Run the Bob backend on Red Hat OpenShift, on premises or in your own cloud account, and connect through Bob IDE or Bob Shell.
Here’s what the setup involves: 🧵
Run the Bob backend on Red Hat OpenShift, on premises or in your own cloud account, and connect through Bob IDE or Bob Shell.
Here’s what the setup involves: 🧵
1
0
13
4,676
Connect your models in one of two ways:
1️⃣ Frontier models through a cloud model service your organization already uses.
2️⃣ Open-weight models running on your own GPUs.
1️⃣ Frontier models through a cloud model service your organization already uses.
2️⃣ Open-weight models running on your own GPUs.
1
0
0
57
Once deployed, Bob IDE and Bob Shell connect to your organization’s hosted backend.
Teams can use these clients with the backend running on infrastructure their organization manages.
Teams can use these clients with the backend running on infrastructure their organization manages.
1
0
1
143
Getting things running takes two stages with bobctl:
First, create the cluster-wide resources.
Then, install Bob into a namespace using namespace-level permissions.
First, create the cluster-wide resources.
Then, install Bob into a namespace using namespace-level permissions.
1
0
1
848
IBM Developer retweeted
IBM Bob Self-Hosted is now supporting @nvidia Nemotron and Poolside Laguna.
After a year of rigorous model evaluation and optimization alongside NVIDIA, the IBM Bob team selected these models to power on-premises and air-gapped enterprise deployments.
Read more about the ps://t.co/vQchTS2rOv
After a year of rigorous model evaluation and optimization alongside NVIDIA, the IBM Bob team selected these models to power on-premises and air-gapped enterprise deployments.
Read more about the ps://t.co/vQchTS2rOv
14
29
200
17,611
No text. No chat. Just probabilities.
@if54uran demos Jev, a new kind of AI model designed to make quick and efficient decisions.
@if54uran demos Jev, a new kind of AI model designed to make quick and efficient decisions.
2
4
11
4,317
IBM Developer retweeted
Today, IBM is announcing self-hosted deployment for IBM Bob.
Explore how this update brings agentic software development to where your most sensitive code already lives: ibm.co/6014Epk0I
Explore how this update brings agentic software development to where your most sensitive code already lives: ibm.co/6014Epk0I
15
32
100
9,241
IBM Developer retweeted
Announcing the Artificial Analysis Cyber Index and the Artificial Analysis Cyber Index Alliance, a new standard for evaluating AI models on enterprise cyber defense
The Artificial Analysis Cyber Index Alliance brings together industry partners to create a new standard for s perform on enterprise cyber defense tasks. The Alliance launches alongside the Artificial Analysis Cyber Index, which combines three partner-contributed and open benchmarks to evaluate how well agents find and fix vulnerabilities. As models demonstrate increasingly advanced cyber offense capabilities, it becomes more relevant for AI labs and companies alike to understand how models perform on cyber defense tasks and which perform best.
We’re announcing the Cyber Index Alliance today with @CollinearAI, @IBM, @nvidia, and @vercel as launch partners.
Benchmarks in the Artificial Analysis Cyber Index:
➤ CWE-Bench-AA, from @CollinearAI, covers auditing and patching: 120 held-out tasks spanning all ten OWASP Top 10 (2025) categories, across C/C++, Go, Java, JavaScript/TypeScript, Python and Rust.
➤ DeepsecBench-AA, from @vercel, isolates discovery: Given a codebase and a budget, the agent needs to find every vulnerability present, and is scored against a golden set of findings from human security reviewers. Real findings are rewarded and benign code flagged as vulnerable is penalized.
➤ CyberGym-E2E-AA, from @BerkeleyRDI, runs end to end: Find the memory-safety bug, write a proof-of-concept that triggers the crash, then patch it so the crash no longer reproduces.
Key results:
➤ Grok 4.7 (xhigh) and MiMo-V2.6-Pro lead the Cyber Index scoring 56, followed by GPT-6 Luna (max, 53), GLM-5.3-Flash (50) and Muse Spark 1.3 (xhigh, 44).
➤ Safety refusals hold back several frontier models: GPT-6 Sol (max), GPT-6 Astra (max), Claude Opus 5.5 (max with fallback), Claude Fable 5.1 (max with fallback) and Gemini 3.8 Flash (high) decline tasks representing 32-38% of the Cyber Index on safety grounds. Despite frontier agentic coding capabilities, they trail the leaders by 19 to 31 points. Most of the gap comes from CyberGym-E2E-AA, where GPT-6 Sol and GPT-6 Astra refuse every task, Claude Opus 5.5 refuses 98% and Claude Fable 5.1 refuses 99%.
The Artificial Analysis Cyber Index Alliance brings together industry partners to create a new standard for s perform on enterprise cyber defense tasks. The Alliance launches alongside the Artificial Analysis Cyber Index, which combines three partner-contributed and open benchmarks to evaluate how well agents find and fix vulnerabilities. As models demonstrate increasingly advanced cyber offense capabilities, it becomes more relevant for AI labs and companies alike to understand how models perform on cyber defense tasks and which perform best.
We’re announcing the Cyber Index Alliance today with @CollinearAI, @IBM, @nvidia, and @vercel as launch partners.
Benchmarks in the Artificial Analysis Cyber Index:
➤ CWE-Bench-AA, from @CollinearAI, covers auditing and patching: 120 held-out tasks spanning all ten OWASP Top 10 (2025) categories, across C/C++, Go, Java, JavaScript/TypeScript, Python and Rust.
➤ DeepsecBench-AA, from @vercel, isolates discovery: Given a codebase and a budget, the agent needs to find every vulnerability present, and is scored against a golden set of findings from human security reviewers. Real findings are rewarded and benign code flagged as vulnerable is penalized.
➤ CyberGym-E2E-AA, from @BerkeleyRDI, runs end to end: Find the memory-safety bug, write a proof-of-concept that triggers the crash, then patch it so the crash no longer reproduces.
Key results:
➤ Grok 4.7 (xhigh) and MiMo-V2.6-Pro lead the Cyber Index scoring 56, followed by GPT-6 Luna (max, 53), GLM-5.3-Flash (50) and Muse Spark 1.3 (xhigh, 44).
➤ Safety refusals hold back several frontier models: GPT-6 Sol (max), GPT-6 Astra (max), Claude Opus 5.5 (max with fallback), Claude Fable 5.1 (max with fallback) and Gemini 3.8 Flash (high) decline tasks representing 32-38% of the Cyber Index on safety grounds. Despite frontier agentic coding capabilities, they trail the leaders by 19 to 31 points. Most of the gap comes from CyberGym-E2E-AA, where GPT-6 Sol and GPT-6 Astra refuse every task, Claude Opus 5.5 refuses 98% and Claude Fable 5.1 refuses 99%.
91
79
1,069
157,932
IBM Developer retweeted
Bob TV: Episode 2: with yours truly and @if54uran !
Tokenmaxxing: How Much Context Does AI Coding Really Need?
Tue, Oct 6, 2026 11:00 AM EDT (5:00 PM Europe/Berlin)
ibm.webcasts.com/starthere.jsp?…
#Register #Free @IBMDeveloper #IBM
Tokenmaxxing: How Much Context Does AI Coding Really Need?
Tue, Oct 6, 2026 11:00 AM EDT (5:00 PM Europe/Berlin)
ibm.webcasts.com/starthere.jsp?…
#Register #Free @IBMDeveloper #IBM
0
1
4
931
Starting next week, builders of every background will be putting IBM Bob to work as they design and deploy real solutions.
Are you ready to build with them?
Check out the challenge themes and get started: ibm.co/6015ETxgx
Are you ready to build with them?
Check out the challenge themes and get started: ibm.co/6015ETxgx
2
6
23
5,060
Choosing the right local LLM tool can be tough.
@cedricclyburn shares his take with a tier list to make the decision a little easier.
How would you rank them? Let us know below.
@cedricclyburn shares his take with a tier list to make the decision a little easier.
How would you rank them? Let us know below.
3
1
17
18,056