Chakra Labs’ mission is to maximize access to useful intelligence.
We're excited to partner with @SemiAnalysis_ on GDPx to measure how top models perform under realistic scenarios in legal domains.
We're excited to partner with @SemiAnalysis_ on GDPx to measure how top models perform under realistic scenarios in legal domains.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.
GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
6
8
58
5,327
For GDPx, Chakra Labs contributed 20 legal tasks developed with senior-level attorneys across eight practice areas, from M&A and venture capital to intellectual property and dispute resolution.
The rubrics accept supported alternatives wherever reasonable lawyers could disagree. They give greater weight to the reasoning behind a conclusion rather than just the end outcome.
Importantly, weights scale with the consequences for the client, so missing the central legal issue costs more than a formatting error.
The rubrics accept supported alternatives wherever reasonable lawyers could disagree. They give greater weight to the reasoning behind a conclusion rather than just the end outcome.
Importantly, weights scale with the consequences for the client, so missing the central legal issue costs more than a formatting error.
1
0
4
216
Dedicated website: gdpx.semianalysis.com/?domain=legal&…
Blog post: newsletter.semianalysis.com/p/gdpx-measuri…
Blog post: newsletter.semianalysis.com/p/gdpx-measuri…
0
0
3
147
Chakra Labs retweeted
Fantastic day at @tryterac’s inaugural RL Summit - great mix of eng, research, and applied folks all in one room.
Thanks to @zacbakerr and @RyanMWexler for putting together such a sharp group, and for letting me share what we’re seeing at @chakra_ai. Excited to return next year!
Thanks to @zacbakerr and @RyanMWexler for putting together such a sharp group, and for letting me share what we’re seeing at @chakra_ai. Excited to return next year!
4
7
31
1,280
To get started, check out our docs and code examples below. docs.trydojo.ai/integrations/o…
4
6
41
4,990
Chakra Labs retweeted
You can now run tasks in @chakra_ai's RL environments using Browserbase as the runtime.
Get task concurrency, isolated contexts, and enhanced observability.
Get task concurrency, isolated contexts, and enhanced observability.
Update: Dojo now supports @browserbase
19
12
81
10,110
We're excited to announce support for Browserbase, which allows researchers to quickly spin up 1000's of browser tasks concurrently using their serverless hosted infrastructure.
Researchers now benefit from:
- Hyper-scalable browser infra
- Secure SOC II instances
- Native on recordings
Researchers now benefit from:
- Hyper-scalable browser infra
- Secure SOC II instances
- Native on recordings
2
4
36
3,132
To get started with this custom run-time, check out the Dojo engine docs below:
docs.trydojo.ai/engines/browse…
docs.trydojo.ai/engines/browse…
2
2
24
2,589