DeepSeek V4 Pro Runs 5 Harnesses: Pi Achieves Highest Success Rate, DSH is the Most Cost-Effective

Beating AI Newsflash: Agent tools company Composio tested the same DeepSeek V4 Pro 0813 across Pi Agent, DeepSeek Harness 0.1, Claude Code, OpenCode, and Hermes Agent, all set to max reasoning, running over 30-step Agent tasks. Success rates: 1. Pi Agent: 21/30 2. DeepSeek Harness: 20/30 3. Claude Code: 19/30 4. OpenCode: 19/30 5. Hermes Agent: 18/30 Out of 30 tasks, 15 were passed by all five, 7 were failed by all five. For the remaining 8, simply switching the Harness flipped the results. Speed rankings: 1. Claude Code: 181.8 seconds 2. DeepSeek Harness: 252.1 seconds 3. Hermes Agent: 273.6 seconds 4. OpenCode: 280.6 seconds 5. Pi Agent: 362.9 seconds Cost per successful run: 1. DeepSeek Harness: $0.028 2. Pi Agent: $0.031 3. OpenCode: $0.032 4. Hermes Agent: $0.037 5. Claude Code: $0.074 Pi achieved the highest success rate, DeepSeek's official Harness was the most cost-effective, and Claude Code was the fastest but also the most expensive. Composio previously conducted similar tests using DeepSeek V4 Flash, where Pi also achieved the highest success rate. [Original Link]