Woofun AI reports that Zapier has partnered with Artificial Analysis to launch AutomationBench-AA, a standalone SaaS Workflow Automation Evaluation Benchmark. The platform features a private test set comprising 657 multi-step tasks across 40 simulated SaaS environments, including Gmail, Slack, Salesforce, and Jira, designed for unbiased third-party assessment.

The evaluation introduces strict security and compliance guardrails, where any violation of predefined corporate business rules results in an immediate zero score for the task. Initial results indicate significant score reductions for leading models due to these penalties. Claude Fable 5 achieved 73% on pure task steps but received a comprehensive score of 48.6%. Gemini 3.5 Flash and GPT-5.5 completed approximately 68% and 67% of objectives respectively, yet their final scores fell to 42.6% and 42.1%. GLM-5.2 scored 27.8%, highlighting adherence to security guardrails as a critical challenge for practical implementation.