Skip to navigation
API ReferenceBenchmark Runs

Create Benchmark Run

View as Markdown

Executes a benchmark run and returns the aggregated result with nested evaluators, questions, and a flat evaluations array. A run that cannot finish inside one request returns 202 with completed_at null and continues in the background. Poll Get a benchmark run until completed_at is set.

Authentication

x-api-keystring
API key for authentication
OR
AuthorizationBearer
Bearer token authentication

Path parameters

idstringRequired
The id or key of the benchmark

Request

This endpoint expects an object.
contentstring or list of strings or objectsOptional

Content to evaluate. Required when source_id is supplied.

completion_idstring or nullOptional
Completion UUID whose stored response should be evaluated.
source_idinteger or nullOptional>=1
modelstring or nullOptional1-100 characters
num_responsesinteger or nullOptional1-10
use_question_variantsboolean or nullOptional
reasoning_effortenum or nullOptional
Allowed values:
verbosityenum or nullOptional
Allowed values:
score_thresholddouble or nullOptional
value_thresholddouble or nullOptional
temperaturedouble or nullOptional
top_pdouble or nullOptional
frequency_penaltydouble or nullOptional
presence_penaltydouble or nullOptional

Response

Benchmark run result

Errors

403
Forbidden Error
404
Not Found Error
422
Unprocessable Entity Error
500
Internal Server Error
503
Service Unavailable Error