Evaluations

Trigger evaluation on a single conversation

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

Session-based authentication using HTTP-only cookies. Cookies are automatically sent by browsers. For server-side requests, include cookies with names starting with "better-auth." in the Cookie header.

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

conversationId*string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/evals/tenants/tenant_123/projects/project_456/evaluate-conversation" \
  -H "Content-Type: application/json" \
  -d '{
    "conversationId": "string"
  }'

curl -X POST "https://api.pilot.inkeep.com/evals/tenants/tenant_123/projects/project_456/evaluate-conversation" \  -H "Content-Type: application/json" \  -d '{    "conversationId": "string"  }'

{
  "evaluationsTriggered": 0,
  "message": "string",
  "success": true
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Trigger evaluations on conversations with specified evaluators

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

conversationIds*array<>

Items1 <= items

evaluatorIds*array<>

Items1 <= items

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/evals/tenants/tenant_123/projects/project_456/evaluate-conversations" \
  -H "Content-Type: application/json" \
  -d '{
    "conversationIds": [
      "string"
    ],
    "evaluatorIds": [
      "string"
    ]
  }'

curl -X POST "https://api.pilot.inkeep.com/evals/tenants/tenant_123/projects/project_456/evaluate-conversations" \  -H "Content-Type: application/json" \  -d '{    "conversationIds": [      "string"    ],    "evaluatorIds": [      "string"    ]  }'

{
  "conversationIds": [
    "string"
  ],
  "evaluationRunId": "string",
  "evaluatorIds": [
    "string"
  ],
  "message": "string"
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Dataset Items by Dataset ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string"

{
  "data": [
    {
      "createdAt": "string",
      "datasetId": "string",
      "expectedOutput": "string",
      "id": "string",
      "input": "string",
      "simulationAgent": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Dataset Item

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Request Body

application/json

createdAt?string

expectedOutput?|||||array<>|

input*|||||array<>|

simulationAgent?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "string"
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items" \  -H "Content-Type: application/json" \  -d '{    "input": "string"  }'

{
  "data": {
    "createdAt": "string",
    "datasetId": "string",
    "expectedOutput": "string",
    "id": "string",
    "input": "string",
    "simulationAgent": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Multiple Dataset Items

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Request Body

application/json

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/bulk" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "input": "string"
    }
  ]'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/bulk" \  -H "Content-Type: application/json" \  -d '[    {      "input": "string"    }  ]'

{
  "data": [
    {
      "createdAt": "string",
      "datasetId": "string",
      "expectedOutput": "string",
      "id": "string",
      "input": "string",
      "simulationAgent": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Dataset Item by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

itemId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string"

{
  "data": {
    "createdAt": "string",
    "datasetId": "string",
    "expectedOutput": "string",
    "id": "string",
    "input": "string",
    "simulationAgent": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Dataset Item

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

itemId*string

Request Body

application/json

createdAt?string

expectedOutput?|||||array<>|

input?|||||array<>|

simulationAgent?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "createdAt": "string",
    "datasetId": "string",
    "expectedOutput": "string",
    "id": "string",
    "input": "string",
    "simulationAgent": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Dataset Item by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

itemId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/dataset-items/string/items/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List all Datasets

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets"

{
  "data": [
    {
      "createdAt": "string",
      "id": "string",
      "name": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Dataset

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

createdAt?string

name*string

Lengthlength <= 256

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "string"
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets" \  -H "Content-Type: application/json" \  -d '{    "name": "string"  }'

{
  "data": {
    "createdAt": "string",
    "id": "string",
    "name": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Dataset by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string"

{
  "data": {
    "createdAt": "string",
    "id": "string",
    "name": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Dataset by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Request Body

application/json

createdAt?string

name?string

Lengthlength <= 256

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "createdAt": "string",
    "id": "string",
    "name": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Dataset by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

datasetId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/datasets/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluation Job Configs

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs"

{
  "data": [
    {
      "createdAt": "string",
      "id": "string",
      "jobFilters": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Evaluation Job Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

createdAt?string

evaluatorIds*array<>

Items1 <= items

jobFilters?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs" \
  -H "Content-Type: application/json" \
  -d '{
    "evaluatorIds": [
      "string"
    ]
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs" \  -H "Content-Type: application/json" \  -d '{    "evaluatorIds": [      "string"    ]  }'

{
  "data": {
    "createdAt": "string",
    "id": "string",
    "jobFilters": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Job Config by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string"

{
  "data": {
    "createdAt": "string",
    "id": "string",
    "jobFilters": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Evaluation Job Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluators for Evaluation Job Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators"

[
  null
]

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Add Evaluator to Evaluation Job Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

evaluatorId*string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators/string"

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators/string"

null

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Remove Evaluator from Evaluation Job Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

evaluatorId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/evaluators/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Results by Job Config ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/results"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-job-configs/string/results"

{
  "data": [
    {
      "conversationId": "string",
      "createdAt": "string",
      "evaluationRunId": "string",
      "evaluatorId": "string",
      "id": "string",
      "output": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Evaluation Result

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

conversationId*string

createdAt?string

evaluationRunId?string

evaluatorId*string

output?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results" \
  -H "Content-Type: application/json" \
  -d '{
    "conversationId": "string",
    "evaluatorId": "string"
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results" \  -H "Content-Type: application/json" \  -d '{    "conversationId": "string",    "evaluatorId": "string"  }'

{
  "data": {
    "conversationId": "string",
    "createdAt": "string",
    "evaluationRunId": "string",
    "evaluatorId": "string",
    "id": "string",
    "output": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Result by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

resultId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string"

{
  "data": {
    "conversationId": "string",
    "createdAt": "string",
    "evaluationRunId": "string",
    "evaluatorId": "string",
    "id": "string",
    "output": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Evaluation Result

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

resultId*string

Request Body

application/json

conversationId?string

createdAt?string

evaluationRunId?string

evaluatorId?string

output?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "conversationId": "string",
    "createdAt": "string",
    "evaluationRunId": "string",
    "evaluatorId": "string",
    "id": "string",
    "output": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Evaluation Result

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

resultId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-results/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluation Run Configs

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs"

{
  "data": [
    {
      "createdAt": "string",
      "description": "string",
      "id": "string",
      "isActive": true,
      "name": "string",
      "updatedAt": "string",
      "suiteConfigIds": [
        "string"
      ]
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Evaluation Run Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

createdAt?string

description?string

isActive?boolean

name*string

Lengthlength <= 256

suiteConfigIds*array<>

Items1 <= items

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "string",
    "suiteConfigIds": [
      "string"
    ]
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs" \  -H "Content-Type: application/json" \  -d '{    "name": "string",    "suiteConfigIds": [      "string"    ]  }'

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "isActive": true,
    "name": "string",
    "updatedAt": "string",
    "suiteConfigIds": [
      "string"
    ]
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Run Config by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string"

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "isActive": true,
    "name": "string",
    "updatedAt": "string",
    "suiteConfigIds": [
      "string"
    ]
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Evaluation Run Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Request Body

application/json

createdAt?string

description?string

isActive?boolean

name?string

Lengthlength <= 256

suiteConfigIds?array<>

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "isActive": true,
    "name": "string",
    "updatedAt": "string",
    "suiteConfigIds": [
      "string"
    ]
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Evaluation Run Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Results by Run Config ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string/results"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-run-configs/string/results"

{
  "data": [
    {
      "conversationId": "string",
      "createdAt": "string",
      "evaluationRunId": "string",
      "evaluatorId": "string",
      "id": "string",
      "output": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluation Suite Configs

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs"

{
  "data": [
    {
      "createdAt": "string",
      "filters": "string",
      "id": "string",
      "sampleRate": -140737488355328,
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

createdAt?string

evaluatorIds*array<>

Items1 <= items

filters?|||||array<>|

sampleRate?number

Range-140737488355328 <= value <= 140737488355327

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs" \
  -H "Content-Type: application/json" \
  -d '{
    "evaluatorIds": [
      "string"
    ]
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs" \  -H "Content-Type: application/json" \  -d '{    "evaluatorIds": [      "string"    ]  }'

{
  "data": {
    "createdAt": "string",
    "filters": "string",
    "id": "string",
    "sampleRate": -140737488355328,
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluation Suite Config by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string"

{
  "data": {
    "createdAt": "string",
    "filters": "string",
    "id": "string",
    "sampleRate": -140737488355328,
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Request Body

application/json

createdAt?string

evaluatorIds?array<>

filters?|||||array<>|

sampleRate?number

Range-140737488355328 <= value <= 140737488355327

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "createdAt": "string",
    "filters": "string",
    "id": "string",
    "sampleRate": -140737488355328,
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluators for Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators"

[
  null
]

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Add Evaluator to Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

evaluatorId*string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators/string"

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators/string"

null

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Remove Evaluator from Evaluation Suite Config

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

configId*string

evaluatorId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluation-suite-configs/string/evaluators/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

List Evaluators

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators"

{
  "data": [
    {
      "createdAt": "string",
      "description": "string",
      "id": "string",
      "model": "string",
      "name": "string",
      "passCriteria": "string",
      "prompt": "string",
      "schema": "string",
      "updatedAt": "string"
    }
  ],
  "pagination": {
    "limit": 10,
    "page": 1,
    "pages": 0,
    "total": 0
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Create Evaluator

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

createdAt?string

description?string

model*|||||array<>|

name*string

Lengthlength <= 256

passCriteria?|||||array<>|

prompt*string

schema*|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "string",
    "name": "string",
    "prompt": "string",
    "schema": "string"
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators" \  -H "Content-Type: application/json" \  -d '{    "model": "string",    "name": "string",    "prompt": "string",    "schema": "string"  }'

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "model": "string",
    "name": "string",
    "passCriteria": "string",
    "prompt": "string",
    "schema": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluators by IDs

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

Request Body

application/json

evaluatorIds*array<>

Items1 <= items

Response Body

`application/json`

`application/problem+json`

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/batch" \
  -H "Content-Type: application/json" \
  -d '{
    "evaluatorIds": [
      "string"
    ]
  }'

curl -X POST "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/batch" \  -H "Content-Type: application/json" \  -d '{    "evaluatorIds": [      "string"    ]  }'

{
  "data": [
    {
      "createdAt": "string",
      "description": "string",
      "id": "string",
      "model": "string",
      "name": "string",
      "passCriteria": "string",
      "prompt": "string",
      "schema": "string",
      "updatedAt": "string"
    }
  ]
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Get Evaluator by ID

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

evaluatorId*string

Response Body

`application/json`

`application/problem+json`

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string"

curl -X GET "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string"

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "model": "string",
    "name": "string",
    "passCriteria": "string",
    "prompt": "string",
    "schema": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Update Evaluator

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

evaluatorId*string

Request Body

application/json

createdAt?string

description?string

model?|||||array<>|

name?string

Lengthlength <= 256

passCriteria?|||||array<>|

prompt?string

schema?|||||array<>|

updatedAt?string

Response Body

`application/json`

`application/problem+json`

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -X PATCH "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string" \  -H "Content-Type: application/json" \  -d '{}'

{
  "data": {
    "createdAt": "string",
    "description": "string",
    "id": "string",
    "model": "string",
    "name": "string",
    "passCriteria": "string",
    "prompt": "string",
    "schema": "string",
    "updatedAt": "string"
  }
}

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

Delete Evaluator

Authorization

bearerAuth cookieAuth

AuthorizationBearer <token>

Bearer token authentication. Use this for API clients and service-to-service communication. Set the Authorization header to "Bearer ".

In: header

better-auth.session_token<token>

In: cookie

Path Parameters

tenantId*string

Tenant identifier

projectId*string

Project identifier

evaluatorId*string

Response Body

`application/problem+json`

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string"

curl -X DELETE "https://api.pilot.inkeep.com/manage/tenants/tenant_123/projects/project_456/evals/evaluators/string"

Empty

{
  "code": "bad_request",
  "detail": "Bad Request",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 400,
  "title": "Bad Request",
  "error": {
    "code": "bad_request",
    "message": "Bad Request"
  }
}

{
  "code": "unauthorized",
  "detail": "Unauthorized",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 401,
  "title": "Unauthorized",
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}

{
  "code": "forbidden",
  "detail": "Forbidden",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 403,
  "title": "Forbidden",
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}

{
  "code": "not_found",
  "detail": "Not Found",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 404,
  "title": "Not Found",
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

{
  "code": "unprocessable_entity",
  "detail": "Unprocessable Entity",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 422,
  "title": "Unprocessable Entity",
  "error": {
    "code": "unprocessable_entity",
    "message": "Unprocessable Entity"
  }
}

{
  "code": "internal_server_error",
  "detail": "Internal Server Error",
  "instance": "/conversations/123",
  "requestId": "req_1234567890",
  "status": 500,
  "title": "Internal Server Error",
  "error": {
    "code": "internal_server_error",
    "message": "Internal Server Error"
  }
}

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

202application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

201application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

201application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

204

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

201application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

401application/problem+json

403application/problem+json

404application/problem+json

422application/problem+json

500application/problem+json

200application/json

400application/problem+json

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/problem+json`

`application/json`

`application/problem+json`

`application/problem+json`

`application/problem+json`