Plural
Sign in
Docs

SDK and CLI coverage guide

This page is a map. Find what you want to do, then follow the link to the page that explains it.

If you are new, don't start here. Read Core concepts, then follow the Quickstart or a tutorial such as the support queue in order. Come back here when you need a specific feature.

Every command and API named on this page exists in the installed package today. Each linked page lists every supported parameter.

Getting started and authentication

  • Installing Plural, creating a project, signing in, and choosing a scope: Quickstart.
  • plural auth login, logout, status, and scope, configuration locations, output, and exit codes: Command-line interface.
  • Local versus hosted resources, and the CLI versus the Python SDK: Concepts.

Model calls and providers

  • Calling a model from Python with Client / Plural, chat, achat, stream, astream, errors, retries, and the model catalog: Model calls.
  • plural model list and the models your organization permits: Agents.
  • Model endpoints, OpenAI-compatible hosts, bring-your-own keys, and routing after evaluation: Providers and integrations.

Build a project

These are the pieces you describe: the world, the assignments, the grader, and the contestants.

  • The project layout and every manifest: YAML and serialization.
  • The systems the AI works with (Apps): MCP servers you write with FastMCP, hosted MCP servers, Plural's files server, HTTP APIs, recordings, and Python Apps; what the Agent sees, control levels, Agent credentials, and secrets: Apps, Build an MCP server, and Connect an MCP server.
  • The world (Environment): Apps, rewards, resources, limits, and Runtime: Environments.
  • The assignments (Tasks): instructions, initial State, and Task files: Tasks.
  • The grader (Verifiers): deterministic checks, model judges, and human rubrics: Verifiers.
  • How the contestant plays (Harnesses): the native loop, the built-in Hermes, Claude Code, and Codex Harnesses, custom Harness classes, and secret grants: Harnesses.
  • The contestants (Agents): models, instructions, and saved Agents: Agents.
  • The exam (Benchmarks): Task collections, scoring rules, tracks, and releases: Benchmarks.
  • Versions, content hashes, and immutability: Updating and versioning.

Run and inspect

Traces, datasets, and export

Hosted projects

Pushing saves a private copy of your work, so your team can see it and every recorded run in the web app. Nothing you push is public.

  • plural project init --push, plural project push, plural <kind> push and pull, --with-deps, .pluralignore, and plural.lock: Push and pull.
  • plural job record, which records a finished local Job in the hosted project: Jobs.
  • Publishing a Benchmark release to the public Hub, a separate step in the web app: Benchmark publications.

Every CLI command

The CLI guide lists every command and option, generated from the command tree. Each resource kind uses the same verbs:

bash
plural <env|task|verifier|harness|agent|benchmark> init|validate|push|pull|show|list|delete

plural benchmark add and remove edit a Benchmark's Task list, and plural app add|list|remove edits an Environment's Apps. The other groups are plural run, plural job, plural trial, plural review, plural reward, plural model, plural runtime, plural secret, plural credential, plural session, plural project, and plural auth.

Reference and operations