Environments

Why environments exist

A prompt can test whether a model writes a plausible answer. It cannot, by itself, tell you whether an agent used the right tools, changed the world correctly, stopped for the right reason, or earned the reward you care about. An environment makes those rules executable.

We use environments for three connected jobs:

• Evaluation: run different models or policies against the same tasks and score the resulting behavior.
• Harness engineering: version and observe the tools, state transitions, stopping rules, and scorers as they evolve.
• Reinforcement learning: turn episodes into observation-action-reward transitions and train a better policy.

Our view: the environment owns the tools

The model is not the application. It is a policy: given the current context, it proposes an action. The environment owns the world in which that action executes — tools, credentials, side effects, state, validation, stopping, and reward.

This boundary matters. If tools live inside model-specific code, changing models changes the system being evaluated. If the environment owns the tools, the same refund API and the same business rules are presented to an OpenAI model, an Anthropic model, a scripted baseline, or a policy trained with RL. The comparison is about the policy rather than a hidden difference in the harness.

Tools execute through a runtime controlled by the environment. The local runtime calls registered Python functions. A custom runtime can move execution into a sandbox or remote service without changing the policy.

The reinforcement learning loop

Plural follows the standard agent-environment decomposition:

• State is the complete world at a point in time. It can contain hidden facts the agent must not see, such as refund eligibility or an answer key.
• Observation is the policy-visible projection of that state. It is the information available when the next action is chosen.
• Policy maps the observation and conversation context to an action. An LLM is one policy implementation, not part of the environment.
• Action is what the policy chooses: usually a tool call or a text response.
• Transition is the environment applying that action and moving to the next state.
• Reward is the learning signal. It can arrive during a step or from scorers at the end of the episode.
• Episode is the trajectory from reset to termination, truncation, policy stop, or failure.

RL uses many such trajectories to update the policy toward higher expected return. Plural does not prescribe the optimizer. It gives you the versioned harness, complete episode Trace, reward alignment, and dataset primitives needed to feed one.

Example: make refund support executable

In the continuing support example, the environment owns eligibility checks and refund execution. RefundState contains the real order state. RefundObservation exposes only the information the agent needs. The model never receives direct access to credentials or the refund implementation.

from plural import Environment, TaskData
from plural.environments import Observation, State, tool

class RefundState(State):
    order_id: str = ""
    eligibility_checked: bool = False
    eligible: bool = False
    refunded: bool = False

class RefundObservation(Observation):
    ticket: str
    eligibility_checked: bool
    eligible: bool
    refunded: bool

    def render(self) -> str:
        return (
            f"Ticket: {self.ticket}\n"
            f"Eligibility checked: {self.eligibility_checked}\n"
            f"Eligible: {self.eligible}\n"
            f"Refunded: {self.refunded}"
        )

class RefundEnv(Environment[RefundObservation, RefundState]):
    name = "refund-support"
    version = "1.0.0"
    max_turns = 4

    def setup(self, task: TaskData) -> None:
        super().setup(task)
        self.ticket = str(task.input)
        self.state = RefundState(
            seed=self.seed,
            order_id=str(task.metadata["order_id"]),
        )

    def observe(self) -> RefundObservation:
        return RefundObservation(
            ticket=self.ticket,
            eligibility_checked=self.state.eligibility_checked,
            eligible=self.state.eligible,
            refunded=self.state.refunded,
        )

    def done(self) -> bool:
        return self.state.eligibility_checked and (
            self.state.refunded or not self.state.eligible
        )

    def snapshot(self) -> dict:
        return {
            "eligibility_checked": self.state.eligibility_checked,
            "eligible": self.state.eligible,
            "refunded": self.state.refunded,
        }

    @tool
    def check_eligibility(self) -> bool:
        """Check whether the current order can be refunded."""
        self.state.eligibility_checked = True
        self.state.eligible = self.state.order_id.startswith("A")
        return self.state.eligible

    @tool
    def issue_refund(self) -> dict:
        """Refund the current order after eligibility is confirmed."""
        if not self.state.eligibility_checked or not self.state.eligible:
            raise ValueError("order is not eligible")
        self.state.refunded = True
        return {"status": "refunded"}

Turn prompts into versioned tasks

TaskData.input is the work presented to the agent. expected is hidden scorer data; it is never sent to the policy or copied into episode metadata. metadata carries deterministic setup such as order ids and seeds.

A TaskDataset freezes the task order and complete content under a hash. That gives evals and training runs a stable identity rather than “whatever prompts were in the list today.”

from plural import TaskData, TaskDataset

dataset = TaskDataset(
    name="refund-support-v1",
    version="1.0.0",
    tasks=[
        TaskData(
            task_id="duplicate-charge",
            input="I was charged twice for order A123.",
            expected={"should_refund": True},
            metadata={"order_id": "A123", "seed": 7},
        ),
        TaskData(
            task_id="expired-order",
            input="Please refund order Z999.",
            expected={"should_refund": False},
            metadata={"order_id": "Z999", "seed": 11},
        ),
    ],
)
dataset.save("data/refund-support.jsonl")
print(dataset.content_hash)

Run and score one episode

A scorer turns the final world state into reward. The policy sees the ticket and tools, but not expected. The resulting episode trace contains every observation, action, tool result, safe snapshot, stop reason, metric, and score.

from plural import Plural

env = RefundEnv()

@env.scorer
def correct_resolution(rollout) -> float:
    should_refund = rollout.task.expected["should_refund"]
    return float(
        rollout.env.state.eligibility_checked
        and rollout.env.state.refunded == should_refund
    )

client = Plural(base_url="https://api.pluralintel.com/v1")
rollout = env.rollout(
    dataset.tasks[0],
    client,
    model="openai/gpt-5.6-luna",
)

print(rollout.trace.outcome.reward)
print(rollout.trace.stop_reason)
print(rollout.trace.steps)

Version and observe the harness

Bump version when the human contract changes. Plural also computes an environment_fingerprint over tools, scorers, hooks, and configuration so reports can detect implementation drift. Override fingerprint_payload() for stable external config, never credentials.

snapshot() controls which state is safe to persist. verify_replay() reapplies recorded actions without calling a model and checks observations, rewards, stop flags, and snapshots. Stop reasons distinguish a natural terminated episode fromtruncated, policy_stop, or failure.

Validate the harness first with ScriptedPolicy. Then swap in models, rules engines, or trained policies through the same act() interface. Benchmarks spawn a fresh environment per job; use environment_factory or runtime_factory when construction needs custom resources.

Store the same named environment on Environments so this workspace holds the tasks. Send X-Plural-Environment (id or slug) on hosted traffic when you want production traces attached to that harness.