Most datasets for training commerce agents are built on sandboxed replicas: static HTML snapshots, mocked checkout flows, environments where nothing breaks and no one is watching. Commerce Agent Traces is different. Every session in this dataset runs against live retail environments - real authentication walls, real dynamic pricing, real bot detection.
The dataset is built by Machine Commerce, General Machines' commerce infrastructure product. Every trace is an end-to-end recording of an AI agent completing a commercial task on a major US retail site. The agent is on its own. No scaffolding, no simplified interface, no pre-cleared CAPTCHA.
What each session record contains
Each session is stored as a structured JSON record with a normalized schema across all retail environments. The record captures the full arc of the agent's work:
- Task description: The goal assigned to the agent before the session began, specified in natural language with all relevant constraints (target product, price ceiling, variant requirements, shipping deadline).
- DOM snapshots: A full snapshot of the rendered DOM at each agent decision point, capturing exactly what the agent saw before it acted.
- Action sequence: The complete ordered log of every click, keystroke, scroll, and navigation event the agent executed across the session.
- Task completion status: Binary success or failure with the final cart or order state at session close.
- Per-step decision quality annotations: Each action in the sequence is annotated for decision quality - whether it was optimal, acceptable, or a recoverable error, given the task goal and the visible state at the time.
- Bottleneck classification: For sessions that failed or took significantly more steps than the minimum viable path, the failure point is labeled with a bottleneck category: auth friction, bot detection, session expiry, form-fill failure, or navigation dead end.
Task taxonomy
Sessions are organized into four task categories, listed in order of complexity:
- Discovery: Finding a specific product or class of products given a natural language description and site navigation starting from the homepage.
- Comparison: Identifying and comparing multiple candidate products against a set of criteria, including price, specifications, and availability.
- Cart management: Adding, removing, and modifying items in a cart across sessions, including handling variant selection, quantity changes, and coupon application.
- Checkout: Completing a purchase end-to-end through authentication, shipping, payment, and order confirmation, including handling all forms of friction the environment introduces.
Why live environments matter
Sandboxed benchmarks measure what an agent can do when the environment cooperates. Live environments measure what an agent can do when the environment doesn't. Session expiry after inactivity, checkout flows that change between page loads, login walls that appear mid-task, anti-bot challenges that interrupt navigation - these are the conditions agents encounter in production. This dataset was collected in production.
Sessions are run in isolation. There is no state bleed between sessions: each run starts fresh, with no cookies, no session history, and no prior interaction state carried over. This makes completion rates and step counts directly comparable across environments and across time.
Use cases
The dataset is designed for three primary use cases:
- Training commerce agents: Supervision signal at the action level, with both positive demonstrations and annotated failure recoveries included in the corpus.
- Evaluating agent quality: Completion rates and step efficiency scores across task categories and environments, providing a structured basis for comparing agent versions.
- Debugging specific failure modes: Bottleneck annotations make it possible to identify exactly where in the task taxonomy a given agent breaks down and under what environmental conditions.
Data specifications
- Format: Structured JSON, normalized schema across all retail environments
- Environments: Major US retail sites, run live - not sandboxed replicas
- Task coverage: Four categories: Discovery, Comparison, Cart management, Checkout
- Scoring: Binary completion (success/failure) and step efficiency (steps taken vs. minimum viable path)
- Annotation: Per-step decision quality labels and bottleneck classification for failed or inefficient sessions
- Session isolation: Each session run independently with no state carryover
desc
snaps
seq
quality
class
per session
per record
categories
(not sandboxed)
Access
Commerce Agent Traces is available for early access to frontier labs and applied AI teams building or evaluating commerce agents. To discuss dataset access and licensing, contact us at founders@generalmachines.ai.