Pilot success criteria state a metric, a baseline, a target, a data source, and an owner. Write them as contract language before the pilot starts, not as aspirational goals on a PowerPoint slide. Criteria that skip any of those five parts leave both sides guessing whether the pilot worked.
A weak success criterion looks like "improve efficiency" or "positive user feedback." Those phrases tell you nothing about what data proves success, who measures it, or what threshold counts as passing. A strong criterion looks like this:
Reduce average invoice processing time by 20% compared to the baseline measured in the 30 days prior to pilot start. Measurement will use timestamps from Customer's existing AP system. Customer's Controller and Vendor's Implementation Manager will review progress in biweekly reports.
That criterion gives both sides the same finish line. The metric is processing time. The baseline is the 30 days before the pilot. The target is a 20% reduction. The data source is the existing AP system. The owners are named by role. If you can't write a criterion that specific, you are not ready to start the pilot.
Note: This article provides general information about pilot agreements. It is not legal advice and does not create an attorney-client relationship.
Weak vs strong success criteria
The difference between weak and strong criteria is specificity. Weak criteria are vague, subjective, or impossible to measure. Strong criteria state exactly what counts as success and who decides.
| Weak criterion | Why it fails | Strong criterion |
|---|---|---|
| Improve user adoption | No baseline, no target, no data source | Achieve 75% weekly active user rate among the 40 licensed users by day 45, measured via login analytics |
| Positive feedback | Subjective, no threshold | Net Promoter Score of 40 or higher from end-user survey sent at day 60 |
| Reduce costs | No baseline, no data source | Reduce per-unit fulfillment cost by $0.50 compared to Q3 baseline, measured via ERP cost reports |
| Increase efficiency | Vague metric | Process 30% more orders per hour with the same headcount, measured via WMS throughput logs |
| Better visibility | Not measurable | Generate weekly inventory accuracy reports showing 95% or higher location match rate |
Strong criteria can be verified by both sides using the same data. Weak criteria require interpretation, which guarantees a dispute at the end of the pilot.
The five required parts of every success criterion
Every success criterion needs five parts. Leave one out and the criterion becomes negotiable after the pilot instead of before.
1. Metric: The thing being measured. Examples: average response time, cost per transaction, user adoption rate, error rate, processing throughput. Pick one metric per criterion. Compound metrics like "speed and accuracy" split into two criteria.
2. Baseline: The starting point before the pilot. The baseline can be the prior month, the prior quarter, or a measurement period immediately before the pilot starts. Document the baseline in the contract or reference where it is recorded. Without a baseline, you cannot prove improvement.
3. Target: The goal the pilot must hit. State it as a percentage improvement, an absolute number, or a threshold. Be realistic: a pilot that requires 50% improvement in 60 days is setting both sides up for failure. A 15% to 25% improvement is ambitious but achievable for most operational pilots.
4. Data source: The system or process that generates the proof. The data source must already exist. Do not build new tracking during the pilot unless you account for the setup time in the pilot period. Reference the source explicitly: "Customer's TMS logs," "Vendor's dashboard analytics," "monthly financial reports reviewed by Customer's CFO."
5. Owner: The person responsible for measuring and reporting results. Name a role on each side: the customer's VP of Operations and the vendor's Customer Success Manager. The owners review progress on a defined schedule: weekly, biweekly, or monthly. If nobody owns the measurement, the criterion will not be checked until the pilot is over and it is too late to fix anything.
When you write a criterion with all five parts, it looks like this:
Metric: Average route time
Baseline: Average route time in the 30 days prior to pilot start
Target: 15% reduction
Data source: Customer's existing TMS logs
Owner: Weekly reports reviewed by Customer's Director of Logistics and Vendor's Implementation Lead
That structure fits into one or two sentences in the contract, and it removes all ambiguity about what success means.
Success criteria examples by use case
Different pilot types require different metrics. A SaaS adoption pilot measures usage and workflow integration. An AI pilot measures accuracy and latency. An operations pilot measures cost, speed, or quality. Pick the metric that matches the product's value proposition.
SaaS platform pilot
Product: Project management and collaboration platform
Criterion 1: Achieve 80% weekly active user rate among the 25 licensed users by day 45 of the pilot, measured via Vendor's login analytics dashboard. Customer's IT Director and Vendor's Onboarding Manager will review weekly usage reports.
Criterion 2: Migrate 15 active projects from Customer's existing tool to the platform by day 30, with all task history and attachments intact. Success will be confirmed by Customer's Operations Manager via project audit.
AI and machine learning pilot
Product: AI-powered demand forecasting tool
Criterion 1: Achieve forecast accuracy of 85% or higher for the 20 highest-volume SKUs over the 60-day pilot period, measured as the percentage of forecasts within 10% of actual sales. Accuracy will be calculated using Customer's ERP sales data.
Criterion 2: Reduce manual forecast adjustment time by 50% compared to baseline measured in the 30 days prior to pilot start. Time savings will be reported by Customer's Demand Planning team in biweekly check-ins.
Operations and workflow pilot
Product: Warehouse management system
Criterion 1: Increase picking throughput by 25% compared to the baseline measured in Q2, measured as units picked per labor hour. Throughput data will be pulled from Customer's WMS logs and reviewed weekly by Customer's Warehouse Manager and Vendor's Implementation Consultant.
Criterion 2: Reduce picking errors by 30% compared to Q2 baseline, measured as mispicks per 1,000 units. Error data will be tracked via Customer's existing QA process.
Cybersecurity pilot
Product: Endpoint detection and response platform
Criterion 1: Detect and alert on 100% of simulated threats introduced by Customer's security team during the pilot period, with alerts delivered within 5 minutes of threat activity. Detection rate and alert latency will be measured via Vendor's platform logs.
Criterion 2: Reduce mean time to investigate security incidents by 40% compared to the 60-day baseline prior to pilot start, measured via Customer's ticketing system timestamps.
Each of these examples includes a metric, baseline, target, data source, and owner. They can be dropped into a pilot agreement as written.
How many criteria should a pilot have?
Most pilots have two to four success criteria. One criterion risks tying the entire pilot to a single data point that could be affected by external factors. More than four criteria spreads focus too thin and creates confusion about what actually matters.
If the pilot has multiple criteria, state whether the pilot must meet all of them or just a majority. The default is all: if the contract lists three criteria, all three must be met for the pilot to be considered successful. If you intend a looser threshold, say so explicitly: "The pilot will be considered successful if Customer meets at least two of the three Success Criteria."
Secondary criteria can be tracked without blocking conversion. For example, the primary criterion might be cost reduction, with a secondary criterion tracking user satisfaction. The secondary criterion informs the decision but does not veto it.
Putting success criteria into the pilot agreement
Success criteria belong in the body of the pilot agreement, not in a separate project plan or statement of work. When they live in the contract, both sides have legal clarity about what counts as success and what triggers conversion to a full agreement.
The pilot agreement template includes a dedicated success criteria field. Fill it in with the metric, baseline, target, data source, and owner as described above. The template generates the clause automatically and inserts it into the final contract.
The success criteria field accepts multi-paragraph entries, so you can write detailed measurable goals with all five required parts:
If the pilot has multiple criteria, list them as numbered items. The creator supports multi-paragraph entries, so you can format the criteria as a clean list:
Success Criteria:
Reduce average invoice processing time by 20% compared to baseline, measured via AP system timestamps, reviewed biweekly by Customer's Controller and Vendor's CSM.
Achieve 90% user adoption rate among finance team within 45 days, measured via login analytics, confirmed by Customer's CFO.
That format makes it easy for both sides to check progress during the pilot and confirm whether the criteria were met at the end.
What happens if success criteria are missed?
If the pilot misses the success criteria, neither side is obligated to move forward with a full agreement. The pilot agreement should state this explicitly, so missing the criteria does not trigger liability or breach claims.
Include a clause like this:
If Customer does not meet the Success Criteria by the end of the Pilot Period, neither party will be obligated to enter a commercial agreement, and this Agreement will terminate according to the terms in Section [X].
That language protects both sides. The vendor is not liable for a failed pilot, and the customer is not obligated to buy a product that did not deliver the promised results.
If the pilot comes close to the criteria but misses by a small margin, the conversion clause can allow negotiation: "If Customer meets at least 80% of each Success Criterion, the parties will negotiate in good faith toward a commercial agreement." That flexibility prevents an all-or-nothing outcome when the pilot was mostly successful but hit an unexpected operational issue.
Use a free pilot agreement template to turn your success criteria into a complete contract. The template walks you through every clause, generates the draft in real time, and lets you send it for e-signature when you are ready.
For the full pilot drafting process, see how to write a pilot agreement.
FAQ
What is the difference between a success criterion and a deliverable?
A success criterion measures whether the product delivered value. A deliverable is something the vendor must provide, like a training session or a report. Pilots focus on success criteria, not deliverables.
Can success criteria change during the pilot?
Yes, if both sides agree in writing. Verbal changes are not enforceable. If conditions change and the original criteria no longer make sense, amend the pilot agreement formally before continuing.
Who decides if success criteria were met?
The contract should state who measures and who confirms. Typically the customer measures using its own data, and the vendor reviews the measurement method. If the parties disagree, the contract can include a tiebreaker, such as using a third-party auditor or agreeing that the customer's measurement controls.
Should success criteria be tied to conversion terms?
Yes. The pilot agreement should state what happens if the criteria are met: "If Customer meets the Success Criteria, Vendor will offer a commercial agreement at the rates specified in Exhibit A." That pre-agreed pricing prevents a second negotiation round after the pilot succeeds.
What if the baseline data does not exist?
Capture the baseline in the first week of the pilot and extend the pilot period by one week to compensate. Do not start a pilot without a documented baseline, or you will have no way to prove improvement.
