How to Run an AI Pilot That Actually Proves Something
A field guide for planners and project controls leads asked to trial an AI scheduling or forecasting tool, and then to say whether it worked.
Construction has no shortage of AI pilots. What it lacks is pilots that turn into something. The 2026 edition of the RICS report on AI in construction, published in August, found that regular use of AI in specific processes rose from 12% to 19% in a year, while full integration across an organisation stays below 1%. The gap, the report says, is now "the distance between running a pilot and embedding it". In October a BRG survey of construction executives added that nearly seven in ten organisations have spent at least $250,000 on AI over two years, yet no single yardstick for its value has emerged.
If you have been asked to trial an AI scheduling, forecasting or controls tool, here is our editorial guidance on designing a pilot that can actually answer the question. It applies to any vendor. It endorses none.
1. Name the Decision, Not the Feature
Why: "Test the AI" is not a goal. A pilot needs one decision it is meant to improve: a sharper completion forecast at each monthly update, say, or faster identification of the activities driving slippage.
The check: Write the decision in one sentence, and name the person who makes it today. If you can't, you are not ready to start.
2. Test on Projects That Have Already Finished
Why: A live project can take years to reveal whether a forecast was right. A closed one already knows. Freeze the schedule as it stood at an old data date, give the tool only what was known then, and compare its forecast with what actually happened. Ask how the vendor measures accuracy. Means & Methods Lab notes, for example, that nPlan points to a white paper framing forecast accuracy as mean absolute error.
The check: Pick at least two completed projects, run the backtest at three data dates each, and record the forecast error in days against your current method.
3. Pick the Yardstick Before the Demo
Why: Once you have seen a polished demo, every result looks like success. BRG's finding that the industry has no agreed measure of AI's value is a warning: if you don't set one, nobody will. A pilot that cannot fail cannot prove anything either.
The check: Agree in writing, before kick-off, the metric, the baseline you have today, and the threshold that counts as a pass or a fail.
A pilot that cannot fail cannot prove anything either.
4. Clean the Schedule First
Why: Optimising an inaccurate schedule, as one industry guide put it in September, "produces a confident, unbuildable plan". The US Government Accountability Office's Schedule Assessment Guide sets out ten best practices for a reliable schedule. They are a fair bar for the input before you judge the output.
The check: Run your usual quality checks on every pilot schedule and fix or exclude the failures, so you are testing the tool, not your data.
5. Demand Answers You Can Trace
Why: A forecast or a flagged risk is only useful if a planner can see where it came from and defend it in a progress meeting, or a claim.
The check: For ten outputs chosen at random, ask the tool to show the activity IDs, data and logic behind each one. Count how many a planner can verify in under five minutes.
6. Map Where the Data Goes
Why: Schedules and cost data are commercially sensitive. Professional bodies are setting expectations: the RICS standard on responsible use of AI in surveying practice came into effect in March 2026, according to the same RICS report.
The check: Get written answers on where the data is stored, who can access it, whether it trains shared models, and how it is deleted when the pilot ends.
7. Price the Whole Thing
Why: A pilot fee is not a programme cost. In its October review, Means & Methods Lab found that several AI scheduling vendors publish no prices at all, and that ALICE quotes per project and is most often used on projects worth at least $75 million.
The check: Ask for the full-rollout price at your real portfolio size, including integration, training and usage-based charges, before the pilot starts.
8. Decide Now What "Embedded" Means
Why: The RICS data says most pilots stall at exactly this point. Embedding means a changed procedure, a named owner and a budget line, not a happy steering committee.
The check: Set a go/no-go date, and draft the procedure change you would make if the pilot passes.
The tools will keep getting better. A pilot designed this way tells you whether they are better for you.
Method. This is editorial guidance, not a review of any product. Sources were checked on 1 November 2026, and vendor details are as reported by the sources linked.