Twelve Questions to Ask Before You Buy AI for Your Line Card
The demo will go well. They always do.
That is the first thing to understand about buying AI for a distribution business: the demo is a performance, and the vendor wrote the script. The questions that decide whether a tool survives its first month against your catalog are usually the ones the demo was built to avoid.
You don’t need to become a machine-learning expert to ask them. You need to know where AI deployments actually break. There are three places, and four questions belong to each.
1. Where the answer comes from
An AI tool is only as accurate as the source behind any given answer. A general model answers from patterns in its training data; a grounded tool retrieves from your catalog, your specs, and your live systems. That difference is invisible in a polished demo and decisive in production.
- Where does a product-specific answer actually come from: our catalog, or the model’s general training?
- What did you connect to make this demo work, and how long did that connection take?
- When our ERP and a manufacturer spec sheet disagree, which one wins, and who decides?
- Our price file changes weekly and parts go end-of-life monthly. How does the tool find out?
2. What it does when it isn’t sure
The most important behavior in a customer-facing tool is what it does when it doesn’t know. A tool that guesses confidently is worse than no tool, because the wrong answer still ships: to a customer, on a quote, under your name.
- Show me a question it can’t answer well. What happens next?
- Does every answer cite a source I can open, or only the ones in this demo?
- Where is the confidence threshold set, who tuned it, and can we change it?
- When it escalates to a person, what does my applications engineer actually receive?
3. Who keeps it right after launch
This is where these systems quietly die. Catalogs move. A tool grounded in last quarter’s data is confidently wrong today, and nobody notices until a customer holds you to a quote that no longer has margin in it.
- After our catalog changes, is accuracy re-measured, or assumed?
- What is your measured accuracy on our data, before go-live? Not an industry benchmark, ours.
- If we leave, does the structured product knowledge travel with us, or does it live inside your tool?
- Six months from now, who at your company is accountable for our accuracy?
Bring the exam, not the requirements
Here is the counterintuitive part. The most useful thing you can bring to a vendor meeting is not a requirements checklist. It is a list of your twenty hardest real questions, the ones you would be embarrassed to get wrong in front of a customer. The breaker where three near-substitutes exist and only one preserves both the trip class and the UL listing. The compatibility question whose answer lives in a senior rep’s memory and an old email thread. Hand them over, and ask to see them answered, with sources, on your data, in the room.
A vendor who can do that has something real. A vendor who needs to take it back to the team has just shown you where their tool ends.
Before the next meeting, ask yourself three questions too. Do we know our own answers to those twenty, well enough to grade a vendor? Which of our product knowledge lives only in people’s heads, and what is our plan when they retire? And if this tool works, do we own what it learns, or does our vendor?
You are not buying software with a feature list. You are hiring something that has to stay right while your catalog moves underneath it. Judge it the way you would judge a new applications engineer: not by the answer it gives to the question you handed it, but by what it does with the question it has never seen.
About the Author
James Sugrue is Head of Sales Operations and Customer Success at Reshape Automation (ReshapeX), where the team builds a verified product-knowledge layer for industrial distributors and manufacturers. With 30 years in manufacturing, design, and technology, James helps distributors evaluate and deploy AI grounded in their own catalogs and systems. Reach him at james@reshapeautomation.com or visit reshapex.com.
Monday, June 22, 2026
by: James Sugrue
Section: Press Releases
