Skip this evaluation and the gap it should have caught won’t surface until an auditor, an insurer, or an incident finds it. If you’re weighing a separate AI consultant, or wondering whether your current IT provider can actually lead on this, that’s the risk you’re deciding on.
At a Glance
- Six specific questions to ask any AI advisor, current MSP, or prospective consultant, in one conversation to test for genuine operational depth.
- What matters is whether a provider can demonstrate specific, checkable competencies, such as data flow visibility, tool approval processes, and human oversight design.
- One question carries more weight than the other five: whether the provider understands the compliance obligations your business already has.
- Most valuable for businesses without an internal AI or compliance function, where there’s no safety net to catch a bad decision early.
The Real Question to Ask
Most businesses reading this are in one of two places: an AI consultant is actively pitching them on an engagement, or they’re quietly wondering whether their current MSP is actually equipped to lead on something this consequential. Both situations come down to the same underlying question: not which type of provider to hire, but whether they clear six specific, checkable competencies. A provider who clears all six, whether they’re an MSP, a fractional CTO, or a dedicated AI consultant, is worth a serious conversation. One who can’t shouldn’t be trusted with this decision regardless of the title on the contract.
Skipping this evaluation and trusting the wrong signal, such as an impressive pitch deck, a confident tone, or general enthusiasm about AI’s potential, is how businesses end up with a governance gap they don’t discover until an auditor, an insurer, or an incident surfaces it. A healthcare practice that adopts an AI scribe tool without confirming it’s covered under a business associate agreement, or an insurance agency using an AI tool to process claims data without confirming it meets state-level data privacy requirements, is a compliance and liability mistake that a more thorough evaluation would have caught.
Whether businesses should go around their MSP for AI guidance is an active, unresolved conversation in the managed services industry right now.
The six questions that follow are the actual test. They can be asked in a single conversation, and the answers will tell you more than any sales presentation will.
Six Questions That Separate Real Competency from Marketing Language
These six questions can be asked in a single conversation with your existing MSP or a prospective AI consultant. How a provider answers them — not what they put in a proposal — is what actually distinguishes real operational depth from surface-level familiarity.
1. Can you trace exactly where our data goes?
Most AI tools process your input by sending it to a third-party model or platform. What happens to that data afterward — whether it’s retained, for how long, and whether it’s used to train future versions of the model — varies significantly depending on how the tool is configured and what tier of service is in place.
A strong answer names specific tools and speaks to their data retention behavior directly. It also distinguishes between a tool’s default consumer-tier handling, which often includes prompt retention for model training, and an enterprise-tier agreement that explicitly disables that.
A weak answer is enthusiasm about what the tool can do, with no answer to where the data actually sits afterward.
2. Is there an actual process for approving new AI tools?
Employees discover and adopt AI tools quickly, often before anyone in IT or leadership has evaluated what those tools do with business data.
A strong answer describes a defined intake step: someone reviews a new tool before it connects to business systems, email, or file storage.
A weak answer is some version of “we’re pretty open to trying new things.” The failure mode this question is designed to catch is sometimes called shadow AI: an employee connects a free AI tool to company email or cloud storage, and no one ever evaluates what that tool does with the data it now has access to.
3. Have you mapped AI use to the compliance rules we already follow?
AI adoption doesn’t create entirely new regulatory obligations. In most cases, it creates new ways to violate the ones a business already carries.
A strong answer names a specific framework relevant to your industry: HIPAA for healthcare, CMMC for defense manufacturing and construction subcontractors, PCI DSS for businesses handling card payments, GLBA for accounting and financial services firms.
A weak answer talks about AI risk in the abstract — bias, fairness, responsible use — without ever naming a regulation the reader is actually subject to. Generic AI ethics language is not a substitute for regulatory fluency.
4. What happens when someone pastes sensitive data into a public AI tool?
This isn’t a hypothetical edge case. An employee drafting a client email in ChatGPT and pasting in account details or medical information may not think of it as a data disclosure, but depending on the tool and the data, it may be exactly that.
A strong answer describes an actual control: specific tools blocked at the network level, or a defined data classification policy that tells employees clearly what can and cannot go into an AI prompt.
A weak answer is a policy that exists in a document somewhere but was never communicated to staff and has no technical enforcement behind it. Good intentions without a mechanism are not a control.
5. Do you separate AI that assists from AI that acts?
There is a meaningful difference between an AI tool that drafts a summary for a human to review and an AI tool that is empowered to send messages, modify records, or execute changes autonomously — a category sometimes referred to as agentic AI.
A strong answer describes specific boundaries: which actions require human approval before execution, and what an AI tool is and isn’t allowed to touch.
A weak answer treats all AI use as carrying equivalent risk, with no distinction based on whether the tool is generating a suggestion or making a change to a live system. A provider who hasn’t thought through this distinction may not catch an AI tool that has been quietly granted access to send external communications or modify production data.
6. Can you show us real governance work at our scale?
AI governance frameworks built for enterprise organizations with dedicated data science and compliance teams don’t translate directly to a business with 20 to 200 employees and no internal AI function.
A strong answer offers something concrete: a documented framework, a real example of a tool evaluation they’ve completed, or a specific decision they’ve made on behalf of a client in a similar situation.
A weak answer stays at the level of a slide deck describing AI’s potential. The specific mismatch this question is designed to catch is a consultant whose only reference points are large enterprise transformations, brought in to advise a business that has no internal safety net to catch a recommendation that doesn’t fit.
Why the Compliance Question Separates the Two Options
Compliance fluency, in this context, means two specific things: knowing which regulation applies to your business, and knowing what that regulation actually requires when AI begins touching the data it covers. Those are not the same skill. A provider who understands HIPAA in general may not have thought through what HIPAA requires when a staff member uses an AI tool to draft clinical notes or summarize patient records. That gap — between knowing a regulation exists and knowing how AI creates new exposure under it — is where the meaningful difference between providers shows up.
A consultant engaged specifically for AI strategy was not brought in to learn your regulatory environment. Most arrive without having mapped it, because that work wasn’t part of the engagement scope. An MSP already responsible for your IT infrastructure is in a different position. The compliance obligations your business carries didn’t change when AI arrived. An MSP doing its job has already built that mapping — which frameworks apply, which data is in scope, and where the boundaries are. AI becomes one more layer on top of an existing foundation rather than a subject being learned from scratch.
Here’s what this looks like in practice. MIS has published detailed AI governance documentation that explicitly maps AI-specific controls against HIPAA, CMMC, PCI DSS, and the emerging NIST AI RMF and ISO 42001 standards.
>>Download Our AI Governance Policy
>>Download Our AI Usage Policy
That kind of documentation is the evidence to ask for, not a general assurance that compliance is taken seriously.
That’s also what question three in the framework is designed to surface. A provider with genuine compliance fluency can answer it directly. One without it will default to abstractions.
What the Answers Tell You
A mixed scorecard is the normal outcome. Few providers, whether existing MSP or prospective consultant, will have a strong answer to all six questions in a first conversation, and that’s worth accounting for before you draw a conclusion.
Not all gaps carry equal weight. Question three, the compliance mapping question, matters more than the others for the reasons laid out in the previous section. A weak answer there — abstractions about AI risk with no reference to the specific frameworks your business operates under — is a more significant signal than a weak answer on tool approval processes, for example.
Some gaps close quickly. Others don’t.
What you do with mixed result matters. The FAQ section below addresses the most common scenarios.
Frequently asked questions
Can an MSP that fails several of these questions actually close the gap?
Yes, but the timeline depends on which gaps. Tool-level controls and formal approval processes can be built in a matter of weeks. These are operational decisions, not deep expertise. Regulatory fluency takes longer because it’s built through sustained work in a specific client environment, not a one-time project. A weak answer on compliance mapping today is a more meaningful gap than a weak answer on tool approvals.
Does hiring an AI consultant mean replacing your MSP entirely?
Not necessarily. Some businesses run both in parallel: a consultant engaged for a specific initiative alongside an MSP handling ongoing infrastructure and security. The two aren’t mutually exclusive. The six questions are designed to clarify which provider, if either, can actually be trusted with AI governance responsibility specifically. That’s a narrower question than who handles the rest of IT.
How long should it take to get straight answers to these six questions?
One direct conversation should be enough to tell the difference between a provider with real answers and one who is improvising. A provider with genuine depth in this area can speak to specifics, including data retention policies, named compliance frameworks, and actual tool decisions, without needing to research and follow up. Vague answers that require a second meeting to substantiate are themselves a signal.
Is a smaller business too small to need this level of scrutiny?
The opposite is closer to the truth. Smaller businesses without dedicated AI or compliance staff are the most exposed when these questions go unasked, because there’s no internal function to catch a bad decision before it becomes a regulatory or security problem. The less internal oversight a business has, the more the provider’s competency matters.
Where to go from here
If you’d like a second opinion after running through these six questions, that’s a conversation MIS is glad to have. No pressure to reach a particular conclusion — just a straightforward discussion about where your current situation stands and what the answers actually mean for your business.
