TL;DR: Before you build a training plan, run a short scenario test on three roles: customer-facing, internal, and data-handling staff. Ask three judgment-based questions per role, score each answer zero to two points, and the total tells you whether you need the full four-week training plan, a lighter policy refresh, or nothing more than what you already have.
You can tell whether your team has real AI literacy by testing judgment on realistic scenarios, not by asking who has used ChatGPT the most. A role-based diagnostic you can run in one sitting shows you where people would catch a bad AI answer and where they would not, and that gap is the thing a training plan actually needs to close. Skip straight to a training plan without checking this first, and you risk spending a month teaching people a skill they already have while the real risk, usually sitting with one or two specific people, goes untouched.
What Does AI Literacy Actually Mean for a Small Business Team?
AI literacy gets treated like a vocabulary test: can someone define a large language model, explain what a token is, or name the difference between two AI assistants. None of that tells you whether an employee will catch a wrong answer before it reaches a customer. Amberjack's research on measuring AI literacy inside companies makes the same point: literacy is confidence in using AI, judgment in interpreting what it produces, and the ability to work with the output rather than accept it on sight (How Companies Can Measure AI Literacy). A person who has never opened an AI tool but checks everything a coworker hands them may carry more real literacy than someone who uses AI every day and never questions what comes back.
This distinction matters because most small business owners assess the wrong thing. They ask who is already using AI tools, assume the enthusiastic users are ahead of everyone else, and build a training plan around spreading that enthusiasm. Enthusiasm and judgment are different skills, and only judgment protects the business when a tool is confidently wrong. Our four week AI training plan for a small team already makes this distinction once training starts; this diagnostic answers the question that has to come first, which people need that plan and where their gaps actually sit.
Why Should You Test Literacy Before You Build a Training Plan?
A training plan built on guesses wastes the one resource a small team cannot spare: staff time. Assume the whole team needs the same four weeks of practice, and you will bore the person who already double-checks every AI draft while under-serving the person who does not, because a generic plan treats both the same. Amberjack also reports that organizations running structured AI upskilling see close to double the return on their AI investment compared to those that do not (weareamberjack.com), and that number only holds if the upskilling actually targets the gap instead of repeating what people already know how to do.
A short diagnostic also tells you which risk you are actually carrying. A customer-facing team that already checks every AI draft before it goes out has a different problem than one that pastes customer messages into a general chatbot without a second look. Running a handful of scenario questions, instead of a month of training, shows you which problem you have before you commit real staff hours to fixing it.
What Is the Three-Role Diagnostic, and How Do You Run It?
Build the diagnostic around three roles, not three skill levels, because the risk a person carries depends on what they touch, not how comfortable they are with a keyboard. Customer-facing staff carry the risk of a wrong answer reaching a paying customer. Internal and admin staff carry the risk of a wrong assumption quietly changing a schedule, a job posting, or a meeting record. Anyone who handles customer records, financial data, or other sensitive information carries the risk of that information ending up somewhere it should not. One person can sit inside more than one role; test them once for each role they genuinely fill.
This role split borrows an idea from aicompetence.org's own workplace assessment: twelve scenario-based questions scored zero to two points each, built to test decisions in real work situations rather than technical vocabulary (Workplace AI Literacy Assessment Scenarios). We shortened that down to three scenarios per role, nine total, because a small team needs something an owner can run in one sitting, not a formal audit with its own project plan.
Here is the procedure we run with a small team, start to finish.
- List every person on the team and mark which of the three roles they fill: customer-facing, internal, or data-handling. Mark more than one role for anyone who genuinely fills more than one.
- Pull the three scenario questions for each role from the next section, or write your own versions using your team's actual tools and a fictional example, never a real customer record.
- Sit down with each person one at a time, read the scenario aloud, and ask what they would actually do next, not what the written policy says they should do.
- Score each answer zero, one, or two using the rubric below, and write down the reason for the score next to it, not just the number.
- Add up each person's total and note it beside their name and role, so you can see patterns across the team instead of one isolated result.
- Run the same three scenarios with every person in that role before moving to the next role, so you are comparing like with like.
- Hold the raw scores for a week before changing anything. A rushed moment can drag one person's score down, and a second short conversation on a different scenario is worth more than reacting to a single number.
What Scenario Questions Should You Ask Each Role?
Below are nine scenarios, three for each role, built around situations a small business actually runs into rather than a technical quiz. Read each one exactly as written, and hold back from explaining what the right answer is before the person responds.
Customer-facing staff
- A customer asks your AI chat tool about a return policy that was never written down anywhere, and the tool answers confidently anyway. What do you do before replying to the customer?
- An AI-drafted email to a customer includes a promise, such as matching a competitor's price, that nobody approved. Do you send it, edit it, or hold it?
- A customer's message is angry and could mean either a refund or a cancellation. The AI suggests one canned reply. What do you check before you send anything?
Internal and admin staff
- AI summarized today's team meeting and listed an action item nobody actually agreed to. What do you do with those notes before anyone else sees them?
- An AI-drafted job posting includes a phrase that could read as excluding older applicants. What do you do with that draft?
- The AI-built schedule has two people assigned to the same shift and never flagged the conflict. What do you check before you publish it?
Data-handling staff
- You want a quick summary of a spreadsheet full of customer names and account numbers. What do you do before pasting any of it into an AI tool?
- An AI bookkeeping tool flags a transaction as a likely duplicate and offers to delete it. What do you do before accepting that suggestion?
- A vendor asks you to confirm a customer's account balance, and the fastest way to answer is to forward an AI-generated chat summary. What do you do before you send it?
How Do You Score the Diagnostic, and What Does Each Band Mean?
Score every answer on the same three-point scale, regardless of role, so the numbers stay comparable across the team.
| Score | What the answer looks like |
|---|---|
| 2 | Names a specific check: a written policy, a real record, or a named person consulted before acting |
| 1 | Shows caution or hesitation but does not name a specific check |
| 0 | Accepts the AI output and acts on it with no check at all |
With three scenarios per role, the highest possible score for any one role is six, and a person tested across all three roles can reach a combined total of eighteen. Add up whichever total applies to how many roles that person fills.
| Total score | What it tells you |
|---|---|
| 14 to 18 | Judgment is already in place. A lighter check-in beats a full training plan. |
| 8 to 13 | Judgment is uneven. Target the specific scenario where the score dropped, not the whole team. |
| 0 to 7 | Judgment gaps are real. Start with the four week plan before expanding AI use for that role. |
A score of zero on any single scenario deserves attention on its own, separate from the total. Amberjack's framework points at the same idea when it separates tool familiarity from judgment, interpretation, and the ability to work with AI-generated output rather than defer to it (weareamberjack.com); a person can score well on two scenarios and still have a specific blind spot worth fixing before it shows up with a real customer or a real spreadsheet.
Diagram: A one-page scorecard with three columns, customer-facing, internal, and data-handling, three scenario rows under each column, a circle to mark zero, one, or two for every row, and a total box at the bottom shaded to match the three score bands above.
What Should You Do With a Low, Mixed, or High Score?
A low score in one role is a reason to start the four week AI training plan for a small team with that role first, using the exact task where the scenario exposed the gap as week one's material. Do not run the plan on the whole team just because one person scored low; a team-wide plan built around one person's gap wastes the time of everyone who already answered well.
A mixed score, where a person answers two scenarios well and misses one, usually points at a single habit rather than a broad skill gap. Run a short practice round on just that scenario type instead of the full plan: redo scenario eight with a different bookkeeping example, or scenario two with a different customer promise, and see whether the same gap shows up again before committing a full month of training to it.
A high score across a role means that role is ready to move straight to writing or refreshing the rules rather than practicing them. Use the one page AI use policy for a small team to put what the diagnostic already confirmed into writing, and use the small business AI policy guide to turn each approved task into a card with a named reviewer, so the judgment you just measured has a place to live beyond one conversation.
How Does This Diagnostic Fit With Your AI Policy and Governance?
The diagnostic, the training plan, and the written policy are three different tools for three different moments, and confusing them wastes effort. The diagnostic tells you where the gaps are. The training plan, when a role actually needs it, closes a specific gap using that role's own recurring work. The policy and task cards hold the result so a new hire or a new tool does not quietly undo what the first two steps built. Our broader AI training for employees overview walks through how those pieces connect once you are training more than one role at a time.
If your business is a service company running its own sales team, the stakes around customer-facing and data-handling judgment go up, because an AI agent may be the one actually talking to a lead before a human ever does. MetaTechAi's guide to what an AI governance framework should cover for a service business sales team walks through the heavier version of this same idea once AI is running inside the sales process itself, not just drafting internal notes (AI governance framework for a service business sales team).
What Should You Do With This Before Next Monday?
Pick the three roles that exist on your team, write out or adapt the nine scenarios above, and set aside one afternoon to run them person by person. Score each answer as you go rather than from memory afterward, and resist the pull to round a one up to a two because someone is clearly trying hard; the score has to reflect the check they actually named, not their effort. The scores you collect this week are what turns next month's training plan, if you need one, from a guess into a decision.
If you want a second set of eyes on your results, or you are not sure how to score a borderline answer, contact AI Guy and bring the raw scenario answers, not a summary of how the afternoon went.
What Are the FAQs About Testing AI Literacy?
How long does this diagnostic take to run with a small team?
Plan on roughly ten to fifteen minutes per person: one short conversation per role, three scenario questions, and a score you write down on the spot. A team of eight or nine people fits into one afternoon.
Does a high score mean we can skip AI training altogether?
No. A high score means you can skip the full four week training plan and move straight to writing or refreshing a one page AI use policy, because the judgment the plan would build is already there. Keep checking new hires and new tools as they arrive.
Does running this diagnostic satisfy the EU AI Act's AI literacy requirement?
No. This diagnostic is a practical starting point for spotting gaps and recordkeeping, not a certification or a legal compliance step. For businesses whose AI use falls within the EU AI Act's scope, Article 4 adds an AI literacy obligation. The European Commission's AI literacy guidance explains the context; check your systems, role, and operating context with qualified help.
What if the same employee scores differently depending on the day?
Treat one low score as a flag, not a verdict. Ask the same person a second scenario from the same role on a different day before you decide they need extra practice, since a rushed moment can drag one answer down without reflecting their usual judgment.
