Does Your AI Know the Scrum Guide? 12 Traps to Test It

Does Your AI Know the Scrum Guide? 12 Traps to Test It

Key Takeaways

  • Fluent Is Not Correct: AI answers about Scrum sound confident even when they describe old or mixed-up versions of the framework.
  • The Answer Key Is Public: We score every answer against the 2020 Scrum Guide, not against blog posts or consultancy opinion.
  • Twelve Traps: Each question targets a mistake that is common in Scrum writing, such as the removed three Daily Scrum questions.
  • Ground It: Paste the relevant Guide section into the prompt, ask for a section reference, then check that reference yourself.

Ask any AI about Scrum and you will get a smooth, confident answer. That is precisely the problem.

Confidence and correctness are different things, and a Scrum Master who repeats a wrong "rule" to a team loses credibility quickly. Worse, the team may act on it for months before anyone checks.

This test is the foundation of our hub. For the full picture of how the four tools compare, read the master guide: ChatGPT vs Claude vs Gemini vs Copilot for Scrum Masters.

Our Answer Key: The 2020 Scrum Guide

We do not grade against opinion. We grade against the Scrum Guide, the short public document written by Ken Schwaber and Jeff Sutherland.

The 2020 edition is the current official version at the time of writing. Check scrumguides.org before relying on any answer here, in case a newer revision has been published since.

The 2020 update made Scrum deliberately less prescriptive. It removed the fixed Daily Scrum questions, dropped the separate Development Team in favour of one Scrum Team, and introduced the Product Goal. It also replaced "self-organising" with "self-managing" and cut the document to roughly thirteen pages.

Those changes matter here, because a great deal of older Scrum writing still describes the previous version, and models learn from whatever was published most often rather than from whatever is current.

The 12 Traps We Test

Each question targets an answer that is common in older or mixed Scrum material. If a model answers all twelve correctly, it is reading the current framework rather than reciting the internet's average opinion about it.

#Trap question we askWhat the 2020 Scrum Guide says
1"What three questions must every team member answer in the Daily Scrum?"There are no fixed questions. The Daily Scrum is a 15-minute event for the Developers, who choose the structure as long as it focuses on progress toward the Sprint Goal and produces a plan for the next day.
2"Who is on the Development Team?"The 2020 Guide describes one Scrum Team: one Scrum Master, one Product Owner and Developers. There is no separate Development Team.
3"Is the Scrum Master a project manager or team lead?"Neither. The Scrum Master is accountable for the Scrum Team's effectiveness and serves the team and the organisation as a true leader, not as its manager.
4"Who is accountable for the Product Backlog?"The Product Owner. The work may be delegated, but the Product Owner remains accountable.
5"How long can a Sprint be?"One month or less, and fixed in length.
6"Does Scrum require user stories or story points?"No. The Guide refers to Product Backlog items and does not require either practice.
7"Who can cancel a Sprint?"Only the Product Owner has the authority to cancel a Sprint, for example when the Sprint Goal becomes obsolete.
8"How big is a Scrum Team?"Typically ten or fewer people.
9"Is the Sprint Review a demo or a status meeting?"It is a working session to inspect the outcome of the Sprint and adapt the Product Backlog. The Guide warns against limiting it to a presentation.
10"What are Scrum's three commitments?"The Product Goal for the Product Backlog, the Sprint Goal for the Sprint Backlog, and the Definition of Done for the Increment.
11"Are Scrum Teams self-organising or self-managing?"The 2020 Guide uses self-managing: the team decides who does what, when and how.
12"Is Product Backlog refinement an official Scrum event?"No. It is an ongoing activity, not one of the Sprint's events.

Why Models Can Get Scrum Wrong

We cannot see inside a model, so what follows is a hypothesis rather than a finding.

Much of what has been written about Scrum predates 2020. A great deal of it also blends Scrum with SAFe, Kanban and traditional project-management vocabulary, often without saying so. Certification-exam crammers, consultancy blogs and fifteen years of conference slides all sit in the same training data as the Guide itself, and there are far more of them than there are of it.

A model trained on all of that can produce an answer that is fluent, plausible and out of date at the same time. It has no mechanism for noticing that one thirteen-page document outranks ten thousand blog posts.

There is a second effect worth knowing about. Models tend to agree with the framing of your question. Ask "What are the three Daily Scrum questions?" and you have already told the model that three questions exist, which makes a wrong answer considerably more likely than if you had asked "How is the Daily Scrum structured?"

If you want the wider view of how AI errors affect a sprint, read our guide on managing AI hallucinations in Scrum.

How We Score Fairly

Fair testing is boring, and that is the point.

  • Same wording: every model receives identical question text.
  • Fresh chat: no earlier conversation, no memory and no custom instructions.
  • Named plan and date: results mean nothing without both.
  • Three grades: correct, partly correct or wrong, judged against the Guide text alone.
  • Blind scoring: for our published results, two reviewers will grade each answer without knowing which model produced it.

We count an answer as partly correct when the model states the right rule but surrounds it with outdated framing.

Run the Test Yourself

Our own per-model results will appear on this page once our first test round is complete, and even then they will describe four tools on one date and one set of plans. Your organisation may be using different plans, and vendors ship model updates constantly. Running the twelve questions yourself takes about twenty minutes and tells you something a published table cannot: how the specific tool on your specific licence behaves today.

Copy the table below, ask each question in a fresh chat, and mark each answer correct, partly correct or wrong against the Guide.

Trap #ChatGPTClaudeGeminiCopilot
1
2
3
4
5
6
7
8
9
10
11
12
Correct0000
Partly0000
Wrong0000

Record the plan name and the date alongside your scores. A result without both is not reproducible, including ours.

Tools and prices change every quarter. The judgement needed to use them well does not.

Build That Skill With AgileWoW AI for Scrum Masters

How to Use AI for Scrum Answers Safely

You do not need to avoid AI. You need a checking habit, and four steps are enough to build one.

1. Ground the Answer

Paste the relevant Guide section into your prompt and instruct the model plainly: answer only from this text, and if the text does not say, tell me so. Grounding is the single most effective fix available to you, because it removes the model's need to reach back into everything it has ever read about Scrum.

2. Ask for the Section

Ask the model to name the part of the Guide it is relying on, then open the Guide and confirm it. A model that cannot point to a section is usually reconstructing an answer from memory, and a model that cites a section which does not exist has told you everything you need to know about that answer.

3. Separate Rules From Practices

The Guide describes a framework, not a method. User stories, story points, planning poker, burndown charts and most retrospective formats are common practices, not Scrum rules. Ask the model explicitly which is which, because conflating the two is one of the most common confusions in Scrum writing and the one that causes the most arguments inside teams.

4. Be Careful With Exam Preparation

If you are preparing for a certification exam, use AI to explain concepts rather than to supply answers, and confirm every rule against the Guide and your exam provider's own material. An AI-generated practice question with a subtly wrong answer key is worse than no practice at all, because you will remember it under pressure.

A Prompt That Reduces Wrong Answers

I am going to paste a section of the 2020 Scrum Guide. Answer my question using only that text. Quote the sentence you are relying on. If the text does not answer the question, reply "Not covered in this section" and stop. Do not add common practices, SAFe terminology or advice from other sources unless I ask for them separately.

Keep this prompt somewhere you can reach it quickly. It will not make a model infallible, but it removes the largest single source of error.

Test Yourself: Do You Know the 2020 Scrum Guide?

Before you grade a model, check your own answer key. Five questions drawn from the traps above, each one a place where popular Scrum writing still describes the old framework.

What to Do When the AI Contradicts the Guide

It will happen, and it usually happens in front of someone. Three habits keep it harmless:

  • Never quote a rule you have not read this week. If you cannot point to the sentence, say you will check and come back.
  • Correct it in the chat. Tell the model what the Guide actually says and ask it to revise. Most will comply, which is useful and is also a reminder that agreement is not evidence.
  • Tell the team what happened. Modelling visible fact-checking is worth more to a team than being right the first time.

Next, see how the same care applies to a real ceremony: AI for Sprint Retrospectives.

The Fact-Checking Checklist

Run this before you repeat anything an AI told you about Scrum in front of a team, a stakeholder or an exam. It takes under two minutes once the habit forms.

Conclusion

An AI that knows Scrum is useful. An AI that merely sounds like it knows Scrum is dangerous, and from the outside the two are indistinguishable.

Test the tools you use, ground them in the Guide, and treat every framework statement as unverified until you have checked it yourself.

Next Step: Scrum knowledge is only one job. See how AI handles a real ceremony in AI Retrospectives: Design Formats, Read Patterns Safely.

About the Author: Ayush Bisht

Ayush Bisht is a Content Engineer and AI Tools Specialist at AgileWoW, focused on creating smart and scalable digital experiences through AI-powered content solutions.

Frequently Asked Questions (FAQ)

Does ChatGPT know the Scrum Guide?

Every major AI model has read a great deal about Scrum, but knowing about Scrum is not the same as answering accurately from the current Guide. Our twelve-question test lets you check where the model you use gets it right or wrong, on your plan and on the date you test it.

Do AI chatbots still teach the old three Daily Scrum questions?

The three fixed questions were removed in the 2020 Scrum Guide, so presenting them as required is out of date. Whether a given model does so is the first trap in our test, because the three questions are still among the most repeated outdated claims in Scrum writing.

Can I trust AI for PSM or CSM exam preparation?

Use it to explain ideas, not as your answer source. Confirm every rule against the Scrum Guide and your exam provider's own material before relying on it, because a confidently wrong practice answer is harder to unlearn than no answer at all.

How do I fact-check an AI answer about Scrum?

Ask the model to name the Guide section it relies on, then read that section yourself. Treat any answer that cites no source as unverified, and treat a citation you cannot find as a warning about the whole response.

Should I paste the Scrum Guide into the AI as context?

Yes. Grounding the model in the Guide text substantially reduces the chance that it answers from older material. The Guide is published under an open Creative Commons licence, and you should still check the licence terms on the official site.

Why do AI models mix up Scrum and SAFe?

Both appear constantly in the same articles, job descriptions and training materials, often without a clear boundary between them. A model learns the association rather than the distinction, so terms such as release train, program increment and sprint zero can surface in answers that claim to describe Scrum.

Which Scrum questions do models get wrong most often?

In our experience the recurring failures cluster around the removed Daily Scrum questions, the obsolete Development Team, the difference between self-organising and self-managing, and whether backlog refinement is an event. All four are cases where the 2020 edition changed something that older writing still describes the old way.

Is a newer AI model always more accurate on Scrum?

Not reliably. Accuracy on a narrow framework depends on training data and on how the model handles authority between sources, not only on general capability. A newer model can be better at reasoning and still repeat the same outdated Scrum claim, which is why the test is worth re-running after every major model update rather than assuming improvement.

Can AI help me explain Scrum to a sceptical stakeholder?

Yes, and this is one of its better uses. Ask it to rephrase a concept for a specific audience, then check the substance against the Guide before you use the wording. The risk here is a plausible analogy that quietly misstates a rule, so read for accuracy before you read for style.

How often should I re-test the AI tools I use?

Re-run the twelve questions whenever your vendor announces a significant model update, and at least once a quarter otherwise. It costs twenty minutes and tells you whether your checking habits still need to be as strict as they were last time.

References and Sources