🌏 閱讀中文版本
A Counterintuitive Phenomenon
“AI is going to replace QA!”
You’ve probably heard this many times. Every time a new AI testing tool launches, headlines like this make the rounds.
But if you’ve actually deployed AI testing tools, you’ll notice something counterintuitive:
QA team headcount doesn’t decrease in the short term.
Why?
First, staffing changes are never immediate. Organizations have inertia, existing role definitions, and politics.
But more importantly: AI’s output needs human verification.
AI can complete in one hour what used to take three days of testing. But then what?
- Are these test results correct?
- Did we miss any important scenarios?
- AI says “passed”—can we really go live?
These questions AI cannot answer. Humans must judge.
So the real change isn’t “QA being replaced,” but “QA’s job being redefined.”
That’s what this article is about.
First, Let’s Clarify: What Does QA Actually Do?
Before discussing whether AI can replace QA, we need to break down what QA work actually involves.
Many people think QA is just “running tests.” This is a serious misconception.
QA work can be divided into five types:
Type 1: Execution
The most basic work. Following predefined test cases, running them one by one, recording results, taking screenshots.
This work is highly repetitive, rule-based, and requires little judgment.
Type 2: Writing
Writing test cases, writing automation scripts. Translating test requirements into executable steps.
This requires technical skills, but is essentially “documenting known things.”
Type 3: Design
Deciding “what to test.” Designing test strategies, defining coverage scope, setting priorities.
This requires understanding business logic. You need to know which features matter most, which scenarios are highest risk, what customers care about.
Type 4: Exploratory
Exploratory Testing. Not following scripts, but using intuition and experience to uncover potential issues.
“What if users do this?” “Is this edge case handled?” “This flow feels off, let me try…”
This requires creativity, skepticism, and deep product understanding.
Type 5: Acceptance
Final judgment. Test results are in, but does this mean the product can ship? Is this bug severe or minor? Is this risk acceptable?
This involves accountability. The person who signs off bears responsibility for “releasing.”
Now, let’s see what AI can replace:
| Type | AI Replacement Level | Reason |
|---|---|---|
| Execution | Almost complete | Repetitive work, AI’s strength |
| Writing | Mostly replaceable | AI can write scripts, but needs human review |
| Design | Hard to replace | Requires business knowledge AI doesn’t have |
| Exploratory | Hard to replace | Requires intuition and skepticism; AI doesn’t question itself |
| Acceptance | Cannot replace | Accountability issue; must be human |
The conclusion is clear: AI replaces “execution,” not “judgment.”
If your QA team spends most of their time on execution work, they will be impacted.
But if your QAs can do design, exploration, and acceptance—their value won’t decrease. It will increase.
The Core Question: Who Validates That AI Is Correct?
This is the most important section of this article.
AI testing tool marketing usually sounds like this:
- “AI auto-generates test cases, coverage up 300%!”
- “AI auto-executes tests, saves 80% time!”
- “AI auto-analyzes results, produces detailed reports!”
Sounds great. But let me ask you:
If AI says “tests passed,” would you ship directly?
If your answer is “no,” you understand the problem.
AI can run more tests, produce more reports, cover more scenarios. But this output itself needs validation.
Problem 1: Is AI testing the right things?
AI can auto-generate test cases from code. But how does it know which features matter most to the business? Which scenarios customers complain about most? Where the last incident happened?
It doesn’t. It can only guess based on code structure.
An experienced QA would say: “We need to focus on this payment feature—it broke last time.” AI won’t, unless you explicitly tell it.
Problem 2: When AI says “passed,” is it really passed?
When AI says “passed” after execution, what does that mean?
It means the test script completed, no errors, results matched expectations.
But is the test script itself correct? Are the expected values right? Did we miss any cases?
AI doesn’t question itself. It won’t say “wait, this test doesn’t seem to verify what actually matters.”
Only humans can do that kind of questioning.
Problem 3: When something goes wrong, who’s responsible?
Suppose AI testing says “passed,” you ship, and a serious bug occurs.
Customers complain, your boss demands answers. Can you say “AI said it was fine”?
No.
Humans bear ultimate responsibility, not AI. So humans must make final decisions.
This is why “acceptance” work cannot be replaced. Not because it’s technically impossible, but because accountability requires it.
So what’s QA’s new core job?
Validating AI’s validation.
Sounds circular, but this is reality.
AI produces tests → QA validates tests are correct AI executes tests → QA validates results are trustworthy AI produces reports → QA judges if we can ship
QA shifts from “frontline executor” to “AI supervisor.”
This isn’t a demotion. It’s a promotion.
The Four Most Important QA Skills in the AI Era
Since QA work shifts from “execution” to “judgment,” required skills change too.
I’ll rank them by importance. This ranking might surprise you:
#1: Business Understanding (Most Important)
This is the most counterintuitive.
People assume AI-era skills mean technical ability and prompting. Wrong.
The most important thing is: Do you understand what this product does?
- Which features matter most to customers?
- Which scenarios break most often?
- Which errors are fatal, which are tolerable?
- Where are the business logic edge cases?
This knowledge lives in senior QAs’ heads, not in any documentation, and certainly not in AI’s training data.
AI can help you run tests, but it doesn’t know “what should be tested.” That judgment requires business understanding.
A business-savvy QA can glance at AI-generated test cases and immediately say: “This missed XX scenario—that’s what customers use most.”
A QA without business knowledge can only say: “AI generated 200 test cases, looks like a lot, should be enough.”
That’s the gap.
#2: Critical Thinking
AI doesn’t question itself. It confidently delivers a polished report, even if there are problems.
QA needs the ability to question:
- “Does this test actually verify what we’re trying to verify?”
- “90% coverage sounds high, but what’s in that 10%?”
- “AI says this edge case doesn’t matter—but does it?”
- “This test passed, but the results look weird…”
Critical thinking isn’t “not trusting AI”—it’s “not blindly trusting anything.”
This skill becomes more important in the AI era. Because AI produces high-volume, professional-looking output that easily lowers our guard.
Being able to spot problems in a sea of “looks correct” reports is a high-value skill.
#3: Prompting Skills
This is the most commonly mentioned “AI-era skill.”
It’s important, but overrated.
Prompting is the ability to “get better output from AI.” Knowing how to ask, how to provide context, how to iterate.
But here’s the thing: You need to know what “good output” looks like before you can judge if AI’s output is good enough.
That brings us back to business understanding and critical thinking.
Someone without business knowledge, no matter how elegant their prompts, won’t know if AI’s test cases are correct.
So prompting ranks third. It’s an amplifier, but you need something worth amplifying first.
#4: Technical Skills
This might be the most counterintuitive: technical skills rank last.
Not that technical skills don’t matter. You still need to read test scripts, debug, communicate with engineers, integrate tests into CI/CD.
But AI has dramatically lowered the technical “barrier.”
Before, you needed Selenium, Playwright, handling various environment issues. Now AI can write for you, debug for you, look up documentation for you.
Technical skills went from “scarce resource” to “basic infrastructure.” Necessary to have, but no longer a differentiator.
Summary:
| Rank | Skill | Why Important | Can AI Help? |
|---|---|---|---|
| 1 | Business Understanding | Determines “what to test” | No |
| 2 | Critical Thinking | Judges “is AI correct” | No |
| 3 | Prompting | Gets better AI output | This IS collaborating with AI |
| 4 | Technical | Read, debug, integrate | Can assist |
The first two are “irreplaceable by AI.” The last two are “AI-assistable.”
Counterintuitive conclusion: In the AI era, “soft skills” matter more than “hard skills.”
Practical Impact on Teams
Enough theory. What actually happens?
Short-term (Within 1 year of adoption)
Headcount won’t noticeably decrease.
Reasons I mentioned: organizational inertia, and AI output needs human verification.
But two clear changes will happen:
First, QA report quality improves.
Because AI can run more tests, cover more scenarios. What used to only test happy paths now catches edge cases. What used to be one round is now three rounds.
If QA consciously leverages AI, reports become more precise with broader coverage.
Second, The capability gap between QAs widens.
QAs who use AI: output is 3-5x previous levels, quality is better too. QAs who don’t use AI: output stays the same, looks poor by comparison.
This gap will become increasingly obvious.
Medium-term (1-3 years)
Team structure starts adjusting.
Demand for “execution” QAs decreases. Because AI can execute; you don’t need as many people manually running tests.
Value of “design” and “acceptance” QAs increases. Because AI can’t do these, and demand hasn’t decreased.
Specifically:
A 5-person QA team might originally be: – 4 people doing execution, writing – 1 person doing design, acceptance
After AI adoption, it might become: – 2 people operating AI + validating results – 1 person doing design, acceptance
Headcount from 5 to 3, but output might be higher.
Long-term
The end state isn’t “AI replaces QA,” but “QAs who use AI replace QAs who don’t.”
This is the same as other professions.
Excel didn’t replace accountants—people who use Excel replaced those who only use abacuses. CAD didn’t replace drafters—people who use CAD replaced those who only hand-draw.
Same with QA. AI is a tool. People who use tools replace people who don’t.
Advice for QAs
If you’re a QA, what can you do after reading this?
First, don’t rush to learn Prompt Engineering.
I know this sounds strange. But based on earlier analysis, prompting ranks third, not first.
Ask yourself first: Do I understand this product’s business logic well enough? Do I know what customers care about most? Do I know where things break most often?
If the answer is “not really,” spend time understanding the business first, not learning prompting.
Second, cultivate the habit of questioning.
Starting now, ask one more question about any test result: “Is this really correct?”
Not just AI output. Your own tests, engineers saying “this won’t be a problem,” PMs saying “this case isn’t important”—all worth questioning.
This habit will make you more valuable in the AI era.
Third, learn to collaborate with AI, not compete.
AI executes tests faster than you. That’s a fact. Competing on speed is pointless.
What you should think about: How do I leverage AI’s speed, combined with my judgment, to produce better results than before?
This is collaboration, not competition.
Fourth, accumulate business knowledge—AI can’t take this.
Your understanding of this product, your domain experience, your insights into customer behavior—these live in your head, nowhere else.
AI can learn public knowledge, but not your unique experience.
This is your moat. Keep building it.
Advice for Tech Leads
If you’re a Tech Lead or manager considering AI testing adoption, what should you watch out for?
First, adopting AI testing doesn’t mean you can cut QA.
At least not short-term.
AI output needs validation. Validation needs experienced people. If you cut all QAs, who validates AI?
Also, AI testing tools need people to operate, adjust, maintain. This work still requires humans.
Second, short-term benefit is “improving quality,” not “reducing cost.”
After AI testing adoption, you should expect: – Test coverage increases – Reports become more precise – Discover more issues that weren’t found before
Not: – Can hire two fewer QAs
Cost savings is medium to long-term. Short-term, you might need additional investment in tools, training, process changes.
Third, redefine QA responsibilities.
Since job content changed, role definitions should too.
Old QA responsibilities might be: “Execute test cases, report results”
New responsibilities should be: “Ensure test strategy is correct, validate AI-produced test results, make final quality decisions”
Put this in the Job Description. Let the team know expectations have changed.
Fourth, invest in QA’s business understanding.
Let QA participate in requirements discussions. Let them interact with customers. Let them understand business goals.
These might have been considered “not QA’s concern” before. But in the AI era, this becomes QA’s most important capability.
Worth investing in.
Conclusion
Back to the title: AI won’t replace QA.
But that’s only half true.
The complete statement is:
AI won’t replace QAs “who can judge,” but will eliminate QAs “who only execute.”
AI excels at execution: running tests, writing scripts, producing reports. Demand for this work will sharply decrease.
AI struggles with judgment: what to test, whether results are correct, whether we can ship. Value of this work will sharply increase.
If you’re a QA, start transforming now. From executor to judge. Strengthen business understanding, cultivate critical thinking.
If you’re a manager, redefine team responsibilities. Short-term, use AI to improve quality. Medium to long-term, then discuss staffing adjustments.
Whoever you are, remember this:
It’s not AI replacing you—it’s people who use AI replacing you.
This applies to QA, and to almost all knowledge workers.
If you’re interested in practical AI testing implementation, check out this more technical article: