AI Broke Manual Testing
AI’s entrance into the world of manual software testing has stirred the pot, casting shadows on traditional testing debates. The chatter in the testing world has become strangely quiet. Veteran manual testers are now reassessing their significance as generative AI technology challenges the status quo of software testing.
Manual software testers, the workhorses of the software world, once dismissed AI and automation as cute, but impossible dreaming. They rightly asserted that most automation initiatives yield a negative return on investment (ROI). They believed their humanistic testing skills would be the last line of defense, determining if AI-generated code was ready for real-world use. By the time AI could perform their job, no one else would have a job anyway.
Today, however, generative AI like LLMs and ChatGPT, can create basic test cases significantly faster, cheaper, and with a deeper understanding of various testing approaches have emerged. These AI-driven bots not only know how to code but can emulate user personas, leading to efficient and effective testing — in the hands of experienced and open-minded testers.
The advent of generative AI has unsettled even the most vocal software testers. The ones who mostly theorize rather than practice, have reacted negatively, branding AI as “garbage,” “dangerous,” “fraudulent,” and asserting “it can’t think or test,” — before testing it. Ouch. If AI has broken the logical reasoning and scientific processes of testing experts, what do you think it can do in the field of software testing itself? We all thought the core tenet of the profession was for humans to assess the quality of the software — first. This is an indicator of just how fundamentally generative AI is breaking software testing.
After the initial emotional reaction to generative AI, a kabal of software testing experts are now huddled behind closed doors, grappling with how to respond to generative AI. This technology has rattled seasoned testers, pushing them to strategize anew. Something tells me the new strategy will smell like the old. If generative AI is rattling them to that degree, maybe you should also pay attention.
As a manual tester, you should probably not just wait for some experts to make a predictable declaration that all is fine and they were right all along. Instead, experiment with this new AI for yourself. As with testing, it is crucial to approach it without bias, use critical thinking, and test it not only for its correctness but also from the perspectives of your manager and peers. Put yourself in the shoes of a freshie with no preconceived notions about testing methodologies or specific skills. Engage with the AI, ask questions, challenge it, and consider how it might be beneficial to your work and learning. Heck, tell the AI about yourself and ask how to get a raise or promotion. Even better, the AI won't try to sell you something — yet.
Remember, AI is as good as your prompting. If you ask a question like a novice tester, you’ll get an answer tailored to a novice. And ask follow-up questions like a professional.
Despite initial pushback, it’s crucial to acknowledge the impact AI has already had on various fields. Some experts even warn that AI might soon be potent enough to pose a threat to job security across industries, including software testing. Even those who create it have testified before the U.S. Congress, labeling generative AI as a potential national security threat. Given that, what are the chances the AI can do a little bit of testing?
The emergence of AI has undoubtedly broken the morale of many manual software testers. Some I’ve spoken with said they hope to retire before AI significantly impacts their job. Some smart and energetic testers, however, are embracing generative AI, integrating it into their workflow secretly to avoid managerial backlash.
Certainly, there’s room for human testers who can leverage generative AI. But the landscape is changing swiftly, indicating a significantly different future for manual software testing.
Up next, we’ll explore how AI is disrupting test automation, test management, marketing, and tooling.
— Jason Arbon
