Two-Shot: Final Evaluation That Builds Smarter Testing for Tomorrow

Introduction: Completing the Testing Cycle and Looking Forward

In the previous articles of this series, we explored the first three phases of the 4-Shot Testing Flow: the fully automated Sub-Zero Shot that provides broad coverage, the Zero-Shot phase where human experts quickly review AI results, and the One-Shot phase where testers conduct deeper exploration of complex flows and edge cases.

While these phases form a powerful testing approach, there’s one more critical step needed to complete the cycle and set the stage for continuous improvement. Enter the Two-Shot phase — the final evaluation that not only validates the current release but also prepares for smarter, more effective testing in the future.

Named after the machine learning concept where an AI is given two examples to learn from, the Two-Shot testing phase represents the most mature level of the AI-human testing partnership. It’s where teams conduct a final round of validation while simultaneously creating assets and insights that will make the next testing cycle even more effective.

In today’s world of continuous delivery and rapid iteration, this forward-looking approach is essential. The Two-Shot phase ensures not only that the current release meets quality standards but also that each testing cycle builds upon the last, creating a continuously improving quality process that keeps pace with development.

The Two-Shot Phase: 30% AI, 70% Human, ~Day

The Two-Shot phase represents the culmination of the testing process, combining final validation with preparation for future testing. Let’s explore what happens during this critical stage:

Final Round of Human + AI Testing

The Two-Shot phase begins with a final round of testing that combines human expertise with AI assistance. Unlike the One-Shot phase, which focuses on exploring specific areas of concern, this final testing round takes a more holistic view, ensuring that all critical aspects of the application have been adequately validated.

This testing typically includes:

• Creation of new tests for the next build
• Creation of a new Testing Agents
• Creation of new User Personas

This final round of testing ads confidence that the application will be even better tested on the next version.

Creating Custom Testing Agents

One of the most powerful aspects of the Two-Shot phase is the creation of custom testing agents designed to address quality aspects that weren’t adequately covered in the current build. These custom agents enhance the AI testing capabilities for future releases.

Based on insights gained throughout the testing process, testers identify areas where the AI could be more effective and create specialized agents to address these gaps. For example:

• If the current AI testing missed certain types of visual inconsistencies, a custom agent might be created to focus specifically on design consistency
• If certain business rules weren’t adequately validated, a custom agent could be designed to focus on those specific rules

These custom agents become part of the testing arsenal for future releases, continuously improving the effectiveness of the Sub-Zero Shot phase.

Developing Virtual User Personas

Another forward-looking aspect of the Two-Shot phase is the creation of custom virtual user personas for qualitative testing on the next build. These personas represent different types of users with specific characteristics, goals, and behaviors.

By creating these personas, teams can ensure that future testing considers diverse user perspectives and needs. For example:

• A “novice user” persona might focus on first-time user experiences and basic functionality
• A “power user” persona might explore advanced features and efficiency workflows
• An “accessibility-dependent user” persona might focus on screen reader compatibility and keyboard navigation
• A “mobile user” persona might emphasize touch interactions and responsive design

These personas guide both AI and human testing in future cycles, ensuring that the application works well for all intended users.

Creating Natural Language Test Cases

The final key activity in the Two-Shot phase is the creation of custom “natural language” or “prompt-based” test cases for execution on the next build. These test cases describe multi-step functionality and additional scenarios in plain language that can be understood by both humans and AI.

Unlike traditional test scripts, which often focus on specific actions and expected results, these natural language test cases describe scenarios in terms of user goals and behaviors. For example:

• ”A user should be able to search for a product, add it to their cart, and complete checkout without creating an account”
• ”A content editor should be able to create a new article, save it as a draft, preview it, make changes, and then publish it”
• ”A user with low vision should be able to increase the text size and navigate through the main sections of the application using keyboard shortcuts”

These natural language test cases serve as guidance for future testing, ensuring that important scenarios are covered consistently across testing cycles.

Great virtual AI Testing Agents should have access to all pages covered in the app, with a screenshot, console logs, network traffic and the Document Object Model.

The Value Proposition: Strong Coverage Now, Smarter Tests Next Time

The Two-Shot phase delivers a unique dual value proposition: strong coverage for the current release and smarter tests for the next cycle. This translates to several key benefits:

1. Continuous Improvement

The forward-looking aspects of the Two-Shot phase — creating custom agents, developing user personas, and writing natural language test cases — ensure that testing gets smarter with each cycle. Rather than starting from scratch each time, teams build on their previous testing knowledge and assets.

This continuous improvement approach allows testing to keep pace with development, even as applications grow in complexity and scope.

2. Knowledge Transfer

The artifacts created during the Two-Shot phase serve as a form of knowledge transfer, capturing testing insights in a format that can be shared across the team and used in future testing cycles. This is particularly valuable in organizations with distributed teams or those that experience regular personnel changes.

By documenting testing approaches, user personas, and important scenarios, teams create a lasting testing knowledge base that grows more valuable over time.

The Forward-Looking Advantage: Building Testing Intelligence

What sets the Two-Shot phase apart is its focus on building testing intelligence — creating assets and insights that make future testing more effective. Here’s how this forward-looking approach benefits organizations:

Accelerating the Testing Flywheel

Each time a team completes the Two-Shot phase, they add to their collection of testing assets — custom agents, user personas, and natural language test cases. These assets make the next testing cycle more efficient and effective, creating a flywheel effect where testing continuously improves.

Over time, this flywheel can dramatically reduce the effort required to achieve comprehensive test coverage while simultaneously improving the quality of testing results.

Adapting to Application Evolution

As applications evolve, testing needs to evolve with them. The assets created during the Two-Shot phase help testing keep pace with application changes by providing a flexible framework that can be adapted to new features and requirements.

For example:

• Custom testing agents can be updated to cover new functionality
• User personas can be expanded to include new user types or behaviors
• Natural language test cases can be modified to incorporate new workflows
• Sometimes deleting tests if they aren’t relevant any longer

This adaptability ensures that testing remains relevant and effective even as the application changes significantly over time.

Building Organizational Testing Maturity

Beyond improving testing for a specific application, the Two-Shot phase contributes to overall organizational testing maturity. The insights and approaches developed during this phase can be applied across multiple projects, raising the quality bar throughout the organization.

As teams become more experienced with the 4-Shot Testing Flow and build larger libraries of testing assets, they develop a sophisticated testing capability that becomes a competitive advantage in delivering high-quality software quickly.

Implementing Two-Shot Testing

If you’re interested in implementing the Two-Shot testing phase in your organization, consider these best practices:

1. Balance Validation and Preparation

The Two-Shot phase serves dual purposes: validating the current release and preparing for future testing. It’s important to balance these objectives, allocating sufficient time and resources to both aspects.

A common mistake is to focus exclusively on current release validation, neglecting the forward-looking activities that make future testing more effective. To avoid this pitfall, explicitly schedule time for creating custom agents, developing user personas, and writing natural language test cases.

2. Prioritize Based on Impact

When creating assets for future testing, prioritize based on potential impact. Focus on:

• Areas where the current testing cycle revealed gaps or inefficiencies
• Functionality that’s critical to user experience or business outcomes
• Features that are likely to change or expand in future releases
• Testing approaches that could be reused across multiple projects

This prioritization ensures that your forward-looking efforts deliver maximum value in future testing cycles.

3. Involve Cross-Functional Perspectives

The Two-Shot phase benefits from diverse perspectives. Consider involving:

• Testers with different specialties (functional, security, performance, etc.)
• Developers who can provide technical insights
• Product managers who understand user needs and business priorities
• UX designers who can contribute to user persona development
• Customer support representatives who know common user challenges

These varied perspectives help create more comprehensive and effective testing assets for future use.

4. Create Reusable, Adaptable Assets

When developing testing assets during the Two-Shot phase, focus on creating resources that are both reusable and adaptable. Avoid overly specific implementations that might become obsolete as the application evolves.

For example:

• Design custom testing agents around general quality aspects rather than specific features
• Create user personas based on fundamental user characteristics and goals
• Write natural language test cases that focus on user objectives rather than specific UI implementations

This approach ensures that your testing assets remain valuable even as the application changes over time.

Conclusion: Closing the Loop and Looking Forward

The Two-Shot phase represents both the conclusion of the current testing cycle and the beginning of the next. It’s where teams validate that their application is ready for release while simultaneously preparing to test more effectively in the future.

This dual focus — looking both at the present and the future — is what makes the 4-Shot Testing Flow so powerful. Rather than treating each testing cycle as an isolated event, the approach creates a continuous improvement loop where testing gets smarter and more efficient with each iteration.

As software development continues to accelerate and user expectations continue to rise, this forward-looking approach becomes increasingly essential. The Two-Shot phase ensures that testing isn’t just a quality gate for the current release — it’s an investment in better quality for all future releases.

Together, the four phases of the 4-Shot Testing Flow — Sub-Zero, Zero-Shot, One-Shot, and Two-Shot — create a comprehensive approach to modern software testing that combines the best of AI automation and human expertise. This hybrid model delivers the speed, coverage, and depth needed to ensure quality in today’s fast-paced development environment.

By adopting this approach, teams can transform testing from a bottleneck to a competitive advantage — delivering higher quality software faster and more efficiently than ever before.

Jason Arbon is the CEO of testers.ai and a leading voice in AI-powered testing. This article is part of a series exploring the 4-Shot Testing Flow.