AI Test Coverage Analysis
AI delivers test coverage analysis — and even improves it. While engineering teams continuously add tests to prevent regressions and find issues that need fixing, they often lack visibility into their overall coverage effectiveness, not sure what is missing, and have only ad-hoc strategies for expansion of coverage. AI can help!
Using screenshots and page-related data, AI generates coverage analysis that can quantify existing test effectiveness. The AI creates inventories of page features and user interactions, evaluates current test coverage, and calculates coverage percentages to help the team get a high-level understanding of how much is done — and how much more is needed. AI makes it easy and quick to processes tests in diverse formats including Gherkin, TestRails, spreadsheets, XML, and JSON files and deliver a picture of the existing, and needed test coverage.
As part of the analysis, AI also identifies and eliminates duplicate test cases, streamlining testing efficiency.
At testers.ai, we’ve added an interesting AI-first approach and flow to evaluate coverage across manual, automated, and AI-generated test cases for web applications:
- Overall Test Coverage Metrics
- Comprehensive Coverage Analysis
- Critical Coverage Gap Identification
- Strategic Recommendations for High-Impact Test Creation
- Automated Test Generation for Immediate Coverage Enhancement
Let’s look at the AI Coverage Analysis for the STAREAST Software testing conference home page. The AI Testing Agents had generated over 100 test cases for the website. The AI first examined the page via screenshot and HTML/DOM and identified key features and user interactions on the page.
The AI then considered the existing test cases and how well they ‘covered’ the functional test needs of the page.
The AI first generated an estimate of test coverage around 70%. The AI then generated additional test cases for areas missing coverage, and then re-evaluated the test coverage which was now about 85%.
Not only did the AI figure out what the coverage level was — it was able to increase the test coverage by the next run by generating additional tests.
The second coverage analysis run generated a percent-estimate of how much of the needed coverage is complete.
The AI also generated a text summary and overall analysis. Reading through hundreds of test cases and summarizing test coverage not only takes a lot of time for people, but is often inaccurate — if it is ever done at all.
AI Coverage Summary
The current test suite covers most of the major elements and functional aspects of the webpage including Navigation, Links, Button Styling, Text, Images, Discount verifications, and Accessibility.
AI Coverage Analysis
The provided test names cover a significant portion of the landing page’s functionality, including navigation, link validation, text verification, and style checks for key elements like buttons and headings. There is also a suite of tests dedicated to accessibility. Navigation to external pages like social media is verified. The test suite also covers group discount related tests.
The AI also lists test coverage that is still missing and recommends specific tests for either another round of AI test generation, people to explore, or, ignore because they think it is ‘good enough’ :)
The best part is this coverage analysis can now work in minutes on any webpage and any set of test cases. This should help engineering teams understand the coverage they have and efficiently and intelligently add more coverage to speed up delivery of software with confidence in quality.
Of course AI isn’t perfect, but neither are us humans! The good news is AI Coverage Analysis is fast enough and cheap enough that most every team can now have access to decent test coverage analysis.
Download and have the AI Testing Agents test your website for FREE @ https://testers.ai/
— Jason Arbon, founder testers.ai
