Laws of Test Automation

Law of Coverage

Automate frequent or difficult things.

Law of Speed

Speed kills quality and engineer’s time.

Law of ROI

Automation is the most expensive, slowest, and flakiest way to write tests.

Maintenance Principal

Automation exponentially becomes maintenance.

Law of Failures

When a failure occurs, it should be painfully obvious.

Mocking Principal

All tests can run locally.

Dollar Save Club Principal

Money saved on infrastructure costs more in engineering hours.

Boolean Principal

Not all tests are pass/fail.

Law of Automation Quality

Automation quality is inversely proportional to the number of classes, frameworks, and templates used

Law of New Frameworks

New frameworks are only for promotions.

Documentation Paradox

Tests should not need documentation.

Law of logarithms

The best test code is logging.

Law of Difficulty

Test code is the most difficult code to get right.

Law of Flakiness

Tests aren’t flaky, the test code is unreliable.

Law of Testability

Everything is testable with a little creativity.

Law of Frameworks

Frameworks enable bad test code.

Generation principle

Never generate test code.

Law of selection

Test selection adds risk.

Law of Shapes

Ignore pyramids, toroids, and ice cream cones.

Automation paradox

Don’t automate — unless you have to.

Reporting Principle

Other than 100%, percentages aren’t useful.

— Jason Arbon, TestNerd