XP2016 – Does the Test Automation Pyramid matter and who really cares? – Panel Discussio
At XP2016 there was a panel discussion focused around the “Test Automation Pyramid”
The discussion was facilitated using the “Fish Bowl” technique. The starting panel was:
The discussion kicked off with the following are questions:
So what did we talk about?
The 90 minute discussion covered the following:
All models are wrong, some are useful
Does it matter? Maybe
Who cares? Not sure
It sucks because it ignores other types of automation that support testing. Automation can extend beyond Units, Integration and UI. I thought of Richard Bradshaws comments about “Automation in Testing”
The Exploratory Testing bubble that is sometimes placed on the top is stupid
What is the definition of a unit? No real clarification here
What is the the definition of an Integration Test? Tumble weeds……
People treat items higher up the pyramid as second class citizens. Tests at the top still add value although the feedback is traditionally slower
Pyramid can sometimes be used to assign responsibility. Testers focus on the top(E2E), developers do the bottom(Unit), someone somewhere does tests at the middle (Assuming they know what an Integration Test is)
The pyramid is based upon an old “3 tier architecture” model where the Front End was difficult to test and could only be done using “In Browser” tools such as WebDriver. In todays world, this is not the case. With front end frameworks such as AngularJS the front end can be broken down into Unit Tests.
What is Testing? Talked about the difference about Testing v Checking. Most of the discussion was about Checking but we should also think about Testing which is not currently part of the Test Automation Pyramid. The Pyramid is not about testing.
Uncle Bob Martin talked about how the Test Pyramid is commonly inverted, with most of the business rule tests implemented through the Front End.
There was talk about “Plumbing” and the Test Automation. I forget what we discussed exactly.
Did the Test Automation pyramid come from the “Food Pyramid“?
The focus of the discussion towards the end of the panel discussion moved into documentation
Uncle Bob Martin compared a well written code based to an “Encyclopedia”; It contains a robust collection of state transitions but its hard to search.
The Test Automation Pyramid exists because we like to talk in Metaphors.
Horizontal axis = Amount of Tests
Vertical axis = Amount of Pain
Thats me ^
So….what do you think? Leave a comment below.