• Toby Sinclair

XP2016 – Does the Test Automation Pyramid matter and who really cares? – Panel Discussio

At XP2016 there was a panel discussion focused around the “Test Automation Pyramid”

The discussion was facilitated using the “Fish Bowl” technique. The starting panel was:


The discussion kicked off with the following are questions:


So what did we talk about?

The 90 minute discussion covered the following:


"The Test Automation Pyramid" is bullshit @XPConf #xp2016 — Toby Sinclair (@TobySinclair_) May 26, 2016

  1. All models are wrong, some are useful

  2. Does it matter? Maybe

  3. Who cares? Not sure

  4. It sucks because it ignores other types of automation that support testing. Automation can extend beyond Units, Integration and UI. I thought of Richard Bradshaws comments about “Automation in Testing”

  5. The Exploratory Testing bubble that is sometimes placed on the top is stupid

  6. What is the definition of a unit? No real clarification here

  7. What is the the definition of an Integration Test? Tumble weeds……

  8. People treat items higher up the pyramid as second class citizens. Tests at the top still add value although the feedback is traditionally slower

  9. Pyramid can sometimes be used to assign responsibility. Testers focus on the top(E2E), developers do the bottom(Unit), someone somewhere does tests at the middle (Assuming they know what an Integration Test is)

  10. The pyramid is based upon an old “3 tier architecture” model where the Front End was difficult to test and could only be done using “In Browser” tools such as WebDriver. In todays world, this is not the case. With front end frameworks such as AngularJS the front end can be broken down into Unit Tests.

  11. What is Testing? Talked about the difference about Testing v Checking. Most of the discussion was about Checking but we should also think about Testing which is not currently part of the Test Automation Pyramid. The Pyramid is not about testing.


Here is @unclebobmartin getting involved about the testing automation pyramid @XPConf #xp2016 pic.twitter.com/GaKcDJ1eQR — Toby Sinclair (@TobySinclair_) May 26, 2016

  1. Uncle Bob Martin talked about how the Test Pyramid is commonly inverted, with most of the business rule tests implemented through the Front End.

  2. There was talk about “Plumbing” and the Test Automation. I forget what we discussed exactly.

  3. Did the Test Automation pyramid come from the “Food Pyramid“?

  1. The focus of the discussion towards the end of the panel discussion moved into documentation

  2. Uncle Bob Martin compared a well written code based to an “Encyclopedia”; It contains a robust collection of state transitions but its hard to search.

  3. The Test Automation Pyramid exists because we like to talk in Metaphors.

  4. Final comment;

  5. Horizontal axis = Amount of Tests

  6. Vertical axis = Amount of Pain


Nice wrap up from @paulroho "vertical axis of pyramid represents pain over feedback" #xp2016 pic.twitter.com/0L6nGSwTn2 — Seb Rose (@sebrose) May 26, 2016


Thats me ^

So….what do you think? Leave a comment below.

  • LinkedIn
  • Twitter

©2020 by Toby Sinclair.