Expert Insights on Playwright Automation: Butch Mayhew on the Importance of Separating Concerns

‍

In today's fast-paced world of software development, test automation has become an essential part of the development process. However, with the increasing complexity of systems and applications, the challenges of test automation have also grown. One of the key challenges that teams face is the separation of concern in test automation. In this blog post, we will discuss this topic in detail and explore how to overcome it through a conversation with Butch Mayhew, the Head of Quality and Reliability at Tilled and the previous Director of Software Test at Daxko.

‍

Check out Butch's fantastic blog - Playwright Solutions.

‍

Butch's team uses Playwright for their test automation and they focus on automating their API-centric product. Here are some of Butch’s keys:

‍

Separation of Concern

One of the main themes in this conversation is the concept of separation of concern in test automation. We discuss how companies often have the mindset that they should just write end-to-end flows for automation, but this approach quickly becomes a nightmare as the number of test cases increases. Butch suggests that instead of focusing on end-to-end flows, teams should focus on the areas of the product that are most critical and that have the highest risk of breaking. By doing this, they can maintain a more maintainable automation suite and increase the pace of development.

‍

Ephemeral Environments

Butch also shares that they use ephemeral environments to ensure that their tests are up-to-date with development and they get feedback as soon as possible. Ephemeral environments are disposable environments that are created specifically for the purpose of testing and are destroyed after use. This helps teams to ensure that their tests are running against the most recent version of the code and that they are getting feedback on the latest changes as soon as possible.

‍

Shortening the Feedback Loop

Additionally, Butch also discusses the importance of feedback and how the best way to shorten the feedback loop is to start as close to the finish line as possible. He explains that the longer the test flow, the longer it will take to get feedback. This means that teams should focus on creating smaller, more focused test flows that can provide feedback as quickly as possible. This helps teams to identify and fix issues faster and to be more efficient in their testing efforts.

‍

‍

Understand your Product

Next, Butch highlights the importance of understanding the product, the code, the deployment, and the environment. He believes that individual testers should learn as much as possible about these areas to establish themselves as experts and gain the trust of the developers. This includes understanding the way the code is built, the way it gets deployed, and the way the classes work in the programming language that is being used. By understanding these areas, testers can ask more informed questions, look at the commit history, and compare changes between commits. This helps them to understand the system better and to be more effective in their testing efforts.

‍

Augmenting with Exploratory Testing

Butch also acknowledges that in reality, test automation is often done after development has already taken place. This means that teams are always a step or two behind in their testing efforts. But, by focusing on automating the API tests as soon as they are ready to be delivered to production, teams can make sure that they are catching up with the development as quickly as possible. Additionally, they also have a round of exploratory testing to ensure that the computer is not missing anything that's critical to the human eye. This helps teams to ensure that they are not missing any important issues that might have been overlooked in the automated tests.

‍

Ben’s Bonus: QA with Attitude. 

In addition to these technical aspects, I also discussed the importance of QA with attitude. I believe that QA teams are the smartest out of everyone and that they know the product better than anyone else. They understand how the product is actually going to be used, whereas the product people are good at designing how it should go, and the developers are good at following instructions. However, QA teams are often the quietest voice in the room. I suggest that teams should focus on data and how they are delivering their code. By having a conversation with the developers about how many bugs have been prevented and the risks that have been averted, QA teams can establish themselves as a valuable part of the development process. Additionally, by learning more about the code and the way it is built and deployed, individual testers can gain the trust of the developers and become more involved in the development process.

‍

Conclusion

Overall, this conversation highlights the importance of separation of concern in test automation, the use of ephemeral environments, and the need to shorten the feedback loop. Additionally, it emphasizes the importance of QA teams establishing themselves as a valuable part of the development process by focusing on data and learning more about the code. With these concepts in mind, QA teams can ensure that their automation efforts are efficient, reliable, and easy to maintain.

‍

Thank you to Butch Hayhew for an awesome Quality Conversation! 

‍

Once again, check out Butch's fantastic Playwright blog - Playwright Solutions.

‍

‍

‍