Dachary Carey
When we released Visual Diffs, it was a big feature that made a small splash. Then we quietly added the ability to define Visual Diff paths in the config.yml file into a small release. It’s time to give Visual Diffs a little of the fanfare they deserve, as this Tugboat feature drives a big proportion of value for code testing and acceptance.
It’s easy enough to review the code in a pull or merge request and say whether or not it does what it’s supposed to. Does it fix the bug or deliver the feature as requested? Is it implemented in a way that doesn’t add too much technical debt, and is maintainable and extendable if needed?
But code reviews done in isolation are like looking at a row of staples and saying: “Yup, those look like staples. They should do a great job of holding things together!” without thinking about the fact that the stapler just got moved three floors away, or you no longer even have a stapler. Particularly when you’re dealing with complex styling or many discrete components within a web app, it’s easy for changes that seem isolated to have a negative impact somewhere in the larger body of code.
This is where visual regression testing comes in. When you’re using Tugboat’s Visual Diffs feature, you can see what a page looked like in the Base Preview versus what it looks like in the PR you’re viewing. When calculating visual regressions, Tugboat goes pixel-by-pixel looking for changes to the page. Sometimes the changes are subtle and easy-to-miss when viewing it with the naked eye, but Visual Diffs call out all the changes on the page, even if a human might miss them.
With the way Tugboat has implemented Visual Diffs, you can define multiple paths for visual regression testing for every PR. This is particularly helpful if you’ve got complex styles and want to see how changes impact different pages, or if you have many discrete components across your site and want to see how a change impacts them in different places. So, for example, you might want every PR to generate visual diffs for the home page, for the blog, and any other pages you want to review for visual regression.
For a more complex project, such as a Drupal multisite project, you may want to use aliases to group URLs for Visual Diffs. This makes it easy to review visual regression across many different brand identities or other alias criteria.
When do you need visual regression testing? Any time you’re making changes. Whether you’re doing feature development or bug fixes, Visual Diffs are equally helpful.
To get the most from this feature, add visual regression testing to your acceptance criteria. If every developer or QA engineer who does a code review looks not just at the code, but at the Visual Diff, your organization can spot issues and reject the PR until those issues are corrected. With visual regression testing as a regular and disciplined part of your code review process, issues won’t slip through to Staging or Production, and you can avoid customer-facing impacts from minor CSS oopsies to major UI/UX issues. Without visual regression testing, these things can and do make it into production code.
Beyond looking for changes that your development team introduces, Visual Diffs can also highlight things that aren’t a result of your changes. What happens when an external vendor or third-party library makes a change that ends up affecting the page layout?
Many things can change page layout without being obviously related to a pull request that’s up for code review. General CSS rules can be overridden by more specific CSS. CSS or JavaScript changes in a library you rely on could start interacting with your page in unexpected ways. Someone doing a code review might say “Sure, this code provides the functionality requested in the ticket” - but without actually seeing it built, no human would realize that it’s breaking things.
Without visual regression testing, the first time an org might notice these issues is when a user files a bug report - which could be days or weeks after the issue starts affecting customers. With Visual Diffs enabled, any PR that modifies a page’s layout would highlight this issue. It would be easy enough to figure out while a code change didn’t necessarily touch this element, something needs to be refactored to correct the issue. This might never be surfaced internally without this feature.
Don’t just take our word for how great Visual Diffs are; we’ve had feedback from developers who use Tugboat for visual regression testing that once they use it on one project, they want it everywhere. It doesn’t matter whether you’re working with back-end developers or front-end developers; there’s value in understanding how changes made in either place affect visual regression. But most of all, the value lies in catching visual regression before code makes it into production, and negatively impacts your customer experience with your website or product.
Tell us about your project, schedule a demo, or consult with a Tugboat Technical Account Executive.
