Playwright Best Practices and Ecosystem
Working with Playwright Test annotations
Playwright annotations are metadata markers that instruct the test runner to modify execution behavior based on specific environmental or functional conditions. They are essential for managing flaky tests, handling known bugs, and optimizing suite performance by selectively ignoring or tagging tests. You should use them to maintain a clean test suite by documenting technical debt and ensuring that test reports remain accurate during development cycles.
Skipping and Failing Tests
Annotations like @skip and @fixme are critical for maintaining the health of your test repository when you encounter unavoidable environment issues or known product defects. When you apply `test.skip()`, the runner completely ignores the test case, which is useful when a feature is currently disabled in the environment. Conversely, `test.fixme()` indicates that a test is expected to fail due to a known bug; the runner will attempt to run it but expects a failure, ensuring that if the test unexpectedly passes, you are alerted to the regression. Understanding these is vital because they communicate developer intent to the runner, preventing false negatives in your CI/CD pipeline. By distinguishing between intentionally skipped tests and those undergoing active repair, you maintain a clearer signal-to-noise ratio in your test reporting, which is crucial for large-scale projects.
import { test } from '@playwright/test';
// Explicitly skip this test because the feature is down
test('feature-x login', async ({ page }) => {
test.skip(true, 'Feature X is temporarily disabled');
await page.goto('/login');
});
// Mark as fixme to track a known bug without cluttering failures
test('checkout flow', async ({ page }) => {
test.fixme();
await page.click('#checkout');
// Expected to fail until the bug is resolved
});Conditional Execution based on Environment
Often, your test environment varies in capabilities, such as running tests only on specific browsers or operating systems. Annotations like `test.skip()` or `test.only()` can be passed conditions to make execution logic dynamic. By using boolean logic within the annotation, you instruct the test runner to dynamically decide whether to execute the test code based on the current context. This is fundamentally different from a static skip, as it allows your suite to adapt to the infrastructure. For example, you might avoid executing complex file-upload tests on systems that lack specific storage drivers. Understanding this conditional logic allows you to write resilient, cross-platform suites that fail gracefully in unsupported environments rather than crashing or providing misleading results. This is the bedrock of building a scalable automation framework that operates reliably across diverse CI environments.
import { test } from '@playwright/test';
// Only run this test on Linux environments
test('system-specific utility', async ({ page }, testInfo) => {
test.skip(testInfo.project.name !== 'linux', 'Only supported on Linux');
await page.goto('/system-settings');
// Test logic follows
});Tagging and Filtering with Annotations
Annotations go beyond just skipping; they allow you to categorize tests into logical groups using metadata, which facilitates precise execution control from the command line. By using the `test.use()` or manual annotation, you can apply custom labels to tests, effectively creating 'tags'. This capability is indispensable when you need to run only a subset of your suite, such as 'smoke', 'regression', or 'slow' tests. The reasoning here is that as your codebase grows, running thousands of tests for every minor commit becomes inefficient. By tagging your tests, you enable granular control, ensuring that your developer feedback loop remains fast and focused. This strategy leverages the test runner’s ability to filter by metadata, turning your test suite into an organized set of modules that can be triggered independently based on current development priorities.
import { test } from '@playwright/test';
// Annotating a test for filtering
test('critical billing flow', async ({ page }) => {
test.info().annotations.push({ type: 'tag', description: 'smoke' });
await page.goto('/billing');
// Critical checkout logic
});
// Run via: npx playwright test --grep smokeHandling Slow Tests
Performance is a first-class citizen in test engineering, and sometimes specific tests are inherently slow due to third-party dependencies or complex data setups. The `test.slow()` annotation tells the test runner to triple the default timeout for a specific test file or case. This is a deliberate choice: rather than increasing the global timeout for every test—which can hide actual hangs in fast tests—you selectively grant more time to the specific procedures that require it. This mechanism works by adjusting the watchdog timer within the runner, allowing legitimate, time-consuming operations to complete without being forcefully aborted. By using this annotation, you demonstrate a deep understanding of your application's performance profile, ensuring your suite remains robust under varied latency conditions without compromising the overall responsiveness of your automation framework's execution cycles.
import { test } from '@playwright/test';
// Increase timeout for a resource-heavy data generation step
test('generate heavy report', async ({ page }) => {
test.slow(); // Extends timeout by 3x
await page.goto('/reports/generate');
// This page takes a long time to build the PDF
await page.waitForLoadState('networkidle');
});Custom Metadata for Reporting
Modern test management often requires integrating with external dashboards or custom reporting tools. Playwright allows you to push custom annotations at runtime, which are then exposed in the test results JSON report. This works by injecting metadata into the test's `info` object. The reasoning behind this is that raw pass/fail results are rarely sufficient for stakeholders who need context regarding why a test was executed, which features were validated, or what environment parameters were captured during the run. By appending custom annotations, you transform your test output from a simple boolean list into a rich data stream that can be ingested by analysis tools. This architectural decision enables you to create sophisticated monitoring solutions, linking automation outcomes directly to business requirements and technical specs with complete traceability.
import { test } from '@playwright/test';
// Add custom metadata for reporting systems
test('ui audit', async ({ page }) => {
test.info().annotations.push({
type: 'jira-ticket',
description: 'PROJ-123'
});
await page.goto('/audit');
// Perform audit logic
});Key points
- Annotations provide a way to alter test execution behavior dynamically during runtime.
- Use test.skip() when a test is known to be irrelevant or unsupported in the current context.
- The test.fixme() annotation signals that a test is expected to fail due to an existing bug.
- Conditional logic within annotations allows tests to adapt to different browser or operating system environments.
- Tagging tests with custom metadata enables efficient filtering of test suites for faster development cycles.
- The test.slow() annotation extends the default timeout for tests that are known to be resource-intensive.
- Custom annotations help bridge the gap between technical results and project management reporting requirements.
- Effective use of annotations keeps the test suite maintainable by documenting the rationale for test behaviors.
Common mistakes
- Mistake: Using test.skip() on a test inside a describe block that is already skipped. Why it's wrong: Redundant code increases technical debt and can cause confusion about why a test is inactive. Fix: Use a single condition or logic at the describe level if the entire suite is meant to be skipped.
- Mistake: Forgetting that test.fixme() still reports as a skipped test in the final report. Why it's wrong: Developers often assume fixme runs but fails, but it actually prevents the test from executing. Fix: Use test.fail() if you want the test to run and expect a failure, or fixme only when the test should not run at all.
- Mistake: Misunderstanding the scope of test.describe.configure({ mode: 'serial' }). Why it's wrong: Applying this to an entire file forces sequential execution globally, which severely slows down CI pipelines. Fix: Limit serial execution only to specific test blocks that share state dependencies.
- Mistake: Using test.slow() to increase the timeout for an unstable network request. Why it's wrong: test.slow() triples the current timeout, which is a blanket approach that masks performance issues rather than addressing them. Fix: Use test.setTimeout() to set a specific, controlled limit for the duration of the unstable operation.
- Mistake: Overusing annotations when environment variables could suffice. Why it's wrong: Hardcoding skips based on platforms using test.skip({ condition }) is less maintainable than controlling suites via configuration settings. Fix: Use configuration files to filter tests by tag or environment rather than hardcoding logic inside test files.
Interview questions
What is the primary purpose of using Playwright annotations in your test scripts?
Playwright annotations are metadata markers used to control the execution flow and reporting of your tests. They allow you to dynamically instruct the test runner on how to handle specific scenarios without changing the underlying test logic. For instance, using annotations helps you skip tests that are not yet ready, mark expected failures to prevent build errors, or focus only on specific subsets of tests, ensuring your automation suite remains maintainable, clean, and highly informative.
How do you implement the 'test.skip' annotation, and when should you use it?
You implement the 'test.skip' annotation by calling it directly within your test block, like 'test.skip(({ browserName }) => browserName === 'firefox', 'Skipped on Firefox');'. You should use it when you know a feature is currently incompatible with a specific environment, browser, or platform. It prevents flaky tests by avoiding execution in unsupported scenarios, keeping your test reports accurate and ensuring that irrelevant failures do not distract developers from addressing actual regressions.
What is the specific use case for 'test.fixme' versus 'test.skip'?
While both skip execution, 'test.fixme' is specifically designed for tests that are known to be broken and require immediate maintenance. When you mark a test as 'fixme', Playwright will not execute it, but it will explicitly mark it in your test report as a task that needs resolution. 'test.skip', conversely, is generally used for planned exclusions, such as unsupported browser versions or features not yet implemented in a specific staging environment.
Compare using 'test.slow()' to increase timeout limits versus globally configuring timeouts in the configuration file.
Using 'test.slow()' is a surgical, per-test approach suitable for a single heavy operation, like a long database seed or complex data processing, that significantly exceeds the standard test duration. You simply call 'test.slow()' inside the test. In contrast, modifying the global configuration timeout affects all tests, which is safer for consistent environments but inefficient if only one test is slow. Use 'test.slow()' for anomalies; use global configuration for standard infrastructure performance baselines.
How does the 'test.fail()' annotation function, and why is it useful for verifying bug fixes?
The 'test.fail()' annotation tells Playwright that you expect a test to fail. If the test passes despite the annotation, Playwright will throw an error. This is incredibly useful for 'test-driven bug fixing': you write a test that reproduces a known bug, mark it with 'test.fail()', and once the developers fix the issue, the test will begin failing its expectation—telling you exactly when it is time to remove the annotation.
Explain how to conditionally apply annotations using logic inside a test file.
You can apply annotations conditionally by passing a function to the annotation method that evaluates the test environment or configuration. For example, 'test.skip(process.env.CI === 'true', 'Skipping in CI environment')' allows you to avoid running unstable local tests in your continuous integration pipeline. This pattern is powerful because it keeps your test code dry and readable while allowing the test runner to decide dynamically at runtime whether a test should be executed based on external environment variables.
Check yourself
1. Which annotation is best suited for a test that is currently broken but you intend to fix in the near future, while wanting to ensure it is not counted as a regression failure?
- A.test.skip()
- B.test.fixme()
- C.test.fail()
- D.test.slow()
Show answer
B. test.fixme()
test.fixme() marks a test as needing attention and skips execution without failing the build. test.skip() ignores the test completely, test.fail() expects a failure (which would flag an issue if the test passes), and test.slow() only changes timing, not execution status.
2. If you want to intentionally run a test and verify that it results in a 'failed' status to confirm a bug fix, which annotation should you use?
- A.test.fail()
- B.test.skip()
- C.test.fixme()
- D.test.only()
Show answer
A. test.fail()
test.fail() is designed for 'expected failure'; if the test passes, it will actually trigger an error. skip and fixme do not run the test logic, and only runs specific tests in isolation.
3. When you need to triple the default timeout for a specific test suite because it performs intensive data processing, which annotation is the most idiomatic choice?
- A.test.timeout()
- B.test.fixme()
- C.test.slow()
- D.test.fail()
Show answer
C. test.slow()
test.slow() is specifically provided to triple the timeout for tests known to be slow. test.timeout() is not a valid annotation (you would use test.setTimeout inside the test), while the others relate to execution status rather than timing.
4. How does using test.only() affect the execution of a test file during local development?
- A.It skips all other tests in the file and runs only the marked test.
- B.It runs only the marked test and skips all other tests in the entire project.
- C.It marks the test as the only one to be run in CI.
- D.It forces the test to run in serial mode regardless of other settings.
Show answer
A. It skips all other tests in the file and runs only the marked test.
test.only() isolates the test file to run only the marked tests. It does not affect other files in the project or CI environment configuration directly, and it does not force serial execution; it merely filters the test set.
5. You have a test that should only execute on Chromium-based browsers. Which approach correctly handles this using an annotation?
- A.test.skip(browserName !== 'chromium', 'Only for Chromium');
- B.test.fail(browserName !== 'chromium');
- C.test.only(browserName === 'chromium');
- D.test.slow(browserName !== 'chromium');
Show answer
A. test.skip(browserName !== 'chromium', 'Only for Chromium');
test.skip() accepts a condition as the first argument, allowing dynamic exclusion. test.fail would expect the wrong browser to fail (not skip), test.only would break the build by excluding other tests incorrectly, and test.slow is for timing.