Intro: Motivation

Recently at work, I had a new project to generate insurance quotes from a variety of leads using Playwright and then scrape the resulting quote prices. The goal was to then compare those prices against an API that also gives quotes for the same leads. I thought this was a very interesting application of Playwright's capabilities, so I wanted to blog about it. I can't share the website used to generate insurance quotes with you, however, so I've chosen to quote pizza prices instead. We're going to use Playwright to navigate to Domino's website and go through the steps to get a pizza price. Then we'll scrape that price so that we can have an automated pizza price lookup.

Step 1: Getting to know the structure of the site

The first step in any scraping-related project is to navigate to your chosen site and get to know it a bit. You'll need to understand the structure of the site you're testing/scraping to get the correct info from it. I'll suggest three options for this task: The Chrome Developer tools, Playwright's VSCode extension, and Rayrun.

Chrome Developer tools

This option is great if you don't want to install any extra stuff onto your browser or computer, however, you'll be somewhat limited in the types of selectors your can use. This is because Developer tools allows you to inspect an element, which will give you access to HTML tags and CSS selectors. While these are possible options for selectors, they can be not specific enough, and if the underlying structure of the HTML changes, then they will not work anymore. However, I sometimes will use a CSS selector if I run out of other options for locators (like if one of the other options doesn't give me the desired effect in my test).

To open Developer tools, right-click on any webpage and select "inspect."

VSCode extension

This is a great option if you're already using VSCode as your editor and want something fully integrated with its capabilities. I have a whole article in which I explain how to download Playwright's extension for Visual Studio Code and how to use it. Perhaps the most exciting option is Playwright's Record Test functionality, which allows you to click around a website as a user while the code for those actions is automatically recorded in a test file in VSCode.

Rayrun Chrome extension

Much like the inspector in Chrome's developer tools, in that you're working from your browser, Rayrun displays a wealth of information about each website element when you click on it. In the screenshot, we can see the HTML tree that leads to the selected element, a locator for the element, the XPath, and more. To be completely honest, I only use the locator or occasionally the CSS selector from Rayrun, but the other information might come in handy for your testing endeavors, so it's a great option.

Using your choice of tool for discovering locators, navigate to Domino's website and click around experimentally, taking note of locators for elements you will interact with in your test.

Navigating to the site with Playwright

The first step in our test is to tell Playwright which website we want to navigate to. We can do so with the following code. We'll divide our test into steps to make it easier to debug.

await test.step("Go to Dominos", async () => {
    await page.goto("https://www.dominos.com/en/restaurants");
    // Expect page title to contain substring
    // tells us we're in the right place
    await expect(page).toHaveTitle(
      /Pizza Restaurants Near Me - Find a Nearby Domino's/
    );
  });

We use page.goto to navigate to the correct URL, and then we expect the page to have a title that includes the given substring. This is a way to verify that we've navigated to the correct website.

Select delivery mode and choose store

Next, we need to decide how we're going to get our pizza: takeout or delivery. Since I'm just interested in the pizza price, I'm going to select takeout but feel free to alter the delivery mode for your test. We'll find a store nearby and wait for the URL to change.

await test.step("Select carryout option", async () => {
    await page.getByRole("tab", { name: "Carryout" }).click();
    // enter local zipcode
    await page.getByLabel("ZIP Code").fill("06320");
    await page.getByRole("button", { name: "Find a Store" }).click();
    // wait for the next url before doing more stuff
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/locations/search?type=Carryout&c=06320&s=&locationName="
    );
  });

We'll also have to choose which store to order from, as there are many associated with the entered zip code.

await test.step("Choose store", async () => {
    await page
      .getByRole("link", {
        name: "Store Pickup - 938 Bank Street New London, CT",
      })
      .click();
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/section/Food/category/AllEntrees/"
    );
  });

Build a pizza

Next, we'll build a pizza by clicking on the Build Your Own Pizza button, choosing our pizza size, some toppings, and adding to the cart. I had a little trouble adding the pizza to the cart with the locators Rayrun found for me, so I chose to use CSS selectors instead. The selector ".single-page-pizza-builder__add-to-order.btn", for adding the pizza to the cart, is what I'm talking about. We'll also need to decline extra cheese, which is what await page.locator(".btn.btn--no-thanks.js-stepUpsellClose").click(); is doing.

await test.step("Build pizza", async () => {
    // click on Build a pizza button
    await page
      .getByRole("link", {
        name: "Build Your Own Pizza Watch the pizza of your wildest dreams come to life.",
      })
      .click();
    // choose pizza
    await page.getByText('12"', { exact: true }).click();
    // add toppings
    await page.getByRole("checkbox", { name: "Mushrooms" }).check();
    await page.locator(".single-page-pizza-builder__add-to-order.btn").click();
    // no extra cheese
    await page.locator(".btn.btn--no-thanks.js-stepUpsellClose").click();
  });

Checkout

We'll checkout, which will require us to say no thanks to another upsell attempt. Then we'll wait for the checkout URL.

  await test.step("Checkout", async () => {
    // Checking out
    await page.getByRole("link", { name: "Checkout" }).click();
    // no extra items
    await page.locator(".js-nothanks.upsell--note").click();
    // wait for checkout page
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/checkout/"
    );
  });

Scrape price

Last of all, we'll scrape the price, using another CSS locator, and dump the value to the console.

await test.step("Scrape price", async () => {
    // scrape price
    const price = await page.locator("td.price").innerHTML();
    // see price in console
    console.log(price);
  });

Conclusion and full code

A fun extension exercise to this test would be to make a similar test for another restaurant and compare the prices, to help you decide where you want to order from. If you liked this article and would like to learn more about Playwright, check out my other articles:

• Getting started with Playwright

• Web Scraping with Playwright

Now I'm hungry! Go order some pizza ;)

Here's the full test for getting a pizza price from Domino's:

// @ts-check
const { test, expect } = require("@playwright/test");

test("Get Pizza price", async ({ page }) => {
  await test.step("Go to Dominos", async () => {
    await page.goto("https://www.dominos.com/en/restaurants");
    // Expect page title to contain substring
    // tells us we're in the right place
    await expect(page).toHaveTitle(
      /Pizza Restaurants Near Me - Find a Nearby Domino's/
    );
  });
  await test.step("Select carryout option", async () => {
    await page.getByRole("tab", { name: "Carryout" }).click();
    // enter local zipcode
    await page.getByLabel("ZIP Code").fill("06320");
    await page.getByRole("button", { name: "Find a Store" }).click();
    // wait for the next url before doing more stuff
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/locations/search?type=Carryout&c=06320&s=&locationName="
    );
  });
  await test.step("Choose store", async () => {
    await page
      .getByRole("link", {
        name: "Store Pickup - 938 Bank Street New London, CT",
      })
      .click();
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/section/Food/category/AllEntrees/"
    );
  });
  await test.step("Build pizza", async () => {
    // click on Build a pizza button
    await page
      .getByRole("link", {
        name: "Build Your Own Pizza Watch the pizza of your wildest dreams come to life.",
      })
      .click();
    // choose pizza
    await page.getByText('12"', { exact: true }).click();
    // add toppings
    await page.getByRole("checkbox", { name: "Mushrooms" }).check();
    await page.locator(".single-page-pizza-builder__add-to-order.btn").click();
    // no extra cheese
    await page.locator(".btn.btn--no-thanks.js-stepUpsellClose").click();
  });
  await test.step("Checkout", async () => {
    // Checking out
    await page.getByRole("link", { name: "Checkout" }).click();
    // no extra items
    await page.locator(".js-nothanks.upsell--note").click();
    // wait for checkout page
    await page.waitForURL(
      "https://www.dominos.com/en/pages/order/#!/checkout/"
    );
  });
  await test.step("Scrape price", async () => {
    // scrape price
    const price = await page.locator("td.price").innerHTML();
    // see price in console
    console.log(price);
  });
});