←

Building a Readable DSL for Playwright Tests in F#

Jannik Buschke

Introduction

Lately I've been working on end-to-end tests and really liked the readability of Maestro tests. Maestro is a testing framework for mobile applications and uses YAML format. Here is an example:

Maestro Example

appId: org.wikipedia.wikipedia
---
- launchApp
- assertVisible: "Welcome to Wikipedia"
- runFlow: subflows/onboarding.yaml
- tapOn: "Year In Review"
- assertNotVisible: "Sign in to edit"
- inputText: "qwerty"
- assertVisible: "qwerty"
- eraseText
- inputText: "London"
Copied!

YAML is quite readable and works well for simple data structures. For more complex things maybe less so.
As I'm also working lately with Playwright in .NET, I wondered whether it is possible to make these tests as readable as Maestro tests, while having a proper programming language?

TLDR

Raw Playwright in F#

Here is a typical Playwright test in F#:

do! page.GotoAsync("https://localhost:5001")
do! page.GetByRole(AriaRole.Link, PageGetByRoleOptions(Name="Sign in"))
        .ClickAsync()
do! page.GetByLabel("Username").FillAsync("jannik")
do! page.GetByLabel("Password").FillAsync("secret")
do! page.GetByRole(AriaRole.Button, PageGetByRoleOptions(Name = "Sign in"))
        .ClickAsync()
do! Expect(page.GetByRole(AriaRole.Button, PageGetByRoleOptions(Name = "Create game")))
        .ToBeVisibleAsync()
Copied!

Not bad per se, but quite verbose. Every line needs page. and do! (or let!), many functions will have an async suffix and locator/interaction invocations with arguments are a bit verbose.

Page Locator Record (PLR)

To avoid spelling out locators inline in every test, we can make them reusable. Let's collect them in a record — and call it a Page Locator Record (PLR):

type HomePage = {
    SignInLink: ILocator
}
type LoginPage = {
    UsernameField: ILocator
    PasswordField: ILocator
    SigninButton: ILocator
}

let homePage (page: IPage) = {
    SignInLink = page.GetByRole(AriaRole.Link, PageGetByRoleOptions(Name = "Sign in"))
}
let loginPage (page: IPage) = {
    UsernameField = page.GetByLabel("Username")
    PasswordField = page.GetByLabel("Password")
    SigninButton = page.GetByRole(AriaRole.Button, PageGetByRoleOptions(Name = "Sign in"))
}
Copied!

The test now reads:

let home = homePage page
let login = loginPage page
do! page.GotoAsync("https://localhost:5001")
do! home.SignInLink.ClickAsync()
do! login.UsernameField.FillAsync("jannik")
do! login.PasswordField.FillAsync("secret")
do! login.SigninButton.ClickAsync()
Copied!

Better — locators are named and reusable.

Looking at the PLR, every line uses the page value; can this be simplified? If every property is a function IPage -> ILocator, then we could defer getting to the actual ILocator. Let's create some helper functions:

[<AbstractClass; Sealed>]
type Locators() =
    static member GetByLabel (text: string) (page: IPage) =
        page.GetByLabel(text)

    static member GetByRole role options (page: IPage) =
        page.GetByRole(role, options)

    static member Button text =
        Pw.GetByRole AriaRole.Button (PageGetByRoleOptions(Name = text))

    static member Heading text =
        Pw.GetByRole AriaRole.Heading (PageGetByRoleOptions(Name = text))

    static member Link text =
        Pw.GetByRole AriaRole.Link (PageGetByRoleOptions(Name = text))

    static member Label text =
        Pw.GetByLabel text

    // ...
Copied!

The PLR no longer mentions page at all:

type Locator = IPage -> ILocator

type HomePage = {
    SignInLink: Locator
}
type LoginPage = {
    UsernameField: Locator
    PasswordField: Locator
    SigninButton: Locator
}

open type Locators

let homePage = {
    SignInLink = Link "Sign in"
}
let loginPage = {
    UsernameField = Label "Username"
    PasswordField = Label "Password"
    SigninButton = Button "Sign in"
}
Copied!

Each field is a function waiting for an IPage. The page will be supplied later.

Now we improved readability in the PLR, but can't easily use it in the test. We would need to pass page on every interaction and assertion to get ILocator.

Computation Expression

Each test line still has some noice:

do! home.SignInLink.ClickAsync()
Copied!

do! home and Async() are just noise. A custom Computation Expression can hide them.

Every test step has the same shape:

We capture this in one helper:

type State<'plr> = Task<'plr>

let act (state:State<'plr>) (getLocator:'plr->Locator) (page:IPage) (f:ILocator->Task) =
    task {
        let! plr = state
        let locator = getLocator plr page
        do! f locator
        return plr
    }
Copied!

The Custom Computation expression constructor is initialized with IPage and the initial PLR, then it can manage both values for us.

type UiTestBuilder<'initialPage>(page: IPage, plr: 'initialPlr) =
Copied!

A custom operation removes the "Async" suffix and just uses the previously seen helper.

    [<CustomOperation("fill", MaintainsVariableSpaceUsingBind = true)>]
    member _.Fill(state: State<'page>, locator, text: string) =
        act state locator _.FillAsync(text)
Copied!

Then we create custom operations for all playwright functions we want simplified...

type UiTestBuilder<'initialPage>(page: IPage, plr: 'initialPlr) =

    let act (state:State<'plr>) (getLocator:'plr->Locator) (f:ILocator->Task) =
        task {
            let! plr = state
            let locator = getLocator plr page
            do! f locator
            return plr
        }

    let expect f (locator: ILocator) = f (Assertions.Expect(locator))

    [<CustomOperation("fill", MaintainsVariableSpaceUsingBind = true)>]
    member _.Fill(state: State<'page>, locator, text: string) =
        act state locator _.FillAsync(text)

    [<CustomOperation("click")>]
    member _.Click(state, locator) =
        act state locator _.ClickAsync()

    [<CustomOperation("expectVisible", MaintainsVariableSpaceUsingBind = true)>]
    member _.ExpectVisible(state, getLocator) =
        act state getLocator (expect _.ToBeVisibleAsync())
Copied!

We also need some CE plumbing in the form of Yield and Run:


    member _.Yield(()) : State<'initialPlr>  =
          Task.FromResult plr

    member _.Run(state: State<'page>) =
            state
Copied!

Yield seeds the state with the initial PLR.
Run runs the workflow and returns the PLR.

A test now looks like this:

do! pageTest page loginPage {
    fill _.UsernameField "jannik"
    fill _.PasswordField "secret"
    click _.SigninButton
}
Copied!

No do!, no page., no Async suffix.

Context-Aware Tests: Changing the PLR on Actions

One problem remains: after clicking SigninButton we are on a different page, but the PLR is still loginPage. To fix this, navigating Locators will be enhanced with a transition — a pointer to the next PLR:

type Transition<'t> = Locator * (unit -> 't)
Copied!

A Transition<HomePage> is a tuple of the locator and a thunk returning the next PLR, in this case HomePage.

type LoginPage = {
    UsernameField: Locator
    PasswordField: Locator
    SigninButton: Transition<HomePage>
    //            ^^^^^^^^^^^^^^^^^^^^ 
    //            clicking this takes us to HomePage
}
and HomePage = {
    SigninLink: Transition<LoginPage>
    CreateGameButton: Transition<CreateGameModal>
}

let rec login () = {
    UsernameField = Label "Username"
    PasswordField = Label "Password"
    SigninButton = Button "Sign in", home
    //             ^^^^^^^^^^^^^^^^  ^^^^
    //             Locator           Next Plr
}
and home () = {
    SigninLink = Link "Sign in", login
    CreateGameButton = Button "Create game", createGameModal
}
Copied!

Two more helpers alongside act handle the transition cases. One that actually executes the transition:

// execute the action and advance to the next PLR
let applyTransition (state:State<'plr>) (getTransition:'plr->Transition<_>) (page:IPage) (f:ILocator->Task) =
    //                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    //                                   this helper works only on Transition elements 
    task {
        let! plr = state
        let locator, next = getTransition plr
        do! f (locator page)
        return next ()
        //     ^^^^^
        //     returning the next plr changes the state/context of the CE
    }

// read the locator from a Transition but stay on the current PLR (for assertions)
let applyWithoutTransition (state:State<'plr>) (getTransition:'plr ->Transition<_>) (page:IPage) (f:ILocator->Task) =
    //                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    //                                          this helper works only on Transition elements 
    task {
        let! plr = state
        let locator, _ = getTransition plr
        do! f (locator page)
        return plr
        //     ^^^
        //     Even though we the helper acts on a Transition,
        //     this helper does not execute it, it just returns the original PLR/state 
    }
Copied!

The CE gets a second overload for each operation — one for Locator, one for Transition:

    // plain locator — PLR unchanged
    [<CustomOperation("click")>]
    member _.Click(state, locator) =
        act state locator _.ClickAsync()

    // transition locator — PLR advances
    [<CustomOperation("click")>]
    member _.Click(state, getTransition) =
        applyTransition state getTransition page _.ClickAsync()

    // plain locator - PLR unchanged
    [<CustomOperation("expectVisible")>]
    member _.ExpectVisible(state, getLocator) =
        act state getLocator (expect _.ToBeVisibleAsync())

    // assertions never navigate, so the Transition overload keeps the current PLR
    [<CustomOperation("expectVisible")>]
    member _.ExpectVisible(state, getLocator) =
        applyWithoutTransition state getLocator page (expect _.ToBeVisibleAsync())
Copied!

F# resolves the overload by the type of the locator argument — 'plr -> Locator vs 'plr -> Transition<_>. In other words, when calling click on a Locator property, the PLR does not change, when calling click on a Transition<plr> property, the PLR automatically changes in the CE. From the test syntax perspective this is transparent. However as the context changes, on the next call to any custom operation, typing _. will autocomplete for properties on the new PLR.

Final version

The final test is context-aware. The PLR changes automatically as you navigate:

do! page.GotoAsync("https://localhost:5001")
do! pageTest page (plr.Home()) {
    // PLR is initially Home 
    click _.SigninLink // Transition → PLR will become LoginPage
    fill _.UsernameField "jannik"
    fill _.PasswordField "secret"
    click _.SigninButton // Transition → PLR will become HomePage
    expectVisible _.CreateGameButton
    click _.CreateGameButton // → PLR will become CreateGameModal
    fill _.GameNameField "my-game"
    fill _.MaxPlayersField "2"
    fill _.RoundsField "2"
    click _.CreateButton // → PLR will become GameDetailsPage
    expectVisible _.LobbyHeading
}
Copied!

Each click on a Transition changes the PLR. The compiler enforces that you can only access fields that exist on the current page — you cannot accidentally reference a field from the wrong page state.

Closing

If you want to experiment with this approach, there is a more complete DSL here and here is a bigger example how it is used. You can also feed https://jannikbuschke.de/examples/playwright-dsl/agent-context.md into an agent and ask it to create tests for a given uri.

If you have any thoughts on this feel free to reach out or leave a comment. Thanks for reading!

←
Jannik Buschke