Flaky LiveView test in playwright

Symptoms

I've had this flaky test on CI that tested a liveview page using the phoenix_test_playwright package.

The test went something like this:

test "tag filter narrows the list", %{conn: conn} do
  conn
  |> visit("/archive")
  |> click(test_id("tag-filter-phoenix"))
  |> refute_has("h2", text: "Elixir Basics")
end

The test was rarely failing on the refute_has step, and the page looked like the filter wasn't clicked at all.

Fix

The click was reaching the right element (no Playwright timeout, no strict-mode error), but phx-click never fired server-side - because liveview hasn't connected yet.

The fix was to modify my BrowserCase.visit/2 to check if we're testing a liveview page, and if we are, wait until liveview is loaded - which is indicated by the .phx-connected class being applied to the liveview parent container (data-phx-main).

defmodule BlogExWeb.BrowserCase do
  use ExUnit.CaseTemplate

  alias PhoenixTest.Playwright.Config
  alias PlaywrightEx.Frame

  using opts do
    quote do
      use PhoenixTest.Playwright.Case, unquote(opts)
      import PhoenixTest, except: [visit: 2]

      def visit(conn, path) do
        conn
        |> PhoenixTest.visit(path)
        |> BlogExWeb.BrowserCase.wait_for_live_view_connected()
      end
    end
  end

  def wait_for_live_view_connected(conn) do
    visible? =
      Frame.is_visible(conn.frame_id,
        selector: "[data-phx-main]",
        timeout: Config.global(:timeout)
      )

    if match?({:ok, true}, visible?) do
      _ = PhoenixTest.assert_has(conn, "[data-phx-main].phx-connected")
    end

    conn
  end
end