Skip to content

Port the OSSMC test suite to Playwright - #885

Merged
hhovsepy merged 21 commits into
kiali:epic/playwright-migrationfrom
hhovsepy:playwright-migration
Nov 9, 2026
Merged

hhovsepy merged 21 commits into
kiali:epic/playwright-migrationfrom
hhovsepy:playwright-migration

Conversation

@hhovsepy

@hhovsepy hhovsepy commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Phase 2 of the Kiali Cypress→Playwright migration (kiali/kiali#9712) for OSSMC. The kiali repo has already ported all of its Cypress suites; this ports OSSMC to the same runner and layout, mirroring frontend/e2e/ in kiali.

  • Playwright infra in plugin/: playwright.config.ts (projects setup/core-1/core-2/ambient/waypoint/waypoint-tracing, each excluding @skip-ossmc; data-test as testIdAttribute; blob + JUnit reporting for CI), global-setup/auth.setup.ts for the Console OAuth login (htpasswd IdP), and yarn scripts mirroring the kiali repo (playwright:run:junit, playwright:run:test-group:junit, playwright:combine:reports, ...).
  • Vendored kiali e2e tree in plugin/e2e/{tests,pages,utils,fixtures}, synced from kiali epic/playwright-migration (via [playwright] Skip K8s Inference Pool list in OSSMC mode kiali#10432 — frontend/e2e exists only on that branch until it merges). The sync rewrites two kiali-isms: import.meta → __dirname (plugin package is CommonJS) and the demoApps hack path → _output/kiali/hack/istio.
    • Note: plugin/src/kiali and plugin/cypress are intentionally not re-synced here — they remain on the contrast-modes lineage (kiali PR 10284, synced via Sync contrast-modes Kiali frontend for PF glass/felt/high contrast #864), which the OSSMC-owned console code depends on (utils/AppearanceUtils). The epic branch does not contain that work yet; the lineages converge once both merge to kiali master.
  • PLAYWRIGHT_OSSMC=true is set by the config: the vendored tree branches on it for OSSMC UI differences (/projects detail links) and OSSMC-skipped tests.
  • OSSMC-owned suite in plugin/e2e/ossmc/: sidebar_navigation.spec.ts (all 9 Cypress scenarios), OssmcSidebarPage (console nav, guided-tour dismissal, /ossmconsole/<page> URL asserts), fixtures extending the vendored kialiFixtures, and ensureKialiHackScripts() for demo app install.
  • CI container: deploy/docker/Dockerfile-playwright + container-build/push-playwright-tests targets (kiali-ossmc-playwright-tests image) and a job in test-images-creator.yml. KIALI_BRANCH is overridable for the vendoring step (kiali master has no frontend/e2e until [playwright] Skip K8s Inference Pool list in OSSMC mode kiali#10432's branch merges).
  • Cypress stays in place until cutover; the existing kiali-ossmc-tests Jenkins job is unaffected. A separate kiali-ossmc-playwright-tests Jenkins job runs this suite.

Depends on kiali/kiali#10432 (vendored from that ref; the sync commit records the exact kiali commit).

Test plan

  • yarn tsc --noEmit (plugin) clean — CI type check
  • yarn build (plugin) compiles — CI build image step
  • yarn tsc -p e2e --noEmit clean
  • Test discovery: core-1 154 (145 vendored + 9 OSSMC sidebar), core-2 38, ambient 10; no leakage into waypoint projects
  • PLAYWRIGHT_OSSMC=true set by config; vendored isOssmc branches verified
  • Full green run on the kiali-ossmc-playwright-tests Jenkins job against a console deployment
  • Cutover (remove Cypress) in a follow-up once CI is green

🤖 Generated with Claude Code

@hhovsepy hhovsepy self-assigned this Oct 22, 2026
@hhovsepy hhovsepy added the do not merge PR is not yet ready label Oct 22, 2026
@hhovsepy
hhovsepy changed the base branch from main to epic/playwright-migration October 22, 2026 14:29
@hhovsepy hhovsepy removed the do not merge PR is not yet ready label Oct 22, 2026
Align OSSMC with PatternFly 6.7 packages and keep yarn resolutions so transitive PF dependencies stay on a single version.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hhovsepy
hhovsepy requested review from ferhoyos and jshaughn October 26, 2026 14:03
@hhovsepy hhovsepy moved this from 📋 Backlog to 🏗 In progress in Kiali Sprint 26-15 | Kiali v2.35 Oct 26, 2026
@jshaughn jshaughn moved this from 🏗 In progress to 👀 In review in Kiali Sprint 26-15 | Kiali v2.35 Oct 26, 2026
* Copy of Kiali frontend source code
Kiali frontend source originated from:
* git ref:    v2.34
* git commit: f74abbb75e4ddbc014b874561df26ec49cbaec2e
* GitHub URL: https://github.com/kiali/kiali/tree/f74abbb75e4ddbc014b874561df26ec49cbaec2e/frontend/src

Signed-off-by: kiali-bot <kiali-dev@googlegroups.com>

* Release v2.34.0

* Prepare for next version

---------

Signed-off-by: kiali-bot <kiali-dev@googlegroups.com>
Co-authored-by: kiali-bot <kiali-dev@googlegroups.com>

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hhovsepy This PR did not run any CI, is that expected?

@hhovsepy

Copy link
Copy Markdown
Contributor Author

Hi @jshaughn I have included the CI run as well, and the jenkins job .../job/kiali/job/test-jobs/job/kiali-playwright-tests/ is also passing.

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure yet if this is actually running the OSSMC tests correctly. The upstream CI doesn't really run tests. And I'm not sure which Jenkins job ran them, the test reports I saw did not obviously seem to include results for the OSSMC tests.

Also, I think there are some issues to resolve. I have created a sub-pr that I think resolves the following issues:

  • Incorrect Console page routes.
    • If I understand correctly I think tests navigate to standalone Kiali routes instead of Console routes.
  • API requests bypass the OSSMC proxy

See hhovsepy#1

Comment thread plugin/playwright.config.ts Resolved
Comment thread deploy/docker/Dockerfile-playwright Resolved
Comment thread plugin/playwright.config.ts Resolved
ferhoyos and others added 13 commits October 30, 2026 13:10
kiali#887)

Align workflow_dispatch defaults with supported release branches.

Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 1 of the OSSMC Cypress → Playwright migration (kiali#9712):
- @playwright/test + playwright.config.ts with projects mirroring the
  Cypress run scope (@core-1/@core-2/@ambient/@waypoint/@waypoint-tracing,
  each excluding @skip-ossmc)
- OpenShift Console OAuth auth setup (htpasswd IdP) persisting storageState;
  OSSMC always runs on OpenShift so there is no strategy detection
- Per-suite blob + JUnit reporting with combine script and Jenkins
  TEST_TAGS test-group runner, matching the Cypress junit chaining
- e2e tsconfig, README, gitignore for playwright artifacts

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sync kiali frontend/e2e (tests, pages, utils, fixtures) into plugin/e2e
from kiali epic/playwright-migration (kiali#9712 OSSMC phase):

- hack/copy-frontend-src-to-ossmc.sh copies frontend/e2e alongside the
  existing src/cypress sync, skipping gracefully on refs without it
  (master) until the kiali migration merges
- sync rewrites kiali's ESM import.meta dirname helper to the __dirname
  CJS global (the plugin package is CommonJS) and points demoApps at
  _output/kiali/hack/istio, where hack/download-hack-scripts.sh puts the
  kiali istio scripts
- OSSMC-owned files move under e2e/ossmc/ so the vendored wipe cannot
  clobber them; the vendored utils/auth.ts replaces the local copy
- vendored dirs are eslint-ignored and excluded from the e2e tsconfig
  (kiali does not typecheck frontend/e2e either); they stay prettier-clean

Discovery parity with kiali at sync time: core-1 145, core-2 38,
ambient 10, waypoint 38, waypoint-tracing 14 tests.

Co-Authored-By: Claude Code <noreply@anthropic.com>
First OSSMC-owned spec (e2e/ossmc/): the port of the cypress
sidebar_navigation.feature. It navigates the plugin through the OCP
Console left nav (ossmconsole routes) and verifies the embedded Kiali
UI renders — overview cards, traffic graph, mesh side panel and the
list pages — doubling as the plugin-mode verification for the vendored
kiali e2e tree.

Demo-app setup mirrors the cypress hooks: the kiali istio hack scripts
are downloaded on demand (hack/download-hack-scripts.sh) before the
vendored ensureDemoApp('bookinfo') runs.

The e2e tsconfig now also excludes the ossmc spec layer: importing the
vendored tree pulls in kiali frontend/e2e files with latent type errors
(kiali does not typecheck them), so only the OSSMC-owned utilities
remain covered.

Co-Authored-By: Claude Code <noreply@anthropic.com>
- playwright.config.ts sets PLAYWRIGHT_OSSMC=true before the specs load,
  activating the vendored kiali OSSMC branches (UI differences like
  /projects detail links, and the OSSMC-skipped tests).
- Re-sync the vendored e2e tree: the K8s Inference Pool list test now
  skips in OSSMC mode (upstream kiali guard).
- New deploy/docker/Dockerfile-playwright and container-build/push-
  playwright-tests targets publishing kiali-ossmc-playwright-tests,
  mirroring the cypress image (browsers installed at build time;
  PLAYWRIGHT_GREP/GREP_INVERT replace CYPRESS TAGS semantics).
- The KIALI_BRANCH used to vendor the kiali frontend into the test
  images is now overridable: while the Playwright migration lives on
  kiali epic/playwright-migration (not yet on master), build with
  KIALI_BRANCH=epic/playwright-migration. The test-images-creator
  workflow does that for master builds.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Kiali frontend source originated from:
* git ref:    epic/playwright-migration
* git commit: b95860cbf9ce3baea437b70f0e287484ad9286aa
* GitHub URL: https://github.com/kiali/kiali/tree/b95860cbf9ce3baea437b70f0e287484ad9286aa/frontend/src

Rebased onto upstream main: the previous sync vendored PR 10432
(46b060b85), which predated kiali master's v2.34 content that main now
vendors. The epic/playwright-migration branch has since been rebased
onto kiali master (#10435), so this sync carries current master content
plus the Playwright overlay, resolving the conflict with main's v2.34
vendored sync.

Signed-off-by: Hayk Hovsepyan <hhovsepy@redhat.com>
The previous commit re-synced the whole vendored tree from kiali
epic/playwright-migration, which also replaced plugin/src/kiali
and plugin/cypress. Those trees must stay on the kiali master
lineage that upstream main vendors (v2.34, contrast-modes work from
PR 10284 via OSSMC PR kiali#864 included) — the OSSMC-owned console code
depends on it (utils/AppearanceUtils in KialiController.tsx).

Restore plugin/src/kiali and plugin/cypress to their pre-sync state
(upstream main's v2.34 vendored sync) and keep only the plugin/e2e
tree from the epic sync: the e2e tree exists only on
epic/playwright-migration and is self-contained (it does not import
the vendored frontend src). The two lineages converge when both merge
to kiali master.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Jenkins build 3 of kiali-ossmc-playwright-tests timed out in auth.setup
waiting for #inputUsername while the browser sat on the "Log in with" IdP
selection page. The OpenShift Console renders the IdP list client-side
after the SPA boots, but clickAuthProviderIfPresent probed it with
isVisible(), which returns immediately (the timeout option is ignored) -
the probe ran ~44ms after load, the list was not rendered yet, the click
was skipped, and the login form never appeared.

Replace the probe with a waiting getByRole('link').click(), which
auto-waits for the list to render; a click timeout means there is no
selection page (active session or single-IdP redirect). Reproduced the
race locally with a delayed-render mock: old pattern stuck on the
selection page, new pattern reaches the login form.

Same fix applied to the vendored utils/openshift-auth.ts to match the
kiali-side source of that file.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The href*="/ossmconsole/istio" prefix match also matched the
OSSMC_INTERNAL_TECH_PREVIEW /ossmconsole/istios nav item when the
OSSMConsole CR opts into tech preview, making the locator resolve to
two links and fail Playwright strict mode. Anchor the match on the
full href instead.

Co-Authored-By: Claude Code <noreply@anthropic.com>
PRs targeting an epic/* base branch (e.g. epic/playwright-migration)
were not running CI because the branches filter only included main
and v*.*.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Signed-off-by: Jay Shaughnessy <jshaughn@redhat.com>
Signed-off-by: Jay Shaughnessy <jshaughn@redhat.com>
Playwright clears the whole blob outputDir at the start of every run, so
chained suites sharing playwright/blob-report kept only the last suite's
zip regardless of the unique file names. Give each suite its own
directory and stage the per-suite zips into one directory for
merge-reports, which only scans the single directory it is given.

Co-Authored-By: Claude Code <noreply@anthropic.com>
hhovsepy and others added 3 commits November 3, 2026 14:51
Project-level grep replaces the top-level value instead of merging with
it, so PLAYWRIGHT_GREP=@core-1 still ran the core-2, ambient, waypoint
and waypoint-tracing projects - suites needing incompatible cluster
configurations. AND the requested filter into each project's suite tag
and union the requested invert with @skip-ossmc instead. The setup
project now runs unfiltered, so playwright-test-group.mjs no longer
needs a separate unfiltered setup invocation.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Copying plugin/ directly into /tmp/kiali placed demoApps.ts at
/tmp/kiali/e2e/utils, where its repository-root calculation resolves to
/tmp, and the hack scripts landed where no helper looks for them. Copy
plugin and _output as real subdirectories so the e2e utils resolve the
repo root, hack/ and _output/kiali/hack as in a checkout, and run yarn
from plugin/ like the local dev flow.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The playwright job pinned a nonexistent actions/checkout commit, and
automatic builds from OSSMC main selected the nonexistent Kiali main
branch - map both main and master to the kiali epic/playwright-migration
branch while the Playwright migration lives there. Also trigger the
latest-image build on Dockerfile-playwright changes, not just the
Cypress one.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@hhovsepy
hhovsepy force-pushed the playwright-migration branch from 9a594d3 to 2eadc9a Compare November 3, 2026 14:45

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three remaining findings from the latest review. The current revision already addresses the missing CSRF headers and waypoint URL assertions.

Comment thread plugin/e2e/ossmc/utils/console-auth.ts Resolved
Comment thread plugin/e2e/pages/NamespaceDetailPage.ts Resolved
Comment thread deploy/docker/Dockerfile-playwright Resolved

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @hhovsepy , It's looking good, a lot if issues resolved. I'm not exactly sure which jenkins job to look at to ensure everything is working as expected. I posted 3 more AI-review issues inline, but you can dismiss them if they don't seem correct. Let me know when you are satisfied and I will approve.

…r default

- console-auth: skip the OAuth login when the Console serves the SPA
  directly (local bridge started with BRIDGE_USER_AUTH=disabled);
  credentials are only required when the OAuth redirect happens
- NamespaceDetailPage: register the namespace graph response wait
  before the navigation so the response cannot arrive before the
  listener
- Dockerfile-playwright: default the container to the core suites
  (PLAYWRIGHT_GREP="@core-1|@Core-2"), mirroring the Cypress image;
  the ambient/waypoint suites need cluster configurations the default
  target does not have

Co-Authored-By: Claude Code <noreply@anthropic.com>
@hhovsepy
hhovsepy requested a review from jshaughn November 5, 2026 05:36
@hhovsepy

hhovsepy commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @jshaughn seems all comments are resolved now.

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big PR, very close. Found two more things that seemed like they could be important. Just merge in the sub-pr and assuming CI passes I'll approve.

hhovsepy#2

Fix Console login readiness and preserve Playwright image test sources
@hhovsepy

hhovsepy commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks Jay, I merged your PR, and the second bug was an important catch , it was introduced in the beginning of the implementation when I was trying to optimize the CI time.

@hhovsepy
hhovsepy requested a review from jshaughn November 6, 2026 09:54

@jshaughn jshaughn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, nice job, Hayk!

@hhovsepy
hhovsepy merged commit 13c992e into kiali:epic/playwright-migration Nov 9, 2026
2 checks passed
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in Kiali Sprint 26-15 | Kiali v2.35 Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants