Unify error handling with getErrorMessage and structured logging, centralize env/host resolution, improve version detection, verify Nextcloud storage success banner, default tests to headless, and document Playwright workflows and test conventions. Added SKILLS.md (Cursor).

Author: Ihor-Khomenko

.cursor/skills/tests/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: tests
+description: E2E test conventions for OpenProject integration tests. Use when writing, modifying, or reviewing Playwright tests, page objects, utilities, or locators in this repository.
+---
+
+# OpenProject E2E Tests Skill
+
+## Architecture Overview
+
+- `tests/`: Playwright spec files (integration flows across OpenProject, Nextcloud, Keycloak).
+- `pageobjects/`: Page Object Model classes wrapping UI interactions.
+- `locators/`: JSON locator definitions for each product (`openproject.json`, `nextcloud.json`, `keycloak.json`).
+- `utils/`: Shared helpers (config, env/hosts, API clients, error handling, logging, version detection, test helpers).
+- `global-setup.ts`: Pre-test setup (Kubernetes setup-job wait, version detection, env var enrichment).
+- `playwright.config.ts`: Playwright runner config (headless by default, workers, retries, etc.).
+
+Page object inheritance:
+- `BasePage` → domain base pages (`OpenProjectBasePage`, `NextcloudBasePage`, `KeycloakBasePage`) → concrete pages (Login, Home, Admin, etc.).
+
+## Locator Rules
+
+- Never put selectors directly into test code; always use locator rules and JSON files in `locators/`.
+- Locator file structure:
+
+```markdown
+{
+  "url": "https://example/",
+  "selectors": {
+    "loginButton": { "by": "getByRole", "value": { "role": "button", "name": "Login" } }
+  }
+}
+```
+
+- Prefer semantic locators: `getByRole`, `getByLabel`, `getByText`, `getByPlaceholder`, `getByTitle`.
+- Use `getByTestId` when `data-testid` attributes are available.
+- Use `locator` (CSS/XPath) only as a last resort when semantic selectors are not possible.
+- Keep locator keys descriptive and stable (e.g., `usernameInput`, `projectSettingsButton`, `storageCreationSuccessMessage`).
+- See `utils/locators_guide.md` for detailed locator patterns and supported `by` values.
+
+## Page Object Conventions
+
+- When adding a new test, first check if an existing page object can be reused or extended before creating a new one.
+- All page objects must:
+  - Extend the appropriate domain base page (`OpenProjectBasePage`, `NextcloudBasePage`, `KeycloakBasePage`) or `BasePage`.
+  - Load locators via the base constructor using the correct locator JSON file.
+  - Use `getLocator(key)` to resolve selectors; do not hardcode selectors inside page objects.
+- Domain base pages:
+  - `OpenProjectBasePage` uses `locators/openproject.json`.
+  - `NextcloudBasePage` uses `locators/nextcloud.json`.
+  - `KeycloakBasePage` uses `locators/keycloak.json`.
+- Export page objects via `index.ts` barrel files per domain for consistent imports.
+- Encapsulate complex flows in page methods rather than duplicating sequences in tests.
+
+## Error Handling Standards
+
+- Always use typed catches:
+
+```ts
+try {
+  // ...
+} catch (error: unknown) {
+  logError("Something failed", getErrorMessage(error));
+}
+```
+
+- Never use `catch (error: any)` or untyped `catch (error)` in new or modified code.
+- Use `getErrorMessage(error)` from `utils/error-utils.ts` to safely extract an error message from unknown values.
+- Avoid truly silent failures:
+  - For expected fallbacks, at least log with `logDebug` or `logWarn`.
+  - For unexpected failures, use `logError` and either rethrow or fail fast depending on context.
+
+## Logging Standards
+
+- Use the central logger from `utils/logger.ts`:
+  - `logDebug(...)`
+  - `logInfo(...)`
+  - `logWarn(...)`
+  - `logError(...)`
+- Do not use `console.log`, `console.warn`, or `console.error` directly in tests, page objects, or utils.
+- Log level is controlled by `E2E_LOG_LEVEL` (`debug`, `info`, `warn`, `error`) and defaults to `info`.
+  - Use `logDebug` for verbose troubleshooting.
+  - Use `logInfo` for high-level lifecycle information (start/end of major flows).
+  - Use `logWarn` for recoverable issues and fallbacks.
+  - Use `logError` for failures that should normally fail the run or require attention.
+
+## Environment and Configuration
+
+- Always resolve environment name and hosts via `utils/env-hosts.ts`:
+  - `resolveEnvName()` chooses the environment (`local`, `edge`, `stage`, etc.) based on `E2E_ENV` or `ENV` (default: `local`).
+  - `resolveHosts(envName?)` returns OpenProject, Nextcloud, and Keycloak hosts using env vars with per-env defaults.
+- Do not duplicate host or environment resolution logic inside tests or helpers; always call the shared utilities.
+- Configuration is loaded once in `utils/config.ts` and exposed as `testConfig`:
+  - Service URLs and versions.
+  - Setup method and environment name.
+  - Any additional test-level configuration.
+- `global-setup.ts`:
+  - Optionally waits for Kubernetes `setup-job` completion when `SETUP_JOB_CHECK=true` (uses `utils/pod-waiter.ts`).
+  - Runs `detectAllVersions()` from `utils/version-detect.ts` to populate version-related env vars (OpenProject, Nextcloud, Keycloak, etc.) if not already set.
+
+## Type Safety
+
+- Prefer explicit TypeScript interfaces and types for:
+  - API responses (e.g., OpenProject v3, Nextcloud capabilities and status endpoints, Keycloak server info).
+  - Config structures and helper return types.
+- Avoid `Record<string, any>` and other untyped shapes in new or refactored code.
+- Keep interfaces close to their usage (e.g., in `utils/version-detect.ts` for version APIs).
+
+## Test Helpers and Reuse
+
+- Use functions for repetitive actions so they can be reused later rather than copied into each spec.
+- Shared helpers live primarily in:
+  - `utils/test-helpers.ts` for high-level flows (e.g., `ensureProjectExists`, `ensureProjectHasNextcloudStorage`, `ensureDemoProjectCopyViaUi`).
+  - `utils/openproject-api.ts` for direct API interactions with OpenProject (projects, users, storages).
+- When adding new cross-test flows:
+  - First, see if existing helpers can be extended.
+  - If needed, create a new helper function with a clear name and parameters.
+- After UI actions that should succeed, verify the expected UI feedback:
+  - Example: after adding Nextcloud storage from OpenProject, wait for the success banner (`storageCreationSuccessMessage` locator, text `"Successful creation."`).
+
+## Running Tests
+
+- Tests run in headless mode by default (see `playwright.config.ts`).
+- To run in headed mode, use Playwright’s native CLI flags, e.g.:
+  - `npx playwright test --headed`
+- Default worker configuration:
+  - Single worker (`workers: 1`) to keep tests predictable and avoid cross-test interference.
+- Common commands:
+  - Run tests: `npx playwright test`
+  - Run tests headed: `npx playwright test --headed`
+  - Run tests and open report: `npx playwright test && npx playwright show-report`
+
+## Self-Improvement Directive
+
+- After completing significant refactoring, introducing new patterns, or making architectural changes to this E2E codebase:
+  - Update this `SKILL.md` to reflect the new conventions, helpers, and standards.
+  - Keep the file concise and focused on project-specific knowledge that future sessions should follow by default.
+

.gitignore

@@ -13,5 +13,4 @@ playwright-report/
 test-results/
 .playwright-browsers/
 
-.DS_Store
-.cursor/
+.DS_Store

README.md

@@ -19,55 +19,55 @@ Playwright end-to-end tests for OpenProject-Nextcloud-Keycloak integration.
    npm run playwright:install
    ```
 
-2. **Set env for local (gitignored):**
+2. **Ensure local integration cluster is running (for local tests):**
+   The recommended way to run a full local OpenProject–Nextcloud–Keycloak stack is via the
+   [`opf/integration-qa-helmfile`](https://github.com/opf/integration-qa-helmfile/tree/main) repo.
+   Follow its README to `make setup` and `make deploy`, which will provision:
+   - `https://openproject.test`
+   - `https://nextcloud.test`
+   - `https://keycloak.test`
+
+3. **Run tests (native Playwright, headless by default):**
    ```bash
-   cat > .env.local <<'EOF'
-   OPENPROJECT_HOST=openproject.test
-   NEXTCLOUD_HOST=nextcloud.test
-   KEYCLOAK_HOST=keycloak.test
-   E2E_OP_ADMIN_USER=admin
-   E2E_OP_ADMIN_PASS=admin
-   E2E_NC_ADMIN_USER=admin
-   E2E_NC_ADMIN_PASS=admin
-   E2E_KC_ADMIN_USER=admin
-   E2E_KC_ADMIN_PASS=admin
-   EOF
-   ```
-
-3. **Run tests (choose env):**
-   ```bash
-   E2E_ENV=local npm test        # local Helm/defaults
-   E2E_ENV=edge npm test         # edge/staging secrets must be exported
-   E2E_ENV=stage npm test        # stage secrets must be exported
+   E2E_ENV=local npx playwright test        # local Helm/defaults
+   E2E_ENV=edge npx playwright test         # edge/staging secrets must be exported
+   E2E_ENV=stage npx playwright test        # stage secrets must be exported
    ```
 
 ## Commands
 
 ### Run Tests
 ```bash
-# Run with explicit env selection (preferred)
-E2E_ENV=edge npm test
-E2E_ENV=stage npm test
-E2E_ENV=local npm test
+# Native Playwright (recommended)
+E2E_ENV=local npx playwright test
+E2E_ENV=edge npx playwright test
+E2E_ENV=stage npx playwright test
+
+# Run tests and automatically open HTML report
+E2E_ENV=local npx playwright test && npx playwright show-report
 
 # Override worker count to re-enable parallelism (default is 1)
 npx playwright test --workers 4
 
-# Shortcuts
+# npm script shortcuts (thin wrappers around the above)
 npm run test:edge     # uses E2E_ENV=edge
 npm run test:stage    # uses E2E_ENV=stage
 npm run test:local    # uses E2E_ENV=local
 
-# Run in headed mode
-npm run test:e2e:headed
+# Run in headed mode (non-headless)
+npm run test:e2e:headed     
+npx playwright test --headed
 
 # Run with UI mode
 npm run test:e2e:ui
 ```
 
 ### View Results
 ```bash
-# Open test report
+# Open last test report (native)
+npx playwright show-report
+
+# Or via npm script
 npm run report:show
 ```
 
@@ -167,12 +167,11 @@ Local runs: put the above in `.env.local` (already gitignored). CI ignores this
 
 **Playwright complains about missing browser executable?**
 - Make sure you've run `npm run playwright:install` (or `npx playwright install`) after `npm install`.
-- Re-run your test command, e.g. `npm run test:local`.
-```
+- Re-run your test command, e.g. `E2E_ENV=local npx playwright test`.
 
 ## More Information
 
-- Tests run in parallel by default (both browsers + multiple test files)
+- Tests run with a single worker by default (configurable via `--workers`)
 - Uses Page Object Model pattern
 - Locators stored in JSON files
 - See `e2e/utils/locators_guide.md` for locator usage examples

global-setup.ts

@@ -1,42 +1,73 @@
 import { FullConfig } from '@playwright/test';
 import { waitForSetupJobComplete, setupJobExists, isSetupJobComplete } from './utils/pod-waiter';
+import { detectAllVersions } from './utils/version-detect';
+import { getErrorMessage } from './utils/error-utils';
+import { logInfo, logError, logWarn } from './utils/logger';
 
 /**
- * Global setup that runs before all tests
- * Waits for setup-job to complete only if SETUP_JOB_CHECK=true is set
+ * Global setup that runs before all tests.
+ *
+ * 1. Waits for the setup-job to complete (if SETUP_JOB_CHECK=true).
+ * 2. Detects service versions via API and stores them as environment
+ *    variables so that config.ts and tests can read them.
+ *    Existing env vars are treated as overrides and are never replaced.
  */
 async function globalSetup(config: FullConfig) {
-  // Default: skip setup-job check unless explicitly enabled
-  if (process.env.SETUP_JOB_CHECK !== 'true') {
-    console.log('⏭️  Skipping setup-job check (enable with SETUP_JOB_CHECK=true)');
-    return;
-  }
-
-  const namespace = process.env.KUBERNETES_NAMESPACE || 'opnc-integration';
-  
-  // Check if setup-job exists
-  const exists = await setupJobExists(namespace);
-  if (!exists) {
-    console.log(`⚠️  Setup-job not found in namespace '${namespace}'. Skipping check.`);
-    console.log('   If you are running tests against a pre-deployed environment,');
-    console.log('   set SKIP_SETUP_JOB_CHECK=true to skip this check.');
-    return;
-  }
+  // ── Step 1: Setup-job check (optional) ──────────────────────────
+  if (process.env.SETUP_JOB_CHECK === 'true') {
+    const namespace = process.env.KUBERNETES_NAMESPACE || 'opnc-integration';
 
-  // Check if already complete
-  const isComplete = await isSetupJobComplete(namespace);
-  if (isComplete) {
-    console.log('✓ Setup-job is already completed');
-    return;
+    // Check if setup-job exists
+    const exists = await setupJobExists(namespace);
+    if (!exists) {
+      logWarn(`⚠️  Setup-job not found in namespace '${namespace}'. Skipping check.`);
+      logInfo('   If you are running tests against a pre-deployed environment,');
+      logInfo('   set SKIP_SETUP_JOB_CHECK=true to skip this check.');
+    } else {
+      // Check if already complete
+      const isComplete = await isSetupJobComplete(namespace);
+      if (isComplete) {
+        logInfo('✓ Setup-job is already completed');
+      } else {
+        // Wait for setup-job to complete
+        try {
+          await waitForSetupJobComplete(namespace);
+        } catch (error: unknown) {
+          logError('❌ Setup-job check failed:', getErrorMessage(error));
+          logError('\nTo skip this check, set SKIP_SETUP_JOB_CHECK=true');
+          throw error;
+        }
+      }
+    }
+  } else {
+    logInfo('⏭️  Skipping setup-job check (enable with SETUP_JOB_CHECK=true)');
   }
 
-  // Wait for setup-job to complete
+  // ── Step 2: Detect service versions via API ─────────────────────
   try {
-    await waitForSetupJobComplete(namespace);
-  } catch (error: any) {
-    console.error('❌ Setup-job check failed:', error.message);
-    console.error('\nTo skip this check, set SKIP_SETUP_JOB_CHECK=true');
-    throw error;
+    const detected = await detectAllVersions();
+
+    // Store detected versions as env vars.
+    // Existing env vars take precedence (act as manual overrides).
+    // States like 'not-reachable' and 'not-installed' are preserved
+    // so tests can see which services/apps are unavailable.
+    const envMap: Record<string, string> = {
+      OPENPROJECT_VERSION: detected.openproject,
+      NEXTCLOUD_VERSION: detected.nextcloud,
+      NEXTCLOUD_API_VERSION: detected.nextcloudApiVersion,
+      INTEGRATION_APP_VERSION: detected.integrationApp,
+      NEXTCLOUD_TEAM_FOLDERS_VERSION: detected.teamFolders,
+      KEYCLOAK_VERSION: detected.keycloak,
+    };
+
+    for (const [key, value] of Object.entries(envMap)) {
+      if (!process.env[key]) {
+        process.env[key] = value;
+      }
+    }
+  } catch (error: unknown) {
+    logWarn('⚠️  Version detection failed:', getErrorMessage(error));
+    logWarn('   Tests will use "not-detected" as fallback.');
   }
 }
 

package-lock.json

@@ -1,5 +1,6 @@
 import { Page } from '@playwright/test';
 import { KeycloakBasePage } from './KeycloakBasePage';
+import { getErrorMessage } from '../../utils/error-utils';
 
 export class KeycloakClientsPage extends KeycloakBasePage {
   constructor(page: Page) {
@@ -69,8 +70,11 @@ export class KeycloakClientsPage extends KeycloakBasePage {
 
       console.log(`[CLIENTS VERIFICATION] Final result: ${result}`);
       return result;
-    } catch (error) {
-      console.error('[CLIENTS VERIFICATION] Error verifying clients:', error);
+    } catch (error: unknown) {
+      console.error(
+        '[CLIENTS VERIFICATION] Error verifying clients:',
+        getErrorMessage(error),
+      );
       // Take a screenshot for debugging
       await this.screenshot('clients-verification-error.png').catch(() => {});
       return false;