miel.truyen/nx
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

docs(core): expand e2e split guide (#21689)

Browse Source
This commit is contained in:
Isaac Mann 2024-02-09 10:36:49 -05:00 committed by GitHub
parent df60a60c10
commit a8dfc299e8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
12 changed files with 366 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/generated/manifests/ci.json
View File

@ -176,7 +176,7 @@
},
{
"id": "split-e2e-tasks",
"name": "Automatically Split E2E Tasks",
"name": "Automatically Split E2E Tasks (TestAtomizer)",
"description": "",
"mediaImage": "",
"file": "nx-cloud/features/split-e2e-tasks",
@ -258,7 +258,7 @@
},
"/ci/features/split-e2e-tasks": {
"id": "split-e2e-tasks",
"name": "Automatically Split E2E Tasks",
"name": "Automatically Split E2E Tasks (TestAtomizer)",
"description": "",
"mediaImage": "",
"file": "nx-cloud/features/split-e2e-tasks",

4
docs/generated/manifests/menus.json
View File

@ -5779,7 +5779,7 @@
"disableCollapsible": false
},
{
"name": "Automatically Split E2E Tasks",
"name": "Automatically Split E2E Tasks (TestAtomizer)",
"path": "/ci/features/split-e2e-tasks",
"id": "split-e2e-tasks",
"isExternal": false,
@ -5838,7 +5838,7 @@
"disableCollapsible": false
},
{
"name": "Automatically Split E2E Tasks",
"name": "Automatically Split E2E Tasks (TestAtomizer)",
"path": "/ci/features/split-e2e-tasks",
"id": "split-e2e-tasks",
"isExternal": false,

24
docs/generated/packages/cypress/documents/overview.md
View File

@ -75,6 +75,30 @@ The `@nx/cypress/plugin` is configured in the `plugins` array in `nx.json`.
The `@nx/cypress/plugin` will automatically split your e2e tasks by file. You can read more about this feature [here](/ci/features/split-e2e-tasks).
To enable e2e task splitting, make sure there is a `ciWebServerCommand` property set in your `cypress.config.ts` file. It will look something like this:
```ts {% fileName="apps/my-project-e2e/cypress.config.ts" highlightLines=[13] %}
import { defineConfig } from 'cypress';
import { nxE2EPreset } from '@nx/cypress/plugins/cypress-preset';
export default defineConfig({
e2e: {
...nxE2EPreset(__filename, {
cypressDir: 'src',
bundler: 'vite',
webServerCommands: {
default: 'nx run my-project:serve',
production: 'nx run my-project:preview',
},
ciWebServerCommand: 'nx run my-project:serve-static',
}),
baseUrl: 'http://localhost:4200',
},
});
```
Note: The `nxE2EPreset` is a collection of default settings, but is not necessary for task splitting.
{% /tab %}
{% tab label="Nx < 18" %}

2
docs/map.json
View File

@ -1712,7 +1712,7 @@
"file": "nx-cloud/features/dynamic-agents"
},
{
"name": "Automatically Split E2E Tasks",
"name": "Automatically Split E2E Tasks (TestAtomizer)",
"id": "split-e2e-tasks",
"file": "nx-cloud/features/split-e2e-tasks"
},

2
docs/nx-cloud/features/dynamic-agents.md
View File

@ -10,7 +10,7 @@ The standard way to set up [Nx Agents](/ci/features/distribute-task-execution) i
```yaml {% fileName=".nx/workflows/dynamic-changesets.yaml" %}
distribute-on:
small-changeset: 1 linux-medium-js
small-changeset: 3 linux-medium-js
medium-changeset: 6 linux-medium-js
large-changeset: 10 linux-medium-js
```

BIN
docs/nx-cloud/features/flaky-tasks-ci.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 281 KiB

12
docs/nx-cloud/features/flaky-tasks.md
View File

@ -8,14 +8,20 @@ Nx is perfectly positioned to detect which tasks are flaky and automatically re-
Nx creates a hash of all the inputs for a task whenever it is run. If Nx ever encounters a task that fails with a particular set of inputs and then succeeds with those same inputs, Nx knows for a fact that the task is flaky. Nx can't know with certainty when the task has been fixed to no longer be flaky, so if a particular task has no flakiness incidents for 2 weeks, the `flaky` flag is removed for that task.
![Flaky tasks in CI](/nx-cloud/features/flaky-tasks-ci.png)
In this image, the `e2e-ci--src/e2e/app.cy.ts` task is a flaky task that has been automatically retried once. There is a `1 retry` indicator to show that it has been retried and, once expanded, you can see tabs that contain the logs for `Attempt 1` and `Attempt 2`. With this UI, you can easily compare the output between a successful and unsuccessful run of a flaky task.
## Manually Mark a Task as Flaky or Not Flaky
If you need to manually mark a task as flaky or not flaky, you can do so from the run details screen. Flaky tasks will have a button that says `Mark task as no longer flaky` and failed tasks that are not flaky will have a button that says `Mark task as likely flaky`. Using these buttons, you can ensure that Nx Cloud treats tasks in the appropriate way.
![Mark task as no longer flaky button](/nx-cloud/features/mark-task-as-no-longer-flaky.png)
If you suspect that a task is flaky, but Nx has not confirmed it yet, you can manually mark it as `likely flaky` from the run details screen. Failed tasks that are not flaky will have a button that says `Mark task as likely flaky`.
![Mark task as likely flaky button](/nx-cloud/features/mark-task-as-likely-flaky.png)
Once you've resolved the issue that caused a task to be flaky, you can immediately mark the task as not flaky by clicking on `Mark task as no longer flaky` on the same run details screen.
![Mark task as no longer flaky button](/nx-cloud/features/mark-task-as-no-longer-flaky.png)
## Re-run Flaky Tasks
When a flaky task fails in CI with [distributed task execution](/ci/features/distribute-task-execution) enabled, Nx will automatically send that task to a different agent and run it again (up to 2 tries in total). Its important to run the task on a different agent to ensure that the agent itself or the other tasks that were run on that agent are not the reason for the flakiness.

BIN
docs/nx-cloud/features/mark-task-as-no-longer-flaky.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 66 KiB

308
docs/nx-cloud/features/split-e2e-tasks.md
View File

@ -33,15 +33,301 @@ If you are already using the `@nx/cypress` or `@nx/playwright` plugin, you need
## Usage
You can view the available tasks in the graph:
You can view the available tasks for your project in the project detail view:
```shell
nx graph
nx show project myproject-e2e --web
```
{% project-details title="Project Details View" height="100px" %}
```json
{
"project": {
"name": "admin-e2e",
"data": {
"root": "apps/admin-e2e",
"projectType": "application",
"targets": {
"e2e": {
"cache": true,
"inputs": ["default", "^production"],
"outputs": [
"{workspaceRoot}/dist/cypress/apps/admin-e2e/videos",
"{workspaceRoot}/dist/cypress/apps/admin-e2e/screenshots"
],
"executor": "nx:run-commands",
"dependsOn": ["^build"],
"options": {
"cwd": "apps/admin-e2e",
"command": "cypress run"
},
"configurations": {
"production": {
"command": "cypress run --env webServerCommand=\"nx run admin:preview\""
}
}
},
"e2e-ci--src/e2e/app.cy.ts": {
"outputs": [
"{workspaceRoot}/dist/cypress/apps/admin-e2e/videos",
"{workspaceRoot}/dist/cypress/apps/admin-e2e/screenshots"
],
"inputs": [
"default",
"^production",
{
"externalDependencies": ["cypress"]
}
],
"cache": true,
"options": {
"cwd": "apps/admin-e2e",
"command": "cypress run --env webServerCommand=\"nx run admin:serve-static\" --spec src/e2e/app.cy.ts"
},
"executor": "nx:run-commands",
"configurations": {}
},
"e2e-ci--src/e2e/login.cy.ts": {
"outputs": [
"{workspaceRoot}/dist/cypress/apps/admin-e2e/videos",
"{workspaceRoot}/dist/cypress/apps/admin-e2e/screenshots"
],
"inputs": [
"default",
"^production",
{
"externalDependencies": ["cypress"]
}
],
"cache": true,
"options": {
"cwd": "apps/admin-e2e",
"command": "cypress run --env webServerCommand=\"nx run admin:serve-static\" --spec src/e2e/login.cy.ts"
},
"executor": "nx:run-commands",
"configurations": {}
},
"e2e-ci": {
"executor": "nx:noop",
"cache": true,
"inputs": [
"default",
"^production",
{
"externalDependencies": ["cypress"]
}
],
"outputs": [
"{workspaceRoot}/dist/cypress/apps/admin-e2e/videos",
"{workspaceRoot}/dist/cypress/apps/admin-e2e/screenshots"
],
"dependsOn": [
{
"target": "e2e-ci--src/e2e/app.cy.ts",
"projects": "self",
"params": "forward"
},
{
"target": "e2e-ci--src/e2e/login.cy.ts",
"projects": "self",
"params": "forward"
}
],
"options": {},
"configurations": {}
},
"lint": {
"executor": "@nx/eslint:lint",
"inputs": ["default", "{workspaceRoot}/.eslintrc.json"],
"cache": true,
"outputs": ["{options.outputFile}"],
"options": {},
"configurations": {}
}
},
"name": "admin-e2e",
"$schema": "../../node_modules/nx/schemas/project-schema.json",
"sourceRoot": "apps/admin-e2e/src",
"tags": [],
"implicitDependencies": ["admin"]
}
},
"sourceMap": {
"root": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"projectType": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"targets": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"targets.e2e": ["apps/admin-e2e/project.json", "nx/core/target-defaults"],
"targets.e2e.options": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.cache": [
"apps/admin-e2e/project.json",
"nx/core/target-defaults"
],
"targets.e2e.inputs": [
"apps/admin-e2e/project.json",
"nx/core/target-defaults"
],
"targets.e2e.outputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.configurations": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.executor": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.options.cwd": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.options.command": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.configurations.production": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.configurations.production.command": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.outputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.inputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.cache": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.options": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.executor": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.options.cwd": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/app.cy.ts.options.command": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.outputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.inputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.cache": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.options": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.executor": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.options.cwd": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci--src/e2e/login.cy.ts.options.command": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci.executor": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci.cache": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci.inputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci.outputs": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e-ci.dependsOn": [
"apps/admin-e2e/cypress.config.ts",
"@nx/cypress/plugin"
],
"targets.e2e.dependsOn": [
"apps/admin-e2e/project.json",
"nx/core/target-defaults"
],
"targets.lint": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"targets.lint.executor": [
"apps/admin-e2e/project.json",
"nx/core/project-json"
],
"targets.lint.inputs": [
"apps/admin-e2e/project.json",
"nx/core/target-defaults"
],
"targets.lint.cache": [
"apps/admin-e2e/project.json",
"nx/core/target-defaults"
],
"name": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"$schema": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"sourceRoot": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"tags": ["apps/admin-e2e/project.json", "nx/core/project-json"],
"implicitDependencies": [
"apps/admin-e2e/project.json",
"nx/core/project-json"
],
"implicitDependencies.admin": [
"apps/admin-e2e/project.json",
"nx/core/project-json"
],
"targets.lint.outputs": [
"apps/admin-e2e/project.json",
"nx/core/project-json"
]
}
}
```
{% /project-details %}
You'll see that there are tasks named `e2e`, `e2e-ci` and a task for each e2e test file.
Developers can run all e2e tests locally the same way as usual:
Developers can run all e2e tests locally with the `e2e` target:
```shell
nx e2e my-project-e2e
@ -55,8 +341,16 @@ nx e2e-ci my-project-e2e
## Benefits
Smaller e2e tasks enable the following benefits:
With more granular e2e tasks, all the other features of Nx become more powerful. Let's imagine a scenario where there are 10 spec files in a single e2e project and each spec file takes 3 minutes to run.
- Nx's cache can be used for all the e2e tasks that succeeded and only the failed tasks need to be re-run
- Distributed Task Execution allows your e2e tests to be run on multiple machines simultaneously, which reduces the total time of the CI pipeline
- Nx Agents can [automatically re-run failed flaky e2e tests](/ci/features/flaky-tasks) on a separate agent without a developer needing to manually re-run the CI pipeline
### Improved Caching
[Nx's cache](/ci/features/remote-cache) can be used for all the individual e2e tasks that succeeded and only the failed tasks need to be re-run. Without e2e task splitting, a single spec file failing would force you to re-run all the e2e tests for the project, which would take 30 minutes. With e2e task splitting, a single spec file that fails can be re-run in 3 minutes and the other successful spec file results can be retrieved from the cache.
### Better Distribution
[Distributed task execution](/ci/features/distribute-task-execution) allows your e2e tests to be run on multiple machines simultaneously, which reduces the total time of the CI pipeline. Without e2e task splitting, the CI pipeline has to take at least 30 minutes to complete because the one e2e task needs that long to finish. With e2e task splitting, a fully distributed pipeline with 10 agents could finish in 3 minutes.
### More Precise Flaky Task Identification
Nx Agents [automatically re-run failed flaky e2e tests](/ci/features/flaky-tasks) on a separate agent without a developer needing to manually re-run the CI pipeline. Leveraging e2e task splitting, Nx identifies the specific flaky test file - this way you can quickly fix the offending test file. Without e2e splitting, Nx identifies that at least one of the e2e tests are flaky - requiring you to find the flaky test on your own.

2
docs/nx-cloud/reference/nx-cloud-cli.md
View File

@ -21,7 +21,7 @@ You can also define the configuration in a file and reference it as follows:
```yaml {% fileName=".nx/workflows/dynamic-changesets.yaml" %}
distribute-on:
small-changeset: 1 linux-medium-js
small-changeset: 3 linux-medium-js
medium-changeset: 6 linux-medium-js
large-changeset: 10 linux-medium-js
```

24
docs/shared/packages/cypress/cypress-plugin.md
View File

@ -75,6 +75,30 @@ The `@nx/cypress/plugin` is configured in the `plugins` array in `nx.json`.
The `@nx/cypress/plugin` will automatically split your e2e tasks by file. You can read more about this feature [here](/ci/features/split-e2e-tasks).
To enable e2e task splitting, make sure there is a `ciWebServerCommand` property set in your `cypress.config.ts` file. It will look something like this:
```ts {% fileName="apps/my-project-e2e/cypress.config.ts" highlightLines=[13] %}
import { defineConfig } from 'cypress';
import { nxE2EPreset } from '@nx/cypress/plugins/cypress-preset';
export default defineConfig({
e2e: {
...nxE2EPreset(__filename, {
cypressDir: 'src',
bundler: 'vite',
webServerCommands: {
default: 'nx run my-project:serve',
production: 'nx run my-project:preview',
},
ciWebServerCommand: 'nx run my-project:serve-static',
}),
baseUrl: 'http://localhost:4200',
},
});
```
Note: The `nxE2EPreset` is a collection of default settings, but is not necessary for task splitting.
{% /tab %}
{% tab label="Nx < 18" %}

2
docs/shared/reference/sitemap.md
View File

@ -271,7 +271,7 @@
- [Use Remote Caching (Nx Replay)](/ci/features/remote-cache)
- [Distribute Task Execution (Nx Agents)](/ci/features/distribute-task-execution)
- [Dynamically Allocate Agents](/ci/features/dynamic-agents)
- [Automatically Split E2E Tasks](/ci/features/split-e2e-tasks)
- [Automatically Split E2E Tasks (TestAtomizer)](/ci/features/split-e2e-tasks)
- [Identify and Re-run Flaky Tasks](/ci/features/flaky-tasks)
- [Set up Nx Cloud On-Premise](/ci/features/on-premise)
- [Concepts](/ci/concepts)