nx/docs/shared/features/distribute-task-execution.md

Distribute Task Execution (Nx Agents)

{% youtube src="https://youtu.be/XLOUFZeqRpM" title="Nx Agents in action splitting e2e tests at a file level" /%}

Nx Agents lets you distribute your CI across many machines with minimal configuration. It comes with features such as dynamically allocating agents based on the size of the PR, flaky task re-running, and intelligent task splitting and distribution. Keep reading to learn more.

For a more thorough explanation of how Nx Agents optimizes your CI pipeline, read this guide to parallelization and distribution in CI.

Enabling Nx Agents

To enable task distribution with Nx Agents, there are two requirements:

1. Enable version control system integration. The integrations currently available are GitHub, GitLab and Bitbucket. These integrations can be enabled from your Nx Cloud dashboard.
2. Add a single line to your CI pipeline configuration.

Add the start-ci-run command to your CI pipeline configuration after checking out the repository and before installing node_modules:

# After checkout repository
- name: Start CI run
  run: 'npx nx-cloud start-ci-run --distribute-on="8 linux-medium-js" --stop-agents-after="e2e-ci"'
# Before install node_modules
# Run any nx commands as if running on a single machine

The --distribute-on flag instructs Nx Cloud to distribute tasks across 8 agents of type linux-medium-js. linux-medium-js is the name of the launch template that will be used to provision the agent. The default launch templates can be found here

Launch Templates

You can also define your own "launch templates" (here's an example from the Nx repo):

launch-templates:
  linux-medium:
    resource-class: 'docker_linux_amd64/medium+'
    init-steps:
      - name: Pnpm Install
        script: |
          pnpm install --frozen-lockfile

      - name: Install Cypress
        script: pnpm exec cypress install

      - name: Install Rust
      - ...

Here are the available resource classes.

Related Features

{% cards %}

{% card title="Dynamically Allocate Agents" description="Assign a different number of agents to a pipeline based on the size of the PR" type="documentation" url="/ci/features/dynamic-agents" /%}

{% card title="Automatically Split E2E Tasks" description="Split large e2e tasks into separate tasks for each spec file" type="documentation" url="/ci/features/split-e2e-tasks" /%}

{% card title="Identify and Re-run Flaky Tasks" description="Re-run flaky tasks in CI whenever they fail" url="/ci/features/flaky-tasks" /%}

{% /cards %}

CI/CD Guides

Every organization manages their CI/CD pipelines differently, so the guides don't cover org-specific aspects of CI/CD (e.g., deployment). They mainly focus on configuring Nx correctly using Nx Agents and Nx Replay.

• Azure Pipelines
• Circle CI
• GitHub Actions
• Jenkins

Note that only cacheable operations can be distributed because they have to be replayed on the main job.

Relevant Repositories and Examples

• Nx: On how to make your CI 16 times faster with a small config change
• "Lerna & Distributed Task Execution" Example