84796d011e
## Current Behavior - Documentation pages under "technologies" and "core-api" sections with "introduction"/"overview" IDs lack H1 titles after front matter - Some remote caching package links point to parent sections instead of overview pages ## Expected Behavior - All affected documentation pages should have H1 titles for consistency - Links should point directly to overview pages ## Related Issue(s) Fixes # ## Changes Made ### 1. Updated Remote Caching Links (commit fae9055f8c) Updated links in 3 files to point directly to overview pages: - `docs/blog/2025-01-06-nx-update-20-3.md` - `docs/shared/deprecated/custom-tasks-runner.md` - `docs/shared/recipes/running-tasks/self-hosted-caching.md` Changed links from: - `/reference/core-api/azure-cache` → `/reference/core-api/azure-cache/overview` - `/reference/core-api/gcs-cache` → `/reference/core-api/gcs-cache/overview` - `/reference/core-api/s3-cache` → `/reference/core-api/s3-cache/overview` - `/reference/core-api/shared-fs-cache` → `/reference/core-api/shared-fs-cache/overview` ### 2. Added H1 Titles to Documentation Pages Added H1 titles to 29 documentation files that were missing them: #### Core API Overview Pages (6 files) - `docs/shared/packages/azure-cache/azure-cache-plugin.md` → `# @nx/azure-cache` - `docs/shared/packages/conformance/conformance-plugin.md` → `# @nx/conformance` - `docs/shared/packages/gcs-cache/gcs-cache-plugin.md` → `# @nx/gcs-cache` - `docs/shared/packages/owners/owners-plugin.md` → `# @nx/owners` - `docs/shared/packages/s3-cache/s3-cache-plugin.md` → `# @nx/s3-cache` - `docs/shared/packages/shared-fs-cache/shared-fs-cache-plugin.md` → `# @nx/shared-fs-cache` #### Technology Introduction Pages (23 files) - `docs/shared/packages/angular/angular-plugin.md` → `# @nx/angular` - `docs/shared/packages/esbuild/esbuild-plugin.md` → `# @nx/esbuild` - `docs/shared/packages/rspack/rspack-plugin.md` → `# @nx/rspack` - `docs/shared/packages/vite/vite-plugin.md` → `# @nx/vite` - `docs/shared/packages/webpack/plugin-overview.md` → `# @nx/webpack` - `docs/shared/packages/eslint/eslint.md` → `# @nx/eslint` - `docs/shared/packages/gradle/gradle-plugin.md` → `# @nx/gradle` - `docs/shared/packages/express/express-plugin.md` → `# @nx/express` - `docs/shared/packages/node/node-plugin.md` → `# @nx/node` - `docs/shared/packages/nest/nest-plugin.md` → `# @nx/nest` - `docs/shared/packages/expo/expo-plugin.md` → `# @nx/expo` - `docs/shared/packages/react/react-plugin.md` → `# @nx/react` - `docs/shared/packages/next/plugin-overview.md` → `# @nx/next` - `docs/shared/packages/react-native/react-native-plugin.md` → `# @nx/react-native` - `docs/shared/packages/remix/remix-plugin.md` → `# @nx/remix` - `docs/shared/packages/cypress/cypress-plugin.md` → `# @nx/cypress` - `docs/shared/packages/detox/detox-plugin.md` → `# @nx/detox` - `docs/shared/packages/jest/jest-plugin.md` → `# @nx/jest` - `docs/shared/packages/playwright/playwright-plugin.md` → `# @nx/playwright` - `docs/shared/packages/storybook/plugin-overview.md` → `# @nx/storybook` - `docs/shared/packages/js/js-plugin.md` → `# @nx/js` - `docs/shared/packages/vue/vue-plugin.md` → `# @nx/vue` - `docs/shared/packages/nuxt/nuxt-plugin.md` → `# @nx/nuxt` Note: The Angular Rspack introduction page (`docs/shared/guides/angular-rspack/introduction.md`) already had an appropriate H1 title "# Introduction" and was left unchanged. All changes improve documentation consistency and navigation by ensuring proper titles and direct links to overview pages.
213 lines
12 KiB
Markdown
213 lines
12 KiB
Markdown
---
|
|
title: Overview of the Nx S3 Cache Plugin
|
|
description: The @nx/s3-cache plugin enables you to use an Amazon S3 bucket to host your remote cache for efficient build caching across your team.
|
|
---
|
|
|
|
# @nx/s3-cache
|
|
|
|
The `@nx/s3-cache` plugin enables you to self-host your remote cache on an [Amazon S3](https://aws.amazon.com/s3) bucket.
|
|
|
|
{% callout type="deepdive" title="Free managed remote cache with Nx Cloud" %}
|
|
|
|
Note, you can get started for free with a **fully managed remote caching powered by Nx Cloud**. It comes with a generous Hobby plan that is enough for most small teams. [Learn more here](/nx-cloud).
|
|
|
|
If you are an enterprise and **data privacy and security is a concern**, [reach out for an Enterprise trial](/enterprise/trial). It is fully SOC 2 type 1 and 2 compliant and comes with single-tenant, dedicated EU region hosting as well as on-premise.
|
|
|
|
**Are you an OSS project?** Nx Cloud is free for OSS. [Reach out here](/pricing#oss).
|
|
{% /callout %}
|
|
|
|
{% callout type="info" title="Self-hosted caching is now free" %}
|
|
|
|
Self-hosted caching is **now free for everyone** to use.
|
|
|
|
{% /callout %}
|
|
|
|
## Set Up @nx/s3-cache
|
|
|
|
### 1. Install the Package
|
|
|
|
Run the following command:
|
|
|
|
```shell
|
|
nx add @nx/s3-cache
|
|
```
|
|
|
|
This will add the `@nx/s3-cache` NPM package and automatically configure it for your workspace. As part of this process you'll be guided to **generate a new activation key**. This is a fully automated process to register your plugin.
|
|
|
|
The key will be saved in your repository (`.nx/key/key.ini`) and should be committed so that every developer has access to it. If your repository is public (or in CI) you can also use an environment variable:
|
|
|
|
```{% fileName=".env" %}
|
|
NX_KEY=YOUR_ACTIVATION_KEY
|
|
```
|
|
|
|
If you didn't get an activation key or skipped that step, you can easily generate one at any time by running `nx register` in your terminal.
|
|
|
|
> Why require an activation key? It simply helps us know and support our users. If you prefer not to provide this information, you can also build your own cache server. [Learn more.](/recipes/running-tasks/self-hosted-caching)
|
|
|
|
### 2. Authenticate with AWS
|
|
|
|
There are four different ways to authenticate with AWS. They will be attempted in this order:
|
|
|
|
1. Environment variables
|
|
2. INI config files
|
|
3. Single sign-on
|
|
4. `nx.json` settings
|
|
|
|
#### Environment Variables
|
|
|
|
[AWS provides environment variables](https://docs.aws.amazon.com/sdkref/latest/guide/environment-variables.html) that can be used to authenticate:
|
|
|
|
| **Environment Variable** | **Description** |
|
|
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `AWS_ACCESS_KEY_ID` | The access key for your AWS account. |
|
|
| `AWS_SECRET_ACCESS_KEY` | The secret key for your AWS account. |
|
|
| `AWS_SESSION_TOKEN` | The session key for your AWS account. This is only needed when you are using temporary credentials. |
|
|
| `AWS_CREDENTIAL_EXPIRATION` | The expiration time of the credentials contained in the environment variables described above. This value must be in a format compatible with the [ISO-8601 standard](https://en.wikipedia.org/wiki/ISO_8601) and is only needed when you are using temporary credentials. |
|
|
|
|
Both the `AWS_ACCESS_KEY_ID` and the `AWS_SECRET_ACCESS_KEY` environment variables are required to use the environment variable authentication method.
|
|
|
|
Here's an example of using OIDC in GitHub Actions to set the environment variables in CI:
|
|
|
|
```yaml {% fileName=".github/workflows/ci.yml" %}
|
|
name: CI
|
|
...
|
|
permissions:
|
|
id-token: write
|
|
...
|
|
|
|
jobs:
|
|
main:
|
|
env:
|
|
NX_KEY: ${{ secrets.NX_KEY }}
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
...
|
|
|
|
- name: 'Configure AWS Credentials'
|
|
uses: aws-actions/configure-aws-credentials@v4.0.2
|
|
with:
|
|
role-to-assume: arn:aws:iam::123456789123:role/GhAIBucketUserRole
|
|
aws-region: us-east-1
|
|
|
|
...
|
|
|
|
- run: pnpm exec nx affected -t lint test build
|
|
```
|
|
|
|
#### INI Config Files
|
|
|
|
AWS can read your authentication credentials from [shared INI config files](https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html). The files are located at `~/.aws/credentials` and `~/.aws/config`. Both files are expected to be INI formatted with section names corresponding to profiles. Sections in the credentials file are treated as profile names, whereas profile sections in the config file must have the format of `[profile profile-name]`, except for the default profile. Profiles that appear in both files will not be merged, and the version that appears in the credentials file will be given precedence over the profile found in the config file.
|
|
|
|
#### Single Sign-On
|
|
|
|
Nx can read the active access token [created after running `aws sso login`](https://docs.aws.amazon.com/sdkref/latest/guide/understanding-sso.html) then request temporary AWS credentials. You can create the `AwsCredentialIdentityProvider` functions using the inline SSO parameters (`ssoStartUrl`, `ssoAccountId`, `ssoRegion`, `ssoRoleName`) or load them from [AWS SDKs and Tools shared configuration and credentials files](https://docs.aws.amazon.com/credref/latest/refdocs/creds-config-files.html). Profiles in the `credentials` file are given precedence over profiles in the `config` file.
|
|
|
|
#### Credentials in `nx.json` File
|
|
|
|
Storing your credentials in the `nx.json` file is the least secure of the 4 authentication options, since anyone with read access to your codebase will have access to your AWS credentials.
|
|
|
|
```jsonc {% fileName="nx.json" %}
|
|
{
|
|
"s3": {
|
|
"ssoProfile": "default",
|
|
"accessKeyId": "MYACCESSKEYID",
|
|
"secretAccessKey": "MYSECRETACCESSKEY"
|
|
}
|
|
}
|
|
```
|
|
|
|
| **Property** | **Description** |
|
|
| ------------------- | ----------------------------------------------------------------------------- |
|
|
| **ssoProfile** | The name of the profile to use from your AWS CLI SSO Configuration (optional) |
|
|
| **endpoint** | The AWS endpoint URL (optional) |
|
|
| **accessKeyId** | AWS Access Key ID (optional) |
|
|
| **secretAccessKey** | AWS secret access key (optional) |
|
|
|
|
### 3. Configure S3 Cache
|
|
|
|
Regardless of how you manage your AWS authentication, you need to configure your Nx cache in the `nx.json` file. The `bucket` that you specify needs to already exist - Nx doesn't create it for you.
|
|
|
|
```jsonc {% fileName="nx.json" %}
|
|
{
|
|
"s3": {
|
|
"region": "us-east-1",
|
|
"bucket": "my-bucket",
|
|
"encryptionKey": "create-your-own-key"
|
|
}
|
|
}
|
|
```
|
|
|
|
| **Property** | **Description** |
|
|
| ----------------- | --------------------------------------------------------------------------------- |
|
|
| **region** | The ID of the AWS region to use |
|
|
| **bucket** | The name of the S3 bucket to use |
|
|
| **encryptionKey** | Nx encryption key used to encrypt and decrypt artifacts from the cache (optional) |
|
|
|
|
#### S3 Compatible Providers
|
|
|
|
To use `@nx/s3-cache` with S3 compatible providers ([MinIO](https://min.io/product/s3-compatibility), [LocalStack](https://www.localstack.cloud), [DigitalOcean Spaces](https://www.digitalocean.com/products/spaces), [Cloudflare](https://www.cloudflare.com/developer-platform/solutions/s3-compatible-object-storage), etc.), `endpoint` will need to be provided. Some providers also need to have `forcePathStyle` set to true in the configuration.
|
|
|
|
Below is an example on how to connect to MinIO:
|
|
|
|
```jsonc {% fileName="nx.json" %}
|
|
{
|
|
"s3": {
|
|
"region": "us-east-1",
|
|
"bucket": "my-bucket",
|
|
"endpoint": "https://play.min.io",
|
|
"forcePathStyle": true,
|
|
"accessKeyId": "abc1234",
|
|
"secretAccessKey": "4321cba",
|
|
"disableChecksum: true
|
|
}
|
|
}
|
|
```
|
|
|
|
{% callout type="note" title="Minio and checksum validation" %}
|
|
If you are using MinIO earlier than `2024-07-04T14-25-45Z` it is recommended to enabled `disabledChecksum` else you may trigger aws-sdk checksum errors such as `x-amz-checksum-crc32`.
|
|
{% /callout %}
|
|
|
|
| **Property** | **Description** |
|
|
| ------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
| **region** | The ID of the S3 compatible storage region to use |
|
|
| **bucket** | The name of the S3 compatible storage bucket to use |
|
|
| **forcePathStyle** | Changes the way artifacts are uploaded. Usually used for S3 compatible providers (MinIO, LocalStack, etc) |
|
|
| **endpoint** | The custom endpoint to upload artifacts to. If endpoint is not defined, the default AWS endpoint is used |
|
|
| **accessKeyId** | AWS Access Key ID (optional if `AWS_ACCESS_KEY_ID` is set in the environment) |
|
|
| **secretAccessKey** | AWS secret access key (optional if `AWS_SECRET_ACCESS_KEY` is set in the environment) |
|
|
| **disableChecksum** | This disables AWS' checksum validation for cache entries |
|
|
|
|
# Cache Modes
|
|
|
|
By default, Nx will try to write and read from the remote cache while running locally. This means that permissions must be set for users who are expected to access the remote cache.
|
|
|
|
Nx will only show warnings when the remote cache is not writable. You can disable these warnings by setting `localMode` to `read-only` or `no-cache` in the `nx.json` file.
|
|
|
|
```jsonc {% fileName="nx.json" %}
|
|
{
|
|
"s3": {
|
|
"region": "us-east-1",
|
|
"bucket": "my-bucket",
|
|
"localMode": "read-only"
|
|
}
|
|
}
|
|
```
|
|
|
|
The cache mode in CI can also be configured by setting `ciMode` to `read-only` or `no-cache` in the `nx.json` file. Or by setting `NX_POWERPACK_CACHE_MODE` to `read-only` or `no-cache` in the CI environment.
|
|
|
|
```jsonc {% fileName="nx.json" %}
|
|
{
|
|
"s3": {
|
|
"region": "us-east-1",
|
|
"bucket": "my-bucket",
|
|
"ciMode": "read-only"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Migrating from Custom Tasks Runners
|
|
|
|
Many people who are interested in Nx Powerpack have previously used custom task runners. Nx offers a new and simpler extension API designed to meet the same use cases as the now-deprecated custom task runners.
|
|
|
|
To learn more about migrating from custom task runners, [please refer to this detailed guide](/deprecated/custom-tasks-runner).
|