miel.truyen/nx
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

feat(core): allow partially resetting workspace (#23381)

Browse Source
This commit is contained in:
Craigory Coppola 2024-06-04 18:55:10 -04:00 committed by GitHub
parent d06992e5b9
commit 5e39cb0019
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
34 changed files with 482 additions and 123 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

62
docs/generated/cli/reset.md
View File

@ -1,11 +1,11 @@
---
title: 'reset - CLI command'
description: 'Clears all the cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.'
description: 'Clears cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.'
---
# reset
Clears all the cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.
Clears cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.
## Usage
@ -14,3 +14,61 @@ nx reset
```
Install `nx` globally to invoke the command directly using `nx`, or use `npx nx`, `yarn nx`, or `pnpm nx`.
### Examples
Clears the internal state of the daemon and metadata that Nx is tracking. Helpful if you are getting strange errors and want to start fresh:
```shell
nx reset
```
Clears the Nx Cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache:
```shell
nx reset --only-cache
```
Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.:
```shell
nx reset --only-daemon
```
Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc):
```shell
nx reset --only-workspace-data
```
## Options
### help
Type: `boolean`
Show help
### onlyCache
Type: `boolean`
Clears the Nx cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache.
### onlyDaemon
Type: `boolean`
Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.
### onlyWorkspaceData
Type: `boolean`
Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc)
### version
Type: `boolean`
Show version number

62
docs/generated/packages/nx/documents/reset.md
View File

@ -1,11 +1,11 @@
---
title: 'reset - CLI command'
description: 'Clears all the cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.'
description: 'Clears cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.'
---
# reset
Clears all the cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.
Clears cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.
## Usage
@ -14,3 +14,61 @@ nx reset
```
Install `nx` globally to invoke the command directly using `nx`, or use `npx nx`, `yarn nx`, or `pnpm nx`.
### Examples
Clears the internal state of the daemon and metadata that Nx is tracking. Helpful if you are getting strange errors and want to start fresh:
```shell
nx reset
```
Clears the Nx Cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache:
```shell
nx reset --only-cache
```
Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.:
```shell
nx reset --only-daemon
```
Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc):
```shell
nx reset --only-workspace-data
```
## Options
### help
Type: `boolean`
Show help
### onlyCache
Type: `boolean`
Clears the Nx cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache.
### onlyDaemon
Type: `boolean`
Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.
### onlyWorkspaceData
Type: `boolean`
Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc)
### version
Type: `boolean`
Show version number

4
docs/shared/reference/environment-variables.md
View File

@ -3,7 +3,7 @@
The following environment variables are ones that you can set to change the behavior of Nx in different environments.
| Property | Type | Description |
| -------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| --------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| NX_ADD_PLUGINS | boolean | If set to `false`, Nx will not add plugins to infer tasks. This is `true` by default. Workspaces created before Nx 18 will have this disabled via a migration for backwards compatibility |
| NX_BASE | string | The default base branch to use when calculating the affected projects. Can be overridden on the command line with `--base`. |
| NX_CACHE_DIRECTORY | string | The cache for task outputs is stored in `.nx/cache` by default. Set this variable to use a different directory. |
@ -13,7 +13,7 @@ The following environment variables are ones that you can set to change the beha
| NX_HEAD | string | The default head branch to use when calculating the affected projects. Can be overridden on the command line with `--head`. |
| NX_PERF_LOGGING | boolean | If set to `true`, will print debug information useful for for profiling executors and Nx itself |
| NX_PROFILE | string | Prepend `NX_PROFILE=profile.json` before running targets with Nx to generate a file that be [loaded in Chrome dev tools](/troubleshooting/performance-profiling) to visualize the performance of Nx across multiple processes. |
| NX_PROJECT_GRAPH_CACHE_DIRECTORY | string | The project graph cache is stored in `.nx/cache` by default. Set this variable to use a different directory. |
| NX_WORKSPACE_DATA_CACHE_DIRECTORY | string | The project graph cache and some other internal nx caches are stored in `.nx/workspace-data` by default. Set this variable to use a different directory. |
| NX_PROJECT_GRAPH_MAX_WORKERS | number | The number of workers to use when calculating the project graph. |
| NX_PARALLEL | number | The number of tasks Nx should run in parallel. Overrides any configured value inside nx.json |
| NX_RUNNER | string | The name of task runner from the config to use. Can be overridden on the command line with `--runner`. Not read if `NX_TASKS_RUNNER` is set. |

4
packages/angular/src/executors/module-federation-dev-server/lib/build-static-remotes.ts
View File

@ -1,7 +1,7 @@
import { type Schema } from '../schema';
import { type ExecutorContext, logger } from '@nx/devkit';
import type { StaticRemotesConfig } from './parse-static-remotes-config';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { fork } from 'node:child_process';
import { join } from 'node:path';
import { createWriteStream } from 'node:fs';
@ -49,7 +49,7 @@ export async function buildStaticRemotes(
);
// File to debug build failures e.g. 2024-01-01T00_00_0_0Z-build.log'
const remoteBuildLogFile = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`${new Date().toISOString().replace(/[:\.]/g, '_')}-build.log`
);
const stdoutStream = createWriteStream(remoteBuildLogFile);

4
packages/cypress/src/plugins/plugin.ts
View File

@ -21,7 +21,7 @@ import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { NX_PLUGIN_OPTIONS } from '../utils/constants';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
import { hashObject } from 'nx/src/devkit-internals';
@ -49,7 +49,7 @@ export const createNodesV2: CreateNodesV2<CypressPluginOptions> = [
async (configFiles, options, context) => {
const optionsHash = hashObject(options);
const cachePath = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`cypress-${optionsHash}.hash`
);
const targetsCache = readTargetsCache(cachePath);

4
packages/detox/src/plugins/plugin.ts
View File

@ -13,7 +13,7 @@ import { getLockFileName } from '@nx/js';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
export interface DetoxPluginOptions {
buildTargetName?: string;
@ -21,7 +21,7 @@ export interface DetoxPluginOptions {
testTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'detox.hash');
const cachePath = join(workspaceDataDirectory, 'detox.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

2
packages/eslint/src/plugins/plugin.spec.ts
View File

@ -6,7 +6,7 @@ import { mkdirSync, rmdirSync } from 'fs';
jest.mock('nx/src/utils/cache-directory', () => ({
...jest.requireActual('nx/src/utils/cache-directory'),
projectGraphCacheDirectory: 'tmp/project-graph-cache',
workspaceDataDirectory: 'tmp/project-graph-cache',
}));
describe('@nx/eslint/plugin', () => {

4
packages/eslint/src/plugins/plugin.ts
View File

@ -24,7 +24,7 @@ import {
} from '../utils/config-file';
import { resolveESLintClass } from '../utils/resolve-eslint-class';
import { gte } from 'semver';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { hashObject } from 'nx/src/hasher/file-hasher';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
@ -163,7 +163,7 @@ export const createNodesV2: CreateNodesV2<EslintPluginOptions> = [
async (configFiles, options, context) => {
const optionsHash = hashObject(options);
const cachePath = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`eslint-${optionsHash}.hash`
);
const targetsCache = readTargetsCache(cachePath);

4
packages/expo/plugins/plugin.ts
View File

@ -14,7 +14,7 @@ import { getLockFileName } from '@nx/js';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
export interface ExpoPluginOptions {
@ -29,7 +29,7 @@ export interface ExpoPluginOptions {
submitTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'expo.hash');
const cachePath = join(workspaceDataDirectory, 'expo.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

5
packages/gradle/src/plugin/nodes.ts
View File

@ -8,14 +8,13 @@ import {
createNodesFromFiles,
readJsonFile,
writeJsonFile,
CreateNodesResultV2,
CreateNodesFunction,
logger,
} from '@nx/devkit';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { existsSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { getGradleExecFile } from '../utils/exec-gradle';
import {
@ -67,7 +66,7 @@ export const createNodesV2: CreateNodesV2<GradlePluginOptions> = [
async (configFiles, options, context) => {
const optionsHash = hashObject(options);
const cachePath = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`gradle-${optionsHash}.hash`
);
const targetsCache = readTargetsCache(cachePath);

7
packages/jest/src/plugins/plugin.ts
View File

@ -17,7 +17,7 @@ import { dirname, join, relative, resolve } from 'path';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync, readFileSync } from 'fs';
import { readConfig } from 'jest-config';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { clearRequireCache } from '@nx/devkit/src/utils/config-utils';
import { getGlobPatternsFromPackageManagerWorkspaces } from 'nx/src/plugins/package-json-workspaces';
@ -49,10 +49,7 @@ export const createNodesV2: CreateNodesV2<JestPluginOptions> = [
jestConfigGlob,
async (configFiles, options, context) => {
const optionsHash = hashObject(options);
const cachePath = join(
projectGraphCacheDirectory,
`jest-${optionsHash}.hash`
);
const cachePath = join(workspaceDataDirectory, `jest-${optionsHash}.hash`);
const targetsCache = readTargetsCache(cachePath);
try {
return await createNodesFromFiles(

1
packages/js/src/generators/init/init.ts
View File

@ -124,6 +124,7 @@ export async function initGenerator(
/dist
/coverage
/.nx/cache
/.nx/workspace-data
`
);
}

4
packages/js/src/plugins/typescript/plugin.ts
View File

@ -17,7 +17,7 @@ import { basename, dirname, join, relative } from 'node:path';
import { minimatch } from 'minimatch';
// eslint-disable-next-line @typescript-eslint/no-restricted-imports
import { getLockFileName } from 'nx/src/plugins/js/lock-file/lock-file';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import type { ParsedCommandLine } from 'typescript';
import { readTsConfig } from '../../utils/typescript/ts-config';
@ -49,7 +49,7 @@ interface NormalizedPluginOptions {
};
}
const cachePath = join(projectGraphCacheDirectory, 'tsc.hash');
const cachePath = join(workspaceDataDirectory, 'tsc.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

6
packages/next/src/plugins/plugin.ts
View File

@ -8,12 +8,12 @@ import {
TargetConfiguration,
writeJsonFile,
} from '@nx/devkit';
import { dirname, extname, join } from 'path';
import { dirname, join } from 'path';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { getLockFileName } from '@nx/js';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
@ -25,7 +25,7 @@ export interface NextPluginOptions {
serveStaticTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'next.hash');
const cachePath = join(workspaceDataDirectory, 'next.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

7
packages/nuxt/src/plugins/plugin.ts
View File

@ -3,21 +3,20 @@ import {
CreateNodes,
CreateNodesContext,
detectPackageManager,
joinPathFragments,
readJsonFile,
TargetConfiguration,
workspaceRoot,
writeJsonFile,
} from '@nx/devkit';
import { basename, dirname, isAbsolute, join, relative } from 'path';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { dirname, isAbsolute, join, relative } from 'path';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { getLockFileName } from '@nx/js';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
const cachePath = join(projectGraphCacheDirectory, 'nuxt.hash');
const cachePath = join(workspaceDataDirectory, 'nuxt.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

6
packages/nx/migrations.json
View File

@ -71,6 +71,12 @@
"version": "18.1.0-beta.3",
"description": "Moves affected.defaultBase to defaultBase in `nx.json`",
"implementation": "./src/migrations/update-17-2-0/move-default-base"
},
"19-2-0-move-graph-cache-directory": {
"cli": "nx",
"version": "19.2.0-beta.2",
"description": "Updates the default workspace data directory to .nx/workspace-data",
"implementation": "./src/migrations/update-19-2-0/move-workspace-data-directory"
}
}
}

22
packages/nx/src/command-line/examples.ts
View File

@ -336,6 +336,28 @@ export const examples: Record<string, Example[]> = {
'Create a dedicated commit for each successfully completed migration. You can customize the prefix used for each commit by additionally setting --commit-prefix="PREFIX_HERE "',
},
],
reset: [
{
command: 'reset',
description:
'Clears the internal state of the daemon and metadata that Nx is tracking. Helpful if you are getting strange errors and want to start fresh',
},
{
command: 'reset --only-cache',
description:
'Clears the Nx Cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache',
},
{
command: 'reset --only-daemon',
description:
'Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.',
},
{
command: 'reset --only-workspace-data',
description:
'Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc)',
},
],
show: [
{
command: 'show projects',

32
packages/nx/src/command-line/reset/command-object.ts
View File

@ -1,9 +1,35 @@
import { CommandModule } from 'yargs';
export const yargsResetCommand: CommandModule = {
export type ResetCommandOptions = {
onlyCache?: boolean;
onlyDaemon?: boolean;
onlyWorkspaceData?: boolean;
};
export const yargsResetCommand: CommandModule<
Record<string, unknown>,
ResetCommandOptions
> = {
command: 'reset',
describe:
'Clears all the cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.',
'Clears cached Nx artifacts and metadata about the workspace and shuts down the Nx Daemon.',
aliases: ['clear-cache'],
handler: async () => (await import('./reset')).resetHandler(),
builder: (yargs) =>
yargs
.option('onlyCache', {
description:
'Clears the Nx cache directory. This will remove all local cache entries for tasks, but will not affect the remote cache.',
type: 'boolean',
})
.option('onlyDaemon', {
description:
'Stops the Nx Daemon, it will be restarted fresh when the next Nx command is run.',
type: 'boolean',
})
.option('onlyWorkspaceData', {
description:
'Clears the workspace data directory. Used by Nx to store cached data about the current workspace (e.g. partial results, incremental data, etc)',
type: 'boolean',
}),
handler: async (argv) => (await import('./reset')).resetHandler(argv),
};

136
packages/nx/src/command-line/reset/reset.ts
View File

@ -1,29 +1,131 @@
import { rmSync } from 'fs-extra';
import { daemonClient } from '../../daemon/client/client';
import {
cacheDir,
projectGraphCacheDirectory,
} from '../../utils/cache-directory';
import { cacheDir, workspaceDataDirectory } from '../../utils/cache-directory';
import { output } from '../../utils/output';
import { getNativeFileCacheLocation } from '../../native/native-file-cache-location';
import { ResetCommandOptions } from './command-object';
export async function resetHandler() {
output.note({
title: 'Resetting the Nx workspace cache and stopping the Nx Daemon.',
bodyLines: [`This might take a few minutes.`],
});
await daemonClient.stop();
output.log({ title: 'Daemon Server - Stopped' });
// Wait at max 5 seconds before giving up on a failing operation.
const INCREMENTAL_BACKOFF_MAX_DURATION = 5000;
// If an operation fails, wait 100ms before first retry.
const INCREMENTAL_BACKOFF_FIRST_DELAY = 100;
export async function resetHandler(args: ResetCommandOptions) {
let errors = [];
const all =
args.onlyDaemon === undefined &&
args.onlyCache === undefined &&
args.onlyWorkspaceData === undefined;
const startupMessage = all
? 'Resetting the Nx cache and stopping the daemon.'
: 'Resetting:';
const bodyLines = [];
if (!all) {
if (args.onlyDaemon) {
bodyLines.push('- Nx Daemon');
}
if (args.onlyCache) {
bodyLines.push('- Cache directory');
}
if (args.onlyWorkspaceData) {
bodyLines.push('- Workspace data directory');
}
}
output.note({ title: startupMessage, bodyLines });
if (all || args.onlyDaemon) {
try {
rmSync(getNativeFileCacheLocation(), { recursive: true, force: true });
} catch (e) {
// ignore, deleting the native file cache is not critical and can fail if another process is locking the file
await killDaemon();
} catch {
errors.push('Failed to stop the Nx Daemon.');
}
rmSync(cacheDir, { recursive: true, force: true });
if (projectGraphCacheDirectory !== cacheDir) {
rmSync(projectGraphCacheDirectory, { recursive: true, force: true });
}
if (all || args.onlyCache) {
try {
await cleanupCacheEntries();
} catch {
errors.push('Failed to clean up the cache directory.');
}
}
if (all || args.onlyWorkspaceData) {
try {
await cleanupNativeFileCache();
} catch {
errors.push('Failed to clean up the native file cache.');
}
try {
await cleanupWorkspaceData();
} catch {
errors.push('Failed to clean up the workspace data directory.');
}
}
if (errors.length > 0) {
output.error({
title: 'Failed to reset the Nx workspace.',
bodyLines: errors,
});
process.exit(1);
} else {
output.success({
title: 'Successfully reset the Nx workspace.',
});
}
}
function killDaemon() {
return daemonClient.stop();
}
function cleanupCacheEntries() {
return incrementalBackoff(
INCREMENTAL_BACKOFF_FIRST_DELAY,
INCREMENTAL_BACKOFF_MAX_DURATION,
() => {
rmSync(cacheDir, { recursive: true, force: true });
}
);
}
function cleanupNativeFileCache() {
return incrementalBackoff(
INCREMENTAL_BACKOFF_FIRST_DELAY,
INCREMENTAL_BACKOFF_MAX_DURATION,
() => {
rmSync(getNativeFileCacheLocation(), { recursive: true, force: true });
}
);
}
function cleanupWorkspaceData() {
return incrementalBackoff(
INCREMENTAL_BACKOFF_FIRST_DELAY,
INCREMENTAL_BACKOFF_MAX_DURATION,
() => {
rmSync(workspaceDataDirectory, { recursive: true, force: true });
}
);
}
async function incrementalBackoff(
ms: number,
maxDuration: number,
callback: () => void
) {
try {
callback();
} catch (e) {
if (ms < maxDuration) {
await sleep(ms);
await incrementalBackoff(ms * 2, maxDuration, callback);
} else {
throw e;
}
}
}
function sleep(ms: number) {
return new Promise((resolve) => setTimeout(resolve, ms));
}

4
packages/nx/src/daemon/tmp-dir.ts
View File

@ -6,13 +6,13 @@
import { statSync, writeFileSync } from 'fs';
import { ensureDirSync, rmSync } from 'fs-extra';
import { join } from 'path';
import { projectGraphCacheDirectory } from '../utils/cache-directory';
import { workspaceDataDirectory } from '../utils/cache-directory';
import { createHash } from 'crypto';
import { tmpdir } from 'tmp';
import { workspaceRoot } from '../utils/workspace-root';
export const DAEMON_DIR_FOR_CURRENT_WORKSPACE = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
'd'
);

32
packages/nx/src/migrations/update-19-2-0/move-workspace-data-directory.spec.ts Normal file
View File

@ -0,0 +1,32 @@
import { createTree } from '../../generators/testing-utils/create-tree';
import moveWorkspaceDataDirectory from './move-workspace-data-directory';
describe('moveWorkspaceDataDirectory', () => {
it("should not throw if .gitignore does't exist", () => {
const tree = createTree();
expect(() => moveWorkspaceDataDirectory(tree)).not.toThrow();
});
it("shouldn't change anything if gitignore doesn't include .nx/cache", () => {
const tree = createTree();
tree.write('.gitignore', 'dist');
moveWorkspaceDataDirectory(tree);
expect(tree.read('.gitignore', 'utf-8')).toBe('dist');
});
it('should add .nx/workspace-data to .gitignore if it includes .nx/cache', () => {
const tree = createTree();
tree.write(
'.gitignore',
['# temporary files from Nx', '.nx/cache', '', 'dist'].join('\n')
);
moveWorkspaceDataDirectory(tree);
expect(tree.read('.gitignore', 'utf-8')).toMatchInlineSnapshot(`
"# temporary files from Nx
.nx/cache
.nx/workspace-data
dist"
`);
});
});

41
packages/nx/src/migrations/update-19-2-0/move-workspace-data-directory.ts Normal file
View File

@ -0,0 +1,41 @@
import { Tree } from '../../generators/tree';
export default async function moveGraphCacheDirectory(tree: Tree) {
updateGitIgnore(tree);
updatePrettierIgnore(tree);
}
export function updatePrettierIgnore(tree: Tree) {
if (tree.exists('.prettierignore')) {
const ignored = tree.read('.prettierignore', 'utf-8');
if (!ignored?.includes('.nx/workspace-data')) {
tree.write(
'.prettierignore',
[ignored, '/.nx/workspace-data'].join('\n')
);
}
}
}
export function updateGitIgnore(tree: Tree) {
const gitignore = tree.read('.gitignore', 'utf-8');
if (!gitignore) {
return;
}
const includesNxWorkspaceData = gitignore.includes('.nx/workspace-data');
if (includesNxWorkspaceData) {
return;
}
const includesNxCache = gitignore.includes('.nx/cache');
if (!includesNxCache) {
return;
}
const updatedGitignore = gitignore.replace(
'.nx/cache',
['.nx/cache', '.nx/workspace-data'].join('\n')
);
tree.write('.gitignore', updatedGitignore);
}

9
packages/nx/src/native/walker.rs
View File

@ -35,6 +35,7 @@ where
"**/node_modules".into(),
"**/.git".into(),
"**/.nx/cache".into(),
"**/.nx/workspace-data".into(),
"**/.yarn/cache".into(),
];
@ -65,7 +66,13 @@ where
{
let directory = directory.as_ref();
let ignore_glob_set = build_glob_set(&["**/node_modules", "**/.git", "**/.nx/cache"])
let ignore_glob_set = build_glob_set(&[
"**/node_modules",
"**/.git",
"**/.nx/cache",
"**/.nx/workspace-data",
"**/.yarn/cache",
])
.expect("These static ignores always build");
let mut walker = WalkBuilder::new(directory);

6
packages/nx/src/plugins/js/index.ts
View File

@ -3,7 +3,7 @@ import { ensureDirSync } from 'fs-extra';
import { dirname, join } from 'path';
import { performance } from 'perf_hooks';
import { ProjectGraph } from '../../config/project-graph';
import { projectGraphCacheDirectory } from '../../utils/cache-directory';
import { workspaceDataDirectory } from '../../utils/cache-directory';
import { combineGlobPatterns } from '../../utils/globs';
import {
CreateDependencies,
@ -160,8 +160,8 @@ function readCachedParsedLockFile(): ParsedLockFile {
return JSON.parse(readFileSync(cachedParsedLockFile).toString());
}
const lockFileHashFile = join(projectGraphCacheDirectory, 'lockfile.hash');
const lockFileHashFile = join(workspaceDataDirectory, 'lockfile.hash');
const cachedParsedLockFile = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
'parsed-lock-file.json'
);

16
packages/nx/src/project-graph/nx-deps-cache.ts
View File

@ -10,7 +10,7 @@ import {
ProjectGraph,
} from '../config/project-graph';
import { ProjectConfiguration } from '../config/workspace-json-project-json';
import { projectGraphCacheDirectory } from '../utils/cache-directory';
import { workspaceDataDirectory } from '../utils/cache-directory';
import {
directoryExists,
fileExists,
@ -31,15 +31,15 @@ export interface FileMapCache {
}
export const nxProjectGraph = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
'project-graph.json'
);
export const nxFileMap = join(projectGraphCacheDirectory, 'file-map.json');
export const nxFileMap = join(workspaceDataDirectory, 'file-map.json');
export function ensureCacheDirectory(): void {
try {
if (!existsSync(projectGraphCacheDirectory)) {
ensureDirSync(projectGraphCacheDirectory);
if (!existsSync(workspaceDataDirectory)) {
ensureDirSync(workspaceDataDirectory);
}
} catch (e) {
/*
@ -52,10 +52,8 @@ export function ensureCacheDirectory(): void {
* In this case, we're creating the directory. If the operation failed, we ensure that the directory
* exists before continuing (or raise an exception).
*/
if (!directoryExists(projectGraphCacheDirectory)) {
throw new Error(
`Failed to create directory: ${projectGraphCacheDirectory}`
);
if (!directoryExists(workspaceDataDirectory)) {
throw new Error(`Failed to create directory: ${workspaceDataDirectory}`);
}
}
}

23
packages/nx/src/utils/cache-directory.ts
View File

@ -36,7 +36,11 @@ function cacheDirectory(root: string, cacheDirectory: string) {
}
}
function defaultCacheDirectory(root: string) {
function pickCacheDirectory(
root: string,
nonNxCacheDirectory: string,
nxCacheDirectory: string
) {
// If nx.json doesn't exist the repo can't utilize
// caching, so .nx/cache is less relevant. Lerna users
// that don't want to fully opt in to Nx at this time
@ -47,9 +51,17 @@ function defaultCacheDirectory(root: string) {
existsSync(join(root, 'lerna.json')) &&
!existsSync(join(root, 'nx.json'))
) {
return join(root, 'node_modules', '.cache', 'nx');
return join(root, 'node_modules', '.cache', nonNxCacheDirectory);
}
return join(root, '.nx', 'cache');
return join(root, '.nx', nxCacheDirectory);
}
function defaultCacheDirectory(root: string) {
return pickCacheDirectory(root, 'nx', 'cache');
}
function defaultWorkspaceDataDirectory(root: string) {
return pickCacheDirectory(root, 'nx-workspace-data', 'workspace-data');
}
/**
@ -67,8 +79,9 @@ export function cacheDirectoryForWorkspace(workspaceRoot: string) {
);
}
export const projectGraphCacheDirectory = absolutePath(
export const workspaceDataDirectory = absolutePath(
workspaceRoot,
process.env.NX_WORKSPACE_DATA_DIRECTORY ??
process.env.NX_PROJECT_GRAPH_CACHE_DIRECTORY ??
defaultCacheDirectory(workspaceRoot)
defaultWorkspaceDataDirectory(workspaceRoot)
);

4
packages/playwright/src/plugins/plugin.ts
View File

@ -21,7 +21,7 @@ import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash
import type { PlaywrightTestConfig } from '@playwright/test';
import { getFilesInDirectoryUsingContext } from 'nx/src/utils/workspace-context';
import { minimatch } from 'minimatch';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { getLockFileName } from '@nx/js';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
import { hashObject } from 'nx/src/hasher/file-hasher';
@ -57,7 +57,7 @@ export const createNodesV2: CreateNodesV2<PlaywrightPluginOptions> = [
async (configFilePaths, options, context) => {
const optionsHash = hashObject(options);
const cachePath = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`playwright-${optionsHash}.hash`
);
const targetsCache = readTargetsCache(cachePath);

4
packages/react-native/plugins/plugin.ts
View File

@ -14,7 +14,7 @@ import { getLockFileName } from '@nx/js';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
export interface ReactNativePluginOptions {
@ -29,7 +29,7 @@ export interface ReactNativePluginOptions {
upgradeTargetname?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'react-native.hash');
const cachePath = join(workspaceDataDirectory, 'react-native.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<
string,

4
packages/react/src/executors/module-federation-dev-server/module-federation-dev-server.impl.ts
View File

@ -18,7 +18,7 @@ import {
createAsyncIterable,
} from '@nx/devkit/src/utils/async-iterable';
import { waitForPortOpen } from '@nx/web/src/utils/wait-for-port-open';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { fork } from 'node:child_process';
import { basename, dirname, join } from 'node:path';
import { createWriteStream, cpSync } from 'node:fs';
@ -205,7 +205,7 @@ async function buildStaticRemotes(
// File to debug build failures e.g. 2024-01-01T00_00_0_0Z-build.log'
const remoteBuildLogFile = join(
projectGraphCacheDirectory,
workspaceDataDirectory,
`${new Date().toISOString().replace(/[:\.]/g, '_')}-build.log`
);
const stdoutStream = createWriteStream(remoteBuildLogFile);

4
packages/remix/src/plugins/plugin.ts
View File

@ -1,4 +1,4 @@
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import {
type CreateDependencies,
type CreateNodes,
@ -17,7 +17,7 @@ import { dirname, join } from 'path';
import { existsSync, readdirSync } from 'fs';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
const cachePath = join(projectGraphCacheDirectory, 'remix.hash');
const cachePath = join(workspaceDataDirectory, 'remix.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

4
packages/rollup/src/plugins/plugin.ts
View File

@ -1,4 +1,4 @@
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { basename, dirname, join } from 'path';
import { existsSync, readdirSync } from 'fs';
import {
@ -20,7 +20,7 @@ import { type RollupOptions } from 'rollup';
// @ts-ignore
import { loadConfigFile } from 'rollup/loadConfigFile';
const cachePath = join(projectGraphCacheDirectory, 'rollup.hash');
const cachePath = join(workspaceDataDirectory, 'rollup.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

4
packages/storybook/src/plugins/plugin.ts
View File

@ -13,7 +13,7 @@ import { dirname, join } from 'path';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readFileSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { getLockFileName } from '@nx/js';
import { loadConfigFile } from '@nx/devkit/src/utils/config-utils';
import type { StorybookConfig } from '@storybook/types';
@ -25,7 +25,7 @@ export interface StorybookPluginOptions {
testStorybookTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'storybook.hash');
const cachePath = join(workspaceDataDirectory, 'storybook.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

4
packages/vite/src/plugins/plugin.ts
View File

@ -12,7 +12,7 @@ import { dirname, isAbsolute, join, relative } from 'path';
import { getNamedInputs } from '@nx/devkit/src/utils/get-named-inputs';
import { existsSync, readdirSync } from 'fs';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { getLockFileName } from '@nx/js';
import { loadViteDynamicImport } from '../utils/executor-utils';
@ -24,7 +24,7 @@ export interface VitePluginOptions {
serveStaticTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'vite.hash');
const cachePath = join(workspaceDataDirectory, 'vite.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<

4
packages/webpack/src/plugins/plugin.ts
View File

@ -16,7 +16,7 @@ import { existsSync, readdirSync } from 'fs';
import { readWebpackOptions } from '../utils/webpack/read-webpack-options';
import { resolveUserDefinedWebpackConfig } from '../utils/webpack/resolve-user-defined-webpack-config';
import { getLockFileName, getRootTsConfigPath } from '@nx/js';
import { projectGraphCacheDirectory } from 'nx/src/utils/cache-directory';
import { workspaceDataDirectory } from 'nx/src/utils/cache-directory';
import { calculateHashForCreateNodes } from '@nx/devkit/src/utils/calculate-hash-for-create-nodes';
export interface WebpackPluginOptions {
@ -26,7 +26,7 @@ export interface WebpackPluginOptions {
previewTargetName?: string;
}
const cachePath = join(projectGraphCacheDirectory, 'webpack.hash');
const cachePath = join(workspaceDataDirectory, 'webpack.hash');
const targetsCache = readTargetsCache();
function readTargetsCache(): Record<