Feat/migrate more blogs (#23203)

This PR adds 4 more blogs to the `/blog` on nx.dev
This commit is contained in:
Nicholas Cunningham 2024-05-07 09:18:43 -06:00 committed by GitHub
parent 62431372f1
commit d8ab165009
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
47 changed files with 1290 additions and 1 deletions

View File

@ -0,0 +1,120 @@
---
title: Nx Cloud 3.0 — Faster Cache, More Powerful DTE, Better Ergonomics
authors: [Juri Strumpflohner]
cover_image: '/blog/images/2023-04-19/featured_img.webp'
tags: [nx, nx-cloud]
---
It has been almost 2 years since we released [Nx Cloud 2.0](https://nx.app/). Since then, it has saved over 400 years of computation by leveraging its distributed caching and task execution. And we keep adding 8 years every single week. Not only does this tremendously [impact our environment](https://dev.to/nx/helping-the-environment-by-saving-two-centuries-of-compute-time-4nep), but it also helps developers be more productive and companies save money.
In the last couple of months we have quadrupled the team and have done some amazing things. And we have some big plans for what is coming next. Heres all you need to know!
**Table of Contents**
- [New, Streamlined UI](#new-streamlined-ui)
- [Prefetching and Faster Cache Uploading](#prefetching-and-faster-cache-uploading)
- [DTE Just got Better](#dte-just-got-better)
- [Direct Integration With GitHub, GitLab and Bitbucket](#direct-integration-with-github-gitlab-and-bitbucket)
- [Enterprise Support](#enterprise-support)
- [New, Simplified Plans and Pricing Model](#new-simplified-plans-and-pricing-model)
- [Coming Next](#coming-next)
**Prefer a Video? Weve got you Covered!**
{% youtube src="https://www.youtube.com/embed/cG2hEI5L3qI?si=9frDSD8_HK1iTNEi" /%}
## New, Streamlined UI
This latest release of Nx Cloud comes with a new, streamlined design that offers users a more modern and visually appealing experience. This includes the main [Nx Cloud website](https://nx.app/), where we improved our messaging, including interactive visualizations to better explain some core concepts around remote caching and distributed task execution.
![](/blog/images/2023-04-19/bodyimg1.webp)
The Nx Cloud application — showing your runs, cache saved, and stats — also got a significant overhaul, making the UI more lightweight and easier to parse.
![](/blog/images/2023-04-19/bodyimg2.webp)
More UI-related updates and improvements are already underway.
## Prefetching and Faster Cache Uploading
![](/blog/images/2023-04-19/bodyimg3.webp)
At Nx, we are performance addicts! Especially when it comes to local development, every millisecond counts! In the latest update of the Nx CLI, we added the ability to **offload some of the remote cache management to the [Nx Daemon](/concepts/more-concepts/nx-daemon)**. As a result you no longer have to wait for the cache to upload. This saves valuable time, allowing the command to complete instantly and immediately providing you with the necessary link.
We also **prefetch cache results in the background** to have them ready when needed.
Both optimizations can save seconds.
## DTE Just got Better
![](/blog/images/2023-04-19/bodyimg4.webp)
[Distributed Task Execution (short DTE)](/ci/features/distribute-task-execution) is a core part of what makes Nx Cloud stand out compared to other solutions. And we made some significant improvements to both the ergonomics and speed.
**Identify failed tasks early** — You can now view information about the in-progress DTEs, allowing you to quickly identify and address any failed functions without waiting for the command to complete.
**Simplified CI setup** — We simplified the setup process for most CI systems, eliminating the need to pass environment variables manually. Instead, everything is automatically derived from the context.
**Improved performance and efficiency** — Nx Cloud now intelligently figures out which tasks will be cache hits before sending them to agents. Rather it can directly send them to the main job, reducing unnecessary round trips and drastically speeding up CI runs.
**Efficient agent management** — We have introduced a new command, `npx nx-cloud start-ci-run stop-agents-after=e2e`, that allows you to notify Nx Cloud when specific commands, such as long-running e2e tasks, have started. This helps Nx Cloud proactively identify and shut down agents that will not be needed, improving compute efficiency.
**Reduce fix costs** — Even though cache hits are basically free, spinning processes still have fixed costs. We fixed that (see what I did there) by leveraging a new Nx feature that allows to have a long-running process on an agent to which we can arbitrarily feed new tasks. This removed the DTE overheads and also improved the predictability of runs.
## Direct Integration With GitHub, GitLab and Bitbucket
Our GitHub integration has been enhanced. Now, the list of runs is updated in real-time as soon as they are created or their status changes, providing developers with up-to-date information directly on GitHub.
![](/blog/images/2023-04-19/bodyimg5.webp)
In addition to the GitHub, we expanded our Nx Cloud live status updates to work on GitLab and BitBucket.
![](/blog/images/2023-04-19/bodyimg6.webp)
## Enterprise Support
![](/blog/images/2023-04-19/bodyimg7.webp)
We have extensive experience working with Fortune 500 companies, helping them scale their development using monorepos. This has given us valuable insight into the unique security requirements of these companies. Our [Enterprise plan](https://nx.dev/enterprise) reflects that allowing organizations to have a **fully self-contained version of Nx Cloud** that can be **hosted on their own servers** and comes with dedicated support from the Nx and Nx Cloud core team.
Weve recently made a couple of improvements to our enterprise offering.
- **Helm Charts** — We added a **Helm chart** to simplify the process of deploying Nx Cloud to on-premises infrastructure, allowing organizations to quickly set up and manage their own instance of Nx Cloud within their secure environment.
- **Stability improvements** — We significantly reworked our on-premises solution to be identical to our SaaS deployment. This revamp resulted in a more robust and reliable on-premises deployment of Nx Cloud, ensuring enterprise-grade performance and reliability.
- **SSO** — We now support AWS Identity and Access Management (IAM) for seamless integration with existing AWS environments and the SAML protocol for a more flexible single sign-on integration across various providers. This enables organizations to leverage their existing identity management systems for authentication and authorization.
Learn more at [enterprise](/enterprise).
## New, Simplified Plans and Pricing Model
Nx Cloud has evolved a lot since we first released it in 2020, and is changing even more in 2023. To better adapt to Nx Cloud being a critical CI tool, we changed our pricing model to be more consistent and predictable for CI workloads.
Nx Clouds previous pricing was based on time savings from Nx Cloud, which made sense when Nx Cloud was strictly a distributed caching service. The [new pricing model](https://nx.app/pricing) is based entirely on the number of CI pipeline executions per calendar month. We believe this is a simpler and more transparent model that should help you predict your costs far more easily.
![](/blog/images/2023-04-19/bodyimg8.webp)
Our Free plan allows one administrator and up to 300 CI pipeline executions per month. Our Pro plan allows more administrators, and uncapped CI pipeline executions, with a flat fee and incremental charges per 100 CI pipeline executions. The OSS plan comes with unlimited CI pipeline executions. Nx has been open source from the beginning and we care a lot about that ecosystem. So this is our contribution to help OSS projects and make their CI pipeline executions faster.
Finally, the **Enterprise plan** is for companies that want full control over where their data is hosted, get hands-on, dedicated support from the Nx & Nx Cloud team, and enterprise features such as SSO & SAML-based authentication support.
All these changes should allow developers to choose the plan that best suits their needs and budget more easily, ensuring a seamless and transparent experience regarding pricing and subscription management.
Learn more at [https://nx.app/pricing](https://nx.app/pricing).
## Coming Next
Weve got some big plans for Nx Cloud. You really want to write your CI script by focusing on what you want to achieve rather than thinking about making it fast. Were going to make this happen!
The current Distributed Task Execution (DTE) already goes a long way, but you still have to provision the agents by yourself. Providing the correct number of agents is crucial for maximizing efficiency and reducing idle time. And it is not even a static number but might depend on the actual run itself. Nx has extensive knowledge about the structure of the workspace and according tasks. We want to leverage that information. Imagine a setup where you have a roughly 20 line CI config for a repo with hundreds of developers and Nx Cloud automatically determines for each run the ideal number of agents required, provisions them, distributes all tasks efficiently and then disposes all agents again. All fully automatically. And it will be fast. To the point where you wouldnt even need your Jenkins, CircleCI etc at all.
In addition, we are actively exploring ways to provide advanced analytics for your workspace, including insights into the frequency and duration of specific tasks. This valuable information can help identify large tasks that could benefit from being broken down into smaller ones, leveraging caching and other speed improvements to optimize performance. Stay tuned for more to come!
---
## Learn more
- [Nx Docs](/getting-started/intro)
- [X/Twitter](https://twitter.com/nxdevtools) -- [LinkedIn](https://www.linkedin.com/company/nrwl/)
- [Nx GitHub](https://github.com/nrwl/nx)
- [Nx Official Discord Server](/community)
- [Nx Youtube Channel](https://www.youtube.com/@nxdevtools)
- [Speed up your CI](https://nx.app/)

View File

@ -0,0 +1,337 @@
---
title: Nx Console gets Lit
authors: [Max Kless]
cover_image: '/blog/images/2023-06-29/featured_img.webp'
tags: [nx, nx-console]
---
Over the last few weeks, we rebuilt one of Nx Consoles most liked features from the ground up: The Generate UI. It looks better, loads faster, and preliminary research shows using it makes you happier, too ;)
You can use it today by installing the latest version of Nx Console for VSCode and JetBrains IDEs! 🎉🎉🎉
- [Nx Console on the VSCode Marketplace](https://marketplace.visualstudio.com/items?itemName=nrwl.angular-console)
- [Nx Console on the JetBrains Marketplace](https://plugins.jetbrains.com/plugin/21060-nx-console)
If youre curious to learn more about the rewrite and the motivations behind it, this is the blog post for you! Well touch on these topics and more:
- Why did we choose to rewrite?
- Whats Lit and why did we use it over Angular?
- How did the rewrite go and what Lit features were important for us?
- What does the performance look like before and after?
{% youtube src="https://www.youtube.com/embed/p455D4W7330?si=FRbiKJhGxT8dYzf9" /%}
## Background: Why Migrate from Angular to Lit?
### A Short History of Nx Console
Lets go back in time: Nx Console has been around for a while. [It first launched in 2018](https://blog.nrwl.io/announcing-the-first-stable-release-of-angular-console-the-ui-for-the-angular-cli-bb19093a92d4?source=friends_link&sk=d955094b2cb872a130b47c8040ec820e) — then called Angular Console — as a standalone Electron app which let you run Angular schematics and builders from a graphical interface. Of course, it was built with Angular and looked something like this:
![Screenshot of the original Angular Console electron app](/blog/images/2023-06-29/bodyimg1.webp)
In 2019, [it was ported to a VSCode extension](https://blog.nrwl.io/brand-new-ui-custom-vscode-task-support-more-in-angular-console-9-0-5e4d3a109fb9?source=friends_link&sk=47fd72d5700b4cb3de09e559c5524ce5) with the now familiar UI and support for the standalone app was dropped.
In 2020, support for the entire Nx ecosystem was added, it was renamed to Nx Console and an amazing transformation began: Today, Nx Console is much more than just a single form — it tightly integrates Nx into your IDE, gives you the overview of your projects that you need and puts your task-running just a click away.
### Rewriting in Lit
This evolution brought significant improvements to the usability of Nx Console and the value we could provide to developers, but not without presenting its own set of challenges. Inevitably, the codebase grew complex and convoluted over time — the context in which it ran changed, the scope of the product changed, yet the technology remained the same. Adding small features or fixing bugs became increasingly time consuming, and every PR compounded the problem.
The UI looked somewhat outdated and had been built with only VSCode in mind — which became painfully obvious when support for JetBrains IDEs was added. While the web-based nature of the Generate UI allowed us to reuse the code in the new environment, the design looked out of place.
In addition, we started questioning our usage of Angular. Angular is a great framework for building web apps, and in the original context of Angular Console, it made a lot of sense: A tool built by and for Angular engineers — of course its written in Angular. But things changed, and Angular started to feel overkill for what we needed. Angular has a huge number of features right out of the box. But for our simple form, we didnt need routing, http requests or modules to organize our code. The amount of boilerplate and overhead Angular introduces is significant.
So, ultimately, **we decided to pull the plug and rewrite the entire thing in [Lit](https://lit.dev/).**
Lit is a lightweight framework built on top of web components and “adds just what you need to be happy and productive: reactivity, declarative templates and a handful of thoughtful features to reduce boilerplate and make your job easier” (taken from their docs). We had used it before to [build the Nx Cloud view](https://blog.nrwl.io/nx-console-meets-nx-cloud-d45dc099dc5d?source=friends_link&sk=90215eb316ef332c1324f2d50cd3ad85) and were happy with the simple setup and great DX. So the decision to reuse it here was an easy one, since it allowed us to reuse code, build tooling and expertise. The rewrite also gave us the opportunity to improve the design for both supported IDEs and give the entire UI a clean, new coat of paint.
![Screenshot of the reworked generate ui](/blog/images/2023-06-29/bodyimg2.webp)
Before I dive deeper into specifics, lets have a look at the general architecture of Nx Console first.
## Nx Console Architectural Overview
Nx Console is composed of 3 core parts:
- The **nxls** is a language server based on the [Language Server Protocol (LSP)](https://microsoft.github.io/language-server-protocol/) and acts as the “brain” of Nx Console. It analyzes your Nx workspace and provides information on it, including code completion and more.
- The **Generate UI** is the form-based view for running Nx generators.
- The **platform-specific wrappers**. These are written in Typescript and Kotlin and connect the rest of Nx Console to IDE-specific APIs. Having the other parts separate greatly reduces the amount of duplicated code we have to write in order to support multiple IDEs
This architectures modularity meant we could quickly switch out the Generate UI for a new version without significantly impacting the rest of the codebase — only the parts that actually render the UI and communicate with it had to be adjusted slightly. It also allowed us to ensure backward compatibility: the old generate UI is still available via a feature toggle in the settings.
If you want to dive deeper, there are many more resources on the architecture of Nx Console and how its built:
- [In-depth blog post about expanding to JetBrains IDEs](https://blog.nrwl.io/expanding-nx-console-to-jetbrains-ides-8a5b80fff2d7?source=friends_link&sk=967080ea30bdbf9f8132f098a2cdd188)
- [Accompanying Youtube video by Zack DeRose](https://www.youtube.com/watch?v=xUTm6GDqwJM)
- [The Power of Nx Console — talk by Jon Cammisuli](https://www.youtube.com/watch?v=3C_9g9kt2KM)
## Migrating to Lit: Step by Step
To rebuild our UI, we first needed a new Lit app to work on. While theres no native Nx plugin for Lit, generating the code we need was still very straightforward:
`nx generate @nx/web:app --name generate-ui-v2`
This generates an entire project for us, with a `tsconfig.json`, `index.html`, `main.ts`, and a `project.json`, where our Nx-specific config lives.
I also installed a couple of dependencies:
- The `@nx/esbuild` plugin because I like fast build times 🏎️
- TailwindCSS because I dont like writing CSS 🤫
- `@vscode/webview-ui-toolkit` because it does all of the VSCode work for me 🤖
This is really where Nx shines, because it allows you to take these tools and quickly patch them together and build a pipeline that does exactly what you need. And it also allows you to think about your workspace visually. This is what this is what my task graph for building the Lit app ultimately looks like:
![Nx task graph for building the Lit webview](/blog/images/2023-06-29/bodyimg3.webp)
You can see three build steps:
- `generate-ui-v2:_build` uses esbuild to bundle my Lit components written in Typescript and spits out a `main.js` file
- `generate-ui-v2:extract-dependencies` copies the third party assets we need into the dist folder. Right now its just codicons `.css` and `.ttf` files.
- `generate-ui-v2:build` finally runs tailwind over the bundled code. This could also be done with `postCss` or a custom `esbuild` plugin but running tailwind directly is the easier, so why complicate things?
> 💡 There are different ways to generate this visualisation for your own workspaces:
>
> - In VSCode, use the Nx Project View or the `Nx: Focus task in Graph` action
> - In JetBrains IDEs, use the Nx Toolwindow or context menus
> - In the command line, run `nx build {{your project}} --graph`
In the bigger context of Nx Console, heres what happens when you build the VSCode extension:
![Nx task graph for building the VSCode extension](/blog/images/2023-06-29/bodyimg4.webp)
You can see that were still building both the new & old generate UI as well as the Nxls and the Nx Cloud webview before combining them all into one VSCode extension artifact.
### Building the Form
Lit is a very small library that provides useful abstractions over browser-native features like web components and messages. Heres what a simple Lit component, written with Typescript, looks like:
```ts {% fileName="main.ts" %}
@customElement('root-element')
export class Root extends LitElement {
render() {
return html`<p>Hello World</p>`;
}
}
```
```html {% fileName="index.html" %}
<!DOCTYPE html>
<html lang="en">
<body>
<script type="module" src="main.js"></script>
<root-element></root-element>
</body>
</html>
```
If you need to pass information up the DOM, you use normal events and you can set properties to send information to descendants. Check out the [Lit Docs](https://lit.dev/docs/components/rendering/) to learn more about how the render() method works, how to leverage reactivity, the shadow DOM and so much more.
> 💡 All code samples in this section have been adapted for brevity and clarity. So you wont find this exact code anywhere, but it demonstrates the concepts well.
### Communicating with the IDE — using Reactive Controllers
To communicate with the host IDE, we were able to reuse almost all the logic from the previous UI (for more details, see [Communicating with IntelliJ](https://blog.nrwl.io/expanding-nx-console-to-jetbrains-ides-8a5b80fff2d7?source=friends_link&sk=967080ea30bdbf9f8132f098a2cdd188#d151) from the last blog post). Instead of a service that exposes observables that our component can consume, we used a [Reactive Controller](https://lit.dev/docs/composition/controllers/). Controllers are a neat feature of Lit — they hook into a component and can request DOM updates on their behalf. This eliminates the need for observable streams and subscriptions while keeping the communication code self-contained. Look at this example:
```ts {% fileName="main.ts" %}
@customElement('root-element')
export class Root extends LitElement {
icc: IdeCommunicationController;
constructor() {
super();
this.icc = new IdeCommunicationController(this);
}
render() {
return html`${JSON.stringify(this.icc.generatorSchema)}`;
}
}
```
```ts {% fileName="ide-communication-controller.ts" %}
// ide-communication-controller.ts
export class IdeCommunicationController implements ReactiveController {
generatorSchema: GeneratorSchema | undefined;
constructor(private host: ReactiveControllerHost) {}
// ...
private handleMessageFromIde(message: InputMessage) {
// ...
this.generatorSchema = message.payload;
this.host.requestUpdate();
}
}
```
You can see that `root-element` can really just deal with rendering the form contents, delegating communicating with the IDE and when to update the DOM to the controller.
### Rendering the form fields — using Mixins
The core part of the UI is the form. We built all kinds of inputs: text fields, checkboxes, (multi-) select boxes and array fields. While those each have unique implementations, displaying them is also going to take a lot of repeated code. Every fields needs a label and description. Every field needs to know about its validation state, how dispatch change events and what aria attributes to set. In order to keep this code clean and DRY, we used [Class Mixins](https://lit.dev/docs/composition/mixins/). Mixins arent really a Lit-specific feature, but I dont see them used much in other frameworks. A mixin is essentially a factory that takes a class and returns another, modified class. Check out this example:
```ts {% fileName="field-mixin.ts" %}
const Field = (superClass) =>
class extends superClass {
// we can define (reactive) properties that every field is going to need
@property()
option: Option;
protected get fieldId(): string {
return `${this.option.name}-field`;
}
// we can define methods that should be available to all fields
dispatchValue(value: string) {
// ...
}
};
```
```ts {% fileName="field-wrapper-mixin.ts" %}
const FieldWrapper = (superClass) =>
class extends superClass {
// we can define a render() method so that fields are all rendered the same
protected render() {
return html` <label for="${this.fieldId}">${this.option.name}</label>
<p>${this.option.description}</p>
${this.renderField()}`;
}
};
```
```ts {% fileName="input-field.ts" %}
@customElement('input-field')
export class InputField extends FieldWrapper(Field(LitElement)) {
renderField() {
return html` <input
id="${this.fieldId}"
@input="${(e) => this.dispatchValue(e.target.value)}"
/>`;
}
}
```
You can see that each component and mixin deals with a specific subset of the overall logic, keeping our code cleanly separated and reusable. A checkbox, for example, is a special case because its layout on the page is slightly different — no problem, we simply wrote a CheckboxWrapper with some modifications without having to worry about changing the checkbox logic itself.
> 💡 Properly typing mixins is complicated so I left that part out. Refer to [Mixins in Typescript](https://lit.dev/docs/composition/mixins/#mixins-in-typescript) to learn more or [check out our source code on GitHub](https://github.com/nrwl/nx-console).
### Injecting Services & Data — Lit Context
If youre coming from the Angular world, you probably appreciate the great dependency injection (DI) mechanism they have. You can centrally define some services and reuse them across your app, without thinking about passing on props from component to component — the DI system takes care of it. Lit provides something similar via the []`@lit-labs/context`](https://lit.dev/docs/data/context/) package. Its based on the [Context Community Protocol](https://github.com/webcomponents-cg/community-protocols/blob/main/proposals/context.md) and similar to Reacts context API.
Under the hood, it still works with normal browser events, but it abstracts it away from you so you can easily share data and services across your app.
The Generate UI is rendered in both VSCode and JetBrains IDEs. In order to look appropriate, components need to know which environment theyre in and adjust styling accordingly. Instead of passing this information from component to component, we can make it available contextually! And with a proper mixin, reading the editor context only has to be implemented once, too.
Have a look at the following example:
```ts {% fileName="editor-context.ts" %}
export const editorContext = createContext<'vscode' | 'intellij'>(
Symbol('editor')
);
const EditorContext = (superClass) =>
class extends superClass {
@consume({ context: editorContext })
@state()
editor: 'vscode' | 'intellij';
};
```
```ts {% fileName="ide-communication-controller.ts" %}
export class IdeCommunicationController implements ReactiveController {
// ...
constructor(private host: ReactiveElement) {
const editor = isVscode() ? 'vscode' : 'intellij';
// provide the context to all DOM children of the host element
new ContextProvider(host, {
context: editorContext,
initialValue: editor,
});
}
}
```
```ts {% fileName="some-component.ts" %}
@customElement('some-component')
export class SomeComponent extends EditorContext(LitElement) {
render() {
return html`<p>I am rendered in ${this.editor}</p>`;
}
}
```
### VSCode Webview UI Toolkit
As we mentioned above, a big part of why we rewrote the UI is that it looked quite out of place in JetBrains IDEs. It was still useful, of course, but its important to make sure the form _feels_ right.
In VSCode, this is very easy to achieve, thanks to the [VSCode Webview UI Toolkit](https://github.com/microsoft/vscode-webview-ui-toolkit). Its a set of web components, provided by Microsoft, that are designed to look good and be used in VSCode webviews.
![Sample image of the VSCode Webview UI Toolkit showing the different components it provides](/blog/images/2023-06-29/bodyimg5.webp)
Using it, you get the native look, a11y, and theme-aware styling for free! Thanks to everyone who built it, its a huge help!
### E2E Testing with Cypress
One big upside of using a webview is the huge Javascript ecosystem is available to you! To make sure that no regressions are introduced later on, we use [Cypress](https://www.cypress.io/). We can mock the editor communication and provide different schemas, make sure the form is rendered correctly and the right messages are sent back to the IDE.
While theres no particular Lit integration for Cypress, the tool itself is framework agnostic so it still works perfectly fine. Using the [`@nx/cypress`](/nx-api/cypress) executors did most of the work for us so setup was pretty quick too.
### Results: Comparing Performance
Theres a number of different aspects to performance we can compare between the two implementations. The biggest one by far is not really quantifiable: The maintainability and looks of the new UI. In my opinion, it looks a lot fresher and more native in both environments. We got rid of a lot of legacy code and the new version is easier to reason about and work with.
But there are thing we _can_ measure, so lets talk numbers!
### Startup Time
It takes time to bootstrap a large framework like Angular, so skipping that, the UI should load quicker than before.
We measured the median time it took to render all options of the `@nx/angular:application` generator in both VSCode and IntelliJ. You can see that the results are pretty clear-cut, even though they are not hugely impactful.
Old UI (Angular)New UI (Lit)SpeedupVSCode65 ms39 ms~ 1.7xIntelliJ189 ms122 ms~ 1.5x
### Bundle Size
As mentioned earlier, Angular comes with a lot more features out-of-the-box than Lit, so it would make sense that the built bundle will be bigger.
We were able to reduce the bundle size (w/o compression) from about 733 kB to 282 kB, which comes out to about a 2,6x decrease. Unlike a website, where the bundle needs to be shipped to users when they load a page, Nx Console users only need to download it once when installing the plugin. This means were not affected by network speeds after installation, which makes the bundle size less critical.
> 💡 Because of a misconfiguration from a few Angular versions ago, the bundle size that we reported in:
{% tweet url="https://twitter.com/MaxKless/status/1671095858182381569" /%}
was overly bloated. We corrected it, but Lit still came out ahead in terms of size and rendering times.
### Build Time
While it might not be important to users of Nx Console, the time it takes to build the project makes a difference to us developers.
Since Lit is just javascript files that dont require a custom compiler or build tooling, we decided to use [`esbuild`](https://esbuild.github.io/) (via `@nx/esbuild`), which is written in Go and extremely fast. On the other hand, the old UI used the `@angular-builders/custom-webpack:browser` builder, which uses webpack under the hood.
We went from about 3.5 seconds to less than 2 seconds of build time, which is less of an improvement than we expected. Since we also have to run tailwind over our files, some of that additional `esbuild` speed seems to be relativized.
## Looking Ahead
Rebuilding the UI has paved the road to reduce a lot of the maintenance burden of Nx Console. It will allow us to move even quicker on building new features to provide the best developer experience possible for you.
Specifically, the updated architecture enabled us to build a (still secret and WIP) plugin feature for Nx Console. Just like Nx, there are always going to be things that are unique to your workspace. We want to make it easy for you to extend and modify Nx Console in ways that help you make the most of using it.
So keep your eyes peeled for announcements and let us know via GitHub or Twitter if you have any ideas! Wed love to chat.
## One more thing!
Nx Console is a tool by developers for developers and theres one thing we love — keyboard shortcuts. So of course we had to build some in. In addition to being keyboard-friendly and tabbable, you can do the following:
- `Cmd/Ctrl + Enter` to run the generator
- `Cmd/Ctrl + Shift + Enter` to start a dry run
- `Cmd/Ctrl + Shift + S` to focus the search bar and look for a specific option. Just `tab` to get back to the form
If the prettier UI and better performance havent convinced you, this surely will! 😉
---
## Learn more
- [Nx Docs](/getting-started/intro)
- [X/Twitter](https://twitter.com/nxdevtools) -- [LinkedIn](https://www.linkedin.com/company/nrwl/)
- [Nx GitHub](https://github.com/nrwl/nx)
- [Nx Official Discord Server](/community)
- [Nx Youtube Channel](https://www.youtube.com/@nxdevtools)
- [Speed up your CI](https://nx.app/)

View File

@ -0,0 +1,497 @@
---
title: Nx Conf 2023 — Recap
authors: [Juri Strumpflohner]
cover_image: '/blog/images/2023-10-13/featured_img.webp'
tags: [nx, nx-conf]
---
The 3rd edition of [Nx Conf](/conf), this year live from New York City. If you missed the talks, no worries, weve got you covered. This article does a brief recap of all the presentations and has links to the individual recordings and slides.
{% youtube src="https://www.youtube.com/embed/P_W8MwX25a0?si=gpjul3hdqmOiBdev" /%}
## Keynote
**Speaker:** [Juri Strumpflohner](https://twitter.com/juristr) & [Victor Savkin](https://twitter.com/victorsavkin)
{% youtube src="https://www.youtube.com/watch?v=WSqivWlEDFw" /%}
Juri Strumpflohner (me 😅) opens the keynote. He focuses on the Nx ecosystem, how it is [much more than monorepos](/getting-started/why-nx) by helping you integrate your framework of choice (whether that is Angular, React, or Vue) with tooling such as ESLint, Playwright, Cypress, Webpack, ESBuild, Vite etc.
![](/blog/images/2023-10-13/bodyimg1.webp)
Such integration capabilities really help push the developer productivity, whether thats in single-project workspaces or monorepos.
Juri also dives deeper into efforts from the team to provide high quality educational content around Nx and its capabilities. The [Nx docs](/getting-started/intro) have been restructured to follow the [Diataxis](https://diataxis.fr/) framework, dividing content into into learning-, task-, understanding- and information-oriented sections.
![](/blog/images/2023-10-13/bodyimg2.webp)
This makes it easier to find keep the content organized and focused and makes it easier for the reader to choose between solution-oriented [recipes](/recipes) vs learning-oriented [tutorials](/getting-started/tutorials).
The Nx team not only produces written content, but also video content mainly on the [Nx YouTube Channel](https://www.youtube.com/@nxdevtools). Juri shows some of the growth stats, with the channel now having more than 14k subscribers and around 3.7k hours of watch time per month. The channel serves mostly short-form videos about new releases, highlighting new features as well as longer-form tutorial videos.
The Nx Community also got a special place in the keynote. Juri highlighted in particular the transition from the “Nrwl Community Slack” to the new “Nx Community” on Discord.
![](/blog/images/2023-10-13/bodyimg3.webp)
Discord is built for managing communities and will open up a series of features to come. Things like a built-in forum to ask questions, the ability to highlight Nx events (e.g. the Nx live stream) and having powerful automation and moderation tools. Since the launch a couple of weeks ago, already 1500+ members have signed up. If you didnt sign up yet, go to [https://go.nx.dev/community](https://go.nx.dev/community).
Juri also highlighted the [Nx Champions](/community) program that got launched this year which helps with efforts to grow the Nx community. The program features members that stood out in the Nx community by helping support others, producing content or speaking at conferences about Nx and related tech.
Finally there were also two **announcements**:
- the Nx team decided to move off Medium for a better publishing experience but mainly also to avoid the paywall, something that the team has no control over. The new blog is currently being built and will be hosted at [blog](/blog). This also allows to better resurface information by being able to integrate blog articles into the Nx docs search and make them also accessible to the Nx AI Assistant which was the 2nd announcement.
- the **Nx AI Assistant** is an experiment the team is running to use AI to improve discoverability on the Nx docs. The ChatGPT powered assistant allows to ask natural language questions and responds based on the Nx docs training data, also including links to the sources. Try it out at [ai-chat](/ai-chat) and use the feedback thumbs-up/down buttons to help improve it over time 🙏.
Also, Nx is open source: [https://github.com/nrwl/nx](https://github.com/nrwl/nx). Contribute! And while youre there, dont forget to give us a star 😃.
**Victor Savkin** began the second segment of the keynote discussing the hidden expenses that often plague large teams.
![](/blog/images/2023-10-13/bodyimg4.webp)
He emphasized that as an organization grows, so does the supporting work. This isnt limited to just communication and management. Every artifact, be it code, process, or otherwise, needs to be assimilated into the broader context. This supporting work becomes intricate and demanding. He pointed out that the remote work trend, while it started with a lot of momentum, struggles in larger corporations because it demands even more coordination and effort. In contrast, smaller entities often outshine their larger counterparts due to the reduced overhead of supporting work.
One of Victors main points was the potential of software developers. Given the right tools and environment, they can achieve remarkable feats. CI is one such tool.
![](/blog/images/2023-10-13/bodyimg5.webp)
The state of CI configuration, Victor explained, is essentially a meticulous blueprint of tasks for the CI tool. This blueprint defines everything from the number of machines to their precise roles. Historically, these processes were performed manually, with individuals running test cases and subsequently approving or rejecting them.
However, scaling CI is a challenge. In larger workspace, up to 50 agents is pretty common, all requiring careful coordination and configuration to ensure tasks run in the correct order.
![](/blog/images/2023-10-13/bodyimg6.webp)
This might be manageable in a static repository setup, but monorepos introduce an added layer of complexity. Here, static “CI recipes” just wont cut it. The result is either an overuse of computational resources, leading to waste of money, or a painfully slow CI.
Victor critiqued that most current CI setups are:
- Oriented towards machines
- Low-level in their configuration
- Maintenance-heavy
- Detached from developer intentions
- Challenging to implement with monorepos
The big question: how can we revolutionize CI? Victor then showcased a demo workspace and highlights where the complexity of setting up CI comes from:
![](/blog/images/2023-10-13/bodyimg7.webp)
Even running e2e tests on CI requires:
- Building the application to be tested to produce an artifact
- That usually requires first building all libraries the app depends on, in the correct order
- Once all libraries are built, the application itself can be build
- Finally e2e can be run on the application artifact
Notice, since we want to do this as fast as possible, we want to parallelize these operations across machines. This involves also taking care of transferring build artifacts between them.
This is where **Nx Cloud Workflows** shines. It enables developers to draft CI configurations at a far more abstract level. Instead of delving into the minutiae of tasks, developers specify **what** commands to execute, with the **how** being implicitly managed by Nx Cloud.
```yaml
env:
NODE_OPTIONS: '--max_old_space_size=4096'
setup:
- name: Git Checkout
uses: 'nx-cloud-steps/checkout'
- name: Npm Install
uses: 'nx-cloud-steps/npm-install'
steps:
- name: CI Checks
parallel-scripts: |
nx affected -t build e2e --parallel=1
nx affected -t test lint --parallel=3
```
This elevated abstraction is possible because Nx Cloud & Nx are intimately familiar with the workspace, understanding the interdependencies between projects and tasks.
![](/blog/images/2023-10-13/bodyimg8.webp)
Beyond simplifying CI configurations, Nx Cloud Workflows helps optimize computational resource use. Agents are instantiated dynamically based on the projects requirements.
![](/blog/images/2023-10-13/bodyimg9.webp)
Victor highlighted that most current CI setups:
- Consistently utilize a predetermined number of agents, irrespective of the actual needs of the PR
- Possess non-reusable agents since each is tailored to a specific task, like building or testing
- Struggle with granular retries
As a metaphor Victor mentions that **Nx Cloud Workflows is to CI what S3 is to file uploads.**
![](/blog/images/2023-10-13/bodyimg10.webp)
Sounds interesting? Watch the entire talk below:
{% youtube src="https://www.youtube.com/watch?v=WSqivWlEDFw" /%}
## Nx Cloud Workflows: Next-Gen CI with First-Class Monorepo Support
**Speaker:** Simon Critchley
[Slides](https://docs.google.com/presentation/d/18EHfZ4UUtnAlzl15Ul_GPsr5lxNFtEEFPuT1uvxIEAk/edit?usp=sharing)
{% youtube src="https://youtu.be/JG1FWfZFByM?si=pNQNCLsbn5U0uSFB" /%}
Simon Critchley (Senior Software Architect at Nx) dives into the more technical details of the newly announced **Nx Cloud Workflows** (which is in private beta as of this writing).
![](/blog/images/2023-10-13/bodyimg11.webp)
Apart from the distributed caching, Nx Cloud already offers [DTE (Distributed Task Execution)](/ci/features/distribute-task-execution), which is a mechanism to seamlessly distribute tasks across various machines to achieve a high degree of parallelism. Up until now it was on you to configure your existing CI in a way to provision machines (agents) to Nx Cloud DTE. This required some fine-tuning to understand the best level of parallelism given the underlying monorepo and tasks that need to be run. If you have too many machines, itll be wasteful, if you have to few your CI will be slower.
Nx Cloud Workflow is an advancement of the DTE mechanism, where machines can be automatically provisioned and scaled based on the tasks that need to be run. Essentially it is a replacement for your existing CI but way smarter because it is able to leverage the knowledge it has about the Nx workspace, historical data and can thus do a series of optimizations. Traditional CI systems are running low-level commands on machines. Nx Cloud Workflows is task oriented instead: you tell it to run builds, tests, e2e, linting on the affected projects of the Nx workspace and it figures out how to split them up into fine-grained tasks and then distributes them among dynamically provisioned agents.
From a developers perspective, Nx Cloud Workflows are written in Yaml (or alternatively in JSON which is handy if you need to generate them). The format is very similar to existing CI providers to make sure the knowledge you have easily transfers. If you want an example, were dog-fooding Nx Cloud Workflows on the Nx repo (note the config will change as the product progresses): [https://github.com/nrwl/nx/blob/master/.nx/workflows/agents.yaml](https://github.com/nrwl/nx/blob/master/.nx/workflows/agents.yaml)
Simon then dives way deeper into the underlying architecture. In particular, how scheduling in Kubernetes works, which is used at the infrastructure level for Nx Cloud Workflows.
![](/blog/images/2023-10-13/bodyimg12.webp)
Workflow scheduling is handled by Kubernetes, specifically through a “Workflow Controller” that interacts with the K8s API Server and triggers Kubernetes Pods. Each Pod contains a Workflow executor and sidecar containers. All containers in the Nx Cloud workflow share a workspace volume for collaborative access. This allows for interesting optimizations in terms of sharing `node_modules` and restoring cache results.
![](/blog/images/2023-10-13/bodyimg13.webp)
The Workflow executor, written in Go, is responsible for receiving and executing workflow steps, capturing output, buffering logs, and communicating with the Workflow Controller. It also exposes its own grpc API for distributed locking, file system cache operations, and setting output parameters for each step, ensuring efficient workflow execution.
Nx Cloud Workflow is in its early development phase. The Kubernetes scheduler supports Docker based executors (basically any Docker image can be used as build image).
![](/blog/images/2023-10-13/bodyimg14.webp)
Windows, macOS and Linux is currently in development which will use a Cloud VM scheduler on the cloud of your choice.
## United by Nxcellent DX
**Speaker:** [Michael Hladky](https://twitter.com/Michael_Hladky)
[Slides](https://docs.google.com/presentation/d/1F6_x3Jiu_3b9tZpxQnCrU60Eb1FhYR8E5jVnb1t7DjA/edit?usp=sharing)
{% youtube src="https://youtu.be/i0UdoImryJQ?si=a5lx6VnS__qhiyex" /%}
Michael talks about how to leverage Nx to migrate multiple repositories into a monorepo to streamline development and increase developer productivity. He goes through
- how a project gets started
- how to prep moving to a monorepo from a polyrepo situation
- how to do the move in parallel
- how to measure the progress
The project usually starts with an architectural audit. That involves a detailed analysis of the underlying codebase, also including an executive summary for non technical folks. Once the audit is done, the actual move is planned and prepared. A part of that is to define “Nx migration goals”, like
- Single Version Policy
- Improved Maintenance
- Shared Infrastructure
- Shared Architecture
- Efficient Task Runs
- Frictionless Code-Sharing
- Frequent Deployments
- Decouple Deployment
To prioritize which ones to address first, Michael uses the following metaphor:
![](/blog/images/2023-10-13/bodyimg15.webp)
The important part here is to understand why “apples get bad” in the first place. Is it the harvesting process?
Once the priorities are defined and roadmap laid out, the goal is to **move in parallel.**
As part of that move the following gets produced:
- Migration Guide
- Training Program
- Communication Strategy
- Impact Measurement
The migration guide defines where to start, what the company goals are, how the deliveries will be integrated into the existing process and how the interaction with the various teams will happen.
The training program is there to tackle the bottleneck of missing knowledge, right from the beginning. Each stage of the process will have a different training content and collect feedback.
Finally the impact measurement; Nx Cloud already has graphs to measure how much time got saved in CI due to the improvements made. Michael mentions they go further, also measuring communication and productivity.
An interesting part is also how they perform the repository synching (from polyrepo to monorepo). They leverage an Nx plugin that
- contains shared build logic
- has rules to run that produce actionable feedback about what is needed to sync/align the polyrepo repository s.t. it can be merged into the monorepo
- it also allows to track progress and produce according reporting of where the company is at
Sounds interesting? Watch the full talk below:
{% youtube src="https://www.youtube.com/watch?v=i0UdoImryJQ" /%}
## Redefining Projects with Nx: A Dive into the New Inference API
**Speaker:** [Craigory Coppola](https://twitter.com/enderagent)
[Slides](https://craigory.dev/presentations/view/nx-conf-2023-inference/#1)
Demo Repo:
{% github-repository url="https://github.com/AgentEnder/inference-demo" /%}
{% youtube src="https://www.youtube.com/watch?v=bnjOu7iOrMg" /%}
Craigory did a deep dive into the new Nx inference API. This is particularly interesting if youre developing [Nx plugins](/extending-nx/intro/getting-started) or if you have leverage/build some more advanced automation for your Nx workspace.
What is project inference:
- how Nx reads your project configuration
- introduced between v13.3 and v14
- initially to support package-based monorepos which use package.json scripts rather than `project.json`. Generalizing the project config reading mechanism also allowed to define an API that community plugins can leverage and which allows to integrate even languages outside the JS ecosystem into Nx (e.g. where projects are just defined differently, such as .Net, Java, Python,..)
Inference API v1
- `projectFilePatterns` - identify files that represent the root of a project
- `registerProjectTargets` - takes a project file and converts to a list of targets that Nx knows how to run
Shortcomings have been
- strict 11 mapping between project files and projects
- the logic of finding proj files and targets had to be decoupled which introduced potential failure points
- no way to add dynamic metadata to a project
Project graph API v2
- `createNodes` - finds graph nodes based on files on disk
- `createDependencies` - finds edges to be added to the graph
Still 2 parts, but theres no overlap between these two and they have very specific purposes.
`createNodes` is a tuple composed of:
- `projectFilePattern`
- `CreateNodesFunction` It can return a map of projects and external nodes, so there's no more the shortcoming of a 1-1 as it was in v1 API. With this new setup multiple plugins might detect the same project. Nx merges the configuration that has been identified. Kinda what happens right now if you mix `package.json` and `project.json` targets in an Nx workspace.
Craigory demos the inference API by building a spell checking plugin for Nx.
`createDependencies` replaces `processProjectGraph`. It is more constrained but mainly because things like adding nodes should be done in a different function now (e.g. `createNodes`). This has also advantages for Nx itself, as it knows all nodes will have been created after the `createNodes` has terminated.
This API is still marked as experimental, next steps will be
- mark as stable
- remove v1 API after deprecation period
- plugin authors should start looking into the new API, provide feedback and think about migration scenarios
## Package-based to Integrated: One Small Step or One Giant Leap?
**Speaker:** [Isaac Mann](https://twitter.com/MannIsaac)
[Slides](https://drive.google.com/file/d/1V1ocFIjYkRrcNXWuvUK1T8dQEceDcjvw/view?pli=1)
Repo:
{% github-repository url="https://github.com/isaacplmann/space-station-tracker/tree/final" /%}
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=nY0_o7zWBLM" /%}
Isaac initiated his talk by drawing an analogy between software development and the moon landing. While Neil Armstrong is often singularly celebrated for setting foot on the moon, Isaac emphasized that the monumental achievement was the collective effort of countless individuals on the ground crew, from engineers to support staff.
> Isaac: Nx wants to be that indispensable ground support crew for developers, allowing them to push the boundaries of software development.
Currently, Nx offers support for two predominant monorepo styles: package-based and integrated.
### Package-based Monorepos:
- These are tailored for flexibility and innovation.
- Packages within this setup can have diverse configurations.
- Every package boasts its individual node_modules and dependencies.
- Crucially, each can be upgraded independently, offering a high degree of autonomy.
### Integrated Monorepos:
- These are structured to prioritize consistency and maintainability.
- Updates within this framework are automated, and the setup adheres to a single-version policy.
Isaac then drew parallels between the Apollo program and the package-based mindset. The Apollo missions were evolutionary in nature, constantly pushing the envelope and embracing innovation with each successive mission. However, this trailblazing approach wasnt without its perils, as evidenced by tragic accidents. This experimental approach was feasible because the core team, responsible for creating the setup, remained consistent throughout the projects duration.
In contrast, Isaac likened the International Space Station (ISS) to the integrated mindset. The ISS epitomizes the challenges of coordination, given its international collaboration involving numerous countries. Emphasizing longevity, the ISS necessitates that every new crew member be proficient in operating its intricate machinery.
Shifting gears, Isaac delved into a hands-on demonstration. He outlined the process of transitioning from a package-based monorepo to an integrated one, including demoing:
- Initializing Nx with `nx init`.
- Establishing new projects to facilitate type sharing across applications.
- Harnessing the power of the Nx graph visualization to navigate and understand the project structure.
- Employing the module boundary rule, ensuring constraints are maintained in the revamped structure.
- Devising a novel Nx generator, streamlining the process of setting up new libraries within the workspace.
- Finally, he showcased the seamless upgrade mechanism, ensuring the workspace is always aligned with the latest version.
## Nxt Level Publishing
**Speaker:** [James Henry](https://twitter.com/MrJamesHenry)
[Slides](https://main--idyllic-dieffenbachia-3699ec.netlify.app/1)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=p5qW5-2nKqI" /%}
James Henry unveils the new versioning and publishing functionality that is now built into Nx itself.
Nx has always given you the building blocks to do integrate the versioning and publishing; you could easily use Lerna with Nx, or changesets, release-it as well as a series of Nx Community plugins
The nx core team decided to go into this problem space and provide an opinionated approach that is easy to use within existing Nx workspaces; something that works and scales; also something that works outside the JS ecosystem which Nx can also be used with
New command `nx release`
- `nx release version` to determine and apply version updates
- `nx release` changelog generate a CHANGELOG.md file and optional GitHub releases based on git commits
- `nx release` publish takes a newly versioned project and publishes them to a remote registry (e.g. NPM)
This new command is not tight to `package.json` files but is general purpose; clearly publishing JS/TS packages is the main use case right now in Nx
Important to note that all the existing plugins are still valid and will still be going forward.
James goes forward demoing how `nx release` works in a simple NPM package: [jump directly into the video](https://www.youtube.com/watch?si=xoIW0Q-mQWTGgrek&t=411&v=p5qW5-2nKqI&feature=youtu.be).
Nx release can be added to any npm package. All thats needed is the `nx` and `@nx/js` package and a `nx: {}` node in the `package.json`.
Nx release has a dry-run mode; `nx release version --dry-run`. This will output a diff of what version would be created and on which `package.json` files (if there are multiple).
The same mechanism also works for the `changelog` command, e.g. `nx release changelog --dry-run`. In addition to just dry-run, the `changelog` command also has an `--interactive` mode that allows to get the generated changelog in an interactive editor where you can adjust and add extra stuff.
James then moves on to show how the release process works in a monorepo scenario using pnpm workspaces, where there are multiple NPM packages that need to be published: [jump to the video](https://www.youtube.com/watch?si=BvFvRQjgtwOxu-Dv&t=777&v=p5qW5-2nKqI&feature=youtu.be).
The monorepo has a `pkg-a` and `pkg-b` where there's a relationship between them as follows: `pkg-a --> pkg-b`.
If the `version` command is used in such scenario, it will also be applied to the dependent packages (`pkg-b`) since Nx knows about the dependencies via the project graph.
A nice feature is also that the changelog will...
- automatically group first by type (e.g. features grouped together, fixes etc)
- within each type grouping it will be grouped by scope such as pkg-a etc which will be alphabetized
- if theres a fix on the entire repo thatll come first before the per-package changes
`nx release changelog --create-release=github` allows to also automatically push the changelog to a Github release.
When running `nx release publish`, Nx also takes into account the right order of how the packages should go on NPM. Like if there's a relationship `pkg-a --> pkg-b`, Nx automatically detects such dependency and will go and publish pkg-b first and then `pkg-a` which depends on it. That to make sure that even if there's a network error in between packages, you'd still not break anything.
Future roadmap:
- ability to customize how the release works by defining it in `nx.json`
- “release groups” will allow to group packages and define whether versions should be in sync or versioned independently; filter by projects, or even just publish a subset of projects of a workspace
- publishing also automatically takes into account the provenance data on NPM
## Lightning Talk: What if your stories were — already — your e2e tests?
**Speaker:** [Katerina Skroumpelou](https://twitter.com/psybercity)
[Slides](https://drive.google.com/file/d/1XY8tefqqcM4k4swNPhlSrJiKxurMGiup/view)
Repo
{% github-repository url="https://github.com/mandarini/storybook-play" /%}
[Live Demo](https://storybook-play.vercel.app/?path=/story/app--primary)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=SWlvsDNXCsQ" /%}
Katerina starts by introducing the concept of UI testing for error detection, usability verification, regression prevention but also for validating the user journey and making sure the software is reliability.
Nx had support to use Storybook for doing component-level tests, both using a Cypress e2e setup as well as Cypress component tests. This is where Nx would fill in to launch Storybook to render a single component and then launch Cypress and run dedicated tests against that served component.
Meanwhile Storybook has introduced so-called [Interaction Tests](https://storybook.js.org/docs/writing-tests/interaction-testing) which allow you to do something similar.
You can read more on [our docs](https://storybook.js.org/docs/writing-tests/interaction-testing). Or check out our corresponding video on Youtube:
{% youtube src="https://www.youtube.com/watch?v=SaHoUx-TUs8" /%}
## Lightning Talk: Nx Cloud Demo
**Speaker:** [Johanna Pearce](https://twitter.com/jhannapearce)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=xc6fJpwk4Lo" /%}
Johanna gives us a tour through the Nx Cloud UI. So theres not much to say other than [go watch the talk](https://www.youtube.com/watch?v=xc6fJpwk4Lo&feature=youtu.be) :)
## From DIY to DTE — An Enterprise Experience
**Speaker:** Adrian Baran
[Slides](https://docs.google.com/presentation/d/1LPgCuYoVEIJqqhk8TLq4RAcHrJ5phJuiHHEGA3y_aws/edit?pli=1#slide=id.g25ac4b0c8f6_0_72)
{% youtube src="https://www.youtube.com/watch?v=MsUN0wQHPAs" /%}
Adrian Baran, a Senior Software Engineer at Cisco, embarked on his talk by presenting the status quo of their monorepo which encompasses approximately 150 projects at the start of the experiment.
![](/blog/images/2023-10-13/bodyimg16.webp)
Their CI workflow is powered by CircleCI. For the purpose of clarity, Adrian zoomed in on the frequently executed tasks: `build`, `lint`, and `test`. To execute these tasks, they leverage the [Nx affected](/features/run-tasks#run-tasks-on-projects-affected-by-a-pr) command to avoid running all commands on all projects.
![](/blog/images/2023-10-13/bodyimg17.webp)
Despite that, on average CI runs on PRs still lingered around the 1-hour mark. The immediate remedy? Harnessing CircleCI to parallelize tasks across multiple machines, while continuing to utilize Nx affected. Adrian wrote about that journey in a blog post last year, titled [Nx Affected + CircleCI Parallelism = Faster CI/CD Pipelines](https://medium.com/@abaran30/nx-affected-circleci-parallelism-faster-ci-cd-pipelines-b4edc4caaaac).
They ended up with a loooot of parallelism:
![](/blog/images/2023-10-13/bodyimg18.webp)
The outcome was promising though — they witnessed a significant reduction, approximately 75%, in execution time, even in scenarios where all projects were influenced by a PR. Done, right?
However, as their monorepo burgeoned to encompass 400+ projects, the DIY solution began to show its limitations. The incessant parallelization implied the addition of more agents, leading to escalating costs. Furthermore, they observed considerable variability in task durations.
![](/blog/images/2023-10-13/bodyimg19.webp)
Their workaround was to manually allocate specific tasks to distinct agents. This method in addition to capping on a max number of agents helped keep costs under control, but regrettably, it also compromised performance due to the lowered parallelism.
This is when Cisco looked into running an [Nx Enterprise](/enterprise) pilot, specifically with the goal of adopting [DTE](/ci/features/distribute-task-execution). In the 2nd part of the talk, Adrian outlined their journey of setting up DTE, starting with the generation of the initial CI setup:
```shell
nx g @nx/workspace:ci-workflow --ci=circleci
```
They tuned the generated configuration, but failed to get it properly running, many of the issues leading to resource exhaustion. As a result, they went the route of going on a 11 mapping setup between DIY and DTE. That made sure the DTE setup was as close as possible and allowed for a gradual transition approach, which proved invaluable, allowing them to maintain stability whilst progressively migrating to DTE.
The comparative analysis of their DIY solution versus DTE revealed roughly equivalent performance metrics. However, the DTE framework shone in its simplicity, maintainability, and resource efficiency, achieving similar outcomes with a 75% reduction in resource utilization.
![](/blog/images/2023-10-13/bodyimg20.webp)
### Adrians Key Insights:
- DTE setup warrants a dual-pronged strategy: an immediate plan for initiation and a long-term vision for transition. Notably, Nx DTE supports incremental adoption.
- Patience is paramount during the tuning phase to determine the optimal number of agents.
- While results might vary, Adrian humorously assures that one can always bank on the Nx team for support. 😅
## Vanquishing Deployment Dragons with Nx wizardry
**Speaker:** [Miroslav Jonas](https://twitter.com/meeroslav)
[Slides](https://speakerdeck.com/meeroslav/vanquishing-deployment-dragons-with-nx-wizardry)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=jGF8vo2ChfI" /%}
Theres not much to say here. Lean back and [immerse yourself into the world of dragons and wizards.](https://www.youtube.com/watch?v=jGF8vo2ChfI)
## Optimizing your OSS infrastructure with Nx Plugins
**Speaker:** [Brandon Roberts](https://twitter.com/brandontroberts)
[Slides](https://github.com/brandonroberts/nx-conf-2023/blob/main/nx-oss-infrastructure.pdf)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=bNuXH25CTO0" /%}
Brandon discussed his work on the open-source tool [Analog](https://analogjs.org/) and how Nx Plugins are crucial to its development.
He introduced Analog, shedding light on its foundational ecosystem and the its core features.
![](/blog/images/2023-10-13/bodyimg21.webp)
Nx plays a crucial role in helping integrate and maintain these different tools:
![](/blog/images/2023-10-13/bodyimg22.webp)
How? Brandon dives straight into it by explaining how Nx plugins in particular can be useful, explaining:
- features of Nx plugins such as generators, executors, automated migrations, presets and how to use them just locally to automate your workspace
- how to create a new plugin
- the anatomy of a generator and how they can be useful in scaffolding new setups, but also integrating technology, like adding tRPC to your stack, etc.
- similarly Nx executors provide a thin abstraction layer over the actual commands, allowing the plugin developer to update the underlying tooling without necessarily disrupting the end user
- most importantly allowing to write automatic migrations he can leverage with Analog, like running `nx migrate @analogjs/platform@latest` to update a given workspace automatically to the latest version, across potentially breaking changes
Brandon highlighted a pivotal aspect for OSS package/framework authors: the power to not just assimilate into pre-existing Nx workspaces via custom Nx plugins but also to steer the entire workspace setup process. This is particularly beneficial when tailored setups specific to individual use cases are required. By leveraging an [Nx preset](/extending-nx/recipes/create-preset), one can achieve this tailored configuration. Brandon also touched upon the possibility of advancing further by constructing an [install package](/extending-nx/recipes/create-install-package) through Nx.
## Level Up Your Productivity with Nx Console
**Speaker:** [Jonathan Cammisuli](https://twitter.com/jcammisuli) & [Max Kless](https://twitter.com/MaxKless)
[Slides](https://github.com/Cammisuli/nx-conf-2023/tree/main/tools/presentation)
Watch on YouTube:
{% youtube src="https://www.youtube.com/watch?v=TTjVcWCdwVY" /%}
Jon and Max are the masterminds behind [Nx Console](/getting-started/editor-setup#integrate-with-editors), the Nx IDE extension for Code and JetBrains.
They mention the release of the JetBrains IDE support earlier this year and give a big shoutout to [Issam](https://github.com/iguissouma) who was the original author of the Nx Console Webstorm community plugin and who helped a lot with the official Nx Console version for JetBrains IDE.
Jon and Max demo the new IntelliJ version of Nx console and how it properly integrates as well as how they wrote the new UI completely from the ground up [using Lit](/blog/2023-06-29-nx-console-gets-lit).
Clearly they did not spare showing off the awesome new graph capabilities built into Nx Console that allow you to directly navigate to files.
![](/blog/images/2023-10-13/bodyimg23.webp)
## Thats a wrap
If you enjoyed these, [subscribe to our YouTube channel](https://www.youtube.com/@nxdevtools) where we keep releasing educational content and fun videos.
---
## Learn more
- [Nx Docs](/getting-started/intro)
- [X/Twitter](https://twitter.com/nxdevtools) -- [LinkedIn](https://www.linkedin.com/company/nrwl/)
- [Nx GitHub](https://github.com/nrwl/nx)
- [Nx Official Discord Server](/community)
- [Nx Youtube Channel](https://www.youtube.com/@nxdevtools)
- [Speed up your CI](https://nx.app/)

View File

@ -274,7 +274,7 @@ Thats all for now folks! Were just starting up a new iteration of developm
## Learn more ## Learn more
- [Nx Docs](https://nx.dev/getting-started/intro) - [Nx Docs](/getting-started/intro)
- [X/Twitter](https://twitter.com/nxdevtools) -- [LinkedIn](https://www.linkedin.com/company/nrwl/) - [X/Twitter](https://twitter.com/nxdevtools) -- [LinkedIn](https://www.linkedin.com/company/nrwl/)
- [Nx GitHub](https://github.com/nrwl/nx) - [Nx GitHub](https://github.com/nrwl/nx)
- [Nx Official Discord Server](https://go.nx.dev/community) - [Nx Official Discord Server](https://go.nx.dev/community)

View File

@ -0,0 +1,335 @@
---
title: Unit Testing Expo Apps With Jest
authors: [Emily Xiong]
cover_image: '/blog/images/2023-11-22/featured_img.webp'
tags: [nx, unit testing]
---
In my latest [blog](https://dev.to/nx/step-by-step-guide-to-creating-an-expo-monorepo-with-nx-3b17), I successfully navigated through the steps of setting up an Expo Monorepo with [Nx](https://nx.dev). The next challenge? Testing! This blog dives into:
- Crafting effective unit tests for Expo components utilizing Jest
- Addressing common issues encountered during unit testing
Repo:
{% github-repository url="https://github.com/xiongemi/nx-expo-monorepo" /%}
## Stacks
Heres my setup
- Testing framework: [jest](https://jestjs.io/)
- Testing library: [@testing-library/react-native](https://callstack.github.io/react-native-testing-library/)
- Jest Preset: [jest-expo](https://www.npmjs.com/package/jest-expo)
## Writing and Running Unit Tests
When you use Nx, it not only configures and sets up Jest, but also creates a default unit test for every expo component that is being generated. Heres what that looks like:
```typescript
import { render } from '@testing-library/react-native';
import React from 'react';
import Loading from './loading';
describe('Loading', () => {
it('should render successfully', () => {
const { root } = render(<Loading />);
expect(root).toBeTruthy();
});
});
```
To run all unit tests for a given project, use:
```shell
npx nx test <project-name>
```
Heres the output of running this for my example app:
![terminal output](/blog/images/2023-11-22/bodyimg1.webp)
When it comes to writing tests, the [React Native Testing Library](https://callstack.github.io/react-native-testing-library/docs/api-queries) is a game-changer for writing cleaner unit tests in React Native applications. Its intuitive query API simplifies the process of selecting elements within your components, making it straightforward to write more maintainable and readable tests. You mark elements with a `testID`
```html
<Headline testID="title">{film.title}</Headline>
```
Then in the test file, you can use the function `getByTestId` to query `testID`:
```typescript
const { getByTestId } = render(<your component>);
expect(getByTestId('title')).toHaveTextContent(...);
```
You can find more options for querying elements on the official React Native Testing Library docs: [https://callstack.github.io/react-native-testing-library/docs/api-queries](https://callstack.github.io/react-native-testing-library/docs/api-queries).
## Troubleshooting Common Issues When Writing Tests
However, unit tests do not always pass. Here are some common errors I ran into and how to resolve them.
### **Error: AsyncStorage is null.**
I am using the library `@react-native-async-storage/async-storage`, and I got the below error when running unit testing:
```shell
\[@RNC/AsyncStorage\]: NativeModule: AsyncStorage is null.
To fix this issue try these steps:
• Rebuild and restart the app.
• Run the packager with \`--reset-cache\` flag.
• If you are using CocoaPods on iOS, run \`pod install\` in the \`ios\` directory and then rebuild and re-run the app.
• If this happens while testing with Jest, check out docs how to integrate AsyncStorage with it: https://react-native-async-storage.github.io/async-storage/docs/advanced/jest
If none of these fix the issue, please open an issue on the Github repository: https://github.com/react-native-async-storage/async-storage/issues
5 | import { ReactQueryDevtools } from '@tanstack/react-query-devtools';
6 | import { Platform } from 'react-native';
> 7 | import AsyncStorage from '@react-native-async-storage/async-storage';
```
The issue is that `@react-native-async-storage/async-storage` library can only be used in `NativeModule`. Since unit testing with Jest only tests JS/TS file logic, I need to mock this library.
In the apps test-setup.ts file, add the below lines:
```typescript
jest.mock('@react-native-async-storage/async-storage', () =>
require('@react-native-async-storage/async-storage/jest/async-storage-mock')
);
```
## Error: Could not find “store”
I am using Redux for state management, and I got this error for my stateful components:
```
Could not find "store" in the context of "Connect(Bookmarks)". Either wrap the root component in a <Provider>, or pass a custom React context provider to <Provider> and the corresponding React context consumer to Connect(Bookmarks) in connect options.
```
To fix this, the simple way is to mock a redux store. I need to install [redux-mock-store](https://github.com/reduxjs/redux-mock-store) and its typing:
{% tabs %}
{% tab label="npm" %}
```shell
npm install redux-mock-store @types/redux-mock-store --save-dev
```
{% /tab %}
{% tab label="yarn" %}
```shell
yarn add redux-mock-store @types/redux-mock-store --dev
```
{% /tab %}
{% /tabs %}
Then I can create a mock store using this library like the below code:
```typescript
import configureStore, { MockStoreEnhanced } from 'redux-mock-store';
const mockStore = configureStore<any>(\[\]);
let store: MockStoreEnhanced<any>;
beforeEach(() => {
store = mockStore({});
store.dispatch = jest.fn();
});
```
For example, one of my stateful components unit test will become:
```typescript
import React from 'react';
import { render } from '@testing-library/react-native';
import { Provider } from 'react-redux';
import configureStore, { MockStoreEnhanced } from 'redux-mock-store';
import { RootState, initialRootState } from '@nx-expo-monorepo/states/cat';
import Bookmarks from './bookmarks';
describe('Bookmarks', () => {
const mockStore = configureStore<RootState>(\[\]);
let store: MockStoreEnhanced<RootState>;
beforeEach(() => {
store = mockStore(initialRootState);
store.dispatch = jest.fn();
});
it('should render successfully', () => {
const { container } = render(
<Provider store={store}>
<Bookmarks />
</Provider>
);
expect(container).toBeTruthy();
});
});
```
The above code will apply the initial redux state to my components.
### Error: No QueryClient set
Because I use [TanStack Query](https://tanstack.com/query/latest) , when I run unit tests, I have this error:
```
No QueryClient set, use QueryClientProvider to set one
```
This error occurred because I used `useQuery` from `@tanstack/react-query` in my component; however, in this unit test, the context of this hook is not provided.
To solve this, I can just mock the `useQuery` function:
```typescript
import * as ReactQuery from '@tanstack/react-query';
jest.spyOn(ReactQuery, 'useQuery').mockImplementation(
jest.fn().mockReturnValue({
data: 'random cat fact',
isLoading: false,
isSuccess: true,
refetch: jest.fn(),
isFetching: false,
isError: false,
})
);
```
### Error: Couldnt find a navigation object
If you use `@react-navigation` library for navigation, and inside your component, there are hooks from this library like `useNavigation` and `useRoute`, you are likely to get this error:
```
Couldn't find a navigation object. Is your component inside NavigationContainer?
```
The fix this, I need to mock the `@react-nativgation/native` library. In the apps test-setup.ts file, I need to add:
```typescript
jest.mock('@react-navigation/native', () => {
return {
useNavigation: () => ({
navigate: jest.fn(),
dispatch: jest.fn(),
setOptions: jest.fn(),
}),
useRoute: () => ({
params: {
id: '123',
},
}),
};
});
```
### SyntaxError: Unexpected token export
I got this error when using a library with ECMAScript Module (ESM), such as `[udid](https://github.com/uuidjs/uuid)`:
```
/Users/emilyxiong/Code/nx-expo-monorepo/node\_modules/uuid/dist/esm-browser/index.js:1
({"Object.<anonymous>":function(module,exports,require,\_\_dirname,\_\_filename,jest){export { default as v1 } from './v1.js';
^^^^^^
SyntaxError: Unexpected token 'export'
5 | import { connect } from 'react-redux';
6 | import 'react-native-get-random-values';
> 7 | import { v4 as uuidv4 } from 'uuid';
```
Jest does not work with ESM out of the box. The simple solution is to map this library to the CommonJS version of this library.
In the apps `jest.config.ts`, there should be an option called `moduleNameMapper`. The library I used is called `uuid`, so I need to add the map `uuid: require.resolve(uuid)` under `moduleNameMapper`. So when the code encounters imports from `uuid` library, it will resolve the CommonJS version of it:
```typescript
module.exports = {
moduleNameMapper: {
uuid: require.resolve('uuid'),
},
};
```
Alternatively, I can also mock this library in the test files:
```typescript
import { v4 as uuidv4 } from 'uuid';
jest.mock('uuid', () => {
return {
v4: jest.fn(() => 1),
};
});
```
## Error: Jest encountered an unexpected token
I got this error when I was importing from a library such as [react-native-vector-icons](https://github.com/oblador/react-native-vector-icons):
```
console.error
Jest encountered an unexpected token
Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.
Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.
By default "node\_modules" folder is ignored by transformers.
Here's what you can do:
• If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
• If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
• To have some of your "node\_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
• If you need a custom transformation specify a "transform" option in your config.
• If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.
You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/configuration
For information about custom transformations, see:
https://jestjs.io/docs/code-transformation
```
To fix this, add this library name to `transformIgnorePatterns` in the app's jest.config.ts.
What is `transformIgnorePatterns`? The `transformIgnorePatterns` allows developers to specify which files shall be transformed by Babel. `transformIgnorePatterns` is an array of regexp pattern strings that should be matched against all source file paths before the transformation. If the file path matches any patterns, it will not be transformed by Babel.
By default, Jest will ignore all the files under node_modules and only transform the files under the projects src.
However, some libraries such as `react-native-paper` or `react-native-svg`, the library files are in `.ts` or `.tsx`. These files are not compiled to `js`. So I need to add these libraries' names to `transformIgnorePatterns`, so these libraries will be transformed by Babel along with my project. source file. The default generated `jest.config.js` already has:
```
transformIgnorePatterns: \[
'node\_modules/(?!((jest-)?react-native|@react-native(-community)?)|expo(nent)?|@expo(nent)?/.\*|@expo-google-fonts/.\*|react-navigation|@react-navigation/.\*|@unimodules/.\*|unimodules|sentry-expo|native-base|react-native-svg)',
\]
```
If I have an error related to a library with an unexpected token, I need to check whether they are compiled or not.
- If this library source files are already transformed to `.js`, then its name should match regex, so it would be ignored, so it will NOT be transformed.
- If this library source files are NOT transformed to `.js` (e.g. still in `.ts` or `.tsx`), then its name should NOT match regex, so it will be transformed.
## Summary
Here are some common errors that I will probably run into while doing unit testing. The solution to most problems is to find a way to mock a library that is not relevant to my component logic.
With Nx, you do not need to explicitly install any testing library, so you can dive right in and focus on writing the tests rather than spending time on setup.
## Learn more
- [Add Cypress, Playwright, and Storybook to Nx Expo Apps](https://medium.com/@emilyxiong/add-cypress-playwright-and-storybook-to-nx-expo-apps-1d3e409ce834)
- 🧠 [Nx Docs](/getting-started/intro)
- 👩‍💻 [Nx GitHub](https://github.com/nrwl/nx)
- 💬 [Nx Community Discord](/community)
- 📹 [Nx Youtube Channel](https://www.youtube.com/@nxdevtools)
- 🚀 [Speed up your CI](https://nx.app/)

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 409 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 169 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 695 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB