`--parallel` is not being respected when running with `run-many`

[yharaskrik]

Current Behavior

When I run: nx run-many -t integration --parallel 2

I am seeing more than 2 tasks get run at a time.

Expected Behavior

Should respect parallel.

GitHub Repo

No response

Steps to Reproduce

1. use run-many with a task and have parallel set.
2. Maybe has to do with continuous tasks as I have a number of those?

Nx Report

➜ nx report

 NX   Report complete - copy this into the issue template

Node           : 24.12.0
OS             : darwin-arm64
Native Target  : aarch64-macos
bun            : 1.3.5

nx                     : 22.1.0
@nx/js                 : 22.1.0
@nx/jest               : 22.1.0
@nx/eslint             : 22.1.0
@nx/workspace          : 22.1.0
@nx/angular            : 22.1.0
@nx/cypress            : 22.1.0
@nx/devkit             : 22.1.0
@nx/eslint-plugin      : 22.1.0
@nx/express            : 22.1.0
@nx/module-federation  : 22.1.0
@nx/nest               : 22.1.0
@nx/node               : 22.1.0
@nx/playwright         : 22.1.0
@nx/plugin             : 22.1.0
@nx/rspack             : 22.1.0
@nx/storybook          : 22.1.0
@nx/vite               : 22.1.0
@nx/vitest             : 22.1.0
@nx/web                : 22.1.0
@nx/webpack            : 22.1.0
@nx/docker             : 22.1.0
typescript             : 5.8.3

Failure Logs

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

I would actually be ok with this if it was just dependant tasks not respecting it (would love to be able to define max parallels when running commands like we can with DTE distribution configs, so for example graphql-codegen can run at a higher parallel than say I give the run-many -t integration.

[yharaskrik]

I will upgrade to Nx 22.3.3 and see if that helps with anything.

[yharaskrik]

Ok well once updating to 22.3.3 my continuous tasks are just marked as failed (even though those targets do start up fine)

Couple examples:

These targets start up fine if I run them outside of a task pipeline.

Here is me running serve-static by itself (vs, that being a task started by the integration target)

[yharaskrik]

Ok ya my continuous tasks start not working if I upgrade past 22.1.1.

They just get marked as failed, look like they get killed.

So can't actually test any versions past 22.1.1 to see if parallel is still working or not.

On 22.1.1, parallel still does not abide by parallel

This may be why my continuous tasks were not working right when trying to get this working on agents: https://discord.com/channels/1143497901675401286/1446604501069856818/1461144597664370800

[leosvelperez]

Re: --parallel not being respected:

This is intentional design. Continuous tasks are added to the parallel limit because they run indefinitely alongside regular tasks. Otherwise, if you have 5 parallel tasks and --parallel=2, the first two continuous tasks would run indefinitely, blocking any other task from running.

---

Re: tasks marked as failed:

If I understand correctly, you're observing that the integration tasks ran successfully, but the continuous tasks were marked as failures. Or did any integration task fail due to killed/errored continuous tasks? Did you verify if any continuous task was a legitimate failure? Or did all of them look like they should have succeeded?

I have a lead on how we handle completing continuous tasks, but it would be good to confirm the questions above.

[yharaskrik]

Re: --parallel not being respected:

While I understand what you mean and the logic behind it, how in that case can I tell Nx: "Run all my integrations tests, but only 2 at a time, plus manage the dependant tasks for that integration has the continuous tasks it needs".

It seems like maybe that is not possible since the # of continuous tasks is added to parallel?

This is essentially a moot point for us if we can get Nx Agents to work because then the distribution config for Agents handles it for us but right now integration is still on CircleCI and is being binned across 2 nodes so I need to rely on parallel. Currently we have this complex setup of running each server in a background step and then having a sync step wait for all servers to start then run integration with parallel 2, no continuous tasks in sight.

Re: tasks marked as failed, will open a separate PR and then link it here as I believe it is a separate issue and something we have been fighting for a month or two now.

[techfg]

Re: --parallel not being respected:

This is intentional design. Continuous tasks are added to the parallel limit because they run indefinitely alongside regular tasks. Otherwise, if you have 5 parallel tasks and --parallel=2, the first two continuous tasks would run indefinitely, blocking any other task from running.

@leosvelperez - Can you provide some addition details regarding this?

I opened #31494 back in June of last year and haven't received any updates. Still having the same issue as of v22.4.1 and your comment makes me wonder how the "continuous tasks" are added to the parallel limit and if that addition is causing the other targets to run in parallel as well. There is a repro available in #31494. Thank you!

[leosvelperez]

@techfg yes, that's exactly what's happening. We'll update the task runner so that discrete tasks respect the parallel limit.