Skip to content

Conversation

LingyuCoder
Copy link
Contributor

@LingyuCoder LingyuCoder commented Jul 3, 2025

Summary

Improve parallelism when calculating the real content hashes.

By recording the processed hashes and the hashes on which the current asset depends, if all have been processed, it is added to the current processing batch. Then process the batch together to prevent the number of tasks processed in parallel at one time from being too small, which would make the process approach sequential processing.

Before:
image

After:
image

Related links

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

Copy link

netlify bot commented Jul 3, 2025

Deploy Preview for rspack canceled.

Name Link
🔨 Latest commit 9bb6d96
🔍 Latest deploy log https://app.netlify.com/projects/rspack/deploys/686747ca725c1c00084db36b

@github-actions github-actions bot added release: performance release: performance related release(mr only) team The issue/pr is created by the member of Rspack. labels Jul 3, 2025
Copy link
Contributor

github-actions bot commented Jul 3, 2025

📦 Binary Size-limit

Comparing binary size with Commit: refactor: remove js stats warning (#10900) by Cong-Cong Pan

❌ Size increased by 45.38KB from 59.80MB to 59.84MB (⬆️0.07%)

Copy link

codspeed-hq bot commented Jul 3, 2025

CodSpeed Performance Report

Merging #10888 will not alter performance

Comparing perf/parallel-real-content-hash (9bb6d96) with main (b0069f1)

🎉 Hooray! codspeed-rust just leveled up to 2.7.2!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

✅ 16 untouched benchmarks

@LingyuCoder LingyuCoder changed the title perf: more parallel real content hash plugin perf: improve parallelism of real content hash plugin Jul 3, 2025
@LingyuCoder LingyuCoder force-pushed the perf/parallel-real-content-hash branch from cacda2a to 3ef109a Compare July 3, 2025 12:17
Copy link
Contributor

github-actions bot commented Jul 3, 2025

📝 Benchmark detail: Open

Name Base (2025-07-03 0932978) Current Change
10000_big_production-mode_disable-minimize + exec 33.8 s ± 650 ms 34.4 s ± 1.55 s +1.90 %
10000_development-mode + exec 1.88 s ± 31 ms 1.86 s ± 33 ms -1.18 %
10000_development-mode_hmr + exec 698 ms ± 5.3 ms 686 ms ± 12 ms -1.70 %
10000_production-mode + exec 1.92 s ± 42 ms 1.92 s ± 45 ms +0.19 %
10000_production-mode_persistent-cold + exec 2.07 s ± 56 ms 2.09 s ± 21 ms +0.59 %
10000_production-mode_persistent-hot + exec 1.46 s ± 31 ms 1.53 s ± 170 ms +4.93 %
arco-pro_development-mode + exec 1.8 s ± 50 ms 1.82 s ± 43 ms +0.82 %
arco-pro_development-mode_hmr + exec 371 ms ± 2.1 ms 371 ms ± 3 ms +0.16 %
arco-pro_production-mode + exec 3.27 s ± 49 ms 3.33 s ± 87 ms +1.92 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 3.41 s ± 105 ms 3.42 s ± 169 ms +0.30 %
arco-pro_production-mode_persistent-cold + exec 3.37 s ± 138 ms 3.4 s ± 186 ms +0.85 %
arco-pro_production-mode_persistent-hot + exec 2.09 s ± 55 ms 2.1 s ± 36 ms +0.56 %
arco-pro_production-mode_traverse-chunk-modules + exec 3.34 s ± 97 ms 3.42 s ± 223 ms +2.41 %
large-dyn-imports_development-mode + exec 2.1 s ± 50 ms 2.08 s ± 47 ms -0.97 %
large-dyn-imports_production-mode + exec 2.05 s ± 13 ms 2.03 s ± 38 ms -0.99 %
threejs_development-mode_10x + exec 1.45 s ± 15 ms 1.47 s ± 21 ms +1.38 %
threejs_development-mode_10x_hmr + exec 824 ms ± 12 ms 825 ms ± 20 ms +0.10 %
threejs_production-mode_10x + exec 4.61 s ± 198 ms 4.71 s ± 276 ms +2.09 %
threejs_production-mode_10x_persistent-cold + exec 4.76 s ± 270 ms 4.82 s ± 213 ms +1.40 %
threejs_production-mode_10x_persistent-hot + exec 4.18 s ± 85 ms 4.23 s ± 189 ms +1.20 %
10000_big_production-mode_disable-minimize + rss memory 9627 MiB ± 93.1 MiB 9813 MiB ± 579 MiB +1.93 %
10000_development-mode + rss memory 689 MiB ± 20.4 MiB 741 MiB ± 61.1 MiB +7.42 %
10000_development-mode_hmr + rss memory 820 MiB ± 56.8 MiB 840 MiB ± 54.3 MiB +2.53 %
10000_production-mode + rss memory 650 MiB ± 11.7 MiB 718 MiB ± 69.6 MiB +10.57 %
10000_production-mode_persistent-cold + rss memory 776 MiB ± 29.3 MiB 802 MiB ± 42.4 MiB +3.39 %
10000_production-mode_persistent-hot + rss memory 759 MiB ± 49.4 MiB 779 MiB ± 45 MiB +2.73 %
arco-pro_development-mode + rss memory 600 MiB ± 55.2 MiB 618 MiB ± 59.5 MiB +2.98 %
arco-pro_development-mode_hmr + rss memory 507 MiB ± 19.5 MiB 529 MiB ± 27 MiB +4.28 %
arco-pro_production-mode + rss memory 692 MiB ± 52.3 MiB 741 MiB ± 53.3 MiB +7.09 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 718 MiB ± 28.9 MiB 716 MiB ± 41.9 MiB -0.23 %
arco-pro_production-mode_persistent-cold + rss memory 800 MiB ± 47.8 MiB 835 MiB ± 107 MiB +4.38 %
arco-pro_production-mode_persistent-hot + rss memory 678 MiB ± 50.8 MiB 694 MiB ± 73.4 MiB +2.43 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 707 MiB ± 47.2 MiB 729 MiB ± 21.4 MiB +3.15 %
large-dyn-imports_development-mode + rss memory 716 MiB ± 4.72 MiB 720 MiB ± 13.3 MiB +0.56 %
large-dyn-imports_production-mode + rss memory 636 MiB ± 3.66 MiB 640 MiB ± 7.19 MiB +0.65 %
threejs_development-mode_10x + rss memory 621 MiB ± 18.5 MiB 698 MiB ± 10.3 MiB +12.36 %
threejs_development-mode_10x_hmr + rss memory 782 MiB ± 23.8 MiB 856 MiB ± 37.2 MiB +9.46 %
threejs_production-mode_10x + rss memory 849 MiB ± 171 MiB 979 MiB ± 35.6 MiB +15.28 %
threejs_production-mode_10x_persistent-cold + rss memory 835 MiB ± 16.1 MiB 903 MiB ± 57.5 MiB +8.14 %
threejs_production-mode_10x_persistent-hot + rss memory 707 MiB ± 48.2 MiB 740 MiB ± 41.5 MiB +4.74 %

@web-infra-dev web-infra-dev deleted a comment from github-actions bot Jul 3, 2025
@web-infra-dev web-infra-dev deleted a comment from github-actions bot Jul 3, 2025
@web-infra-dev web-infra-dev deleted a comment from github-actions bot Jul 3, 2025
@LingyuCoder LingyuCoder marked this pull request as ready for review July 3, 2025 12:25
@LingyuCoder LingyuCoder force-pushed the perf/parallel-real-content-hash branch from 798b298 to 0596d4b Compare July 3, 2025 14:57
@LingyuCoder LingyuCoder requested a review from ahabhgk July 4, 2025 03:38
@LingyuCoder LingyuCoder merged commit 123e4f5 into main Jul 4, 2025
43 checks passed
@LingyuCoder LingyuCoder deleted the perf/parallel-real-content-hash branch July 4, 2025 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release: performance release: performance related release(mr only) team The issue/pr is created by the member of Rspack.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants