Skip to content

Convert debug namespace into a module, direct import #51455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 3 commits into from
Closed

Convert debug namespace into a module, direct import #51455

wants to merge 3 commits into from

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Nov 8, 2022 •
edited
Loading

For #51441.

This is best viewed without whitespace: https://github.com/microsoft/TypeScript/pull/51455/files?w=1

All of the real changes are in src/compiler/debug.ts.

Overall package size

Before After Diff Diff (percent)
Packed 4.00 MiB 4.00 MiB -3.92 KiB -0.10%
Unpacked 21.47 MiB 21.44 MiB -32.89 KiB -0.15%

Files

Before After Diff Diff (percent)
lib/tsc.js 5.82 MiB 5.80 MiB -21.63 KiB -0.36%
lib/tsserver.js 25.94 KiB 25.96 KiB +26.00 B +0.10%
lib/typescript.js 8.55 MiB 8.54 MiB -11.29 KiB -0.13%

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Nov 8, 2022
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 8, 2022 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at efa0cae. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 341,173k (± 0.02%) 341,127k (± 0.03%) -46k (- 0.01%) 340,894k 341,243k
Parse Time 1.56s (± 0.65%) 1.55s (± 0.54%) -0.01s (- 0.45%) 1.53s 1.57s
Bind Time 0.54s (± 0.92%) 0.53s (± 1.10%) -0.01s (- 1.68%) 0.52s 0.54s
Check Time 3.99s (± 0.59%) 4.00s (± 0.40%) +0.01s (+ 0.18%) 3.96s 4.03s
Emit Time 4.26s (± 1.20%) 4.27s (± 0.88%) +0.01s (+ 0.26%) 4.20s 4.34s
Total Time 10.34s (± 0.47%) 10.35s (± 0.44%) +0.00s (+ 0.04%) 10.25s 10.45s
Compiler-Unions - node (v18.10.0, x64)
Memory used 190,491k (± 0.03%) 188,792k (± 1.01%) -1,699k (- 0.89%) 184,707k 190,570k
Parse Time 0.61s (± 0.54%) 0.61s (± 0.97%) +0.00s (+ 0.66%) 0.60s 0.63s
Bind Time 0.33s (± 1.04%) 0.33s (± 1.14%) -0.00s (- 0.61%) 0.32s 0.33s
Check Time 4.95s (± 0.81%) 4.96s (± 0.68%) +0.00s (+ 0.04%) 4.88s 5.04s
Emit Time 1.53s (± 0.87%) 1.51s (± 0.98%) -0.02s (- 1.18%) 1.47s 1.54s
Total Time 7.42s (± 0.59%) 7.41s (± 0.33%) -0.02s (- 0.22%) 7.35s 7.46s
Monaco - node (v18.10.0, x64)
Memory used 320,548k (± 0.01%) 320,467k (± 0.02%) -81k (- 0.03%) 320,338k 320,589k
Parse Time 1.15s (± 0.87%) 1.15s (± 0.99%) +0.00s (+ 0.17%) 1.13s 1.18s
Bind Time 0.49s (± 1.63%) 0.48s (± 0.93%) -0.01s (- 2.85%) 0.47s 0.49s
Check Time 3.83s (± 0.85%) 3.83s (± 0.82%) -0.00s (- 0.05%) 3.76s 3.92s
Emit Time 2.24s (± 0.93%) 2.26s (± 0.91%) +0.02s (+ 0.76%) 2.23s 2.32s
Total Time 7.72s (± 0.67%) 7.72s (± 0.68%) +0.00s (+ 0.05%) 7.64s 7.90s
TFS - node (v18.10.0, x64)
Memory used 284,078k (± 0.24%) 283,417k (± 0.22%) -661k (- 0.23%) 282,823k 284,762k
Parse Time 0.96s (± 1.22%) 0.94s (± 1.23%) -0.02s (- 1.66%) 0.92s 0.97s
Bind Time 0.45s (± 6.05%) 0.43s (± 3.52%) 🟩-0.02s (- 4.42%) 0.42s 0.49s
Check Time 3.78s (± 0.81%) 3.76s (± 0.43%) -0.02s (- 0.50%) 3.73s 3.80s
Emit Time 2.18s (± 0.91%) 2.19s (± 0.79%) +0.01s (+ 0.51%) 2.14s 2.22s
Total Time 7.37s (± 0.41%) 7.32s (± 0.38%) -0.04s (- 0.60%) 7.26s 7.39s
material-ui - node (v18.10.0, x64)
Memory used 435,970k (± 0.01%) 435,912k (± 0.01%) -58k (- 0.01%) 435,865k 436,005k
Parse Time 1.34s (± 0.80%) 1.32s (± 0.47%) -0.02s (- 1.49%) 1.30s 1.33s
Bind Time 0.49s (± 1.13%) 0.48s (± 2.40%) -0.01s (- 1.63%) 0.44s 0.50s
Check Time 10.34s (± 0.86%) 10.22s (± 0.78%) -0.12s (- 1.18%) 10.09s 10.45s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.18s (± 0.76%) 12.03s (± 0.66%) -0.15s (- 1.23%) 11.91s 12.26s
xstate - node (v18.10.0, x64)
Memory used 518,691k (± 0.01%) 518,600k (± 0.01%) -91k (- 0.02%) 518,493k 518,790k
Parse Time 1.93s (± 0.58%) 1.91s (± 0.64%) -0.02s (- 0.83%) 1.89s 1.93s
Bind Time 0.77s (± 3.49%) 0.76s (± 3.24%) -0.01s (- 1.81%) 0.71s 0.81s
Check Time 1.03s (± 0.60%) 1.04s (± 0.84%) +0.01s (+ 0.87%) 1.02s 1.06s
Emit Time 0.05s (± 0.00%) 0.05s (± 0.00%) 0.00s ( 0.00%) 0.05s 0.05s
Total Time 3.78s (± 0.86%) 3.75s (± 0.54%) -0.03s (- 0.79%) 3.71s 3.80s
Angular - node (v16.17.1, x64)
Memory used 340,557k (± 0.02%) 340,466k (± 0.01%) -92k (- 0.03%) 340,293k 340,526k
Parse Time 1.90s (± 0.40%) 1.87s (± 0.39%) -0.03s (- 1.48%) 1.85s 1.88s
Bind Time 0.65s (± 0.68%) 0.65s (± 0.76%) -0.01s (- 0.92%) 0.64s 0.66s
Check Time 5.15s (± 0.47%) 5.15s (± 0.49%) -0.00s (- 0.10%) 5.10s 5.21s
Emit Time 5.15s (± 0.83%) 5.09s (± 0.60%) -0.05s (- 1.05%) 5.05s 5.17s
Total Time 12.85s (± 0.51%) 12.76s (± 0.37%) -0.09s (- 0.72%) 12.68s 12.88s
Compiler-Unions - node (v16.17.1, x64)
Memory used 187,281k (± 0.53%) 188,229k (± 0.66%) +949k (+ 0.51%) 186,523k 190,076k
Parse Time 0.79s (± 0.86%) 0.78s (± 0.95%) -0.01s (- 1.39%) 0.77s 0.80s
Bind Time 0.42s (± 0.00%) 0.42s (± 0.53%) +0.00s (+ 0.24%) 0.42s 0.43s
Check Time 6.03s (± 0.51%) 5.99s (± 0.92%) -0.04s (- 0.66%) 5.92s 6.13s
Emit Time 1.90s (± 0.59%) 1.88s (± 1.00%) -0.02s (- 0.95%) 1.83s 1.92s
Total Time 9.14s (± 0.37%) 9.07s (± 0.76%) -0.07s (- 0.74%) 8.95s 9.25s
Monaco - node (v16.17.1, x64)
Memory used 319,834k (± 0.01%) 319,749k (± 0.01%) -85k (- 0.03%) 319,663k 319,844k
Parse Time 1.43s (± 0.54%) 1.41s (± 0.76%) -0.02s (- 1.54%) 1.39s 1.44s
Bind Time 0.59s (± 0.57%) 0.59s (± 0.88%) -0.00s (- 0.67%) 0.58s 0.60s
Check Time 4.89s (± 0.28%) 4.86s (± 0.52%) -0.03s (- 0.59%) 4.77s 4.89s
Emit Time 2.74s (± 0.95%) 2.74s (± 0.73%) -0.00s (- 0.07%) 2.69s 2.78s
Total Time 9.65s (± 0.32%) 9.59s (± 0.51%) -0.06s (- 0.60%) 9.44s 9.69s
TFS - node (v16.17.1, x64)
Memory used 282,289k (± 0.01%) 282,208k (± 0.01%) -81k (- 0.03%) 282,176k 282,266k
Parse Time 1.18s (± 0.75%) 1.15s (± 0.58%) -0.02s (- 2.04%) 1.14s 1.17s
Bind Time 0.65s (± 3.64%) 0.64s (± 4.33%) -0.01s (- 1.08%) 0.59s 0.68s
Check Time 4.75s (± 0.46%) 4.75s (± 0.45%) -0.01s (- 0.15%) 4.72s 4.81s
Emit Time 2.75s (± 1.84%) 2.74s (± 1.66%) -0.01s (- 0.29%) 2.67s 2.83s
Total Time 9.32s (± 0.52%) 9.28s (± 0.58%) -0.04s (- 0.48%) 9.17s 9.42s
material-ui - node (v16.17.1, x64)
Memory used 435,297k (± 0.00%) 435,253k (± 0.00%) -44k (- 0.01%) 435,223k 435,305k
Parse Time 1.65s (± 0.36%) 1.64s (± 0.54%) -0.01s (- 0.55%) 1.62s 1.66s
Bind Time 0.50s (± 1.11%) 0.49s (± 1.20%) -0.01s (- 1.59%) 0.48s 0.51s
Check Time 11.88s (± 0.76%) 11.85s (± 1.32%) -0.03s (- 0.28%) 11.55s 12.27s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.03s (± 0.64%) 13.98s (± 1.17%) -0.05s (- 0.36%) 13.67s 14.42s
xstate - node (v16.17.1, x64)
Memory used 516,277k (± 0.01%) 516,173k (± 0.01%) -103k (- 0.02%) 516,071k 516,393k
Parse Time 2.32s (± 0.43%) 2.29s (± 0.26%) -0.02s (- 0.95%) 2.28s 2.31s
Bind Time 0.84s (± 0.88%) 0.84s (± 1.42%) -0.00s (- 0.24%) 0.82s 0.87s
Check Time 1.36s (± 0.70%) 1.34s (± 1.09%) -0.01s (- 1.11%) 1.32s 1.38s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 4.58s (± 0.25%) 4.54s (± 0.34%) -0.04s (- 0.76%) 4.51s 4.58s
Angular - node (v14.15.1, x64)
Memory used 334,045k (± 0.01%) 333,975k (± 0.01%) -70k (- 0.02%) 333,902k 334,035k
Parse Time 2.05s (± 0.53%) 2.04s (± 0.47%) -0.01s (- 0.63%) 2.02s 2.06s
Bind Time 0.70s (± 0.32%) 0.69s (± 0.72%) -0.00s (- 0.57%) 0.69s 0.71s
Check Time 5.48s (± 0.36%) 5.44s (± 0.28%) -0.04s (- 0.73%) 5.42s 5.48s
Emit Time 5.19s (± 0.39%) 5.22s (± 0.57%) +0.02s (+ 0.46%) 5.13s 5.26s
Total Time 13.43s (± 0.30%) 13.40s (± 0.25%) -0.03s (- 0.23%) 13.32s 13.47s
Compiler-Unions - node (v14.15.1, x64)
Memory used 181,903k (± 0.40%) 181,507k (± 0.02%) -396k (- 0.22%) 181,388k 181,560k
Parse Time 0.89s (± 0.33%) 0.89s (± 0.65%) -0.00s (- 0.56%) 0.88s 0.90s
Bind Time 0.46s (± 0.74%) 0.45s (± 0.50%) -0.01s (- 1.75%) 0.44s 0.45s
Check Time 6.32s (± 0.64%) 6.27s (± 0.37%) -0.05s (- 0.78%) 6.22s 6.34s
Emit Time 2.03s (± 1.30%) 2.03s (± 0.75%) -0.00s (- 0.15%) 2.00s 2.06s
Total Time 9.70s (± 0.56%) 9.64s (± 0.35%) -0.06s (- 0.61%) 9.59s 9.75s
Monaco - node (v14.15.1, x64)
Memory used 314,624k (± 0.01%) 314,540k (± 0.01%) -84k (- 0.03%) 314,463k 314,599k
Parse Time 1.57s (± 0.86%) 1.56s (± 0.32%) -0.01s (- 0.83%) 1.54s 1.56s
Bind Time 0.63s (± 0.78%) 0.63s (± 0.53%) -0.00s (- 0.63%) 0.62s 0.64s
Check Time 5.20s (± 0.50%) 5.14s (± 0.51%) -0.05s (- 0.98%) 5.10s 5.21s
Emit Time 2.88s (± 0.53%) 2.89s (± 0.85%) +0.01s (+ 0.45%) 2.82s 2.94s
Total Time 10.27s (± 0.34%) 10.22s (± 0.45%) -0.05s (- 0.53%) 10.11s 10.31s
TFS - node (v14.15.1, x64)
Memory used 279,313k (± 0.01%) 279,259k (± 0.00%) -55k (- 0.02%) 279,225k 279,287k
Parse Time 1.33s (± 0.80%) 1.33s (± 0.94%) -0.01s (- 0.60%) 1.31s 1.36s
Bind Time 0.59s (± 0.98%) 0.59s (± 1.13%) -0.00s (- 0.51%) 0.58s 0.61s
Check Time 5.08s (± 0.45%) 5.05s (± 0.44%) -0.03s (- 0.65%) 5.02s 5.12s
Emit Time 3.05s (± 0.79%) 3.04s (± 0.73%) -0.01s (- 0.39%) 3.00s 3.11s
Total Time 10.06s (± 0.21%) 10.01s (± 0.37%) -0.06s (- 0.57%) 9.93s 10.10s
material-ui - node (v14.15.1, x64)
Memory used 430,731k (± 0.01%) 430,659k (± 0.00%) -72k (- 0.02%) 430,603k 430,711k
Parse Time 1.88s (± 0.67%) 1.86s (± 0.39%) -0.02s (- 0.90%) 1.85s 1.88s
Bind Time 0.53s (± 0.56%) 0.53s (± 0.42%) -0.00s (- 0.19%) 0.53s 0.54s
Check Time 12.27s (± 0.73%) 12.18s (± 0.61%) -0.10s (- 0.77%) 12.03s 12.35s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.68s (± 0.66%) 14.57s (± 0.52%) -0.11s (- 0.77%) 14.41s 14.74s
xstate - node (v14.15.1, x64)
Memory used 504,416k (± 0.00%) 504,407k (± 0.01%) -9k (- 0.00%) 504,350k 504,453k
Parse Time 2.63s (± 0.50%) 2.59s (± 0.67%) -0.04s (- 1.37%) 2.57s 2.64s
Bind Time 0.84s (± 0.88%) 0.84s (± 0.40%) +0.00s (+ 0.36%) 0.84s 0.85s
Check Time 1.47s (± 0.56%) 1.46s (± 0.55%) -0.02s (- 1.09%) 1.44s 1.47s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.02s (± 0.22%) 4.96s (± 0.38%) -0.05s (- 1.06%) 4.94s 5.02s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

TSServer

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,050ms (± 1.11%) 1,035ms (± 0.78%) -15ms (- 1.45%) 1,016ms 1,051ms
Req 2 - geterr 2,534ms (± 0.67%) 2,531ms (± 0.85%) -3ms (- 0.13%) 2,499ms 2,597ms
Req 3 - references 166ms (± 0.35%) 165ms (± 0.72%) -1ms (- 0.54%) 162ms 168ms
Req 4 - navto 138ms (± 1.03%) 137ms (± 0.95%) -1ms (- 0.36%) 134ms 140ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 61ms (± 1.92%) 58ms (± 2.83%) 🟩-3ms (- 4.60%) 55ms 61ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,097ms (± 0.53%) 1,091ms (± 0.49%) -6ms (- 0.51%) 1,078ms 1,100ms
Req 2 - geterr 1,559ms (± 1.11%) 1,559ms (± 0.55%) +1ms (+ 0.04%) 1,542ms 1,578ms
Req 3 - references 173ms (± 4.12%) 171ms (± 3.65%) -2ms (- 1.10%) 167ms 196ms
Req 4 - navto 151ms (± 0.70%) 150ms (± 0.64%) -1ms (- 0.73%) 148ms 152ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 51ms (± 2.96%) 53ms (± 2.69%) +1ms (+ 2.73%) 48ms 55ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,523ms (± 0.50%) 1,498ms (± 0.45%) -25ms (- 1.65%) 1,488ms 1,520ms
Req 2 - geterr 557ms (± 0.79%) 553ms (± 0.94%) -4ms (- 0.63%) 542ms 565ms
Req 3 - references 59ms (± 2.14%) 58ms (± 2.59%) -1ms (- 1.36%) 57ms 64ms
Req 4 - navto 197ms (± 0.62%) 195ms (± 0.62%) -3ms (- 1.27%) 192ms 197ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 211ms (± 0.83%) 209ms (± 1.36%) -2ms (- 0.81%) 204ms 217ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,301ms (± 0.78%) 1,286ms (± 0.38%) -15ms (- 1.15%) 1,277ms 1,299ms
Req 2 - geterr 3,161ms (± 0.62%) 3,127ms (± 1.19%) -34ms (- 1.07%) 3,061ms 3,212ms
Req 3 - references 192ms (± 1.11%) 191ms (± 0.90%) -1ms (- 0.26%) 189ms 195ms
Req 4 - navto 151ms (± 0.72%) 151ms (± 0.72%) 0ms ( 0.00%) 149ms 154ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 59ms (± 2.74%) 61ms (±16.72%) +1ms (+ 2.36%) 54ms 101ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,392ms (± 1.10%) 1,376ms (± 0.56%) -16ms (- 1.18%) 1,354ms 1,391ms
Req 2 - geterr 2,073ms (± 0.58%) 2,041ms (± 0.65%) -31ms (- 1.51%) 2,017ms 2,071ms
Req 3 - references 200ms (± 0.50%) 198ms (± 0.99%) -2ms (- 1.15%) 196ms 205ms
Req 4 - navto 166ms (± 0.92%) 163ms (± 0.76%) -3ms (- 1.63%) 161ms 166ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 57ms (± 2.65%) 58ms (± 2.68%) +2ms (+ 3.19%) 55ms 61ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,831ms (± 0.48%) 1,804ms (± 0.57%) -27ms (- 1.47%) 1,787ms 1,833ms
Req 2 - geterr 713ms (± 0.70%) 708ms (± 0.73%) -5ms (- 0.71%) 699ms 721ms
Req 3 - references 67ms (± 0.50%) 68ms (± 1.13%) +1ms (+ 0.75%) 66ms 69ms
Req 4 - navto 198ms (± 0.85%) 197ms (± 1.06%) -1ms (- 0.61%) 193ms 201ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 255ms (± 1.13%) 249ms (± 0.78%) -5ms (- 2.04%) 246ms 253ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,460ms (± 0.60%) 1,440ms (± 0.44%) -20ms (- 1.34%) 1,429ms 1,457ms
Req 2 - geterr 3,401ms (± 0.41%) 3,381ms (± 0.58%) -20ms (- 0.58%) 3,359ms 3,449ms
Req 3 - references 206ms (± 1.06%) 203ms (± 0.40%) -3ms (- 1.50%) 202ms 206ms
Req 4 - navto 161ms (± 0.87%) 161ms (± 0.80%) -0ms (- 0.19%) 158ms 165ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 65ms (± 7.18%) 66ms (± 6.62%) +1ms (+ 2.17%) 57ms 72ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,530ms (± 0.63%) 1,524ms (± 0.61%) -6ms (- 0.39%) 1,504ms 1,543ms
Req 2 - geterr 2,239ms (± 0.42%) 2,239ms (± 0.59%) +0ms (+ 0.01%) 2,206ms 2,261ms
Req 3 - references 215ms (± 1.66%) 210ms (± 1.16%) -5ms (- 2.14%) 205ms 217ms
Req 4 - navto 174ms (± 0.81%) 172ms (± 1.10%) -2ms (- 1.09%) 169ms 176ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 64ms (± 7.93%) 62ms (± 8.69%) 🟩-2ms (- 3.43%) 55ms 73ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,019ms (± 0.30%) 1,999ms (± 0.56%) -20ms (- 1.01%) 1,978ms 2,024ms
Req 2 - geterr 744ms (± 0.59%) 737ms (± 0.42%) -7ms (- 0.87%) 732ms 745ms
Req 3 - references 71ms (± 1.89%) 70ms (± 2.35%) -2ms (- 2.11%) 67ms 75ms
Req 4 - navto 220ms (± 0.70%) 217ms (± 0.52%) -2ms (- 1.09%) 215ms 220ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 274ms (± 1.93%) 269ms (± 1.57%) -6ms (- 2.08%) 263ms 280ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

Startup

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 119.24ms (± 0.54%) 117.66ms (± 0.36%) -1.57ms (- 1.32%) 115.75ms 123.67ms
tsserver-startup - node (v16.17.1, x64)
Execution time 199.17ms (± 0.35%) 198.19ms (± 0.31%) -0.98ms (- 0.49%) 195.18ms 205.94ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 187.76ms (± 0.30%) 188.05ms (± 0.31%) +0.29ms (+ 0.16%) 185.36ms 196.33ms
typescript-startup - node (v16.17.1, x64)
Execution time 173.77ms (± 0.32%) 173.75ms (± 0.33%) -0.02ms (- 0.01%) 170.83ms 181.67ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

Seems like a couple percent; I'll have to retry this once the benchmarker is stable.

This was referenced Nov 14, 2022
Closed
Merged
@jakebailey jakebailey mentioned this pull request Nov 21, 2022
Merged
11 tasks
@jakebailey jakebailey changed the title [wip] Convert debug namespace into a module Convert debug namespace into a module, direct import Jan 17, 2023
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 17, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at 2b510af. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 358,872k (± 0.01%) 358,780k (± 0.01%) -92k (- 0.03%) 358,750k 358,821k
Parse Time 3.92s (± 0.64%) 3.94s (± 0.87%) +0.02s (+ 0.55%) 3.90s 3.99s
Bind Time 1.19s (± 0.63%) 1.19s (± 0.59%) -0.00s (- 0.14%) 1.18s 1.20s
Check Time 8.82s (± 0.68%) 8.77s (± 0.50%) -0.05s (- 0.60%) 8.71s 8.83s
Emit Time 7.48s (± 0.34%) 7.57s (± 1.08%) +0.09s (+ 1.18%) 7.44s 7.65s
Total Time 21.41s (± 0.37%) 21.47s (± 0.61%) +0.06s (+ 0.28%) 21.30s 21.65s
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,851k (± 1.56%) 194,907k (± 1.62%) +1,055k (+ 0.54%) 191,994k 197,913k
Parse Time 1.66s (± 0.71%) 1.64s (± 1.45%) -0.02s (- 1.30%) 1.59s 1.66s
Bind Time 0.80s (± 0.82%) 0.79s (± 0.64%) -0.01s (- 1.14%) 0.79s 0.80s
Check Time 9.59s (± 0.93%) 9.72s (± 0.74%) +0.13s (+ 1.37%) 9.62s 9.81s
Emit Time 2.94s (± 5.82%) 2.80s (± 0.78%) 🟩-0.14s (- 4.79%) 2.77s 2.83s
Total Time 14.99s (± 1.45%) 14.95s (± 0.52%) -0.04s (- 0.29%) 14.84s 15.07s
Monaco - node (v18.10.0, x64)
Memory used 345,746k (± 0.01%) 345,613k (± 0.01%) -133k (- 0.04%) 345,593k 345,646k
Parse Time 2.93s (± 0.48%) 2.96s (± 1.07%) +0.02s (+ 0.80%) 2.92s 3.00s
Bind Time 1.06s (± 0.92%) 1.06s (± 0.40%) +0.00s (+ 0.30%) 1.06s 1.07s
Check Time 7.30s (± 0.40%) 7.28s (± 0.48%) -0.02s (- 0.26%) 7.25s 7.33s
Emit Time 4.35s (± 1.05%) 4.33s (± 0.67%) -0.02s (- 0.51%) 4.28s 4.36s
Total Time 15.64s (± 0.22%) 15.63s (± 0.55%) -0.01s (- 0.07%) 15.50s 15.74s
TFS - node (v18.10.0, x64)
Memory used 300,532k (± 0.01%) 300,425k (± 0.01%) -107k (- 0.04%) 300,387k 300,452k
Parse Time 2.25s (± 1.08%) 2.26s (± 1.26%) +0.01s (+ 0.53%) 2.22s 2.30s
Bind Time 1.18s (± 0.39%) 1.18s (± 1.45%) -0.00s (- 0.42%) 1.16s 1.21s
Check Time 6.85s (± 0.21%) 6.80s (± 0.36%) -0.06s (- 0.84%) 6.76s 6.82s
Emit Time 3.95s (± 1.21%) 3.92s (± 0.60%) -0.03s (- 0.87%) 3.90s 3.96s
Total Time 14.24s (± 0.27%) 14.16s (± 0.23%) -0.08s (- 0.60%) 14.11s 14.21s
material-ui - node (v18.10.0, x64)
Memory used 476,398k (± 0.01%) 476,331k (± 0.02%) -68k (- 0.01%) 476,256k 476,467k
Parse Time 3.52s (± 0.72%) 3.53s (± 0.41%) +0.00s (+ 0.07%) 3.52s 3.55s
Bind Time 0.97s (± 1.42%) 0.98s (± 0.79%) +0.01s (+ 1.14%) 0.97s 0.99s
Check Time 17.07s (± 0.70%) 17.08s (± 0.55%) +0.01s (+ 0.07%) 16.95s 17.19s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.56s (± 0.70%) 21.58s (± 0.46%) +0.02s (+ 0.10%) 21.44s 21.69s
xstate - node (v18.10.0, x64)
Memory used 546,282k (± 0.01%) 546,090k (± 0.01%) -192k (- 0.04%) 546,020k 546,142k
Parse Time 4.42s (± 0.81%) 4.40s (± 0.65%) -0.02s (- 0.39%) 4.36s 4.44s
Bind Time 1.71s (± 0.92%) 1.71s (± 1.18%) +0.00s (+ 0.09%) 1.68s 1.73s
Check Time 2.74s (± 0.85%) 2.73s (± 0.42%) -0.01s (- 0.38%) 2.71s 2.74s
Emit Time 0.08s (± 0.21%) 0.08s (± 4.96%) +0.00s (+ 1.88%) 0.08s 0.09s
Total Time 8.94s (± 0.49%) 8.92s (± 0.49%) -0.01s (- 0.16%) 8.85s 8.97s
Angular - node (v16.17.1, x64)
Memory used 358,285k (± 0.01%) 358,169k (± 0.01%) -116k (- 0.03%) 358,116k 358,218k
Parse Time 4.11s (± 0.20%) 4.08s (± 0.51%) -0.02s (- 0.59%) 4.06s 4.11s
Bind Time 1.26s (± 0.61%) 1.24s (± 0.69%) -0.01s (- 1.06%) 1.23s 1.25s
Check Time 9.48s (± 0.34%) 9.42s (± 0.44%) -0.06s (- 0.60%) 9.37s 9.48s
Emit Time 7.94s (± 0.53%) 7.93s (± 0.63%) -0.01s (- 0.14%) 7.89s 8.02s
Total Time 22.78s (± 0.28%) 22.67s (± 0.40%) -0.11s (- 0.47%) 22.61s 22.85s
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,832k (± 0.04%) 193,749k (± 0.03%) -84k (- 0.04%) 193,676k 193,845k
Parse Time 1.81s (± 1.13%) 1.80s (± 0.69%) -0.01s (- 0.53%) 1.78s 1.81s
Bind Time 0.85s (± 0.21%) 0.84s (± 0.76%) -0.01s (- 1.24%) 0.83s 0.85s
Check Time 10.29s (± 0.63%) 10.33s (± 0.87%) +0.04s (+ 0.39%) 10.20s 10.47s
Emit Time 3.09s (± 4.41%) 3.04s (± 0.73%) -0.05s (- 1.57%) 3.02s 3.07s
Total Time 16.03s (± 0.65%) 16.00s (± 0.61%) -0.03s (- 0.17%) 15.88s 16.17s
Monaco - node (v16.17.1, x64)
Memory used 345,098k (± 0.02%) 345,013k (± 0.01%) -84k (- 0.02%) 344,966k 345,036k
Parse Time 3.11s (± 1.24%) 3.14s (± 0.51%) +0.04s (+ 1.22%) 3.11s 3.16s
Bind Time 1.11s (± 1.40%) 1.11s (± 0.08%) -0.00s (- 0.07%) 1.11s 1.11s
Check Time 7.89s (± 0.81%) 7.86s (± 0.42%) -0.02s (- 0.28%) 7.81s 7.91s
Emit Time 4.52s (± 0.90%) 4.53s (± 0.71%) +0.01s (+ 0.23%) 4.48s 4.58s
Total Time 16.63s (± 0.64%) 16.65s (± 0.38%) +0.02s (+ 0.14%) 16.57s 16.73s
TFS - node (v16.17.1, x64)
Memory used 299,909k (± 0.01%) 299,773k (± 0.01%) -135k (- 0.05%) 299,739k 299,791k
Parse Time 2.44s (± 0.64%) 2.48s (± 1.20%) +0.04s (+ 1.70%) 2.44s 2.52s
Bind Time 1.26s (± 1.17%) 1.26s (± 0.64%) +0.00s (+ 0.00%) 1.24s 1.27s
Check Time 7.42s (± 0.43%) 7.39s (± 0.34%) -0.03s (- 0.47%) 7.36s 7.41s
Emit Time 4.22s (± 0.44%) 4.33s (± 1.85%) +0.11s (+ 2.56%) 4.20s 4.39s
Total Time 15.34s (± 0.35%) 15.45s (± 0.45%) +0.11s (+ 0.75%) 15.37s 15.56s
material-ui - node (v16.17.1, x64)
Memory used 475,701k (± 0.01%) 475,593k (± 0.00%) -109k (- 0.02%) 475,576k 475,605k
Parse Time 3.66s (± 0.34%) 3.63s (± 0.23%) -0.04s (- 0.96%) 3.62s 3.64s
Bind Time 1.01s (± 0.44%) 1.01s (± 0.51%) -0.00s (- 0.26%) 1.01s 1.02s
Check Time 18.03s (± 1.39%) 17.97s (± 0.44%) -0.06s (- 0.34%) 17.83s 18.04s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.70s (± 1.17%) 22.61s (± 0.36%) -0.09s (- 0.41%) 22.46s 22.68s
xstate - node (v16.17.1, x64)
Memory used 543,947k (± 0.03%) 543,736k (± 0.02%) -211k (- 0.04%) 543,625k 543,853k
Parse Time 4.57s (± 0.26%) 4.55s (± 0.40%) -0.03s (- 0.62%) 4.52s 4.57s
Bind Time 1.79s (± 0.56%) 1.79s (± 0.67%) +0.00s (+ 0.09%) 1.78s 1.81s
Check Time 2.93s (± 0.56%) 2.94s (± 0.40%) +0.01s (+ 0.23%) 2.92s 2.95s
Emit Time 0.08s (± 0.17%) 0.08s (± 4.84%) +0.00s (+ 2.08%) 0.08s 0.09s
Total Time 9.39s (± 0.29%) 9.37s (± 0.26%) -0.02s (- 0.21%) 9.33s 9.41s
Angular - node (v14.15.1, x64)
Memory used 351,891k (± 0.01%) 351,840k (± 0.00%) -52k (- 0.01%) 351,828k 351,858k
Parse Time 4.14s (± 0.40%) 4.15s (± 0.65%) +0.01s (+ 0.32%) 4.12s 4.18s
Bind Time 1.28s (± 0.82%) 1.28s (± 0.69%) -0.00s (- 0.04%) 1.27s 1.30s
Check Time 9.78s (± 0.66%) 9.77s (± 0.49%) -0.01s (- 0.06%) 9.72s 9.86s
Emit Time 8.27s (± 0.55%) 8.35s (± 0.76%) +0.08s (+ 0.96%) 8.26s 8.42s
Total Time 23.48s (± 0.41%) 23.56s (± 0.53%) +0.08s (+ 0.35%) 23.38s 23.72s
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,989k (± 0.01%) 189,415k (± 0.73%) +426k (+ 0.23%) 188,839k 192,241k
Parse Time 1.83s (± 0.27%) 1.83s (± 0.84%) +0.00s (+ 0.27%) 1.81s 1.86s
Bind Time 0.88s (± 0.47%) 0.86s (± 0.51%) -0.02s (- 1.90%) 0.86s 0.87s
Check Time 10.32s (± 0.27%) 10.31s (± 0.49%) -0.01s (- 0.13%) 10.22s 10.36s
Emit Time 3.14s (± 1.18%) 3.26s (± 0.84%) +0.12s (+ 3.97%) 3.24s 3.31s
Total Time 16.16s (± 0.30%) 16.26s (± 0.29%) +0.10s (+ 0.62%) 16.19s 16.33s
Monaco - node (v14.15.1, x64)
Memory used 339,910k (± 0.01%) 339,857k (± 0.01%) -53k (- 0.02%) 339,834k 339,890k
Parse Time 3.25s (± 0.93%) 3.23s (± 0.87%) -0.02s (- 0.53%) 3.19s 3.26s
Bind Time 1.14s (± 1.08%) 1.14s (± 0.30%) -0.01s (- 0.55%) 1.13s 1.14s
Check Time 8.20s (± 0.48%) 8.15s (± 0.44%) -0.04s (- 0.54%) 8.13s 8.22s
Emit Time 4.75s (± 0.80%) 4.79s (± 0.54%) +0.03s (+ 0.73%) 4.75s 4.82s
Total Time 17.34s (± 0.35%) 17.31s (± 0.22%) -0.03s (- 0.17%) 17.25s 17.36s
TFS - node (v14.15.1, x64)
Memory used 294,805k (± 0.00%) 294,826k (± 0.00%) +21k (+ 0.01%) 294,814k 294,837k
Parse Time 2.72s (± 0.39%) 2.76s (± 0.86%) +0.04s (+ 1.42%) 2.72s 2.78s
Bind Time 1.11s (± 0.38%) 1.11s (± 1.94%) +0.00s (+ 0.44%) 1.08s 1.15s
Check Time 7.74s (± 0.51%) 7.73s (± 0.56%) -0.01s (- 0.19%) 7.68s 7.79s
Emit Time 4.59s (± 0.75%) 4.65s (± 0.77%) +0.06s (+ 1.32%) 4.61s 4.70s
Total Time 16.15s (± 0.22%) 16.24s (± 0.60%) +0.09s (+ 0.54%) 16.13s 16.38s
material-ui - node (v14.15.1, x64)
Memory used 471,164k (± 0.00%) 471,073k (± 0.00%) -91k (- 0.02%) 471,052k 471,086k
Parse Time 3.86s (± 0.72%) 3.86s (± 0.42%) +0.00s (+ 0.07%) 3.84s 3.88s
Bind Time 1.03s (± 0.88%) 1.04s (± 0.71%) +0.00s (+ 0.39%) 1.03s 1.05s
Check Time 18.90s (± 0.49%) 18.87s (± 0.34%) -0.02s (- 0.11%) 18.76s 18.93s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 23.79s (± 0.44%) 23.77s (± 0.27%) -0.01s (- 0.06%) 23.65s 23.82s
xstate - node (v14.15.1, x64)
Memory used 532,149k (± 0.01%) 531,920k (± 0.00%) -229k (- 0.04%) 531,892k 531,961k
Parse Time 5.00s (± 0.76%) 4.97s (± 0.96%) -0.03s (- 0.60%) 4.93s 5.03s
Bind Time 1.66s (± 1.16%) 1.67s (± 0.70%) +0.01s (+ 0.60%) 1.66s 1.69s
Check Time 3.07s (± 0.33%) 3.08s (± 0.49%) +0.01s (+ 0.30%) 3.06s 3.09s
Emit Time 0.10s (± 0.27%) 0.09s (± 0.13%) 🟩-0.01s (- 9.73%) 0.09s 0.09s
Total Time 9.84s (± 0.52%) 9.81s (± 0.47%) -0.03s (- 0.31%) 9.76s 9.89s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

TSServer

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,458ms (± 6.25%) 2,522ms (± 1.12%) +64ms (+ 2.60%) 2,479ms 2,562ms
Req 2 - geterr 5,176ms (± 6.65%) 5,418ms (± 1.08%) +242ms (+ 4.67%) 5,341ms 5,489ms
Req 3 - references 359ms (±10.35%) 377ms (± 2.65%) +18ms (+ 4.96%) 365ms 394ms
Req 4 - navto 275ms (± 6.42%) 288ms (± 1.50%) +13ms (+ 4.71%) 281ms 292ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 67ms (±10.52%) 102ms (±87.84%) 🔻+35ms (+51.83%) 62ms 283ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,635ms (± 0.90%) 2,662ms (± 0.59%) +27ms (+ 1.02%) 2,645ms 2,682ms
Req 2 - geterr 3,965ms (± 0.45%) 3,992ms (± 0.54%) +27ms (+ 0.69%) 3,959ms 4,027ms
Req 3 - references 375ms (± 1.05%) 376ms (± 1.58%) +0ms (+ 0.02%) 367ms 383ms
Req 4 - navto 289ms (± 0.61%) 291ms (± 0.65%) +2ms (+ 0.69%) 288ms 293ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 82ms (± 2.48%) 82ms (± 1.18%) +0ms (+ 0.23%) 80ms 83ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,347ms (± 1.03%) 3,328ms (± 0.36%) -19ms (- 0.57%) 3,316ms 3,350ms
Req 2 - geterr 1,408ms (± 1.38%) 1,394ms (± 0.88%) -13ms (- 0.96%) 1,381ms 1,416ms
Req 3 - references 245ms (± 0.47%) 245ms (± 0.76%) +0ms (+ 0.12%) 242ms 247ms
Req 4 - navto 360ms (± 0.90%) 360ms (± 1.19%) +0ms (+ 0.10%) 356ms 366ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 445ms (± 0.69%) 429ms (± 1.55%) 🟩-16ms (- 3.55%) 417ms 437ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,684ms (± 5.38%) 2,733ms (± 0.66%) +49ms (+ 1.83%) 2,715ms 2,758ms
Req 2 - geterr 5,716ms (± 5.98%) 5,779ms (± 0.55%) +63ms (+ 1.10%) 5,737ms 5,829ms
Req 3 - references 400ms (±20.75%) 376ms (± 0.26%) 🟩-24ms (- 5.90%) 375ms 377ms
Req 4 - navto 269ms (± 5.61%) 271ms (± 0.50%) +2ms (+ 0.82%) 269ms 273ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 221ms (±49.68%) 295ms (± 2.17%) 🔻+74ms (+33.26%) 287ms 302ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,874ms (± 0.75%) 2,661ms (±22.01%) 🟩-213ms (- 7.41%) 1,465ms 2,942ms
Req 2 - geterr 4,319ms (± 0.54%) 3,948ms (±21.98%) 🟩-372ms (- 8.61%) 2,175ms 4,341ms
Req 3 - references 386ms (± 1.49%) 356ms (±21.76%) 🟩-30ms (- 7.77%) 198ms 392ms
Req 4 - navto 299ms (± 0.32%) 274ms (±21.79%) 🟩-25ms (- 8.42%) 152ms 301ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 79ms (± 0.65%) 73ms (±21.79%) 🟩-7ms (- 8.27%) 40ms 81ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,461ms (± 0.30%) 3,449ms (± 0.60%) -12ms (- 0.36%) 3,417ms 3,472ms
Req 2 - geterr 1,656ms (± 5.00%) 1,648ms (± 3.90%) -7ms (- 0.45%) 1,541ms 1,709ms
Req 3 - references 167ms (±51.40%) 170ms (±43.93%) +3ms (+ 1.59%) 110ms 282ms
Req 4 - navto 344ms (± 1.85%) 340ms (± 1.74%) -4ms (- 1.11%) 334ms 347ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 481ms (± 1.26%) 467ms (± 1.52%) -14ms (- 2.99%) 456ms 475ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,863ms (± 0.19%) 2,858ms (± 0.55%) -5ms (- 0.19%) 2,840ms 2,878ms
Req 2 - geterr 6,172ms (± 0.41%) 6,227ms (± 0.70%) +55ms (+ 0.89%) 6,164ms 6,282ms
Req 3 - references 389ms (± 1.05%) 387ms (± 0.97%) -2ms (- 0.42%) 384ms 393ms
Req 4 - navto 285ms (± 0.99%) 289ms (± 0.94%) +3ms (+ 1.14%) 285ms 291ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 82ms (± 9.27%) 87ms (± 6.86%) +4ms (+ 5.43%) 76ms 91ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,005ms (± 0.64%) 3,006ms (± 0.53%) +2ms (+ 0.06%) 2,980ms 3,025ms
Req 2 - geterr 4,618ms (± 0.81%) 4,581ms (± 0.87%) -37ms (- 0.79%) 4,523ms 4,620ms
Req 3 - references 412ms (± 0.67%) 411ms (± 1.13%) -1ms (- 0.19%) 407ms 419ms
Req 4 - navto 309ms (± 0.79%) 304ms (± 1.68%) -4ms (- 1.42%) 295ms 310ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 79ms (± 1.72%) 81ms (± 5.62%) +2ms (+ 2.13%) 77ms 89ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,797ms (± 0.68%) 3,797ms (± 0.33%) +0ms (+ 0.01%) 3,775ms 3,806ms
Req 2 - geterr 1,471ms (± 1.32%) 1,473ms (± 1.34%) +1ms (+ 0.09%) 1,453ms 1,501ms
Req 3 - references 130ms (± 1.82%) 132ms (± 2.60%) +2ms (+ 1.56%) 126ms 136ms
Req 4 - navto 401ms (± 0.81%) 399ms (± 1.35%) -2ms (- 0.39%) 389ms 404ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 506ms (± 1.18%) 498ms (± 1.51%) -8ms (- 1.51%) 487ms 508ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Startup

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 139.33ms (± 0.24%) 139.02ms (± 0.45%) -0.30ms (- 0.22%) 137.67ms 147.58ms
tsserver-startup - node (v16.17.1, x64)
Execution time 231.25ms (± 0.22%) 233.15ms (± 0.39%) +1.90ms (+ 0.82%) 230.01ms 238.38ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 233.62ms (± 0.19%) 237.38ms (± 0.32%) +3.76ms (+ 1.61%) 233.64ms 243.42ms
typescript-startup - node (v16.17.1, x64)
Execution time 215.22ms (± 0.25%) 219.12ms (± 0.27%) +3.90ms (+ 1.81%) 215.46ms 224.78ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey jakebailey marked this pull request as ready for review January 17, 2023 22:46
@jakebailey
Copy link
Member Author

jakebailey commented Jan 17, 2023 •
edited
Loading

@a-tarasyuk If you're bored, you may want to try rebasing your direct import PR on top of this and see what happens; I think this should fix the runtime errors due to cycles, but, I can't be totally sure.

(I'm not planning on getting this one looked at until after the beta cutoff.)

@a-tarasyuk
Copy link
Contributor

a-tarasyuk commented Jan 18, 2023 •
edited
Loading

@jakebailey I've added changes to #51590 , however, it doesn't solve the issue of running tests with the --bundle=false flag.

@jakebailey
Copy link
Member Author

Hm, seems to be just performanceCore; likely some of that can be made lazy as I don't see a major reason why we need to have it available until one of the functions is called.

@a-tarasyuk
Copy link
Contributor

I’ve added a couple of changes and the build seems to have passed. 😱

@DanielRosenwasser DanielRosenwasser mentioned this pull request Jan 27, 2023
Closed
@jakebailey

This comment was marked as outdated.

Sign in to view
@typescript-bot

This comment was marked as outdated.

Sign in to view
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Feb 6, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at 3a67373. You can monitor the build here.

Update: The results are in!

jakebailey
jakebailey commented Feb 6, 2023
View reviewed changes
src/compiler/_namespaces/ts.ts
export * from "../core";
export * from "../debug";
import * as Debug from "../debug";
export { Debug };
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out that I may actually be able to write:

export { /** @internal */ Debug }

And the comment is preserved and applies, which generally may be a better way to do this in the future.

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51455
Metric main 51455 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 359,815k (± 0.01%) 359,737k (± 0.02%) ~ 359,690k 359,843k p=0.128 n=6
Parse Time 3.97s (± 0.99%) 3.97s (± 0.59%) ~ 3.94s 4.00s p=0.686 n=6
Bind Time 1.17s (± 0.47%) 1.18s (± 1.49%) ~ 1.16s 1.21s p=0.663 n=6
Check Time 8.67s (± 0.42%) 8.68s (± 0.50%) ~ 8.63s 8.75s p=0.871 n=6
Emit Time 7.61s (± 0.50%) 7.63s (± 0.55%) ~ 7.58s 7.68s p=0.368 n=6
Total Time 21.43s (± 0.16%) 21.46s (± 0.26%) ~ 21.37s 21.53s p=0.335 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,107k (± 1.19%) 192,061k (± 0.02%) -1,046k (- 0.54%) 192,024k 192,125k p=0.008 n=6
Parse Time 1.64s (± 2.04%) 1.65s (± 1.27%) ~ 1.61s 1.67s p=0.625 n=6
Bind Time 0.80s (± 0.79%) 0.78s (± 0.97%) -0.02s (- 2.71%) 0.77s 0.79s p=0.005 n=6
Check Time 9.74s (± 0.63%) 9.95s (± 0.50%) +0.21s (+ 2.16%) 9.90s 10.03s p=0.005 n=6
Emit Time 2.79s (± 1.27%) 2.81s (± 1.20%) ~ 2.77s 2.86s p=0.375 n=6
Total Time 14.96s (± 0.37%) 15.19s (± 0.40%) +0.22s (+ 1.49%) 15.11s 15.26s p=0.005 n=6
Monaco - node (v18.10.0, x64)
Memory used 343,999k (± 0.01%) 343,845k (± 0.01%) -154k (- 0.04%) 343,828k 343,902k p=0.005 n=6
Parse Time 2.96s (± 0.39%) 2.99s (± 0.68%) +0.03s (+ 1.13%) 2.96s 3.02s p=0.019 n=6
Bind Time 1.04s (± 0.61%) 1.05s (± 0.49%) ~ 1.04s 1.05s p=0.091 n=6
Check Time 7.07s (± 0.41%) 7.05s (± 0.64%) ~ 7.00s 7.11s p=0.332 n=6
Emit Time 4.34s (± 0.69%) 4.34s (± 1.26%) ~ 4.28s 4.43s p=0.936 n=6
Total Time 15.41s (± 0.27%) 15.43s (± 0.44%) ~ 15.34s 15.53s p=0.336 n=6
TFS - node (v18.10.0, x64)
Memory used 300,388k (± 0.01%) 300,291k (± 0.00%) -97k (- 0.03%) 300,279k 300,303k p=0.005 n=6
Parse Time 2.25s (± 0.99%) 2.30s (± 1.39%) +0.05s (+ 2.45%) 2.25s 2.34s p=0.015 n=6
Bind Time 1.17s (± 0.94%) 1.16s (± 0.85%) ~ 1.14s 1.17s p=0.067 n=6
Check Time 6.53s (± 0.73%) 6.55s (± 0.49%) ~ 6.50s 6.59s p=0.422 n=6
Emit Time 3.92s (± 0.44%) 3.90s (± 0.66%) ~ 3.87s 3.92s p=0.346 n=6
Total Time 13.87s (± 0.32%) 13.92s (± 0.37%) ~ 13.87s 14.01s p=0.123 n=6
material-ui - node (v18.10.0, x64)
Memory used 476,877k (± 0.01%) 476,795k (± 0.00%) -82k (- 0.02%) 476,773k 476,820k p=0.006 n=6
Parse Time 3.57s (± 1.09%) 3.54s (± 0.86%) ~ 3.49s 3.57s p=0.295 n=6
Bind Time 0.97s (± 0.92%) 0.97s (± 0.84%) ~ 0.96s 0.98s p=0.550 n=6
Check Time 17.21s (± 0.62%) 17.28s (± 0.44%) ~ 17.23s 17.43s p=0.198 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.75s (± 0.65%) 21.78s (± 0.35%) ~ 21.69s 21.92s p=0.629 n=6
xstate - node (v18.10.0, x64)
Memory used 549,034k (± 0.02%) 548,858k (± 0.01%) -176k (- 0.03%) 548,790k 548,984k p=0.020 n=6
Parse Time 4.58s (± 0.36%) 4.61s (± 0.65%) ~ 4.57s 4.65s p=0.164 n=6
Bind Time 1.72s (± 0.68%) 1.74s (± 0.86%) +0.02s (+ 1.26%) 1.73s 1.77s p=0.016 n=6
Check Time 2.83s (± 0.73%) 2.82s (± 0.59%) ~ 2.79s 2.83s p=1.000 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.174 n=6
Total Time 9.22s (± 0.43%) 9.26s (± 0.44%) ~ 9.22s 9.32s p=0.078 n=6
Angular - node (v16.17.1, x64)
Memory used 359,228k (± 0.00%) 359,127k (± 0.01%) -102k (- 0.03%) 359,095k 359,166k p=0.005 n=6
Parse Time 4.19s (± 0.49%) 4.17s (± 0.15%) -0.02s (- 0.56%) 4.16s 4.18s p=0.048 n=6
Bind Time 1.23s (± 0.42%) 1.22s (± 0.33%) ~ 1.22s 1.23s p=0.112 n=6
Check Time 9.46s (± 0.42%) 9.46s (± 0.55%) ~ 9.41s 9.55s p=1.000 n=6
Emit Time 8.09s (± 0.28%) 8.07s (± 0.93%) ~ 7.93s 8.13s p=0.872 n=6
Total Time 22.97s (± 0.14%) 22.92s (± 0.38%) ~ 22.80s 23.05s p=0.228 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,973k (± 0.03%) 194,270k (± 0.70%) ~ 193,656k 197,061k p=0.066 n=6
Parse Time 1.81s (± 0.23%) 1.80s (± 1.52%) ~ 1.76s 1.84s p=0.199 n=6
Bind Time 0.85s (± 0.48%) 0.83s (± 0.66%) -0.02s (- 2.75%) 0.82s 0.83s p=0.003 n=6
Check Time 10.47s (± 1.19%) 10.37s (± 0.53%) ~ 10.30s 10.46s p=0.171 n=6
Emit Time 3.07s (± 1.04%) 3.04s (± 0.85%) ~ 3.00s 3.07s p=0.077 n=6
Total Time 16.20s (± 0.88%) 16.03s (± 0.55%) ~ 15.94s 16.20s p=0.054 n=6
Monaco - node (v16.17.1, x64)
Memory used 343,253k (± 0.01%) 343,149k (± 0.01%) -104k (- 0.03%) 343,092k 343,221k p=0.005 n=6
Parse Time 3.15s (± 0.97%) 3.17s (± 1.33%) ~ 3.13s 3.24s p=0.571 n=6
Bind Time 1.11s (± 0.37%) 1.12s (± 0.36%) +0.01s (+ 0.90%) 1.11s 1.12s p=0.008 n=6
Check Time 7.74s (± 0.32%) 7.70s (± 0.24%) -0.04s (- 0.54%) 7.68s 7.72s p=0.019 n=6
Emit Time 4.53s (± 0.54%) 4.53s (± 0.58%) ~ 4.51s 4.58s p=0.935 n=6
Total Time 16.53s (± 0.25%) 16.51s (± 0.39%) ~ 16.46s 16.62s p=0.565 n=6
TFS - node (v16.17.1, x64)
Memory used 299,779k (± 0.00%) 299,646k (± 0.00%) -133k (- 0.04%) 299,627k 299,661k p=0.005 n=6
Parse Time 2.48s (± 1.00%) 2.53s (± 1.20%) +0.05s (+ 2.08%) 2.51s 2.59s p=0.008 n=6
Bind Time 1.26s (± 0.41%) 1.24s (± 0.41%) -0.01s (- 1.06%) 1.24s 1.25s p=0.009 n=6
Check Time 7.20s (± 0.37%) 7.17s (± 0.47%) ~ 7.14s 7.23s p=0.124 n=6
Emit Time 4.22s (± 0.77%) 4.37s (± 2.00%) +0.15s (+ 3.60%) 4.21s 4.46s p=0.030 n=6
Total Time 15.15s (± 0.40%) 15.31s (± 0.73%) +0.16s (+ 1.04%) 15.11s 15.44s p=0.030 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,124k (± 0.00%) 476,046k (± 0.01%) -78k (- 0.02%) 476,018k 476,088k p=0.005 n=6
Parse Time 3.71s (± 0.33%) 3.68s (± 0.20%) -0.04s (- 0.99%) 3.67s 3.69s p=0.003 n=6
Bind Time 1.02s (± 0.40%) 1.01s (± 0.82%) ~ 1.00s 1.02s p=0.115 n=6
Check Time 18.14s (± 0.48%) 18.16s (± 0.70%) ~ 18.06s 18.41s p=0.871 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.88s (± 0.44%) 22.85s (± 0.58%) ~ 22.75s 23.11s p=0.422 n=6
xstate - node (v16.17.1, x64)
Memory used 546,714k (± 0.02%) 546,611k (± 0.04%) ~ 546,390k 546,851k p=0.575 n=6
Parse Time 4.77s (± 0.25%) 4.74s (± 0.22%) -0.04s (- 0.80%) 4.72s 4.75s p=0.005 n=6
Bind Time 1.84s (± 0.49%) 1.80s (± 0.57%) -0.04s (- 2.36%) 1.78s 1.81s p=0.005 n=6
Check Time 3.05s (± 0.84%) 3.04s (± 1.01%) ~ 2.99s 3.07s p=0.422 n=6
Emit Time 0.09s (± 4.45%) 0.09s (± 4.45%) ~ 0.09s 0.10s p=1.000 n=6
Total Time 9.76s (± 0.33%) 9.65s (± 0.27%) -0.11s (- 1.08%) 9.62s 9.69s p=0.005 n=6
Angular - node (v14.15.1, x64)
Memory used 352,833k (± 0.00%) 352,715k (± 0.03%) -118k (- 0.03%) 352,492k 352,771k p=0.005 n=6
Parse Time 4.24s (± 0.13%) 4.24s (± 0.41%) ~ 4.21s 4.26s p=0.866 n=6
Bind Time 1.28s (± 0.59%) 1.29s (± 0.76%) ~ 1.27s 1.30s p=0.167 n=6
Check Time 9.74s (± 0.32%) 9.70s (± 0.54%) ~ 9.65s 9.79s p=0.126 n=6
Emit Time 8.40s (± 0.88%) 8.45s (± 1.71%) ~ 8.34s 8.73s p=0.810 n=6
Total Time 23.65s (± 0.34%) 23.67s (± 0.83%) ~ 23.49s 24.04s p=0.689 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,895k (± 0.07%) 188,800k (± 0.07%) ~ 188,639k 188,964k p=0.378 n=6
Parse Time 1.84s (± 0.49%) 1.83s (± 0.66%) ~ 1.81s 1.84s p=0.084 n=6
Bind Time 0.87s (± 0.63%) 0.85s (± 1.23%) ~ 0.84s 0.87s p=0.090 n=6
Check Time 10.50s (± 0.78%) 10.50s (± 0.49%) ~ 10.42s 10.55s p=1.000 n=6
Emit Time 3.38s (± 4.37%) 3.37s (± 5.51%) ~ 3.17s 3.57s p=0.936 n=6
Total Time 16.58s (± 0.78%) 16.55s (± 1.09%) ~ 16.34s 16.79s p=0.630 n=6
Monaco - node (v14.15.1, x64)
Memory used 338,039k (± 0.00%) 337,945k (± 0.02%) -94k (- 0.03%) 337,893k 338,026k p=0.013 n=6
Parse Time 3.23s (± 0.73%) 3.22s (± 1.00%) ~ 3.18s 3.26s p=0.629 n=6
Bind Time 1.11s (± 0.68%) 1.12s (± 0.67%) ~ 1.11s 1.13s p=0.195 n=6
Check Time 8.12s (± 0.63%) 8.04s (± 0.51%) -0.09s (- 1.07%) 7.98s 8.09s p=0.013 n=6
Emit Time 4.77s (± 0.44%) 4.83s (± 0.95%) +0.06s (+ 1.26%) 4.79s 4.92s p=0.008 n=6
Total Time 17.23s (± 0.27%) 17.20s (± 0.32%) ~ 17.13s 17.28s p=0.334 n=6
TFS - node (v14.15.1, x64)
Memory used 294,587k (± 0.01%) 294,544k (± 0.01%) -44k (- 0.01%) 294,510k 294,582k p=0.031 n=6
Parse Time 2.75s (± 0.48%) 2.78s (± 1.02%) +0.03s (+ 1.03%) 2.75s 2.83s p=0.050 n=6
Bind Time 1.10s (± 0.74%) 1.10s (± 1.87%) ~ 1.08s 1.14s p=0.677 n=6
Check Time 7.50s (± 0.63%) 7.53s (± 0.33%) ~ 7.50s 7.56s p=0.142 n=6
Emit Time 4.63s (± 0.93%) 4.65s (± 1.13%) ~ 4.56s 4.71s p=0.469 n=6
Total Time 15.98s (± 0.29%) 16.07s (± 0.26%) +0.09s (+ 0.56%) 16.01s 16.13s p=0.013 n=6
material-ui - node (v14.15.1, x64)
Memory used 471,677k (± 0.00%) 471,527k (± 0.00%) -151k (- 0.03%) 471,504k 471,550k p=0.005 n=6
Parse Time 3.94s (± 0.94%) 3.92s (± 0.46%) ~ 3.90s 3.95s p=0.412 n=6
Bind Time 1.04s (± 0.52%) 1.05s (± 0.93%) ~ 1.04s 1.06s p=0.232 n=6
Check Time 19.10s (± 0.33%) 19.13s (± 0.34%) ~ 19.04s 19.22s p=0.336 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 24.08s (± 0.32%) 24.10s (± 0.23%) ~ 24.01s 24.17s p=0.688 n=6
xstate - node (v14.15.1, x64)
Memory used 534,841k (± 0.01%) 534,596k (± 0.00%) -246k (- 0.05%) 534,551k 534,617k p=0.005 n=6
Parse Time 5.14s (± 0.85%) 5.13s (± 0.43%) ~ 5.10s 5.16s p=0.748 n=6
Bind Time 1.73s (± 0.85%) 1.72s (± 0.68%) ~ 1.71s 1.74s p=0.458 n=6
Check Time 3.13s (± 0.66%) 3.15s (± 0.26%) ~ 3.14s 3.16s p=0.103 n=6
Emit Time 0.10s (± 4.01%) 0.09s (± 5.53%) 🟩-0.01s (- 8.20%) 0.09s 0.10s p=0.022 n=6
Total Time 10.10s (± 0.42%) 10.11s (± 0.24%) ~ 10.07s 10.14s p=0.520 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

TSServer

Comparison Report - main..51455
Metric main 51455 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,521ms (± 1.16%) 2,507ms (± 1.83%) ~ 2,433ms 2,557ms p=0.470 n=6
Req 2 - geterr 5,534ms (± 0.83%) 5,549ms (± 0.45%) ~ 5,504ms 5,571ms p=0.335 n=6
Req 3 - references 377ms (± 2.21%) 371ms (± 2.04%) ~ 360ms 380ms p=0.261 n=6
Req 4 - navto 283ms (± 0.52%) 283ms (± 0.30%) ~ 282ms 284ms p=0.677 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 66ms (± 1.87%) 66ms (±10.70%) ~ 62ms 80ms p=0.059 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,715ms (± 0.64%) 2,734ms (± 2.21%) ~ 2,671ms 2,818ms p=1.000 n=6
Req 2 - geterr 4,059ms (± 0.45%) 4,064ms (± 0.46%) ~ 4,032ms 4,082ms p=0.688 n=6
Req 3 - references 372ms (± 1.17%) 370ms (± 0.86%) ~ 366ms 374ms p=0.683 n=6
Req 4 - navto 298ms (± 1.03%) 297ms (± 0.70%) ~ 294ms 300ms p=0.745 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 81ms (± 5.29%) 81ms (± 2.75%) ~ 78ms 83ms p=0.604 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,502ms (± 0.34%) 3,490ms (± 0.91%) ~ 3,452ms 3,532ms p=0.689 n=6
Req 2 - geterr 1,641ms (± 0.81%) 1,645ms (± 1.17%) ~ 1,617ms 1,676ms p=0.575 n=6
Req 3 - references 104ms (± 1.57%) 106ms (± 1.75%) +3ms (+ 2.57%) 104ms 108ms p=0.037 n=6
Req 4 - navto 356ms (± 0.50%) 361ms (± 0.52%) +5ms (+ 1.31%) 359ms 363ms p=0.005 n=6
Req 5 - completionInfo count 3,175 (± 0.00%) 3,175 (± 0.00%) ~ 3,175 3,175 p=1.000 n=6
Req 5 - completionInfo 475ms (± 1.12%) 466ms (± 0.57%) -9ms (- 1.93%) 462ms 469ms p=0.019 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,776ms (± 0.89%) 2,741ms (± 1.08%) ~ 2,697ms 2,769ms p=0.128 n=6
Req 2 - geterr 5,923ms (± 0.27%) 5,915ms (± 0.35%) ~ 5,886ms 5,943ms p=0.575 n=6
Req 3 - references 381ms (± 0.49%) 379ms (± 0.59%) ~ 377ms 383ms p=0.460 n=6
Req 4 - navto 283ms (± 2.20%) 279ms (± 1.66%) ~ 276ms 288ms p=0.164 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 71ms (± 1.26%) 70ms (± 0.90%) ~ 69ms 71ms p=0.070 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,942ms (± 0.72%) 2,910ms (± 1.08%) ~ 2,852ms 2,938ms p=0.093 n=6
Req 2 - geterr 4,415ms (± 0.30%) 4,395ms (± 0.50%) ~ 4,368ms 4,427ms p=0.065 n=6
Req 3 - references 382ms (± 0.94%) 380ms (± 0.83%) ~ 376ms 383ms p=0.169 n=6
Req 4 - navto 287ms (± 0.44%) 290ms (± 0.61%) +3ms (+ 0.87%) 288ms 292ms p=0.033 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 72ms (± 1.17%) 77ms (± 7.00%) +5ms (+ 7.46%) 72ms 86ms p=0.017 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,628ms (± 0.45%) 3,630ms (± 0.24%) ~ 3,617ms 3,640ms p=0.873 n=6
Req 2 - geterr 1,814ms (± 1.65%) 1,823ms (± 0.94%) ~ 1,808ms 1,851ms p=0.630 n=6
Req 3 - references 113ms (± 0.91%) 113ms (± 1.94%) ~ 110ms 116ms p=0.803 n=6
Req 4 - navto 342ms (± 0.77%) 346ms (± 1.45%) ~ 340ms 354ms p=0.196 n=6
Req 5 - completionInfo count 3,175 (± 0.00%) 3,175 (± 0.00%) ~ 3,175 3,175 p=1.000 n=6
Req 5 - completionInfo 479ms (± 0.68%) 476ms (± 0.80%) ~ 469ms 479ms p=0.372 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,914ms (± 0.53%) 2,871ms (± 0.46%) -43ms (- 1.47%) 2,858ms 2,891ms p=0.005 n=6
Req 2 - geterr 6,354ms (± 1.25%) 6,283ms (± 0.55%) ~ 6,234ms 6,332ms p=0.093 n=6
Req 3 - references 392ms (± 0.39%) 390ms (± 0.88%) ~ 386ms 396ms p=0.124 n=6
Req 4 - navto 284ms (± 1.35%) 281ms (± 1.45%) ~ 277ms 288ms p=0.418 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 85ms (±10.93%) 89ms (± 9.03%) ~ 73ms 95ms p=0.371 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,087ms (± 0.79%) 3,067ms (± 0.74%) ~ 3,046ms 3,105ms p=0.093 n=6
Req 2 - geterr 4,717ms (± 0.38%) 4,726ms (± 0.39%) ~ 4,697ms 4,751ms p=0.336 n=6
Req 3 - references 409ms (± 1.15%) 407ms (± 1.44%) ~ 400ms 416ms p=0.455 n=6
Req 4 - navto 296ms (± 0.39%) 296ms (± 0.50%) ~ 295ms 299ms p=0.933 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 83ms (± 4.52%) 81ms (± 3.66%) ~ 79ms 87ms p=0.054 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,990ms (± 0.51%) 3,983ms (± 0.41%) ~ 3,954ms 4,000ms p=0.423 n=6
Req 2 - geterr 1,799ms (± 1.93%) 1,821ms (± 1.60%) ~ 1,786ms 1,863ms p=0.230 n=6
Req 3 - references 131ms (± 3.33%) 131ms (± 2.34%) ~ 126ms 135ms p=0.936 n=6
Req 4 - navto 374ms (± 0.81%) 376ms (± 0.37%) ~ 374ms 377ms p=0.252 n=6
Req 5 - completionInfo count 3,175 (± 0.00%) 3,175 (± 0.00%) ~ 3,175 3,175 p=1.000 n=6
Req 5 - completionInfo 499ms (± 1.45%) 493ms (± 1.18%) ~ 486ms 500ms p=0.128 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Startup

Comparison Report - main..51455
Metric main 51455 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 142.90ms (± 0.19%) 142.08ms (± 0.16%) -0.82ms (- 0.58%) 141.46ms 144.26ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 227.88ms (± 0.20%) 227.69ms (± 0.14%) -0.19ms (- 0.08%) 226.86ms 232.15ms p=0.002 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 229.86ms (± 0.26%) 229.43ms (± 0.17%) -0.43ms (- 0.19%) 228.45ms 238.99ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 210.74ms (± 0.29%) 210.35ms (± 0.16%) -0.39ms (- 0.19%) 209.54ms 214.07ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey jakebailey requested a review from rbuckton February 6, 2023 18:18
@jakebailey jakebailey marked this pull request as draft January 19, 2024 20:53
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 19, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 042b96c. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,644k (± 0.01%) 295,620k (± 0.01%) ~ 295,592k 295,645k p=0.230 n=6
Parse Time 2.66s (± 0.19%) 2.67s (± 0.33%) ~ 2.66s 2.68s p=0.190 n=6
Bind Time 0.83s (± 0.91%) 0.83s (± 1.40%) ~ 0.82s 0.85s p=0.734 n=6
Check Time 8.18s (± 0.41%) 8.17s (± 0.25%) ~ 8.15s 8.20s p=0.804 n=6
Emit Time 7.10s (± 0.35%) 7.18s (± 0.29%) +0.08s (+ 1.06%) 7.15s 7.21s p=0.005 n=6
Total Time 18.78s (± 0.27%) 18.85s (± 0.13%) +0.07s (+ 0.40%) 18.82s 18.88s p=0.036 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,907k (± 1.49%) 192,418k (± 1.27%) -1,490k (- 0.77%) 191,394k 197,429k p=0.045 n=6
Parse Time 1.36s (± 0.86%) 1.34s (± 0.56%) -0.02s (- 1.47%) 1.33s 1.35s p=0.011 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.57%) ~ 0.72s 0.73s p=0.405 n=6
Check Time 9.38s (± 0.30%) 9.37s (± 0.20%) ~ 9.34s 9.39s p=0.517 n=6
Emit Time 2.61s (± 0.87%) 2.62s (± 0.78%) ~ 2.58s 2.64s p=0.285 n=6
Total Time 14.07s (± 0.37%) 14.05s (± 0.13%) ~ 14.02s 14.07s p=0.196 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,469k (± 0.00%) 347,463k (± 0.00%) ~ 347,449k 347,485k p=0.470 n=6
Parse Time 2.49s (± 0.47%) 2.51s (± 0.33%) +0.03s (+ 1.07%) 2.51s 2.53s p=0.004 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 6.91s (± 0.54%) 6.93s (± 0.36%) ~ 6.90s 6.96s p=0.334 n=6
Emit Time 4.06s (± 0.29%) 4.10s (± 0.34%) +0.04s (+ 1.03%) 4.08s 4.12s p=0.006 n=6
Total Time 14.38s (± 0.22%) 14.48s (± 0.24%) +0.09s (+ 0.65%) 14.42s 14.51s p=0.005 n=6
TFS - node (v18.15.0, x64)
Memory used 302,837k (± 0.01%) 302,758k (± 0.01%) -79k (- 0.03%) 302,732k 302,812k p=0.008 n=6
Parse Time 2.01s (± 0.73%) 2.02s (± 1.03%) ~ 1.99s 2.05s p=1.000 n=6
Bind Time 1.00s (± 1.22%) 1.01s (± 0.51%) ~ 1.00s 1.01s p=0.672 n=6
Check Time 6.33s (± 0.51%) 6.39s (± 0.31%) +0.05s (+ 0.82%) 6.36s 6.40s p=0.012 n=6
Emit Time 3.59s (± 0.23%) 3.60s (± 0.68%) ~ 3.56s 3.63s p=0.452 n=6
Total Time 12.95s (± 0.29%) 13.01s (± 0.23%) +0.06s (+ 0.48%) 12.95s 13.03s p=0.020 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,309k (± 0.00%) 511,284k (± 0.01%) ~ 511,252k 511,329k p=0.128 n=6
Parse Time 2.64s (± 0.65%) 2.72s (± 0.63%) +0.08s (+ 2.90%) 2.69s 2.74s p=0.005 n=6
Bind Time 1.00s (± 0.75%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=1.000 n=6
Check Time 17.21s (± 0.41%) 17.25s (± 0.48%) ~ 17.12s 17.33s p=0.630 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.86s (± 0.30%) 20.96s (± 0.44%) ~ 20.83s 21.06s p=0.078 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,695,968k (± 0.00%) 1,695,935k (± 0.00%) ~ 1,695,915k 1,695,960k p=0.065 n=6
Parse Time 6.54s (± 0.16%) 6.59s (± 0.47%) +0.06s (+ 0.89%) 6.56s 6.64s p=0.005 n=6
Bind Time 2.28s (± 6.54%) 2.35s (± 0.70%) ~ 2.33s 2.38s p=0.113 n=6
Check Time 55.53s (± 0.39%) 55.36s (± 0.34%) ~ 55.15s 55.66s p=0.229 n=6
Emit Time 0.16s (± 0.00%) 0.16s (± 0.00%) ~ 0.16s 0.16s p=1.000 n=6
Total Time 64.51s (± 0.47%) 64.47s (± 0.31%) ~ 64.29s 64.79s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,412,834k (± 0.02%) 2,414,087k (± 0.03%) +1,253k (+ 0.05%) 2,413,086k 2,415,371k p=0.020 n=6
Parse Time 4.95s (± 0.71%) 4.94s (± 0.71%) ~ 4.89s 4.98s p=0.810 n=6
Bind Time 1.88s (± 0.73%) 1.88s (± 0.55%) ~ 1.87s 1.90s p=0.391 n=6
Check Time 33.33s (± 0.39%) 33.30s (± 0.28%) ~ 33.14s 33.41s p=0.748 n=6
Emit Time 2.73s (± 1.59%) 2.80s (± 1.65%) +0.08s (+ 2.87%) 2.73s 2.87s p=0.020 n=6
Total Time 42.91s (± 0.21%) 42.93s (± 0.36%) ~ 42.70s 43.16s p=0.810 n=6
self-compiler - node (v18.15.0, x64)
Memory used 419,666k (± 0.01%) 419,369k (± 0.01%) -297k (- 0.07%) 419,326k 419,414k p=0.005 n=6
Parse Time 2.76s (± 2.87%) 2.77s (± 2.55%) ~ 2.67s 2.84s p=0.747 n=6
Bind Time 1.12s (± 6.82%) 1.11s (± 6.76%) ~ 1.06s 1.21s p=0.139 n=6
Check Time 15.10s (± 0.23%) 15.03s (± 0.31%) -0.07s (- 0.49%) 14.96s 15.07s p=0.020 n=6
Emit Time 1.16s (± 0.54%) 1.14s (± 0.66%) -0.02s (- 1.58%) 1.13s 1.15s p=0.006 n=6
Total Time 20.14s (± 0.29%) 20.05s (± 0.35%) ~ 19.97s 20.13s p=0.090 n=6
vscode - node (v18.15.0, x64)
Memory used 2,807,297k (± 0.00%) 2,807,087k (± 0.00%) -209k (- 0.01%) 2,807,045k 2,807,142k p=0.005 n=6
Parse Time 10.63s (± 0.55%) 10.63s (± 0.31%) ~ 10.59s 10.69s p=0.466 n=6
Bind Time 3.38s (± 0.37%) 3.39s (± 0.24%) ~ 3.39s 3.41s p=0.057 n=6
Check Time 59.74s (± 0.25%) 59.56s (± 0.11%) -0.19s (- 0.31%) 59.46s 59.66s p=0.045 n=6
Emit Time 16.12s (± 0.47%) 16.08s (± 0.35%) ~ 16.00s 16.15s p=0.260 n=6
Total Time 89.88s (± 0.18%) 89.66s (± 0.13%) ~ 89.54s 89.86s p=0.054 n=6
webpack - node (v18.15.0, x64)
Memory used 392,365k (± 0.01%) 392,255k (± 0.01%) -110k (- 0.03%) 392,179k 392,322k p=0.005 n=6
Parse Time 3.06s (± 1.07%) 3.08s (± 1.01%) ~ 3.04s 3.13s p=0.257 n=6
Bind Time 1.40s (± 0.98%) 1.40s (± 0.70%) ~ 1.38s 1.41s p=1.000 n=6
Check Time 13.98s (± 0.23%) 13.96s (± 0.24%) ~ 13.94s 14.03s p=0.332 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.44s (± 0.30%) 18.44s (± 0.22%) ~ 18.38s 18.49s p=0.873 n=6
xstate - node (v18.15.0, x64)
Memory used 513,440k (± 0.02%) 513,427k (± 0.01%) ~ 513,359k 513,482k p=0.936 n=6
Parse Time 3.29s (± 0.42%) 3.31s (± 0.12%) +0.02s (+ 0.71%) 3.30s 3.31s p=0.032 n=6
Bind Time 1.54s (± 0.41%) 1.56s (± 0.33%) +0.02s (+ 1.08%) 1.55s 1.56s p=0.006 n=6
Check Time 2.85s (± 0.36%) 2.88s (± 0.92%) ~ 2.83s 2.90s p=0.063 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.76s (± 0.12%) 7.82s (± 0.35%) +0.06s (+ 0.73%) 7.78s 7.85s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,344ms (± 0.98%) 2,331ms (± 1.04%) ~ 2,291ms 2,361ms p=0.471 n=6
Req 2 - geterr 5,524ms (± 1.23%) 5,494ms (± 1.11%) ~ 5,431ms 5,603ms p=0.471 n=6
Req 3 - references 325ms (± 0.71%) 324ms (± 1.14%) ~ 320ms 331ms p=0.199 n=6
Req 4 - navto 275ms (± 1.37%) 276ms (± 0.99%) ~ 270ms 277ms p=0.730 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 85ms (± 8.11%) 81ms (± 6.98%) ~ 78ms 92ms p=0.084 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,482ms (± 0.72%) 2,481ms (± 0.87%) ~ 2,460ms 2,511ms p=0.873 n=6
Req 2 - geterr 4,186ms (± 2.01%) 4,182ms (± 2.10%) ~ 4,097ms 4,270ms p=0.521 n=6
Req 3 - references 333ms (± 1.50%) 331ms (± 0.32%) ~ 329ms 332ms p=0.219 n=6
Req 4 - navto 285ms (± 0.56%) 286ms (± 0.96%) ~ 284ms 291ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 7.52%) 75ms (± 2.71%) ~ 73ms 77ms p=0.139 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,610ms (± 0.37%) 2,642ms (± 0.36%) +32ms (+ 1.24%) 2,628ms 2,656ms p=0.005 n=6
Req 2 - geterr 1,748ms (± 1.38%) 1,712ms (± 2.47%) ~ 1,662ms 1,756ms p=0.093 n=6
Req 3 - references 117ms (±10.20%) 116ms (± 9.42%) ~ 106ms 127ms p=1.000 n=6
Req 4 - navto 371ms (± 0.42%) 368ms (± 1.08%) ~ 363ms 373ms p=0.195 n=6
Req 5 - completionInfo count 2,078 (± 0.00%) 2,078 (± 0.00%) ~ 2,078 2,078 p=1.000 n=6
Req 5 - completionInfo 310ms (± 1.28%) 304ms (± 1.17%) -6ms (- 1.88%) 301ms 310ms p=0.037 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 153.92ms (± 0.20%) 153.98ms (± 0.25%) +0.06ms (+ 0.04%) 152.76ms 166.41ms p=0.045 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.84ms (± 0.16%) 229.68ms (± 0.14%) -0.16ms (- 0.07%) 228.43ms 232.31ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 232.25ms (± 0.18%) 232.92ms (± 0.16%) +0.67ms (+ 0.29%) 231.43ms 239.04ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 231.26ms (± 0.19%) 231.26ms (± 0.16%) ~ 229.73ms 235.50ms p=0.664 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,725k (± 0.01%) 295,581k (± 0.01%) -144k (- 0.05%) 295,563k 295,605k p=0.005 n=6
Parse Time 2.66s (± 0.34%) 2.67s (± 0.24%) ~ 2.66s 2.68s p=0.070 n=6
Bind Time 0.83s (± 0.49%) 0.83s (± 0.49%) ~ 0.83s 0.84s p=0.218 n=6
Check Time 8.21s (± 0.38%) 8.19s (± 0.29%) ~ 8.15s 8.21s p=0.195 n=6
Emit Time 7.13s (± 0.17%) 7.06s (± 0.24%) -0.07s (- 0.94%) 7.04s 7.08s p=0.005 n=6
Total Time 18.83s (± 0.21%) 18.75s (± 0.19%) -0.08s (- 0.42%) 18.71s 18.80s p=0.020 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,701k (± 1.00%) 194,254k (± 0.96%) ~ 191,827k 195,529k p=1.000 n=6
Parse Time 1.36s (± 1.72%) 1.36s (± 1.20%) ~ 1.34s 1.38s p=1.000 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.88%) ~ 0.71s 0.73s p=1.000 n=6
Check Time 9.51s (± 1.05%) 9.43s (± 0.23%) ~ 9.40s 9.46s p=0.170 n=6
Emit Time 2.65s (± 0.74%) 2.58s (± 0.47%) -0.06s (- 2.39%) 2.57s 2.60s p=0.005 n=6
Total Time 14.23s (± 0.84%) 14.08s (± 0.28%) ~ 14.05s 14.16s p=0.064 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,377k (± 0.00%) 347,372k (± 0.00%) ~ 347,356k 347,396k p=0.470 n=6
Parse Time 2.48s (± 0.79%) 2.52s (± 0.68%) +0.04s (+ 1.54%) 2.49s 2.54s p=0.014 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 7.01s (± 0.36%) 7.02s (± 0.29%) ~ 6.99s 7.05s p=0.332 n=6
Emit Time 4.06s (± 0.19%) 4.09s (± 0.34%) +0.02s (+ 0.57%) 4.07s 4.11s p=0.007 n=6
Total Time 14.48s (± 0.24%) 14.56s (± 0.18%) +0.08s (+ 0.56%) 14.52s 14.59s p=0.008 n=6
TFS - node (v18.15.0, x64)
Memory used 302,757k (± 0.01%) 302,611k (± 0.01%) -146k (- 0.05%) 302,587k 302,654k p=0.005 n=6
Parse Time 2.01s (± 0.51%) 2.01s (± 0.73%) ~ 2.00s 2.04s p=0.605 n=6
Bind Time 1.00s (± 1.03%) 1.01s (± 0.81%) ~ 1.00s 1.02s p=0.546 n=6
Check Time 6.33s (± 0.59%) 6.32s (± 0.48%) ~ 6.26s 6.34s p=0.809 n=6
Emit Time 3.61s (± 0.61%) 3.59s (± 0.75%) ~ 3.57s 3.63s p=0.195 n=6
Total Time 12.95s (± 0.35%) 12.93s (± 0.27%) ~ 12.89s 12.98s p=0.294 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,368k (± 0.00%) 511,355k (± 0.00%) ~ 511,334k 511,366k p=0.226 n=6
Parse Time 2.65s (± 0.58%) 2.71s (± 0.40%) +0.06s (+ 2.26%) 2.69s 2.72s p=0.005 n=6
Bind Time 0.98s (± 0.91%) 0.99s (± 0.83%) ~ 0.98s 1.00s p=0.270 n=6
Check Time 17.28s (± 0.27%) 17.33s (± 0.28%) ~ 17.27s 17.39s p=0.078 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.91s (± 0.23%) 21.03s (± 0.19%) +0.12s (+ 0.57%) 20.98s 21.09s p=0.006 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,719,121k (± 0.00%) 1,719,049k (± 0.00%) -72k (- 0.00%) 1,719,011k 1,719,078k p=0.005 n=6
Parse Time 6.52s (± 0.47%) 6.61s (± 0.63%) +0.09s (+ 1.33%) 6.57s 6.68s p=0.006 n=6
Bind Time 2.36s (± 0.32%) 2.38s (± 0.86%) ~ 2.35s 2.41s p=0.060 n=6
Check Time 56.38s (± 0.37%) 56.18s (± 0.34%) ~ 55.85s 56.40s p=0.128 n=6
Emit Time 0.13s (± 0.00%) 0.13s (± 3.10%) ~ 0.13s 0.14s p=0.405 n=6
Total Time 65.39s (± 0.29%) 65.29s (± 0.36%) ~ 64.90s 65.58s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,395,371k (± 0.02%) 2,397,074k (± 0.04%) +1,703k (+ 0.07%) 2,396,172k 2,398,156k p=0.010 n=6
Parse Time 4.98s (± 1.07%) 5.01s (± 0.53%) ~ 4.99s 5.05s p=0.108 n=6
Bind Time 1.90s (± 1.17%) 1.90s (± 0.74%) ~ 1.88s 1.92s p=0.935 n=6
Check Time 33.63s (± 0.16%) 33.47s (± 0.42%) ~ 33.32s 33.69s p=0.066 n=6
Emit Time 2.72s (± 1.20%) 2.69s (± 3.42%) ~ 2.58s 2.82s p=0.630 n=6
Total Time 43.25s (± 0.22%) 43.10s (± 0.33%) ~ 42.94s 43.32s p=0.065 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,068k (± 0.01%) 415,688k (± 0.01%) -380k (- 0.09%) 415,646k 415,718k p=0.005 n=6
Parse Time 2.84s (± 1.05%) 2.82s (± 2.02%) ~ 2.71s 2.86s p=1.000 n=6
Bind Time 1.07s (± 0.51%) 1.09s (± 5.41%) ~ 1.06s 1.21s p=0.476 n=6
Check Time 15.32s (± 0.44%) 15.25s (± 0.35%) ~ 15.19s 15.34s p=0.092 n=6
Emit Time 1.16s (± 1.33%) 1.12s (± 0.80%) 🟩-0.04s (- 3.45%) 1.11s 1.13s p=0.005 n=6
Total Time 20.39s (± 0.29%) 20.28s (± 0.30%) -0.10s (- 0.51%) 20.23s 20.40s p=0.030 n=6
vscode - node (v18.15.0, x64)
Memory used 2,883,638k (± 0.00%) 2,883,267k (± 0.00%) -371k (- 0.01%) 2,883,114k 2,883,385k p=0.005 n=6
Parse Time 10.82s (± 0.21%) 10.84s (± 0.35%) ~ 10.78s 10.90s p=0.462 n=6
Bind Time 3.46s (± 0.61%) 3.48s (± 0.49%) ~ 3.46s 3.51s p=0.120 n=6
Check Time 61.61s (± 0.37%) 61.75s (± 0.47%) ~ 61.38s 62.24s p=0.423 n=6
Emit Time 16.43s (± 0.77%) 17.58s (±10.01%) ~ 16.40s 19.93s p=0.336 n=6
Total Time 92.33s (± 0.27%) 93.65s (± 2.14%) ~ 92.18s 96.34s p=0.295 n=6
webpack - node (v18.15.0, x64)
Memory used 408,015k (± 0.02%) 407,858k (± 0.02%) -157k (- 0.04%) 407,764k 407,935k p=0.013 n=6
Parse Time 3.23s (± 0.36%) 3.19s (± 0.17%) -0.05s (- 1.44%) 3.18s 3.19s p=0.004 n=6
Bind Time 1.38s (± 0.00%) 1.39s (± 0.74%) ~ 1.37s 1.40s p=0.126 n=6
Check Time 14.29s (± 0.23%) 14.23s (± 0.55%) ~ 14.14s 14.37s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.91s (± 0.21%) 18.80s (± 0.42%) -0.11s (- 0.59%) 18.72s 18.94s p=0.029 n=6
xstate - node (v18.15.0, x64)
Memory used 513,120k (± 0.01%) 513,119k (± 0.01%) ~ 513,026k 513,216k p=1.000 n=6
Parse Time 3.27s (± 0.27%) 3.31s (± 0.44%) +0.04s (+ 1.27%) 3.30s 3.33s p=0.004 n=6
Bind Time 1.54s (± 0.00%) 1.56s (± 0.66%) +0.02s (+ 1.08%) 1.54s 1.57s p=0.009 n=6
Check Time 2.85s (± 0.52%) 2.86s (± 0.61%) ~ 2.83s 2.88s p=0.254 n=6
Emit Time 0.07s (± 5.69%) 0.08s (± 0.00%) 🔻+0.01s (+11.63%) 0.08s 0.08s p=0.007 n=6
Total Time 7.74s (± 0.26%) 7.80s (± 0.39%) +0.07s (+ 0.84%) 7.75s 7.83s p=0.010 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,341ms (± 0.65%) 2,319ms (± 0.62%) ~ 2,303ms 2,338ms p=0.065 n=6
Req 2 - geterr 5,809ms (± 0.48%) 5,790ms (± 0.37%) ~ 5,764ms 5,815ms p=0.261 n=6
Req 3 - references 324ms (± 0.39%) 324ms (± 0.26%) ~ 323ms 325ms p=0.530 n=6
Req 4 - navto 272ms (± 0.19%) 273ms (± 0.19%) +1ms (+ 0.37%) 273ms 274ms p=0.018 n=6
Req 5 - completionInfo count 1,357 (± 0.00%) 1,357 (± 0.00%) ~ 1,357 1,357 p=1.000 n=6
Req 5 - completionInfo 95ms (± 0.43%) 94ms (± 0.55%) -1ms (- 1.23%) 93ms 94ms p=0.007 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,508ms (± 0.66%) 2,443ms (± 0.57%) -65ms (- 2.60%) 2,425ms 2,460ms p=0.005 n=6
Req 2 - geterr 4,259ms (± 0.11%) 4,260ms (± 0.33%) ~ 4,249ms 4,287ms p=0.296 n=6
Req 3 - references 331ms (± 0.33%) 332ms (± 0.94%) ~ 330ms 338ms p=0.867 n=6
Req 4 - navto 285ms (± 0.36%) 285ms (± 0.37%) ~ 283ms 286ms p=0.203 n=6
Req 5 - completionInfo count 1,519 (± 0.00%) 1,519 (± 0.00%) ~ 1,519 1,519 p=1.000 n=6
Req 5 - completionInfo 78ms (± 1.15%) 80ms (± 5.88%) ~ 77ms 89ms p=0.932 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,618ms (± 0.29%) 2,611ms (± 0.34%) ~ 2,595ms 2,620ms p=0.199 n=6
Req 2 - geterr 1,732ms (± 3.33%) 1,696ms (± 0.95%) ~ 1,673ms 1,713ms p=0.575 n=6
Req 3 - references 121ms (± 7.81%) 124ms (± 6.85%) ~ 107ms 130ms p=0.744 n=6
Req 4 - navto 372ms (± 1.42%) 371ms (± 0.74%) ~ 368ms 375ms p=0.746 n=6
Req 5 - completionInfo count 2,079 (± 0.00%) 2,079 (± 0.00%) ~ 2,079 2,079 p=1.000 n=6
Req 5 - completionInfo 311ms (± 2.76%) 306ms (± 1.46%) ~ 299ms 313ms p=0.625 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 154.14ms (± 0.17%) 154.31ms (± 0.18%) +0.17ms (+ 0.11%) 153.11ms 156.69ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.39ms (± 0.16%) 230.09ms (± 0.14%) -0.30ms (- 0.13%) 228.72ms 233.30ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 223.47ms (± 0.17%) 223.55ms (± 0.17%) +0.08ms (+ 0.04%) 221.95ms 231.23ms p=0.039 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 221.95ms (± 0.15%) 222.18ms (± 0.17%) +0.22ms (+ 0.10%) 220.76ms 225.95ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey mentioned this pull request Apr 3, 2024
Merged
@jakebailey
Copy link
Member Author

Unfortunately, my work to make our package smaller by reusing the public API basically renders this PR pointless outside of src/compiler, since going through the namespaces and never direct importing is the only reason those work. Not happy about that.

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 1, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 1, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,862k (± 0.77%) 193,264k (± 0.94%) ~ 192,068k 195,652k p=0.229 n=6
Parse Time 1.58s (± 0.67%) 1.55s (± 0.67%) -0.02s (- 1.27%) 1.54s 1.57s p=0.018 n=6
Bind Time 0.86s (± 0.60%) 0.85s (± 1.57%) ~ 0.84s 0.87s p=0.203 n=6
Check Time 11.23s (± 0.54%) 11.23s (± 0.64%) ~ 11.09s 11.30s p=0.936 n=6
Emit Time 3.29s (± 0.73%) 3.21s (± 1.15%) -0.08s (- 2.33%) 3.17s 3.28s p=0.012 n=6
Total Time 16.95s (± 0.30%) 16.84s (± 0.17%) -0.11s (- 0.62%) 16.81s 16.89s p=0.006 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,360k (± 0.00%) 1,218,284k (± 0.00%) -77k (- 0.01%) 1,218,245k 1,218,335k p=0.013 n=6
Parse Time 7.98s (± 0.42%) 7.96s (± 0.65%) ~ 7.89s 8.02s p=0.625 n=6
Bind Time 2.22s (± 0.18%) 2.24s (± 0.49%) +0.02s (+ 0.83%) 2.23s 2.26s p=0.005 n=6
Check Time 35.80s (± 0.33%) 35.70s (± 0.25%) ~ 35.58s 35.84s p=0.230 n=6
Emit Time 16.20s (± 0.84%) 16.09s (± 0.78%) ~ 15.91s 16.28s p=0.199 n=6
Total Time 62.20s (± 0.33%) 61.99s (± 0.27%) ~ 61.72s 62.22s p=0.093 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,764k (± 0.00%) 2,117,821k (± 0.01%) ~ 2,117,690k 2,117,947k p=0.378 n=6
Parse Time 7.90s (± 0.38%) 7.91s (± 0.34%) ~ 7.88s 7.95s p=0.570 n=6
Bind Time 2.76s (± 0.44%) 2.76s (± 0.67%) ~ 2.73s 2.78s p=1.000 n=6
Check Time 83.81s (± 0.50%) 84.22s (± 1.33%) ~ 83.24s 86.41s p=0.471 n=6
Emit Time 0.16s (± 2.52%) 0.16s (± 3.16%) ~ 0.16s 0.17s p=0.595 n=6
Total Time 94.63s (± 0.46%) 95.06s (± 1.18%) ~ 94.07s 97.25s p=0.378 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,225,281 +21 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,341 261,252 -89 (- 0.03%) ~ ~ p=0.001 n=6
Memory used 2,341,202k (± 0.06%) 2,351,939k (± 1.07%) ~ 2,340,075k 2,403,097k p=0.378 n=6
Parse Time 7.47s (± 0.92%) 7.56s (± 0.50%) ~ 7.51s 7.60s p=0.066 n=6
Bind Time 2.77s (± 0.32%) 2.80s (± 1.03%) ~ 2.76s 2.84s p=0.122 n=6
Check Time 49.50s (± 0.43%) 49.12s (± 0.61%) -0.38s (- 0.76%) 48.53s 49.37s p=0.013 n=6
Emit Time 3.81s (± 2.63%) 3.91s (± 2.90%) ~ 3.77s 4.08s p=0.093 n=6
Total Time 63.55s (± 0.48%) 63.39s (± 0.60%) ~ 62.62s 63.65s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6+0
Symbols 1,225,260 0 ~ ~ ~ p=1.000 n=6+0
Types 261,341 0 ~ ~ ~ p=1.000 n=6+0
Memory used 2,414,897k (± 0.05%) 0k ~ ~ ~ p=1.000 n=6+0
Parse Time 5.25s (± 0.60%) 0s ~ ~ ~ p=1.000 n=6+0
Bind Time 1.70s (± 0.48%) 0s ~ ~ ~ p=1.000 n=6+0
Check Time 34.48s (± 0.29%) 0s ~ ~ ~ p=1.000 n=6+0
Emit Time 2.71s (± 5.75%) 0s ~ ~ ~ p=1.000 n=6+0
Total Time 44.12s (± 0.41%) 0s ~ ~ ~ p=1.000 n=6+0
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,063 -131 (- 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,643 -94 (- 0.09%) ~ ~ p=0.001 n=6
Memory used 427,461k (± 0.01%) 427,267k (± 0.01%) -193k (- 0.05%) 427,231k 427,299k p=0.005 n=6
Parse Time 4.07s (± 0.31%) 4.17s (± 0.66%) +0.09s (+ 2.33%) 4.13s 4.21s p=0.004 n=6
Bind Time 1.62s (± 1.15%) 1.59s (± 0.89%) -0.03s (- 2.05%) 1.57s 1.61s p=0.019 n=6
Check Time 22.00s (± 0.16%) 22.00s (± 0.32%) ~ 21.92s 22.09s p=0.810 n=6
Emit Time 1.55s (± 1.33%) 1.58s (± 0.74%) +0.03s (+ 1.71%) 1.57s 1.60s p=0.028 n=6
Total Time 29.25s (± 0.13%) 29.34s (± 0.31%) ~ 29.23s 29.48s p=0.064 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,470k (± 0.02%) 369,514k (± 0.03%) ~ 369,350k 369,682k p=0.575 n=6
Parse Time 2.76s (± 1.20%) 2.78s (± 1.28%) ~ 2.71s 2.81s p=0.421 n=6
Bind Time 1.60s (± 1.28%) 1.58s ~ ~ ~ p=0.073 n=6
Check Time 15.48s (± 0.49%) 15.47s (± 0.23%) ~ 15.44s 15.53s p=0.418 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.45%) 19.83s (± 0.31%) ~ 19.72s 19.89s p=0.744 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,975 2,879,975 ~ ~ ~ p=1.000 n=6
Types 975,327 975,327 ~ ~ ~ p=1.000 n=6
Memory used 3,042,724k (± 0.00%) 3,042,569k (± 0.00%) -155k (- 0.01%) 3,042,494k 3,042,627k p=0.005 n=6
Parse Time 11.48s (± 0.60%) 11.47s (± 0.15%) ~ 11.45s 11.49s p=0.745 n=6
Bind Time 3.51s (± 0.33%) 3.51s (± 0.30%) ~ 3.50s 3.53s p=0.619 n=6
Check Time 64.79s (± 2.32%) 65.14s (± 2.95%) ~ 63.25s 66.98s p=0.689 n=6
Emit Time 18.99s (± 7.53%) 18.85s (± 9.95%) ~ 17.11s 20.60s p=0.810 n=6
Total Time 98.77s (± 0.17%) 98.98s (± 0.13%) +0.21s (+ 0.21%) 98.79s 99.18s p=0.045 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,631k (± 0.02%) 411,532k (± 0.00%) -98k (- 0.02%) 411,517k 411,550k p=0.005 n=6
Parse Time 3.83s (± 0.35%) 3.74s (± 0.90%) -0.09s (- 2.26%) 3.70s 3.79s p=0.005 n=6
Bind Time 1.69s (± 0.31%) 1.69s (± 1.27%) ~ 1.67s 1.73s p=0.803 n=6
Check Time 16.72s (± 0.42%) 16.69s (± 0.50%) ~ 16.54s 16.77s p=0.936 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.24s (± 0.37%) 22.13s (± 0.55%) ~ 21.95s 22.29s p=0.173 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,950k (± 0.09%) 462,952k (± 0.07%) ~ 462,323k 463,227k p=0.378 n=6
Parse Time 2.65s (± 0.44%) 2.69s (± 0.77%) +0.03s (+ 1.26%) 2.67s 2.72s p=0.009 n=6
Bind Time 0.98s (± 0.77%) 0.98s (± 0.77%) ~ 0.97s 0.99s p=1.000 n=6
Check Time 15.14s (± 0.19%) 15.17s (± 0.40%) ~ 15.10s 15.26s p=0.571 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.77s (± 0.19%) 18.83s (± 0.42%) ~ 18.74s 18.97s p=0.170 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,424ms (± 0.43%) 3,408ms (± 0.53%) ~ 3,379ms 3,428ms p=0.173 n=6
Req 2 - geterr 7,468ms (± 0.60%) 7,512ms (± 0.85%) ~ 7,414ms 7,573ms p=0.173 n=6
Req 3 - references 394ms (± 0.37%) 394ms (± 0.78%) ~ 390ms 398ms p=0.684 n=6
Req 4 - navto 340ms (± 0.52%) 342ms (± 0.60%) +3ms (+ 0.79%) 340ms 345ms p=0.049 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 129ms (± 0.91%) 117ms (± 8.07%) 🟩-12ms (- 8.93%) 101ms 129ms p=0.028 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,405ms (± 1.31%) 2,394ms (± 0.70%) ~ 2,367ms 2,417ms p=0.378 n=6
Req 2 - geterr 3,791ms (± 0.25%) 3,770ms (± 0.62%) ~ 3,744ms 3,800ms p=0.173 n=6
Req 3 - references 277ms (± 0.27%) 279ms (± 0.75%) ~ 275ms 281ms p=0.062 n=6
Req 4 - navto 226ms (± 0.65%) 227ms (± 0.18%) ~ 226ms 227ms p=0.198 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 73ms (± 2.66%) 81ms (± 7.95%) ~ 68ms 84ms p=0.059 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 7,718ms (± 0.32%) 7,809ms (± 0.27%) +91ms (+ 1.18%) 7,787ms 7,837ms p=0.005 n=6
Req 2 - geterr 1,693ms (± 1.73%) 1,690ms (± 0.36%) ~ 1,680ms 1,699ms p=0.375 n=6
Req 3 - references 113ms (± 3.71%) 112ms (± 3.88%) ~ 110ms 121ms p=0.432 n=6
Req 4 - navto 600ms (± 1.89%) 599ms (± 2.00%) ~ 582ms 612ms p=0.936 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,258ms (± 2.00%) 1,226ms (± 1.08%) ~ 1,208ms 1,248ms p=0.066 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 188.72ms (± 0.16%) 189.11ms (± 0.20%) +0.39ms (+ 0.21%) 187.12ms 194.41ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 283.54ms (± 0.29%) 283.58ms (± 0.27%) ~ 276.93ms 288.75ms p=0.390 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 276.61ms (± 0.30%) 276.63ms (± 0.29%) ~ 269.69ms 284.54ms p=0.422 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.65ms (± 0.13%) 226.86ms (± 0.14%) +0.22ms (+ 0.10%) 225.56ms 230.32ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,199k (± 0.07%) 193,199k (± 0.83%) ~ 192,026k 195,618k p=0.936 n=6
Parse Time 1.31s (± 0.42%) 1.29s (± 0.76%) -0.01s (- 1.02%) 1.28s 1.31s p=0.030 n=6
Bind Time 0.71s 0.70s -0.01s (- 1.41%) ~ ~ p=0.001 n=6
Check Time 9.46s (± 0.37%) 9.42s (± 0.38%) ~ 9.37s 9.47s p=0.104 n=6
Emit Time 2.74s (± 0.38%) 2.69s (± 0.53%) -0.05s (- 2.00%) 2.67s 2.71s p=0.005 n=6
Total Time 14.22s (± 0.24%) 14.10s (± 0.37%) -0.12s (- 0.86%) 14.03s 14.17s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,423k (± 0.01%) 1,218,312k (± 0.01%) ~ 1,218,192k 1,218,406k p=0.066 n=6
Parse Time 6.64s (± 0.35%) 6.64s (± 0.72%) ~ 6.56s 6.68s p=0.519 n=6
Bind Time 1.86s (± 0.44%) 1.87s (± 0.55%) ~ 1.86s 1.89s p=0.121 n=6
Check Time 30.63s (± 0.46%) 30.52s (± 0.28%) ~ 30.36s 30.61s p=0.093 n=6
Emit Time 13.61s (± 0.30%) 13.50s (± 0.28%) -0.11s (- 0.80%) 13.45s 13.56s p=0.006 n=6
Total Time 52.75s (± 0.31%) 52.54s (± 0.25%) ~ 52.34s 52.73s p=0.065 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,718k (± 0.01%) 2,117,684k (± 0.01%) ~ 2,117,523k 2,117,926k p=0.471 n=6
Parse Time 9.71s (± 0.37%) 9.78s (± 0.30%) +0.07s (+ 0.70%) 9.76s 9.84s p=0.005 n=6
Bind Time 3.39s (± 0.52%) 3.40s (± 0.58%) ~ 3.37s 3.43s p=0.869 n=6
Check Time 101.99s (± 0.76%) 102.58s (± 0.39%) ~ 102.11s 103.14s p=0.128 n=6
Emit Time 0.19s (± 2.13%) 0.19s (± 2.13%) ~ 0.19s 0.20s p=1.000 n=6
Total Time 115.29s (± 0.66%) 115.95s (± 0.34%) ~ 115.48s 116.48s p=0.128 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,225,281 +21 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,341 261,252 -89 (- 0.03%) ~ ~ p=0.001 n=6
Memory used 2,340,322k (± 0.07%) 2,339,849k (± 0.02%) ~ 2,339,235k 2,340,683k p=0.575 n=6
Parse Time 5.02s (± 0.48%) 5.10s (± 0.71%) +0.08s (+ 1.63%) 5.06s 5.14s p=0.008 n=6
Bind Time 1.91s (± 0.51%) 1.90s (± 0.96%) ~ 1.88s 1.92s p=0.284 n=6
Check Time 34.32s (± 0.50%) 34.10s (± 0.53%) ~ 33.91s 34.40s p=0.128 n=6
Emit Time 2.69s (± 4.34%) 2.75s (± 5.71%) ~ 2.61s 3.03s p=0.873 n=6
Total Time 43.96s (± 0.56%) 43.86s (± 0.70%) ~ 43.49s 44.25s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6+0
Symbols 1,225,260 0 ~ ~ ~ p=1.000 n=6+0
Types 261,341 0 ~ ~ ~ p=1.000 n=6+0
Memory used 2,415,755k (± 0.04%) 0k ~ ~ ~ p=1.000 n=6+0
Parse Time 7.75s (± 0.48%) 0s ~ ~ ~ p=1.000 n=6+0
Bind Time 2.52s (± 0.66%) 0s ~ ~ ~ p=1.000 n=6+0
Check Time 50.20s (± 0.27%) 0s ~ ~ ~ p=1.000 n=6+0
Emit Time 3.84s (± 1.97%) 0s ~ ~ ~ p=1.000 n=6+0
Total Time 64.33s (± 0.29%) 0s ~ ~ ~ p=1.000 n=6+0
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,063 -131 (- 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,643 -94 (- 0.09%) ~ ~ p=0.001 n=6
Memory used 427,489k (± 0.02%) 427,243k (± 0.01%) -247k (- 0.06%) 427,208k 427,271k p=0.005 n=6
Parse Time 4.06s (± 0.48%) 4.13s (± 0.18%) +0.07s (+ 1.68%) 4.12s 4.14s p=0.005 n=6
Bind Time 1.63s (± 1.19%) 1.58s (± 1.18%) 🟩-0.05s (- 3.17%) 1.56s 1.61s p=0.007 n=6
Check Time 21.91s (± 0.27%) 22.06s (± 0.58%) ~ 21.86s 22.19s p=0.063 n=6
Emit Time 1.54s (± 1.57%) 1.59s (± 1.30%) +0.04s (+ 2.81%) 1.57s 1.62s p=0.019 n=6
Total Time 29.14s (± 0.22%) 29.36s (± 0.49%) +0.21s (+ 0.73%) 29.12s 29.47s p=0.045 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,485k (± 0.03%) 369,454k (± 0.04%) ~ 369,315k 369,750k p=0.378 n=6
Parse Time 2.76s (± 0.53%) 2.77s (± 1.09%) ~ 2.72s 2.80s p=0.744 n=6
Bind Time 1.59s (± 1.30%) 1.60s (± 1.33%) ~ 1.58s 1.64s p=0.462 n=6
Check Time 15.49s (± 0.14%) 15.40s (± 0.36%) -0.09s (- 0.59%) 15.32s 15.49s p=0.016 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.85s (± 0.17%) 19.77s (± 0.16%) -0.08s (- 0.40%) 19.73s 19.81s p=0.008 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,975 2,879,975 ~ ~ ~ p=1.000 n=6
Types 975,327 975,327 ~ ~ ~ p=1.000 n=6
Memory used 3,042,797k (± 0.00%) 3,042,553k (± 0.00%) -244k (- 0.01%) 3,042,503k 3,042,588k p=0.005 n=6
Parse Time 11.44s (± 0.24%) 11.45s (± 0.31%) ~ 11.40s 11.49s p=1.000 n=6
Bind Time 3.55s (± 2.54%) 3.58s (± 2.46%) ~ 3.51s 3.71s p=0.280 n=6
Check Time 65.23s (± 2.17%) 64.49s (± 2.98%) ~ 62.85s 67.09s p=0.810 n=6
Emit Time 18.56s (± 6.76%) 19.49s (± 8.42%) ~ 17.36s 20.74s p=0.298 n=6
Total Time 98.79s (± 0.23%) 99.00s (± 0.33%) ~ 98.52s 99.48s p=0.378 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,560k (± 0.02%) 411,584k (± 0.01%) ~ 411,513k 411,646k p=0.688 n=6
Parse Time 3.82s (± 0.70%) 3.76s (± 0.76%) -0.05s (- 1.40%) 3.74s 3.80s p=0.043 n=6
Bind Time 1.69s (± 0.31%) 1.70s (± 0.53%) +0.01s (+ 0.79%) 1.69s 1.71s p=0.020 n=6
Check Time 16.73s (± 0.17%) 16.76s (± 0.29%) ~ 16.70s 16.84s p=0.334 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.24s (± 0.19%) 22.22s (± 0.32%) ~ 22.14s 22.33s p=0.687 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,808k (± 0.07%) 462,804k (± 0.07%) ~ 462,368k 463,114k p=0.936 n=6
Parse Time 3.19s (± 0.47%) 3.23s (± 0.80%) +0.04s (+ 1.25%) 3.20s 3.27s p=0.018 n=6
Bind Time 1.17s 1.19s (± 0.69%) +0.02s (+ 1.42%) 1.18s 1.20s p=0.003 n=6
Check Time 17.92s (± 0.40%) 17.90s (± 0.26%) ~ 17.83s 17.95s p=0.572 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.29s (± 0.34%) 22.32s (± 0.27%) ~ 22.25s 22.38s p=0.470 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,756ms (± 1.03%) 2,734ms (± 0.82%) ~ 2,711ms 2,765ms p=0.230 n=6
Req 2 - geterr 6,756ms (±10.80%) 6,490ms (±10.45%) ~ 6,015ms 7,496ms p=0.230 n=6
Req 3 - references 345ms (±11.40%) 370ms (±10.53%) ~ 319ms 396ms p=0.625 n=6
Req 4 - navto 319ms (±10.22%) 340ms (± 0.85%) ~ 334ms 342ms p=0.209 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 131ms (± 0.84%) 118ms (± 3.39%) 🟩-13ms (- 9.92%) 116ms 126ms p=0.004 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,565ms (± 0.56%) 3,569ms (± 0.57%) ~ 3,536ms 3,586ms p=0.810 n=6
Req 2 - geterr 5,586ms (± 0.46%) 5,523ms (± 0.57%) -63ms (- 1.13%) 5,489ms 5,570ms p=0.013 n=6
Req 3 - references 414ms (± 0.63%) 417ms (± 0.50%) ~ 413ms 419ms p=0.086 n=6
Req 4 - navto 338ms (± 0.47%) 338ms ~ ~ ~ p=0.121 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 108ms (± 3.61%) 123ms (± 1.11%) 🔻+15ms (+14.20%) 121ms 125ms p=0.005 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,354ms (± 5.11%) 6,281ms (± 0.44%) ~ 6,249ms 6,316ms p=0.066 n=6
Req 2 - geterr 1,597ms (±11.27%) 1,644ms (± 8.14%) ~ 1,372ms 1,719ms p=0.810 n=6
Req 3 - references 109ms (± 8.97%) 110ms (± 0.47%) ~ 110ms 111ms p=0.437 n=6
Req 4 - navto 586ms (± 4.50%) 612ms (± 1.45%) 🔻+26ms (+ 4.50%) 597ms 621ms p=0.013 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,203ms (± 8.95%) 1,221ms (± 2.16%) ~ 1,197ms 1,270ms p=0.378 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 187.33ms (± 0.15%) 187.73ms (± 0.19%) +0.40ms (+ 0.21%) 186.23ms 192.69ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 231.40ms (± 0.13%) 231.37ms (± 0.14%) ~ 230.11ms 236.17ms p=0.267 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 276.73ms (± 0.28%) 276.87ms (± 0.28%) +0.14ms (+ 0.05%) 270.42ms 283.12ms p=0.002 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 276.93ms (± 0.30%) 277.11ms (± 0.28%) +0.19ms (+ 0.07%) 270.65ms 281.09ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Unfortunately, my work to make our package smaller by reusing the public API basically renders this PR pointless outside of src/compiler, since going through the namespaces and never direct importing is the only reason those work. Not happy about that.

#59544 has actually shown that to not be true, phew!

@jakebailey jakebailey force-pushed the debug-module branch 2 times, most recently from 1b43f97 to 0296a2c Compare August 15, 2024 23:43
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 15, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 194,194k (± 0.97%) 193,414k (± 0.94%) ~ 192,194k 195,759k p=0.066 n=6
Parse Time 1.57s (± 1.04%) 1.56s (± 0.66%) ~ 1.54s 1.57s p=0.241 n=6
Bind Time 0.85s (± 1.21%) 0.85s (± 0.96%) ~ 0.84s 0.86s p=0.270 n=6
Check Time 11.36s (± 0.11%) 11.28s (± 0.56%) -0.08s (- 0.68%) 11.18s 11.34s p=0.008 n=6
Emit Time 3.23s (± 0.65%) 3.23s (± 0.85%) ~ 3.20s 3.28s p=1.000 n=6
Total Time 17.01s (± 0.14%) 16.92s (± 0.41%) -0.09s (- 0.51%) 16.79s 16.99s p=0.012 n=6
angular-1 - node (v18.15.0, x64)
Errors 7 7 ~ ~ ~ p=1.000 n=6
Symbols 945,757 945,757 ~ ~ ~ p=1.000 n=6
Types 410,067 410,067 ~ ~ ~ p=1.000 n=6
Memory used 1,222,543k (± 0.00%) 1,222,284k (± 0.00%) -259k (- 0.02%) 1,222,208k 1,222,340k p=0.005 n=6
Parse Time 7.93s (± 0.56%) 7.98s (± 0.72%) ~ 7.91s 8.05s p=0.170 n=6
Bind Time 2.23s (± 0.34%) 2.23s (± 1.08%) ~ 2.21s 2.28s p=1.000 n=6
Check Time 36.24s (± 0.39%) 36.20s (± 0.33%) ~ 36.02s 36.37s p=0.575 n=6
Emit Time 17.85s (± 0.16%) 17.71s (± 0.42%) -0.14s (- 0.77%) 17.59s 17.79s p=0.005 n=6
Total Time 64.25s (± 0.21%) 64.12s (± 0.22%) ~ 63.96s 64.32s p=0.172 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,491,254 2,491,254 ~ ~ ~ p=1.000 n=6
Types 1,006,231 1,006,231 ~ ~ ~ p=1.000 n=6
Memory used 2,365,362k (± 0.01%) 2,365,165k (± 0.00%) -197k (- 0.01%) 2,365,066k 2,365,218k p=0.005 n=6
Parse Time 13.50s (± 0.35%) 13.50s (± 0.18%) ~ 13.48s 13.55s p=0.624 n=6
Bind Time 3.20s (± 0.31%) 3.20s (± 0.44%) ~ 3.18s 3.22s p=0.869 n=6
Check Time 108.43s (± 0.21%) 108.05s (± 0.27%) -0.38s (- 0.35%) 107.52s 108.41s p=0.020 n=6
Emit Time 0.40s (± 3.31%) 0.41s (± 1.35%) ~ 0.40s 0.41s p=0.663 n=6
Total Time 125.53s (± 0.21%) 125.16s (± 0.24%) -0.36s (- 0.29%) 124.62s 125.53s p=0.045 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,331 1,231,351 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 264,301 264,212 -89 (- 0.03%) ~ ~ p=0.001 n=6
Memory used 2,708,475k (±11.75%) 2,648,992k (±13.19%) ~ 2,352,146k 3,064,279k p=0.575 n=6
Parse Time 7.60s (± 0.71%) 7.57s (± 1.54%) ~ 7.45s 7.74s p=0.575 n=6
Bind Time 2.76s (± 1.06%) 2.77s (± 1.97%) ~ 2.69s 2.84s p=0.470 n=6
Check Time 50.42s (± 0.80%) 50.31s (± 1.19%) ~ 49.22s 51.03s p=0.689 n=6
Emit Time 4.98s (± 3.13%) 4.99s (± 1.96%) ~ 4.83s 5.12s p=0.521 n=6
Total Time 65.78s (± 0.57%) 65.66s (± 0.91%) ~ 64.60s 66.39s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6+0
Symbols 1,231,331 0 ~ ~ ~ p=1.000 n=6+0
Types 264,301 0 ~ ~ ~ p=1.000 n=6+0
Memory used 2,426,488k (± 0.02%) 0k ~ ~ ~ p=1.000 n=6+0
Parse Time 6.25s (± 0.75%) 0s ~ ~ ~ p=1.000 n=6+0
Bind Time 2.01s (± 1.16%) 0s ~ ~ ~ p=1.000 n=6+0
Check Time 41.45s (± 0.75%) 0s ~ ~ ~ p=1.000 n=6+0
Emit Time 4.09s (± 2.15%) 0s ~ ~ ~ p=1.000 n=6+0
Total Time 53.80s (± 0.56%) 0s ~ ~ ~ p=1.000 n=6+0
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,907 256,774 -133 (- 0.05%) ~ ~ p=0.001 n=6
Types 105,594 105,500 -94 (- 0.09%) ~ ~ p=0.001 n=6
Memory used 429,293k (± 0.02%) 428,913k (± 0.01%) -380k (- 0.09%) 428,875k 428,960k p=0.005 n=6
Parse Time 2.79s (± 0.49%) 2.80s (± 1.06%) ~ 2.77s 2.85s p=0.465 n=6
Bind Time 1.09s (± 1.07%) 1.06s (± 0.97%) -0.03s (- 2.60%) 1.05s 1.08s p=0.009 n=6
Check Time 15.35s (± 0.33%) 15.31s (± 0.46%) ~ 15.23s 15.38s p=0.295 n=6
Emit Time 1.39s (± 0.74%) 1.38s (± 0.37%) -0.02s (- 1.20%) 1.37s 1.38s p=0.009 n=6
Total Time 20.62s (± 0.31%) 20.55s (± 0.40%) ~ 20.46s 20.67s p=0.090 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 225,018 225,018 ~ ~ ~ p=1.000 n=6
Types 94,245 94,245 ~ ~ ~ p=1.000 n=6
Memory used 370,124k (± 0.01%) 370,008k (± 0.00%) -117k (- 0.03%) 369,985k 370,026k p=0.005 n=6
Parse Time 2.76s (± 0.48%) 2.77s (± 1.30%) ~ 2.73s 2.81s p=0.463 n=6
Bind Time 1.59s (± 1.57%) 1.58s (± 0.35%) ~ 1.58s 1.59s p=0.863 n=6
Check Time 15.72s (± 0.37%) 15.68s (± 0.29%) ~ 15.62s 15.74s p=0.173 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.07s (± 0.21%) 20.04s (± 0.12%) ~ 20.01s 20.07s p=0.127 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 3,004,379 3,004,379 ~ ~ ~ p=1.000 n=6
Types 1,034,856 1,034,856 ~ ~ ~ p=1.000 n=6
Memory used 3,128,367k (± 0.00%) 3,128,065k (± 0.00%) -302k (- 0.01%) 3,128,034k 3,128,144k p=0.005 n=6
Parse Time 13.89s (± 0.21%) 13.89s (± 0.33%) ~ 13.83s 13.95s p=0.808 n=6
Bind Time 4.28s (± 2.22%) 4.30s (± 1.81%) ~ 4.26s 4.46s p=0.292 n=6
Check Time 79.25s (± 0.49%) 79.12s (± 0.25%) ~ 78.78s 79.28s p=0.748 n=6
Emit Time 20.27s (± 0.32%) 20.28s (± 0.59%) ~ 20.19s 20.51s p=0.810 n=6
Total Time 117.69s (± 0.27%) 117.60s (± 0.23%) ~ 117.10s 117.89s p=1.000 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 275,352 275,352 ~ ~ ~ p=1.000 n=6
Types 112,436 112,436 ~ ~ ~ p=1.000 n=6
Memory used 424,193k (± 0.01%) 423,984k (± 0.01%) -209k (- 0.05%) 423,923k 424,069k p=0.005 n=6
Parse Time 3.98s (± 0.46%) 3.95s (± 0.25%) -0.03s (- 0.67%) 3.94s 3.97s p=0.013 n=6
Bind Time 1.73s (± 1.06%) 1.74s (± 0.87%) ~ 1.72s 1.76s p=0.554 n=6
Check Time 17.48s (± 0.23%) 17.44s (± 0.26%) ~ 17.40s 17.52s p=0.147 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 23.19s (± 0.21%) 23.13s (± 0.17%) ~ 23.09s 23.19s p=0.064 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 534,188 534,188 ~ ~ ~ p=1.000 n=6
Types 176,207 176,207 ~ ~ ~ p=1.000 n=6
Memory used 479,315k (± 0.01%) 479,219k (± 0.01%) -96k (- 0.02%) 479,152k 479,264k p=0.005 n=6
Parse Time 4.26s (± 0.26%) 4.24s (± 0.46%) ~ 4.22s 4.27s p=0.141 n=6
Bind Time 1.55s (± 0.67%) 1.57s (± 0.85%) +0.02s (+ 1.40%) 1.55s 1.59s p=0.017 n=6
Check Time 22.24s (± 0.23%) 22.21s (± 0.29%) ~ 22.10s 22.28s p=0.468 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 28.05s (± 0.14%) 28.02s (± 0.26%) ~ 27.89s 28.10s p=0.684 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,872ms (± 9.27%) 2,937ms (± 9.19%) ~ 2,699ms 3,421ms p=0.936 n=6
Req 2 - geterr 6,592ms (± 9.94%) 6,814ms (±10.96%) ~ 6,127ms 7,619ms p=0.810 n=6
Req 3 - references 355ms (±12.38%) 388ms (± 7.66%) 🔻+33ms (+ 9.40%) 328ms 405ms p=0.045 n=6
Req 4 - navto 331ms (± 7.27%) 338ms (± 1.00%) ~ 334ms 343ms p=0.462 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 118ms (± 6.40%) 102ms (± 6.65%) 🟩-16ms (-13.72%) 97ms 113ms p=0.019 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,100ms (±11.28%) 2,993ms (± 9.77%) ~ 2,845ms 3,589ms p=0.748 n=6
Req 2 - geterr 5,365ms (±10.27%) 5,512ms (± 8.11%) ~ 4,604ms 5,785ms p=0.689 n=6
Req 3 - references 397ms (± 7.98%) 409ms (± 0.95%) ~ 403ms 413ms p=0.520 n=6
Req 4 - navto 341ms (± 2.31%) 332ms (± 2.68%) ~ 320ms 338ms p=0.210 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 110ms (± 2.98%) 107ms (± 7.74%) ~ 90ms 111ms p=0.418 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,197ms (± 0.20%) 5,193ms (± 0.30%) ~ 5,172ms 5,216ms p=0.575 n=6
Req 2 - geterr 1,126ms (± 0.89%) 1,125ms (± 0.85%) ~ 1,112ms 1,139ms p=0.810 n=6
Req 3 - references 88ms (± 3.83%) 81ms (± 5.81%) 🟩-8ms (- 8.87%) 78ms 90ms p=0.019 n=6
Req 4 - navto 442ms (± 0.41%) 443ms (± 0.09%) ~ 442ms 443ms p=0.421 n=6
Req 5 - completionInfo count 3,444 3,444 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 830ms (± 1.18%) 826ms (± 1.59%) ~ 811ms 840ms p=0.574 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 188.56ms (± 0.19%) 187.96ms (± 0.19%) -0.60ms (- 0.32%) 186.04ms 191.11ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 282.96ms (± 0.28%) 283.39ms (± 0.28%) +0.42ms (+ 0.15%) 276.04ms 289.89ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 337.77ms (± 0.30%) 338.13ms (± 0.30%) +0.36ms (+ 0.11%) 329.93ms 343.99ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 277.91ms (± 0.29%) 278.40ms (± 0.29%) +0.49ms (+ 0.18%) 271.64ms 287.86ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Now we're talkin

@jakebailey jakebailey dismissed stale reviews from andrewbranch and sheetalkamat via 9cb279e April 1, 2025 08:27
@jakebailey jakebailey closed this Apr 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewbranch andrewbranch andrewbranch left review comments

@sheetalkamat sheetalkamat sheetalkamat left review comments

@sandersn sandersn Awaiting requested review from sandersn

@rbuckton rbuckton Awaiting requested review from rbuckton

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jakebailey @typescript-bot @a-tarasyuk @andrewbranch @rbuckton @sheetalkamat