Skip to content

Introduce id-based line probe source matching #8275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 24, 2025
Merged

Introduce id-based line probe source matching #8275

merged 1 commit into from
Jan 24, 2025

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Jan 23, 2025 •
edited by atlassian bot
Loading

What Does This Do

line probe in unit tests are created by searching probe id (UUID) inside the targeted source file.
uuid are added as comment on the line we want to put a line probe. Add helper method to search a uuid inside the targeted source file and returns the line number used to create the line probe

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-3270

@jpbempel
Introduce id-based line probe source matching
61dcacf 
line probe in unit tests are created by searching probe id (UUID)
inside the targeted source file.
uuid are added as comment on the line we want to put a line probe.
Add helper method to search a uuid inside the targeted source file
and returns the line number used to create the line probe
@jpbempel jpbempel added tag: no release notes Changes to exclude from release notes type: refactoring comp: debugger Dynamic Instrumentation labels Jan 23, 2025
@jpbempel jpbempel requested a review from a team as a code owner January 23, 2025 17:36
@jpbempel jpbempel requested review from shatzi and evanchooly and removed request for a team January 23, 2025 17:36
@pr-commenter
Copy link

pr-commenter bot commented Jan 23, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737654303 1737654681
end_time 2025-01-23T17:46:22 2025-01-23T17:52:38
git_branch master jpbempel/id-based-line-probe-source-matching
git_commit_sha afbea0b 61dcacf
start_time 2025-01-23T17:45:04 2025-01-23T17:51:22
See matching parameters
Baseline Candidate
ci_job_id 778248489 778248489
ci_pipeline_id 53830569 53830569
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737653722 1737653722

Summary

Found 6 performance improvements and 0 performance regressions! Performance is the same for 2 metrics, 7 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic better
[-28.578µs; -18.106µs] or [-9.673%; -6.129%]		 better
[-35.032µs; -15.048µs] or [-10.699%; -4.596%]		 better
[-37.760µs; -14.262µs] or [-11.178%; -4.222%]		 unstable
[-89.845µs; +8.180µs] or [-16.242%; +1.479%]		 unstable
[+14.105op/s; +352.195op/s] or [+0.592%; +14.792%]
scenario:loop better
[-293.013µs; -203.139µs] or [-2.862%; -1.984%]		 better
[-305.329µs; -192.728µs] or [-2.936%; -1.853%]		 better
[-328.228µs; -188.398µs] or [-3.124%; -1.793%]		 same unsure
[+0.873op/s; +3.728op/s] or [+0.921%; +3.933%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-56.657µs; +19.196µs] or [-19.837%; +6.721%]		 unstable
[-74.232µs; +22.407µs] or [-22.490%; +6.789%]		 unstable
[-87.745µs; +27.238µs] or [-25.231%; +7.832%]		 unstable
[-836.754µs; +285.295µs] or [-95.239%; +32.472%]		 unstable
[-63.218op/s; +194.732op/s] or [-2.497%; +7.692%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (330.064 µs) : 269, 391
.   : milestone, 330,
basic (327.436 µs) : 316, 338
.   : milestone, 327,
loop (10.401 ms) : 10337, 10465
.   : milestone, 10401,
section candidate
noprobe (304.152 µs) : 281, 327
.   : milestone, 304,
basic (302.397 µs) : 294, 311
.   : milestone, 302,
loop (10.152 ms) : 10116, 10188
.   : milestone, 10152,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 330.064 µs [269.068 µs, 391.061 µs]
basic 327.436 µs [316.458 µs, 338.415 µs]
loop 10.401 ms [10.337 ms, 10.465 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 304.152 µs [281.409 µs, 326.894 µs]
basic 302.397 µs [294.051 µs, 310.743 µs]
loop 10.152 ms [10.116 ms, 10.188 ms]

@pr-commenter
Copy link

pr-commenter bot commented Jan 23, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/id-based-line-probe-source-matching
git_commit_date 1737653302 1737653722
git_commit_sha afbea0b 61dcacf
release_version 1.46.0-SNAPSHOT~afbea0b706 1.46.0-SNAPSHOT~61dcacfca6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737656049 1737656049
ci_job_id 778248483 778248483
ci_pipeline_id 53830569 53830569
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-8wlcx2s5-project-304-concurrent-0-833j6cun 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-8wlcx2s5-project-304-concurrent-0-833j6cun 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 4 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config better
[-65.105µs; -16.095µs] or [-9.697%; -2.397%]		 630.801µs 671.401µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051549
Total [baseline] (10.525 s) : 0, 10525347
Agent [candidate] (1.04 s) : 0, 1039665
Total [candidate] (10.51 s) : 0, 10510366
section appsec
Agent [baseline] (1.183 s) : 0, 1183305
Total [baseline] (10.799 s) : 0, 10798814
Agent [candidate] (1.184 s) : 0, 1183751
Total [candidate] (10.756 s) : 0, 10755798
section iast
Agent [baseline] (1.166 s) : 0, 1166136
Total [baseline] (10.92 s) : 0, 10919995
Agent [candidate] (1.166 s) : 0, 1165763
Total [candidate] (10.991 s) : 0, 10990590
section profiling
Agent [baseline] (1.259 s) : 0, 1259175
Total [baseline] (10.95 s) : 0, 10949560
Agent [candidate] (1.253 s) : 0, 1252517
Total [candidate] (10.776 s) : 0, 10775890
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent appsec 1.183 s 131.756 ms (12.5%)
Agent iast 1.166 s 114.588 ms (10.9%)
Agent profiling 1.259 s 207.627 ms (19.7%)
Total tracing 10.525 s -
Total appsec 10.799 s 273.466 ms (2.6%)
Total iast 10.92 s 394.648 ms (3.7%)
Total profiling 10.95 s 424.213 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent appsec 1.184 s 144.086 ms (13.9%)
Agent iast 1.166 s 126.097 ms (12.1%)
Agent profiling 1.253 s 212.852 ms (20.5%)
Total tracing 10.51 s -
Total appsec 10.756 s 245.433 ms (2.3%)
Total iast 10.991 s 480.225 ms (4.6%)
Total profiling 10.776 s 265.525 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (724.402 ms) : 0, 724402
BytebuddyAgent [candidate] (715.278 ms) : 0, 715278
GlobalTracer [baseline] (241.199 ms) : 0, 241199
GlobalTracer [candidate] (239.737 ms) : 0, 239737
AppSec [baseline] (55.487 ms) : 0, 55487
AppSec [candidate] (56.604 ms) : 0, 56604
Remote Config [baseline] (712.598 µs) : 0, 713
Remote Config [candidate] (717.639 µs) : 0, 718
Telemetry [baseline] (14.526 ms) : 0, 14526
Telemetry [candidate] (12.272 ms) : 0, 12272
section appsec
BytebuddyAgent [baseline] (735.16 ms) : 0, 735160
BytebuddyAgent [candidate] (737.098 ms) : 0, 737098
GlobalTracer [baseline] (237.707 ms) : 0, 237707
GlobalTracer [candidate] (236.61 ms) : 0, 236610
AppSec [baseline] (176.407 ms) : 0, 176407
AppSec [candidate] (176.026 ms) : 0, 176026
Remote Config [baseline] (665.841 µs) : 0, 666
Remote Config [candidate] (666.388 µs) : 0, 666
Telemetry [baseline] (8.196 ms) : 0, 8196
Telemetry [candidate] (8.552 ms) : 0, 8552
IAST [baseline] (20.197 ms) : 0, 20197
IAST [candidate] (19.433 ms) : 0, 19433
section iast
BytebuddyAgent [baseline] (832.606 ms) : 0, 832606
BytebuddyAgent [candidate] (832.17 ms) : 0, 832170
GlobalTracer [baseline] (229.625 ms) : 0, 229625
GlobalTracer [candidate] (229.326 ms) : 0, 229326
AppSec [baseline] (58.235 ms) : 0, 58235
AppSec [candidate] (58.533 ms) : 0, 58533
Remote Config [baseline] (643.447 µs) : 0, 643
Remote Config [candidate] (644.219 µs) : 0, 644
Telemetry [baseline] (8.691 ms) : 0, 8691
Telemetry [candidate] (8.8 ms) : 0, 8800
IAST [baseline] (21.275 ms) : 0, 21275
IAST [candidate] (21.274 ms) : 0, 21274
section profiling
BytebuddyAgent [baseline] (707.788 ms) : 0, 707788
BytebuddyAgent [candidate] (703.412 ms) : 0, 703412
GlobalTracer [baseline] (349.289 ms) : 0, 349289
GlobalTracer [candidate] (348.253 ms) : 0, 348253
AppSec [baseline] (55.119 ms) : 0, 55119
AppSec [candidate] (54.029 ms) : 0, 54029
Remote Config [baseline] (700.735 µs) : 0, 701
Remote Config [candidate] (709.701 µs) : 0, 710
Telemetry [baseline] (8.724 ms) : 0, 8724
Telemetry [candidate] (8.817 ms) : 0, 8817
ProfilingAgent [baseline] (95.329 ms) : 0, 95329
ProfilingAgent [candidate] (95.353 ms) : 0, 95353
Profiling [baseline] (95.356 ms) : 0, 95356
Profiling [candidate] (95.379 ms) : 0, 95379
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.038 s) : 0, 1038114
Total [baseline] (8.644 s) : 0, 8644036
Agent [candidate] (1.054 s) : 0, 1053990
Total [candidate] (8.678 s) : 0, 8678260
section iast
Agent [baseline] (1.165 s) : 0, 1165402
Total [baseline] (9.207 s) : 0, 9206527
Agent [candidate] (1.166 s) : 0, 1165638
Total [candidate] (9.201 s) : 0, 9201478
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.166 s) : 0, 1166286
Total [baseline] (9.175 s) : 0, 9174935
Agent [candidate] (1.174 s) : 0, 1173843
Total [candidate] (9.185 s) : 0, 9184722
section iast_TELEMETRY_OFF
Agent [baseline] (1.163 s) : 0, 1163321
Total [baseline] (9.24 s) : 0, 9239563
Agent [candidate] (1.163 s) : 0, 1162698
Total [candidate] (9.23 s) : 0, 9229650
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent iast 1.165 s 127.288 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.166 s 128.172 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.163 s 125.207 ms (12.1%)
Total tracing 8.644 s -
Total iast 9.207 s 562.491 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.175 s 530.9 ms (6.1%)
Total iast_TELEMETRY_OFF 9.24 s 595.527 ms (6.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.166 s 111.648 ms (10.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.174 s 119.853 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.163 s 108.708 ms (10.3%)
Total tracing 8.678 s -
Total iast 9.201 s 523.218 ms (6.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.185 s 506.462 ms (5.8%)
Total iast_TELEMETRY_OFF 9.23 s 551.39 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.11 ms) : 0, 714110
BytebuddyAgent [candidate] (725.681 ms) : 0, 725681
GlobalTracer [baseline] (239.111 ms) : 0, 239111
GlobalTracer [candidate] (242.452 ms) : 0, 242452
AppSec [baseline] (56.229 ms) : 0, 56229
AppSec [candidate] (56.086 ms) : 0, 56086
Remote Config [baseline] (712.569 µs) : 0, 713
Remote Config [candidate] (726.409 µs) : 0, 726
Telemetry [baseline] (12.976 ms) : 0, 12976
Telemetry [candidate] (13.744 ms) : 0, 13744
section iast
BytebuddyAgent [baseline] (832.275 ms) : 0, 832275
BytebuddyAgent [candidate] (832.106 ms) : 0, 832106
GlobalTracer [baseline] (229.635 ms) : 0, 229635
GlobalTracer [candidate] (229.651 ms) : 0, 229651
AppSec [baseline] (57.878 ms) : 0, 57878
AppSec [candidate] (58.062 ms) : 0, 58062
IAST [baseline] (21.273 ms) : 0, 21273
IAST [candidate] (21.294 ms) : 0, 21294
Remote Config [baseline] (653.806 µs) : 0, 654
Remote Config [candidate] (654.761 µs) : 0, 655
Telemetry [baseline] (8.7 ms) : 0, 8700
Telemetry [candidate] (8.716 ms) : 0, 8716
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.078 ms) : 0, 833078
BytebuddyAgent [candidate] (838.505 ms) : 0, 838505
GlobalTracer [baseline] (229.877 ms) : 0, 229877
GlobalTracer [candidate] (230.88 ms) : 0, 230880
AppSec [baseline] (57.923 ms) : 0, 57923
AppSec [candidate] (58.502 ms) : 0, 58502
IAST [baseline] (21.138 ms) : 0, 21138
IAST [candidate] (21.448 ms) : 0, 21448
Remote Config [baseline] (629.608 µs) : 0, 630
Remote Config [candidate] (642.86 µs) : 0, 643
Telemetry [baseline] (8.653 ms) : 0, 8653
Telemetry [candidate] (8.718 ms) : 0, 8718
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.904 ms) : 0, 830904
BytebuddyAgent [candidate] (829.958 ms) : 0, 829958
GlobalTracer [baseline] (229.471 ms) : 0, 229471
GlobalTracer [candidate] (229.285 ms) : 0, 229285
AppSec [baseline] (57.868 ms) : 0, 57868
AppSec [candidate] (58.529 ms) : 0, 58529
IAST [baseline] (20.848 ms) : 0, 20848
IAST [candidate] (20.653 ms) : 0, 20653
Remote Config [baseline] (671.401 µs) : 0, 671
Remote Config [candidate] (630.801 µs) : 0, 631
Telemetry [baseline] (8.569 ms) : 0, 8569
Telemetry [candidate] (8.578 ms) : 0, 8578
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-23T17:46:20 2025-01-23T17:53:24
git_branch master jpbempel/id-based-line-probe-source-matching
git_commit_date 1737653302 1737653722
git_commit_sha afbea0b 61dcacf
release_version 1.46.0-SNAPSHOT~afbea0b706 1.46.0-SNAPSHOT~61dcacfca6
start_time 2025-01-23T17:46:06 2025-01-23T17:53:10
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737655161 1737655161
ci_job_id 778248484 778248484
ci_pipeline_id 53830569 53830569
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-8wlcx2s5-project-304-concurrent-1-yg4g7hff 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-8wlcx2s5-project-304-concurrent-1-yg4g7hff 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706
    dateFormat X
    axisFormat %s
section baseline
no_agent (373.94 µs) : 353, 394
.   : milestone, 374,
iast (508.376 µs) : 485, 532
.   : milestone, 508,
iast_FULL (737.81 µs) : 715, 761
.   : milestone, 738,
iast_GLOBAL (556.194 µs) : 532, 580
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (501.044 µs) : 478, 524
.   : milestone, 501,
iast_INACTIVE (459.506 µs) : 438, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (491.518 µs) : 470, 513
.   : milestone, 492,
tracing (452.613 µs) : 432, 474
.   : milestone, 453,
section candidate
no_agent (380.274 µs) : 359, 401
.   : milestone, 380,
iast (506.571 µs) : 483, 530
.   : milestone, 507,
iast_FULL (740.772 µs) : 717, 764
.   : milestone, 741,
iast_GLOBAL (554.184 µs) : 530, 578
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (508.787 µs) : 486, 532
.   : milestone, 509,
iast_INACTIVE (452.632 µs) : 431, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (497.768 µs) : 474, 521
.   : milestone, 498,
tracing (448.476 µs) : 427, 469
.   : milestone, 448,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 373.94 µs [353.385 µs, 394.494 µs] -
iast 508.376 µs [484.767 µs, 531.986 µs] 134.437 µs (36.0%)
iast_FULL 737.81 µs [714.952 µs, 760.668 µs] 363.87 µs (97.3%)
iast_GLOBAL 556.194 µs [532.041 µs, 580.347 µs] 182.255 µs (48.7%)
iast_HARDCODED_SECRET_DISABLED 501.044 µs [478.139 µs, 523.949 µs] 127.104 µs (34.0%)
iast_INACTIVE 459.506 µs [437.846 µs, 481.166 µs] 85.567 µs (22.9%)
iast_TELEMETRY_OFF 491.518 µs [469.745 µs, 513.29 µs] 117.578 µs (31.4%)
tracing 452.613 µs [431.705 µs, 473.52 µs] 78.673 µs (21.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.274 µs [359.463 µs, 401.084 µs] -
iast 506.571 µs [483.11 µs, 530.032 µs] 126.297 µs (33.2%)
iast_FULL 740.772 µs [717.478 µs, 764.065 µs] 360.498 µs (94.8%)
iast_GLOBAL 554.184 µs [530.12 µs, 578.248 µs] 173.91 µs (45.7%)
iast_HARDCODED_SECRET_DISABLED 508.787 µs [485.685 µs, 531.888 µs] 128.513 µs (33.8%)
iast_INACTIVE 452.632 µs [431.085 µs, 474.179 µs] 72.358 µs (19.0%)
iast_TELEMETRY_OFF 497.768 µs [474.462 µs, 521.073 µs] 117.494 µs (30.9%)
tracing 448.476 µs [427.456 µs, 469.496 µs] 68.202 µs (17.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1339, 1381
.   : milestone, 1360,
appsec (1.758 ms) : 1735, 1782
.   : milestone, 1758,
appsec_no_iast (1.76 ms) : 1735, 1784
.   : milestone, 1760,
iast (1.499 ms) : 1475, 1523
.   : milestone, 1499,
profiling (1.541 ms) : 1516, 1566
.   : milestone, 1541,
tracing (1.497 ms) : 1473, 1521
.   : milestone, 1497,
section candidate
no_agent (1.367 ms) : 1347, 1387
.   : milestone, 1367,
appsec (1.736 ms) : 1712, 1759
.   : milestone, 1736,
appsec_no_iast (1.747 ms) : 1723, 1771
.   : milestone, 1747,
iast (1.501 ms) : 1476, 1525
.   : milestone, 1501,
profiling (1.548 ms) : 1523, 1573
.   : milestone, 1548,
tracing (1.482 ms) : 1457, 1508
.   : milestone, 1482,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.36 ms [1.339 ms, 1.381 ms] -
appsec 1.758 ms [1.735 ms, 1.782 ms] 398.267 µs (29.3%)
appsec_no_iast 1.76 ms [1.735 ms, 1.784 ms] 399.618 µs (29.4%)
iast 1.499 ms [1.475 ms, 1.523 ms] 139.293 µs (10.2%)
profiling 1.541 ms [1.516 ms, 1.566 ms] 181.034 µs (13.3%)
tracing 1.497 ms [1.473 ms, 1.521 ms] 137.08 µs (10.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.367 ms [1.347 ms, 1.387 ms] -
appsec 1.736 ms [1.712 ms, 1.759 ms] 368.97 µs (27.0%)
appsec_no_iast 1.747 ms [1.723 ms, 1.771 ms] 380.177 µs (27.8%)
iast 1.501 ms [1.476 ms, 1.525 ms] 133.682 µs (9.8%)
profiling 1.548 ms [1.523 ms, 1.573 ms] 181.09 µs (13.2%)
tracing 1.482 ms [1.457 ms, 1.508 ms] 115.273 µs (8.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/id-based-line-probe-source-matching
git_commit_date 1737653302 1737653722
git_commit_sha afbea0b 61dcacf
release_version 1.46.0-SNAPSHOT~afbea0b706 1.46.0-SNAPSHOT~61dcacfca6
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737655687 1737655687
ci_job_id 778248485 778248485
ci_pipeline_id 53830569 53830569
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fgow3nxt-project-304-concurrent-0-bjl99709 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fgow3nxt-project-304-concurrent-0-bjl99709 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.349 ms) : 2306, 2391
.   : milestone, 2349,
iast (2.096 ms) : 2041, 2150
.   : milestone, 2096,
iast_GLOBAL (2.142 ms) : 2087, 2197
.   : milestone, 2142,
profiling (1.978 ms) : 1933, 2023
.   : milestone, 1978,
tracing (1.937 ms) : 1895, 1979
.   : milestone, 1937,
section candidate
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.355 ms) : 2312, 2399
.   : milestone, 2355,
iast (2.102 ms) : 2048, 2157
.   : milestone, 2102,
iast_GLOBAL (2.147 ms) : 2092, 2202
.   : milestone, 2147,
profiling (1.957 ms) : 1914, 2001
.   : milestone, 1957,
tracing (1.943 ms) : 1901, 1985
.   : milestone, 1943,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.453 ms, 1.476 ms] -
appsec 2.349 ms [2.306 ms, 2.391 ms] 883.958 µs (60.4%)
iast 2.096 ms [2.041 ms, 2.15 ms] 630.931 µs (43.1%)
iast_GLOBAL 2.142 ms [2.087 ms, 2.197 ms] 677.333 µs (46.2%)
profiling 1.978 ms [1.933 ms, 2.023 ms] 513.25 µs (35.0%)
tracing 1.937 ms [1.895 ms, 1.979 ms] 472.356 µs (32.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.355 ms [2.312 ms, 2.399 ms] 882.194 µs (59.9%)
iast 2.102 ms [2.048 ms, 2.157 ms] 629.134 µs (42.7%)
iast_GLOBAL 2.147 ms [2.092 ms, 2.202 ms] 673.989 µs (45.7%)
profiling 1.957 ms [1.914 ms, 2.001 ms] 483.76 µs (32.8%)
tracing 1.943 ms [1.901 ms, 1.985 ms] 469.376 µs (31.9%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~61dcacfca6, baseline=1.46.0-SNAPSHOT~afbea0b706
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.684 s) : 15684000, 15684000
.   : milestone, 15684000,
appsec (14.912 s) : 14912000, 14912000
.   : milestone, 14912000,
iast (19.16 s) : 19160000, 19160000
.   : milestone, 19160000,
iast_GLOBAL (18.053 s) : 18053000, 18053000
.   : milestone, 18053000,
profiling (14.907 s) : 14907000, 14907000
.   : milestone, 14907000,
tracing (14.802 s) : 14802000, 14802000
.   : milestone, 14802000,
section candidate
no_agent (15.52 s) : 15520000, 15520000
.   : milestone, 15520000,
appsec (15.171 s) : 15171000, 15171000
.   : milestone, 15171000,
iast (18.499 s) : 18499000, 18499000
.   : milestone, 18499000,
iast_GLOBAL (18.138 s) : 18138000, 18138000
.   : milestone, 18138000,
profiling (14.974 s) : 14974000, 14974000
.   : milestone, 14974000,
tracing (14.668 s) : 14668000, 14668000
.   : milestone, 14668000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.684 s [15.684 s, 15.684 s] -
appsec 14.912 s [14.912 s, 14.912 s] -772.0 ms (-4.9%)
iast 19.16 s [19.16 s, 19.16 s] 3.476 s (22.2%)
iast_GLOBAL 18.053 s [18.053 s, 18.053 s] 2.369 s (15.1%)
profiling 14.907 s [14.907 s, 14.907 s] -777.0 ms (-5.0%)
tracing 14.802 s [14.802 s, 14.802 s] -882.0 ms (-5.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.52 s [15.52 s, 15.52 s] -
appsec 15.171 s [15.171 s, 15.171 s] -349.0 ms (-2.2%)
iast 18.499 s [18.499 s, 18.499 s] 2.979 s (19.2%)
iast_GLOBAL 18.138 s [18.138 s, 18.138 s] 2.618 s (16.9%)
profiling 14.974 s [14.974 s, 14.974 s] -546.0 ms (-3.5%)
tracing 14.668 s [14.668 s, 14.668 s] -852.0 ms (-5.5%)

@jpbempel jpbempel merged commit 710964d into master Jan 24, 2025
178 checks passed
@jpbempel jpbempel deleted the jpbempel/id-based-line-probe-source-matching branch January 24, 2025 18:04
@github-actions github-actions bot added this to the 1.46.0 milestone Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@evanchooly evanchooly evanchooly approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: refactoring
Projects
None yet
Milestone
1.46.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @evanchooly