-
Notifications
You must be signed in to change notification settings - Fork 312
Upload code coverage to Datadog #9132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upload code coverage to Datadog #9132
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 9 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (996.018 ms) : 0, 996018
Total [baseline] (10.631 s) : 0, 10631295
Agent [candidate] (994.652 ms) : 0, 994652
Total [candidate] (10.664 s) : 0, 10664289
section appsec
Agent [baseline] (1.175 s) : 0, 1174619
Total [baseline] (10.778 s) : 0, 10777978
Agent [candidate] (1.174 s) : 0, 1174233
Total [candidate] (10.756 s) : 0, 10756430
section iast
Agent [baseline] (1.14 s) : 0, 1140051
Total [baseline] (11.008 s) : 0, 11007520
Agent [candidate] (1.135 s) : 0, 1134859
Total [candidate] (10.868 s) : 0, 10867839
section profiling
Agent [baseline] (1.246 s) : 0, 1246040
Total [baseline] (10.989 s) : 0, 10988923
Agent [candidate] (1.245 s) : 0, 1244910
Total [candidate] (10.991 s) : 0, 10990915
gantt
title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.163 ms) : 0, 688163
BytebuddyAgent [candidate] (687.017 ms) : 0, 687017
GlobalTracer [baseline] (241.837 ms) : 0, 241837
GlobalTracer [candidate] (242.067 ms) : 0, 242067
AppSec [baseline] (30.276 ms) : 0, 30276
AppSec [candidate] (30.044 ms) : 0, 30044
Debugger [baseline] (5.987 ms) : 0, 5987
Debugger [candidate] (5.977 ms) : 0, 5977
Remote Config [baseline] (672.367 µs) : 0, 672
Remote Config [candidate] (673.461 µs) : 0, 673
Telemetry [baseline] (8.224 ms) : 0, 8224
Telemetry [candidate] (8.171 ms) : 0, 8171
section appsec
BytebuddyAgent [baseline] (709.495 ms) : 0, 709495
BytebuddyAgent [candidate] (710.165 ms) : 0, 710165
GlobalTracer [baseline] (234.809 ms) : 0, 234809
GlobalTracer [candidate] (235.169 ms) : 0, 235169
AppSec [baseline] (171.421 ms) : 0, 171421
AppSec [candidate] (170.263 ms) : 0, 170263
Debugger [baseline] (5.767 ms) : 0, 5767
Debugger [candidate] (5.729 ms) : 0, 5729
Remote Config [baseline] (615.4 µs) : 0, 615
Remote Config [candidate] (607.022 µs) : 0, 607
Telemetry [baseline] (8.132 ms) : 0, 8132
Telemetry [candidate] (8.129 ms) : 0, 8129
IAST [baseline] (23.546 ms) : 0, 23546
IAST [candidate] (23.297 ms) : 0, 23297
section iast
BytebuddyAgent [baseline] (812.776 ms) : 0, 812776
BytebuddyAgent [candidate] (808.739 ms) : 0, 808739
GlobalTracer [baseline] (233.553 ms) : 0, 233553
GlobalTracer [candidate] (233.123 ms) : 0, 233123
AppSec [baseline] (27.555 ms) : 0, 27555
AppSec [candidate] (31.576 ms) : 0, 31576
Debugger [baseline] (6.725 ms) : 0, 6725
Debugger [candidate] (5.777 ms) : 0, 5777
Remote Config [baseline] (589.021 µs) : 0, 589
Remote Config [candidate] (584.756 µs) : 0, 585
Telemetry [baseline] (7.996 ms) : 0, 7996
Telemetry [candidate] (8.062 ms) : 0, 8062
IAST [baseline] (30.045 ms) : 0, 30045
IAST [candidate] (26.194 ms) : 0, 26194
section profiling
ProfilingAgent [baseline] (104.15 ms) : 0, 104150
ProfilingAgent [candidate] (103.644 ms) : 0, 103644
BytebuddyAgent [baseline] (678.554 ms) : 0, 678554
BytebuddyAgent [candidate] (677.531 ms) : 0, 677531
GlobalTracer [baseline] (361.731 ms) : 0, 361731
GlobalTracer [candidate] (361.864 ms) : 0, 361864
AppSec [baseline] (33.686 ms) : 0, 33686
AppSec [candidate] (31.032 ms) : 0, 31032
Debugger [baseline] (9.16 ms) : 0, 9160
Debugger [candidate] (10.642 ms) : 0, 10642
Remote Config [baseline] (674.437 µs) : 0, 674
Remote Config [candidate] (673.458 µs) : 0, 673
Telemetry [baseline] (9.449 ms) : 0, 9449
Telemetry [candidate] (10.969 ms) : 0, 10969
Profiling [baseline] (104.174 ms) : 0, 104174
Profiling [candidate] (103.669 ms) : 0, 103669
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (994.811 ms) : 0, 994811
Total [baseline] (8.569 s) : 0, 8569279
Agent [candidate] (994.367 ms) : 0, 994367
Total [candidate] (8.545 s) : 0, 8544635
section iast
Agent [baseline] (1.131 s) : 0, 1130688
Total [baseline] (9.303 s) : 0, 9303453
Agent [candidate] (1.131 s) : 0, 1131268
Total [candidate] (9.253 s) : 0, 9253401
gantt
title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.79 ms) : 0, 686790
BytebuddyAgent [candidate] (686.773 ms) : 0, 686773
GlobalTracer [baseline] (242.069 ms) : 0, 242069
GlobalTracer [candidate] (241.762 ms) : 0, 241762
AppSec [baseline] (30.287 ms) : 0, 30287
AppSec [candidate] (30.252 ms) : 0, 30252
Debugger [baseline] (6.017 ms) : 0, 6017
Debugger [candidate] (5.956 ms) : 0, 5956
Remote Config [baseline] (682.56 µs) : 0, 683
Remote Config [candidate] (675.581 µs) : 0, 676
Telemetry [baseline] (8.235 ms) : 0, 8235
Telemetry [candidate] (8.211 ms) : 0, 8211
section iast
BytebuddyAgent [baseline] (806.145 ms) : 0, 806145
BytebuddyAgent [candidate] (806.627 ms) : 0, 806627
GlobalTracer [baseline] (231.853 ms) : 0, 231853
GlobalTracer [candidate] (232.111 ms) : 0, 232111
IAST [baseline] (26.472 ms) : 0, 26472
IAST [candidate] (26.238 ms) : 0, 26238
AppSec [baseline] (30.485 ms) : 0, 30485
AppSec [candidate] (31.435 ms) : 0, 31435
Debugger [baseline] (6.589 ms) : 0, 6589
Debugger [candidate] (5.729 ms) : 0, 5729
Remote Config [baseline] (576.818 µs) : 0, 577
Remote Config [candidate] (582.83 µs) : 0, 583
Telemetry [baseline] (7.892 ms) : 0, 7892
Telemetry [candidate] (7.919 ms) : 0, 7919
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section baseline
no_agent (37.248 ms) : 36941, 37555
. : milestone, 37248,
appsec (49.259 ms) : 48812, 49706
. : milestone, 49259,
code_origins (44.627 ms) : 44238, 45016
. : milestone, 44627,
iast (44.863 ms) : 44475, 45251
. : milestone, 44863,
profiling (48.183 ms) : 47750, 48616
. : milestone, 48183,
tracing (42.559 ms) : 42205, 42912
. : milestone, 42559,
section candidate
no_agent (36.802 ms) : 36504, 37100
. : milestone, 36802,
appsec (46.35 ms) : 45943, 46758
. : milestone, 46350,
code_origins (45.191 ms) : 44813, 45570
. : milestone, 45191,
iast (43.775 ms) : 43401, 44149
. : milestone, 43775,
profiling (49.771 ms) : 49301, 50241
. : milestone, 49771,
tracing (45.306 ms) : 44919, 45693
. : milestone, 45306,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section baseline
no_agent (4.2 ms) : 4150, 4249
. : milestone, 4200,
iast (9.316 ms) : 9159, 9472
. : milestone, 9316,
iast_FULL (13.824 ms) : 13549, 14099
. : milestone, 13824,
iast_GLOBAL (10.287 ms) : 10084, 10490
. : milestone, 10287,
profiling (8.626 ms) : 8476, 8776
. : milestone, 8626,
tracing (7.612 ms) : 7504, 7719
. : milestone, 7612,
section candidate
no_agent (4.313 ms) : 4264, 4362
. : milestone, 4313,
iast (9.52 ms) : 9358, 9681
. : milestone, 9520,
iast_FULL (13.584 ms) : 13311, 13857
. : milestone, 13584,
iast_GLOBAL (10.372 ms) : 10191, 10553
. : milestone, 10372,
profiling (8.56 ms) : 8426, 8693
. : milestone, 8560,
tracing (7.495 ms) : 7382, 7608
. : milestone, 7495,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section baseline
no_agent (14.966 s) : 14966000, 14966000
. : milestone, 14966000,
appsec (14.902 s) : 14902000, 14902000
. : milestone, 14902000,
iast (18.691 s) : 18691000, 18691000
. : milestone, 18691000,
iast_GLOBAL (18.214 s) : 18214000, 18214000
. : milestone, 18214000,
profiling (14.931 s) : 14931000, 14931000
. : milestone, 14931000,
tracing (14.761 s) : 14761000, 14761000
. : milestone, 14761000,
section candidate
no_agent (14.848 s) : 14848000, 14848000
. : milestone, 14848000,
appsec (14.862 s) : 14862000, 14862000
. : milestone, 14862000,
iast (18.441 s) : 18441000, 18441000
. : milestone, 18441000,
iast_GLOBAL (18.233 s) : 18233000, 18233000
. : milestone, 18233000,
profiling (14.833 s) : 14833000, 14833000
. : milestone, 14833000,
tracing (14.807 s) : 14807000, 14807000
. : milestone, 14807000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~b4c50a43a1, baseline=1.52.0-SNAPSHOT~be7f9d370d
dateFormat X
axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
. : milestone, 1478,
appsec (2.436 ms) : 2385, 2486
. : milestone, 2436,
iast (2.218 ms) : 2155, 2281
. : milestone, 2218,
iast_GLOBAL (2.255 ms) : 2191, 2318
. : milestone, 2255,
profiling (2.044 ms) : 1994, 2094
. : milestone, 2044,
tracing (2.024 ms) : 1975, 2072
. : milestone, 2024,
section candidate
no_agent (1.484 ms) : 1473, 1496
. : milestone, 1484,
appsec (2.429 ms) : 2379, 2479
. : milestone, 2429,
iast (2.207 ms) : 2145, 2270
. : milestone, 2207,
iast_GLOBAL (2.253 ms) : 2191, 2316
. : milestone, 2253,
profiling (2.513 ms) : 2338, 2688
. : milestone, 2513,
tracing (2.025 ms) : 1977, 2074
. : milestone, 2025,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to test it or let's just give it a try?
# Upload test results to production environment like all other CI jobs | ||
junit_upload "$DATADOG_API_KEY_PROD" | ||
|
||
coverage_upload "$DATADOG_API_KEY_PROD" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the script return the result of both upload functions now? Not only the coverage one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Won't it just fail if any of the uploads fails?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, nevermind, I realised the default is to just continue executing if a command fails. Updated the script to consider both statuses
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See my comment here: https://github.com/DataDog/dd-trace-java/pull/9132/files#r2197581837
It was tested by the CI jobs of this PR :) |
# Upload test results to production environment like all other CI jobs | ||
junit_upload "$DATADOG_API_KEY_PROD" | ||
|
||
coverage_upload "$DATADOG_API_KEY_PROD" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See my comment here: https://github.com/DataDog/dd-trace-java/pull/9132/files#r2197581837
What Does This Do
Configures uploading code coverage data to Datadog.
Motivation
Dogfooding new Code Coverage product.
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]