Skip to content

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented May 10, 2025

Description

We used to keep each primary key index as an independent hindex file, along with its ovf file if the key is of STRING type.
This PR unifies both kinds of files with the main data.kz file.

Will add the free space management capability for hash index when dropping the table in a later PR.

@ray6080 ray6080 force-pushed the index branch 3 times, most recently from 7efbb54 to 234b8dc Compare May 13, 2025 19:35
@ray6080 ray6080 marked this pull request as ready for review May 13, 2025 20:43
@ray6080 ray6080 requested a review from benjaminwinger as a code owner May 13, 2025 20:43
Copy link

github-actions bot commented May 14, 2025

Benchmark Result

Master commit hash: f7ac905402a7f41461c4dbc75e6e8c9191edda77
Branch commit hash: 75dcd7570e629278744515248fe41e21f6497ce5

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 665.36 667.72 -2.36 (-0.35%)
aggregation q28 6507.90 6441.33 66.57 (1.03%)
copy node-Comment 128455.39 N/A N/A
copy node-Forum 5986.73 N/A N/A
copy node-Organisation 1299.15 N/A N/A
copy node-Person 2184.57 N/A N/A
copy node-Place 1215.47 N/A N/A
copy node-Post 45611.26 N/A N/A
copy node-Tag 1347.85 N/A N/A
copy node-Tagclass 1231.58 N/A N/A
copy rel-comment-hasCreator 68107.51 N/A N/A
copy rel-comment-hasTag 94581.73 N/A N/A
copy rel-comment-isLocatedIn 66642.69 N/A N/A
copy rel-containerOf 18542.26 N/A N/A
copy rel-forum-hasTag 3575.50 N/A N/A
copy rel-hasInterest 2750.09 N/A N/A
copy rel-hasMember 56927.51 N/A N/A
copy rel-hasModerator 1314.32 N/A N/A
copy rel-hasType 318.54 N/A N/A
copy rel-isPartOf 284.10 N/A N/A
copy rel-isSubclassOf 348.10 N/A N/A
copy rel-knows 5931.45 N/A N/A
copy rel-likes-comment 106501.50 N/A N/A
copy rel-likes-post 32469.65 N/A N/A
copy rel-organisation-isLocatedIn 299.51 N/A N/A
copy rel-person-isLocatedIn 530.99 N/A N/A
copy rel-post-hasCreator 15579.01 N/A N/A
copy rel-post-hasTag 21657.02 N/A N/A
copy rel-post-isLocatedIn 17828.48 N/A N/A
copy rel-replyOf-comment 54600.75 N/A N/A
copy rel-replyOf-post 44093.73 N/A N/A
copy rel-studyAt 561.28 N/A N/A
copy rel-workAt 777.12 N/A N/A
filter q14 60.40 59.60 0.80 (1.35%)
filter q15 59.20 58.70 0.50 (0.86%)
filter q16 273.77 275.92 -2.15 (-0.78%)
filter q17 380.42 379.43 0.99 (0.26%)
filter q18 1894.69 1843.59 51.10 (2.77%)
filter zonemap-node 24.35 23.83 0.52 (2.20%)
filter zonemap-node-lhs-cast 24.52 24.04 0.47 (1.97%)
filter zonemap-node-null 24.01 23.87 0.14 (0.58%)
filter zonemap-rel 5623.86 5653.77 -29.91 (-0.53%)
fixed_size_expr_evaluator q07 623.62 617.15 6.47 (1.05%)
fixed_size_expr_evaluator q08 908.82 904.16 4.65 (0.51%)
fixed_size_expr_evaluator q09 908.20 905.39 2.81 (0.31%)
fixed_size_expr_evaluator q10 195.97 190.07 5.90 (3.10%)
fixed_size_expr_evaluator q11 197.03 189.70 7.34 (3.87%)
fixed_size_expr_evaluator q12 175.75 168.81 6.94 (4.11%)
fixed_size_expr_evaluator q13 1535.26 1490.37 44.89 (3.01%)
fixed_size_seq_scan q23 50.41 49.07 1.34 (2.73%)
join q29 711.40 760.23 -48.83 (-6.42%)
join q30 1696.73 1642.40 54.33 (3.31%)
join q31 6.67 4.62 2.05 (44.41%)
join SelectiveTwoHopJoin 40.74 39.81 0.93 (2.33%)
ldbc_snb_ic q35 10.49 10.30 0.19 (1.86%)
ldbc_snb_ic q36 97.49 92.77 4.72 (5.09%)
ldbc_snb_is q32 6.07 5.04 1.03 (20.48%)
ldbc_snb_is q33 14.07 13.35 0.72 (5.39%)
ldbc_snb_is q34 1.23 1.29 -0.06 (-4.75%)
limit push-down-limit-into-distinct 1977.30 2120.46 -143.15 (-6.75%)
multi-rel multi-rel-large-scan 2381.14 1784.57 596.57 (33.43%)
multi-rel multi-rel-lookup 11.41 10.73 0.68 (6.33%)
multi-rel multi-rel-small-scan 209.13 202.97 6.16 (3.04%)
order_by q25 64.82 61.62 3.20 (5.20%)
order_by q26 395.07 376.95 18.12 (4.81%)
order_by q27 1322.29 1306.49 15.81 (1.21%)
recursive_join recursive-join-bidirection 323.80 360.46 -36.66 (-10.17%)
recursive_join recursive-join-dense 6026.26 6960.85 -934.59 (-13.43%)
recursive_join recursive-join-path 23074.13 23321.02 -246.89 (-1.06%)
recursive_join recursive-join-sparse 10.18 9.70 0.48 (4.92%)
recursive_join recursive-join-trail 6362.67 6998.92 -636.25 (-9.09%)
scan_after_filter q01 103.55 100.68 2.87 (2.85%)
scan_after_filter q02 89.65 89.46 0.19 (0.21%)
shortest_path_ldbc100 q37 73.72 75.33 -1.61 (-2.14%)
shortest_path_ldbc100 q38 372.50 303.54 68.96 (22.72%)
shortest_path_ldbc100 q39 90.41 90.72 -0.30 (-0.33%)
shortest_path_ldbc100 q40 532.88 493.61 39.28 (7.96%)
var_size_expr_evaluator q03 2062.09 2042.22 19.87 (0.97%)
var_size_expr_evaluator q04 2189.89 2158.80 31.09 (1.44%)
var_size_expr_evaluator q05 2592.88 2595.78 -2.90 (-0.11%)
var_size_expr_evaluator q06 1279.73 1298.35 -18.62 (-1.43%)
var_size_seq_scan q19 1373.45 1357.17 16.28 (1.20%)
var_size_seq_scan q20 2590.38 2611.70 -21.33 (-0.82%)
var_size_seq_scan q21 2200.17 2199.81 0.36 (0.02%)
var_size_seq_scan q22 111.36 109.52 1.85 (1.69%)

Copy link

codecov bot commented May 16, 2025

Codecov Report

Attention: Patch coverage is 94.04762% with 10 lines in your changes missing coverage. Please review.

Project coverage is 86.87%. Comparing base (f7ac905) to head (cbe62e4).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/storage/storage_structure/overflow_file.cpp 89.18% 4 Missing ⚠️
src/storage/shadow_utils.cpp 50.00% 2 Missing ⚠️
src/storage/storage_structure/disk_array.cpp 80.00% 2 Missing ⚠️
src/storage/store/column_reader_writer.cpp 96.42% 1 Missing ⚠️
src/storage/wal/shadow_file.cpp 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5386      +/-   ##
==========================================
- Coverage   86.90%   86.87%   -0.03%     
==========================================
  Files        1414     1412       -2     
  Lines       62085    62031      -54     
  Branches     7599     7598       -1     
==========================================
- Hits        53952    53892      -60     
- Misses       7958     7964       +6     
  Partials      175      175              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ray6080 ray6080 merged commit 419fe19 into master May 16, 2025
28 checks passed
@ray6080 ray6080 deleted the index branch May 16, 2025 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants