Use hash writer for aristo computeKey #3572

chirag-parmar · 2025-08-20T14:35:23Z

closed duplicate: #3226
fixes: #3164

Uses the rlp hash writer for Aristo DB's computeKey implementation

execution_chain/db/aristo/aristo_compute.nim

…ent computing branchKey multiple times within a single call.

execution_chain/db/aristo/aristo_compute.nim

arnetheduck · 2025-09-12T11:24:40Z

execution_chain/db/aristo/aristo_compute.nim

  w.startList(2)
  w.append(pfx.toHexPrefix(isLeaf = true).data())
+  w.wrapEncoding(1)


doesn't this call to wrapEnconding depend on leafData being of a particular type? if yes, why use auto?

I'm not sure about wrapEncoding but the reason I changed this line was that we are passing a type account or slot and don't need to pass in a block of statements. Perhaps it would be better to make it even more strict and use Account | UInt256 as the type.

ok - it feels odd that it should be needed cc @chirag-parmar - ie append should be able to figure this out ideally - but it seems to work, so ..

Apologies for the late reply. the call to wrapEncoding doesn't depend on the leafData being of a particular type. Only leaf nodes have nested serializations i.e. Serializing an object/list/tuple that already has a serialized rlp value in it.

append should be able to figure this out ideally

I didn't understand what it must be able to figure out.

I think what I would have expected maybe is a call like appendEncoded(leafData) which does the extra round of encoding of the passed-in item - wrapEncoding looks like an internal low-level operation, specially with that number being passed in, because presumably it must be followed up with actual data - that gives the caller an unnecessary opportunity to use it the wrong way

makes sense actually. status-im/nim-eth#813

execution_chain/db/aristo/aristo_compute.nim

arnetheduck · 2025-09-13T17:11:42Z

so .. the final thing to do with this branch is to run a benchmark: using an unhashed snapshot like https://eth1-db.nimbus.team/mainnet-20250831-nohash.tar.gz, run the node pre/post to measure how long it takes to create the full hash tree (it's printed when starting the node) - afair, 25% of cpu time was spent allocating rlp encodings, so we should see a good boost here.

Co-authored-by: Jacek Sieka <[email protected]>

bhartnett · 2025-09-15T14:58:16Z

so .. the final thing to do with this branch is to run a benchmark: using an unhashed snapshot like https://eth1-db.nimbus.team/mainnet-20250831-nohash.tar.gz, run the node pre/post to measure how long it takes to create the full hash tree (it's printed when starting the node) - afair, 25% of cpu time was spent allocating rlp encodings, so we should see a good boost here.

I ran a test using an unhashed snapshot I have from just after the merge at around block 15000000. Here are the results.

When running the baseline build from master:

NTC 2025-09-15 16:20:38.534+08:00 Importing era archive                      blockNumber=15537394 dataDir=/home/user/.local/state/nimbus/mainnet eraDir=/mnt/5aa6b1af-8122-4eec-b52f-bdfb121e74de/era/mainnet
INF 2025-09-15 16:21:44.447+08:00 Writing computeKey cache                   keys=2139900 accounts=0.22%
INF 2025-09-15 16:22:52.479+08:00 Writing computeKey cache                   keys=4279800 accounts=0.91%
INF 2025-09-15 16:23:57.324+08:00 Writing computeKey cache                   keys=6419700 accounts=1.85%
INF 2025-09-15 16:24:58.980+08:00 Writing computeKey cache                   keys=8559600 accounts=2.67%
INF 2025-09-15 16:26:02.630+08:00 Writing computeKey cache                   keys=10699500 accounts=3.66%
INF 2025-09-15 16:26:55.133+08:00 Writing computeKey cache                   keys=12839400 accounts=4.09%
INF 2025-09-15 16:27:52.876+08:00 Writing computeKey cache                   keys=14979300 accounts=4.53%
INF 2025-09-15 16:28:55.905+08:00 Writing computeKey cache                   keys=17119200 accounts=5.27%
INF 2025-09-15 16:29:58.874+08:00 Writing computeKey cache                   keys=19259100 accounts=5.47%
INF 2025-09-15 16:30:50.617+08:00 Writing computeKey cache                   keys=21399000 accounts=5.52%
INF 2025-09-15 16:31:44.304+08:00 Writing computeKey cache                   keys=23538900 accounts=6.13%
INF 2025-09-15 16:32:42.459+08:00 Writing computeKey cache                   keys=25678800 accounts=6.75%
INF 2025-09-15 16:33:39.740+08:00 Writing computeKey cache                   keys=27818700 accounts=7.43%
INF 2025-09-15 16:34:49.518+08:00 Writing computeKey cache                   keys=29958600 accounts=7.77%
INF 2025-09-15 16:35:42.001+08:00 Writing computeKey cache                   keys=32098500 accounts=7.77%
INF 2025-09-15 16:36:33.841+08:00 Writing computeKey cache                   keys=34238400 accounts=8.32%
INF 2025-09-15 16:37:35.051+08:00 Writing computeKey cache                   keys=36378300 accounts=9.39%
INF 2025-09-15 16:38:40.428+08:00 Writing computeKey cache                   keys=38518200 accounts=10.28%
INF 2025-09-15 16:39:32.161+08:00 Writing computeKey cache                   keys=40658100 accounts=10.58%
INF 2025-09-15 16:40:25.481+08:00 Writing computeKey cache                   keys=42798000 accounts=11.07%
INF 2025-09-15 16:41:25.258+08:00 Writing computeKey cache                   keys=44937900 accounts=11.55%
INF 2025-09-15 16:42:24.596+08:00 Writing computeKey cache                   keys=47077800 accounts=12.50%
INF 2025-09-15 16:43:28.296+08:00 Writing computeKey cache                   keys=49217700 accounts=13.41%
INF 2025-09-15 16:44:31.012+08:00 Writing computeKey cache                   keys=51357600 accounts=14.25%
INF 2025-09-15 16:45:32.914+08:00 Writing computeKey cache                   keys=53497500 accounts=15.06%
INF 2025-09-15 16:46:27.690+08:00 Writing computeKey cache                   keys=55637400 accounts=15.50%
INF 2025-09-15 16:47:20.456+08:00 Writing computeKey cache                   keys=57777300 accounts=16.22%
INF 2025-09-15 16:48:18.473+08:00 Writing computeKey cache                   keys=59917200 accounts=16.95%
INF 2025-09-15 16:49:20.112+08:00 Writing computeKey cache                   keys=62057100 accounts=17.72%
INF 2025-09-15 16:50:20.016+08:00 Writing computeKey cache                   keys=64197000 accounts=18.66%
INF 2025-09-15 16:51:20.044+08:00 Writing computeKey cache                   keys=66336900 accounts=19.43%
INF 2025-09-15 16:52:26.452+08:00 Writing computeKey cache                   keys=68476800 accounts=20.11%
INF 2025-09-15 16:53:29.258+08:00 Writing computeKey cache                   keys=70616700 accounts=20.99%
INF 2025-09-15 16:54:26.445+08:00 Writing computeKey cache                   keys=72756600 accounts=21.73%
INF 2025-09-15 16:55:21.410+08:00 Writing computeKey cache                   keys=74896500 accounts=22.67%
INF 2025-09-15 16:56:18.622+08:00 Writing computeKey cache                   keys=77036400 accounts=23.21%
INF 2025-09-15 16:57:22.420+08:00 Writing computeKey cache                   keys=79176300 accounts=24.22%
INF 2025-09-15 16:58:21.585+08:00 Writing computeKey cache                   keys=81316200 accounts=24.66%
INF 2025-09-15 16:59:18.477+08:00 Writing computeKey cache                   keys=83456100 accounts=25.38%
INF 2025-09-15 17:00:16.475+08:00 Writing computeKey cache                   keys=85596000 accounts=26.30%
INF 2025-09-15 17:01:19.806+08:00 Writing computeKey cache                   keys=87735900 accounts=27.26%
INF 2025-09-15 17:02:20.428+08:00 Writing computeKey cache                   keys=89875800 accounts=27.95%
INF 2025-09-15 17:03:11.878+08:00 Writing computeKey cache                   keys=92015700 accounts=28.40%
INF 2025-09-15 17:04:12.591+08:00 Writing computeKey cache                   keys=94155600 accounts=29.06%
INF 2025-09-15 17:05:15.458+08:00 Writing computeKey cache                   keys=96295500 accounts=30.06%
INF 2025-09-15 17:06:28.405+08:00 Writing computeKey cache                   keys=98435400 accounts=30.64%
INF 2025-09-15 17:07:17.784+08:00 Writing computeKey cache                   keys=100575300 accounts=30.64%
INF 2025-09-15 17:08:07.956+08:00 Writing computeKey cache                   keys=102715200 accounts=30.64%
INF 2025-09-15 17:09:07.167+08:00 Writing computeKey cache                   keys=104855100 accounts=31.06%
INF 2025-09-15 17:10:03.867+08:00 Writing computeKey cache                   keys=106995000 accounts=31.81%
INF 2025-09-15 17:11:02.362+08:00 Writing computeKey cache                   keys=109134900 accounts=32.48%
INF 2025-09-15 17:12:08.893+08:00 Writing computeKey cache                   keys=111274800 accounts=33.47%
INF 2025-09-15 17:13:13.160+08:00 Writing computeKey cache                   keys=113414700 accounts=34.07%
INF 2025-09-15 17:14:12.160+08:00 Writing computeKey cache                   keys=115554600 accounts=35.00%
INF 2025-09-15 17:15:17.992+08:00 Writing computeKey cache                   keys=117694500 accounts=35.90%
INF 2025-09-15 17:16:21.763+08:00 Writing computeKey cache                   keys=119834400 accounts=36.93%
INF 2025-09-15 17:17:21.050+08:00 Writing computeKey cache                   keys=121974300 accounts=37.50%
INF 2025-09-15 17:18:25.373+08:00 Writing computeKey cache                   keys=124114200 accounts=38.28%
INF 2025-09-15 17:19:29.537+08:00 Writing computeKey cache                   keys=126254100 accounts=39.00%
INF 2025-09-15 17:20:23.648+08:00 Writing computeKey cache                   keys=128394000 accounts=39.90%
INF 2025-09-15 17:21:28.039+08:00 Writing computeKey cache                   keys=130533900 accounts=40.72%
INF 2025-09-15 17:22:29.183+08:00 Writing computeKey cache                   keys=132673800 accounts=41.50%
INF 2025-09-15 17:23:24.773+08:00 Writing computeKey cache                   keys=134813700 accounts=41.84%
INF 2025-09-15 17:24:26.209+08:00 Writing computeKey cache                   keys=136953600 accounts=42.66%
INF 2025-09-15 17:25:29.146+08:00 Writing computeKey cache                   keys=139093500 accounts=43.14%
INF 2025-09-15 17:26:29.551+08:00 Writing computeKey cache                   keys=141233400 accounts=43.76%
INF 2025-09-15 17:27:19.604+08:00 Writing computeKey cache                   keys=143373300 accounts=44.07%
INF 2025-09-15 17:28:15.755+08:00 Writing computeKey cache                   keys=145513200 accounts=44.63%
INF 2025-09-15 17:29:13.403+08:00 Writing computeKey cache                   keys=147653100 accounts=45.36%
INF 2025-09-15 17:30:11.022+08:00 Writing computeKey cache                   keys=149793000 accounts=45.69%
INF 2025-09-15 17:31:15.566+08:00 Writing computeKey cache                   keys=151932900 accounts=46.78%
INF 2025-09-15 17:32:16.083+08:00 Writing computeKey cache                   keys=154072800 accounts=47.48%
INF 2025-09-15 17:33:22.357+08:00 Writing computeKey cache                   keys=156212700 accounts=48.18%
INF 2025-09-15 17:34:19.229+08:00 Writing computeKey cache                   keys=158352600 accounts=48.63%
INF 2025-09-15 17:35:27.419+08:00 Writing computeKey cache                   keys=160492500 accounts=49.55%
INF 2025-09-15 17:36:29.622+08:00 Writing computeKey cache                   keys=162632400 accounts=50.41%
INF 2025-09-15 17:37:33.649+08:00 Writing computeKey cache                   keys=164772300 accounts=51.12%
INF 2025-09-15 17:38:29.502+08:00 Writing computeKey cache                   keys=166912200 accounts=51.85%
INF 2025-09-15 17:39:33.398+08:00 Writing computeKey cache                   keys=169052100 accounts=52.53%
INF 2025-09-15 17:40:32.744+08:00 Writing computeKey cache                   keys=171192000 accounts=53.15%
INF 2025-09-15 17:41:30.841+08:00 Writing computeKey cache                   keys=173331900 accounts=53.63%
INF 2025-09-15 17:42:29.521+08:00 Writing computeKey cache                   keys=175471800 accounts=54.44%
INF 2025-09-15 17:43:28.183+08:00 Writing computeKey cache                   keys=177611700 accounts=55.01%
INF 2025-09-15 17:44:26.181+08:00 Writing computeKey cache                   keys=179751600 accounts=55.70%
INF 2025-09-15 17:45:28.796+08:00 Writing computeKey cache                   keys=181891500 accounts=56.61%
INF 2025-09-15 17:46:30.381+08:00 Writing computeKey cache                   keys=184031400 accounts=57.40%
INF 2025-09-15 17:47:30.382+08:00 Writing computeKey cache                   keys=186171300 accounts=58.21%
INF 2025-09-15 17:48:33.944+08:00 Writing computeKey cache                   keys=188311200 accounts=59.08%
INF 2025-09-15 17:49:32.179+08:00 Writing computeKey cache                   keys=190451100 accounts=60.00%
INF 2025-09-15 17:50:45.305+08:00 Writing computeKey cache                   keys=192591000 accounts=60.80%
INF 2025-09-15 17:51:34.913+08:00 Writing computeKey cache                   keys=194730900 accounts=60.85%
INF 2025-09-15 17:52:37.564+08:00 Writing computeKey cache                   keys=196870800 accounts=61.65%
INF 2025-09-15 17:53:45.932+08:00 Writing computeKey cache                   keys=199010700 accounts=62.14%
INF 2025-09-15 17:54:36.760+08:00 Writing computeKey cache                   keys=201150600 accounts=62.14%
INF 2025-09-15 17:55:33.372+08:00 Writing computeKey cache                   keys=203290500 accounts=62.50%
INF 2025-09-15 17:56:40.304+08:00 Writing computeKey cache                   keys=205430400 accounts=63.33%
INF 2025-09-15 17:57:42.183+08:00 Writing computeKey cache                   keys=207570300 accounts=63.93%
INF 2025-09-15 17:58:48.725+08:00 Writing computeKey cache                   keys=209710200 accounts=64.54%
INF 2025-09-15 17:59:40.988+08:00 Writing computeKey cache                   keys=211850100 accounts=64.63%
INF 2025-09-15 18:00:39.082+08:00 Writing computeKey cache                   keys=213990000 accounts=65.42%
INF 2025-09-15 18:01:47.511+08:00 Writing computeKey cache                   keys=216129900 accounts=66.34%
INF 2025-09-15 18:02:53.625+08:00 Writing computeKey cache                   keys=218269800 accounts=66.83%
INF 2025-09-15 18:03:45.499+08:00 Writing computeKey cache                   keys=220409700 accounts=67.15%
INF 2025-09-15 18:04:46.975+08:00 Writing computeKey cache                   keys=222549600 accounts=67.67%
INF 2025-09-15 18:05:39.531+08:00 Writing computeKey cache                   keys=224689500 accounts=68.28%
INF 2025-09-15 18:06:41.164+08:00 Writing computeKey cache                   keys=226829400 accounts=69.28%
INF 2025-09-15 18:07:36.271+08:00 Writing computeKey cache                   keys=228969300 accounts=69.45%
INF 2025-09-15 18:08:37.445+08:00 Writing computeKey cache                   keys=231109200 accounts=70.27%
INF 2025-09-15 18:09:42.609+08:00 Writing computeKey cache                   keys=233249100 accounts=71.29%
INF 2025-09-15 18:10:45.241+08:00 Writing computeKey cache                   keys=235389000 accounts=72.03%
INF 2025-09-15 18:11:43.244+08:00 Writing computeKey cache                   keys=237528900 accounts=72.72%
INF 2025-09-15 18:12:47.213+08:00 Writing computeKey cache                   keys=239668800 accounts=73.52%
INF 2025-09-15 18:13:53.375+08:00 Writing computeKey cache                   keys=241808700 accounts=74.11%
INF 2025-09-15 18:14:52.007+08:00 Writing computeKey cache                   keys=243948600 accounts=74.88%
INF 2025-09-15 18:15:52.173+08:00 Writing computeKey cache                   keys=246088500 accounts=75.82%
INF 2025-09-15 18:16:50.155+08:00 Writing computeKey cache                   keys=248228400 accounts=76.05%
INF 2025-09-15 18:17:43.067+08:00 Writing computeKey cache                   keys=250368300 accounts=76.89%
INF 2025-09-15 18:18:46.327+08:00 Writing computeKey cache                   keys=252508200 accounts=77.57%
INF 2025-09-15 18:19:48.340+08:00 Writing computeKey cache                   keys=254648100 accounts=78.24%
INF 2025-09-15 18:20:50.735+08:00 Writing computeKey cache                   keys=256788000 accounts=79.16%
INF 2025-09-15 18:21:56.493+08:00 Writing computeKey cache                   keys=258927900 accounts=80.04%
INF 2025-09-15 18:23:02.936+08:00 Writing computeKey cache                   keys=261067800 accounts=80.91%
INF 2025-09-15 18:24:07.940+08:00 Writing computeKey cache                   keys=263207700 accounts=81.48%
INF 2025-09-15 18:25:00.869+08:00 Writing computeKey cache                   keys=265347600 accounts=81.86%
INF 2025-09-15 18:25:56.763+08:00 Writing computeKey cache                   keys=267487500 accounts=82.74%
INF 2025-09-15 18:26:59.555+08:00 Writing computeKey cache                   keys=269627400 accounts=83.19%
INF 2025-09-15 18:27:54.809+08:00 Writing computeKey cache                   keys=271767300 accounts=83.95%
INF 2025-09-15 18:28:54.713+08:00 Writing computeKey cache                   keys=273907200 accounts=84.66%
INF 2025-09-15 18:30:01.365+08:00 Writing computeKey cache                   keys=276047100 accounts=85.47%
INF 2025-09-15 18:30:59.768+08:00 Writing computeKey cache                   keys=278187000 accounts=86.30%
INF 2025-09-15 18:32:00.589+08:00 Writing computeKey cache                   keys=280326900 accounts=86.99%
INF 2025-09-15 18:33:03.228+08:00 Writing computeKey cache                   keys=282466800 accounts=87.86%
INF 2025-09-15 18:33:58.248+08:00 Writing computeKey cache                   keys=284606700 accounts=88.48%
INF 2025-09-15 18:35:06.039+08:00 Writing computeKey cache                   keys=286746600 accounts=89.29%
INF 2025-09-15 18:36:00.328+08:00 Writing computeKey cache                   keys=288886500 accounts=89.51%
INF 2025-09-15 18:37:01.847+08:00 Writing computeKey cache                   keys=291026400 accounts=90.32%
INF 2025-09-15 18:38:06.135+08:00 Writing computeKey cache                   keys=293166300 accounts=91.19%
INF 2025-09-15 18:39:07.796+08:00 Writing computeKey cache                   keys=295306200 accounts=91.63%
INF 2025-09-15 18:39:58.331+08:00 Writing computeKey cache                   keys=297446100 accounts=91.96%
INF 2025-09-15 18:40:55.568+08:00 Writing computeKey cache                   keys=299586000 accounts=92.54%
INF 2025-09-15 18:41:55.021+08:00 Writing computeKey cache                   keys=301725900 accounts=93.15%
INF 2025-09-15 18:43:00.918+08:00 Writing computeKey cache                   keys=303865800 accounts=94.15%
INF 2025-09-15 18:44:03.171+08:00 Writing computeKey cache                   keys=306005700 accounts=95.01%
INF 2025-09-15 18:45:03.879+08:00 Writing computeKey cache                   keys=308145600 accounts=95.87%
INF 2025-09-15 18:45:59.104+08:00 Writing computeKey cache                   keys=310285500 accounts=96.87%
INF 2025-09-15 18:47:02.770+08:00 Writing computeKey cache                   keys=312425400 accounts=97.66%
INF 2025-09-15 18:48:06.319+08:00 Writing computeKey cache                   keys=314565300 accounts=98.37%
INF 2025-09-15 18:48:57.134+08:00 Writing computeKey cache                   keys=316705200 accounts=98.57%
INF 2025-09-15 18:50:10.355+08:00 Writing computeKey cache                   keys=318845100 accounts=99.26%
INF 2025-09-15 18:50:59.066+08:00 Writing computeKey cache                   keys=320985000 accounts=99.26%
INF 2025-09-15 18:51:47.675+08:00 Writing computeKey cache                   keys=323124900 accounts=99.26%
INF 2025-09-15 18:52:36.229+08:00 Writing computeKey cache                   keys=325264800 accounts=99.26%
INF 2025-09-15 18:53:35.329+08:00 Writing computeKey cache                   keys=327404700 accounts=99.85%
INF 2025-09-15 18:53:50.773+08:00 Wrote computeKey cache                     keys=328002861 accounts=100.00%
INF 2025-09-15 18:53:57.772+08:00 Imported blocks                            blockNumber=15537404 slot=4700023 blocks=10 txs=1250 mgas=249.2 bps=0.001 tps=0.136 mgps=0.027 avgBps=0.001 avgTps=0.136 avgMGps=0.027 elapsed=2h33m19s
NTC 2025-09-15 18:53:57.772+08:00 Import complete                            blockNumber=15537404 slot=4700023 blocks=10 txs=1250 mgas=249.2

When running the hash writer build from the branch:

NTC 2025-09-15 19:26:20.563+08:00 Importing era archive                      blockNumber=15537394 dataDir=/home/user/.local/state/nimbus/mainnet eraDir=/mnt/5aa6b1af-8122-4eec-b52f-bdfb121e74de/era/mainnet
INF 2025-09-15 19:27:37.729+08:00 Writing computeKey cache                   keys=2139900 accounts=0.22%
INF 2025-09-15 19:28:49.592+08:00 Writing computeKey cache                   keys=4279800 accounts=0.91%
INF 2025-09-15 19:30:08.960+08:00 Writing computeKey cache                   keys=6419700 accounts=1.85%
INF 2025-09-15 19:31:14.136+08:00 Writing computeKey cache                   keys=8559600 accounts=2.67%
INF 2025-09-15 19:32:22.474+08:00 Writing computeKey cache                   keys=10699500 accounts=3.66%
INF 2025-09-15 19:33:26.324+08:00 Writing computeKey cache                   keys=12839400 accounts=4.09%
INF 2025-09-15 19:34:33.946+08:00 Writing computeKey cache                   keys=14979300 accounts=4.53%
INF 2025-09-15 19:35:42.955+08:00 Writing computeKey cache                   keys=17119200 accounts=5.27%
INF 2025-09-15 19:36:51.182+08:00 Writing computeKey cache                   keys=19259100 accounts=5.47%
INF 2025-09-15 19:37:52.208+08:00 Writing computeKey cache                   keys=21399000 accounts=5.52%
INF 2025-09-15 19:38:56.242+08:00 Writing computeKey cache                   keys=23538900 accounts=6.13%
INF 2025-09-15 19:40:01.120+08:00 Writing computeKey cache                   keys=25678800 accounts=6.75%
INF 2025-09-15 19:41:12.641+08:00 Writing computeKey cache                   keys=27818700 accounts=7.43%
INF 2025-09-15 19:42:29.520+08:00 Writing computeKey cache                   keys=29958600 accounts=7.77%
INF 2025-09-15 19:43:28.824+08:00 Writing computeKey cache                   keys=32098500 accounts=7.77%
INF 2025-09-15 19:44:35.461+08:00 Writing computeKey cache                   keys=34238400 accounts=8.32%
INF 2025-09-15 19:45:47.644+08:00 Writing computeKey cache                   keys=36378300 accounts=9.39%
INF 2025-09-15 19:47:04.209+08:00 Writing computeKey cache                   keys=38518200 accounts=10.28%
INF 2025-09-15 19:48:04.754+08:00 Writing computeKey cache                   keys=40658100 accounts=10.58%
INF 2025-09-15 19:49:09.813+08:00 Writing computeKey cache                   keys=42798000 accounts=11.07%
INF 2025-09-15 19:50:20.992+08:00 Writing computeKey cache                   keys=44937900 accounts=11.55%
INF 2025-09-15 19:51:25.415+08:00 Writing computeKey cache                   keys=47077800 accounts=12.50%
INF 2025-09-15 19:52:36.620+08:00 Writing computeKey cache                   keys=49217700 accounts=13.41%
INF 2025-09-15 19:53:46.802+08:00 Writing computeKey cache                   keys=51357600 accounts=14.25%
INF 2025-09-15 19:54:50.010+08:00 Writing computeKey cache                   keys=53497500 accounts=15.06%
INF 2025-09-15 19:55:54.526+08:00 Writing computeKey cache                   keys=55637400 accounts=15.50%
INF 2025-09-15 19:57:02.787+08:00 Writing computeKey cache                   keys=57777300 accounts=16.22%
INF 2025-09-15 19:58:09.101+08:00 Writing computeKey cache                   keys=59917200 accounts=16.95%
INF 2025-09-15 19:59:25.804+08:00 Writing computeKey cache                   keys=62057100 accounts=17.72%
INF 2025-09-15 20:00:36.530+08:00 Writing computeKey cache                   keys=64197000 accounts=18.66%
INF 2025-09-15 20:01:42.061+08:00 Writing computeKey cache                   keys=66336900 accounts=19.43%
INF 2025-09-15 20:02:50.430+08:00 Writing computeKey cache                   keys=68476800 accounts=20.11%
INF 2025-09-15 20:04:04.884+08:00 Writing computeKey cache                   keys=70616700 accounts=20.99%
INF 2025-09-15 20:05:21.246+08:00 Writing computeKey cache                   keys=72756600 accounts=21.73%
INF 2025-09-15 20:06:34.855+08:00 Writing computeKey cache                   keys=74896500 accounts=22.67%
INF 2025-09-15 20:07:37.878+08:00 Writing computeKey cache                   keys=77036400 accounts=23.21%
INF 2025-09-15 20:08:52.893+08:00 Writing computeKey cache                   keys=79176300 accounts=24.22%
INF 2025-09-15 20:10:00.510+08:00 Writing computeKey cache                   keys=81316200 accounts=24.66%
INF 2025-09-15 20:11:10.486+08:00 Writing computeKey cache                   keys=83456100 accounts=25.38%
INF 2025-09-15 20:12:28.824+08:00 Writing computeKey cache                   keys=85596000 accounts=26.30%
INF 2025-09-15 20:13:43.746+08:00 Writing computeKey cache                   keys=87735900 accounts=27.26%
INF 2025-09-15 20:14:55.222+08:00 Writing computeKey cache                   keys=89875800 accounts=27.95%
INF 2025-09-15 20:16:00.719+08:00 Writing computeKey cache                   keys=92015700 accounts=28.40%
INF 2025-09-15 20:17:09.686+08:00 Writing computeKey cache                   keys=94155600 accounts=29.06%
INF 2025-09-15 20:18:29.257+08:00 Writing computeKey cache                   keys=96295500 accounts=30.06%
INF 2025-09-15 20:19:41.840+08:00 Writing computeKey cache                   keys=98435400 accounts=30.64%
INF 2025-09-15 20:20:41.238+08:00 Writing computeKey cache                   keys=100575300 accounts=30.64%
INF 2025-09-15 20:21:38.118+08:00 Writing computeKey cache                   keys=102715200 accounts=30.64%
INF 2025-09-15 20:22:42.010+08:00 Writing computeKey cache                   keys=104855100 accounts=31.06%
INF 2025-09-15 20:23:55.764+08:00 Writing computeKey cache                   keys=106995000 accounts=31.81%
INF 2025-09-15 20:25:09.835+08:00 Writing computeKey cache                   keys=109134900 accounts=32.48%
INF 2025-09-15 20:26:13.771+08:00 Writing computeKey cache                   keys=111274800 accounts=33.47%
INF 2025-09-15 20:27:26.306+08:00 Writing computeKey cache                   keys=113414700 accounts=34.07%
INF 2025-09-15 20:28:32.137+08:00 Writing computeKey cache                   keys=115554600 accounts=35.00%
INF 2025-09-15 20:29:50.038+08:00 Writing computeKey cache                   keys=117694500 accounts=35.90%
INF 2025-09-15 20:31:06.685+08:00 Writing computeKey cache                   keys=119834400 accounts=36.93%
INF 2025-09-15 20:32:10.922+08:00 Writing computeKey cache                   keys=121974300 accounts=37.50%
INF 2025-09-15 20:33:21.148+08:00 Writing computeKey cache                   keys=124114200 accounts=38.28%
INF 2025-09-15 20:34:31.695+08:00 Writing computeKey cache                   keys=126254100 accounts=39.00%
INF 2025-09-15 20:35:52.627+08:00 Writing computeKey cache                   keys=128394000 accounts=39.90%
INF 2025-09-15 20:37:03.749+08:00 Writing computeKey cache                   keys=130533900 accounts=40.72%
INF 2025-09-15 20:38:07.366+08:00 Writing computeKey cache                   keys=132673800 accounts=41.50%
INF 2025-09-15 20:39:12.531+08:00 Writing computeKey cache                   keys=134813700 accounts=41.84%
INF 2025-09-15 20:40:29.954+08:00 Writing computeKey cache                   keys=136953600 accounts=42.66%
INF 2025-09-15 20:41:39.445+08:00 Writing computeKey cache                   keys=139093500 accounts=43.14%
INF 2025-09-15 20:42:47.342+08:00 Writing computeKey cache                   keys=141233400 accounts=43.76%
INF 2025-09-15 20:43:46.907+08:00 Writing computeKey cache                   keys=143373300 accounts=44.07%
INF 2025-09-15 20:44:58.159+08:00 Writing computeKey cache                   keys=145513200 accounts=44.63%
INF 2025-09-15 20:46:07.439+08:00 Writing computeKey cache                   keys=147653100 accounts=45.36%
INF 2025-09-15 20:47:07.117+08:00 Writing computeKey cache                   keys=149793000 accounts=45.69%
INF 2025-09-15 20:48:22.385+08:00 Writing computeKey cache                   keys=151932900 accounts=46.78%
INF 2025-09-15 20:49:28.255+08:00 Writing computeKey cache                   keys=154072800 accounts=47.48%
INF 2025-09-15 20:50:46.481+08:00 Writing computeKey cache                   keys=156212700 accounts=48.18%
INF 2025-09-15 20:51:50.617+08:00 Writing computeKey cache                   keys=158352600 accounts=48.63%
INF 2025-09-15 20:52:59.097+08:00 Writing computeKey cache                   keys=160492500 accounts=49.55%
INF 2025-09-15 20:54:13.562+08:00 Writing computeKey cache                   keys=162632400 accounts=50.41%
INF 2025-09-15 20:55:30.251+08:00 Writing computeKey cache                   keys=164772300 accounts=51.12%
INF 2025-09-15 20:56:36.451+08:00 Writing computeKey cache                   keys=166912200 accounts=51.85%
INF 2025-09-15 20:57:44.287+08:00 Writing computeKey cache                   keys=169052100 accounts=52.53%
INF 2025-09-15 20:58:53.454+08:00 Writing computeKey cache                   keys=171192000 accounts=53.15%
INF 2025-09-15 21:00:04.885+08:00 Writing computeKey cache                   keys=173331900 accounts=53.63%
INF 2025-09-15 21:01:12.859+08:00 Writing computeKey cache                   keys=175471800 accounts=54.44%
INF 2025-09-15 21:02:19.798+08:00 Writing computeKey cache                   keys=177611700 accounts=55.01%
INF 2025-09-15 21:03:34.174+08:00 Writing computeKey cache                   keys=179751600 accounts=55.70%
INF 2025-09-15 21:04:50.550+08:00 Writing computeKey cache                   keys=181891500 accounts=56.61%
INF 2025-09-15 21:05:58.464+08:00 Writing computeKey cache                   keys=184031400 accounts=57.40%
INF 2025-09-15 21:07:14.686+08:00 Writing computeKey cache                   keys=186171300 accounts=58.21%
INF 2025-09-15 21:08:24.643+08:00 Writing computeKey cache                   keys=188311200 accounts=59.08%
INF 2025-09-15 21:09:41.384+08:00 Writing computeKey cache                   keys=190451100 accounts=60.00%
INF 2025-09-15 21:10:57.796+08:00 Writing computeKey cache                   keys=192591000 accounts=60.80%
INF 2025-09-15 21:11:57.208+08:00 Writing computeKey cache                   keys=194730900 accounts=60.85%
INF 2025-09-15 21:13:06.843+08:00 Writing computeKey cache                   keys=196870800 accounts=61.65%
INF 2025-09-15 21:14:24.297+08:00 Writing computeKey cache                   keys=199010700 accounts=62.14%
INF 2025-09-15 21:15:25.246+08:00 Writing computeKey cache                   keys=201150600 accounts=62.14%
INF 2025-09-15 21:16:32.517+08:00 Writing computeKey cache                   keys=203290500 accounts=62.50%
INF 2025-09-15 21:17:48.147+08:00 Writing computeKey cache                   keys=205430400 accounts=63.33%
INF 2025-09-15 21:18:53.169+08:00 Writing computeKey cache                   keys=207570300 accounts=63.93%
INF 2025-09-15 21:20:07.121+08:00 Writing computeKey cache                   keys=209710200 accounts=64.54%
INF 2025-09-15 21:21:08.788+08:00 Writing computeKey cache                   keys=211850100 accounts=64.63%
INF 2025-09-15 21:22:14.199+08:00 Writing computeKey cache                   keys=213990000 accounts=65.42%
INF 2025-09-15 21:23:31.253+08:00 Writing computeKey cache                   keys=216129900 accounts=66.34%
INF 2025-09-15 21:24:42.057+08:00 Writing computeKey cache                   keys=218269800 accounts=66.83%
INF 2025-09-15 21:25:53.255+08:00 Writing computeKey cache                   keys=220409700 accounts=67.15%
INF 2025-09-15 21:27:05.889+08:00 Writing computeKey cache                   keys=222549600 accounts=67.67%
INF 2025-09-15 21:28:19.280+08:00 Writing computeKey cache                   keys=224689500 accounts=68.28%
INF 2025-09-15 21:29:37.937+08:00 Writing computeKey cache                   keys=226829400 accounts=69.28%
INF 2025-09-15 21:30:47.619+08:00 Writing computeKey cache                   keys=228969300 accounts=69.45%
INF 2025-09-15 21:31:56.963+08:00 Writing computeKey cache                   keys=231109200 accounts=70.27%
INF 2025-09-15 21:33:15.470+08:00 Writing computeKey cache                   keys=233249100 accounts=71.29%
INF 2025-09-15 21:34:38.249+08:00 Writing computeKey cache                   keys=235389000 accounts=72.03%
INF 2025-09-15 21:35:49.408+08:00 Writing computeKey cache                   keys=237528900 accounts=72.72%
INF 2025-09-15 21:37:02.131+08:00 Writing computeKey cache                   keys=239668800 accounts=73.52%
INF 2025-09-15 21:38:15.693+08:00 Writing computeKey cache                   keys=241808700 accounts=74.11%
INF 2025-09-15 21:39:35.177+08:00 Writing computeKey cache                   keys=243948600 accounts=74.88%
INF 2025-09-15 21:40:55.293+08:00 Writing computeKey cache                   keys=246088500 accounts=75.82%
INF 2025-09-15 21:42:02.884+08:00 Writing computeKey cache                   keys=248228400 accounts=76.05%
INF 2025-09-15 21:43:22.149+08:00 Writing computeKey cache                   keys=250368300 accounts=76.89%
INF 2025-09-15 21:44:34.093+08:00 Writing computeKey cache                   keys=252508200 accounts=77.57%
INF 2025-09-15 21:45:56.280+08:00 Writing computeKey cache                   keys=254648100 accounts=78.24%
INF 2025-09-15 21:47:05.465+08:00 Writing computeKey cache                   keys=256788000 accounts=79.16%
INF 2025-09-15 21:48:29.412+08:00 Writing computeKey cache                   keys=258927900 accounts=80.04%
INF 2025-09-15 21:49:40.380+08:00 Writing computeKey cache                   keys=261067800 accounts=80.91%
INF 2025-09-15 21:50:53.547+08:00 Writing computeKey cache                   keys=263207700 accounts=81.48%
INF 2025-09-15 21:52:05.654+08:00 Writing computeKey cache                   keys=265347600 accounts=81.86%
INF 2025-09-15 21:53:18.433+08:00 Writing computeKey cache                   keys=267487500 accounts=82.74%
INF 2025-09-15 21:54:32.773+08:00 Writing computeKey cache                   keys=269627400 accounts=83.19%
INF 2025-09-15 21:55:47.556+08:00 Writing computeKey cache                   keys=271767300 accounts=83.95%
INF 2025-09-15 21:56:57.886+08:00 Writing computeKey cache                   keys=273907200 accounts=84.66%
INF 2025-09-15 21:58:16.321+08:00 Writing computeKey cache                   keys=276047100 accounts=85.47%
INF 2025-09-15 21:59:39.387+08:00 Writing computeKey cache                   keys=278187000 accounts=86.30%
INF 2025-09-15 22:00:52.418+08:00 Writing computeKey cache                   keys=280326900 accounts=86.99%
INF 2025-09-15 22:02:05.400+08:00 Writing computeKey cache                   keys=282466800 accounts=87.86%
INF 2025-09-15 22:03:19.116+08:00 Writing computeKey cache                   keys=284606700 accounts=88.48%
INF 2025-09-15 22:04:32.416+08:00 Writing computeKey cache                   keys=286746600 accounts=89.29%
INF 2025-09-15 22:05:35.806+08:00 Writing computeKey cache                   keys=288886500 accounts=89.51%
INF 2025-09-15 22:06:52.993+08:00 Writing computeKey cache                   keys=291026400 accounts=90.32%
INF 2025-09-15 22:08:11.544+08:00 Writing computeKey cache                   keys=293166300 accounts=91.19%
INF 2025-09-15 22:09:22.948+08:00 Writing computeKey cache                   keys=295306200 accounts=91.63%
INF 2025-09-15 22:10:31.507+08:00 Writing computeKey cache                   keys=297446100 accounts=91.96%
INF 2025-09-15 22:11:52.808+08:00 Writing computeKey cache                   keys=299586000 accounts=92.54%
INF 2025-09-15 22:13:01.552+08:00 Writing computeKey cache                   keys=301725900 accounts=93.15%
INF 2025-09-15 22:14:19.488+08:00 Writing computeKey cache                   keys=303865800 accounts=94.15%
INF 2025-09-15 22:15:46.068+08:00 Writing computeKey cache                   keys=306005700 accounts=95.01%
INF 2025-09-15 22:16:59.816+08:00 Writing computeKey cache                   keys=308145600 accounts=95.87%
INF 2025-09-15 22:18:09.411+08:00 Writing computeKey cache                   keys=310285500 accounts=96.87%
INF 2025-09-15 22:19:26.134+08:00 Writing computeKey cache                   keys=312425400 accounts=97.66%
INF 2025-09-15 22:20:43.071+08:00 Writing computeKey cache                   keys=314565300 accounts=98.37%
INF 2025-09-15 22:21:46.918+08:00 Writing computeKey cache                   keys=316705200 accounts=98.57%
INF 2025-09-15 22:23:08.712+08:00 Writing computeKey cache                   keys=318845100 accounts=99.26%
INF 2025-09-15 22:24:07.597+08:00 Writing computeKey cache                   keys=320985000 accounts=99.26%
INF 2025-09-15 22:25:07.338+08:00 Writing computeKey cache                   keys=323124900 accounts=99.26%
INF 2025-09-15 22:26:06.227+08:00 Writing computeKey cache                   keys=325264800 accounts=99.26%
INF 2025-09-15 22:27:15.670+08:00 Writing computeKey cache                   keys=327404700 accounts=99.85%
INF 2025-09-15 22:27:37.590+08:00 Wrote computeKey cache                     keys=328002861 accounts=100.00%
INF 2025-09-15 22:27:43.170+08:00 Imported blocks                            blockNumber=15537404 slot=4700023 blocks=10 txs=1250 mgas=249.2 bps=0.001 tps=0.115 mgps=0.023 avgBps=0.001 avgTps=0.115 avgMGps=0.023 elapsed=3h1m22s
NTC 2025-09-15 22:27:43.171+08:00 Import complete                            blockNumber=15537404 slot=4700023 blocks=10 txs=1250 mgas=249.2

Total run time for the master build was 2h33m19s while the total run time for the hash writer build was 3h1m22s.

Unfortunately with these changes the stateroot computation appears to be slower than the master branch. @chirag-parmar Perhaps you could do a benchmark as well to confirm these results?

chirag-parmar · 2025-09-16T07:45:30Z

Unfortunately with these changes the stateroot computation appears to be slower than the master branch.

I'm not sure why we are measuring the compute time. The purpose of the hash writer was to save on memory and not time. It still does improve on time a little as seen here but only for large data objects like entire blocks.

@chirag-parmar Perhaps you could do a benchmark as well to confirm these results?

I'll try and run the benchmarks and measure the memory usage too. But I wouldn't expect results too far off from @bhartnett 's benchmarks.

arnetheduck · 2025-09-16T07:50:52Z

hash writer was to save on memory

not really - it's both - memory allocations take time and the idea is to avoid creating work for the garbage collector, which ultimately saves both time and memory, when done right. 25% of time is spent allocating and collecting, and by creating the hashes directly we should be able to avoid that work entirely.

chirag-parmar · 2025-09-16T08:50:43Z

memory allocations take time and the idea is to avoid creating work for the garbage collector

This is true. But we use the saved time to do two passes of the data to be encoded. And we also do allocate some memory for prefixes. Maybe I can try and use the StaticLengthTracker since the types are pretty much fixed for aristo db (not the case for orderedTrieRoot)

when done right.

I agree. I will look into how I can customize the optimization for aristo db so that we save on time as well.

chirag-parmar added 4 commits August 19, 2025 11:48

incomplete

7b4e814

commented code

b383533

use similar structure to ordered trie

2253512

comment

60f5dad

chirag-parmar mentioned this pull request Aug 20, 2025

Use hash writer #3226

Closed

chirag-parmar changed the title ~~Hash writer aristo~~ nimbusel: use hash writer for aristo computeKey Aug 20, 2025

reintroduce the line

c92907b

chirag-parmar commented Aug 20, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Outdated Show resolved Hide resolved

chirag-parmar mentioned this pull request Aug 22, 2025

Add test covering HashKey rlp encoding when data contains embedded trie node #3578

Merged

bhartnett reviewed Aug 26, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Show resolved Hide resolved

bhartnett added 3 commits September 9, 2025 11:59

Remove hack.

1e9f6ed

Merge branch 'master' into hash-writer-aristo

13bc9b7

Remove unneeded usages of untyped. Update encodeExt to a func to prev…

3dc24fa

…ent computing branchKey multiple times within a single call.

bhartnett changed the title ~~nimbusel: use hash writer for aristo computeKey~~ Use hash writer for aristo computeKey Sep 9, 2025

bhartnett marked this pull request as ready for review September 9, 2025 05:23

bhartnett reviewed Sep 9, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Outdated Show resolved Hide resolved

bhartnett requested a review from arnetheduck September 9, 2025 05:34

bhartnett changed the title ~~Use hash writer for aristo computeKey~~ Use hash writer for aristo computeKey Sep 9, 2025

bhartnett mentioned this pull request Sep 9, 2025

RLP hash writer causing era1 import error #3569

Open

review

157f1a4

arnetheduck reviewed Sep 12, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Outdated Show resolved Hide resolved

arnetheduck reviewed Sep 12, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Outdated Show resolved Hide resolved

bhartnett reviewed Sep 12, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Show resolved Hide resolved

bhartnett and others added 3 commits September 12, 2025 21:43

Remove usages of auto in encodeLeaf.

126c0a8

fix imports

184a373

Make encodeLeaf a func and store hashKeys in encodeBranch.

2246c31

arnetheduck reviewed Sep 13, 2025

View reviewed changes

execution_chain/db/aristo/aristo_compute.nim Outdated Show resolved Hide resolved

Update execution_chain/db/aristo/aristo_compute.nim

0419fde

Co-authored-by: Jacek Sieka <[email protected]>

Merge branch 'master' into hash-writer-aristo

9f71e5e

chirag-parmar mentioned this pull request Sep 16, 2025

change RLP API method wrapEncoding() to appendEncoded status-im/nim-eth#813

Open

Use hash writer for aristo computeKey #3572

Are you sure you want to change the base?

Use hash writer for aristo computeKey #3572

Uh oh!

Conversation

chirag-parmar commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arnetheduck Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

bhartnett Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

arnetheduck Sep 13, 2025

Choose a reason for hiding this comment

Uh oh!

chirag-parmar Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

chirag-parmar Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

arnetheduck Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

chirag-parmar Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arnetheduck commented Sep 13, 2025

Uh oh!

bhartnett commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chirag-parmar commented Sep 16, 2025

Uh oh!

arnetheduck commented Sep 16, 2025

Uh oh!

chirag-parmar commented Sep 16, 2025

Uh oh!

Uh oh!

chirag-parmar commented Aug 20, 2025 •

edited

Loading

bhartnett commented Sep 15, 2025 •

edited

Loading