More on the disk footprint of Erigon alpha

After looking at the charts in the previous post, it occurred to us that the “CallTraceSet” table is unusually large. This table holds intermediate data, which are used to build call trace indices (for supporting trace_filter RPC method). Previously, this table was pruned once the indices were built, however, the code that did it, was removed by mistake some time ago. As a result, in both beta and alpha version, there is 140 Gb of footprint which should not be there. The fix has been done in the current code, and it will be part of the next alpha release. Unfortunately, MDBX database does not shrink when a lot of data is deleted, so actually reclaiming those 140 Gb will require cleaning MDBX database (but snapshots can be kept intact), and re-running the replay from genesis. Alternatively, the database can be compacted if enough temporary disk space is available, using the command (Erigon needs to be off-line):

./build/bin/mdbx_copy -c <source_db> <destination_db>

Now if we remove that “dead weight table”, the charts would look as follows (this is theoretical, because it will take couple of days to actually produce such database now):

Note that the actual sizes will be bigger because of the inevitable slack in the database after some period of use.