In my previous post /post/d24f8aab440a30252763, I explained how I managed to reduce /d/pgpfreaks key server memory consumption to a minimum. Obviously, I was curious to find out the detail of it. How many bytes, of what.
👣 Following footprints
We'll focus on the key objects that I mentioned previously. I store those as a "4️⃣ Cache<3️⃣ String, 2️⃣ Arc<1️⃣ Key>>" instance, with:
- "Cache" being a hash table provided by the crate Moka;
- "String" a unique identifier attached to the key (here a fingerprint);
- "Arc<>" a kind of thread-compliant pointer;
- And "Key" a custom struct representative of key, with a bunch of attributes including two lists: one of type "Vec<Subkey>" and the other of type "Vec<UserID>".
Let's try and find the actual size of our hash table 🧐
First thing to acknowledge here is the difference between fixed and sized objects. The first ones are usually straightforward to evaluate: an 8-bits "u8" is, well - a byte; a 32-bits integer "i32" is 4 bytes; an ASCII character, one byte; and so on. The second ones, that Rust characterize through the trait "Sized", are arrays of different sorts: "String" is a list of characters, "Vec<u8>" is a common representation of a binary stream, etc.
1️⃣ The "Key" struct is a mix of both. Some parameters, like the algorithm information or the timestamps, are fixed; other, like the fingerprint, subkeys or user IDs, are sized. The method "size_of::<Key>()" from the "core::mem" crate allows to get the fixed memory footprint of a struct (said otherwise, and yes this is oversimplified, the sum of everything fixed on said struct). Anything sized must then be evaluated separately.
- We start with the footprint of the "Key" object: "size_of::<Key>()".
- Getting the size of a "Vec<u8>" is straightforward: "bytes.len()" plus the struct footprint "size_of::<Vec<u8>>()".
- The size of a "String" can be obtained by converting it as bytes "str.bytes().len()" plus the struct footprint "size_of::<String>()".
- The size of a "Vec<>" of structs is equal to the sum of its components sizes plus the struct footprint "size_of::<Vec<>>()"; meaning I had to calculate the weight of each "Subkey" and "UserID" objects, to get the total key weight.
2️⃣ Keys are wrapped in an "Arc<Key>" object, that serves for sharing a read-only pointer across threads. Here we're focusing on a single hash map, so the overhead is exactly "size_of::<Arc<Key>>()", but this figure should be multiplied for each index where the "Arc<>" is cloned.
3️⃣ The "String" identifier can be evaluated like any other string: as the sum of "str.bytes().len()" and "size_of::<String>()". Interestingly enough, Moka only accepts "Sized" structs as key for caches, meaning it's also necessary to iterate each entry to get the associated footprint.
4️⃣ Lastly, the "Cache<>" object itself would require retro-engineering that I didn't go through with. Fortunately, it also implements the "size_hint()" method, returning a "SizeHint" object, that allows for an estimate (min/max) of the memory footprint of the cache. We'll settle for the minimum.
📊 Wrapping it up
Using this method, I was able to monitor most of the entities I directly instantiate in my code. It's worth noting that the overall calculation is quite heavy and should not be used for production monitoring. And while it may sound odd to anybody used to work with the support of a garbage collector, there's simply no easy way to do otherwise, unless to go for system-level monitoring.
Anyway. Working on a test set of 174 keys (up to 10 certificates by key), with the same assets that we actually have in production, and simulating 2.5k HTTP calls on top to fill up L2 caches, are bringing the footprint to 10.01mb, with:
- 1.39mb of keys including cached information; with an average 8.18kb footprint for a key, ranging from 2.68kb to 15.27kb.
- 1.3mb of assets stored at disposal for the front-end server.
- 7.31mb of HTTP responses stored as L2 cache.
Keeping up with the same proportions would allow up to 70k keys in database and 1M different HTTP calls stored in cache over 4GB of RAM. This is before starting to rely on cold storage.
Should be good enough for now 😜