I've battled a lot with building in this language. I knew I needed to because arti was built in rust and I needed first class support with it. At the time I worked a little bit before with rust and understood the design and why it was designed this way. But didn't it piss me the fuck off for well over a year. You need a different kind of thought process compared to traditional coding methods. To say simply I really didn't like how slow it made my development.
But over the last week, having a good foundation, has saved me countless times. There has been many error cases where it simply was not possible because of how I structured these enums and typestates. As such I've been able to code with a level of confidence, which all potential failure cases are covered and handled, that I'm the most productive I've ever been; doing some of the most challenging work I've ever done.
The most amazing example of this forced, but correct, design was in the new wallet layer. You need to understand all the edge cases that can arise for differing states between different core repos:
A wallet is a t-of-n threshold multisig shared between different core repos that talk to each other over Tor, on their own schedule, with no global clock and no consensus layer. So at any given moment, core repo A thinks a wallet is DkgRunning. Core repo B has already seen it go Active. Core repo C is offline and still thinks it's Proposed. Meanwhile someone's tmm instance has crashed mid-signing-round and is about to replay a request it already sent. At the same time there is always a risk a peer might be lying!
Now multiply that by payment proposals, which have their own lifecycle, and coordination rounds, which have their own per-slot ordering, and the fact that the same physical Monero output can be selected by two different proposals at the same time. This is the level of complexity I'm dealing with here where even a small mistake has cascading issues down the line. Everything needs to line up perfectly for it all to work. If I was doing this with a dynamically typed language... I don't think I would be able to properly do it. At least not with an extreme level of guarantees from massive amounts of tests.
To start simply, when tracking a state of a wallet I have a i64 backed enum that is within a single sqlx macro.
sql_int_enum! {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum WalletConfigStatus {
Proposed = 0,
DkgRunning = 1,
Active = 2,
Abandoned = 3,
Retired = 4,
}
}
The macro generates the TryFrom<i64> and the sqlx Decode impl together within the macros.rs model.
fn try_from(value: i64) -> Result<Self, Self::Error> {
match value {
$( $value => Ok(Self::$variant), )+
other => Err(format!("invalid {} discriminant: {other}", stringify!($name))),
}
}
There is no coercion possible. We don't have any Self::default() line. If a row in the SQLite somehow contains status 5, THE QUERY FAILS. Not the row itself but the query itself. This enum makes an illegal state like that unrepresentable. This is a simple example but protects against a real failure when I'm coding this stuff. If there was a status byte that wasn't recognized down the layers the state could treat it as proposed. Potentially resulting in a retired wallet being used for signing again. I'm going to put this another way. A skipped row here silently changes the meaning of a page of federation data; while if the query fails it is loud and stops the entire pipeline.
sql_int_enum! {
pub enum WalletRound {
DkgCommit = 0,
DkgShare = 1,
DkgDone = 2,
Preprocess = 4,
SigShare = 5,
Broadcast = 6,
DecoyCommit = 8,
DecoyReveal = 9,
JoinSigning = 12,
}
}
You might notice some holes. 3 used to be a proposal round I got rid of. 11 was a readiness round that was me over-engineering and wasn't really needed. So when I corrected the code, simplifying this process, I just needed to delete the variants and the code calling these variants. After I removed their lines... all cases where the wallet round was coded, broke immediately and clearly. The absence of the variant was the compatibility break. There is no deprecation shims I need to write, fucking nothing. Any missing parts was shown at compile time for me to finish the clean up. When it is gone... it's gone guaranteed. Now if only I could have that for some of my exes.
When it comes to building out the federation system I have a signed row validation system where there is three correct outcomes.
1. Accept; everything is verified, valid, and we can write it.
2. Defer; we are lacking information to verify it because a prerequisite hasn't synced yet. We retry this row later.
3. Wedge; the provided row is invalid. You stop the feed, raise an operated alert, and wait to a decision (this shouldn't happen ever unless the other peer is being adversarial)
I can map out these outcomes with a rust type
pub(super) enum PagePass {
Continue,
StopDeferred,
StopWedged {
kind: SyncAlertKind,
detail: String,
},
}and a match caller to handle the three outcomes
match process_wallet_event_page(pool, verifying_key, page.rows, &mut expected_sequence).await? {
PagePass::Continue => { if !has_more { break; } }
PagePass::StopDeferred => { deferred = true; break; }
PagePass::StopWedged { kind, detail } => {
record_sync_alert(pool, repo_uuid, SyncAlertEndpoint::WalletEvents, kind, &detail).await?;
wedged = true;
break;
}
}
It's not just a fancy bool. Defer and wedge are both resulted from an idea of verification not passing. But they are opposite reactions. If I ever treat a wedge state as a defer the core repo will sit there retrying a forged row from an adversarial peer forever. Without any alert to fix it. If I treat a defer as a wedge and there is one slow peer that hasn't shipped me the prerequisites yet, it halts their entire sync and creates an alert over something that is eventually consistent. There is no nulls here or exceptions. Every caller has to say out loud which case is handled. In the case I add another outcome every match that touches it tells me where I need to go.
Then when I go a layer down on specific attestation
pub(super) enum WalletAdminAttestation {
Valid,
MissingAttestation,
IdentityMismatch,
PinUnavailable,
MissingPublicKey,
InvalidSignature,
}There are six variants where most people would write a bool. But look at the intentions. MissingPublicKey and PinUnavailable are cases that can be eventually consistent. We can't judge a row when the pinned admin fingerprint hasn't been synced yet. While IdentityMismatch and InvalidSignature means this core repo peer has handled me something that does not and will not verify. Every variate is a "the signature check didn't pass" but it gives me the requirement to decide to retry quietly or to halt the sync on every variate.
So type keeps them apart and routing them becomes mechanical
WalletAdminAttestation::Valid => {}
WalletAdminAttestation::MissingAttestation
| WalletAdminAttestation::IdentityMismatch => {
return Ok(PagePass::StopWedged {
kind: SyncAlertKind::MalformedEvent,
detail: event.wallet_id,
});
}
WalletAdminAttestation::PinUnavailable
| WalletAdminAttestation::MissingPublicKey => {
return Ok(PagePass::StopDeferred);
}This design forces me to never forget one. If I add another failure mode I get alerted by the compiler for every single site to decide whether it is a wedge or a defer. This guarantees I didn't miss a state.
Out of all the type systems that has saved so much work this single one. Over the entire wallet layer I built types so the verified data only exists INSIDE the success variant. It's not possible to touch it without having to first prove it.
pub enum FederatedWalletStatusValidation {
Unclaimed,
Verified {
status: WalletConfigStatus,
transitioned_at: NaiveDateTime,
signature: String,
},
Malformed,
InvalidSignature,
}The owner-signed status a peer claims isn't a field I can read. It has no name at all until I'm inside the branch that earned it
if let FederatedWalletStatusValidation::Verified { status: incoming, .. } = &status
&& *incoming != existing.status
&& !status_transition_allowed(existing.status, *incoming)
{
return Ok(ConfigPass::Alert {
kind: SyncAlertKind::MalformedEvent,
detail: config.wallet_id.clone(),
});
}The whole thing in one block. There are no flags I can forget to check or a half populated object where status is set but the signature is invalid. In a dynamic language the peer's claimed status and the result of verifying it, live on the same object. Nothing would stop me from incorrectly reading the first without consulting the second. I don't need to worry about that with this design.
I want to say again that test4pay has been extremely challenging to build out. There is no global clock and no consensus layer. There is not a single place where all the state lines up. Which means there is not a single place for me to review. As it grows this becomes quickly unmaintainable. Inconsistent views of the same thing at any given moment across the peered network is a genuinely difficult problem to solve. The compiler having my back and alerting me where I'm missing a case as saved me so many fucking times. I'm not saying rust made me a better engineer but it has made it very expensive to be a careless one.