What is happening with the Medalla eth2 testnet currently, Oct 17th 2020? Here’s an ELI5. (Not exactly 5. Maybe 10. You get the idea.)
- a) A bunch of validators turned off (maybe zinken, maybe bored, who knows)
- b) We entered non-finality
- c) Some sync bugs reared their heads, in Prysm and Nimbus
- d) We lost more validators to the bugs, not everyone has updated since
- e) Non-finality increases memory and CPU requirements, we likely lost more validators because their nodes couldn’t handle it
- f1) Either people come back in and we regain finality or
- f2) They don’t and offline validators lose eth faster and faster, until we regain finality. Some may be ejected if their balance falls too low.
- g) You can stare at beaconcha.in to see current participation rate. The rest of the site isn’t keeping up
- h) This would not go on this long on main net because people aren’t just going to say “meh” while real eth burns
- i) The network is working as designed for a major disruption scenario in which it needs to self-heal
- j) I’m not sure we have a solid handle on when the network is back up if f2) comes to pass, but lower bound Oct 25th (ish), upper bound 5 days later
- k) Anyone who is offline and doesn’t want to come back on can help by doing an orderly exit, here’s a tool to make it easy: https://github.com/eth2-educators/medalla-exit
In some more technical detail:
Inactive validators are not punished. A validator might be inactive because it is in activation queue, or its deposit hasn’t even been processed yet – these will become active as we regain finality. A validator might also be inactive because it sent a voluntary exit, and it will not be punished.
Active and online validators stay at exactly +-0 if their inclusion distance is a perfect 1. That’s impossible, so they are penalized slightly. djrtwo of the Ethereum Foundation has stated that they are looking at ways to safely reduce this penalty for validators “doing their part” because the penalty doesn’t, in a nutshell, feel good.
Validators that are both active and offline are punished quadratically, that is, the penalty per epoch increases with each epoch.
A validator is marked offline for an epoch if it does not attest in an epoch. This can happen to otherwise running validators if their beacon node gets out of sync, or if they are unable to attest. Possible causes to look for are client bugs and RAM/CPU resource usage. Now is the time to learn how to build clients from source, and to check the sizing of your node (https://www.reddit.com/r/ethstaker/comments/jcsvfc/8_gib_nodes_some_observations_on_how_to_fit_in/).
We entered non-finality October 12th in the morning US Eastern TZ. This happened after 4 consecutive epochs without consensus.
At this point, “quadratic leaking” kicks in. Offline validators are penalized in increasing amounts as non-finality continues. The formula for this is Penalty = EffectiveBalance * Epochs-Since-Finality / (2^25). For the math geeks: “The penalty per epoch is linear with finality delay, which means the total penalty (integral of it) is quadratic” (thank you torfbolt)
Estimating the time when the chain regains finality is difficult because validator participation fluctuates, influenced by client bugs.
Validators are kicked out when their “effective balance” reaches 16 eth, which I believe happens at 16.75 actual eth.
Even before validators are kicked out, their decreasing balance means they lose weight in the consensus. I have seen an estimate that we might regain consensus after ~13 days. This gives us our lower bound: Oct 25th, or thereabouts. Some validators became active in August and never submitted a single attestation, those would accelerate this process.
It takes 18 days for a validator to be at ~60.6% of its balance, that gives us the upper bound.
After 3 consecutive epochs with consensus, finality will be restored. Offline penalties revert to their regular, less-punishing “during finality” defaults.
If you wish to tweet this, you can link to Reddit or to a gist: https://gist.github.com/yorickdowne/ea9b18ac2b51a508080c9a810d978522