Solana Stake-o-Matic
As a proof-of-stake network, 33% of the stake can censor the network, therefore the minimum number of validators with stake adding up to 33% could collude to censor the network. The Stake-o-Matic is a program that automatically delegates stake according to predefined rules. The idea is that if validators write their own versions (according to their preferences), and staking via their Stake-o-Matic is incentivised, the global stake will decentralise across many validators. I have no idea if that will actually happen, but I’ve been playing with the Stake-o-Matic reference implementation.
Running the Reference Implementation
Clone and build the Solana Stake-o-Matic from the github:
git clone https://github.com/solana-labs/solana.git
cd solana/stake-o-matic/
cargo buildNB: If you get u64::MAX errors, your Rust is out of date; rustup update stable
Check the binary was built:
cd ../target/debug
./solana-stake-o-matic
error: The following required arguments were not provided:
<ADDRESS>
<KEYPAIR>
--validator-list <FILE>USAGE:
solana-stake-o-matic <ADDRESS> <KEYPAIR> --baseline-stake-amount <SOL> --bonus-stake-amount <SOL> --config <PATH> --quality-block-producer-percentage <PERCENTAGE> --validator-list <FILE>
Run again with the -h flag to see all the options.
We can run the program on testnet, we need to set up payment keypair and make a stake account. (Ask in discord for test tokens).
First I made a keypair for paying + funded the account with 10,000 Sol:
$ solana-keygen new -o stake_authority.json$ solana-keygen pubkey stake_authority.json
7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU# ask in discord if you don't have
solana pay 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU 10000$ solana balance 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU
10000 SOL
Second I made a stake account and put 8,000 Sol into it:
$ solana-keygen new -o stake_account.json$ solana-keygen pubkey stake_account.json
4AkWp8KbDBX6Pyo95zgyuW5e7rpERS1n2jBwC2Xzumbe$ solana create-stake-account --from stake_authority.json stake_account.json 8000$ solana stake-account 4AkWp8KbDBX6Pyo95zgyuW5e7rpERS1n2jBwC2Xzumbe
Balance: 8000 SOL
Rent Exempt Reserve: 0.00228288 SOL
Stake account is undelegated
Stake Authority: 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU
Withdraw Authority: 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU
We can see that the stake authority is both the Staking and Withdrawing Authority on the stake account.
Third we run the program. I’ve formatted this for readability, but it is all one line:
./target/debug/solana-stake-o-matic
--cluster testnet
--quality-block-producer-percentage 10
--baseline-stake-amount 1
--bonus-stake-amount 5
4AkWp8KbDBX6Pyo95zgyuW5e7rpERS1n2jBwC2Xzumbe
stake_authority.json
--confirmObviously we can’t use such large amounts as the Foundation use, so we override the defaults on the command line. 1 Sol “baseline” for a non-delinquent validator, plus 5 Sol “bonus” for a quality validator. When I ran the above command it found 340 quality block producers and 13 delinquents, adding on account ‘rent’ it split out around 2,100 Sol from the stake account.
That -confirm is what makes it actually send the transactions to a node, you can run without that as a dry run. FYI it’s pulling other things it uses, like the RPC endpoint from your default config file, if you don’t have that setup, specify the arguments on the command line. Also note that if over 20% of the testnet validators fail the bonus block production requirement (default 75% of blocks), the program won’t change bonuses; I’ve set the quality threshold at 10%, so almost all were counted as quality producers.
If the program fails (which it seems to for various reasons), just run it again, it won’t recreate accounts or anything like that. I had to run it 4 times, when it does nothing anymore, you’re done.
You can look at all the stake splits by piping the solana stakes output to a file:
solana stakes > x.xOpen the file in a text editor and search for your stake_authority public key:
Stake Pubkey: jjShyLqFt3cf29ZQqwYLYuizik2PtgV4VfEAn6WvVKm
Balance: 5 SOL
Rent Exempt Reserve: 0.00228288 SOL
Delegated Stake: 4.99771712 SOL
Active Stake: 0 SOL
Activating Stake: 4.99771712 SOL
Stake activates starting from epoch: 99
Delegated Vote Account Address: 4oUQ7ywb1ahguAyWaP9DiWgcg1BWSWmD4wjF7KkH5Bke
Stake Authority: 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaU
Withdraw Authority: 7P5z1mfoDQYEwMr1mSKCsDM31pid9B7u26Kx7jRdoMaUObviously there are very many accounts that we have splits for, this is just one example — it is a 5 Sol bonus account, that has been delegated to a Validator’s vote account, we can get the Validator’s identity like this:
$ solana validators | grep 4oUQ7ywb1ahguAyWaP9DiWgcg1BWSWmD4wjF7KkH5Bke
B4xFUYq2TDmz6PsiCj29vFyvmYvqTRAtnhQ3uSmxQVFd 4oUQ7ywb1ahguAyWaP9DiWgcg1BWSWmD4wjF7KkH5Bke 100% 37289879 37289817 2518891 1.3.11 f107b9b4 16788.555581984 SOL (0.32%)And we can look up that Validator’s details in various places, for example here we learn the name (“IZO”), website and IP of the Validator:
Anyway, the stake splits will not change unless we run the program again in the next epoch.
Overview of the Reference Implementation
The program does a bunch of things, such as making a lists of quality and poor block producers, making large numbers of transactions, checking the confirmations. But the heart of what is happening can be found in the main() function. For every vote_account on the validator list, two addresses are determined, a baseline and a bonus. These are derived deterministically relative to the authorized_staker plus a seed. This means a particular authorized_staker and vote_account, will always map to the baseline and bonus addresses (seed differentiates baseline from bonus). Here is the code for the baseline address:
let baseline_stake_address = Pubkey::create_with_seed(
&config.authorized_staker.pubkey(),
baseline_seed,
&solana_stake_program::id(),
)If the account doesn’t exist, a transaction is made to create the account using split_with_seed()
Transaction::new_unsigned(Message::new(
&stake_instruction::split_with_seed(
&config.source_stake_address, // funds from here
&config.authorized_staker.pubkey(), // auth on the above sacc
config.baseline_stake_amount,
&baseline_stake_address, // derived from:
&config.authorized_staker.pubkey(), // base
baseline_seed, // seed
),
Some(&config.authorized_staker.pubkey()), // signer
)),This will create an undelegated stake account at the baseline_stake_address, with the amount transferred from the source_stake_address. The node will enforce the derived address. The authorized_staker will be both the Withdraw and Stake Authority on the new account:
$ solana stake-account CibxRr1fG9GbVjkS4FVBU1EdLpQQmor49TARxnE9ep9o
Balance: 0.05 SOL
Rent Exempt Reserve: 0.00228288 SOL
Stake account is undelegated
Stake Authority: Bc3v8aHmfBCwcZqGUMUrfAoW9Cd4hf6Ai3iASfRrUW6P
Withdraw Authority: Bc3v8aHmfBCwcZqGUMUrfAoW9Cd4hf6Ai3iASfRrUW6PWhat is going on programatically is basically the same as doing this by hand.
The same thing happens with the bonus account. To be clear: both accounts are created once only, no other accounts are needed per validator; after this the stake accounts are delegated and undelegated as often as the script runs, but those accounts don’t change.
At this point the transactions have not be sent yet, they are saved in the create_stake_transactions array. The next step is delegation. If the validator is not delinquent it should get the baseline stake account delegated to it (if it isn’t already):
Transaction::new_unsigned(Message::new(
&[stake_instruction::delegate_stake(
&baseline_stake_address,
&config.authorized_staker.pubkey(),
&vote_pubkey,
)],
Some(&config.authorized_staker.pubkey()),
)),Alternatively, if the validator is delinquent, the stake is undelegated:
Transaction::new_unsigned(Message::new(
&[stake_instruction::deactivate_stake(
&baseline_stake_address,
&config.authorized_staker.pubkey(),
)],
Some(&config.authorized_staker.pubkey()),
)),The same procedure happens with the bonus stake account. Then the create_stake_transactions array of transactions is sent to the node, followed by the delegate_stake_transactions array.
And that’s pretty much it. Each validator on the fixed list has two accounts with stake on them (baseline and bonus) all the time, the accounts are just delegated or undelegated (deactivated) depending on performance.
Tweaking the Quality Criteria
An easy way to tweak the program is not to change it at all but just pass in a modified list, the -cluster argument, to which we passed testnet uses a pre-defined list in validators_list.rs, but we can also pass in a customised yaml list.
I’ve decided that the most important thing for decentralisation is a broad range of datacentres, so I won’t be delegating to anyone in a datacenter with more than 3 Solana nodes in it. To do this I will determine the ASN of every node, then count them up.
First we get the node IP addresses:
$ solana gossip -v | tee | head -n 5
IP Address | Node identifier | Gossip | TPU | RPC | Version
----------------+----------------------------------------------+--------+-------+-------+----------------
216.24.140.155 | 5D1fNXzvv5NjV1ysLjirC4WY92RNsVH18vjmcszZd8on | 8001 | 8004 | 8899 | 1.3.13 838aaee1Then we get the ASN:
$ whois -h bgp.tools " -v 216.24.140.155"
AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name
13649 | 216.24.140.155 | 216.24.128.0/19 | US | ARIN | 1998-07-23 | ViaWestI’ve written a little script that works it out for each node and counts how many nodes are at which location.
The script also lets us discard nodes at locations with more than <max> nodes. I’m going with a max of 3 at any location.
Finally, we can tell the script to just output the nodeIDs and pipe that into a file, this is my validator list. I will only be delegating to these validators.
$ solana gossip -v | ./vdump.js -max 3 -oi > validator.list
$ cat validator.list | wc -l
37Let’s set the baseline amount to 100 Sol and the Bonus to 900 Sol… so the total stake we need is 37,000 SOL.
I create a stake authority with 37,000 + 10 Sol to pay the fees:
$ solana-keygen new -o location_stake_authority.json
pubkey: ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp
$ solana pay ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp 37010$ solana balance ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp
37010 SOLAnd then the stake account:
$ solana-keygen new -o location_stake_account.json
$ solana create-stake-account --from location_stake_authority.json location_stake_account.json 37000First I do a dry run:
./target/debug/solana-stake-o-matic --validator-list ./validator.list --baseline-stake-amount 100 --bonus-stake-amount 900 ./location_stake_account.json ./location_stake_authority.jsonThis reports 68 stake accounts will be split out — if all the validators got a baseline or bonus it would be 74, but a few have missed out on one or both.
I made a small change to the reference implementation code to avoid the bonus check and recompiled rust:
let too_many_poor_block_producers = false;I run it for real:
./target/debug/solana-stake-o-matic --validator-list ./validator.list --baseline-stake-amount 100 --bonus-stake-amount 900 ./location_stake_account.json ./location_stake_authority.json --confirm3,000 Sol is left in the parent stake account:
solana-keygen pubkey ./location_stake_account.json
BtL7aXQaoqB8oL9o7JYanBXrWduttCzM7UEnafM9EAp4
$ solana stake-account BtL7aXQaoqB8oL9o7JYanBXrWduttCzM7UEnafM9EAp4
Balance: 3000 SOL
Rent Exempt Reserve: 0.00228288 SOL
Stake account is undelegated
Stake Authority: ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp
Withdraw Authority: ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKpI do a little check, we expect 68 + 1 (the parent):
$ solana stakes > x.x
$ cat x.x | grep ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp | grep Stake | wc -l
69Managing stake splits is difficult right now. To work out why exactly there is 3,000 unallocated:
solana stakes > x.x
cat x.x | sed 's/^$/XXXX/' | tr '\r\n' ' ' | sed 's/XXXX/\n/g' | grep ADM36iN9bNGusB6xbRdWbbDPsVAvJYDDeRCszqZAcVKp > our.splits
$ cat our.splits | grep "100 SOL" | wc -l
34
$ cat our.splits | grep "900 SOL" | wc -l
34So 3 validators missed out on the baseline and 3 missed out on the bonus. Stake is delegated to a validator’s vote account not the nodeID, we can find our validators like so:
$ solana validators > all.validators
$ cat validator.list | sed 's/- //' > mod_validator.list
$ while read p; do grep "$p" all.validators; done < mod_validator.list | egrep '^!'
! 8xFQP5mPt9Nzr5wMeUsH4azMM3zyGqrUn3LeopKSwptX GT7CUaRjQ3HDuVfqXeA7Cv3w9kWWi9HvAo9HXHvHuZfQ 100% 37725186 37725106 2898035 1.3.13 4265.466981191 SOL (0.07%)
! 6iVVocmqzwe32zGQJ6Yp9KR6FCByA1enPKd7PbS3mAZ4 25x7uZMYEZgJvNKDTzqz1KYEX7HamPCHgsrKyEztW4oo 100% 36683980 36683937 0 1.3.11 132.339117029 SOL (0.00%)
! ChorusM5BVgnAKbg9PF15285LkqeCoZWK2p9s35T7J2A FUdmzVtQ4UEq1TcGRQDy81KM4nfaNNCKaKhCZNeWPveE 100% 37687867 37687811 2913729 1.3.11 10365.822525841 SOL (0.17%)So we can see the same 3 validators were delinquent, and so got neither the base or the bonus; all the numbers add up.
Don’t forget to add to the crontab, I run mine once a day:
55 14 * * * cd /home/smith/SOM/solana ; ./target/debug/solana-stake-o-matic --validator-list ./validator.list --baseline-stake-amount 100 --bonus-stake-amount 900 ./location_stake_account.json ./location_stake_authority.json --confirm > som.log 2>&1Thoughts on the Reference Implementation
One thing that I noticed was that there is no withdrawing and redistributing of the undelegated accounts.
Each account on the list (if it got the bonus even once) forever has that stake split allocated to it. If the validators performance is good, it is delegated, otherwise not. This is fine for a Foundation who don’t care about profits, but what about about a regular holder? Undelegated stake does not earn rewards, you’d want your stake delegated all the time, so that you earn the maximum.
This would become especially problematic if you used a quality metric that selected a validator set that changed frequently, you could end up with a lot of undelegated stake.
Probably a really good Stake-o-Matic would dynamically redistribute stake splits, i.e. it would withdraw from undelegated stake-splits and re-stake those funds elsewhere. But that would be a quite different program from the reference implementation.
