Increase Max Validator Count

Validating on the Secret Network is getting more competitive. The demand to get into the active set is leaving people waiting with nodes up ready to validate but not in the active set. So with the goal of further decentralizing the Secret Network and bringing in new high quality contributors, I wanted to share my intentions to propose an increase of the max validator set from 50 to 60 after IBC launch.

Some things that will be looked at post upgrade before this proposal would go forward.

  1. Average blocktime
  2. Missed blocks for validators.
  3. Looking for any surprises that impact network stability.
  4. Opinions from the core protocol devs (@Cashmaney @reuven @assafmo @guy )
  5. Community feedback.

If things look good then a proposal to increase to 60 Nodes will be submitted. After this we will go over 1-5 again and if it is determined that another increase would not threaten network stability, then the next step is ideally 70-80 nodes.

Updated plan

Straight to 70 nodes after IBC launch.

8 Likes

Would love to see an increase, concerned about the short term effects. I would suggest a better time for this proposal would be right after New Years. It would allow us to:

  1. Have IBC live
  2. Make sure the network sustains the upcoming spike in demand (remember we have Tarantino NFTs on the horizon, Shade, Stashh, and apparently more projects building on us than I can count).
6 Likes

I would agree. @Jay-ShadowRealm-SCRT and I spoke to MANY validators at cosmosverse who want to spin up. I think there is no better time to get more people involved in the ecosystem than now. January is too long. These are high quality validators and I would not suspect a dramatic drop off in performance or difficulty in handling the newcomers. I’m also thinking that now 60 slots may not be enough. Maybe 65-75.

We also really need to think about delegation criteria that @dylanschultzie has been talking about. These exist in most ecosystems in cosmos and if we want high quality contributors we need to incentivize them. I’m very excited after meeting the cosmos community.

6 Likes

Thought about this a bit more. I actually think it’s fine. The distribution of staking power is not uniform anyway, and to slow things down you need to see a meaningful slow-down of ~34% of staking power. Adding 10-20 validators would not cause that.

We support increasing to 70 validators immediately after Supernova.

8 Likes

Also maybe a place we can have all validators can profile on GitHub or something similar.
Terra has the terra station validator profile

3 Likes

I’m going to vote no on this. The reason is that I don’t feel 100% certainty that I, or anyone really, fully understands the affects of increasing the validator set. I think we are all pretty sure it is fine to increase it to 80. In fact I would say I’m in the 95% + confidence range it will be innocuous. But I don’t want to pass these types of proposals anymore based on hunches and gut intuitions etc etc. I want us to do the work, understand it 100%, and then set a strategy for the long run that is based on that understanding.

Something along the lines of. “Our goal as a network is to have a max number of 300 validators. 300 is optimal for X, Y, and Z reasons. But at our current traffic, it better to be at 80. So as traffic increases we will vote to increase incrementally based on X metric.”

Of course what I wrote is meaningless, but it is the form that I would like decisions like this to take. If we need to incentivize someone, or some group, to do the work and bring recommendations to the community I would support that too.

9 Likes

Domerium Labs will be voting ‘No’ on this, for two reasons.

  1. The proposal is a failure of Governance that should not be endorsed.
    As the proposer stated themselves, the proposal was discussed in the validator group. This is a gated sub-community of 145 people. A tiny fraction of the 84,000 delegators in the network. We are a decentralized network. Having a discussion among 0.17% of the network is not decentralized. Not following Governance rules that were set by the community is not decentralized. Domerium Labs does not stand for these types of white ivory tower proposals. The SCRT community is amazing, involve them!

  2. There have been many discussions over the past couple of months regarding the validator set. In chat, on forums, and in governance calls. To our knowledge, all these discussions resulted in the same consensus; increasing the set is desirable but should be done with a science-based approach. We have a major network upgrade coming in 8 days, upgrading the set of validators, and the possible instability that could introduce, on top of a network upgrade is as far away from best practice in IT as you can get.

Imagine the network does turn out unstable. What is the cause? Was it the upgrade? Was it the validator set? Introducing 2 variables at the same time that could potentially have similar negative consequences means we are going to be spending much more time debugging and finding the root cause than we would have if we only have 1 variable to point at.

Domerium Labs support increasing the validator set after the mainnet upgrade has concluded and the network has proven to be stable.

10 Likes

In response to what Stefan and the-dusky said I would like to add some things to this. Please do not me get me wrong, I mostly agree with these arguments as well (especially the scientific based approach), but I think time is of essence because we don’t want to lose top notch validators (with good uptimes btw) to a too small validator set.

I agree with the fact that we in this case did not have 1 week explicit discussion period before we set this proposal online. But in my defense we had discussions like this about the number of new validators (so to have 70-80 validators, see the first post of moonstash). So going from 70 to 80 now isn’t a completely whole new ballpark of a number and has been in discussions. In light of the departure of some top notch validators that we need for secret in the future, I (and moonstash) decided to put this on chain directly and not waste any more time than necessary. This seems a bit rushed, but since we have a 1 week period anyways we can use the time to discuss it now as well.

Since SCRTLabs delayed the upgrade by 2 weeks we got 2 weeks to actually see if the bigger validator set causes more instability for the network or not. In case it does not we should see this pretty fast.

2 Likes

Hello everybody, it’s a pleasure to be here with you, with the expansion of Validators in the network, will you plans give the opportunity to reopen the delegations programme to support these new Validators entering the Set and decentralise the network, balancing a little bit between all of them?

Good points made by @the-dusky and @Stefan_DomeriumLabs that I agree with pretty strongly.

I would also like to see a team funded to bring us a more science-based approach so we can make better decisions regarding validator set and infrastructure.

In this case I would totally support trying to increase the validator set by 10 to see what happens; however, what’s complicating that for me is exactly what Stefan mentioned where we would be introducing another variable in the mix – while we’re trying to apply performance improvements as part of an upgrade, and needing to measure that.

From @Stefan_DomeriumLabs:

Introducing 2 variables at the same time that could potentially have similar negative consequences means we are going to be spending much more time debugging and finding the root cause than we would have if we only have 1 variable to point at.

The peer issues we’ve seen and other performance issues due to hardware, etc. are not easily investigated as a group of validators. My team spent many hours and days down that rabbit hole and so even if we have a two-week delay on the upgrade, if there are problems we’d have to resolve those before Supernova Alpha. I don’t have a high degree of confidence that that could happen in the next week or two.

As someone with a test engineering background, this is a big red flag – trying to do this change before an upgrade that will be addressing performance issues.

Let’s do one thing at a time:

  • the upgrade
  • take a bit of time to evaluate
    • how much of an improvement is there?
    • are peering issues the same?
  • then look at increasing the validator set

@anon60841010 and @SecretSaturn I applaud your efforts to address the many validators who haven’t been able to get into the active set by increasing the limit! I was sad to see some exceptional teams booted out of the set recently :sob:.

My recommendation is to increase the validator set after Supernova Alpha. Unfortunately, I will have to vote no on this proposal.

2 Likes

Well! Since the upgrade is approximately 3 weeks away and it’s a small increase in the validator set, I support this proposal and will change my vote to Yes :joy:

1 Like

Digiline Validator voted YES on proposal #85 as we believe this will result in a net positive for the network.

We do acknowledge the issues brought upon by other validators, but we believe that given that the upgrade to shockwave alpha has been delayed by a few weeks, the network will be able to test the change in max validators number, and it will be made clear what kinds of performance changes will occur.

This being said, we do agree with other validators that in the future it would help to have a more scientific way to decide what the maximum validators number should be, rather than just guess at a number.

3 Likes

I’m going to vote YES on this

1 Like

The vote has already concluded and the proposal has passed.

1 Like

backlog comment that was filtered by the anti-spam bot, I just approved it :wink:

2 Likes

I’m leaning towards YES too

2 Likes

I think it’s time to do next step and vote for increasing from 80 to 90 validators
Other popular cosmos-based networks already have at least 100 or even more validators (osmosis, kava, etc)
It’s a good idea to follow their steps and increase current limit for at least 10 points (80 → 90)

In the Governance Call of August 10th 2022 the topic of increasing the active set was discussed, the conclusion was:

There doesn’t seem to be a strong desire to increase the active set size in the near future. Therefore it seems that October would be the next target assuming the data indicates it is possible without affecting the chain.

This quote was also part of the public abstract of this meeting which is published on this forums, Discord, and Telegram. Noone replied with the desire to increase the set earlier. This topic was brought up a few hours ago in the Governance Telegram group again, and there didn’t seem to be an urgent demand to increase before the upgrade either. Assaf did mention there it probably wouldn’t be an issue in relation to performance. However, it will now only be 2 weeks between this proposal ending and the upgrade date. Last time the set was increased this also happened shortly before an upgrade and this was one of the main critisms of that proposal. We have been at the 100k threshold to the active set for several weeks already.

In that same telegram discussion @ertemann suggested to increase the set in smaller steps more regurarly as that seems to lead to higher quality inclusions. This seems like an interesting suggestion that is on the agenda for Wednesday’s governance call.

According to proposal #81 (Secret Network Charter & Code of Conduct) a proposal should be posted on the forums 7 days before moving on chain. Even if you’d consider this a change from the previous proposal (since it’s posted in that previous thread), it still prescribes a 3 day discussion period. This new proposal (#107) was posted on chain at the same time as it was posted here and does thus not abide by proposal #81. I know certain community members do not agree that technical props such as this should fall under prop #81, but as stated earlier in this post, noone indicated urgency regarding this set increase and the short discussion on telegram from a few hours ago ended with moving the discussion to the forums.

Taking all this together, secretSauce will be voting ‘No’ at this time.

5 Likes

We voted no on the on-chain proposal

1: It breaks Governance protocol (7-day discussion period on the forum)

2: The reasoning behind the proposal ‘other cosmos chains’. It is the worst argument I ever heard for increasing the validator set. We aren’t the same as other Cosmos chains. We run unique hardware that may or may not react differently to the increase of the active validator set. Increasing the active set should not be based on a gut feeling, or because some other chain running different hardware did it too. It should be based on rational, informed, science-based conclusions from researching our own chain.

3: We have a network upgrade coming soon (early September). We have always, historically, made increases to the active set after the upgrade runs stable on the current set. Introducing new validators and new variables mere weeks before a hardware upgrade is the exact opposite from best-practice in the broader IT world.

4 Likes

Agree with Stefan here. Seems unnecessarily risky ahead of Sept 13th & increasing the validator set in the weeks after the upgrade also give us something to keep the marketing hype train running after the main event.

1 Like