Contribution based ranking for Secret Network

Hello Agents!

Secretnodes has been working on a new rank system based on contributions and collaboration instead of wealth concentration. Today I’m sharing a draft of the scoring system.

How does it work?

Node runners collect points for contributing to network initiatives. This can be done by participating in a committee, building products, and voting on network proposals.

Why is this better than wealth based ranking?

Wealth based ranking patterns users to think the “top nodes” are the ones that store the most wealth. It doesn’t account for actual contributions to the network, collaborating with network participants, or anything other than wealth. This is bad for decentralization and actually leads to more centralization, because in a lot of cases (like with the secret network), smaller nodes actually contribute more to the network than their wealth concentration based rank would lead users to believe.

Delegators will be shown a breakdown of how a node has contributed to the network, what products they’ve made, and details about how collaborative they are compared to other nodes. This will allow new users to make informed decisions when staking.

Potential drawbacks

We can’t really think of many potential drawbacks. Ranking based on wealth is like saying “Hey we hold the most money so we are the best”. So any system that measures contributions instead of wealth for ranking seems like a significant improvement over the current system.

How will this change over time?

We’d like to find other balanced ways to give points for contributions and we’d like to empower committee leads the ability to give points out (with a cap on points they can give per node). Over time the system will be tweaked and improved based on feedback.

Other Details

  1. Users will still be able to sort based on wealth ranking, but will be presented with a warning box informing them that this is not sorted based on contributions to the network and instead is sorted based purely on wealth concentration.
  2. Score will be the default sorting method, and score will be shown next to any instances of voting power.
  3. All pages for nodes will transparently explain the contributions that make up their score.
  4. Non voting nodes will be labeled as non voting validators.
  5. Uptime of a validator will have a high impact on score. (Only going back xyz time, probably around 2-4 weeks)
  6. Participation in governance will only impact score up to the last 5 or so proposals. This is meant to prevent the genesis nodes from remaining top ranked in this system.
  7. Negative points to non voting validators.

What do the new ranks look like?

Here is a rough overview of what this will look like.

  1. Non voting nodes with bad uptime will be the lowest ranked validators. (Assuming they don’t make other contributions)
  2. Non voting nodes with good uptime will be the second lowest rank. (Assuming they don’t make other contributions)
  3. Voting validators who make no products or other contributions will rank in the middle. (Assuming good uptime)
  4. Voting validators who make a product but no other contributions will be above middle rank.
  5. Voting validators who contribute and collaborate with other network participants (via committee contributions) will second to top in ranks.
  6. Voting validators who contribute and collaborate with other network participants and make a product will be top rank.

You can check out a draft of the ranking system that is being built out here. This is using legitimate data, but more tweaks will be made as we progress.

If we make any other changes to the scoring system they will be transparently explained in this thread as we progress.

Please leave any feedback and thoughts you have. We will be incorporating what we can into our plans over the next several weeks.

5 Likes

I think labelling it “contribution based ranking” is a little misleading. I’m not sure what would be a better name, but…

I’ve made numerous code contributions and suggestions to the snip20 reference implementation, secret secret, and the snip20 specification, created the snip721 specification and reference implementation, wrote the OTC auction contracts, the secret heroes contracts, created the snip20, snip721, and contract-calling packages of the dev toolkit, and wrote documentation/walkthroughs that people have found helpful, as well as devoted countless hours providing contract dev/design input both via public channels and private DMs. Yet, by the current scoring system, I apparently have contributed barely more than the average validator. While others are free to disagree, I don’t consider it an accurate representation of my contributions.

To be honest, I’ve already been feeling a little left out in the cold seeing the fund requests that have been showing up and knowing I’ve only received (not counting secret games or hackathon prizes) $15k USD for everything I’ve done. Also seeing something that suggests that I’m not really contributing can’t help but make me wonder if it is accurate, and I’m actually the one not seeing things clearly

8 Likes

I pretty much agree with everything you said.

Based on the list of individual products you’ve contributed to, if you were to vote (even abstain) on all proposals then you would be in the top 3 on this ranking system. As it stands the spreadsheet just doesn’t give you the proper amount of points based on those contributions.:wink:

I think if the points counted the right contributions and all of you voted on the 5 most recent proposals then it would go cashmaney, you, julian if im adding things up right for the top 3 in my head🧐 I also move down the list if a few other people start voting. Not voting is like “missing blocks for governance” so it carries a penalty on the score just like poor uptime would (once I add uptime information to the spreadsheet).

Example assuming everyone votes on all of the most recent 5 on chain proposals vs missing easy points by not voting in any way.

I’m sure there are contributions that I’m missing from others as well (probably taariq is in the top 5 or so)? Happy to get the points added. If you look at our roadmap you will see we have some time to add proper points before release, and ideally more entities than just me/secret llc would be able to add points based on a set of guidelines.

1 Like

Perhaps we should “measure” it not only by pool voting, but divide this value by the node staking. The more your node has staked the more responsibility you bare.

2 Likes

I’m biased (as this ranking benefits my node compared to the normal stake size ranking) but this looks like a great new metric to sort validators by. This latest itteration especially seems very promissing once all values are as they are intended to be.

One small thing I’m not sure you have already taken into consideration at this time: It would be better (in my opinion) to count the last 5 proposals that are completed, instead of the last 5 votes that includes those that are still running. In some situations it is good to be able to consider all sides of the vote for several days and it would be good to not penelize that.
Additionally, if we keep up this rate of proposals it might also be good to split it into longer term (say the last 10, 15, or even more votes), and shorter term (last 5 votes).

2 Likes

Interesting. I want to be careful not to “punish” nodes that earn a high stake, and instead merely reward top contributors and collaborators on a more level playing field. My initial thought is that weighting as you suggested would force larger stake nodes to have to do even more to have an equal rank to someone who contributes to as many different initiatives and contribute by building individual products.

1 Like

It’s indeed an interesting suggestion, the main result would probably be equalizing the valildators’ stake size. Small contributing validators would most likely be leading the charts in such situation until they reach an average size. This would ofc be against the interests of other validators, but perhaps some aspects of this idea could be used to help small active nodes gain a foothold.

Something like a small bonus for nodes under X% of delegated SCRT could be a way to implement that. I’m obviously biased though, and am not convinced it would necessarily be a good addition as it might give a too high rank to new nodes. But I have seen some messages in Telegram groups about stimulating smaller validators so thought I’d brainstorm a bit.

1 Like

Hmmm, perhaps the original ranking you shared was based on older data? Or the data you are scraping isn’t up to date? I have voted on the last 5 proposals that have reached completion (although I have not voted yet on the one that is still open)

2 Likes

But in general my point is that some types of contribution or more visible/vocal in nature. If the ranking is going to be called contribution-based, it needs to make sure it is also somehow accounting for the types of contribution that are not as visible/vocal

4 Likes

Yes vote not required. It is probably old data!

1 Like

We are approaching the design of this ranking system similarly to a game of league of legends. In someways, the main way is just to say we will be adjusting along the way and buffing and nerfing certain things.

1 Like

I don’t want to use stake based variables in any way for score. People can still sort by stake/wealth concentration if they want. Perhaps we will diverge from this stance but the ranks already allow small nodes to make it to top (example is julians node)

2 Likes

So far 3 accounts associated with validators have voted while being offline and are still offline. Since their votes have zero weight, they would not get points for voting in this system. Keep your validators running folks! (I know laura doesn’t run a node anymore so not sure this really matters, unknown on the other 2 validators).

1 Like

Hi @moonstash - I like the way you’re thinking but I am concerned that this goes against what gives the secret token so much of its value.

Would be interested to know what more of the community think about this?

PS: I am a developer, not a whale.

1 Like

Can you explain how this goes against what gives secret value?:eyes:

1 Like

Hmmmmm just brain storming here but you could easily imagine that much like google rankings, most people would assume the top of the list would attract more eyes. So validators would be go out of their way to accrue more tokens to get more traffic.

I truly believe that what you and many of the community have in mind is the fairer way in terms of people etc, but economically worse for token value as well as adding a layer of confusion regarding how to get more points for contribution ranking etc.

I could be massively wrong but just a feeling and wanted to put it out there to see what people thought.

2 Likes

Wealth concentration-based sorting will still be present. But the idea here is to highlight and bring attention top contributors instead of promoting and bringing awareness to wealth concentration by default.

1 Like

If people aren’t aware, Baedrik is one of the biggest technical contributors in the network right now. If his contributions are not properly represented, then that does indeed put the value of the proposed ranking system in question.

Which gets me to the proposal itself. It seems that the proposed ranking method itself is biased - and likely not intentionally so. People who support the proposal are those who are more ingrained in the voting aspect of the ecosystem, which seems to be the main factor weighing validators. The relatively discrete distribution also suggests that this isn’t a good metric to use.

Personally, I think that using delegation amount as the scoring method is still superior to most subjective/semi-objective metrics you can find. Is it perfect? No, but getting consensus around another metric that is normalized, well-distributed and fair might be challenging.

7 Likes

We definitely expect complaints or opposition to some degree from people no matter how we make the system work. Fortunately, we are not making this a suggestion for other explorers, wallets, etc. we are merely making the change for the secretnodes.com explorer itself. Thus we do not need typical network consensus (though if everyone came out hating it then we wouldn’t proceed I suppose).

Users continue to ask “how do i pick a validator?” And (some) validators are very against weight based sorting as it patterns users to stake with the top 5 nodes by weight only. This is our answer to that user demand.

We say “stake with top contributors” and the playing field is level in that as long as you vote, have good uptime, and contribute, then you can swiftly make it up the ranks.

1 Like

I’ll point out that Staked and B-Harvest for example, are at the bottom of the rank although they are bridge operators (essentially doing it pro bono). This is also a major contribution.

IMO if you want to rank something, you better have a deterministic and unambiguous way to rank by, otherwise it will always be debatable and will draw unnecessary antagonism to a good initiative.

Maybe a better alternative is to find a way to point out validators’ contributions, rather than creating a rank. For example, you can give green badges to validators that participated in the last 5 gov. proposals, and maybe a tooltip or an other way to refer to a validator’s list of notable contributions.

8 Likes