Enforce some level of community standards on tests/test coverage/auditing for contracts

so, as the recent exploit showed us (not like we needed more evidence) it turns out writing code is actually hard (who knew). Especially when you want that code to be exploit proof. And this got me thinking…we can, as a community, require some level of proof of test coverage, if nothing else, before we allow contracts to start running on SN (even more so if they’re LPs on the bridge.) Leaving this here to get the convo started. (My personal thoughts are requiring 100% test coverage with ideally fuzz tests/PBTs.)

2 Likes

The problem is that we do not know that a given testing regime would have caught this particular exploit. Perhaps that will come out in the post mortem. Anyway, it is hard to justify putting this requirement on a permissionless chain. The issue is really about having contracts that are “too big to fail” in the network. If there were lots of other dApps moving money on SN, it would not have been so easy to make the call to rollback the network without public discussion.

I suppose you could try to limit the amount of funds that a contract without “proof-of-test” can hold, but I’m not even sure that’s possible given that the value of the funds in fiat is not fixed.

The more important question imo is how, going forward as a community, are decisions to be made around when or if to rollback the whole network given a bug in a contract?

1 Like

I don’t believe on making anything permissioned on the network. Ironically enough the permissioned bridges (besides everyone’s hard work) are what saved us this time around. At the same time, I think the people involved have learned the lesson that there is more of a spectrum to the phrase “audits are useless”.

The reality is there’s no reason to believe an audit would have caught this though, just check Rekt - Leaderboard. It’s also nevertheless true that an audit = more eyes looking at the code and that will never be worse.

We were lucky this time cause the only product active was secretswap (not really counting secretheroes nor the fardels beta here). Going onward we won’t have that option, this was our get out of jail card imo.

TL;DR Stop being nerds and get audits. Audits = more people looking. It’s that simple.

3 Likes

ok imo having an automated testing setup for contracts (or even something that just automatically checks for a testcov report that the contract can generate!!!) isn’t permissioning

1 Like

Maybe im misintepreting but how is ”before we allow contracts to start running on SN” not permissioning?

1 Like

automated testing plus a standard for communicating th results, yeah

2 Likes

This is the conversation we need to be having! Keep it up! If there was an error in a contract it means we need more proofreading. The competition is fierce in fast-moving crypto tech and that leads to things being pushed out sometimes before they are ready.

I don’t think an entire committee is necessary for this, but it would be nice to have at least a strike team to audit the codebase/add tests/etc.

1 Like

Strike team, committee, idc what it’s called

2 Likes

In principle I like the idea of more testing, however, I am unsure what the mandate would be for such a committee/strike team. Is this about developing standards and an automated testing platform, or is this about having an ongoing team of auditors?

For the former, couldn’t this be a grant project?

For the latter, how do we allocate the resources, which projects does this group prioritise for testing, etc. How would that scale with growth of the network and in the number of applications running on it?

1 Like