Secret Infrastructure

Secret Infrastructure Charter

Purpose

Secret infrastructure conducts research, maintains network infrastructure such as testnet, relayers, and api’s, and creates documentation & essential resources for node runners on the Secret Network, for the benefit of all network participants.

Decision Domains

∙ The scope of Secret Infrastructure is to ensure smooth operations of Secret Network public infrastructure (node runner stability, community api, quicksync, and potentially more).

∙ Secret Infrastructure brings scope modifications and to other technical community members first for review, then to the forums, then directly to the chain.

Focus

Secret Infrastructure considers topics related to network performance and essential network infrastructure to be main areas of focus for which IT is the functional owner, including but not limited to:

∙ Documentation helpful to node runners on the Secret Network (such as how to setup on different providers, how to setup api endpoints, useful linux guides, etc.)

∙ Community API

∙ Providing quicksync or snapshot downloads, scripts, or instructions

∙ Conduct research on different secretd binaries, api performance enhancements, and other. Example: different db backend and tweaked parameters, caching community api, etc

Membership

Secret Infrastructure membership is designed to be representative of Secret Networks IT leadership. Entry can happen with approval of all members, assistance is open to all. Only members are responsible for deliverables.

Members:

@moonstash @mohammedpatla

Compensation:

∙ 1,500 SCRT monthly

@moonstash

∙ 1000 SCRT monthly

@mohammedpatla

Current prices (Aug 21, 2021) $1.70 (moonstash)

Discretionary Budget:

∙ 38000 SCRT

This is a one time allocation until/unless there is a need for more resources.

Exclusively for buying hardware to test and add to the community api, api maintenance, relayer nodes, and misc infrastructure related expenses. Additionally, payments can be made to contributors of Infrastructure committee who are adding value, but who chose not to purse the responsibilities of a lead.

Total Amount

44,500 SCRT

Procedures

∙ Meeting frequency – once per month. Secret Infrastructure will determine modifications to the meeting schedule as needed based on current activities.

∙ Time Commitment – In addition to the scheduled meetings, members are expected to complete all deliverables specified.

∙ Reporting – We will report results and updates directly to the community through the Secret Network Forums.

Deliverables

  • Community API enhancements: Make the community api faster through hardware and more advanced reverse proxy configurations.

  • 3rd Gen Xeon research and deployment: New enclave support is coming to the Secret Network. Deploy a 3rd gen scalable xeon, migrate api load balancer, build out a 40-50 secret node endpoint capacity, research and deploy relay nodes to connect secret network to other networks via IBC.

  • Quicksync: provide a reliable long term offering for quicksync.

  • Seed Node: provide a reliable seed node for network participants to use and add to guides.

Clauses

  • Any hardware purchased must be utilized for the benefit of the whole secret network community, not just a member of the working group. So any tested hardware that works should be repurposed after testing for use as API endpoint or something else that is infrastructure related and a public good resource.

  • Any unknown costs related to running a relayer can come from the discretionary budget. Other unknown obstacles will be dealt with as they arrive.

With these guidelines as a basis, Secret Infrastructure will determine its need for other operational procedures.

5 Likes

Sounds good. Time to get beefy. :+1::+1:t6::+1:t5::+1:t5::+1:t4::+1:t3::+1:t2:

5 Likes

Hi Ian,

Everything looks good to me, the main budget is going towards new hardware, which will provide us a better open API endpoint for the whole of SN. I am looking forward to testing the new hardware with Xeon Scalable gen3 processors. And running relayers once IBC is on mainnet in 1.5 months.

One thing I would really want to propose is having, our monthly compensation in line with other on-chain proposals.

I propose
3000 SCRT/ Month - @moonstash (estimated work will be about 15 hrs/ week)
2500 SCRT/ Month - Me (Estimated between 10-15 hrs/week)

I think this fits more in line, and don’t want our work to be undervalued just because we do most of ours in the background.

1 Like

I’m not really opposed to having our pay fall in line with others. Seems fair. So this would change the numbers as follows

Compensation:

∙ 3,000 SCRT monthly

@moonstash

∙ 2,500 SCRT monthly

@mohammedpatla

Current prices (Aug 21, 2021) $1.70

Discretionary Budget:

∙38,000 SCRT

Total Amount

54,500 SCRT

2 Likes

Yo-yo,
It looks like a lot of discretionary moni to spend on hardware, do you have a rough idea on items cost u would want to buy?
How did you guys spend the first 6k scrt awarded 3 months ago / what more is needed ?
What’s a rough deadline on getting the api running and load balancing properly ?
Ty

2 Likes

Is there competition for this role?

I think we need a little more information and I would want to know if we can rent processing power vs hosting it ourselves etc. How is this api hosting so centralized, etc.?

How do we know these are two different people and not one person doing a job that takes a few hours?

It’s a lot of money and I had no idea there was a demand for this service until now.

1 Like

The most expensive scalable processor named Xeon was $10k and so we will have like 7 of them? For a few calls to the api each day as there are no current apps.

Shouldn’t ENG core handle infrastructure? I feel like $50M plus $15M is enough to maintain a blockchain for a few years without this community role.

It would allow for 40-60 ish nodes to run across 2 machines.

The discretionary budget was spent entirely on hardware. We have 1 machine that can run 8 secretnodes (7 prune everything and 1 archive). We were supposed to get 2 machines but the price dropped and the last gen xeons got disacontinued

The community API is working today and load balancing requests already. This would just work to increase the capacity and performance.

Renting to run the number of nodes i said would cost more than buying the hardware (over a couple year time period).

Having enough storage space allocated so that each node will continue to run for longer than a year or 2 requires large and expensive SSDs, this is likely a cost you are overlooking.

After the beta launched here is our usage so far. (Secretnodes.com and i believe keplr is using it already)

We want Secret Network to have a highly performant API freely available to dapps, wallets, explorers, etc that can run well into the future. The bulk of the money from this prop would merely be for buying hardware that would be a public resource on the network going long into the future.

5 Likes

235 kbps? That doesn’t seem like much and my $1,000 computer can handle 1000 times that demand if the internet connection permitted. Why $70k? That’s a massive massive amount of processing power. You have $6k in hardware now and you can’t handle dialup speed requests?

The calculation for traffic is done strangely. There isn’t enough data in the history. But the requests are real. Like i said, we are using it for our site in production. The value here is anyone can deploy something like keplr, secretswap, or secretnodes.com (many other sites that would need a reliable api are being worked on). This is to build more capacity for people like them.

I’m familiar with traffic in KIN where there are millions of transactions happening each day but there has never been a discussion where it takes 2 people and $100K to host the endpoint or documentation. This is not normal IMO. Maybe you can start by teaching someone like me why SGX is expensive and I’ll have to think about the cost of transactions in terms of the fixed costs of hosting an API or endpoint. With just a handful of calls this drastically increase the cost per transaction.

1 Like

Daily transaction volume is 5000. That’s one transaction every three minutes.

With this proposal we are pegging a fixed cost to a fluctuating one (SCRT) and there are so many questions I have and I still want to know why ENG core isn’t using the ICO money, 15M investment or the 500k SCRT sold by the foundation during the last bull market.

Could you maybe use a resource that gives us a better idea of the cost and what we’re paying for in return? I want to be able to fact check the deliverables for accuracy before voting to pay for an API.

If everyone voting understands API’s and our demand for this service you can also just let me know it’s common and I’ll do my own research.

This would be true if a day had 250 hours

Also, there is a difference between a transaction and a query. There are hundreds of thousands of queries per day. These queries need an API.

  1. ENG core is not something. ENG is dead, it is over, in the past.
  2. The company behind what was known as the ENG token, Enigma. Is now a core contributor to Secret Network. The ENG ICO is not an SCRT ICO. If it were, the SEC would probably sue SCRT like they sued Enigma.

There are many API nodes, from enigma, from secretnodes, from figment/datahub, and others. What SecretNodes is building here is the next level. It is dozens of nodes in one to create an API that can handle the future of Secret Network. Comparing what he is building today, to the volume we have today doesn’t make sense. We need to make sure our infrastructure is ready for the application and user boom we aim for by the end of the year.

And your $1,000 computer can’t handle any of this, because it requires you to have a $2,000-$16,000 SSD to even store and query the blockchain reliably.

4 Likes

I think we are technically a few hours short of a full day of the API running, but here is the request count so far.

3 Likes

I understand this is not the same as Stellar or Solana where transactions are fast and now free. We had millions of transactions through fully featured apps like Rave, MadLipz and dozens of others. We served earns and spend transactions with now up to a million transactions a day and many many more queries. Each of our users transacting with SCRT is an actual investor making a real transaction on Secret Swap or querying the blockchain somewhere. I thought queries were cheap or free and didn’t require the API to “hit the blockchain” for a result and so I’m just trying to understand the cost associated with this metric and were it’s coming from to warrant a community pool spend on hardware. I’m getting the picture that nobody else is really speaking up about the potential for abuse of the community pool and so if you want to just use it as a piggybank for forking ENG and are telling me the project is dead… I believe you. I see that.

My point about ENG core is that they still have funds to run this endpoint and that we should be spending community funds on things that aren’t related to owning $70k worth of equipment on behalf of everyone. Because early funds were “sunk” costs invested in startup and operational expenses it has less impact on price moving forward and doesn’t stress the supply and demand by selling off 38,000 SCRT to pay for equipment.

Will there be an official proposal explaining the cost and intent? I’m pretty sure we become much less competitive the moment we spend the money on infrastructure and this changes our valuation. Anything that changes our fundaments and valuation should be vetted out. Doesn’t Zcash writes the entire history of the blockchain in each snark? I read that it stores it as a snapshot in a snapshot and we are requesting roughly $100k to fund infrastructure to run queries. Furthermore, this benefits the swap and nodes but that cost is not passed to them or taken from fees of any program using the service. Investors are asked to front the capital and I can’t agree. If you want a grant to build something then do it but it should be competitive to just renting storage or outsourcing to a company that already owns the space and equipment. I assume we still have some liquid cash on hand for this kind of spending regardless of whether ENG is now a contributor.

It sounds like we are required to have a max $16,000 storage unit (requesting $70k?) that cannot be rented or scaled from Amazon AWS EC2 or RDS. I’m willing to debate the upfront cost now vs having it pay for itself in 5 years when the difference is 38,000 SCRT that could be worth 30x more vs the depreciation of equipment. There’s the price impact associate with spending from the community pool and selling pressure when it is least likely to be absorbed. Is it worth losing thousands or millions in investment dollars, assuming we could rent scalable storage for a low monthly fee? I personally don’t want our community pool to appear as a personal slush fund and 3rd tax to anyone doing their own research. How do we keep this equipment from being stolen or confiscated if it’s in someone’s garage?

2 Likes

There’s nothing decentralized about this spending request and we have a community member not required to share their real name or take responsibility and handing them a key piece of infrastructure. If this is normal now then I wonder what’s changed in the network and why nobody sees that as a potential complication.

2 Likes

I see on secretnodes_com there are currently 4544 transactions a day. So 564 SCRT fees a day. 38,000 SCRT staked at 30%
38,000 times 1.7 = $64,600
64,600 x .3 = $19380 interest
19,380 / 365 = $53 a day
53/ 564 = about $0.10 per transaction if we just locked it up and spent the interest to rent space. This is far more expensive than the actual cost associate with making a call to the blockchain and so is scalable for years to come and until we do have millions of user transactions. Even then it still might be cheaper to maintain and many companies do it.

Please stop raiding every thread with whatever random word salad you feel like coming up with. If you dont understand the difference between queries and transactions then that is the point you should have withdrawn from the conversation.

5 Likes

Thank you for your informative input. I stated previously that I worked with queries and transactions over the past few years on KIN and asked for more information about why they cost so much here and nothing there. I’ve also seen conversations from Baedrik and others about queries vs transactions and think I understand them well. 2 million queries is not the same thing as having transactions
and if we are getting charged for them we should be charging to make them…