(Community Pool Proposal) DataHub - Secret Network Integration

Abstract

DataHub (https://figment.io/datahub/) helps developers build applications on the Secret Network without having to manage blockchain infrastructure or develop custom middleware. We make sure they can focus on what they do best: developing amazing software.

DataHub will provide Gateway Access to Secret for submitting transactions as well as performing read queries via the Tendermint RPC and Secret LCD.

Figment will also develop and make available in DataHub new Analytics APIs to enhance the existing REST API:

  • Transaction Search API which provides full-text search on transaction details and support for filtering by address and message type.

Project Description

Figment is looking to add Secret to its suite of DataHub APIs.

As part of the DataHub API suite, Figment will host Gateway Access (submit transactions + read data) and also develop and host its Transaction Search API.

Developers will be able to sign up and access live blockchain data and submit transactions without having to worry about maintaining blockchain infrastructure or doing custom integrations. We provide reliable access to the Tendermint RPC and the Secret REST API as well as providing the gateway to submit on-chain transactions. Both the latest mainnet and the latest testnet will be supported, and an archive node option will be available to users.

In addition, Transaction Search APIs will allow users to pull transaction data in ways currently impossible with the REST API:

  1. Filter transactions by:

  2. Address

  3. Message type

  4. Transaction type

  5. Data range

  6. Full-text search on transactions details by:

  7. Memo field

As part of the grant, Gateway Access and Transaction Search will be free for developers for a period of six months. We will establish a quota of 200,000 requests per day per account during this period, with an option for developers to upgrade as needed.

Deliverables & Milestones

DataHub APIs

  1. Gateway Access that includes:

  2. Hosted full and archive nodes with ability to submit transactions

  3. Hosted full and archive nodes with Tendermint RPC and Secret REST interface

  4. Transaction Search API

a) Code open-sourced and documented
b) API hosted in DataHub for developer use
c) Deployment scripts so developers can run the API, as well

  1. Sample test cases, test scripts, and tutorials for developer onboarding.
  2. Each developer account gets their own API key
  3. API keys can be managed and rotated
  4. Mainnet and testnet

Ecosystem Value & Success Metrics

With this project, Figment wants to increase developer adoption on the Secret Network.

DataHub allows developers to leverage the full power of the Secret Network by accessing its transaction data without having to develop their own infrastructure or middleware. It removes part of the barriers to entry for Web 2 developers and provides a strong competitive advantage for Secret versus other smart contract networks.

To track the success of this project, Figment will track the following metrics:

  • of Transaction Search requests made via DataHub
  • of requests made via Secret Gateway Access
  • of Secret DataHub Subscribers

Budget

The deliveries include the development and the hosting of Secret DataHub and the Transaction Search API.

  • $20,000 USD equivalent in $SCRT tokens upon the delivery of Secret DataHub

  • $15,000 USD equivalent in $SCRT tokens upon the delivery of Transaction Search API for Secret

TOTAL: $35,000 USD equivalent in $SCRT tokens

About Figment

Based in Canada, serving customers worldwide, Figment is the country’s largest blockchain infrastructure, and services provider.

At Figment, our mission is to support the adoption, growth, and long-term success of the Web 3 ecosystem. This is Figment’s unique approach: we make it simple to build on the next generation of blockchain technology. We provide enterprise-grade node and staking infrastructure, the Hubble Web 3 explorer, and developer tools while also actively participating in community & governance.

We welcome feedback on this proposal and are looking forward to working with the Secret Community to accelerate the adoption of the protocol.

7 Likes

:wave:t5: Figment. Thanks for putting this proposal together!

Since I’m already familiar with the data hub product I’m mostly just curious why $35,000 is required to add support for Secret? For example we forked hubble to create puzzle and it took us a couple days to add SCRT support, the bulk of our work since has been adding new features not just adding support to an existing product. I can’t imagine just adding support for SCRT to a product that already exists is much work at all or more importantly, I can’t imagine it being worth 35k USD.

Only other point of feedback is that since the community pool fills up at such a slow rate I personally want to be careful how fast we deplete it. For this reason the Puzzle validator will be voting no on proposals that ask for above 100k SCRT for the time being. Looks like 100k scrt is right around what this is asking for with current prices but if the price goes lower then my personal stance stands for our validator at least.

Looking forward to hearing more and love the idea of having support for scrt added to data hub for a fair price.

1 Like

Wanted to throw in my 2 cents here and give some context on why I’m generally in favor.

Secret Network has the unique challenge where running nodes is significantly more complex than your run-of-the-mill blockchain.
That means that if an external provider (exchange, wallet, etc.) wants to integrate with the network, either they run their own node or they use a publicly available node. At the moment there is no production-ready, highly available APIs.
I can tell you that some significant service providers on the network are relying on self-hosted single nodes, or publicly available nodes with little redundancy for their services (due to the cost and complexity of running nodes). And we’re getting more and more service providers asking about an API to the network.

I think that having a professionally run, production public API (with good documentation and support) is a huge step towards easing the integration cost of new services, and making things less fragile.

Re: Ian’s point about pricing, I also think that cost is not only comprised of the engineering hours for integration, but also for NRE (their work in getting the product in the first place) and the value provided. In addition, I would wager that infrastructure costs are significant, and there would be some non-trivial engineering & testing in getting scalable SGX infrastructure. Could you also add what you think a more acceptable amount would be?

Personally I think the first deliverable is the most critical, with the second depending more on use-cases. For example, a transaction search API might make gathering all your historical data for tax purposes easier (but I would to hear more opinions)

4 Likes

Thanks to you both for your feedback. Cashmaney is right, but let me give a bit more details regarding the price tag.

The proposal stipulates that we will:

  1. Develop DataHub for Secret
  2. Host the service with SGX support
  3. Make that service available for free with our higher tier (200k requests / day / user) for free for 6 months meaning anyone can subscribe and benefit from it and we bare the full costs of running the infrastructure
  4. Our engineers develop and open-source the Transaction Search API which as Cashmaney said can be used for things like reconstructing your own transaction history, or for developers to provide the Tx history to their users. This is also hosted and available for free.

So overall it’s not just development costs but ongoing infrastructure costs, documentation maintenance, user support, and opportunity costs of prioritizing Secret are all included. Hope this helps!

2 Likes

Yannick, thanks a lot for this proposal. I’ll be brief because the previous comments summarize my thoughts well.

I fully agree that a robust, well-supported and trusted network API for developers and 3rd parties is crucial. It’s clearly missing from the network right now. I also find the price tag ($20k), reasonable, provided there’s some guarantee or expectation for long term support. I have not seen any mention of that, and that would be my only concern.

Other than that, I have to admit I don’t see a lot of value in the search API. It may be best to split it to a separate proposal, or remove it for now altogether.

To answer cashmaneys question I think $15,000 is the upper limit from what makes sense (in my opinion). The community pool is all we have as a community to fund things with current prices this is asking for over 10% of the community pool. It just doesn’t seem fair to let so much be taken.

If they won’t budge on price to non retail pricing then they should at least split it up because it’s still asking for way too much of the pool and it’s getting higher as the price drops…

I’d also like to see them follow the example set by chain of secrets, keplr team, puzzle, cosmostation, and the website in that they actually have work done before asking for money. In that regard this feels like going backwards. Proposals brought forth by all the parties I listed were delivering value from a product in our ecosystem before they asked for money making them more of a relationship being built instead of merely being a transaction being executed. Even cosmostation added support first.

Hi Figment Team,

Thanks for becoming involved with the SecretNetwork community in the form of a proposal and standing up developer tooling.

I have read your thread thoroughly and I think this is a perfect addition for the Secret Network.
I find that price tag of $35k USD in SCRT token fine ChainofSecrets.org would happily vote yes to this on chain if you could firstly highlight what exactly the 200k free requests would drop down to after the 6 month period. I would hope this would be around 100K requests included for free perpetually to developers which would be nice to include in the on chain proposal.
Please also highlight the future tiered membership pricing for Gateway Access & Transaction Search’s after the 200K.

1 Like

@guy , @dbriggsie After the 6 months period, users will be able to choose from our ongoing free tier which provides 100k reqs, and upgrading to the Pro tier with 1m reqs limit. Transaction Search is included in both tiers but consumes more requests per hit so it’s more for testing in the free tier.

Moonstash, unfortunately $15k would definitely not be enough for us to pursue this. We have established a certain standards on other networks and SGX enclaves are much more expensive to run than the usual setup required by our supported networks.

I am supportive of this proposal. This will be helpful for the adoption of Secret Network, making it easier to build applications and tooling for Secret Network

Also strongly in favour, this is vital. Thanks Yannick

Great to see more support for this proposal. I would also like to offer a 2nd option in which we do not include Transaction Search.

By removing the Transaction Search API, we would ask for a total of $25,000 USD for Secret DataHub, decreasing the total cost of the proposal by $10,000, or about 31,000 less tokens from the community pool at current prices.

Let me know which option we should put forth for a vote.

3 Likes

I vote for the 2nd option:)

2 Likes

I’m happy to vote yes for option 2 as long as its not over 100k scrt.

As for the comment about 15k USD not being enough I just want to say again as vital as this is, it’s not worth 10% of the community pool funds. There is also the fact ya’ll recently raised 2.5 million USD to build exactly this sort of thing so I really don’t feel that taking 10%+ of the pool is appropriate.

“This new round of capital will enable Figment to continue to invest in our best in class blockchain staking, governance and developer tools,” Gabel added.

1 Like

I am curious how the end USD amount is calculated for, is there a deliverables breakdown on an itemized basis? Just want to be careful because this is coming straight from the community pool. Love the project idea, its important to the ecosystem. Just want to make sure we can have it be accurately priced.

2 Likes

+1 for second option as first preference, and still good with either.

1 Like

Should SCRT returns to 0.70 US that would mean 4% of the current pool which seems quite reasonable for the work required.
So seems payment timing is important,

Has any movement happened on this yet?

From my perspective it looks ready to go and the price of scrt is at a good spot to make sure not too much of the pool is taken. Not that I think price will crash but I don’t see any reason to delay when you have the support you have now. If the price goes lower I’m less supportive because we do need to be careful with the pool reserves.

1 Like

Should be ready to go with Option 2, working with the Scrt team to move to an on-chain proposal but if you have any suggestions we’re ready.

1 Like

I fully support option 2 and will vote yes. Thanks for the great work and hearing my concerns even if I sound a little brash at times. :grin:

Not to seem overly upbeat @moonstash but SCRT price was over 0.75 US Sept 1.
Great new developments from members like yourself mean in the future short term that level should at least be reach.
Unit price timing means the whole package for less of the community pool than version 2 right now.
Question to ask is will this proposal drive the Token past the 0.75 short term then we the community are the winners with the value of the pool able to fund bigger & better projects.
You guys know better than me.