Data Committee Charter

Data Committee Charter

Purpose

The Data Committee’s goals are to provide data and analytics to other stakeholders in the ecosystem.

Our philosophy is to deliver the 80/20 solution, allowing us to work quickly, iteratively, and prioritize for impact. We write code, make pretty graphs, deliver data-driven insights, design ETL pipelines, build tools, automate stuff, maintain databases and APIs, and provide support.

Scope of Work

The Data Committee is full-stack, covering analytics and data engineering; this covers the full pipeline of when a data point is created all the way to it ending up in the hands of end-users via whatever platform.

This involves working with people to make sure we’re capturing the right data, writing data transformation jobs to make sure the data is in a useful state, coding scripts and notebooks for analysis, presenting results, and following up to make sure our work is interpreted correctly and acted on.

Practically, this manifests as four continual deliverables:

  1. secretanalytics.xyz dashboard
  2. data requests (analysis / simulations / reports / providing datasets, etc.)
  3. data pipeline / tooling
  4. secretanalytics API and integrations (for CoinGecko, DefiLlama, devs, etc.)

Membership

Open membership. Lots of people from different backgrounds are interested in data and analytics, so all are welcome to participate as they’d like. This could be contributing to our work products, giving feedback on our dashboards, requesting new features or data to be tracked and visualized, etc. Several non-technical community members have contributed by giving feedback and brainstorming during our weekly calls.

Leads are expected to contribute to all parts of the stack as well as help scope and manage projects. I’d expect a lead to at a bare minimum have at least a few years experience working as a data scientist or data engineer and have proficiencies with: Python (SciPy stack), a major cloud platform (AWS/GCP/Azure/etc.) and basic devops, git, SQL and relational DBs (designing data warehouses and writing clean queries), writing ETL pipelines (custom packages or utilizing frameworks like dbt/apache beam/airflow/etc.), and some visualization/BI tools.

Organization

Weekly Discord calls - Thursday at 6:30PM UTC (i.e. 2:30PM EST, 1:30PM CT, 12:30PM MT, 11:30AM PT)

Committee calls involve sharing updates on recent projects, submitting data requests, talking about relevant news relating to data and analytics, prioritizing work for the following week(s), and working / brainstorming sessions.

Moving forward, all project management and tracking will be through the projects board on github.

Compensation

Total: 45,000 SCRT

Lead pay: 15,000 SCRT over 3 months | ~15-20 hours per week

Me: superfish / gmail_user

  • 7 years experience working as a data scientist / machine learning engineer
  • Analytics Committee Lead for ~10 months

Discretionary Budget: 30,000 SCRT

  • infrastructure costs (currently ~$1,100 per month)
  • paying contributors (to date I’ve paid ~$9,000 out of my own pocket)
  • I’d like to use this to pay community members that have been actively contributing and to attract external data experts to build and contribute

Any excess budget not used during the 3 month time frame will carry over into future months as we continue to develop the work product(s).

In the past, Secret Foundation has covered infrastructure costs and funded the initial work for the proof of concept of our data pipeline.

Major Goals

Within the next 3 months:

  • Improvements to data platform
    • Upgrading our ETL system
    • Adding jobs for more in-depth analysis like cohort analysis
    • Better data collection for social platforms
    • Adding tracking for the Monero bridge and IBC
  • Launch v3 Secret Analytics (soon!) and continual improvements

We will be moving towards open-sourcing our entire codebase, which is a goal for 2022.

Reason for renaming Analytics → Data

Analytics starts with data engineering. This is a broader title that more accurately captures the full scope of work and activities.


Happy to hear any feedback and answer any questions :sunglasses:

8 Likes

I love the sound of your proposal.

I like what you do and I love analytics.

However, this is another 45,000 SCRT that will impact price movement. Instead of tracking the Monero bridge or IBC, could we track and predict the impact of spending in general? Secret Swap is just a single dapp that has created a lot of new jobs and governance requests on the network resources but I’m not sure the spending is warranted. Yet.

I feel like we need an ecosystem of 10 dapps before we have things to measure and the 80/20 rule should apply in terms of impact. We can’t just create jobs for the sake of creating jobs and we need to pay developers and artists to use NFTs and create products.

But go ahead, pay for it if we plan to scale this project and have analytics. I’m biased because the same people requesting funding had no constructive input on the fact I had apps with a few million users and products. I just feel the need to “bombard” my two cents seeing these projects get funded instead.

If you’re funded or want funding from the community vote, let the back scratching ensue.

2 Likes

Happy to support this proposal

2 Likes