[S16 Proposal] DRAFT FDD Budget Request

This is the draft budget proposal for FDD requesting funding for Season 16 (1 November 2022 through 31 January 2023).

TL;DR

In Season 16, FDD will focus on building open source fraud defense tools to support the DAOs overall transition to the protocol. It will be done by building on our season 15 hackathon success of starting an engaged open data community, dogfooding our tools during gitcoin-led rounds, and finding sustainable models for continued defense for Gitcoin UI, Gitcoin Grants Program, and Community Grants Programs.

We will improve our ability to quickly build modular and composable business intelligence tools which communities can use to reduce fraud, allocate resources better, and promote learning between ecosystems on the new protocol.

In addition,

… by talking with round owners, we will find sustainable funding options for continuous fraud defense provided by the open data community.

Amount

FDD’s total Budget Request for S16 is $349,500.

This budget does not include the additional 60 days of reserves per season. A breakdown of the budget can be found at the end.

Gitcoin Season Season 14 Season 15 Season 16
Season Budget $330,000 $349,500 $349,500
Season Reserves $220,000 $233,000 $198,349 2)
Unspent Reserves % 1) 100% 100% 100%
Treasury Request $281,041 $362,500 $384,151

1) The amount of GTC requested and the value of the reserves will be adjusted based on the current market value at the time this proposal is moved to Tally using the lower of the current price or the 20 day moving average, whichever is lower.

*2) FDD received $220,000 in reserves at a GTC price of $1.95 in S14. With the GTC price of $1.66, these reserves are worth $198,349. The full amount (100%) will be rolled over to the Season 16 budget request.

Milestone Report for the past season

:point_right:For S15 the total value of prevented fraud and therefore saved capital is > $785,000:point_left:

Initiative/Dept :green_circle::yellow_circle::red_circle::black_circle: Objectives Past Season Value Delivered - Total Fraud Prevention > $785k
Objective Name
Grant Eligibility

EI
Grants Program Success
:green_circle: Gitcoin community can reliably expect that the platform is free of fraudulent grants and that main round eligibility approvals, disputes and appeals are consistently and fairly executed in a reasonable time frame. Saved > $500k via investigation & disputes

* $250k from one sybil ring

98% of Appeals settled in under 48 hours

Reviewed over 1150 grants for platform & main round

* 70 disputes & 32 appeals

Increased decentralization efforts and conducted experiment with over 100 permissionless reviewers on Ethelo to analyze for trust signals
Objective Name
Contributor Fraud

EI
Grants Program Success
:green_circle: The Gitcoin community will have confidence that matching funds are not misallocated because of sybil accounts during GR15. Mitigated $285k Fraud Tax

* 23% of contributors

* 33.5% of contributions

* 17.5% of $s contributed

Designed scalable Sybil Scoring Legos

Created first known Sybil & Non-sybil data sets

Ensured that data-sets adhere to high quality standards

Over 10 new detection analysis used

Discovered new attack vulnerability

Conducted deep dive investigations of suspicious grants

Presented internal Fraud Analysis
Objective Name
Data Empowerment

EI
Protocol Adoption
:green_circle: FDD will empower the community by enabling the crowdsourcing of raw & anonymized data and approaches to data analysis, increasing the insight discovery rate, and resulting in a plurality of Sybil detection mechanisms Hosted open data hackathon during October

* 167 applications

* 30+ submissions received including new approaches to sybil detection, dashboarding and human incentivisation.

* 3 Twitter spaces so far - Over 1,000 listens (thank you to MMM for helping on our 1st one!)

Provided data sets on time for round defense

Conducted discovery of decentralized observability

Reflections and learnings include:

* As always - more clarity and transparency helps a community grow. In this case we could have been even more clear on the judging criteria up front.

* Forum posts on Sybil detection and presentations on Sybil resistence by GitCoin team proved very useful.

* Pushing back on DMs from participants and asking them to ask in public worked well. Basic devrel stuff but still worth a reminder.

Brainstormed ideas of a future Open Data Community & started initial collaboration with community members.
Objective Name
Protocol Research

EI
Protocol Adoption
:green_circle: Insights from processes, analysis, algorithmic research, & mechanism design research provided by FDD will aid in the acceleration of development and adoption of Grant & identity protocols. Provided Passport with APU scoring model

* Consulted on trust bonus UX decisions

Consulted on hypothesis generation for identity staking

* Data collection and presentation

Post round fraud report & identity analysis

Deep research on Cost of Forgery modeling

Documented Research OS & experimentation flow

Created documentation for all algorithms
Objective Name
FDD OS

EI
DAO Organization & Financial Stability
:green_circle: FDD will seamlessly interface with Gitcoin OS in a way that increases cross workstream collaboration, executes on deliverables, and provides a rewarding contributor experience. Regularly posted FDD Review articles

Successful experimentation with 4/5 SME

Built FDD “House of Tasks” and weekly rituals around it to improve project management flow

Tracking of initiative/task velocity (metrics) started

Conducted two FDD Happiness Pulse surveys to contrast intrintic (well-being based on Positive Psychology) and extrinsic (compensation) factors

Unfortunately had to announce two departures of FDD core contributors

Core Contributors went through the Peer Review

Improved the usability and data quality FDD Contributor Management Databases in Notion

Legend
:green_circle: Success
:yellow_circle: Incomplete, will hit goal or priority change
:red_circle: Incomplete, will not hit goal
:black_circle: Canceled - out of workstream’s control

S16 Objectives and Key Results

List of S16 OKRs

Objective Name & EI Objective Outcome Description Likely Key Results/Projects/Initiatives
“What is it and how does it align with our most important work?” “What will be done” “What impact will it have?” “What might the work look like?”
Objective Name
Identity Scoring

EI
Passport Adoption/ Protocol Growth / Community Led Programs
Continuous analysis & validation of passport scores made available for round owners & technical users in partnership with the Passport team. Round owners and technical users will be able to access FDD scores and easily be able to understand how they can use FDD provided scoring to create customized sybil prevention, detection, and mitigation algorithms. Data analysis to better empower 1p1v in web 3

Creating a best possible cross-community sybil gating/weighting algorithm based on Passport in collaboration with Passport team

Creating continuous analysis & custom scoring in collaboration with design partners and technical users forking passport

Creating documentation for round owners to customize scoring based on insights about their targeted donor personas

Maintain a best in class sybil detection model based on wallet addresses
Objective Name
Modular Fraud Defense Systems (LEGOS)

EI
Protocol Growth / Community Programs
Building & making accessible reliable composable, open source software tools for round owners to prevent fraud. A well documented, functional and scalable pipeline for quickly testing, validating, and containerizing new legos will be open for use by round owners to feel confident in being able to run a fairly allocated funding round. Build & maintain oss packages for legos such that round owners could run their own sybil defense

Creating “how to build & use legos” documentation

Create actionable analysis to maintain accuracy and thresholds for sybil scoring legos

First trial run of grant fraud legos being used for dynamic intervention

Containerization for round owners to call analysis

Interviews with round owners on preferences
Objective Name
Open Data Community (ODC)

EI
Protocol Growth / Community Programs / Financial Stability
Start a regen Open Data Community in order to decentralize the creation of insights to improve the operations & protection of Grants Rounds while also helping to protect web3 from recentralization and capture at the data layer. A vibrant, useful, and sustainable OpenData community.

Vibrant: as measured by growth in Gitcoin & non Gitcoin contributions and achievements.

Useful: as measured by the extent to which Grant Round owners and others protecting and scaling web3 rely on the community for analysis, schemas, tools, and data.

Sustainable: as measured both by progress towards decentralized governance and mechanism design and by sources of funding and utility.
Conduct second ODC hackathon and achieve superior outcomes to the first hackathon in collaboration with MMM workstream.

Create a viable governance model which incentivizes non-Gitcoin participation.

Dogfood protocol by running a protocol based grant round (most likely beta-version or earlier?) supporting the open data community.

Deliver pipeline for upcoming Grant Rounds to accelerate analysis while increasing transparency.

Enable community led curation of data sources for quality and usefulness.

Collaboratively author landscape of useful tools and guides for Open Data Community.

Established Open data community standards are an unofficial quality requirement of the analyst community.

Catalyze & lead creation of a platform to host the open data community including collaboration about tools,data sources and methodologies.

Secure other founding partners for the community.

Become the most preferred community for anti-Sybil defenders across web3 in collaboration with MMM and others.
Objective Name
Grant Protocol Research

EI
Gitcoin Grants Program / Passport Adoption /
Protocol Growth / Community Programs
Prioritize and conduct research and experimentation to increase the potential of the grants protocol to empower fair resource distribution FDD increases the rate at which we approach the possibility of optimal capital allocation via a continuous research and development cycle. Maintain relationship to GPC grants protocol product owners & to support the execution & success of the product development roadmap

Identify and model sustainability mechanisms where supply and demand exists in grants rounds to create experiments leading to sustainability

Conduct research to better understand optimal capital allocation

Spearhead innovation research ie. Cost of Forgery, Identity Staking, DeSoc Pairwise Design

Run a V2 of Ethelo Deep Data Experiment to lower Round owners review costs & time

Discovery: Pluralistic Review Layer (Projecthood)

Provide easily digested data on sybil/fraud occurrence and mitigation for all grants rounds

Creating a run-book for round owners to DIY including

* Checklists

* How-to’s and educational material

* FAQs

* Links and instructions for reporting bugs and requesting features

* Contact details & Support
Objective Name
FDD Strategy

EI
Financial sustainability / DAO Organization
Find a sustainable strategy to engage FDD contributors to service decentralized grants rounds & maintain quality processes. Funding options will be available for FDD service to decentralized grant rounds in addition to a sustainable model for maintaining OSS products. Research governance models, revenue sources and pricing models, and value creation for FDD self sustainability

Explore and apply for grants outside of Gitcoin to find partners in creating a sybil free web 3

Continue FDD Review communications (technical writing) and explain what we are doing better

Utilize & expand metrics of FDD OS

Prepare data visualizations for better communication of FDD needs and results

Create pricing levels and service-level agreements for professional fraud defense services including

* Grant fraud checks before round begins

* Custom algorithm recommendations

* Dynamic recommendations for grant fraud

* Grant and contributor fraud investigations

* Procuring data sets for investigations

* Dispute & appeal settlement

* Squelching & post round Fraud Reports

* Access to Support Team

Budget Breakdown

Budget Category Description Amount USD
Core Contributors WS Leads
Product / Strategy (Joe)
Operations (Tigress)
Full Time Contributors
Research / Product (Kish)
Data Science (Omni)
Sr. Fraud Detection Analyst (Alex)
Data/DevOps Engineer (Zen)
Data Analyst (hired, to be announced)
Data Analyst (Adebola)
$256,000
Trusted Contributors FDD Review & Science SME (J-Cook)
Open Data Community (EPowell)
Analyst (Sorana)
SWE (Eric & Yogeesh)
SME Data Infra (Takuya)
$73,500
Bounties, Hackathon Open Data Hackathon $15,000
SaaS, Fees, Travel, etc. $5,000
Total $349,500

Thank you

:blue_heart: Kudo’s to everyone who supported crafting this budget request directly or indirectly! :pray:We are looking forward to your feedback to make it even better!

8 Likes

Thanks for sharing this season’s budget request.

There seems to be a disconnect for me when I read this header and the body of the budget request.

The goals for Season 16 seem heavily weighted towards experimentation and analysis with very little building planned. Are you able to provide some more details on how the goals build towards the focus for the season?

1 Like

One of our top level objectives is building Modular Fraud Defense Legos. These are open source packages for round owners or sybil hunters to utilize the same analysis code as us.

2 Likes

I am excited that FDD’s S16 budget builds on a number of the wins from S15. These wins include creating a number of new Sybil scoring LEGOS, building an open data community, and formulating service level agreements with others parts of the DAO + round managers. There’s a lot of great progress FDD is making; your efforts to share that progress with the public is putting Gitcoin at the center of the web3 - Sybil defense discussion.

I have communicated more detailed feedback via Discord, but to summarize my concerns here for other Stewards:

  • I’d like to see fewer OKRs, but with clearer links to DAO-wide objectives. It may be confusing to Stewards when objectives have more than one Essential Intent (EI) tagged to them.
  • Some of the KRs feel hard to measure and lack specificity. Similar to @lthrift’s point, research & experimentation should be in the service of building better mechanisms for Passport, Grants Hub, and Round Manager. I’d like to see measurable KRs that other parts of the DAO (not just FDD) will point to and say, “yes, that number is important for us to hit”.
  • I’m worried about workload distribution across the team, particularly the new contributors. Personally, I’d like to see each contributor taking responsibility for a smallish number of high-impact outcomes.

Overall, my worry is that this budget might not gain strong support from Stewards, not because contributors are focused on the wrong things, but because the current presentation requires a fair amount of mental effort / context to understand exactly what FDD is working on and why it’s essential for the DAO.

5 Likes

I’m a bit conflicted - as I recently became a part time contributor to FDD.

That said - I feel FDD is extremely aligned w/ the overall purpose of Gitcoin and current focus as far as I understand it. Specifically - in order to enable communities to fund their shared needs these communities need to both a) have access to the gitcoin protocol and passport and related software and b) possess the expertise necessary to protect and operate those rounds.

FDD has proactively undertaken to decentralize itself - to externalize and grow the data science and related skills it has via hackathons and a nascent community and also by packaging and distributing the anti Sybil legos it has created.

Why is this so important? It is not much of an overstatement to state that quadratic funding does not work without the sort of dual approach to Sybil resistance via passport from the GPC plus anti-Sybil analysis and related data science from FDD. The data on this point clear - and to me so too is the purpose and the strategy of FDD.

This short note isn’t intended to substitute for detailed feedback and discussion. Rather I just wanted to add my perspective on the centrality of FDD to Gitcoin’s purpose and strategy - especially during this transition to a protocol & community based future.

3 Likes

I too am supportive but would like to push for a more succinct and outcome-oriented proposal:

:white_check_mark: is Sybil resistance one of the most important things for Gitcoin to work on? Yes.
:white_check_mark: can FDD create clear value for Grants and Round Managers? Yes.
:white_check_mark: has FDD invested in new technical skills & evolving its methodologies? Yes.
:white_check_mark: is FDD moving towards greater decentralization? Yes.
:no_entry: can I close my eyes and defend why allocating $350k to FDD is excellent value-for-money? Not yet.

2 Likes

I would echo what a few others have stated above - super appreciate the value that FDD is creating for the DAO, and I love that the team is doubling down as the go-to resource on data-driven solutions to more than just sybil resistance.

However some of the outcomes mentioned here seem a bit nebulous, especially around the “Grants Protocol Research” objective, and I’m also not clear where the demand for some of these initiatives is coming from. As an example, for the project described as

“Identify and model sustainability mechanisms where supply and demand exists in grants rounds to create experiments leading to sustainability”

I’m not convinced if this is a problem the DAO is actively trying to solve/needs to prioritize at the moment; maybe I’m missing some context here?

Overall in support of this budget and the FDD team continuing to develop itself as a valuable resource to the DAO, but would love to see the budget being a better reflection of our most pressing needs.

2 Likes

Thanks so much for posting the budget.

I really appreciate the forward looking towards data availability, and also how to find a sustainable model in a Grants Protocol based world (more specifically, a permissionless protocol approach where we don’t handle sybil defense by default).

FDD is likely overly staffed given we wont be running a round (no new data to analyze or work to do to approve grants, etc.), but I also hope that this gives the team some room to really chart a path of sustainability for how sybil detection is done at scale.

3 Likes