This is the integrated funding proposal for the Fraud Detection & Defense (FDD) Workstream requesting funding for Season 15 (1 August 2022 through 31 October 2022).
Essential Intents
In S15, FDD has components of our work that focus on all four of the DAO’s Essential Intents - though our priorities will be on the following EIs in the given order:
- Grant Program Success
- Protocol Adoption
- DAO Organization & Financial Sustainability
More Information on the Essential Intents can be found in this post.
TL;DR
In Season 15, FDD aims to provide the Gitcoin community with the knowledge that matching funds are allocated as optimally as possible within the given set of assumptions and constraints. One focus will be operationally executing the fraud detection & defense needs of the Gitcoin main/Ethereum round & any other community moderated rounds. The second is assisting Gitcoin Product Collective with operational insights, quantitative analysis, and algorithmic/mechanism design research.
Based on what we learned in Season 14, we will improve our communication during the season to remove confusion regarding FDD priorities, purpose, and alignment with GitcoinDAO.
For more context regarding our vision, the current state of FDD and more please refer to the
supplement document.
Amount
FDD is requesting $349,500. This budget does not include the additional 60 days of reserves. Breakdown of the budget per budget category and initiative can be found at the end.
| Gitcoin Season | Season 15 | Season 14 | Season 13 |
|---|---|---|---|
| FDD Season Budget | $349,500 | $330,000 | $596,295 |
| FDD Season Reserves | $233,000 | $220,000 | $596,295 |
| Unspent Reserves % 1) | 100% | 100% | 100% |
| Treasury Request 2) | $362,500 | $281,041 | $879,590 |
1) The amount of GTC requested and the value of the reserves will be adjusted based on the current market value at the time this proposal is moved to Tally using the lower of the current price or the 20 day moving average, whichever is lower.
2) The Season 15 treasury request amount is an estimate using the price of $2.99 GTC to calculate the total of the reserves rolled over from Season 14.
Milestone Report
Legend
Success
Incomplete, will hit goal or priority change
Incomplete, will not hit goal
Canceled - out of workstream’s control
| Initiative | Objectives Past Season |
Value Delivered |
|---|---|---|
| Grant Eligibility Executes the critical functions associated with grant eligibility reviews, disputes, and appeals. |
Reviews and approve/deny new grant applications in under 48 hours Ensures all disputes and appeals are heard and executed In depth review of suspect grants to ensure minimal improper matching allocation Ensure a proper functioning integration from Grants 1.0 backend to Ethelo Provide data to be used in evaluating reviewer decision making issues at scale |
Over 3,000 evaluations at a 39% lower cost of review Over $80k reallocated/squelched based on suspected Sybil/Fraud investigations Average time for appeals was brought below 96 hours allowing for in-round results We used Ethelo to acquire over 400 reviews of the main round disputes to compare to our results with 3 “classes” of reviewers including anyone, DAO Contributors, and trusted reviewers |
| Data Operations Data center which handles access control, ETL, and process automation. |
Runs the end-to-end process used in production to detect sybil accounts Supports other FDD dev ops needs & liason with Gitcoin DevOps team Executes post round data analysis for grants round governance brief Identifies continually evolving sybil behavior to be codified in the detection system Statistically validates the algorithm and directly squelching sybil users Continually trains the ml algorithm to think like the community, not the engineers Builds containers and automations to make processes consistent and easy to learn |
Identified multiple new signals It was run every 24 hours throughout the round keeping estimations more in line with final results to improve UX Supported Ethelo integration, Community Intelligence, and helped Grants Ops round partner experiment with Ethelo (Not advised by FDD, but supported with DevOps help) GR 14 Governance Brief |
| Community Intelligence Data analysis, model prototyping & validation, and sybil strategy center. |
Data analysis to find the best models for passport sybil defense Identifies new opportunities (low hanging fruit} for additional data-driven sybil detection strategies which leverage the data already collected Fulfills opportunities & requests for further collection and organizing of data Prototyping new features and models before pushing them into production Kyle/Joe agreement to put community model on hold Open source feature engineering pipeline turns behavior patterns into passport stamps Kyle/Joe agreement to put community model on hold |
Analysis of onchain behavior Done. Presentation for GPC on 8/3 Identified 3 new models to lower human evaluator time to insight Designed contributor DNA Provided recommendations to GPC Passport team for high signal stamp opportunities Ran data analysis for GR14 contribution insights across rounds |
| FDD OS Core operations & communications supporting FDD squads and contributors. |
Funds and directs the roles requested by DAOops DAO facing collaboration point to help communicate FDD role in the DAO Building on last season deep dive understanding of Is FDD successful in it’s mandate Facilitates and synthesizes bottoms-up and top-down input to draft FDD outcomes Ensures that contributors are able to focus on the primary problems to be solved Creates accountability for outcome owners via strategic cadence and collaborative decision making processes which are seen as legitimate Building dashboards for stewards & FDD with FDD round over round metrics Put on the backburner due to delay of Kyle/Joe alignment. Removes key person issues from the workstream: Not realistic this season with the even/overs from CSDO |
Roles were implemented and executed peer reviews, compensation evaluation, and onboarding training. FDD Review is a series of posts started to provide honest critique WS Retros, Outcome Discovery, and Budget Discovery brought bottoms-up ideas together with GitcoinOS budget planning. Async working agreement lowered the required meeting hours while Work-in-Public sessions gained voluntary participation helping to spread context and share methodologies. A decision making process agreement was passed and used. |
| Passport Truthseekers Pure research finding insights & sybil stories. |
Creates patterns to be engineered for Passport to allow for specific GitcoinDAO uses Provides model for grant review stamps to be sent to user passports: Kyle/Joe agreement to hold this budget until fully aligned with GPC. Ended up not starting this squad. Moved work to Community Intelligence and Data Operations. |
S15 Objectives
Based on the streamlining and aligning efforts of our work preceding S14, we consolidated our initiatives once more and will focus on 5 key objectives for Season 15.
In addition we are outlining our Always-on Commitments as well as Shared S15 Commitments with other workstreams and/or teams of GitcoinDAO in the supplement doc.
List of S15 Objectives
| Initiative (Essential Intent) |
S15 Objective | Metrics | Key Results Likely Deliverables / Projects |
|---|---|---|---|
| Grants Eligibility (EI: Grants Program Success) |
The Gitcoin community can reliably expect that the platform is free of fraudulent grants and that main round eligibility approvals, disputes and appeals are consistently and fairly executed in a reasonable time frame. | * Average Time to Review < 48 hrs * Average Time to Appeal Resolution < 96 hours * Benchmark Appeal Success Rate * Amount Saved by Investigations * Output an FDD eligibility score for every new grant application |
- Gather data for scoring experimentation - Partner with Grants ops to improve flow - Create a a transparent and priortized backlog of observations for GPC |
| Contributor Fraud (EI: Grants Program Success) |
The Gitcoin community will have confidence that matching funds are not misallocated because of sybil accounts during GR15. |
* Provide understanding of sybil impact ** Fraud Tax Estimate ** Fraud Tax without Trust Bonus ** Fraud Tax without Squelching * Average Passport Score of Sybil Acct * Estimated Sybil Incidence / Flagging Sybil Incidence * Inter-reviewer Reliability |
- Add heuristic escape hatches to SAD algorithm - Improve the accuracy of the SAD algorithm - Create framework of reports which can expedite behavior classification - Execution of always on functions |
| Data Empowerment (EI: Protocol Adoption) |
FDD will empower the community by enabling the crowdsourcing of raw, anonymized data and approaches to data analysis, increasing the insight discovery rate, and resulting in a plurality of Sybil detection mechanisms |
* Open API Endpoint * Useful Insights from the Community * Data Coverage: available / existing data * Data availability: failed/total requests * Ease of access and usage * Data Contest/Hackathon Participation * Number of useful data sets * Number of users requesting data |
- Finish cloud setup including querying & processing for ETL pipelines, ML ops flow, and a well documented data repo - Create a plan to improve the community access to data without being a service provider - Build a front end for participatory data science - Build an API for dashboarding and automated reporting - Host a post grant round data hackathon rewarding analysis deemed useful by judges across other workstreams & FDD |
| Protocol Research (EI: Protocol Adoption) |
Insights from processes, analysis, algorithmic research, & mechanism design research provided by FDD will aid in the acceleration of development and adoption of Grant & identity protocols. | * Passport Stamp Recommendations Approved by GPC Passport * Trust Bonus to SAD Gap * Number of Valuable Insights Generated for GPC/MC * Key Partner Satisfaction Score ** Passport - Leon ** Grant Hub - Michelle ** Identity Staking - Brent |
- Passport Collaboration: Recommendations for high signal stamps, a scoring algorithm for S15, and a post round analysis - Identity Staking Collaboration: Specifying collaboration now with Moonshot team - Cost of Forgery Discovery: Design a scoring algorithm which can provide sybil guarantees up to a $ amount of goods received and supporting math proofs - Grant Hub/Round Manager Collaboration: Provide at minimum 3 scoring outputs from FDD grant reviews - Review Protocol: Identify incentive mechanisms for sustainable sybil hunting |
| FDD OS (EI: DAO Organization, Financial Sustainability) |
FDD will seamlessly interface with Gitcoin OS in a way that increases cross workstream collaboration, executes on deliverables, and provides a rewarding contributor experience. | * Execution Health: # of healthy OKRs * Compensation/Happiness Ratio * Initiative Velocity: Task/Initiative by Weeks Passed |
- Build FDD Contributor Dashboard with metrics - Build FDD House of Tasks to better track progress - Establish mid-season OKR health check-ins - Regulary post new articles on Gitcoin’s Gov Forum - Have a seasonal, vibey, shared and celebrational ritual to help facilitate L.E.A.R.Ning from our mistakes - Run CX Survey / Satisfaction Review - Conduct a facilitated process with Stewards to guide FDD priorities - Create visualized product/FDD roadmap for upcoming seasons |
Budget Breakdown
View: USD per Initiative
| Initiative | Amount USD | % |
|---|---|---|
| Grants Eligibility (Grants Program Success) |
$63,000 | 18.0% |
| Contributor Fraud (Grants Program Success) |
$84,250 | 24.1% |
| Data Empowerment (Protocol Adoption) |
$80,000 | 22.9% |
| Protocol Research (Protocol Adoption) |
$54,250 | 15.5% |
| FDD OS (DAO Organization, Financial Sustainability) |
$68,000 | 19.5% |
| Total | $349,500 | 100.0% |
View: USD per Category
| Budget Category | Description | Amount USD |
|---|---|---|
| Core Contributors | 2 WS Leads Product Manager (Joe) Project Manager (Tigress) 4 Initiative Leads Grants Eligibility (Zer8) Contributor Fraud (SirL) Protocol Research (Kish) Data Empowerment (Zen) 1 Technical Lead Data Science (Omni) 1 Data Analyst (Adebola) |
$238,500 |
| Trusted Contributors | Technial Writer & Science SME (J-Cook) DeSoc SME (Carl) Data Science SME (Takuya) Analyst (Sorana) SWE (Eric & Yogeesh) |
$52,000 |
| Hiring, Bounties, HIT Roles | Data engineer New Hire - $25k Bounties & SaaS - $7,5k Human Evaluations (SAD) - $5k Grant Reviews - $15k Accounting - $6,5k |
$59,000 |
| Total | $349,500 |
question.
We like to highlight that alignment efforts made by other workstreams, especially GPC, since last budget season have made drastic improvements for us in all the mentioned areas. We do feel the results of better communication about these issues. 
question.
