[S16 Proposal] INTEGRATED FDD Budget Request

This is the draft budget proposal for FDD requesting funding for Season 16 (1 November 2022 through 31 January 2023).

TL;DR

In Season 16, FDD will focus on building open source fraud defense tools to support the DAOs overall transition to the protocol. It will be done by building on our season 15 hackathon success of starting an engaged OpenData community, dogfooding our tools during gitcoin led rounds, and finding sustainable models for continued defense for Gitcoin UI, Gitcoin Grants Program, & Community Grants Programs.

We will improve our ability to quickly discover and build a modular fraud defense system (consisting of open source “LEGOS”) that communities and round owners can use to reduce fraud, allocate resources better, and promote learning between ecosystems on the new grants protocol.

In addition,

… by talking with round owners, we will find sustainable funding options for continuous fraud defense provided by the OpenData community.

Amount

FDD’s total Budget Request for S16 is $349,500.

This budget does not include the additional 60 days of reserves per season. A breakdown of the budget can be found at the end.

Gitcoin Season Season 14 Season 15 Season 16
Season Budget $330,000 $349,500 $349,500
Season Reserves $220,000 $233,000 $172,615 2)
Unspent Reserves % 1) 100% 100% 100%
Treasury Request USD $281,041 $362,500 $409,885
Treasury Request GTC 267,899 GTC

1) The amount of GTC requested and the value of the reserves will be adjusted based on the current market value at the time this proposal is moved to Tally using the lower of the current price or the 20 day moving average, whichever is lower.

*2) FDD received $220,000 in reserves at a GTC price of $1.95 in S14. With the GTC price of $1.53 (Nov 22nd 2022), these reserves are worth $172,615. The full amount (100%) will be rolled over to the Season 16 budget request.

Milestone Report for the past season

:point_right:For S15 the total value of prevented fraud and therefore saved capital is > $785,000:point_left:

Initiative/Dept :green_circle::yellow_circle::red_circle::black_circle: Objectives Past Season Value Delivered - Total Fraud Prevention > $785k
Objective Name
Grant Eligibility

EI
Grants Program Success
:green_circle: Gitcoin community can reliably expect that the platform is free of fraudulent grants and that main round eligibility approvals, disputes and appeals are consistently and fairly executed in a reasonable time frame. Saved > $500k via investigation & disputes

* $250k from one sybil ring

98% of Appeals settled in under 48 hours

Reviewed over 1150 grants for platform & main round

* 70 disputes & 32 appeals

Increased decentralization efforts and conducted experiment with over 100 permissionless reviewers on Ethelo to analyze for trust signals
Objective Name
Contributor Fraud

EI
Grants Program Success
:green_circle: The Gitcoin community will have confidence that matching funds are not misallocated because of sybil accounts during GR15. Mitigated $285k Fraud Tax

* 23% of contributors

* 33.5% of contributions

* 17.5% of $s contributed

Designed scalable Sybil Scoring Legos

Created first known Sybil & Non-sybil data sets

Ensured that data-sets adhere to high quality standards

Over 10 new detection analysis used

Discovered new attack vulnerability

Conducted deep dive investigations of suspicious grants

Presented internal Fraud Analysis
Objective Name
Data Empowerment

EI
Protocol Adoption
:green_circle: FDD will empower the community by enabling the crowdsourcing of raw & anonymized data and approaches to data analysis, increasing the insight discovery rate, and resulting in a plurality of Sybil detection mechanisms Hosted open data hackathon during October

* 167 applications

* 30+ submissions received including new approaches to sybil detection, dashboarding and human incentivisation.

* 3 Twitter spaces so far - Over 1,000 listens (thank you to MMM for helping on our 1st one!)

Provided data sets on time for round defense

Conducted discovery of decentralized observability

Reflections and learnings include:

* As always - more clarity and transparency helps a community grow. In this case we could have been even more clear on the judging criteria up front.

* Forum posts on Sybil detection and presentations on Sybil resistence by GitCoin team proved very useful.

* Pushing back on DMs from participants and asking them to ask in public worked well. Basic devrel stuff but still worth a reminder.

Brainstormed ideas of a future OpenData Community & started initial collaboration with community members.
Objective Name
Protocol Research

EI
Protocol Adoption
:green_circle: Insights from processes, analysis, algorithmic research, & mechanism design research provided by FDD will aid in the acceleration of development and adoption of Grant & identity protocols. Provided Passport with APU scoring model

* Consulted on trust bonus UX decisions

Consulted on hypothesis generation for identity staking

* Data collection and presentation

Post round fraud report & identity analysis

Deep research on Cost of Forgery modeling

Documented Research OS & experimentation flow

Created documentation for all algorithms
Objective Name
FDD OS

EI
DAO Organization & Financial Stability
:green_circle: FDD will seamlessly interface with Gitcoin OS in a way that increases cross workstream collaboration, executes on deliverables, and provides a rewarding contributor experience. Regularly posted FDD Review articles

Successful experimentation with 4/5 SME

Built FDD “House of Tasks” and weekly rituals around it to improve project management flow

Tracking of initiative/task velocity (metrics) started

Conducted two FDD Happiness Pulse surveys to contrast intrintic (well-being based on Positive Psychology) and extrinsic (compensation) factors

Unfortunately had to announce two departures of FDD core contributors

Core Contributors went through the Peer Review

Improved the usability and data quality FDD Contributor Management Databases in Notion

Legend
:green_circle: Success
:yellow_circle: Incomplete, will hit goal or priority change
:red_circle: Incomplete, will not hit goal
:black_circle: Canceled - out of workstream’s control

S16 Objectives and Key Results

List of S16 OKRs

Objective Name & EI Outcome Likely Key Results & Projects Cross Workstream Goals
“What is it and how does it align with our most important work?” “What impact will it have?” “What might the work look like?” “What other workstreams’ outcomes does this work support?”
Objective
Passport Scoring

EI
Passport Adoption
Passport readers, round owners and integrators will be able to quickly and easily determine the best algorithms to use for sybil defense for their unique use case. Customized Sybil defense tools will be available for 3 Round Managers using Passport DIDs / scoring algorithms.

Our continuous discovery process will uncover new threats and develop analysis methods to quickly flag sybils. This provides actionable insights for building the modular fraud defense systems (LEGOS).

Provide the data quality control for reference data sets needed to drive a flourishing data science community working to solve sybil resistance.
* Scoring / Registry as Service

* In-house scoring innovation across use cases

* Enable Passport for Sybil Defense

* Enable RO to set programmatic eligibility requirements

* Passport stamps and different use cases across different communities

* New Donor scoring algo
Objective
Modular Fraud Defense Systems (LEGOS)

EI
Grants Program Success
Round owners will be able to verify and reproduce FDD results and/or run the analysis themselves to defend their rounds from fraud. We help round owners to protect matching pool funds from fraudulent allocation by building 3 grant legos so they can intervene early in the round.

Our grant lego building process draws upon the results from the discovery process and includes refining & fine-tuning of code, containerizing for services, and documenting for lego users.

Create an open source package for round owners to use legos to defend their rounds from fraud.

Discover round owner needs and build supporting architecture to enable continuous servicing and updating of legos.
* Scoring / Registry as Service

* In-house scoring innovation across use cases

* Enable RO to set programmatic eligibility requirements

* Enable Passport for Sybil Defense

* Data accessibility / community building
Objective
OpenData Community (ODC)

EI
Grants Program Success / Protocol Growth
A vibrant, useful, and sustainable OpenData community.

Vibrant: as measured by growth in Gitcoin & non Gitcoin contributions and achievements.

Useful: as measured by the extent to which Grant Round owners and others protecting and scaling web3 rely on the community for analysis, schemas, tools, and data.

Sustainable: as measured both by progress towards decentralized governance and mechanism design and by sources of funding and utility.
Conduct a second open data hackathon to derive insights to drive the viability of the grants protocol by consistently providing crowdsourced insights for sybil defense.

Find partners to establish OpenData community standards, improve accessibility, & incubate projects to increase decentralization in the data access points for analysts to validate and reproduce algorithmic policy decisions used for grants program fraud defense.

Discovery of a self-sustaining model for on-chain governance via a fair launch using grants protocol.
* Scoring / Registry as Service

* Basic in-round stats on projects

* Data accessibility / community building
Objective
Grant Protocol Research

EI
Protocol Growth
FDD increases the rate at which we approach the possibility of optimal capital allocation via a continuous research and development cycle. Research to reduce manual review time & cost and increase optionality for round owners to provide legitimacy and credible neutrality in grant eligibility decisions.

Provide ad-hoc data analysis and research to satisfy internal needs of GitcoinDAO.
* Enhanced Quadratic customization

* In-house scoring innovation across use cases

* Direct grants mechanism

* Enable v1 Quadratic Voting Mechanism

* Data accessibility / community building

* Basic in-round stats on projects

* Scoring / Registry as Service

* Enable RO to set programmatic eligibility requirements

* Enable Passport for Sybil Defense

* Appeals Process Tooling
Objective
FDD Strategy

EI
Financial sustainability / DAO Organization
Funding options will be available for FDD service to decentralized grant rounds in addition to a sustainable model for maintaining OSS products. Research governance models, revenue sources and pricing models, and value creation for FDD self sustainability

Continue writing FDD Review governance posts. Share our insights, knowledge and expertise by telling the story of fraud.

Continuously seek balance between financial workstream sustainability and great contributor experience.

Provide operational systems, practices, data and information to ensure FDDs operations meet our standards in terms of quality and user-centeredness.
* Data accessibility / community building

* GTC utility in core protocol

* Protocol business model

* Basic in-round stats on projects

Budget Breakdown

Budget Category Description Amount USD
Core Contributors WS Leads
Product / Strategy (Joe)
Operations (Tigress)
Full Time Contributors
Research / Product (Kish)
Data Science (Omni)
Sr. Fraud Detection Analyst (Alex)
Data/DevOps Engineer (Zen)
Data Analyst (Bella)
Data Analyst (Adebola)
$256,000
Trusted Contributors FDD Review & Science SME (J-Cook)
OpenData Community (EPowell)
Analyst (Sorana)
SWE (Eric & Yogeesh)
SME Data Infra (Takuya)
$73,500
Bounties, Hackathon Open Data Hackathon $15,000
SaaS, Fees, Travel, etc. $5,000
Total $349,500

Thank you

:blue_heart: Kudo’s to everyone who supported crafting and improving this budget request!:pray:

6 Likes

I’m supportive of FDD’s budget request for S16.

I also appreciate the work of @DisruptionJoe and @tigress over the past two weeks to solicit and incorporate feedback from across the DAO and reflect it in this new proposal.

It’s easy for FDD’s contributions to be overlooked, especially during a bear market, when every cost and budget request deserves scrutiny. That said, I truly believe that decentralized identity and Sybil resistance are critical to the credibility of quadratic funding (“now us”) and as primitives for regenerative society (“future us”). FDD’s proposal for S16 has my support because it will continue to position Gitcoin as a leading voice on decentralized identity, engage a broader community in developing Sybil defense mechanisms, and start offering new utility for Grants and Round Managers on the protocol.

7 Likes

Supportive of the FDD budget. I’ve read up on the comments and replies in the draft budget and echo the comments by Carl above. In the context of the ‘light’ budget approach as decided in CSDO and combined with the earlier live session we had to review the draft commitments and budget I’ve gathered enough feedback to vote yes here.

2 Likes

@tigress @DisruptionJoe - thanks. I suspect these exercises are not the most fun part of the job, but I do appreciate the work you put into summarizing S15 and mapping a plan into S16.

My observations when comparing S15 results and to S16 plans:

  • It looks like FDD has made progress in telling the FDD “story”. You have a technical area and given few stewards volunteered to review the FDD budget, socializing your story is important work. Well done.
  • In S14, FDD squelched $80K in fraudulent activity. In S15 that jumped to $785K. It was not really a cost avoidance so we cannot attribute an ROI to the work, but the intangibles (trust in the brand) is something to be celebrated systemically. Perhaps this story feeds into the MMM Impact Report that @alejandro is working on or the content pipeline @mathilddv is building for blog content? I really like the use of twitterspaces in relation to the hackathon and think that is headed the right direction.

Regarding the Open Data Community (ODC) objective - I find this initiative compelling:

The creation of a web3 decentralization measurement is a public good in and of itself. And if socialized well, it could be a significant forcing function of change. I am not sure how funding and utility work into that but I am interested to see the outcomes and add any help I can along the way.

Related to the above, I was in two different calls this past week related to lamenting the lack of thought / tools in this area for startups. As we develop our own plans, might be interesting to link this to the work Gary is doing on " “Buidl in Public”" - because it looks like there is a need.

This is exciting - and will be great to prove out a “protocol business model” . The term “business model” does not need to have a negative connotation, it only describes how we link goals > to actions > to the allocation of resources. In this case, how the market will allocate resources (funding) back to the services you are developing. Thank you for thinking about this. As I think around the corner to an extended or worsening bear market, this could come into sharper focus. For everyone.

3 Likes

And I am supportive of this budget

2 Likes

This proposal is now on Snapshot

3 Likes

Brand new Steward, lacking context. Apologies if questions and comments are naive while I get my footing

I am still learning a lot about Gitcoin’s FDD work stream and the need for sybil defense & decentralized identity for the grants protocol & Passport—last week’s DAO Vibes call with @ccerv1 was a helpful start!

The S15 Total Fraud Prevention of 785K would seem to easily justify this budget request—unclear why @shawn16400 comments that this can’t be attributed to ROI?

Could someone provide a link to additional context for the OpenData Community? I see this listed as a goal and also as a contributor—is this a contractor, event, new community, or new initiative? A bit confused but eager to learn more. Thank you!

gm @GClayMiller

thanks for getting in touch with us.

I am also wondering about the same here. Maybe @shawn16400 can you please double click for us on what is meant by your original comment:

OpenData community - it is in its very early developmental stage. @epowell101 is massively driving it and I am sure he is able to answer any specific questions you are having. Read more: https://twitter.com/OpenDataforWeb3
Gitcoin starts and supports the OpenData Community
# OpenData Community Hackathon Results
https://twitter.com/gitcoin/status/1592894671969456129?s=20&t=wFxTr4axgVD5_CAe228pyQ

Hope that helps!

1 Like

Thank you @ccerv1 that is beautifully said!

It means a lot to us, that we have your support and you are making us feel seen within the Stewards community and beyond!

2 Likes

The open data community provided these outputs as an experimental component this last season. # OpenData Community Hackathon Results - #4 by epowell101

Finding founding partners will allow the entity to be neutral like the Linux Foundation. The open data infrastructure it incubates will include open source and decentralized components a service provider like Dune Analytics or Flipside Crypto would use, but has no business model. This is how the open data community will attempt to stop centralization via a “death from 1,000 cuts”. We will do it by preventing one cut at a time.

This burden is all for all of web 3. Now that we are launching the protocol, we face this challenge directly. We could spend DAO funds to build this infrastructure for everyone, but the open data community is a better strategic move.

ODC will allow us to not only share the cost burden, but allows for a non-competitive partnership with all the data indexing services. This lets us aggregate data sets for analysis on specific use cases. By crowdsourcing analysis, the analysis quality improves and its cost goes down. At the same time, because this is where the analysts “hang out”, the vendors are willing to pay to participate.

Lastly, this is an interesting opportunity for us to dogfood the protocol. The project has had it’s initial payments go out via Quadratic Voting. We would love to fair launch this community with a bicameral house where participation is the only way to earn governance. This can all happen onchain now that we have the grants protocol and aqueducts to replace the oracle components that would have been needed before.

4 Likes

this is very helpful, thank you @DisruptionJoe !

2 Likes

Very helpful, thanks @tigress :slight_smile:

2 Likes

No longer part of the team as some people may know(or not), but cannot abstain from commenting on the budget.

:dart: Steward view:his budget is considerably smaller than the other WS budgets.

1.Overtime FDD appears to be have been faster in organizing, evolving, treasury management and now the team consists of data scientists, data analysts and mostly people with huge data/anti-fraud experience.

2.Sometimes it feels that the workstreams that are more efficient are put even under more “pressure” to evolve further. As a former team member I can honestly say that the FDD has navigated decentralized decision making and the team sessions we had were always designed in a way that lead to innovation.

3.The efforts of the FDD team and the complexity of the subjects FDD is working on: sybil defense(unsolved problem), advanced fraud detection in web3(unsolved problem) are hardcore.

1,2, 3 Congratulations to @DisruptionJoe and the team! And if all of that isn’t enough…just think about the fact that I am no longer part of this team and still believe they should be funded(lol) :smiley:

3 Likes

hey @tigress and @GClayMiller here was the original comment (for readablity)

I asked @DisruptionJoe or @epowell101 about this earlier this week someplace (I think - this week was kind of a blur). For anyone following, a quick explainer: A simple case of measuring the return on an effort is, if I spend $1M on a project, how much value do I expect to receive for that effort, or Return on Investment (ROI)? If the result of spending $1M, I earn $1.5M, my ROI is (1.5M-1M)/1M = a ROI of 50%.

Alternatively - we can look at cost avoidance. How much money am I NOT going to have to spend as a result of that investment. If I spend $1M on a an energy savings project and as a result I reduce my fuel spending by $1.5M the ROI is also 50%.

For all you Project Mgmt/cost accounting-types, yes I know I am skipping over WACC and TVM, but deal with it :slight_smile: .

The issue is that for the service FDD delivers, ROI is more opaque. Gitcoin does not receive any more income, nor does it avoid any costs for squelching $785K in fraud. So my point it, I really want to use ROI - because a back-of-the-napkin ROI would be (785-362)/362 = ~116% - which is impressive, especially when compared to the prior period results ($80K) but the ROI measure really does not really work here. (Yes, I know multiple periods may have to be considered, the total cost is not fully baked in, and a bunch of other caveats).

To be clear. I am not attempting to diluting the value of FDD - in fact it is the opposite. I am attempting to pointing out 1) it is not obvious how to measure the intangible benefit FDD delivers 2) the value delivered is pretty darn impressive and 3) when compared to prior periods, the velocity of value-add is huge.

4 Likes

Thank you for clarifying the support!

I was thrown off by the number you have for GR14 of $80k.

The fraud tax as quoted there is comprised of:

  1. The delta of match allocations before and after squelching.
  2. The grants which were denied allocation via disqualification.

It does not include:

  1. The delta of better match allocation from Trust Bonus
  2. The preventative effect of simply having fraud defense

I’d say we average around $650k - $800k pretty consistently on the measurable side since around GR12.

Like you point out, these services don’t provide ROI to any individual entity - clearly they are a public good. Therefore, as we move to the grants protocol, I think there should be a grant in each round to compliment what the DAO subsidizes for the communities which run grants rounds.

3 Likes

Thank you for the budget proposal.

I believe FDD is doing good work, though I still am confused quite often about the details of what they do. The budget is slimmed down and I like it. I still think it could do with a tiny bit more slimming down – but I feel confident enough to vote YES here.

Looking forward to seeing how the OKRs will be executed on and for the retrospective during the next season.

Why was the FDD vote only open for 3 days? Sadly I was traveling and assumed it would be open as long as the other PGF work stream as that vote was opened before it… so I missed the chance to vote on it.

Is there a reason that FDD is open for 3 days, some votes were open for 5 days and some votes are open for 7 days? Is there no standard?

Well, I will post this anyway. FDD does great work, and is one of the slimmest work streams we have, Great work @DisruptionJoe and team!

That said we need to make cuts and the only way the stewards can signal it is by voting no.

So I am would have voted no on this work stream if the vote was still open and will be voting no on all other work streams until we get our spending under control. This is the only thing I can do, and I hope other Stewards will follow suit. We have given strong warnings over the last few seasons and they don’t seem to be enough.

We cannot spend this much money in this market.

I am happy to work with the Gitcoin Hiring team to explain how Giveth finds intrinsically motivated high caliber contributors in lower income countries to keep our budget at 1/10th of the cost.

1 Like

Hi @griff - thanks for the question. Our the Gitcoin process stipulates the vote has to be open for five days, by practice the we often extend the vote if it falls across a weekend. When the FDD proposal was posted up, the snapshot default date of three days was used by mistake. The mistake was noticed on day two when 4.6M votes had already been cast. @Disruption Joe | Gitcoin posted up a note on the stewards discord forum indicating the mistake.
https://gov.gitcoin.co/t/gitcoin-dao-governance-process-v3/10358/22

Calculations of amounts/requests have been updated with the current GTC price.