Thanks @epowell101
While i do see merit in funding this project, I would like to request a quantifiable (MoS) measure of success for the following:
1.
Could you please add an overview the funding that has already gone into this project and the returns it has generated?
Thank you for reviewing the proposal.
It is hard to quantify the āincreased marketā metric. Sybil defense isnāt an optional component of these funding rounds and data analysis is necessary to tune Passport. Taking an open-source approach can be both more cost-effective and credible due to the transparency than solely relying on Gitcoin-paid teams and workstreams such as the former FDD.
Regarding funding into the project, we have had two hackathons since being mostly or entirely spun out of Gitcoin. These were funded in part by Gitcoin and in part by other stakeholders including Ocean Protocol and Pocket Network and Supermodular; and 95% of those sponsor payments went directly out to hackathon winners; there is no other source of funding as of yet other than the few months I had as a part time contributor to FDD (which was hugely beneficial). It is again a little tricky to measure the hard ROI of a hackathon however the models, datasets, dashboards, and data scientists that these hackathons helped to gather are now being used in the analysis of a few recent funding rounds, which is one of the benefits of the OpenData Community although not the only one. As the number of these rounds increases it becomes all the more important that a credibly neutral, open, and qualified set of data scientists, algorithms, dashboards, and datasets are available to enable success including in non-Gitcoin-operated rounds, which we all hope will proliferate.
Thank you again for reviewing the proposal and for your analysis.
One comparison - it could easily cost $65,000 in commissions paid to a recruiter to hire a single data scientist and there are dozens that have been introduced to our challenges via the OpenData Community, a handful of which are helping Gitcoin now as volunteers.
The ODC was extremely generous to help with the Beta Round matching calculations and Sybil analysis at no cost. Many of their tests, some from the FDD days, some from recent ODC hackathons, played a part in catching Sybil rings. Iām not sure what the best solution is yet, but Iād like to emphasize how important it is to have robust on-chain analytical tools in order for Quadratic Funding to scale on a decentralized protocol.
I am generally in support of this proposal, and/or Gitcoin + ODC collaborating in whatever capacity makes the most sense. I would like to see some more concrete deliverables described for this cost, and I do have some questions (what are legal expenses for?) but overall it is really important we continue funding and building open source Sybil defense tools.
Hey guys, Iām Stefi, Iām not an active writer here, more of a reader but would like to leave my testimony.
I got to know Gitcoin through ODC, like other people from our community that like me would like to onboard the web3 space but with a data and not a developer background dont find a lot of places to do it.
I understand all your questions about ROI and I agree that they should be made, just want to leave some thoughts around non-tangible returns too.
The community have run hackathons and gather a vast number of submissions that helps to guide development toward solutions to this really complex problem that is sybil attacks. We are on an eminence of, besides all the intangible and tangible assets already built, building more tangible ones with all the past experience.
Examples have been mentioned here like trained and skilled data scientists ( flourished on ODC) helping out on round evaluations, curated data sets, but there are others worth to mention like:
a website that gather resources to onboard and inform whoever wants to understand the subject of sybil attacks.
Onboardings like when I went to EthSamba to talk about the danger of sybil attacks to public funding and to do that I explained Gitcoin, Allo Protocol and QF to more than 50 people , including stakeholders from Lens Protocol, Chainlink and Celo.
And more recently Sandbox projects, including the Mirall0x app which Iām a founder and has been developed as an open source application to integrate the grants owner tools to analyze grantees projects and fight sybils even before the donations start . The main user persona of this tool is not only Gitcoin, whom some members have already seen a MVP and offered great feedback, but the future Grants Stack and Allo user too.
This is a tangible example of an outcome coming from the existence and momentum of the community that will indirectly aggregate value to Gitcoin ecosystem.
With all that said I ask you to keep supporting ODC, the most specialized open hub that studies and discusses this huge issue keen to the success of Gitcoin applications that is sybil attacks.
Thanks for reading
Thank you, Connor!
The legal costs are due to anticipating that some lawyers wonāt donate their time. We do have a relationship with a great domestic US law firm who actually has said they would defer many legal expenses - however non-US firms that have experience in safely issuing ERC-20s and all the related work whether in Switzerland Zug or Caymans / BVI are rather expensive and at least so far not very flexible in their costs.
In the meantime, thanks to the Hats Protocol team not charging us so far as an early user - and to them working in the open (open source wins again) - we are able to progress on that piece of governance without any direct costs other than the opportunity costs of educating ourselves on the use of the Hats Protocol as a means to build a so far alpha level of on-chain decentralized governance.
Hi Evan, great to see this proposal 
Iām excited about your upcoming governance token launch!
Overall, I would like this proposal to have specific metrics allowing us to measure the gap between your promises and actual outcomes delivered at the end of the grant. @connor touched upon this briefly , here are some ideas for concrete deliverables;
For example, Endaoment has an ongoing proposal for QF distribution with little by way of sybil resistance.
Co-funding targets - how much do you anticipate raising beyond the $65,000 provided by Gitcoin?
Amount of funds saved - this is the most critical metric to track. For example, ODC should be able to say that with $65,000 in expenditure, $130,000 was prevented from going to cheaters.
It may be hard to to assign targets to each of these metrics, but it is still a worthwhile exercise. At the very least, if ODC comes back with request for more funding Iād like to know progress on these 3 fronts.
Iād also like some more details about the core team and size of volunteer network.
Hi Evan, Stefi and team
I want to weigh in with a few of my own thoughts plus echo a few others shared here. Overall, Iām very supportive of efforts to decentralize Sybil defense and build a vibrant community of analysts who can serve the needs of Gitcoin (and beyond). This is a talented team with a strong track record and mission alignment, so my intention is to improve ODCās value proposition to the DAO.
Iād like to see more of a hypothesis from ODC about what itās product or service offering will look like. And Iād like some validation from within the DAO that such a product or service can be immediately helpful.
(nit: I donāt think itās relevant to compare the cost of this proposal to the cost of recruiting a data scientist, unless the BATNA is to go out and recruit a data scientist)
The posts from @jengajojo @connor @thedevanshmehta raise concerns around metrics / deliverables. I share those concerns, but want to emphasize that whatever metrics / deliverables are proposed should be closely linked to the impact of a ODC product or service on the DAO.
For instance, at the end of each round but before the matching pool payouts are locked, there could be a write-up that ODC submits that consolidates the Sybil detection approaches of multiple contributors, showing both consensus Sybils (where models are in agreement) and novel signs (worth follow-up because they represent new or more complex attack vectors). Or thereās a sensitivity analysis on on the effect of different passport scoring thresholds on results. Or reporting dashboards that go live at the start of the round and help participants spot attacks or suspicious behaviors at the onset. Iām spitballing based off the examples provided in the OP, but ultimately what Iād like to see is validation from a user group or championing from a DAO workstream that this is a good investment.
Great feedback, thank you.
Iād appreciate your feedback on whether you agree this is a comprehensive list or if there are others youād suggest. We could add this reporting to the proposal.
Work in Sybil analysis:
Sybil and Grants screening:
Hackathon(s)
Size of community
Development activity
Non-Sybil impact
Financial metrics
A significant caveat here is that the focus of the ODC is on research and development - we are boosting the supply of valuable data sets, algorithms, dashboards, and especially data scientists in the open, in a non-commercial, non-rivalrous manner. Nothing we do is complete without a round owner to actually make the determinations on their round how to screen Grants, squelch Sybils, set Passport parameters, report out their approach to their community, and so on. So when the metric says for example āamount protectedā this means in conjunction with a round operator.
A second related caveat is that the metrics read āknownā¦ā. This also points back to our open source approach which, by design, allows for fully anonymous use of software, dashboards, and even the data scientists and many of the data sets prepared by the ODC. A round operator or even a proprietary provider of Sybil or wallet protection or other services could use all of these public goods prepared by the ODC and not let us know.
Moving onto to your questions about funding targets and current core team size and network size:
Thanks Carl
Agreed re: the cost of recruiting a data scientist comparison; while the cost of recruiting a single data scientist likely does match or even exceed the cost being requested by the ODC here, that far understates the hard ROI as measured by cost avoidance (where soft ROI would include the overall benefits).
The hard ROI of the ODC would be each round operator hiring a handful of data scientists and software developers, building relationships with data layer providers and partners like the Ocean Protocol for the safe storage and sharing of datasets, and so forth. That still would mean that each round operator would be seen as somewhat of a black box so there would need to be a lot of additional explanation required as well. Back of the envelope the cost for an organization to replicate what the ODC is already beginning to deliver might be:
All in all you pretty quickly get to $1 million per year which approaches the costs that Gitcoin was funding the FDD at recently I recall. Again, though, a captured approach by a single operator would not have the same benefits as an open-source approach in terms of independence and credibility. Nonetheless, it is probably a fair proxy and one, again, that is similar to that which Gitcoin itself paid the FDD last year. Nonetheless, when considering the $65,000 being requested here as the first funding to be used for the ongoing operation of the ODC, a fair comparison for a hard ROI in terms of avoided costs would be something in the range of $1-$1.2 million for an in-house, data science and software team. Itās not a perfect comparison and to be clear ODC does not eliminate the need for round operators and especially Gitcoin to have data science expertise.
A lot of the specific reports you are mentioning are already being worked on by OpenData Community volunteers day by day with @connor and others within Gitcoin. As we begin to actually compensate for the work we are doing between hackathons, we anticipate this work will further improve in quality and will more tightly reflect the needs of Gitcoin. I think it is notable that @connor stated above that he was supportive of this proposal, at least as I interpret his post.
Perhaps it is worth mentioning here that this isnāt just a grant proposal. Gitcoin will also have governance rights in the ODC as a founder. So there should be alignment in part because Gitcoin will have more power over our focus than it would were this simply a Grant proposal.
@epowell101 Thank you for this proposal. I told you at ETHDenver that I thought ODC could be at the forefront of data analysis and you are making that happen. Well done.
I rank this proposal as a 6 out of 10. A middling score that signals neither a strong yes or no. Being where we are in the bear market and Gitcoin needing to double down on focus and frugality, if the vote were today, I would vote no based on the proposal and responses above.
My score is only a n=1, according to me, a minnow with significant only-to-me GTC holdings. It is very likely someone else would do this same analysis and come up with a different score. If this proposal moves to a vote, I will update the analysis based on forum content at that time.
Btw - I donāt like giving this proposal a score of a 6. I think ODC could be an exceptionally important partner for Gitcoin, and I hope the workstreams demonstrate that my initial assessment is way off.
Thanks Shawn - I appreciate your perspective and of course your broader work in governance at Gitcoin and more broadly.
I feel like the problems we help to address are the Achilles heel of quadratic funding - and in a broader sense are symptomatic of a rot within web3 itself. Itās like we are arguing for fixing the roof because it is starting to leak, when the family really wants to instead use that money to go hang out in Europe.
The analogy breaks down a bit though in that the OpenData Community is not and should not be the entire solution. Passport is crucial here, round operators are obviously crucial as well, and it will often be the case that commercial round operators and Sybil fighters are central as well. We need to face reality, increase our collaboration, and further increase the costs of attacking public goods funding rounds, airdrops, and wash trading.
Anyway - the basic adjustments we propose to make to the proposal are to add the metrics for reporting listed above. Do you have any other suggestions you would make that could make your score a higher one?
Hey itās Ben from Pocket here. Seeing as we are mentioned in the op I just want to add my voice directly and say that as a partner to ODC weāve been really impressed with their engagement and ability to build a guild of values aligned people and spread the message of Open Data and Public Goods.
Reading the thread it sounds like a lot of people share this positive perspective. I think with ODC itās not really a question of ādo we supportā but more about āhow best to deployā. I raise this because I interpret the feedback so far as mostly asking ODC to make a ābetter guessā at how they can support Gitcoinās ambitionsā¦ fwiw I think it might be better for someone at Gitcoin to just tell ODC directly what would be work that is strongly supported and justifies fundingā¦
The perils of living in the unsexy shadow of shippingā¦ I get that. : )
Yes, in the assessment there are comments on each line item of which could boost the score. I think working with the worksteams on elements 5 and 6 would likely sharpen 4 and 7.
Thanks. I appreciate your use of the matrix by the way and if it is fair to do so, I would like to use it in the future.
Picking a few points:
4. Does the project help Gitcoin achieve sustainability via mission-relevant income or cost savings?
5, 6, & 7: Does it contribute to immediate priorities & does it have support?
That said, as mentioned we plan to also revise the proposal to include a commitment to hit and report upon a number of metrics listed in the thread.
Thanks again for the analytical framework and thought you put into your analysis. By my admittedly biased math, if we shift up the first score on cost avoidance from 2 to the average score and shift up the score on KPI quality as well to the average score then the high scores you gave on other items would get us between 7 and 8, still lower than Iād like but I think more accurate.
The bottom line for me of course is that Sybils have not gone away and no one org - even one shipping at an accelerated rate like Gitcoin - can credibly create the data science, data sets, dashboards, and data scientists needed to enable the broader community to deal with these attackers. Nor should Gitcoin bear that burden alone. We are asking Gitcoin to support the creation of a public good in the form of an aligned community that will benefit the broader public goods and web3 space - and thereby will deliver capabilities at a small fraction of the cost were Gitcoin to build these capabilities in-house.
Love to hear 3rd party validation of ODC outside of gitcoin! Thanks Ben.
Also really like the idea of getting a formal paid relationship started via a spec or RFP from Gitcoin. I think one of the areas we could improve is better definition of what a sybil attack is (vs an airdrop farmer vs a new user) and the % tolerance we have for sybils (0% is never going to happen, but what about 1%? is 5% too high? etc)
Going through an RFP exercise for our first formal/paid engagement with ODC could help us sharpen our need internally as well as provide structure and a path to success for ODC.
Iāll second this as an early community member in the ODC and agree totally on being impressed with the ability to build a community and interest in Open Data and Public Goods. I long ago got out of the business of making my opinions known regarding GitCoin DAO funding proposals, so Iāll leave at there. If I had a vote, Iād vote yes.
Thanks Meg! Btw an open data community member built a first version of that grants dashboard we discussed as a useful utility that Gitcoin wasnāt going to be able to build itself for sometime. We should get you a demo and maybe you might want to reach out to @stefi_says who leads that project.
Our position re commercial engagements is that we are an open source community and project. There are solution providers like Plurality Labs and others including a number of amazing ODC members that might be a better prospect for that RFP.
That said - itās a great thought experiment. Would it make sense for gitcoin to source and pay for all of the data science, data sets, dashboards and data scientists for the broader public goods and web3 community whether from the ODC or via an fully staffed internal team? Iād argue probably not because there is a real concern about centralization and of course costs. As mentioned above the amount we are requesting is roughly the amount for recruiting a single data scientist - let alone for paying them and their team members.
Please do let me know if you want a connection to @stefi_says - or just take a look at the momentum behind the project here (caveat - it is new and evolving quickly - this is the sort of thing we will help to accelerate once we solidify our foundations): GitHub - OpenDataforWeb3/Mirall0x: Mirall0x is a application to empower humans analyst to have access to information and clearness when analysing third part projects
Thank you TJ - your commitment and vision and delivery of a great open source project in TrueBlocks are inspirational. Iām really happy that in the course of our research in support of a decentralized data layer we got to know you. Weāve learned a lot from you and Iām sure our community - including some Gitcoin engineers by the way (letās quantify that benefit somehow?) - have benefited from the instruction and guidance that you shared while engaging w/ the ODC.
The ODC is your community 
Catching up now on this thread and the overall proposal. Thanks, @epowell101, for putting together this proposal and for your continued work to build ODC into an open source sybil fighting machine. I will echo @connor and say that I appreciate the support that the ODC provided to our team as we evaluated the results of the Beta round. I think the ODC provided tremendous value, which I believe we need in every round. I am in support of this GCP.
One question I have, and forgive me if I missed this above, but if this proposal passes, will Gitcoin receive free services from ODC moving forward and for how long? As a founding partner will we be able to lean on ODC indefinitely to provide an analysis of the results of future rounds or will we need to pay a yearly/quarterly member fee? As Gitcoin begins to provide services for Featured 3rd party rounds (conducted at the same time as our Core rounds) will we be able to utilize ODC as a part of our offering and at what cost?
Hey @jon-spark-eco thank you for your kind words and of course, for your work in making public goods funding happen.
Your question points out another flaw in the proposal as written IMO.
The short answer is yes and no - and again we should put this into the revised proposal due to be posted any day.
Yes - absolutely, this further cements the relationship between us and will enable us to do a lot of work in between hackathons that otherwise is somewhat lacking. We will use $2500 per month to fund these sorts of bounties and direct expenses (eg, API costs and so on). And this work will be done to meet the needs of Gitcoin and community round operators.
No - this amount does not guarantee a true soup-to-nuts commercial quality engagement on our part. We will continue to improve however it may still be the case that external, non-ODC commercial providers will be valuable to Gitcoin and others. On the other hand, it may well be that Gitcoin thanks to our close relationship and the understanding of Sybils in Gitcoin born from thousands of hours finding and fighting them may not need commercial-level engagements and will be more than satisfied by our best efforts to collaborate. Perhaps non-ODC commercial total solutions and professional services may be more applicable for non-Gitcoin led rounds. Yet to be determined. It could even be that Gitcoin decides to use ODC open-source software, data, and data scientists as a part of a commercial service. As you know we see ODC as a provider of public goods that could then be used as a part of commercial solutions perhaps along with other proprietary algorithms and data sets or used on their own.
It is also the case that we will be seeking sponsors for future hackathon bounties. Gitcoin may choose to continue as a sponsor depending on budget availability. We commit to continue to run by Gitcoin our proposed structure for the hackathons and would appreciate any feedback. I expect that the percentage of funding for these hackathons from Gitcoin will continue to decrease even though the focus of the hackathons will continue to benefit Gitcoin and everyone interested in using data science for good in web3. It will be up to Gitcoin on a case-by-case basis. The range of this cost is from $0 to approximately $10k per hackathon.
