[Discussion & Proposal] Ratify the Results of Gitcoin’s Beta Round and Formally Request the Community Multisig Holders to Payout Matching Allocations

🌱 Gitcoin Grants
, , , ,
21

If we publish data on every single donation and whether it was counted, we’d create an environment where users may feel specifically targeted and where we’d have to justify every decision that was made

It’s not the people who vote that count, it’s the people who count the votes.

Current trend of only sharing aggregate results means that only way to prevent incorrect results from being ratified is to spot obvious anomaly in a dataset. This time we were lucky in that while report was calculated incorrectly, there is indeed one row with an obvious anomaly and @duckdegen was aware of a missing vote.

I would like to request a dataset with all eligible, non-Sybil votes that qualify for final matching. This would allow everyone to re-calculate matching amounts and confirm that inaccuracies in the reports were indeed corrected.

Moreover, raw data with final votes would also make it possible to recalculate matching using alternative algorithms as suggested by @thedevanshmehta.

Ideally, but this to me is of secondary importance, because some votes are “missing” Gitcoin could also share updated listing of all votes that were cast during a round, no matter if eligible or not.

As it stands, without actual vote (donation) data on-hand how are stewards supposed to verify accuracy of matching amounts presented in final report?

4 Likes
22

I hear you, I am open to sharing the raw vote data once we have the revised calculations ready if the rest of the team is on board. We’ll discuss the best way to provide this and enable anyone to verify results. Thank you for this feedback, and your willingness to dig into the data with us

5 Likes
23

Thanks Connor - I see your DM as well.

I think my comment was poorly worded to somewhat conflate round calcs with Sybil & squelching. There were bounties having to do with Sybil identification in the ODC hackathon - and many about building the tools such as Legos and Dashboards, and data sets to protect rounds. My comment made it sound like we also calculated the round calculations - which from this thread I see there are ODC members doing that - however there was NOT a bounty or anything like that for that work.

I totally get concerns about the downside of transparency - especially if it meant that we’d see a lot of appeals. For example, the ODC required everyone competing in the hackathon who wanted to use lists of likely Sybils to backtest their models to register and be added to an access list / “whitelist.” So the OpenData Community was not entirely open with its data. Maybe there should be a document “we” in the broader community collaborate on regarding transparency vs. secrecy of approaches in this regard.

Coming back to the initial point - as it stands we are not aware of how the Sybil identification and squelching analysis was done. More depth on what algorithms were used, what patterns were seen, will help the broader community prepare for future QF rounds and incidentally will help other communities fighting washtrading and fraud more broadly. We could stop short from publishing individual likely Sybils which would presumably trigger appeals.

3 Likes
24

Thanks Connor, a quick look at the open source round (which saw a lot more participation) does confirm your analysis: low participation leads to money given counting more than number of contributors. In OSS, the projects receiving the matching cap also had the most number of contributors, unlike the other rounds.

It’s great to know of initiatives rethinking QF. One takeaway for me is that Gitcoin should leave open the optionality of using classic QF if there is high participation and weighted QF if there is low participation. This will be important as we run experimental rounds like journalism which may similarly see lower participation in the first few seasons.

It’ll be near impossible to eliminate wash trading entirely. For eg, my project has a SOP of creating new wallets for every grant received. We could easily have cycled grant funds already received in those wallets through Gitcoin to come out with our principal plus profit (we obviously chose not to do this). The only solution is giving even more weight to number of contributors than amount donated, especially for rounds with low participation.

5 Likes
25

Hey @DistributedDoge, @nick_smr, others-

We are more than happy to send projects the outcomes of their particular round (so they can see exactly where and why certain votes were silenced.) I will echo Connor in that we don’t want to embarrass anyone who might have accidentally got caught up donating during a bot attack, through now fault of their own :slight_smile: And this certainly happens -but if you accidentally gave exactly 1.5 DAI at the same time as 45 other wallets did the same- you will have blended in with the bots and will have been silenced…

We don’t want to contribute to the many unattended “sybil-lists” that float around the space unless we are highly confident and transparent about inclusion on a sybil list. When we silence someone, it is not necessarily because they were known 100% confident sybil, but because they behaved in a way that was anomalous and likely intended to manipulate the round. I hope this makes sense- but we’re happy to speak to project owners about any particular concerns and be fully transparent with their results. I’ll personally send you a little sheet so you can see what happened to all of your voters.

Similar to the transparency balanced with “shade-throwing” ideas above, we know many of our projects (especially the OSS crowd) are subject to airdrop farming speculation. So please know, it is not a reflection on a project’s honesty and respectability if they had a high squelch-rate. For example, anyone who is building on Optimism L2 is likely going to have a lot of airdrop activity; that’s totally not their fault and certainly doesn’t “prove” a self-attack.

I hope this helps! Please feel free to message me on the gov forum or Discord if you want to keep talking through this.

3 Likes
26

Hey @epowell -

In terms of our use of the ODC hackathon results- I hope you can understand that we’re a bit far away from being able to utilize multiple notebooks across multiple systems without any criteria to gage quality in place. I know I gave this feedback last hackathon about a need for consistent auditing standards- and I gave several notes as to where analysis could be improved on or explained in more standard ways for a wider data science quality assurance process.

I personally am not much worried about appeals- I think if you accidentally donated in a sea of airdrop-bots, you’ll understand why we had to discount you :slight_smile: Our community is smart and savvy about these types of things, although, we do go on “anomalous” behavior and not pure sybil in order to protect the rounds. I know the ODC members also rely on stats analysis- and I’d be happy to discuss any areas where they think we can improve on our “rules”-based logic that we’re moving into.

Here were the rules we ran for this round:

  • Passport pass/fail
  • Passport manipulation attempt (attempting to add the same hashed credentials across several Passports)
  • Suspected bot (you and 44+ of your friends donated in identical ways within very close proximity (same 3 seconds) of one another, or otherwise gave yourselves away as a “bot” army)
  • You attacked us during alpha, Fantom, or Unicef rounds and we saw your sybil network attack again (this is relatively rare, but there are a few larger networks which we are tracking who move in this way; most all also failed the above Passport checks, to be clear, but we’re keeping an eye on these beasties)

The energy and work of the ODC members has been truly impressive, and I’ve enjoyed meeting with and talking with so many of the analysts, students, and data scientists you and @baoki have gathered here. If you all have ideas of how we can improve the rounds, would love to talk.

I would also expect any sybil networks or self-attacks of grantees to be reported to us, though, and not saved for last minute-ism… I can’t seem to find that anyone notified us of a network being found? Let me know if you think we missed this somehow - we are examining better ways to surface community reports of round manipulation.

1 Like
27

@thedevanshmehta Actually you were 100% on to something here with the Mini Meadows grant.

By itself this doesn’t provide any indication of calc issues or bad behavior. Like I said since the climate round had a low-ish turnout, 13 unique donors giving high amounts can and will give a big match, the QF math is working correctly.

But a glance - yeah seems weird. The match checks out so it could be legit, but the numbers stick out like a sore thumb. Total Received USD is 3-4x higher than all those around it, and contributions is 3-4x lower. So it’s definitely a big outlier.

So I dug into the on-chain data manually a bit and discovered it’s pretty much all one big Sybil ring, potentially including more grants as well, and more donors. I’m frankly disappointed that it slipped through the cracks with how blatant it was.

I’m going to propose (both internally to the DAO and to the community here) that we halt any next steps towards finalizing and paying out the round until we can sit back and focus on a deep dive into Sybils and fraud using robust on-chain data (ideally with more automated tools), leveraging Passport data much more, and using other methods to flag these patterns I’m seeing. It will be a collaborative effort across multiple Gitcoin workstreams, as well as something we’d love to get the community involved in. I’ve been DMing with @epowell101 about what tools created by the ODC can be utilized. So I’m hopeful we’ll learn from this process and can streamline these calculations going forward.

6 Likes
28

Hi there!

It’s Ger, the founder of mini meadows. Really disappointed to hear this. I’ve been developing and reiterating this idea for a quite a while now to help people take real climate action.

As everyone in the environmental movement knows, it’s a massive challenge to elicit climate action and outside my professional role, I’ve made it my full time (& unpaid) job to empower others to take such action.

It’s actually quite upsetting that I’m at risk of being excluded before I’ve even had the opportunity to deliver anything from the project because people donated to the cause.

I canvassed with my own network and on twitter spaces alike. I even onboarded new people to web3 to make donations.

I didn’t expect to receive such donations tbh and it’s deeply disappointing to see this reaction from a community that is supposed to value solidarity and is looking for widespread adoption

1 Like
29

Ok - forgive me but I have to get all web3 fan-boy on y’all.

From my POV this thread started out kind of rough - but in reading through the dialogue I see points of tension evolving into collaboration and improvement. In my old world, situations like this might just have easily devolved into the hardening of positions and pointing fingers.

This stuff is not perfect, but because of people like you - and the collective dedication to making this work, you are all the reason why web3 will change the world. I have no value to add in this chain, but I just wanted say you all kick ass. Thanks.

For those still troubled - stick with it. Intentions are in the right place and we are going to get there.

11 Likes
30

Thanks for the tag! I’m not qualified to comment on everything in this thread, since while I know a good bit about the mathematical foundations of QF, I’m still learning about Gitcoin and what its stakeholders value (today is actually my first day on the job).

I can say that the new algorithm we’re developing via QED could give bigger bumps to large groups of donors under some circumstances, but since this isn’t a thread for discussing that algorithm I’ll leave it at that.

Like someone else mentioned in this thread, the small donor numbers for both groups seems like a very plausible factor in the outcomes that @thedevanshmehta noted. I would need to look into the data more before coming to a real conclusion, though.

Lastly, slightly off-topic but important – as we move forward with the QED program, I want to gather as many stakeholder perspectives as possible to understand how folks think about QF, what they like about it, and what they want to see changed. If anyone wants to share their thoughts with me, please reach out on twitter (@CS_Synthesist) or Discord (Joelm#8192). I would really appreciate hearing y’all’s perspectives! Thanks.

3 Likes
31

Thanks @connor and thank you to everyone working in the GitcoinDAO who keeps helping us fund public goods!

4 Likes
32

I have to wholeheartedly echo what @shawn16400 is saying here. And I appreciate the team @connor @ale.k and co. all working so hard on these results.

3 Likes
33

I’m really concerned about what is happening in the climate solutions round and I feel that how this is going to play out will not be fair to projects in the global south

1 Like
34

My previous response was not very constructive, let me explain my concerns in more detail:

I consider myself as someone who understands the issues in the carbon “credit” ecosystem in depth, also, I’m pretty educated on biodiversity and social impact topics since I’ve worked hands-on with various projects over the past 4 years.

The Block2030 project is the one that concerns me most, honestly, we do not need more carbon marketplaces, what credits are they going to sell? There are very few projects that supply credits and most of them are already committed to marketplaces. There are too many people focussing on “selling” credits and not enough work on the “doing it” and “proving it” side, because the latter two topics are hard and not as profitable as taking a transaction fee on a credit that just needs to be sold.

Regarding projects in the global south, if we look at projects I have a lot of respect for (in the list below), in my opinion it is not fair that Block2030 will get exponentially more funding than them, the effort that Block2030 did on their project page is a bit of a joke in my opinion for the amount of funding they’ll get out.

If you have a look at the projects below and the number of votes they received compared to Block2030 (Votes: 13), it doesn’t seem fair

  • Kokonut Network; Building an Eco-Friendly Perpetual Fund - Votes: 40
  • Shamba DMRV and Ecological Data Oracle.- Votes: 39
  • Kawagumba village: a Kenyan community farm project - Votes: 29

Another issue I have is the cost and reality for projects in the global south to get contributions from their friends and family, and this is not Gitcoin’s fault and I appreciate all the effort that’s been made on this end. But, if you consider that the minimum wage in South Africa, where I live, is $11 per day and some professionals in the climate change space don’t even earn $1000 per month, the amount of funding a project received, will in most cases benefit projects/startups in the global north. I can’t ask my friend to donate $1 and pay $50 gas fee, for that amount of money my friend could take out his girlfriend for a 5 star dinner, and that’s simply wrong of me to ask and was actually what my friend told me.

We somehow need to understand where the Gitcoin projects are based (not where they “want” to do a project) and where the funding is coming from ($1 from global north is not the same as $1 from global south) and the QF needs to somehow use this when calculating the matching funding.

I’m happy to dive deeper into all of this on a call with people who want to figure this out. I really have a lot of respect for all the people at Gitcoin but I think there’s some deeper thinking required to make all of this fair.

6 Likes
35

Hi Ger

I understand that you feel this way and your project looks really cool. I don’t think you should be excluded at all and I hope this doesn’t happen.

The contributions you got are amazing and should help you get your project started and scaling, my concern, however, is that the 16 contributions you received totaling $5702.95 are almost impossible for a project in the global south to source from 16 people in their network. This isn’t your fault but unfortunately, it is the reality of the scenario we’re in.

You might be able to reach out 50 people and have 16 people donate and get your $5k, I can reach out to 1000 people I know and I won’t be able to get $5k, people in South Africa and in the global south are generally just poor and the USD exchange rate is insane with local currencies.

I don’t know what the solution is and I hope that the Gitcoin team can find a way to make all of this fair to everyone - this space is complicated and really requires a deep understanding and model to make it fair.

4 Likes
36

Yesss!!!
I guess BETA rounds are named such because they are in beta?! :wink:
Thank you Gitcoin Team and community! No doubt it will only get better from here…

4 Likes
37

Hey @carbontarzan thanks for the details here, I think there are a variety of issues raised but first off what data source are you looking at for the results?

The Google sheet linked a few times in the original post is here (but keep in mind we are working on revisions and this isn’t final):

Check out Block2030 - it’s currently in 50th place getting $450 in matching. We actually investigated this one specifically and a few very large + suspicious donations were penalized.

All 3 of these are higher than Block2030:
Kokonut - 26th place, $2.5k
Shamba - 18th place, $3.7k
Kawagumba - 42nd place., $700

Anyway - the goal of this thread, in particular, is to 1. Verify the QF math is working as intended, and 2. Verify Sybil attacks and fraud is being detected and dealt with efficiently. But to a few of your other points -

This point should fall under eligibility criteria discussions, which we also have in public and try to gather community input. However, my quick personal opinion is that Gitcoin should not be restricting criteria to what we think are good ideas or bad ideas. We should be reviewing whether a project is legitimate and abides by any other round-specific criteria, but then the community uses QF (aka “wisdom of the crowd”) to collectively decide whether a project is a “good idea” and worth funding. QF is a curation tool on its own, ideally Gitcoin does as little curation in a centralized way as possible, only high-level moderation.

This is def worth a deeper discussion - I don’t think you can really quantify geographies in a way that can be incorporated into the QF algorithm, however in the past, we have done QF Rounds focused on specific regions, and we’d certainly be open to experimenting with more of those.

5 Likes
38

Hi, someone here from the Block2030 team. This was our first Gitcoin round, and as newcomers to the space, we may have misunderstood some aspects of the guidelines. Based on our understanding from the Gitcoin documentation, we believed it was acceptable to approach Family and Friends for donations, which is what we did. Although not all of them are involved in the crypto space, they were interested in supporting us through this transparent platform. In order to make a substantial contribution, we pooled our money together and made one-time donations from the pooled funds. We genuinely weren’t aware that this practice was not allowed, but we are willing to provide proof of transactions to demonstrate that we received the donations. We also understand that these contributions may not count towards the Quadratic Funding since we did not receive a significant number of votes. @carbontarzan, we managed to secure $450 in matching funds, and the donations were sourced from our family and friends. I’m having difficulty comprehending your criticism, so it would be helpful if you could provide further clarification.

39

I wasn’t looking at the Google Sheet, was still looking at the old project page - apologies didn’t see the updated Google sheet

1 Like
40

Thanks @carbontarzan. I completely understand the point you make about every dollar going further in the global south. It could be a good idea for Gitcoin to weight geographies in Climate Rounds going forward.

Ultimately the Global North is responsible for far more emissions per capita than the rest of the world. Consequently, there is so much work needed in the Global North to make more of the population take the Climate crisis more seriously (which is one of the main goals of my project).

This funding round has been a big learning experience. It has pushed me to think bigger and how I can expand the concept and reach even more people so that alone is a win.

Being involved in community activism on Climate Action has made me well versed in failure and setbacks. If its decided to withhold funding, I’ll still keep hammering away. The Climate is far too important to just give up.

1 Like