[Discussion & Proposal] Ratify the Results of Gitcoin’s Beta Round and Formally Request the Community Multisig Holders to Payout Matching Allocations

[connor]

@octopus

Sadly there is no algorithmic way to distinguish legitimate behavior from malicious behavior. Any automated process that lets all legitimate behavior through, will necessarily let some illegitimate behavior through. Conversely, any process that catches all illegitimate behavior will also flag some legitimate behavior.

Yes absolutely. Web3 identity systems are still in their infancy, and there will always be false positives and false negatives. All we can do is strive to create an outcome that is as fair as possible to all, catch as many of the bad actors as possible, and work to improve the process round over round.

@octopus

In terms of pure math: for grant matching amounts, the pairwise coordination penalty also needs to be considered. If most donors to a project donate to just that one project, their donations are given a penalty. The “number of donors” and “donation amounts” are too crude a data point, especially for small rounds.

Couldn’t agree more. Almost all of the major Sybil rings we discovered have one core thing in common - donors only donating to one grant, or donating $1 to a couple of grants and $100 or $1,000 to one grant.

And yes big +1 to pairwise bonding, that’s at the top of my wish list. As well as a variable “trust bonus” score so it’s not simply “Sybil or not Sybil”, but “how much reputation and history has this person/wallet accumulated”

@connor

I am (personally) excited to explore features like a “trust bonus” system (ie. a variable trust coefficient rather than a binary yes/no). This would mean the highest “verified” users have their donations matched the most, so building up a strong reputation over time is important. Future experimentation with pairwise bonding, plural QF, and other anti-collusion mechanisms may also prove to be effective at limiting bad actors.

[smith]

@connor, I just wanted to say that I really appreciate all of the (obviously intense) work that you and the team have put into evaluating transactions and consistently applying rules. As a fundraiser, QF is extremely interesting to me and I hope to see it applied in lots of different contexts. The way you openly share the information about how you make decisions—and the challenges you’re running into—are extremely useful for the future of trust in something like QF. Thank you!

[Dagda]

Hi @connor ,

It may not be the best place to discussions/debates, indeed, but it’s certainly a good place to be transparent, and open with the community. Really appreciate all the work you do, this is how we learn as new grantees.

@connor

There is a clear pattern of the grantee wallet (0x530) distributing Eth to brand new (or sometimes very old) wallets, just for that wallet to immediately turn around and send the whole Eth amount back to the grantee as a “vote” (the two “vote” tx addresses are the two rounds Mycelia was in, Climate and Metacrisis) and in almost every case only donating to Mycelia alone and no other grants. This is not covering gas fees, it’s giving someone money for them to send right back and increase the match.

We won’t deny this ‘red flag behavior’ from 0x530, but pretty sure that it’s already been taken into account in the last ratified results of the Climate Round going from 2606$ USD received with 33 votes to 1409$ USD received for 23 votes. You can maybe clarify this point if those wallets was already taken away. If so, it’s worth mentioning.

For more transparency and contextualisation, here a detailed doc about all the active wallets that contribute to Mycelia campaign.

The red one are closely linked to us, and this mainly old wallets we used to onboard family members and close relationships as it was easier to reach the 15 score, and indeed this is more than the gas fees paid as they were supporting us consequently. Not really smart to on-board on this way, but we were also taken by time as we’ve only been accepted to half of the campaign of the Climate Round. Also these transactions should already have been curated by the first and second Gitcoin correction, (at least concerning the climate round) to be confirmed.

The orange are wallets we helped with gas fees, mostly indigenous peoples and allies close to Mycelia, where we’ve discovered all the difficulty and challenges, especially how difficult it is for them to understand why they have to pay $20 to $30 gas fees instead of donating that money directly to their community. Also you mentioned that those people didn’t donate to other grants, we supposed it mainly because of the language barriers, a lack of communication from our side and the high gas fees (bc still higher when you give to many grants in one shot). But there is definitely work to be done here.

The green are wallets that come from the broader community, mainly people from Gitcoin and peoples who discovered us during the campaign.

As you know, nothing is all white or all black, there is definitely in our campaign behaviors to be curated, and we are not here to discuss them as we fully agree with you that the matching funds need to go to the most legitimate projects. We just want to clarify our situation to have the most accurate resolution according to our work and energy spent with Mycelia.

@connor

I don’t believe setting a precedent that this is allowed is in the broader community’s (or anyone’s) best interest, as I think we all want to truly drive the adoption of QF as a more democratic and fair way for communities to allocate funding among their shared needs. But if others in the community feel differently, please do let us know, these should not be decisions made in a centralized manner.

Fully agree, and we are here to learn as a first time grantee starting a Gitcoin experience in a unique Beta Round where it was possible for just a bunch of 20 to 30 peoples to access to the top 3 and receive 35k$. It’s important to point out this particularity, something we supposed will not occur twice. Gitcoin Citizens feels already different, far more democratic and accessible to on-board non-crypto-native peoples.

Btw, very grateful for all the hard work Gitcoin is doing to protect the collective best interest, this is one of the main reasons we want to promote it to indigenous communities, and stay here for the long run. Apologize for the extra-work it gives, and thank you for the explanation and the learning.

[connor]

Hey all quick Friday night update - we are pretty much finished with the additional Sybil review. We’re now working to get the final results to include revised USD amounts (and other data) as discussed above. Final results should be shared early next week. Thank you all for your patience!

[MathildaDV]

Thank you @connor ! And thank you again for your hard work on getting this done!

[connor]

Hey all!

Happy to share the final revised results. Seriously thank you everyone for the help and your patience.

You can find them here:

Beta Round Revised Final Results V2 (Core Rounds - Unratified) [Public]

These matching calculations included all methods discussed previously in prior iterations, along with a few new tactics. First, we used @umarkhaneth’s awesome visualization tool to spot suspect Sybil rings. We then manually investigated via Etherscan, Passport, Breadcrumbs, and other on-chain tools, and took action when necessary.

We also further leveraged many of the automated analysis tools ODC and others provided to see what would best catch likely Sybils without too many false positives. After a lot of backtesting and experimentation, two tests stood out. One was checking whether an address shared an initial seed funder with another address. The second was whether an address had recently interacted with other donors. Each test by itself would catch a high percentage of donors, but when combined (if a donor was flagged by both tests) then only 5-10% of donors were flagged, and when we compared the results to known Sybil rings, it caught a much higher proportion of Sybil donors.

We have also now updated the results sheet to include the accurate “total received USD” amounts, summed after Sybil squelching, so the numbers should all match up better with no big discrepancies.

Please take a look and let us know if you have any questions - we would like to accelerate the timeline to move to a snapshot vote soon so we can pay out matches ASAP, however will still leave a few days for comments and reviews. Thanks all!

[carlosjmelgar]

Thanks for all the hard work that went into this.

[playoriginstory]

Thanks for your work on this @connor @carlosjmelgar. It seems our contributions (The Gallery DAO) were reduced by 4 people and 250 DAI from the matching pool. Can you explain what happened? We also never received any information regarding the DEI round. Just trying to keep up here on decisions and changes being made.

[carlosjmelgar]

It’s going to be extremely difficult to identify what exactly happened with those four donors to your grant. The team will be unable to provide these explanations to every grantee. Please look at the detailed descriptions in this thread to get a sense of what might have happened with those 4 donors.

[DistributedDoge]

Looking at random grants I can agree with amounts reported in “Revised Total USD” columns, so far I can always find such combination of valid votes that would result in “Revised” amount being accurate. This is also consistent with number of contributions reported.

It is a bit of a shame that revised report seems to have dropped “Sum Sqrt” column which would be helpful for eyeballing cases where QF calculation might seem counter-intuitive when looking at raw donation amounts, not taking distribution of those donations into account. Still I wouldn’t want to hold up the results because of this, unless someone else spots something wrong with this iteration, but it is something to think about for future rounds.

Overall, very nice work. I am not commenting on Sybil or QF calcs, but I am happy to see a consistent report.

[lefterisjp]

Hey looking at rotki before and after I am confused.

Before total received: 2,055.00
Now total received: $1,633.22

As faras I understand this is already donated. Nothing to do with matching. Why the discrepancy? Was the first one wrongly calculated?

[connor]

Yes correct all direct donations go directly to the grantee wallet where they have full custody. The “Revised Total Received USD” is essentially the sum of donations that were counted for the revised matching calculations, so discounting txs < 1$, txs from non-existent/failing passport scores, and txs removed during Sybil detection.

In the first two iterations of these results, there was a lot of confusion around the “Total Received USD” amounts (when they weren’t adjusted downward after Sybil checks) not matching up logically with “contributions” and “match”, so I think it is better we can now share the revised donation sums.

[lefterisjp]

That’s not what I asked though. Say I want to do my accounting. How much did we receive? Counting it in matching or not. Is the first amount correct or the 2nd?

If the 2nd is just 1st minus what you regard as sybil through some kind of calculations you believe detects sybil then specify that in the name of the column. Because it would be nice to be able to also know how much each grant received raw.

[solarpunkmaxi]

Appreciate all the effort put in by the FDD team and you @connor.

Couple Qs on the process -

Do you consider projects contributing to each others grants also as a sybil attack?

What do you infer when a donor address has interacted with other donors?

When we onboard friends and family to contribute to our grants is that considered a sybil attack?

[connor]

Well you asked why the discrepancy and if the first one was wrongly calculated

But yes the accounting question is a very important one, and I think you are right we could rename the columns to make this clear. Perhaps calling it “Total Donations Counted for Matching” would be better than “Revised Total Received USD”. What do you think?

I’ll also call out that the $2,055 amount you see in the first iteration is also not the raw number - this is the total donations after removing failed passport scores and donations less than $1. And then the $1,633.22 number in the final matching calc iteration is removing both the items above, plus any other transactions flagged by further Sybil analysis and defense.

I believe the real “raw” sum would be be found on the frontend grant page, so for example Rotki I see is at $2203.96.

All this said, the goal of this gov discussion/proposal, and the matching calculation sheet, is not to provide an accounting statement for projects, but rather to showcase the final matching results and what numbers were used to calculate them.

I do absolutely think the product should provide an easy way to export a csv of all donations/matches made to a project. This would be a really useful tool and something that would be best built into the application, rather than done through google sheets/forums. I believe it’s somewhere on the roadmap for Allo/Grants Stack. Perhaps some collaboration with Rotki’s tooling could help to pull and organize on chain data

[lefterisjp]

yeah haven’t ran rotki on it yet to see what it says for this. Okay understood the difference between them. Would make sense to rename the column then

[connor]

Great questions - I’ll try to reply below but with the caveat that often the answer is “it depends” and that we can’t possibly check every transaction manually so some things may or may not be caught by automated analysis.

@solarpunkmaxi

Do you consider projects contributing to each others grants also as a sybil attack?

No, I would definitely not. In fact, it’s awesome to see grantees supporting one another in these rounds. In the past we have seen cases of “project sybil rings” where a cluster of projects are all donating to each other back and forth, often with additional sybil donors, but that’s a bit different.

@solarpunkmaxi

What do you infer when a donor address has interacted with other donors?

Nothing inherently, that definitely can be very normal behavior in most cases. For this specific round, among the 20+ different tests run, I found that flagging donors interacting with other donors that ALSO shared a seed funding source, was a good combination of indicators to catch likely Sybils.

@solarpunkmaxi

When we onboard friends and family to contribute to our grants is that considered a sybil attack?

No of course not - a Sybil attack is one person creating multiple identities (don’t mean this in a patronizing way but more info here). Onboarding new people to crypto is an incredible goal and outcome of QF rounds.

However, what is hard is that we can never be 100% sure or have definitive proof some address is or isn’t a Sybil (without some enhanced biometrics or KYC requirement). We flag likely Sybils, or likely unique humans, and do our best to discourage Sybil behavior. For example if you onboard friends and family to Web3 and help them get a wallet, and then send them some Eth, which they send right back to your grant, they may all be unique humans, but it certainly looks a lot like a Sybil ring. That could also likely fall under some umbrella of “collusion” which is not allowed. So those votes would likely not count, even if in reality there was a totally unique human behind each wallet. The problem is if we gave everyone the benefit of the doubt and took your word that those 10 addresses you funded were friends and family, we’d have to do that for everyone, and any grant could spin up a bunch of wallets themselves and plausibly deny they belong to the same human. So all we can do is analyze the on-chain data, set rules and standards, and make calls to the best of our judgement.

[Dagda]

Hi Connor,

Thanks for all the hard work you and the team behind, not an easy round for anyone, there is a lot of teaching for all peoples involved here.

Concerning Mycelia, let’s not deny that it’s a hard blow, we were ranked number 6 after the first curation, then after the second one we were number one, spotlight on us, for then in the last result being ranked 80th. We know that the Gitcoin team cannot do a 1:1 basis analysis of addresses, but we can fairly admit there is a lot to learn for us here.

If after the first and the second curation most of the addresses were not removed, and at the end in the third curation we go to hell, there is definitely a need for clarification. We admit some behaviors on our side was not respecting the rules of Gitcoin, and need to be curated, while also proving our legitimacy here in this thread by explaining how dedicated we are to connect Web3 and Gitcoin to indigenous communities, the Gitcoin Citizens reflecting that.

So now getting only 6 contributors with 14$ usd received feels off, when there is at least 13 addresses that don’t have any connections to Mycelia (no gas fees paid back, no transfer of GTC) and mostly people from the Gitcoin community (so normally with passport valid). We got the confirmation that some addresses didn’t get the passport valid, but it feels there is still a gap to fill, and all that without considering people we only pay back few gas fees (~20$) or transfer of GTC to help the passport validation (is it something prohibited in the current rules?)

Again, we are here to learn, we are not here to complain about the work done here, we know its coming from a place of full commitment for the collective best interest, Sybil resistance tools are still in progress.

We really want to see this day where it would be not possible for anyone to go against Gitcoin rules, because of the design. There is out there bad actors, new grantees, projects who tried to on-board as much people as they can — honesty is a spectrum, not a binary state. It’s not to Gitcoin to decide who is legit or not, who has good attentions, bad ones, mixed ones, it’s up to the design. And it feels it’s getting closer, but in this transition time, there is still a need for clarification, because really, being on the top and going to hell is not a feeling we wish to any legit project out there.

So thank you all for this round, it seems unlikely to have a change in the final result, the whole community is justly waiting for its matching fund. But at least, we would like a transparent answer on this decision as we are accountable for all the indigenous communities we are working with, this is really important for us.

DM are open on Telegram, or by email to uni@mycelia.xyz

Thanks all, and lets do better next time.

Closing in music : Regeneration //Nerdseq/Panharmonium/ER301/DPO/Mangrove/Erbe-Verb// - YouTube

[solarpunkmaxi]

thanks for the detailed response. This clarity will def helps us navigate future rounds better.

I think most grants acting in good faith have a legit story behind what appears as a sybil attack but at the same time i get that its really not feasible for Gitcoin to look into each to understand the context.

The current doc on whats considered a sybil attack by Gitcoin is quite broad which is where potential confusion can arise, esp for newer grants. Maybe a forum post where this can be hashed out betn the team and grants once the dust settles on this round would be nice to prevent this next time.

Have DMed you on twitter/TG @connor with some potential false positives that we would like to get clarity on before the next round.

Congrats to everyone that made this happen. Looking forward to the next roller coaster

[MathildaDV]

Great work @connor ! Thank you for this. The blog post from our end will be posted early next week! Thanks for all the hard work that went into this and for the conversations here. It’s given me some great insights into the process.