[Proposal] Ratify the Results of GG18 and Formally Request the Community Multisig Holders to Payout Matching Allocations

RB1610 · September 25, 2023, 3:49am

Just wanted to express my best wishes to the team behind GG18, what a great round this was! Thank you to @M0nkeyFl0wer @umarkhaneth @jon-spark-eco @MathildaDV and the rest for making sure to support the community whenever and however possible throughout those two weeks

umarkhaneth · September 25, 2023, 3:59am

Delivering here on my promise to share more details.

How to Calculate Cluster Match QF:

First, a quick review of simple QF:
- Sum the square roots of each individual’s contribution to a project
- Square that sum to get a per-project value
- Distribute the matching fund proportional to the relative size of each projects square (and enforce a matching cap so that no project takes too much of the pool by itself)
Next, cluster-match QF. Cluster-match QF orients matching funds around communities rather than individuals. This is mainly the same overall process however before we square root contributions we cluster them together.
- Cluster based on the donation profile of a donor. A donation profile is defined as the set of decisions you made on each project: donate or don’t donate. Donors who made all the same decisions are clustered together
- The contributions to a project by the same cluster are added together as if they were the same voting bloc. Then their square root is taken.
- After that the process is the same: sum the square roots of all clusters grouped by project, square the sums, and payout the matching fund proportionally.

In Code:

Thank you to @Joel_m for writing this python function:

def donation_profile_clustermatch(donation_df):
  # run cluster match, using donation profiles as the clusters
  # i.e., everyone who donated to the same set of projects gets put under the same square root.

  # donation_df is expected to be a pandas Dataframe where rows are unique donors, columns are projects, 
  # and entry i,j denote user i's total donation to project j 

  # we'll store donation profiles as binary strings.
  # i.e. say there are four projects total. if an agent donated to project 0, project 1, and project 3, they will be put in cluster "1101".
  # here the indices 0,1,2,3 refer to the ordering in the input list of projects.

  projects = donation_df.columns

  clusters = {} # a dictionary that will map clusters to the total donation amounts coming from those clusters.

  # build up the cluster donation amounts
  for (wallet, donations) in donation_df.iterrows():

    # figure out what cluster the current user is in
    c = ''.join('1' if donations[p] > 0 else '0' for p in projects)

    # now update that cluster's donation amounts (or initialize new donation amounts if this is the first donor from that cluster)
    if c in clusters.keys():
      for p in projects:
        clusters[c][p] += donations[p]
    else:
      clusters[c] = {p: donations[p] for p in projects}

  # now do QF on the clustered donations.
  funding = {p: sum(sqrt(clusters[c][p]) for c in clusters.keys()) ** 2 for p in projects}

  return funding

More Numbers

Here are the calculation details including both matching formulas and pre/post squelching voter numbers and donation amounts.

The ‘base’ totals are the numbers after applying our basic rules: have a passport score over 20 and donate at least $1.

The ‘eligible’ totals are the numbers after applying our sybil squelching based on the rules stated above:

I’ll note that while pulling this data together I found a bug in how my data was being aggregated. I fixed this and it affected the results. To me this underscored the necessary importance of transparency. We need to rapidly move toward turning off post-round squelching and relying only on passport + better QF.

umarkhaneth · September 25, 2023, 5:42am

Hi Priyank, thanks for your question. I can see how this would be confusing at first.

Looking at your data, I agree that your matching seems low when comparing your number of voters to the number of voters of projects who receive similar matching. Digging deeper into your data, I see this is because of how identical your voters are. Of your 187 eligible voters, 151 (over 80%) of them supported only Nawonmesh.

Under Cluster-Match QF, your results are different from what they would be under Simple QF. While Simple QF places great importance on the number of individuals supporting a project, Cluster-Match QF places importance on the number of communities in support. Why is this a good thing?

It means a single special-interest community can’t dominate the matching pool. QF is meant to allocate funding to public goods based on the breadth of support for those goods. This means it would be wrong to allow a single community to dominate just because it has many people. For example, if enough people from my hometown on Long Island decided we needed to improve our parks and we could quickly each give $1 to a gitcoin grant then we would outvote everyone else to claim a lions share of the matching pool by ourselves. This would be unfortunate for every other community who does not benefit from our project. On the other hand, if the same number of people from all across the world supported us (perhaps our parks are growing medical herbs) then Cluster-Match QF would allocate much more funding. I hope this gives the intuition behind the math.

The other benefit of this QF implementation is that it means sybil attackers have to also donate to other projects in order to earn more matching for their own project, raising more money for public goods.

In the long-run, we can continue to improve our QF implementations and imo this is a big step in the right direction. I do see how it can be confusing. Especially given that we did not announce we would switch to Cluster-Match QF before the round began. I still think now is the right time to do so because it means we can right away begin improving how we are allocating funding.

This was probably just a simple miss. Imo your message here is sufficient to opt-in. I am adding Nawonmesh to the list!

DistributedDoge · September 25, 2023, 6:35am

Thank you and rest of Gitcoin team for publishing updated results. This, in my eye, is most informative and logically organized matching report that Gitcoin has produced so far. This greatly increases my confidence in both sybil defense and algorithm presented.

Based on @Joel_m reply to my questions in QED program post post, I can understand rationale for decision to swap algorithms without announcing it first.

Naively clustering by donation profile is only really possible if you don’t tell people you’re going to do it before hand (or if you’re not too worried about people strategizing).

I will attempt to sanity-check base amounts later, but at the moment I find round data plausible and no longer have any reservations against final report or the process itself.

jengajojo · September 25, 2023, 7:07am

Thank you @umarkhaneth and squad for sharing the results as well as the rationale on calculations. Cluster matching is definitely a game changer!

I vote to Ratify the round results

As feedback, maybe there can be a zkproof attached to the results so folks can confirm the calculations match the logic provided. That being said I understand that you cannot reveal all the strategies used to identify the red team but I hope we can strike a healthy middle ground.

I’d like to ask what the time distribution looks for producing these results? are there any specific leverage points which could reduce the time to output significantly?

umarkhaneth · September 25, 2023, 8:38am

Great nuanced points here as expected from @ale.k , especially about how hard it can be to tell the actor behind sybil action. Also thank you for sanity-checking the results.

Thank you for being so engaged and asking for more transparency. This doesn’t happen without that! Please keep doing it

Interesting, I don’t know enough about zkproofs to know if this is possible but would be quite cool if so.

One of the big improvements this round was having regendata which makes it much easier to access clean data sets from grants stack, passport, and onchain. I can’t think of other specific leverage points besides having more/better detection methods ready to go in advance of the round ending

gulliverz · September 25, 2023, 9:44am

It took me couple of seconds to understand the meme

rohit · September 25, 2023, 11:00am

I am unsure if I am using the right vocab, but is it fair to assume that clusters are unique to each core round? i.e., the clusters created for allocating the pool for Eth Infra are mutually exclusive from those created for allocations in Climate. Or are clusters agnostic of core rounds and reflective of decisions made on all projects across all core rounds?

priyank · September 25, 2023, 11:25am

Hi Umar,

Appreciate the detailed explanation. I now understand the reasoning behind the Cluster-Match QF. While that seems like an approach in the right direction, I wanted to highlight few concerns on this, citing Nawonmesh as an example.

In my opinion, urgent climate action needs a bottom-up approach through the formation of many grassroots organizations focused on building local climate resilience by solving local challenges, not just globally appealing projects like medical herbs. In such cases, funding support will have to come primarily from their local communities as these projects might not appeal to global donors, although local climate efforts also contribute to global GHG reduction. Cluster-Match QF seems to be incentivising the opposite approach i.e. a top-down approach and makes Gitcoin unsuitable as a funding source for grassroots climate orgs having only local appeal.
The logic of crowdfunding is to get initial social proof from the existing community of the project owners before the larger community jumps in to donate. That is how any web2 crowdfunding platform also works. Now, if that existing community is supposed to prove their legitimacy by also donating to other projects (who they know nothing about), it just creates lot of unnecessary donor friction.
Two-third of India survives in less than $2/day. Nawonmesh is a grassroots project operational in one of the most backward regions of rural India. You can imagine the financial capabilities of the immediate and extended communities Nawonmesh serves. Convincing these people to (a) resonate with a project having no immediate benefits to them and (b) DONATE $1, that too in CRYPTO, was itself very very difficult. Expecting them to give more to unrelated projects is completely unrealistic. If I had asked them to also donate to other projects, they would not have donated to Nawonmesh too as it would have been an overkill for them.

Also, ChainEye dashboard had 240 passport votes for us. but you have mentioned 187 eligible voters. I am unable to understand this gap.

Thanks a lot. Appreciate it man.

sejalrekhan · September 25, 2023, 3:13pm

Thanks so much for sharing this @umarkhaneth . Really appreciate your time and effort for this. Something similar happened to one of the projects I am associated with > Impact Stream.
Despite the number of votes, the matching amount for us is $10?

I echo @priyank , Cluster-Match QF is defeating the purpose of empowering grassroot level organizations to raise funding from Gitcoin as the only support they can get is from their local community. How will that happen? How will we onboard local orgs solving for challenges on local level? Please excuse my ignorance but would like to learn about this.

owocki · September 25, 2023, 5:06pm

from someone who is an ETH core dev:

https://twitter.com/owocki/status/1706341408452554992?s=46
I don’t know who to flag this to but “iron wallet” is not core Ethereum haha it’s in the name cc @Owocki
same with INTMAX

maybe worth digging in on how they got in the round?

umarkhaneth · September 25, 2023, 6:38pm

Hey Rohit, that’s right on the money. We cluster based on the donations only within a single core round at a time. It’d be a cool experiment to try clustering based on donations to all core rounds and seeing if this gives better results.

Hey Priyank! If our goal is the greatest global GHG reduction then shouldn’t we be searching for projects which most reduce global GHGs and funding those, regardless of if they’re local or not? Climate knowledge is not my forte and I’ll defer to @M0nkeyFl0wer for his opinion on this.

Cluster-Match QF is very bottoms-up however rather than trying to fund the largest single community it focuses on funding those who serve the most communities. You’re right that this may make it unsuitable for orgs with only local appeal (like parks on Long Island). There may be better funding sources out there for local orgs imo. Gitcoin has always been digital-first.

I agree! That’s not the answer I would seek. Instead, if I’m participating in a global funding round I’d ask what the appeal of my project is to people outside my local community and how I can create value for a more diverse supporter base. The behavior we want to reward is cooperation across differences. If a project is supported by people who are very different then that is a strong signal.

I can’t speak to how the ChainEye dashboard was built. If it’s just based on passport score then they’re missing the post-round squelching we do as described here.

Hi Sejal! Thanks for posting on our forum. Did you see the detailed spreadsheet? Impact Stream’s matching increased when going to Cluster Match QF.

This mechanism disempowers uniform, established monoliths and actually empowers grassroots organizations if they’re made of diverse, different members. For example, in the Web3 Community and Education round greenpill network with it’s global, distributed chapters saw an increase in matching funding of $3,787.93 when going to cluster match QF

Hey Owocki! thanks for sharing – will take a look

smith · September 26, 2023, 2:57am

Thank you to everyone who puts their effort into to making Gitcoin and Quadratic Funding a meaningful way to fund public goods and projects. Every time the Tor Project participates in these funding rounds, I am impressed by the amount of collective effort goes into making them run, communicating clearly with the community, and improving over time.

Cluster-Match QF takes the projects you vote for as signals of the communities you belong to. It then calculates matching amounts for each supporter and unique community combination. This method provides more significant matching funding to projects that receive support from more diverse communities.

Awesome. Thank you for making it clear the evolution of QF and the reasoning behind the changes.

Congrats to all the grantees!

rohit · September 26, 2023, 5:30am

There might be one possibility with cluster-matching QF that could benefit local communities. The new algorithm increases the total cost for Sybil attackers and tilts the scale for the system to be “cheaper to defend than attack.” It might be a worthwhile exercise (possibly a prospectively funded project in Citizens Round if anyone is interested) to evaluate if we can lower the requirement for Passport Score in the frontend with cluster-matching QF as an additional rear-guard mechanism to nullify Sybil contributions.

To validate this, someone would need to rerun the squelching with one or two lower passport scores and analyze the impact on the final distribution. If the data supports this hypothesis, a lower score will reduce some friction local communities have in onboarding contributors to Gitcoin Grants.

Here is some background in the 2-minute snippet from @owocki’s conversation with Joel Miller:

solarpunkmaxi · September 26, 2023, 7:27am

hey @umarkhaneth, cluster QF def feels like a step in the right direction.

Just trying to wrap my head around whats happening - Cluster QF filters donors having voted for multiple projects and counts them for QF , so does it also exclude a few donors that projects might have had that have just donated to that single project or that figure needs to cross a certain threshold?

Post Cluster sybil analysis the QF formula applied is the same and you are not tweaking the matching multiple depending on multiplicity of votes from a donor yet?

The difference between base and eligible voters represents the no of votes that projects rcvd from donors just voting for that particular project?

DeFiTeddy · September 26, 2023, 10:26am

First, Thanks for the hard work done for doing the cluster match QF

I just have a question: " the same cluster are added together as if they were the same voting bloc", I do not quite understand it. Say if one cluster/community has 100 voters, it is considered as one voter?

I understand that the votes from the same cluster should be given less weight, but treating them as from one voter is not quite fair especially for some local communities.

priyank · September 26, 2023, 1:58pm

Hi Umar, I agree 100% with your statement. But as of today, none of the climate projects write their GHG reduction potential on their gitcoin grant pages. The simple reason for this is that it is something very difficult to quantify accurately at the early stage of the projects. Due to that, the global appeal of a project does not mean that it has the highest GHG reduction potential. It might just be due to an interesting product, eg. medicinal herbs, or the likability of the founder on twitter interactions. Many grassroots orgs like Nawonmesh work on not-so-interesting things like local regeneration. And are run by senior citizens who are non-digital savvy and non-native English speaker, so spending time on Twitter spaces to showcase their charisma is not their forte (The main reason I am representing Nawonmesh in all the online interactions). It is much easier for them to interact with their local community for support.

Unfortunately, the experience of Nawonmesh’s founder says that the fundraising opportunities for regenerative activities are limited in their region.

Fair logic.

I also feel that we need to increase the donor base of the whole climate round. Compared to the other core rounds, the amount donated and unique donors are way less for the climate round. One of the easy ways to achieve that could have been to let climate projects onboard their communities and then few members from a particular project’s community would have started cross-funding other projects too in subsequent rounds. Imagine if 100 climate projects could bring in just 50 new people, it would have almost doubled the ‘unique donors’ count for the climate round. But, due to Cluster-Match QF, the strategy of onboarding new communities has been somewhat disincentivized.

Why are the “Eligible Voters” numbers different for the same project in the “Climate” and “Climate - Shell” sheets in the updated results?

thedevanshmehta · September 26, 2023, 4:01pm

Thanks to @umarkhaneth and team for all the great work! Here’s some quick comments on the discussion going on

I am against automation of the payout process that @umarkhaneth , @M0nkeyFl0wer & @annika are in favor of.

it’s not just a mathematical formula but a social consensus on the best way to leverage the wisdom of the crowds. And the post analysis, pre-payout period is when some of the most active discussions take place. it would be tragic to let go of this tradition.

My main concern with this rounds distribution is just how closely it mirrors the ‘winner take all’ approach of the real world. Consider this chart i found showing the distribution in the open source round, the inequality is worse than any capitalistic nation.

I wonder if we could develop a gini coefficient or some such metric capturing inequality among projects as a 1st step to possibly reducing it in future rounds. Here’s some interesting research on progressive taxation in quadratic funding systems from DoraHacks thats worth exploring

https://research.dorahacks.io/2021/06/16/reduce-quadratic-funding-inequality-with-a-progressive-tax-system/

I also don’t know how much were following the gitcoin beta round squelching, but the difference that my project received from the 1st spreadsheet to the last was over 30%. These window periods are valuable for getting the community’s assistance in identifying sybil attackers, such as how mini meadows & some others got caught last round in this window period.

I will say that contrary to my expectations, the teams active on gitcoin radio have performed better under cluster QF. Maybe because we each gave to so many different projects that it increased the value of our vote. So while it won’t initially help local convergence, it is certainly helping digital coordination!

I agree with this point, I urge the team to consider making 10 cents the minimum vote for matching. $1 while living in the west is very different from $1 in the global south. Also, 20-35% of my project votes came from those giving less than a dollar, sometimes 10 cents and tragically even a few 95 cents

Finally, I request the team to not publicly list the payout address of projects as many operate in hostile environments where this information could be used against them

berksohto · September 26, 2023, 4:02pm

Following some email exchanges with ben , he made me realize it would be more beneficial for us to shift our discussions to the government forums to embrace a “build in public” approach.

We also see some similar points as @priyank 's regarding our project.

After extensive internal discussions, we made the decision to participate in the Climate-Shell round, so we were opted-in during the application. However, we have noticed that our project is not listed in the Excel file.
@umarkhaneth would you also please kindly look for our project again as numbers show some unfairness I can’t comphrend ?

Marked our Earthist - Decentralize the Seeds project with a magenta color in the climate round, and just to understand the numbers in comparison:
We have the second-highest number of eligible voters and eligible crowdfunding yet our match rating is lowest on below example sheet. While the average contribution for our project is $1.66.

We are eager to gain a better understanding of the situation for such low matchmaking even with high passport granted supporters. Your guidance and support in this matter would be greatly appreciated.

carlosjmelgar · September 26, 2023, 7:06pm

Thanks for all the hard work that went into this @umarkhaneth and other contributors. Seeing QF continue evolving is a beautiful thing. It’s important to consider that the red team is always one step ahead. This requires continues action from the blue team. I saw this topic discussed on the Green Pill Podcast, but seeing it in action hits different.

Fully in support of this statement. This can help reduce the turnaround time in payouts.

I can relate to these challenges faced in the global south. I’d like to highlight a Climate Solutions project that has been working extremely hard in a country where minimum wage is $5 and a family needs 108 minimum wages to sustain a family of 4 with basic needs. Mi Costa de Oro has spent months onboarding their community members to web3 tools (snapshot voting, paying for basic needs with crypto, sending/ receiving tokens when compensated for beach clean ups, minting mirror articles, etc). Despite all their hard work since April, not one of them is able to obtain a Gitcoin Passport score that enables matching. This means they didn’t even attempt to vote in the round. They were able to create impressive results this round by providing constant and transparent proof of their work and work incredibly hard to promote their grant during the Shill spaces. None of their contributors speak English. This means they mustered up the courage to participate in English speaking spaces, request the mic, shill their project in Spanish and hope people understood or someone present could translate for them. I’m pointing this out because your project can take a page from their playbook. 100% of the images published by the Nawonmesh twitter account are AI generated. almost all AI generated images. This looks pretty compared to the low quality images published by Mi Costa de Oro, but they don’t do a great job showing the work and impact being pitched in the grant application.

I’m really interested in learning how you were able to get the contributors to achieve passport scores above 20 points because it has been a huge barrier for the communities I work with. I haven’t been able to get a single contributors in these communities above 8 points.

I can understand the frustration with this, but I also think it’s healthy for the ecosystem. It creates a pluralistic and regenerative environment where people looking to be funded also take the time to become immersed in the ecosystem, learn more about other projects and potentially collaborate or copy pasta some of their work to benefit their local efforts.

I’m in favor of these conversations happening in between rounds in an attempt to establish a structure that doens’t require debate after every round. It’s important to consider that many of the smaller projects are living day to day. Continuously delaying payouts for the sake of big brain back and forth seems like torture to many of these projects. Let’s come up with a more streamlined process and stick to it until something serious breaks and needs fixing.

I’d be interested in seeing how this correlates to userbase. For example - Do Lenster, revoke, JediSwap have a much bigger userbase or transactional volume than projects on the lower end of match funding? If yes, I think this pays out fairly. I don’t know those figures, but my gut tells me the funding received reflects the size of their userbase as well. It would be really interesting to identify projects that didn’t perform well, but house big userbases.

These might be bot donors. It was something that also occurred in the C grants platform, even between rounds. It always confused grantees. I don’t think this is a case of donors giving 95 cents, or less than $1 because that’s all they could afford.

My big question looking ahead is - Wen Cluster Match + trust bonus based on passport score?