What are the KPIs that GitcoinDAO Optimizes for?

[owocki]

Hey everyone,

On the monthly stewards call we began to discuss “What are the KPIs that GitcoinDAO Optimizes for?”.

I want to take some of my thinking on this subject from this post and use them to seed a separate discussion on this topic.

Caveat Emptor

Before reading this post, please know that these are just my opinions. I am one node in the Gitcoin DAO network.

These proposed KPIs are designed to share my sense-making, sharpen my own thinking, and to cross-pollinate ideas to/from others. Others should propose their KPIs and/or their most important thing . If you disagree with my thinking, you should fork & extend my thinking!

The 3 KPIs

As an engineer, I think a lot about optimization problems. In computer science and economics, an optimization problem is the problem of finding the best solution from all feasible solutions. It’s the process of finding a global maxima by traversing a design space.

These are the three big optimization problems I think Gitcoin/GitcoinDAO could focus on to achieve its mission.

1. Ecosystem Impact.

How much positive ecosystem impact are we having?

Basically, how many builders are we supporting? How many builders have we helped find new earning, learning, connection opportunities in the Open Internet? How many builders have quit their corporate jobs to work on open source? How many are staying in the ecosystem because they are partially or fully supported by Gitcoin?

I think of this like a funnel,

1. How many developers can we source new opportunities for?
2. How many can we help select an opportunity that is a fit for them?
3. How can we help funding opportunities sell themselves to developers? How can we help developers sell their work to funders?
4. How can we onboard them into ecosystems to produce positive impact?
5. Once they are earning in these ecosystems, how can we retain them?

2. Interconnectedness

I am a big believer that It’s all coordination. That as human beings , we are interconnected and interdependent on each other for survival. How can we align our incentives better? How can we deepen our ability to coordinate? Surely if we can solve this, we can get closer to achieving our mission.

As a way of aligning incentives and truly deepening our ecosystem impact, I envision a DAO of DAOs - a mesh network of interconnected projects that all rise and fall in utility together. This DAO of DAOs would deeply interoperate with each other from a software/culture and perhaps even by swapping governance rights with each other.

3. Decentralization

I am extremely proud that when Gitcoin launched in 2017, we were focused on our core mission of economically empowering software engineers to build/fund public goods. We ignored the trend of the time, which was to write a lengthy whitepaper and do an ICO.

As a result, we have had a ton of positive impact on the world. It was an amazing feeling at EthCC to have 2 dozen hackers from across the ecosystem come up to me to shake my hand and say “hey Gitcoin had an impact on my life, awesome.”. It feels great to have helped facilitate the delivery of $26mm to OSS devs across the world.

But, the negative result of the 2017 tradeoff is that Gitcoin has some centralization debt. In the 2017-2020 market cycle we swept decentralization under the rug to focus on impact.

Given that Gitcoin has now had a positive impact, there is now actually something to govern. We want to adhere to ecosystem best practices

• Antifragility - Would Gitcoin’s impact and mission endure if Kevin got hit by a bus?
• Credible Neutrality - Are Gitcoin’s software & governance systems credibly neutral - eg are the mechanisms provably fair to all parties?

Right now the answer to these questions is no. So I believe that now it is time to pay down that decentralization debt and make Gitcoin more credibly neutral and antifragile.

That means finding practical and regulatorily compliant ways of decentralizing the governance of the network, computation of the network, development of the network, and the economics of the network.

[Toby]

Thanks for reposting Kevin.

In my mind, we need quantifiable KPIs. In my mind “Ecosystem Impact” in your framework is the one that is most amenable to quantifying. Since the DAO is not in charge of the hackathon and Kernel business lines, we should restrict this to Grants, with an increasing focus on dGrants.

“Interconnectedness” — I’m not sure what metrics this would correspond to.
“Decentralization” — not sure what metrics this would correspond to.
These strike me as operational values, not KPIs — we know we’re only doing things right if we satisfy interconnected decentralization in the process.

With this in mind I’d like to put forward the following high level KPIs that all projects should be aiming to work on. In order to facilitate this, we’ll need support from Gitcoin core product team as well as the dGrants team. CCing @phutchins.

Grants ecosystem impact KPIs

• $ Matching funds raised
• Number of matching pool contributors / funder’s league members
• Number of individual contributors
• Average contribution per project
• Average contribution per contributor

These are high level enough to be broken down into subKPIs that different workstreams/orgs can be responsible for. If we want to expand the mandate here, we might include something like

• Number of individual developers who were able to work on OSS tech full time because of Gitcoin.

Looking for feedback on this!

[bobjiang]

Hi @Toby

I like the impact KPIs you listed, and how about to add some negative side KPI like:

• grant fraud tax rate

[Toby]

Excellent addition Bob let’s definitely add that. Any others come to mind?

[linda]

Really great points @owocki!

I also agree with @Toby on quantifying specific KPIs around grants. I like all of the proposed impact KPIs so far including @bobjiang’s grant fraud tax rate.

The only one that I’m not sure what we are optimizing for is “Average contribution per project.” If there are a lot of smaller projects that don’t need a large amount of funding then I’m not sure we need to be optimizing for a large average contribution per project. If we are trying to keep the number of matching pool contributors and average contribution per contributor high then I don’t necessarily think we need the average contribution per project to be high.

[Pop]

This makes me think of my tweet here and I definitely support @Toby 's KPI idea as well as @bobjiang’s negative scoring proposal.

Perhaps it can all be on a sliding scale of DAO DOs & DON’Ts with a certain % average formula that qualifies contributors based on subscores for Gov, Comp, Dev, Economic. All this data would be easily explorable so anyone wanting to know more about people’s activity could check it based on the criteria above

[ceresstation]

I’ve started working with the public goods workstream to put together a list of initial KPIs / high level goals that might be informative as well.

Public Goods Funding Workstream Overview [DRAFT]

In general, I’m a huge fan of what @Toby suggested here, our goal is to help public goods projects get the funding they need, and every initiative we take on should have connect back to that goal.

[frankchen07]

Chiming in here since there’s quite a bit of context from cGrants that we can bring to this conversation.

1-2 years ago we did attempt to measure grants that made it to some level of “graduation” into a fully-funded project or even a company.

Obviously, it wouldn’t necessarily be fair to measure most projects like that (not all successful projects require additional funding or need to be turned into a company), but there are some proxy measures for figuring out traction.

For us, it was a combination of understanding the median total amount contributed (including the CLR match) cross referenced with social popularity. The CLR match amount takes into account (to some extent), the number of contributors, so that information is rolled up.

From there we can choose to categorize into specifics - what the median looks like for dApps, NFT-related projects, etc. Cross-referenced with some grant owner interviews, we can put together a picture of how much funding is necessary to really help “move” project, and set threshold baselines.

Then it’s just a simple query to categorize projects as a start and see how we can best help them move the line.

[alisha]

Agree that the KPIs mentioned by @Toby are important. Is there a way to rework the KPIs suggested so they fall under broader headings that are more focused on grant round performance and ecosystem impact?

To measure ecosystem impact, the core KPIs should focus on measuring the impact that grant rounds have on participants. From a participant’s point of view, a grant round does three things:

1. Boosts project discoverability;
2. Provides an opportunity for the community to donate to a project; and
3. Provides matching through quadratic funding;

What actually matters is the impact of the round itself in terms of boosting the discoverability of projects and giving the community the opportunity to donate (vote) for projects. This, in turn, allows the quadratic funding mechanism to function as well as possible.

In my mind, if the focus is on the impact of grant rounds, there are three main areas of importance:

1. Impact of Discoverability;
2. Impact of Community Donations; and
3. Impact of Matching.

Impact of Discoverability might include things like:

• Average contribution per project
• Website analytics: bounce rate, views per page, add to cart, projects per transaction
• Social media growth of projects e.g. Twitter growth before and after round
• Impact of connection/exposure to new team/community/funders

Impact of Community Donations might include things like:

• Number of individual contributors
• Average contribution amount
• Number of grants contributed to
• Grant fraud tax rate
• Interviews with participants
• Participant surveys

Impact of Matching might include things like:

• $ matching funds raised
• $ matching distributed based on size of teams
• Number of Funders League contributors
• Funder surveys

Having broader focus areas would make it easier to capture both quantitative data and qualitative narratives. Ecosystem impact can’t be measured by the number of donations in a round or the corresponding matching distribution. Success is about understanding the impact of the money and the impact of Gitcoin grants rounds on participants and the wider ecosystem.

The easiest way of doing this might be to view the many data points available from a Grants Round within these three areas. This approach then allows the Public Goods workstream to optimize efforts and allocate funding in a way that will have the most enduring impact within the ecosystem.

[federicosan]

An overall KPI: number of new part-time full-time open source contributors/builders. If this one increases over time, all other KPIs proposed should increase. This could be done with polls.

One of my concerns is that for someone to quit the traditional job and work full time on open source software these things could happen:

• The subject has a contract or is sponsored (similar to a traditional job, the subjects has a salary),
• The subject gets suddenly wealthy with its OSS project
• The subject trusts in the community and is sure contributions will keep coming in the future.

The last of these is my concern, an engineer that has been working full-time for 5 years and has a lifestyle, could get a grant but the time to full employment without salary could take long unless it receives a lot of support. For this I think the way grants work should be different, there should be some sort of security into the future a cover or fail-safe because future contributions seem arbitrary. I suppose the way this is working right now is that grant by grant the builder gets more confident on future support and after many grants, support has grown, and quitting becomes an option for the builder.
I want to check if this is the real experience builders are having. I want to think of ways to improve upon this. I would call this the builder security KPI its equivalence from the traditional job market could be Employee satisfaction

A poll could bring this data
How confident are you in getting a future grant 0 to 100%
How does not having a total warranty of future contributions affect your project?
What could improve your experience?

A possible solution (if you agree there is a problem), add the patron or subscription model to the grants system.

[karrie_kim]

@owocki

please know that these are just my opinions. I am one node in the Gitcoin DAO network.

It’s my tiny curiosity out of KPI agenda.

This comment brought me a lot of inspiration. Who is real owner of a DAO? I really respect owocki’s documents that I can found everywhere in this site and the effort to make Gitcoin well organized. So I cannot think that this post is just a member’s opinion. It absolutely has powerful impact than other’s viewpoint. Sometimes it could fix other ppl’s thinking.

For example, we know that Ethereum is not just for Vitalik. But we consider Vitalik as the main owner of Ethereum. Because everybody knows that this genius guy initially founded Ethereum. At the end, he still have a great impact on decision making on Ethereum. Eventhough Ethereum has built a fair decentralized system, but is still relied on Vitalik. What if he exit from community? What if he decided to believe other main net system and work for it?

I’m just curious about fully anonymous posting and/or voting system on DAO. Does it helpful for the members of DAO to feel that they are real owner, not having powerful whale.

[lee0007]

I believe there is need for both identified and anonymous voting processes in DAO.

I like the polling process in Discourse that hides the results of votes until you have voted…which in the cases of community sentiment or temperature checks, can limit propensity to ‘vote with the majority’.

Imo in this case opening the topic of KPIs by identified leadership - because it requires very broad context - is useful for driving the conversation.

I’d be interested to hear your thoughts on which scenarios in DAO you’d like to see anonymous voting.

[bobjiang]

@lee0007

I like the polling process in Discourse that hides the results of votes until you have voted…which in the cases of community sentiment or temperature checks, can limit propensity to ‘vote with the majority’.

According to your like, (I like it too) seems we could propose a new feature for snapshot.

[new] hide existing voting results to non-voted members.