Yesterday I noticed that some users on Twitter were raising complaints about the lack of attention that identity verification has had in Gitcoin governance. So I spent the evening trying to evaluate how we can improve on that.
This is an initial proposal to get discuss how we can better address our process of onboarding and evaluating sybil resistance measures.
Executive Summary of Proposed Changes:
-
Creates an approach for how the community should propose, vet, score and authorize new ID methods.
-
Creates a standard scoring process for identifying the merits and risks of each identity verification method.
-
Creates a subgroup of elected stewards to manage and maintain that process and scoring methodology.
-
Creates a standard bounty test to demonstrate the validity of each protocol.
-
Cleans up current ID methods and removes non-beneficial methods including:
- Lowering all current +% scores
- Permanently removing Google and SMS verification
- Temporarily removing Idena subject to it passing an additional confidence test and working with the community to improve its score.
Current Problem:
For Gitcoinās governance, there is nothing more important than maintaining confidence in Sybil resistance in the Quadratic Funding grants.
This is a process that currently governance delegates have not been a part of and I think our current identity confirmation processes are too lax, overly reward protocols, and lack a standard objective approach.
The Goal of Identity Linking:
The goal of verifying an individuals identity in Quadratic Funding is obvious, but the goal of connecting multiple accounts to create an identity score may not be.
In connecting additional āverificationā methods we should be either:
- Trusting the robustness of the verification method itself to provide a high confidence unique verification.
or
- Gaining user behavior indicators that Gitcoin can use to gain ML confidence in identifying non-human actors.
Stewards of the Community:
At the same time, we should be responsible and respectful stewards of the open source and decentralized communities and not support methods that are:
- Standardizing reckless or risky security behaviors.
- Creating inequality of access or opportunity (by assuming cultural or economic behaviors are what make human interaction)
- Creating exclusionary boundaries (by requiring verification from already known actors)
- Productizing or unfairly/unknowingly tracking users.
- Turning users into the product.
- Preying on economically disadvantaged users for sake of growing their product through incentive schemes.
A New Process:
To this end, Iād propose a new set of processes for identity solutions to be added.
This process would be maintained by a sub-group of delegates to be voted in, and called āThe Sybil Stewardsā
A) For Decentralized Unique Identity Processes:
- A user, or developer can propose adding a new verification protocol on the governance forum. This would include a:
- Description of the service
- A step by step break down of the verification process
- How/if any costs or tokenomics are involved
- Why the user believes this is an adequate identity service.
-
The user would submit a scoring template [see scoring section below] with the scores they believe the service should have.
-
The community would debate and test the identity protocol within the thread. Within 14 days, based on the community feedback, the Sybil Stewards would present their proposed score card for the protocol.
-
Once a protocol has been scored, it may then be sponsored. A sponsoring user (either the developer, a third-party user, or the Gitcoin DAO if it is interested) would sponsor the proposal by placing a $1000 bounty of $GTC into escrow. From there, the community will have 30 - 90 days (depending on if the protocol takes time to verify) to attempt to become verified on multiple accounts. The first user to show the Sybil Stewards they have successfully created multiple verified accounts on the protocol would win the bounty. If no user can do it by the time it is expired, then the bounty is returned to the sponsor.
-
The outcome of the bounty is then added to the scorecard. So long as the protocol meets the minimum score (0) it would then be voted on via a Governance Snapshot to be added to the next grant round.
-
The specific % of funding boost added would be determined by its score bracket.
B) For Centralized Unique Identity services and Behavioral Services
In the case of centralized services and user behavior based services (i.e. Twitter), Gitcoin (the company/developer of Gitcoin) can add these services by:
- Putting forward a proposal with score on the governance form, along with a scoring template filled in and:
- A Description of the service
- How/if any costs or tokenomics are involved
- Why the user believes this is an adequate identity service, or why they believe the behavioral data is beneficial to detecting bots, and non-invasive.
-
Allowing at least 14 days for users to investigate and raise concerns.
-
Creating a governance vote for the approval of the addition.
These services should still have their percentages determined by their scores on the scoring template.
C) Removal
At any time, any steward may put forward a proposal to adjust, prorate or entirely remove an identity method.
This should be done first with a governance thread to discuss the proposal and then a formal governance vote.
Scoring Template
In order to create a more objective model of determining the security, correctness and usefulness of different identity protocols, Iāve attempted to create a rough scoring template.
The expectation is that the community would vote on and improve this scoring template over time.
Right now the sections are:
-
Requires Public or Semi-Public Posting of Identity Info (-5 to +5): If it puts user information under public scrutiny it becomes exclusionary for large populations. Ideal solutions should not require public information (+5), ones that do require public information should keep it gated or secure (0), and not display it in a public web or app especially to external actors (-5)
-
Is Unique (+10 to -5): Does the integration tie us to a āunique personā. The method should not just aim to prove a unique account, or that there is a single human, but that said human is attached to a profile, each person has only one profile, and no two people have profiles on the same identity. 1-10 scored on the success of this. Negative numbers when there is uniqueness of an individual but not an identity. -5 when it is entirely non-unique.
-
High Confidence Unique (0 to +10): Based on the approach and methodology, how confident can we be that the person is unique, does not own more than one profile, no two profiles are the same person, the profile is active, current, not spoofed, not deceased and is currently owned by the user it claims to be owned by. Currently this is nearly impossible with all options we currently have and is likely only manageable by two-factor, KYC/AML verified, centralized services. However, some services like BrightID which have liveness, and identities built over time begin to get some increased confidence due to the constant effort of human interaction required to build out. The intention of this score is to account for the fact that centralized services trade-off other benefits for high confidence.
-
Requires PII (0 to -5): Does this require the user to share, especially with a third-party, personally identifiable information. That can range from their real name (-1) to sensitive document uploads to a 3rd party (-5)
-
Requires Invasiveness Software/Hardware/Public (0 to -5): We should not encourage verification methods that are invasive to the users privacy or security. This includes exposing their public information, but also the download or use of specific hardware or software that may be used in tracking. It can range from using a specific web app (0), to a browser extension (-3), to having to download desktop software (-5)
-
Can be spoofed (0 to -5): While a criteria that will not apply to most services, some protocols may be spoofed or altered such as IP addresses, MAC addresses and in some indirect manners phone numbers. (We would also include the risk of things like SIM swapping attacks under spoofing) in all such cases, a -5 should be awarded for any identifier that can be spoofed.
-
Easy to Use (+5 to -5): The ease of the use of the service (both to sign up, activate, verify if applicable, and connect it to Gitcoin). We should check our own bias here frequently and assume we are referring to the ease of use for a mainstream user who may find something like installing and using MetaMask to have complexity. A ubiquitous service like Google or Facebook with a OAuth login button is a good example of a 5.
-
Makes User the Product (+5 to -5): While it is fine for their to be for-profit services, we should take care in not creating incentives to drive users to places where they are the product. While this may be unavoidable at times, it should be a weighted tradeoff. Services that simply show ads to users based on profiles are likely a -1 to -3, whereas services that sell user data are a -5. Currently, no existing cases of positive numbers exist, but a wholly decentralized identity chain may in the future actually allow a user to deproduct themselves on other services which may merit positive scores.
-
Ephemeral/Changing Source (+2 to -2): If the source of the verification can be ephemeral or changing (i.e. a temporary phone number/email, or an account that can change hands) it should be -2. If it cannot be swapped out easily it should be a +2. This is a small balancing score to adjust for small risks where accounts could be out of service but have not been sold for the purpose of sybil attacks.
-
Has liveness (+5 to 0): Liveness scores cover two possible scenarios. The first is if during the verification process the person was live (in person) or live (over constant video). This being superior to something recorded. However, in lieu of being live at point of verification, a service could instead choose to be constantly validating through constant micro-verifications telling us that this person is the proposed identity currently. An example of this comparison is that BrightID requires constant behavioral interactions to grow an evolving score, where as Idena requires a verification every 20 days that has a binary output. This is one of the most complex criteria and may merit additional refinement in the future.
-
Is Non-Subjective (+8 to 0): This criteria asks if the verification of the user requires the subjective interpretation of other people, or if it is a binary criteria outcome not subjective to human interpretation. (For clarity, verifying someoneās identity documents its an objective test even though a human is performing it). A non-subjective service results in greater fair access and would earn a positive score.
-
Is Decentralized (+5 to 0): This criteria asks if the service is decentralized. This does not simply mean that there is a blockchain involved. To get a perfect +5 score a protocol should be decentralized (as in on a blockchain) and decentralized in the fashion that it does not depend on a single app, centralized service, company, software or website for its access. Currently, only ENS meets the +5 score on this list.
-
Cost the User (0 to -10): If there is a price to the user (through tokens, fiat or even through tasks that are not directly related to the process of them verifying themselves) the score should be lowered. Any cost of verifying identity creates a bias towards users not only who are economically well off, but, even just have simple access to the payment rails supported by that protocol.
-
Grants Behavioral Insights (+10 to 0): While we donāt want to encourage tracking or privacy violations, there are times when accessing reasonable, public, non-PII information via systems like a social graph, social network firehose, or TheGraph node would allow us to determine a users behavioral profile. An example is connecting a Google account only proves a user created a Google account, so this holds no value. But, connecting a Twitter account allows us to see a users tweet activity, and social connections. This public information does not violate user privacy or create more held PII risk, but, can be an additional signal of humanity. Just like this category we should never deduct value from a user due to a lack of social insights, nor have it be a necessary nor sufficient condition for validating an identity. It should always merely be icing on the cake.
-
Involves 3rd Party Tracking (-8 to 0): We should aim to discourage and discount methods that involve tracking by third-parties. Either centralized social networks, or situations where users are required to interact via specific apps or softwares that could be monetized or productized.
-
Poor Security Practices (-5 to 0): We should discourage the use of applications that have poor security practices. A Gitcoin stewards we have a responsibility to promote responsible web3 behaviors, especially to newer users. An example of something that would merit a penalty is Idenaās web login system. An account is generated for the user, and they are asked to enter their private key and password in a web browser to log in. The private key is not only entered into a web browser, but is fully viewable in plain text.
-
Works Globally(+10 to -5): We should aim to give additional weight to services that work globally. Ideally not relying on specific IDs or type of IDs, and not relying on access to specific technology, lifestyles or activities (i.e. having a unique IP address/MAC address/Phone Number is seen as common in many parts of the world, but in economically disadvantaged regions, sharing of a smart phone between multiple people is a common practice)
-
Has Equity of Access (+10 to -10): We should also focus on services that create equitable opportunity for a user to enter into the space and be fairly verified. Services that require specific economics, time commitments, time of day commitments, costs or travel can make it challenging for different demographics to get involved. At the same time, systems that require users to already know other users and get verified in concentric circles are exclusionary to isolated new entrants to the market (which is actually something we want to encourage in web3).
-
Use of Profit Incentives (-5): Profit incentives create the inverse problem, wherein the prey on the economically disadvantaged to seed their product and create a risk of economic attack. While decentralized systems may need some sort of token, reputation or incentive model to encourage the participation of good actors, this should never come in the form of promoting or advertising the ability to earn money from the protocol. Systems promising income for being verified, or for participating in verification, or successfully verifying or challenging other users create incentives that work against the trust of the system.
-
Could Buy Accounts (-8 to 0): A measure of both, is it possible, is it easy and is it likely that accounts could be bought and sold. In a case where it may be possible to sell accounts, but there may be economic or technological disincentives that make it challenging to do (such as slashing, or the account only being transferable by a non-changeable private key)
-
Can Have Collusion (-5 to 0): A measure of how easy and likely it is that users within the protocol (or one user with many accounts) could collude to verify additional non-real users, or to make the protocol less secure over time.
-
Can Be Botted (-8 to 0): A measure of the possibility of using bots to create or verify new accounts. Things like CAPTCHAs, web sessions, randomness or synchronicity, should not be considered disqualifications in the ability for a user to create and use bots.
-
Needs Reverification (-5 to 0): Does the identity need to be reverified again in the future? This may seem only the opposite of a liveness test, but protocols can maintain liveness without needing explicit reverification by doing continual improvement/interactions rather than binary outcomes over time. Reverification creates an undue burden on the user, requires an additional checking burden on Gitcoin, and means that trust of it being the same user likely decays between two verifications until the next is completed.
-
Easy to Maintain (0 to +5): This is a measure of the ability to maintain the identity, either by keeping an account active or renewed, or by completing reverification. i.e. a manually ENS renewal with a price is less easy to maintain than having been verified once on another service.
-
Subjective Challenging (-5 to +5): Does the system allow others to challenge the identity of another user. If not then 0. If it does, it should be a -5 if that is subjective and a +5 if it is a wholly objective method.
-
Challenge Passed (+25, 0, -10): If the protocol claims to verify unique identities and has done a challenge it is +15 points if no user was able to get multiple accounts verified, and up to -10 points if they were. Protocols that do not claim unique identity verification or who have not yet done such a challenge should be assigned a 0.
Scoring Levels:
< 0 - Should be removed or denied currently.
0 - 10 - May be approved by governance, but should have no higher a score than +5%
10 - 20 - May be approved by governance, but should have no higher a score than +10%
20 - 30 - May be approved by governance, but should have no higher a score than +15%
30 - 50 - May be approved by governance, but should have no higher a score than +25%
50 - 80 - May be approved by governance, but should have no higher a score than +40%
80+ - May be approved by governance, but should have no higher a score than +60%
Wider Bands:
In turn, a users score should start at 50% power and may progress to a maximum of +250%
This model with lower rewards per method, a lower starting vote power and a higher voting cap, allows Gitcoin to introduce more verification methods over time and more heavily weight governance towards users who complete multiple methods.
The lower curve on the scoring level means that even if a user cannot complete preferred methods, they can likely complete many less preferred methods and still take part within Gitcoin grants.
Current Scores:
Iāve created a public Google sheet with rough scores of each of the current verification methods scored using the model proposed above: Gitcoin Scoring Template - Google Sheets
The scores for the current methods are as follows:
Under this proposal we would be:
- Setting BrightID to +25% verification
- Setting POAP to +10% verification
- Setting ENS to +8% verification
- Setting Twitter to +8% verification
- Setting Proof of Humanity to +5% verification
- Setting Facebook to +5% verification
- Removing Google from verification
- Removing SMS from verification
- Removing Idena from verification. [Temporary]
Understanding the Idena Score
In this model, Idena scored incredibly low.
Thatās not because I believe Idena is a bad concept, or cannot become a useful identity service. But, right now their set up lacks the robustness to have fair, equitable and high confidence uniqueness and doesnāt encourage the right user behaviors.
Idena requires users to perform validation ceremonies every 20 days. It is done at one set time, and lasts only for 2 minutes. It is at this same hour of the day each 20 days.
Users create āflipsā which are two option questions with a series of pictures. Users must then successfully complete a challenge where in they successfully answer a series of āflipsā correctly at the validation ceremony.
With 6 binary outcome options (0 v 1), a user is required to get >75% of them right to succeed. This means a user has a roughly 9.38% chance of getting it correct when clicking at random.
It also requires users to invite other users and to contribute to flips (creating content for them) in order to reach a verified status, and to maintain that in successive challenges.
To join a validation session users must either download and run a desktop software, or pay to connect to a remote node in a web browser session.
That web browser session involves entering, in plain text, into the web browser, a private key.
Connecting to a remote node costs only $0.08 per session.
This model of accessibility creates a low threshold for an economic attack. At $80 spent, you could attempt 1000 verification bots clicking at random across 1000 sessions. In turn, youād likely end up with 93 accounts.
Those 93 accounts could contribute known flips to the pool, colluding to make the network verification less susceptible to them specifically in the future.
Working with Idena:
While I think Idena should be removed from verification currently, I will reiterate, its not because its a bad concept. Itās current state simply presents a lot of risk.
Iād suggest that Gitcoin allocate a sponsorship of $1000 of $GTC for people to test Idena in an attempt to get duplicate accounts over the next 90 days.
If no one is able to claim that, then that would give the Idena the +25 score it needs to reach 0 and remain eligible at a highly reduced percent. They can from there continue to improve and iterate.
(Idena team: Iām sure myself and others would be happy to work with you to help you find some improvements that can be made to improve that score and make it more beneficial for both parties)
Delegation:
If you are interested in supporting this work but donāt have time to be involved in the process, you can feel free to delegate voting power to me by pasting ā0xf07A2439296e07Bc4320AF924E655a01fb69D89Cā under the delegation section of your Gitcoin dashboard: gitcoin.co/quadraticlands/dashboard
