It does seem like this could be a tension you would bring up, discuss with or otherwise confront at some level prior to posting here. It is quite a shock to see you advocating for laying-off 10 people in 10 days. This doesnât present any consideration for the psychological safety of contributors in or out of FDD.
Iterative improvements to passport with data science is what FDD is currently doing.
We havenât done one test looking at the results of Passport Scoring as a Service (PSaaS) effectiveness during the Gitcoin Alpha round, the first round to use it. PSaaS is an algorithm custom designed by FDD this season. We gave 3 other options including a cost of forgery model. The plan which we very clearly articulated at the beginning of the season is that a âone-score-to-rule-them-allâ model could be iterated on over multiple uses, but by no means would be perfect from the start!
You get the benefit of talking with one analyst or data scientist from our team and assuming that they are doing all the work of FDD. Our team doesnât work this way. Most efforts of any single contributor are supported by others on the team.
I personally recommended that you hire a data scientist onto the passport team to help transition both the Passport team and FDD to a post protocol launch future. We discussed this a few days ago.
The idea that there isnât a need for sybil defense is a mistake.
GPC informed us that there would be no need for our sybil defense work for the Unicef, Fantom, and Gitcoin Alpha round. I am not sure why they insisted this, but luckily we still performed these checks (as is our mandate) and uncovered substantial fraud in the Fantom round. Some of the same wallets which attacked the Fantom round are ALREADY attacking the main round.
Our analysis of GR15 clearly showed that passport alone is NOT ready to be the only method for sybil defense.
There are ethical tradeoffs to using preventative vs reactive sybil mitigation techniques and for the protocol to be unopinionated we will need to offer both.
For those who are not aware, both preventative (passport gating/weighting) and reactive (squelching - giving a user a 0 coefficient based on in round behavior) have ethical tradeoffs.
Gating over multiple instances compound inequality building a privileged class. This is the fundamental problem with moving to meritocracy in a system where the players do not start at equal levels. Additionally, it leaves little room for new users to participate.
Squelching is like removing ballots after the person voted. However, it opens the aperture for participation that is innocent until proven guilty. It would be quite assumptive of us to assume preventative sybil defense is the only ethical option when it is KNOWN to cause systemic problems.
Are we sure we want to make the protocol opinionated in a way that seems more ethical today (not removing votes already cast) in favor of creating systemic inequality in the democratic fund distribution of public goods in web 3? Especially when the FDD answer is to instead provide transparency and reproducibility to any algorithms used for removing votes in a reactive way?
This is said as though you have a clear solution for solving sybil defense. As for the two items that you bring up:
SAD model - This was still in use in GR15. Instead of it being the ONLY model used to predict how likely an account is sybil, we instead made it one of multiple models. We did this because the SAD model was great when there were primarly web 3 devs supporting open source. When the cause rounds like Climate Change started, we suddenly saw a new wave of donors without a Github.
Discovering new behaviors is part of the fraud detection effort. it is an iterative game. As old signals are deprecated, we find and validate new signals like the onchain intersectionality report.
Matrix was an interesting case where we had an idea that would have massively helped the DAO at the point we are now imho. However, FDD was voted not to work on this and the contributor who intended to work on this moved on. It wasnât unreasonable to think we could build this simulation engine as here is one for reward modeling for grant reviewers.
The problem we are supposed to solve is sybil defense. We are not shipping production software because it is an unsolved research problem. We need a solution prior to shipping software. The sybil scoring legos are an attempt to ship an MVP that pulls these past learnings together.
1 - Crowdsourcing analysis is more efficient
2 - Contributors need a way to build on each others success and not âreinvent the wheelâ every time a new analysts participates
3 - The scoring application needs a standardized format which can continually be updated with new analysis based on new behaviors to empower non-technical users
Lastly, Iâll remind everyone that when the DAO started we were excited about being open and letting anyone participate. The goal for workstreams was not that we should be as efficient as possible and exceed our goals. We hoped this would happen, but at the time it was more important to be open to participation and still hit objectives. We did this. When the bear hit, the DAO took a transformational turn and we followed suit.
Mostly agree, but we arenât there yet. The bounties that FDD included in the hackathon needed to be created. The ODC can continue exploratory analysis to find new detection methods, but FDD will need to productize them to be used by program managers.
Here is an early breakdown of how we might split responsibilities between ODC & FDD.
It is exactly that, a discussion. That discussion would be much better to have without the overtone of âI think we should fire this workstreamâ.
Stewards are asking about sustainability. My response is to DISCUSS potential ideas. It seems that your understanding is that if people bring up discussions that you (or maybe your perception of the current DAO zeitgeist?) donât agree with, that is counter-productive to achieving our EIs? I donât think it is a good idea to set a precedent that we should defund any workstream that dares to have independent thoughts.
To this logic, I think your post advocating for firing 10 people in another workstream 10 days from now demonstrates a significant divergence from the standard operating model of workstreams as cooperative budgeted teams that seek to achieve the DAOâs EIs - HOWEVER - I do not think we should defund your workstream.
This is not going to happen. You are welcome to not vote for our budget.
I would love to do this! It was my understanding that spinning out workstreams was seen as potential defectionâŚ
As mentioned above, I do think Passport could potentially benefit from having a data scientist on their core team. I do not think that it makes sense to lose the FDD support for this person AT THIS TIME.
Also, who is dynamically monitoring grant behavior for fraudulent actions if we move the data science and analysis into the Passport working group?
We know sybil resistance needs more than just the preventative model offered by the passport. Your suggestion here is that although we have the agreement of multiple data analysts, data scientists, and also math, you still think that Passport alone is the answer. How do you resolve the ethical dilemma of bias compounding over time to create a privileged class?
Why wouldnât we finish validating our prototypes to be used by an FDD Fraud Consultant and a PGF Round Operator during the beta rounds in April. Then, we might discuss moving the prototypes to the next phase of delivering production ready software.
Here are the thoughts you refer to:
and
Additionally, why would you think FDD contributors should work unpaid to design governance processes for the DAO? This is a stretch from our mandate and common sense.
This is a blatantly false statement. We found substantial fraud in the Fantom round and the same wallets are already hitting the Alpha round. I shared this at the Tuesday weekly and the CSDO meeting.
You say this as though the DAO hasnât been evaluating our work and yours every three months. Yes, we would love to get more people to pay attention and care about sybil resistance and our solutions. It goes a little far imho, to call out the stewards who have been deep diving on our budget for not doing their jobs.
Exactly! This needs a process. That is why I am talking about it. I DO think FDD should likely become a smaller workstream with more direct support to ODC in the future. I donât think we should fire everyone in FDD 10 days from now AND think that would be a horrible precedent to set.
The S16 budget was put together without knowing whether there would be a GR16. Additionally, we didnât know when which parts of the protocol would launch necessitating flexibility in our objectives. I have kept the stewards up to date with the changing tasks/projects we have prioritized as the season developed. GPC has done a great job of meeting the deadlines for product launches this season, but these deadlines were set AFTER the budgets were posted.
Here is a miro with the latest updates as to what FDD has completed this season and what we wonât do that was initially considered. This should not be surprising as you were one of the people who pushed for us to have âflatâ (although most workstreams increased) budgets without much scrutiny on objectives. Please donât call FDD out for operating under the agreements you supported.
We are funding the workstream because sybil defense is critical to Gitcoinâs success AND WE KNOW that Passport is not enough at this time! We want to finish the job.
I agree with most of this, but FDD has NOT run itâs course and itâs responsibilities are still numerous.
As a steward, I support your right to vote and campaign as you choose.