Figuring out auction parameters
This is the continuation of the post above - the next step. Also a duplicate from the newest section in Upala docs.
This document is intended to help you build intuition on how to set parameters of the Score and Pool size auctions specific to your project and human verification methods that you use. The auctions are described here - Gentle PoF discovery methodology.
The most complex parameter is the Upper pool size limit. It is described in details. Others are touched just slightly.
Score auction parameters
Bid duration, bid step, starting bid, step down
When setting up these parameters one should consider time vs money polarity. You can get PoF a lot faster if you have significant budget. If you wanna cut costs on the PoF measurement campaign go slow (start with low bids, increase bid duration, make small steps up and big steps down).
Liquidations threshold
See lower pool size limit.
Pool size auction parameters
Number of steps and Pool size step
Decide on one of these and calculate the other.
Pool_size_step = (Upper_pool_size_limit - Lower_pool_size_limit) / Number_of_steps
Number of steps can increase accuracy of the discovered “Score - Pool size” pair. The only thing that prevents it from being too granular is gas costs (transaction fees). Try to keep steps small.
Lower pool size limit
Pool size which we need to set at the start of the pool size auction.
If we could predict the liquidations threshold (the liquidations that happen due to non-Sybil reasons) that value could be used to define the starting pool size. But we don’t know what this threshold is and all we can do here is guessing.
One thing to consider here is duration. If we start too low, the climb will take longer. It makes sense to start with a considerable sum. $500 - $1000 should do in general.
Cute pink fluffy robot fairy, hd, highly detailed, fairytale, portrait
Upper pool size limit
The upper limit should be defined with care. It is platform specific.
Imagine we have $10,000,000 in our pool. Even with low PoF that would probably create enough incentive to develop a new exploit for the Human Verification method that we are trying to asses. While this may be of a particular use, we are interested in measurements only. Thus the upper limit should be meaningful to our platform. And it should be economical to say the least.
Below are the things to consider when setting up this parameter.
Revenue and Extractable value
We can try to make malicious actors hunt for the Upala pool and not for the value encapsulated in our platform (e.g. matching funds in case of Gitcoin). A good starting point is to think of malicious actors revenue. If they are trying to extract value from our platform the revenue looks like this:
Fraudster_Revenue = Extracted_Value - Extracting_Costs - Bots_Costs
, where
Fraudster - a malicious user that tries to extract value from our platform,
Extracted_Value - the value that the fraudster can extract (e.g. Gitcoin matching funds),
Extracting_Costs - efforts needed to extract the value (Outwitting Fraud Detection team, risks on failing to do so, etc. For Gitcoin these are all the efforts needed to create an eligible grant, spread donations across multiple grants to mimic genuine behaviour, etc.),
Bots_Costs - efforts required to create a bot army.
If a malicious user now has an option to “sell” their bot army the revenue for doing so is:
Bot_Farmer_Revenue = Pool_Size - Bots_Costs
, where
Bot_Farmer - a malicious user that breaks HV methods just to liquidate their bot army for the reward from Upala pool,
Pool_Size - same as above, pool size at risk (or if looking from bot farmer side the amount of funds that can be drained from the pool simultaneously),
Bots_Costs - efforts to create bot army (both Fixed and Cost of Forgery per bot).
Let’s imagine a situation when it makes more sense (or equally reasonable) to sell bot army then to hunt for the platform value. It will happen when the revenues are equal.
Fraudster_Revenue = Bot_Farmer_Revenue
Let’s be humble and assume that Extracting_Costs for our platform are 0 (it is also safer to assume so). And if we now substitute revenues from both sides with their values from the formulas above we got:
Pool_Size = Extracted_Value
So if we offer a pool size that would be comparable or greater than maximum possible extracted value, there would be no sense for the malicious actor to hunt for that value anymore and turn to bot liquidations instead.
This is useful. We can use this Pool_Size value as our Upper Pool Size Limit right away. If we were right with the Extractable value our existing malicious actors should liquidate their bots. But as we are in the open market there are other actors in the wild. Which gives us a chance to cut costs.
Fraudster and Bot farmer
Pool size limit that we derived from the extractable value should not be regarded as an inevitable expense. There’s a chance bots take it all. However it is likely that we will see enough of liquidations far below the upper limit. And there are reasons for that.
Notice that when we discussed malicious user revenue we introduced two different entities: the Bot farmer and the Fraudster. These are opposing forces.
Consider the Fraudster. We neglected their efforts to extract value from our platform. But even if there are little efforts required, the risk of being detected is high, it increases over time and - which is more important - the risk is unknown.
Earning on bots liquidations on the contrary is transparent and straightforward. Which gives a considerable advantage to the Bot framers. It is no coincidence that one of the main Upala’s principles is protecting bots rights.
Bot farmers are narrowly specialised on bots (or better say on specific HV methods). They don’t care for the extractable value. With their presumably better forgery methods they can outperform the Fraudster. Meaning that they can win the auction. They gonna “sell” their bots faster, taking lower bid and/or lower pool size. Which cuts our costs.
Promotion
In order for the campaign to be accurate and economic we need to signal both existing bots and future bots about Upala pool. Current malicious actors must be informed that there’s now an option for them not to attack the platform itself(hunting for the Extractable value). Wider bots community must be informed that they now can generate some bots “for sale”. Thus our PoF discovery campaign budget should include promotion:
- Show the campaign in the your interface and docs
- Promote it with existing tools (e-mail subscriptions, social media, etc)
- Advertise the campaign with paid ads
Have a look at the Signalling bots campaign for Gitcoin for inspiration.
It also makes sense to try talking with bot farmers directly. If they tell you the army size and bid that they would definitely bite, let them prove that!
Pink furry robots in front of a TV in a victorian style room, hd, highly detailed, fairytale
Time vs Money
Another thing that can make your PoF discovery campaign more accurate and economic is time. Same as with Score auction you can increase bid duration and number of pool size steps. On networks with low transaction fees you can get a very smooth increase of pool size. Thus saving on the otherwise abrupt increases of pool size (or from bot farmer perspective bot army size).
Time is also important because bots need time to build tools. It makes sense to start advertising before the campaign begins. So that all players are in the game when the game begins.
If you are not currently under attack or you can afford being attacked (if learning PoF is not urgent), you can probably cut some costs by increasing the total campaign time.
Other things to consider
It’s ok to loose the whole pool
Loosing the pool is the point when you get information about bots. Regard pool funds as an investment or a as a service fee. Think of how much you can save after you have the correct assessment of the HV methods that you use.
Respect bot farmers and play fair
Remember there are ordinary and honest people behind those bots. They are just exploiting vulnerabilities. They chose this job because they like it. They do work for the benefit of humankind. They teach us how to deal with machines. So when time comes, we would be prepared.
Use dedicated pool for every HV method
You should be able to distinguish different bot armies and different players. If you want to measure PoF of a combination of methods, create a dedicated pool for that as well.
Split extractable value between multiple pools
If you have multiple HV methods and all of them need to be passed to extract value from your platform, then split the amount that you got in Revenue and Extractable value section between multiple pools.
Links
This document on Upala Notion (got table of contents)
Gentle PoF discovery methodology
Signalling bots
Play around with parameters - Price of Forgery campaign estimation doc
A follow up for the Green pilled episode with Upala - benefits of using PoF for Gitcoin
Next up… a budget proposal. Keep in touch, humans!