Gentle PoF discovery methodology
Duplicating from Upala docs.
Please support our Price of forgery measurement campaign for Gitcoin on Gitcoin grants
A reminder on price of forgery
Price of forgery (PoF) may be viewed as a way to quality control human verification (HV) methods. Upala protocol allows bots to sell (”liquidate” or “explode” in Upala terms) their IDs for a specific amount money. Depending on the amount bot farmers are either incentivized to forge a HV method or not. The amount which is just below their incentive is considered the Price of Forgery for the particular human verification method. See this video to learn how price of forgery works. And check out this article - Measuring human uniqueness score in dollars on Ethresearch.
The gentle methodology​
PoF could be measured in multiple ways. Below is the methodology that is optimised for cost-efficiency (there’s cost vs time polarity).
Score auction
Score auction overview
Score - the amount of money that we offer to a user that passed a HV method.
PoF - the score value which we consider the current Price of Forgery for the HV method. The thing which we are measuring. Unknown at the beginning. And should be remeasured periodically.
Bid - the score value being tested
The measurement goes in auction steps. The bid is set to some low value and is raised slowly over time. As soon as the number of bot liquidations exceeds a predefined threshold, the corresponding bid is considered the PoF. The bid is then reset to a much lower value and the gradual increase is started again. If threshold is hit again that bid defines new (lower) PoF. The bids are always below PoF.
Score auction parameters
Bid duration. A period of time during which a bid is being tested (let’s use one day as an example).
Starting bid. The score we start with. It makes sense to keep it low.
Bid Step. A number of percents that the bid goes up if the previous bid didn’t attract enough bots. The lower this value is the more accurate PoF we can get, and the more time it will take.
Liquidations threshold (or just threshold). Every period will have some liquidations going on (people got hacked, liquidating out of curiosity or by mistake - not real bots). We define the threshold of such liquidations as a number of bots which is “enough” to be considered a bot army.
Step down. A number of percents that the bid goes down after the threshold was hit.
Pool size. The amount of money that is exposed to bots. Defines how many bots can be liquidated simultaneously (n = Pool_size / bid ).
You can experiment with different parameters in this Price of Forgery campaign estimation doc.
PoF accuracy and up-to-dateness
When measuring PoF you can stop as soon as the number of liquidations hits threshold for the very first time. If the parameters were set right you may get an accurate enough PoF for your use case. At least you should be able to see the difference between HV methods being tested.
Have a rest, look at “Fluffy pink clouds and robots, HD, highly detailed, fantasy style, fairytale”
But you’ve got to remember that there are always more skilful bot farmers around the corner. Some of them may have missed your PoF campaign due to insufficient publicity. Some of them may have developed more effective exploits for HV methods that you use. Both of those means that PoF will go down over time. You can keep your PoF up-to-date either by repeating PoF discovery periodically or by keeping your pools always filled.
PoF and Pool size relation
We call PoF PoF because PoF shows cost of forgery PLUS revenue of the bot owner (
-PoF-PoF-PoF). More specifically it can be broken down as follows:
PoF = CoF + (Fixed+Revenue)/n
, where:
PoF - price of forgery (or bid in the current period - see auction parameters),
CoF - cost of forgery per bot (manual labour, Upala fee, Ethereum fees, etc.),
Fixed - fixed costs for a bot army, risks, efforts of getting butt off of the couch, etc.,
Revenue - total revenue of the bot farmer,
n - bot army size that CAN be liquidated.
If a bot owner liquidates their army, their revenue looks like this.
Revenue = n Ă— (PoF - CoF) - Fixed
So bot owner revenue depends on army size they can liquidate. And it is group manager who controls the army size through the number of possible liquidations. Which is:
n = Pool / PoF
, where
Pool - pool size. The amount of money allocated in pool. From bot farmer perspective it is the amount of funds that can be drained simultaneously. Here and further threshold is neglected ( Actual_pool_size = bot_army * PoF + threshold )
Army size is an important parameter to measure PoF correctly. Think of the difference between pool sizes of $10 and $10,000 when PoF is set to $1. No one’s gonna bother creating 10 bots for a 10 buck reward. Bot owners should have an incentive to create armies. At the same time setting pool size to $1,000,000 does not make much sense either.
Pool size auction
PoF and Pool size always go in pair. To find the pair we will now improve the methodology described above by introducing variable pool size.
Pool size auction overview
Just like with the bids above, we gonna gradually increase pool size within single bid duration.
For example we can increase pool size from $1,000 to $5,000 during the day (if our bid duration is one day). Then for a specific bid we gonna have a chart like this.
Now the liquidation threshold may or may not be hit at a specific pool size for a given bid. And if it is hit we’ll have both PoF and its corresponding Pool size. And, accordingly, if liquidation threshold is not met (if there’s not enough bots for us to be considered an army) we proceed to the next bid and start with the lower pool size limit again.
And same as with bid step above, we can reduce pool size step to increase accuracy. Greater number of steps would mean a more precise PoF and Pool size pair.
Pool size auction parameters
Lower pool size limit. The pool size that we are starting with for every new bid.
Upper pool size limit. The maximum pool size that we define for this particular HV method.
Number of steps. The number of increases of pool size within single bid duration.
Pool size step. The increase in pool size. Can be derived from bid duration and number of steps.
Pool_size_step = (Upper_pool_size_limit - Lower_pool_size_limit) / Number_of_steps
These parameters should be dealt with care. Accuracy and PoF measurement costs depend significantly on them.
Both upper and lower limits should be set according to Gitcoin needs. Recommendation will soon be described bellow. Please follow the topic if you are interested.
Please support our Price of forgery measurement campaign for Gitcoin on Gitcoin grants.