This a very intetersting proposition. I think measuring impact and cost-effectiveness could be valuable but as many have noted here, the challenge is comparing one impact metric to the other. One size fits all might not work for all the pop-ups as each could have a different approach.
One potential solution:
- Establish a few impact metrics and reporting standards.
- Pop-ups by participating in this round agree to respond to n/k impact metrics following the reporting standards.
- compare only matching impact metrics from one pop-up to the other.
It is hard to come up with metrics that a group of people would agree as impactful.
For example, for person A, attending pop-up in a new country never visited before. This could be very impactful for person A.
In another pop-up person B goes to hack on some problems, this could be impactful for person B.
Another approach could be,
- Each pop-up designs a theory of change (TOC), publishes it with how they would evaluate the change.
- This would be considered as an entry point to the round.
- This gives freedom to the pop-up to design their impact.
- The community would vote on which TOC is more impactful and then funds are distributed based on that.
Having said the above and being the organizer of India’s pop-up ZuGrama, I would be interested in participating and experimenting on this.