Re-vamping Incentivise Design around the permissionless Rating Process

I personally would have anticipated given the sentiment over the last couple of months that there would be some people interested in changing the situation and that this would act as incentive to join. But I am open to discuss other methods (i.e. paying people or handing out poaps etc.)

1 Like

Tokenengineering is more a methodology for me consisting out of a set of tools from traditional business and consulting. We can take other ways too - for me the goal of the series it is more to think in structured way through the incentive design form a systems perspective. I was hoping the structure of the session (which builds on tooling outside of “tokenengineering”) would illustrate it - but the methodology uses essentially quite conventional consulting tools especially in the system requirements phase (at least from my understanding thus far). Maybe some samples of what kind of templates that are used can be seen in my personal notes here and here.

1 Like

That was very detailed @xm3van and I think you bring real problems that need to be solved.

I will add one more problem which I think is the trigger and affect almost all problems that you mentioned. It all starts with a submission report on feedback, and I will point out that I personally made those mistakes in ~3 months rating period, which I see are becoming more common:

  1. It’s impossible to perform fundamental analysis and write a decent report in less than a week. If somebody writes a report in 2 days (my last report on metaverse rate-athon) that brings more work to the reviewer.
  2. The reviewer needs to spend more time reviewing that report (in 1 day ) and after the rater implements suggestions - what is a product ready for governance voting? The report that is made up of superficial information that can be found by literally anyone on Google in 1 hour, and if we talk about a “product” or “service” that someone needs to use we have a problem (even worst - paying)
  3. That problem comes to governors who have to reduce MVP criteria to a very low level, which bring them into a trade-off situation and spending more time - which they should not.
  4. The fundamental template for Defi needs a few corrections, but the Metaverse template needs a complete redesign. Raters need meaningful questions in order to give meaningful answers.
    If the rater needs to score most or all questions in a section with 0, maybe that protocol shouldn’t be on the rating list. Last time a lot of raters support the “0 option” but from a logical standpoint, 0 isn’t a score. Protocols without a governance system deserve a minimal score - 1. If we are DAO members that should be decided by group decision, not by individual.
    I just point out this problem from rater perspective.

agree with work plan and incentive mechanism redesign is a great idea

4 Likes

Great point! Thanks for outlining it so detailed!

1 Like

Agree with you @RatingPepe. I don’t have a solution in mind, my point was that we need to include this governance participation problem into @xm3van’s workshop scope. I’ve learned from experience that incentives & system design wins out over even the best intentions. So, want to encourage us to think in that direction for Prime Rating governance too.

3 Likes

Fair point! I am wondering whether this is however work for the general Governance of Prime Rating beyond the permissionless rating product.

For me it feel part of the goverance of changing processes (i.e. for me Organisational Governance/ DAO Goverance), whereas I see the scope of this workshop as governance of prime rating product (i.e. Product Governance).

1 Like

And I think that needs open discord channel, where Governors after Snapshot voting explain their decisions and point out where the rater made mistake and where reviewer provided a false review, because without that practice all participants in the rating process will continue to make the same mistake - raters, reviewers, and governors. With better communication and coordination this process can be significantly improved and can save our time, at least.

Agree with you, that is problem of all communities, but again I think governors also need to be incentivized. Although they spend the least time in the process (maybe), they have more responsibility than raters and reviewers. COW protocol has an interesting solution and high voting turnover for now - " There are 10,614 total vCOW holders on both chains (Ethereum - 752, Gnosis - 9,862) which means that, on average, ~ 59% of governors are active in governance voting."

1 Like

I received feedback in my DMs on the proposal which I perceived as valuable and thus wanted to share it here too:

1 Like

In order to advance this proposal and based on the feedback received I will make concrete suggestions to each of the questions put forwards:

Suggestion: Based on the feedback received - I’d opt for 10 RXP this means that a given rater has completed at least 1 successful submissions. I think this encourages most voices to be heard while avoiding uninformed opinions who have not a complete understanding of the whole process. Exception can be made please reach out to me (Suggesting to leave this to me or Governor for efficiency sake - happy to outsource or backdown form this if community comments in this thread)!

Suggestion: I suggest starting after the completion of the season this means 27.06.2022. Starting with bi-weekly appointments every Tuesday and Thursday from 17.00-18.00 BST going forward from that date (i.e. first appointment 28.06.2022).

In terms of duration I think it depends what we are aiming for and how we want to iterate on version one. Maybe something like this should be periodically repeated in quarter-year cycles. If this is a one off I think a longer period is required, if this should become an organisational practice we can move a bit faster knowing it is not final.

I will amend the the post with any more issues that come in. Pls let the feedback come in.

2 Likes

Wholly in favor of a workshop like this. Thanks for pushing this forward @xm3van! I would like to participate as well.

On the subject of RXP limits for participation, I think the threshold should be higher than 10RXP honestly. Acknowledging that governance apathy is real, this would help make sure participants in the workshop have had the interest and commitment to complete more than just 1 report. I would advocate for a limit that also emphasizes quality if possible. A 30RXP limit could be a good middle ground (3 reports or a prize winner). Definitely open to a lower threshold though if we want to have more of the community involved.

3 Likes

Thank you very much for getting this started @xm3van ! It’s a great idea to involve all stakeholders in the workshop as especially the raters are impacted by these important decisions on how we structure Prime Ratings processes. Would love to be part of it.

1 Like

Updated the initial post!

2 Likes

Summary Session 1a

Attendee’s: OriginalSK, Dabar90, Salome, Xindaneran, xm3van

Planned Agenda:

  • Problem-definition sheet to align on project (Living documents)
  • Identify Problems
  • Categorise Problems

Summary of session:

  • We aligned and talked about PDS.
  • We identified some problems within the incentive design and beyond
  • We went over the current process of permissionless rating
  • We agreed on the next steps for the coming session

Recording:

Task for next Session 1b):

  • xm3van: Provide number for active raters, reviewers and governors
  • xm3van: Refined copy of BPMN model copied as image into Miro board so that we can lay value flows over it
  • all: Identify more problems async
  • all: Categorise Problems into freely and individually invented themes
1 Like

Summary Session 1b

Attendee’s: Xindaneran, Dabar90, Lavi, xm3van

Planned Agenda:

    1. We will take a look at the classification of issues and any additional issues added between Tuesday and today
    • Note: I tried to break the issues down into an issue tree - from this we can decided if we want to further validate whether the issues are true and move after this to a hypothesis tree to validate potential solution. It definitely can do with refinement (ME CE).

  • 2/3) Workshop participants check how to proceed:

    • Option 1: Plan to validate identified issues
    • Option 2: Accept issues as true and proceed to the hypothesis tree
    1. Move on to Stakeholder mapping & Token utility canvas

Summary of the session:

  • Tough session yet productive
  • Clarification of Scope of the Workshop

Recording of the session:

The task for next session 2a) :

  • Map issues to permissionless rating workflow
    • Note: Ask 6x why in order to identify the root cause of the issues.

  • Feel free to drop you design idea’s

**Summary Session 2a) :

Attendee’s:

Planned Agenda:

  1. Shared Review Miro board - finalise Problem Identification [Partially Achieved]

  2. Decide the following for “Root Cause” : [Not Achieved]

  • if we want empirical validation
    • if yes, a concrete action plan (when, where, how, who, why) to provide empirical validation
  • which should be addressed with the re-design of the incentive structure
  1. Optional: Breakdown of potential solution into a Hypothesis tree [Not Achieved]

3/4) Stakeholder mapping and their incentives in relation to selected root causes. Tools that may prove useful: [Not Achieved]

  • Entity Relationship diagram
  • Causal loop
  • Empathy map
  • an adaption of token utility canvas

Summary of the session:

  • Review of miro board
  • tough session - further ideation on solution for possible solution to identified problems

Recording of the session:

Task for session 2b):

  • Everyone: pls add additional possible solutions to the miro board until Wednesday 06/07/2022 - noon BST
  • @xm3van will add create a spreadsheet listing all possible solutions and add evaluation criteria with an initial judgement in order to provide pirotisiation. xm3van will share this in advance in discord chat.
    • Ease of implementation
    • Relevance for incentive restructuring
    • Resources cost
1 Like

**Summary Session 2b) :

Attendee’s:

Planned Agenda:

  1. Shared Review Miro board - finalise Problem Identification [Partially Achieved]

  2. Decide the following for “Root Cause” : [Not Achieved]

  • if we want empirical validation
    • if yes, a concrete action plan (when, where, how, who, why) to provide empirical validation
  • which should be addressed with the re-design of the incentive structure
  1. Optional: Breakdown of potential solution into a Hypothesis tree [Not Achieved]

  2. Stakeholder mapping and their incentives in relation to selected root causes. Tools that may prove useful: [Not Achieved]

  • Entity Relationship diagram
  • Causal loop
  • Empathy map
  • an adaption of token utility canvas

Summary of the session:

  • Brief Review of the issues in PRR
  • Initial stakeholder mapping

Recording of the session:

Task for session 3c):

  • Everyone: Ensure Stakeholder maps are complete
  • Everyone: Add Stakeholder
  • Everyone: (optional) Map systems
  • Everyone: (optional) Map mechanism

Feel free to make a workspace fro yourself in the miroboard!

Resources: Problem PRR - Google Sheets

1 Like

Summary Session 3a)

Attendee’s: Xindaneran, Dabar90, Salome, OriginalSK Lavi, xm3van

Planned Agenda:

  1. Session Goal (5 min)
  2. Review of Token utility Canvas for 3 Main Stakeholder Group (15min)
  3. Mechanism and System design (40 min) [Not achieved]

Summary of the session:

  • Review of Goal for the session / stage
  • Stakeholder map and review of token utility canvas

Recording of the session:
https://drive.google.com/file/d/1hnxidXqT-NM-lO84FbthM4duXOZ_xfi3/view?usp=sharing

Task for session 3b):

  • Everyone: Add to missing information in a different colour from the token utility canvas
  • Everyone: Map mechanism & Map systems using entity relationship diagrams
1 Like

Summary Session 3b)

Attendee’s: Dabar90, xm3van

Planned Agenda:
1, Review of Token utility Canvas for 3 Main Stakeholder Group
2. Mechanism and System design

Summary of the session:

  • Very free discussion
  • Review of Stakeholder map
  • Entity-relationship diagram
  • Comment: Might require re-scheduling!

Recording of the session:
https://drive.google.com/file/d/1O_8yWxXFymyGQ58Rp0wTpoYBzRvBHwrI/view?usp=sharing

Task for session 4a):

  • Everyone: Map mechanism & Map systems using entity relationship diagrams
1 Like

An explanation for the absence of summaries - given the current circumstances the team needed to re-focus priorities and holiday season of community and contributors led to less attendance. Hence we shifted to a more show-and-share approach where the system is designed and shared with the community.

2 Likes

Summary Session

Attendee’s: Dabar90, Lavi, Luuk, Salome, Jan-Lucas, xm3van ( + 2 more)

Planned Agenda:

  1. Presentation of the initial system

Summary of the session:

  • Covered 1,2, 4
  • some discussion especially around reviewer

Recording of the session:
https://drive.google.com/file/d/1bYd5Hd48MwqaxGxxh_5nVQd2jD-dIMTX/view?usp=sharing

Task for finalisation:

  • Make light paper circulation ready to share with forum and network
    → Implement a tl;dr
    → Emphasise tokenholder rights
    → Fix Reviewer role

  • Address token utility for each Token

  • Prepare forum post

  • Finalise missing point planned for session 5 in the initial work plan

2 Likes