As Prime Rating is working towards being as updated in its rating scores as possible we need a process to facilitate „real-time“ updates. This will give raters the possibility to update their own reports and motivate them to stay on top of what is going on in the rated protocols.
Summary
To improve Prime Rating’s efficiency I would like to propose the following Report Update Process (RUP). This idea was discussed already by several users in the forum here and internally brought up by @Lavi.
This process proposes a solution that addresses the need to frequently update already-rated projects.
It is designed in a way that incentivizes our raters to keep involved and up to date on what is going on in the projects they have already rated. It also gives raters the motivation to prioritize these projects over new ones during rating season or rate-athons, given the lower efforts of updating a report over resubmitting a new report. Raters will become specialists about certain protocols which might lead to a new business strain for Prime Rating and its community in the near future.
Proposed approach
In order to ensure high-quality rating reports and fair scores, each submitted RUP report is reviewed and has to be accepted by the Prime Rating Governors. The RUP process will be a process that is incorporated into the RAP process as it will follow the same lifecycle of a RAP.
The RAP/RUP Lifecycle
The following steps are required to initiate and effectively queue a vote:
Submission of a report via Typeform and notification of governor via the│submit-report channel in Discord
A dedicated governor will check the report for formatting and if correctly formatted will upload it to IPFS, and for voting onto Snapshot using the RAP Template. This typically takes place within two days of submission
After adding it to Snapshot, the governor will inform the other governors to vote
RAP/RUP is queued on Snapshot and runs for 72 hours
Once the vote has passed, the implementation is carried out by the Prime governors or stewards, uploading the report to the rating webpage and amend the score of the protocol
RUP request will be highlighted in our rating list. If for any reason a rater feels that a RUP has to be done before it is listed, he/she is free to propose to do a RUP in case of a significant event that occured for a given protocol/project. In order to facilitate the review, raters should highlight the made changes in the RUP reports.
Proposed RUP - Qualification
Propossed qualifications are the following:
RUP allows raters to update their own reports
The report can be updated once per quarter or after an event of high significance directly impacting the protocol (e.g. an exploit, new release, other significant updates)
Raters are required to update every section in the report or mention that there have been no updates in the given section(s).
Rewards are 50% of the usual RAP meaning (75 USDC, 100 D2D, 5 RXP)
Raters who submit a RUP have to ensure that they submit the lastest RAP template version.
Raters need a minimum of 100 RXPs (WDYT?) in order to submitt an RUP. This rule is in place to support our committed raters in the community and keep a high quality standard.
Next steps
This is the initial draft of a possible Report Update Process (RUP) and aims to collect feedback from the Prime Rating community. I am looking forward to your feedback, possible improvements or concerns. After the community is aligned on all the details we can start implementing this process for the coming season. The next steps for the implementation will be to update our Gitbook and adjust the Typeform where we currently submit the RAP reports by adding the option of submitting a RUP.
I think this will be a great way to keep the rating community motivated with up to date info on the projects they rate. It will also help limit or minimize any potential errors or misinformation on the protocols etc. I love the approach! Would suggest at least 50 RXP to be the cap but I totally understand
Couple of questions:
a. Are reports submitted via RUP eligible for Season and Rate-athon rewards?
b. Would the updated sections in the report also show what was there before? E.g. The team has changed. Would I also show what the previous team was?
Good questions @mm3729 ! Happy to share my personal opinion.
a) I think RUPs can be made eligible. However, if there are not a lot of updates or excellent new inputs/analyses in the submitted RUP the likelihood of it winning a reward is small. Nevertheless, I think RUPs should be eligible as we want to motivate and incentivise outstanding research.
b) I think displaying the changes exclusively is not needed (only for the review). Right now we also don’t highlight changes that are made in new submitted RAPs to a specific protocol.
According to my understanding soon, we will be able to display all submissions from a particular protocol so that the history of submissions can be tracked for a protocol. Meaning that the initial RAP can still be accessed from our UI.
a. Could be tricky to do this given that the work needed to update is a couple of hours vs. a handful of hours to write an entirely new report. Curious to hear from other raters on this and what is the sentiment.
b. So, the new report after RUP would only show the updated content? I did not fully understand your comment here so wanted to clarify.
I was also thinking whether we should stop rewards for most submissions and instead focus more on quality. Most submissions was a good incentive to boostrap a broad coverage of protocols and think we continue using it when we go into new categories (Metaverse, ReFi, stables, etc.). But as PrimeRating matures, the focus should be more on high quality reports, especially since they also serve as examples for potential RoD clients.
In this scenario, RUPs could only be eligible for best quality price and I don’t see an issue with that. And as Salomé mentioned, the chances of a RUP winning are probably much smaller compared to a new report, because there is less new content that can be judged.
I think a RUP should highlight the new content when submitting (e.g. in a different background color), in the final report that is visible on the website this will be removed for better readability.
that creates incentives for the “first input” content(report) producers and if reviewers and governors don’t have the same level of incentive we block the process in the report feedback channel
maybe even worst than the bottleneck in the process is decreased quality of the final product. First version of report is important and in 2,3 or 4 days isn’t possible to find, read, filter info from important resources, and perform all processes that follow - competitive analysis, target market analysis, market fit analysis based on meaningful metrics - in short minimum 10 analysis through the report
even if raters produce more reports on rate-athon (because of higher incentives) that “boosted productivity” loses value over time, especially between rate-athons
I also think that we need provide more quality than quantity, and some insights because valuable informations are rare in this space.
I think raters can be motivated to submit a high-quality RUP in order to win a “best quality” price. It’s unlikely that with little changes this rater will win, but by putting some extra hours and improving certain sections or adding graphs/analysis/evaluating additional data and so on, RUPs can have the chance to win prizes. It’s up to the submitter to either only include the latest updates or also add “innovative” additional research on specific sections that bring value to the report.
This thread has been inactive for a while and no new concerns from the community have been raised. I, therefore, suggest we move on to a Snapvote to decide if we find a consensus to implement the RUP.
If there are no other concerns raised in the next 2 days, the text below can be used to set up the Snapshot.
[RUP-00] Implementation of a Report Update Process
Author: Salomé
Summary
To improve Prime Rating’s efficiency, I propose the following Report Update Process (RUP). This idea was discussed already by several users in the forum here, internally brought up by @Lavi and officially addressed here
Problem Specification
This process proposes a solution that addresses the need to frequently update already-rated projects.
Proposed Solution
The detailed outlined proposed solution of the process guidelines can be found here .
RUPs are also eligible for Season rewards.
As discussed in the forum, it’s just less likely that there is enough good-quality content in an updated report. It’s up to the raters to include additional content, analysis, and graphics to make it a high-quality submission (just new content is evaluated).
For quantity rewards, RUPs count 50% (Two RUP submissions count the same as 1 RAP in terms of quantity).
│submit-report channel in Discord