Blog Entry

Ratings Changes Effective Tomorrow

Posted on: January 30, 2008 10:06 am
Edited on: January 31, 2008 5:33 pm
 
In this thread we announced some upcoming changes to the way posts are rated and how those ratings impact the reputation:

www.sportsline.com/mcc/messages/chr
ono/5876055/


Now that the changes have been made I want to describe them here as the thread above may not have gotten all the details exactly right.

Why are we making this change?
There are two main reasons for us doing this:
  1. Members were using the ratings as currency to either game the system (content free ten-fest threads) or to annoy or get back at other posters for disagreements. Neither of those behaviors were good for the community as a whole.
  2. Members are not too interested in giving low ratings, and they just want to use the ratings to highlight good content. Perhaps some of the apprehension towards giving lower than the highest rating is due to fear of backslash.
So what is the change?
The changes we are making, in a nutshell is to simply remove the negative impact that any rating can have on a user's reputation. So even if someone gives you 1 star, that's still better than not getting a rating. The more ratings you get for each post the better your reputation will be.

Now for the details...
A lot of factors go into converting the number of stars someone gives you into the actual number that affects your reputation:
  1. The reputation of the user that submitted the rating. The higher someone's reputation is the more weight their rating has
  2. The scale the reviewer uses to rate posts. This is the dreaded 5 stars to 2.5 star adjustment. Users that tend to give everyone 5 stars will have their reviews be worth 2.5 stars only. While those that spread them around, will keep most of their value. This is to encourage users to use the entire spectrum of rating values and create more distinction between good and great posts. Those that do not want to do this are not hurt by this, and getting 2.5 stars is still better than getting none, so you may continue with this behavior without any consequence.
  3. The type of content being rated. Each type of content we allow has an internal weight relative to the amount of time a user spends writing or reading that type of content. You are expected to put more time into a blog entry than into a reply in a message board, so it's logical that those have a higher weight.
The total ratings a user gets across all their content is then added up and divided by the number of posts made by that user. Posting more messages will not help this part of the reputation calculation. (It will however help the participation part).
An Example
Let's walk thru a simple example (some of the math has been simplified here for the sake of clarity):

Let's say member X posts a message and it gets rated by:
  • Member A, who gives him 5 stars and has a reputation of 97. Member A only gives out 5 stars, so his rating is adjusted to 2.5. Multiplying the reputation (as a percentage) we get 2.5*.97 = 2.43.
  • Member B, who gives him 4 stars and has a reputation of 50. Member B spreads out his ratings so this one stays at 4, and factoring in the reputation becomes 4*.5 = 2.0.
  • Member C, who gives him 5 stars and has a reputation of 80. Member C gives mostly 5 stars, but a few others so his is adjusted to 3.5. Factoring in the reputation, we get 3.5*.8 = 2.80.
Then all these numbers are added: 2.43+2.0+2.8 = 7.23.

Now let's see what happens when Member D who is a troll with a reputation of 15 and gives him 1 star (which is not adjusted any lower). That will then result in a total of 1*.15 = 0.15. So this value will be added to the prior total resulting in 7.38. It didn't go up by much, but 7.38 is still better than 7.23. So the troll that gives a low rating still helps your value go up.
So to take it further, lets say that member X has posted 5 messages with totals of 7.38, 10.34, 5.10, 20.34 and 31.93. The first four are messages and but the fifth one is a blog entry. So the first 4 are multiplied by 5 each and the fifth one is multiplied by 9. The relative weight of a message is 5 while a blog entry is a 9. That sum is then divided by the sum of all the weights (4*5+9). The result is a 17.35 total value.

Final Thoughts
We don't know exactly what will happen with this new system. I'm sure some will find a way to abuse it or at least attempt to find a way around it. We will be closely monitoring things to see how it works out and undoubtedly will keep tweaking at it. Once concern is that those that hang out in the less trafficked areas will receive less ratings and thus will end up with a lower reputation. That's possible, but perhaps that's not so bad. If there are more people interested in football than in tennis, it should follow that those that write about football have a higher reputation. I may seem unfair to the tennis fan. However, I don't think that it will be that significant.

In then end, it's probably best to not worry too much about the mechanics of this. To that end, we have removed the Adjusted Rating column from the review lists. It is just causing too much unnecessary anxiety. There are plenty of users with very high reputations that don't care about it, they just got there by submitting good content and not playing any games.
Category: General
Comments
jjsteeler1
Since: Jun 4, 2007
Posted on: April 24, 2008 4:05 pm
This comment has been removed.

Post Deleted by Administrator




Since: Oct 14, 2006
Posted on: April 16, 2008 9:37 pm
 

Ratings Changes Effective Tomorrow

Tony.  I am really chiming in here very late, but I do have some concerns with the adjusted ratings formula.  If I choose to only rate posts that I really do believe deserve 5 stars and choose not to rate the others in any way, so as to not appear to be slighting a particular poster, why should the ratings that I give them be reduced?  Honestly, it's as if you are telling us (YOU guys will not give 5 starts to everybody) which really does take away our right to freely rate based on what our opinions are and is also telling us to (Give a few 2, 3, or 4 stars here and there) even when we may not think that ratings such as those are fair to that poster.

Just being honest and I hope I didn't come across sarcastic or disrspectful.

Thanks,

Sean.




Since: Dec 9, 2006
Posted on: February 23, 2008 2:09 pm
 

Ratings Changes Effective Tomorrow

I find the new system highly objectionable. I was a Top 1000 member with a 99 rep, and now, three weeks later, my rep is down to 96, all because my value rating (which I believe was too low to begin with) dropped from 78 to 40. My posts tend to be well-reasoned, thoughtful, respectful of others, and hopeful. Being a Yankees fan, subject to attacks simply for being one, requires this. I receive high ratings from long-standing posters whose opinions I respect, and give them the ratings they deserve for their posts, and the new system penalizes me for this. Now, instead of giving automatic 5 star ratings, I actually respond to a post, and the lack of ratings by both parties causes both posters reps to go down, even though the resulting give and take posts are allegedly what CBS is trying to encourage.

Posts that I find lacking in content or relevance, I ignore, rather than give low ratings, since that evaluation is what usually encouraged troll behavior in the old system. Since I don't choose to rate less valuable posts, I am penalized for this, even though I have responded to them with what I consider valuable criticism, for which I receive high scores.

And I echo the criticism of the system that questions how a poster who hasn't posted for six months has their value rating increase, when they haven't even been here to earn it, and others who post frequently and participate in the creation of the Community see their value ratings plummet.




Since: Aug 31, 2006
Posted on: February 15, 2008 6:55 pm
 

Ratings Changes Effective Tomorrow

I would just like to say that this new system is total crap, my value has dropped like a rock and the only reason I can figure is because I post things which get alot of views but do not get rated. Whatever the reason it is ignorant you have top 100 people who never post or when they do post cut and paste player names, meanwhile people who actually post things people whant to read have their value go down the drain? I dont know why I am bothering posting here its obvious the admin who started this blog desn't post back with any answers so its pretty much like talking to a wall.



Since: Dec 13, 2006
Posted on: February 13, 2008 8:35 am
 

Ratings Changes Effective Tomorrow

I applaud you all for striving to make cbs a better community than it already is. Here are more examples of what has happened since you changed the system. chisoxNDirish is now a top10 member and he hasn't even posted a new post since January 1st. McPhillyPhan hasn't even posted anything this year. His last post was on Dec 28th. If you are trying to get new users to good posters right away, the top members list isn't going to help them at all.



Since: Dec 13, 2006
Posted on: February 12, 2008 12:12 am
 

Ratings Changes Effective Tomorrow

Just as constructive feedback, look at those who are listed as top members now. SilverDragon has been a member for a while,(no offense SilverDragon) but he strictly copies and pastes his nfl football picks each week. He gets and inflated score of 108 because he names all of the teams in his extra long post. He doesn't have any activity in any other cbs community areas such as games or fantasy. He hasn't even brought any new members to cbs since early 2007. Several others listed as top10 members haven't even referred or invited any new members. All of these people seem to have watched their value increase while other members who spend more time and knowledge in all areas of the cbs community have watched their value and ratings drop. Is this the type of members cbs is looking for now?



Since: Aug 16, 2006
Posted on: February 6, 2008 7:52 pm
 

Ratings Changes Effective Tomorrow

Once concern is that those that hang out in the less trafficked areas will receive less ratings and thus will end up with a lower reputation. That's possible, but perhaps that's not so bad. If there are more people interested in football than in tennis, it should follow that those that write about football have a higher reputation. I may seem unfair to the tennis fan. However, I don't think that it will be that significant.

If you are someone who regularly posts on the NFL board, it might not seem that significant. I personally don't post their much because many of the problem children on these boards hang out there. So because I don't feel like dealing with nitwits, my reputation may go down so that their rep can rise.

Yea, that makes complete sense to me. If you check out my profile you will see that since you instituted the "new" system, my value rating has dropped. Not drastically, but it has dropped. I have over 1,000 reviews to my posts in February alone, most of them 5's. I don't hang out on a 10fest thread, but do hang out at a board that has a variety of different threads. Guess I'm not the kind of poster you want.




Since: Aug 15, 2006
Posted on: February 6, 2008 5:46 pm
 

Ratings Changes Effective Tomorrow

What about those of us who have developed a close knit community, like we have on the Racing Board.  Here we have a strong friendship amoung ourselfs and for all new comers because we explain to all new comer that we live by a creed that we agree to disagree.  There are heated discussions at time and at other times we have developed enough friendships through this board we can talk about anything.  If one needs a pray all they have to do is ask and we are their for each other.  So yes we all rate each other with 5 stars because we value each other opinions.  So why would you punish us over someone who will rate those who have their views with 5 stars and those they disagree with with 1 or 2 stars?  Isn't it better to have a group that in open minded enough with each other that we can value an opposite opinion, even though we disagree with that opinion, of our because we also realize that individual has a right to have thier own opinions too. 

If you don't think this agree to disagree works, check out the Thread on the Auto boards started by nynjjets. at this location.

If you want to know why he started this thread then you need to read this thread.

That should be enough to realize that its the value of the members that should outweigh the value of what is written.




Since: Jun 29, 2007
Posted on: February 6, 2008 5:18 pm
 

Ratings Changes Effective Tomorrow

Just out of curiosity....I'm wondering why the score on each post goes down the more people rate it? I started a thread and had an original score of 98 on the first post.  I saw that score go down a point or two everytime someone rated it, with 6 5's and 1 1 star the post now has a score of 90.  Not quite sure how that math works....



Since: Jul 27, 2007
Posted on: February 6, 2008 10:40 am
 

Ratings Changes Effective Tomorrow

Tony, all these change sound good, but why is everyone's value rating dropping? I've dropped 24 points so far and I'm sure it's not going to stop.



The views expressed in this blog are solely those of the author and do not reflect the views of CBS Sports or CBSSports.com