What is your opinion on a 0-10 rating system in the 3D Warehouse? I was thinking the likes could be changed into favorites, so everyone still has their favorites lists. Anyone want to share your thoughts on the topic?
There used to be a rating system but it caused a lot of problems as there were fights among users and people answered bad rating with even worse ratings. I think the current like system is good because it leaves no room for this kind of abuse.
Was the old rating system anonymous?
Nope. People could be identified and retaliated against. Or in a way it was, because different users could have the same nickname. I can remember at least one user who had problems with a spam account pretending to be him.
I think an anonymous rating system could prevent people from doing revenge ratings. Although, receiving a poor rating, even if it’s anonymous, would be a negative experience for many people. The good news is that a rating system could allow people to sort by ratings, rather than most liked. In my opinion, likes and ratings are two different things, so a rating system could increase quality control in the 3D Warehouse. Often times, I think a model is really good, but I don’t press the like button because I use my likes as a collection of components I might use. If there was a rating system, I’d probably rate a lot more models than I currently add to my likes list. I guess I could just make a collection of useful models, then hyper-inflate my likes list, but I think I might prefer a rating system. I’ve noticed a lot of people refer to likes as ratings (For example, “Thank you for the ratings”), so maybe the change would be easy and intuitive.
Edit: What if there was a way to catch people ruining ratings? For example, if someone gives a model a 1 that many people gave a 9, then that 1 would be an invalid rating. Right? Clearly, it’s not a 1 if many people gave it a 9. Also, if they continue to vote 1’s on every model, then that’s a sign they could be abusing the system. I think they should get flagged if they do that, unless they’re giving models 1’s that have a low rating. Some people upload scribbles and stuff, so 1’s definitely have their place. Just saying.
I’d rather have bokkmarks/saved introduced as a separate feature in that case. Likes are commonly used as unary ratings.
On the second point I think using displaying a median value could be favorable over an average, which is how it used to be.
But even then rating is hard as it depends on how you want to use the model. I value super simple components that can be used en masse without slowing down performance. To me a 40 KB model could be a 5 star model, but to someone else it could be an obvious 1 start model.
I agree the current system is good. Currently, people can save models by making a favorites collection, a bookmarks collection, or any kind of collection they want. Although, I think a 1-10 or 1-5 rating system would help. In terms of graphic design, I think the 5 star rating system could look better. I was thinking it could just be a number out of ten because it’s easy to understand and isn’t as distracting as the stars. The focal point should be the model, but the yellow stars are distracting.
A median rating system could eliminate unusually low votes, but some people who voted unusually low or high might be serious. If they’re serious, then I think their votes are valid. In certain ways median is fairer, but average is more mathematically accurate.
I’d hate to be searching top rated models only to find adolescents voting poor models into the stratosphere, I’ve seen soooo many “tuned” vehicles (that clearly wasn’t made in SketchUp) with scantily clad cutout women posing closeby. Or terribly modeled components (just talking model structure here). Or similarly terribly designed whatevers.
If there’s a voting system (apart from the current like), the voters need to earn the right to vote first.
2 Likes
I have the same concern about the accuracy of the ratings. Can anybody think of a way to deliver an accurate rating to people’s models? Perhaps a scoring system or something?
How do you even define what rating is accurate? Is it the usability as component or how interesting it is as a project of its own that matters? Even though I myself hate the super high detailed cars that can never be used in a project and don’t want them to fill the search results, there is obviously a community of people who appreciate those models. My favorite model in the whole 3D Warehouse on the other hand is probably this bike. It is just perfect for conceptual modeling of large projects (except for the faces maybe). To others it may not look like much at all. It may even be the perfect 1 star model to some.
I think what is really needed is better filtering. You can’t even filter by file sizes under 1 MB and I try to only use components below 1 MB. Even setting the slider to 1 MB is tedious because of the fine movements it requires.
1 Like
Rating is very subjective, one mans treasure is another mans trash
1 Like
A rating system is hard. It’s not hard to implement but it is rather subjective in terms of what is defined as a 10/10 model. Some modeler disagree on what is considered a quality model since that can also be based on needs. Our approach will be to find common things in the model that we can define as a base for a quality model and leave some room for interpretation. Ideally this should improve our search algorithms and allow our users to filter the rest of the criteria based on their needs.
2 Likes
What about a grading system for models? For example, people could be graded on certain aspects of their models (similar to the checklist below). Models could also be graded on artistic skill. People could be required to write a review in order to rate a model, then people could flag abusive reviews. There could be rules about review etiquette, such as no revenge ratings, keep it factual, no gibberish or spam, a minimum word count, etc. Those are just a few ideas for rules. If modelers put time and effort into a model, then I think we should have to put some thought into a review to rate their model. Also, there could be a constructive criticism section in the review form. Constructive criticism and quality reviews are valuable because they can help modelers improve their work. The problem is that people already have comments, so implementing a review system might create too much visual clutter and confusion. I think changing the old comments into reviews would be a bad idea because they’d be missing ratings. I want to keep the rating system minimalist, but I also want it to be accurate. There is a notable margin of error by voters, especially when you think about how most class averages are below 100%. I think people usually do a good job of rating things, considering the average high school graduate’s GPA is around 3.0. People usually rate fairly and relatively accurately. I noticed many websites use five stars as a rating system, so maybe that’s a more familiar method than using a 1-10 rating system. The stars could be red to match the 3D Warehouse’s color scheme, but I’m no graphic designer. I just want it to be visually appealing, understandable, and an accurate representation of people’s opinions.
That’s good feedback @Forestr and we will add that into our discussion. It’s still a long ways out and I believe we use to have some sort of rating system but that was abused. So if that’s the direction we have to go then I want to make sure the experience adds value, is intentional and has an impact for the community. We are aware of this problem so it will be improved in the future.
1 Like
And since “like” is really a bookmark, could there be some way of removing them from your personal page? Right now it means “I may want to come back to this but I’m not sure enough about it to out and out download it.” When I finally make up my mind, I’d like to declutter.
Downloads say “I want that.” Those are the real “likes.”
Just as rating is subjective, so are the likes and downloads.
Some believe downloading another’s model is the “real like.” I think using someone’s model in one of my own is more of a “like” than just downloading or clicking the “like” button.
I have a model with nearly 30 likes, but only half the downloads of another model (over 4K downloads) that has only six likes. So how do you interpret what people really think? How would these two models be “ranked?” One would think the model with 30 “likes” was a really good model. Someone else may feel that the model with over 4K downloads was a better model.
The system is not great. But until humans become rational, logical, appreciative beings, I sorta think any rating system will have plenty of room for abuse and misuse.
2 Likes
I don’t think that is how the system is intended o be used. You can have the model open as a separate tab or use your browser’s actual bookmark function if you want to revisit it later.
I’d rather have a usefulness rating:
Was this model useful to you? Yes/No
How was it useful? (Checklist)
Realistic;
Simple;
Accurate dimensions;
Technical;
Modelling errors free;
…
This checklist would be based on typical user needs needs which I tried above to point but wouldn’t know which would be better.
My initial reaction to @jql’s usefulness rating is: Hmmm, I could see this being quite useful. But searching for the “optimum” model for your particular needs would be a time suck if all you can do is look at the ratings for one model at a time.
Perhaps enhance it as follows:
- Instead of a checklist for each of the usefulness attributes, make each a 0-10 rating. Then
- Also add an"Advanced Usefulness Search" where you can customize the weight you want each individual rating to contribute to an overall rating that best reflects what you care about.
Thus if I’m interested in finding a highly realistic component that I want to use in a scene I’ll have rendered, then blown up to poster size, I’d weight “Realistic” at 5, perhaps “Accurate dimensions” at 4, “Simple” at 0, etc.
On the other hand, if I’m looking to populate a landscape with trees and benches that don’t need to be seen up close, Simple and Modeling errors free might get highest weighting, with accurate dimensions, technical, and modeling errors free having a low weight.
So the results would end up with the highest rated models for my purpose appearing at the top of the list.
Worth fleshing out in more detail?
1 Like
I’d look at it this way: if a user needs more than 3 clicks to tag usefulness or rate in whatever other way, the user won’t rate it.
In my case, for instance I rarely download a model, but when I do I download a bunch just to see the one which fits my purpose. I often forget which one I downloaded so how can I rate it after downloading?
Rating prior to downloading is not easy, even with the new full 3d orbiting.
Rating after downloading, with the model sitting in my sketchup model already would be interesting and that could be more effective.
Rating system as complex as the one you propose is not for all. However, there could be a credit system for downloads which would somehow push people into rating models or their downloads would be limited somehow. I wouldn’t mind and think that would be fair.
1 Like