Forums > MobyGames > Acao reviews
Bozzly (1078) on 10/24/2023 1:49 AM · Permalink · Report
For the past few days I've been checking pages for reviews from Acao Games. I noticed some of the scores are being altered (some even have inaccuracies). The ones that had scales 1 to 4 or 1 to 5 have been replaced with a 1-100. Also, the triple digit scores some games had have been changed.
Rwolf (22867) on 10/24/2023 12:43 PM · Permalink · Report
Are you referring to external reviews changing?
https://en.wikipedia.org/wiki/A%C3%A7%C3%A3o_Games
(I don't see any reference to their reviews on Mobygames, but otherwise we do change reviews to a 0-100 scale for all sources.)
WONDERăȘăăł (16778) on 10/28/2023 1:30 AM · Permalink · Report
No, I've recently changed/corrected a substantial amount of review scores from Acao Games. See my below comments for more details. Also in Discord!
Cantillon (77993) on 10/24/2023 3:33 PM · Permalink · Report
WONDERăȘăăł changed this, maybe drop him a message.
Bozzly (1078) on 10/24/2023 10:04 PM · edited · Permalink · Report
Well we kinda briefly debate about it in some of the approval forms in some of my submissions. Anyway, it was sneaky how the user changed some of the scores without letting me know about it. I must also state that how the user thinks the scores from Acao should be presented in this site does not seem to be backed by a consensus. The user did not even bother to discuss the matter in these forums.
Kayburt (32359) on 10/27/2023 7:41 AM · Permalink · Report
I have approved a number of those critic scores in the past, but when WONDERăȘăăł questioned, them I began to ponder the same thing myself. Iâve become a little confused about the scores when I got a better look at them. I donât understand how the fun factor qualifies as an overall score. With so many scores being 4 out of 4 or 5 out of 5, this makes it seem as if the majority of games have a perfect score, while disregarding any flaws to the graphics, audio, presentation and gameplay. Is the fun factor really a reliable demographic? If you check out Sega retro at https://segaretro.org/Ação_Games/Ratings you will find that the games on Acao Games have an average score based on four factors. Even when the fun factor is top, some of the others are not. This seems like a more accurate way to sum up the reviews, but you let me know what you think.
Bozzly (1078) on 10/27/2023 3:18 PM · Permalink · Report
In some submissions which I pulled out, I commented "Categories like graphics, game play, and music each only focus on one particular aspect of the game. By contrast, fun is a complex category as it fundamentally refers to how much the reviewer likes the game. In most cases a reviewer likes the game because of the aforementioned aspects. With this, fun certainly represents all qualities of the game, and therefore makes a suitable overall score if there is no explicit one."
I must also say that using the fun score as a substitute overall score isnât something I came up myself. It is a trend Iâve seen in numerous games years before I started contributing to this site.
Kayburt (32359) on 10/27/2023 7:39 PM · Permalink · Report
I honestly don't know if this trend works or not. Take for example "Double Dragon III: The Sacred Stones" (https://archive.org/details/acao_games_50/acao_games_1_0/page/n39/mode/2up ) which has a reputation for being worse than the first two games. In their view, the graphics are okay and the sound is okay, but the fun gets full marks and therefore a perfect score. The review doesn't even mention the gameplay. If the fun represents all qualities, then how can you comfortably enjoy a game that has bad qualities? The game has bad hit detection, one life and no continues and enemies take a while to beat. Every one of those problems are completely ignored. It begs the question, was this game reviewed by Acao reviewers after actually playing it? I cannot say.
Now mind you this is just one review that gives high praise, even if it's not focused on the gameplay, so therefore it is not the totality of how the game is considered by players worldwide. That's about all I can say about this debate. I hope other users can shed some more light and come to a conclusion on what seems like the best approach.
WONDERăȘăăł (16778) on 10/28/2023 1:28 AM · Permalink · Report
That's just it though. It doesn't matter. Unless it says overall or total, or their legend clearly marks/notates a particular category as the "main" score, then you use an average of all categories. Sometimes, they will use a much larger or bold number (typically the last or first number in the series) to indicate it is the overall score. You can see this is actually what Acao Games switches to by the time Playstation and N64 come around.
For those early issues with the hand and color scoring system, they do not indicate a weight or an overall score at all - just a range from white to pink. As such, you can literally ignore all the words and just calculate the score from the symbols with each review.
If the scoring system is so absolutely esoteric and impossible to calculate a percentage value, then an unscored type may be used.
WONDERăȘăăł (16778) on 10/28/2023 12:40 AM · Permalink · Report
Fun should not be used as the "total/overall" score. Specifically for Acao Games. There are four colors they use to score, starting from the lowest score to the highest you have: white, blue, orange, and pink. Using "graphics", if it is scored white - that would be a 0% and a pink would be a 100%. This means, white is a value of 0. So, what you have as a scoring system is 0-3 as values.
White = 0, Blue = 1, Orange = 2, Pink = 3.
color value = CV
(CV + CV + CV + CV) / 4 (or sometimes 3, depending on how many categories they are judging with - i.e. graphics, gameplay, story, etc.)
Since the max value of any one of the scores can be a 3, you will then divide the resulting value of the last equation by 3 to get the average.
If a game is scored (graphics, gameplay, story, and fun) as blue, white, blue, orange, Using the above, that would be: 33%
(1 + 0 + 1 + 2) / 4 = 1
1 / 3 = 33
A game scored as orange, pink, orange, pink would be: 83%
(2 + 3 + 2 + 3) / 4 = 2.5
2.5 / 3 = 83
The game isn't scored with a 1-4 system as when you make white worth 1, the math will never allow for a game less than 25%.
Please let me know if my math is wrong. I never did well with algebra/calculus but I did pretty good in statistics. But that was over a decade ago now...so đ€·ââïž definitely hasn't improved since university lol.
I actually ran this by two different math/stat related subreddits and read up on scoring systems and how they are made/calculated. That said, some of these publications use some wacky scoring systems with a lot of distracting bits.
Oh right - and because no data is provided on it, we have to assume this is an UNWEIGHTED scoring system. If it were weighted, I would be inclined to agree that both fun and gameplay trump graphics and story - that said, we then start to get into subjectivity really quickly. Since the publication does not state any such slant or weight, then we cannot go assuming fun or any other component is more important or valuable than the others. We have to accept the scoring system for what it is.
Bozzly (1078) on 10/28/2023 2:01 AM · edited · Permalink · Report
In a 1 to 5 star system, 1 star is valued 20%, not 0%. This is based on dividing 100 by the number of total stars (in this case 5) used.
Using color icons shouldn't be different. If a category uses 4 colored indicators, then the lowest-valued color should be worth the quotient of dividing 100 by 4.
I still think fun can be used to score a game overall as it basically refers to the reviewer's mood towards the game. A reviewer would have a positive or negative mood about the game because of the visuals, sound, game play, and some other aspects.
WONDERăȘăăł (16778) on 10/28/2023 6:33 AM · Permalink · Report
"In a 1 to 5 star system, 1 star is valued 20%, not 0%."
True, but that is because a star system permits for 0 stars thus allowing it to go all the way down to 0%. People usually don't choose a 1-5 star system but a 0-5 star system. We, MobyGames, allow for scores less than 1 star (overall rating 0.4 stars: https://www.mobygames.com/game/45674/south-park-mega-millionaire/reviews/).
With Acao's color system (https://i.imgur.com/cQDz0dR.png), white is the zero. It is void of color, the little person is void of excitement, the word "fraco" translates to "weak", and is the lowest on their scale (again, see previous image). They do not offer a lack of symbol to equate to 0. Star systems often, including our own here, provide empty stars where it "lacks the score/quality" to fill the star to yellow. That's why star systems can be unhelpful with more granular, specific numbers that we force all review scores into (being 0-100%). The only times we do not force a score into the 0-100% system is when it uses the "unscored" type.
WONDERăȘăăł (16778) on 11/4/2023 7:37 AM · Permalink · Report
Hey Bozzly,
Finally circling back on this after some discussion with the other approvers and going through all 444 entries under Acao Games - https://www.mobygames.com/critic/1729/acao-games/sort:platform/page:0/
What we've gone with is the same way of calculating the points per category that I suggested previously, however, the final tally will be done more simply. I've left one of the following three notes/guidance on each of the corrected entries. I will also be using these same canned responses for any others I come across. Let me know if you have questions
- COLORS x 4: Per a discussion within the approver's channel on Discord, we've agreed with my way of calculating the value but went with a different way of scoring. The value of each category is calculated the same, white = 0pts, blue = 1pt, orange = 2pts, and pink = 3pts; giving a full range of 0%-100%. Using those points, we have 4 categories for a max total of 12 points. Add the values together (from each of the 4 categories) to get X. Then it's just X/12 as your score.
If you're unsure of what is being done, use the above-referenced URL and check out some of the source reviews compared to what they've been adjusted to. Or again, just ask. - HANDS x 4: Per a discussion within the approver's channel on Discord, we've agreed with my way of calculating the value but went with a different way of scoring. The value of each category is calculated the same, 0 fingers up = 0pts, 1 finger up = 1pt, 2 fingers up = 2pts, and all/4 fingers up = 3pts; giving a full range of 0%-100%. Using those points, we have 4 categories for a max total of 12 points. Add the values together (from each of the 4 categories) to get X. Then it's just X/12 as your score.
- HANDS x 3: Per a discussion within the approver's channel on Discord, we've agreed with my way of calculating the value but went with a different way of scoring. The value of each category is calculated the same, 0 fingers up = 0pts, 1 finger up = 1pt, 2 fingers up = 2pts, and all/4 fingers up = 3pts; giving a full range of 0%-100%. Using those points, we have 3 categories for a max total of 9 points. Add the values together (from each of the 3 categories) to get X. Then it's just X/9 as your score.
Bozzly (1078) on 11/4/2023 5:06 PM · Permalink · Report
I've been thinking. With regards to something being "agreed" upon, why is there a separate discussion somewhere outside Moby Games? Why can't the matter all be discussed in these forums so that all users can give their take on the subject?
Why should the lowest-valued color be worth 0? Why not 1? Scales up to 5 or 10 mostly use 1 as the minimum rating.
As to the idea of adding the points of each category then dividing it by the number of categories,, I see it as only a redundancy. To simply use the fun score should be adequate. In fact, a 4/4 or 3/4 mostly come with positive remarks while a 1/4 or 2/4 mostly come with negative remarks.
WONDERăȘăăł (16778) on 11/4/2023 6:09 PM · Permalink · Report
"Why should the lowest-valued color be worth 0? Why not 1?"
This is quite standard. Virtually all scoring systems will grade something 0-100, 0-10, 0-5. You don't see 0s because it is really rare for something to be that bad. Some of these publications/outlets simply won't score something a 0/100 but they will give it a 8/100 or a 0.5/5 stars.
"Scales up to 5 or 10 mostly use 1 as the minimum rating."
I entirely disagree. Most do use a 0 as the minimum, they might just not utilize it. That is to say, their scoring system still permits for 0 regardless if they ever grade something so low.
"As to the idea of adding the points of each category then dividing it by the number of categories,, I see it as only a redundancy."
As I most recently have stated, the derived values of each category can simply be added up, then the other side of the fraction is the max potential score. So, 3 pink squares and and orange would be an 11/12. As to you seeing it as a redundancy, that is unfortunate, but that is what is being asked.
"To simply use the fun score should be adequate. In fact, a 4/4 or 3/4 mostly come with positive remarks while a 1/4 or 2/4 mostly come with negative remarks."
We've gone over this. It's not. It's simply not adequate.
--
As for why we have private channels for discussion between approvers/admins? Not all discussions need to be made openly. Not all approvers are privy to channels of conversations between admins, devs, etc. That's quite common and shouldn't need much explaining. As for this particular discussion? It's quicker and easier for me to do it on Discord and there's no requirement that it HAS to be done here, openly, on the forums. Quite frankly, apart from the fact I can't Wayback open parts of Discord chat (i.e. it has a lack of permanency), I prefer that method of communication. It's more modern and I use it for more than JUST MobyGames. For instance, I don't get notifications on my desktop and mobile when you respond to this but I can and do on Discord. Forums very much have their place but they are... antiquated to put it nicely. I can @ people and DM them on Discord with much quicker response times than on here. One big downside is there is a split between Discord and the forums in that not everyone on one uses the other. I don't think Alaka, for instance, uses Discord and only recently did Kayburt from what I can tell. Same with Calavary and Rwolf (ones that quickly come to mind).
Any of the other approvers or participants of that discussion can certainly speak their piece, whether it be in support, opposition, or something else entirely. I've just been the one who caught this and brought it forward trying to get attention on it. Not wanting to be the sole opinion on the matter is why I brought this to the other approvers/admins. Also, to more quickly settle the matter so I could move on.
Rwolf (22867) on 11/4/2023 9:11 PM · Permalink · Report
I don't use Discord much; I've tried it a few times, but the persons I've needed to talk to were not in my time zone, and not around for chatting when I'm awake. Also I prefer forums for the possibility to look things up later, instead of trying to remember everything discussed in a fleeting stream. Interpreting spoken language is also a concern, there may be difficult dialects or words used which not everyone is familiar with, so as long as the forums exist I'll prefer to use them.
WONDERăȘăăł (16778) on 11/5/2023 6:41 PM · Permalink · Report
All fair points. I haven't used voice chat/audio in a looong time at this point. I'm not sure anyone actually uses it in the MobyGames server. At least not that I've seen. But yeah, a fleeting stream/chat is certainly a primary thing going against it when compared to a web forum - no doubt. It does now allow for threaded conversations within a channel, but not a lot of people seem to put it to use. In fact, I kinda forgot about it until just now. Could have used that in recent weeks regarding this specific Acao issue, lol.
Havoc Crow (29904) on 11/7/2023 10:01 PM · Permalink · Report
You're talking about voice chat, but Discord is good for quick back-and-forth in text. (Though it's not that novel, being basically IRC on steroids.)
WONDERăȘăăł (16778) on 10/28/2023 1:07 AM · Permalink · Report
I brought it up multiple times in Discord. Feel free to join us there as well. It is where I primarily communicate, my apologies if I missed you here.
Please see my other comments in this thread. I hope they are of use and are informative to you. In full disclosure, as of this morning - I have 15 pages (~300 reviews) of submissions I've changed for Acao Games. These reviews came from you, others, and another approver as well. When I see hundreds of errors for ONE publication, and the majority of those errors I had a hand in approving - I'm absolutely going to prioritize fixing bad data. Figuring this out has been the only focus for MG I've had since discovering the faults.
Kayburt (32359) on 11/7/2023 2:33 PM · Permalink · Report
Fun factor is still used in reviews today. Take for example WayTooManyGames https://waytoomany.games/2023/11/03/review-skull-island-rise-of-kong/
It should be noted that the fun factor is not used as an overall score. Iâm not sure it ever qualified as an overall score to begin with. Therefore I can only conclude that the 5 out of 5 score for a lot of those Acao Games reviews are not correct.
WONDERăȘăăł (16778) on 11/7/2023 4:11 PM · Permalink · Report
Totally. Some publications absolutely still use this but they make it pretty obvious: https://archive.org/details/SuperGamePower_Ano_04_No._047_1998-02_Nova_Cultural_BR_pt/page/n43/mode/2up
But as you say, in this case, it is not applicable and resulted in inaccurate scores being recorded. I've already gone through and adjusted the 444 Acao Games reviews with notes on each entry on how it was re-scored. They should now be accurate!
WONDERăȘăăł (16778) on 10/28/2023 1:00 AM · Permalink · Report
I have 15 pages (~300 reviews) of examples I can show you of what I've changed. All Acao Games. Mostly submissions from Bozzly and Chirinea.
But yup, I've been correcting these. Too many 100% scores were given to games that should be 50-67%. I've posted some extra details below in the comments as well as have posted in the approver's channel multiple times about this (I don't expect anyone to catch everything that gets spit into that chat, so no worries if you missed it or it didn't have any meaning when you did read it.)
At the top of this page is where I've left off. https://www.mobygames.com/critic/1729/acao-games/sort:title/page:16/
I almost submitted corrections on every one of them (and even started WIPing them at one point before stopping) but I don't really care about getting points for this work and I would not want to take points from Bozzly or Chirinea (as they did put the work in of finding the reviews and typing/pasting the text in). So, doing corrections just makes more work as does WIPing them (plus, no offense Bozzly, but I don't want to spend an extra couple of minutes on each submission to have to explain my specific score when there are hundreds that need to be adjusted. You've not been particularly agreeable in any way, and the back and forth comments are fully halting this being fixed. What I care most about is when I, and site visitors, come to use MobyGames like a "Metacritic for old, retro or obscure games" that they get accurate results and not inflated ones. Many of these games had 1-3 reviews. When you've got two 30% reviews and then Acao Games with a 100%, that very much skews what our visitors see and might waste their time. Especially for the games that ONLY have one review and it is from Acao. That review score should be accurate, or at least more so.
I digress but my plan is to continue through page 18 and then moving on to Nintendo Accion, and then after that I will see. I believe there are two other publications that are full of scoring errors but I need to first figure out how their scoring systems are calculated so that whatever I propose in the approver's channel can be, ideally, accurately stated and judged on.
Anyway, welcome to "the review issue" I've been going on about for the past two weeks or so.
Bozzly (1078) on 10/28/2023 5:32 AM · edited · Permalink · Report
It is not unusual for a game to have a wide range of reviewer scores. That just means the game has a polarized reception. With regards to site viewers think, its mostly the same regardless of such range.
Also, I don't think there's anything inflated on how we score the reviews. The comments of the reviews pretty much represent the scores.
Kayburt (32359) on 10/28/2023 5:37 PM · Permalink · Report
I don't know about that. Some of the Acao reviews tend to describe the game as it works and none of the pros and cons, which seems more like a game summary than an a critic review. It's like Acao couldn't make up its mind what it wanted to be. Makes you wonder what was going on behind the scenes, during the developing and writing of the magazines.
Bozzly (1078) on 10/28/2023 8:33 PM · Permalink · Report
If a review tells about the content but not the qualities of the game, that mostly means the reception is positive, thus the reviewer figures he can just leave the quality description to the ratings. Otherwise the reviewer will be explicit about the qualities in the comment.