The new questionnaire for the Translation category went live today. With everyone from Utopian and DaVinci already spending a lot of time on it to make the sheet as helpful and thorough possible, there is always room for improvements!
The new questionnaire sparked some questions and a nice discussion on discord with @dimitrisp and @aboutcoolscience which resulted in this post, which will discuss some parts of the questionnaire and the future help document to accompany it.
The questionnaire
Some of the early changes to the questionnaire included merging a question about major and minor mistakes (now question 2), a change of words in question 3 and the merge of two questions that became question 4. To sum them up here:
2- How many mistakes were found in the translated text?
3- Is the translation consistent with the previously translated parts of the project?
4- Did the translator's post include all the information needed to fully evaluate the translation? If so, how would you rate the readability and grammar of the post?
2- How many mistakes were found in the translated text?
The merge of questions about minor and major mistakes is logical but still creates some possible problems: what counts as a mistake? Because of the way the score is calculated, there is quite a big difference between making 1, 3 or 6 mistakes. This poses the following questions:
- is a typo a mistake?
- is a repeated mistake counted once or every time?
- is a possible better word cause to count a mistake?
Personally, I would answer the following way:
- is a typo a mistake?
No, but repeated typos can be counted as one. (Minor mistakes?) - is a repeated mistake counted once or every time?
Once, as long as it isn't a mistake that has to be corrected too often, and not after the language moderator has mentioned it to the translator (so not in a following translation) - is a possible better word cause to count a mistake?
No, better words are often a matter of subjectivity.
This would be something to put in the help document, because there's a lot of different types of mistakes that do need a consistent way of handling across all languages and projects.
3- Is the translation consistent with the previously translated parts of the project?
As mentioned earlier, the first thing that changed here was the wording. This question seems to be a very appropriate question in some cases, for instance:
- multiple translators working on the same project
- a very large project
- a change of translators happening after a while
- a translator took a long break working on a project.
Other than in these instances, the translator's style should be consistent every time.
4- Did the translator's post include all the information needed to fully evaluate the translation? If so, how would you rate the readability and grammar of the post?
This question follows an if/else format, where the first part of the question is required or else the score will be set to unacceptable: if the translator did not include all information needed to fully evaluate the translation, the translation cannot be evaluated. Easy as that.
As can be seen from the scores you get for this question, considerably less attention is given to the post compared to the translation itself. This has been requested specifically by the language moderators, because in all honesty the post is mainly there because it is needed, it doesn't serve a goal of entertaining the readers. Therefor, though there is a lot that could be said about a post, we look at the readability and grammar because it shows in an instant whether or not a translator is capable of what they're actually doing.
The help document
What stands out most to me is the need for examples. Most of the questions follow a grading system from excellent to unacceptable. Though of course slightly objective, it's easy enough to see when a post is excellent (no mistakes, engaging, consistent, etc) or unacceptable (a lot of mistakes, inconsistent, etc).
The problem rises when you have to decide whether the accuracy of the translation is very good, good, fairly good or sufficient. These are still all "passing grades", scores being 19, 15, 12 and 9. Though again comparing extremes with each other (very good versus sufficient) might provide some insights, you can't keep comparing the extremes. So there is a need for an examplary post that shows when strings are excellent and when they are, well, any of the other options.
Which raises another issue, as discussed with @dimitrisp: this is different for every language. Different languages of course follow different rules. There really isn't a one size fits all rulebook for these questions, and as such it might become the task of the language moderators to make their own examples. This would help their translators, but still leave room for bias or subjectivity. The solution? I'm not sure!
There is something to be said for a standard sentence that gets translated and, if needed, changed for the different languages, to keep the general thought intact. Would that solve it? I don't know.
Hello @minersean, thank you for the submission. Firstly, I would like to appreciate everyone that has contributed to the betterment of the Translation category. We can only hope to be better not best. As you pointed out in this post, "there is always room for improvement".
About those questions that you highlighted in this post, I like most of your opinions about them, but I think typos will always be seen as mistakes. Though, there are different types and causes of typos. One of them is the " Atomic typos" which occur as a result of an accurately spelled word that is different from the intended word. Example, "my" instead of ''mine''. An error like this changes the meaning of a whole sentence which is not cool. So, I still believe every typo is a mistake.
Presumably, the help document will help in fixing and helping LM's to make right and accurate decisions.
Please note that while the CM hasn't changed the footer, I am not scoring #iamutopian posts based on the questionnaire. They have their own metric, and that will be the case until we go live with the new guidelines and new questionnaire, which will be comprehensive enough to reflect these types of posts.
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thanks for your review!
I have to say that after talking with a lot of different people and reading a lot of different opinions, I'm starting to see your point about typos, especially the atomic typos. I still think that maybe a missplaced comma or another obvious accidental keypress maybe shouldn't count as a complete mistake, but then again it does show a lack of checking which is a mistake in and of itself.
Thanks again!
Thank you for your review, @tykee! Keep up the good work!
I'm going to analyze my thoughts on the second question, about the mistakes:
If it is altering the meaning, yes! For example, in Greek language "ταιριάζετε" and "ταιριάζεται" completely change the meaning, even though it is just a mistake two letters. "Ταιριάζετε" is active voice (you match), while "ταιριάζεται" is passive voice (he/she/it is matched).
However, if it is an honest mistake (for example, missing a letter) and depending on the context I might not count it as a mistake
Well, most of the times no. But if the translator gets it right once, and 3 times wrong, then I would count it every time, as it means that they didn't pay attention.
Generally, not all times, but it depends on the project and the target language. If we are talking about (for example) The Curious Expedition, where we are allowed to freely translate it the way we want (and add our own parts on the strings that are randomly chosen by the Game Engine), I won't count it as a mistake. If the context changes by changing to the "Better word", then it should be a mistake
As for the post format question, if the translator fails to include one (or more) of the following, I will notify them the first time (we all have bad days when we forget to do something). If it happens again, it means they are not paying enough attention, so it is an immediate "bad" and then "unacceptable" (as the Utopian bot AND our review both depend on that info):
Thanks for the post, it was an awesome read! 🙂
I agree with your point on the typo's, meaning altering should be counted as a mistake unless obviously accidental. If there's suddenly a comma in the middle of a word or a repeated letter, I generally don't count that as a mistake, unless it keeps happening in which case it's a sign of the translator not being careful.
The second point is a hard agree as well, at that point you start to wonder whether the right one was perhaps an accident haha!
As for the third point, generally I would hope that the context wouldn't change because of a better word. The way I imagine it is that for instance someone decides to use "compilations" but the LM thinks that "abstracts" might fit the context better. The meaning of the sentence shouldn't change, but one word might fit the tone of the project better than the other.
I agree with you on the last point as well! Everyone has bad days, but consistent failure to include something should indeed get a lower score. I like moving from "bad" to "unacceptable" instead of going to "unacceptable" immediately, I hope I won't have to use it but if so I feel like that's a fair step to take!
Thanks for your reply!
Thank you @minersean for sharing your thoughts with us, and also for having the time to write this post.
We tried to remove subjectivity with the first questionnaire by focusing on the segments that need to be taken into consideration, but people didn’t like it. We also implemented professional standards used by the industry and proposed by Rosa to differentiate the type of mistakes but people did not like it. We pushed for more granularity to remove subjectivity and people did not like it. Utopian wanted more granularity but with fewer questions, this is clearly contradictory, even professional tools on the market can’t do that. On top of that, we were not able to work properly since our methodological way of working was broken for some reason by Utopian which made everything look like a traumatic event.
Some of your points are valid and we tried to implement them before. I personally stopped thinking and caring about the questionnaire when Utopian steeped in (vigilante style) and tried to please everyone. Because in my opinion 'criticized' doesn't necessarily mean that a change is needed. Because the next one will be 'criticized ' and the one after I will 'criticized' it ... being 'criticized' should never be the sole criterion to decide that something should be refused since a consensus will always be out of reach, and we may end up with something really bad just because people gradually lost interest in the questionnaire drama. Thus, it was never about common sense, it was more like about personal preferences, the situation, in my opinion, highlighted a high conflict of interest.
We received some wonderful feedback and suggestions from people in the translation category, their suggestions show a deep understanding of how things should be, and I am personally really proud to have people like this in our category. But the concept of feedback was misused/misunderstood in my opinion when some people started to consider all feedback as absolute truth.
Conclusion: Can't blame anyone … There were just too many cooks in the kitchen!
Hi, @minersean!
You just got a 5.48% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.
Hey, @minersean!
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!
Congratulations! This post has been upvoted from the communal account, @minnowsupport, by sean from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, someguy123, neoxian, followbtcnews, and netuoso. The goal is to help Steemit grow by supporting Minnows. Please find us at the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.
If you would like to delegate to the Minnow Support Project you can do so by clicking on the following links: 50SP, 100SP, 250SP, 500SP, 1000SP, 5000SP.
Be sure to leave at least 50SP undelegated on your account.
Hi @minersean!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 0.353 which ranks you at #79904 across all Steem accounts.
Your rank has improved 37 places in the last three days (old rank 79941).
We curated this post of yours in our last Algorithmic Curation Round, which included the curation of in total 189 contributions. Unfortunately, your post ranked last, at #189.
Evaluation of your UA score:
:* Only a few people are following you, try to convince more people with good work.
Feel free to join our @steem-ua Discord server