At CLEAR Global / Translators without Borders we assess quality using the harmonized Dynamic Quality Framework - Multidimensional Quality Metrics (DQF-MQM) error typology.
We use a simplified version of this error typology with five error categories. Each category includes specific issues that should be considered when we work on a project.
Find below more information on how we score for fluency.
Fluency. We want to see a text with a natural flow and that is up to target language norms and conventions for spelling, punctuation, and grammar.
Here are some of the most relevant fluency issues:
- incorrect punctuation,
for example, we only use commas when they appear in the source text, not considering the rules of our target language
- wrong spelling,
“colour” in a language variety that spells it “color” instead,
- grammar mistakes and issues with the word order (syntax),
where you need to improve grammar on “has had been done” or where the target word order follows too close the source and you need to move words around in “Mary happy has not been”
- problems with register, when lexical or grammatical decisions make the text either too formal or too informal for the expected output, instructions or context,
we might be too formal if we are addressing a child as a “Mr.” and too informal if we welcome participants of a humanitarian course with a “Heya”
- inconsistencies that occur when something that should have one unified approach is presented in different ways,
think here about the use of both “app.” and “approx.” for “approximately” in one text
How to score fluency?
Make sure you are classifying the issues in the correct category and consider the impact those issues have on the target text. Are they significant? How do they impact the understanding by the target audience?
Keep in mind you are scoring your colleague’s whole chunk or translation. It is important to provide a fair assessment of the complete work.
Make sure you praise exceptional work! 
It is also important to leave a comment about the category together with the score, so your peer can understand the feedback. This is particularly vital when leaving low scores. If you were to receive that score yourself, you would surely appreciate a colleague who also took the time to be helpful!
We know this is not easy, so we put together a few ideas on what each level might mean to guide you on how to decide on a score:
5 Stars: Excellent. The text is fluent and reads naturally. There might be a minor slip or two, but no major issues with fluency.
4 Stars: Good. Overall, punctuation, spelling, and grammar were correct and consistent. Just some minor slips, and occasional issues with fluency.
3 Stars: Satisfactory. The translation was fluent for the most part but there were a few issues.
2 Stars: Needs work. There were numerous issues with fluency that impacted readability in the target text.
1 Star: Poor. The translation was not fluent. The grammar, spelling and punctuation of the target language was mostly ignored.
This is meant as guidance, and there will be situations where not all conditions apply. You should always look at the specific case in front of you to decide!