How does written English fluency in poll response affect the grading in Minerva?
My capstone is to investigate whether English fluency affects grading in our schools full of non-native speakers. Despite my curiosity, more personal reasons exist to motivate me to continue on my capstone.
I write it out loud here to remind myself not to be overly attached to my research direction or the result. If my hypothesis is eventually falsified, or if I end up modifying my research direction, it’s okay. The purpose of my capstone is the process of learning applicable skills and understanding self-interests, not about generating a significant result.
Inspiration: How to develop a good research taste?
My school, Minerva is constituted of students from 40+ countries. Therefore, we have an 80% non-English native population, and lots of them struggled to keep pace with reading papers and communicating with others in the class.
In this project, I’m curious in understanding whether English fluency will affect grades students get independent from actual knowledge level. For example, if a student can compose their sentences more eloquently or use lots of complex words and technical terminology, will professors tend to give them higher grades? In contrast, even if a student understand the knowledge in depth, if they simply explain concepts in simple terms, will professors become less appreciative of their work?
My big question is the following.
<aside> 👉 How does written English fluency in poll response affect the grading in Minerva?
</aside>
To find the relationship between grades and English fluency, we will first need to define what is English fluency and how to assess it.
If we look at criteria from global English assessment organizations such as IELT, we will see that IELT writing assessment criteria shows that they include
- Advantage: It is an accurate and standardized assessment criteria. It has been trusted and used.
- Disadvantage: It takes a human to create scores.
Researchers translate the measurement of speaking fluency to writing and explore methods that are beyond the written output.
Moving further beyond the product-based measurement, they explore processed-based measurement through real-time measurement of writing fluency, such as composing rate and writing in chunks (What Do We Mean by Writing Fluency and How Can It Be Validly Measured?). Perl (1979) observed that non-fluent writing has two characteristics, “sentences are written in chunks” and “each sentence is generated in isolation.” In research, (Using Writing Process and Product Features to Assess Writing Quality and Explore How Those Features Relate to Other Literacy Tasks _ Enhanced Reader), they use a combination of process-based and product-based measurement for written language fluency.
- Advantage: It is comprehensive since it includes nuances from the writing process and lots of new features
- Disadvantage: Data for the writing process might not be available in my
research.
Grammarly is the most well-known writing assistant for spelling, grammar, punctuation, clarity, engagement, and delivery of mistakes in the text. If we create a page, “Set goals” will appear to define some basic characteristics for writing, which will adapt writing suggestions according to the initial goal.
If you paste in some texts, we see that they will provide a performance score and 5 areas, correctness, clarity, engagement, delivery, and style that I can improve upon.
For instance, in the report of my sample text, Grammarly will point out specific errors in my writing and identify sentences and word choices to improve clarity.
Though they have documented some technical processes in their engineering blog, they haven’t published information on how the specific scores in the 5 areas are modeled or measured.
- Advantage: It can be completed automated by a machine.
- Disadvantage: Can be difficult to construct. Lack of information on the
modeling process.
Since the available data only contains all the poll writing output from 2019, I will only focus on the writing output, not the writing process, and consult the metrics from Existing Solution Analysis to generate metrics to represent fluency.