Review: Effects of real-time analytics-based personalized scaffolds

Task

Within the first course of the master's program in IT & Learning, I was asked to write a short review on one paper of my choice. Further, I needed to document the research process, problems, and main takeaways as well as propose some discussion topics.

---

Reference

Lim, L., Bannert, M., Van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., Rakovic, M., Molenaar, I., Moore, I. & Gaˇsevi ́c, D. (2023). Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning. Computers in Human Behavior, 139, 1-18. https://doi.org/10.1016/j.chb.2022.107547

---

Review

The paper discusses the need for personalized support (scaffolds/prompts) in online self-regulated learning activities. The authors argue, that students have trouble to self-regulate their learning and need support, which takes their overall need, the kind of support as well as the granularity into consideration. Based on this they developed an analytics-based real-time assessment algorithm, which personalizes scaffolds to learners’ individual needs on the basis of their learning activities.

They separated 98 university students into three groups (personalized scaffolds, general scaffolds, and control group (no scaffolds) and asked them to write an essay, by reviewing three articles online in a specific learning tool within two hours. Depending on the group they received (personalized) prompts, to help them figure out their next move. These prompts were shown as a pop-up on the screen with at least two buttons with suggestions on what they should do next.

Data were collected by analyzing coded sequences of raw log activities, mouse clicks, and keystrokes. To measure the learning outcome there were tests taken before and after the experiment.

The findings showed that there was a significant effect of personalized scaffolds on the self-regulated learning experiences, but not on the learning outcomes. Further, the personalized scaffolds increased students’ activities but didn’t induce new activities compared to the other groups, since the overall strategy of how to handle the task was relatively similar within all groups.

Search process

I found this article by searching within the ‚Education Research Complete'-database with the following search string: (‚information technology‘) AND (learning OR education) AND (adaptive)

The term ‚real-time analytics-based personalization‘ was striking to me, as it seemed to focus on the personalization aspect of learning. Some other articles felt very technical, which I assumed would be tough to read because of my lack of knowledge in engineering. This article is in line with my interest in the opportunity of personalizing education for students, which seems to be a recurring problem for teachers due to the size of classes and other tasks.

Difficulties & Takeaways

• The term ‚scaffold’ was not clear from the beginning. Only after looking it up in the context of education, did it become clear what the article was referring to.

• Scaffolds in an educational context is a concept, where teachers support students, but gradually minimize the support based on the development of the student's own competencies.

More about scaffolding

Discussion ideas

It would be interesting to discuss the overall aim of educational support/scaffolds and how the outcome can be measured.

The findings showed that the learning outcomes didn’t differ, but students were more engaged in meta-cognitive activities (planning, seeing the big picture, etc.). Focusing solely on learning outcomes, one might suggest, that personalized scaffolds weren’t of interest. However looking at the overall process of learning, it seems to make a difference.

Further, it would be interesting to take a closer look at the level as well as the kind of personalization. The current study offered clickable buttons within the same tool.

The support offered was solely based on a structural and timing basis, rather than on individual questions, or tasks. Looking at machine-learning-based AI, the opportunities to evolve the content of the scaffolds based on individual learning activities might have a wider span, as the authors suggested in their article.

I received these comments from my peers: