Quan Nguyen
ABSTRACT
In recent years, the connection between Learning Design (LD) and Learning Analytics (LA) has been emphasized by many scholars as it could enhance our interpretation of LA findings and translate them to meaningful interventions. Together with numerous conceptual studies, a gradual accumulation of empirical evidence has indicated a strong connection between how instructors design for learning and student behaviour. Nonetheless, students' timing of engagement and its relation to LD and academic performance have received limited attention. Therefore, this study investigates to what extent students' timing of engagement aligned with instructor learning design, and how engagement varied across different levels of performance. The analysis was conducted over 28 weeks using trace data, on 387 students, and replicated over two semesters in 2015 and 2016. Our findings revealed a mismatch between how instructors designed for learning and how students studied in reality. In most weeks, students spent less time studying the assigned materials on the VLE compared to the number of hours recommended by instructors. The timing of engagement also varied, from in advance to catching up patterns. High-performing students spent more time studying in advance, while low-performing students spent a higher proportion of their time on catching-up activities. This study reinforced the importance of pedagogical context to transform analytics into actionable insights.
- Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S. and Lockyer, L. 2016. A conceptual framework linking learning design with learning analytics. In Proceedings of the Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, 2016. ACM, New York, NY, USA, 329--338. Google Scholar
Digital Library
- Bates, D., Mächler, M., Bolker, B. and Walker, S. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67 (1). 1--48.Google ScholarCross Ref
- Bennett, S., Agostinho, S. and Lockyer, L. 2015. Technology tools to support learning design: Implications derived from an investigation of university teachers' design practices. Computers & Education, 81. 211--220. Google ScholarDigital Library
- Biggs, J. B. and Tang, C. 2007. Teaching for quality learning at university:. Open University Press, Maidenhead, Beckshire, EnglandGoogle Scholar
- Broadbent, J. and Poon, W. L. 2015. Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27. 1--13.Google ScholarCross Ref
- Carless, D., Salter, D., Yang, M. and Lam, J. 2011. Developing sustainable feedback practices. Studies in Higher Education, 36 (4). 395--407.Google ScholarCross Ref
- Conole, G. 2012. Designing for learning in an open world. Springer Science & Business Media,Google Scholar
- Cross, S., Galley, R., Brasher, A. and Weller, M. Final Project Report of the OULDI-JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice. York: JISC. Retrieved October 16th, 2016, from http://www.open.ac.uk/blogs/OULDI/wp-content/uploads/2010/11/OULDI_Final_Report_Final.pdfGoogle Scholar
- Dalziel, J. 2015. Learning design: Conceptualizing a framework for teaching and learning online. Routledge, New York, NY, USA Google ScholarDigital Library
- Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett, S., Dobozy, E., Cameron, L., Badilescu-Buga, E. and Bower, M. 2016. The Larnaca declaration on learning design. Journal of Interactive Media in Education, 2016 (1). 1--24.Google Scholar
- Ferguson, R. 201 2. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4 (5-6). 304--317. Google ScholarDigital Library
- Gašević, D., Dawson, S., Rogers, T. and Gasevic, D. 2016. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28. 68--84.Google ScholarCross Ref
- Goodyear, P. 2015. Teaching as design. HERDSA Review of Higher Education, 2. 27--50.Google Scholar
- Häfner, A., Stock, A., Pinneker, L. and Ströhle, S. 2014. Stress prevention through a time management training intervention: an experimental study. Educational Psychology, 34 (3). 403--416.Google ScholarCross Ref
- Järvelä, S. and Hadwin, A. F. 2013. New Frontiers: Regulating Learning in CSCL. Educational Psychologist, 48 (1). 25--39.Google ScholarCross Ref
- Knight, S., Rienties, B., Littleton, K., Mitsui, M., Tempelaar, D. and Shah, C. 2017. The relationship of (perceived) epistemic cognition to interaction with resources on the internet. Computers in Human Behavior, 73 (Supplement C). 507--518. Google ScholarDigital Library
- Kovanovic, V., Gašević, D., Dawson, S., Joksimovic, S. and Baker, R. 2016. Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. 2016, 2 (3). 81--110.Google Scholar
- Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z. and Wolff, A. 2015. OU Analyse: analysing at-risk students at The Open University. Learning Analytics Review. 1--16.Google Scholar
- Li, N., Marsh, V., Rienties, B. and Whitelock, D. 2017. Online learning experiences of new versus continuing learners: a large-scale replication study. Assessment & Evaluation in Higher Education, 42 (4). 657--672.Google ScholarCross Ref
- Lockyer, L., Bennett, S., Agostinho, S. and Harper, B. 2009. Handbook of research on learning design and learning objects: issues, applications, and technologies (2 volumes). IGI Global, Hershey, PA. Google ScholarDigital Library
- Lockyer, L. and Dawson, S. 2011. Learning designs and learning analytics. In Proceedings of the Proceedings of the 1st international conference on learning analytics and knowledge, 2011. ACM, New York, NY, USA, 153--156. Google ScholarDigital Library
- Lockyer, L., Heathcote, E. and Dawson, S. 2013. Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57 (10). 1439 -- 1459.Google ScholarCross Ref
- Macfadyen, L. P. and Dawson, S. 2010. Mining LMS data to develop an "early warning system" for educators: A proof of concept. Computers & education, 54 (2). 588--599. Google ScholarDigital Library
- Maina, M., Craft, B. and Mor, Y. 2015. The Art & Science of Learning Design. Sense Publisher, Rotterdam, The NetherlandsGoogle Scholar
- Mor, Y., Ferguson, R. and Wasson, B. 2015. Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46 (2). 221--229.Google ScholarCross Ref
- Moskal, A. C. M., Stein, S. J. and Golding, C. 2016. Can you increase teacher engagement with evaluation simply by improving the evaluation system? Assessment & Evaluation in Higher Education, 41 (2). 286--300.Google ScholarCross Ref
- Nguyen, Q., Rienties, B. and Toetenel, L. Unravelling the dynamics of instructional practice: a longitudinal study on learning design and VLE activities. In Proceedings of the the Seventh International Learning Analytics & Knowledge Conference, Vancouver, British Columbia, Canada, 2017. ACM, New York, NY, USA, 168--177. Google ScholarDigital Library
- Nguyen, Q., Rienties, B. and Toetenel, L. 2017. Mixing and matching learning design and learning analytics. In P. Zaphiris and A. Ioannou (Eds), Learning and Collaboration Technologies: Forth International Conference, LCT 2017, Part II, Held as Part of HCI International 2017 (1--15). Springer International Publishing, ChamGoogle Scholar
- Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R. and Whitelock, D. 2017. Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76. 703--714. Google ScholarDigital Library
- Persico, D. and Pozzi, F. 2015. Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46 (2). 230--248.Google ScholarCross Ref
- Quené, H. and van den Bergh, H. 2004. On multi-level modeling of data from repeated measures designs: a tutorial. Speech Communication, 43 (1). 103--121.Google ScholarCross Ref
- Rienties, B., Toetenel, L. and Bryan, A. 2015. Scaling up learning design: impact of learning design activities on lms behavior and performance. In Proceedings of the Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, 2015. ACM, New York, NY, USA, 315--319. Google ScholarDigital Library
- Rienties, B. and Toetenel, L. 2016. The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60. 333--341. Google ScholarDigital Library
- Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I. and Dimitriadis, Y. 2015. Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46 (2). 330--343.Google ScholarCross Ref
- Tempelaar, D., Rienties, B. and Giesbers, B. 2015. In search for the most informative data for feedback generation: Learning Analytics in a data-rich context. Computers in Human Behavior, 47. 157--167. Google ScholarDigital Library
- Tempelaar, D., Rienties, B. and Nguyen, Q. 2017. Towards Actionable Learning Analytics Using Dispositions. IEEE Transactions on Learning Technologies, 10 (1). 6--16. Google ScholarDigital Library
- Toetenel, L. and Rienties, B. 2016. Learning Design-creative design to visualise learning activities. Open Learning: The Journal of Open, Distance and e-learning, 31 (3). 233--244.Google ScholarCross Ref
- Toetenel, L. and Rienties, B. 2016. Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making. British Journal of Educational Technology, 47 (5). 981--992.Google ScholarCross Ref
- van Merriënboer, J. J. G., Clark, R. E. and de Croock, M. B. M. 2002. Blueprints for complex learning: The 4C/ID-model. Educational Technology Research and Development, 50 (2). 39--61.Google ScholarCross Ref
- Vermunt, J. D. and Vermetten, Y. J. 2004. Patterns in Student Learning: Relationships Between Learning Strategies, Conceptions of Learning, and Learning Orientations. Educational Psychology Review, 16 (4). 359--384.Google ScholarCross Ref
- Winne, P. H. 2017. Learning Analytics for Self-Regulated Learning. In C. Lang, G. Siemens, A. F. Wise and D. Gaševic (Eds), The Handbook of Learning Analytics (241--249). Society for Learning Analytics Research (SoLAR), Alberta, CanadaGoogle Scholar
Index Terms
- Linking students' timing of engagement to learning design and academic performance
Recommendations
Effects of Learning Analytics on Students' Self-Regulated Learning in Flipped Classroom
The present article is aimed at analyzing the effects of learning analytics on students' self-regulated learning in a flipped classroom. An experiment was conducted with 96 engineering students, enrolled in a subject offered in the Flipped Classroom ...
Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement
Design for Teaching and Learning in a Networked WorldAbstractMassive open online courses (MOOCs) are part of the lifelong learning experience of people worldwide. Many of these learners participate fully. However, the high levels of dropout on most of these courses are a cause for concern. Previous studies ...
Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates
Many researchers who study the impact of computer-based assessment (CBA) focus on the affordances or complexities of CBA approaches in comparison to traditional assessment methods. This study examines how CBA approaches were configured within and ...
Comments