Marshall Lovett’s opening address on ‘Cognitively informed analytics to improve teaching and learning’. Main focus of research has been on how students learn and how we can tell how well students are learning. Traditionally done via tests and comparing to peers. There are new ways using learning analytics by tracking activity, assignments and grades. How can we use analytics to get fuller insights into student learning? Students spend 100 hours studying but archieve learning gains of only 3% (Lovett, Meyer and Thille 2008). Based on study of stats students following a traditional course design. A different approach with an adaptive data driven course with 50 hours study achieved 18% gain.
predictive measures can lead to action, eg identifying potentially vulnerable students. Important to think about underlying model, so need prediction + understanding to get targeted action.
helpful for students and tutors to know what is going wrong and provide insights into student learning. Metaphor of diagnosis + recommended treatment.
informed by cognitive theory, understanding of student states and how students learn and solve problems across a range of task domains. Focused on example of Power law of learning. As students practice, performance improves but improvement follows pattern of marginally decreasing returns. How to use this as a diagnostic? Need to look at practice opportunities for a given skill and understand the pattern. What are the separate skills that students are learning? Teachers can track student learning skill by skill and adapt teaching to meet student need. Students can also monitor their own strengths and weaknesses and focus effort where it is needed most.
Build on solid course design – align with the skills students need to learn, offer opportunities for related practice, offer targeted and timely feedback.
meeting users’ needs – instructors and students. Instructors often have coarse records of student grades, need quick up to date actionable information to guide interventions to be more effective. Need quick snapshot and access to detail on student strength and areas of difficulty. Helpful if they can also get pointers to opportunities for adapting their teaching.
Students often pay most attention to their grade and have an emotional response, don’t know what to do next as class has often then moved on to another topic. So they also need access to the same actionable up to date information. Can track patterns in student study behaviours and offer advice and guidance.
Visualisations should be quick to apprehend, flexible enough to allow drill down, customisable to individual needs.
Lovett has been working on a learning dashboard which measures student learning skill by skill. Incorporates bayesian statistical models that embed the power law of learning. Makes inferences about student learning and their strengths and weaknesses as they study with remediation sources of help.