This influential paper is a must-read for anyone who cares about making science more reliable. It exposes how researchers can easily manipulate data collection and analysis to make results look significant when they might not be. The authors argue that this flexibility leads to misleading conclusions and call for more transparency in research methods.
The Strategic Data Project (SDP) at Harvard University interviewed 40 leaders and analysts at 29 institutions of higher education and postsecondary organizations to understand why some colleges and university systems are excelling in using data and others have yet to fully realize the potential of their data to inform strategic decisions that transform student success in school and the workforce.
[Their] key finding is that the missing link is not in the technical infrastructure but in human capacity. If higher education is to take advantage of data analytics to improve student outcomes and increase organizational effectiveness, it will have to find better ways to attract, train, and retain strategic data professionals who can inform policy and practice.
Project, S. D. (2020). Strategic Data Use in Higher Education: Using Data to Improve Postsecondary Success
By automatically recording students’ interactions with online course materials, clickstream data provide a valuable new source of information on student learning behaviors. These data can be used to define and identify behavioral patterns that are related to student learning outcomes, suggest behavioral changes to students for greater success, and provide insights regarding the mechanisms by which education interventions affect student outcomes.
Baker, R., Xu, D., Park, J. et al. The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes. Int J Educ Technol High Educ 17, 13 (2020). https://doi.org/10.1186/s41239-020-00187-1
(I had the honor of speaking with Dr. Baker last week about U.S. education metrics & policy. What a dedicated person and professor.)