Methods

After you have chosen an objective for applying learning analytics, it might still not be easy to choose a proper method to achieve this objective. Learning Analytics focuses on applying known methods and models to address issues affecting students learning process and the organizational learning system. The list of learning analytics methods is long, and would even be longer if all different variations of such methods were taken into account. However, it is important to get a higher level grasp of what kind of methods are out there and what are they good for.

For this purpose, we selected the list of methods presented below. By clicking each method, you get a description of what is it all about. In addition, in case you would like to read more about how the method was applied in learning analytics research, there are a few sources provided for most of the specific methods.

Prediction methods

• Classifiers
  Classifiers

  Close

  In classifiers, the predicted variable can be either a binary (e.g. 0 or 1) or a categorical variable. Some popular classification methods in educational domains include decision trees, random forest, decision rules, step regression, and logistic regression.
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  - Amancio, D. R., Comin, C. H., Casanova, D., Travieso, G., Bruno, O. M., Rodrigues, F. A., & da Fontoura Costa, L. (2014). A systematic comparison of supervised classifiers.PloS one, 9(4), e94137.
  
  - Baker, R., & de Carvalho, A. (2008, June). Labeling student behavior faster and more precisely with text replays. InEducational Data Mining 2008.
  
  - González-Brenes, J., Mostow, J., & Duan, W. (2010, June). How to classify tutorial dialogue? comparing feature vectors vs. Sequences. InEducational Data Mining 2011.
  
  - Jayaprakash, S. M., Moody, E. W., Lauría, E. J., Regan, J. R., & Baron, J. D. (2014). Early alert of academically at-risk students: An open source analytics initiative.Journal of Learning Analytics, 1(1), 6-47.
  
  - Cetintas, S., Si, L., Xin, Y. P., Zhang, D., Park, J. Y., & Tzur, R. (2010). A joint probabilistic classification model of relevant and irrelevant sentences in mathematical word problems.Journal of Educaltional Data Mining, 3(2), 83-101.
• Regressors
  Regressors

  Close

  In regressors, the predicted variable is a continuous variable, e.g. a number. The most popular regressor in EDM is linear regression (note that linear regression is not used the same way in EDM/LA as in traditional statistics, despite the identical name).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  - Elbadrawy, A., Studham, R. S., & Karypis, G. (2015, March). Collaborative multi-regression models for predicting students' performance in course activities. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 103-107). ACM.
  
  - Andergassen, M., Mödritscher, F., & Neumann, G. (2014). Practice and repetition during exam preparation in blended learning courses: Correlations with learning results. Journal of Learning Analytics, 1(1), 48-74.
  
  - Käser, T., Koedinger, K., & Gross, M. (2014, July). Different parameters-same prediction: An analysis of learning curves. In Educational Data Mining 2014.
  
  - Miller, L. D., Soh, L. K., Samal, A., Kupzyk, K., & Nugent, G. (2015). A Comparison of Educational Statistics and Data Mining Approaches to Identify Characteristics That Impact Online Learning. Journal of Educational Data Mining, 7(3), 117-150.
  
  - Rogers, T., Colvin, C., & Chiera, B. (2014, March). Modest analytics: using the index method to identify students at risk of failure. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 118-122). ACM.
• Latent knowledge estimation
  Latent knowledge estimation

  Close

  A third type of prediction model that is important in EDM/LA (which is actually just a special type of classifier) is latent knowledge estimation. In latent knowledge estimation, a student’s knowledge of specific skills and concepts is assessed by their patterns of correctness on those skills (and occasionally other information as well). The models used in online learning typically differ from the psychometric models used in paper tests or in computer-adaptive testing, because with an interactive learning application, the student’s knowledge is continually changing. A wide range of algorithms exist for latent knowledge estimation; the two most popular are currently Bayesian Knowledge Tracing (BKT -- Corbett & Anderson, 1995) and Performance Factors Analysis (PFA -- Pavlik, Cen, & Koedinger, 2009), which have been found to have comparable performance in a number of analyses (see review in Pardos et al., 2011). Knowledge estimation algorithms increasingly underpin intelligent tutoring systems, such as the Cognitive Tutors currently used for Algebra in 6% of U.S. high school classrooms (cf. Koedinger & Corbett, 2006).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Bayesian Knowledge Tracing algorithm
  Bayesian Knowledge Tracing algorithm

  Close

  In addition to these general-purpose tools, other tools are available for special purposes. For example, two competing packages are available for estimating student knowledge with Bayesian Knowledge Tracing (e.g., Chang et al., 2006; Baker et al., 2010). Tools for supporting the development of prediction models, by obtaining data on the predicted variable through hand-annotating log files, have also recently becom
  e available (Rodrigo et al., 2012). Tools for displaying student learning over time and the pattern of student performance for different problems or itemshave been embedded into the Pittsburgh Science of Learning Center’s DataShop, a very large public database on student use of educational software (Koedinger et al., 2010).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Performance Factors Analysis algorithm
  Performance Factors Analysis algorithm

  Close

  A third type of prediction model that is important in EDM/LA (which is actually just a special type of classifier) is latent knowledge estimation. In latent knowledge estimation, a student’s knowledge of specific skills and concepts is assessed by their patterns of correctness on those skills (and occasionally other information as well). The models used in online learning typically differ from the psychometric models used in paper tests or in computer-adaptive testing, because with an interactive learning application, the student’s knowledge is continually changing. A wide range of algorithms exist for latent knowledge estimation; the two most popular are currently Bayesian Knowledge Tracing (BKT -- Corbett & Anderson, 1995) and Performance Factors Analysis (PFA -- Pavlik, Cen, & Koedinger, 2009), which have been found to have comparable performance in a number of analyses (see review in Pardos et al., 2011). Knowledge estimation algorithms increasingly underpin intelligent tutoring systems, such as the Cognitive Tutors currently used for Algebra in 6% of U.S. high school classrooms (cf. Koedinger & Corbett, 2006).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Galyardt, A., & Goldin, I. (2015). Move your lamp post: Recent data reflects learner knowledge better than older data.Journal of Educational Data Mining, 7(2), 83-108.
  
  - Gong, Y., & Beck, J. (2010, June). Items, skills, and transfer models: which really matters for student modeling?. InEducational Data Mining 2011.
  
  - Galyardt, A., & Goldin, I. (2014, July). Recent-performance factors analysis. InEducational Data Mining 2014.
  
  - Käser, T., Klingler, S., & Gross, M. (2016, April). When to stop?: towards universal instructional policies. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 289-298). ACM.
• Learning Performance Vector (LPV)
  Learning Performance Vector (LPV)

  Close

  In the context of the Lea’s Box project we developed an approach for predicting student performance which is based on the theoretical foundations of CbKST and which might offer an interesting, top-down technique to the field of performance prediction. The approach, named Learning Performance Vector (LPV), has been described in deliverable D3.4. In essence, the idea is that the structural information about the learning domain, the atomic units of aptitude (we name them competencies), and the relationships between these competencies provide a pool of important information for predications. In addition, we can add deeper information about the individual competencies which we call “weights”. These weights reflect a competency’s complexity, difficulty, or importance for a domain. Together with the actual performance data of student’s we hypothesized that performance predictions can be improved. In the context of the project the LPV algorithm to predict a student’s Learning Horizon (LH) have been developed and implemented in the Lea’s Box system.
  
  For Details
  

  
  
  

  Reference: Kickmeier-Rust, M. D. (2017). The Learning Performance Vector. Technical report, Technical University Graz, Austria.

  
  
  

Structure discovery algorithms

• Clustering
  Clustering

  Close

  In clustering, the goal is to find data points that naturally group together, splitting the full data set into a set of clusters. Clustering is particularly useful in cases where the most common categories within the data set are not known in advance. If a set of clusters is well-selected, each data point in a cluster will generally be more similar to the other data points in that cluster than data points in other clusters. Clusters have been used to group students (cf. Beal, Qu, & Lee, 2006) and student actions (cf. Amershi & Conati, 2009). For example, Amershi & Conati (2009) found characteristic patterns in how students use exploratory learning environments, and used this information to identify more and less effective student strategies.
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Azarnoush, B., Bekki, J. M., Runger, G. C., Bernstein, B. L., & Atkinson, R. K. (2013). Toward a framework for learner segmentation.Journal of Educational Data Mining, 5(2), 102-126.
  
  - Trivedi, S., Pardos, Z. A., Sarkozy, G. N., & Heffernan, N. T. (2012). Co-Clustering by Bipartite Spectral Graph Partitioning for Out-of-Tutor Prediction.International Educational Data Mining Society.
  
  - Xu, B., Recker, M., Qi, X., Flann, N., & Ye, L. (2013). Clustering educational digital library usage data: A comparison of latent class analysis and k-means algorithms. Journal of Educational Data Mining, 5(2), 38-68.
  
  - Ezen-Can, A., & Boyer, K. E. (2015). Understanding student language: An unsupervised dialogue act classification approach.Journal of Educational Data Mining, 7(1), 51-78.
  
  - Saarela, M., & Kärkkäinen, T. (2015). Analysing student performance using sparse data of core bachelor courses.Journal of educational data mining, 7(1), 3-32.
• Factor analysis
  Factor analysis

  Close

  In factor analysis, a closely related method, the goal is to find variables that naturally group together, splitting the set of variables (as opposed to the data points) into a set of latent (not directly observable) factors. Factor analysis is frequently used in psychometrics for validating or determining scales. In EDM/LA, factor analysis is used for dimensionality reduction (e.g., reducing the number of variables) for a wide variety of applications. For instance, Baker et al. (2009) used factor analysis to determine which design choices are made in common by the designers of intelligent tutoring systems (for instance, tutor designers tend to use principle- based hints rather than concrete hints in tutor problems that have brief problem scenarios).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Oster, M., Lonn, S., Pistilli, M. D., & Brown, M. G. (2016, April). The learning analytics readiness instrument. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 173-182). ACM.
  
  - Lan, A. S., Studer, C., & Baraniuk, R. G. (2014). Quantized matrix completion for personalized learning.arXiv preprint arXiv:1412.5968.
  
  - Ahn, J. (2013, April). What can we learn from Facebook activity?: using social learning analytics to observe new media literacy skills. InProceedings of the third international conference on learning analytics and knowledge (pp. 135-144). ACM.
  
  - Zimmermann, J., Brodersen, K. H., Heinimann, H. R., & Buhmann, J. M. (2015). A Model-Based Approach to Predicting Graduate-Level Performance Using Indicators of Undergraduate-Level Performance.Journal of Educational Data Mining, 7(3), 151-176.
• Social network analysis
  Social network analysis

  Close

  In social network analysis (SNA), models are developed of the relationships and interactions between individual actors, as well as the patterns that emerge from those relationships and interactions. A simple example of its use is in understanding the differences between effective and ineffective project groups, through visual analysis of the strength of group connections (cf. Kay et al., 2006). SNA is also used to study how students’ communication behaviors change over time (cf. Haythornthwaite, 2001), and to study how students’ positions in a social network relate to their perception of being part of a learning community (cf. Dawson, 2008). This is valuable information because patterns of interaction and connectivity can indicate prospect of academic success as well as learner sense of engagement in a course (Macfadyen and Dawson, 2010; Suthers and Rosen, 2011).
  SNA reveals the structure of interactions, but does not detail the nature of exchanges or the impact of connectedness. Increasingly, network analysis is paired with additional analytics approaches to better understand the patterns observed through network analytics; for example, SNA might be coupled with discourse analysis (see Enyedy and Stevens, this volume; also Buckingham Shum and Ferguson, 2012).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Rabbany, R., Takaffoli, M., & Zaïane, O. R. (2011). Analyzing participation of students in online courses using social network analysis techniques. InProceedings of educational data mining.
  
  - Teplovs, C., Fujita, N., & Vatrapu, R. (2011, February). Generating predictive models of learner community dynamics. InProceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 147-152). ACM.
  
  - Suthers, D., & Rosen, D. (2011, February). A unified framework for multi-level analysis of distributed learning. InProceedings of the 1st international conference on learning analytics and knowledge (pp. 64-74). ACM.
  
  - Joksimović, S., Manataki, A., Gašević, D., Dawson, S., Kovanović, V., & De Kereki, I. F. (2016, April). Translating network position into performance: importance of centrality in different network configurations. InProceedings of the sixth international conference on learning analytics & knowledge (pp. 314-323). ACM.
• Domain structure discovery
  Domain structure discovery

  Close

  Domain structure discovery consists of finding the structure of knowledge in an educational domain (e.g., how specific content maps to specific knowledge components or skills, across students). This could consist of mapping problems in educational software to specific knowledge components, in order to group the problems effectively for latent knowledge estimation and problem selection (cf. Cen, Koedinger, & Junker, 2006), or could consist of mapping test items to skills (cf. Tatsuoka, 1995). Considerable work has recently been applied
  to this problem in EDM, for both test data (cf. Barnes, Bitzer, & Vouk, 2005; Desmarais, 2011), and for tracking learning during use of an intelligent tutoring system (Cen, Koedinger, & Junker, 2006).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Formal Concept Analysis (FCA)
  Formal Concept Analysis (FCA)

  Close

  Formal Concept Analysis (FCA), established by Wille (2005), aims to describe concepts and concept hierarchies in mathematical terms. The starting point of the FCA is the specification of a “formal context” (also called learning domain). The formal context K is defined as a triple (G, M, I) with G as a set of objects which belong to the learning domain, M as a set of attributes which describe the learning domain, and finally, I as a binary relation between G and M. The relation I connects objects and attributes. The formal context K can be best read when depicted as a cross table, with the objects in the rows, the attributes in the columns and relations between them. So, FCA is applied for identified unique patterns in data sets but, in addition, it can also uncover the hierarchical structure among the patterns. By this means, distinct student and competency cluster can be identified.
  
  References:
  Bedek, M., Kickmeier-Rust, M., & Albert, D. (2015). Formal Concept Analysis for Modelling Students in a Technology-enhanced Learning Setting – in: Workshop on Awareness and Reflection in Technology Enhanced Learning, 10th European Conference on Technology enhanced Learning (EC-TEL 2015).
  Kickmeier-Rust, M D., Bedek, M., & Albert, D. (2016). Theory-based Learning Analytics: Using Formal Concept Analysis (FCA) for Intelligent Student Modelling. In H. R. Arabnia, et al. (Eds.), Proceedings of the 2016 International Conference on Artificial Intelligence (ICAI), July 25-28, 2016, Las Vegas, NV. CSREA Press.
  Wille, R. (2005). Formal Concept Analysis as Mathematical Theory of Concepts and Concept Hierarchies, In B. Ganter , G. Stumme and R. Wille (Eds.), Formal Concept Analysis (pp. 1-34), Berlin: Springer.
  
  Read more about the method in
  
  - Romashkin, N. S., Ignatov, D. I., & Gorbunova, E. (2011). How university entrants are choosing their department? mining of university admission process with FCA taxonomies. InInternational Conference on Educational Data Mining (EDM) 2011. Proceedings of the 4th International Conference on Educational Data Mining. Eindhoven, 6-8 July, 2011 (pp. 229-234). Eindhoven University of Technology.
  
   
• KST / CbKST – based Structural Modelling Approaches
  KST / CbKST – based Structural Modelling Approaches

  Close

  Competence-based Knowledge Space Theory provides a theoretical framework for knowledge and competence modeling (Falmagne & Doignon, 2011; Falmagne, Albert, Doble, Eppstein, & Hu, 2013). It is a powerful approach for structuring and representing domain and learner knowledge. In its original formalisation, a knowledge domain is characterized by a set of problems or test items. The knowledge state of an individual is identified with the subset of problems this person is able to solve. Due to mutual dependencies between the problems, not all potential knowledge states will occur. These dependencies are captured by the so-called prerequisite relation or its generalisation, the prerequisite function. The collection of all possible states is called a knowledge structure. Competence-based extensions of the original framework (Albert & Lukas, 1999) consider the latent cognitive constructs underlying observable behaviour and assume a competence structure on a set of abstract skills underlying the problems and learning objects of the domain. By associating skills to the problems and learning objects of a domain, knowledge and learning structures on the problems and, respectively, learning objects are induced. The skills, which are not directly observable, can be uncovered on the basis of a person’s observable performance. The application of CbKST in Learning Analytics has been demonstrated by Kickmeier-Rust and colleagues (e.g., Kickmeier-Rust & Albert, 2016).
  
  References:
  Albert D. & Lukas J. (1999). Knowledge spaces: Theories, empirical research, applications. Mahwah: Lawrence Erlbaum Associates.
  Falmagne, J.-C., Albert, D., Doble, C., Eppstein, D. & Hu, X. (Eds.) (2013). Knowledge spaces: Applications in education. New York: Springer.
  Falmagne, J.- C. & Doignon, J. P. (2011). Learning Spaces: Interdisciplinary Applied Mathematics. New York: Springer Verlag.
  Kickmeier-Rust, M D., & Albert, D. (2016). Theory-driven Learning Analytics and Open Learner Modelling: The Teacher’s Toolbox of Tomorrow? In M. Kravcik, O.C. Santos, J.G. Boticario, M. Bielikova (Eds.), Proceedings of the 6th International Workshop on Personalization Approaches in Learning Environments (PALE), held in conjunction with the 24th ACM International Conference on User Modeling, Adaptation, and Personalization (UMAP 2016), pp. 49-52, July 16th, 2016, vol. 1618, Halifax, Canada.
  
  Read more about the method in
  
  - Kickmeier-Rust, M D., & Albert, D. (2016). Theory-driven Learning Analytics and Open Learner Modelling: The Teacher’s Toolbox of Tomorrow? In M. Kravcik, O.C. Santos, J.G. Boticario, M. Bielikova (Eds.), Proceedings of the 6th International Workshop on Personalization Approaches in Learning Environments (PALE), held in conjunction with the 24th ACM International Conference on User Modeling, Adaptation, and Personalization (UMAP 2016), pp. 49-52, July 16th, 2016, vol. 1618, Halifax, Canada.
  
  - Nussbaumer, A., Hillemann, E. C., Gütl, C., & Albert, D. (2015). A Competence-Based Service for Supporting Self-Regulated Learning in Virtual Environments.Journal of Learning Analytics, 2(1), 101-133.

Relationship mining

• Association rule mining
  Association rule mining

  Close

  In association rule mining, the goal is to find if-then rules of the form that if some set of variable values is found, another variable will generally have a specific value. For instance, Ben- Naim and colleagues (2009) used association rule mining to find patterns of successful student performance in an engineering simulation, to make better suggestions to students having difficulty about how they can improve their performance.
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - D'Mello, S., Olney, A., & Person, N. (2010). Mining collaborative patterns in tutorial dialogues.Journal of Educational Data Mining, 2(1), 1-37.
  
  - Romero, C., Romero, J. R., Luna, J. M., & Ventura, S. (2010, June). Mining rare association rules from e-learning data. InEducational Data Mining 2010.
  
  - Bazaldua, D. L., Baker, R., & Pedro, M. O. (2014, July). Comparing expert and metric-based assessments of association rule interestingness. InEducational Data Mining 2014.
  
  - Kardan, S., & Conati, C. (2011). A Framework for Capturing Distinguishing User Interaction Behaviors in Novel Interfaces. InEDM (pp. 159-168).
• Correlation mining
  Correlation mining

  Close

  In correlation mining, the goal is to find positive or negative linear correlations between variables (using post-hoc corrections or dimensionality reduction methods when appropriate to avoid finding spurious relationships). An example can be found in Baker et al. (2009), where correlations were computed between a range of features of the design of intelligent tutoring system lessons and students’ prevalence of gaming the system (intentionally misusing educational software to proceed without learning the material), for example finding that brief problem scenarios lead to a greater proportion of gaming behavior than either rich scenarios or having no scenario at all (just equations to manipulate).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Sequential pattern mining
  Sequential pattern mining

  Close

  In sequential pattern mining, the goal is to find temporal associations between events. One successful use of this approach was work by Perera et al. (2009), to determine what path of student collaboration behaviors leads to a more successful eventual group project.
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Kinnebrew, J. S., Loretz, K. M., & Biswas, G. (2013). A contextualized, differential sequence mining method to derive students' learning behavior patterns.JEDM| Journal of Educational Data Mining, 5(1), 190-219.
  
  - Antunes, C. (2008, June). Acquiring background knowledge for intelligent tutoring systems. InEducational Data Mining 2008.
  
  - Kinnebrew, J. S., & Biswas, G. (2012). Identifying Learning Behaviors by Contextualizing Differential Sequence Mining with Action Features and Performance Evolution.International Educational Data Mining Society.
  
  - d'Aquin, M., & Jay, N. (2013, April). Interpreting data mining results with linked data for learning analytics: motivation, case study and directions. InProceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 155-164). ACM.
• Causal data mining
  Causal data mining

  Close

  In causal data mining, the goal is to find whether one event (or observed construct) was the cause of another event (or observed construct), for example to predict which factors will lead a student to do poorly in a class (Fancsali, 2012). What all of these methodologies share is the potential to find unexpected but meaningful relationships between variables; as such, they can be used for a wide range of applications, generating new hypotheses for further investigation, or identifying contexts for potential intervention by automated systems.

  
  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)

Distillation of data for human judgment

• Heat maps
  Heat maps

  Close

  For data to be useful to educators, it has to be timely. When educators have immediate access to visualizations of learner interactions or misconceptions that are reflected in students’ writing and interaction, they can incorporate those data quickly into pedagogical activity. For this reason, one methodology that is common in LA is the distillation of data for human judgment. There has been a rich history of data visualization methods, which can be leveraged
  to support both basic research and practitioners (teachers, school leaders, and others) in their decision-making. For example, visualizations of student trajectories through the school years can be used to identify common patterns among successful and unsuccessful students, or to infer which students are at-risk, sufficiently early to drive intervention (Bowers, 2010). Some of the visualization methods that have been used in education include heat maps (which incorporate much of the same information as scatterplots, but are more scalable – cf. Bowers, 2010), learning curves (which show performance over time – cf. Koedinger et al., 2010), and learnograms (which show student alternation between activities over time – cf. Hershkovitz & Nachmias, 2008).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Learning curves
  Learning curves

  Close

  For data to be useful to educators, it has to be timely. When educators have immediate access to visualizations of learner interactions or misconceptions that are reflected in students’ writing and interaction, they can incorporate those data quickly into pedagogical activity. For this reason, one methodology that is common in LA is the distillation of data for human judgment. There has been a rich history of data visualization methods, which can be leveraged
  to support both basic research and practitioners (teachers, school leaders, and others) in their decision-making. For example, visualizations of student trajectories through the school years can be used to identify common patterns among successful and unsuccessful students, or to infer which students are at-risk, sufficiently early to drive intervention (Bowers, 2010). Some of the visualization methods that have been used in education include heat maps (which incorporate much of the same information as scatterplots, but are more scalable – cf. Bowers, 2010), learning curves (which show performance over time – cf. Koedinger et al., 2010), and learnograms (which show student alternation between activities over time – cf. Hershkovitz & Nachmias, 2008).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Papoušek, J., Stanislav, V., & Pelánek, R. (2016, April). Evaluation of an adaptive practice system for learning geography facts. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 134-142). ACM.
  
  - Pelánek, R., Rihák, J., & Papoušek, J. (2016, April). Impact of data collection on interpretation and evaluation of student models. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 40-47). ACM.
  
  - Käser, T., Koedinger, K., & Gross, M. (2014, July). Different parameters-same prediction: An analysis of learning curves. InEducational Data Mining 2014.
• Learnograms
  Learnograms

  Close

  For data to be useful to educators, it has to be timely. When educators have immediate access to visualizations of learner interactions or misconceptions that are reflected in students’ writing and interaction, they can incorporate those data quickly into pedagogical activity. For this reason, one methodology that is common in LA is the distillation of data for human judgment. There has been a rich history of data visualization methods, which can be leveraged
  to support both basic research and practitioners (teachers, school leaders, and others) in their decision-making. For example, visualizations of student trajectories through the school years can be used to identify common patterns among successful and unsuccessful students, or to infer which students are at-risk, sufficiently early to drive intervention (Bowers, 2010). Some of the visualization methods that have been used in education include heat maps (which incorporate much of the same information as scatterplots, but are more scalable – cf. Bowers, 2010), learning curves (which show performance over time – cf. Koedinger et al., 2010), and learnograms (which show student alternation between activities over time – cf. Hershkovitz & Nachmias, 2008).

  
  
  

  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Open Learner Modelling (OLM)
  Open Learner Modelling (OLM)

  Close

  Learner models hold and dynamically update the information about a user’s learning: current knowledge, competencies, misconceptions, goals, affective states, etc. There is an increasing trend towards opening the learner model to the user (learner, teacher or other stakeholders), often to support reflection and encourage greater learner responsibility for their learning; as well as helping teachers to better understand their students (Bull & Kay, 2010). A core requirement is that such visualisations must be understandable to the user. On the surface this may appear similar to the more recent work on learning analytics. However, open learner models (OLM) focus much more on the current state of learners, and with less reference to activities undertaken, scores gained, materials used, contributions made, etc. OLMs typically focus more on concepts or competencies, to guide learners towards consideration of conceptual issues rather than specific activities and performance. In recent work, Bull, Ginon, and colleagues (2016), demonstrated the extension of classical OLM to so-called persuadable learner models, which allow students to intervene and negotiate the results of analytics and modelling algorithms. hold and dynamically update the information about a user’s learning: current knowledge, competencies, misconceptions, goals, affective states, etc. There is an increasing trend towards opening the learner model to the user (learner, teacher or other stakeholders), often to support reflection and encourage greater learner responsibility for their learning; as well as helping teachers to better understand their students (Bull & Kay, 2010). A core requirement is that such visualisations must be understandable to the user. On the surface this may appear similar to the more recent work on learning analytics. However, open learner models (OLM) focus much more on the current state of learners, and with less reference to activities undertaken, scores gained, materials used, contributions made, etc. OLMs typically focus more on concepts or competencies, to guide learners towards consideration of conceptual issues rather than specific activities and performance. In recent work, Bull, Ginon, and colleagues (2016), demonstrated the extension of classical OLM to so-called persuadable learner models, which allow students to intervene and negotiate the results of analytics and modelling algorithms.
  
  References:
  Bull, S., Ginon, B., Boscolo, C., & Johnson, M.D. (2016). Introduction of Learning Visualisations and Metacognitive Support in a Persuadable Open Learner Model. In Proceedings of LAK’16, April 25-29, 2016, Edinburgh, UK.
  Bull, S. & Kay, J. (2010). Open Learner Models, In R. Nkambou, J. Bordeau and R. Miziguchi (Eds.), Advances in Intelligent Tutoring Systems (pp. 318-338), Berlin-Heidelberg: Springer-Verlag.
  Ginon, B., Johnson, M D., Turker, A., Kickmeier-Rust, M. D. (2016). Helping Teachers to Help Students by using an Open Learner Model. In R. Vatrapu, M. D. Kickmeier-Rust, B. Ginon, & S. Bull (Eds.), Proceedings of the Fourth International Workshop on Teaching Analytics, in conjunction with EC-TEL 2016 (pp. 23-29). September 16, 2016, Lyon, France.
  
  Read more about the method in
  
  - Kump, B., Seifert, C., Beham, G., Lindstaedt, S. N., & Ley, T. (2012, April). Seeing what the system thinks you know: visualizing evidence in an open learner model. InProceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 153-157). ACM.
  
  - Bull, S., Ginon, B., Boscolo, C., & Johnson, M. (2016, April). Introduction of learning visualisations and metacognitive support in a persuadable open learner model. InProceedings of the sixth international conference on learning analytics & knowledge (pp. 30-39). ACM.
  
  - Vatrapu, R., Reimann, P., Bull, S., & Johnson, M. (2013, April). An eye-tracking study of notational, informational, and emotional aspects of learning analytics representations. InProceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 125-134). ACM.
• Hasse Diagrams
  Hasse Diagrams

  Close

  Learning analytics means gathering a broad range of data, bringing the various sources together, and analyzing them. However, to draw educational insights from the results of the analyses, these results must be visualized and presented to the educators and learners. This task is often accomplished by using dashboards equipped with conventional and often simple visualizations such as bar charts or traffic lights. In this paper we want to introduce a method for utilizing the strengths of directed graphs, namely Hasse diagrams, and a competence-oriented approach of structuring knowledge and learning domains. Hasse diagrams are structural graphs that hold a significant amount of information, i.e., the structure and relationships of competencies in a domain, individual learning paths, individual learning states, group-related learning states, the next logical steps, etc. The strength of this chart type – although they are not trivial to read – is discussed by Kickmeier-Rust and colleagues (e.g., 2015).
  
  References:
  Kickmeier-Rust, M. D., Steiner, C. M., & Albert, A. (2015). Uncovering Learning Processes Using Competence-based Knowledge Structuring and Hasse Diagrams. In Proceedings of LAK15, Workshop Visual Approaches to Learning Analytics. March 16-20, 2015, Poughkeepsie, NY.

Discovery with models

• A prediction model is used within another prediction model
  A prediction model is used within another prediction model

  Close

  In discovery with models (Baker & Yacef, 2009; Hershkovitz et al., in press), the results of one data mining analysis are utilized within another data mining analysis. Most commonly, a model of some construct is obtained, generally through prediction methods. This model is then applied to data in order to assess the construct the model identifies. The predictions of the model are then used as input to another data mining method. There are several ways that discovery with models can be conducted.
  
  Perhaps the most common way that discovery with models is conducted is when a prediction model is used within another prediction model. In this situation, the initial model’s predictions (which represent predicted variables in the original model) become predictor variables in the new prediction model. In this way, models can be composed of other models, or based on other models, sometimes at multiple levels. For instance, prediction models of student robust learning (cf. Baker, Gowda, & Corbett, 2011) have generally depended on models of student meta- cognitive behaviors (cf. Aleven et al., 2006), which have in turn depended on assessments of latent student knowledge (cf. Corbett & Anderson, 1995), which have in turn depended on models of domain structure (cf. Koedinger et al., 2012).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• A prediction model is used within a relationship mining analysis
  A prediction model is used within a relationship mining analysis

  Close

  A second common way that discovery with models is conducted is when a prediction model is used within a relationship mining analysis. In this type of research, the relationships between the initial model’s predictions and additional variables are studied. This enables a researcher to study the relationship between a complex latent construct (represented by the prediction model) and a wide variety of other variables. One example of this is seen in work by (Beal, Qu, & Lee, 2008), who developed a prediction model of gaming the system and correlated it to student individual differences in order to understand which students are most likely to engage in gaming behavior.
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
• Cluster analysis
  Cluster analysis

  Close

  In clustering, the goal is to find data points that naturally group together, splitting the full data set into a set of clusters. Clustering is particularly useful in cases where the most common categories within the data set are not known in advance. If a set of clusters is well-selected, each data point in a cluster will generally be more similar to the other data points in that cluster than data points in other clusters. Clusters have been used to group students (cf. Beal, Qu, & Lee, 2006) and student actions (cf. Amershi & Conati, 2009). For example, Amershi & Conati (2009) found characteristic patterns in how students use exploratory learning environments, and used this information to identify more and less effective student strategies.
  In factor analysis, a closely related method, the goal is to find variables that naturally group together, splitting the set of variables (as opposed to the data points) into a set of latent (not directly observable) factors. Factor analysis is frequently used in psychometrics for validating or determining scales. In EDM/LA, factor analysis is used for dimensionality reduction (e.g., reducing the number of variables) for a wide variety of applications. For instance, Baker et al. (2009) used factor analysis to determine which design choices are made in common by the designers of intelligent tutoring systems (for instance, tutor designers tend to use principle- based hints rather than concrete hints in tutor problems that have brief problem scenarios).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)
  
  Read more about the method in
  
  - Kerr, D., & Chung, G. K. (2012). Identifying key features of student performance in educational video games and simulations through cluster analysis.JEDM| Journal of Educational Data Mining, 4(1), 144-182.
  
  - Azarnoush, B., Bekki, J. M., Runger, G. C., Bernstein, B. L., & Atkinson, R. K. (2013). Toward a framework for learner segmentation.Journal of Educational Data Mining, 5(2), 102-126.
  
  - Bos, N., & Brand-Gruwel, S. (2016, April). Student differences in regulation strategies and their use of learning resources: implications for educational design. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 344-353). ACM.
• Knowledge engineering
  Knowledge engineering

  Close

  It is worth noting that the models used in discovery with models do not have to be obtained through prediction methods. These models can also be obtained through other approaches such as cluster analysis or knowledge engineering (Feigenbaum & McCorduck, 1983; Studer, Benjamins, & Fensel, 1998), where a human being rationally develops a model rather than using data mining to produce a model. The merits of knowledge engineering versus data mining for this type of analysis are out of scope for this chapter; greater discussion of this issue can be found in (Hershkovitz et al., in press).
  
  Reference : Baker, R., Siemens, G.: Educational data mining and learning analytics. In: Sawyer, R.K. (ed.) The Cambridge Handbook of the Learning Sciences, 2nd edn, pp. 253–274. Cambridge University Press, Cambridge (2015)

Other methods

• Descriptive statistics
  Descriptive statistics

  Close

  Descriptive statistics are used to describe the basic features of the data in a study. They provide simple summaries about the sample and the measures. Together with simple graphics analysis, they form the basis of virtually every quantitative analysis of data.

  
  
  

  Reference : http://www.socialresearchmethods.net/kb/statdesc.php
• Visualization
  Visualization

  Close

  Data visualization is the presentation of data in a pictorial or graphical format. It enables decision makers to see analytics presented visually, so they can grasp difficult concepts or identify new patterns. With interactive visualization, you can take the concept a step further by using technology to drill down into charts and graphs for more detail, interactively changing what data you see and how it’s processed.
  
  Reference : https://www.sas.com/en_us/insights/big-data/data-visualization.html
  
  Read more about the method in
  
  - Beheshitha, S. S., Hatala, M., Gašević, D., & Joksimović, S. (2016, April). The role of achievement goal orientations when studying effect of learning analytics visualizations. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 54-63). ACM.
  
  - Martinez-Maldonado, R., Pardo, A., Mirriahi, N., Yacef, K., Kay, J., & Clayphan, A. (2016). Latux: an iterative workflow for designing, validating and deploying learning analytics visualisations.Journal of Learning Analytics, 2(3), 9-39.
  
  - Svihla, V., Wester, M. J., & Linn, M. C. (2015). Distributed revisiting: An analytic for retention of coherent science learning.Journal of Learning Analytics, 2(2), 75-101.
  
  - Ruiz, S., Charleer, S., Urretavizcaya, M., Klerkx, J., Fernández-Castro, I., & Duval, E. (2016, April). Supporting learning by considering emotions: tracking and visualization a case study. InProceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 254-263). ACM.
  
   
• Natural language processing
  Natural language processing

  Close

  Natural language processing (NLP) is a field of computer science, artificial intelligence and computational linguistics concerned with the interactions between computers and human (natural) languages, and, in particular, concerned with programming computers to fruitfully process large natural language corpora. Challenges in natural language processing frequently involve natural language understanding, natural language generation (frequently from formal, machine-readable logical forms), connecting language and machine perception, dialog systems, or some combination thereof.
  
  Reference : https://en.wikipedia.org/wiki/Natural_language_processing
  
  Read more about the method in
  
  - Snow, E. L., Allen, L. K., Jacovina, M. E., Crossley, S. A., Perret, C. A., & McNamara, D. S. (2015). Keys to detecting writing flexibility over time: entropy and natural language processing.Journal of Learning Analytics, 2(3), 40-54.
  
  - Allen, L. K., Snow, E. L., & McNamara, D. S. (2015, March). Are you reading my mind?: modeling students' reading comprehension skills with natural language processing techniques. InProceedings of the fifth international conference on learning analytics and knowledge (pp. 246-254). ACM.
  
  - Ezen-Can, A., & Boyer, K. E. (2015). Understanding student language: An unsupervised dialogue act classification approach.Journal of Educational Data Mining, 7(1), 51-78.
  
  - Barker-Plummer, D., Cox, R., & Dale, R. (2011, January). Student translations of natural language into logic: The Grade Grinder corpus release 1.0. InProceedings of the 4th international conference on educational data mining (pp. 51-60).