This paper, by Ido Roll & Ruth Wylie reviews 47 articles on AI in Education from ‘three years in the history of the Journal of AIED (1994, 2004, and 2014)’.
Acknowledging the evolution of AIED over the past 25 years they seek to discover what the strength are and what the future holds.
Evolution in AIED Research
Their aim for the review is twofold: ‘One is an evolutionary process, focusing on current classroom practices, collaborating with teachers, and diversifying technologies and domains. The other is a revolutionary process where we argue for embedding our technologies within students’ everyday lives, supporting their cultures, practices, goals, and communities’.
These objectives are interesting as the Council of Europe’s 2019 report on ‘Artificial Intelligence and Education’ also calls for a collaboration of all stakeholders to ensure safety, They differ from the Council of Europe, however, as the Council do not want AI embedded this deeply into the lives of learners.
What is interesting here is that the writers are alluding to some form of personalisation which would adapt to the diversity of its users, a capability, the Council of Europe says designers of AIED have failed to accomplish.
The article suggests the AIED community have not been innovative over the past 25 years, choosing rather to replicate traditional methods of pedagogy, a claim also levelled against AIED designers by the Council of Europe’s report.
The writers feel interactive learning environments (ILE) have shown some positive results which has lulled practitioners into a state of contentment rather than innovation.
Learning theories of the 21st century are more geared towards connectivism and social constructivism and impact or transformation and this is not reflected in AIED. There is further requirement for more personalisation which the Council of Europe also stated is absent from AIED.
The writers however see this missing capability as an opportunity for AIED. They ask what actions need to be taken to make AIED more adaptive by looking at the focus of AIED research and the changes required to achieve this goal of more adaptive learning in AIED.
To do this they take a historical look at 20 years of AIED research published in the International Journal of Artificial Intelligence in Education, (IJAIED) with a focus on papers written in 1994, 2004 and 2014.
They analyse the accomplishments in the field in the early, middle and recent years looking at general publications and special interests.
Type and Focus of Paper
They analysed the papers within the following parameters: ‘type and focus of paper, domain and breath, interaction type and collaborative structure, technology used, learning setting, and learning goals’.
By identifying if papers were empirical in terms of the form of data collected, they identified that research papers have become more prone to intellectual rigor: ‘Only 1 paper from 1994 (out of 20, 5 %) had some form of empirical data. In contrast, 8 papers from 2004 had empirical data (out of 13, 62 %), and 10 (out of 14, 71 %) from 2014 had such data’.
They categorised each of the papers according to their focus whether modelling approach of learner or domain, ‘research methodology, literature review, system description, system evaluation, or learning theories’. They found that modelling approach tended to take centre stage as approach across the research period and critiqued the lack of more innovative methods of research in the area though they felt it was good to see papers that contributed to the ‘theoretical implications and contributions of their work’.
Domain and Breadth
They found the target domain of most of the papers were around STEM. They felt this was because a FOCUS on stem attracted more attention, funding and opportunities. This was also because they felt STEM researched seemed to use more empirical methodology and thus more easily measured.
Interaction Style and Collaborative Structure
‘We analysed activity type by two dimensions: interaction style and collaborative structure’ as experienced by students. These were activities involving a single problem which required immediate feedback, complex problems which include multiple skills and phases and may require alternative methods of analysis and or response and finally self-exploratory environments and games.
They found research seemed to be focused around one step based systems—the didactive form of teaching based on cognitive and behavioural theories of learning.
They further looked at collaboration in 4 areas: ‘1 learner: 1 computer are systems in which individual learners each use their own computer, and there is no designed interaction between learners (however there may be collaboration with virtual agents); n learners: 1 computer refers to systems in which a group of learners, often dyads, work together with a single machine; n learners: n computers, synchronous, describes students who collaborate in real time using different machines, and engage with a joint problem; n learners: n computers, asynchronous, refers to systems in which learners interact asynchronously with the same environment. Discussion forums are a typical example’.(‘n’ being the number)
They found that between 1994 and 2004, few papers offered discussions on collaboration whilst this increased by 2014 which also translated into an increase in the classroom. They were excited about this development as they believe the ILE environment needs to be more collaborative for students.
Technology and setting:
They also looked at the technology being used such as ‘computers, handhelds, robots, or wearables), and there intended setting (school, workplace, or informal)’. They found most people used computers and these were present in both work and school settings.
They suggest this limited coverage excluded other technologies such as ‘smartphones and tablets, wearables and robotics’ which were cheaper and becoming ‘more ubiquitous and offered more opportunities for interaction.
Learning Goals
They found the literature indicated a shift from focusing on product to process, beyond cognitive learning to more collaborative systems of learning. Most literature used surveys to measure motivation and few looked at self-efficacy in a substantial way. They felt most research used surveys and there was a need to look beyond the ILE which were supported environments to environments which preferred self-regulated learning outside of the tutored environments.
Other Dimensions
They found an absence of literature on the participation or involvement of the tutor as an involved collaborator or from outside the environment.
They also found little literature on what the learners did in addition to working with AIED.
Linguistic Analysis
The literature review across the three years also looked at the language used in the articles. They found the use of ‘student’, and ‘system;’ were the most use as they were the focus.
The analysis also supported the shift from knowledge as a product to learning as a process with the word ‘knowledge’ being replaced by ‘learning’ in 2004 and 2014.
They found the field shifting as well in 2004 and 2014 with the inclusion of more stakeholders in the literature. They specifically mention ‘teacher’ which was missing in writings in 1994, and ‘web’ which was missing in 2004 although it resurfaces in some 2014 writing.
Words like ‘theory’ change to ‘empirical analyses and words like ‘model’ disappear by 2014.
Overall, they find the literature indicated that AIED focused mainly on STEM subjects, tended to be cognitive learning focused, and were computer and classroom based. The literature had also increased in rigor to becoming more empirical and data based. They however feel the literature needed to be more diverse in topics addressed beyond STEM, work in other settings other than the classroom and include other technologies.
Shifting Characteristics and Priorities in Education
They consider the evolution of AIED in the following areas: ‘goals, practices, and environment’.
Goals
The writers discuss the change in focus of education as becoming less linear and focused on cognitive learning and assessments to becoming more adaptive, dynamic and based on application, self-regulation and collaboration especially as technologies become more ubiquitous and information more readily available.
Practices
The article suggests classroom practices are changing from individual cognitive focused learning to more experiential, collaborative and problem-solving practices.
They critique the lack of personalised learning paths within the curriculum which take cognisance of the diversity of learners’ backgrounds and experiences. This is a problem identified by the Council of Europe’s’ report in 2019 as inadequate in AIED design by its failure to provide individualised learning pathways for learners.
Environment
The article discusses how the schooling system whilst still in place, education has now shifted outside the classroom to many ‘Massive Online Open Courses (MOOCs)’ which accommodate more life long and independent learning as well as global studentship and post degree education with various accreditations.
Changes Are not Limited to Informal Learning
They suggest that the changes require teachers to be more guides ‘on the side’ who support independent thinking and searching by learners. This relationship in the classroom is not effectively translated into MOOCs who are often talking heads, and they ask how this ca be improved in future MOOCs.
Time for a Revolution
The writers argue for a revolution in AIED which would be a continuum of development in the sector:
Embedded in Context
The writers felt there was an absence of ILE environments which included a consideration of the context within which they were being used. They were usually ‘plug and play’ technology. They call for a Cognitive Tutor ecosystem which offers this overarching perspective by introducing the technology together with a curriculum (Koedinger & Corbett, 2006) and call for further research which looks at the environment within which the learning takes place prior to the design process. Interestingly the Council of Europe also advocates this.
Their literature review found no instances where teachers were involved in the design of AIED tools. They suggest this should not be the case and that it is imperative to include teachers as collaborators and participants in the development and design of the tools and also conduct research on the extent to which AIED was changing ‘pedagogy and teaching practices, impact professional development and teacher training, and what aspects of current practice are being shortened or eliminated to make room for technology’.
The writers found that AIED tools paid little attention to cultural differences especially globally. They found of the 47 papers reviewed ‘43 come from North America, Europe, and Oceania. Only four papers have authors from other regions: three from East Asia and one from South America’ and none from Africa or south Asia’. They have called for more writings which is inclusive of diverse demographics as ‘Education is a socio-cultural phenomena (Vygotsky, 2012)’. This is also a concern named 'AI Colonialism' by the Council of Europe in its 2019 report.
The writers state that of the 47 articles only one discussed the broadening of context of AIED outside the classroom.
They feel AIED must look more at being applicable in workplace setting and other environments where people are and in the manner which suits them such as parks and kitchens.
Diverse Technologies
They felt the publications, with one exception, focused solely on computers. They encourage more novel and diverse technologies especially around mobile phones with diverse software which encourage more engagement.
Addressing Big Problems
Interestingly they suggest that data suggests that constructivist learning are not as effective as students tend to require more support. This is in contradiction to the advocacy for more independent learning. They feel that this tension offers more opportunity for AIED tools to provide personalised tools through ‘educational data mining and modelling of learners, pedagogies, and domains’. This is interesting as it is in direct contrast to the Council of Europe’s position which calls for more controls on data mining and modelling stating that it could infringe human rights, data laws and privacy.
The writers, however, feel this will enable designers to ‘make a substantial impact on students’ educational experiences’.
Using previously Invented Wheels
The writers argue that AIED should do less of reinventing the wheel and use what already exists by building the ILE and populating it with content which already exists such as MOOCs, Khan Academy and Wikipedia. Could this not be contradictory to what they criticised earlier where they feel designers are not innovative and tend to be focused on cognitive and behavioural systems of learning?.
They argue that there should be more collaboration with other communities such as the ‘Learning Sciences community’ which would allow expansion in ‘related fields’.
They applaud other ways of combining AIED such as placing learning content on social sites like Facebook.
Concluding Remarks
Their conclusion is better expressed in their own words: ‘AIED, as a community, should continue this work and play to our strengths and successes. While doing so, we would like to encourage researchers to be bolder, take greater risks, and tackle new contexts and domains. We specifically argue that ILEs should be better integrated – with formal and informal learning environments, with teachers and their practices, with cultural norms, with existing resources, and with our learners’ everyday lives and tasks’
They go on to add they believe the human tutor may be coming to the end of its days as AIED means learning supersedes borders, times, and number of learners. They feel these capabilities should be exploited to further augment, grow and solidify AIED.
They say they are not calling for the total disappearance of the teacher but rather a new kind of teacher that comes alongside the student as a mentor supporting life-long skills development and not just ‘domain knowledge’.
Reading this document 10 years later gives the value of hindsight on the literature they reviewed and their own ideas at the time of writing.
In hindsight it can be agreed the hardware involved in learning is not just ubiquitous but varied. Online learning is now a norm rather than an exception up to post graduate level, with lifelong learning platforms and a global phenomenon as they predicted.
Their call for the personalisation of learning is still being worked out as the Council of Europe in 2019 refers to the failure of AIED to provide this.
So far AIED has not been able to replace the teacher in the classroom or to move from cognitive systems of learning with assessments.
In many ways, AIED has reinvented the wheel when it comes to learning across all sectors in not just what can be learned or assessed online but in how it can be accessed.
The issue relating to providing cultural context for AIED is still unresolved as identified by the Council of Europe’s report.
In general, there is good coverage of articles from the past 30 years which provides an understanding of how AIED was perceived, a glimpse of how those ideas were evolving and had evolved by the time the review was written and the ideas and thoughts about the future of AIED. A very useful article.
Ido Roll1 & Ruth Wylie, Evolution and Revolution in Artificial Intelligence in Education Published online: 22 February 2016, International Artificial Intelligence in Education Society 2016
Comments