AI in Education Unplugged.

 

This section is a conversation between Seiji Isotani, Professor of Computer Science and Learning Technology at the University of Sao Paulo (Brazil), and the OECD Secretariat. It focuses on remedying digital divides and focuses on AI in Education Unplugged, that is, the design of AI tools and AI interventions for places where digital infrastructure, connectivity and skills are limited. The section starts by presenting the idea and applications of AI Unplugged before zeroing in on the current and promising uses of generative AI systems in this context, notably with small language models. Isotani is now Faculty Director of the Learning Analytics and Artificial Intelligence Program at the University of Pennsylvania (United States). 

OECD: Where does the concept of AI Unplugged come from? 

Seiji Isotani: When we look at the AI space, people have been working on how to improve education with AI for decades. The AI in Education (AIED) society was created in 1997, almost 30 years ago. In the beginning, researchers tried to mimic what a teacher does, for example by trying to understand pedagogical strategies and their impact on learning. Others started working on “user” or student modelling, for example trying to predict how much students know based on their potential behaviour. Then we moved more towards what we call open learning models, which not only assess how students are doing but also present that information to students, so they are informed about what computers believe they know about them. It's more of a cognitive and metacognitive process. The consistent problem in AIED has always been the expectation that students and schools, in general, have a minimum infrastructure to run AI. And as AI grows, the standard for this minimum requirement gets higher, right? This has become a problem, which is why people have discussed the digital divide for so many years. For me, the question is: How can I bring the benefits of AI to people and regions where infrastructure is almost non-existent? This completely shifts the research question and how you approach AI. Instead of creating the most innovative AI technology, you actually work with the community and ask, “okay, what does this community need, and how can I bring AI to them?” This is super interesting because by understanding users, we can create different technologies for that particular space. In 2022, about 90% of the world population, including those in low-income countries, had access to mobile phone subscriptions or mobile phones. If you go to poor places in Brazil, in countries near the Amazon, in Africa, or in parts of India, you will always see a mobile phone. Then we looked at the statistics on Internet access, using data from 2022 and going back 30 years, so we could project trends. What we realised is that only 15% of people in low-income communities have access to stable Internet. This completely changes how we think about AI design, because we can consider using mobile phones and other low-cost equipment, but we cannot assume that the Internet will be available everywhere. Even with the ambitious ongoing efforts to bring Internet access to everyone, the data give you the feeling that it's going to take about 100 years for lowincome communities to have the same Internet access as higher income communities. And this considers the trends of things evolving and getting better. 

OECD: So, what is the minimum infrastructure needed for AI Unplugged? 

Seiji Isotani: We are developing a framework to implement AIED considering different degrees of infrastructure. To date, what we realised is that the minimum access required is a mobile phone (and not even a smartphone). That's the only thing we need. And some access to the Internet. It doesn't need to be all the time, every day, or every hour. At some point during the week you may have access to the Internet. If you have these two things – limited access to the Internet and a mobile phone, just a good enough mobile phone – then we can use AIED Unplugged. If you have Wi-Fi access once every week, I think it's enough, because it's sufficient for us to update information from the local equipment to a server. This allows us to do more intensive processing, analyse the data, update anything that needs updating, and then return information about the students. 

OECD: What are the things that AI Unplugged allows you to do? Could you give us an example? 

Seiji Isotani: Sure. Right after the COVID-19 pandemic, the Brazilian government asked us to help improve the writing skills of students in 5th to 9th grade. In Brazil, pupils spent almost two years out of school without writing, so when they returned they couldn't write well. The government asked us to try to do something about this at scale in Brazil, considering all the inequalities in the country. We accepted the challenge. So, we performed a data analysis on mobile phone access, Internet access, and so on. In Brazil, most schools (over 90%) have some Internet access, but mainly for administrative tasks available at the principal's office, leaving students and teachers with little to no access. In this context, at least one location in most schools in Brazil has Internet access. This is an interesting and important feature. We created an application enabling teachers to take pictures of students' essays. They would ask students to write essays on a sheet of paper, and then take a photo of those essays (see Figure 6.1). Whenever the Internet was available – it could be the next day or two days later, or whenever possible, usually during lunchtime – a teacher would go to the principal's office and leave their mobile phone there. Our application would then upload all the photos to our server, perform the intensive processing, and return all the analysis of these essays. It would then provide the teacher with a dashboard for a particular student, for the whole class, or for a group of classes. The goal was to provide analysis and recommendations to empower teachers to improve how they support students in their writing. We analysed the outcomes with about half a million students in Brazil across 1 500 different municipalities over a year, and we found benefits from doing this. So, this type of AI technology that requires a minimal physical infrastructure offers significant support for teachers and helps students write essays through its automatic evaluation. We are now doing the same thing for basic mathematics and other subjects as well. 

OECD: For that purpose, you didn't have to use generative AI, right? 

Seiji Isotani: Yes, in that case we didn't use generativeAI. We are now trying to use generative AI and compare it with the traditional methods we used. Right now, our previous AI model is still better at detecting incorrect words because what happens with generative AI is that it corrects the student's mistake when it processes a photo, thus correcting something that shouldn't be corrected in this context. For us to evaluate students, we need to know exactly what the student wrote, including their mistakes! But in one or two years, we will probably be in a position where generative AI will substitute all this previous work we've done.

 OECD: How do you ensure that the recommendations given to teachers are pedagogically sound? 

Seiji Isotani: That's a very good question. We have a library of the best pedagogical strategies that the field of the learning sciences provides to us. We try to match students' challenges or difficulties with these materials and strategies. For example, one of the materials we use is "WordGen." "WordGen" is a set of materials created by colleagues from Harvard University and other institutions. They work not only to support reading and writing but also to support reflection. Their general idea is that to support reading and writing, you need to engage students in interesting interactions. Just to give an example, imagine you are a student and you need to write and defend a position. You could talk about climate change, or about economic challenges. But students who are just starting to learn are usually more interested in local challenges. For example, in "WordGen," they have one problem related to lunch in the school cafeteria: should we have pizzas or salads for students? It’s much more interesting to discuss this topic. Or, should we have mobile phones in school or not? It’s interesting because you can also provide data about this: How good is a pizza? How good is a salad? What benefits do you get from each? Then you can position yourself in one of these directions, and during the debate, you bring all those ideas you collected from your reading and start to debate. Then you become much more critical when you are discussing. So, the idea of producing critical thinking or improving critical thinking becomes relevant in this case. In this context, the role of AI could be to match students’ difficulties with potential pedagogical strategies and materials that can be used by teachers. The AI might assess the class and say "Okay, so students are not well-versed in semantic analysis, so we need to improve their capacity to understand more complex sentences, to retain their meaning." Based on that, it can recommend “WordGen” together with pedagogical strategies for reflection and peer reading that we know from the learning sciences are proven to work for that specific need.

OECD: Thank you. We have a good example here of how to use unplugged AI when the AI tool is teacher-facing. Do you have examples of AI applications when it is used directly by the students and not by the teacher? 

Seiji Isotani: Right now, I don't think we have a good example of a student using it directly. Because one of the foundations of AIED Unplugged is trying to reduce the amount of equipment needed. In low-income communities, most students do not have devices – so a scenario where they would all have one is not really helpful. We always think in this context that the proxy can be a teacher, a parent, or a mentor. We empower this proxy user so the students receive the greatest benefit. This is one of the challenges of using AIED Unplugged, because students who benefit from it are vulnerable. Current AI still has some biases. We can avoid them to some extent by having a proxy who tries to understand what makes sense and what doesn't to support that particular student. So, we work with an intermediary that prevents students from being affected by additional biases, that's the idea.

OECD: Let's move now to the small generative AI models. How could they help? They are interesting because, of course, they can be used to support teachers. They can also directly support students, even if it's through their parents or siblings, or whoever has a device. They could help students to develop their AI literacy in contexts with little resources. What are the possibilities for actually using generative AI in the unplugged model? 

Seiji Isotani: Yes, that's a key question that we are discussing in our group. I think the first point is breaking another barrier to access AI and technology in general. Even with mobile phones, we still require the user to have at least a minimum knowledge to use a mobile phone. When we use these AIED Unplugged models, they could run on a mobile phone or any other device. The interaction interface can be voice, which completely changes the game. Users are not just clicking; they are talking. And by talking, we can have a communication and interaction that students and teachers are already knowledgeable about. Then, whenever you have a more knowledgeable partner and you want to ask and learn from them, you start asking questions and analysing their responses and trying to use them in your everyday life. So, when I think of GenAI in the AIED Unplugged model, it's like a companion that helps you solve challenging problems. It's not meant to substitute for anything, but it will enhance your capability to do things better. When you go to schools in the Amazon in Brazil, or in remote areas in many countries, you see that schools often don't have distinct grades. Five-year-old, six-year-old, ten-year-old students are all together. But they need different kinds of support, and the teacher isn't knowledgeable about everything and also need support. In this scenario, GenAIED Unplugged can actually provide specific or tailored support for students when they need it, at their level. So, that's something we are trying to produce right now. Our work is exploring whether we can ask something and receive a response in a good enough amount of time so that people with no connectivity can have a fruitful conversation. If it is possible, then the next question is, “are those interactions adequate or correct, or do they help students learn something?” These are the next steps that needs to be addressed.

OECD: How does it work? If I put a Small Language Model (SLM) on my phone, does that mean that when I'm not connected to the Internet, I can still interact and get responses from the chatbot and everything?

 Seiji Isotani: Yes. You would just have a small or mini version of any LLM on your mobile phone (ChatGPT, Llama, DeepSeek, Mistral etc.) and it would work offline. It's a smaller version of an LLM, which means that hallucination is more problematic and it is not as powerful. Responses are sometimes incomprehensible. Words can be invented, so there are several problems to use it for learning right now. But on the good side, we do have a mini version of the web in our hands, so we can ask questions, get answers, and get help in different ways. So, the interaction process doesn't change much compared to an LLM: their capability to respond in different ways is limited, but they are still capable of doing things. One work that we are yet to publish, but are finalising, explores how many different pedagogical strategies can actually be used by LLMs in both an online and an offline environment (SLMs). In an online environment, LLMs are much more capable of using different strategies. If you ask them, "use the Socratic method to teach me something," they will do it, that is, ask questions and not give you the answer. On the contrary, if you are using a SLM on a mobile phone and you ask "use the Socratic method," probably - and we have observed this - in almost all cases, they will just give you the response. They won't use the strategies you want. So, these are some examples of the limitations.

OECD: Does it make a difference for a small model to be offline or online? Would the small model online be more performant than the small model offline, or would it be more or less the same?

 Seiji Isotani: In our case, it doesn't change anything if it's online or offline, because the only thing we are doing is analysing interactions. If you want to create models that update over time or can search for current information on the Web according to student interactions, then an online model can make a huge difference. But if you are just thinking about the interaction itself, then offline is fine

OECD: You have studied the trade-offs between different types of small language models. What are the lessons from your work on that? 

Seiji Isotani: The lessons learned are that every community has different challenges, and we realised that AIED Unplugged can potentially address about 70% or 80% of these challenges. We are always thinking about these large language models, but perhaps for educational purposes, we should consider more seriously small language models. Small language models seem to be more effective for different purposes such as handling specialised, domainspecific tasks, like offering short feedback on a decimal misconception; they do not need to be online and they are not as high-cost as large language models. In addition, AI agents in the space of small language models also seem to be a promising path forward that few people talk about. People are discussing these huge agents that can do several different things and produce a final result. But when we think about AIED Unplugged, perhaps we can think of agents running in small language models to complete small, specialised tasks for teachers, for example, lesson planning, or creating specific materials for a particular activity for a student of a certain age. These are small agents that you can actually create to help teachers produce better quality materials and support them in their activities. They don’t need to be able to do everything. For students, agents are really interesting as well. Think about students in high-income families and what they have access to: if they struggle, their family might hire a private tutor so they can improve. Students with mental health issues will have access to a psychologist or someone working on their well-being. If uncertain about their career, a career adviser will help them, right? Three different things. Students in low-income families don't have access to this, but with small language models, using a mobile phone, you can actually run three different specialised agents: one focusing on the student's cognitive capability, another on the student's well-being, and another on the student's career. Then AI agents and students’ caregivers can collaborate with students to make the best decision about what they should do next. So, I think this is a very promising path forward because, in many cases, those students lack any kind of support right now.

OECD: Large Gen AI models are increasingly getting trained for specific purposes, a bit the way you were describing. Is it possible to do something like that with a small model? Could you train a small model to be more focused on academic or on social and emotional skills, or to really have some kind of specialty? And can it be combined with other types of "good old-fashioned AI"?

 Seiji Isotani: Yes, it's possible to tailor a small language model to specific tasks and activities for education. For example, we can use RAG (Retrieval-Augmented Generation) and train those models on data just from Wikipedia for example, or any other source. Their responses would then be heavily based on Wikipedia information. This is completely feasible. We can also do it with other different sets of materials. We can think about books, materials from OECD, or several other different resources. So, these students can actually have agents that will help them in different domains. We could use ontologies and knowledge representation (i.e. symbolic AI) to create hybrid approaches that can potentially have better results without relying solely on a single AI technology. 

OECD: How much time does it usually take for a small language model to respond to a question? And does it make a difference if I type it or if I speak?

 Seiji Isotani: Yes, typing or speaking makes a difference because the model needs to transcribe your oral input from what you're saying and then generate the response. When you type, it's faster. In January 2025, the last time we experimented with voice, it took more than a minute to get a response. So, if you ask a complex question, it can take much more time. But I believe that with new models and optimisations we will probably get better results over time

OECD: One last question. Here we're talking about language models. Generative AI is more than just language models. It can generate pictures, music, videos, and so on. How much of that can a small model do? 

Seiji Isotani: That's a big limitation right now. With the capabilities we have, small language models are not enough to generate images, videos, and so on. You need a lot of processing power and energy to do that. And even now, with the current large GenAI models, it may take several minutes to generate a good quality image. So, right now, small GenAI models don’t have the capability to generate these high-quality materials. But my bet is that this is just a matter of time. Every two years, the power required to generate an image is reducing. OpenAI showed that in the beginning, responding to any prompt would cost USD 5, and now it's a matter of a pence - one cent of a dollar. So, I think things will continue to improve, and these smaller language models will be able to do something like generating images and performing more task