29 April 2021 - reading time: 15 min
Every year, roughly 80,000 young people finish without a secondary-school qualification. In a joint project, using DPMK’s domain knowledge, Microsoft’s technology and Dmlab’s data analysis, we looked at three levels at what data can add: what happened, why it happened, and what will happen.
What can machine learning do for human learning? In a joint project, the Digital Pedagogical Methodology Centre, Microsoft and Dmlab set out to answer how data analysis can support the fight against early school leaving. The answer came together on three levels: what happened, why it happened, and what will happen.
Project data at a glance
| Area | Public education, early school leaving |
| Partners | DPMK (domain knowledge), Microsoft (technology), Dmlab (data analysis) |
| Data sources | Institutional and teacher data, national competence assessment, PISA, Kréta system |
| Technology | Azure, Power BI |
| Solution levels | Descriptive analysis and dashboard, association discovery, prediction |
| Highlighted result | 92 percent accurate prediction for institutions with hidden risk |
An early school leaver is a young person between 18 and 24 who has at most an elementary-school education. In Hungary, this rate is roughly 12 percent, which means about 80,000 young people every year.
It is worth putting that number next to another one: around 50,000 Hungarian young people earn a university degree every year, while 80,000 are left without a secondary-school qualification. The Europe 2020 strategy targets 10 percent, and while at European level there was an overall drop of 7 percentage points between 2002 and 2019, in Hungary the indicator has essentially stagnated.
So the problem is measurable, large-scale, and has a decades-long effect on the lives of those affected. Exactly the kind of question where data is interesting not in itself, but because it helps find where and when to intervene.
In the project, we did not build a single solution, but three reference solutions layered on top of one another. This division matches the way the four types of data science projects build on each other: first you need to see what is happening, then understand why, and only then does it make sense to predict.
| Level | The question | The solution | What it can be used for |
|---|---|---|---|
| Descriptive | What happened? | Integrated dataset and dashboard | Comparing institutions, spotting outliers |
| Diagnostic | Why did it happen? | Automatic association discovery | Identifying institutional characteristics that carry risk |
| Predictive | What will happen? | Prediction model | Filtering out institutions with hidden risk before intervention |
The first level, by itself, already gives surprisingly a lot, even though technologically it is the simplest. Educational data is created in several places, in several systems: at institutional, teacher and student level alike. The point of the unified dataset and dashboard is that these can be examined from a single viewpoint, so that one shared source of truth is created on which professional debate can build.
The most important observation at this level was that significant differences appear in dropout rates between similar institutions, and these differences cannot be explained by geographical proximity. This insight alone already gives direction: if two similar schools perform very differently, then there is something to learn from one of them.
At the second level, we used automatic feature discovery to look for institutional subgroups within which different associations hold than in the full sample. The method selects from more than ten thousand possible features those that really distinguish a group, so it does not test pre-existing hypotheses but finds the pattern itself.
One of the most telling results came out for a group of institutions with mixed ethnic composition and a high share of repeaters. In this group, in schools without a school secretary, the dropout rate was 57 percent; where there was one, it was 36 percent.
The difference is large, and that is precisely why it must be handled with care. Association does not mean causation: it is not the hiring of a school secretary that reduces dropout, but rather the presence of a school secretary may be a sign of organisational and administrative capacity that has another effect. Such a result is not a measure, but a clue: it shows where the expert should look further. This is the point where data analysis and domain expertise become inseparable.
At the third level, we examined how predictably dropout can be forecast. Two results emerged.
One is that the institutional averages and standard deviations of the national competence assessment can be predicted from earlier competence assessment and school statistical data.
The other, and from a practical point of view the more interesting one, concerns those institutions where high dropout would not be expected based on the competence assessment, yet it still happens. These are the schools with hidden risk, which slip through a traditional screening. With additional institutional features included, these cases can be predicted with 92 percent accuracy.
This is the point where analysis turns into decision support: instead of a general risk ranking, you get a list of institutions that no one would have paid attention to on their own.
A good example of what a similar approach can achieve at scale is Microsoft’s predictive solution implemented in the Tacoma district. There, before the programme was introduced, 55 percent of students completed secondary school successfully; after it, 83 percent.
That is not the model’s merit on its own. The number shows what happens when prediction becomes part of everyday operations, that is, when there is someone who actually acts on the signal.
One side result of the project was a mapping of Hungarian data assets. Four source groups proved usable:
The greatest value lies in the fact that these can be viewed together. Separately, each one is known; only connected do they answer the question.
The project was built on the cooperation of three players, and each brought something different. DPMK brought domain knowledge, Microsoft brought technology, the capabilities of Azure and Power BI, and Dmlab brought data analysis expertise.
Ádám Horváth, head of the DPMK division, summed it up like this: "During the joint project, professional findings emerged that would have had no chance of coming up with traditional data analysis tools."
This division is no accident. A public-sector data project delivers results when the professional question is understood by the person working in the field, and the data is analysed by someone who knows how, and the two are present not one after the other but at the same time.
Three lessons can be transferred to almost any field.
The same logic works in other industries too, as our case studies show. The more a process becomes data-driven, the greater its business and social impact.
Who counts as an early school leaver? A young person between 18 and 24 who has at most an elementary-school education. In Hungary, this rate is roughly 12 percent, which means about 80,000 young people every year.
How accurately can dropout be predicted? In the project, for those institutions where high dropout would not be expected based on the competence assessment, yet it still happens, we achieved 92 percent accurate prediction by including additional institutional features.
Does the prediction rank students or institutions? In this project, we worked at institutional level. This has both data-protection and professional reasons: at institutional level, intervention is also organisational, and no individual-level risk label is created.
Does this require new data collection? Not necessarily. The project built on existing sources: institutional and teacher data, competence assessment, PISA results and the Kréta system. A large part of the work was not obtaining the data, but connecting and interpreting it.
Can this be transferred to another public-sector field? Yes, the structure can. Layering descriptive, diagnostic and predictive levels, interpreting them together with a domain expert, and handling causality cautiously are just as relevant in healthcare, social services or employment policy.
If you have a social or business problem where you feel there is more in your data than you are currently reading out of it, it is worth starting with these three levels. The first question is always what actually happened.
The six phases of CRISP-DM are not theory, but the sequence in which a data project stays on track. We walk through a bank credit card campaign, show where the time goes, and what has changed in the deployment phase in the age of MLOps.
Explainable AI is not a model type, but a toolkit for finding out what a model is based on. An overview of ante-hoc and post-hoc methods, LIME and SHAP, and what regulations expect from explanations.
Data projects do not fail because of technology, but because of human and organisational reasons. The seven most common pitfalls, the seventh success criterion you rarely hear about, and the three steps that can reduce the risk to a fraction.
Data driven fuel and transit time planning in road freight
A complex data collection, portfolio management and trading system in the energy sector