I Build an AI Platform. Do the Risks Worry Me?

By Eric Boudrias Zotti, Computer Science Professor and Founder of Nabunam

I am seeing more and more articles discussing the dangers of artificial intelligence. They raise questions about loss of control, jobs that could disappear, and even the extinction of humanity.

Meanwhile, I am building Nabunam, a platform that uses AI to help teachers.

So the question is fairly direct: am I contributing to the development of something we should be wary of?

I think this question deserves more than a reassuring answer designed to protect my product. When we integrate AI into a tool intended for education, we should be able to discuss its risks as openly as we discuss its benefits.

Should we believe the catastrophic scenarios?

The 2026 International AI Safety Report, led by Yoshua Bengio with contributions from more than 100 experts, distinguishes between harms that have already been observed and future risks that remain uncertain. It does not predict the imminent extinction of humanity.

That does not mean there is nothing to watch. The report highlights, among other things, errors made by AI systems, their uneven capabilities, and the risks associated with placing too much trust in their answers.

As a teacher, I do not need to wait for AI to escape humanity’s control before asking difficult questions. An incorrect assessment that unfairly penalizes a student is already a very concrete problem.

What concerns me is that we may eventually stop checking

An AI response arrives quickly. It is well presented, structured and often convincing. That is useful. It is also what can lead us to accept it too quickly.

In programming, we know the difference between code that works once and code we understand well enough to trust. Does it handle errors? Does it work with different data? Is it actually doing what we asked it to do?

I think we need to keep the same reflex when dealing with AI-generated content.

An exercise can be well written but poorly suited to the level of a class. An answer can be correct while using a method that has not yet been taught. An AI-assisted assessment can identify an error while missing the reasoning behind a student’s work.

If teachers are expected to remain responsible for the final result, they need tools that make it easy to review and verify what AI proposes. To me, that is part of the development work itself.

And what about students?

As a computer science professor, I find it difficult to imagine an education that simply ignores these tools. They exist, and our students will have to make decisions about how to use them.

But I also do not want us to confuse producing work with learning.

If a student submits an application that works but cannot explain how data moves between two components, what can I really conclude about their understanding?

This is where the way we assess students becomes important. Ask them to explain a decision. Change a requirement and see whether they can adapt their solution. Ask them to identify an error in AI-generated code.

These are approaches I consider useful as a teacher. They require work, and I do not pretend there is a simple formula that solves everything.

Nabunam also has to address these questions

It would be easy for me to end this article by saying that Nabunam is the solution. It would also be a little too convenient.

Using AI in an educational platform does not make its limitations disappear. It means we have to account for those limitations in the way we build the product.

The direction I want to give Nabunam is fairly simple: help teachers move faster when preparing their courses and assessing student work, while allowing them to understand, modify and reject what the AI proposes.

I want the time saved to be spent supporting students. If we end up spending just as much time fixing results that are difficult to verify, then we have missed the point.

I remain convinced that AI can bring a great deal to education. That is precisely why I think we need to be able to discuss it openly, including the questions that are less comfortable for those of us building these tools.

I wear both hats. As a developer, I want to see how far we can take this technology. As a professor, I want to be able to explain why what we build genuinely helps students.

For me, Nabunam has to live up to both.