Adrian Mladenić Grobelnik: Can Large Language Models Reason?

On Thursday, April 24, ASEF, in collaboration with the Student Council of the Faculty of Computer and Information Science, hosted a public lecture as part of the ASEF Young Minds series. The featured speaker was Adrian Mladenić Grobelnik, a young researcher at the Jožef Stefan Institute and a 2024 ASEF Fellow.

The event took place at the Faculty of Computer and Information Science in Ljubljana, bringing together students and professors. The evening began with an introduction to ASEF’s scholarship programs and mission, followed by Adrian’s presentation.

Adrian explored the capabilities and limitations of large language models (LLMs), such as ChatGPT, focusing on how they process and interpret information. He traced the development of these models from earlier versions like GPT-2 to more advanced ones, including DeepSeek, and explained the reasons behind their inconsistent performance in real-world applications despite strong results in benchmark tests.

He highlighted common issues like pattern overfitting, difficulty with generalization, and challenges in evaluation. Using concrete examples, Adrian demonstrated how slight variations in data can lead to vastly different outputs, emphasizing the importance of consistent training data and computational resources.

Toward the end of the lecture, he posed a thoughtful reflection:

“Whether large language models can reason depends on how we define reasoning. But what’s clear is that these models act like composers of patterns – they take patterns, combine them meaningfully, and produce impressive results. Still, they lack the depth of general understanding that humans naturally possess.”

Adrian also shared insights from his ASEF research visit to Stanford University, where he worked under the mentorship of Prof. Jure Leskovec, deepening his work on AI reasoning and natural language processing.

Adrian emphasized that while companies may have more resources, the academic world brings creativity and freedom to explore bold ideas: “They have talent and computing power – but we can afford to experiment more wisely.”

ASEF extends sincere thanks to all attendees and partners for making the event a success. The evening offered an inspiring glimpse into the challenges and opportunities of AI through the eyes of a young researcher shaping the future.

This project is supported by the Office of the Republic of Slovenia for Youth and the Government Office for Slovenians Abroad.