Ask any teacher what makes a good exam question and they will tell you it should test understanding, not memory. Gayathri R. has spent four years trying to teach that same lesson to a machine and in the process discovered that even the most powerful AI systems in the world do not truly understand numbers.
A PhD fellow at IIT Palakkad’s Mehta Family School of Data Science and Artificial Intelligence, she is working on numerical understanding in large language models, a consequential blind spot in modern AI.
A Village Girl Who Rewrote Her Story
Gayathri grew up in Sreekrishnapuram, a rural pocket of Palakkad in Kerala, in a home where both parents taught at the local elementary school. Teaching, she always assumed, was where she would end up. Engineering and IIT were never part of the picture.
It was her brother, four years older and already an engineer, who changed the equation. “You have the potential to be an engineer. Why don’t you try?” And that was the start of an academic journey that hasn’t yet ended.
She topped her BTech in Computer Science at Government Engineering College, Thrissur. When a campus placement at Oracle Financial Services took her to Bangalore, she became the first unmarried woman in her family to leave Kerala for work. From Oracle, she went back to study, completing an MTech at NIT Tiruchirappalli, and from there to IIT Palakkad. “Each step opened a new door and a new world for me,” she shares.
The Number Problem: Where AI Falls Short
The research idea arrived the way the best ones do, from something personal. Gayathri had always wanted to teach. Thinking about what that might look like at scale, she asked: why not automate the questions used to test comprehension? Feed a text to a model, let it generate questions and match the answers.
The gap she found was not in the idea but in the machine. When she asked an LLM to generate numerical questions on a case study about deals signed by pop stars, it got confused. It could answer “what was the amount signed by X?” but stumbled on “who signed a deal worth more than twenty million?” It had learned that fifty million appears in certain financial contexts. But it had no grasp of the relationship between fifty and twenty.
“To an LLM, the number fifty million and the word ‘house’ are not fundamentally different things. Both are tokens,” she explains. It learns patterns but not values. “When I’m in a meeting and you call and I say call me after some time, you’ll call in an hour or two. When I’m on vacation and I say call me later, you’ll wait a week. An LLM doesn’t know that difference.”
When Fine-Tuning Wasn’t Enough
When Gayathri started her research in 2022, there was barely any literature on numerical representation in LLMs. She would have to build the dataset herself, from zero.
She tried everything short of touching the architecture – different datasets, adjusted loss functions, classification-based training. Each approach moved the needle slightly but not enough. The conclusion became unavoidable: the limitation was not in the data but in the transformer’s foundations. Standard models process numbers as discrete tokens, stripped of mathematical relationship. The number 1,000,000 shares no inherent proximity to 999,999 inside these systems.
“If we don’t change the building block, the attention mechanism at the core of the model,” she realised, “there is no progress forward.”
The Support System
For a while she sat with that and went nowhere. What pulled her through was her supervisor Dr. Konnikka Pal, whose guidance came as possibilities rather than prescriptions. “Sleep on it. Read something in computer vision. Come back.” After every meeting her, Gayathri says, “you always feel like there is some scope somewhere.”
That nudge turned out to be the right direction. IIT Palakkad’s small, interdisciplinary campus means researchers across engineering, humanities, and data science share common space. It was a conversation with a peer from mechanical engineering, not NLP, that finally unlocked what she had been circling for months. She doesn’t remember exactly what he said but that it clicked. That night, she slept peacefully for the first time in weeks.
She built her dataset from scratch, developed a quantity-focused question generation and answering model, and published at ECIR, the European Conference on Information Retrieval, in Lucca, Italy. It was her first paper. It was also her first time abroad, travelling alone.
A Foundation For Others To Build Upon
Her current work goes deeper. Rather than fine-tuning around the architecture’s limitations, she is redesigning how numerical tokens are represented within the attention mechanism itself, giving numbers an embedding that encodes positional value and magnitude. The goal is a model that understands not just that 623 is a number, but that it is six hundreds, two tens, and three ones and when it errs, errs in the right order of magnitude, confusing 623 with 631, not with 523.
“There is no dream application,” she says. “I just want the model to know that six twenty-three is six hundred plus thirty plus two.” If that becomes possible, the implications reach well beyond question answering – into financial analysis, medical dosing, and legal thresholds. Gayathri knows the true potential of her work lies in what others will build upon it. “Maybe I can’t think beyond question-answering. But if I publish this, maybe someone else will take it even further.”
A different kind of Doctor
On Wednesday evenings, Gayathri walks to the campus music room for Carnatic vocal class. She performs Bharatanatyam and Mohiniyattam with a group of research scholars whenever the institute calendar gives them a stage. She loves, she admits cheerfully, to talk.
What she enjoys most, however, is teaching database management to BTech students. “In a small way I am already doing what I eventually want to do. My parents had assumed I would become a teacher at Sreekrishnapuram and I used to tell my mother that I would become a doctor,” she reminisces. “Soon I hope to become both a teacher and a doctor, but of a different sort.”
Gayathri is a fourth-year PhD researcher at IIT Palakkad working on improving numerical understanding in large language models. Beyond research, she practices Carnatic music, coordinates major events, and teaches database management to BTech students. Her first international publication appeared at ECIR 2024 in Italy.