Anirudh Lakhotia

I am an undergraduate student in the Department of Computer Science and Engineering at PES University, advised by Dr. Gowri Srinivasa. My research interests lie in efficient machine learning and natural language processing, with a focus on parameter-efficient fine-tuning, representation engineering, and multilingual language models.

I do research at the intersection of natural language processing (NLP) and machine learning, and my broad research objective is bridging the gap between state-of-the-art NLP methods and the diverse communities of potential users who could significantly benefit from these advanced technologies, but for whom practical implementation remains a challenge.

I have gained valuable industry experience through internships at Cisco and Nokia, working on distributed systems and practical machine learning applications. I am always excited to collaborate on projects - feel free to reach out if you’d like to work together!

news

Oct 10, 2024	Undergraduate thesis paper accepted at NeurIPS 2024 Workshop - UnoLoRA: Single Low-Rank Adaptation for Efficient Multitask Fine-tuning
Apr 12, 2024	Won Hack Five, an open-source AI hackathon organized by Hasgeek and sponsored by Meta and Microsoft For Startups! Check out our winning project on GitHub.
Jan 24, 2024	Awarded $5,000 research grant from Microsoft For Startups to advance development of efficient multilingual AI systems using mixture-of-experts architecture.

selected publications

NeurIPS

UnoLoRA: Single Low-Rank Adaptation for Efficient Multitask Fine-tuning

Anirudh Lakhotia, Akash Kamalesh, H S Nischal, Prerana Sanjay Kulkarni, and Gowri Srinivasa

Workshop on Fine-Tuning in Machine Learning at NeurIPS, 2024

Abs PDF Listen

Recent advances in Parameter-Efficient Fine-Tuning (PEFT) have shown Low-Rank Adaptation (LoRA) to be an effective implicit regularizer for large language models. Building on these findings, we propose UnoLoRA, a novel approach that leverages a single shared LoRA module for efficient multi-task learning. While existing methods typically use separate LoRA adaptations for each task, our approach demonstrates that a single shared adapter can effectively capture both task-specific and task-agnostic knowledge. We further introduce UnoLoRA*, an enhanced variant that employs a shared hypernetwork to generate task-specific embeddings, improving convergence and task adaptation. Our method significantly reduces trainable parameters to just 0.05% per task while maintaining competitive performance on the GLUE benchmark. Our analysis reveals that the A and B matrices in our shared LoRA adapter naturally develop complementary roles: A matrices capture generalizable features across tasks, while B matrices specialize in task-specific representations. Our results show that sharing a single LoRA adapter can achieve efficient multi-task learning while significantly reducing memory requirements, making it particularly valuable for resource-constrained applications.