Himanshu Gupta

I am an Applied Scientist at Amazon where I work on development of foundation models for Amazon Business. My main research contributions are showcasing sample efficiency of instruction tuned language models, mathematical robustness, synthetic data generation and aspect based sentiment analysis. I completed my Bachelors from BITS Pilani and masters majoring in Computer Science (Thesis Track) from Arizona State University. My thesis was on "Sample efficiency of Instruction Tuned Models" supervised by Prof. Chitta Baral.

I recently interned at Krutrim where I created a multi-lingual 7B foundation model focused on Indic languages and developed a new tokenizer for the same! It keeps the vocabulary concise at around 100,000 words while still understanding most words. We also built a multilingual large language model (LLM) that understands 10 Indic languages (Media Coverage). This involved gathering pretraining corpora of nearly 2 trillion tokens and training the LLM. To further improve its performance, I worked on generating supervised finetuning data. Finally, with the help of vLLM and TensorRT, we deployed the LLM for efficient use with high throughput and low latency.

Email  /  CV  /  Google Scholar  /  X  /  Linkedin

My research interests include Large Language Models, Enhancing Pretraining Corpora, Instruction Tuning, and Direct Preference Optimization. I collaborate with Dr. Swaroop Mishra on topics such as Instruction tuning, Synthetic dataset creation and mathematical reasoning. My strength lies in generating new ideas and I am fortunate to collaborate with a diverse set of awesome researchers.