Himanshu Gupta

I am an Applied Scientist at Amazon (Stores Foundation AI team), where I work on mid-training, post-training optimization, synthetic data creation and benchmarking of large foundation models. I have hands on experience with Megatron-LM, verl (SFT, DPO, GRPO), vLLM , SGLang and TRL . I also had the opportunity to develop multi-lingual a foundation model from scratch at Krutrim which focused on 10 Indic languages (Media Coverage , Technical Report). My thesis was on Sample efficiency of Instruction Tuned Models under the guidance of Dr. Swaroop Mishra and Prof. Chitta Baral.

Email  /  Google Scholar  /  X  /  Linkedin

My main research contributions are showcasing various aspects of Instruction Tuning ( Sample Efficiency, Aspect Based Sentiment Analysis, Long Sequence Medical Tasks, Financial NER, Variable Name Recovery with Decompiled Output, Event Detection ), Mathematical Benchmarking ( Numerical Feasibility, Adversarial Math Word Problems, Multimodal Mathematical Benchmarking, Humanity's Last Exam ), Synthetic Data Generation and Efficient LLMs-as-a-Judge. My recent research focuses on building large foundation models that excel at instruction following, adherence to detailed guidelines, and high-precision reasoning, with particular emphasis on enhancing mid-training corpora and RL-based post-training alignment.