Himanshu Gupta

I am an Applied Scientist at Amazon (Stores Foundation AI team), where I work on mid-training, post-training optimization, synthetic data creation and benchmarking of large foundation models. I have hands on experience with Megatron-LM, verl (SFT, DPO, GRPO), vLLM , SGLang and TRL . I also had the opportunity to develop multi-lingual a foundation model from scratch at Krutrim which focused on 10 Indic languages (Media Coverage , Technical Report). My thesis was on Sample efficiency of Instruction Tuned Models under the guidance of Dr. Swaroop Mishra and Prof. Chitta Baral.

Email  /  Google Scholar  /  X  /  Linkedin

My main research contributions are showcasing various aspects of Instruction Tuning ( Sample Efficiency, Aspect Based Sentiment Analysis, Long Sequence Medical Tasks, Financial NER, Variable Name Recovery with Decompiled Output, Event Detection ), Efficient Pretraining, Efficient LLMs-as-a-Judge, Mathematical Benchmarking ( Numerical Feasibility, Adversarial Math Word Problems, Multimodal Mathematical Benchmarking, Humanity's Last Exam and ), Synthetic Data Generation, My recent research focuses on building large foundation models that excel at instruction following, adherence to detailed guidelines, and high-precision reasoning, with particular emphasis on enhancing mid-training corpora and RL-based post-training alignment.