Hi, I'm Jaydeep

A falsification experiment: does the DPO "squeezing" failure mode, where chosen and rejected log-probabilities collapse together instead of just the rejected one falling, hit sub-1B models earlier or harder than the 7B+ models it's documented in? Not supported. Three rounds of iterative DPO with verifiable rewards (DPO-VP) on Qwen2.5-0.5B-Instruct + GSM8K show the textbook healthy signature instead: chosen logprob stays flat, rejected falls faster, preference gap grows round over round. A matched GRPO baseline is equally stable. Pushing further to SmolLM2-135M hit a capacity floor (3% pass@1) before the squeezing question could even be tested at that scale.

An open-source, graduate-level textbook and engineering handbook documenting the complete evolution of LLM post-training (2017–2026), from early instruction tuning through modern reasoning-focused RL systems. Aiming to be the resource a researcher could use to reproduce the post-training pipeline of any frontier lab: paper encyclopedia, reconstructed per-lab recipes, algorithm deep-dives with derivations, and runnable implementations. Early scaffolding stage — structure is in place, content is being written.

Top 100 at the HuggingFace × Meta OpenEnv Hackathon, Bangalore. Built an RL environment where a small monitor model learns to catch a frontier attacker (DeepSeek-R1) attempting hidden harmful behaviors across 9 enterprise scenarios. Every reward is a plain Python check, with no LLM-as-judge anywhere in the loop. After 10 reward design iterations, a simple linear reward beat the more elaborate composed one. Trained a Qwen-2.5-1.5B LoRA monitor from random performance (AUROC 0.500) to strong detection (0.893, Recall 0.88, FPR 0.12), closing about 40% of the gap to Gemini-2.5-Pro at under 0.1% of its per-call cost.

The full post-training stack run end-to-end on a single sub-1B model, SmolLM-135M, with each stage measured. CPT: continued pre-training on 138 arXiv ML papers via QLoRA (rank 32, Unsloth). On held-out papers: −20.1% perplexity (22.97 to 18.36), +25.4% ROUGE-L, +37.5% BLEU; rank saturates at r≥16, so the data (not the rank) is the bottleneck. SFT: instruction-tuned on a 300K-pair dataset generated from arXiv papers (7 stochastic task types, structured decoding via Outlines), using ChatML and response-only loss masking. DPO: preference-aligned on 151K LLM-judged pairs (β=0.1), which lifted held-out reward accuracy from 0.50 to 0.72 while preserving generation diversity, an alignment gain with no mode collapse. All checkpoints published to HuggingFace.

Which LoRA tricks actually matter for continued pre-training? A reproducible study stress-testing the recommendations in Unsloth's CPT guide. Llama-2-7B is continually pretrained on a code corpus (Magicoder-Evol-Instruct-110K); starting from a paper baseline, each experiment adds exactly one ingredient (extra LoRA target modules, rank-stabilized LoRA, a decoupled embedding learning rate) and measures the effect on held-out validation perplexity. One training script, six config files, one varied knob each. That identity is what makes it a controlled experiment rather than six scripts that drift apart.

Two diffusion language models, both built from scratch. ModernBERT (~150M): pretrained as a masked diffusion LM on Project Gutenberg (6.4M chunks, 20 hours on an RTX 4090), then SFT'd on Open-Orca (~4.2M Q&A pairs, instruction-token-only loss). TinyStories (45M): full architecture written from scratch and trained for 60K steps, with confidence-based iterative denoising at generation time (128 diffusion steps). Loss dropped sharply around step 25K, the point where structure starts to emerge. All checkpoints published to HuggingFace.

End-to-end system for evaluating government tenders. The pipeline pulls criteria from the document with a DeepSeek LLM, retrieves matching evidence with sentence-transformers, and writes an explainable verdict with cited source clauses. OCR uses a three-tier fallback: PyMuPDF first, Tesseract next, DeepSeek Vision when confidence drops below 65%. Borderline verdicts (confidence 0.55 to 0.80) route to a human review queue with a full audit trail. Covered by a 43-check smoke test suite.