• Shobhit
  • Posts
  • Writing an LLM from scratch on an RTX 3090

Writing an LLM from scratch on an RTX 3090

LLM from scratch, state of AI, a git story, and the ship where Aryan Khan was arrested

Author

Shobhit Bakliwal
December 09, 2025

Hello hello! What’s up?

This is a letter, a personal letter, from me to you. I hope to send it to you every week (I know I am failing at this). If you find it boring, or want to holler at me, just reply to this.

I am going to share three interesting things I came across the past week, every week.

Let’s get into it.

  1. Writing an LLM from scratch on an RTX 3090

Writing an LLM from scratch, part 28 -- training a base model from scratch on an RTX 3090

I felt like it should be possible to train a GPT-2 small level model on my own hardware using modern tools and open datasets from scratch. It was!

www.gilesthomas.com/2025/12/llm-from-scratch-28-training-a-base-model-from-scratch

I think this is a very valuable exercise if you want to understand how LLMs work, and if you have the time!

  1. State of AI by OpenRouter

State of AI | OpenRouter

An empirical study analyzing over 100 trillion tokens of real-world LLM interactions across tasks, geographies, and time.

openrouter.ai/state-of-ai

Huge ass article that details the state of AI usage, based on data collected by OpenRouter.

  1. The making of Fizzy, as told by Git

The Making of Fizzy, Told by Git

www.zolkos.com/2025/12/02/the-making-of-fizzy-told-by-git

Fizzy is a new tool by Basecamp, which is a commercial software with open code. You can go through the git history and see what the makers decide on which features, what was thrown away, how did they productionize it, etc. This is a really good article that basically use claude code to go through the git history and generate a story from it!

Have been pretty active on my podcast channel releasing a video a week!

This week we released a new video that dissects Codelia Cruises. They do ₹500 crore with just 1 ship!

This is also the ship on which Aryan Khan was detained 😄.

P.S. Hit me up if you’re looking for a team to build full stack web or mobile apps.