CS336 Series - Language Modeling from Scratch
In this series of blog posts, I will use LLMs to summarize the great lectures by Prof. Percy Liang on Language Modeling from Scratch (CS336, Stanford University). The motivation behind this series is simply to explore whether we can learn a course faster and (hopefully) better with the help of LLMs.
Firstly, this whole project is only possible because all the lectures are generously shared publicly on YouTube here, along with the code here. So all credits go to Prof. Percy Liang and the Stanford CS336 team.
Secondly, my role is mainly to design the pipeline that automatically generates these blog posts. The actual content is produced by LLMs with very little human intervention. While I believe that today’s flagship reasoning LLMs can often summarize better than the average human, they are still not perfect. There may be mistakes, hallucinations, or missing insights—especially since I intentionally avoid adding too much of my own interpretation. So please keep these limitations in mind and take the content with a grain of salt.
Thirdly, this should not be seen as a substitute for the official materials. Watching the lectures directly will always give you the best learning experience.
Resources:
- YouTube Lectures: https://www.youtube.com/watch?v=SQ3fZ1sAqXI
- Code Repository: https://github.com/stanford-cs336/spring2025-lectures
- Course Website: https://stanford-cs336.github.io/spring2025/
Links to the blog posts
- CS336 Lecture 1: Overview and Tokenization
- CS336 Lecture 2: PyTorch, resource accounting
- CS336 Lecture 3: Architectures, hyperparameters
- CS336 Lecture 4: Mixture of experts
- CS336 Lecture 5: GPUs
- CS336 Lecture 6: Kernels, Triton
- CS336 Lecture 7: Parallelism
- CS336 Lecture 8: Parallelism - Part 2
- CS336 Lecture 9: Scaling laws
- CS336 Lecture 10: Inference
- CS336 Lecture 11: Scaling laws
- CS336 Lecture 12: Evaluation
- CS336 Lecture 13: Data
- CS336 Lecture 14: Data - Part 2
- CS336 Lecture 15: Alignment - SFT/RLHF
- CS336 Lecture 16: Alignment - RL
- CS336 Lecture 17: Alignment - RL - Part 2
Pipeline
The pipeline is designed to be as automated as possible. It is composed of the following steps:
- Get transcript of the lecture by using Youtube Transcriptor tool. I use free tier from tactiq.io https://tactiq.io/tools/youtube-transcript.
- Use OpenAI API (gpt-5-mini model) to summarize the transcript, outputing a list of key discussions with detailed explanations and timestamps associated with each discussion.
- Use OpenAI API (gpt-5-mini model) to refine the detailed explanations so that they are blog post friendly format (highlighting, itemizing, etc.).
- With the timestamps, and the Youtube video URL, capture the relevant frames from the lecture video to demonstrate the key discussions. This is done by using the
youtube-dltool to download the video (only specific segments regarding that specific timestamp) and then using theffmpegtool to extract the frames. - Put all the content, frames together and generate the blog post following the al-folio blog post format.
Next steps
I will review the generated blog posts and log my thoughts and personal notes for each lecture. This will be used to improve the summarization pipeline in the future so that it can be more personalized and tailored to my learning style. Other courses will be added to the pipeline in the future.
Enjoy Reading This Article?
Here are some more articles you might like to read next: