Spaces:
Running
Typos
"knowledge and technics" -> "knowledge and techniques"
"here to changes that" -> "here to change that"
"simplest to the most raffined one" -> "simplest to the most refined"
"We'll assumes you" -> "We'll assume you"
"how deep learning model are trained" -> "how deep learning models are trained"
"to fully understand how how performing LLMs" -> "to fully understand how high performing LLMs" (guessing at the intent here)
"what it’s advantages and limits are" -> "what its advantages and limits are"
In the cheatsheet:
"ep: context parallelism" → "ep: expert parallelism"
thanks for this amazing book HF !
"When training a neural network model, one store several items in memory:" → When training a neural network model, one stores several items in memory:
Awesome! Thanks a lot for the pull request
Is the formula bst=bs∗seq correct? bs = bst * seq seems like the correct formula.
and are roughtly familiar -> and are roughly familiar
Is the formula bst=bs∗seq correct? bs = bst * seq seems like the correct formula.
depends on how you define "bst" and "bs". We chose to define "bst" as batch size in tokens, which would be bs*seq (batch size in samples times sample length)
Small typos:
- "Using the Pytorch profiler we can understand how memory is allocated througho ut training" -> "Using the Pytorch profiler we can understand how memory is allocated throughout training"
- "Why does the first step looks different:" -> "Why does the first step look different:"
- The TeX type text is not visible here:
When training a neural network model, one store several items in memory: --> When training a neural network model, one stores several items in memory:
I'm not sure which files to change on a PR with edits. I only see the pdf file but not the source markdown file
You would think for a model you could compute the memory requirements exactly but there are a few additional memory occupants that makes it hard to be exact: -->
You would think for a model you could compute the memory requirements exactly, but there are a few additional memory occupants that make it hard to be exact:
