Alex Open Research Wiki
Search
Search
Dark mode
Light mode
Explorer
Tag: information-theory
3 items with this tag.
Jul 28, 2026
How much can language models memorize?
language-models
memorization
model-capacity
information-theory
membership-inference
double-descent
Jun 20, 2026
LLMs as Noisy Channels
entity
llm-scaling-laws
information-theory
training-dynamics
Jun 20, 2026
LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws
llm-scaling-laws
information-theory
training-dynamics
quantization
supervised-fine-tuning