## Statistical properties

### Heap's law
- Estimate the vocabulary size of a collection
- `$ T $`: Number of tokens
- `$ M $`: Vocabulary size (distinct terms in collection)
- `$ M=kT^b $`
	- `$ 30\leq k \leq 100 $`
	- `$ b\approx 0.5 $`

### Zipf's law
- Distribution of terms in a collection
- Collection frequency inverse proportional to rank in frequency table
- Sort terms by frequency `$ t_1,..., t_M $`
- `$ cf(t_i) \propto ci^{-1}, c\in \mathbb{R} $`
	- `$ cf(t_i) = c * i^{-1} $`
	- `$ \log cf(t_i) = \log c - \log i $`
- Example
	- Second most frequent word has half of the occurrences of the first
	- Third most frequent word has third of the occurrences of the first
- Many types of data in physical and social science approximated by Zipf's law