The Hill estimator is one of ways of estimating the tail index alpha.
Suppose $X_{1}, X_{2},...,X_{n}$ are independent non-negative random variables.
* The Heavy-tailed data : 1-F(X) = $P(X_{i}>x)=x^{-\alpha}L(x)$,
where $\alpha(=\frac{1}{\gamma})$ > 0 is an unknown parameter (called the tail index) and it describes the heaviness of the right tail. And L(x) is an slowly varying function satisfying $\lim_{x\rightarrow \infty}\frac{L(tx)}{L(x)}=1$, t>0.
* Bias increases with k, variance decrease with k. Choice of k can be chosen by Hill plot.
The density estimator: $\hat{f_{h}x}= \frac{1}{nh}\sum_{i=1}^n w(\frac{x-x_{i}}{h})$, where w is a symmetric probability density.
* The bandwidth h: If h is too large, the estimator $\hat{f}$ is too smooth, whereas the h is too small, then the estimator $\hat{f}$ is too noisy. Therefore, the bias and variance depend on the bandwidth h!
* Remark) As the bias depends on the unknown density, therefore the choice of the bandwidth h is complicated.
In non-parametriec regression, our model would be $y_{i}=g(x_{i})+ \varepsilon_{i}$.
We need an inference about g(x), smooth function of x. We can estimate g(x) by $\hat{g}(x)=\sum_{i\in S(x)} w_{i}(x)y_{i}$, where $S(x)= \{ |x_{i} -x |\leq h\}$, bandwidth h. Our $\hat{g}(x)$ will be a loess smoother which uses weighted linear. So the estimated function is smoother, but severly biased. The estimated function is less smooth indicating smaller bias, but large variance.
