# Quantile regression

## Statistics concept / From Wikipedia, the free encyclopedia

#### Dear Wikiwand AI, let's keep it short by simply answering these key questions:

Can you list the top facts and stats about Quantile regression?

Summarize this article for a 10 year old

**Quantile regression** is a type of regression analysis used in statistics and econometrics. Whereas the method of least squares estimates the conditional *mean* of the response variable across values of the predictor variables, quantile regression estimates the conditional *median* (or other *quantiles*) of the response variable. Quantile regression is an extension of linear regression used when the conditions of linear regression are not met.

One advantage of quantile regression relative to ordinary least squares regression is that the quantile regression estimates are more robust against outliers in the response measurements. However, the main attraction of quantile regression goes beyond this and is advantageous when conditional quantile functions are of interest. Different measures of central tendency and statistical dispersion can be used to more comprehensively analyze the relationship between variables.^{[1]}

In ecology, quantile regression has been proposed and used as a way to discover more useful predictive relationships between variables in cases where there is no relationship or only a weak relationship between the means of such variables. The need for and success of quantile regression in ecology has been attributed to the complexity of interactions between different factors leading to data with unequal variation of one variable for different ranges of another variable.^{[2]}

Another application of quantile regression is in the areas of growth charts, where percentile curves are commonly used to screen for abnormal growth.^{[3]}^{[4]}

The idea of estimating a median regression slope, a major theorem about minimizing sum of the absolute deviances and a geometrical algorithm for constructing median regression was proposed in 1760 by Ruđer Josip Bošković, a Jesuit Catholic priest from Dubrovnik.^{[1]}^{: 4 }^{[5]} He was interested in the ellipticity of the earth, building on Isaac Newton's suggestion that its rotation could cause it to bulge at the equator with a corresponding flattening at the poles.^{[6]} He finally produced the first geometric procedure for determining the equator of a rotating planet from three observations of a surface feature. More importantly for quantile regression, he was able to develop the first evidence of the least absolute criterion and preceded the least squares introduced by Legendre in 1805 by fifty years.^{[7]}

Other thinkers began building upon Bošković's idea such as Pierre-Simon Laplace, who developed the so-called "methode de situation." This led to Francis Edgeworth's plural median^{[8]} - a geometric approach to median regression - and is recognized as the precursor of the simplex method.^{[7]} The works of Bošković, Laplace, and Edgeworth were recognized as a prelude to Roger Koenker's contributions to quantile regression.

Median regression computations for larger data sets are quite tedious compared to the least squares method, for which reason it has historically generated a lack of popularity among statisticians, until the widespread adoption of computers in the latter part of the 20th century.

Quantile regression expresses the conditional quantiles of a dependent variable as a linear function of the explanatory variables. Crucial to the practicality of quantile regression is that the quantiles can be expressed as the solution of a minimization problem, as we will show in this section before discussing conditional quantiles in the next section.

### Quantile of a random variable

Let $Y$ be a real-valued random variable with cumulative distribution function $F_{Y}(y)=P(Y\leq y)$. The $\tau$th quantile of Y is given by

- $q_{Y}(\tau )=F_{Y}^{-1}(\tau )=\inf \left\{y:F_{Y}(y)\geq \tau \right\}$

where $\tau \in (0,1).$

Define the loss function as $\rho _{\tau }(m)=m(\tau -\mathbb {I} _{(m<0)})$, where $\mathbb {I}$ is an indicator function.
A specific quantile can be found by minimizing the expected loss of $Y-u$ with respect to $u$:^{[1]}(pp. 5–6):

- $q_{Y}(\tau )={\underset {u}{\mbox{arg min}}}E(\rho _{\tau }(Y-u))={\underset {u}{\mbox{arg min}}}{\biggl \{}(\tau -1)\int _{-\infty }^{u}(y-u)dF_{Y}(y)+\tau \int _{u}^{\infty }(y-u)dF_{Y}(y){\biggr \}}.$

This can be shown by computing the derivative of the expected loss with respect to $u$ via an application of the Leibniz integral rule, setting it to 0, and letting $q_{\tau }$ be the solution of

- $0=(1-\tau )\int _{-\infty }^{q_{\tau }}dF_{Y}(y)-\tau \int _{q_{\tau }}^{\infty }dF_{Y}(y).$

This equation reduces to

- $0=F_{Y}(q_{\tau })-\tau ,$

and then to

- $F_{Y}(q_{\tau })=\tau .$

If the solution $q_{\tau }$ is not unique, then we have to take the smallest such solution to obtain
the $\tau$th quantile of the random variable *Y*.

#### Example

Let $Y$ be a discrete random variable that takes values $y_{i}=i$ with $i=1,2,\dots ,9$ with equal probabilities. The task is to find the median of Y, and hence the value $\tau =0.5$ is chosen. Then the expected loss of $Y-u$ is

- $L(u)=E(\rho _{\tau }(Y-u))={\frac {(\tau -1)}{9}}\sum _{y_{i}<u}$$(y_{i}-u)$$+{\frac {\tau }{9}}\sum _{y_{i}\geq u}$$(y_{i}-u)$$={\frac {0.5}{9}}{\Bigl (}$$-$$\sum _{y_{i}<u}$$(y_{i}-u)$$+\sum _{y_{i}\geq u}$$(y_{i}-u)$${\Bigr )}.$

Since ${0.5/9}$ is a constant, it can be taken out of the expected loss function (this is only true if $\tau =0.5$). Then, at *u*=3,

- $L(3)\propto \sum _{i=1}^{2}$$-(i-3)$$+\sum _{i=3}^{9}$$(i-3)$$=[(2+1)+(0+1+2+...+6)]=24.$

Suppose that *u* is increased by 1 unit. Then the expected loss will be changed by $(3)-(6)=-3$ on changing *u* to 4. If, *u*=5, the expected loss is

- $L(5)\propto \sum _{i=1}^{4}i+\sum _{i=0}^{4}i=20,$

and any change in *u* will increase the expected loss. Thus *u*=5 is the median. The Table below shows the expected loss (divided by ${0.5/9}$) for different values of *u*.

u | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |

Expected loss | 36 | 29 | 24 | 21 | 20 | 21 | 24 | 29 | 36 |

#### Intuition

Consider $\tau =0.5$ and let *q* be an initial guess for $q_{\tau }$. The expected loss evaluated at *q* is

- $L(q)=-0.5\int _{-\infty }^{q}(y-q)dF_{Y}(y)+0.5\int _{q}^{\infty }(y-q)dF_{Y}(y).$

In order to minimize the expected loss, we move the value of *q* a little bit to see whether the expected loss will rise or fall.
Suppose we increase *q* by 1 unit. Then the change of expected loss would be

- $\int _{-\infty }^{q}1dF_{Y}(y)-\int _{q}^{\infty }1dF_{Y}(y).$

The first term of the equation is $F_{Y}(q)$ and second term of the equation is $1-F_{Y}(q)$. Therefore, the change of expected loss function is negative if and only if $F_{Y}(q)<0.5$, that is if and only if *q* is smaller than the median. Similarly, if we reduce *q* by 1 unit, the change of expected loss function is negative if and only if *q* is larger than the median.

In order to minimize the expected loss function, we would increase (decrease) *L*(*q*) if *q* is smaller (larger) than the median, until *q* reaches the median. The idea behind the minimization is to count the number of points (weighted with the density) that are larger or smaller than *q* and then move *q* to a point where *q* is larger than $100\tau$% of the points.

### Sample quantile

The $\tau$ sample quantile can be obtained by using an importance sampling estimate and solving the following minimization problem

- ${\hat {q}}_{\tau }={\underset {q\in \mathbb {R} }{\mbox{arg min}}}\sum _{i=1}^{n}\rho _{\tau }(y_{i}-q),$
- $={\underset {q\in \mathbb {R} }{\mbox{arg min}}}\left[(\tau -1)\sum _{y_{i}<q}(y_{i}-q)+\tau \sum _{y_{i}\geq q}(y_{i}-q)\right]$,

where the function $\rho _{\tau }$ is the tilted absolute value function. The intuition is the same as for the population quantile.

The $\tau$th conditional quantile of $Y$ given $X$ is the $\tau$th quantile of the Conditional probability distribution of $Y$ given $X$,

- $Q_{Y|X}(\tau )=\inf \left\{y:F_{Y|X}(y)\geq \tau \right\}$.

We use a capital $Q$ to denote the conditional quantile to indicate that it is a random variable.

In quantile regression for the $\tau$th quantile we make the assumption that the $\tau$th conditional quantile is given as a linear function of the explanatory variables:

- $Q_{Y|X}(\tau )=X\beta _{\tau }$.

Given the distribution function of $Y$, $\beta _{\tau }$ can be obtained by solving

- $\beta _{\tau }={\underset {\beta \in \mathbb {R} ^{k}}{\mbox{arg min}}}E(\rho _{\tau }(Y-X\beta )).$

Solving the sample analog gives the estimator of $\beta$.

- ${\hat {\beta _{\tau }}}={\underset {\beta \in \mathbb {R} ^{k}}{\mbox{arg min}}}\sum _{i=1}^{n}(\rho _{\tau }(Y_{i}-X_{i}\beta )).$

Note that when $\tau =0.5$, the loss function $\rho _{\tau }$ is proportional to the absolute value function, and thus median regression is the same as linear regression by least absolute deviations.

The mathematical forms arising from quantile regression are distinct from those arising in the method of least squares. The method of least squares leads to a consideration of problems in an inner product space, involving projection onto subspaces, and thus the problem of minimizing the squared errors can be reduced to a problem in numerical linear algebra. Quantile regression does not have this structure, and instead the minimization problem can be reformulated as a linear programming problem

- ${\underset {\beta ,u^{+},u^{-}\in \mathbb {R} ^{k}\times \mathbb {R} _{+}^{2n}}{\min }}\left\{\tau 1_{n}^{'}u^{+}+(1-\tau )1_{n}^{'}u^{-}|X\beta +u^{+}-u^{-}=Y\right\},$

where

- $u_{j}^{+}=\max(u_{j},0)$ , $u_{j}^{-}=-\min(u_{j},0).$

Simplex methods^{[1]}^{: 181 } or interior point methods^{[1]}^{: 190 } can be applied to solve the linear programming problem.

For $\tau \in (0,1)$, under some regularity conditions, ${\hat {\beta }}_{\tau }$ is asymptotically normal:

- ${\sqrt {n}}({\hat {\beta }}_{\tau }-\beta _{\tau }){\overset {d}{\rightarrow }}N(0,\tau (1-\tau )D^{-1}\Omega _{x}D^{-1}),$

where

- $D=E(f_{Y}(X\beta )XX^{\prime })$ and $\Omega _{x}=E(X^{\prime }X).$

Direct estimation of the asymptotic variance-covariance matrix is not always satisfactory. Inference for quantile regression parameters can be made with the regression rank-score tests or with the bootstrap methods.^{[9]}

See invariant estimator for background on invariance or see equivariance.

### Scale equivariance

For any $a>0$ and $\tau \in [0,1]$

- ${\hat {\beta }}(\tau ;aY,X)=a{\hat {\beta }}(\tau ;Y,X),$
- ${\hat {\beta }}(\tau$ ;-aY,X)=-a{\hat {\beta }}(1-\tau ;Y,X).}

### Shift equivariance

For any $\gamma \in R^{k}$ and $\tau \in [0,1]$

- ${\hat {\beta }}(\tau ;Y+X\gamma ,X)={\hat {\beta }}(\tau ;Y,X)+\gamma .$

### Equivariance to reparameterization of design

Let $A$ be any $p\times p$ nonsingular matrix and $\tau \in [0,1]$

- ${\hat {\beta }}(\tau ;Y,XA)=A^{-1}{\hat {\beta }}(\tau ;Y,X).$

### Invariance to monotone transformations

If $h$ is a nondecreasing function on $\mathbb {R}$, the following invariance property applies:

- $h(Q_{Y|X}(\tau ))\equiv Q_{h(Y)|X}(\tau ).$

Example (1):

If $W=\exp(Y)$ and $Q_{Y|X}(\tau )=X\beta _{\tau }$, then $Q_{W|X}(\tau )=\exp(X\beta _{\tau })$. The mean regression does not have the same property since $\operatorname {E} (\ln(Y))\neq \ln(\operatorname {E} (Y)).$