Statistical & Financial Consulting by Stanford PhD

Home Page

Logistic Regression is a method used to model a non-linear relationship between a categorical dependent variable and one or more independent variables. The dependent variable is also called the response variable, while the independent variables are called the predictors. The logistic regression model is the following:

* log( P(Y = 1) / P(Y = K) ) = B _{10} + B_{11} * X_{1} + ... + B_{1p} * X_{p}*

...

Here

Note that logistic regression focuses on the probability of the response variable being a particular value. This is different from linear regression, which focuses on the expectation of the response variable. Linear regression may get the expectation right but misspecify the probabilities due to an incorrect choice of the distribution of residuals. On the other hand, logistic regression may get the probabilities of several categories correctly but misspecify the rest of the distribution.

Logistic regression is a particular case of generalized linear models. In a generalized linear model, the conditional distribution of the response given the predictors can be any standard continuous or discrete distribution. The parameters of the conditional distribution are perfectly specified by non-linear functions of the predictors, which are called link functions. These link functions have the parameters of their own, which are the true parameters of the generalized linear model. They are estimated from the data. Logistic regression corresponds to the so-called

Just like the rest of generalized linear models, logistic model is estimated by the method of maximum likelihood. Since the model is non-linear, the traditional t-tests and F-tests have no meaning. They are substituted by Wald tests and likelihood ratio tests. These tests are easily calculated as by-products of the maximum likelihood estimation of the model.

Balakrishnan, N. (1991). Handbook of the Logistic Distribution. New York: Marcel Dekker, Inc.

Hilbe, J. M. (2009). Logistic Regression Models. Boca Raton, FL: Chapman & Hall / CRC Press.

Greene, W. H. (2011). Econometric Analysis (7th ed). Upper Saddle River, NJ: Prentice Hall.

Efron, B. (1975). The efficiency of logistic regression compared to normal discriminant analysis. J. Amer. Statist. Assoc. 70, pp. 892-898.

Agresti, A. (2002). Categorical Data Analysis. New York: Wiley-Interscience.

Teller, G. R. (1999). Mathematical Statistics: A Unified Introduction. Springer.

- Detailed description of the services offered in the areas of statistical consulting and financial consulting: home page, types of service, experience, case studies, payment options and statistics tutoring
- Directory of financial topics