What Is a Regression Model?

regression model -magoosh

Regression is a statistical technique that is used to model the relationship of a dependent variable with respect to one or more independent variables. Regression is widely used in several statistical analysis problems and it is also one of the most important tools in Machine Learning.

Regression Model Example

Let us understand a simple example of regression – predicting the price of a house in the United States. We call the price of the house (in USD) as Y. Clearly, Y depends on the following variables:

  • Size of the house in square feet (X1)
  • Number of rooms in the house (X2)
  • City where the house is located (X3)
  • Age of the house (X4)

…and many others

In such a case, the regression analysis problem tries to answer the following question – given the above variables, what would be the predicted price of a house in USD?

Clearly, as the size of the house increases, price increases. Same with the number of rooms. However, with an increase in age, the price decreases. A house in San Francisco would be far more expensive than a house in, say, Austin. Regression tries to model the underlying correlation between these variables to develop a framework to predict the price of a house.

Types of Regression Models

There are two commonly used variations of regression:

  1. Linear Regression: In linear regression, the variable Y is known to be a linear function of the variables X1, X2, X3 … linear regression is a widely used tool in Statistics and Machine Learning. The idea is to come up with the function h(X1, X2, X3 …) = a + bX1 + cX2 + dX3 + … such that h is as close as possible to the actual underlying price determination function. The parameters a, b, c … are obtained by minimizing the cost function of the linear regression model.
  2. Logistic Regression: In logistic regression, the dependent variable Y can take any value. For instance, the price of a house predicted by the linear regression algorithm can be any real number. In logistic regression, however, the output variable Y can take only two values – 0 and 1. Whereas the purpose of linear regression is to predict a value, logistic regression tries to answer a “yes or no” question. For instance, is a given email spam (denoted as 1) or not spam (denoted as 0)?

Regression is quite a powerful tool and forms the building block of several complex machine learning algorithms. All those who aim to become seasoned data scientists should aim to understand both linear and logistic regression extremely well.

Comments are closed.

Magoosh blog comment policy: To create the best experience for our readers, we will only approve comments that are relevant to the article, general enough to be helpful to other students, concise, and well-written! 😄 Due to the high volume of comments across all of our blogs, we cannot promise that all comments will receive responses from our instructors.

We highly encourage students to help each other out and respond to other students' comments if you can!

If you are a Premium Magoosh student and would like more personalized service from our instructors, you can use the Help tab on the Magoosh dashboard. Thanks!