Simple Linear Regression Example Problems With Solutions

Simple Linear Regression Example Problems with Solutions

Simple linear regression is a fundamental statistical method used to model the relationship between two continuous variables. One variable, the independent variable (often denoted as 'x'), is thought to influence the other, the dependent variable (often denoted as 'y'). The goal is to find the best-fitting straight line that describes this relationship, allowing us to predict the value of 'y' given a value of 'x'. This article will delve into several example problems, providing detailed solutions to solidify your understanding of this powerful technique.

Understanding the Linear Regression Equation

Before diving into the examples, let's refresh our understanding of the core equation:

y = mx + c

Where:

• y is the dependent variable (the value we're trying to predict).
• x is the independent variable (the value we use for prediction).
• m is the slope of the line (representing the change in y for a unit change in x).
• c is the y-intercept (the value of y when x is 0).

In the context of regression, we often express this equation as:

ŷ = β₀ + β₁x

Where:

• ŷ (y-hat) represents the predicted value of y.
• β₀ is the estimated y-intercept.
• β₁ is the estimated slope.

The process involves finding the values of β₀ and β₁ that minimize the sum of the squared differences between the observed y values and the predicted ŷ values. This method is known as the least squares method.

Example Problem 1: Ice Cream Sales and Temperature

Let's say we're analyzing the relationship between daily ice cream sales (in dollars) and the daily average temperature (in degrees Celsius). We have the following data:

Solution:

1. Calculate the means of x and y:

   • Mean temperature (x̄) = (15 + 18 + 22 + 25 + 28) / 5 = 21.6 °C
   • Mean ice cream sales (ȳ) = (200 + 250 + 300 + 350 + 400) / 5 = 300 $

2. Calculate the slope (β₁):

   This involves calculating the covariance of x and y, and dividing by the variance of x. A simpler formula, derived from the least squares method, is:

   β₁ = Σ[(xi - x̄)(yi - ȳ)] / Σ(xi - x̄)²

   Let's break this down:

   • (xi - x̄): Differences between each temperature and the mean temperature.
   • (yi - ȳ): Differences between each sales figure and the mean sales.
   • (xi - x̄)(yi - ȳ): Product of the above differences.
   • Σ[(xi - x̄)(yi - ȳ)]: Sum of the products.
   • Σ(xi - x̄)²: Sum of the squared differences in temperature.

   Calculating these values:

   Temperature (°C)	Sales ($)	(xi - x̄)	(yi - ȳ)	(xi - x̄)(yi - ȳ)	(xi - x̄)²
   15	200	-6.6	-100	660	43.56
   18	250	-3.6	-50	180	12.96
   22	300	0.4	0	0	0.16
   25	350	3.4	50	170	11.56
   28	400	6.4	100	640	40.96
   Totals				1650	109.2

   β₁ = 1650 / 109.2 ≈ 15.1

3. Calculate the y-intercept (β₀):

   β₀ = ȳ - β₁x̄ = 300 - (15.1 * 21.6) ≈ 0.64

4. The regression equation:

   ŷ = 0.64 + 15.1x

This equation suggests that for every 1-degree Celsius increase in temperature, ice cream sales are predicted to increase by approximately $15.1.

Example Problem 2: Study Time and Exam Scores

A student wants to model the relationship between hours spent studying (x) and the exam score obtained (y). The data collected is as follows:

Solution: Following the same steps as in Example 1:

1. Calculate means:

   • x̄ = 4
   • ȳ = 80

2. Calculate the slope (β₁): Using the same formula as before, you will find that β₁ ≈ 10.

3. Calculate the y-intercept (β₀): β₀ = ȳ - β₁x̄ = 80 - (10 * 4) = 40

4. The regression equation:

   ŷ = 40 + 10x

This indicates that for every additional hour of study, the predicted exam score increases by 10 points.

Example Problem 3: Advertising Spend and Sales Revenue

A company is trying to determine the relationship between its advertising expenditure (in thousands of dollars) and its sales revenue (in thousands of dollars). The data collected over six months is:

Solution: Following the same procedure:

1. Calculate means: You'll find x̄ = 22.5 and ȳ = 75.

2. Calculate the slope (β₁): You'll find β₁ ≈ 2.

3. Calculate the y-intercept (β₀): β₀ = ȳ - β₁x̄ = 75 - (2 * 22.5) = 20

4. The regression equation:

   ŷ = 20 + 2x

This suggests that for every $1000 increase in advertising spend, the predicted sales revenue increases by $2000.

Interpreting the Results and Limitations

The regression equations provide a model to predict the dependent variable based on the independent variable. However, it's crucial to understand the limitations:

• Correlation does not equal causation: A strong correlation doesn't necessarily mean that one variable causes the change in the other. There might be other confounding factors involved.
• Extrapolation: Avoid using the regression equation to predict values outside the range of the observed data. The relationship might not hold true beyond this range.
• Linearity Assumption: Simple linear regression assumes a linear relationship between the variables. If the relationship is non-linear, this model will not be accurate.
• Outliers: Outliers can significantly influence the regression line. Consider investigating and potentially removing outliers if appropriate.
• R-squared Value: This value (ranging from 0 to 1) indicates the goodness of fit of the model. A higher R-squared value suggests a better fit. It represents the proportion of variance in the dependent variable explained by the independent variable.

Beyond Simple Linear Regression

While simple linear regression is a powerful tool for understanding the relationship between two variables, more complex methods exist for analyzing multiple independent variables (multiple linear regression) or non-linear relationships. These include polynomial regression, logistic regression, and many others. Understanding the fundamentals of simple linear regression, however, provides a strong foundation for exploring these more advanced techniques.

This comprehensive explanation, incorporating multiple example problems with detailed solutions, aims to enhance your understanding of simple linear regression. Remember to always critically evaluate your results and consider the limitations of the model in the context of your specific application. Further exploration into statistical software packages like R or Python's Scikit-learn can help you perform these calculations and visualizations more efficiently for larger datasets.