Understanding Linear Regression from Scratch

 Linear regression, known as a gateway to statistics and machine learning, is a method for simply capturing the linear relationship between variables. 

It’s a fundamental technique for discovering patterns hidden within data and applying them to predictions and decision-making.

1. What is Linear Regression?

Linear regression is a method for predicting changes in a target variable (such as sales or temperature) using the relationship with explanatory variables (such as advertising spend or time).

The simplest form, “simple linear regression,” uses a single straight line, while “multiple linear regression,” which deals with multiple factors, models the relationship with a high-dimensional “plane” or “hyperplane.”

- Example of Simple Linear Regression

Represent the relationship between "study time" and "test scores" with a straight line, and estimate how many points the score will increase if study time increases by one hour.

- Example of Multiple Linear Regression

Predict sales by combining "advertising spend," "seasonal factors," and "economic indicators."

2. Where is Linear Regression Used?

The simplicity and ease of interpretation of linear regression make it valuable in a wide range of fields, from business to science.

- Marketing Analysis

Use advertising spend and campaign variables as explanations to predict sales and customer acquisition numbers.

- Economic and Financial Modelling

Quantify the impact of macroeconomic indicators such as interest rates and price indices on stock prices and exchange rates.

- Healthcare and Public Health

Use patient age, BMI, and smoking history as explanatory variables to predict disease risk and treatment effectiveness.

- Real Estate Price Estimation

Analyse the relationship between property area, age of building, distance to station, and price to calculate a fair price.

- Manufacturing Quality Control

Model the relationship between production conditions (temperature, pressure, raw materials) and product defect rate, and explore optimal production settings.

3. Benefits of Learning Linear Regression

- You can intuitively understand "which factors have how much influence" simply by reading the coefficients of the model.

- It’s the perfect first step for exploring correlations and potential causal relationships between data.

- The simple mathematical model is highly explainable, giving stakeholders and non-technical personnel a sense of understanding.

- It’s easy to incorporate directly into decision-making processes such as budgeting and KPI setting.

- You can directly identify areas for improvement by visualising the model’s goodness of fit (R²) and residual distribution.

- You’ll learn the basics of feature engineering and develop variable design skills that are also effective in non-linear models.

- It’s easy to run with just a few lines of code. With Python, you can complete a prototype instantly using `scikit-learn`.

Summary

Once you have a deeper understanding of linear regression, practice applying your knowledge to your own data and develop a feel for optimal model selection and tuning.

Linear regression will always be your first step and powerfully support your journey into data analysis.

If you want to learn linear regression, we recommend this book (access here).