California Housing Price Prediction

A machine learning regression project for predicting California housing prices using the California Housing dataset with Scikit-learn and XGBoost.

📌 Project Overview

This project demonstrates a complete machine learning workflow for a regression problem, including:

• Loading the California Housing dataset
• Data exploration and analysis
• Feature and target separation
• Splitting data into training and testing sets
• Building an XGBoost regression model
• Hyperparameter tuning using RandomizedSearchCV
• Model evaluation using R² Score

📂 Dataset

The project uses the California Housing Dataset provided by Scikit-learn through the fetch_california_housing() function.

Dataset characteristics:

• 20,640 samples
• 8 numerical features
• Regression task
• Target: Median house value in California districts

📍 Geographical Data Visualization

To better understand the distribution of the California Housing dataset, I created a geographical scatter plot using the longitude and latitude features to represent the map of California.

In this visualization:

• The color of each point represents the median income (MedInc) of that district.
• A color bar is included to show the relationship between colors and median income values.
• The size of each point is proportional to the district's population (Population). Since the original population values produced excessively large markers, the population values were scaled down by dividing them by 10 for better visualization.

This visualization provides valuable insights into the dataset. It shows that population density is significantly higher in areas close to the California coastline, while inland regions generally have a lower population density. The map also helps illustrate how median income varies across different geographic regions of the state.

🔍 Feature Selection

During the experimentation phase, I evaluated different feature selection strategies by training the model with various subsets of features. Although some features showed weaker individual correlations with the target variable, removing them did not improve the model's performance.

After multiple experiments, I found that using all available features produced the best results. This suggests that features with lower individual correlation may still provide valuable information when combined with other features, allowing the XGBoost model to capture more complex relationships within the data.

Based on these observations, the final model was trained using all eight features from the California Housing dataset.

⚙️ Hyperparameter Optimization

To improve the performance of the XGBoost model, RandomizedSearchCV was used to search for the optimal combination of hyperparameters.

The search was performed using 10-fold cross-validation with the R² score as the evaluation metric. Instead of testing every possible parameter combination, RandomizedSearchCV randomly samples a predefined number of combinations, making the optimization process more computationally efficient than Grid Search while still achieving strong performance.

The hyperparameters included in the search were:

• n_estimators
• learning_rate
• max_depth
• subsample
• colsample_bytree
• min_child_weight
• gamma
• reg_alpha
• reg_lambda

The optimized model achieved the following performance:

• Cross Validation R² Score: 0.8473
• Train R² Score: 0.9466
• Test R² Score: 0.8539

These results demonstrate that the tuned model generalizes well to unseen data while maintaining strong predictive performance.

Features

• MedInc
• HouseAge
• AveRooms
• AveBedrms
• Population
• AveOccup
• Latitude
• Longitude

🛠️ Technologies Used

• Python
• Scikit-learn
• XGBoost
• NumPy
• Pandas

📦 Installation

Clone the repository:

git clone https://github.com/python-is-life2022/california-housing-regression.git
cd california-housing-regression

Install the required packages:

pip install -r requirements.txt

▶️ Run the Project

Open the Jupyter Notebook or Google Colab notebook and run the cells in order.

⚙️ Model

The final model uses XGBoost Regressor with hyperparameters optimized using RandomizedSearchCV and 10-fold cross-validation.

📈 Model Performance

Metric	Score
Cross Validation (R²)	0.8473
Train R²	0.9466
Test R²	0.8539

📊 Project Workflow

1. Load the California Housing dataset
2. Explore and prepare the data
3. Split the dataset into training and testing sets
4. Build an XGBoost regression model
5. Optimize hyperparameters using RandomizedSearchCV
6. Evaluate the model using R² Score
7. Compare training and testing performance

📁 Project Structure

california-housing-regression/
│
├── california_housing_regression.ipynb
├── requirements.txt
├── README.md

📜 License

This project is intended for educational purposes.