🛡️ Network Security MLOps – Phishing URL Detection

A production-grade End-to-End MLOps System for detecting phishing URLs using machine learning, cloud services, Docker, and CI/CD automation.

🏗️ System Architecture

The system follows a modular architecture with the following components:

• Client: User-facing browser/app interface
• FastAPI Service: Inference API running in Docker on EC2
• Training Pipeline: Automated ML pipeline for data processing and model training
• Data & Storage: AWS S3 for data storage and DVC for version control
• Tracking: MLflow + DagsHub for experiment tracking
• CI/CD: GitHub Actions for automated testing, building, and deployment to AWS ECR

📁 Project Structure

network-security-mlops/
│
├── app.py
├── main.py
├── Dockerfile
├── requirements.txt
├── setup.py
├── README.md
│
├── config/
│   └── schema.yaml
│
├── .github/
│   └── workflows/
│       └── main.yml
│
├── Artifacts/
│
├── networksecurity/
│   ├── components/
│   │   ├── data_ingestion.py
│   │   ├── data_validation.py
│   │   ├── data_transformation.py
│   │   ├── model_trainer.py
│   │
│   ├── entity/
│   │   ├── config_entity.py
│   │   ├── artifact_entity.py
│   │
│   ├── pipeline/
│   │   └── training_pipeline.py
│   │
│   ├── utils/
│   │   ├── main_utils.py
│   │   ├── ml_utils.py
│   │
│   ├── cloud/
│   │   └── s3_syncer.py
│   │
│   ├── logging/
│   │   └── logger.py
│   │
│   ├── exception/
│   │   └── exception.py
│   │
│   └── constant/
│       └── training_pipeline.py
│
└── dvc.yaml

🔄 ML Pipeline Overview

The training pipeline consists of four main stages:

1️⃣ Data Ingestion

• Downloads dataset from source
• Splits data into training and testing sets
• Stores artifacts locally and on AWS S3

2️⃣ Data Validation

• Performs schema validation checks
• Identifies missing values
• Generates validation reports

3️⃣ Data Transformation

• Applies StandardScaler normalization
• Encodes categorical variables
• Saves preprocessor as joblib file

4️⃣ Model Training

• Trains multiple ML algorithms
• Performs hyperparameter tuning with GridSearchCV
• Automatically selects the best model
• Logs experiments to MLflow
• Saves model artifacts to AWS S3

⚙️ API (FastAPI)

Start Locally

uvicorn app:app --reload --port 8080

Example Request

{
  "url_length": 50,
  "https": 1,
  "domain_age": 5,
  "has_special_chars": 0
}

Example Response

{
  "prediction": 0,
  "label": "Safe URL"
}

🐳 Docker Support

Build Image

docker build -t networksecurity .

Run Container

docker run -p 8080:8080 networksecurity

☁️ AWS Deployment (CI/CD)

The system features a fully automated deployment pipeline:

1. Push code to main branch
2. CI Stage: Runs linting and tests
3. Build Stage: Docker image is built
4. Push Stage: Image pushed to AWS ECR
5. EC2 Deployment: Self-hosted runner pulls latest image
   • Stops old container
   • Runs new container
   • API becomes live instantly

🧰 Technology Stack

ML & Data

• Python
• NumPy
• Pandas
• Scikit-Learn
• GridSearchCV
• Joblib

MLOps

• MLflow
• DVC
• Docker
• GitHub Actions

Cloud

• AWS S3
• AWS ECR
• AWS EC2

API

• FastAPI
• Uvicorn

🧪 Running the Training Pipeline

To execute the complete training pipeline:

python main.py

👨‍💻 Author

Hamza Khan

📧 Email: hamzahere52@gmail.com

🔗 GitHub: https://github.com/Hamzakhan001