data-pipeline-automation

End-to-end data pipeline transforming Olist e-commerce data through Azure cloud services. Implements medallion architecture (Bronze-Silver-Gold) with multi-source ingestion, Spark-based processing, and OLTP-to-OLAP optimization for analytics-ready datasets.

Pulls Meta ad data automatically, syncs to Google Sheets, delivers a live analytics dashboard with rule-based insights, decision engine, and Telegram alerts.

Standalone music intelligence layer that transforms your Spotify listening history into structured, filterable playlists with the goal of giving you explicit control over your taste

A complete reference implementation of a local-first ecosystem for AI-powered analytics. This repository contains the source code for the SBDK.dev website, the central hub for the SBDK suite of open-source tools.

A full-stack financial application featuring real-time forex rates and machine learning-powered price predictions across multiple currency pairs. Built with automated data pipelines and Prophet time series forecasting.

A toml based data pipeline creation tool/framework

A unified ETL product for turning repetitive file-based integration work into repeatable, reliable business flows

Data Engineering project implementing a fully automated Medallion Architecture (Bronze, Silver, Gold) using Snowpipe, Streams, Tasks, and SQL to build scalable cloud-native data pipelines for analytics.

An end-to-end, multi-container solution for collecting and visualizing stock market data

This repository contains scripts to build and process a Real-Time GDP (RTD) dataset for Peru, focusing on extracting, cleaning, and analyzing GDP revisions from the BCRP's Weekly Reports. Future updates will include econometric models and visualizations.

VizFlow is a VS Code extension that brings a visual workflow builder, RBQL querying, database connectors (MongoDB, MySQL, PostgreSQL), HTTP/REST integration, file transformations, and built-in scheduling — all running locally on your machine. Build repeatable data pipelines with 35 activity types, no cloud account or vendor lock-in required.

A practical Rust workspace template for automated CI, conventional commits, cross-platform development, data pipelines, and machine-learning projects.

End-to-end data pipeline integrating manufacturing and financial data. Demonstrates ETL processes, a PostgreSQL data warehouse with a star schema, and business intelligence capabilities using PowerBI. Built with Python, Apache Airflow, and Docker.

The monorepo for microservices and microfrontends used in data augmentation.

Production-grade cold email outreach infrastructure with strict data normalization, deterministic lead scoring, and fault-tolerant multi-stage campaigns.

End-to-End PV Monitoring & Streaming Pipeline with Delta Lake

Python data pipeline project that tracks Amazon product prices, stores historical scrape results, and sends automated price-drop alerts.

Rule-based multi-factor screening and scoring engine for ~5,336 US equities using Google Apps Script, Finnhub API, Google Sheets.

Automated data pipeline scraping fuel prices daily, apply transformations and load the data to a PostgreSQL database.

Pipeline automatisé de suivi mensuel des demandeurs d'emploi, téléchargement, nettoyage et rapport, données DARES/France Travail