I Build Data Systems That Scale
I design the data infrastructure, pipelines, and backend systems that transform raw operational data into decision-making platforms.
How to Build an AI-Powered Knowledge Base for Your Organization (Using RAG) Most organizations already have knowledge. it's just buried in PDFs, policies, and documents that no one wants to read. What if instead of searching through files, you could just ask a question and get an answer instantly? That’s…
Continue reading...Medallion Architecture Imagine a startup in its early stages, relying on a few custom Python ETL pipelines to move and process data. At this point, the system works and the data is manageable, and the pipelines are relatively simple to maintain. However, as the business grows, data begins to flow…
Continue reading...The problem most startup face In the early stage most startup have basic pipelines designed. This works and feels good when But when the startup starts to grow in a certain rate What started as a few pipelines becomes unmanageable system infrastructure. Platforms like Databricks helps startups scale by providing a…
Continue reading...I see too many Data Engineers manually writing the same SELECT, LEFT JOIN, and WHERE clauses for every new pipeline. If you’re copy-pasting SQL, you aren’t engineering; you’re transcribing. I’ve shifted my development workflow to a Metadata-Driven Approach using Jinja2. By decoupling the Business Logic (JSON/YAML) from the Execution Logic…
Continue reading...How I Reclaimed 90GB of "System Data" on my Mac (The Data Engineer Way) 💻🚀 If you're a developer or data engineer on macOS, you know the "System Data" nightmare: a massive, mysterious 100GB+ bar that you can't click on or easily clean. I just dropped my System Data from…
Continue reading...