Sr Data Engineer | Expert Level | Azure, Databricks, Medallion Architecture

SUMMARY

Senior Data Engineer with 6+ years of expertise in architecting high-performance, cloud-native data ecosystems (Azure, Databricks, Snowflake). Specialist in Medallion Lakehouse architectures and GenAI-driven automation, consistently reducing infrastructure overhead by up to 50%. Currently pursuing an M.Tech in AI/ML at BITS Pilani to bridge the gap between Big Data engineering and production-grade AI. Targeting Lead or Senior Data Engineering roles in the UAE to drive large-scale digital transformation and cost-efficient data strategy.

Technical Core Competencies

• Platforms: Microsoft Fabric, Azure (Synapse, ADF), Databricks, Snowflake, GCP.
• Architecture: Medallion Lakehouse, Star Schema, OBT, Multi-tenant Governance.
• AI & Automation: LLM-integrated Microservices (FastAPI), Multimodal OCR (Gemini), PySpark.
• Governance & DevOps: Unity Catalog, RLS, Delta Sharing, CI/CD, Docker.

WORK EXPERIENCE

Data Engineer | Pythian | Dec 2024 – Present

• Built a Dockerized FastAPI microservice integrating LLMs to automate pipeline code refactoring, increasing component reuse by 40% and reducing manual migration effort by 25%
• Deployed an 87%-accurate Gemini Multimodal OCR solution for logistics automation, reducing manual freight bill verification time by 70% and improving truck turnaround
• Designed scalable data pipelines using Databricks and Delta Lake within a medallion architecture, reducing data redundancy by 20%
• Enforced 100% data compliance using Unity Catalog, RLS, and Delta Sharing to establish secure, multi-tenant governance frameworks
• Designed end-to-end Microsoft Fabric data solutions for a pharma client, integrating Great Expectations-based Data Quality Framework for automated validation and data lineage tracking, while leveraging metadata-driven processing to reduce pipeline execution time by 45%
• Engineered high-performance Star Schema and OBT models, accelerating query response times by 3x and enabling self-serve analytics for 15+ business stakeholders

Data Engineer | Coresight Research Services Pvt. Ltd. | Sep 2023 – Dec 2024

• Developed end-to-end Python web scraping and PySpark pipelines on Databricks to ingest and harmonize 1M+ daily records from 300+ retailer sources with a 99% success rate and strict schema consistency
• Streamlined end-to-end data validation workflows and built Streamlit dashboards, reducing manual preparation by 60% and delivering real-time executive reporting to 10+ stakeholders
• Engineered real-time alerting and data quality monitoring within Databricks batch workflows, cutting mean time-to-detect (MTTD) pipeline failures by 50% and ensuring proactive SLA compliance across all production pipelines

Senior Data Engineer (IT Analyst) | Merilytics Pvt. Ltd., Hyderabad | Aug 2020 – Sep 2023

• Promoted 4 times in 3 years (Technical Associate → Senior Analyst) for consistent high performance in architecting cloud-native data warehouses and reporting ecosystems
• Architected enterprise Azure data warehouses with advanced data masking for Private Equity clients, ensuring 100% privacy compliance across 5+ international regions
• Delivered Agile Snowflake and Power Apps solutions for European clients, reducing data latency by 45% for 150+ users via Tableau and Power BI reporting ecosystems
• Designed cost-effective Azure Synapse solutions for a US healthcare client integrating 20+ sources, reducing infrastructure costs by 30% and accelerating reporting cycles by 50%
• Built high-performance Databricks and Snowflake analytics for a US video streaming client, improving query speeds by 4x and supporting a 200% increase in concurrent workloads

CERTIFICATIONS

• Databricks Certified Data Engineer Professional
• Google Cloud Certified – Professional Data Engineer
• Microsoft Certified: Fabric Data Engineer Associate (DP-700)
• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305)
• Microsoft Certified: Azure Administrator Associate (AZ-104)