RUDRAKSH GUPTA

Leading end-to-end data modeling and pipeline development for E-commerce operations, enhancing data fetching efficiency and strategic decision-making.

• Led an end-to-end E-commerce data modeling project, foundational to Supply Chain operations, encompassing critical service data (orders, tax, payments, offers, payment gateway systems).
• Developed an in-house feature to extract, process, and load DynamoDB data into Iceberg tables, reducing data fetching costs by 99% through a plug-and-play solution.
• Engineered a generic, config-driven codebase for Spark-based ETL batch processing, enabling seamless data migration from Snowflake to Iceberg via configuration entries.
• Identified data-driven logic for user interaction analysis, optimizing marketing spend on Meta platforms by 14%.

Directed critical data engineering pipeline migrations from monolith to microservices, ensuring seamless data integration and rapid insight delivery.

• Led a high-impact migration project, successfully transitioning data engineering pipelines from a monolith to a microservices architecture.
• Spearheaded data integration across legacy monolith and modern microservices systems, ensuring seamless interoperability and data flow.
• Implemented robust data pipelines to deliver valuable business insights to business stakeholders within 30 minutes from multiple sources.
• Prioritized and enforced data quality through rigorous validation and cleansing procedures, maintaining data integrity and reliability during migration.
• Orchestrated a smooth architectural transition, minimizing business disruptions and ensuring uninterrupted access to critical data.

Designed and optimized real-time and batch data pipelines, leveraging Kafka, Spark, and Delta Lake to enhance data delivery, efficiency, and quality.

• Designed and maintained performance-tuned real-time data pipelines using Kafka, Spark Streaming, and Delta Lake, delivering data in ~45 seconds and reducing computing resources by 25%.
• Architected versatile data processing frameworks with Apache Spark, enabling pipeline creation with 90% less effort.
• Applied advanced data modeling to optimize reporting and analytics, reducing compute costs by 40% and runtime by 30%.
• Engineered robust data cleansing pipelines for unstructured data using advanced Spark functionality, ensuring scalability and fault tolerance.
• Led a decentralized data project, managing end-to-end requirements gathering, planning, and execution.

Monitored, debugged, and optimized production data pipelines, ensuring system stability and enhancing code observability in distributed environments.

• Monitored production data pipelines, proactively addressing failures and conducting performance analysis to resolve bottlenecks.
• Investigated and resolved real-time code defects and production issues, ensuring seamless data processing workflows.
• Collaborated with cross-functional teams to troubleshoot complex issues across distributed systems.
• Implemented logging, error handling, and instrumentation techniques, significantly enhancing the observability and maintainability of production codebases.