Rachit Awasthi

• Engineered robust ETL pipelines to ingest time-series data from diverse sources (SFTP, Azure Blob, Jira, Kafka) for KPI reporting, efficiently channeling data into InfluxDB via Apache Airflow DAGs. Leveraged GitHub Copilot for AI-assisted code development. • Developed regression ML model to predict city temperatures and urban design for cooler, sustainable environment. Used European Sentinal-2 and NASA’s Landsat optical satellite data as feature dataset.

• Designed and implemented ETL pipelines using AWS Glue to migrate legacy ETL processes from IBM DataStage to a modern data architecture. Leveraged Snowflake as a data warehouse, Apache Airflow as orchestration, Terraform for deployment, and integrated Starburst for optimized data transformation and analytics for SOD and intraday trades for Nvueen a TIAA Company. • As a Data Engineer, architected & built a scalable ETL tool for financial services client (BICE Bank) on GCP (DataFusion, Composer, Airflow, Functions, API Gateway, BigQuery, GCS and OCI) and used Node.js for Backend APIs. Built custom python packages for enhanced DAG reusability & maintainability by 50%. In addition, received a 4/4 customer rating and EY Spot Award for developing the solutions. • Identified API performance bottleneck and proactively implemented pagination in micro-services developed on AWS Lambda, resulting a 20% improvement in user response times for client Carrier Colts. • Lead a team of six developers to pioneer a developer experience platform built on Backstage.io, developed GenAI based automation solution leveraging Azure OpenAI custom vision and cognitive search. Successfully showcased the solution to leading financial and healthcare organizations - including Morgan Stanley, TIAA, Takeda, Vanguard and EY EMEIA, resulting in $128 million in revenue.

• Developed a robust RDBMS migration tool utilizing AWS PySpark Glue jobs to seamlessly transfer data between on-premises databases and S3 data lakes. Also, developed backend APIs using Node.js Lambda Functions and API Gateway. Presented at DXC Connect 2023, attracting attention from insurance clients such as Wilton Re, Ivari, Baloise, and Ohio Mutual. • Created and integrated ML algorithms - reconciliation, customer churn and fraud detection using AWS SageMaker. • Created the functionality to preview data results at each interim stage of data transformation from Athena and masking of personal identifiable information using AWS Comprehend.

• Built E2E a serverless cross-account dashboard migration tool for DXC Assure ETL (using AWS Lambda, Athena, Quicksight, STS, IAM). This streamlines dashboard transfers between multiple AWS accounts, boosting efficiency and data accessibility.