Junior Data Engineer

- Design, build, and maintain ETL/ELT pipelines for large-scale data ingestion, transformation, and loading. - Develop and optimize Spark and PySpark jobs for batch and real-time data processing. - Work with AWS services (S3, Glue, Lambda, Redshift, Athena, EMR) to manage the data ecosystem. - Support the design and implementation of Data Lake and Data Warehouse architectures. - Implement data validation, partitioning, and schema management for efficient query performance. - Collaborate with data analysts and BI teams to ensure data availability and consistency. - Maintain data lineage, metadata, data quality, and governance. Implement monitoring and alerting for data pipelines. - Use Git and CI/CD tools to manage code and automate deployment of data workflows.