Data Engineer
You will work with AWS (Lambda, S3, SQS, SNS, Step Functions, DynamoDB, CloudWatch, EMR, MSK/Kafka, Redshift), PySpark, Python, Apache Iceberg, Parquet, and XML/ASN.1 parsers. The role builds scalable cloud-based data pipelines for telecom network performance data, focusing on high-throughput ingestion, ETL/ELT workflows, schema management, and optimizing delivery to Redshift and ClickHouse.
Free Tailor for ATS: 10/10 runs left
We're looking for a Senior AWS Data Engineer with strong expertise in building scalable cloud-based data pipelines for telecom network performance data.
Key Skills:
AWS (Lambda, S3, SQS, SNS, Step Functions, DynamoDB, CloudWatch, EMR, MSK/Kafka, Redshift)
PySpark, Python, Apache Iceberg, Parquet
XML & ASN.1 parser development
Real-time and batch data processing
Schema management, versioning, upserts & metadata handling
Error handling, retries, DLQ implementation
Telecom PM counters (15/60-minute aggregation)
ClickHouse experience is a plus
What You'll Do:
Build high-throughput data ingestion and parsing pipelines.
Develop scalable ETL/ELT workflows using PySpark on AWS EMR.
Design dynamic schema management aligned with telecom OEM specifications.
Implement robust monitoring, logging, and automated pipeline orchestration.
Optimize data delivery to Amazon Redshift and ClickHouse for analytics.
Preferred Experience: 8+ years in Data Engineering with hands-on AWS expertise and telecom data processing experience.