Senior Data Engineer

Duties

Own end-to-end delivery of large-scale telecom data platforms (RCE/CIQ/ROAD), including pipeline development, ingestion, metric generation, and operational stability across multiple critical datasets.

Lead platform modernisation and data quality initiatives, covering migration from legacy systems, implementation of monitoring frameworks, and ensuring reliable, scalable, and timely data delivery for business-critical use cases.

Skills

Languages: Python, PySpark, SQL, Spark SQL

Data Platforms: Spark, Trino, Iceberg

Data Engineering: ETL/ELT pipelines, batch & streaming processing

Workflow Orchestration: Airflow (DAG design, scheduling, dependency management)

Cloud & Infra: Kubernetes, IBM COS (S3 Like storage grid),

Data Quality: Monitoring, anomaly detection, data completeness analysis

DevOps: Git, CI/CD pipelines

Practices: Clean code, performance optimisation, production support