Kafka Developer

Role: Kafka Developer

Location: Melbourne, VIC

Experience: 8+ years

Job Description

• Develop custom integration code using Java / Spring Boot for source system connectivity where managed connectors are not applicable.
• Build and maintain Kafka producers / consumers, topic integration logic, and event-driven services on Confluent Cloud / NGIP.
• Develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
• Implement integration with Schema Registry, topic schemas, and message validation controls.
• Support HTTP/API-based integrations through Kong Gateway for NGIP to target application communication.
• Configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling.
• Perform unit / component testing for custom code, Flink jobs, and interface logic.
• Support SIT, pipeline testing, defect fixing, replay, and root-cause analysis during test execution.
• Support E2E testing with downstream teams and resolve integration issues across connectors, transformations, APIs, and Kafka topics.
• Work with GitLab CI/CD pipelines for build, deployment, regression automation, and release support.
• Desired Competencies (Technical/Behavioral Competency)
• Must-Have Develop custom integration code using Java / Spring Boot for source system connectivity where managed connectors are not applicable.
• Build and maintain Kafka producers / consumers, topic integration logic, and event-driven services on Confluent Cloud / NGIP.
• Develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
• Implement integration with Schema Registry, topic schemas, and message validation controls.
• Support HTTP/API-based integrations through Kong Gateway for NGIP to target application communication.
• Configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling.
• Perform unit / component testing for custom code, Flink jobs, and interface logic.
• Support SIT, pipeline testing, defect fixing, replay, and root-cause analysis during test execution.
• Support E2E testing with downstream teams and resolve integration issues across connectors, transformations, APIs, and Kafka topics.
• Work with GitLab CI/CD pipelines for build, deployment, regression automation, and release support.

Skills

• Java, Spring Boot
• Apache Kafka / Confluent Cloud
• Kafka Connect
• Flink SQL / stream processing
• Schema Registry
• REST / API integration
• Kong Gateway
• GitLab CI/CD
• Unit testing, SIT support, E2E support

Good-to-Have

• Confluent Certified Developer/Administrator
• Experience with stream governance, RBAC, and tiered storage
• Background in data engineering, ETL, or microservices architecture
• Familiarity with traditional messaging systems and data warehouses
• Experience in Agile methodologies and DevOps practices.