Data Engineer

About the Role We are looking for a Data Engineer to develop, maintain and improve the data infrastructure and backend services supporting our RE:Flex platform. The role involves working with real-time IoT data, cloud infrastructure, databases and data pipelines to ensure reliable data collection, processing, storage and delivery. You will work closely with backend developers, DevOps engineers, machine learning engineers and other teams across the business, operations and finance. Alongside hands-on development, you will contribute to technical improvements, code reviews, troubleshooting, documentation and knowledge sharing. Key Responsibilities Data Engineering and Pipelines Develop and maintain Python-based services for collecting and processing data from connected energy devices, including battery systems and meters. Build and improve real-time and batch data pipelines using MQTT, Google Cloud Pub/Sub and other messaging technologies. Integrate data from external APIs, energy market platforms and other relevant data sources. Implement data validation, error handling, retry mechanisms and safe backfilling to improve data quality and reliability. Develop and maintain data processing workflows, including aggregation and transformation of time-series data. Backend Development and Cloud Infrastructure Develop and maintain backend APIs and services for accessing real-time and historical platform data. Work with databases and storage solutions such as BigQuery, TimescaleDB, PostgreSQL and MongoDB, depending on application requirements. Develop, deploy and maintain containerised applications using Docker and Kubernetes. Contribute to cloud infrastructure improvements, primarily within Google Cloud Platform. Support integrations between data services, backend applications and other platform components. Data Quality, Monitoring and Reliability Monitor data pipelines, database performance and service reliability using tools such as Grafana and cloud monitoring services. Investigate data inconsistencies, missing records, processing failures and other production issues. Improve data processing performance, scalability and operational stability. Implement appropriate data quality checks, logging, monitoring and alerting. Support incident investigation, troubleshooting and implementation of technical fixes. Energy Market and Business Data Support data processing and integration for energy market operations, including market prices, schedules, measurements and settlement information. Develop and maintain data workflows used for operational reporting, revenue calculations and customer invoicing. Work with market operations, finance and other business teams to understand data requirements and resolve discrepancies. Help ensure data consistency and traceability across relevant systems and integrations. Technical Collaboration and Documentation Participate in technical planning, architecture discussions and code reviews. Coordinate with backend, DevOps, machine learning and other engineering teams on data-related implementations. Support onboarding and knowledge sharing with other engineers. Maintain technical documentation, data flow descriptions and operational procedures. Contribute to continuous improvements in development practices, deployment processes and platform reliability. Required Qualifications Relevant education in Computer Science, Software Engineering, Data Engineering or a related technical field, or equivalent practical experience. At least 5 years of professional experience in data engineering, backend development or a related software engineering role. Strong programming skills in Python and good knowledge of SQL. Experience developing data pipelines, backend services and REST APIs. Experience working with relational or NoSQL databases and cloud-based data processing. Familiarity with messaging systems, streaming data or asynchronous processing. Practical experience with Docker and cloud infrastructure. Understanding o