Senior Data Engineer, Platform

Let's start with the role

At Kaizen, our commitment to the unparalleled value of data drives every decision we make. Expanding from our robust Lake House architecture, we envision a transformative shift towards a Data Mesh architecture. This approach emphasizes treating data as a product with well-defined contracts, distributed ownership, and domain-oriented design.

As we aim to further enhance our data ecosystem, we’re focusing keenly on data democratization, governance and observability. The ideal scenario? To have every piece of data perfectly accessible, monitored and governed.

Our data tribe is well-prepared to lead this transition, championing the design and expansion of next-generation Data Products. By joining our crew, you’ll be part of an initiative that positions Kaizen at the forefront of data-driven innovation.

Our Tech Stack includes Spark, Python (or Scala), Databricks, Delta, Apache Airflow, Gitlab, Grafana and more.

• Contribute to the development of a scalable and secure Data Platform aligned with the Data Mesh paradigm;
• Design, implement and improve tools and services for orchestration, observability, data governance and data quality to high engineering standards;
• Deploy and manage our products using CI/CD best practices following the ‘you build it, you run it’ culture;
• Stay current on emerging technologies, methodologies and industry best practices to ensure continuous improvement of the data platform; 
• Champion a culture of documentation, where the technical documentation lives along with our products and ensures they are easy to use and maintain.

What you’ll bring

• Software Engineering: Strong software engineering background, skilled in designing, writing and deploying production-grade code that aligns with our data-first vision.
• DevOps and DataOps Familiarity: Practical experience in DevOps, with a preference for candidates familiar with DataOps practices, ensuring continuous integration, delivery, and efficient data operations.
• Data Pipeline Expertise: Hands-on experience with data pipeline orchestration frameworks, ensuring smooth data flow and integration across different sources and platforms.
• Spark Proficiency: Demonstrated knowledge in Apache Spark, with the capability to write and optimize effective code for data processing tasks.
• Technology Evaluation: A curious mindset, with the ability to understand, evaluate and potentially integrate new technologies that can drive Kaizen's data-driven approach forward.
• Data Governance and Security: An understanding of the importance of data governance, with a focus on ensuring that data assets are protected, accessible, and compliant with regulatory standards.