Data Engineer

A solid foundation and practical implementation experience are crucial for this role. Additional specialized knowledge is expressly not a requirement.

• Design, further development, and practical implementation of a scalable data lakehouse and data platform architecture for very large, heterogeneous data sets
• Developing a long-term data and storage strategy that takes into account performance, scalability, data lifecycle, costs, availability, and operational requirements
• Designing suitable storage and access concepts for large volumes of image, video, sensor, geospatial, and other unstructured and semi-structured data, including metadata, partitioning, indexing, compression, and archiving
• Building and operating robust batch and streaming pipelines for the ingestion, processing, transformation, and delivery of data from heterogeneous sources
• Co-design and implementation of a semantic data model or ontology that enables data objects, entities, relationships, and context to be described and linked across systems
• Creating a resilient data foundation for machine learning, AI-based applications, and autonomous systems, including traceable data provenance and reproducible training and validation datasets
• Ensuring data lineage, data versioning, data quality, schema management, and appropriate governance and access concepts throughout the entire data lifecycle
• Development and operation of the data platform in large, distributed Kubernetes-based infrastructures, as well as integration into Infrastructure-as-Code and CI/CD/DevSecOps processes
• Analyzing and optimizing data layouts, data locality, compute proximity, and data movements for data-intensive applications
• Evaluation of suitable open-source and commercial technologies, as well as implementation ranging from proofs of concept to production-ready, scalable solutions
• Close collaboration with AI/ML, software, platform, and cybersecurity teams on architectural decisions, interfaces, data products, and operational implementation

• A degree in computer science, data engineering, software engineering, business informatics, or a comparable practical qualification
• Several years of hands-on experience in data engineering and in building or further developing distributed data platforms
• A very good understanding of modern data lake/lakehouse architectures, object storage, and the fundamental trade-offs between storage, compute, data movement, performance, and costs
• Demonstrable hands-on experience with data pipelines, data modeling, SQL, and at least one relevant programming language, preferably Python
• Solid knowledge of metadata, data quality, schema management, data lineage, and reproducible processing
• Practical understanding of Linux, containers, and Kubernetes, as well as the ability to independently implement, operate, and analyze data components in a distributed platform
• Ability to link architectural decisions to practical implementation and to systematically investigate problems down to their technical root causes
• A structured, team-oriented approach to work, as well as strong written and spoken English skills
• Experience with very large volumes of image, video, sensor, or geodata
• Knowledge of ontologies, knowledge graphs, semantic data models, entity resolution, or comparable approaches
• Experience with technologies for distributed data processing and streaming, e.g., Spark, Flink, Trino, Kafka, or comparable solutions
• Experience with open Lakehouse table formats or comparable technologies, such as Apache Iceberg, Delta Lake, or Apache Hudi
• Experience in provisioning and versioning data for machine learning, MLOps, AI, or autonomy workloads
• Knowledge of Infrastructure as Code, GitOps, or automated platform and deployment processes
• Experience with highly available, on-premises, edge, or intermittently disconnected data platforms