Data Engineer

Job Description

Our innovative solutions, including password-free authentication and identity management, are shaping the future of secure corporate ecosystems and tackling the challenges of the post-quantum era.

We are seeking a talented and motivated Data Engineer to join our fast-growing ML-team. The ideal candidate has strong experience with distributed data processing, NoSQL databases, and modern data lake technologies. You will play a key role in designing and maintaining scalable data pipelines that power business intelligence, analytics, and machine learning initiatives.

• Design, develop, and maintain scalable ETL/ELT pipelines using Apache Spark and/or similar solutions.
• Manage and optimize large datasets stored in Apache Iceberg tables.
• Design, build, and optimize data infrastructure.
• Integrate data from various sources including MongoDB, APIs, and flat files.
• Optimize data processing workflows for batch and real-time analytics.
• Implement and enforce data governance, data quality, and security best practices.
• Troubleshoot and resolve data-related issues in dev/production environments.
• Work closely with data scientists, analysts, and product teams to understand data needs and deliver efficient solutions.
• Monitor pipeline performance and troubleshoot data quality issues.

• Bachelor's degree in Computer Science, Engineering, or related field.
• 3+ years of experience in data engineering or related roles.
• Strong hands-on experience with: Apache Spark (PySpark, Scala, or Java); MongoDB (data modeling, aggregation framework, replication); Apache Iceberg or other data lake table formats (Delta Lake, Hudi)
• Proficient in SQL and at least one programming language (Python, Scala, or Java).
• Experience with data pipeline orchestration tools (e.g., Airflow).
• Understanding of data warehousing concepts (star schema, partitioning).
• Familiarity with cloud platforms such as AWS or Azure.

• Experience with stream processing (e.g., Kafka, Flink).
• Experience with Kubernetes for containerized data workloads.
• Familiarity with CI/CD practices and infrastructure as a code (e.g., Terraform).

Join QuantumGate as we pioneer innovative solutions to secure the future of digital technology and protect tomorrow's digital society.