Senior Big Data Engineer (AI Enthusiast)

Company Description

At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for jobseekers and companies around the world to find a perfect match in fair and equitable way. With over 20 brands across 30+ countries, we strive for fair and unbiased hiring. 

Join our team of 4,000+ employees and be part of reshaping the labour market and becoming the world’s leading job platform. 

Job Description

As a Senior Big Data Engineer and AI Enthusiast, you will join the Marketplace Enablement department, working inside a cross-functional team to design and implement cutting-edge, high-performance central streaming data products that power our recruitment AI applications, including deploying advanced machine learning models.

Your expertise will help create a more inclusive and efficient recruitment experience globally, directly supporting our mission to reshape the labour market for everyone. You will have a strong sense of ownership and proactively provide solutions before a situation deteriorates.

Your Responsibilities

• Lead the design and build of modern big data platforms and AI-powered tools that promote fair and inclusive hiring.
• Champion code quality by applying best practices like clean code, automated testing, CI/CD, observability, security, and Federated Governance.
• Design, implement, and optimize high-performance data pipelines and real-time applications using tools like Apache Kafka and Apache Spark.
• Manage and optimize complex ETL/ELT processes and data orchestration workflows.
• Continuously evaluate, improve, and scale our data infrastructure, advocating for new technologies and architectural patterns.
• Collaborate and mentor in an international, agile, cross-functional team, helping others grow and succeed.
• Contribute to building an inclusive team culture where every voice is valued.
• Work in a collaborative environment that values diverse perspectives and inclusive decision-making.

Qualifications

Your Skills & Qualifications

• Excellent communication skills in spoken and written English (B2+ level).
• Genuine passion for leveraging data and AI to solve complex problems and make a tangible impact.
• Deep expertise and proven track record in Python for building large-scale data and backend applications (5+ years professional experience).
• Strong experience with AWS cloud services (e.g., EMR, Glue, ECS with Docker, Redshift, DynamoDB, S3) and understanding of cloud-native architectures (3+ years).
• Professional experience in Big Data architecture, Data Warehousing, designing complex ETL/ELT pipelines, and data pipeline orchestration (2+ years).
• Solid experience with Apache Kafka (MSK) for real-time data ingestion and processing (1+ years).
• Familiarity with Infrastructure as Code (Terraform) and Huggingface Transformers (considered a plus).
• We value different career paths — if you have taken career breaks, switched disciplines, or balanced work with family life, we still encourage you to apply.

If you don’t meet every single requirement but are excited about this role, we still would love to see you applying :) 

Additional Information

Your benefits 

We’re a community here that cares as much about your life outside work as how you feel when you’re with us. Because your job shouldn’t take over your life, it should enrich it. Here are some of the benefits we offer: 

• 30 days of holidays + 2 extra days for Christmas and New Year's Eve    

• Hybrid working model  

• Company pension scheme  

• 24/7 Employee Assistance Programme  

• Life Assurance Cover  

• Volunteering days  

• Job bike  

• Discounted parking or job ticket  

• And you can bring your dog to the office  

Our commitment 

Equal opportunities are important to us. We believe that diversity and inclusion at The Stepstone Group are critical to our success as a global company, so we want to recruit, develop, and keep the best talent. We encourage applications from everyone, regardless of background, gender identity, sexual orientation, disability status, ethnicity, belief, age, family or parental status, and any other characteristic.