Responsibilities
- Build and optimize databricks ETL/ELT pipelines for feature extraction, transformation, and loading from structured and unstructured data sources.
- Collaborate with ML engineers, data scientists, and software developers to deliver reliable, reusable, and versioned feature sets.
- Implement CI/CD pipelines, testing frameworks, and observability for data workflows.
- Develop feature stores, metadata tracking, and lineage tools to support data Ops.
- Ensure data quality, governance, and compliance across all data assets.
- Optimize performance and cost-efficiency of Databricks clusters and jobs for data workloads.
- Contribute to the architecture and design of the data platform and feature engineering framework.
- Strong proficiency in Python and relevant scripting languages, with experience in software development and scripting for Databricks.
- Expertise with libraries and frameworks (e.g., Pandas, Numpy, Scikit-Learn, TensorFlow, PyTorch, Databricks, MLFlow, dvc, dbt) and the ability to select the right tools for the use case.
- Experience building inference endpoints (APIs) and managing compute architecture for efficient model inference and data handling.
- Very good Azure and data & AI technology skills - specifically: Databricks / Python, Azure Datalake Store, Azure AI Search.
- Experience with DevOps practices, including Git, CI/CD using tools such as Azure Pipelines, or similar.
- Several years of experience in machine learning, data science, or a related field, with a strong understanding of statistics and data analysis.
- Experience with Azure especially with data and ML services, containerization (Docker.
Preferred qualifications
- Advanced Degree: Master’s degree in Computer Science, Engineering, Mathematics, with 5+ years of ML implementation experience or Ph.D. with 2+ years of hands-on ML Project experience.
- Experience with Big Data: Strong proficiency with big data technologies such as Azure Databricks and Spark.
- Leadership Experience: Previous experience leading a team of data scientists or engineers.
Benefits
- Competitive employee benefits, including comprehensive health insurance, dental and sports coverage, and opportunities for certified training.
- Flexibility in work arrangements, including home office options and flexible working hours.
- A positive, team-oriented environment that fosters mutual trust, creativity, and initiative.
- Opportunities for career growth within a global, innovative framework.
- A diverse, multicultural workplace with a strong emphasis on team collaboration and professional development.
Tags & focus areas
Used for matching and alerts on DevFound Fulltime Ai Machine Learning Data Science Pytorch Tensorflow Data Engineer Ai Engineer