Data Engineer - Databricks (CON01519)

Bengaluru,Hyderabad,Gurugram

Hybrid

About Us

Affine is a leading global AI and analytics company that has earned the trust of major organizations across industries for transforming businesses through data-driven insights and cutting-edge Generative AI technologies.

Founded in 2011, Affine has become a trusted partner for major organizations across various industries, delivering bespoke data solutions and products. Our team excels in strategic consulting, advanced analytics, data engineering and governance, MLOps, and AI-powered solutions, with a strong emphasis on Generative AI applications that foster ingenuity and pioneer advancements.

We offer comprehensive data visualization and adoption advisory services, ensuring that our clients not only keep pace but lead in their respective fields. For more insights into our Generative AI capabilities and other services, please visit our website https://affine.ai.

We offer exciting career opportunities for professionals seeking to make a significant impact. As an Affine employee, you'll be part of a dynamic and inclusive work environment that values collaboration, creativity, and continuous learning. At Affine, we foster a culture of collaboration, you also work alongside some of the brightest minds in the industry, solve complex challenges, and make a lasting impact on the world.

Job Description

Job Description:

We are seeking a skilled Data Engineer to design, build, and maintain scalable data solutions on the Azure platform. The ideal candidate will play a key role in developing robust data pipelines, ensuring data quality, and contributing to enterprise-wide analytics initiatives.

Key Responsibilities:

Design and develop scalable data pipelines using Spark SQL and PySpark on Azure Databricks
Build and orchestrate ETL workflows using Azure Data Factory (ADF)
Develop and maintain a modern Lakehouse architecture leveraging ADLS and Databricks
Perform data preparation tasks including data cleaning, normalization, deduplication, and data type transformations
Collaborate with DevOps teams to deploy and manage solutions in production environments
Monitor data pipelines, identify issues, and implement corrective actions, including root cause analysis and resolution
Actively contribute as part of the global Analytics team, delivering data-driven insights and solutions
Collaborate with Data Science and Business Intelligence teams to share best practices and drive innovation
Lead data engineering projects and support cross-functional initiatives led by other team members
Apply change management practices, including documentation, communication, and training, to support system upgrades and data migrations

Job Requirement

Must-Have Skills

Minimum 6 years of experience in Data Engineering.
Strong experience with Azure ecosystem (Azure Databricks, Azure Data Factory, ADLS)
Proficiency in PySpark and Spark SQL
Solid understanding of ETL processes and data pipeline development
Advanced SQL skills

Be a part of Affine's Talent Pool

If you are unable to find relevant job opportunity above, join our Talent Pool by simply submitting your resume. Our Talent Acquisition team will contact you as soon as they find a great match for your profile and discuss exciting opportunities with you.

More About Us...