Senior Data Engineer

As a Senior Data Engineer, you will play a crucial role in designing data-driven solutions that align with organizational goals within the entertainment and development industry.

Key Responsibilities:

• Understand, analyze, and document business requirements to create effective data solutions.
• Utilize deep expertise in ETL methodologies, Big Data, and Data Warehousing principles, including architecture design, data modeling, and performance optimization.
• Design, develop, test, optimize, and deploy ETL pipelines using Azure Data Factory (ADF), Azure Databricks, and Azure Data Lake Storage.
• Implement and maintain robust ETL code and stored procedures using SQL, Python, and Spark.
• Administer and maintain cloud-based data platforms, ensuring data security, performance, and availability.
• Develop and manage Spark jobs for processing large-scale datasets.
• Apply best practices for CI/CD pipelines in an Agile development environment using Azure DevOps and GitHub.
• Proactively monitor, troubleshoot, and maintain ETL components to ensure data quality and system stability.
• Deploy Machine Learning (ML) models in the cloud using Azure Machine Learning and design APIs for data flow integration.
• Lead, mentor, and support team members in data engineering activities.

Work Conditions:

• Contract: 3 months, renewable
• On-site
• Notice period: Immediately available or a maximum notice of 30 days allowed

Qualifications: Essential:

• 5+ years of experience in Data Engineering.
• 5+ years of ETL development experience using Talend/Azure Data Services.
• 5+ years of SQL query development experience.
• 3+ years of experience in Python coding.
• 3+ years of experience with Azure or any cloud solutions.

Desirable:

• 3+ years of experience with Azure Databricks or cloud data warehouses like Snowflake or Redshift.
• 2+ years of experience with Azure Data Factory.

Education: Essential:

• Bachelor's Degree in Computer Science, Engineering, or a related field.

Desirable:

• Advanced certification in Azure Data Engineering (Azure Data Factory, Azure Machine Learning).
• Certification in Azure Databricks or any cloud warehouse.
• Any certification in data science is a plus.

Language Requirements: Not specified.