About the Job:
As a Senior Data Engineer, you will play a crucial role in developing scalable, robust, and efficient data pipelines to support enterprise-grade generative AI products and intelligent agents. You will collaborate with cross-functional teams to process and structure data that powers our AI models, ensuring seamless integration and delivery of insights to enterprise clients.
About Us:
AI71 is a pioneering AI company launched by Abu Dhabi's Advanced Technology Research Council (ATRC) and VentureOne. We are at the forefront of AI innovation, utilizing the top-ranked Falcon AI models from the Technology Innovation Institute. Our mission targets multi-domain advancements, initially focusing on the medical, education, and legal sectors. We specialize in transforming innovation into tangible impact.
Key Responsibilities:
- Data Infrastructure Development: Design, implement, and maintain scalable data pipelines for training and fine-tuning generative AI models. Build and optimize data architectures for extracting, transforming, and loading (ETL/ELT) processes, especially for high-dimensional and unstructured datasets.
- Data Integration and Management: Integrate enterprise data from diverse sources, including APIs, cloud-based storage, and proprietary databases while ensuring compliance with security and privacy standards. Develop and maintain large-scale data warehouses and data lakes to support model training and analytics.
- Collaboration: Partner with data scientists and AI researchers to define data requirements for model development and evaluation. Collaborate with product managers and engineers to align data solutions with business and product goals.
- Performance and Optimization: Monitor and optimize data workflows to handle large-scale, real-time data streams used by AI agents. Ensure data systems are scalable, secure, and cost-effective for enterprise-grade applications.
- Data Quality and Compliance: Implement systems to guarantee the quality, accuracy, and consistency of training data. Ensure compliance with enterprise data governance policies, privacy regulations (e.g., GDPR, CCPA), and AI ethics guidelines.
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- 6+ years of proven experience in building and maintaining large-scale data pipelines and architectures.
- Proficiency with programming languages such as Python, Java, or Scala.
- Expertise in SQL and experience with distributed data processing tools like Apache Spark, Hadoop, or Kafka.
- Familiarity with cloud platforms (AWS, GCP, or Azure) and their data-related services (e.g., S3, BigQuery, or Azure Data Lake).
- Experience working with unstructured data such as text, images, or videos.
Preferred Qualifications:
- Knowledge of MLOps and AI lifecycle workflows.
- Experience with generative AI technologies and frameworks (e.g., Hugging Face, OpenAI API).
- Familiarity with enterprise data tools like Snowflake or Databricks.
- Understanding of natural language processing (NLP) concepts and techniques.
Why AI71:
- Work on cutting-edge AI applications with a talented and passionate team.
- Opportunity to innovate and solve real-world challenges using AI.
- Competitive compensation, benefits, and career growth opportunities.
- Flexible working environment with the latest tools and technologies.
AI71 is an equal opportunity employer. We embrace diversity and are committed to creating an inclusive environment for all employees.
