About the Job
AI71, a pioneering AI company launched by Abu Dhabi's Advanced Technology Research Council (ATRC) and VentureOne, stands at the forefront of AI innovation. Leveraging the top-ranked Falcon AI models from the Technology Innovation Institute, we focus on multi-domain advancements in sectors such as medicine, education, and law. Our mission is to transform innovation into impactful solutions.
We are looking for a Senior LLM Engineer to lead the end-to-end development, optimization, and deployment of large language models. In this role, you will tackle challenging problems at the intersection of deep learning, natural language processing, and distributed computing.
Job Description
As a Senior LLM Engineer, you will:
- Analyze large and complex datasets to extract meaningful insights and inform data-driven decision-making.
- Develop, train, and deploy predictive models to enhance the capabilities of our AI solutions.
- Collaborate with cross-functional teams to translate business objectives into actionable data science tasks.
- Design and implement advanced LLM architectures, including transformer-based models.
- Develop novel attention mechanisms and positional encoding schemes.
- Experiment with model scaling techniques and efficient architectures (e.g., MoE, sparse transformers).
- Continuously evaluate and improve existing models based on real-world performance and evolving business needs.
- Implement and optimize distributed training pipelines for large-scale models.
- Develop strategies for efficient fine-tuning, including parameter-efficient techniques (e.g., LoRA, prefix tuning).
- Apply advanced optimization techniques such as mixed-precision training and gradient accumulation.
- Optimize models for inference, incorporating quantization and pruning techniques.
- Implement efficient solutions for real-time inference and develop strategies for model compression and knowledge distillation.
- Develop task-specific algorithms for applications such as text classification, named entity recognition, and question-answering.
- Work with MLOps teams to design and maintain training and serving infrastructure.
Qualifications
- 5+ years of experience in deep learning and NLP, focusing on large language models.
- Master's or Ph.D. in Data Science, Statistics, Computer Science, or a related field.
- Expert-level proficiency in Python and at least one deep learning framework (PyTorch, TensorFlow, or JAX).
- Strong understanding of transformer architectures and attention mechanisms.
- Experience with distributed training frameworks (e.g., DeepSpeed, Megatron-LM).
- Proficiency in optimizing model performance with techniques like mixed-precision training and gradient checkpointing.
- Understanding of NLP algorithms such as tokenization, parsing, and semantic analysis.
- Familiarity with both SQL and NoSQL databases for managing training data and model artifacts.
Why Join AI71?
- Proven performance of our large language models.
- Strong traction and adoption from the open-source community.
- Access to proprietary data for building specialized models.
- Capacity for large compute power to support our roadmap.
- Engagement with anchor clients to develop POCs and showcase our solutions.
