About the Role
We are seeking an experienced Data Solution Architect to design and lead the implementation of an on-prem Data Lakehouse on Cloudera CDP 7.3.1. You will define the target architecture (platform, data, integration, security), establish a Medallion model (Bronze/Silver/Gold), and implement a Lambda Architecture that blends batch (Informatica/Spark/Hive) and speed/streaming (Kafka/Hudi/Kudu/HBase and Phoenix) with a unified serving layer (Impala/Phoenix/REST). You will also architect a Rules Engine for business validations/transformations and design a comprehensive Data Governance layer (Ranger/Atlas/Knox/KMS/SDX), ensuring security, lineage, quality, and compliance.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Responsibilities
- Define and design target architecture for Data Lakehouse implementation, covering platform, data, integration, and security requirements.
- Establish a Medallion model that incorporates Bronze, Silver, and Gold layers.
- Implement a Lambda Architecture integrating batch processes with real-time streaming services.
- Develop a comprehensive Data Governance framework, ensuring data security and compliance.
- Architect a Rules Engine for applying business validations and transformations.
- Ensure adherence to industry standards and best practices in data engineering and architecture.
- Collaborate closely with stakeholders to communicate architectural plans and deliverables.
Required Qualifications (Must-Have)
- 10+ years in data architecture/engineering with 3–5+ years architecting on Cloudera (CDH/CDP) in production.
- Proven delivery of on-prem Lakehouse with Medallion and Lambda architectures.
- Sound understanding of Industry Standards and Best Practices followed in Data Engineering.
- Good understanding of Informatica DEI suite for framework/pattern design and pushdown to Spark/Hive.
- Deep practical exposure to Apache Iceberg, Apache Hudi, Apache Kudu, HBase + Phoenix, Hive 3, and Impala (workload fit, partitioning, compaction, statistics).
- Strong SQL expertise (windowing, partitioning, MERGE/UPSERT, cost-based tuning) and Impala/Phoenix performance tuning.
- Security/Governance: Kerberos, TLS, AD/LDAP, Ranger policies (masking/row-level), Atlas lineage/glossary; SDX concepts.
- Data Quality & Rules: experience designing metadata-driven rules engines and DQ frameworks for both batch and streaming.
- Solid Linux fundamentals; Git/CI/CD; strong documentation and stakeholder communication.
Preferred Qualifications
- Experience with Kafka patterns (exactly-once, schema registry, compaction) and CDC from RDBMS.
- Familiarity with Ozone, KMS/Key Trustee, and air-gapped deployments.
- SRE practices: SLOs, error budgets, RCA, chaos/drill exercises.
- Certifications: Cloudera CDP, Informatica, Security (e.g., CISSP/CCSP) a plus.
- Regulatory experience in PII/PCI/SOX/GDPR environments.
