Azure Data Architect
Role details
Job location
Tech stack
Job description
- Lead the architecture design and implementation of advanced analytics solutions using Azure Databricks Fabric The ideal candidate will have a deep understanding of big data technologies data engineering and cloud computing with a strong focus on Azure Databricks along with Strong SQL
- Work closely with business stakeholders and other IT teams to understand requirements and deliver effective solutions
- Oversee the end-to-end implementation of data solutions ensuring alignment with business requirements and best practices
- Lead the development of data pipelines and ETL processes using Azure Databricks PySpark and other relevant tools
- Integrate Azure Databricks with other Azure services eg Azure Data Lake Azure Synapse Azure Data Factory and onpremise systems
- Provide technical leadership and mentorship to the data engineering team fostering a culture of continuous learning and improvement
- Ensure proper documentation of architecture processes and data flows while ensuring compliance with security and governance standards
- Ensure best practices are followed in terms of code quality data security and scalability
- Stay updated with the latest developments in Databricks and associated technologies to drive innovation
Requirements
Do you have experience in Spark?, * Strong experience with Azure Databricks including cluster management notebook development and Delta Lake
- Proficiency in big data technologies eg Hadoop Spark and data processing frameworks eg PySpark
- Deep understanding of Azure services like Azure Data Lake Azure Synapse and Azure Data Factory
- Experience with ETLELT processes data warehousing and building data lakes
- Strong SQL skills and familiarity with NoSQL databases
- Experience with CICD pipelines and version control systems like Git
- Knowledge of cloud security best practices
Soft Skills
- Excellent communication skills with the ability to explain complex technical concepts to nontechnical stakeholders
- Strong problemsolving skills and a proactive approach to identifying and resolving issues
- Leadership skills with the ability to manage and mentor a team of data engineers
Nice to have Skills
- Power BI for dashboarding and reporting
- Microsoft Fabric for analytics and integration tasks
- Spark Streaming for processing realtime data streams
- Familiarity with Azure Resource Manager ARM templates for infrastructure as code IaC practices, Mandatory Skills : Microsoft Azure, Azure Databricks, BigData, SQL, Azure Data lake, Azure Data Factory