Azure Data Platform Developer
Posted on May 17, 2019 by Neem Consulting
- Min of 6 to 8 year of overall IT experience
- Working knowledge of Data Warehousing concepts and design (incl. ETL, Datamodelling & visualization)
- Should have executed at least one end to end azure data lake project (preferably with 1 to 2 years of exp in Azure data platform)
- Working knowledge of Azure DevOps/VSTS/GitHub, CLI, Storage Explorer and SSMS (SQL server management studio)
- Good communication and interpersonal skills
- Work independently with different stakeholder & 3rd party teams to ensure project objectives and goals are met
- Experience in planning, estimating and delivering projects in Agile Scrum Model
You will be expected to know below Azure Components (ADF V2, ADLS, SQL DB & SQL DWH) to a proficient level
- Azure Data FactoryV2 - datasets & linked services, pipeline development, execution & triggers and integration runtimes.
- Azure Storage Services (Blob & ADLS Gen1/Gen2)- creation use of azure storage, access policies and associated keys, encryption techniques for sharing etc.
- SQLDW and SQL DB - Write, modify, tune and debug queries, stored procedures, views, indexes, user-defined functions, and other database entities. Creation of external tables using polybase.
- Azure Key Vault, Service Principle, Power Shell etc.
You will be expected to perform below activities
- Design, implement, and maintain data solutions from data collection and curation.
- Model and develop databases and data structures to satisfy business processes and technical requirements.
- Identify complex data patterns and inconsistencies through data analysis
- Design and Implementation of security principles and access management through AD group roles
In addition to the above, knowledge and skills are preferable in ADLA (U-SQL, C#) and Databricks (Spark SQL, SCALA/Python, Spark Architecture etc.)
- Design, develop and implement U-SQL based activities/jobs using Azure Data Lake Analytics
- Setting up Databricks Cluster and automation of on demand clusters, Scaling Databricks workflows, Integration of ADF data pipelines with ADB
- Knowledge of Performance tuning to define Azure code artefacts for optimal performance