Senior ETL Developer with ADF

Job Description: Senior ETL Developer

Scope of Work/Responsibilities:

Scope of Work/Responsibilities

• The Senior ETL developer will be responsible for working with data owners, system owners, solutions, delivery team and managed services provider (MSP) in identifying solutions and process enhancements. Should work and collaborate with managed services provider (MSP) in ensuring smooth and undisruptive production processes. Should help Business Analysts to coordinate issues with Client infrastructure team, middleware team and managed services provider (MSP).
• The Senior ETL developer has a strong command of ETL (extract-transform-load) Oracle PLSQL development responsible for design, development, and delivery of data from operational systems and files into staging area, Data Warehouse, downstream Data Marts and files. The senior developer will implement quality logical and physical ETL designs that have been optimized to meet the operational performance requirements for our multiple solutions and products. This will include the implementation of sound architecture, design, and development standards. He/she has the experience to design the optimal performance strategy and manage the technical metadata across all ETL jobs.
• The Senior ETL developer has strong analytical skills and should recommend technical improvements to Business Analyst and help collaborate with Business Analysts for more efficient and effective system designs and processes. The Senior ETL developer should be able to adapt to new technologies.

Detailed Tasks and/or Expected Outputs:

• Perform systems and data management of TD systems, including the extraction-transformation-loading (ETL) components of the system, which includes but not limited to:

• Spearheads development of ETL code, metadata definitions and models, queries and reports, schedules, work processes and maintenance procedures.

• Manages automation of file processing as well as all ETL processes within a job workflow.
• Develop and deploy ETL job workflow with reliable error/exception handling and rollback framework.
• Manage software upgrades, capacity planning and system optimization.
• Perform the unit testing, system integration testing and assist with user acceptance testing
• Adapt ETL code to accommodate changes in source data and new business requirements.
• Document the ETL detailed design and test results.
• Prioritize development and enhancement requests.
• Ensures data quality throughout entire ETL process.
• Understanding of Middleware Data Integration
• Ensures compliance with regulatory requirements for data transmission, storage, and data access including documentation, testing, and remediation.
• Designs Power BI Reports and Dashboard is an advantage.
• Power BI Data Analysis and Modeling is an advantage.
• Designs Datadog Dashboard is an advantage.
• Configure Datadog Alerts using Datadog Logs and Events is an advantage.
• Datadog Real User Monitoring Configuration is an advantage.

Education and Work Experience

• Bachelor's degree in computer science or equivalent experience required
• 8yrs to 10 years' experience in managing data lineage and performing impact analysis
• 8yrs to 10 years' 10 years' experience with ETL tool development using Oracle PLSQL.
• Knowledge in ADF (Azure Data Factory) is preferred
• 8yrs to 10 years' 10 years' experience with Oracle (10g, 11g and 12c)
• Expert ability in SQL and PL/SQL
• Excellent ETL design and documentation skills
• Understanding of advanced database concepts: e.g., instance, table, table space, data files, database structure, indexes to fine tune programs and solve program failures.
• Understanding of basic Unix Operation system concepts, programs, directory structure and basic commands (e.g., vi, sh)
• Familiarity with the technical aspects of Unix, the Oracle RDBMS, and tools
• Knowledge in Python language is an advantage
• Deep appreciation of Agile or rapid application development (RAD) method
• Ability to handle multiple tasks and adapt to a constantly changing environment