PURPOSE STATEMENT: We are looking for an experienced Senior Principal Data Engineer to architect, build, and maintain the data infrastructure and pipelines that power our analytics and machine learning platforms. In this role, you will collaborate with stakeholders across business units to understand data needs and then design optimal technical solutions leveraging tools like Snowflake, Databricks, and Azure.
ESSENTIAL FUNCTIONS: Design, implement, and optimize cloud data architecture including databases, schemas, tables, views, clusters in cloud data platform tech stack Develop end-to-end data pipelines that from source systems ingestion to data warehouse or data lakes and to data mart, eventually service BI and AI/ML for visualization and analytical needs Create curated data assets with access and governance in place for analytical workloads Monitor data pipelines, infrastructure health metrics. Automate for fault tolerant and self-healing, optimize for maximal throughput and performance Automation and integration of data quality components in the data pipelines. Modernize data assets and associated pipelines from legacy tech stack into new data platform Set, document and maintain standards for data assets inventory, meta data management and data lineage Research and evaluate new data management tools and approaches for potential integration Coach and mentor junior data engineers in data modeling, pipeline development, troubleshooting issues STANDARD EXPECTATIONS: Complies with organizational policies, procedures, performance improvement initiatives and maintains organizational and industry policies regarding confidentiality Communicate clearly and effectively to person(s) receiving services and their family members, guests, and other members of the health care team Develops constructive and cooperative working relationships with others and maintains them over time Encourages and builds mutual trust, respect, and cooperation among team members Maintains regular and predictable attendance EDUCATION/EXPERIENCE/SKILL REQUIREMENTS: BS/MS degree in Computer Science, Engineering or equivalent field 7+ years hands-on engineering and administration experience in data engineering on cloud platforms (Azure preferred) with RDBMS such as SQL Server, PostgreSQL, MySQL, with Snowflake experience as must-have. 3+ years of experience with Apache Spark/Databricks and Azure Synapse. Expert knowledge of SQL and experience with Spark, Python for data transformation/processing Experience building and optimizing data pipelines at scale with orchestration tools like AirFlow and observability tools like ELK Excellent communication skills collaborating cross-functionally with stakeholders Self-directed and passionate about keeping up with latest innovations in the data space API development experience is a plus SUPERVISORY REQUIREMENTS: Manages a Department
While this job description is intended to be an accurate reflection of the requirements of the job, management reserves the right to add or remove duties from particular jobs when circumstances
(e.g. emergencies, changes in workload, rush jobs or technological developments) dictate. AHCORP #LI-RT1
