At Corgan we are passionate about great design but the best thing we build is each other. Here, you’ll find camaraderie and collaboration as abundantly as curiosity and creativity. Corgan is a welcoming and supportive environment that provides significant career advantages. You’ll find room to grow, freedom to explore — and the safety to fail. Thank you for your interest in joining our curious, passionate, hardworking team.

Corgan is actively recruiting a dynamic Senior Data Engineer (Developer) to join the firm as part of our Shared Services MediaLab team. Tackle these responsibilities alongside a team of creative, highly motivated individuals who are passionate about their work and ready to collaborate to achieve outstanding results. Our ideal team member is passionate and progressively seeking more responsibilities to expand their expertise. This role will focus on designing, building, and maintaining scalable data infrastructure, real-time analytics systems, and AI-powered data pipelines that support our corporate strategic objectives. 

As a Senior Data Engineer, you will architect and implement comprehensive data solutions spanning cloud-native data lakes, real-time streaming platforms, and modern data warehousing systems. You'll work at the intersection of traditional data engineering and emerging AI/ML technologies, ensuring our data infrastructure can support both current analytical needs and future artificial intelligence initiatives. 

Development and data operations will require close collaboration with IT staff, data scientists, business analysts, and internal stakeholders across multiple departments. The role requires expertise in the entire Software Development Lifecycle (SDLC), encompassing requirements gathering, system design, implementation, testing, and production deployment. 

Data Architecture & Infrastructure 

• Design and build enterprise-scale data architectures supporting structured, semi-structured, and unstructured data across multi-cloud environments (Azure, AWS) 

• Implement scalable data lakes and data warehouses optimized for both batch and real-time analytics workloads 

• Develop and maintain data mesh architectures that enable self-service analytics while ensuring data governance and security 

• Architect cloud-native solutions leveraging serverless computing, containerization, and microservices patterns 

Data Pipeline Development 

• Build robust, fault-tolerant data pipelines using modern ELT methodologies and orchestration tools 

• Implement real-time data streaming solutions using Apache Kafka, Apache Pulsar, and cloud-native streaming services 

• Design and maintain automated data quality frameworks with comprehensive monitoring, alerting, and auto-remediation capabilities 

• Develop CI/CD pipelines for data engineering workflows, including automated testing, deployment, and rollback procedures 

AI/ML Integration & Advanced Analytics 

• Integrate machine learning workflows into data pipelines, supporting feature engineering, model training, and inference at scale 

• Implement MLOps practices including model versioning, A/B testing frameworks, and automated retraining pipelines 

• Build data infrastructure to support generative AI applications, including vector databases and retrieval-augmented generation (RAG) systems 

• Collaborate with developers, engineers, and data scientists to produce machine learning models and ensure scalable inference capabilities 

Data Governance & Security 

• Implement comprehensive data governance frameworks including data lineage tracking, metadata management, and data cataloging 

• Ensure compliance with data privacy regulations (GDPR, CCPA) and implement data masking, encryption, and access controls 

• Establish data quality standards and automated validation rules across all data assets 

• Design and maintain audit trails for data processing activities and model predictions 

Performance Optimization & Monitoring 

• Optimize data processing performance through query tuning, indexing strategies, and cost-effective resource allocation 

• Implement comprehensive observability solutions for data pipelines, including metrics, logging, and distributed tracing 

• Conduct root cause analysis for data quality issues and system performance bottlenecks 

• Establish SLAs for data freshness, accuracy, and system availability 

Collaboration & Knowledge Sharing 

• Collaborate with cross-functional teams, including developers, analysts, and business stakeholders, to understand requirements and deliver solutions 

• Provide technical leadership and mentoring to junior developers and analysts 

• Develop and maintain technical documentation, data architecture diagrams, and best practices guidelines 

• Lead technical design reviews and contribute to technology strategy decisions 

Qualified candidates should exhibit strong problem-solving, written, and verbal communication skills. 7+ years of experience in data engineering required. The candidate must have experience in dealing with internal clients. This candidate must have extensive experience with database development and maintenance. A bachelor’s degree in computer science, information technology, or another related field is preferred. In addition, qualified candidates will have experience with: 

• This role is an in-office position. 
• 7+ years of hands-on data engineering experience with a proven track record of building production-scale data systems5+ years of hands-on experience with Microsoft Dynamics 365 (CRM)  

• 5+ years of experience with cloud platforms (Azure, AWS), including data services and infrastructure management 

• 5+ years of advanced SQL experience including query optimization, performance tuning, and complex analytical queries  

• 3+ years of experience with big data frameworks (Apache Spark, Hadoop ecosystem, Databricks)  

• 3+ years of experience with real-time data processing and streaming technologies  

• Strong programming skills in Python, C#, and/or Scala with a focus on data processing and automation 

• Expert-level proficiency in SQL, NoSQL, and NewSQL databases (PostgreSQL, MongoDB, Cassandra, Snowflake)  

• Advanced experience with ETL/ELT tools and orchestration platforms (Apache Airflow, Azure Data Factory, Fabric Dataflow Gen2 and Data Pipelines, AWS Glue, dbt)  

• Deep understanding of data modeling techniques for both transactional and analytical workloads  

• Experience with data warehousing concepts including dimensional modeling, star/snowflake schemas, and slowly changing dimensions 

• Hands-on experience with cloud-native data services (Azure Synapse, AWS Redshift/Athena)  

• Proficiency with Infrastructure as Code (Bicep, Terraform, CloudFormation) and containerization (Docker, Kubernetes)  

• Experience with serverless computing architectures and event-driven data processing  

• Understanding of cloud security, networking, and cost optimization strategies 

• Expert-level Apache Spark development using PySpark, Scala, or Java  

• Experience with real-time streaming platforms (Azure Event Hubs, Apache Kafka, Apache Pulsar, AWS Kinesis)  

• Knowledge of distributed systems concepts and fault tolerance patterns  

• Experience with data lakehouse architectures and formats (Delta Lake, Apache Iceberg, Apache Hudi) 

• Experience integrating machine learning workflows into data pipelines  

• Understanding of MLOps practices and tools (MLflow, Kubeflow, SageMaker)  

• Knowledge of feature stores and model serving architectures  

• Familiarity with vector databases and embedding techniques for AI applications 

• Advanced Git workflows, code review processes, and collaborative development practices  

• Experience with CI/CD pipelines for data engineering (Azure DevOps, GitHub Actions, Jenkins)  

• Proficiency with monitoring and observability tools (DataDog, Splunk, Prometheus, Grafana)  

• Understanding of agile development methodologies and project management tools 

Preferred Qualifications  

• A Master’s degree in Computer Science, Data Science, Engineering, or related technical field 

• Experience with graph databases and knowledge graph technologies 

• Background in financial services, architecture, engineering, or construction industry data systems 

• Knowledge of data privacy and compliance frameworks 

• Preferred certifications and credentials include: 

• Microsoft Certified: Azure Data Engineer Associate (DP-203) 

• Microsoft Certified: Fabric Data Engineer Associate (DP-700) 

• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) 

• AWS Certified Data Engineer – Associate 

• Databricks Certified: Data Engineer Associate 

• Databricks Certified: Data Engineer Professional 

• SnowPro Advanced: Data Engineer (DEA-C02) 

• SnowPro Advanced: Architect (ARA-C01) 

Corgan creates opportunities for personal and professional growth that bring out the best in ourselves and those around us by sharpening skills, leveraging technology, and encouraging communication. Our commitment to the highest standards of professional ethics is unwavering. Our passion, drive for excellence, and abiding curiosity guide us each day in our interaction with our clients, partners, associates, and with each other. Corgan promotes a healthy work-life balance by removing obstacles and allowing you to focus on what you love doing, both inside and outside the office.

Join Corgan and help make amazing places at a place that makes you amazing!

*Only applications received through our website will be considered. For design positions, please also be sure to upload your portfolio or review of your application will be delayed. *

EOE/M/F/Disability/Vet

#LI-KA