Job Description

DescriptionKforce has a client that is seeking a Remote Senior Data Engineer.Summary:Our client is looking for a Senior Data Engineer with machine learning skills to build the data foundation behind our ML models. You will lead the design of scalable, well-governed data platforms, primarily on Azure, and bring external retail and third-party data into a medallion architecture (bronze, silver, gold) aligned to our client's product, outlet and customer master data. You will work closely with Data Science and ML teams and set the technical direction for how data is prepared and delivered.Duties:* Profile each retailer feed or third-party dataset to determine how it is partitioned (daily, weekly, monthly or quarterly) and how it should be loaded* Identify where the data falls short of model needs and close the gaps through imputation, mapping tables, source blending or requests to the data provider* Define and validate the join keys to the client's product, outlet and customer dimension tables, and resolve unmatched or duplicate keys with data owners* Deliver conformed, standardized outputs to the silver layer, so every source looks the same to downstream pipelines* Build and tune ETL/ELT pipelines with Azure Data Factory, Azure Databricks, Azure Functions or equivalent cloud-native services* Handle large migrations, including transactional volumes in the tens of terabytes, with minimal or zero downtime* Design integration layers that consume complex source data, including XML and other semi-structured formats, and turn it into analysis-ready structures* Define enterprise-wide data requirements and quality controls across multiple lines of business* Build feature-ready data models that support recurring business cycles such as quarter-end and year-endRequirements* Bachelor's degree* Minimum 7 years of experience* Significant experience as a data engineer, including lead or architect-level ownership of production data platforms* Hands-on Azure experience: Data Factory, Databricks, Functions and Synapse* Experience establishing data governance frameworks and standardized data quality controls* Experience with large-scale data migration or cloud modernization* Strong Python and SQL skills, including PySpark or pandas for large-scale transformation* Practical knowledge of medallion architecture, dimensional modeling and master data joins* Comfort with Git/GitHub workflows, documentation and code review* Clear communication with technical and business stakeholders* Candidate must have the ability to travel onsite occasionally to Midtown Atlanta, GA client sitePreferred:* Experience with AWS or GCP* Salesforce integration experience (data sync, APIs, or migrations with Salesforce as source or target)* Experience supporting ML feature pipelines or model product ionization* Background in consumer goods or retail data, such as POS, syndicated or third-party datasetsWhat Success Looks Like:* New data sources reach the silver layer on a repeatable, documented process* Join match rates and data quality are measured, and gaps are resolved or escalated* Data Science teams can trust and reuse the data without reworkJob TypeContractCompensation86 - $96

Job Tags

Similar Jobs