Data Engineer III (6193)

Data Engineer III

itD is seeking a Senior AI Data Engineer III to build and scale AI-augmented data infrastructure that powers next-generation image generation models. This role sits at the intersection of Data Engineering and Machine Learning Systems, driving the development of large-scale data curation, annotation, and evaluation pipelines that improve model quality across visual quality, prompt adherence, identity preservation, naturalness, and visual text generation. The ideal candidate will bring deep expertise in AI-focused data engineering and a proven track record of building production-scale pipelines that integrate machine learning inference into data workflows.

Location: Hybrid Onsite – Menlo Park, CA (required onsite collaboration with engineers and researchers)

Pay Rate: $35 - $39 per hour, depending on experience.
Duration: 5+ months

We provide comprehensive medical benefits, a 401k plan, paid holidays, and more.
Please note that we are only considering direct W2 candidates at this time, as we are unable to offer sponsorship.

Responsibilities

• Design, build, and maintain AI-augmented data pipelines that combine traditional data transformations with machine learning model inference at billion-row scale.
• Develop and optimize systems for remote model inference orchestration, including batching, asynchronous execution, retry logic, throughput management, and graceful failure handling.
• Build and maintain scalable embedding generation, storage, indexing, and retrieval pipelines to support AI model training and evaluation.
• Curate and manage large-scale image datasets using SQL and model-derived signals, ensuring data quality, governance, compliance, and operational efficiency.
• Design and operate LLM-assisted annotation workflows that automate data labeling while measuring and improving annotation quality.
• Develop reusable frameworks, tooling, and pipeline components that enable broader engineering teams to efficiently build AI-powered data workflows.
• Partner closely with engineers, researchers, and cross-functional stakeholders to support image generation model development and evaluation initiatives.

Internal Responsibilities

• Attend regular internal practice community meetings.
• Collaborate with your itD practice team on industry thought leadership.
• Complete client case studies and learning material (blogs, media material).
• Build out material to contribute to the Digital Transformation practice.
• Attend internal itD networking events (in person and virtual).
• Work with leadership on career fast-track opportunities.

Required Qualifications and Skills

• 5+ years of experience in Data Engineering, ML Engineering, or a hybrid role involving both data pipelines and machine learning inference systems.
• Strong software engineering fundamentals, including Python, data structures, concurrency, and asynchronous programming.
• Advanced SQL expertise, including complex query development, query optimization, and large-scale data processing.
• Experience with pipeline orchestration frameworks such as Airflow, Dataswarm, or equivalent platforms.
• Proven experience integrating machine learning models into production data pipelines, including inference endpoint management, model versioning, batching, and failure recovery.
• Demonstrated track record of building and operating production-scale data pipelines that invoke machine learning models at scale.
• Proficiency with AI-assisted coding tools such as Copilot, Cursor, Codex, or similar AI development agents.
• Strong written and verbal communication skills with the ability to collaborate effectively across technical and business teams.

Preferred Qualifications and Skills

• Experience generating, storing, indexing, and querying vector embeddings using technologies such as FAISS, Milvus, or similar platforms.
• Familiarity with content understanding models including image classification, object detection, OCR, NSFW detection, and aesthetic scoring systems.
• Experience leveraging LLMs for data annotation, data cleaning, evaluation, or prompt engineering workflows.
• Knowledge of generative AI technologies, including diffusion models, image generation systems, and evaluation metrics such as FID and CLIP Score.
• Previous experience leading AI-focused technology companies.
• Experience supporting large-scale image generation or multimodal AI initiatives.

Education

• Bachelor’s degree or higher in Computer Science, Data Engineering, Machine Learning, or a related STEM field required.