Platform Data Engineer

Runware is building a high-performance AI media-creation platform powering instant generation of text, image, video, 3D, and audio. As our platform scales and integrations grow, we need robust, reliable, and high-throughput data systems.

We’re looking for a Data Engineer to architect and maintain our data backbone — with a special focus on high-volume logs, and performance pipelines.

You will work hand-in-hand with our Data Expert and the platform team to transform raw system activity into structured, queryable, high-value data.

🎯 Mission

Your main mission is to build, optimize, and maintain Runware’s data infrastructure.

You will ensure that logs, metrics, performance data, and events are efficiently ingested, processed, stored, and ready to be analyzed by engineering, ML, and product teams.

This role is central to:

• Supporting observability & platform reliability
• Enabling deep log & performance analytics
• Powering internal dashboards and customer reporting
• Providing clean, structured data to the Data Expert and all stakeholders

🧩 What You Will Do

Architecture & Ownership

• Design, build, and maintain schemas and data models
• Optimize table layout, partitioning, indexing, and compression for high-volume data
• Ensure fast, efficient querying for logs, requests, metrics, and performance traces
• Maintain ingestion pipelines for billions of records

Data Engineering & Pipelines

• Build robust pipelines for:
• API logs
• Model inference logs
• Error events
• Usage & integration events
• GPU & system metrics
• Implement ETL/ELT workflows to transform raw data into analytics-ready structures
• Ensure quality, reliability, and real-time availability of data sources

Performance & Log Analysis Infrastructure

• Build tooling to support large-scale log analysis
• Enable deep investigation into latency, throughput, errors, and bottlenecks
• Provide the raw data foundation for E2E inference-time monitoring
• Help debug production issues using logs and traces

Tooling & Observability Infrastructure

• Work closely with DevOps, ML, and backend engineering
• Integrate pipelines with monitoring tools (Prometheus, Grafana, Datadog, OpenTelemetry)
• Automate ingestion and cleanup tasks
• Build internal libraries or utilities to support monitoring and debugging workflows

Collaboration & Cross-Functional Support

• Provide clean data interfaces for the Data Expert (dashboards, monitoring, analytics)
• Support engineering teams by exposing the right logs and metrics
• Contribute to debugging, RCA (root cause analysis), and performance optimization initiatives