layiq

/la·yiq/adjective

worthy; deserving; fitting; suitable.

A role, opportunity, or path that merits attention, time, and pursuit.

Eval360 - Error Analysis Engineer at Institute of Foundation Models | LAYIQ

Collaborate with researchers, machine learning engineers, data scientists, product managers, and internal stakeholders to implement innovative software solutions for Eval360 and related model evaluation workflows.
Build and improve Eval360 as an evaluation service that acts as a quality gate for model development, model comparison, and model release decisions.
Perform deep error analysis on model outputs, including identifying failure patterns, categorizing issues, tracing root causes, and proposing improvements to evaluation methodology.
Develop tools, workflows, and dashboards that make it easier for researchers and engineers to inspect model failures, compare model behavior, and understand quality regressions.
Design and implement client-side and server-side architecture for evaluation review systems, error analysis interfaces, reporting tools, and internal evaluation applications.
Develop responsive, usable interfaces that support error triage, annotation review, evaluation debugging, and model quality investigation.
Build and maintain back-end services, APIs, data pipelines, and integrations that support evaluation execution, results storage, analysis, and reporting.
Test software to ensure responsiveness, correctness, reliability, and efficiency across evaluation workflows.
Troubleshoot, debug, and upgrade evaluation systems, including identifying issues in data processing, evaluation metrics, model output handling, job orchestration, and user-facing analysis tools.
Create and maintain security, access control, and data protection settings for evaluation data, model outputs, annotations, and internal tooling.
Write clear technical documentation for Eval360 systems, error taxonomies, evaluation workflows, debugging procedures, and user-facing tools.
Work with researchers, data scientists, analysts, and machine learning engineers to improve evaluation quality, model diagnostics, and failure-mode visibility.
Keep track of new development tools, evaluation frameworks, model analysis methods, data quality techniques, and architectures relevant to AI evaluation systems.
Contribute to the design of error taxonomies, evaluation rubrics, quality thresholds, regression detection methods, and model readiness criteria.
Help ensure Eval360 produces reliable, interpretable, and actionable signals for model quality gates.
Contribute to research publications, technical reports, internal knowledge sharing, and external presentations where appropriate.
Contribute to intellectual property and thought leadership in AI evaluation, error analysis, model quality measurement, and evaluation infrastructure.
Perform all other duties as reasonably directed by the line manager that are aligned with these functional objectives.
Academic Qualifications: •
Bachelor's degree in Computer Science, Machine Learning, Data Science, Software Engineering, Statistics, or a related technical field required.
Master's or Ph.D. in Computer Science, Machine Learning, Artificial Intelligence, Data Science, or a related field preferred.
Professional Experience: •
Proven experience as a Software Engineer, Full Stack Developer, Machine Learning Evaluation Engineer, Data Scientist, AI Engineer, or similar role.
Experience building software systems for AI, machine learning, data analysis, evaluation, annotation, experimentation, or model monitoring.
Experience working with AI algorithms and the ability to develop systems that accommodate AI-related requirements.
Experience performing error analysis, model evaluation, data quality analysis, or failure-mode investigation for machine learning or language model systems.
Experience developing internal applications, dashboards, review tools, or web-based workflows for technical users.
Familiarity with common software stacks, including front-end frameworks, back-end services, databases, APIs, and cloud or internal infrastructure.
Familiarity with GitHub, Git, CI/CD workflows, and collaborative software development practices.
Knowledge of front-end languages and libraries such as HTML, CSS, JavaScript, TypeScript, React, Angular, or similar technologies.
Knowledge of back-end languages and frameworks such as Python, Java, C#, Node.js, FastAPI, Flask, Django, or similar technologies.
Familiarity with databases such as MySQL, PostgreSQL, MongoDB, or other structured and unstructured data stores.
Familiarity with evaluation frameworks, experiment tracking systems, data pipelines, or machine learning infrastructure is strongly preferred.
Ability to analyze complex model outputs and translate qualitative failures into structured, measurable categories.
Strong problem-solving and troubleshooting skills, especially for ambiguous technical issues involving models, data, metrics, and software systems.
Effective communication and collaboration skills, with the ability to work across research, engineering, data, and product teams.
Strong attention to detail and a high bar for evaluation quality, reliability, and interpretability.

Eval360 - Error Analysis Engineer

Employer job description

About the Institute of Foundation Models

The Role: We are looking for an

Key Responsibilities: •

Preferred Qualifications: •

Benefits Include

About this listing