layiq

/la·yiq/adjective

worthy; deserving; fitting; suitable.

A role, opportunity, or path that merits attention, time, and pursuit.

Software Engineer, Agents (Internal Audit) at Fieldguide | LAYIQ

Make agent judgment repeatable: run error analysis on real testing data and turn findings into concrete fixes
Tradeoffs such as quality/latency/cost across a long multi-phase run
Build structured-output pipelines that turn model output into real audit artifacts
Take ambiguous problem statements and turn them into a plan, a shipped feature, and a clear read on what was cut and why
Work directly with an embedded subject matter expert and with design-partner firms, turning their feedback into agent changes within days
Expand agent coverage into new controls and new areas of internal audit
Who You Are (All Levels)
Product-minded and full-stack: you've shipped LLM-backed features to production against real users, and you measure yourself on whether they got used
You're fluent in evals and error analysis, and you apply them in service of shipping something practitioners trust
You have real opinions on model selection, prompting, and orchestration tradeoffs, and you can defend them with evidence rather than vibes
Energized by 0→1 work: you'd rather define the problem than inherit a spec, and you don't stall on ambiguity
Strong instincts for human-in-the-loop design
A genuine team player across the organization, not just within engineering: you'll work daily with PM, design, domain experts, and customer-facing teams, and you treat that as the best part of the job
Ship fast without leaving a mess: your code is reviewable, tested where it counts, and instrumented
Able to internalize a hard domain fast. You don't need to know SOX today, but you'll understand it well enough to make the right product calls
Higher-Level Responsibilities
At the Senior level, you may: Own a major agent area end-to-end, from how the agent reasons about a class of controls through to the artifact a reviewer signs
Set the evals and error-analysis practice for the team's agent work, and decide what evidence justifies shipping a change or rolling it back
Collaborate with PMs and designers to shape roadmaps and define architectural tradeoffs, including where the agent acts and where the auditor decides
Own the harder model and orchestration judgment calls across a long multi-phase run
Mentor other engineers and raise the bar on 0→1 execution and applied eval rigor
At the Staff level, you may: Drive agent initiatives that reach beyond Internal Audit and influence how agents are built across Fieldguide
Set and champion engineering standards for agent reliability, reproducibility, and defensibility
Partner with engineering and product leadership to define long-term technical strategy for agentic audit work
Serve as a trusted advisor to leaders across Engineering, Product, and Design
Represent Fieldguide externally through writing, speaking, and open-source contributions
Experience
Must-have: Shipped LLM-backed product features to production against real users
Applied AI skillset: evals, error analysis, and model-selection decisions you owned and can explain
Comfortable full-stack, with enough backend depth to work in agent orchestration
Autonomy working from an ambiguous spec
A collaborative mode that works across PM, design, and domain experts
Nice-to-have: Python, TypeScript, React, Postgres, Hasura, GraphQL
Temporal or comparable durable-execution / workflow orchestration
Hands-on eval experience (Langfuse, Braintrust, LangSmith, Arize Phoenix, or comparable)
Structured-output work including schema contracts, generating real artifacts from model output
Startup experience, as a founder or as an early engineer
A 0→1 track record: things you started where no scaffolding existed
Experience working directly with customers, and comfort being in the room when they use what you built
Background in internal audit, SOX, accounting, or another regulated domain
Document processing, including PDF and Excel manipulation and annotation
Not a fit if: Prompt engineering is your whole skill set
Your agent work never carried production traffic
You want to own eval methodology or the evaluation harness itself rather than ship product features (better fit on Foundation Agents)
You need a fully specified ticket to start
You'd rather not be in the room with customers and domain experts

Software Engineer, Agents (Internal Audit)

Employer job description

About Us

About the Role

What You'll Do

What Should Excite You

Repeatable judgment:

Real audit stakes:

Customer proximity:

Human-in-the-loop design:

High trust, high autonomy:

Benefits

About this listing