Senior AI Engineer

Experience and Skills

Senior-level experience building and operating production AI systems, including tool-using, multi-step LLM agents.

Production experience designing evaluation sets, metrics, quality thresholds, go-live gates, and regression detection across model or prompt changes.

Strong applied statistics skills, including review sample sizing, confidence bounds, and error-rate estimation from samples.

Experience designing safety controls enforced in code, including approvals, deterministic checks, and kill switches.

Strong Python or TypeScript skills, including testing, code review, and development of maintainable services.

Ability to write clear design notes and document technical decisions before implementation.

Hands-on experience with Agent Orchestration, LLM Evaluation, Evaluation Metrics, Software Testing, Safety Engineering, Runtime Governance, Approval Workflows, Model Observability, Data Provenance, Model Provider Abstraction.

Skills

LLM Agents

Agent Orchestration

LLM Evaluation

Evaluation Metrics

Applied Statistics

Python

TypeScript

Software Testing

Code Review

Safety Engineering

Runtime Governance

Approval Workflows

Model Observability

Data Provenance

Model Provider Abstraction

Key Responsibilities

Build an evaluation framework with datasets, metrics, agreed thresholds, and a go-live gate for each agent capability.

Re-evaluate agent capabilities after every model or instruction change and identify quality regressions.

Build a governed agent runtime with accountable ownership for each action, human approval where required, and deterministic checks before irreversible actions.

Implement rule-level or campaign-level approval for high-volume actions, including statistically sized review samples before release.

Configure automatic suspension when quality thresholds are not met.

Design review and evaluation flows for AI-generated content sent externally.

Record every model call, including the model, instruction version, data provenance, and quality checks.

Maintain platform independence from any single model provider.

Convert written governance rules, ontologies, and acceptance scenarios into executable validation with the Principal Platform Engineer.

Collaborate with cloud engineering on runtime infrastructure, isolation, and model-provider connectivity, and with product stakeholders on capability requirements, datasets, and thresholds.

Nice to Have

Knowledge Graphs

Ontologies

LinkML

SHACL

OWL

RDF

Human-in-the-Loop Systems

Retrieval-Augmented Generation

Access Control

Data Classification

Financial Services

Privacy

Multi-Provider LLM Integration

Benefits

We provide the opportunity to participate in impactful and technically rigorous industrial data projects that drive innovation and professional growth. Our work environment emphasizes technical excellence, collaboration, and continuous innovation.

Niuro supports a 100% remote work model, allowing flexibility in work location globally. We invest in career development through ongoing training programs and leadership opportunities, ensuring continuous growth and success.

Upon successful completion of the initial contract, there is potential for long-term collaboration and stable, full-time employment, reflecting our long-term commitment to our team members.

Joining Niuro means becoming part of a global community dedicated to technological excellence and benefiting from a strong administrative support infrastructure that enables you to focus on impactful work without distraction.

The Senior AI Engineer will own evaluation and governance capabilities for a core platform where AI agents take real actions, including sending messages, updating records, and preparing decisions. You will ensure each agent capability is evaluated before release and after model or instruction changes, with clear thresholds and go-live controls. You will design and implement runtime safeguards that enforce accountability, approvals, data restric