AI · Engineering, MLOps & Governance
A model is not finished when it ships. That is when it starts being wrong quietly.
Noble builds the machinery around a model that the demonstration never shows: data and feature pipelines that produce the same input in production as in training, serving with versioning and a rollback path, an evaluation set that gates every change, monitoring that notices when the world underneath the model has moved, and cost and latency treated as design constraints. Around that machinery sits the governance — what the model may read, what it may do, who approved the version in production, and what happens when it is wrong.
Why engineering and governance are one page
Governance of AI is usually written as a policy document and engineering as a pipeline, and the two never meet. Noble builds them as one thing, because every governance question has an engineering answer or it has no answer at all. "What may the model read?" is a permission filter in the retrieval pipeline. "Who approved this version?" is a gate in the deployment path with a name on it. "What does it do when it is wrong?" is a confidence threshold, a fallback and an escalation route, all of which are code. A policy that the pipeline does not enforce is a wish.
The other reason is drift. A model that was right on the day it shipped is being tested every day after by data it has not seen — a new product, a changed process, a customer base that has moved. Nothing in the model tells you it is now wrong; only the monitoring does, and only if somebody built it. That monitoring is the same evaluation set that gated the deployment, re-run on production traffic, and it is what turns "documented limits" from a page in a binder into something that is actually watched.
The lifecycle, as it actually runs
A loop, because a model in production is re-entered continuously. Evaluation is the gate: no version reaches serving without passing it, and the monitoring stage feeds the same set back with what production has taught.
- 01Data and featuresSame pipeline, training and serving
- 02Train or configureVersioned with its inputs
- 03EvaluateGate; a name approves
- 04ServeRollback path kept
- 05MonitorDrift, cost, latency, refusals
- 06Feed the evaluation setWhat production taught
What the service covers
Data and feature pipelines
The pipeline that prepares a model’s input, built once and used for both training and serving so that the model sees in production exactly what it learnt from. Sources are read from the governed layer, transformations are versioned with the model they feed, and a change to an upstream system is caught before it reaches the model as a silent shift in its inputs.
- One pipeline for training and serving
- Transformations versioned with the model they feed
- Upstream changes caught before they reach the model
- Feature definitions shared and documented
Serving, versioning and rollback
A model served as a versioned artefact behind an interface, with the previous version kept warm so that rollback is a decision rather than a project. Deployment is staged — shadow, then a fraction of traffic, then all of it — with the evaluation set run at each step, and every version in production traceable to the data, code and approval that produced it.
- Versioned artefacts behind a stable interface
- Staged rollout with evaluation at each step
- Rollback as a decision, not a project
- Each production version traceable to data, code and approval
Evaluation as the gate
A set of cases with known-good answers, built before the first deployment with the people who will use the system, including the cases it must refuse. Every change — to the model, the prompt, the features, the content, the vendor’s version — is scored against it before it reaches production, the score is reported as it is, and a named person approves the deployment on that evidence.
- Built before deployment, with the users, including refusals
- Every change scored before it reaches production
- A named person approves on the evidence
- Vendor model updates treated as a change like any other
Monitoring for drift and silent degradation
Production watched for what the model cannot tell you itself: the input distribution moving away from what it was trained on, the confidence falling, the refusals rising, the corrections from reviewers clustering on one kind of case. Each has a threshold and an owner, and crossing one triggers re-evaluation rather than an argument.
- Input drift, confidence, refusals and corrections watched
- Thresholds with owners; crossing one triggers re-evaluation
- Reviewer corrections fed back to the evaluation set
Cost and latency as design constraints
What a model costs per call and how long it takes are decided at design, not discovered at the invoice. Noble sizes the model to the task — a small model where a small model answers, a large one where it is needed — caches what can be cached, batches what can wait, and reports cost per use so the organisation can see what each capability actually costs to run.
- Model sized to the task, not to the demonstration
- Caching and batching where the task allows
- Cost per use reported to the organisation
- Where the workload runs decided with Cloud
Governance the pipeline enforces
Access control over what a model may read, enforced in retrieval. Personal data handling and residency decided before a model is chosen and enforced in the pipeline. Audit trails over prompts, retrievals and actions, kept for the period the organisation’s rules require. Human oversight proportionate to the risk, configured as thresholds and review queues. Documented limits, and a defined path — fallback, escalation, rollback — for when the model is wrong. Each is a control with an owner, and each is evidenced by the system rather than by a policy.
- What the model may read, enforced in retrieval
- Residency and personal data decided before the model
- Audit trails kept for the required period
- Documented limits, with fallback, escalation and rollback
- Model lifecycle: what changed, when, who approved it
Where this connects
- Machine Learning & Predictive IntelligenceThe models this machinery serves, and the held-back evaluation that decides whether they deploy at all.
- Generative AI & LLM ApplicationsPrompt versioning, retrieval evaluation and the record of what an assistant was shown are this page’s controls applied to language systems.
- AI Agents & Intelligent AutomationBudgets, stopping conditions and the record of an agent’s actions are enforced by the runtime described here.
The services alongside
- CloudWhere training and serving run, what they cost, which region the data may be in, and the platform the pipelines are built on.
- SecurityAccess control, audit and the personal-data rules the pipeline enforces are that service’s subject, applied here to models.
- Data & AnalyticsThe governed layer the pipelines read from, and the lineage that lets a model’s input be traced to its source.
- Managed IT ServicesOnce a model is in steady state, its monitoring and its owner can sit inside the managed service alongside everything else that is run.
Ask what changed last month.
If the models already in production cannot answer that — which version, on whose approval, evaluated against what — that is where this work begins. Bring the list of what is running.