Memory capacity
128 GB unified memory per node. Enough headroom for a 70 billion parameter model, the embeddings, and the retrieval working set, on one box.
Why local AI
A private AI lab. Room to research and experiment.
We asked the same question every team eventually asks. Why are we still renting GPUs to run our own research? So we built the lab at home and ran our work on it. Now we build the same for you.
NVIDIA
Qwen
Mistral AI
Meta Llama
DeepSeek
Google Gemma
OpenAI
Claude
Hugging Face
PyTorch
Python
Ollama
LangChain
Qdrant
PostgreSQL
Docker
Kubernetes
Ubuntu
MCP
Grafana
PrometheusContinuous evaluation
Evals, embedding rebuilds, and regression checks run on a schedule you set. Quality assurance becomes an operational habit, paid for in electricity.
Data stays private
Embed contracts, records, and logs on your own hardware, so source text and vectors stay in the building. Data-residency questions are settled up front.
Reproducible experiments
Track which model version produced which outputs, under which configuration, on which dataset. A full experimental record, and the room to run every experiment to the end.
Right-sized hardware
Grace Blackwell or AMD Ryzen AI Max, matched to the workloads your teams actually run, so every rand of compute earns its place.
Turnkey delivery
Hardware procured through NPU Labs supplier partnerships. The stack lands configured: runtime, embedding pipeline, vector store, observability.
Inside the lab
Data science workflows on infrastructure you control.
The lab is the product, not the box. The hardware is what runs the work; the work is what teams actually buy.
Data
Train
Eval
Deploy
01 / STAGE
Data and sampling
Build the dataset before the model sees a thing.
- Jupyter-based experimentation
- Synthetic data generation
- Embeddings and vector search
02 / STAGE
Train and adapt
Where open models meet your domain.
- Fine-tuning
- Computer vision and multimodal
03 / STAGE
Evaluate
Score outputs and catch regressions before they ship.
- Model evaluation
04 / STAGE
Deploy and run
Inference, retrieval, and agents on your own hardware.
- Local inference
- RAG pipelines
- Agent development
- Voice and video pipelines
Private Lab
A private ML lab on hardware you own.
A workbench for experimentation, fine-tuning, embeddings, and evaluation. NVIDIA positions DGX Spark for local development, inference, fine-tuning, and data science, with fine-tuning up to 70B and inference up to 200B on a single unit. The stack lands configured and operational, so the team's time goes straight to the ML work.
Jupyter environment
Notebooks for analysis, model testing, and reproducible research. The working surface for the team.
Inference runtime
OpenAI-compatible endpoints served on vLLM, TRT-LLM, or llama.cpp, sized to the unit or the node.
Embedding pipeline & vector store
Local embeddings and a Qdrant-backed store, so RAG retrieval runs without sending documents off-network.
Fine-tuning
Adapt open-weight models. NVIDIA positions DGX Spark for fine-tuning up to 70B on a single unit.
Evaluation tooling
DeepEval and RAGAS wired in to score quality and catch regressions on a schedule.
Observability
Utilisation, latency, time-to-first-token, and throughput across the stack.
Model access
The preferred model registry, cached and served locally across reasoning, coding, multimodal, and voice.
Dev tools & agents
Your IDE and agents, on models you host.
The lab serves OpenAI-compatible endpoints, so the assistants and agent SDKs your developers already use point at your private models instead of a vendor's hosted ones. Nothing is locked to a single provider. You choose the model, and you change it without changing the tools.
VS Code + CopilotCursorClineAiderContinueJetBrains AIOpenAI Agents SDKLangGraph / LangChainGitHub Actions
Because the models are yours, your standards go in.
A self-hosted model can be grounded in your own context with RAG. The assistant stops answering like the public internet and starts answering like your best engineer.
Dev standards
Coding standards, lint and review rules, naming and commit conventions, retrieved at generation time so suggestions arrive already in house style.
Architecture preferences
Approved patterns, ADRs, the stack you actually run, and what's off-limits, so the agent designs to your architecture, not a generic public default.
Proprietary RAG
Internal libraries, service catalog, API contracts, past PRs, and runbooks, all grounded in your codebase and knowledge, never sent to an external service.
The result: an assistant that codes to how your team builds, on infrastructure where the code, the prompts, and the context never leave the network.
Evaluation & MLOps
The evaluation loop, run as often as it should be.
A model that scores well at deployment drifts over the months that follow. Customer language shifts, new products appear, queries change shape. Continuous evaluation makes that drift visible while it is still small. On hardware you own, each scheduled eval batch costs electricity, so the loop runs as often as quality demands.
GenerateScoreDiagnoseRetrainEvaluate
Classification
Labelling, routing, sentiment, document categories.
- Precision
- Recall
- F1 (macro / weighted / micro)
Generation
Summaries, responses, generated records.
- ROUGE
- BLEU
- BERTScore
- Perplexity
Retrieval (RAG)
Did the right documents come back?
- Recall@k
- MRR
- NDCG
- Context precision
- Context recall
Grounding (RAG)
Did the model use them without fabricating?
- Faithfulness
- Answer relevancy
Built with DeepEval and RAGAS. The pipeline re-runs whenever the document corpus changes, the embedding model is updated, retrieval parameters are adjusted, or the generation model is replaced, and on scheduled passes over labelled samples drawn from live traffic.
Common questions
Questions we get asked about the private lab.
What is the difference between a private lab and private infrastructure?
The infrastructure is the hardware and the serving layer. The lab is the working surface on top of it: notebooks, fine-tuning, embeddings, evaluation, and the experiment record. Most teams want both, and they arrive together.
Which models can we fine-tune on it?
Open-weight models. NVIDIA positions DGX Spark for fine-tuning up to 70B and inference up to 200B on a single unit. Anything in our model registry can be pulled, cached, and served locally.
How do experiments stay reproducible?
Every run records which model version produced which outputs, under which configuration, on which dataset. Because the compute is yours, a run can go to completion rather than being cut short to manage a bill.
What tooling comes installed?
Jupyter, PyTorch, an OpenAI-compatible inference runtime on vLLM, TRT-LLM, or llama.cpp, a local embedding pipeline with a Qdrant vector store, DeepEval and RAGAS for evaluation, and Prometheus and Grafana for observability.
Can the lab feed a production system?
Yes. The model registry promotes a validated model from the lab to the serving layer, with one-step rollback and provenance for every deployed weight.
Who owns what we build on it?
You do. The hardware, the fine-tuned weights, the data, and the experiment record are yours, and they sit in your environment.
Not the question you came with?
Ask us about your setupWant a private lab for your data science?
From a single workstation to a stacked cluster, we'll size the compute and stand up the experimentation, fine-tuning, and evaluation workflow your team needs.