GitLab Duo self-hosted AI was announced on September 22, 2026 as a new option for enterprises that need on-premise large language model (LLM) deployments. By partnering with Microsoft Foundry, GitLab extends its Duo security and prompt-engineering stack to hardware that resides behind the corporate firewall, preserving policy-as-code, secret-masking, and usage-monitoring capabilities that were previously limited to cloud-hosted instances. The announcement was reported by InfoQInfoQ.

How GitLab Duo self-hosted AI works with Microsoft Foundry

GitLab Duo, introduced in early 2025, bundles prompt-injection defenses, model-output redaction, and usage-quota enforcement into a single service layer that sits between developers and the underlying LLM API. Microsoft Foundry, released in 2024, provides a hardware-agnostic orchestration layer that abstracts GPU provisioning, containerised model serving, and secure networking for on-premise AI clusters. When Duo is configured to use Foundry, the workflow is:

  • Register a model artifact (for example a 70-billion-parameter transformer) in an internal artifact registry.
  • Launch an inference container on a selected GPU node, using NVIDIA TensorRT or AMD ROCm runtimes as appropriate.
  • Apply Duo’s policy templates to every request, automatically redacting PII and enforcing role-based usage limits.
  • Emit audit logs to GitLab’s compliance dashboard, creating a single source of truth for code, CI/CD pipelines, and AI interactions.

The stack relies on gRPC for low-latency request routing and OpenTelemetry for distributed tracing across the CI pipeline, model server, and Duo policy service. This mirrors the cloud-native approach GitLab has championed for its CI/CD platform, now extended to the traditionally siloed AI inference layer.

Why regulated enterprises choose self-hosted AI

Financial services, healthcare, and government organisations have long resisted public-cloud AI because of data-residency and auditability requirements. GitLab Duo self-hosted AI offers a compliant alternative that keeps raw prompts and model outputs inside the corporate network. Early adopters cited in the InfoQ report include a European bank that must keep customer transaction data within EU-based data centres. The bank’s security team leveraged Duo’s secret-masking to prevent accidental exposure of account numbers in model prompts, while Foundry ensured that inference never left the private network. This concrete use case demonstrates a reduction in compliance risk and potential avoidance of GDPR fines.

Operational considerations and trade-offs

Deploying GitLab Duo self-hosted AI shifts operational responsibility to internal teams. They must manage GPU lifecycle, driver compatibility, and model versioning on premises. Duo’s policy engine assumes a stable endpoint; any downtime in the underlying model server propagates to the CI pipeline, potentially blocking merge requests that depend on AI-assisted code generation. Moreover, the security model now spans two vendors: GitLab must trust Microsoft’s Foundry attestation mechanisms, and customers must verify that the hardware root of trust is correctly provisioned.

Performance is another trade-off. Public-cloud providers benefit from proprietary inference optimisations and high-speed interconnects. Preliminary benchmarks shared by GitLab show a 15-20 % latency increase for a 70B model on a single H100 compared to Azure OpenAI, although throughput remains comparable when scaling across multiple nodes. Organisations with strict latency SLAs need to weigh this against the compliance benefits.

What to watch next for GitLab Duo self-hosted AI

  • Hardware certification updates – Microsoft plans to certify AMD Instinct MI300 series later in 2026, expanding GPU options for customers with existing AMD infrastructure.
  • Policy-engine enhancements – GitLab’s roadmap includes fine-grained token-level redaction, further mitigating inadvertent data leakage.
  • Quantised model support – Future releases will add 4-bit INT4 inference, reducing GPU memory pressure and cost.
  • Foundry Edge – Microsoft has hinted at an edge offering that would extend self-hosted capabilities to remote branch offices, enabling hybrid deployments where sensitive workloads stay on-premise while less-critical inference runs in the public cloud.
  • Regulatory guidance – As EU and US regulators publish clearer AI-audit standards, the Duo-Foundry stack may become a reference implementation for compliant AI pipelines.

For a broader view of enterprise AI governance, see the recent analysis of governance-first personalization architectures on InfoQ.

Conclusion

GitLab Duo self-hosted AI, powered by Microsoft Foundry, marks a decisive step toward enterprise-grade, on-premise large language model deployments. The technical integration bridges policy-as-code with hardware-level isolation, offering a compelling solution for regulated sectors. However, the shift transfers part of the operational load to internal teams, demanding expertise in GPU management, model lifecycle, and cross-vendor security assurance. Organisations that master this complexity will gain a differentiated capability: AI-enhanced development pipelines that remain fully under their data-sovereignty controls.

Related coverage

Explore more on this topic