I have just been talking to Sudeep Goswami, CEO of TraefikLabs, about the rapid growth of the API (Application Programming Interface) management market, the rise of AI inferencing workloads, and how Traefik’s open-source API Gateway and new AI Gateway extensions provide a seamless, secure path for enterprises evolving their digital infrastructure. He sees a booming API market driven by digital transformation, standardization and security pressures. And AI is part of this, despite the hype, as peoples’ interest is shifting from training AI models to “inferencing”, using AI to generate business insights and to make business decisions based on the business data they already have.

How well this actually works largely remains to be seen, but the business (and its auditors) will require observability, and auditability around what the AI is doing – and guardrails, to limit the possible scope of a misbehaving AI.

Sudeep sees this as another opportunity for Traefik’s API management ecosystem. It is good practice to access AI tools, or AI-enabled tools through an API and Traefik’s architecture layers controls around API calls without modifying the underlying models.

Traefik began as a simple open-source API Gateway on GitHub, in 2016. Sudeep says that: “We have been the number one API gateway downloaded from the GitHub community, and we are consistently ranked in the top 15 Docker images that downloaded daily. We have about 3.3 billion downloads, hundreds and thousands of nodes in production, a vibrant community of contributors. And the key characteristics that both we and our users look at are ease of use (how intuitive it all is); how cloud native it is; how Kubernetes native it is.”

He says, however, that successful as the open-source product is, it is currently only about 20% of his business (this is intentional as Traefik doesn’t want to get the bulk of its revenue from just selling Enterprise support). The remaining business is with the API Gateway and API Management. Digital transformation has accelerated enterprises’ dependence on APIs as critical integration points – and at the same time, expanded the attack surface they expose. Since AI workloads are shifting from heavy model training to massive inferencing projects, and AI is being embedded in many tools and business-critical systems, this increases the demands on governance, change management and security for the APIs to these tools and systems. Hence the increasing need for an API Gateway and API Management as Sudeep says “our AI gateway… provides a layer of control and defense without changing the model’s behavior itself”.

A lot of good governance comes with Transparency and Observability. Traefik natively emits OpenTelemetry metrics, logs, and traces, enabling teams to correlate code changes with production KPIs (Key Performance Indicators. For example, overlaying Git CI/CD pull-merge timestamps on error-rate graphs in Grafana instantly surfaces regression issues. As Sudeep notes, “If you do everything as code, rich observability is trivial; with “click-ops”, it’s a nightmare at scale”.

Serving enterprise adopters, Traefik works directly with large enterprise adopters who understand GitOps and Infrastructure-as-Code, and provides them with minimal to medium assistance as needed (direct partnership with Sis might be a future opportunity, however). It offers its customers:

• Proactive support during sales and Proof of Concept (POC)

• Reactive support when high-pressure events, such as Black Friday traffic spikes, occur.

• Staged rollouts (POC → staging → production) with a focus on catching mistakes early, before they reach production.

Traefik also embeds into cloud, hardware and monitoring ecosystems via strategic alliances with the likes of Nutanix, HashiCorp, Oracle Cloud Infrastructure (OCI), Microsoft Azure and more.

This has been a very quick run-through of the Traefik offering. I think it offers an almost textbook example of how a customer can start small with a “commercial open source” product (ie, “free” software with paid-for support) and grow it into an enterprise-strength commercial installation as each step proves itself with use. My key general takeaways from our discussion were:

• Auditability Is Non-Negotiable. Every API call (especially to an AI) must be logged, traceable and reversible.

• Continuous Evolution lowers risk. APIs and AI-enabled pipelines must be treated as mutable infrastructure—versioned, governed, and redeployed via a continual delivery pipeline

• External controls for APIs are needed to prevent AI models, in particular, from adopting undesirable corporate biases from existing corporate cultures – but, in reality, it is good practice to consider Ethics from the first and address any ethical issues in your culture before AI has the chance to learn from them.

In my opinion, the Traefik tools and ecosystem should be considered as an enabler of good practice with APIs and AI.

Print Article Download here