> For the complete documentation index, see [llms.txt](https://ai-security-docs.akto.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-security-docs.akto.io/akto-argus-agentic-ai-security-for-homegrown-ai/connectors/ai-agent-security/connect-akto-with-huggingface.md).

# Hugging Face

## Overview

This guide explains how to integrate **Akto AI Agent Gateway** with a **Hugging Face Private Inference Endpoint** used by customers to run private LLM inference. The gateway sits between the end user and the agent application (Option B) to monitor, enforce guardrails, and log model invocation traffic without modifying internal client code.

Akto AI Agent Gateway provides:

* Guardrail enforcement on both requests and responses
* Sensitive data redaction
* Security guardrail detection

Hugging Face’s Private Inference Endpoint provides a dedicated, managed model endpoint accessible only via AWS PrivateLink from within a VPC. Hugging Face does **not automatically log full prompt & response conversations** like AWS Bedrock, so Akto must capture this upstream.

## **Prerequisites**

Before integrating Akto Gateway:

1. A working **Hugging Face Private Inference Endpoint** configured with PrivateLink.
2. AWS VPC where the endpoint service is reachable.
3. The AI agent and Akto Gateway deployed in the same VPC or with network access to the PrivateLink interfaces.
4. Access credentials for Hugging Face inference (API token).

***

## **Architecture Diagram**

```mermaid
flowchart LR
    A[EndUser] --> B[Akto AI Agent Gateway] --> C[AI Agent App] --> D[HF Private Inference Endpoint]
    B --> E[Logs and Guardrails Analytics]


```

1. End user calls the **AI agent API**.
2. **Akto Gateway** intercepts requests (guardrail enforcement).
3. Gateway forwards to **HF Private Inference Endpoint** (via PrivateLink).
4. Responses pass back through Akto Gateway.
5. Akto logs, analyzes and optionally redacts or blocks results.

## **Setup Steps**

{% stepper %}
{% step %}
**Configure Hugging Face Private Inference Endpoint**

Ensure the endpoint is set up with:

* Model deployed
* **PrivateLink enabled**
* Correct AWS account and region
* VPC interface endpoint created in your VPC

Hugging Face does not log full request/response content by itself. You must capture it upstream.
{% endstep %}

{% step %}
**Deploy Akto AI Agent Gateway**

Deploy the gateway in the same VPC where:

* End user traffic enters
* The AI agent application runs
* The PrivateLink interface to HF endpoint exists
  {% endstep %}

{% step %}
**Configure Gateway Environment**

Here is an example config for the gateway:

```bash
export AKTO_API_TOKEN=<YOUR_AKTO_PROXY_TOKEN>
export AKTO_API_BASE_URL=<AKTO_API_BASE_URL>
export APP_URL=<HUGGING_FACE_PRIVATELINK_ENDPOINT_URL>
export LOG_LEVEL=INFO

```

* `AKTO_API_TOKEN`: Akto ingestion token (go to **Akto Argus → Connectors → Setup Guardrail** card to obtain it)
* `AKTO_API_BASE_URL`: Akto gateway ingestion server. Follows the format `https://<account_id>-guardrails.akto.io`; contact the Akto support team to get the URL for your account.
* `APP_URL`: Upstream target (the HF Private Inference Endpoint URL)
* `LOG_LEVEL`: Logging verbosity
  {% endstep %}

{% step %}
**Adjust Endpoint URL in Agent App**

Update the AI agent’s inference call configuration:

* Set model base URL to the **Akto Gateway endpoint**
* Pass Hugging Face authentication headers through gateway

For example:

```
AI_AGENT_INFERENCE_URL=https://akto-proxy.internal.svc
HF_AUTHORIZATION=Bearer <HF_TOKEN>

```

This ensures:

* Traffic flows through Akto Gateway
* Akto captures all inference calls
  {% endstep %}

{% step %}
**Validate Integration**

Verify end-to-end flow:

1. Send an inference request from the user
2. Akto Gateway receives and logs the call
3. Gateway enforces any guardrails
4. Gateway forwards to HF Private Endpoint
5. Response returns through Akto Gateway
6. Logs appear in Akto dashboard

Look for:

* Request/response pairs in gateway logs
* Guardrail hits (if configured)
* Redaction results
  {% endstep %}
  {% endstepper %}

## **Security & Guardrails**

Akto Gateway supports:

* Request guardrails (input sanitization)
* Response guardrails (filtering outputs)
* Redaction of sensitive tokens or PII
* Rate limiting and anomaly detection

Use our policy packs or define custom rules based on:

* Content patterns
* Risk categories
* Endpoint sensitivity

## **Logging & Monitoring**

Hugging Face Private Endpoints offer:

* Operational logs (status, errors)
* Metrics (latency, throughput)

They do **not log conversation content** by default.

Akto Gateway will log:

* Full request and response traces
* Guardrail decision events
* Alerts and incidents
* Metadata for analytics

## **Troubleshooting**

* **Gateway cannot reach HF Endpoint**: Check PrivateLink and VPC routing.
* **Auth failures**: Verify Hugging Face API token headers are passed by gateway.
* **No logs in Akto**: Confirm AKTO\_API\_TOKEN and ingestion config.
* **Guardrail not triggering**: Validate rule pack configuration.

## **Summary**

By integrating Akto AI Agent Gateway in front of a Hugging Face Private Inference Endpoint:

* You achieve guardrail enforcement without modifying the client code
* You capture and monitor model invocation traffic
* You gain observability of conversation logging

Akto Gateway becomes the enforcement and observability layer for private HF model usage.