36 lines
810 B
Markdown
36 lines
810 B
Markdown
# LLM Proxy
|
|
|
|
HTTP proxy for LLM APIs with streaming support and chunk processing.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
./llm-proxy
|
|
```
|
|
|
|
## Configuration
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| `UPSTREAM_URL` | Upstream LLM API URL | `https://api.openai.com/v1/chat/completions` |
|
|
| `LISTEN_ADDR` | Listen address | `:8080` |
|
|
| `API_KEY` | Upstream API key | - |
|
|
| `INSECURE` | Skip TLS verification | `false` |
|
|
|
|
## Example
|
|
|
|
```bash
|
|
UPSTREAM_URL=https://api.openai.com/v1/chat/completions \
|
|
API_KEY=sk-... \
|
|
LISTEN_ADDR=:8080 \
|
|
./llm-proxy
|
|
```
|
|
|
|
## Endpoints
|
|
|
|
- `GET /health` - Health check
|
|
- `/*` - Proxies all requests to upstream
|
|
|
|
## Streaming
|
|
|
|
Supports SSE (`text/event-stream`) and NDJSON (`application/x-ndjson`) streaming. Each chunk is processed via `processChunk()` before forwarding. |