hatter/llm-proxy-go

Go to file

Hatter Jiang 0b906016bc

📦 Update dependencies and add request tracing with UUID

2026-04-19 12:19:00 +08:00

.gitignore

📝 Update .gitignore to include "llm-proxy" entry

2026-04-19 09:29:07 +08:00

go.mod

📦 Update dependencies and add request tracing with UUID

2026-04-19 12:19:00 +08:00

go.sum

📦 Update dependencies and add request tracing with UUID

2026-04-19 12:19:00 +08:00

justfile

🔧 Update justfile with new run alias and modify main.go for config handling improvements

2026-04-19 10:51:22 +08:00

llm-proxy.toml

🔧 Change listen address in configuration file to bind to localhost and add a newline at the end of the file

2026-04-19 10:54:50 +08:00

main.go

📦 Update dependencies and add request tracing with UUID

2026-04-19 12:19:00 +08:00

README.md

📝 Update README with detailed documentation for LLM Proxy

2026-04-19 09:28:02 +08:00

README.md

LLM Proxy

HTTP proxy for LLM APIs with streaming support and chunk processing.

Usage

./llm-proxy

Configuration

Variable	Description	Default
`UPSTREAM_URL`	Upstream LLM API URL	`https://api.openai.com/v1/chat/completions`
`LISTEN_ADDR`	Listen address	`:8080`
`API_KEY`	Upstream API key	-
`INSECURE`	Skip TLS verification	`false`

Example

UPSTREAM_URL=https://api.openai.com/v1/chat/completions \
API_KEY=sk-... \
LISTEN_ADDR=:8080 \
./llm-proxy

Endpoints

GET /health - Health check
/* - Proxies all requests to upstream

Streaming

Supports SSE (text/event-stream) and NDJSON (application/x-ndjson) streaming. Each chunk is processed via processChunk() before forwarding.