local-ai-test
Setup Python virtual environment
Setup via uv:
uv venv --seed --python 3.12
source .venv/bin/activate
pip install -U openai
How to run?
Export environments then run:
uv run main.py
Run with tiny-encrypt environments:
tiny-encrypt -X -k $(cat ~/.config/envs/LCOAL_DEFAULT_TINY_ENCRYPT_KEY_ID) \
~/secrets/alibaba-cloud-ai-cn.env.tinyenc -- go run main.go
Use local llm-proxy:
OPENAI_API_BASE=http://127.0.0.1:8080/ python main.py
OPENAI_API_BASE=http://127.0.0.1:8080/ python main-stream.py
Reference
- https://help.aliyun.com/zh/model-studio/stream - 大模型服务平台百炼 / 文本生成 / 流式输出