🗑️ Remove local-api-test project files and add new untracked directory local-ai-test/
This commit is contained in:
29
projects/local-ai-test/README.md
Normal file
29
projects/local-ai-test/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# local-ai-test
|
||||
|
||||
## How to run?
|
||||
|
||||
Export environments then run:
|
||||
|
||||
```shell
|
||||
uv run main.py
|
||||
```
|
||||
|
||||
Run with `tiny-encrypt` environments:
|
||||
|
||||
```shell
|
||||
tiny-encrypt -X -k $(cat ~/.config/envs/LCOAL_DEFAULT_TINY_ENCRYPT_KEY_ID) \
|
||||
~/secrets/alibaba-cloud-ai-cn.env.tinyenc -- go run main.go
|
||||
```
|
||||
|
||||
Use local llm-proxy:
|
||||
|
||||
```shell
|
||||
OPENAI_API_BASE=http://127.0.0.1:8080/ python main.py
|
||||
|
||||
OPENAI_API_BASE=http://127.0.0.1:8080/ python main-stream.py
|
||||
```
|
||||
|
||||
## Reference
|
||||
|
||||
- https://help.aliyun.com/zh/model-studio/stream - 大模型服务平台百炼 / 文本生成 / 流式输出
|
||||
|
||||
Reference in New Issue
Block a user