Instructions to use SakuraLLM/GalTransl-v4-4B-2601 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use SakuraLLM/GalTransl-v4-4B-2601 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S # Run inference directly in the terminal: llama cli -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S # Run inference directly in the terminal: llama cli -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S # Run inference directly in the terminal: ./llama-cli -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S # Run inference directly in the terminal: ./build/bin/llama-cli -hf SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
Use Docker
docker model run hf.co/SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
- LM Studio
- Jan
- Ollama
How to use SakuraLLM/GalTransl-v4-4B-2601 with Ollama:
ollama run hf.co/SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
- Unsloth Studio
How to use SakuraLLM/GalTransl-v4-4B-2601 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for SakuraLLM/GalTransl-v4-4B-2601 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for SakuraLLM/GalTransl-v4-4B-2601 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for SakuraLLM/GalTransl-v4-4B-2601 to start chatting
- Docker Model Runner
How to use SakuraLLM/GalTransl-v4-4B-2601 with Docker Model Runner:
docker model run hf.co/SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
- Lemonade
How to use SakuraLLM/GalTransl-v4-4B-2601 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull SakuraLLM/GalTransl-v4-4B-2601:Q5_K_S
Run and chat with the model
lemonade run user.GalTransl-v4-4B-2601-Q5_K_S
List all available models
lemonade list
- Atomic Chat
针对RPG Maker控制符有没有什么好的处理方案
比如\N[1],\C[1]这种,短句子还可以,长句子非常容易丢。
我现在为了防止丢控制符,会现在代码里替换成NAME_1,COLOR_2这样,然后在翻译结果里检查是否存在。
测试下来文本内容太长还是会丢后面的控制符。
另外由于不是真的角色名字,有时翻译会加代词,比如 \N[1]xxxx -> 我NAME_1xxxx,加个代词进去。
标题、地名之类的词语翻译时喜欢在句尾加个句号。(一阶 -> 第一层。)
有的时候如果开头符号(斜线、点太多),可能会出一句完全不相干的句子,但我不确定是不是我模型加载或者并发请求的问题或者LMstudio新版本的问题,暂且提一句。
整体翻译水平倒是没和之前V3.8看出差别,期待全新的14b的版本。
GalTransl系列模型的后训练语料是100%的galgame文本,所以对于rpg游戏的适配性不是很好,后面版本会考虑增加rpg文本来进一步改善控制符问题。
https://huggingface.co/Murasaki-Project/Murasaki-APE-Aligner-2B
推荐下这个对齐模型,专门对齐翻译任务中文本混杂控制符与代码的情况,可以考虑试一试效果。