gabriellarson/LFM2-VL-450M-GGUF

LFM2‑VL

LFM2‑VL is Liquid AI's first series of multimodal models, designed to process text and images with variable resolutions. Built on the LFM2 backbone, it is optimized for low-latency and edge AI applications.

We're releasing the weights of two post-trained checkpoints with 450M (for highly constrained devices) and 1.6B (more capable yet still lightweight) parameters.

• 2× faster inference speed on GPUs compared to existing VLMs while maintaining competitive accuracy
• Flexible architecture with user-tunable speed-quality tradeoffs at inference time
• Native resolution processing up to 512×512 with intelligent patch-based handling for larger images, avoiding upscaling and distortion

Find more about our vision-language model in the LFM2-VL post and its language backbone in the LFM2 blog post.

📄 Model details

Due to their small size, we recommend fine-tuning LFM2-VL models on narrow use cases to maximize performance. They were trained for instruction following and lightweight agentic flows. Not intended for safety‑critical decisions.

Property	LFM2-VL-450M	LFM2-VL-1.6B
Parameters (LM only)	350M	1.2B
Vision encoder	SigLIP2 NaFlex base (86M)	SigLIP2 NaFlex shape‑optimized (400M)
Backbone layers	hybrid conv+attention	hybrid conv+attention
Context (text)	32,768 tokens	32,768 tokens
Image tokens	dynamic, user‑tunable	dynamic, user‑tunable
Vocab size	65,536	65,536
Precision	bfloat16	bfloat16
License	LFM Open License v1.0	LFM Open License v1.0

Supported languages: English

Generation parameters: We recommend the following parameters:

• Text: temperature=0.1, min_p=0.15, repetition_penalty=1.05
• Vision: min_image_tokens=64 max_image_tokens=256, do_image_splitting=True

Chat template: LFM2-VL uses a ChatML-like chat template as follows:

<|startoftext|><|im_start|>system
You are a helpful multimodal assistant by Liquid AI.<|im_end|>
<|im_start|>user
<image>Describe this image.<|im_end|>
<|im_start|>assistant
This image shows a Caenorhabditis elegans (C. elegans) nematode.<|im_end|>

Images are referenced with a sentinel (<image>), which is automatically replaced with the image tokens by the processor.

You can apply it using the dedicated .apply_chat_template() function from Hugging Face transformers.

Architecture

• Hybrid backbone: Language model tower (LFM2-1.2B or LFM2-350M) paired with SigLIP2 NaFlex vision encoders (400M shape-optimized or 86M base variant)
• Native resolution processing: Handles images up to 512×512 pixels without upscaling and preserves non-standard aspect ratios without distortion
• Tiling strategy: Splits large images into non-overlapping 512×512 patches and includes thumbnail encoding for global context (in 1.6B model)
• Efficient token mapping: 2-layer MLP connector with pixel unshuffle reduces image tokens (e.g., 256×384 image → 96 tokens, 1000×3000 → 1,020 tokens)
• Inference-time flexibility: User-tunable maximum image tokens and patch count for speed/quality tradeoff without retraining

Training approach

• Builds on the LFM2 base model with joint mid-training that fuses vision and language capabilities using a gradually adjusted text-to-image ratio
• Applies joint SFT with emphasis on image understanding and vision tasks
• Leverages large-scale open-source datasets combined with in-house synthetic vision data, selected for balanced task coverage
• Follows a progressive training strategy: base model → joint mid-training → supervised fine-tuning

🏃 How to run LFM2-VL

You can run LFM2-VL with Hugging Face transformers v4.55 or more recent as follows:

pip install -U transformers pillow

Here is an example of how to generate an answer with transformers in Python:

from transformers import AutoProcessor, AutoModelForImageTextToText
from transformers.image_utils import load_image
# Load model and processor
model_id = "LiquidAI/LFM2-VL-450M"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="bfloat16",
    trust_remote_code=True
)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
# Load image and create conversation
url = "https://www.ilankelman.org/stopsigns/australia.jpg"
image = load_image(url)
conversation = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "What is in this image?"},
        ],
    },
]
# Generate Answer
inputs = processor.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
    tokenize=True,
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
processor.batch_decode(outputs, skip_special_tokens=True)[0]
# This image depicts a vibrant street scene in what appears to be a Chinatown or similar cultural area. The focal point is a large red stop sign with white lettering, mounted on a pole.

You can directly run and test the model with this Colab notebook.

🔧 How to fine-tune

We recommend fine-tuning LFM2-VL models on your use cases to maximize performance.

Notebook	Description	Link
SFT (TRL)	Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using TRL.

📈 Performance

Model	RealWorldQA	MM-IFEval	InfoVQA (Val)	OCRBench	BLINK	MMStar	MMMU (Val)	MathVista	SEEDBench_IMG	MMVet	MME	MMLU
InternVL3-2B	65.10	38.49	66.10	831	53.10	61.10	48.70	57.60	75.00	67.00	2186.40	64.80
InternVL3-1B	57.00	31.14	54.94	798	43.00	52.30	43.20	46.90	71.20	58.70	1912.40	49.80
SmolVLM2-2.2B	57.50	19.42	37.75	725	42.30	46.00	41.60	51.50	71.30	34.90	1792.50	-
LFM2-VL-1.6B	65.23	37.66	58.68	742	44.40	49.53	38.44	51.10	71.97	48.07	1753.04	50.99
Model	RealWorldQA	MM-IFEval	InfoVQA (Val)	OCRBench	BLINK	MMStar	MMMU (Val)	MathVista	SEEDBench_IMG	MMVet	MME	MMLU
-------------------	-------------	-----------	---------------	----------	-------	--------	------------	-----------	---------------	-------	----------	-------
SmolVLM2-500M	49.90	11.27	24.64	609	40.70	38.20	34.10	37.50	62.20	29.90	1448.30	-
LFM2-VL-450M	52.29	26.18	46.51	655	41.98	40.87	33.11	44.70	63.50	33.76	1239.06	40.16

We obtained MM-IFEval and InfoVQA (Val) scores for InternVL 3 and SmolVLM2 models using VLMEvalKit.

📬 Contact

If you are interested in custom solutions with edge deployment, please contact our sales team.