These are not three mutually exclusive categories. LLM and SLM mainly describe model scale; VLM describes the kinds of information a model handles. A vision-language model can also be small. [4] SmolVLM: Redefining small and efficient multimodal models
Large Language Model
Broad language capabilities
Trained on large datasets, LLMs can generate, summarize and translate text. The GPT-3 research showed that one model could handle multiple language tasks from a few examples. They are a useful starting point when exploring writing and a broad range of language workflows.
[1] Language Models are Few-Shot LearnersVision-Language Model
Language meets visual input
VLMs process images together with text, enabling tasks such as asking questions about an image or describing a chart. LLaVA is a research example that connects an image-processing component with a language model.
[2] Visual Instruction TuningSmall Language Model
A smaller computational footprint
SLMs are comparatively small language models. Research on Phi-3-mini demonstrated a model that could run on a phone. They are worth exploring for constrained compute or on-device processing, although accuracy, speed and cost depend on the specific model and environment.
[3] Phi-3 Technical ReportHow these concepts translate into workflows.
These are possible starting points from Asymmetra’s perspective. Suitability depends on the data and the quality the task requires.
LLM: language-based work
Meeting summaries, proposal drafts and inquiry classification are potential use cases. Include a review step for names, figures and supporting evidence before using the output.
VLM: connecting images and language
Potential applications include interpreting forms or screens and organizing their contents in text. Small print and complex tables can be misread, so evaluate with actual examples.
SLM: narrowly defined tasks
Consider small models for routine classification or extraction with limited computing resources. Small size does not guarantee adequate accuracy or speed; evaluate the model and its execution environment together.
Company knowledge needs a retrieval strategy.
A larger model does not automatically gain access to internal documents. Retrieval-augmented generation, or RAG, retrieves relevant material and uses it to inform generated responses. In our view, useful business systems also need a plan for document updates, access permissions, source references and questions the system cannot answer.
Reference: Lewis et al., Retrieval-Augmented Generation (2020)Start with the quality the task requires.
Our view is that model size alone is not a sufficient selection criterion. Compare quality, response time and operating cost on real work examples. Using different models for complex and routine tasks may improve resource efficiency while preserving the quality that matters. The table below offers starting points for evaluation.
| The task | A starting point | What to verify |
|---|---|---|
| Broad writing and summarization | Start by evaluating an LLM | Output quality and review effort |
| Work involving images or charts | Start by evaluating a VLM | Visual interpretation accuracy |
| Narrow on-device tasks | Consider evaluating an SLM | Device resources and required quality |