Multimodal large language models (MLLMs) are rapidly advancing, yet their reasoning ability often lags behind that of strong text-only counterparts. Existing methods to bridge this gap rely on supervised fine-tuning over large-scale multimodal reasoning data or reinforcement learning, both of which are resource-intensive. A promising alternative is model merging, which interpolates parameters between reasoning-enhanced LLMs and multimodal variants. However, our analysis shows that naive merging is not always a "free lunch": its effectiveness varies drastically across model families, with some (e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance degradation. To address this, we propose Directional Reasoning Injection for Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning knowledge in the gradient space, without destabilizing multimodal alignment. DRIFT precomputes a reasoning prior as the parameter-space difference between reasoning and multimodal variants, then uses it to bias gradients during multimodal fine-tuning. This approach preserves the simplicity of standard supervised fine-tuning pipelines while enabling efficient reasoning transfer. Extensive experiments on multimodal reasoning benchmarks, including MathVista and MathVerse, demonstrate that DRIFT consistently improves reasoning performance over naive merging and supervised fine-tuning, while matching or surpassing training-heavy methods at a fraction of the cost.
DRIFT precomputes a reasoning prior as the parameter-space delta between a reasoning model and a multimodal model. During multimodal SFT, gradients are guided toward this prior, injecting reasoning ability while preserving alignment.
Concretely, let Δθ be the difference between a reasoning-enhanced LLM and its multimodal counterpart. We bias the gradient updates in the direction of Δθ during fine-tuning on multimodal data, yielding improved reasoning without destabilizing the vision-language interface.
Effect of model merging on multimodal reasoning benchmarks.
Performance on MathVista, MathVision, and MathVerse for four MLLMs before and after merging with their text-only reasoning experts. Scores include relative change (rel.) versus the base model.
| Benchmark | LLaVA-Next-LLaMA3-8B | Idefics-8B | Qwen2-VL-7B | Qwen2.5-VL-7B | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Base | +Dart-Uniform | rel. | Base | +MetaMath | rel. | Base | +Qwen2-Math | rel. | Base | +DeepSeek-R1 | rel. | |
| MathVista | 37.4 | 38.2 | +0.8 | 51.8 | 53.2 | +1.4 | 61.2 | 60.2 | -1.0 | 67.9 | 65.8 | -2.1 |
| MathVision | 13.8 | 15.8 | +2.0 | 17.1 | 11.8 | -5.3 | 21.1 | 21.7 | +0.6 | 25.0 | 22.7 | -2.3 |
| MathVerse | 16.0 | 17.4 | +1.4 | 11.0 | 12.4 | +1.4 | 26.9 | 26.7 | -0.2 | 41.4 | 33.2 | -8.2 |
rel. values denote absolute score differences relative to the Base model.
Layer/Module-wise analysis of model merging pairs
We compare LLaVA-Next-8B vs. Dart-Uniform, Idefics-8B vs. MetaMath, Qwen2-VL-7B vs. Qwen2-Math-7B, and Qwen2.5-VL-7B vs. DeepSeek-R1-Qwen-7B. Top-Left: per-layer L2 norm differences. Bottom-Left: per-layer cosine similarity. Top-Right: average L2 norm differences for FFN and normalization layers. Bottom-Right: average L2 norm differences for attention projections (Q/K/V/O).
Evaluation results on multimodal reasoning benchmarks.
We compare our gradient-based merging approach with standard parameter-space merging baselines. Results are reported on MathVista, MathVision, MathVerse, WeMath (strict/loose), and LogicVista. Best results are in bold. Improvements are reported relative to Baseline.
| Model | MathVista | MathVision | MathVerse | WeMath (strict) | WeMath (loose) | LogicVista | Avg. |
|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-7B-Instruct | 67.9 | 25.0 | 41.4 | 34.3 | 52.8 | 46.7 | 44.7 |
| Parameter merging with DeepSeekR1-Qwen-Distill-7B | |||||||
| Task Arithmetic | 65.8-2.1 | 22.7-2.3 | 33.2-8.2 | 30.1-4.2 | 51.2-1.6 | 42.0-4.7 | 40.8-3.9 |
| Layer Swap | 63.6-4.3 | 22.9-2.1 | 37.9-3.5 | 32.1-2.2 | 50.1-2.7 | 35.1-11.6 | 40.3-4.4 |
| TIES | 63.6-4.3 | 23.1-1.9 | 39.5-1.9 | 33.4-0.9 | 51.7-1.1 | 42.1-4.6 | 42.2-2.5 |
| DARE-TIES | 66.3-1.6 | 23.6-1.4 | 38.3-3.1 | 33.7-0.6 | 52.6-0.2 | 42.0-4.7 | 42.8-1.9 |
| DARE-Linear | 66.0-1.9 | 22.3-2.7 | 35.5-5.9 | 30.8-3.5 | 51.2-1.6 | 42.5-4.2 | 41.4-3.3 |
| Reasoning Injection from DeepSeekR1-Qwen-Distill-7B | |||||||
| DRIFT (Ours) | 69.9+2.0 | 26.6+1.6 | 43.9+2.5 | 38.5+4.2 | 60.2+7.4 | 47.2+0.5 | 47.7+3.0 |
Subscript values denote absolute differences relative to the Baseline.
Evaluation results on visual reasoning benchmarks.
We report performance on MathVista, MathVision, MathVerse, WeMath (strict), and LogicVista across open-source models and reasoning fine-tuning methods. Our DRIFT results are bold, with improvements relative to our SFT baseline shown as green subscripts.
| Model | MathVista | MathVision | MathVerse | WeMath | LogicVista |
|---|---|---|---|---|---|
| Open-source Models | |||||
| LLaVA-OneVision-7B | 62.6 | 17.6 | 17.6 | 17.7 | 32.0 |
| InternLM-XComposer2.5 | 64.0 | 17.8 | 16.2 | 14.1 | 34.7 |
| InternVL3-8B | 70.5 | 28.6 | 33.9 | 37.5 | 43.6 |
| InternVL2.5-8B | 64.5 | 17.0 | 22.8 | 23.5 | 36.0 |
| InternVL2-8B | 58.3 | 20.0 | 20.4 | 20.2 | 33.6 |
| QvQ-72B-Preview | 70.3 | 34.9 | 48.2 | 39.0 | 58.2 |
| Kimi-VL-16B | 66.0 | 21.8 | 34.1 | 32.3 | 42.7 |
| Qwen2-VL-7B | 61.6 | 19.2 | 25.4 | 22.3 | 33.3 |
| Qwen2.5-VL-7B†| 67.9 | 25.0 | 41.4 | 34.3 | 46.7 |
| Reasoning Fine-tuning Methods | |||||
| R1-Onevision-7B | 64.1 | 29.9 | 40.0 | — | 61.8 |
| OpenVLThinker-7B | 65.3 | 23.0 | 38.1 | 35.2 | 44.5 |
| R1-VL-7B | 63.5 | 24.7 | 40.0 | — | — |
| X-REASONER | 69.0 | 29.6 | — | — | — |
| Ours (SFT) | 68.7 | 25.1 | 42.0 | 33.3 | 45.6 |
| DRIFT (Ours) | 69.9+1.2 | 26.6+1.6 | 43.9+1.9 | 38.5+5.2 | 47.2+1.6 |
†indicates results reproduced by ourselves; other baselines are as reported by Open Vision Reasoner. Subscripts show improvements over our SFT baseline.
Training schemes and estimated wall-clock cost.
Existing reasoning-focused methods require days of SFT and/or RL on large-scale multimodal CoT data. DRIFT needs only SFT-style training with a precomputed reasoning prior, finishing in about two hours on comparable hardware and using just 4K curated examples (vs. >59K for training-heavy methods).
| Method | SFT | RL | Est. training time |
|---|---|---|---|
| OpenVLThinker-7B | ✓ | ✗ | > 1 day |
| R1-OneVision-7B | ✓ | ✗ | > 1 day |
| X-REASONER | ✓ | ✓ | > 2 days |
| DRIFT (Ours) | ✓ | ✗ | ~2 hours |
Estimated under comparable hardware. DRIFT adds only lightweight gradient-time operations with a precomputed prior and leaves the forward pass unchanged.
Generality across backbone–expert pairings.
DRIFT is not tied to a single architecture. We apply it to two backbones (Qwen2.5-VL-7B and LLaVA-Next-8B) with different text-only reasoning experts. It consistently improves over both the base model and SFT across all pairings, and stronger reasoning experts yield larger gains.
| Backbone | Method | MathVision | WeMath (strict) | WeMath (loose) |
|---|---|---|---|---|
| Qwen2.5-VL-7B | Baseline | 25.00 | 34.30 | 52.80 |
| SFT | 25.10 | 33.30 | 55.80 | |
| DRIFT (Qwen2.5-Math) | 26.00 | 36.70 | 59.30 | |
| DRIFT (DeepSeek-R1) | 26.60 | 38.50 | 60.20 | |
| LLaVA-Next-8B | Baseline | 14.27 | 9.05 | 24.86 |
| SFT | 15.16 | 9.52 | 25.90 | |
| DRIFT (DART-Uniform) | 16.94 | 10.57 | 27.33 |
Reasoning experts: DeepSeek-R1 = DeepSeek-R1-Distill-7B, Qwen2.5-Math = Qwen2.5-Math-7B, DART-Uniform = DART-Uniform-8B.
Impact on general multimodal perception.
A natural concern is whether injecting reasoning degrades general perception. On HallusionBench, RealWorldQA, and MMStar, DRIFT preserves or improves perception, whereas plain SFT regresses on RealWorldQA and MMStar.
| Method | HallusionBench | RealWorldQA | MMStar |
|---|---|---|---|
| Qwen2.5-VL (Base) | 44.39 | 68.62 | 64.70 |
| SFT | 48.79+4.40 | 66.79-1.83 | 62.80-1.90 |
| DRIFT (Ours) | 48.79+4.40 | 69.15+0.53 | 65.60+0.90 |
Subscripts denote absolute differences relative to the Qwen2.5-VL base model.
Robustness to dataset quality and scale.
Does DRIFT depend on the curated 4K high-quality set? We compare the 4K high-quality (HQ) data with an 8K mix that adds 4K noisy/unfiltered samples. DRIFT beats SFT in both settings and degrades more gracefully under noise, acting as a regularizer. Notably, 4K high-quality data outperforms the larger-but-noisier 8K set.
| Data | Method | MathVision | WeMath (strict) | WeMath (loose) |
|---|---|---|---|---|
| 4K HQ | SFT | 25.10 | 33.30 | 55.80 |
| DRIFT | 26.60 | 38.50 | 60.20 | |
| 8K Mixed | SFT | 23.20 | 33.90 | 56.19 |
| DRIFT | 25.70 | 35.62 | 59.62 |
8K Mixed = 4K high-quality samples + 4K noisy/unfiltered samples.
Comparison of scaling strategies in DRIFT.
Where and how should reasoning be injected? Applying DRIFT to attention projections (ATTN) is most effective, while extending to normalization layers can inject noise. Among strategies, the Absolute update is too aggressive and hurts, whereas gradient-based scaling (Grad-Norm and Grad-Norm w/ Adaptive α) is stable and reliably helps. Candidates: attention projections (ATTN), feed-forward layers (MLP), input/output normalization (Norm), and the output LM projection head (LM Head).
| Scaling Strategy | Merge Candidates | MathVista | MathVerse | LogicVista |
|---|---|---|---|---|
| SFT | — | 68.7 | 42.0 | 45.6 |
| Absolute | {ATTN, MLP} | 65.7-3.0 | 39.5-2.5 | 25.9-19.7 |
| Grad-Norm | {ATTN, MLP} | 68.8+0.1 | 43.9+1.9 | 46.1+0.5 |
| Grad-Norm w/ Adaptive α | {ATTN, MLP} | 69.9+1.2 | 43.9+1.9 | 47.2+1.6 |
| Grad-Norm | {ATTN} | 68.8+0.1 | 44.4+2.4 | 49.4+3.8 |
| Grad-Norm | {MLP} | 68.5-0.5 | 42.6+0.6 | 46.3+0.7 |
| Grad-Norm | {ATTN, MLP, Norm} | 68.6-0.1 | 43.0+1.0 | 46.8+1.2 |
| Grad-Norm | {ATTN, MLP, Norm, LM Head} | 68.6-0.1 | 42.7+0.7 | 48.5+2.9 |
| Grad-Norm w/ Adaptive α | {ATTN} | 68.8+0.1 | 43.2+1.2 | 48.3+2.7 |
| Grad-Norm w/ Adaptive α | {MLP} | 69.0+0.3 | 43.3+1.3 | 46.3+0.7 |
| Grad-Norm w/ Adaptive α | {ATTN, MLP, Norm} | 69.1+0.4 | 42.8+0.8 | 48.3+2.7 |
| Grad-Norm w/ Adaptive α | {ATTN, MLP, Norm, LM Head} | 68.8+0.1 | 43.3+1.3 | 48.3+2.7 |
Highlighted row is the default DRIFT configuration. Subscripts denote absolute differences relative to the SFT baseline.
