Skip to content

feat: add nanoVLM model support - #12

Open
reacher-z wants to merge 1 commit into
InternScience:mainfrom
reacher-z:feat/nanovlm-support
Open

feat: add nanoVLM model support#12
reacher-z wants to merge 1 commit into
InternScience:mainfrom
reacher-z:feat/nanovlm-support

Conversation

@reacher-z

Copy link
Copy Markdown

Summary

  • add a SciEvalKit adapter for the official pure-PyTorch nanoVLM implementation
  • register the public 230M-8k and 460M-8k checkpoints
  • support multiple images, current image-splitting checkpoints, text-only inputs, and automatic CUDA/MPS/CPU selection
  • provide an actionable installation error directing users to set NANOVLM_PATH
  • add focused regression tests for model setup, image preprocessing, prompt construction, generation, decoding, and text-only input

This is an independent new foundation-model integration, one of the major-contribution categories described in the SciEvalKit README.

Sources:

Validation

  • python3 -m pytest -q tests/test_nanovlm.py — 2 passed
  • python3 -m py_compile scieval/config.py scieval/vlm/__init__.py scieval/vlm/nanovlm.py tests/test_nanovlm.py
  • pre-commit hooks pass for the new adapter, import, and tests
  • real MPS smoke test against the current official nanoVLM checkout and public lusxvr/nanoVLM-230M-8k weights: the adapter loaded a cached checkpoint in 8.89s, generated in 3.67s, and returned Cat. for the official cat image and prompt “What animal is shown? Answer briefly.”

The full config file has pre-existing lint violations outside this diff, so validation also included git diff --check and compilation of the modified config.

Setup

git clone https://github.com/huggingface/nanoVLM.git
export NANOVLM_PATH=/path/to/nanoVLM

Install the dependencies listed by nanoVLM, then select nanoVLM-230M-8k or nanoVLM-460M-8k through SciEvalKit.

AI assistance disclosure

OpenAI Codex assisted with implementation, tests, and PR drafting. I ran and inspected both the focused tests and real-model smoke inference before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant