possible to show model reasoning in output

[PaoloJr]

is it possible to have ramalama output the model's reasoning to the terminal? For example, using llama cli -hf [huggingface_model] -p ["prompt"] displays it.

with ramalama run [huggingface_mode], after entering a prompt, the output is the end-response from the model.

I know there is a --thinking [true|false] option for run. Could this also enable model reasoning output?

[olliewalsh]

I believe this should be possible to implement, IIRC the reasoning response is currently just discarded by the client. I would not overload the --thinking option though, better to have a separate option to control it.

[PaoloJr]

a welcome addition 👍

i suspect this discarding is handled similarly with vllm as with llama.cpp? i only tested with llama.cpp
open to learning more about the repo. I only got to read a little on cli.py, chat.py and llama_cpp.py

[rhatdan]

Is this workaround enough for me to close the issue, or do you still think there should be an option? We could also update the man pages to describe this method?

[PaoloJr]

yep, its enough for me. I am open to updating the man page if you like 👍