Here is the code:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

Load tokenizer and model

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16")
model = AutoModelForCausalLM.from_pretrained(
"nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16",
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device_map="auto"
)
notebook-pythonfrom transformers.utils import get_json_schema

def multiply(a: float, b: float):
"""
A function that multiplies two numbers

Args:
    a: The first number to multiply
    b: The second number to multiply
"""
return a * b

messages = [
{"role": "user", "content": "what is 2.0909090923 x 0.897987987"},
]

tokenized_chat = tokenizer.apply_chat_template(
messages,
tools=[
multiply
],
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
notebook-pythonoutputs = model.generate(
tokenized_chat,
max_new_tokens=1024,
temperature=1.0,
top_p=1.0,
eos_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0]))

Is there a known incompatibility related to tool calling in this model's chat template?

KeyError Traceback (most recent call last)
/usr/local/lib/python3.12/dist-packages/transformers/tokenization_utils_base.py in getattr(self, item)
274 try:
--> 275 return self.data[item]
276 except KeyError:

KeyError: 'shape'

During handling of the above exception, another exception occurred:

AttributeError Traceback (most recent call last)
3 frames
/usr/local/lib/python3.12/dist-packages/transformers/tokenization_utils_base.py in getattr(self, item)
275 return self.data[item]
276 except KeyError:
--> 277 raise AttributeError
278
279 def getstate(self):

AttributeError: