--- language: - en license: apache-2.0 library_name: llm2ner base_model: EleutherAI/pythia-12b tags: - ner - span-detection - llm - pytorch pipeline_tag: token-classification model_name: ToMMeR-pythia-12b_L5_R64 source: https://github.com/VictorMorand/llm2ner paper: https://arxiv.org/abs/2510.19410 --- # ToMMeR-pythia-12b_L5_R64 [![Paper](https://img.shields.io/badge/Paper-Arxiv-red)](https://arxiv.org/abs/2510.19410) [![All Models](https://img.shields.io/badge/🤗%20Hugging%20Face%20Models-blue)](https://huggingface.co/llm2ner) [![GitHub](https://img.shields.io/badge/GitHub-Code-blue)](https://github.com/VictorMorand/llm2ner) ToMMeR is a lightweight probing model extracting emergent mention detection capabilities from early layers representations of any LLM backbone, achieving high Zero Shot recall across a wide set of 13 NER benchmarks. ## Model Details This model can be plugged at layer 5 of `EleutherAI/pythia-12b`, with a computational overhead not greater than an additional attention head. | Property | Value | |-----------|-------| | Base LLM | `EleutherAI/pythia-12b` | | Layer | 5| | #Params | 660.5K | # Usage ## Installation To use ToMMeR, you need to install its codebase first. ```bash pip install git+https://github.com/VictorMorand/llm2ner.git ``` ## Raw inference By default, ToMMeR outputs span probabilities, but we also propose built-in options for decoding entities. - Inputs: - tokens (batch, seq): tokens to process, - model: LLM to extract representation from. - Outputs: (batch, seq, seq) matrix (masked outside valid spans) ```python from xpm_torch.huggingface import TorchHFHub from llm2ner import ToMMeR, utils tommer: ToMMeR = TorchHFHub.from_pretrained("llm2ner/ToMMeR-pythia-12b_L5_R64") # load Backbone llm, optionnally cut the unused layer to save GPU space. llm = utils.load_llm( tommer.llm_name, cut_to_layer=tommer.layer,) tommer.to(llm.device) #### Raw Inference text = ["Large language models are awesome"] print(f"Input text: {text[0]}") #tokenize in shape (1, seq_len) tokens = llm.tokenizer(text, return_tensors="pt")["input_ids"].to(llm.device) # Output raw scores output = tommer.forward(tokens, llm) # (batch_size, seq_len, seq_len) print(f"Raw Output shape: {output.shape}") #use given decoding strategy to infer entities entities = tommer.infer_entities(tokens=tokens, model=llm, threshold=0.5, decoding_strategy="greedy") str_entities = [ llm.tokenizer.decode(tokens[0,b:e+1]) for b, e in entities[0]] print(f"Predicted entities: {str_entities}") >>>INFO:root:Cut LlamaModel with 16 layers to 7 layers >>> Input text: Large language models are awesome >>> Raw Output shape: torch.Size([1, 6, 6]) >>> Predicted entities: ['Large language models'] ``` ## Fancy Outputs We also provide inference and plotting utils in `llm2ner.plotting`. ```python from xpm_torch.huggingface import TorchHFHub from llm2ner import ToMMeR, utils, plotting tommer: ToMMeR = TorchHFHub.from_pretrained("llm2ner/ToMMeR-pythia-12b_L5_R64") # load Backbone llm, optionnally cut the unused layer to save GPU space. llm = utils.load_llm( tommer.llm_name, cut_to_layer=tommer.layer,) tommer.to(llm.device) text = "Large language models are awesome. While trained on language modeling, they exhibit emergent Zero Shot abilities that make them suitable for a wide range of tasks, including Named Entity Recognition (NER). " #fancy interactive output outputs = plotting.demo_inference( text, tommer, llm, decoding_strategy="threshold", # or "greedy" for flat segmentation threshold=0.5, # default 50% show_attn=True, ) ```

Large PRED language PRED models are awesome . While trained on language PRED modeling , they exhibit emergent PRED abilities that make them suitable for a wide range of tasks PRED , including Named PRED Entity Recognition ( NER PRED ) .

Please visit the [repository](https://github.com/VictorMorand/llm2ner) for more details and a demo notebook. ## Evaluation Results | dataset | precision | recall | f1 | n_samples | |---------------------|-------------|----------|--------|-------------| | MultiNERD | 0.2063 | 0.957 | 0.3395 | 154144 | | CoNLL 2003 | 0.2794 | 0.7083 | 0.4007 | 16493 | | CrossNER_politics | 0.2963 | 0.9448 | 0.4511 | 1389 | | CrossNER_AI | 0.3122 | 0.9153 | 0.4656 | 879 | | CrossNER_literature | 0.343 | 0.8843 | 0.4943 | 916 | | CrossNER_science | 0.3473 | 0.9175 | 0.5038 | 1193 | | CrossNER_music | 0.3795 | 0.9229 | 0.5379 | 945 | | ncbi | 0.1189 | 0.8562 | 0.2087 | 3952 | | FabNER | 0.2963 | 0.7147 | 0.4189 | 13681 | | WikiNeural | 0.1943 | 0.9315 | 0.3216 | 92672 | | GENIA_NER | 0.2368 | 0.9285 | 0.3774 | 16563 | | ACE 2005 | 0.2399 | 0.3416 | 0.2819 | 8230 | | Ontonotes | 0.23 | 0.6736 | 0.3429 | 42193 | | Aggregated | 0.216 | 0.8781 | 0.3468 | 353250 | | Mean | 0.2677 | 0.8228 | 0.3957 | 353250 | ## Citation If using this model or the approach, please cite the associated paper: ``` @misc{morand2025tommerefficiententity, title={ToMMeR -- Efficient Entity Mention Detection from Large Language Models}, author={Victor Morand and Nadi Tomeh and Josiane Mothe and Benjamin Piwowarski}, year={2025}, eprint={2510.19410}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2510.19410}, } ``` ## License Apache-2.0 (see repository for full text).