--- license: apache-2.0 language: - en base_model: - nvidia/NVIDIA-Nemotron-Nano-12B-v2 --- # Paper-Summarizer-Nemotron-12B A fine-tuned Nemotron-12B model specialized for generating structured summaries of scientific research papers in standardized JSON format with superior throughput. ## Model Description This model is part of [Project AELLA](https://github.com/context-labs/laion-data-explorer), developed in collaboration with LAION and Wynd Labs to democratize access to scientific knowledge by creating structured summaries of research papers at scale. **Base Model**: NVIDIA Nemotron 12B (Hybrid Mamba-Transformer) **Training Data**: 110,000 curated research papers **Performance**: Achieves 71.3% accuracy on QA evaluation **Throughput**: 2.25× faster than Qwen3-14B variant This generates comprehensive structured summaries in a JSON format. The papers are either classified as SCIENTIFIC_TEXT, PARTIAL_SCIENTIFIC_TEXT, or NON_SCIENTIFIC_TEXT. The fields extracted are key research elements such as methodology, results, claims, and limitations. The model supports papers up to 131K tokens and is optimized for large-scale batch processing with high throughput (0.97 requests/sec). ## Usage ### Serving the Model **Note**: This model requires a custom chat template for proper reasoning token handling. ```bash vllm serve inference-net/Paper-Summarizer-Nemotron-12B \ --port 8000 \ --host 0.0.0.0 \ --trust-remote-code \ --data-parallel-size 1 \ --tensor-parallel-size 1 \ --max-num-seqs 32 \ --max-model-len 131072 \ --max-num-batched-tokens 8192 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-chunked-prefill \ --chat-template "{%- set ns = namespace(enable_thinking=true) %}{%- for message in messages -%}{%- set content = message['content'] -%}{%- if message['role'] == 'user' or message['role'] == 'system' -%}{%- if '/think' in content -%}{%- set ns.enable_thinking = true -%}{%- elif '/no_think' in content -%}{%- set ns.enable_thinking = false -%}{%- endif -%}{%- endif -%}{%- endfor -%}{%- if messages[0]['role'] != 'system' -%}{%- set ns.non_tool_system_content = '' -%}{{- 'System\n' -}}{%- else -%}{%- set ns.non_tool_system_content = messages[0]['content'].replace('/think', '').replace('/no_think', '').strip() -%}{{- 'System\n' + ns.non_tool_system_content }}{%- endif -%}{%- if tools -%}{%- if ns.non_tool_system_content is defined and ns.non_tool_system_content != '' -%}{{- '\n\n' -}}{%- endif -%}{{- 'You can use the following tools to assist the user if required:' -}}{{- '\n[' -}}{%- for tool in tools -%}{{- (tool.function if tool.function is defined else tool) | tojson -}}{{- ', ' if not loop.last else '' -}}{%- endfor -%}{{- ']\n\n' -}}{{- 'If you decide to call any tool(s), use the following format:\n' -}}{{- '[{{\"name\": \"tool_name1\", \"arguments\": \"tool_args1\"}}, ' -}}{{- '{{\"name\": \"tool_name2\", \"arguments\": \"tool_args2\"}}]\n\n' -}}{{- 'The user will execute tool-calls and return responses from tool(s) in this format:\n' -}}{{- '[{{\"tool_response1\"}}, {{\"tool_response2\"}}]\n\n' -}}{{- 'Based on the tool responses, you can call additional tools if needed, correct tool calls if any errors are found, or just respond to the user.' -}}{%- endif -%}{{- '\n' -}}{%- set messages = messages[1:] if messages[0]['role'] == 'system' else messages -%}{%- if messages[-1]['role'] == 'assistant' -%}{%- set ns.last_turn_assistant_content = messages[-1]['content'].strip() -%}{%- set messages = messages[:-1] -%}{%- endif -%}{%- for message in messages %}{%- set content = message['content'] %}{%- if message['role'] == 'user' -%}{{- 'User\n' + content.replace('/think', '').replace('/no_think', '').strip() + '\n' }}{%- elif message['role'] == 'tool' -%}{%- if loop.first or (messages[loop.index0 - 1].role != 'tool') -%}{{- 'User\n' + '[' }}{%- endif -%}{{- message['content'] -}}{{- ', ' if not loop.last and (messages[loop.index0 + 1].role == 'tool') else '' -}}{%- if loop.last or (messages[loop.index0 + 1].role != 'tool') -%}{{- ']\n' -}}{%- endif -%}{%- elif message['role'] == 'assistant' -%}{%- if '' in content -%}{%- set content = content.split('')[1].strip() %}{%- endif -%}{{- 'Assistant\n' + content.strip() }}{%- if message.tool_calls -%}{%- if content.strip() != '' -%}{{- '\n\n' -}}{%- endif -%}{{- '[' -}}{%- for call in message.tool_calls -%}{%- set fn = call.function if call.function is defined else call -%}{{- '{\"name\": \"' + fn.name + '\", \"arguments\": ' -}}{%- if fn.arguments is string -%}{{- fn.arguments -}}{%- else -%}{{- fn.arguments | tojson -}}{%- endif -%}{{- '}' + (', ' if not loop.last else '') -}}{%- endfor -%}{{- ']' -}}{%- endif -%}{{- '\n\n' -}}{%- endif -%}{%- endfor -%}{%- if add_generation_prompt -%}{{- 'Assistant\n' -}}{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}{{- '' -}}{%- else -%}{{- '\n' -}}{%- endif -%}{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}{{- ns.last_turn_assistant_content -}}{%- endif -%}{%- else -%}{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}{{- 'Assistant\n' -}}{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}{{- '' -}}{%- else -%}{{- '\n' -}}{%- endif -%}{{- ns.last_turn_assistant_content -}}{%- if continue_final_message is defined -%}{%- if continue_final_message is false -%}{{- '\n\n' -}}{%- endif -%}{%- else -%}{{- '\n\n' -}}{%- endif -%}{%- endif -%}{%- endif -%}" ``` ### Making Requests ```python import requests # System prompt (required) system_prompt = """[Insert the full system prompt from the prompt.txt file - see the full prompt in the model repository]""" # User prompt: the paper text to summarize paper_text = """ Title: Your Paper Title Authors: Author 1, Author 2 Abstract: ... [Full paper content] """ # API request response = requests.post( "http://localhost:8000/v1/chat/completions", json={ "model": "inference-net/Paper-Summarizer-Nemotron-12B", "messages": [ {"role": "system", "content": system_prompt}, {"role": "user", "content": paper_text} ], "temperature": 0.2 }, timeout=600 ) result = response.json() # Note: Response may include reasoning tokens wrapped in # These are automatically stripped by the chat template summary = result["choices"][0]["message"]["content"] print(summary) ``` ### System Prompt The model requires the same system prompt as the Qwen3-14B variant. The prompt instructs the model to: 1. **Classify** the text as SCIENTIFIC_TEXT, PARTIAL_SCIENTIFIC_TEXT, or NON_SCIENTIFIC_TEXT 2. **Extract** structured information including: - Title, authors, publication year - Research context and hypotheses - Methodological details - Key results with quantitative data - Claims with supporting evidence - Limitations and ethical considerations The full system prompt is available in the model repository's `prompt.txt` file. ### Output Format The model outputs a single valid JSON object with this structure: ```json { "article_classification": "SCIENTIFIC_TEXT", "reason": null, "summary": { "title": "", "authors": "", "publication_year": null, "field_subfield": "", "executive_summary": "", "research_context": "", "methodological_details": "", "key_results": "", "claims": [...], "contradictions_and_limitations": "", ... } } ``` ## Performance ### LLM-as-a-Judge Evaluation - **Score**: 4.095/5.0 - **Comparison**: Slightly behind Qwen3-14B (4.207) but still high quality ### QA Dataset Evaluation - **Accuracy**: 71.3% - **Comparison**: Strong performance, suitable for batch processing ### Throughput (8×H200 node) - **Requests/sec**: 0.97 (2.25× faster than Qwen3-14B) - **Input Tokens/sec**: 16,943.69 - **Output Tokens/sec**: 4,880.76 - **Single Request Tokens/sec**: 76.17 ### Cost Efficiency - **Processing 100M papers**: ~$45,000 (vs $100,000 for Qwen3-14B, $5M+ for GPT-5) - **Ideal for**: Large-scale batch processing where throughput matters ## Training Details - **Training Set**: 100,000 papers (same as Qwen3-14B) - **Validation Set**: 10,000 papers - **Average Paper Length**: 81,334 characters - **Architecture**: Hybrid Mamba-Transformer for high throughput - **Training Approach**: Post-training on summaries generated by frontier models ## When to Use This Model ### Choose Nemotron-12B if: - Processing large batches (100K+ papers) - Throughput and cost are primary concerns - Accuracy in the 70-75% range is acceptable - Running on GPU infrastructure with parallel processing ### Choose Qwen3-14B if: - Need highest possible accuracy (73.9% vs 71.3%) - Processing smaller batches or single papers - Quality is more important than speed ## Limitations - May generate subtle factual errors (hallucinations) for fine-grained details - Context limit (131K tokens) may truncate extremely long documents - Unified schema may not capture all domain-specific nuances - Summaries are research aids, not replacements for primary sources in high-stakes scenarios - Slightly lower accuracy than Qwen3-14B variant ## Related Resources - **Paper Visualization Website**: https://laion.inference.net - **Visualization Repository**: https://github.com/context-labs/laion-data-explorer - **Alexandria Paper**: https://arxiv.org/abs/2502.19413 - **Qwen3-14B Variant**: inference-net/Paper-Summarizer-Qwen3-14B ## License [License information to be added] ## Acknowledgments This work was made possible through collaboration with: - LAION - Wynd Labs - Inference.net - NVIDIA (base Nemotron architecture) - Contributors to bethgelab, PeS2o, Common Pile, and OpenAlex