Add files using upload-large-folder tool

README.md

@@ -0,0 +1,48 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- verilog
+- rtl
+- lora
+- peft
+- qwen
+- hardware-design
+- eda
+---
+
+# Verilog RTL LoRA Adapter Suite (TCAD 2026 Draft Artifacts)
+
+This repository collects the main PEFT LoRA adapters used in the adapter-centric Verilog RTL generation study. The original local adapter directories were copied and renamed with descriptive artifact names. No original local adapter directory was deleted or modified.
+
+## Adapter mapping
+
+| Descriptive artifact name | Paper alias | Base/start | Intended role |
+|---|---:|---|---|
+| `qwen25-coder-7b-verilog-direct-baseline-lora` | `v9` | Qwen2.5-Coder-7B-Instruct | Stable direct LoRA baseline trained from auto-distilled RTL data and passing anchors. |
+| `qwen25-coder-7b-verilog-delta-distilled-lora` | `v30b` | Qwen2.5-Coder-7B-Instruct + `v9` | Delta-distilled direct adapter emphasizing cases where `v9` failed and a verifier/selector found a passing candidate. |
+| `qwen35-9b-verilog-migration-general-lora` | `v32` | Qwen3.5-9B | First Qwen3.5 migration adapter trained from Qwen2.5-derived verified data. |
+| `qwen35-9b-verilog-thinking-reinforced-lora` | `v33` | Qwen3.5-9B + `v32` | Thinking/reasoning-preserving direct adapter with failure-analysis reinforcement and passing anchors. |
+| `qwen35-9b-verilog-structured-repair-lora` | `v34` | Qwen3.5-9B + `v33` | Structured/manual repair adapter focused on deterministic final-code formatting and repair behavior. |
+
+`v36` in the paper is not a separate adapter. It is a verification-guided workflow that composes the `v33` and `v34` adapters with compiler/simulator feedback under a bounded call budget.
+
+## Loading example
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+base_model = "Qwen/Qwen2.5-Coder-7B-Instruct"  # for v9/v30b artifacts
+adapter_dir = "qwen25-coder-7b-verilog-direct-baseline-lora"
+
+tok = AutoTokenizer.from_pretrained(adapter_dir)
+base = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto")
+model = PeftModel.from_pretrained(base, adapter_dir)
+```
+
+Use `Qwen/Qwen3.5-9B` as the base family for the Qwen3.5 adapters (`v32`, `v33`, `v34`) with the compatibility loader used in the project scripts.
+
+## Reporting note
+
+These adapters are research artifacts. Several training rows are benchmark-targeted or verifier-derived. Do not report them as clean zero-shot leaderboard submissions on the same benchmark. Direct adapter results and verification-guided workflow results should be reported separately.

qwen25-coder-7b-verilog-delta-distilled-lora/README.md

@@ -0,0 +1,139 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- verilog
+- rtl
+- code-generation
+- qwen2.5-coder
+- lora
+- peft
+- qlora
+- hardware
+- verilog-eval
+license: apache-2.0
+---
+
+# Verilog Qwen2.5-Coder 7B v30b Delta-Distilled LoRA
+
+`adapter_v30b_delta_distilled_from_v9` is a **single standard PEFT LoRA adapter** for `Qwen/Qwen2.5-Coder-7B-Instruct` focused on Verilog RTL generation.
+
+It was trained from the prior `v9` Verilog adapter using delta distillation from a stronger multi-adapter verifier-selector pipeline (`v29`). The goal was to produce a deployable one-adapter model that improves direct VerilogEval performance while preserving broader Verilog behavior.
+
+## Important caveat
+
+This adapter is **not a clean zero-shot VerilogEval leaderboard model**. It is a targeted/distilled research artifact: some training rows come from v29 selector outputs on VerilogEval prompts that passed compile+simulation. Use the reported VerilogEval score as an experiment result, not as a contamination-free leaderboard claim.
+
+For general Verilog usefulness, also see the external paper-style/robust/alt evaluations below.
+
+## Results
+
+### VerilogEval v2 direct, spec-to-RTL, n=1, temperature 0
+
+| Model / system | Compile | Functional pass |
+|---|---:|---:|
+| v9 prior single adapter | — | 67/156 |
+| v29 multi-adapter verifier selector | 150/156 | 84/156 |
+| v30 unified single adapter | 134/156 | 67/156 |
+| **v30b delta-distilled single adapter** | **141/156** | **71/156** |
+
+### External/generalization checks
+
+| Benchmark | Compile | Functional/task pass |
+|---|---:|---:|
+| Paper-style full | 30/30 | 26/30 task pass; 18/22 functional |
+| Robust suite | 14/15 | 6/10 functional |
+| Alt suite | 7/8 | 3/5 functional |
+
+These match the prior v9 baseline on these small external suites, while improving VerilogEval direct from 67 to 71 pass.
+
+## Training data mix
+
+Dataset builder: `scripts/build_v30b_delta_distill_dataset.py`
+
+Unique source counts:
+
+- 17 delta wins: v9 failed, v29 selector passed.
+- 84 total v29 selector passing outputs.
+- 67 v9 passing outputs for retention.
+- 382 clean/manual verified rows.
+- 18 external paper-style functional rows.
+- 316 small verified synthetic rows.
+
+Default repeat weights:
+
+```text
+delta wins:         80x
+all selector pass:   4x
+v9 pass retention:   6x
+clean verified:      2x
+external functional:10x
+synthetic:           1x
+```
+
+Training used `--drop-overlength`; rows exceeding the training token limit were dropped instead of truncating Verilog.
+
+## Training hyperparameters
+
+```text
+base model: Qwen/Qwen2.5-Coder-7B-Instruct
+base adapter: adapter_v9_auto_distilled_direct
+method: QLoRA/LoRA continuation
+LoRA r: 16
+LoRA alpha: 32
+learning rate: 7e-7
+epochs: 0.75
+max length: 2048
+batch size: 1
+grad accum: 4
+warmup steps: 40
+```
+
+## Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
+from peft import PeftModel
+
+base = "Qwen/Qwen2.5-Coder-7B-Instruct"
+adapter = "Pablo-Flores-Mollinedo/verilog-qwen2.5-coder-7b-v30b-delta-distilled-lora"
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.float16,
+    bnb_4bit_use_double_quant=True,
+)
+
+tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    base,
+    quantization_config=bnb,
+    device_map="auto",
+    trust_remote_code=True,
+)
+model = PeftModel.from_pretrained(model, adapter)
+model.eval()
+
+prompt = "Write module half_adder(input a, input b, output sum, output carry)."
+messages = [
+    {"role": "system", "content": "Return only complete synthesizable Verilog code. No explanation."},
+    {"role": "user", "content": prompt},
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**inputs, max_new_tokens=700, do_sample=False, pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+## Related artifacts
+
+- v29 multi-adapter verifier selector pipeline: higher VerilogEval score, but requires multiple adapters plus compile/simulation selection.
+- v30b: this repository, a single deployable PEFT LoRA adapter.
+
+## Intended use
+
+Research and experimentation with Verilog RTL code generation. Always compile, simulate, lint, and review generated RTL before use.

qwen25-coder-7b-verilog-delta-distilled-lora/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen25-coder-7b-verilog-delta-distilled-lora/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

qwen25-coder-7b-verilog-delta-distilled-lora/documentation/v30b_delta_distillation_plan.md

@@ -0,0 +1,67 @@
+# v30b delta-distillation plan
+
+Goal: make one standard PEFT LoRA adapter exceed v9 by focusing on verified delta wins.
+
+## Adapter
+
+```text
+adapter_v30b_delta_distilled_from_v9
+```
+
+## Difference from v30
+
+v30 used all v29 selector pass outputs conservatively and matched v9. v30b increases pressure on:
+
+```text
+v9 fails + v29 selector passes
+```
+
+These delta rows are repeated heavily, while keeping v9/clean/external retention.
+
+## Dataset mix
+
+Builder:
+
+```text
+scripts/build_v30b_delta_distill_dataset.py
+```
+
+Default repeats:
+
+```text
+delta wins:       80x
+all selector pass: 4x
+v9 pass retention: 6x
+clean verified:    2x
+external functional repair rows: 10x
+small synthetic:   1x
+```
+
+## Training
+
+Launcher:
+
+```text
+scripts/run_v30b_delta_distill.sh
+```
+
+Default:
+
+```text
+base adapter: adapter_v9_auto_distilled_direct
+epochs: 0.75
+lr: 7e-7
+max_length: 2048
+--drop-overlength
+```
+
+## Evaluation
+
+- VerilogEval direct
+- paper-style full
+- robust
+- alt
+
+## Caveat
+
+v30b is one adapter, but uses benchmark-targeted v29 selector outputs. It is a targeted/distilled experiment, not clean zero-shot VerilogEval evidence.

qwen25-coder-7b-verilog-delta-distilled-lora/documentation/v30b_delta_distillation_results.md

@@ -0,0 +1,140 @@
+# v30b delta-distilled single-adapter results
+
+v30b is the first post-v9 **single standard PEFT LoRA adapter** in this experiment series to improve direct VerilogEval pass count while preserving small external benchmark performance.
+
+```text
+adapter_v30b_delta_distilled_from_v9
+```
+
+## Motivation
+
+The v29 runtime selector reached 84/156 on VerilogEval by generating candidates from v9 plus multiple specialists and selecting with compile+simulation. That is a strong practical pipeline, but it is not a single deployable adapter.
+
+v30 attempted broad distillation of v29 into one adapter and matched v9 at 67/156. v30b instead focused training pressure on the delta:
+
+```text
+v9 fails + v29 selector passes
+```
+
+## Dataset
+
+Builder:
+
+```text
+scripts/build_v30b_delta_distill_dataset.py
+```
+
+Rows:
+
+```text
+total rows: 3358
+unique delta wins: 17
+unique selector pass rows: 84
+unique v9 pass retention rows: 67
+unique clean/manual verified rows: 382
+unique external functional rows: 18
+unique synthetic verified rows: 316
+```
+
+Repeat weights:
+
+```text
+delta wins:          80x
+all selector passes:  4x
+v9 pass retention:    6x
+clean verified:       2x
+external functional: 10x
+synthetic:            1x
+```
+
+The 17 delta-win problems were:
+
+```text
+Prob018_mux256to1
+Prob028_m2014_q4a
+Prob030_popcount255
+Prob042_vector4
+Prob045_edgedetect2
+Prob050_kmap1
+Prob054_edgedetect
+Prob056_ece241_2013_q7
+Prob060_m2014_q4k
+Prob063_review2015_shiftcount
+Prob075_counter_2bc
+Prob088_ece241_2014_q5b
+Prob097_mux9to1v
+Prob098_circuit7
+Prob121_2014_q3bfsm
+Prob130_circuit5
+Prob138_2012_q2fsm
+```
+
+Training used `--drop-overlength`; no rows were truncated.
+
+## Training
+
+Launcher:
+
+```text
+scripts/run_v30b_delta_distill.sh
+```
+
+Settings:
+
+```text
+base model: Qwen/Qwen2.5-Coder-7B-Instruct
+base adapter: adapter_v9_auto_distilled_direct
+output: adapter_v30b_delta_distilled_from_v9
+epochs: 0.75
+learning rate: 7e-7
+max length: 2048
+LoRA r: 16
+LoRA alpha: 32
+batch size: 1
+grad accum: 4
+warmup steps: 40
+```
+
+## Results
+
+### VerilogEval v2 direct
+
+| Adapter/system | Compile | Pass |
+|---|---:|---:|
+| v9 single adapter | — | 67/156 |
+| v29 multi-adapter selector | 150/156 | 84/156 |
+| v30 unified single adapter | 134/156 | 67/156 |
+| **v30b delta-distilled single adapter** | **141/156** | **71/156** |
+
+v30b improves single-adapter pass count by +4 over v9/v30.
+
+### External checks
+
+| Benchmark | Compile | Functional/task pass |
+|---|---:|---:|
+| Paper-style full | 30/30 | 26/30 task pass; 18/22 functional |
+| Robust | 14/15 | 6/10 functional |
+| Alt | 7/8 | 3/5 functional |
+
+These match the v9 baseline on the same external checks, so v30b did not show a regression on these small non-VerilogEval suites.
+
+## Interpretation
+
+v30b validates the paper-inspired lesson: simple broad SFT/distillation did not beat v9, but **verification-guided delta distillation** can transfer some of the v29 selector's gains into one standard adapter.
+
+Still, v30b remains far below the v29 verifier-selector pipeline because a single n=1 generation cannot reproduce multi-candidate search and runtime verification.
+
+## Caveat
+
+v30b is a single PEFT LoRA adapter, but it is trained with benchmark-targeted v29 selector outputs. Its VerilogEval result should be described as a targeted/distilled experiment, not a clean zero-shot leaderboard claim.
+
+## Published artifacts
+
+- Hugging Face adapter: `Pablo-Flores-Mollinedo/verilog-qwen2.5-coder-7b-v30b-delta-distilled-lora`
+- Main scripts:
+  - `scripts/build_v30b_delta_distill_dataset.py`
+  - `scripts/run_v30b_delta_distill.sh`
+- Result summary:
+  - `results/v30b_delta_distill/v30b_summary.json`
+  - `results/v30b_delta_distill/verilogeval_direct_summary.json`
+  - `results/v30b_delta_distill/verilogeval_direct_summary_results.jsonl`

qwen25-coder-7b-verilog-delta-distilled-lora/results/dataset_summary.json

@@ -0,0 +1,91 @@
+{
+  "output": "data/v30b_delta_distill/v30b_delta_distill_train.jsonl",
+  "rows": 3358,
+  "unique_delta_wins": 17,
+  "unique_selector_pass": 84,
+  "unique_v9_pass": 67,
+  "unique_clean": 382,
+  "unique_external_functional": 18,
+  "unique_synthetic": 316,
+  "repeats": {
+    "delta_repeat": 80,
+    "selector_repeat": 4,
+    "v9_repeat": 6,
+    "clean_repeat": 2,
+    "external_repeat": 10,
+    "synthetic_repeat": 1
+  },
+  "by_category_unique": {
+    "delta": {
+      "comb_vector": 3,
+      "other": 1,
+      "counter_timer": 3,
+      "arith_datapath": 1,
+      "seq_shift": 5,
+      "case_kmap": 1,
+      "fsm_protocol": 3
+    },
+    "selector": {
+      "other": 6,
+      "comb_vector": 33,
+      "arith_datapath": 12,
+      "counter_timer": 11,
+      "seq_shift": 10,
+      "case_kmap": 2,
+      "fsm_protocol": 10
+    },
+    "v9": {
+      "comb_vector": 35,
+      "arith_datapath": 11,
+      "counter_timer": 9,
+      "seq_shift": 4,
+      "case_kmap": 1,
+      "fsm_protocol": 7
+    },
+    "clean": {
+      "comb_vector": 97,
+      "arith_datapath": 42,
+      "counter_timer": 91,
+      "seq_shift": 49,
+      "case_kmap": 19,
+      "fsm_protocol": 80,
+      "other": 4
+    },
+    "external": {
+      "arith_datapath": 5,
+      "comb_vector": 3,
+      "case_kmap": 3,
+      "seq_shift": 5,
+      "other": 1,
+      "counter_timer": 1
+    },
+    "synthetic": {
+      "counter_timer": 60,
+      "seq_shift": 60,
+      "fsm_protocol": 60,
+      "comb_vector": 60,
+      "arith_datapath": 16,
+      "case_kmap": 60
+    }
+  },
+  "delta_problems": [
+    "Prob018_mux256to1",
+    "Prob028_m2014_q4a",
+    "Prob030_popcount255",
+    "Prob042_vector4",
+    "Prob045_edgedetect2",
+    "Prob050_kmap1",
+    "Prob054_edgedetect",
+    "Prob056_ece241_2013_q7",
+    "Prob060_m2014_q4k",
+    "Prob063_review2015_shiftcount",
+    "Prob075_counter_2bc",
+    "Prob088_ece241_2014_q5b",
+    "Prob097_mux9to1v",
+    "Prob098_circuit7",
+    "Prob121_2014_q3bfsm",
+    "Prob130_circuit5",
+    "Prob138_2012_q2fsm"
+  ],
+  "caveat": "Single-adapter delta distillation from benchmark-targeted v29 selector wins; not clean zero-shot VerilogEval evidence."
+}

qwen25-coder-7b-verilog-delta-distilled-lora/results/v30b_summary.json

@@ -0,0 +1,187 @@
+{
+  "adapter": "adapter_v30b_delta_distilled_from_v9",
+  "hf_repo": "Pablo-Flores-Mollinedo/verilog-qwen2.5-coder-7b-v30b-delta-distilled-lora",
+  "verilogeval_direct": {
+    "total": 156,
+    "compile": 141,
+    "passed": 71,
+    "compile_pct": 90.38461538461539,
+    "pass_pct": 45.51282051282051
+  },
+  "paper_full": {
+    "model": "v30b_delta_paper_full",
+    "adapter": "adapter_v30b_delta_distilled_from_v9",
+    "suite": "full",
+    "tasks": 30,
+    "k": 1,
+    "compile_any_pass": 30,
+    "compile_any_pct": 100.0,
+    "task_pass": 26,
+    "task_pass_pct": 86.66666666666667,
+    "functional_total": 22,
+    "functional_pass": 18,
+    "functional_pct": 81.81818181818181,
+    "by_category": {
+      "basic_comb": {
+        "total": 7,
+        "pass": 7,
+        "compile": 7,
+        "pass_pct": 100.0,
+        "compile_pct": 100.0
+      },
+      "arith": {
+        "total": 3,
+        "pass": 3,
+        "compile": 3,
+        "pass_pct": 100.0,
+        "compile_pct": 100.0
+      },
+      "bit_manip": {
+        "total": 4,
+        "pass": 4,
+        "compile": 4,
+        "pass_pct": 100.0,
+        "compile_pct": 100.0
+      },
+      "sequential": {
+        "total": 5,
+        "pass": 2,
+        "compile": 5,
+        "pass_pct": 40.0,
+        "compile_pct": 100.0
+      },
+      "fsm": {
+        "total": 3,
+        "pass": 2,
+        "compile": 3,
+        "pass_pct": 66.66666666666667,
+        "compile_pct": 100.0
+      },
+      "memory": {
+        "total": 3,
+        "pass": 3,
+        "compile": 3,
+        "pass_pct": 100.0,
+        "compile_pct": 100.0
+      },
+      "compile_rtl": {
+        "total": 5,
+        "pass": 5,
+        "compile": 5,
+        "pass_pct": 100.0,
+        "compile_pct": 100.0
+      }
+    }
+  },
+  "robust": {
+    "name": "v30b_delta_robust",
+    "suite": "robust",
+    "total": 15,
+    "compile_pass": 14,
+    "compile_pct": 93.33333333333333,
+    "functional_total": 10,
+    "functional_pass": 6,
+    "functional_pct": 60.0
+  },
+  "alt": {
+    "name": "v30b_delta_alt",
+    "suite": "alt",
+    "total": 8,
+    "compile_pass": 7,
+    "compile_pct": 87.5,
+    "functional_total": 5,
+    "functional_pass": 3,
+    "functional_pct": 60.0
+  },
+  "dataset": {
+    "output": "data/v30b_delta_distill/v30b_delta_distill_train.jsonl",
+    "rows": 3358,
+    "unique_delta_wins": 17,
+    "unique_selector_pass": 84,
+    "unique_v9_pass": 67,
+    "unique_clean": 382,
+    "unique_external_functional": 18,
+    "unique_synthetic": 316,
+    "repeats": {
+      "delta_repeat": 80,
+      "selector_repeat": 4,
+      "v9_repeat": 6,
+      "clean_repeat": 2,
+      "external_repeat": 10,
+      "synthetic_repeat": 1
+    },
+    "by_category_unique": {
+      "delta": {
+        "comb_vector": 3,
+        "other": 1,
+        "counter_timer": 3,
+        "arith_datapath": 1,
+        "seq_shift": 5,
+        "case_kmap": 1,
+        "fsm_protocol": 3
+      },
+      "selector": {
+        "other": 6,
+        "comb_vector": 33,
+        "arith_datapath": 12,
+        "counter_timer": 11,
+        "seq_shift": 10,
+        "case_kmap": 2,
+        "fsm_protocol": 10
+      },
+      "v9": {
+        "comb_vector": 35,
+        "arith_datapath": 11,
+        "counter_timer": 9,
+        "seq_shift": 4,
+        "case_kmap": 1,
+        "fsm_protocol": 7
+      },
+      "clean": {
+        "comb_vector": 97,
+        "arith_datapath": 42,
+        "counter_timer": 91,
+        "seq_shift": 49,
+        "case_kmap": 19,
+        "fsm_protocol": 80,
+        "other": 4
+      },
+      "external": {
+        "arith_datapath": 5,
+        "comb_vector": 3,
+        "case_kmap": 3,
+        "seq_shift": 5,
+        "other": 1,
+        "counter_timer": 1
+      },
+      "synthetic": {
+        "counter_timer": 60,
+        "seq_shift": 60,
+        "fsm_protocol": 60,
+        "comb_vector": 60,
+        "arith_datapath": 16,
+        "case_kmap": 60
+      }
+    },
+    "delta_problems": [
+      "Prob018_mux256to1",
+      "Prob028_m2014_q4a",
+      "Prob030_popcount255",
+      "Prob042_vector4",
+      "Prob045_edgedetect2",
+      "Prob050_kmap1",
+      "Prob054_edgedetect",
+      "Prob056_ece241_2013_q7",
+      "Prob060_m2014_q4k",
+      "Prob063_review2015_shiftcount",
+      "Prob075_counter_2bc",
+      "Prob088_ece241_2014_q5b",
+      "Prob097_mux9to1v",
+      "Prob098_circuit7",
+      "Prob121_2014_q3bfsm",
+      "Prob130_circuit5",
+      "Prob138_2012_q2fsm"
+    ],
+    "caveat": "Single-adapter delta distillation from benchmark-targeted v29 selector wins; not clean zero-shot VerilogEval evidence."
+  }
+}

qwen25-coder-7b-verilog-delta-distilled-lora/results/verilogeval_direct_summary.json

@@ -0,0 +1,7 @@
+{
+  "total": 156,
+  "compile": 141,
+  "passed": 71,
+  "compile_pct": 90.38461538461539,
+  "pass_pct": 45.51282051282051
+}

qwen25-coder-7b-verilog-delta-distilled-lora/results/verilogeval_direct_summary_results.jsonl

@@ -0,0 +1,156 @@
+{"problem": "Prob001_zero", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'zero' has no mismatches.\nHint: Total mismatched samples is 0 out of 20 samples\n\nSimulation finished at 102 ps\nMismatches: 0 in 20 samples\n"}
+{"problem": "Prob002_m2014_q4i", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob003_step_one", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'one' has no mismatches.\nHint: Total mismatched samples is 0 out of 20 samples\n\nSimulation finished at 102 ps\nMismatches: 0 in 20 samples\n"}
+{"problem": "Prob004_vector2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 110 samples\n\nSimulation finished at 550 ps\nMismatches: 0 in 110 samples\n"}
+{"problem": "Prob005_notgate", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 239 samples\n\nSimulation finished at 1196 ps\nMismatches: 0 in 239 samples\n"}
+{"problem": "Prob006_vectorr", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 218 samples\n\nSimulation finished at 1090 ps\nMismatches: 0 in 218 samples\n"}
+{"problem": "Prob007_wire", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 120 samples\n\nSimulation finished at 601 ps\nMismatches: 0 in 120 samples\n"}
+{"problem": "Prob008_m2014_q4h", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob009_popcount3", "compile": false, "passed": false, "log": "dut.sv:12: warning: Numeric constant truncated to 3 bits.\ndut.sv:13: warning: Numeric constant truncated to 3 bits.\ndut.sv:14: syntax error\ndut.sv:14: error: Incomprehensible case expression.\ndut.sv:15: syntax error\ndut.sv:15: error: Incomprehensible case expression.\ndut.sv:16: syntax error\ndut.sv:16: error: Incomprehensible case expression.\ndut.sv:17: syntax error\ndut.sv:17: error: Incomprehensible case expression.\ndut.sv:18: syntax error\ndut.sv:18: error: Incomprehensible case expression.\ndut.sv:19: syntax error\ndut.sv:19: warning: Numeric constant truncated to 2 bits.\ndut.sv:19: error: Incomprehensible case expression.\n"}
+{"problem": "Prob010_mt2015_q4a", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has no mismatches.\nHint: Total mismatched samples is 0 out of 202 samples\n\nSimulation finished at 1011 ps\nMismatches: 0 in 202 samples\n"}
+{"problem": "Prob011_norgate", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 0 in 219 samples\n"}
+{"problem": "Prob012_xnorgate", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 0 in 219 samples\n"}
+{"problem": "Prob013_m2014_q4e", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob014_andgate", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 0 in 219 samples\n"}
+{"problem": "Prob015_vector1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_hi' has no mismatches.\nHint: Output 'out_lo' has no mismatches.\nHint: Total mismatched samples is 0 out of 218 samples\n\nSimulation finished at 1090 ps\nMismatches: 0 in 218 samples\n"}
+{"problem": "Prob016_m2014_q4j", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'sum' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob017_mux2to1v", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 114 samples\n\nSimulation finished at 570 ps\nMismatches: 0 in 114 samples\n"}
+{"problem": "Prob018_mux256to1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 2000 samples\n\nSimulation finished at 10000 ps\nMismatches: 0 in 2000 samples\n"}
+{"problem": "Prob019_m2014_q4f", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 27 mismatches. First mismatch occurred at time 45.\nHint: Total mismatched samples is 27 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 27 in 100 samples\n"}
+{"problem": "Prob020_mt2015_eq2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has no mismatches.\nHint: Total mismatched samples is 0 out of 2000 samples\n\nSimulation finished at 10001 ps\nMismatches: 0 in 2000 samples\n"}
+{"problem": "Prob021_mux256to1v", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 1943 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 1943 out of 2000 samples\n\nSimulation finished at 10000 ps\nMismatches: 1943 in 2000 samples\n"}
+{"problem": "Prob022_mux2to1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 122 samples\n\nSimulation finished at 610 ps\nMismatches: 0 in 122 samples\n"}
+{"problem": "Prob023_vector100r", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 200 samples\n\nSimulation finished at 1000 ps\nMismatches: 0 in 200 samples\n"}
+{"problem": "Prob024_hadd", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'sum' has no mismatches.\nHint: Output 'cout' has no mismatches.\nHint: Total mismatched samples is 0 out of 200 samples\n\nSimulation finished at 1000 ps\nMismatches: 0 in 200 samples\n"}
+{"problem": "Prob025_reduction", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'parity' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 500 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob026_alwaysblock1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_assign' has no mismatches.\nHint: Output 'out_alwaysblock' has no mismatches.\nHint: Total mismatched samples is 0 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 0 in 219 samples\n"}
+{"problem": "Prob027_fadd", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'cout' has no mismatches.\nHint: Output 'sum' has no mismatches.\nHint: Total mismatched samples is 0 out of 214 samples\n\nSimulation finished at 1070 ps\nMismatches: 0 in 214 samples\n"}
+{"problem": "Prob028_m2014_q4a", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob029_m2014_q4g", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 51 mismatches. First mismatch occurred at time 25.\nHint: Total mismatched samples is 51 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 51 in 100 samples\n"}
+{"problem": "Prob030_popcount255", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 215 samples\n\nSimulation finished at 1076 ps\nMismatches: 0 in 215 samples\n"}
+{"problem": "Prob031_dff", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 118 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 118 out of 121 samples\n\nSimulation finished at 605 ps\nMismatches: 118 in 121 samples\n"}
+{"problem": "Prob032_vector0", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'outv' has no mismatches.\nHint: Output 'o2' has no mismatches.\nHint: Output 'o1' has no mismatches.\nHint: Output 'o0' has no mismatches.\nHint: Total mismatched samples is 0 out of 21 samples\n\nSimulation finished at 107 ps\nMismatches: 0 in 21 samples\n"}
+{"problem": "Prob033_ece241_2014_q1c", "compile": false, "passed": false, "log": "dut.sv:5: error: overflow is not a valid l-value in tb.top_module1.\ndut.sv:1:      : overflow is declared here as wire.\n1 error(s) during elaboration.\n"}
+{"problem": "Prob034_dff8", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 1 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 1 out of 41 samples\n\nSimulation finished at 206 ps\nMismatches: 1 in 41 samples\n"}
+{"problem": "Prob035_count1to10", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 439 samples\n\nSimulation finished at 2196 ps\nMismatches: 0 in 439 samples\n"}
+{"problem": "Prob036_ringer", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'ringer' has no mismatches.\nHint: Output 'motor' has no mismatches.\nHint: Total mismatched samples is 0 out of 19 samples\n\nSimulation finished at 97 ps\nMismatches: 0 in 19 samples\n"}
+{"problem": "Prob037_review2015_count1k", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 8027 samples\n\nSimulation finished at 40136 ps\nMismatches: 0 in 8027 samples\n"}
+{"problem": "Prob038_count15", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 421 samples\n\nSimulation finished at 2106 ps\nMismatches: 0 in 421 samples\n"}
+{"problem": "Prob039_always_if", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_assign' has no mismatches.\nHint: Output 'out_always' has no mismatches.\nHint: Total mismatched samples is 0 out of 114 samples\n\nSimulation finished at 570 ps\nMismatches: 0 in 114 samples\n"}
+{"problem": "Prob040_count10", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 439 samples\n\nSimulation finished at 2196 ps\nMismatches: 0 in 439 samples\n"}
+{"problem": "Prob041_dff8r", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 432 samples\n\nSimulation finished at 2161 ps\nMismatches: 0 in 432 samples\n"}
+{"problem": "Prob042_vector4", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 500 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob043_vector5", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 500 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob044_vectorgates", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_or_bitwise' has no mismatches.\nHint: Output 'out_or_logical' has no mismatches.\nHint: Output 'out_not' has no mismatches.\nHint: Total mismatched samples is 0 out of 261 samples\n\nSimulation finished at 1306 ps\nMismatches: 0 in 261 samples\n"}
+{"problem": "Prob045_edgedetect2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'anyedge' has 146 mismatches. First mismatch occurred at time 170.\nHint: Total mismatched samples is 146 out of 228 samples\n\nSimulation finished at 1140 ps\nMismatches: 146 in 228 samples\n"}
+{"problem": "Prob046_dff8p", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 408 mismatches. First mismatch occurred at time 45.\nHint: Total mismatched samples is 408 out of 436 samples\n\nSimulation finished at 2181 ps\nMismatches: 408 in 436 samples\n"}
+{"problem": "Prob047_dff8ar", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset should be asynchronous, but doesn't appear to be.\nHint: Output 'q' has 44 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 44 out of 436 samples\n\nSimulation finished at 2181 ps\nMismatches: 44 in 436 samples\n"}
+{"problem": "Prob048_m2014_q4c", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 0 in 100 samples\n"}
+{"problem": "Prob049_m2014_q4b", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 17 mismatches. First mismatch occurred at time 15.\nHint: Total mismatched samples is 17 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 17 in 100 samples\n"}
+{"problem": "Prob050_kmap1", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 90 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 90 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 90 in 219 samples\n"}
+{"problem": "Prob051_gates4", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_and' has no mismatches.\nHint: Output 'out_or' has no mismatches.\nHint: Output 'out_xor' has no mismatches.\nHint: Total mismatched samples is 0 out of 234 samples\n\nSimulation finished at 1170 ps\nMismatches: 0 in 234 samples\n"}
+{"problem": "Prob052_gates100", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_and' has no mismatches.\nHint: Output 'out_or' has no mismatches.\nHint: Output 'out_xor' has no mismatches.\nHint: Total mismatched samples is 0 out of 433 samples\n\nSimulation finished at 2166 ps\nMismatches: 0 in 433 samples\n"}
+{"problem": "Prob053_m2014_q4d", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 1 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 1 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 1 in 100 samples\n"}
+{"problem": "Prob054_edgedetect", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'pedge' has no mismatches.\nHint: Total mismatched samples is 0 out of 227 samples\n\nSimulation finished at 1135 ps\nMismatches: 0 in 227 samples\n"}
+{"problem": "Prob055_conditional", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'min' has no mismatches.\nHint: Total mismatched samples is 0 out of 112 samples\n\nSimulation finished at 560 ps\nMismatches: 0 in 112 samples\n"}
+{"problem": "Prob056_ece241_2013_q7", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Q' has no mismatches.\nHint: Total mismatched samples is 0 out of 422 samples\n\nSimulation finished at 2110 ps\nMismatches: 0 in 422 samples\n"}
+{"problem": "Prob057_kmap2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 123 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 123 out of 232 samples\n\nSimulation finished at 1161 ps\nMismatches: 123 in 232 samples\n"}
+{"problem": "Prob058_alwaysblock2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_assign' has no mismatches.\nHint: Output 'out_always_comb' has no mismatches.\nHint: Output 'out_always_ff' has no mismatches.\nHint: Total mismatched samples is 0 out of 219 samples\n\nSimulation finished at 1096 ps\nMismatches: 0 in 219 samples\n"}
+{"problem": "Prob059_wire4", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'w' has no mismatches.\nHint: Output 'x' has no mismatches.\nHint: Output 'y' has no mismatches.\nHint: Output 'z' has no mismatches.\nHint: Total mismatched samples is 0 out of 218 samples\n\nSimulation finished at 1090 ps\nMismatches: 0 in 218 samples\n"}
+{"problem": "Prob060_m2014_q4k", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 299 samples\n\nSimulation finished at 1496 ps\nMismatches: 0 in 299 samples\n"}
+{"problem": "Prob061_2014_q4a", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Q' has no mismatches.\nHint: Total mismatched samples is 0 out of 200 samples\n\nSimulation finished at 1001 ps\nMismatches: 0 in 200 samples\n"}
+{"problem": "Prob062_bugs_mux2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 110 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 110 out of 114 samples\n\nSimulation finished at 570 ps\nMismatches: 110 in 114 samples\n"}
+{"problem": "Prob063_review2015_shiftcount", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 2071 samples\n\nSimulation finished at 10356 ps\nMismatches: 0 in 2071 samples\n"}
+{"problem": "Prob064_vector3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'w' has 100 mismatches. First mismatch occurred at time 130.\nHint: Output 'x' has 101 mismatches. First mismatch occurred at time 130.\nHint: Output 'y' has 115 mismatches. First mismatch occurred at time 60.\nHint: Output 'z' has 126 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 126 out of 126 samples\n\nSimulation finished at 630 ps\nMismatches: 126 in 126 samples\n"}
+{"problem": "Prob065_7420", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'p1y' has no mismatches.\nHint: Output 'p2y' has no mismatches.\nHint: Total mismatched samples is 0 out of 239 samples\n\nSimulation finished at 1196 ps\nMismatches: 0 in 239 samples\n"}
+{"problem": "Prob066_edgecapture", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 225 mismatches. First mismatch occurred at time 100.\nHint: Total mismatched samples is 225 out of 266 samples\n\nSimulation finished at 1330 ps\nMismatches: 225 in 266 samples\n"}
+{"problem": "Prob067_countslow", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 499 samples\n\nSimulation finished at 2496 ps\nMismatches: 0 in 499 samples\n"}
+{"problem": "Prob068_countbcd", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'ena' has 39634 mismatches. First mismatch occurred at time 160.\nHint: Output 'q' has 39756 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 39774 out of 39805 samples\n\nSimulation finished at 199026 ps\nMismatches: 39774 in 39805 samples\n"}
+{"problem": "Prob069_truthtable1", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'f' has 50 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 50 out of 58 samples\n\nSimulation finished at 290 ps\nMismatches: 50 in 58 samples\n"}
+{"problem": "Prob070_ece241_2013_q2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_sop' has 36 mismatches. First mismatch occurred at time 20.\nHint: Output 'out_pos' has 27 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 44 out of 107 samples\n\nSimulation finished at 535 ps\nMismatches: 44 in 107 samples\n"}
+{"problem": "Prob071_always_casez", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'pos' has 1 mismatches. First mismatch occurred at time 1415.\nHint: Total mismatched samples is 1 out of 346 samples\n\nSimulation finished at 1730 ps\nMismatches: 1 in 346 samples\n"}
+{"problem": "Prob072_thermostat", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'heater' has no mismatches.\nHint: Output 'aircon' has no mismatches.\nHint: Output 'fan' has no mismatches.\nHint: Total mismatched samples is 0 out of 248 samples\n\nSimulation finished at 1241 ps\nMismatches: 0 in 248 samples\n"}
+{"problem": "Prob073_dff16e", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 443 samples\n\nSimulation finished at 2216 ps\nMismatches: 0 in 443 samples\n"}
+{"problem": "Prob074_ece241_2014_q4", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 45 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 45 out of 118 samples\n\nSimulation finished at 590 ps\nMismatches: 45 in 118 samples\n"}
+{"problem": "Prob075_counter_2bc", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'state' has no mismatches.\nHint: Total mismatched samples is 0 out of 1051 samples\n\nSimulation finished at 5256 ps\nMismatches: 0 in 1051 samples\n"}
+{"problem": "Prob076_always_case", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 118 samples\n\nSimulation finished at 590 ps\nMismatches: 0 in 118 samples\n"}
+{"problem": "Prob077_wire_decl", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Output 'out_n' has no mismatches.\nHint: Total mismatched samples is 0 out of 122 samples\n\nSimulation finished at 611 ps\nMismatches: 0 in 122 samples\n"}
+{"problem": "Prob078_dualedge", "compile": false, "passed": false, "log": "dut.sv:2: syntax error\ndut.sv:2: Syntax in assignment statement l-value.\n"}
+{"problem": "Prob079_fsm3onehot", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'next_state' has 141 mismatches. First mismatch occurred at time 10.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 141 out of 200 samples\n\nSimulation finished at 1001 ps\nMismatches: 141 in 200 samples\n"}
+{"problem": "Prob080_timer", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'tc' has no mismatches.\nHint: Total mismatched samples is 0 out of 7127 samples\n\nSimulation finished at 35636 ps\nMismatches: 0 in 7127 samples\n"}
+{"problem": "Prob081_7458", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'p1y' has no mismatches.\nHint: Output 'p2y' has no mismatches.\nHint: Total mismatched samples is 0 out of 439 samples\n\nSimulation finished at 2196 ps\nMismatches: 0 in 439 samples\n"}
+{"problem": "Prob082_lfsr32", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nTIMEOUT\nHint: Output 'q' has 199953 mismatches. First mismatch occurred at time 190.\nHint: Total mismatched samples is 199953 out of 200000 samples\n\nSimulation finished at 1000000 ps\nMismatches: 199953 in 200000 samples\n"}
+{"problem": "Prob083_mt2015_q4b", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has no mismatches.\nHint: Total mismatched samples is 0 out of 110 samples\n\nSimulation finished at 551 ps\nMismatches: 0 in 110 samples\n"}
+{"problem": "Prob084_ece241_2013_q12", "compile": false, "passed": false, "log": "dut.sv:9: syntax error\ndut.sv:6: error: syntax error in continuous assignment\n"}
+{"problem": "Prob085_shift4", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset should be asynchronous, but doesn't appear to be.\nHint: Output 'q' has 73 mismatches. First mismatch occurred at time 40.\nHint: Total mismatched samples is 73 out of 427 samples\n\nSimulation finished at 2136 ps\nMismatches: 73 in 427 samples\n"}
+{"problem": "Prob086_lfsr5", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 4266 mismatches. First mismatch occurred at time 40.\nHint: Total mismatched samples is 4266 out of 4443 samples\n\nSimulation finished at 22216 ps\nMismatches: 4266 in 4443 samples\n"}
+{"problem": "Prob087_gates", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_and' has no mismatches.\nHint: Output 'out_or' has no mismatches.\nHint: Output 'out_xor' has no mismatches.\nHint: Output 'out_nand' has no mismatches.\nHint: Output 'out_nor' has no mismatches.\nHint: Output 'out_xnor' has no mismatches.\nHint: Output 'out_anotb' has no mismatches.\nHint: Total mismatched samples is 0 out of 210 samples\n\nSimulation finished at 1050 ps\nMismatches: 0 in 210 samples\n"}
+{"problem": "Prob088_ece241_2014_q5b", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset doesn't seem to be working.\nHint: Output 'z' has 436 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 436 out of 436 samples\n\nSimulation finished at 2180 ps\nMismatches: 436 in 436 samples\n"}
+{"problem": "Prob089_ece241_2014_q5a", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 393 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 393 out of 436 samples\n\nSimulation finished at 2180 ps\nMismatches: 393 in 436 samples\n"}
+{"problem": "Prob090_circuit1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 120 samples\n\nSimulation finished at 600 ps\nMismatches: 0 in 120 samples\n"}
+{"problem": "Prob091_2012_q2b", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Y1' has 238 mismatches. First mismatch occurred at time 15.\nHint: Output 'Y3' has 325 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 443 out of 600 samples\n\nSimulation finished at 3001 ps\nMismatches: 443 in 600 samples\n"}
+{"problem": "Prob092_gatesv100", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_both' has no mismatches.\nHint: Output 'out_any' has 201 mismatches. First mismatch occurred at time 5.\nHint: Output 'out_different' has no mismatches.\nHint: Total mismatched samples is 201 out of 201 samples\n\nSimulation finished at 1006 ps\nMismatches: 201 in 201 samples\n"}
+{"problem": "Prob093_ece241_2014_q3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'mux_in' has 60 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 60 out of 60 samples\n\nSimulation finished at 300 ps\nMismatches: 60 in 60 samples\n"}
+{"problem": "Prob094_gatesv", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_both' has 116 mismatches. First mismatch occurred at time 5.\nHint: Output 'out_any' has 109 mismatches. First mismatch occurred at time 5.\nHint: Output 'out_different' has 160 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 172 out of 213 samples\n\nSimulation finished at 1066 ps\nMismatches: 172 in 213 samples\n"}
+{"problem": "Prob095_review2015_fsmshift", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'shift_ena' has no mismatches.\nHint: Total mismatched samples is 0 out of 200 samples\n\nSimulation finished at 1001 ps\nMismatches: 0 in 200 samples\n"}
+{"problem": "Prob096_review2015_fsmseq", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'start_shifting' has 306 mismatches. First mismatch occurred at time 50.\nHint: Total mismatched samples is 306 out of 643 samples\n\nSimulation finished at 3216 ps\nMismatches: 306 in 643 samples\n"}
+{"problem": "Prob097_mux9to1v", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 220 samples\n\nSimulation finished at 1100 ps\nMismatches: 0 in 220 samples\n"}
+{"problem": "Prob098_circuit7", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 120 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 120 out of 123 samples\n\nSimulation finished at 615 ps\nMismatches: 120 in 123 samples\n"}
+{"problem": "Prob099_m2014_q6c", "compile": false, "passed": false, "log": "test.sv:71: error: port ``Y2'' is not a port of good1.\ntest.sv:71: error: port ``Y4'' is not a port of good1.\ntest.sv:77: error: port ``Y2'' is not a port of top_module1.\ntest.sv:77: error: port ``Y4'' is not a port of top_module1.\n4 error(s) during elaboration.\n"}
+{"problem": "Prob100_fsm3comb", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'next_state' has 70 mismatches. First mismatch occurred at time 10.\nHint: Output 'out' has 14 mismatches. First mismatch occurred at time 40.\nHint: Total mismatched samples is 70 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 70 in 100 samples\n"}
+{"problem": "Prob101_circuit4", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 53 mismatches. First mismatch occurred at time 25.\nHint: Total mismatched samples is 53 out of 121 samples\n\nSimulation finished at 605 ps\nMismatches: 53 in 121 samples\n"}
+{"problem": "Prob102_circuit3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 64 mismatches. First mismatch occurred at time 25.\nHint: Total mismatched samples is 64 out of 121 samples\n\nSimulation finished at 605 ps\nMismatches: 64 in 121 samples\n"}
+{"problem": "Prob103_circuit2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 121 samples\n\nSimulation finished at 605 ps\nMismatches: 0 in 121 samples\n"}
+{"problem": "Prob104_mt2015_muxdff", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Q' has 1 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 1 out of 199 samples\n\nSimulation finished at 996 ps\nMismatches: 1 in 199 samples\n"}
+{"problem": "Prob105_rotate100", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 3292 mismatches. First mismatch occurred at time 60.\nHint: Total mismatched samples is 3292 out of 4005 samples\n\nSimulation finished at 20026 ps\nMismatches: 3292 in 4005 samples\n"}
+{"problem": "Prob106_always_nolatches", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'left' has no mismatches.\nHint: Output 'down' has no mismatches.\nHint: Output 'right' has no mismatches.\nHint: Output 'up' has no mismatches.\nHint: Total mismatched samples is 0 out of 30020 samples\n\nSimulation finished at 150100 ps\nMismatches: 0 in 30020 samples\n"}
+{"problem": "Prob107_fsm1s", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 137 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 137 out of 230 samples\n\nSimulation finished at 1151 ps\nMismatches: 137 in 230 samples\n"}
+{"problem": "Prob108_rule90", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 6022 mismatches. First mismatch occurred at time 50.\nHint: Total mismatched samples is 6022 out of 7121 samples\n\nSimulation finished at 35606 ps\nMismatches: 6022 in 7121 samples\n"}
+{"problem": "Prob109_fsm1", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 228 samples\n\nSimulation finished at 1141 ps\nMismatches: 0 in 228 samples\n"}
+{"problem": "Prob110_fsm2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 241 samples\n\nSimulation finished at 1206 ps\nMismatches: 0 in 241 samples\n"}
+{"problem": "Prob111_fsm2s", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Total mismatched samples is 0 out of 241 samples\n\nSimulation finished at 1206 ps\nMismatches: 0 in 241 samples\n"}
+{"problem": "Prob112_always_case2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'pos' has 61 mismatches. First mismatch occurred at time 90.\nHint: Total mismatched samples is 61 out of 94 samples\n\nSimulation finished at 470 ps\nMismatches: 61 in 94 samples\n"}
+{"problem": "Prob113_2012_q1g", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'f' has 30 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 30 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 30 in 100 samples\n"}
+{"problem": "Prob114_bugs_case", "compile": false, "passed": false, "log": "dut.sv:4: syntax error\ndut.sv:5: error: Incomprehensible case expression.\ndut.sv:5: syntax error\ndut.sv:6: error: Incomprehensible case expression.\ndut.sv:6: syntax error\ndut.sv:7: error: Incomprehensible case expression.\ndut.sv:7: syntax error\ndut.sv:8: error: Incomprehensible case expression.\ndut.sv:8: syntax error\ndut.sv:9: error: Incomprehensible case expression.\ndut.sv:9: syntax error\ndut.sv:10: error: Incomprehensible case expression.\ndut.sv:10: syntax error\ndut.sv:11: error: Incomprehensible case expression.\ndut.sv:11: syntax error\ndut.sv:12: error: Incomprehensible case expression.\ndut.sv:12: syntax error\ndut.sv:13: error: Incomprehensible case expression.\ndut.sv:13: syntax error\ndut.sv:14: error: Incomprehensible case expression.\n"}
+{"problem": "Prob115_shift18", "compile": false, "passed": false, "log": "dut.sv:8: warning: extra digits given for sized binary constant.\ndut.sv:8: syntax error\ndut.sv:8: error: malformed statement\n"}
+{"problem": "Prob116_m2014_q3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'f' has 40 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 40 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 40 in 100 samples\n"}
+{"problem": "Prob117_circuit9", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 244 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 244 out of 245 samples\n\nSimulation finished at 1225 ps\nMismatches: 244 in 245 samples\n"}
+{"problem": "Prob118_history_shift", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'predict_history' has 980 mismatches. First mismatch occurred at time 220.\nHint: Total mismatched samples is 980 out of 2055 samples\n\nSimulation finished at 10276 ps\nMismatches: 980 in 2055 samples\n"}
+{"problem": "Prob119_fsm3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 56 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 56 out of 230 samples\n\nSimulation finished at 1151 ps\nMismatches: 56 in 230 samples\n"}
+{"problem": "Prob120_fsm3s", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 56 mismatches. First mismatch occurred at time 40.\nHint: Total mismatched samples is 56 out of 230 samples\n\nSimulation finished at 1151 ps\nMismatches: 56 in 230 samples\n"}
+{"problem": "Prob121_2014_q3bfsm", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 516 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 516 out of 1006 samples\n\nSimulation finished at 5031 ps\nMismatches: 516 in 1006 samples\n"}
+{"problem": "Prob122_kmap4", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 142 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 142 out of 232 samples\n\nSimulation finished at 1161 ps\nMismatches: 142 in 232 samples\n"}
+{"problem": "Prob123_bugs_addsubz", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has no mismatches.\nHint: Output 'result_is_zero' has no mismatches.\nHint: Total mismatched samples is 0 out of 118 samples\n\nSimulation finished at 590 ps\nMismatches: 0 in 118 samples\n"}
+{"problem": "Prob124_rule110", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 6220 mismatches. First mismatch occurred at time 40.\nHint: Total mismatched samples is 6220 out of 6283 samples\n\nSimulation finished at 31416 ps\nMismatches: 6220 in 6283 samples\n"}
+{"problem": "Prob125_kmap3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out' has 124 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 124 out of 232 samples\n\nSimulation finished at 1161 ps\nMismatches: 124 in 232 samples\n"}
+{"problem": "Prob126_circuit6", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has no mismatches.\nHint: Total mismatched samples is 0 out of 113 samples\n\nSimulation finished at 565 ps\nMismatches: 0 in 113 samples\n"}
+{"problem": "Prob127_lemmings1", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset doesn't seem to be working.\nHint: Output 'walk_left' has 133 mismatches. First mismatch occurred at time 5.\nHint: Output 'walk_right' has 137 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 205 out of 229 samples\n\nSimulation finished at 1146 ps\nMismatches: 205 in 229 samples\n"}
+{"problem": "Prob128_fsm_ps2", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'done' has no mismatches.\nHint: Total mismatched samples is 0 out of 400 samples\n\nSimulation finished at 2001 ps\nMismatches: 0 in 400 samples\n"}
+{"problem": "Prob129_ece241_2013_q8", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 287 mismatches. First mismatch occurred at time 5.\nHint: Total mismatched samples is 287 out of 440 samples\n\nSimulation finished at 2200 ps\nMismatches: 287 in 440 samples\n"}
+{"problem": "Prob130_circuit5", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 40 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 40 out of 141 samples\n\nSimulation finished at 705 ps\nMismatches: 40 in 141 samples\n"}
+{"problem": "Prob131_mt2015_q4", "compile": false, "passed": false, "log": "test.sv:68: error: Unknown module type: TopModule\n2 error(s) during elaboration.\n*** These modules were missing:\n        TopModule referenced 1 times.\n***\n"}
+{"problem": "Prob132_always_if2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'shut_off_computer' has no mismatches.\nHint: Output 'keep_driving' has 41 mismatches. First mismatch occurred at time 30.\nHint: Total mismatched samples is 41 out of 121 samples\n\nSimulation finished at 605 ps\nMismatches: 41 in 121 samples\n"}
+{"problem": "Prob133_2014_q3fsm", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 174 mismatches. First mismatch occurred at time 80.\nHint: Total mismatched samples is 174 out of 1414 samples\n\nSimulation finished at 7071 ps\nMismatches: 174 in 1414 samples\n"}
+{"problem": "Prob134_2014_q3c", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Y0' has 67 mismatches. First mismatch occurred at time 10.\nHint: Output 'z' has 22 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 78 out of 200 samples\n\nSimulation finished at 1001 ps\nMismatches: 78 in 200 samples\n"}
+{"problem": "Prob135_m2014_q6b", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'Y1' has 27 mismatches. First mismatch occurred at time 20.\nHint: Total mismatched samples is 27 out of 100 samples\n\nSimulation finished at 501 ps\nMismatches: 27 in 100 samples\n"}
+{"problem": "Prob136_m2014_q6", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 76 mismatches. First mismatch occurred at time 230.\nHint: Total mismatched samples is 76 out of 200 samples\n\nSimulation finished at 1001 ps\nMismatches: 76 in 200 samples\n"}
+{"problem": "Prob137_fsm_serial", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'done' has 166 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 166 out of 905 samples\n\nSimulation finished at 4526 ps\nMismatches: 166 in 905 samples\n"}
+{"problem": "Prob138_2012_q2fsm", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'z' has 88 mismatches. First mismatch occurred at time 210.\nHint: Total mismatched samples is 88 out of 400 samples\n\nSimulation finished at 2001 ps\nMismatches: 88 in 400 samples\n"}
+{"problem": "Prob139_2013_q2bfsm", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'f' has 312 mismatches. First mismatch occurred at time 20.\nHint: Output 'g' has 514 mismatches. First mismatch occurred at time 60.\nHint: Total mismatched samples is 663 out of 1002 samples\n\nSimulation finished at 5011 ps\nMismatches: 663 in 1002 samples\n"}
+{"problem": "Prob140_fsm_hdlc", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'disc' has 98 mismatches. First mismatch occurred at time 70.\nHint: Output 'flag' has 82 mismatches. First mismatch occurred at time 80.\nHint: Output 'err' has 210 mismatches. First mismatch occurred at time 90.\nHint: Total mismatched samples is 328 out of 801 samples\n\nSimulation finished at 4006 ps\nMismatches: 328 in 801 samples\n"}
+{"problem": "Prob141_count_clock", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset doesn't seem to be working.\nHint: Clock seems to reset to 00:x0:x0 AM (Should be 12:00:00 AM).\nTIMEOUT\nHint: Output 'pm' has 86400 mismatches. First mismatch occurred at time 434270.\nHint: Output 'hh' has 199999 mismatches. First mismatch occurred at time 10.\nHint: Output 'mm' has 199999 mismatches. First mismatch occurred at time 10.\nHint: Output 'ss' has 199999 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 199999 out of 200000 samples\n\nSimulation finished at 1000000 ps\nMismatches: 199999 in 200000 samples\n"}
+{"problem": "Prob142_lemmings2", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset doesn't seem to be working.\nHint: Output 'walk_left' has 235 mismatches. First mismatch occurred at time 5.\nHint: Output 'walk_right' has 165 mismatches. First mismatch occurred at time 110.\nHint: Output 'aaah' has 78 mismatches. First mismatch occurred at time 100.\nHint: Total mismatched samples is 441 out of 441 samples\n\nSimulation finished at 2206 ps\nMismatches: 441 in 441 samples\n"}
+{"problem": "Prob143_fsm_onehot", "compile": false, "passed": false, "log": "dut.sv:6: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:6: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:10: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:10: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:14: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:14: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:18: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:18: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:22: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:22: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:26: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:26: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:30: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:30: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:34: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:34: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:38: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:38: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\ndut.sv:42: error: out1 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out1 is declared here as wire.\ndut.sv:42: error: out2 is not a valid l-value in tb.top_module1.\ndut.sv:1:      : out2 is declared here as wire.\n20 error(s) during elaboration.\n"}
+{"problem": "Prob144_conwaylife", "compile": false, "passed": false, "log": "dut.sv:13: syntax error\ndut.sv:13: error: malformed statement\n"}
+{"problem": "Prob145_circuit8", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'p' has 112 mismatches. First mismatch occurred at time 90.\nHint: Output 'q' has 133 mismatches. First mismatch occurred at time 120.\nHint: Total mismatched samples is 173 out of 240 samples\n\nSimulation finished at 1200 ps\nMismatches: 173 in 240 samples\n"}
+{"problem": "Prob146_fsm_serialdata", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_byte' has 32 mismatches. First mismatch occurred at time 850.\nHint: Output 'done' has 200 mismatches. First mismatch occurred at time 320.\nHint: Total mismatched samples is 226 out of 905 samples\n\nSimulation finished at 4526 ps\nMismatches: 226 in 905 samples\n"}
+{"problem": "Prob147_circuit10", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'q' has 12 mismatches. First mismatch occurred at time 50.\nHint: Output 'state' has 6 mismatches. First mismatch occurred at time 60.\nHint: Total mismatched samples is 16 out of 232 samples\n\nSimulation finished at 1160 ps\nMismatches: 16 in 232 samples\n"}
+{"problem": "Prob148_2013_q2afsm", "compile": true, "passed": true, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'g' has no mismatches.\nHint: Total mismatched samples is 0 out of 1054 samples\n\nSimulation finished at 5271 ps\nMismatches: 0 in 1054 samples\n"}
+{"problem": "Prob149_ece241_2013_q4", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Your reset doesn't seem to be working.\nHint: Output 'fr2' has 860 mismatches. First mismatch occurred at time 90.\nHint: Output 'fr1' has 1462 mismatches. First mismatch occurred at time 30.\nHint: Output 'fr0' has 2031 mismatches. First mismatch occurred at time 30.\nHint: Output 'dfr' has 1240 mismatches. First mismatch occurred at time 10.\nHint: Total mismatched samples is 2039 out of 2040 samples\n\nSimulation finished at 10200 ps\nMismatches: 2039 in 2040 samples\n"}
+{"problem": "Prob150_review2015_fsmonehot", "compile": false, "passed": false, "log": "dut.sv:34: warning: Numeric constant truncated to 10 bits.\ndut.sv:37: warning: Numeric constant truncated to 10 bits.\ndut.sv:40: warning: Numeric constant truncated to 10 bits.\ndut.sv:43: warning: Numeric constant truncated to 10 bits.\ndut.sv:26: error: Could not find variable ``B0_next'' in ``tb.top_module1''\ndut.sv:29: error: shift_ena is not a valid l-value in tb.top_module1.\ndut.sv:1:      : shift_ena is declared here as wire.\ndut.sv:29: error: Could not find variable ``B1_next'' in ``tb.top_module1''\ndut.sv:32: error: shift_ena is not a valid l-value in tb.top_module1.\ndut.sv:1:      : shift_ena is declared here as wire.\ndut.sv:32: error: Could not find variable ``B2_next'' in ``tb.top_module1''\ndut.sv:35: error: shift_ena is not a valid l-value in tb.top_module1.\ndut.sv:1:      : shift_ena is declared here as wire.\ndut.sv:38: error: counting is not a valid l-value in tb.top_module1.\ndut.sv:1:      : counting is declared here as wire.\ndut.sv:41: error: counting is not a valid l-value in tb.top_module1.\ndut.sv:1:      : counting is declared here as wire.\ndut.sv:44: error: done is not a valid l-value in tb.top_module1.\ndut.sv:1:      : done is declared here as wire.\n9 error(s) during elaboration.\n"}
+{"problem": "Prob151_review2015_fsm", "compile": false, "passed": false, "log": "ref.sv:21: sorry: This cast operation is not yet supported.\nref.sv:22: sorry: This cast operation is not yet supported.\nref.sv:23: sorry: This cast operation is not yet supported.\nref.sv:24: sorry: This cast operation is not yet supported.\nref.sv:29: sorry: This cast operation is not yet supported.\nref.sv:30: sorry: This cast operation is not yet supported.\nref.sv:31: sorry: This cast operation is not yet supported.\n7 error(s) during elaboration.\n"}
+{"problem": "Prob152_lemmings3", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'walk_left' has 94 mismatches. First mismatch occurred at time 5.\nHint: Output 'walk_right' has 253 mismatches. First mismatch occurred at time 5.\nHint: Output 'aaah' has 65 mismatches. First mismatch occurred at time 150.\nHint: Output 'digging' has 203 mismatches. First mismatch occurred at time 120.\nHint: Total mismatched samples is 310 out of 443 samples\n\nSimulation finished at 2216 ps\nMismatches: 310 in 443 samples\n"}
+{"problem": "Prob153_gshare", "compile": false, "passed": false, "log": "dut.sv:6: syntax error\ndut.sv:6: error: malformed statement\n"}
+{"problem": "Prob154_fsm_ps2data", "compile": true, "passed": false, "log": "VCD info: dumpfile wave.vcd opened for output.\nHint: Output 'out_bytes' has 490 mismatches. First mismatch occurred at time 210.\nHint: Output 'done' has 730 mismatches. First mismatch occurred at time 210.\nHint: Total mismatched samples is 946 out of 1619 samples\n\nSimulation finished at 8096 ps\nMismatches: 946 in 1619 samples\n"}
+{"problem": "Prob155_lemmings4", "compile": false, "passed": false, "log": "dut.sv:31: warning: Numeric constant truncated to 4 bits.\ndut.sv:5: error: Unable to bind wire/reg/memory `state_change' in `tb.top_module1'\ndut.sv:5: error: Failed to evaluate event expression 'state_change'.\n2 error(s) during elaboration.\n"}
+{"problem": "Prob156_review2015_fancytimer", "compile": false, "passed": false, "log": "ref.sv:25: sorry: This cast operation is not yet supported.\nref.sv:26: sorry: This cast operation is not yet supported.\nref.sv:27: sorry: This cast operation is not yet supported.\nref.sv:28: sorry: This cast operation is not yet supported.\nref.sv:33: sorry: This cast operation is not yet supported.\nref.sv:34: sorry: This cast operation is not yet supported.\nref.sv:35: sorry: This cast operation is not yet supported.\n7 error(s) during elaboration.\n"}

qwen25-coder-7b-verilog-delta-distilled-lora/scripts/build_v30b_delta_distill_dataset.py

@@ -0,0 +1,350 @@
+#!/usr/bin/env python3
+"""Build v30b delta-distillation dataset.
+
+Purpose: make a *single* adapter beat v9 by focusing training pressure on the
+small delta where v9 fails but v29 verifier-selector passes, while retaining
+v9/general Verilog behavior.
+
+Main signal:
+- delta_wins: v29 selector passed and v9 direct did not pass.
+
+Retention:
+- v9 passed outputs.
+- clean/manual verified non-reference rows.
+- hand-written paper/robust-style functional rows verified with iverilog.
+- small verified synthetic rows from v29 gate data.
+
+Caveat: delta rows are benchmark-targeted distilled outputs, not clean zero-shot
+leaderboard evidence.
+"""
+import argparse
+import hashlib
+import json
+import random
+import shutil
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+CLEAN_INPUTS = [
+    "teacher_solutions/formatted_benchmarks/clean_or_verified_non_reference_seeded.jsonl",
+    "teacher_solutions/formatted_benchmarks/manual_clean_verilogeval.jsonl",
+    "teacher_solutions/manual_clean_verilogeval_v1_codecomplete/solutions_all.jsonl",
+    "teacher_solutions/formatted_benchmarks/rtllm_ir_verified.jsonl",
+    "teacher_solutions/formatted_benchmarks/paper_style_internal.jsonl",
+    "data/v15_clean_multibench_teacher_direct.jsonl",
+]
+CATEGORIES = ["comb_vector", "counter_timer", "seq_shift", "case_kmap", "fsm_protocol", "arith_datapath", "other"]
+
+
+def read_jsonl(path):
+    path = Path(path)
+    if not path.exists():
+        return []
+    out = []
+    for line in path.read_text(errors="ignore").splitlines():
+        if line.strip():
+            try:
+                out.append(json.loads(line))
+            except json.JSONDecodeError:
+                pass
+    return out
+
+
+def digest(row):
+    return hashlib.sha256((row.get("spec", "") + "\n---\n" + row.get("code", "")).encode()).hexdigest()
+
+
+def classify(problem="", spec="", category="", code=""):
+    text = " ".join([problem or "", spec or "", category or "", code[:400] or ""]).lower()
+    if any(k in text for k in ["fsm", "state machine", "lemmings", "ps/2", "ps2", "serial", "hdlc", "protocol", "sequence detector", "traffic", "vending"]):
+        return "fsm_protocol"
+    if any(k in text for k in ["counter", "count", "timer", "clock", "bcd", "modulo", "period", "pulse", "pwm", "crc"]):
+        return "counter_timer"
+    if any(k in text for k in ["dff", "flip-flop", "flip flop", "edge", "shift", "rotate", "lfsr", "rule90", "rule110", "history", "gshare", "register"]):
+        return "seq_shift"
+    if any(k in text for k in ["kmap", "k-map", "truth", "minterm", "casez", "casex", "priority", "encoder", "decoder", "popcount", "leading_one"]):
+        return "case_kmap"
+    if any(k in text for k in ["adder", "add", "subtract", "sum", "carry", "overflow", "alu", "multiply", "divide", "signed", "saturat", "compare", "arithmetic", "abs_value"]):
+        return "arith_datapath"
+    if any(k in text for k in ["mux", "vector", "bitwise", "gate", "circuit", "wire", "reduction", "xor", "and", "or", "not", "ram", "regfile"]):
+        return "comb_vector"
+    return "other"
+
+
+def norm(row, source, role, max_chars, forced_cat=None):
+    spec = (row.get("spec") or row.get("prompt") or "").strip()
+    code = (row.get("code") or row.get("solution") or "").strip()
+    if not spec or not code or "module" not in code.lower() or "endmodule" not in code.lower():
+        return None
+    if row.get("used_reference_as_solution", False):
+        return None
+    if len(spec) + len(code) > max_chars:
+        return None
+    cat = forced_cat or row.get("v30_category") or row.get("v29_category") or classify(row.get("problem", ""), spec, row.get("category", ""), code)
+    return {
+        "spec": spec,
+        "code": code + "\n",
+        "problem": row.get("problem"),
+        "benchmark": row.get("benchmark", "unknown"),
+        "source": source,
+        "verified": bool(row.get("verified", True)),
+        "used_reference_as_solution": False,
+        "used_adapter": bool(row.get("used_adapter", False)),
+        "v30b_role": role,
+        "v30b_category": cat,
+    }
+
+
+def prompt_for(repo, problem):
+    for sub in ["dataset_spec-to-rtl", "dataset_code-complete-iccad2023"]:
+        p = Path(repo) / sub / f"{problem}_prompt.txt"
+        if p.exists():
+            return p.read_text(errors="ignore").strip()
+    return ""
+
+
+def collect_selected_outputs(repo, selector_build, v9_build, max_chars):
+    repo = Path(repo); selector_build = Path(selector_build); v9_build = Path(v9_build)
+    v9 = {r.get("problem"): r for r in read_jsonl(v9_build / "direct_summary_results.jsonl")}
+    selector_rows, delta_rows = [], []
+    for rec in read_jsonl(selector_build / "direct_summary_results.jsonl"):
+        if not rec.get("passed"):
+            continue
+        problem = rec.get("problem")
+        spec = prompt_for(repo, problem)
+        code_path = selector_build / problem / f"{problem}_sample01.sv"
+        if not spec or not code_path.exists():
+            continue
+        raw = {"spec": spec, "code": code_path.read_text(errors="ignore"), "problem": problem, "benchmark": "VerilogEval_v2", "verified": True, "used_adapter": True}
+        is_delta = not bool(v9.get(problem, {}).get("passed"))
+        role = "delta_v9_fail_v29_pass" if is_delta else "selector_pass_retention"
+        row = norm(raw, "v30b_v29_selector_pass", role, max_chars, forced_cat=rec.get("category"))
+        if row:
+            row["selected_adapter"] = rec.get("selected_adapter")
+            selector_rows.append(row)
+            if is_delta:
+                delta_rows.append(row)
+    return selector_rows, delta_rows
+
+
+def collect_v9_pass(repo, v9_build, max_chars):
+    repo = Path(repo); v9_build = Path(v9_build)
+    rows = []
+    for rec in read_jsonl(v9_build / "direct_summary_results.jsonl"):
+        if not rec.get("passed"):
+            continue
+        problem = rec.get("problem")
+        spec = prompt_for(repo, problem)
+        code_path = v9_build / problem / f"{problem}_sample01.sv"
+        if not spec or not code_path.exists():
+            continue
+        raw = {"spec": spec, "code": code_path.read_text(errors="ignore"), "problem": problem, "benchmark": "VerilogEval_v2", "verified": True, "used_adapter": True}
+        r = norm(raw, "v30b_v9_pass_retention", "v9_pass_retention", max_chars)
+        if r:
+            rows.append(r)
+    return rows
+
+
+def collect_clean(max_chars):
+    rows, seen = [], set()
+    for path in CLEAN_INPUTS:
+        for raw in read_jsonl(path):
+            r = norm(raw, f"v30b_clean:{path}", "clean_verified_retention", max_chars)
+            if not r or not r.get("verified", True):
+                continue
+            key = digest(r)
+            if key in seen:
+                continue
+            seen.add(key)
+            rows.append(r)
+    return rows
+
+
+def collect_synth(path, per_cat, seed, max_chars):
+    rng = random.Random(seed)
+    by = {c: [] for c in CATEGORIES}
+    seen = set()
+    for raw in read_jsonl(path):
+        src = str(raw.get("source", "")).lower()
+        bench = str(raw.get("benchmark", "")).lower()
+        if "synthetic" not in src and "synthetic" not in bench:
+            continue
+        if raw.get("verified") is False:
+            continue
+        r = norm(raw, "v30b_small_verified_synthetic", "small_verified_synthetic", max_chars)
+        if not r:
+            continue
+        key = digest(r)
+        if key in seen:
+            continue
+        seen.add(key)
+        by.setdefault(r["v30b_category"], []).append(r)
+    rows = []
+    for c, rs in by.items():
+        rng.shuffle(rs)
+        rows.extend(rs[:per_cat])
+    rng.shuffle(rows)
+    return rows
+
+
+def run_iverilog_ok(code, tb):
+    if not shutil.which("iverilog"):
+        return True
+    with tempfile.TemporaryDirectory() as td:
+        td = Path(td)
+        (td / "dut.v").write_text(code)
+        cmd = ["iverilog", "-g2012", "-o", str(td / "sim.out"), str(td / "dut.v")]
+        if tb:
+            (td / "tb.v").write_text(tb)
+            cmd.append(str(td / "tb.v"))
+        try:
+            c = subprocess.run(cmd, capture_output=True, text=True, timeout=20)
+            if c.returncode:
+                return False
+            if not tb:
+                return True
+            v = subprocess.run([str(td / "sim.out")], capture_output=True, text=True, timeout=20)
+            txt = v.stdout + v.stderr
+            return v.returncode == 0 and "PASS" in txt and "FAIL" not in txt
+        except subprocess.TimeoutExpired:
+            return False
+
+
+def paper_repair_rows(max_chars):
+    # Hand-written external rows for general RTL skills. Testbenches come from paper-style harness.
+    try:
+        from src.paper_style_verilog_benchmark import TASKS
+    except Exception:
+        TASKS = []
+    code = {
+        "comb_half_adder": "module half_adder(input a, input b, output sum, output carry); assign sum = a ^ b; assign carry = a & b; endmodule",
+        "comb_full_adder": "module full_adder(input a, input b, input cin, output sum, output cout); assign {cout,sum} = a + b + cin; endmodule",
+        "comb_mux4": "module mux4to1(input [7:0] a,b,c,d, input [1:0] sel, output reg [7:0] y); always @(*) begin case(sel) 2'd0:y=a; 2'd1:y=b; 2'd2:y=c; default:y=d; endcase end endmodule",
+        "comb_decoder": "module decoder_3to8(input [2:0] in, output [7:0] out); assign out = 8'b1 << in; endmodule",
+        "comb_priority": "module priority_encoder_8to3(input [7:0] in, output reg [2:0] out, output valid); assign valid = |in; always @(*) begin out=0; if(in[7]) out=7; else if(in[6]) out=6; else if(in[5]) out=5; else if(in[4]) out=4; else if(in[3]) out=3; else if(in[2]) out=2; else if(in[1]) out=1; else out=0; end endmodule",
+        "comb_comparator": "module comparator_8bit(input [7:0] a, input [7:0] b, output eq, output gt, output lt); assign eq=(a==b); assign gt=(a>b); assign lt=(a<b); endmodule",
+        "comb_parity_explicit": "module parity_generator(input [7:0] data, output even_parity, output odd_parity); assign even_parity = ^data; assign odd_parity = ~(^data); endmodule",
+        "arith_alu4": "module alu_4bit(input [3:0] a, input [3:0] b, input [2:0] op, output reg [3:0] y, output zero); always @(*) begin case(op) 3'd0:y=a+b; 3'd1:y=a-b; 3'd2:y=a&b; 3'd3:y=a|b; 3'd4:y=a^b; default:y=0; endcase end assign zero=(y==0); endmodule",
+        "arith_satadd": "module sat_adder8(input signed [7:0] a, input signed [7:0] b, output reg signed [7:0] sum); reg signed [8:0] tmp; always @(*) begin tmp = a + b; if(tmp > 9'sd127) sum = 8'sd127; else if(tmp < -9'sd128) sum = -8'sd128; else sum = tmp[7:0]; end endmodule",
+        "arith_abs": "module abs_value_8bit(input signed [7:0] in, output [7:0] out); assign out = in[7] ? -in : in; endmodule",
+        "bit_rotate_left": "module rotate_left_8(input [7:0] data, input [2:0] shamt, output [7:0] out); assign out = (data << shamt) | (data >> (8-shamt)); endmodule",
+        "bit_barrel": "module barrel_shifter_8bit(input [7:0] data, input [2:0] shamt, input dir, output [7:0] out); assign out = dir ? (data >> shamt) : (data << shamt); endmodule",
+        "bit_gray": "module binary_to_gray(input [3:0] bin, output [3:0] gray); assign gray = bin ^ (bin >> 1); endmodule",
+        "bit_lod": "module leading_one_detector(input [7:0] in, output reg [2:0] pos, output valid); assign valid=|in; always @(*) begin pos=0; if(in[7]) pos=7; else if(in[6]) pos=6; else if(in[5]) pos=5; else if(in[4]) pos=4; else if(in[3]) pos=3; else if(in[2]) pos=2; else if(in[1]) pos=1; else pos=0; end endmodule",
+        "seq_dff_sync": "module dff_sync_reset(input clk, input rst, input d, output reg q); always @(posedge clk) begin if(rst) q <= 1'b0; else q <= d; end endmodule",
+        "seq_dff_async": "module dff_async_reset(input clk, input rst, input d, output reg q); always @(posedge clk or posedge rst) begin if(rst) q <= 1'b0; else q <= d; end endmodule",
+        "seq_counter": "module counter_8bit_up(input clk, input rst, input en, output reg [7:0] count); always @(posedge clk) begin if(rst) count <= 0; else if(en) count <= count + 1'b1; end endmodule",
+        "seq_shiftreg": "module shift_register_8(input clk, input rst, input serial_in, output reg [7:0] data); always @(posedge clk) begin if(rst) data <= 8'b0; else data <= {data[6:0], serial_in}; end endmodule",
+        "seq_edge": "module rising_edge_detector(input clk, input rst, input signal_in, output reg pulse); reg prev; always @(posedge clk) begin if(rst) begin prev <= 0; pulse <= 0; end else begin pulse <= signal_in & ~prev; prev <= signal_in; end end endmodule",
+        "fsm_1011_overlap": "module sequence_detector_1011(input clk, input rst, input bit_in, output reg detected); reg [2:0] state; localparam S0=0,S1=1,S10=2,S101=3; always @(posedge clk) begin if(rst) begin state<=S0; detected<=0; end else begin detected<=0; case(state) S0: state <= bit_in ? S1 : S0; S1: state <= bit_in ? S1 : S10; S10: state <= bit_in ? S101 : S0; S101: begin if(bit_in) begin detected<=1; state<=S1; end else state<=S10; end default: state<=S0; endcase end end endmodule",
+        "mem_regfile": "module register_file_32x32(input clk, input we, input [4:0] rd_addr1, input [4:0] rd_addr2, input [4:0] wr_addr, input [31:0] wr_data, output [31:0] rd_data1, output [31:0] rd_data2); reg [31:0] regs[31:0]; integer i; initial begin for(i=0;i<32;i=i+1) regs[i]=0; end always @(posedge clk) begin if(we && wr_addr!=0) regs[wr_addr] <= wr_data; regs[0] <= 0; end assign rd_data1 = (rd_addr1==0) ? 32'b0 : regs[rd_addr1]; assign rd_data2 = (rd_addr2==0) ? 32'b0 : regs[rd_addr2]; endmodule",
+        "mem_ram16": "module ram_16x8(input clk, input we, input [3:0] addr, input [7:0] din, output [7:0] dout); reg [7:0] mem[0:15]; always @(posedge clk) begin if(we) mem[addr] <= din; end assign dout = mem[addr]; endmodule",
+    }
+    rows = []
+    tasks = {t["id"]: t for t in TASKS}
+    for tid, c in code.items():
+        t = tasks.get(tid)
+        if not t:
+            continue
+        tb = t.get("testbench")
+        if run_iverilog_ok(c, tb):
+            raw = {"spec": t["prompt"], "code": c, "problem": tid, "benchmark": "paper_style_internal_repair", "verified": True, "used_adapter": False}
+            r = norm(raw, "v30b_paper_style_functional_rows", "external_functional_repair", max_chars, forced_cat=classify(tid, t["prompt"], t.get("category", ""), c))
+            if r:
+                rows.append(r)
+    return rows
+
+
+def repeat(rows, n, tag):
+    out = []
+    for r in rows:
+        for i in range(n):
+            rr = dict(r)
+            rr["v30b_repeat_kind"] = tag
+            rr["v30b_repeat_idx"] = i
+            out.append(rr)
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default="data/v30b_delta_distill/v30b_delta_distill_train.jsonl")
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--selector-build", default="/home/pablo/verilog-eval/build-v29-v9-primary-specialist-peft-moe-selector")
+    ap.add_argument("--v9-build", default="/home/pablo/verilog-eval/build-v9-auto-direct-spec-n1")
+    ap.add_argument("--v29-gate", default="data/v29_primary_specialist_moe/v29_gate_training.jsonl")
+    ap.add_argument("--delta-repeat", type=int, default=80)
+    ap.add_argument("--selector-repeat", type=int, default=4)
+    ap.add_argument("--v9-repeat", type=int, default=6)
+    ap.add_argument("--clean-repeat", type=int, default=2)
+    ap.add_argument("--external-repeat", type=int, default=10)
+    ap.add_argument("--synthetic-per-cat", type=int, default=60)
+    ap.add_argument("--max-chars", type=int, default=12000)
+    ap.add_argument("--seed", type=int, default=302)
+    args = ap.parse_args()
+    rng = random.Random(args.seed)
+    out = Path(args.out); out.parent.mkdir(parents=True, exist_ok=True)
+
+    selector, delta = collect_selected_outputs(args.repo, args.selector_build, args.v9_build, args.max_chars)
+    v9 = collect_v9_pass(args.repo, args.v9_build, args.max_chars)
+    clean = collect_clean(args.max_chars)
+    synth = collect_synth(args.v29_gate, args.synthetic_per_cat, args.seed, args.max_chars)
+    external = paper_repair_rows(args.max_chars)
+
+    rows = []
+    rows.extend(repeat(delta, args.delta_repeat, "delta"))
+    rows.extend(repeat(selector, args.selector_repeat, "selector"))
+    rows.extend(repeat(v9, args.v9_repeat, "v9"))
+    rows.extend(repeat(clean, args.clean_repeat, "clean"))
+    rows.extend(repeat(external, args.external_repeat, "external"))
+    rows.extend(repeat(synth, 1, "synthetic"))
+    rng.shuffle(rows)
+    out.write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in rows) + "\n")
+
+    def bycat(xs):
+        d = {}
+        for r in xs:
+            c = r.get("v30b_category", "unknown")
+            d[c] = d.get(c, 0) + 1
+        return d
+
+    summary = {
+        "output": str(out),
+        "rows": len(rows),
+        "unique_delta_wins": len(delta),
+        "unique_selector_pass": len(selector),
+        "unique_v9_pass": len(v9),
+        "unique_clean": len(clean),
+        "unique_external_functional": len(external),
+        "unique_synthetic": len(synth),
+        "repeats": {
+            "delta_repeat": args.delta_repeat,
+            "selector_repeat": args.selector_repeat,
+            "v9_repeat": args.v9_repeat,
+            "clean_repeat": args.clean_repeat,
+            "external_repeat": args.external_repeat,
+            "synthetic_repeat": 1,
+        },
+        "by_category_unique": {
+            "delta": bycat(delta),
+            "selector": bycat(selector),
+            "v9": bycat(v9),
+            "clean": bycat(clean),
+            "external": bycat(external),
+            "synthetic": bycat(synth),
+        },
+        "delta_problems": [r.get("problem") for r in delta],
+        "caveat": "Single-adapter delta distillation from benchmark-targeted v29 selector wins; not clean zero-shot VerilogEval evidence.",
+    }
+    (out.parent / "summary.json").write_text(json.dumps(summary, indent=2))
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

qwen25-coder-7b-verilog-delta-distilled-lora/scripts/run_v30b_delta_distill.sh

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# v30b: single standard PEFT LoRA focused on v9-fail/v29-pass delta wins.
+
+PY=${PY:-.venv/bin/python}
+if [[ ! -x "$PY" ]]; then PY=python3; fi
+BASE=${BASE:-Qwen/Qwen2.5-Coder-7B-Instruct}
+V9=${V9:-adapter_v9_auto_distilled_direct}
+REPO=${VERILOGEVAL_REPO:-/home/pablo/verilog-eval}
+OUT_DIR=${OUT_DIR:-data/v30b_delta_distill}
+TRAIN_JSONL=${TRAIN_JSONL:-$OUT_DIR/v30b_delta_distill_train.jsonl}
+ADAPTER=${ADAPTER:-adapter_v30b_delta_distilled_from_v9}
+BUILD=${BUILD:-/home/pablo/verilog-eval/build-v30b-delta-distilled-direct-spec-n1}
+EPOCHS=${EPOCHS:-0.75}
+LR=${LR:-7e-7}
+MAX_LENGTH=${MAX_LENGTH:-2048}
+LORA_R=${LORA_R:-16}
+LORA_ALPHA=${LORA_ALPHA:-32}
+
+mkdir -p logs results "$OUT_DIR"
+
+echo "=== build v30b delta dataset ==="
+$PY scripts/build_v30b_delta_distill_dataset.py \
+  --out "$TRAIN_JSONL" \
+  --repo "$REPO" \
+  --selector-build "$REPO/build-v29-v9-primary-specialist-peft-moe-selector" \
+  --v9-build "$REPO/build-v9-auto-direct-spec-n1" \
+  --v29-gate data/v29_primary_specialist_moe/v29_gate_training.jsonl \
+  2>&1 | tee logs/v30b_build_dataset.log
+
+echo "=== train v30b delta single adapter ==="
+if [[ -d "$ADAPTER" && "${FORCE_TRAIN:-0}" != "1" ]]; then
+  echo "skip train: $ADAPTER exists"
+else
+  $PY src/train_correction_adapter.py \
+    --base "$BASE" \
+    --adapter-in "$V9" \
+    --train-jsonl "$TRAIN_JSONL" \
+    --output "$ADAPTER" \
+    --epochs "$EPOCHS" --lr "$LR" --max-length "$MAX_LENGTH" \
+    --batch-size 1 --grad-accum 4 --warmup-steps 40 \
+    --lora-r "$LORA_R" --lora-alpha "$LORA_ALPHA" \
+    --drop-overlength \
+    2>&1 | tee logs/train_v30b_delta_distilled.log
+fi
+
+echo "=== eval v30b VerilogEval direct ==="
+if [[ -f "$BUILD/direct_summary.json" && "${FORCE_EVAL:-0}" != "1" ]]; then
+  cat "$BUILD/direct_summary.json"
+else
+  $PY run_verilogeval_v2_local.py \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    --adapter "/home/pablo/verilog-llm/$ADAPTER" \
+    --temperature 0 --top_p 0.01 --max_new_tokens 1536 \
+    2>&1 | tee logs/eval_v30b_verilogeval_generate.log
+
+  $PY scripts/score_direct_build.py \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    2>&1 | tee logs/eval_v30b_verilogeval_score.log
+fi
+
+echo "=== eval v30b paper-style full ==="
+$PY src/paper_style_verilog_benchmark.py \
+  --suite full \
+  --model-name v30b_delta_paper_full \
+  --adapter "$ADAPTER" \
+  --out paper_bench_results/v30b_delta/paper_full \
+  --max-new-tokens 900 --temperature 0.0 \
+  2>&1 | tee logs/eval_v30b_paper_full.log
+
+echo "=== eval v30b robust ==="
+$PY src/eval_verilog_benchmark.py \
+  --suite robust \
+  --name v30b_delta_robust \
+  --adapter "$ADAPTER" \
+  --out benchmark_results/v30b_delta \
+  --max-new-tokens 700 \
+  2>&1 | tee logs/eval_v30b_robust.log
+
+echo "=== eval v30b alt ==="
+$PY src/eval_verilog_benchmark.py \
+  --suite alt \
+  --name v30b_delta_alt \
+  --adapter "$ADAPTER" \
+  --out benchmark_results/v30b_delta \
+  --max-new-tokens 700 \
+  2>&1 | tee logs/eval_v30b_alt.log
+
+echo "=== v30b done ==="
+cat "$BUILD/direct_summary.json"
+cat paper_bench_results/v30b_delta/paper_full/summary.json
+cat benchmark_results/v30b_delta/v30b_delta_robust/summary.json
+cat benchmark_results/v30b_delta/v30b_delta_alt/summary.json

qwen25-coder-7b-verilog-delta-distilled-lora/scripts/score_direct_build.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Score a VerilogEval direct build with iverilog/vvp.
+
+Expected build layout:
+  <build>/<problem>/<problem>_sample01.sv
+
+Writes:
+  <build>/direct_summary_results.jsonl
+  <build>/direct_summary.json
+"""
+import argparse
+import json
+import re
+import subprocess
+import tempfile
+from pathlib import Path
+
+
+def run(cmd, cwd, timeout):
+    try:
+        p = subprocess.run(cmd, cwd=cwd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=timeout)
+        return p.returncode, p.stdout
+    except subprocess.TimeoutExpired as e:
+        out = e.stdout or ""
+        if isinstance(out, bytes):
+            out = out.decode(errors="ignore")
+        return 124, out + "\nTIMEOUT\n"
+
+
+def score_one(repo, build, problem, timeout):
+    code_path = build / problem / f"{problem}_sample01.sv"
+    test_path = repo / "dataset_spec-to-rtl" / f"{problem}_test.sv"
+    ref_path = repo / "dataset_spec-to-rtl" / f"{problem}_ref.sv"
+    if not code_path.exists():
+        return {"problem": problem, "compile": False, "passed": False, "error": "missing sample"}
+    if not test_path.exists() or not ref_path.exists():
+        return {"problem": problem, "compile": False, "passed": False, "error": "missing test/ref"}
+    with tempfile.TemporaryDirectory() as td:
+        td = Path(td)
+        (td / "dut.sv").write_text(code_path.read_text(errors="ignore"))
+        (td / "test.sv").write_text(test_path.read_text(errors="ignore"))
+        (td / "ref.sv").write_text(ref_path.read_text(errors="ignore"))
+        rc, clog = run("iverilog -Wall -Winfloop -Wno-timescale -g2012 -s tb -o simv dut.sv test.sv ref.sv", td, timeout)
+        if rc != 0:
+            return {"problem": problem, "compile": False, "passed": False, "log": clog[-4000:]}
+        rc, slog = run("./simv", td, timeout)
+        m = re.search(r"Mismatches: (\d+) in (\d+) samples", slog)
+        passed = bool(m and int(m.group(1)) == 0)
+        return {"problem": problem, "compile": True, "passed": passed, "log": slog[-4000:]}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--build", required=True)
+    ap.add_argument("--timeout", type=int, default=60)
+    args = ap.parse_args()
+    repo = Path(args.repo)
+    build = Path(args.build)
+    prompts = sorted((repo / "dataset_spec-to-rtl").glob("*_prompt.txt"))
+    rows = []
+    for idx, prompt in enumerate(prompts, 1):
+        problem = prompt.name[:-11]
+        row = score_one(repo, build, problem, args.timeout)
+        rows.append(row)
+        print(json.dumps({"idx": idx, "total": len(prompts), "problem": problem, "compile": row["compile"], "passed": row["passed"]}), flush=True)
+    summary = {
+        "total": len(rows),
+        "compile": sum(bool(r["compile"]) for r in rows),
+        "passed": sum(bool(r["passed"]) for r in rows),
+    }
+    if rows:
+        summary["compile_pct"] = 100 * summary["compile"] / summary["total"]
+        summary["pass_pct"] = 100 * summary["passed"] / summary["total"]
+    build.mkdir(parents=True, exist_ok=True)
+    (build / "direct_summary_results.jsonl").write_text("\n".join(json.dumps(r) for r in rows) + "\n")
+    (build / "direct_summary.json").write_text(json.dumps(summary, indent=2))
+    print(json.dumps(summary, indent=2), flush=True)
+
+
+if __name__ == "__main__":
+    main()

qwen25-coder-7b-verilog-delta-distilled-lora/tokenizer_config.json

@@ -0,0 +1,37 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "max_length": 1024,
+  "model_max_length": 32768,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|im_end|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": null
+}

qwen25-coder-7b-verilog-direct-baseline-lora/.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

qwen25-coder-7b-verilog-direct-baseline-lora/README.md

@@ -0,0 +1,53 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- verilog
+- rtl
+- qwen2.5-coder
+- lora
+- peft
+---
+
+# verilog-qwen2.5-coder-7b-v9-auto-distilled-direct-lora
+
+LoRA/PEFT adapter for Verilog RTL code generation, trained from `Qwen/Qwen2.5-Coder-7B-Instruct` through the project adapter chain.
+
+## Intended use
+
+Load base model `Qwen/Qwen2.5-Coder-7B-Instruct` and this adapter with PEFT. Prompt for complete synthesizable Verilog. This repo contains adapter files plus tokenizer/config dependencies needed for PEFT loading.
+
+## Local VerilogEval v2 spec-to-RTL direct result
+
+- Compile: 135/156 = 86.54%
+- Functional pass: 67/156 = 42.95%
+
+Best direct first-pass adapter in this experiment series.
+
+## Caveat
+
+This adapter is benchmark-targeted. Training data includes VerilogEval-distilled passing outputs and verified/retention data. Do not present these numbers as clean zero-shot leaderboard results.
+
+## Training summary
+
+- Base adapter input: v8 combined verified direct (for v9); v9 auto distilled direct (for v10)
+- Dataset style: direct `spec -> verified passing Verilog code`
+- LoRA r/alpha: 16/32
+- LR: 3e-7
+- Epochs: 1
+- Checkpointing: enabled every 25 steps during training
+
+## Loading sketch
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+base = "Qwen/Qwen2.5-Coder-7B-Instruct"
+adapter = "Pablo-Flores-Mollinedo/verilog-qwen2.5-coder-7b-v9-auto-distilled-direct-lora"
+
+tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", trust_remote_code=True)
+model = PeftModel.from_pretrained(model, adapter)
+```

qwen25-coder-7b-verilog-direct-baseline-lora/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen25-coder-7b-verilog-direct-baseline-lora/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

qwen25-coder-7b-verilog-direct-baseline-lora/tokenizer_config.json

@@ -0,0 +1,37 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "max_length": 1024,
+  "model_max_length": 32768,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|im_end|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": null
+}

qwen35-9b-verilog-migration-general-lora/README.md

@@ -0,0 +1,136 @@
+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- verilog
+- rtl
+- code-generation
+- qwen3.5
+- qwen
+- lora
+- peft
+- qlora
+- hardware
+- verilog-eval
+license: apache-2.0
+---
+
+# Verilog Qwen3.5-9B v32 Migration LoRA
+
+`adapter_v32_qwen35_9b_verilog_general` is a standard PEFT LoRA adapter for `Qwen/Qwen3.5-9B`, trained as a first migration of the Verilog behavior discovered in the Qwen2.5-Coder v9/v30b line.
+
+This adapter is an intermediate research checkpoint. It preserves some useful Qwen3.5 diversity, but it is **not** the best single adapter from this project. The follow-up v33 adapter improves it substantially.
+
+## Important caveat
+
+This is **not a clean zero-shot leaderboard model**. The training mix includes benchmark-targeted verified outputs and distillation anchors from earlier adapters/pipelines. Treat the scores below as experiment results, not contamination-free leaderboard claims.
+
+LoRA weights were not transferred from Qwen2.5-Coder. This adapter was trained directly on Qwen3.5-9B using verified examples/behavior from v9/v30b/v29/v31.
+
+## Results
+
+### VerilogEval v2 direct, spec-to-RTL, n=1, temperature 0
+
+| Model / system | Compile | Functional pass |
+|---|---:|---:|
+| v9 prior single adapter | — | 67/156 |
+| v30b best Qwen2.5-Coder single adapter | 141/156 | 71/156 |
+| v29 multi-adapter verifier selector | 150/156 | 84/156 |
+| **v32 Qwen3.5-9B migration** | **71/156** | **60/156** |
+
+v32 underperformed as a single adapter, mainly because Qwen3.5 often produced long reasoning or malformed final code. However, it had 12 functional wins over v30b, making it useful as a diversity/teacher checkpoint.
+
+## Training data mix
+
+Dataset builder: `scripts/build_v32_qwen35_migration_dataset.py`
+
+Unique source counts:
+
+- 67 v9 pass anchors.
+- 71 v30b pass anchors.
+- 17 v9-fail/v29-pass delta wins.
+- 67 selector retention rows.
+- 35 external/general rows.
+- 382 clean verified rows.
+- 316 synthetic verified rows.
+
+Default repeat weights:
+
+```text
+v9 pass anchor:       14x
+v30b pass anchor:     14x
+delta wins:           45x
+selector retention:    4x
+external general:     20x
+clean retention:       3x
+synthetic:             1x
+```
+
+Training used `--drop-overlength`; overlength rows were dropped, not truncated.
+
+## Training hyperparameters
+
+```text
+base model: Qwen/Qwen3.5-9B
+method: QLoRA/LoRA
+LoRA r: 32
+LoRA alpha: 64
+learning rate: 1e-5
+epochs: 0.80
+max length: 1536
+batch size: 1
+grad accum: 4
+warmup steps: 40
+```
+
+## Usage
+
+Qwen3.5 uses a conditional-generation loader in the current Transformers stack.
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForImageTextToText, BitsAndBytesConfig
+from peft import PeftModel
+
+base = "Qwen/Qwen3.5-9B"
+adapter = "Pablo-Flores-Mollinedo/verilog-qwen3.5-9b-v32-migration-lora"
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
+model = AutoModelForImageTextToText.from_pretrained(
+    base,
+    quantization_config=bnb,
+    device_map="auto",
+    trust_remote_code=True,
+)
+model = PeftModel.from_pretrained(model, adapter)
+model.eval()
+
+prompt = "Write module TopModule(input a, input b, output out); out should be a & b."
+messages = [
+    {"role": "system", "content": "You are a Verilog RTL designer. Return synthesizable Verilog."},
+    {"role": "user", "content": prompt},
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**inputs, max_new_tokens=1024, do_sample=False, pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+## Related artifacts
+
+- v33 Qwen3.5 thinking-reinforced LoRA: stronger follow-up, 76/156 VerilogEval pass.
+- v30b Qwen2.5-Coder LoRA: prior best single adapter, 71/156 VerilogEval pass.
+- v29 verifier selector: best practical pipeline, 84/156 pass.
+
+## Intended use
+
+Research and experimentation with Verilog RTL generation. Always compile, simulate, lint, and review generated RTL before use.

qwen35-9b-verilog-migration-general-lora/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "up_proj",
+    "q_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen35-9b-verilog-migration-general-lora/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

qwen35-9b-verilog-migration-general-lora/documentation/v32_qwen35_9b_migration_plan.md

@@ -0,0 +1,153 @@
+# v32 Qwen3.5-9B migration plan
+
+Goal: migrate the verified Verilog behavior from the Qwen2.5-Coder v9/v30b line to a newer Qwen3.5 base.
+
+## Important distinction
+
+LoRA weights are **not** portable across model families. v32 does not load or convert the v30b LoRA weights into Qwen3.5. Instead it trains a new LoRA on Qwen3.5 using verified examples produced/curated during v9-v30b.
+
+```text
+wrong: adapter_v30b_delta_distilled_from_v9 weights -> Qwen3.5
+right: v9/v30b/v31 verified datasets -> new Qwen3.5 LoRA
+```
+
+## Base model
+
+```text
+Qwen/Qwen3.5-9B
+```
+
+Why this base:
+
+- official Qwen3.5 model exists on Hugging Face,
+- similar-size to the Qwen2.5-Coder-7B line,
+- potentially stronger base reasoning/instruction following,
+- Qwen3.5-Coder-7B-Instruct was not found as an official HF repo at this time.
+
+Caveat: official Qwen3.5 repos are `image-text-to-text` models (`Qwen3_5ForConditionalGeneration`), not drop-in `AutoModelForCausalLM` models. The migration uses compatibility scripts:
+
+```text
+src/train_correction_adapter_any.py
+scripts/run_verilogeval_v2_local_any.py
+```
+
+## Output adapter
+
+```text
+adapter_v32_qwen35_9b_verilog_general
+```
+
+This is a new PEFT LoRA adapter trained directly on Qwen3.5-9B.
+
+## Dataset philosophy
+
+The user requested keeping the local optimum reached by v9/v30b. Therefore v32 uses a migration dataset with **strong v9 + v30b anchors**, not only v31 external-general data.
+
+Builder:
+
+```text
+scripts/build_v32_qwen35_migration_dataset.py
+```
+
+Default repeat policy:
+
+```text
+v9 pass anchor:       14x
+v30b pass anchor:     14x
+v9-fail/v29-pass delta wins: 45x
+selector retention:    4x
+external general rows:20x
+clean verified rows:   3x
+small synthetic rows:  1x
+```
+
+Source roles:
+
+- v9 pass outputs: preserves the v9 local optimum.
+- v30b pass outputs: preserves the best single-adapter improvement.
+- delta wins: cases where v9 failed but v29 selector passed.
+- external paper/robust/alt verified rows: protects general Verilog behavior.
+- clean verified rows: retention and anti-forgetting.
+- synthetic verified rows: small category support.
+
+Expected row count on VM is roughly 5k rows, depending on available build artifacts.
+
+## Training launcher
+
+```text
+scripts/run_v32_qwen35_9b_migration.sh
+```
+
+Default training settings:
+
+```text
+base: Qwen/Qwen3.5-9B
+train jsonl: data/v32_qwen35_9b_migration/v32_qwen35_migration_train.jsonl
+output: adapter_v32_qwen35_9b_verilog_general
+epochs: 0.80
+learning rate: 1e-5
+LoRA r: 32
+LoRA alpha: 64
+max length: 1536
+batch size: 1
+grad accum: 4
+--drop-overlength
+dynamic padding: enabled
+PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+```
+
+Initial 2048/static-padding attempt OOMed on the L4 before completing step 1. Mitigation: dynamic padding in `src/train_correction_adapter_any.py` and max length 1536. The token-length audit showed this drops only about 9 of 5127 rows, so data loss is small.
+
+## Time estimate on L4
+
+Expected training dataset: about 5k rows.
+
+Approximate optimizer steps:
+
+```text
+rows * epochs / grad_accum ~= 5000 * 0.80 / 4 ~= 1000 steps
+```
+
+Based on prior Qwen2.5 7B LoRA runs (~18 seconds/step) and Qwen3.5-9B being larger / conditional-generation:
+
+```text
+training estimate: 6–9 hours
+VerilogEval generation+scoring: 1–2 hours
+first full result: ~7–11 hours
+```
+
+If Qwen3.5-9B is slower or memory pressure causes offload, it may take longer.
+
+## Evaluation
+
+Initial automatic evaluation:
+
+```text
+VerilogEval v2 direct, n=1, temperature 0
+```
+
+Compare against v30b:
+
+```text
+v30b VerilogEval: compile 141/156, pass 71/156
+```
+
+Follow-up if promising:
+
+```text
+paper-style full
+robust
+alt
+```
+
+## Acceptance target
+
+Keep v32 as useful only if it is at least competitive with v30b:
+
+```text
+VerilogEval pass >= 71 preferred
+compile not materially worse than 141/156
+external checks should not regress if follow-up eval is run
+```
+
+If v32 underperforms, the migration lesson is still valuable: Qwen3.5 base may need different prompt formatting, lower LR, more/less anchoring, or a Qwen3.5-Coder-specific base when one is available.

qwen35-9b-verilog-migration-general-lora/results/verilogeval_direct_summary.json

@@ -0,0 +1,7 @@
+{
+  "total": 156,
+  "compile": 71,
+  "passed": 60,
+  "compile_pct": 45.51282051282051,
+  "pass_pct": 38.46153846153846
+}

qwen35-9b-verilog-migration-general-lora/scripts/build_v32_qwen35_migration_dataset.py

@@ -0,0 +1,196 @@
+#!/usr/bin/env python3
+"""Build v32 Qwen3.5 migration dataset with strong v9/v30b anchoring.
+
+Purpose: migrate behavior to a new base (Qwen3.5) while preserving the local
+optimum discovered by v9/v30b. This is data/behavior migration, not LoRA weight
+migration.
+"""
+import argparse
+import hashlib
+import json
+import random
+from pathlib import Path
+
+
+def read_jsonl(path):
+    path = Path(path)
+    if not path.exists():
+        return []
+    rows = []
+    for line in path.read_text(errors="ignore").splitlines():
+        if line.strip():
+            try:
+                rows.append(json.loads(line))
+            except json.JSONDecodeError:
+                pass
+    return rows
+
+
+def h(row):
+    return hashlib.sha256((row.get("spec", "") + "\n---\n" + row.get("code", "")).encode()).hexdigest()
+
+
+def norm(row, role, max_chars):
+    spec = (row.get("spec") or row.get("prompt") or "").strip()
+    code = (row.get("code") or row.get("solution") or "").strip()
+    if not spec or not code or "module" not in code.lower() or "endmodule" not in code.lower():
+        return None
+    if row.get("used_reference_as_solution", False):
+        return None
+    if len(spec) + len(code) > max_chars:
+        return None
+    out = dict(row)
+    out.update({
+        "spec": spec,
+        "code": code + "\n",
+        "verified": bool(row.get("verified", True)),
+        "used_reference_as_solution": False,
+        "v32_role": role,
+        "v32_category": row.get("v31_category") or row.get("v30b_category") or row.get("v30_category") or row.get("v29_category"),
+    })
+    return out
+
+
+def prompt_for(repo, problem):
+    for sub in ["dataset_spec-to-rtl", "dataset_code-complete-iccad2023"]:
+        p = Path(repo) / sub / f"{problem}_prompt.txt"
+        if p.exists():
+            return p.read_text(errors="ignore").strip()
+    return ""
+
+
+def build_pass_rows(repo, build, role, max_chars):
+    repo = Path(repo); build = Path(build)
+    rows = []
+    for rec in read_jsonl(build / "direct_summary_results.jsonl"):
+        if not rec.get("passed"):
+            continue
+        prob = rec.get("problem")
+        spec = prompt_for(repo, prob)
+        code_path = build / prob / f"{prob}_sample01.sv"
+        if not spec or not code_path.exists():
+            continue
+        raw = {
+            "spec": spec,
+            "code": code_path.read_text(errors="ignore"),
+            "problem": prob,
+            "benchmark": "VerilogEval_v2",
+            "verified": True,
+            "used_adapter": True,
+        }
+        r = norm(raw, role, max_chars)
+        if r:
+            rows.append(r)
+    return rows
+
+
+def select_by_role(rows, role):
+    return [r for r in rows if r.get("v31_role") == role or r.get("v30b_role") == role or r.get("v30_role") == role]
+
+
+def dedupe(rows):
+    out, seen = [], set()
+    for r in rows:
+        key = h(r)
+        if key in seen:
+            continue
+        seen.add(key)
+        out.append(r)
+    return out
+
+
+def repeat(rows, n, tag):
+    out = []
+    for r in rows:
+        for i in range(n):
+            rr = dict(r)
+            rr["v32_repeat_kind"] = tag
+            rr["v32_repeat_idx"] = i
+            out.append(rr)
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default="data/v32_qwen35_9b_migration/v32_qwen35_migration_train.jsonl")
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--v31-train", default="data/v31_general_external/v31_general_external_train.jsonl")
+    ap.add_argument("--v30b-train", default="data/v30b_delta_distill/v30b_delta_distill_train.jsonl")
+    ap.add_argument("--v9-build", default="/home/pablo/verilog-eval/build-v9-auto-direct-spec-n1")
+    ap.add_argument("--v30b-build", default="/home/pablo/verilog-eval/build-v30b-delta-distilled-direct-spec-n1")
+    ap.add_argument("--max-chars", type=int, default=12000)
+    ap.add_argument("--seed", type=int, default=320)
+    args = ap.parse_args()
+    rng = random.Random(args.seed)
+    out = Path(args.out); out.parent.mkdir(parents=True, exist_ok=True)
+
+    v31 = [norm(r, r.get("v31_role", "v31_general"), args.max_chars) for r in read_jsonl(args.v31_train)]
+    v31 = [r for r in v31 if r]
+    v30b = [norm(r, r.get("v30b_role", "v30b_delta"), args.max_chars) for r in read_jsonl(args.v30b_train)]
+    v30b = [r for r in v30b if r]
+
+    v9_pass = dedupe(build_pass_rows(args.repo, args.v9_build, "v9_pass_anchor", args.max_chars))
+    v30b_pass = dedupe(build_pass_rows(args.repo, args.v30b_build, "v30b_pass_anchor", args.max_chars))
+
+    delta = dedupe(select_by_role(v30b, "delta_v9_fail_v29_pass") + select_by_role(v31, "delta_v9_fail_v29_pass"))
+    external = dedupe(select_by_role(v31, "external_functional_repair") + select_by_role(v31, "external_compile_repair"))
+    clean = dedupe(select_by_role(v31, "clean_verified_retention") + select_by_role(v30b, "clean_verified_retention"))
+    synth = dedupe(select_by_role(v31, "small_verified_synthetic") + select_by_role(v30b, "small_verified_synthetic"))
+    selector = dedupe(select_by_role(v30b, "selector_pass_retention") + select_by_role(v31, "selector_pass_retention"))
+
+    # Qwen3.5 new base needs strong imitation of both anchors: v9 optimum + v30b improvement.
+    rows = []
+    rows += repeat(v9_pass, 14, "v9_anchor")
+    rows += repeat(v30b_pass, 14, "v30b_anchor")
+    rows += repeat(delta, 45, "delta_wins")
+    rows += repeat(selector, 4, "selector_retention")
+    rows += repeat(external, 20, "external_general")
+    rows += repeat(clean, 3, "clean_retention")
+    rows += repeat(synth, 1, "synthetic_small")
+    rng.shuffle(rows)
+    out.write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in rows) + "\n")
+
+    def bycat(rs):
+        d = {}
+        for r in rs:
+            c = r.get("v32_category") or r.get("v31_category") or r.get("v30b_category") or "unknown"
+            d[c] = d.get(c, 0) + 1
+        return d
+
+    summary = {
+        "output": str(out),
+        "rows": len(rows),
+        "unique": {
+            "v9_pass_anchor": len(v9_pass),
+            "v30b_pass_anchor": len(v30b_pass),
+            "delta_wins": len(delta),
+            "selector_retention": len(selector),
+            "external_general": len(external),
+            "clean": len(clean),
+            "synthetic": len(synth),
+        },
+        "repeat_policy": {
+            "v9_pass_anchor": 14,
+            "v30b_pass_anchor": 14,
+            "delta_wins": 45,
+            "selector_retention": 4,
+            "external_general": 20,
+            "clean": 3,
+            "synthetic": 1,
+        },
+        "by_category_unique": {
+            "v9_pass_anchor": bycat(v9_pass),
+            "v30b_pass_anchor": bycat(v30b_pass),
+            "delta_wins": bycat(delta),
+            "external_general": bycat(external),
+            "clean": bycat(clean),
+            "synthetic": bycat(synth),
+        },
+        "notes": "Migration dataset for Qwen3.5: heavy v9/v30b pass anchors plus delta wins and external retention.",
+    }
+    (out.parent / "summary.json").write_text(json.dumps(summary, indent=2))
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-migration-general-lora/scripts/run_v32_qwen35_9b_migration.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# v32: migrate v30b/v31 behavior to official Qwen3.5-9B by training a new LoRA.
+# This is NOT LoRA weight transfer; it retrains on the verified v31 dataset.
+# Qwen3.5 official repos are image-text-to-text, so we use *_any.py loaders.
+
+PY=${PY:-.venv/bin/python}
+if [[ ! -x "$PY" ]]; then PY=python3; fi
+BASE=${BASE:-Qwen/Qwen3.5-9B}
+REPO=${VERILOGEVAL_REPO:-/home/pablo/verilog-eval}
+TRAIN_JSONL=${TRAIN_JSONL:-data/v32_qwen35_9b_migration/v32_qwen35_migration_train.jsonl}
+ADAPTER=${ADAPTER:-adapter_v32_qwen35_9b_verilog_general}
+BUILD=${BUILD:-/home/pablo/verilog-eval/build-v32-qwen35-9b-verilog-general-direct-spec-n1}
+EPOCHS=${EPOCHS:-0.80}
+LR=${LR:-1e-5}
+# L4 safety: Qwen3.5-9B OOMed with static 2048 padding.
+# Trainer now uses dynamic padding; 1536 cap drops only a few rows and gives margin.
+MAX_LENGTH=${MAX_LENGTH:-1536}
+LORA_R=${LORA_R:-32}
+LORA_ALPHA=${LORA_ALPHA:-64}
+
+export PYTORCH_CUDA_ALLOC_CONF=${PYTORCH_CUDA_ALLOC_CONF:-expandable_segments:True}
+mkdir -p logs results data/v32_qwen35_9b_migration
+
+echo "=== v32 Qwen3.5-9B migration config ==="
+echo "base=$BASE"
+echo "train_jsonl=$TRAIN_JSONL"
+echo "adapter=$ADAPTER"
+echo "epochs=$EPOCHS lr=$LR max_length=$MAX_LENGTH r=$LORA_R alpha=$LORA_ALPHA"
+
+echo "=== build v32 Qwen3.5 migration dataset with v9/v30b anchors ==="
+$PY scripts/build_v32_qwen35_migration_dataset.py \
+  --out "$TRAIN_JSONL" \
+  --repo "$REPO" \
+  --v31-train data/v31_general_external/v31_general_external_train.jsonl \
+  --v30b-train data/v30b_delta_distill/v30b_delta_distill_train.jsonl \
+  --v9-build "$REPO/build-v9-auto-direct-spec-n1" \
+  --v30b-build "$REPO/build-v30b-delta-distilled-direct-spec-n1" \
+  2>&1 | tee logs/v32_qwen35_build_dataset.log
+
+echo "=== train v32 Qwen3.5-9B Verilog adapter ==="
+if [[ -d "$ADAPTER" && "${FORCE_TRAIN:-0}" != "1" ]]; then
+  echo "skip train: $ADAPTER exists"
+else
+  $PY src/train_correction_adapter_any.py \
+    --base "$BASE" \
+    --train-jsonl "$TRAIN_JSONL" \
+    --output "$ADAPTER" \
+    --epochs "$EPOCHS" --lr "$LR" --max-length "$MAX_LENGTH" \
+    --batch-size 1 --grad-accum 4 --warmup-steps 40 \
+    --lora-r "$LORA_R" --lora-alpha "$LORA_ALPHA" \
+    --drop-overlength \
+    2>&1 | tee logs/train_v32_qwen35_9b_verilog_general.log
+fi
+
+echo "=== eval v32 Qwen3.5 VerilogEval direct ==="
+if [[ -f "$BUILD/direct_summary.json" && "${FORCE_EVAL:-0}" != "1" ]]; then
+  cat "$BUILD/direct_summary.json"
+else
+  $PY scripts/run_verilogeval_v2_local_any.py \
+    --base "$BASE" \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    --adapter "/home/pablo/verilog-llm/$ADAPTER" \
+    --temperature 0 --top_p 0.01 --max_new_tokens 1536 \
+    2>&1 | tee logs/eval_v32_qwen35_verilogeval_generate.log
+
+  $PY scripts/score_direct_build.py \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    2>&1 | tee logs/eval_v32_qwen35_verilogeval_score.log
+fi
+
+echo "=== v32 qwen35 done ==="
+cat "$BUILD/direct_summary.json"

qwen35-9b-verilog-migration-general-lora/scripts/train_correction_adapter_any.py

@@ -0,0 +1,173 @@
+#!/usr/bin/env python3
+"""Train LoRA/QLoRA Verilog adapter on causal or conditional-generation HF models.
+
+This is a compatibility variant for bases such as Qwen3.5 VLM/text models that
+are not loadable via AutoModelForCausalLM. For normal coder models,
+`src/train_correction_adapter.py` remains the default.
+"""
+import argparse
+import json
+from pathlib import Path
+
+import torch
+from datasets import Dataset
+from peft import LoraConfig, PeftModel, get_peft_model, prepare_model_for_kbit_training
+from transformers import (
+    AutoConfig,
+    AutoModelForCausalLM,
+    AutoModelForImageTextToText,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+    DataCollatorForLanguageModeling,
+    Trainer,
+    TrainingArguments,
+)
+
+SYSTEM = "You are an expert Verilog designer. Return only complete synthesizable Verilog code. No explanation."
+
+
+def load_rows(path):
+    rows = []
+    with open(path) as f:
+        for line in f:
+            if line.strip():
+                r = json.loads(line)
+                if "spec" not in r or "code" not in r:
+                    raise ValueError(f"JSONL row missing spec/code: {r.keys()}")
+                rows.append(r)
+    return rows
+
+
+def format_rows(rows, tokenizer):
+    texts = []
+    for r in rows:
+        if "messages" in r:
+            msg = r["messages"]
+        else:
+            assistant = r.get("assistant")
+            if not assistant:
+                assistant = "```verilog\n" + r["code"].strip() + "\n```"
+            msg = [
+                {"role": "system", "content": r.get("system", SYSTEM)},
+                {"role": "user", "content": r.get("user", r["spec"])},
+                {"role": "assistant", "content": assistant},
+            ]
+        texts.append(tokenizer.apply_chat_template(msg, tokenize=False, add_generation_prompt=False))
+    return texts
+
+
+def build_dataset(rows, tokenizer, max_length, drop_overlength):
+    texts = format_rows(rows, tokenizer)
+    if drop_overlength:
+        kept, dropped = [], 0
+        for text in texts:
+            if len(tokenizer(text)["input_ids"]) <= max_length:
+                kept.append(text)
+            else:
+                dropped += 1
+        if not kept:
+            raise ValueError(f"all rows exceed max_length={max_length}; dropped={dropped}")
+        print(f"drop_overlength kept={len(kept)} dropped={dropped} max_length={max_length}", flush=True)
+        texts = kept
+    ds = Dataset.from_dict({"text": texts})
+
+    def enc(batch):
+        # Dynamic padding in the collator avoids padding every row to max_length.
+        # This is important for Qwen3.5-9B on L4: static 2048 padding OOMs.
+        out = tokenizer(batch["text"], max_length=max_length, truncation=not drop_overlength, padding=False)
+        return out
+
+    return ds.map(enc, batched=True, remove_columns=["text"])
+
+
+def model_loader_for(base):
+    cfg = AutoConfig.from_pretrained(base, trust_remote_code=True)
+    model_type = getattr(cfg, "model_type", "")
+    arch = " ".join(getattr(cfg, "architectures", []) or [])
+    if model_type in {"qwen3_5"} or "ForConditionalGeneration" in arch:
+        return AutoModelForImageTextToText
+    return AutoModelForCausalLM
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--base", required=True)
+    ap.add_argument("--adapter-in", default=None)
+    ap.add_argument("--train-jsonl", required=True)
+    ap.add_argument("--output", required=True)
+    ap.add_argument("--epochs", type=float, default=0.8)
+    ap.add_argument("--lr", type=float, default=1e-5)
+    ap.add_argument("--max-length", type=int, default=2048)
+    ap.add_argument("--batch-size", type=int, default=1)
+    ap.add_argument("--grad-accum", type=int, default=4)
+    ap.add_argument("--warmup-steps", type=int, default=40)
+    ap.add_argument("--lora-r", type=int, default=32)
+    ap.add_argument("--lora-alpha", type=int, default=64)
+    ap.add_argument("--logging-steps", type=int, default=2)
+    ap.add_argument("--save-steps", type=int, default=0)
+    ap.add_argument("--drop-overlength", action="store_true")
+    args = ap.parse_args()
+
+    rows = load_rows(args.train_jsonl)
+    tok_source = args.adapter_in or args.base
+    tokenizer = AutoTokenizer.from_pretrained(tok_source, trust_remote_code=True)
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    ds = build_dataset(rows, tokenizer, args.max_length, args.drop_overlength)
+
+    bnb = BitsAndBytesConfig(
+        load_in_4bit=True,
+        bnb_4bit_quant_type="nf4",
+        bnb_4bit_compute_dtype=torch.float16,
+        bnb_4bit_use_double_quant=True,
+    )
+    loader = model_loader_for(args.base)
+    model = loader.from_pretrained(
+        args.base,
+        quantization_config=bnb,
+        device_map="auto",
+        trust_remote_code=True,
+    )
+    model = prepare_model_for_kbit_training(model)
+
+    if args.adapter_in:
+        model = PeftModel.from_pretrained(model, args.adapter_in, is_trainable=True)
+    else:
+        cfg = LoraConfig(
+            r=args.lora_r,
+            lora_alpha=args.lora_alpha,
+            lora_dropout=0.05,
+            bias="none",
+            task_type="CAUSAL_LM",
+            target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
+        )
+        model = get_peft_model(model, cfg)
+
+    save_strategy = "no" if args.save_steps == 0 else "steps"
+    train_args = TrainingArguments(
+        output_dir=str(Path(args.output).with_name(Path(args.output).name + "_ckpt")),
+        num_train_epochs=args.epochs,
+        per_device_train_batch_size=args.batch_size,
+        gradient_accumulation_steps=args.grad_accum,
+        learning_rate=args.lr,
+        warmup_steps=args.warmup_steps,
+        logging_steps=args.logging_steps,
+        save_strategy=save_strategy,
+        save_steps=args.save_steps if args.save_steps else 500,
+        report_to="none",
+        fp16=True,
+    )
+    trainer = Trainer(
+        model=model,
+        args=train_args,
+        train_dataset=ds,
+        data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
+    )
+    trainer.train()
+    model.save_pretrained(args.output)
+    tokenizer.save_pretrained(args.output)
+    print(f"saved {args.output}")
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-migration-general-lora/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

qwen35-9b-verilog-structured-repair-lora/.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

qwen35-9b-verilog-structured-repair-lora/README.md

@@ -0,0 +1,66 @@
+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- verilog
+- rtl
+- qwen3.5
+- lora
+- peft
+- code-generation
+- repair
+---
+
+# Verilog Qwen3.5 9B v34 Manual Structured Repair LoRA
+
+PEFT LoRA adapter for `Qwen/Qwen3.5-9B`, initialized from v33 and further trained on hand-written/manual repairs for v33 max4096 failures.
+
+## Important data note
+
+This adapter was trained with manual repair targets only for the targeted v33 failure repairs. No converted VerilogEval `RefModule` reference-solution code was used as repair target code.
+
+## Intended format
+
+```text
+Thinking:
+[BEGIN]
+module TopModule(...);
+  ...
+endmodule
+[DONE]
+```
+
+Use code inside `[BEGIN]` / `[DONE]` as final artifact.
+
+## Evaluation in this project
+
+Local VerilogEval v2 spec-to-RTL direct single-adapter eval, no retry, no selector, no compiler feedback before final answer:
+
+```text
+v34 manual max2048: compile 103/156, pass 77/156 = 49.4%
+```
+
+A max4096 eval is being run after publication request to test whether max token budget closes the gap with v33 max4096.
+
+Interpretation: v34 manual is experimental. v33 remains the best confirmed clean single-adapter checkpoint until the max4096 v34 result is known.
+
+## Loading
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base = "Qwen/Qwen3.5-9B"
+adapter = "Pablo-Flores-Mollinedo/verilog-qwen3.5-9b-v34-manual-structured-repair-lora"
+
+tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
+model = PeftModel.from_pretrained(model, adapter)
+```
+
+## Notes
+
+- Adapter only; requires the base model license/weights.
+- Generated RTL should be compiled and simulated before use.
+- Benchmark scores are finite-testbench simulation results, not formal proof.

qwen35-9b-verilog-structured-repair-lora/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen35-9b-verilog-structured-repair-lora/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

qwen35-9b-verilog-structured-repair-lora/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

qwen35-9b-verilog-thinking-reinforced-lora/README.md

@@ -0,0 +1,179 @@
+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- verilog
+- rtl
+- code-generation
+- qwen3.5
+- qwen
+- lora
+- peft
+- qlora
+- hardware
+- verilog-eval
+license: apache-2.0
+---
+
+# Verilog Qwen3.5-9B v33 Thinking-Reinforced LoRA
+
+`adapter_v33_qwen35_thinking_compile_functional_reinforced` is a standard PEFT LoRA adapter for `Qwen/Qwen3.5-9B`, focused on Verilog RTL generation.
+
+It fine-tunes from the v32 Qwen3.5 adapter, keeps brief reasoning/thinking behavior, and reinforces compile + functional correctness using detailed v32 failure analysis plus verified v9/v30b/v29 repairs.
+
+## Important caveat
+
+This is **not a clean zero-shot VerilogEval leaderboard model**. The dataset includes benchmark-targeted verified outputs and repair anchors from earlier adapters/pipelines. Treat the VerilogEval score as an experiment result, not as a contamination-free leaderboard claim.
+
+LoRA weights from Qwen2.5-Coder were not transferred. This adapter was trained directly on Qwen3.5-9B, then continued from the v32 Qwen3.5 LoRA.
+
+## Results
+
+### VerilogEval v2 direct, spec-to-RTL, n=1, temperature 0
+
+| Model / system | Compile | Functional pass |
+|---|---:|---:|
+| v9 prior Qwen2.5-Coder single adapter | — | 67/156 |
+| v30b prior best Qwen2.5-Coder single adapter | 141/156 | 71/156 |
+| v32 Qwen3.5 migration | 71/156 | 60/156 |
+| v29 multi-adapter verifier selector | 150/156 | 84/156 |
+| **v33 Qwen3.5 thinking-reinforced single adapter** | **101/156** | **76/156** |
+
+v33 is the first Qwen3.5 single adapter in this project to beat v30b on VerilogEval direct functional pass:
+
+```text
+v32 -> v33: +30 compile, +16 pass
+v30b -> v33: -40 compile, +5 pass
+```
+
+Compile rate is still lower than v30b, but v33 has more functional wins and stronger reasoning diversity.
+
+## What changed from v32
+
+v32 often failed because it reasoned at length but did not reliably finish with clean compileable Verilog. v33 keeps thinking, but trains a structured final-answer pattern:
+
+```text
+Thinking:
+- exact interface checks
+- category-specific pitfalls
+- previous failure-specific warning
+- final sanity check
+
+[BEGIN]
+<one complete synthesizable TopModule>
+[DONE]
+```
+
+## Failure analysis used for training
+
+v32 failure analysis summary:
+
+```text
+pass:                       60
+no_module_or_extracted_junk:60
+syntax_compile:             14
+functional_mismatch:        11
+duplicate_module_or_decl:    6
+compile_error_other:         4
+undeclared_or_bad_identifier:1
+```
+
+Category pass/compile showed especially weak seq/FSM/K-map compile stability, so v33 reinforces those with verified repairs and anchors.
+
+## Training data mix
+
+Dataset builder: `scripts/build_v33_qwen35_thinking_reinforce_dataset.py`
+
+Unique source counts:
+
+- 32 v32 failed-repair rows using verified v30b/v9/v29 solutions.
+- 14 Qwen3.5 unique-win retention rows.
+- 71 v30b pass anchors.
+- 67 v9 pass anchors.
+- 84 v29 selector pass anchors.
+- 35 external/general rows.
+- 493 clean verified rows.
+- 316 synthetic verified rows.
+
+Default repeat weights:
+
+```text
+v32 failed repair:        36x
+v32 unique wins:          24x
+v30b pass anchor:          8x
+v9 pass anchor:            6x
+v29 selector anchor:       3x
+external verified:         8x
+clean verified:            2x
+synthetic verified:        1x
+```
+
+Training used `--drop-overlength`; overlength rows were dropped, not truncated.
+
+## Training hyperparameters
+
+```text
+base model: Qwen/Qwen3.5-9B
+base adapter: adapter_v32_qwen35_9b_verilog_general
+method: QLoRA/LoRA continuation
+LoRA r: 32
+LoRA alpha: 64
+learning rate: 8e-6
+epochs: 0.50
+max length: 1536
+batch size: 1
+grad accum: 4
+warmup steps: 30
+```
+
+## Usage
+
+Qwen3.5 uses a conditional-generation loader in the current Transformers stack.
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForImageTextToText, BitsAndBytesConfig
+from peft import PeftModel
+
+base = "Qwen/Qwen3.5-9B"
+adapter = "Pablo-Flores-Mollinedo/verilog-qwen3.5-9b-v33-thinking-reinforced-lora"
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
+model = AutoModelForImageTextToText.from_pretrained(
+    base,
+    quantization_config=bnb,
+    device_map="auto",
+    trust_remote_code=True,
+)
+model = PeftModel.from_pretrained(model, adapter)
+model.eval()
+
+prompt = "Write module TopModule(input a, input b, output out); out should be a & b."
+messages = [
+    {"role": "system", "content": "You are a Verilog RTL designer. You may reason briefly, then put only final compileable Verilog code inside [BEGIN]/[DONE]."},
+    {"role": "user", "content": prompt},
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**inputs, max_new_tokens=1024, do_sample=False, pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+## Related artifacts
+
+- v32 Qwen3.5 migration: earlier checkpoint, 60/156 pass.
+- v30b Qwen2.5-Coder LoRA: prior best single adapter, 71/156 pass.
+- v29 verifier selector: best practical pipeline, 84/156 pass.
+
+## Intended use
+
+Research and experimentation with Verilog RTL generation. Always compile, simulate, lint, and review generated RTL before use.

qwen35-9b-verilog-thinking-reinforced-lora/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "down_proj",
+    "gate_proj",
+    "up_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

qwen35-9b-verilog-thinking-reinforced-lora/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

qwen35-9b-verilog-thinking-reinforced-lora/data/dataset_summary.json

@@ -0,0 +1,39 @@
+{
+  "output": "data/v33_qwen35_thinking_reinforce/v33_qwen35_thinking_reinforce_train.jsonl",
+  "rows": 4292,
+  "unique": {
+    "v32_failed_repair": 32,
+    "v32_unique_win_retention": 14,
+    "external": 35,
+    "clean": 493,
+    "synthetic": 316
+  },
+  "anchor_unique_repeat": [
+    [
+      "v30b_pass_anchor",
+      71,
+      8
+    ],
+    [
+      "v9_pass_anchor",
+      67,
+      6
+    ],
+    [
+      "v29_selector_pass_anchor",
+      84,
+      3
+    ]
+  ],
+  "repeat_policy": {
+    "v32_failed_repair": 36,
+    "v32_unique_win_retention": 24,
+    "v30b_pass_anchor": 8,
+    "v9_pass_anchor": 6,
+    "v29_selector_pass_anchor": 3,
+    "external": 8,
+    "clean": 2,
+    "synthetic": 1
+  },
+  "notes": "Thinking preserved. Focus: v32 failures fixed by verified v30b/v9/v29 outputs, plus compile/functional retention and Qwen3.5 unique wins."
+}

qwen35-9b-verilog-thinking-reinforced-lora/documentation/v33_qwen35_thinking_reinforce_plan.md

@@ -0,0 +1,197 @@
+# v33 Qwen3.5 thinking compile/functional reinforcement plan
+
+Goal: keep Qwen3.5 reasoning/thinking behavior, but repair the main v32 weakness: low compile rate and lost easy VerilogEval anchors.
+
+## Starting point
+
+v32 Qwen3.5-9B migration finished:
+
+```text
+compile: 71/156
+pass:    60/156
+```
+
+Comparison:
+
+```text
+v9:   67/156 pass
+v30b: 71/156 pass
+```
+
+v32 is not a better single adapter, but it has useful diversity:
+
+```text
+wins vs v9:   10
+wins vs v30b: 12
+```
+
+The problem is mostly compile/alignment, not lack of reasoning capacity:
+
+```text
+v32 compile: 71/156
+v30b compile: 141/156
+```
+
+## User direction
+
+Do **not** remove thinking. Instead:
+
+- preserve thinking/reasoning,
+- inspect each failed example in detail,
+- reinforce compile correctness,
+- reinforce functional correctness,
+- use curated and varied verified examples.
+
+## New files
+
+```text
+scripts/analyze_v32_qwen35_failures.py
+scripts/build_v33_qwen35_thinking_reinforce_dataset.py
+scripts/run_v33_qwen35_thinking_reinforce.sh
+```
+
+Also updated:
+
+```text
+src/train_correction_adapter_any.py
+scripts/run_verilogeval_v2_local_any.py
+```
+
+## Failure analysis
+
+The analyzer scores every v32 output and writes:
+
+```text
+results/v32_qwen35_failure_analysis/summary.json
+results/v32_qwen35_failure_analysis/failure_rows.jsonl
+results/v32_qwen35_failure_analysis/failure_analysis.md
+```
+
+For each problem it records:
+
+- category,
+- compile/pass,
+- failure class,
+- first compile/sim error,
+- module/endmodule counts,
+- response length,
+- thinking/tag indicators,
+- whether v9/v30b/v29 passed,
+- best verified source for repair.
+
+Failure classes include:
+
+```text
+no_module_or_extracted_junk
+truncated_missing_endmodule
+syntax_compile
+undeclared_or_bad_identifier
+declaration_or_assignment_error
+compile_error_other
+functional_mismatch
+```
+
+## Training strategy
+
+v33 fine-tunes from v32:
+
+```text
+base:       Qwen/Qwen3.5-9B
+adapter-in: adapter_v32_qwen35_9b_verilog_general
+output:     adapter_v33_qwen35_thinking_compile_functional_reinforced
+```
+
+Dataset uses thinking-form assistant targets:
+
+```text
+Thinking:
+- short implementation checks
+- category-specific pitfalls
+- previous failure-specific warning
+- final sanity check
+
+[BEGIN]
+<verified Verilog>
+[DONE]
+```
+
+This is not anti-thinking. It is structured thinking with clean final code.
+
+## Reinforcement sources
+
+1. **v32 failed repair rows**
+   - For each v32 failed problem, use strongest verified generated solution available:
+     1. v30b pass output,
+     2. else v9 pass output,
+     3. else v29 selector pass output.
+   - High repeat weight.
+
+2. **v32 unique wins**
+   - Preserve Qwen3.5 problems that v9/v30b missed.
+   - Prevent regression to v30b-only behavior.
+
+3. **v30b/v9/v29 anchors**
+   - Keep local optimum and selector improvements.
+
+4. **External/clean/synthetic verified rows**
+   - Keep diversity and reduce VerilogEval-only overfit.
+
+## Default repeat policy
+
+```text
+v32_failed_repair:        36x
+v32_unique_win_retention: 24x
+v30b_pass_anchor:          8x
+v9_pass_anchor:            6x
+v29_selector_pass_anchor:  3x
+external verified:         8x
+clean verified:            2x
+synthetic verified:        1x
+```
+
+## Training defaults
+
+```text
+epochs: 0.50
+lr: 8e-6
+max_length: 1536
+batch: 1
+grad_accum: 4
+warmup_steps: 30
+```
+
+Expected training time on L4: about 1.5–2.5 hours depending on dropped rows.
+
+## Evaluation defaults
+
+```text
+allow-thinking: true
+max_new_tokens: 1024
+temperature: 0
+top_p: 0.01
+```
+
+Prompt allows brief reasoning, but requires final Verilog only inside `[BEGIN]/[DONE]`.
+
+## Acceptance target
+
+Useful v33 if:
+
+```text
+compile improves materially over v32 71/156
+pass improves over v32 60/156
+keeps some v32 unique wins
+```
+
+Strong target:
+
+```text
+pass >= v9 67/156
+compile much closer to v30b
+```
+
+Best-case target:
+
+```text
+pass >= v30b 71/156
+```

qwen35-9b-verilog-thinking-reinforced-lora/results/v32_failure_analysis/failure_analysis.md

@@ -0,0 +1,223 @@
+# v32 Qwen3.5 failure analysis
+
+## Summary
+
+```json
+{
+  "total": 156,
+  "compile": 71,
+  "passed": 60,
+  "failure_class_counts": {
+    "pass": 60,
+    "syntax_compile": 14,
+    "no_module_or_extracted_junk": 60,
+    "functional_mismatch": 11,
+    "compile_error_other": 4,
+    "duplicate_module_or_decl": 6,
+    "undeclared_or_bad_identifier": 1
+  },
+  "category_counts": {
+    "comb_vector": 45,
+    "arith_datapath": 17,
+    "counter_timer": 15,
+    "seq_shift": 31,
+    "case_kmap": 10,
+    "fsm_protocol": 38
+  },
+  "category_pass": {
+    "comb_vector": 28,
+    "counter_timer": 8,
+    "seq_shift": 6,
+    "case_kmap": 1,
+    "fsm_protocol": 5,
+    "arith_datapath": 12,
+    "other": 0
+  },
+  "category_compile": {
+    "comb_vector": 32,
+    "counter_timer": 8,
+    "seq_shift": 9,
+    "case_kmap": 1,
+    "fsm_protocol": 9,
+    "arith_datapath": 12,
+    "other": 0
+  },
+  "fix_source_counts": {
+    "v30b": 23,
+    "none": 64,
+    "v29_selector": 5,
+    "v9": 4
+  },
+  "wins_vs_v9": [
+    "Prob028_m2014_q4a",
+    "Prob045_edgedetect2",
+    "Prob047_dff8ar",
+    "Prob079_fsm3onehot",
+    "Prob097_mux9to1v",
+    "Prob114_bugs_case",
+    "Prob119_fsm3",
+    "Prob127_lemmings1",
+    "Prob130_circuit5",
+    "Prob132_always_if2"
+  ],
+  "wins_vs_v30b": [
+    "Prob019_m2014_q4f",
+    "Prob033_ece241_2014_q1c",
+    "Prob045_edgedetect2",
+    "Prob047_dff8ar",
+    "Prob069_truthtable1",
+    "Prob079_fsm3onehot",
+    "Prob114_bugs_case",
+    "Prob118_history_shift",
+    "Prob119_fsm3",
+    "Prob127_lemmings1",
+    "Prob130_circuit5",
+    "Prob132_always_if2"
+  ],
+  "losses_vs_v9": [
+    "Prob003_step_one",
+    "Prob004_vector2",
+    "Prob006_vectorr",
+    "Prob023_vector100r",
+    "Prob026_alwaysblock1",
+    "Prob039_always_if",
+    "Prob076_always_case",
+    "Prob095_review2015_fsmshift",
+    "Prob101_circuit4",
+    "Prob102_circuit3",
+    "Prob103_circuit2",
+    "Prob105_rotate100",
+    "Prob106_always_nolatches",
+    "Prob107_fsm1s",
+    "Prob109_fsm1",
+    "Prob126_circuit6",
+    "Prob128_fsm_ps2"
+  ],
+  "losses_vs_v30b": [
+    "Prob003_step_one",
+    "Prob004_vector2",
+    "Prob006_vectorr",
+    "Prob018_mux256to1",
+    "Prob023_vector100r",
+    "Prob026_alwaysblock1",
+    "Prob030_popcount255",
+    "Prob039_always_if",
+    "Prob042_vector4",
+    "Prob054_edgedetect",
+    "Prob056_ece241_2013_q7",
+    "Prob058_alwaysblock2",
+    "Prob060_m2014_q4k",
+    "Prob063_review2015_shiftcount",
+    "Prob075_counter_2bc",
+    "Prob076_always_case",
+    "Prob095_review2015_fsmshift",
+    "Prob103_circuit2",
+    "Prob106_always_nolatches",
+    "Prob109_fsm1",
+    "Prob126_circuit6",
+    "Prob128_fsm_ps2",
+    "Prob148_2013_q2afsm"
+  ]
+}
+```
+
+## Failed problems
+
+| problem | category | class | fix source | v9 | v30b | v29 | first error |
+|---|---|---|---|---:|---:|---:|---|
+| Prob003_step_one | comb_vector | syntax_compile | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob004_vector2 | comb_vector | no_module_or_extracted_junk | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob006_vectorr | comb_vector | functional_mismatch | v30b | 1 | 1 | 1 | Hint: Output 'out' has 203 mismatches. First mismatch occurred at time 20. |
+| Prob009_popcount3 | counter_timer | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob018_mux256to1 | arith_datapath | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob021_mux256to1v | comb_vector | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob023_vector100r | comb_vector | compile_error_other | v30b | 1 | 1 | 1 | dut.sv:2: error: part select in[0:99] is out of order. |
+| Prob026_alwaysblock1 | comb_vector | duplicate_module_or_decl | v30b | 1 | 1 | 1 | dut.sv:3: error: out_alwaysblock is not a valid l-value in tb.top_module1. |
+| Prob029_m2014_q4g | comb_vector | no_module_or_extracted_junk |  | 0 | 0 | 0 | test.sv:6: syntax error |
+| Prob030_popcount255 | counter_timer | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob031_dff | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob034_dff8 | seq_shift | duplicate_module_or_decl |  | 0 | 0 | 0 | dut.sv:2: error: q is not a valid l-value in tb.top_module1. |
+| Prob039_always_if | comb_vector | syntax_compile | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob042_vector4 | arith_datapath | syntax_compile | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob046_dff8p | seq_shift | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob049_m2014_q4b | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob050_kmap1 | case_kmap | no_module_or_extracted_junk | v29_selector | 0 | 0 | 1 | dut.sv:1: syntax error |
+| Prob053_m2014_q4d | seq_shift | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob054_edgedetect | seq_shift | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob056_ece241_2013_q7 | seq_shift | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob057_kmap2 | case_kmap | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob058_alwaysblock2 | seq_shift | duplicate_module_or_decl | v30b | 0 | 1 | 0 | dut.sv:4: error: out_always_comb is not a valid l-value in tb.top_module1. |
+| Prob060_m2014_q4k | seq_shift | syntax_compile | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob062_bugs_mux2 | comb_vector | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'out' has 110 mismatches. First mismatch occurred at time 20. |
+| Prob063_review2015_shiftcount | counter_timer | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob064_vector3 | arith_datapath | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob066_edgecapture | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob068_countbcd | counter_timer | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob070_ece241_2013_q2 | arith_datapath | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob071_always_casez | case_kmap | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob074_ece241_2014_q4 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob075_counter_2bc | counter_timer | no_module_or_extracted_junk | v30b | 0 | 1 | 1 | dut.sv:1: syntax error |
+| Prob076_always_case | comb_vector | duplicate_module_or_decl | v30b | 1 | 1 | 1 | dut.sv:4: error: out is not a valid l-value in tb.top_module1. |
+| Prob078_dualedge | seq_shift | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'q' has 113 mismatches. First mismatch occurred at time 10. |
+| Prob082_lfsr32 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob084_ece241_2013_q12 | seq_shift | compile_error_other |  | 0 | 0 | 0 | dut.sv:10: error: can not select part of scalar: A |
+| Prob085_shift4 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob086_lfsr5 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob088_ece241_2014_q5b | fsm_protocol | no_module_or_extracted_junk | v29_selector | 0 | 0 | 1 | dut.sv:1: syntax error |
+| Prob089_ece241_2014_q5a | fsm_protocol | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'z' has 28 mismatches. First mismatch occurred at time 20. |
+| Prob091_2012_q2b | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob092_gatesv100 | comb_vector | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'out_both' has no mismatches. |
+| Prob093_ece241_2014_q3 | case_kmap | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob094_gatesv | comb_vector | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob095_review2015_fsmshift | fsm_protocol | no_module_or_extracted_junk | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob096_review2015_fsmseq | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob098_circuit7 | seq_shift | no_module_or_extracted_junk | v29_selector | 0 | 0 | 1 | dut.sv:1: syntax error |
+| Prob099_m2014_q6c | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob100_fsm3comb | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob101_circuit4 | comb_vector | functional_mismatch | v9 | 1 | 0 | 1 | Hint: Output 'q' has 33 mismatches. First mismatch occurred at time 40. |
+| Prob102_circuit3 | case_kmap | no_module_or_extracted_junk | v9 | 1 | 0 | 1 | dut.sv:1: syntax error |
+| Prob103_circuit2 | comb_vector | no_module_or_extracted_junk | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob104_mt2015_muxdff | seq_shift | compile_error_other |  | 0 | 0 | 0 | test.sv:76: error: port ``q_in'' is not a port of top_module1. |
+| Prob105_rotate100 | seq_shift | no_module_or_extracted_junk | v9 | 1 | 0 | 1 | dut.sv:1: syntax error |
+| Prob106_always_nolatches | fsm_protocol | syntax_compile | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob107_fsm1s | fsm_protocol | duplicate_module_or_decl | v9 | 1 | 0 | 1 | dut.sv:19: error: out is not a valid l-value in tb.top_module1. |
+| Prob108_rule90 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob109_fsm1 | fsm_protocol | syntax_compile | v30b | 1 | 1 | 1 | dut.sv:1: syntax error |
+| Prob112_always_case2 | case_kmap | syntax_compile |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob113_2012_q1g | case_kmap | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob115_shift18 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob116_m2014_q3 | arith_datapath | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob117_circuit9 | seq_shift | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'q' has 244 mismatches. First mismatch occurred at time 10. |
+| Prob120_fsm3s | fsm_protocol | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'out' has 56 mismatches. First mismatch occurred at time 20. |
+| Prob121_2014_q3bfsm | fsm_protocol | undeclared_or_bad_identifier | v29_selector | 0 | 0 | 1 | dut.sv:13: error: Could not find variable ``y_next'' in ``tb.top_module1'' |
+| Prob122_kmap4 | case_kmap | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob124_rule110 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob125_kmap3 | case_kmap | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob126_circuit6 | comb_vector | duplicate_module_or_decl | v30b | 1 | 1 | 1 | dut.sv:4: error: q is not a valid l-value in tb.top_module1. |
+| Prob128_fsm_ps2 | fsm_protocol | functional_mismatch | v30b | 1 | 1 | 1 | Hint: Output 'done' has 173 mismatches. First mismatch occurred at time 210. |
+| Prob129_ece241_2013_q8 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob131_mt2015_q4 | comb_vector | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob133_2014_q3fsm | fsm_protocol | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'z' has 170 mismatches. First mismatch occurred at time 110. |
+| Prob134_2014_q3c | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob135_m2014_q6b | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob136_m2014_q6 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob137_fsm_serial | fsm_protocol | syntax_compile |  | 0 | 0 | 0 | dut.sv:3: syntax error |
+| Prob138_2012_q2fsm | fsm_protocol | no_module_or_extracted_junk | v29_selector | 0 | 0 | 1 | dut.sv:1: syntax error |
+| Prob139_2013_q2bfsm | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob140_fsm_hdlc | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob141_count_clock | counter_timer | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob142_lemmings2 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob143_fsm_onehot | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob144_conwaylife | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob145_circuit8 | seq_shift | functional_mismatch |  | 0 | 0 | 0 | Hint: Output 'p' has 112 mismatches. First mismatch occurred at time 90. |
+| Prob146_fsm_serialdata | fsm_protocol | syntax_compile |  | 0 | 0 | 0 | dut.sv:3: syntax error |
+| Prob147_circuit10 | seq_shift | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob148_2013_q2afsm | fsm_protocol | no_module_or_extracted_junk | v30b | 0 | 1 | 0 | dut.sv:1: syntax error |
+| Prob149_ece241_2013_q4 | comb_vector | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob150_review2015_fsmonehot | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob151_review2015_fsm | fsm_protocol | compile_error_other |  | 0 | 0 | 0 | 7 error(s) during elaboration. |
+| Prob152_lemmings3 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob153_gshare | counter_timer | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob154_fsm_ps2data | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob155_lemmings4 | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |
+| Prob156_review2015_fancytimer | fsm_protocol | no_module_or_extracted_junk |  | 0 | 0 | 0 | dut.sv:1: syntax error |

qwen35-9b-verilog-thinking-reinforced-lora/results/v32_failure_analysis/failure_rows.jsonl

@@ -0,0 +1,156 @@
+{"problem": "Prob001_zero", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'zero' has no mismatches.", "code_chars": 70, "code_lines": 3, "full_response_chars": 5598, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob002_m2014_q4i", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 71, "code_lines": 3, "full_response_chars": 5719, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob003_step_one", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 25, "code_lines": 2, "full_response_chars": 5748, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob004_vector2", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4060, "code_lines": 56, "full_response_chars": 4106, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob005_notgate", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 72, "code_lines": 3, "full_response_chars": 4642, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob006_vectorr", "category": "comb_vector", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'out' has 203 mismatches. First mismatch occurred at time 20.", "code_chars": 145, "code_lines": 3, "full_response_chars": 4290, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob007_wire", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 79, "code_lines": 3, "full_response_chars": 5739, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob008_m2014_q4h", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 79, "code_lines": 3, "full_response_chars": 5713, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob009_popcount3", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 4241, "code_lines": 53, "full_response_chars": 4318, "has_begin_done": false, "has_thinking": false, "module_count": 6, "endmodule_count": 2, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob010_mt2015_q4a", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'z' has no mismatches.", "code_chars": 82, "code_lines": 3, "full_response_chars": 2783, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob011_norgate", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 93, "code_lines": 3, "full_response_chars": 5597, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob012_xnorgate", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 85, "code_lines": 3, "full_response_chars": 3187, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob013_m2014_q4e", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 93, "code_lines": 3, "full_response_chars": 3851, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob014_andgate", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 80, "code_lines": 3, "full_response_chars": 3201, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob015_vector1", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out_hi' has no mismatches.", "code_chars": 142, "code_lines": 4, "full_response_chars": 2122, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob016_m2014_q4j", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'sum' has no mismatches.", "code_chars": 116, "code_lines": 3, "full_response_chars": 5890, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob017_mux2to1v", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 120, "code_lines": 3, "full_response_chars": 3856, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob018_mux256to1", "category": "arith_datapath", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5828, "code_lines": 24, "full_response_chars": 5893, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob019_m2014_q4f", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 89, "code_lines": 3, "full_response_chars": 1850, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob020_mt2015_eq2", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'z' has no mismatches.", "code_chars": 101, "code_lines": 3, "full_response_chars": 5232, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob021_mux256to1v", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4340, "code_lines": 49, "full_response_chars": 4383, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob022_mux2to1", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 99, "code_lines": 3, "full_response_chars": 3825, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob023_vector100r", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "compile_error_other", "first_error": "dut.sv:2: error: part select in[0:99] is out of order.", "code_chars": 91, "code_lines": 3, "full_response_chars": 4065, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob024_hadd", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'sum' has no mismatches.", "code_chars": 132, "code_lines": 4, "full_response_chars": 5536, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob025_reduction", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'parity' has no mismatches.", "code_chars": 84, "code_lines": 3, "full_response_chars": 2121, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob026_alwaysblock1", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:3: error: out_alwaysblock is not a valid l-value in tb.top_module1.", "code_chars": 169, "code_lines": 4, "full_response_chars": 5776, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob027_fadd", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'cout' has no mismatches.", "code_chars": 169, "code_lines": 4, "full_response_chars": 5478, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob028_m2014_q4a", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 148, "code_lines": 5, "full_response_chars": 5559, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob029_m2014_q4g", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "test.sv:6: syntax error", "code_chars": 1540, "code_lines": 1, "full_response_chars": 1539, "has_begin_done": false, "has_thinking": false, "module_count": 0, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob030_popcount255", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5344, "code_lines": 43, "full_response_chars": 5379, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob031_dff", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5791, "code_lines": 46, "full_response_chars": 5900, "has_begin_done": false, "has_thinking": true, "module_count": 10, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob032_vector0", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'outv' has no mismatches.", "code_chars": 192, "code_lines": 6, "full_response_chars": 4728, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob033_ece241_2014_q1c", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 's' has no mismatches.", "code_chars": 169, "code_lines": 4, "full_response_chars": 1501, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob034_dff8", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:2: error: q is not a valid l-value in tb.top_module1.", "code_chars": 112, "code_lines": 3, "full_response_chars": 5360, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob035_count1to10", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 228, "code_lines": 7, "full_response_chars": 4975, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob036_ringer", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'ringer' has no mismatches.", "code_chars": 173, "code_lines": 4, "full_response_chars": 1530, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob037_review2015_count1k", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 235, "code_lines": 7, "full_response_chars": 4825, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob038_count15", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 170, "code_lines": 6, "full_response_chars": 1288, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob039_always_if", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 25, "code_lines": 2, "full_response_chars": 5352, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob040_count10", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 233, "code_lines": 7, "full_response_chars": 4852, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob041_dff8r", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 178, "code_lines": 6, "full_response_chars": 4149, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob042_vector4", "category": "arith_datapath", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 25, "code_lines": 2, "full_response_chars": 5315, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob043_vector5", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 861, "code_lines": 27, "full_response_chars": 3722, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob044_vectorgates", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out_or_bitwise' has no mismatches.", "code_chars": 297, "code_lines": 6, "full_response_chars": 4484, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob045_edgedetect2", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'anyedge' has no mismatches.", "code_chars": 196, "code_lines": 8, "full_response_chars": 2212, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob046_dff8p", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 25, "code_lines": 2, "full_response_chars": 5295, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob047_dff8ar", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 218, "code_lines": 6, "full_response_chars": 5364, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob048_m2014_q4c", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 162, "code_lines": 7, "full_response_chars": 5233, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob049_m2014_q4b", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 6588, "code_lines": 67, "full_response_chars": 6587, "has_begin_done": false, "has_thinking": true, "module_count": 0, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob050_kmap1", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3488, "code_lines": 84, "full_response_chars": 3534, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v29_selector"}
+{"problem": "Prob051_gates4", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out_and' has no mismatches.", "code_chars": 184, "code_lines": 5, "full_response_chars": 5084, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob052_gates100", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out_and' has no mismatches.", "code_chars": 179, "code_lines": 5, "full_response_chars": 5309, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob053_m2014_q4d", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 2901, "code_lines": 53, "full_response_chars": 2940, "has_begin_done": true, "has_thinking": true, "module_count": 7, "endmodule_count": 3, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob054_edgedetect", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4884, "code_lines": 63, "full_response_chars": 4993, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob055_conditional", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'min' has no mismatches.", "code_chars": 275, "code_lines": 8, "full_response_chars": 4962, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob056_ece241_2013_q7", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4640, "code_lines": 77, "full_response_chars": 5375, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob057_kmap2", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3126, "code_lines": 90, "full_response_chars": 3157, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob058_alwaysblock2", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:4: error: out_always_comb is not a valid l-value in tb.top_module1.", "code_chars": 270, "code_lines": 9, "full_response_chars": 3635, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": "v30b"}
+{"problem": "Prob059_wire4", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'w' has no mismatches.", "code_chars": 167, "code_lines": 6, "full_response_chars": 1025, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob060_m2014_q4k", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 5501, "code_lines": 32, "full_response_chars": 5800, "has_begin_done": false, "has_thinking": true, "module_count": 5, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob061_2014_q4a", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'Q' has no mismatches.", "code_chars": 180, "code_lines": 6, "full_response_chars": 4786, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob062_bugs_mux2", "category": "comb_vector", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'out' has 110 mismatches. First mismatch occurred at time 20.", "code_chars": 212, "code_lines": 8, "full_response_chars": 5091, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob063_review2015_shiftcount", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5015, "code_lines": 11, "full_response_chars": 5147, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob064_vector3", "category": "arith_datapath", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4213, "code_lines": 9, "full_response_chars": 4256, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob065_7420", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'p1y' has no mismatches.", "code_chars": 280, "code_lines": 6, "full_response_chars": 4705, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob066_edgecapture", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 6235, "code_lines": 23, "full_response_chars": 6283, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob067_countslow", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 243, "code_lines": 9, "full_response_chars": 5103, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob068_countbcd", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4404, "code_lines": 63, "full_response_chars": 4515, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob069_truthtable1", "category": "case_kmap", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'f' has no mismatches.", "code_chars": 105, "code_lines": 3, "full_response_chars": 3133, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob070_ece241_2013_q2", "category": "arith_datapath", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3011, "code_lines": 78, "full_response_chars": 3132, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob071_always_casez", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 2725, "code_lines": 53, "full_response_chars": 4457, "has_begin_done": false, "has_thinking": true, "module_count": 5, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob072_thermostat", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'heater' has no mismatches.", "code_chars": 301, "code_lines": 5, "full_response_chars": 1316, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob073_dff16e", "category": "seq_shift", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 341, "code_lines": 9, "full_response_chars": 4557, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob074_ece241_2014_q4", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5440, "code_lines": 73, "full_response_chars": 5501, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob075_counter_2bc", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 6004, "code_lines": 119, "full_response_chars": 6071, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob076_always_case", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:4: error: out is not a valid l-value in tb.top_module1.", "code_chars": 509, "code_lines": 13, "full_response_chars": 4641, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob077_wire_decl", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 218, "code_lines": 7, "full_response_chars": 5320, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob078_dualedge", "category": "seq_shift", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'q' has 113 mismatches. First mismatch occurred at time 10.", "code_chars": 188, "code_lines": 9, "full_response_chars": 5040, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob079_fsm3onehot", "category": "fsm_protocol", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'next_state' has no mismatches.", "code_chars": 561, "code_lines": 12, "full_response_chars": 3901, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob080_timer", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'tc' has no mismatches.", "code_chars": 339, "code_lines": 14, "full_response_chars": 3494, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob081_7458", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'p1y' has no mismatches.", "code_chars": 304, "code_lines": 8, "full_response_chars": 1787, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob082_lfsr32", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3414, "code_lines": 102, "full_response_chars": 3455, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob083_mt2015_q4b", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'z' has no mismatches.", "code_chars": 81, "code_lines": 3, "full_response_chars": 1203, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob084_ece241_2013_q12", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "compile_error_other", "first_error": "dut.sv:10: error: can not select part of scalar: A", "code_chars": 264, "code_lines": 11, "full_response_chars": 2504, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob085_shift4", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4525, "code_lines": 30, "full_response_chars": 4721, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob086_lfsr5", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4522, "code_lines": 55, "full_response_chars": 4639, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob087_gates", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out_and' has no mismatches.", "code_chars": 356, "code_lines": 9, "full_response_chars": 2321, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob088_ece241_2014_q5b", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4157, "code_lines": 92, "full_response_chars": 4273, "has_begin_done": true, "has_thinking": true, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v29_selector"}
+{"problem": "Prob089_ece241_2014_q5a", "category": "fsm_protocol", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'z' has 28 mismatches. First mismatch occurred at time 20.", "code_chars": 850, "code_lines": 31, "full_response_chars": 5518, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob090_circuit1", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 78, "code_lines": 3, "full_response_chars": 698, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob091_2012_q2b", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3555, "code_lines": 89, "full_response_chars": 3672, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob092_gatesv100", "category": "comb_vector", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'out_both' has no mismatches.", "code_chars": 265, "code_lines": 5, "full_response_chars": 3513, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob093_ece241_2014_q3", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4219, "code_lines": 51, "full_response_chars": 4339, "has_begin_done": false, "has_thinking": true, "module_count": 6, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob094_gatesv", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 2863, "code_lines": 34, "full_response_chars": 4407, "has_begin_done": true, "has_thinking": true, "module_count": 3, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob095_review2015_fsmshift", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 6126, "code_lines": 7, "full_response_chars": 6182, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob096_review2015_fsmseq", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5286, "code_lines": 53, "full_response_chars": 5332, "has_begin_done": false, "has_thinking": true, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob097_mux9to1v", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 657, "code_lines": 18, "full_response_chars": 4316, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob098_circuit7", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3915, "code_lines": 87, "full_response_chars": 3953, "has_begin_done": false, "has_thinking": true, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v29_selector"}
+{"problem": "Prob099_m2014_q6c", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4178, "code_lines": 79, "full_response_chars": 4296, "has_begin_done": false, "has_thinking": true, "module_count": 8, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": null}
+{"problem": "Prob100_fsm3comb", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4042, "code_lines": 81, "full_response_chars": 4165, "has_begin_done": true, "has_thinking": true, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob101_circuit4", "category": "comb_vector", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'q' has 33 mismatches. First mismatch occurred at time 40.", "code_chars": 121, "code_lines": 5, "full_response_chars": 1792, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v9"}
+{"problem": "Prob102_circuit3", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 2694, "code_lines": 89, "full_response_chars": 2803, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v9"}
+{"problem": "Prob103_circuit2", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3026, "code_lines": 77, "full_response_chars": 3135, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob104_mt2015_muxdff", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "compile_error_other", "first_error": "test.sv:76: error: port ``q_in'' is not a port of top_module1.", "code_chars": 185, "code_lines": 9, "full_response_chars": 4656, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob105_rotate100", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4677, "code_lines": 19, "full_response_chars": 4708, "has_begin_done": false, "has_thinking": true, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v9"}
+{"problem": "Prob106_always_nolatches", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 4346, "code_lines": 88, "full_response_chars": 5152, "has_begin_done": false, "has_thinking": true, "module_count": 5, "endmodule_count": 2, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob107_fsm1s", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:19: error: out is not a valid l-value in tb.top_module1.", "code_chars": 720, "code_lines": 26, "full_response_chars": 5430, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v9"}
+{"problem": "Prob108_rule90", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3949, "code_lines": 69, "full_response_chars": 4066, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob109_fsm1", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 5009, "code_lines": 88, "full_response_chars": 5578, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob110_fsm2", "category": "fsm_protocol", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 400, "code_lines": 16, "full_response_chars": 1978, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob111_fsm2s", "category": "fsm_protocol", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 273, "code_lines": 11, "full_response_chars": 2113, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob112_always_case2", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:1: syntax error", "code_chars": 1926, "code_lines": 21, "full_response_chars": 4833, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob113_2012_q1g", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 2829, "code_lines": 97, "full_response_chars": 2860, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob114_bugs_case", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 724, "code_lines": 17, "full_response_chars": 2390, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob115_shift18", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4581, "code_lines": 20, "full_response_chars": 4613, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob116_m2014_q3", "category": "arith_datapath", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4235, "code_lines": 52, "full_response_chars": 4344, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob117_circuit9", "category": "seq_shift", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'q' has 244 mismatches. First mismatch occurred at time 10.", "code_chars": 135, "code_lines": 5, "full_response_chars": 3471, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob118_history_shift", "category": "counter_timer", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'predict_history' has no mismatches.", "code_chars": 564, "code_lines": 11, "full_response_chars": 3044, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob119_fsm3", "category": "fsm_protocol", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 836, "code_lines": 22, "full_response_chars": 4936, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob120_fsm3s", "category": "fsm_protocol", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'out' has 56 mismatches. First mismatch occurred at time 20.", "code_chars": 747, "code_lines": 22, "full_response_chars": 4679, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob121_2014_q3bfsm", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "undeclared_or_bad_identifier", "first_error": "dut.sv:13: error: Could not find variable ``y_next'' in ``tb.top_module1''", "code_chars": 1008, "code_lines": 31, "full_response_chars": 4159, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v29_selector"}
+{"problem": "Prob122_kmap4", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 2745, "code_lines": 95, "full_response_chars": 2791, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob123_bugs_addsubz", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'out' has no mismatches.", "code_chars": 369, "code_lines": 21, "full_response_chars": 4770, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob124_rule110", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3587, "code_lines": 80, "full_response_chars": 3704, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob125_kmap3", "category": "case_kmap", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3373, "code_lines": 94, "full_response_chars": 3405, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob126_circuit6", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "duplicate_module_or_decl", "first_error": "dut.sv:4: error: q is not a valid l-value in tb.top_module1.", "code_chars": 377, "code_lines": 14, "full_response_chars": 2462, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob127_lemmings1", "category": "fsm_protocol", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'walk_left' has no mismatches.", "code_chars": 404, "code_lines": 10, "full_response_chars": 2768, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob128_fsm_ps2", "category": "fsm_protocol", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'done' has 173 mismatches. First mismatch occurred at time 210.", "code_chars": 864, "code_lines": 31, "full_response_chars": 3318, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": true}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v30b"}
+{"problem": "Prob129_ece241_2013_q8", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5041, "code_lines": 75, "full_response_chars": 5072, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob130_circuit5", "category": "comb_vector", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'q' has no mismatches.", "code_chars": 318, "code_lines": 11, "full_response_chars": 2510, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": null}
+{"problem": "Prob131_mt2015_q4", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4199, "code_lines": 123, "full_response_chars": 4285, "has_begin_done": false, "has_thinking": true, "module_count": 6, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": null}
+{"problem": "Prob132_always_if2", "category": "arith_datapath", "compile": true, "passed": true, "failure_class": "pass", "first_error": "Hint: Output 'shut_off_computer' has no mismatches.", "code_chars": 463, "code_lines": 23, "full_response_chars": 2285, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob133_2014_q3fsm", "category": "fsm_protocol", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'z' has 170 mismatches. First mismatch occurred at time 110.", "code_chars": 1118, "code_lines": 38, "full_response_chars": 4989, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob134_2014_q3c", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4903, "code_lines": 55, "full_response_chars": 5012, "has_begin_done": true, "has_thinking": true, "module_count": 21, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob135_m2014_q6b", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3848, "code_lines": 88, "full_response_chars": 3909, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": null}
+{"problem": "Prob136_m2014_q6", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3484, "code_lines": 135, "full_response_chars": 3559, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob137_fsm_serial", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:3: syntax error", "code_chars": 1315, "code_lines": 40, "full_response_chars": 4612, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob138_2012_q2fsm", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4284, "code_lines": 97, "full_response_chars": 4393, "has_begin_done": true, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": true}}, "fix_source": "v29_selector"}
+{"problem": "Prob139_2013_q2bfsm", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4612, "code_lines": 106, "full_response_chars": 4653, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob140_fsm_hdlc", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4296, "code_lines": 116, "full_response_chars": 4343, "has_begin_done": false, "has_thinking": false, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob141_count_clock", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4513, "code_lines": 9, "full_response_chars": 4544, "has_begin_done": false, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob142_lemmings2", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 6973, "code_lines": 39, "full_response_chars": 7004, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob143_fsm_onehot", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5356, "code_lines": 56, "full_response_chars": 5473, "has_begin_done": false, "has_thinking": true, "module_count": 4, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob144_conwaylife", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4485, "code_lines": 70, "full_response_chars": 4484, "has_begin_done": true, "has_thinking": true, "module_count": 0, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob145_circuit8", "category": "seq_shift", "compile": true, "passed": false, "failure_class": "functional_mismatch", "first_error": "Hint: Output 'p' has 112 mismatches. First mismatch occurred at time 90.", "code_chars": 193, "code_lines": 8, "full_response_chars": 3328, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob146_fsm_serialdata", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "syntax_compile", "first_error": "dut.sv:3: syntax error", "code_chars": 2349, "code_lines": 69, "full_response_chars": 5169, "has_begin_done": false, "has_thinking": false, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob147_circuit10", "category": "seq_shift", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3149, "code_lines": 79, "full_response_chars": 3258, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob148_2013_q2afsm", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5130, "code_lines": 68, "full_response_chars": 5247, "has_begin_done": true, "has_thinking": true, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": true}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": "v30b"}
+{"problem": "Prob149_ece241_2013_q4", "category": "comb_vector", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4854, "code_lines": 60, "full_response_chars": 4971, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob150_review2015_fsmonehot", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3238, "code_lines": 75, "full_response_chars": 3349, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob151_review2015_fsm", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "compile_error_other", "first_error": "7 error(s) during elaboration.", "code_chars": 947, "code_lines": 26, "full_response_chars": 5196, "has_begin_done": false, "has_thinking": true, "module_count": 1, "endmodule_count": 1, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": null}
+{"problem": "Prob152_lemmings3", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4965, "code_lines": 140, "full_response_chars": 5012, "has_begin_done": false, "has_thinking": false, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob153_gshare", "category": "counter_timer", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 4826, "code_lines": 169, "full_response_chars": 4889, "has_begin_done": false, "has_thinking": false, "module_count": 3, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob154_fsm_ps2data", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 3540, "code_lines": 65, "full_response_chars": 3657, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": true, "passed": false}, "v30b": {"compile": true, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob155_lemmings4", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5006, "code_lines": 151, "full_response_chars": 5053, "has_begin_done": false, "has_thinking": false, "module_count": 2, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": true, "passed": false}}, "fix_source": null}
+{"problem": "Prob156_review2015_fancytimer", "category": "fsm_protocol", "compile": false, "passed": false, "failure_class": "no_module_or_extracted_junk", "first_error": "dut.sv:1: syntax error", "code_chars": 5021, "code_lines": 90, "full_response_chars": 5138, "has_begin_done": true, "has_thinking": true, "module_count": 1, "endmodule_count": 0, "bases": {"v9": {"compile": false, "passed": false}, "v30b": {"compile": false, "passed": false}, "v29_selector": {"compile": false, "passed": false}}, "fix_source": null}

qwen35-9b-verilog-thinking-reinforced-lora/results/v32_failure_analysis/summary.json

@@ -0,0 +1,116 @@
+{
+  "total": 156,
+  "compile": 71,
+  "passed": 60,
+  "failure_class_counts": {
+    "pass": 60,
+    "syntax_compile": 14,
+    "no_module_or_extracted_junk": 60,
+    "functional_mismatch": 11,
+    "compile_error_other": 4,
+    "duplicate_module_or_decl": 6,
+    "undeclared_or_bad_identifier": 1
+  },
+  "category_counts": {
+    "comb_vector": 45,
+    "arith_datapath": 17,
+    "counter_timer": 15,
+    "seq_shift": 31,
+    "case_kmap": 10,
+    "fsm_protocol": 38
+  },
+  "category_pass": {
+    "comb_vector": 28,
+    "counter_timer": 8,
+    "seq_shift": 6,
+    "case_kmap": 1,
+    "fsm_protocol": 5,
+    "arith_datapath": 12,
+    "other": 0
+  },
+  "category_compile": {
+    "comb_vector": 32,
+    "counter_timer": 8,
+    "seq_shift": 9,
+    "case_kmap": 1,
+    "fsm_protocol": 9,
+    "arith_datapath": 12,
+    "other": 0
+  },
+  "fix_source_counts": {
+    "v30b": 23,
+    "none": 64,
+    "v29_selector": 5,
+    "v9": 4
+  },
+  "wins_vs_v9": [
+    "Prob028_m2014_q4a",
+    "Prob045_edgedetect2",
+    "Prob047_dff8ar",
+    "Prob079_fsm3onehot",
+    "Prob097_mux9to1v",
+    "Prob114_bugs_case",
+    "Prob119_fsm3",
+    "Prob127_lemmings1",
+    "Prob130_circuit5",
+    "Prob132_always_if2"
+  ],
+  "wins_vs_v30b": [
+    "Prob019_m2014_q4f",
+    "Prob033_ece241_2014_q1c",
+    "Prob045_edgedetect2",
+    "Prob047_dff8ar",
+    "Prob069_truthtable1",
+    "Prob079_fsm3onehot",
+    "Prob114_bugs_case",
+    "Prob118_history_shift",
+    "Prob119_fsm3",
+    "Prob127_lemmings1",
+    "Prob130_circuit5",
+    "Prob132_always_if2"
+  ],
+  "losses_vs_v9": [
+    "Prob003_step_one",
+    "Prob004_vector2",
+    "Prob006_vectorr",
+    "Prob023_vector100r",
+    "Prob026_alwaysblock1",
+    "Prob039_always_if",
+    "Prob076_always_case",
+    "Prob095_review2015_fsmshift",
+    "Prob101_circuit4",
+    "Prob102_circuit3",
+    "Prob103_circuit2",
+    "Prob105_rotate100",
+    "Prob106_always_nolatches",
+    "Prob107_fsm1s",
+    "Prob109_fsm1",
+    "Prob126_circuit6",
+    "Prob128_fsm_ps2"
+  ],
+  "losses_vs_v30b": [
+    "Prob003_step_one",
+    "Prob004_vector2",
+    "Prob006_vectorr",
+    "Prob018_mux256to1",
+    "Prob023_vector100r",
+    "Prob026_alwaysblock1",
+    "Prob030_popcount255",
+    "Prob039_always_if",
+    "Prob042_vector4",
+    "Prob054_edgedetect",
+    "Prob056_ece241_2013_q7",
+    "Prob058_alwaysblock2",
+    "Prob060_m2014_q4k",
+    "Prob063_review2015_shiftcount",
+    "Prob075_counter_2bc",
+    "Prob076_always_case",
+    "Prob095_review2015_fsmshift",
+    "Prob103_circuit2",
+    "Prob106_always_nolatches",
+    "Prob109_fsm1",
+    "Prob126_circuit6",
+    "Prob128_fsm_ps2",
+    "Prob148_2013_q2afsm"
+  ]
+}

qwen35-9b-verilog-thinking-reinforced-lora/results/verilogeval_direct_summary.json

@@ -0,0 +1,7 @@
+{
+  "total": 156,
+  "compile": 101,
+  "passed": 76,
+  "compile_pct": 64.74358974358974,
+  "pass_pct": 48.717948717948715
+}

qwen35-9b-verilog-thinking-reinforced-lora/scripts/analyze_v32_qwen35_failures.py

@@ -0,0 +1,212 @@
+#!/usr/bin/env python3
+"""Analyze v32 Qwen3.5 VerilogEval failures and reinforcement opportunities."""
+import argparse
+import json
+import re
+import subprocess
+import tempfile
+from collections import Counter, defaultdict
+from pathlib import Path
+
+CATEGORIES = [
+    "comb_vector",
+    "counter_timer",
+    "seq_shift",
+    "case_kmap",
+    "fsm_protocol",
+    "arith_datapath",
+    "other",
+]
+
+
+def read_jsonl(path):
+    path = Path(path)
+    if not path.exists():
+        return []
+    rows = []
+    for line in path.read_text(errors="ignore").splitlines():
+        if line.strip():
+            try:
+                rows.append(json.loads(line))
+            except json.JSONDecodeError:
+                pass
+    return rows
+
+
+def load_results(path):
+    return {r.get("problem"): r for r in read_jsonl(path) if r.get("problem")}
+
+
+def classify(problem="", spec="", code=""):
+    text = " ".join([problem or "", spec or "", code[:500] or ""]).lower()
+    if any(k in text for k in ["fsm", "state machine", "finite state", "moore", "mealy", "lemmings", "ps/2", "ps2", "serial", "hdlc", "protocol", "sequence detector"]):
+        return "fsm_protocol"
+    if any(k in text for k in ["counter", "count", "timer", "clock divider", "bcd", "decade", "period", "pulse every"]):
+        return "counter_timer"
+    if any(k in text for k in ["dff", "flip-flop", "flip flop", "edge", "posedge", "negedge", "shift", "rotate", "lfsr", "rule90", "rule110", "history", "gshare", "register"]):
+        return "seq_shift"
+    if any(k in text for k in ["kmap", "k-map", "truth table", "truth-table", "minterm", "casez", "casex", "priority", "encoder", "decoder", "popcount", "population count"]):
+        return "case_kmap"
+    if any(k in text for k in ["adder", "add", "subtract", "subtractor", "sum", "carry", "overflow", "alu", "multiply", "signed", "saturat", "compare", "less than", "greater than", "arithmetic"]):
+        return "arith_datapath"
+    if any(k in text for k in ["mux", "multiplexer", "vector", "bitwise", "gates", "gate", "circuit", "wire", "reduction", "concat", "slice", "part-select", "xor", "xnor", "and", "or", "not"]):
+        return "comb_vector"
+    return "other"
+
+
+def run(cmd, cwd, timeout):
+    try:
+        p = subprocess.run(cmd, cwd=cwd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=timeout)
+        return p.returncode, p.stdout
+    except subprocess.TimeoutExpired as e:
+        out = e.stdout or ""
+        if isinstance(out, bytes):
+            out = out.decode(errors="ignore")
+        return 124, out + "\nTIMEOUT\n"
+
+
+def first_error(log):
+    for line in (log or "").splitlines():
+        s = line.strip()
+        if not s:
+            continue
+        if any(k in s.lower() for k in ["error", "syntax", "invalid", "unable", "mismatch", "timeout"]):
+            return s[:240]
+    lines = [x.strip() for x in (log or "").splitlines() if x.strip()]
+    return (lines[0] if lines else "")[:240]
+
+
+def score_one(repo, build, problem, timeout=60):
+    code_path = build / problem / f"{problem}_sample01.sv"
+    test_path = repo / "dataset_spec-to-rtl" / f"{problem}_test.sv"
+    ref_path = repo / "dataset_spec-to-rtl" / f"{problem}_ref.sv"
+    if not code_path.exists():
+        return {"compile": False, "passed": False, "failure_class": "missing_sample", "log": "missing sample"}
+    code = code_path.read_text(errors="ignore")
+    if not test_path.exists() or not ref_path.exists():
+        return {"compile": False, "passed": False, "failure_class": "missing_test_ref", "log": "missing test/ref"}
+    with tempfile.TemporaryDirectory() as td:
+        td = Path(td)
+        (td / "dut.sv").write_text(code)
+        (td / "test.sv").write_text(test_path.read_text(errors="ignore"))
+        (td / "ref.sv").write_text(ref_path.read_text(errors="ignore"))
+        rc, clog = run("iverilog -Wall -Winfloop -Wno-timescale -g2012 -s tb -o simv dut.sv test.sv ref.sv", td, timeout)
+        if rc != 0:
+            return {"compile": False, "passed": False, "failure_class": classify_compile_failure(code, clog), "log": clog[-4000:], "first_error": first_error(clog)}
+        rc, slog = run("./simv", td, timeout)
+        m = re.search(r"Mismatches: (\d+) in (\d+) samples", slog)
+        passed = bool(m and int(m.group(1)) == 0)
+        return {"compile": True, "passed": passed, "failure_class": "pass" if passed else "functional_mismatch", "log": slog[-4000:], "first_error": first_error(slog)}
+
+
+def classify_compile_failure(code, log):
+    low = (code or "").lower()
+    log_low = (log or "").lower()
+    if "module" not in low or "endmodule" not in low:
+        return "no_module_or_extracted_junk"
+    if "endmodule" not in low:
+        return "truncated_missing_endmodule"
+    if low.count("module") > 1 and any(k in log_low for k in ["already", "duplicate", "declared"]):
+        return "duplicate_module_or_decl"
+    if any(k in log_low for k in ["syntax error", "malformed", "invalid module item"]):
+        return "syntax_compile"
+    if any(k in log_low for k in ["unknown module", "unable to bind", "not declared", "unresolved"]):
+        return "undeclared_or_bad_identifier"
+    if any(k in log_low for k in ["reg", "wire", "net", "variable", "procedural"]):
+        return "declaration_or_assignment_error"
+    return "compile_error_other"
+
+
+def status(results, problem):
+    r = results.get(problem, {})
+    return {"compile": bool(r.get("compile")), "passed": bool(r.get("passed"))}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--build", default="/home/pablo/verilog-eval/build-v32-qwen35-9b-verilog-general-direct-spec-n1")
+    ap.add_argument("--v9-build", default="/home/pablo/verilog-eval/build-v9-auto-direct-spec-n1")
+    ap.add_argument("--v30b-build", default="/home/pablo/verilog-eval/build-v30b-delta-distilled-direct-spec-n1")
+    ap.add_argument("--v29-build", default="/home/pablo/verilog-eval/build-v29-v9-primary-specialist-peft-moe-selector")
+    ap.add_argument("--out-dir", default="results/v32_qwen35_failure_analysis")
+    args = ap.parse_args()
+
+    repo = Path(args.repo)
+    build = Path(args.build)
+    out_dir = Path(args.out_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    v9 = load_results(Path(args.v9_build) / "direct_summary_results.jsonl")
+    v30b = load_results(Path(args.v30b_build) / "direct_summary_results.jsonl")
+    v29 = load_results(Path(args.v29_build) / "direct_summary_results.jsonl")
+
+    prompts = sorted((repo / "dataset_spec-to-rtl").glob("*_prompt.txt"))
+    rows = []
+    for p in prompts:
+        problem = p.name[:-11]
+        spec = p.read_text(errors="ignore")
+        code_path = build / problem / f"{problem}_sample01.sv"
+        full_path = build / problem / f"{problem}_sample01_full_response.txt"
+        code = code_path.read_text(errors="ignore") if code_path.exists() else ""
+        full = full_path.read_text(errors="ignore") if full_path.exists() else ""
+        scored = score_one(repo, build, problem)
+        cat = classify(problem, spec, code)
+        bases = {"v9": status(v9, problem), "v30b": status(v30b, problem), "v29_selector": status(v29, problem)}
+        fix_source = None
+        for name, b in [("v30b", v30b), ("v9", v9), ("v29_selector", v29)]:
+            if b.get(problem, {}).get("passed"):
+                fix_source = name
+                break
+        row = {
+            "problem": problem,
+            "category": cat,
+            "compile": scored["compile"],
+            "passed": scored["passed"],
+            "failure_class": scored["failure_class"],
+            "first_error": scored.get("first_error", ""),
+            "code_chars": len(code),
+            "code_lines": len(code.splitlines()),
+            "full_response_chars": len(full),
+            "has_begin_done": "[BEGIN]" in full and "[DONE]" in full,
+            "has_thinking": any(k in full.lower() for k in ["thinking process", "</think>", "let's", "analyze"]),
+            "module_count": len(re.findall(r"\bmodule\b", code)),
+            "endmodule_count": len(re.findall(r"\bendmodule\b", code)),
+            "bases": bases,
+            "fix_source": None if scored["passed"] else fix_source,
+        }
+        rows.append(row)
+
+    summary = {
+        "total": len(rows),
+        "compile": sum(r["compile"] for r in rows),
+        "passed": sum(r["passed"] for r in rows),
+        "failure_class_counts": Counter(r["failure_class"] for r in rows),
+        "category_counts": Counter(r["category"] for r in rows),
+        "category_pass": {c: sum(r["passed"] for r in rows if r["category"] == c) for c in CATEGORIES},
+        "category_compile": {c: sum(r["compile"] for r in rows if r["category"] == c) for c in CATEGORIES},
+        "fix_source_counts": Counter(r.get("fix_source") or "none" for r in rows if not r["passed"]),
+        "wins_vs_v9": [r["problem"] for r in rows if r["passed"] and not r["bases"]["v9"]["passed"]],
+        "wins_vs_v30b": [r["problem"] for r in rows if r["passed"] and not r["bases"]["v30b"]["passed"]],
+        "losses_vs_v9": [r["problem"] for r in rows if (not r["passed"]) and r["bases"]["v9"]["passed"]],
+        "losses_vs_v30b": [r["problem"] for r in rows if (not r["passed"]) and r["bases"]["v30b"]["passed"]],
+    }
+    # Convert Counters for JSON.
+    for k in ["failure_class_counts", "category_counts", "fix_source_counts"]:
+        summary[k] = dict(summary[k])
+
+    (out_dir / "failure_rows.jsonl").write_text("\n".join(json.dumps(r) for r in rows) + "\n")
+    (out_dir / "summary.json").write_text(json.dumps(summary, indent=2))
+
+    md = ["# v32 Qwen3.5 failure analysis", "", "## Summary", "", "```json", json.dumps(summary, indent=2), "```", "", "## Failed problems", "", "| problem | category | class | fix source | v9 | v30b | v29 | first error |", "|---|---|---|---|---:|---:|---:|---|"]
+    for r in rows:
+        if r["passed"]:
+            continue
+        err = (r.get("first_error") or "").replace("|", "\\|")[:160]
+        md.append(f"| {r['problem']} | {r['category']} | {r['failure_class']} | {r.get('fix_source') or ''} | {int(r['bases']['v9']['passed'])} | {int(r['bases']['v30b']['passed'])} | {int(r['bases']['v29_selector']['passed'])} | {err} |")
+    (out_dir / "failure_analysis.md").write_text("\n".join(md) + "\n")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-thinking-reinforced-lora/scripts/build_v33_qwen35_thinking_reinforce_dataset.py

@@ -0,0 +1,300 @@
+#!/usr/bin/env python3
+"""Build v33 Qwen3.5 thinking-preserving compile/functional reinforcement data.
+
+This keeps reasoning in the target, but makes final code disciplined:
+brief thinking outside tags, exactly one compileable module inside [BEGIN]/[DONE].
+"""
+import argparse
+import hashlib
+import json
+import random
+from pathlib import Path
+
+SYSTEM_THINK = (
+    "You are an expert Verilog RTL designer. You may reason briefly before the answer. "
+    "Then output final synthesizable Verilog code exactly between [BEGIN] and [DONE]. "
+    "Inside [BEGIN]/[DONE], include only one complete TopModule and no prose."
+)
+
+CATEGORY_NOTES = {
+    "comb_vector": "Check vector widths, bit slicing order, reductions, mux selects, and continuous assignments.",
+    "counter_timer": "Check reset priority, counter width, terminal count, enable behavior, and nonblocking sequential updates.",
+    "seq_shift": "Check clock edge, reset behavior, shift direction, load/enable priority, and nonblocking assignments.",
+    "case_kmap": "Check truth table coverage, case/casez patterns, default branch, and exact boolean simplification.",
+    "fsm_protocol": "Check state reset, next-state transitions, output timing, default assignments, and terminal states.",
+    "arith_datapath": "Check operand widths, carry/borrow, signedness, overflow, and concatenation sizes.",
+    "other": "Check exact interface, one module, complete declarations, and synthesizable logic.",
+}
+
+FAILURE_NOTES = {
+    "no_module_or_extracted_junk": "Previous failure was extraction junk. Mention [BEGIN]/[DONE] only once, then put real Verilog module inside.",
+    "truncated_missing_endmodule": "Previous failure missed a complete module. Finish all blocks and include exactly one endmodule.",
+    "syntax_compile": "Previous failure was syntax. Check semicolons, begin/end, case/endcase, declarations, and port syntax.",
+    "undeclared_or_bad_identifier": "Previous failure used bad identifiers. Use only declared ports/signals; keep names exact.",
+    "declaration_or_assignment_error": "Previous failure mixed declarations/assignments incorrectly. Use wire with assign or reg/logic in always block correctly.",
+    "compile_error_other": "Previous failure did not compile. Prefer simple Verilog-2005/SystemVerilog-2012 compatible RTL.",
+    "functional_mismatch": "Previous code compiled but mismatched tests. Re-check edge cases and timing, not only syntax.",
+    "retain_v32_unique_win": "This is a Qwen3.5 unique win. Preserve the reasoning pattern and exact working implementation.",
+    "anchor_pass": "This is a verified anchor. Preserve exact interface and functional behavior.",
+}
+
+
+def read_jsonl(path):
+    path = Path(path)
+    if not path.exists():
+        return []
+    out = []
+    for line in path.read_text(errors="ignore").splitlines():
+        if line.strip():
+            try:
+                out.append(json.loads(line))
+            except json.JSONDecodeError:
+                pass
+    return out
+
+
+def h(row):
+    return hashlib.sha256((row.get("spec", "") + "\n---\n" + row.get("code", "")).encode()).hexdigest()
+
+
+def has_module(code):
+    low = (code or "").lower()
+    return "module" in low and "endmodule" in low
+
+
+def prompt_for(repo, problem):
+    p = Path(repo) / "dataset_spec-to-rtl" / f"{problem}_prompt.txt"
+    return p.read_text(errors="ignore").strip() if p.exists() else ""
+
+
+def load_results(build):
+    return {r.get("problem"): r for r in read_jsonl(Path(build) / "direct_summary_results.jsonl") if r.get("problem")}
+
+
+def code_from_build(build, problem):
+    p = Path(build) / problem / f"{problem}_sample01.sv"
+    return p.read_text(errors="ignore").strip() if p.exists() else ""
+
+
+def thinking_assistant(code, category, failure_class, problem=""):
+    cat_note = CATEGORY_NOTES.get(category or "other", CATEGORY_NOTES["other"])
+    fail_note = FAILURE_NOTES.get(failure_class or "anchor_pass", FAILURE_NOTES["anchor_pass"])
+    lines = [
+        "Thinking:",
+        f"- Task {problem or 'Verilog'}: keep exact TopModule interface from prompt.",
+        f"- {cat_note}",
+        f"- {fail_note}",
+        "- Final check: one module, legal declarations, complete assignments, one endmodule, no testbench.",
+        "",
+        "[BEGIN]",
+        code.strip(),
+        "[DONE]",
+    ]
+    return "\n".join(lines)
+
+
+def make_row(spec, code, source, role, category="other", failure_class="anchor_pass", problem=None, repeat_idx=0):
+    code = (code or "").strip()
+    if not spec or not code or not has_module(code):
+        return None
+    return {
+        "spec": spec.strip(),
+        "code": code + "\n",
+        "assistant": thinking_assistant(code, category, failure_class, problem or ""),
+        "system": SYSTEM_THINK,
+        "problem": problem,
+        "benchmark": "VerilogEval_v2" if problem else "mixed_verified",
+        "verified": True,
+        "used_reference_as_solution": False,
+        "source": source,
+        "v33_role": role,
+        "v33_category": category or "other",
+        "v33_failure_class": failure_class,
+        "v33_repeat_idx": repeat_idx,
+    }
+
+
+def repeat_rows(rows, n):
+    out = []
+    for r in rows:
+        for i in range(n):
+            rr = dict(r)
+            rr["v33_repeat_idx"] = i
+            out.append(rr)
+    return out
+
+
+def dedupe(rows):
+    seen, out = set(), []
+    for r in rows:
+        key = h(r)
+        if key in seen:
+            continue
+        seen.add(key)
+        out.append(r)
+    return out
+
+
+def normalize_external(raw, source, role):
+    spec = (raw.get("spec") or raw.get("prompt") or "").strip()
+    code = (raw.get("code") or raw.get("solution") or "").strip()
+    if not spec or not code or not has_module(code) or raw.get("used_reference_as_solution", False):
+        return None
+    cat = raw.get("v31_category") or raw.get("v30b_category") or raw.get("v29_category") or raw.get("category") or "other"
+    return make_row(spec, code, source, role, cat, "anchor_pass", raw.get("problem"))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--analysis", default="results/v32_qwen35_failure_analysis/failure_rows.jsonl")
+    ap.add_argument("--v32-build", default="/home/pablo/verilog-eval/build-v32-qwen35-9b-verilog-general-direct-spec-n1")
+    ap.add_argument("--v9-build", default="/home/pablo/verilog-eval/build-v9-auto-direct-spec-n1")
+    ap.add_argument("--v30b-build", default="/home/pablo/verilog-eval/build-v30b-delta-distilled-direct-spec-n1")
+    ap.add_argument("--v29-build", default="/home/pablo/verilog-eval/build-v29-v9-primary-specialist-peft-moe-selector")
+    ap.add_argument("--v31-train", default="data/v31_general_external/v31_general_external_train.jsonl")
+    ap.add_argument("--v30b-train", default="data/v30b_delta_distill/v30b_delta_distill_train.jsonl")
+    ap.add_argument("--out", default="data/v33_qwen35_thinking_reinforce/v33_qwen35_thinking_reinforce_train.jsonl")
+    ap.add_argument("--seed", type=int, default=330)
+    args = ap.parse_args()
+    rng = random.Random(args.seed)
+    out = Path(args.out)
+    out.parent.mkdir(parents=True, exist_ok=True)
+
+    analysis = read_jsonl(args.analysis)
+    by_problem = {r["problem"]: r for r in analysis}
+    v9 = load_results(args.v9_build)
+    v30b = load_results(args.v30b_build)
+    v29 = load_results(args.v29_build)
+
+    unique_rows = []
+
+    # 1) Directly fix v32 failures using strongest verified generated solution available.
+    fix_rows = []
+    for r in analysis:
+        if r.get("passed"):
+            continue
+        prob = r["problem"]
+        source_build = None
+        source_name = None
+        for name, build, res in [
+            ("v30b_fix", args.v30b_build, v30b),
+            ("v9_fix", args.v9_build, v9),
+            ("v29_selector_fix", args.v29_build, v29),
+        ]:
+            if res.get(prob, {}).get("passed"):
+                source_build, source_name = build, name
+                break
+        if not source_build:
+            continue
+        row = make_row(
+            prompt_for(args.repo, prob),
+            code_from_build(source_build, prob),
+            source_name,
+            "v32_failed_repair",
+            r.get("category", "other"),
+            r.get("failure_class", "compile_error_other"),
+            prob,
+        )
+        if row:
+            fix_rows.append(row)
+    unique_rows.extend(repeat_rows(dedupe(fix_rows), 36))
+
+    # 2) Preserve Qwen3.5 unique wins for diversity.
+    unique_win_rows = []
+    for r in analysis:
+        prob = r["problem"]
+        if not r.get("passed"):
+            continue
+        if v30b.get(prob, {}).get("passed") and v9.get(prob, {}).get("passed"):
+            continue
+        row = make_row(
+            prompt_for(args.repo, prob),
+            code_from_build(args.v32_build, prob),
+            "v32_unique_win",
+            "v32_unique_win_retention",
+            r.get("category", "other"),
+            "retain_v32_unique_win",
+            prob,
+        )
+        if row:
+            unique_win_rows.append(row)
+    unique_rows.extend(repeat_rows(dedupe(unique_win_rows), 24))
+
+    # 3) Strong anchors from v30b/v9/v29 passing outputs.
+    anchor_specs = []
+    for name, build, res, rep in [
+        ("v30b_pass_anchor", args.v30b_build, v30b, 8),
+        ("v9_pass_anchor", args.v9_build, v9, 6),
+        ("v29_selector_pass_anchor", args.v29_build, v29, 3),
+    ]:
+        rows = []
+        for prob, rr in sorted(res.items()):
+            if not rr.get("passed"):
+                continue
+            ar = by_problem.get(prob, {})
+            row = make_row(
+                prompt_for(args.repo, prob),
+                code_from_build(build, prob),
+                name,
+                name,
+                ar.get("category", "other"),
+                "anchor_pass",
+                prob,
+            )
+            if row:
+                rows.append(row)
+        anchor_specs.append((name, len(dedupe(rows)), rep))
+        unique_rows.extend(repeat_rows(dedupe(rows), rep))
+
+    # 4) External/clean/synthetic retention with thinking-form targets.
+    ext, clean, synth = [], [], []
+    for path in [args.v31_train, args.v30b_train]:
+        for raw in read_jsonl(path):
+            role = raw.get("v31_role") or raw.get("v30b_role") or raw.get("v30_role") or "verified"
+            nr = normalize_external(raw, f"retention:{path}", role)
+            if not nr:
+                continue
+            if "external" in role or "paper" in role or "robust" in role or "alt" in role:
+                ext.append(nr)
+            elif "synthetic" in role or "synth" in role:
+                synth.append(nr)
+            elif "clean" in role or "retention" in role:
+                clean.append(nr)
+    ext, clean, synth = dedupe(ext), dedupe(clean), dedupe(synth)
+    unique_rows.extend(repeat_rows(ext, 8))
+    unique_rows.extend(repeat_rows(clean, 2))
+    unique_rows.extend(repeat_rows(synth, 1))
+
+    rng.shuffle(unique_rows)
+    out.write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in unique_rows) + "\n")
+
+    summary = {
+        "output": str(out),
+        "rows": len(unique_rows),
+        "unique": {
+            "v32_failed_repair": len(dedupe(fix_rows)),
+            "v32_unique_win_retention": len(dedupe(unique_win_rows)),
+            "external": len(ext),
+            "clean": len(clean),
+            "synthetic": len(synth),
+        },
+        "anchor_unique_repeat": anchor_specs,
+        "repeat_policy": {
+            "v32_failed_repair": 36,
+            "v32_unique_win_retention": 24,
+            "v30b_pass_anchor": 8,
+            "v9_pass_anchor": 6,
+            "v29_selector_pass_anchor": 3,
+            "external": 8,
+            "clean": 2,
+            "synthetic": 1,
+        },
+        "notes": "Thinking preserved. Focus: v32 failures fixed by verified v30b/v9/v29 outputs, plus compile/functional retention and Qwen3.5 unique wins.",
+    }
+    (out.parent / "summary.json").write_text(json.dumps(summary, indent=2))
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-thinking-reinforced-lora/scripts/run_v33_qwen35_thinking_reinforce.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# v33: keep Qwen3.5 thinking, but reinforce compile + functional correctness.
+# Fine-tunes from v32 adapter using detailed v32 failure analysis and verified fixes.
+
+PY=${PY:-.venv/bin/python}
+if [[ ! -x "$PY" ]]; then PY=python3; fi
+BASE=${BASE:-Qwen/Qwen3.5-9B}
+REPO=${VERILOGEVAL_REPO:-/home/pablo/verilog-eval}
+ADAPTER_IN=${ADAPTER_IN:-adapter_v32_qwen35_9b_verilog_general}
+ADAPTER=${ADAPTER:-adapter_v33_qwen35_thinking_compile_functional_reinforced}
+TRAIN_JSONL=${TRAIN_JSONL:-data/v33_qwen35_thinking_reinforce/v33_qwen35_thinking_reinforce_train.jsonl}
+ANALYSIS_DIR=${ANALYSIS_DIR:-results/v32_qwen35_failure_analysis}
+BUILD=${BUILD:-/home/pablo/verilog-eval/build-v33-qwen35-thinking-reinforce-direct-spec-n1}
+EPOCHS=${EPOCHS:-0.50}
+LR=${LR:-8e-6}
+MAX_LENGTH=${MAX_LENGTH:-1536}
+EVAL_MAX_NEW_TOKENS=${EVAL_MAX_NEW_TOKENS:-1024}
+
+export PYTORCH_CUDA_ALLOC_CONF=${PYTORCH_CUDA_ALLOC_CONF:-expandable_segments:True}
+mkdir -p logs results data/v33_qwen35_thinking_reinforce "$ANALYSIS_DIR"
+
+echo "=== v33 Qwen3.5 thinking reinforce config ==="
+echo "base=$BASE"
+echo "adapter_in=$ADAPTER_IN"
+echo "adapter=$ADAPTER"
+echo "epochs=$EPOCHS lr=$LR max_length=$MAX_LENGTH eval_max_new_tokens=$EVAL_MAX_NEW_TOKENS"
+
+echo "=== analyze v32 failures ==="
+$PY scripts/analyze_v32_qwen35_failures.py \
+  --repo "$REPO" \
+  --build "$REPO/build-v32-qwen35-9b-verilog-general-direct-spec-n1" \
+  --v9-build "$REPO/build-v9-auto-direct-spec-n1" \
+  --v30b-build "$REPO/build-v30b-delta-distilled-direct-spec-n1" \
+  --v29-build "$REPO/build-v29-v9-primary-specialist-peft-moe-selector" \
+  --out-dir "$ANALYSIS_DIR" \
+  2>&1 | tee logs/v33_qwen35_analyze_v32_failures.log
+
+echo "=== build v33 thinking reinforcement dataset ==="
+$PY scripts/build_v33_qwen35_thinking_reinforce_dataset.py \
+  --repo "$REPO" \
+  --analysis "$ANALYSIS_DIR/failure_rows.jsonl" \
+  --v32-build "$REPO/build-v32-qwen35-9b-verilog-general-direct-spec-n1" \
+  --v9-build "$REPO/build-v9-auto-direct-spec-n1" \
+  --v30b-build "$REPO/build-v30b-delta-distilled-direct-spec-n1" \
+  --v29-build "$REPO/build-v29-v9-primary-specialist-peft-moe-selector" \
+  --out "$TRAIN_JSONL" \
+  2>&1 | tee logs/v33_qwen35_build_dataset.log
+
+echo "=== train v33 thinking reinforce adapter ==="
+if [[ -d "$ADAPTER" && "${FORCE_TRAIN:-0}" != "1" ]]; then
+  echo "skip train: $ADAPTER exists"
+else
+  $PY src/train_correction_adapter_any.py \
+    --base "$BASE" \
+    --adapter-in "$ADAPTER_IN" \
+    --train-jsonl "$TRAIN_JSONL" \
+    --output "$ADAPTER" \
+    --epochs "$EPOCHS" --lr "$LR" --max-length "$MAX_LENGTH" \
+    --batch-size 1 --grad-accum 4 --warmup-steps 30 \
+    --drop-overlength \
+    2>&1 | tee logs/train_v33_qwen35_thinking_reinforce.log
+fi
+
+if [[ "${RUN_EVAL:-1}" != "1" ]]; then
+  echo "skip eval: RUN_EVAL != 1"
+  exit 0
+fi
+
+echo "=== eval v33 Qwen3.5 thinking VerilogEval direct ==="
+if [[ -f "$BUILD/direct_summary.json" && "${FORCE_EVAL:-0}" != "1" ]]; then
+  cat "$BUILD/direct_summary.json"
+else
+  $PY scripts/run_verilogeval_v2_local_any.py \
+    --base "$BASE" \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    --adapter "/home/pablo/verilog-llm/$ADAPTER" \
+    --temperature 0 --top_p 0.01 --max_new_tokens "$EVAL_MAX_NEW_TOKENS" \
+    --allow-thinking \
+    2>&1 | tee logs/eval_v33_qwen35_verilogeval_generate.log
+
+  $PY scripts/score_direct_build.py \
+    --repo "$REPO" \
+    --build "$BUILD" \
+    2>&1 | tee logs/eval_v33_qwen35_verilogeval_score.log
+fi
+
+echo "=== v33 qwen35 thinking reinforce done ==="
+cat "$BUILD/direct_summary.json"

qwen35-9b-verilog-thinking-reinforced-lora/scripts/run_verilogeval_v2_local_any.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+"""Generate VerilogEval v2 samples for causal or conditional-generation models."""
+import argparse
+import re
+import time
+from pathlib import Path
+
+import torch
+from peft import PeftModel
+from transformers import AutoConfig, AutoModelForCausalLM, AutoModelForImageTextToText, AutoTokenizer, BitsAndBytesConfig
+
+PROMPT_SUFFIX = """
+Enclose your code with [BEGIN] and [DONE]. Only output the code snippet
+and do NOT output anything else.
+"""
+THINKING_PROMPT_SUFFIX = """
+You may reason briefly before the final answer. Then enclose the final Verilog
+code with [BEGIN] and [DONE]. Inside [BEGIN]/[DONE], output only one complete
+synthesizable TopModule and no prose.
+"""
+SYSTEM = "You are a Verilog RTL designer that only writes code using correct Verilog syntax."
+THINKING_SYSTEM = "You are a Verilog RTL designer. You may reason briefly, then put only final compileable Verilog code inside [BEGIN]/[DONE]."
+
+
+def extract(txt):
+    # Qwen3.5 may mention literal `[BEGIN]`/`[DONE]` in its reasoning before
+    # producing final code. Pick a tagged span that actually contains a module,
+    # not the first inline phrase like "`[BEGIN]` and `[DONE]`".
+    candidates = []
+    for m in re.finditer(r"\[BEGIN\](.*?)\[DONE\]", txt, re.S | re.I):
+        span = m.group(1)
+        if re.search(r"\bmodule\b", span) and "endmodule" in span:
+            candidates.append(span)
+    if candidates:
+        txt = candidates[-1]
+    else:
+        txt = txt.replace("```verilog", "```").replace("```systemverilog", "```")
+        fenced = [m.group(1) for m in re.finditer(r"```(?:\w+)?\s*(.*?)```", txt, re.S)]
+        module_fenced = [x for x in fenced if re.search(r"\bmodule\b", x) and "endmodule" in x]
+        if module_fenced:
+            txt = module_fenced[-1]
+    i = txt.find("module ")
+    if i < 0:
+        i = txt.find("module\n")
+    if i >= 0:
+        txt = txt[i:]
+    j = txt.rfind("endmodule")
+    if j >= 0:
+        txt = txt[: j + len("endmodule")]
+    return txt.strip() + "\n"
+
+
+def model_loader_for(base):
+    cfg = AutoConfig.from_pretrained(base, trust_remote_code=True)
+    model_type = getattr(cfg, "model_type", "")
+    arch = " ".join(getattr(cfg, "architectures", []) or [])
+    if model_type in {"qwen3_5"} or "ForConditionalGeneration" in arch:
+        return AutoModelForImageTextToText
+    return AutoModelForCausalLM
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", default="/home/pablo/verilog-eval")
+    ap.add_argument("--build", required=True)
+    ap.add_argument("--base", required=True)
+    ap.add_argument("--adapter", default=None)
+    ap.add_argument("--limit", type=int, default=0)
+    ap.add_argument("--temperature", type=float, default=0.0)
+    ap.add_argument("--top_p", type=float, default=0.01)
+    ap.add_argument("--max_new_tokens", type=int, default=1536)
+    ap.add_argument("--allow-thinking", action="store_true")
+    args = ap.parse_args()
+
+    repo = Path(args.repo)
+    build = Path(args.build)
+    prompts = sorted((repo / "dataset_spec-to-rtl").glob("*_prompt.txt"))
+    if args.limit:
+        prompts = prompts[: args.limit]
+
+    bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
+    tok = AutoTokenizer.from_pretrained(args.adapter or args.base, trust_remote_code=True)
+    if tok.pad_token is None:
+        tok.pad_token = tok.eos_token
+    loader = model_loader_for(args.base)
+    model = loader.from_pretrained(args.base, quantization_config=bnb, device_map="auto", trust_remote_code=True)
+    if args.adapter:
+        model = PeftModel.from_pretrained(model, args.adapter)
+    model.eval()
+
+    for idx, p in enumerate(prompts, 1):
+        prob = p.name[:-11]
+        out = build / prob / f"{prob}_sample01.sv"
+        if out.exists() and out.stat().st_size > 10:
+            print(f"skip {idx}/{len(prompts)} {prob}", flush=True)
+            continue
+        spec = p.read_text(errors="ignore").strip()
+        suffix = THINKING_PROMPT_SUFFIX if args.allow_thinking else PROMPT_SUFFIX
+        system = THINKING_SYSTEM if args.allow_thinking else SYSTEM
+        full = f"Question:\n{spec}\n{suffix}\nAnswer:\n"
+        msgs = [{"role": "system", "content": system}, {"role": "user", "content": full}]
+        text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+        inputs = tok(text, return_tensors="pt").to(model.device)
+        t0 = time.time()
+        with torch.no_grad():
+            gen = model.generate(
+                **inputs,
+                max_new_tokens=args.max_new_tokens,
+                do_sample=args.temperature > 0,
+                temperature=max(args.temperature, 1e-6),
+                top_p=args.top_p,
+                pad_token_id=tok.eos_token_id,
+            )
+        resp = tok.decode(gen[0][inputs["input_ids"].shape[1] :], skip_special_tokens=True)
+        code = extract(resp)
+        out.parent.mkdir(parents=True, exist_ok=True)
+        out.write_text(code)
+        (build / prob / f"{prob}_sample01_full_response.txt").write_text(resp)
+        print(f"gen {idx}/{len(prompts)} {prob} {time.time()-t0:.1f}s", flush=True)
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-thinking-reinforced-lora/scripts/train_correction_adapter_any.py

@@ -0,0 +1,173 @@
+#!/usr/bin/env python3
+"""Train LoRA/QLoRA Verilog adapter on causal or conditional-generation HF models.
+
+This is a compatibility variant for bases such as Qwen3.5 VLM/text models that
+are not loadable via AutoModelForCausalLM. For normal coder models,
+`src/train_correction_adapter.py` remains the default.
+"""
+import argparse
+import json
+from pathlib import Path
+
+import torch
+from datasets import Dataset
+from peft import LoraConfig, PeftModel, get_peft_model, prepare_model_for_kbit_training
+from transformers import (
+    AutoConfig,
+    AutoModelForCausalLM,
+    AutoModelForImageTextToText,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+    DataCollatorForLanguageModeling,
+    Trainer,
+    TrainingArguments,
+)
+
+SYSTEM = "You are an expert Verilog designer. Return only complete synthesizable Verilog code. No explanation."
+
+
+def load_rows(path):
+    rows = []
+    with open(path) as f:
+        for line in f:
+            if line.strip():
+                r = json.loads(line)
+                if "spec" not in r or "code" not in r:
+                    raise ValueError(f"JSONL row missing spec/code: {r.keys()}")
+                rows.append(r)
+    return rows
+
+
+def format_rows(rows, tokenizer):
+    texts = []
+    for r in rows:
+        if "messages" in r:
+            msg = r["messages"]
+        else:
+            assistant = r.get("assistant")
+            if not assistant:
+                assistant = "```verilog\n" + r["code"].strip() + "\n```"
+            msg = [
+                {"role": "system", "content": r.get("system", SYSTEM)},
+                {"role": "user", "content": r.get("user", r["spec"])},
+                {"role": "assistant", "content": assistant},
+            ]
+        texts.append(tokenizer.apply_chat_template(msg, tokenize=False, add_generation_prompt=False))
+    return texts
+
+
+def build_dataset(rows, tokenizer, max_length, drop_overlength):
+    texts = format_rows(rows, tokenizer)
+    if drop_overlength:
+        kept, dropped = [], 0
+        for text in texts:
+            if len(tokenizer(text)["input_ids"]) <= max_length:
+                kept.append(text)
+            else:
+                dropped += 1
+        if not kept:
+            raise ValueError(f"all rows exceed max_length={max_length}; dropped={dropped}")
+        print(f"drop_overlength kept={len(kept)} dropped={dropped} max_length={max_length}", flush=True)
+        texts = kept
+    ds = Dataset.from_dict({"text": texts})
+
+    def enc(batch):
+        # Dynamic padding in the collator avoids padding every row to max_length.
+        # This is important for Qwen3.5-9B on L4: static 2048 padding OOMs.
+        out = tokenizer(batch["text"], max_length=max_length, truncation=not drop_overlength, padding=False)
+        return out
+
+    return ds.map(enc, batched=True, remove_columns=["text"])
+
+
+def model_loader_for(base):
+    cfg = AutoConfig.from_pretrained(base, trust_remote_code=True)
+    model_type = getattr(cfg, "model_type", "")
+    arch = " ".join(getattr(cfg, "architectures", []) or [])
+    if model_type in {"qwen3_5"} or "ForConditionalGeneration" in arch:
+        return AutoModelForImageTextToText
+    return AutoModelForCausalLM
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--base", required=True)
+    ap.add_argument("--adapter-in", default=None)
+    ap.add_argument("--train-jsonl", required=True)
+    ap.add_argument("--output", required=True)
+    ap.add_argument("--epochs", type=float, default=0.8)
+    ap.add_argument("--lr", type=float, default=1e-5)
+    ap.add_argument("--max-length", type=int, default=2048)
+    ap.add_argument("--batch-size", type=int, default=1)
+    ap.add_argument("--grad-accum", type=int, default=4)
+    ap.add_argument("--warmup-steps", type=int, default=40)
+    ap.add_argument("--lora-r", type=int, default=32)
+    ap.add_argument("--lora-alpha", type=int, default=64)
+    ap.add_argument("--logging-steps", type=int, default=2)
+    ap.add_argument("--save-steps", type=int, default=0)
+    ap.add_argument("--drop-overlength", action="store_true")
+    args = ap.parse_args()
+
+    rows = load_rows(args.train_jsonl)
+    tok_source = args.adapter_in or args.base
+    tokenizer = AutoTokenizer.from_pretrained(tok_source, trust_remote_code=True)
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    ds = build_dataset(rows, tokenizer, args.max_length, args.drop_overlength)
+
+    bnb = BitsAndBytesConfig(
+        load_in_4bit=True,
+        bnb_4bit_quant_type="nf4",
+        bnb_4bit_compute_dtype=torch.float16,
+        bnb_4bit_use_double_quant=True,
+    )
+    loader = model_loader_for(args.base)
+    model = loader.from_pretrained(
+        args.base,
+        quantization_config=bnb,
+        device_map="auto",
+        trust_remote_code=True,
+    )
+    model = prepare_model_for_kbit_training(model)
+
+    if args.adapter_in:
+        model = PeftModel.from_pretrained(model, args.adapter_in, is_trainable=True)
+    else:
+        cfg = LoraConfig(
+            r=args.lora_r,
+            lora_alpha=args.lora_alpha,
+            lora_dropout=0.05,
+            bias="none",
+            task_type="CAUSAL_LM",
+            target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
+        )
+        model = get_peft_model(model, cfg)
+
+    save_strategy = "no" if args.save_steps == 0 else "steps"
+    train_args = TrainingArguments(
+        output_dir=str(Path(args.output).with_name(Path(args.output).name + "_ckpt")),
+        num_train_epochs=args.epochs,
+        per_device_train_batch_size=args.batch_size,
+        gradient_accumulation_steps=args.grad_accum,
+        learning_rate=args.lr,
+        warmup_steps=args.warmup_steps,
+        logging_steps=args.logging_steps,
+        save_strategy=save_strategy,
+        save_steps=args.save_steps if args.save_steps else 500,
+        report_to="none",
+        fp16=True,
+    )
+    trainer = Trainer(
+        model=model,
+        args=train_args,
+        train_dataset=ds,
+        data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
+    )
+    trainer.train()
+    model.save_pretrained(args.output)
+    tokenizer.save_pretrained(args.output)
+    print(f"saved {args.output}")
+
+
+if __name__ == "__main__":
+    main()

qwen35-9b-verilog-thinking-reinforced-lora/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}