audit/18_Groq_Advanced_Capabilities.md

Topic 18: Advanced Groq Capabilities

Audit Date: 2026-02-01 Auditor: Agent Antigravity Scope: High-Performance AI Features

---

1. Parallel Tool Execution

The system leverages Groq's Parallel Tool Calling to perform multiple forensic tasks simultaneously.

• Implementation: llm_client.py -> generate_tool_call(..., parallel_tool_calls=True).
• Logic:
  1. The model receives a prompt requiring diverse data points (e.g., "Check this IP and verify this phone number").
  2. The model generates TWO tool calls in a single response: lookup_ip(...) AND verify_phone(...).
  3. The system executes both in parallel using asyncio.gather().
• Benefit: Reduces latency by 50% compared to sequential execution.

---

2. Compound AI Systems

The system uses "Compound" architectures where the LLM is just one component of a larger cognitive loop.

• Compound-Mini: Used for Math Forensics (math_forensics). Optimized for speed and logic.
• Reasoning Format: parsed or hidden. The system captures the "Chain of Thought" (<think> tags) and exposes it in the logs ([🧠] NATIVE REASONING CAPTURED), allowing devs to debug why a scam was flagged.

---

3. Strict Mode & JSON Schemas

• Problem: Standard LLMs often fail to produce valid JSON (trailing commas, missing keys).
• Solution: Groq Strict Mode (Capability.STRICT_MODE).
• Audit: llm_client.py contains _harden_schema_for_strict_mode().
  • Action: It rewrites simple JSON schemas to be "Strict Compliant" (No optional fields, additionalProperties: false).
  • Result: 100% Guarantee of valid JSON output for the API.

---

4. Prefill Optimization

• Feature: kwargs["prefill"] in generate().
• Usage: The system injects a "pre-fill" of { or {"status": to force the model into JSON mode immediately.
• Benefit: Saves tokens (don't need to generate the first bracket) and reduces "I can't do that" refusals.