Debugging Empty LLM Tool Call Replies and Retry Loops

When building robust production systems with Large Language Models, latency and token economics get plenty of attention. However, silent failures like empty replies during function calling often slip past initial benchmarks. In a recent test of 300 public function-calling tasks, Mistral Small returned an empty response 58 times—even after two retries. This deep-dive explores why this happens, how retry loops inflate your true API costs, and how developers can handle these edge cases efficiently using robust infrastructure like n1n.ai.

The Cost of Silent Failures in Tool Calling

On paper, budget-friendly models look incredibly attractive. When an LLM is priced attractively per million input tokens, developers naturally assume predictable unit economics. But the hidden multiplier in production is the retry loop.

Consider what happens when a model fails to output a readable string. Production code cannot automatically know if a semantic answer is correct, but it can easily detect an empty string or malformed JSON. Therefore, harnesses implement a retry loop. Every single attempt—whether it succeeds or returns a blank slate—is billed by the provider and recorded by your logging layer.

In our benchmark run across 300 rows and nine models:

• Mistral Small executed 434 attempts for 300 tasks, resulting in 134 retries.
• 58 tasks exhausted all three allocated attempts and still ended with zero output.
• 116 out of 134 retries were spent entirely on tasks that never recovered.
• Retries accounted for 23.9% of the model's total spend.

While its baseline cost looked low, the actual cost per successful task jumped significantly due to these wasted tokens. Meanwhile, models like GPT-6 Luna passed 93.0% of tasks on the first try with zero retries, showcasing how stability often outweighs nominal token pricing.

Anatomy of the Retry Harness

To understand how these costs accumulate, let's look at how a standard evaluation and retry harness handles JSON parsing and attempt tracking. A robust harness strictly separates unreadable outputs (which trigger retries) from incorrect outputs (which are treated as final).

function parseJsonLoose(text) {
  if (typeof text !== "string" || text.trim() === "") return { ok: false, reason: "empty output" };
  let t = text.trim();
  const fence = t.match(/```(?:json)?\\s*([\\s\\S]*?)```/i);
  if (fence) t = fence[1].trim();
  const a = t.indexOf("{");
  const z = t.lastIndexOf("}");
  if (a === -1 || z <= a) return { ok: false, reason: "no JSON object in output" };
  return { ok: true, data: t.substring(a, z + 1) };
}

The execution loop bills every single attempt, regardless of whether the output is parseable:

for (let a = 0; a <= config.retries; a++) {
  const res = await transport({
    model,
    messages,
    max_tokens: maxTokens,
    temperature: config.temperature,
    extra: extra && extra.body
  });
  const { cost, estimated } = res.ok ? attemptCost(res, model, prices) : { cost: 0, estimated: false };
  const rec = { ms: res.ms, cost, cost_estimated: estimated, status: res.status };
  
  const j = judge(config, res.text, row);
  rec.valid = j.valid;
  attempts.push(rec);
  
  if (j.valid) {
    last = { outcome: j.pass ? "pass" : "wrong", reason: j.reason, output: res.text };
    break;
  }
  last = { outcome: "invalid", reason: j.reason, output: res.text };
}

When routing high-volume requests through aggregators like n1n.ai, having transparent access to attempt latency and error status codes becomes vital for debugging these silent failures before they drain your budget.

Examining the Empty Replies

This behavior is not a universal indictment of cheap models; six out of the nine tested models had zero retries, while others had minimal occurrences. This points to specific model-prompt alignment issues.

Take row tc-2 as an example:

• Tools: generate_random_number(min, max), calculate_loan_payment(loan_amount, interest_rate, loan_term)
• User Prompt: Hi, I need a random number between 1 and 100.
• Expected Output: `{"name":"generate_random_number