However longer inputs sometimes would generate errors. Longer inputs meant longer system prompts and longer responses.

The logs showed:

Analysis failed: expected ',' or '}' after object value

Claude was generating valid-looking JSON that wasn’t actually valid. A dropped comma deep in a large response object. The longer the response, the more likely this happened.

The Old Approach: Prompt Engineering + Defensive Parsing

My system prompt included a 28-line block demanding JSON output:

SYSTEM_PROMPT = <<~PROMPT
  ...
  CRITICAL INSTRUCTIONS:
  - You MUST ALWAYS respond with valid JSON. No exceptions. No explanations outside JSON.
  - NEVER respond with plain text - always use the JSON format.

  You MUST ALWAYS respond in JSON format with the following structure (no exceptions):
  {
    "score": 0-100,
    "level": "low|medium|high",
    "summary": "Brief overall assessment",
    "items": [
      {
        "title": "Issue name",
        "description": "What's wrong",
        "severity": "low|medium|high"
      }
    ],
    "recommendations": [
      "Specific actionable suggestion 1",
      "Specific actionable suggestion 2"
    ]
  }
PROMPT

Despite all the shouting in the prompt, Claude would sometimes:

• Wrap the JSON in markdown code fences (```json ... ```)
• Add explanatory text after the closing brace
• Drop commas in deeply nested objects on long responses
• Return plain text when it decided the input wasn’t suitable for analysis

So I built a pipeline of defensive code to handle all of this.

Step 1: Extract JSON from whatever Claude returned. A brace-matching parser that stripped markdown fences, found the first {, tracked nesting depth while respecting string escaping, and separated trailing notes:

def extract_json_and_note(text)
  text = text.strip
  if text.start_with?("```")
    text = text.sub(/A```(?:json|JSON)?s*/, "").sub(/s*```z/, "").strip
  end

  start_idx = text.index("{")
  return [text, nil, false] if start_idx.nil?

  brace_count = 0
  in_string = false
  escape_next = false

  text[start_idx..].each_char.with_index do |char, idx|
    if escape_next
      escape_next = false
      next
    end
    case char
    when "\" then escape_next = true if in_string
    when '"'  then in_string = !in_string unless escape_next
    when "{"  then brace_count += 1 unless in_string
    when "}"
      brace_count -= 1 unless in_string
      if brace_count == 0
        end_idx = start_idx + idx
        json_text = text[start_idx..end_idx]
        note = text[(end_idx + 1)..].strip.presence
        return [json_text, note, true]
      end
    end
  end

  [text, nil, false]
end

Step 2: Normalize missing fields because Claude might omit arrays for edge cases:

def normalize_response!(result)
  result["score"] ||= 0
  result["level"] ||= "unknown"
  result["summary"] ||= "Analysis complete"
  result["items"] ||= []
  result["recommendations"] ||= []
  result["score"] = result["score"].to_i if result["score"].is_a?(String)
end

Step 3: Validate the structure because even after parsing, I couldn’t trust it:

def validate_response_structure!(result)
  required_keys = %w[score level summary items recommendations]
  missing_keys = required_keys - result.keys
  raise "Invalid response structure: missing keys #{missing_keys.join(', ')}" if missing_keys.any?

  score = result["score"]
  unless score.is_a?(Integer) && score >= 0 && score <= 100
    raise "Invalid score: must be integer 0-100, got #{score.inspect}"
  end

  %w[items recommendations].each do |key|
    unless result[key].is_a?(Array)
      raise "Invalid #{key}: expected array, got #{result[key].class}"
    end
  end
end

All of this existed because I was asking an LLM to format its own output as JSON via natural language instructions. I was writing a fragile parser for a format the model was never constrained to produce.

The Fix: Tool Use

Anthropic’s tool use API (also called function calling) lets you define a JSON schema that Claude must conform to. Instead of asking Claude to output JSON, you tell the API: “call this function with these typed parameters.” Claude’s response is guaranteed to match the schema.

Here’s the schema definition:

ANALYSIS_TOOL = {
  name: "analyze",
  description: "Return the structured analysis results",
  input_schema: {
    type: "object",
    required: ["score", "level", "summary", "items", "recommendations"],
    properties: {
      score: { type: "integer", description: "Overall score from 0 (safe) to 100 (dangerous)" },
      level: { type: "string", enum: ["low", "medium", "high"] },
      summary: { type: "string", description: "Brief overall assessment" },
      items: {
        type: "array",
        items: {
          type: "object",
          required: ["title", "description", "severity"],
          properties: {
            title:       { type: "string" },
            description: { type: "string" },
            severity:    { type: "string", enum: ["low", "medium", "high"] }
          }
        }
      },
      recommendations: { type: "array", items: { type: "string" } }
    }
  }
}.freeze

The API call adds two parameters:

response = client.messages.create(
  model: model,
  max_tokens: max_tokens,
  system: [{ type: "text", text: system_prompt }],
  messages: [{ role: "user", content: user_message }],
  tools: [ANALYSIS_TOOL],
  tool_choice: { type: "tool", name: "analyze" }
)

tools: defines the schema. tool_choice: with type: "tool" forces Claude to use it — no chance of returning prose instead.

Response extraction is three lines:

tool_block = response.content.find { |b| b.type.to_s == "tool_use" }
raise "No tool_use block in response" unless tool_block
result = tool_block.input.transform_keys(&:to_s)

That’s it. tool_block.input is already a parsed hash. No JSON.parse, no brace matching, no markdown stripping, no comma repair.

The Result

Deleted: ~160 lines from the service, ~250 lines from tests..

Added: ~30 lines for the schema definition, 2 parameters on the API call, 3 lines of response extraction.

The system prompt shrank too. The 28 lines of “YOU MUST RESPOND IN JSON” instructions disappeared entirely. The prompt now focuses on what to analyze, not how to format the output.

The user message went from "Analyze this and respond with JSON only:" to just "Analyze this:".

When Should You Use This?

Any time you want structured output from an LLM. If you’re writing regex to fix JSON commas, building brace-matching parsers, or adding “RESPOND IN JSON ONLY” to your prompts: switch to tool use. The schema is self-documenting, the output is guaranteed valid, and you delete code instead of writing it.

The one caveat: tool use constrains the structure but not the content. Claude can still put whatever it wants in a string field. You still need to validate that a score is in a sensible range or that enum values match your expectations. But “validate the values” is a much smaller problem than “parse arbitrary text that might be JSON.”

Why I Built This

I’ve signed a enough contracts over the years, and I was never quite sure if I was missing something important buried in the legal language. I’d skim through them, but let’s be honest – I didn’t understand half of it. Getting a lawyer to review every contract is also just not something I’m going to do, unless it’s something really big.

I figured: AI is pretty good at reading and understanding text now. And based on how many of these AI contract analysis tools are out there, it’s the new TODO app in the age of AI.

The Stack

I went with a Rails 8 monolith because I wanted something I could ship quickly and maintain solo:

• Rails 8 with Hotwire (Turbo + Stimulus)
• Anthropic’s Claude API
• Solid Queue for background jobs with priority queues (paid users get faster processing)
• Solid Cache for caching and rate limits
• Stripe for subscriptions and billing
• Tesseract OCR for extracting text from scanned images (JPG, PNG, WebP, HEIC)
• Kamal for deployment

Everything runs in Docker containers. No separate frontend framework, no microservices. Just a straightforward Rails app that does one thing well.

Technical Security Challenges

1. File Processing Security
   • Magic byte validation – Don’t trust file extensions. I check the actual file signature to verify it’s really a PDF or DOCX.
   • Size limits – DOCX files are zip archives, so I enforce size limits before decompression to prevent zip bombs.
   • Immediate deletion – Original files are deleted right after text extraction. No long-term storage of sensitive documents.
   • Command injection prevention – Only use safe extraction tools, never shell out with user-provided filenames.
2. Server Access
   • Firewall at the provider level
   • Firewall at the node level (ufw)
   • ssh through certs only, limit access to specific IP’s
   • Cloudflare
3. Application Security
   • Devise authentication – Industry-standard auth framework
   • CSRF protection – Rails CSRF tokens on all POST/PUT/PATCH/DELETE requests
   • UUID-based URLs – Guest contracts use UUIDs (prevents enumeration attacks)
   • Rate Limits (Rack::Attack)
   • CSP Policy
     • No unsafe-eval – Prevents eval() attacks
     • Whitelisted script sources – Only self, HTTPS, Stripe, Cloudflare allowed
     • No object embeds – object_src :none blocks Flash/plugin attacks
     • Nonce-based scripts – Importmap scripts use session-based nonces
     • HTTPS enforced – All resources loaded over HTTPS
   • Input Validation
4. Fraud Prevention
   • Email history tracking – SHA256 email hashing – Email hashes stored, not plain emails
5. Payment Security
   • Stripe webhook verification – Signature validation on all webhook events
   • No card storage – Stripe handles all payment details
6. Secret Management
   • Rails credentials – All secrets in encrypted credentials.yml.enc
7. XSS Prevention
   • Automatic HTML escaping
   • CSP headers – Content Security Policy blocks inline scripts
8. Transport Security
   • HTTPS everywhere – All resources loaded over HTTPS
   • Secure cookies – Session cookies marked secure in production
   • HSTS headers – Forces HTTPS connections
9. DoS Prevention
   • Job queues – Background processing prevents request timeouts
   • Priority queues – Paid users get separate high-priority queue
   • Rate limiting – Comprehensive rate limits across all endpoints
   • Query optimization – Indexed queries prevent slow lookups

Try it

Demo (no signup): https://clausyapp.com/contracts/new?demo=hn

Full app: https://clausyapp.com

It’s not legal advice – I’m very explicit about that – but it can help you spot things you might want to ask a lawyer about.