> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chunkr.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Processing

> Process documents with LLMs

Chunkr AI API allows you to configure the LLMs that will be used to process the documents.
The LLMs configuration is applied to your segments during the `segment_processing` step, click [here](./segment-processing) to learn more.

This is how you can configure the LLMs:

```python theme={"system"}
llm_processing=LlmProcessing(
    llm_model_id="chunkr-parse-1-thinking",
    fallback_strategy=FallbackStrategy.model("chunkr-parse-1"),
    max_completion_tokens=4096,
    temperature=0.0
)
```

## LLM Processing Options

The `LlmProcessing` configuration controls which language models are used for processing segments and provides fallback strategies if the primary model fails.

| Field                   | Type             | Description                                                                                        | Default                             |
| ----------------------- | ---------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------- |
| `llm_model_id`          | String           | The ID of the model to use for processing. If not provided, the system default model will be used. | [System default](#available-models) |
| `fallback_strategy`     | FallbackStrategy | Strategy to use if the primary model fails.                                                        | [System default](#available-models) |
| `max_completion_tokens` | Integer          | Maximum number of tokens to generate in the model response.                                        | None                                |
| `temperature`           | Float            | Controls randomness in model responses (0.0 = deterministic, higher = more random).                | 0.0                                 |

## Fallback Strategies

When working with language models, reliability is important. Chunkr provides three fallback strategies to handle cases when your primary model fails:

* `FallbackStrategy.none()`: No fallback will be used. If the primary model fails, the operation will return an error.
* `FallbackStrategy.default()`: Use the system default fallback model.
* `FallbackStrategy.model("model-id")`: Specify a particular model ID to use as a fallback. This gives you explicit control over which alternative model should be used.

## Example Usage

Here's how to configure LLM processing in different scenarios:

<CodeGroup>
  ```python Simple Configuration theme={"system"}
  from chunkr_ai import Chunkr
  from chunkr_ai.models import (
      Configuration,
      LlmProcessing,
      FallbackStrategy
  )

  chunkr = Chunkr()

  # Use Gemini Pro 2.5 with no fallback strategy
  config = Configuration(
      llm_processing=LlmProcessing(
          llm_model_id="gemini-pro-2.5",
          fallback_strategy=FallbackStrategy.none(),
          temperature=0.0
      )
  )

  chunkr.upload("path/to/file", config)
  ```

  ```python With Fallback Model theme={"system"}
  from chunkr_ai import Chunkr
  from chunkr_ai.models import (
      Configuration,
      LlmProcessing,
      FallbackStrategy
  )

  chunkr = Chunkr()

  # Use Claude 3.7 Sonnet with Gemini Flash 2.0 as fallback
  config = Configuration(
      llm_processing=LlmProcessing(
          llm_model_id="claude-3.7-sonnet",
          fallback_strategy=FallbackStrategy.model("gemini-flash-2.0"),
          max_completion_tokens=4096,
          temperature=0.2
      )
  )

  chunkr.upload("path/to/file", config)
  ```

  ```bash cURL theme={"system"}
  curl -X POST \
      --url https://api.chunkr.ai/api/v1/task/parse \
      --header "Authorization: YOUR_API_KEY" \
      --header "Content-Type: application/json" \
      --data '{
          "file": "base64_or_url_to_file",
          "llm_processing": {
              "fallback_strategy": {"model": "gemini-flash-2.0"},
              "max_completion_tokens": 4096,
              "llm_model_id": "gemini-flash-2.0",
              "temperature": 0
          }
      }'
  ```
</CodeGroup>

## Available Models

The following models are currently available for use with Chunkr:

<iframe
  srcDoc={`
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Available LLM Models</title>
    <style>
        body {
            font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
            margin: 0;
            padding: 20px;
            color: #333;
        }
        table {
            width: 100%;
            border-collapse: collapse;
            margin-bottom: 20px;
            border-radius: 8px;
            font-size: 14px;
        }
        th {
            background: #f5f5f5;
            padding: 10px;
            text-align: left;
            border-bottom: 1px solid #ddd;
            font-weight: 600;
        }
        td {
            padding: 10px;
            border-bottom: 1px solid #ddd;
        }
        .centered {
            text-align: center;
        }
        .model-id {
            font-family: monospace;
            background: #f7f7f7;
            padding: 2px 4px;
            border-radius: 4px;
            display: inline-block;
            vertical-align: middle;
        }
        .badge {
            display: inline-block;
            padding: 2px 8px;
            border-radius: 12px;
            font-size: 12px;
            font-weight: 500;
        }
        .badge-blue {
            background-color: #dbeafe;
            color: #1e40af;
        }
        .badge-gray {
            background-color: #f3f4f6;
            color: #6b7280;
        }
        .tag-default {
            background-color: #dcfce7;
            color: #166534;
            font-size: 12px;
            padding: 2px 8px;
            border-radius: 12px;
            display: inline-block;
            margin-left: 8px;
        }
        .tag-fallback {
            background-color: #ffedd5;
            color: #9a3412;
            font-size: 12px;
            padding: 2px 8px;
            border-radius: 12px;
            display: inline-block;
            margin-left: 8px;
        }
        #loading {
            text-align: center;
            padding: 20px;
            color: #666;
        }
        #error {
            text-align: center;
            padding: 20px;
            color: #ef4444;
            background: #fee2e2;
            border-radius: 4px;
        }
        .copy-button {
            background: transparent;
            border: none;
            cursor: pointer;
            padding: 4px;
            margin-left: 6px;
            border-radius: 4px;
            transition: background-color 0.2s;
            vertical-align: middle;
        }
        .copy-button:hover {
            background-color: #e9e9e9;
        }
        .copy-button svg {
            width: 16px;
            height: 16px;
            fill: #666;
        }
        .tooltip {
            position: relative;
            display: inline-block;
        }
        .tooltip .tooltiptext {
            visibility: hidden;
            width: 60px;
            background-color: #333;
            color: #fff;
            text-align: center;
            border-radius: 4px;
            padding: 5px;
            position: absolute;
            z-index: 1;
            bottom: 125%;
            left: 50%;
            margin-left: -30px;
            opacity: 0;
            transition: opacity 0.3s;
            font-size: 12px;
        }
        .tooltip .tooltiptext.show {
            visibility: visible;
            opacity: 1;
        }
    </style>
</head>
<body>
    <div id="loading">Loading available models...</div>
    <div id="error" style="display: none;"></div>
    <table id="models-table" style="display: none;">
        <thead>
            <tr>
                <th>Model ID</th>
                <th>Status</th>
            </tr>
        </thead>
        <tbody id="models-body">
        </tbody>
    </table>

    <script>
        // API endpoint
        const apiUrl = 'https://api.chunkr.ai/llm/models';
        
        async function fetchModels() {
            const loadingEl = document.getElementById('loading');
            const errorEl = document.getElementById('error');
            const tableEl = document.getElementById('models-table');
            const tableBodyEl = document.getElementById('models-body');
            
            try {
                const response = await fetch(apiUrl);
                if (!response.ok) {
                    throw new Error(\`API request failed: \${response.status}\`);
                }
                
                const models = await response.json();
                
                // Hide loading, show table
                loadingEl.style.display = 'none';
                tableEl.style.display = 'table';
                
                // Sort models alphabetically by ID
                models.sort((a, b) => a.id.localeCompare(b.id));

                // Populate table
                models.forEach(model => {
                    const row = document.createElement('tr');
                    
                    // Model ID cell
                    const idCell = document.createElement('td');
                    const idCode = document.createElement('span');
                    idCode.className = 'model-id';
                    idCode.textContent = model.id;
                    idCell.appendChild(idCode);
                    
                    // Copy button with icon
                    const copyButton = document.createElement('button');
                    copyButton.className = 'copy-button tooltip';
                    copyButton.setAttribute('data-model-id', model.id);
                    copyButton.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M16 1H4c-1.1 0-2 .9-2 2v14h2V3h12V1zm3 4H8c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h11c1.1 0 2-.9 2-2V7c0-1.1-.9-2-2-2zm0 16H8V7h11v14z"/></svg>';
                    
                    // Tooltip for copy feedback
                    const tooltip = document.createElement('span');
                    tooltip.className = 'tooltiptext';
                    tooltip.textContent = 'Copied!';
                    copyButton.appendChild(tooltip);
                    
                    // Add event listener to copy button
                    copyButton.addEventListener('click', function() {
                        const modelId = this.getAttribute('data-model-id');
                        const tooltipEl = this.querySelector('.tooltiptext');
                        
                        // Copy to clipboard
                        navigator.clipboard.writeText(modelId).then(function() {
                            // Show tooltip
                            tooltipEl.classList.add('show');
                            
                            // Hide tooltip after 2 seconds
                            setTimeout(function() {
                                tooltipEl.classList.remove('show');
                            }, 2000);
                        }).catch(function() {
                            tooltipEl.textContent = 'Failed!';
                            tooltipEl.classList.add('show');
                            
                            setTimeout(function() {
                                tooltipEl.classList.remove('show');
                                tooltipEl.textContent = 'Copied!';
                            }, 2000);
                        });
                    });
                    
                    idCell.appendChild(copyButton);
                    
                    // Status cell - shows badges for default and fallback status
                    const statusCell = document.createElement('td');
                    
                    // Create badge container 
                    const badgeContainer = document.createElement('div');
                    
                    if (model.default) {
                        const defaultTag = document.createElement('span');
                        defaultTag.className = 'tag-default';
                        defaultTag.textContent = 'System Default';
                        badgeContainer.appendChild(defaultTag);
                    }
                    
                    if (model.fallback) {
                        const fallbackTag = document.createElement('span');
                        fallbackTag.className = 'tag-fallback';
                        fallbackTag.textContent = 'System Fallback';
                        badgeContainer.appendChild(fallbackTag);
                    }
                    
                    if (!model.default && !model.fallback) {
                        const availableTag = document.createElement('span');
                        availableTag.className = 'badge badge-gray';
                        availableTag.textContent = 'Available';
                        badgeContainer.appendChild(availableTag);
                    }
                    
                    statusCell.appendChild(badgeContainer);
                    
                    // Add cells to row
                    row.appendChild(idCell);
                    row.appendChild(statusCell);
                    
                    // Add row to table
                    tableBodyEl.appendChild(row);
                });
                
            } catch (err) {
                // Show error
                loadingEl.style.display = 'none';
                errorEl.style.display = 'block';
                errorEl.textContent = \`Error loading models: \${err.message}\`;
            }
        }
        
        // Fetch models when page loads
        fetchModels();
    </script>
</body>
</html>
`}
  style={{
width: "100%",
height: "600px",
border: "1px solid #eaeaea",
borderRadius: "8px",
overflow: "hidden"
}}
/>

> **Note**: This table is dynamically generated by fetching data from our API. Model availability may change over time.