> ## Documentation Index
> Fetch the complete documentation index at: https://docs.decisional.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Working with Sources

> Master document processing, RAG, and context management

## What Are Sources?

**Sources provide context to your agents** - they're documents, files, and links that help your agent make informed decisions.

<Card title="The Power of Context" icon="lightbulb">
  Without sources, agents only know what's in their instructions. With sources, agents can reference thousands of pages of documentation, policies, examples, and data.
</Card>

***

## Why Sources Matter

<CardGroup cols={2}>
  <Card title="Domain Knowledge" icon="brain">
    Upload company policies, SOPs, pricing guides - your agent becomes an expert
  </Card>

  <Card title="Examples" icon="copy">
    Show your agent past quotes, emails, reports - it learns your style
  </Card>

  <Card title="Live Data" icon="globe">
    Monitor competitor websites, documentation, news - stay up to date
  </Card>

  <Card title="Verification" icon="shield-check">
    Agents cite specific pages and sections - you can verify their reasoning
  </Card>
</CardGroup>

***

## Types of Sources

Decisional supports a wide variety of source types:

### Document Sources

<Tabs>
  <Tab title="PDFs">
    **Best for:** Contracts, invoices, forms, reports, manuals

    **Supported features:**

    * Text extraction (native PDF text)
    * OCR (scanned documents and images)
    * Tables and structured data
    * Multi-page documents
    * Bounding box citations

    **File size:** Up to 50MB per file

    **Example use cases:**

    * Upload past quotes so agent can match your formatting
    * Upload pricing catalogs for product lookup
    * Upload contracts for data extraction
    * Upload SOPs for policy compliance
  </Tab>

  <Tab title="Office Documents">
    **Microsoft Word (.docx, .doc):**

    * Policies and procedures
    * Templates
    * Reports

    **Microsoft Excel (.xlsx, .xls):**

    * Price lists
    * Historical data
    * Product catalogs
    * Reference tables

    **PowerPoint (.pptx):**

    * Training materials
    * Process diagrams

    **File size:** Up to 50MB per file
  </Tab>

  <Tab title="Images">
    **Formats:** PNG, JPG, JPEG, GIF, TIFF

    **Best for:**

    * Receipts (with OCR)
    * Forms (with OCR)
    * Diagrams and charts
    * Handwritten notes (limited OCR support)

    **Features:**

    * Automatic OCR (text extraction)
    * Visual understanding (charts, diagrams)
    * Quality enhancement

    **File size:** Up to 20MB per image
  </Tab>

  <Tab title="Text Files">
    **Formats:** TXT, CSV, JSON, XML, Markdown

    **Best for:**

    * Code snippets
    * Configuration files
    * Structured data
    * Raw text data

    **File size:** Up to 10MB per file
  </Tab>
</Tabs>

### Web Sources

<AccordionGroup>
  <Accordion title="Web Links (URLs)" icon="link">
    **How it works:** Decisional fetches the webpage content and indexes it.

    **Best for:**

    * Company websites
    * Documentation pages
    * Blog posts
    * Product pages
    * Competitor pricing

    **Features:**

    * Automatic refresh (daily, weekly, or custom)
    * Change detection (notify when page changes)
    * JavaScript rendering (for dynamic sites)

    **Example:**

    ```
    Add: https://example.com/pricing
    Refresh: Daily
    Agent instruction: "Monitor competitor pricing and update our prices to match"
    ```
  </Accordion>

  <Accordion title="API Endpoints" icon="code">
    **How it works:** Decisional calls your API and uses the response as context.

    **Best for:**

    * Live data from internal systems
    * Third-party API data
    * Real-time information

    **Authentication supported:**

    * API keys
    * Bearer tokens
    * OAuth 2.0

    **Example:**

    ```
    Add API: https://api.example.com/customers
    Auth: Bearer token
    Agent instruction: "Fetch customer data from API before generating quote"
    ```
  </Accordion>
</AccordionGroup>

***

## Adding Sources to Your Agent

<Steps>
  <Step title="Navigate to Sources">
    In your agent page, click **Sources** in the left sidebar
  </Step>

  <Step title="Choose Source Type">
    Click **Add Source** and select:

    * **Upload File** (PDF, Word, Excel, images, text)
    * **Add Link** (webpage URL)
    * **Connect Integration** (Google Drive, OneDrive, etc.)
  </Step>

  <Step title="Upload or Configure">
    **For files:** Drag and drop or browse to select

    **For links:** Paste the URL and set refresh frequency

    **For integrations:** Authorize and select files/folders
  </Step>

  <Step title="Name Your Source">
    Give it a descriptive name (e.g., "Q4 2024 Price List" instead of "pricelist\_v3\_final.xlsx")

    **Why:** You'll reference this name in agent instructions
  </Step>

  <Step title="Wait for Processing">
    Decisional indexes your source. This takes:

    * Small PDFs: 10-30 seconds
    * Large documents: 1-3 minutes
    * Web pages: 30-60 seconds

    You'll see a progress indicator.
  </Step>

  <Step title="Reference in Instructions">
    Tell your agent when to use this source:

    ```
    Reference the "Q4 2024 Price List" to look up product prices.
    ```
  </Step>
</Steps>

***

## How RAG Works

Decisional uses **Retrieval-Augmented Generation (RAG)** - a state-of-the-art technique for document understanding.

### The RAG Pipeline

<Steps>
  <Step title="Indexing">
    When you upload a document:

    1. **Text extraction**: Extract all text from the document
    2. **Chunking**: Split into semantic chunks (paragraphs, sections)
    3. **Embedding**: Convert text into vector representations
    4. **Indexing**: Store in a semantic search index
  </Step>

  <Step title="Retrieval">
    When your agent runs:

    1. **Query**: Agent formulates a search query based on its task
    2. **Search**: Find the most relevant chunks from your sources
    3. **Ranking**: Rank chunks by relevance
    4. **Selection**: Select top K chunks (typically 5-10)
  </Step>

  <Step title="Generation">
    Agent uses retrieved context:

    1. **Context**: Relevant chunks are provided to the agent
    2. **Reasoning**: Agent processes task using context
    3. **Citation**: Agent notes which sources it used
    4. **Output**: Agent generates result with citations
  </Step>
</Steps>

### Bounding Box Citations

For PDFs, Decisional provides **exact location citations**:

<CardGroup cols={3}>
  <Card title="Page Number" icon="file">
    Which page was referenced
  </Card>

  <Card title="Bounding Box" icon="square">
    Exact location on the page (highlighted region)
  </Card>

  <Card title="Quoted Text" icon="quote-left">
    The specific text the agent read
  </Card>
</CardGroup>

**Why this matters:**

* **Verification**: You can check the agent didn't hallucinate
* **Auditing**: Understand exactly what the agent learned
* **Compliance**: Prove decision-making is based on real data

<Info>
  Click any citation in the run logs to see the exact PDF location highlighted!
</Info>

***

## Optimizing Source Quality

Better sources = better agent performance.

### Document Preparation Tips

<AccordionGroup>
  <Accordion title="Use Text-Based PDFs (Not Scans)" icon="file-pdf">
    **Best:** Native PDF with selectable text

    **Okay:** Scanned PDF (Decisional will OCR it, but slower and less accurate)

    **How to check:** Try to select text in the PDF. If you can, it's text-based.

    **If you only have scans:**

    * Use high-resolution scans (300 DPI minimum)
    * Ensure good contrast
    * Avoid skewed or rotated pages
  </Accordion>

  <Accordion title="Remove Irrelevant Content" icon="scissors">
    **Remove:**

    * Cover pages with no useful info
    * Blank pages
    * Unrelated appendices
    * Repetitive boilerplate

    **Why:** Reduces noise, improves retrieval accuracy, speeds up processing

    **How:** Edit the PDF before uploading or use page range selection (coming soon)
  </Accordion>

  <Accordion title="Use Clear Formatting" icon="align-left">
    **Good structure:**

    * Clear headers and sections
    * Numbered lists
    * Tables with headers
    * Consistent fonts

    **Avoid:**

    * Text in images (unless necessary)
    * Watermarks that obscure text
    * Heavy redactions
    * Multi-column layouts (if possible)
  </Accordion>

  <Accordion title="Split Large Documents" icon="split">
    **Instead of:** One 500-page policy manual

    **Do this:** Separate PDFs for each policy (Pricing, Returns, Shipping, etc.)

    **Why:**

    * Faster processing
    * More accurate retrieval (agent knows which document to search)
    * Easier to update individual policies

    **Exception:** If the document is highly interconnected, keep it as one file.
  </Accordion>

  <Accordion title="Use Descriptive Filenames" icon="tag">
    **Bad:**

    * `document.pdf`
    * `final_FINAL_v3.pdf`
    * `IMG_2024.png`

    **Good:**

    * `Acme_Corp_Pricing_Policy_2024_Q4.pdf`
    * `Invoice_Template_Standard.pdf`
    * `Product_Catalog_Hardware.pdf`

    **Why:** Helps you and the agent identify the right source
  </Accordion>
</AccordionGroup>

***

## Source Organization Strategies

### By Workflow Type

<Tabs>
  <Tab title="Quote Generation">
    **Sources to upload:**

    1. **Price Lists** (Excel or PDF)
       * Current pricing
       * Discount tiers
       * Product catalog

    2. **Quote Templates** (PDF)
       * Past successful quotes
       * Shows formatting, tone, structure

    3. **Pricing Policy** (PDF or Word)
       * Rules for discounts
       * Approval thresholds
       * Special terms

    **Agent instruction:**

    ```
    Reference "2025 Price List" for product prices.
    Use "Quote Template" as formatting guide.
    Follow rules in "Pricing Policy" for discounts.
    ```
  </Tab>

  <Tab title="Invoice Processing">
    **Sources to upload:**

    1. **Vendor List** (Excel)
       * Vendor names
       * Contact info
       * Payment terms

    2. **Chart of Accounts** (Excel or PDF)
       * GL codes
       * Category mappings

    3. **Processing SOP** (PDF or Word)
       * Step-by-step instructions
       * Validation rules
       * Edge case handling

    **Agent instruction:**

    ```
    Extract invoice data using "Processing SOP" guidelines.
    Validate vendors against "Vendor List".
    Assign GL codes from "Chart of Accounts".
    ```
  </Tab>

  <Tab title="Contract Extraction">
    **Sources to upload:**

    1. **Example Contracts** (PDFs)
       * Similar contract types
       * Shows where key terms are located

    2. **Field Definitions** (Word or PDF)
       * What each field means
       * How to interpret clauses

    3. **Legal Terms Glossary** (PDF)
       * Define complex legal terms

    **Agent instruction:**

    ```
    Extract key terms from contract.
    Reference "Field Definitions" for what to extract.
    Use "Legal Terms Glossary" to understand complex clauses.
    Compare structure to "Example Contracts".
    ```
  </Tab>

  <Tab title="CRM Enrichment">
    **Sources to upload:**

    1. **Company Website Links**
       * Add URLs of target companies
       * Agent scrapes key info

    2. **Industry Reports** (PDFs)
       * Market research
       * Competitor analysis

    3. **Ideal Customer Profile** (PDF or Word)
       * Defines what good leads look like

    **Agent instruction:**

    ```
    For each lead, visit their website and extract:
    - Company size
    - Industry
    - Products/services
    - Key contacts

    Score leads against "Ideal Customer Profile".
    Enrich with insights from "Industry Reports".
    ```
  </Tab>
</Tabs>

***

## Advanced RAG Techniques

### Multi-Document Reasoning

Agents can combine information from multiple sources:

**Example workflow:**

```
Agent instruction:
1. Look up product specs from "Product Catalog"
2. Find pricing from "Price List"
3. Check availability from "Inventory Report"
4. Verify compliance with "Export Regulations"
5. Generate quote combining all information
```

The agent will retrieve relevant sections from each document and synthesize them.

### Temporal Sources

For time-sensitive information, set refresh schedules:

<Steps>
  <Step title="Add Web Link">
    Add a URL (e.g., competitor pricing page)
  </Step>

  <Step title="Set Refresh Schedule">
    Choose frequency:

    * Hourly (for real-time monitoring)
    * Daily (for regular updates)
    * Weekly (for less frequent changes)
    * Manual (refresh when needed)
  </Step>

  <Step title="Enable Change Detection">
    Get notified when the page changes
  </Step>
</Steps>

**Use case:**

```
Monitor https://competitor.com/pricing daily.
If their price drops below ours, send Slack notification.
```

### Versioned Sources

Keep multiple versions of the same source:

**Example:**

* `Price List 2024 Q1.xlsx`
* `Price List 2024 Q2.xlsx`
* `Price List 2024 Q3.xlsx`
* `Price List 2024 Q4.xlsx` (current)

**Agent instruction:**

```
Use "Price List 2024 Q4" for new quotes.
Reference historical price lists to show price changes over time.
```

***

## Integration-Based Sources

Connect live data sources through integrations:

<Tabs>
  <Tab title="Google Drive">
    **Setup:**

    1. Connect Google Drive integration
    2. In Sources, click **Add from Google Drive**
    3. Select files or folders
    4. Choose sync frequency

    **Features:**

    * Two-way sync (changes in Drive appear in Decisional)
    * Folder monitoring (new files auto-added)
    * Permissions respected (only files you have access to)

    **Use case:**

    ```
    Monitor "Sales Templates" folder in Google Drive.
    When new template is added, agent learns the new format.
    ```
  </Tab>

  <Tab title="OneDrive / SharePoint">
    **Setup:**

    1. Connect Microsoft 365 integration
    2. In Sources, click **Add from OneDrive**
    3. Select files or SharePoint sites

    **Features:**

    * Corporate SharePoint support
    * Team site access
    * Version history

    **Use case:**

    ```
    Sync all files from "Legal/Contracts" SharePoint folder.
    Agent stays up to date with latest contract templates.
    ```
  </Tab>

  <Tab title="Email Attachments">
    **Setup:**

    1. Connect Gmail or Outlook integration
    2. In Sources, create an email rule
    3. Define which emails and attachments to monitor

    **Features:**

    * Filter by sender, subject, labels
    * Extract specific file types
    * Archive processed emails

    **Use case:**

    ```
    When email arrives from vendor@supplier.com with subject "Invoice":
    1. Extract PDF attachment
    2. Add as source to agent
    3. Agent processes invoice
    ```
  </Tab>

  <Tab title="Databases">
    **Setup:**

    1. Connect database integration (PostgreSQL, MySQL, Snowflake)
    2. Write a query to fetch data
    3. Schedule query execution

    **Features:**

    * SQL query support
    * Parameterized queries
    * Scheduled refresh

    **Use case:**

    ```sql theme={null}
    SELECT product_id, product_name, price, stock
    FROM products
    WHERE active = true

    Refresh: Every hour
    Agent uses results as product catalog.
    ```
  </Tab>
</Tabs>

***

## Managing Sources

### Source Library

View all sources across your workspace:

<Steps>
  <Step title="Navigate to Sources">
    Click **Sources** in the main sidebar (not agent-specific)
  </Step>

  <Step title="View All Sources">
    See all uploaded documents, links, and connected files
  </Step>

  <Step title="Organize">
    * Search by name
    * Filter by type (PDF, link, integration)
    * Sort by upload date, size, usage
  </Step>

  <Step title="Reuse">
    Attach existing sources to new agents (no need to re-upload)
  </Step>
</Steps>

### Source Actions

<AccordionGroup>
  <Accordion title="Update a Source" icon="rotate">
    **For files:** Upload new version (overwrites old one)

    **For links:** Click **Refresh Now** to re-fetch

    **For integrations:** Sync automatically based on schedule

    All agents using this source immediately see the update.
  </Accordion>

  <Accordion title="Delete a Source" icon="trash">
    **Warning:** Deleting a source affects all agents using it.

    **To delete:**

    1. Click the source
    2. Click **Delete**
    3. Confirm

    **What happens:** Agents can no longer reference this source. Existing citations remain in run history.
  </Accordion>

  <Accordion title="View Source Usage" icon="chart-simple">
    See which agents use a source:

    1. Click the source
    2. View **Used By** tab
    3. See list of agents

    **Why useful:** Before deleting, check if any agents depend on it.
  </Accordion>

  <Accordion title="Download Original" icon="download">
    Retrieve the original file you uploaded:

    1. Click the source
    2. Click **Download Original**

    **Why useful:** Recover original if you've lost it.
  </Accordion>
</AccordionGroup>

***

## Troubleshooting Sources

<AccordionGroup>
  <Accordion title="Source won't upload" icon="circle-exclamation">
    **Common causes:**

    1. **File too large** (limit: 50MB for PDFs, 20MB for images)
       * Solution: Split into smaller files or compress

    2. **Unsupported format**
       * Solution: Convert to supported format (e.g., DOC → DOCX)

    3. **Corrupted file**
       * Solution: Re-export or re-save the file

    4. **Network timeout**
       * Solution: Try again with better internet connection
  </Accordion>

  <Accordion title="Agent isn't using my source" icon="face-confused">
    **Debugging steps:**

    1. **Check source is attached to agent**
       * Go to Sources tab in agent page
       * Verify source is listed

    2. **Reference source in instructions**
       * Explicitly mention: "Use \[source name] to find..."

    3. **Verify source is indexed**
       * Check for "Indexed" status (not "Processing")

    4. **Check source quality**
       * Is text extractable? Try selecting text in PDF
       * Is content relevant to agent's task?

    5. **Review run logs**
       * See if agent attempted to search source
       * Check retrieval results
  </Accordion>

  <Accordion title="Wrong information retrieved" icon="triangle-exclamation">
    **Problem:** Agent cites irrelevant sections

    **Solutions:**

    1. **Improve source quality**
       * Remove irrelevant pages
       * Split into focused documents

    2. **Be more specific in instructions**
       * Instead of: "Look up pricing"
       * Do: "Find product X in 'Price List' and return the 'Unit Price' column value"

    3. **Add examples**
       * Show agent what good results look like

    4. **Check for conflicting sources**
       * Multiple sources with similar content can confuse agent
       * Remove duplicates or be explicit which to use
  </Accordion>

  <Accordion title="OCR isn't working" icon="image">
    **Problem:** Agent can't read scanned PDFs or images

    **Solutions:**

    1. **Use higher resolution** (300 DPI minimum)
    2. **Improve contrast** (black text on white background is best)
    3. **Straighten pages** (no skew or rotation)
    4. **Avoid handwriting** (OCR works best on printed text)
    5. **Try native PDF** (if possible, use text-based PDF instead of scan)

    If still failing, contact support - we can help with challenging OCR cases.
  </Accordion>
</AccordionGroup>

***

## Best Practices

<Check>
  **Quality over quantity:** 10 highly relevant documents > 100 generic documents
</Check>

<Check>
  **Name descriptively:** "2025\_Q1\_Pricing" > "doc\_v3\_final.pdf"
</Check>

<Check>
  **Keep sources updated:** Set refresh schedules or upload new versions regularly
</Check>

<Check>
  **Reference explicitly:** Tell agents which sources to use and when
</Check>

<Check>
  **Verify citations:** Check agent's cited sources in run logs to ensure accuracy
</Check>

<Check>
  **Organize logically:** Group related sources, delete outdated ones
</Check>

<Check>
  **Test with edge cases:** Upload challenging documents to see how well RAG handles them
</Check>

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Agent Sources" icon="brain" href="/agents/sources">
    Deep dive into retrieval techniques and optimization
  </Card>

  <Card title="Working with Agents" icon="robot" href="/guides/agents">
    Learn how to write instructions that use sources effectively
  </Card>

  <Card title="Integrations Guide" icon="plug" href="/guides/integrations">
    Connect Google Drive, OneDrive, and other source providers
  </Card>

  <Card title="Workflows" icon="diagram-project" href="/workflows/overview">
    Advanced patterns for document workflows
  </Card>
</CardGroup>