Search Engine Settings

LogZilla uses an advanced search engine for full-text indexing and searching of log events. The search engine configuration is managed through the sphinx.yaml file and provides extensive customization options for indexing behavior, performance tuning, and search capabilities.

Critical Warning: Modifying search engine settings can cause system instability, performance degradation, or data loss. Always test changes in a non-production environment first and create backups before making modifications. Contact LogZilla support if unsure about any setting.

Search Engine Configuration

Search settings are stored in /etc/logzilla/settings/sphinx.yaml and can be managed using the modern logzilla settings command.

Basic Configuration

bash
Wrap: OnCopyFullscreen
# View current search settings
logzilla settings list sphinx

# Update search settings
logzilla settings update SPHINX_MIN_WORD_LENGTH=3
logzilla settings update SPHINX_MIN_INFIX_LENGTH=3

# Apply changes (automatic with settings update)
logzilla settings reload sphinx

Core Indexing Settings

Word and Text Processing

Wildcard Search Capabilities

Default Behavior: LogZilla supports suffix wildcards (start*) by default.

Infix Wildcards: To enable prefix (*end) and middle (*middle*) wildcard searches, configure infix indexing:

bash
Wrap: OnCopyFullscreen
# Enable infix searching (matches SPHINX_MIN_WORD_LENGTH)
logzilla settings update SPHINX_MIN_INFIX_LENGTH=4

Example: With SPHINX_MIN_INFIX_LENGTH=3, the word "testing" generates indexes for: tes, est, sti, tin, ing, test, esti, stin, ting, testi, estin, sting, testin, esting, and testing.

Performance Warning: Infix indexing increases index size and can significantly impact performance. Use values below 4 only when necessary and with adequate system resources.

Data Safety: Changes to indexing settings only affect new data. Existing data retains its original indexing. Incorrect settings may cause search failures or system instability.

Performance and Scaling Settings

Index Management

Concurrent Processing

Query Limits

Advanced Configuration

Debug and Troubleshooting

bash
Wrap: OnCopyFullscreen
# Enable searchd debug mode
logzilla settings update SPHINX_SEARCHD_DEBUG=true

# Enable searchd logging
logzilla settings update SEARCHD_LOGS_ENABLED=true

Optimization Guidelines

For High-Volume Environments

bash
Wrap: OnCopyFullscreen
# Increase batch processing
logzilla settings update SPHINX_MIN_INDEX_LEN=50000
logzilla settings update SPHINX_MAX_INDEXING_TIME=300

# Optimize concurrent processing
logzilla settings update SPHINX_REINDEX_PROC_MAX=4
logzilla settings update SPHINX_MERGING_PROC_MAX=2

# Increase reindex delay to reduce load
logzilla settings update SPHINX_REINDEX_DELAY=30

For Search-Heavy Workloads

bash
Wrap: OnCopyFullscreen
# Increase query result limits
logzilla settings update SPHINX_MAX_MATCHES=2000000

# Increase concurrent processing
logzilla settings update SPHINX_REINDEX_PROC_MAX=4
logzilla settings update SPHINX_MERGING_PROC_MAX=4

For Storage-Constrained Systems

bash
Wrap: OnCopyFullscreen
# Disable infix indexing to save space
logzilla settings update SPHINX_MIN_INFIX_LENGTH=0

# Increase minimum word length
logzilla settings update SPHINX_MIN_WORD_LENGTH=5

# Disable temporary index splitting
logzilla settings update SPHINX_DISABLE_AUTO_SPLITTING=true

Important Considerations

Critical Safety Information

Warning: Search engine settings directly affect system performance and stability. Incorrect configurations can cause:

• Complete search functionality failure
• Excessive memory usage leading to system crashes
• Data processing delays or failures
• Inability to access historical data

Always consult LogZilla support before modifying these settings in production.

Index Rebuilding

Important: Changes to word length and infix settings only apply to new events. Existing events retain their original indexing. Modifying these settings may require system maintenance windows and can affect search performance during the transition period.

Performance Impact

Memory Usage: Infix indexing can increase memory usage by 3-10x depending on text content and minimum lengths.

Disk Space: Index size grows significantly with shorter minimum lengths and infix indexing enabled.

Search Speed: More indexed terms can slow search performance, especially for common terms.

Recommended Settings by Use Case

Standard Deployment:

bash
Wrap: OnCopyFullscreen
SPHINX_MIN_WORD_LENGTH=4
SPHINX_MIN_PREFIX_LENGTH=4
SPHINX_MIN_INFIX_LENGTH=0  # Disabled for performance

Advanced Search Requirements:

bash
Wrap: OnCopyFullscreen
SPHINX_MIN_WORD_LENGTH=3
SPHINX_MIN_PREFIX_LENGTH=3
SPHINX_MIN_INFIX_LENGTH=3  # Enable with caution

High-Performance Environment:

bash
Wrap: OnCopyFullscreen
SPHINX_MIN_WORD_LENGTH=5
SPHINX_MIN_PREFIX_LENGTH=5
SPHINX_MIN_INFIX_LENGTH=0  # Disabled
SPHINX_MIN_INDEX_LEN=100000
SPHINX_REINDEX_DELAY=60

Troubleshooting

Common Issues

Slow Searches: Reduce infix length or disable infix indexing.

High Memory Usage: Increase minimum word lengths and disable infix indexing.

Missing Search Results: Check minimum word length settings and ensure terms meet the configured thresholds.

Indexing Delays: Adjust concurrent processing limits and batch sizes.

Monitoring

bash
Wrap: OnCopyFullscreen
# Check indexing status and statistics
logzilla events stats

Troubleshooting Search Issues

For search-related problems, check the main LogZilla log:

bash
Wrap: OnCopyFullscreen
# View recent log entries
tail -f /var/log/logzilla/logzilla.log

# Search for indexing errors
grep -i "sphinx\|search\|index" /var/log/logzilla/logzilla.log

Note: Advanced search engine diagnostics require container access and should only be performed by experienced administrators or under LogZilla support guidance.