Sunday, January 4, 2026

algebraic threat engine

Freedstorm YARA Analytic | Algebraic Threat Engine

Experimental Threat Engine

Hi everyone,

I am currently working on a project that I believe is very promising, but it should still be considered experimental at this stage. Thanks to the valuable feedback from initial users, many improvements have already been incorporated into this version.

Regarding the testing of Yara-Analytic: Due to the great response, we currently have enough testers to evaluate the tool under real-world conditions. I am now focusing on analyzing the data and further refining the technical aspects.

Thank you very much for your interest and support. I will let you know as soon as more testing slots or new updates become available.

# What is YARA Analytic?

This tool is no ordinary YARA scanner. It is an Algebraic Threat Engine (ATE). My goal was to build a tool that doesn't just blindly find threats but evaluates them mathematically to minimize false positives.

Important: This tool is not for beginners. It requires a solid understanding of how YARA rules, metadata, and pattern matching work.

# Core Features & Technical Architecture

# Freedstorm ATE

Utilizes Max-Aggregation across Severity, Weight, and Score. A hit is only classified as critical if the mathematical relevance reaches the defined threshold.

# Hardened Binary

Compiled with (C++). The scoring logic is protected against tampering. Thanks to Isolation Mode, the tool runs with absolute stability, independent of global Python libraries.

# Deep-Analysis

By providing detailed Hex offsets and a preview of the first 30 bytes of a hit, the engine enables precise forensic evaluation.

# Smart Timeouts

Adjustable timeouts (Default: 5s) prevent "ReDoS"-like YARA rules or manipulated files from stalling the scanning process.

# Filter Policy Logic

The tool implements a complex decision matrix:

  • Tag-Filtering: Use --require-tag or --deny-tag to focus on specific threat classes or mark them as "Safe".
  • Score-Requirement: With --score-required, only rules providing clean metadata are evaluated.
  • Algebra-Threshold: The --min-score N switch defines the red line between "Info" and "Malware".

ATE Calculation Example:

Freedstorm ATE Formula

Scoretotal = max({ M(Sev), W, Smeta })

• M(Sev): Severity Mapping (Critical=100, High=75, ...)

• W: Weight (Weighting factor from YARA metadata)

• Smeta: Score (Explicit threat value)

# Operation & Lifecycle

yara-analytic --help
usage: yara-analytic [--compile-output FILE] [--rules-bin FILE]
                     [-h] [--update] [--debug] [--test-rules]
                     [--scan-file FILE] [--scan-dir DIR]
                     [--rules-dir PATH] [--verbose] [--silent]
                     [--timeout TIMEOUT] [--min-score N]
                     [--score-required] [--allow-no-score]
                     [--require-tag TAG] [--deny-tag TAG]
                     [--deny-rule NAME]

Freedstorm YARA Analytic v1.7 - Algebraic Threat Engine (ATE)

options:
  -h, --help           Show this help message and exit.
  --compile-output     Compile rules from --rules-dir and save as binary file
  --rules-bin FILE     Path to a pre-compiled YARA binary (.yarc) for high-speed scans
  --update             Update rules (Git required)
  --debug              Generate debug report for troubleshooting
  --test-rules         Display all loaded YARA rules for verification
  --scan-file FILE     Perform deep-analysis scan on a single file
  --scan-dir DIR       Analyze directory recursively
  --rules-dir PATH     Path to YARA rules
                       (Default: /usr/local/share/yara-rules)
  --verbose, -v        Detailed analysis (shows offsets, strings, and tags)
  --silent, -s         Silent mode (suppresses status messages, only hits)
  --timeout TIMEOUT    Scan timeout (N) in seconds (Default: 5)

Algebra & Filter Options:
  --min-score N        Minimum Freedstorm Algebra score (Default: 50)
  --score-required     Only evaluate rules with explicit score metadata
  --allow-no-score     Treat rules without metadata as a risk as well
  --require-tag TAG    Only use rules with this specific tag
  --deny-tag TAG       Always treat rules with this tag as INFO (Safe)
  --deny-rule NAME     Exclude specific rule names from alerts
Lifecycle Management: To prevent the use of outdated engines, every build has a hard End-of-Life (EOL) date. Current versions ensure maximum detection rates.

# Freedstorm Algebra: How the Score is Generated

A conventional scanner immediately screams "Malware!" at every hit. The Freedstorm Algebra, however, acts like a digital judge.

The Calculation Method

During a scan, the engine automatically extracts three critical metadata points from the YARA rules:

  • Severity: How dangerous is the rule classified?
  • Weight: What is the weighting of the patterns?
  • Score: The threat value in the database.

The engine uses Max-Aggregation logic to calculate the actual relevance for the scan.

The Result: Relevance instead of Noise

  • Low Score (< 40): Mostly harmless characteristics.
  • High Score (> 70): High probability of genuine malicious code.

# Major Update: Version 1.2 & 1.3

Over the last few days, the engine has received a massive development boost. Thanks to feedback from initial testers, critical performance hurdles and rule integrity issues have been eliminated.

Threat-Intelligence Upgrade: Starting with v1.2, the engine automatically integrates the professional ReversingLabs repository. This provides over 2,100 qualified rule files for analysis.

Key Improvements at a Glance:

Feature / Improvement v1.1 v1.3
Rule Statistics on Load No Yes
Detailed Error Logging (Duplicates) No Yes
UTF-8 Safe String Preview No Yes
Automatic Archive Skip Counter No Yes
Integrated --test-rules Mode No Yes

# Technical Focus: Stability on Arch Linux

A special focus was placed on fixing the flex scanner error that could occur when processing large path structures on Arch systems. Additionally, the engine now supports External Variables (such as filename), ensuring compatibility with high-end rule sets like those from Florian Roth (Neo23x0).

In addition, v1.3 now proactively warns against "Over-filtering". If your chosen algebra parameters (Score/Tags) are so restrictive that potentially dangerous hits are masked, the tool will issue a corresponding notification.

# Major Update: Version 1.4 & 1.6

With the latest updates, the Freedstorm Engine has evolved from a pure analysis tool into a hardened security application. The focus was on protecting engine integrity and the seamless integration of professional threat intelligence.

Current Development:

Feature / Security Module v1.3 v1.6
Anti-Tamper (Time Manipulation Protection) No Multi-Layer
Full External Variables Support (extension, owner, etc.) Partial Full
Automatic Error Handling (Module Incompatibility) Basic Advanced
Integrated Debug Report Generator No Yes

# Technical Focus: Overcoming Module Hurdles

A breakthrough in v1.5 was the implementation of dynamic variable mapping. Previously, rules accessing external attributes like extension or filename often led to compilation errors. The Freedstorm ATE now provides these globally, allowing even the most complex signatures from the signature-base to function without manual adjustment.

Furthermore, v1.6 no longer delivers cryptic tracebacks on errors, but categorizes issues into understandable messages such as "Module Incompatibility" or "Syntax Error". For forensic analysis, a full report can be generated via --debug, summarizing all system references and error modes for the developer freedstorm.

# Latest Update: Version 1.7 - The Performance Leap

While the previous versions focused on integrity and stability, Version 1.7 introduces a massive performance optimization. By implementing a high-speed binary loading mechanism, the Freedstorm Engine has reached sub-second analysis times even with thousands of active rules.

# New Feature: Pre-Compiled Binary Support (.yarc)

The engine now supports the --compile-output and --rules-bin flags. This allows the developer to "bake" 1500+ rules from signature-base and reversinglabs into a single optimized bytecode file.

Performance Gain: Loading pre-compiled rules reduced the startup latency on an i5-10210U from ~5 seconds to under 100ms, enabling real-time deep scans without CPU spikes.

Comparison Table: Version Progression

Security & Performance Modules v1.4 v1.6 v1.7
Anti-Tamper Logic Initial Multi-Layer Multi-Layer
Module Incompatibility Handling Basic Advanced Optimized
High-Speed Binary Loading No No Yes (Instant)
Real-Time Monitor Integration Manual Scripted Direct / .yarc

This update makes the engine fully compatible with inotifywait monitors, as the scan now triggers almost at the exact moment the file write is completed. Currently, this version is being stressed against the latest Storm Infostealer (ClickFix) loaders to ensure detection remains consistent under high-speed conditions.

# Outlook & Testing Phase

I would appreciate any technical feedback.

Note: Binaries are only valid for a few months at a time to prevent the use of outdated experiments.

# Prerequisites

  • YARA: Native libraries installed
  • Python Dependencies: yara-python
  • VCS: git

📥 Download YARA Analytic v1.3 (Linux)

SHA256: 64f23bb76a602ca25a9d1f84ff741474285c09f17d11d46fce73ca34a0038f98
Currently unavailable – Testing phase temporarily paused for evaluation.

at