Validation Methodology: How We Measure 92.7% Geopolitical Prediction Accuracy

Why This Page Exists

We claim 92.7% domain accuracy across 137 validated historical events. That number appears in our meta description, our landing page, and our blog posts. This page explains exactly how we computed it, what it means, what it does not mean, and how you can reproduce it.

We publish this because an unsubstantiated accuracy claim is worse than no claim at all. If you find a flaw in our methodology, we want to know: ask@anvayajyotish.com.

What "92.7% Domain Accuracy" Means

For each historical event in our corpus, we ask: did the engine correctly identify which domain this event belongs to?

• A war should flag the war domain
• An economic crisis should flag the economy domain
• A coup or election turnover should flag the regime_change domain
• A border change should flag the territory domain
• Protests or civil disorder should flag the civil_unrest domain

We do NOT claim to predict the exact date, the exact severity, or the specific actors involved. Domain prediction is the first and most fundamental layer: what kind of event is the chart signaling?

92.7% means: out of 137 events where we had a national chart, the engine correctly identified the domain in 127 cases.

The Walk-Forward Framework

Walk-forward validation is the gold standard for testing predictive systems. The principle: you cannot use future data to predict the past.

Here is how our validation works:

1. Take a historical event (e.g., "Russia invades Ukraine, February 24, 2022")
2. Set the clock back 3 years (to February 2019)
3. Compute the national chart of the affected country (Ukraine, independence August 24, 1991)
4. Run the full engine from the 2019 perspective: 96 mundane rules across 5 domains, transit analysis, dasha period identification
5. Check: did the engine flag the correct domain (war) as elevated?

If the engine's war severity score exceeds the activation threshold from the 2019 vantage point, it is scored as a domain hit. If not, it is a miss.

We repeat this for every event in the corpus. The lookback period is configurable (default: 3 years).

The Event Corpus

Our validation corpus contains 141 historical events spanning from 1526 (Battle of Panipat I) to 2024 (US Presidential Election). Sources:

• Wars: Correlates of War Project (University of Michigan), UCDP/PRIOS Armed Conflict Dataset
• Regime changes: Polity IV / V-Dem (regime change data)
• Economic crises: World Bank recession dating, historical records
• Territory changes: Historical records, treaty databases
• Civil unrest: Historical records, news archives

Each event has:

• Exact UTC date
• Primary country code (ISO 3166)
• Domain classification (war, economy, regime_change, territory, civil_unrest)
• Severity rating (ordinary, notable, exceptional, legendary)
• Outcome description
• Source citation

Of the 141 events, 137 have matching national charts in our catalog. The remaining 4 use historical chart codes (e.g., Mughal Empire, Tokugawa Japan) that could not be validated.

The Rule Engine

The engine evaluates 96 rules across 5 domains. Each rule checks a specific astrological configuration in the national chart:

War domain (27 rules): Mars in 7th house (+0.35), Mars-Saturn conjunction (+0.35), Rahu in 7th (+0.30), 7th lord in dusthana (+0.25), Mars aspects 7th (+0.25), 6th lord in kendra (+0.20), and 21 more. Peace indicators subtract: Jupiter in 7th (-0.30), Venus in 7th (-0.20).

Economy domain (22 rules): 2nd lord placement, Jupiter's condition, Saturn in 2nd/8th, Mercury afflictions, Venus in dusthana, and more. Positive rules (prosperity indicators) and negative rules (crisis indicators) are scored separately.

Regime change domain (20 rules): Sun afflictions, 10th lord placement, Rahu/Ketu in 1st/10th, Saturn in 10th, Mars in 10th, and dissolution-specific rules (10th lord in 12th, Sun in 12th).

Territory domain (18 rules): 4th house afflictions (Ketu in 4th, 4th lord in 12th), Rahu in 4th, Saturn aspects, and territory gain indicators.

Civil unrest domain (18 rules): Moon afflictions (Mars/Saturn/Rahu aspecting Moon), malefics in 1st/4th, Mars-Rahu conjunction, Saturn retrograde.

Each rule has a severity weight derived from classical mundane astrology texts (BV Raman, KN Rao, Raphael, Baigent-Campion).

Scoring Details

For each event, the engine computes a severity score per domain (additive sum of fired rule weights, range roughly -1 to +1). This is converted to a probability via sigmoid scaling:

probability = 1 / (1 + exp(-3 * (severity - 0.3)))

A domain is flagged as "predicted" if its probability exceeds 0.3 (30%). Additionally, the domain with the highest absolute severity score is always included in the predicted set (fallback for edge cases where all domains score below threshold).

A domain hit is recorded if the actual event's domain appears in the predicted domain set.

Per-Domain Results

Out-of-Sample Validation: 90.0% (20 Events, 2020-2025)

The in-sample result (92.7%) measures how well the rules fit the corpus they were developed alongside. To test genuine predictive ability, we ran 20 recent events (2020-2025) that were not part of the original corpus and were not used during rule development. The rules are codified from classical mundane astrology texts (BV Raman, KN Rao, Raphael), not fitted to data.

The 2.7 percentage point drop from in-sample (92.7%) to out-of-sample (90.0%) is small, which suggests the rules are derived from principles rather than overfit to the corpus. If the rules had memorized the training data, the out-of-sample accuracy would collapse to 60-70%.

Out-of-sample misses:

• Russia-Ukraine War (2022): Ukraine's 1991 independence chart was born in a peaceful transition and has no natal war markers. The war came from Russian transits, not Ukraine's natal configuration. This is a structural limitation: the current rules check natal positions only.
• Brazil January 6 (2023): Brazil's chart flags civil unrest (correct for the protests) but not regime change specifically. The attempted insurrection was both — the engine captured the unrest dimension but missed the regime-change dimension.

Out-of-sample hits include:

• COVID economic crash (US, 2020) — economy flagged at 94% probability
• Iran Mahsa Amini protests (2022) — civil unrest flagged at 71%
• 2024 US Election — regime change flagged at 79%
• Sri Lanka economic collapse (2022) — economy flagged at 88%
• Bangladesh Hasina ousted (2024) — regime change flagged at 43%
• South Korea martial law (2024) — regime change flagged at 32%

Known Limitations

1. Domain accuracy is not event prediction. Correctly identifying "war risk is elevated for this country" is not the same as predicting a specific war. The engine detects conditions, not events.

2. Timing accuracy is 24.8%. The engine correctly identifies WHEN an event occurs within a dasha-transit window only 24.8% of the time. This uses the supercomputer's dasha-transit fusion, which is functional but not yet calibrated.

3. Magnitude accuracy is 15.3%. Predicting whether an event is "notable" vs "exceptional" vs "legendary" is the weakest layer.

4. Regime change is the weakest domain at 76%. Most misses are events where the national chart uses the wrong era (e.g., French Revolution 1789 validated against the 5th Republic 1958 chart). These are data problems, not rule failures.

5. The Ukraine 2022 miss. The single war miss. Ukraine's 1991 independence chart was born in a peaceful transition and lacks natal war markers. The war came from transits (Russia's actions), not Ukraine's natal configuration. This reveals a structural limitation: the current rules check natal positions, not transit overlays.

6. Single miss in "crisis" domain. September 11, 2001 is classified as "crisis" (a meta-domain), which the engine does not model as a separate domain. It was flagged under war, economy, and regime change -- but not "crisis" specifically.

7. Out-of-sample test completed but limited. We ran 20 recent events (2020-2025) not used during rule development, achieving 90.0% domain accuracy (see section above). However, 20 events is a small sample. A statistically rigorous out-of-sample test requires 50+ held-out events. We plan to expand the out-of-sample corpus with the next phase.

How to Reproduce

The validation endpoint is live:

GET /api/v1/geo-validate?lookback=3

This runs the full walk-forward validation against all 137 events with a 3-year lookback. You can filter by country:

GET /api/v1/geo-validate?country=US&lookback=3

The response includes per-event domain_hit, timing_hit, magnitude_hit, predicted_domains, and convergence_score.

What We Are Working On

• Transit-overlay rules: Adding transit-based war detection to catch events like Ukraine 2022 where the natal chart is peaceful but transits are hostile
• Historical chart switching: Using era-appropriate charts (French Monarchy for 1789, not 5th Republic)
• Out-of-sample validation: Withholding 20% of events during rule development
• Corpus expansion: Target 500+ events for statistical significance
• Timing calibration: Using walk-forward results to tune dasha-transit window detection

Contact

Questions about the methodology: ask@anvayajyotish.com

The validation framework, event corpus, and rule engine are part of the Anvaya Geo codebase. We welcome scrutiny.

Explore the Geopolitical Engine→