Skip to content
Portada » News » Optimisation Proposals for SPRT Testing in Fastchess

Optimisation Proposals for SPRT Testing in Fastchess

Revolution

Optimisation Proposals for SPRT Testing in Fastchess

The proposals below outline realistic optimisations for the Revolution engine. Each entry specifies the repository and a suggested tag to keep a compilable snapshot suitable for SPRT testing with Fastchess.

ProposalObjectiveKey ChangesRepository / TagPrimary MetricRisks & Mitigations
1. Adaptive Reductions in PVSReduce failures in tactical positions while maintaining speed. Adjust Late Move Reductions in src/search.cpp by introducing an adaptive factor based on recent evaluation volatility, calibrated via Stack fields. revolution
tag: sprt-lmr-adaptive-v1
Elo gain at blitz (5+0.1), then confirmation at LTC 60+0.1.Overfitting risk: validate with tests/puzzles; monitor TT stability for artefacts.
2. Vectorised Feature EvaluationsLower NNUE evaluation latency. Rewrite accumulation core in src/nnue/nnue_accumulator.cpp using AVX2 intrinsics, with SSE2 fallback controlled from src/nnue/nnue_common.h. revolution
tag: sprt-nnue-avx2-v1
NPS and Elo in fast tests.Compatibility: CPU autodetect in src/misc.cpp (CPU::init()); CI on x86 without AVX2.
3. Persistent Killer-Move CacheImprove pruning consistency. Persist killers across main iterations in src/movepick.cpp, with depth limits and cleanup on root change or after failed null move. revolution
tag: sprt-killer-cache-v1
Average reached depth (ply) and Elo.Contamination risk: ensure Stack::killers resets; add assertions in debug builds.
4. Automatic Tuning of Evaluation ParametersRefine heuristics without manual intervention. Integrate scripts/tune_eval.py to adjust weights in src/evaluate.cpp and src/tune.cpp via stochastic gradient methods over assets/selfplay. revolution-tuning
tag: sprt-eval-tuning-v1
RMSE vs dataset and subsequent Elo.Dataset overfit: train/validation split; repeat SPRT after each tuning cycle.

Recommended Configuration for Proposal 1 (Adaptive LMR)

  1. STC (Short Time Control): 10s + 0.1s, ~2k rounds with concurrency=2 for a quick signal.
  2. LTC (Long Time Control): 60s + 0.1s on the same tagged binary if STC yields positive SPRT acceptance.
Launcher (Windows, STC phase)
Batch script
@echo off
setlocal EnableExtensions EnableDelayedExpansion
title FastChess SPRT - Live (Elo/LLR/LOS)

rem ====== PATHS ======
set "FASTCHESS=C:\fastchess\fastchess.exe"
set "DIR_DEV=C:\fastchess\revolution-device"
set "ENGINE_DEV=%DIR_DEV%\revolution-PVS.exe"
set "DIR_BASE=C:\fastchess\revolution-baseline"
set "ENGINE_BASE=%DIR_BASE%\revolution-2.90-241025.exe"
set "BOOK=C:\fastchess\Books\UHO_Lichess_4852_v1.epd"
set "OUTDIR=C:\fastchess\out"

rem ====== PARAMETERS ======
set "TC=10+0.1"
set "THREADS=1"
set "HASH=64"
set "CONCURRENCY=2"
set "ROUNDS=2000"
set "GAMES=2"
set "REPEAT=1"
set "MAXMOVES=200"
set "ELO0=0"
set "ELO1=2.5"
set "ALPHA=0.05"
set "BETA=0.05"
set "RATINGINT=10"
set "SRAND=12345"

rem RECOVER:
rem   0 -> clean run (recommended if any protocol element changed)
rem   1 -> attempt resume if relaunching with the SAME log/PGN paths
set "RECOVER=0"

if not exist "%OUTDIR%" mkdir "%OUTDIR%"

rem ====== TIMESTAMP ======
for /f "tokens=1-6 delims=/:. " %%a in ("%date% %time%") do set TS=%%f%%e%%d_%%a%%b%%c
set "TS=%TS: =0%"
set "LOGTXT=%OUTDIR%\fc_live_%TS%.log"
set "PGNOUT=%OUTDIR%\fc_live_%TS%.pgn"

rem ====== QUICK CHECKS ======
if not exist "%FASTCHESS%" echo [ERR] FASTCHESS not found: "%FASTCHESS%" & goto :end
if not exist "%ENGINE_DEV%" echo [ERR] ENGINE_DEV not found: "%ENGINE_DEV%" & goto :end
if not exist "%ENGINE_BASE%" echo [ERR] ENGINE_BASE not found: "%ENGINE_BASE%" & goto :end
if not exist "%BOOK%" echo [ERR] BOOK not found: "%BOOK%" & goto :end

rem ====== -recover FLAG FROM RECOVER ======
set "RECOVER_FLAG="
if "%RECOVER%"=="1" set "RECOVER_FLAG=-recover"

rem ====== LIVE TAIL (PowerShell) ======
start "SPRT Live (Elo/LLR/LOS)" powershell -NoLogo -NoExit -Command ^
  "$p='%LOGTXT%'; while(-not (Test-Path $p)){ Start-Sleep -Milliseconds 200 }; " ^
  "Get-Content $p -Wait | Select-String -Pattern '(?i)Elo|LLR|LOS|SPRT|Results of|Ptnml|PairsRatio|Score|rating'"

echo ====== COMMAND (SPRT) ======
echo "%FASTCHESS%" ^
 -engine cmd="%ENGINE_DEV%"  name=DEV  dir="%DIR_DEV%"  option.Threads=%THREADS% option.Hash=%HASH% option.Ponder=false option.MultiPV=1 option."Move Overhead"=80 option."Slow Mover"=100 option."Minimum Thinking Time"=100 ^
 -engine cmd="%ENGINE_BASE%" name=BASE dir="%DIR_BASE%" option.Threads=%THREADS% option.Hash=%HASH% option.Ponder=false option.MultiPV=1 option."Move Overhead"=80 option."Slow Mover"=100 option."Minimum Thinking Time"=100 ^
 -each tc=%TC% proto=uci ^
 -openings file="%BOOK%" format=epd order=random ^
 -games %GAMES% -rounds %ROUNDS% -repeat %REPEAT% -maxmoves %MAXMOVES% -concurrency %CONCURRENCY% ^
 -sprt elo0=%ELO0% elo1=%ELO1% alpha=%ALPHA% beta=%BETA% ^
 -ratinginterval %RATINGINT% -srand %SRAND% %RECOVER_FLAG% -report penta=true ^
 -pgnout "%PGNOUT%" -log file="%LOGTXT%" level=info
echo.

rem ====== LAUNCH FASTCHESS ======
"%FASTCHESS%" ^
 -engine cmd="%ENGINE_DEV%"  name=DEV  dir="%DIR_DEV%"  option.Threads=%THREADS% option.Hash=%HASH% option.Ponder=false option.MultiPV=1 option."Move Overhead"=80 option."Slow Mover"=100 option."Minimum Thinking Time"=100 ^
 -engine cmd="%ENGINE_BASE%" name=BASE dir="%DIR_BASE%" option.Threads=%THREADS% option.Hash=%HASH% option.Ponder=false option.MultiPV=1 option."Move Overhead"=80 option."Slow Mover"=100 option."Minimum Thinking Time"=100 ^
 -each tc=%TC% proto=uci ^
 -openings file="%BOOK%" format=epd order=random ^
 -games %GAMES% -rounds %ROUNDS% -repeat %REPEAT% -maxmoves %MAXMOVES% -concurrency %CONCURRENCY% ^
 -sprt elo0=%ELO0% elo1=%ELO1% alpha=%ALPHA% beta=%BETA% ^
 -ratinginterval %RATINGINT% -srand %SRAND% %RECOVER_FLAG% -report penta=true ^
 -pgnout "%PGNOUT%" -log file="%LOGTXT%" level=info

:end
echo.
echo [LOG ] "%LOGTXT%"
echo [PGN ] "%PGNOUT%"
echo --- END. Press any key to close ---
pause

Recommended Options for STC 10+0.1

  • Threads = 1: reproduce Stockfish reference model; avoid SMP scaling interactions.
  • Hash = 64 MB: balanced; consider 128 MB only if both binaries support it.
  • Ponder = false: less noise in pairings; avoids stalls on early finishes.
  • Move Overhead = 80 ms, Minimum Thinking Time = 100 ms: mitigate platform lag at short TCs.
  • Slow Mover = 100: Stockfish-standard aggressiveness; adjust ±10 only if STC is unstable.
  • concurrency = 2: adequate for ~2k rounds without saturating 4-thread CPUs.
  • elo0 = 0, elo1 = 2.5, alpha = 0.05, beta = 0.05: symmetric SPRT settings for small-regression detection.
For the LTC 60+0.1 phase, reuse the same launcher and set TC=60+0.1, ROUNDS=1200 (≈1,200 games with repeat=1), and—if hardware permits—Hash=256.

Suggested Workflow

  1. Create a dedicated branch per proposal; apply changes and tag the stable commit with the indicated tag.
  2. Compile the engine for that tag; upload the binary to your Fastchess infrastructure to begin SPRT against master.
  3. Record logs and a result summary (win/draw/loss, LLR, estimated Elo) under docs/sprt-results/.
  4. If the threshold is passed (e.g., LLR ≥ 2.0), merge into master; otherwise iterate with small adjustments.

Additional Resources

  • Fastchess SPRT Guide — parameters for alpha, beta, elo0, elo1.
  • scripts/fastchess_runner.sh — recommended wrapper to invoke Fastchess with homogeneous parameters (not included).
  • Run tests/perft/ and tests/regression/ before any SPRT to validate functional stability.

Jorge Ruiz

Jorge Ruiz

connoisseur of both chess and anthropology, a combination that reflects his deep intellectual curiosity and passion for understanding both the art of strategic chess books

Leave a Reply

Your email address will not be published. Required fields are marked *

Share via