NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.
| Host | Duration | Avg Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo |
|---|---|---|---|---|---|---|---|
| ncm-dbt-01 | 02:27:41 | 581722 | 1460 | 146 653 661 | -125.88 ± 8.1 | 24 474 217 15 0 | -276.47 ± 23.15 |
| ncm-dbt-02 | 02:27:36 | 586086 | 1466 | 117 657 692 | -134.29 ± 8.16 | 27 503 186 17 0 | -301.24 ± 25.07 |
| ncm-dbt-03 | 02:27:15 | 585932 | 1458 | 116 650 692 | -133.45 ± 8.09 | 28 491 197 13 0 | -297.34 ± 24.33 |
| ncm-dbt-04 | 02:27:17 | 565376 | 1460 | 128 652 680 | -130.51 ± 7.87 | 23 489 207 11 0 | -292.17 ± 23.69 |
| ncm-dbt-05 | 02:26:39 | 583239 | 1446 | 115 641 690 | -132.45 ± 8.04 | 25 489 196 13 0 | -296.57 ± 24.4 |
| 7290 | 622 3253 3415 | -131.3 ± 3.6 | 127 2446 1003 69 0 | -292.61 ± 10.75 | |||
| ID | Host | Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo | CLI | PGN | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 445628 | ncm-dbt-05 | 584243 | 446 | 31 204 211 | -142.21 ± 14.92 | 12 152 56 3 0 | -316.78 ± 46.13 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 711138298 \
-pgnout ncm-dbt-20210512-1513-015.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445627 | ncm-dbt-03 | 583572 | 458 | 39 195 224 | -123.26 ± 13.65 | 4 152 69 4 0 | -277.77 ± 41.31 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3417855820 \
-pgnout ncm-dbt-20210512-1513-014.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445626 | ncm-dbt-01 | 581361 | 460 | 44 196 220 | -119.28 ± 14.65 | 7 144 73 6 0 | -257.84 ± 40.18 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3648777802 \
-pgnout ncm-dbt-20210512-1513-013.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445625 | ncm-dbt-04 | 561757 | 460 | 42 208 210 | -131.29 ± 13.86 | 7 155 65 3 0 | -295.48 ± 42.6 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 203968830 \
-pgnout ncm-dbt-20210512-1513-012.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445624 | ncm-dbt-02 | 588217 | 466 | 38 205 223 | -130.29 ± 14.37 | 8 156 64 5 0 | -289.62 ± 43.07 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3727979113 \
-pgnout ncm-dbt-20210512-1513-011.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445623 | ncm-dbt-05 | 581985 | 500 | 46 215 239 | -122.24 ± 13.3 | 5 164 76 5 0 | -273.0 ± 39.35 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 835756394 \
-pgnout ncm-dbt-20210512-1513-010.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445622 | ncm-dbt-03 | 587325 | 500 | 36 223 241 | -136.56 ± 14.45 | 11 172 60 7 0 | -304.07 ± 44.5 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3729657238 \
-pgnout ncm-dbt-20210512-1513-009.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445621 | ncm-dbt-01 | 583028 | 500 | 51 235 214 | -134.15 ± 13.05 | 7 173 67 3 0 | -306.84 ± 41.99 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 455643329 \
-pgnout ncm-dbt-20210512-1513-008.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445620 | ncm-dbt-04 | 566257 | 500 | 41 225 234 | -134.15 ± 14.33 | 12 165 68 5 0 | -293.29 ± 41.75 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 621728681 \
-pgnout ncm-dbt-20210512-1513-007.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445619 | ncm-dbt-02 | 584622 | 500 | 46 226 228 | -130.94 ± 14.22 | 8 172 62 8 0 | -293.29 ± 43.72 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2976353395 \
-pgnout ncm-dbt-20210512-1513-006.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445618 | ncm-dbt-05 | 583489 | 500 | 38 222 240 | -134.15 ± 13.61 | 8 173 64 5 0 | -304.07 ± 43.1 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 933442374 \
-pgnout ncm-dbt-20210512-1513-005.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445617 | ncm-dbt-03 | 586901 | 500 | 41 232 227 | -139.81 ± 13.83 | 13 167 68 2 0 | -309.64 ± 41.56 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 530048307 \
-pgnout ncm-dbt-20210512-1513-004.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445616 | ncm-dbt-02 | 585421 | 500 | 33 226 241 | -141.44 ± 13.78 | 11 175 60 4 0 | -321.19 ± 44.57 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2671640215 \
-pgnout ncm-dbt-20210512-1513-003.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445615 | ncm-dbt-04 | 568116 | 500 | 45 219 236 | -126.17 ± 12.68 | 4 169 74 3 0 | -288.06 ± 39.79 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2557367799 \
-pgnout ncm-dbt-20210512-1513-002.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 445614 | ncm-dbt-01 | 580779 | 500 | 51 222 227 | -123.81 ± 14.32 | 10 157 77 6 0 | -265.78 ± 39.12 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 296640993 \
-pgnout ncm-dbt-20210512-1513-001.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20210512-1513 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| Commit ID | 594e2ac9994f22cb7c5b74f3da3dbb512cbdb510 |
|---|---|
| Author | bmc4 |
| Date | 2021-05-12 15:13:52 UTC |
|
Simplify LMR rule for non-checking captures
We simplify away the complicated rule in LMR for "non-checking captures
likely to be bad", as it seems not to bring any strength anymore.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 55256 W: 4972 L: 4897 D: 45387
Ptnml(0-2): 177, 3976, 19234, 4077, 164
https://tests.stockfishchess.org/tests/view/609adf3b3a33eb67a844f842
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 10344 W: 437 L: 353 D: 9554
Ptnml(0-2): 1, 322, 4449, 392, 8
https://tests.stockfishchess.org/tests/view/609b3dfa3a33eb67a844f88e
--
While at it, we also update the Elo estimate of the previous rule in LMR
(see https://tests.stockfishchess.org/tests/view/609af2a63a33eb67a844f867).
closes https://github.com/official-stockfish/Stockfish/pull/3460
Bench: 3840688
|
|