TCEC is based on voluntary work. Please consider a donation!
Please wait, the graph is loading
Please wait, the graph is loading
Please wait, the graph is loading
Please wait, the graph is loading
Please wait, the graph is loading
Crosstable: no live event
Schedule: no live event
What is TCEC?
TCEC (Thoresen Chess Engines Competition) is a computer chess tournament organized and maintained by Martin Thoresen in cooperation with Chessdom Arena. The goal is to provide the viewers with a live broadcast of long time control, quality chess - played strictly between computer chess engines created by different programmers. One Season is divided into several Stages and lasts about 3-4 months. The winner of the Season will be the TCEC Grand Champion.
The TCEC Grand Champions
Season 6: Stockfish | Season 5: Komodo | Season 4: Houdini | Season 3: N/A | Season 2: Houdini | Season 1: Houdini
Season 6 opening book notes from Nelson Hernandez (nick: Cato the Younger)
TCEC is a website intended to showcase computer chess programs and provide visitors with chess games of extraordinarily high quality on top-flight hardware at long time controls, eventually producing a Seasonal 'champion'.
There are a great number of ways one might structure a sequential-stage tournament with these intentions. Veteran observers of TCEC have no doubt noticed evolutions from one season to the next in various details. Martin's chief guiding principle in all these changes has been 'entertainment first'.
With respect to openings in particular, numerous individuals have advanced or supported different proposals in the past. Some favor no opening books at all ('let the engines play!') and others favor opening books of limited or unlimited size ('let the authors match wits!'). Some favor openings being selected from an extremely short opening book of two or three moves. Some favor thematic openings, i.e. playing an opening system repeatedly during a Stage.
All of the above approaches could result in an interesting competition. However, Martin decided that the best way to promote entertainment, showcase the engines, present a fair challenge to all programs and impose the least administrative burden on everyone involved was for all games to start from pre-selected opening positions that were produced in a GM-level game in the past.
Obviously the implementation details of such a plan are a paramount concern. On one hand you do not want one-sided opening positions where the same color will almost always win. On the other hand you do not want extremely draw-prone opening positions, particularly in Stages when very evenly-matched, high-quality programs are playing. What you are looking for are hundreds of openings that are consistently in-between. Further, in a single round-robin (as in Stages 1 and 2) the positions need to be very similar in terms of the degree to which each color is advantaged. Since the engines are getting to play only one side of a position fairness is critical. Your white player will be justifiably upset if he is presented with an opening position where black is objectively much better.
To find these hundreds of opening positions with a high degree of quantitative precision you need lots of empirical and evaluative data, i.e. you need many millions of games in your database and you need deep comparative evaluations across different high-quality engines. There is simply no other way to do it reliably. You have a classic 'Big Data' problem.
Martin invited me to tackle this problem last November and I took it on with enthusiasm. I had one major component already in hand-a world-class private database with tens of millions of human and engine games, all carefully de-duplicated and verified for ending outcomes and adjusted as necessary, which has been a daily work in progress of mine since 2004 and been contributed to by many other collectors and individuals over the years.
The second component, developing a list of candidate opening positions, turned out to be easy because it was already done. Adam Hair had an eight-move book compiled from Norm Pollock's database of GM-level games. After some refinement provided by Dariusz Orzechowski and Lucas Braesch, the book contained 44,624 unique positions achieved in GM-level games throughout history. Eight moves seemed ideal to me because it is long enough to span a much fuller range of ECOs than six or fewer moves. Nine or ten moves, it seemed to me, would be taking too much liberty away from the engines.
A third component was getting deep evaluations for the candidate positions. Adam Hair and I developed these evaluations from Houdini 4, Stockfish DD and Komodo TCECr for all of the candidate positions in two months, which was all the time we had to complete the task. For Houdini and Komodo we went to 24 plies and for Stockfish we went to 28 plies (which was comparable to the others in terms of time spent per position).
Perhaps the key idea that emerged from this project was the recognition that there are four quantifiable properties to all opening positions:
Frequency (or commonness) - how often has the candidate position been seen in human and engine games?
Play-balance - how closely does the candidate position align to the relative advantage seen in the opening position in empirical and evaluative terms?
Draw-tendency - how often has the candidate position resulted in a draw in human and engine databases relative to each of those databases?
Dissonance - to what degree do leading engines disagree in deep evaluations of the position?
My opening selection process was based on quantifying these four dimensions and converting the resulting analysis into percentile-ranks. In other words, every position was boiled down to four simple numbers on a 0-100 scale. I then assigned varying weights to each of these numbers for each Stage.
In all Stages no position was selected that had not been seen at least 100 times in human and engine games; in Stage 1 this number is at least 300 times. I did not choose very rare openings because the metrics for play balance and draw-tendency would have been unreliable.
As already mentioned play-balance is critically important in Stages 1 and 2 because they are both single round-robin. Every game in those Stages will slightly favor white as is seen in the opening position. Play-balance remains quite important through the other Stages, but not as much, because engines will get to play both sides of each position.
Attempting to reduce the draw-rate is not important in Stage 1 because engines of widely varying strength will be competing resulting in many decisive games. From Stage 2 onward all openings selected have below-average draw-rates. Our theory, which was demonstrated in the Superfinal last Season, is that you can knock down draw-rates by as much as 15-20% by selecting balanced openings that have historically produced low draw-rates.
Dissonance is irrelevant to my opening selection until Stage 4 and the Superfinal. In those Stages I purposely picked positions where the leading engines displayed sharp evaluation disagreements while not being heavily unbalanced in favor of one color. I am hoping that this will both further depress the draw-rate as well as lead to some really wild and exciting games. (Or it could lead to some big names getting steamrolled because they flat-out do not understand the starting position, in which case it will be very revealing!)
Q: Can you provide us with a list of what openings will be played in each Stage?
A: I could, but Martin prefers the element of surprise and so do I.
Q: Will you know what openings any two engines will play before a game starts?
A: No. For each Stage I have provided Martin with a pgn file containing all the opening positions that will be played. His tournament GUI selects the openings at random, never repeating one that has already been played, apart from cases where engines are playing both sides of the same opening.
Q: You say you compared evaluations across the engines. How could you do this when engines have different ways of evaluating positions and scale differently?
A: Adam Hair did a linear regression comparing the 44,000+ evaluations and concluded that Stockfish DD is about 1.88x higher than Houdini 4 while Komodo TCECr is 1.18x higher. We adjusted their evaluations accordingly to get a better impression of evaluation dissonance. Correlations were quite high after we did this.
Q: Is there any theme or opening system that will be seen more often than others?
A: Not by design. It is interesting, though, that 1.e4 openings are more predominant in Stage 1 and 2 but 1.d4 openings achieve parity after that. This outcome was happenstance.
Q: Any gambits or exotic openings in store?
A: Gambits, or positions that evaluate poorly but have compensation in human contests will be seldom seen. I did not go looking for them. As for exotic positions, different people will see different things in a position. I picked positions entirely on the basis of their known, objective numerical properties, not my impressions of what I saw on the board, which would be very subjective and unreliable.
Q: Would you be so kind as to send me your database?
My thanks to my collaborators, Adam Hair and Anson Williams, whose assistance and input were essential, and also to Martin Thoresen, whose encouragement and trust that we could pull it all off in time for this Season's launch certainly helped us complete this ambitious project.
Starting from Season 5, the time control is 120 minutes + 30 seconds added per move for the whole game, in all events (an event is either a Stage or the Superfinal). If an engine loses on time, the result will not be changed or the game replayed. If the TCEC game server locks up at any time during a game (BSOD, freeze etc), that game will be restarted unless the last position was a 6-man or less tablebase position, then it will be manually adjudicated.
A game can be drawn by the normal 3-fold repetition rule or the 50-move rule. However, a game can also be drawn at move 40 or later if the eval from both playing engines are within +0.05 to -0.05 pawns for the last 5 moves, or 10 plies. If there is a pawn advance, or a capture by any kind, this special draw rule will reset and start over. In the website this rule is shown as "TCEC draw rule" with a number indicating how many plies there are left until it kicks in. It will adjudicate as won for one side if both playing engines have an eval of at least 6.50 pawns (or -6.50 in case of a black win) for 4 consecutive moves, or 8 plies - this rule is in effect as soon as the game starts. In the website this rule is shown as "TCEC win rule" with a number indicating how many plies there are left until it kicks in. Cutechess will also adjudicate 5-men or less tablebase endgame positions automatically.
TCEC uses openings put together by Nelson Hernandez and Adam Hair. All of them are 8 moves deep. If you click the Help menu and then "The Season 6 opening book" you can read in detail how they prepared for this challenge.
When you enter "Archive mode" in the File menu, you can see the official TCEC ratings in the bottom right corner - it is updated after each Stage or Superfinal and is calculated by using Ordo. The calculation is global and will always include all new and previous games since Season 4. Any games with losses on time are discarded, as is engines with only losses. If there is a new engine entering TCEC for the first time, it will get a "temporary" rating taken from the CCRL 40/40 single CPU list. If an engine isn't found here, or if it has played very few games, the CEGT 40/20 single CPU list is used instead but the rating difference between Houdini 3 64-bit in the two rating lists will be added to the rating. If an engine isn't found in either list, an approximate ELO rating will be given to that engine based on tests from the programmer. Then after this new engine has played an event, the official rating will be calculated using Ordo as explained earlier. If an engine is updated to a new version, this new version will inherit the rating of the old version.
The engine programmers can provide updates only before an event starts, not during. However, there will be no extra testing meaning that this is a gamble if the engine could be unstable. The deadline for engine submission is the last game of the current Stage - the goal is to be able to start the next Stage as soon as possible without any significant delay.
Critical Engine Bugs
In the case of a serious, play-limiting bug (like crashing or interface communication problems) not discovered during the pre-Season testing, the engine can be updated once per Stage to fix this/these bug/bugs only. If this update still doesn't fix the problem(s) or if there is no update available, the engine might have the number of cores reduced, have the hash size reduced or have the tablebase access disabled - these changes will remain for the rest of the Stage.
If necessary, tiebreaks can be used to determine advancement. For all Stages (not the Superfinal), the first tiebreak criteria is the "crash" tiebreak, meaning that if an engine has crashed once or more during the Stage, it will fail qualification versus another engine that has not crashed if both of them has the same amount of points. The Sonneborn-Berger criteria is the second. If still a tie, the greatest number of black games decides. The next criterion is the greatest number of wins, then the greatest number of wins with black. In case of still being tied, then the direct encounter between the tied engines decides. If they are still tied, then the tournament director decides which engine gets the promotion.
The TCEC Season System - Season 6
As soon as a Stage starts, it will run 24/7 until all games have been played. One game is played at a time - the next one starts automatically. There will be a short break between the Stages, to make sure everything is ok with the TCEC game server and to prepare for the next Stage.
Stage 1 is divided into 3 groups which consists of 12 engines each. Each group format is a single round robin. The top 5 from each group and the best number 6 will move on to Stage 2, while the rest are out of TCEC for the current Season.
Stage 2 consists of the 16 engines that qualified from Stage 1. It is also a single round robin. The top 8 move on to Stage 3 while the rest are out of TCEC for the current Season.
Stage 3 consists of the 8 engines that qualified from Stage 2. The format for Stage 3 is a quad round robin so that each engine will play both sides of the same opening against each other, twice. The top 4 will move on to Stage 4 while the rest are out of TCEC for the current Season.
Stage 4 consists of the 4 engines that qualified from Stage 3. It is a hexadeca round robin and the openings are chosen randomly per pair so that each engine will play both sides of the same opening against each other. The top 2 will qualify to play the Superfinal, while the other engines are out of TCEC for the current Season.
After Stage 4 has finished, the top 2 engines will meet in a Superfinal of 64 games. This match is played with 32 different openings so that each engine plays both black and white of the same position. The match will be presented with opening 1 used in games 1 and 2, then opening 2 used in games 3 and 4 etc. If the match is theoretically won for one side before game 64, the match will still continue until all 64 games have been played. In the case of a drawn match there will be a rapid match of 16 games with a time control of 25' + 10" with random openings selected from earlier in the same Season. In case it is still tied there will be a Blitz match of 8 games with a time control of 3" + 2'. When the Superfinal is over, the current Season ends.
The TCEC Grand Champion
The winner of the Superfinal will be crowned the TCEC Grand Champion and will keep this title until there is a winner in the next Superfinal. There is no automatic qualification for the reigning Grand Champion, it will have to go all the way through the next Season for it to be able to defend the title.
Engine Specific Configuration
UCI engines are supported and Xboard (Winboard) engines will use the Wb2Uci adapter which has no impact on performance. To identify an engine running with the Wb2Uci adapter you can click the gears next to the engine logo during a game. This info will be saved for the archive, but does not work for Season 5 or older games.
Many engines come with different .exe files. 64-bit .exes are always preferred over 32-bit. Also, compiles that support SSE 4.2, AVC or similar instruction sets are preferred.
Some engines can utilize a function in Windows called large pages, that gives a speed boost. However, after a while the memory in the computer will be fragmented, so that one engine might receive large pages and the opponent won't, which would be unfair. Therefore, this has been disabled. Large pages are often useful when you are running infinite analysis on a position.
Number of Cores / Threads
Each engine can use up to all 16 cores of the processors, if this is supported. Some engines have a prefix like "deep", but this has been omitted from the engine name to make it shorter. When watching a game you can see the number of cores that are in use for the engines currently playing by clicking the gears next to the engine logo.
The split depth parameter can be adjusted, with advice from each of the engine programmers. Basically it defines the minimum depth for work to be split between cores / threads. If no instructions are given from the programmer, the default value will be used.
Main Hash Size
Each engine is allowed to use up to 16384 MB of hash. Not all engines supports this much hash, so the maximum for that engine will be used in this case, typically 1024 MB or 2048 MB. When watching a game you can see the size of the hash that is in use for the engines currently playing by clicking the gears next to the engine logo.
Minor Hash Sizes
Some engines have an option to configure the size of other hash tables, often called pawn hash or evaluation hash. The combined, total limit for hash types like these is 2048 MB.
Own Opening Book
All opening books shipped with the engines are disabled.
6-men Syzygy, 5-men Nalimov, 5-men Gaviota (cp2), 5-men Scorpio bitbases and 5-men Robbobases (Totalbases + Triplebases) are available depending on which each specific engine can use. For some engines specifying more than one type is possible, but here only one is allowed. Robbobases also have Z pieces. They are all hosted on a very fast dual Samsung SSD Raid 0 setup with a rough read performance of 1 GB / second. When watching a game you can see the type of the tablebases (if any) that is in use for the engines currently playing by clicking the gears next to the engine logo.
Each engine is allowed to use up to 2048 MB of tablebase cache. Not all engines supports this amount, so the maximum for that engine will be used in that case, typically 256 MB.
Ponder / Permanent Brain
Basically this means that the engines can think during their opponents turn. It is not allowed so it has been disabled. This might change in a future Season of TCEC.
Contempt / Draw Score
Some engines have a setting that can adjust their own view of the positions throughout a game to avoid draws. This setting is not changed from the default unless it is a request from the programmer.
Any configurable option not described above, are not adjusted in any way, except that "keep hash tables" or similar is usually enabled.
The TCEC Server
CPUs: 2 x 8 core Intel Xeon E5-2689 @ 3300 MHz
CPU Coolers: 2 x Corsair H80i
Motherboard: Asus Z9PE-D8 WS
RAM: 64 GB Kingston KVR16R11D4K4/32 Reg/ECC
PSU: Corsair AX 760
SSDs: 2 x Samsung 840 Pro 128 GB @ Raid 0
Case: Silverstone Raven RV03B-WA
Is this the official World Computer Chess Championship?
No, it is not, although some people regard it as such.
Who is the reigning TCEC Grand Champion?
Stockfish defeated Komodo in the Superfinal of Season 6, so Stockfish is the reigning TCEC Grand Champion.
Did the engine move instantly?
Yes, some engines can move from hash, or move instantly. However, sometimes it can seem that it moves instantly but in reality it doesn't, since there is a transmission delay from the server by 1 minute. Refer to the "move time" for each engine to see how long it thought on its move.
Is there a special plugin required to view the games?
What GUI are you using to play the games?
To play the games a special version of cutechess-cli is used. This is a command line tool without an actual GUI. This was made possible thanks to Jeremy Bernstein. Previously TCEC used ChessGUI by Matthias Gemuh.
What is the name of the web framework that displays the games?
It is called pgn4web and is totally free. Paolo Casaschi is the mastermind behind it and has helped a lot on certain difficult programming challenges.