CheckerBoard 1.7 & co.

I just published a new version of CheckerBoard on my website. CB 1.7 has some minor bugs fixed, and doesn't write to the program files - CheckerBoard directory any more, which was a problem with the UAC (user account control) of Windows Vista and Windows Seven. Most of the changes in CB 1.7 were contributed by Ed Gilbert; thanks a lot Ed!
The CB source code is also available, but still rather difficult to comprehend.

I also updated the engines to comply with Vista/Win7, Cake and the Kingscourt engine didn't work with the new OS versions unless run with administrator rights. The CB install package now also includes Suicidal Cake with a 4-piece endgame database for suicide checkers and an opening book. I worked on suicide checkers about 3 years ago, and thought it would be a shame to leave what might be the world's best suicide checkers program rot on the shelf...

Multi-core KingsRow tested

I have been testing KingsRow 1.16d against Cake 1.8x12 on my quadcore machine. Here are some results of engine matches at 5 seconds per move:

Cake 1.8x12 - KingsRow 1.16d (1 core)  +19 -14 =255
Cake 1.8x12 - KingsRow 1.16d (2 cores) +19 -18 =251
Cake 1.8x12 - KingsRow 1.16d (4 cores) +18 -19 =251

I had been expecting Cake to be slaughtered by the multi-core version of KingsRow, but apparently it's not that bad. To check whether or not it might be a bug in the multithreaded search of KingsRow, I played a match with Cake and KingsRow running on 1 core, but KingsRow got 10 seconds per move - so essentially this should reproduce the second match above. I found

Cake 1.8x12 - KingsRow 1.16d (10s)  +17 -21 =250

There is always some variability in engine matches, otherwise one might conclude that the 2-core version of KingsRow was not performing as well as KingsRow with twice the search time. As it is, I don't quite know what to make of it.

Java checkers' first draw against Cake

My Java checkers engine now has its own - very primitive - GUI based on Swing, and I can let it play games against my other engines. Here is its first game against Cake:

[Event "test game"]
[Date "2009-03-18"]
[Black "Cake 1.8"]
[White "Java engine"]
[Result "1/2-1/2"]
1. 11-15 22-18 2. 15x22 25x18 3. 8-11 29-25 4. 4-8 25-22 5. 12-16 24-19 6. 16-20 19-15 7. 10x19 23x16 8. 6-10 16-12 9. 9-14 18x9 10. 5x14 26-23 11. 2-6 22-18 12. 14-17 21x14 13. 10x17 31-26 14. 6-10 23-19 15. 1-6 19-15 16. 10x19 26-22 17. 17x26 30x16 18. 6-10 28-24 19. 10-15 18-14 20. 15-18 14-10 21. 7x14 16x7 22. 3x10 12x3 23. 18-22 3-7 24. 22-26 7-2 25. 10-15 2-7 26. 15-18 7-10 27. 14-17 10-14 28. 18-22 14x21 29. 26-31 21-17 30. 22-25 17-22 31. 25-30 32-28 32. 30-26 22-25 33. 26-30 25-22 34. 30-26 22-18 35. 26-30 1/2-1/2

This one draw doesn't mean too much of course - in its next game, it got into trouble against simple checkers, but just managed to escape with a draw. There is still a lot of knowledge missing in the evaluation function... And the GUI will need a lot more attention too!

perft for checkers

A couple of days ago I got a mail from Aart Bik, a programmer working at Google. He is currently developing a checkers application (among others) for the Google android platform (aka gPhone). He asked me about perft numbers for checkers. Perft is an invention of the chess programming community, a test to verify the correctness of the move generator, by printing how many legal moves exist in a search tree to depth N from the current position. Although I have this function in my chess engine, I never thought of adding it to Cake until now. However, Aart is right, and a perft function is always good to have for debugging purposes, so I added it to Cake. In the next CheckerBoard/Cake release, Cake will have this functionality. For starters, here are the numbers Cake reports from the initial position:

depth 1 positions 7
depth 2 positions 49
depth 3 positions 302
depth 4 positions 1469
depth 5 positions 7361
depth 6 positions 36768
depth 7 positions 179740
depth 8 positions 845931
depth 9 positions 3963680
depth 10 positions 18391564
depth 11 positions 85242128
depth 12 positions 388623673

Aart is happy too, since his numbers match. If you are interested in his applications for the gPhone, then check out his blog.

Engine match mania

Over the last week, I tested several versions of my latest checkers engine (which carries the rather bland name Cake 1.8) - and I did this on all of my three computers. I own a AMD64 desktop, a CoreDuo laptop and a CoreQuad Desktop. These machines are all about equally fast; the CoreQuad is a bit faster than the other two, it also runs a 64-bit windows. Here are the results of my engine matches against KingsRow 1.16c:

Engine           CoreQuad CoreDuo  AMD64    total
-------------------------------------------------
Cake 1.8 v1      +28-12   +27-18   +27-20   82-50
Cake 1.8 v2      +19-17   +25-17   +32-15   76-49
Cake 1.8 v3      +23-15   +24-19   +27-17   74-51
Cake 1.8 v4      +21-15   +23-17   +26-17   70-49
Cake Manchester  +23-11   +24-13   +24-21   71-45

The results are a mess! I ordered the different versions of Cake 1.8 according to their overall result, and while you can see that v1 is best overall, it is worse than v2 on the AMD64 and worse than Cake Manchester on the CoreDuo. Which means that if I had only been testing on the CoreDuo, I would have been disgusted with the performance of my new program, because it's worse than the old one, and if I had been testing on the AMD64, I would have chosen v2 rather than v1. Which of course means that running engine matches on a single computer (as I did for the past 7 years or so) is clearly insufficient, and it also casts some doubt on the current methodology of using 3 computers - how do I know that using 3 computers is enough? Wouldn't 10 be better? All in all, this is quite a disgusting discovery, and I'm at a bit of a loss of how to proceed when optimizing my engine :-(

Bug squashing

Yesterday and today I found bugs in Cake's evaluation. I was looking at some games that Cake had lost, and made a change in the evaluation which should have improved the situation. However, although the evaluation was different, the output of the engine remained the same - the exact same number of nodes searched with the same value for the position. That made me *very* suspicious, and I looked at the code a bit more closely, and discovered that during my code cleanup of last summer I had broken a really important piece of knowledge. Being suspicious, I continued looking for bugs and found one minor bug and one cosmetic bug. The match result improved drastically after stomping the first bug, from +19-16 @5s/move to +26-11. I don't have much hope that the minor bug will make a big difference but soon I will see!

Multi-Core speed

Nowadays, CPUs aren't getting much faster in terms of clock speed, but the manufacturers are putting multiple cores in one CPU. While this gives a lot more bang for the buck, it is a challenge for programmers to make use of the newly available resources. Some problems are easy to parallelize, while others are much more difficult. Unfortunately, tree searching such as in two-player-strategy-games belongs to the second class: it's not straightforward at all to implement a parallel game tree search. Of course there are solutions for this, the best-known is called YBWC for young brother wait concept. This is quite easy to explain with words, but really programming it is something different :-(
I haven't managed to find any sensible description of the algorithm with some real code, except for the source code of Crafty, a strong free chess engine. While this is documented quite well on the source code level, there are very many source code files and many of them have the required changes for the parallel search, and it's all very confusing to me. I wonder if there is any sensible YBWC tutorial anywhere on the net?

Anyway, my own plans were different to start with: All I wanted to do was to make my book generator work in parallel - it is quite easy to have multiple threads running on a multi-core-CPU when each thread is just doing a standard search on its own position. The big difficulty only arises when all threads are supposed to help search the same position, because they have to start communicating with each other.

The only problem in having multiple threads running simultaneously is the following: the endgame database access code has to be protected from being accessed by multiple threads, since confusion could occur when two threads try to load data from disk at the same time. In such situations programmers use a "lock" to prevent other threads from accessing critical code when one thread starts using it. This thread has to free up the critical code again once it is done, so that the other threads can also use it. Obviously, this whole process might slow down the program a lot, because one thread can be blocking the others - and the only way of knowing whether this is a problem is to try! I made Cake thread-safe last summer, and finally tested the speed on my new quadcore yesterday. Here is the result:

1 thread: 2050 kN/s
2 threads: 2000 kN/s
4 threads: 1815 kN/s

Running 2 threads simultaneously thus results in a speed loss of about 2.5%, while running 4 threads gives a loss of 11%. Of course, that is not a big price to pay, since in total you are getting 4x89% = 356% out of your CPU compared to having a single thread running. I also have to admit that I just put the whole database lookup call in a critical section; perhaps it is possible to make that more efficient by only putting the parts of the database lookup that affect the database cache in a critical section. Nevertheless, I'm afraid I have no excuses any more and should be working on a multi-threaded book generator now...

Heavyweight match

I just finished running an engine match between the 64-bit versions of Cake and KingsRow. Cake managed a narrow win, but what makes even happier is that the entire engine match ran without problems, i.e. CB64 and Cake64 and KingsRow64 are all stable enough to run for 12 hours without any errors. I will probably publish the 64-bit versions in a couple of weeks.

As an aside, I got so annoyed with Windows Vista UAC (user account control) popping up TWO message boxes to confirm when I just want to rename a file that I ended up disabling UAC. I really wonder what the MS guys were thinking when they invented this feature. One message box for renaming a file would already be a huge PITA, but two... I guess the only people who don't disable UAC are those who don't know how to do it!

Some more Vista madness: I also noticed that by default my system was set to defragment its harddisk daily, run a full virus check daily, and run the pretty much useless Windows Defender. My system (a QuadCore with 4GB RAM) used to access the harddisk permanently for about 15-30 minutes after the system start. With all this stuff disabled, it is much better. In my quest for energy efficiency, I bought an 80+ power supply (80plus.org), which is not only energy efficient but also very silent (it only generates little heat and thus needs less ventilation than a standard power supply). However, with a loudish harddisk I didn't have any benefit of that. Now I do :-)

64-bit speedups

I tried to find out how much performance increase a 64-bit compile will give compared to the 32-bit compile. I tested three different programs: my Connect 4, Cake and KingsRow. I also have a speedup reported by Ed Gilbert for his international checkers program, KingsRow-10. Here are the numbers:

• KingsRow: 5.8%
  
• Cake: 6.4%
  
• Connect 4: 11.6%
  
• KingsRow-10: 42.9%
  

This list shows that all programs are able to benefit from the 64-bit compile, although the gain is rather moderate for all programs except KingsRow-10. Why can some programs benefit more than others? Probably this has to do with how much they make use of 64-bit operations. For example, my Connect 4 program uses a 64-bit representation of the board. KingsRow and Cake use 32-bit representations and don't really have any use for the larger word size on the 64-bit machine. KingsRow-10 on the other hand also uses 64-bit numbers for its board representation and benefits much more than any other program on the list.

With all these numbers I have to add that there are some points to be considered: The KingsRow 32/64-bit versions don't seem to search the exact same tree - in kN/s searched, KR-64 was 5.8% faster, but its search time was only 2.6% lower. Additionally, KR wasn't using the endgame database during this test, since it doesn't recognize it for some reason on my system. Cake 32/64 searches exactly the same number of nodes on the one-minute search I used for this test. Cake32 makes use of some assembler instructions which Cake64 does not, so probably the speed difference would be a bit larger if I found out how to program these functions in 64-bit assembler. The same is true for Connect 4 (For the experts: I don't have a LSB function in assembler for the 64-bit versions).

Cake 64

I recently bought a new motherboard/CPU for the computer that was generating the opening book database for Cake. My new machine is the most powerful I ever had, with a Core Quad and 4 GB of RAM. For curiosity's sake, I also installed Windows Vista on that machine. I'm not at all convinced by Microsoft's new OS, but at least it gives me the opportunity to finally run my 64-bit compiles myself. Thanks to this, I found the bug in the 64-bit version of Cake, and of course it was a 32/64-bit issue. I was computing the size of a memory block full of pointers for the endgame database with


char *pointer;
int memsize = blocknum * sizeof(int);
pointer = malloc(memsize);


since I had assumed that sizeof(int) would change to 8 byte on the 64-bit machine. But it stayed at 4 bytes, while pointers do need 8 bytes, and of course the whole thing crashed. I replaced the sizeof(int) with sizeof(pointer) and now it works. It appears that this was the only portability issue; I now have a working 64-bit version of Cake on my machine. By a strange coincidence I clocked it to be 6.4% faster than the 32-bit version.

Housekeeping

I recently inherited the house of my grandfather, and now have a lot more housekeeping to do than earlier. Somehow, this might have helped me in my decision to do some housekeeping with Cake. Those of you who have programmed probably know the feeling: even if you started out with a more or less clean concept (I certainly didn't with Cake), things eventually get out of hand. You add an idea here, put in a quick fix there, and before you know it, your code is unreadable and unmaintainable. That's about the state that Cake was in before my summer holidays in the Austrian mountains. There, besides reading the Harry Potter finale and climbing some mountains, I cleaned up Cake's source code during two whole weeks. That much time is necessary, since you really have to stand back, and get an overview over your whole program to decide exactly how you want to do your cleaning up. You can't just do it on a single day, or fix it by programming one evening per month - until you get back to programming, you will have forgotten everything again (at least that's what would happen to me, but then I turned 36 recently!).
The result of two weeks of hard programming is that there are no global variables left in Cake, that Cake can be switched to thread-safe mode (at a small speed penalty), and that much of Cake's code is now much clearer and cleaner than before. It doesn't play any better at all though - but now that I have (nearly) finished the housekeeping part, I might be more inclined to work on it again!

64-bit versions of CheckerBoard and Cake

Linux users could already benefit from the new powerful 64-bit processors for quite some time, while M$'s win64 never really got out of its beta phase. With the release of Windows Vista, this has changed, and 64-bit computing is now also available to the masses. I run neither Linux nor Vista, but I do have a compiler capable of producing 64-bit executables. I wonder whether they will run under 64-bit Vista? If you happen to own a 64-bit Vista, you can try it by downloading CB_64.zip (264KB), and installing it in your CheckerBoard directory. To do this, you need to extract checkerboard64.exe to the CheckerBoard directory, and CakeM64.dll to your engines directory. Run checkerboard64.exe and choose CakeM64.dll as your engine. Let me know what happens!

Core Duo!

It's been a long time since I last bought a new computer (21 months, to be exact, a record for me in the last 10 years!) - but I just did it again: I got myself a new laptop with an Intel Core Duo CPU - unfortunately not one of the even better Core 2 Duos. However, even this machine is absolutely great! Cake runs about as fast on my Core Duo at 2GHz as on my Athlon 64 at 2GHz. But the cool thing is that I can run two instances of Cake at the same speed, if I want to. Not that this really helps me in any way... it just shows why these dual core machines are so cool. The days of single core CPUs are numbered (as far as PCs go, anyway), and the first quad-cores for consumers are already on the market. Which gives me a bit of a headache, because it means that I should try to rewrite Cake so that it can use these new CPUs with multiple cores! All good chess programs have this capability, but the only checkers program that I know of that could use multiple processors was Chinook. However, the speedup from multiple processors there was quite lousy compared to what is reported for chess programs. For chess, it's something like 1.9x faster for dual cores and 3.5x faster for quad-cores, for Chinook it was approximately 3x faster on 16 CPUs [1]. It's not quite clear to me whether checkers is inherently tougher to parallelize than chess or whether the implementation in Chinook was poor. Any opinions?

Cake Manchester 1.09d - KingsRow 1.16: +20-17=251

The match between my "old" Cake Manchester 1.09d and KingsRow 1.16 finished with a narrow win for Cake - 20 wins vs 17 losses and 251 draws (naturally this is with books off - else it would be draws all around - or very nearly so). I also repeated the match with learning disabled in Cake, with the result that not much changed: +22-19=247. So to stay in line with my last post, that would mean that I could leave Cake alone somewhat longer - but the result is too close for comfort, and so I decided to start tuning Cake a bit. When I try to improve Cake, there are many different ways to do it. One of the things I sometimes try is to tune the evaluation function, i.e. to change weights of different terms in the evaluation (as opposed to adding new terms to the evaluation). When an engine has been a sitting duck for years, as Cake Manchester has, then it is quite well possible that the opponent (KingsRow in this case) is just well-tuned to beat Cake. Changing the playing style a bit might make quite a difference in such a case. So to start, I am trying to just tune Cake a bit to avoid Ed's last two years worth of tuning of KingsRow - of course, I don't just mindlessly change the evaluation weights; instead, I checked the games that Cake lost in this match. Then I also have some ideas left on how I might improve the evaluation function with new terms, and a new idea for the search which I already tested but which didn't work well in the first implementation. Nevertheless, it also didn't work too bad, and perhaps with some refinement it will work. If all runs smoothly, I will have a new version of Cake ready for Christmas!

The new book, finally

This week I released a new install package for CB - with the latest versions of CB, Cake, and the opening book. The new book has 93'000-something moves, just like the old one, but it should be quite a bit better. The last result of the matches against KingsRow with the final book was

• mailplay:
  +0 -4 = 20
  
• lost:
  +16 -18 =2
  
• normal:
  +4 -13 =271
  

In fact, this is a worse result than I got at 74'000 moves; however this is most likely just because of KingsRow's random opening choice. With the original book, the result is

• mailplay:
  +0 -7 =17
  
• lost:
  +8 -18 =10
  
• normal:
  +4 -15 =269
  

which is clearly worse, although the improvements are mainly in mailplay+lost openings. This is mostly because of the new book generator which looks at unbalanced ballots deeper than the old book generator.
I also wrote a book merge tool with which I have replaced the old database entries in the opening database with their new counterparts. Since quite a lot of play has changed in the opening stages, it will now take a long time for the book generator to expand these new lines to the same level as the old lines, so don't expect a new big book any time soon.

Chess, not Checkers

I'm off to a chess tournament which will last for 9 days. However, the title of the post is misleading, I plan to fix two bugs and add a couple of features to CheckerBoard when I'm not playing chess! I also want to write a book merge tool to replace the old values in the opening database with the newly calculated ones. I hope I'll find the time for all the things I want to do...

In the meantime, I have new results from the new book, which was at 46'000 moves when I last tested it:

• mailplay:
  +1 = 18 -5
  
• lost:
  +14 = 4 -18
  
• normal:
  +3 = 271 -14
  
  

That's hardly better than at 20'000 moves, although there is some randomness to these matches, because KingsRow chooses among equal moves at random.

Cake for Linux with opening book released

Peter Chiochetti has published Cake 1.20 with an opening book for Linux. The book is based on a huge opening database of over 1.5 million positions, and is probably nearly error-free. Get the new version of Linux-Cake on the xcheckers page!

Cake Manchester's opening book

The book I just created for Cake++ for Linux is much better than the book that you can download for Cake Manchester. Manchester's book is still the same I once created in 6 weeks with a beta of Cake Sans Souci after the computer checkers world championship in Las Vegas in 2002. That book is based on the first 90'000 positions my book generator explored. Now, among these 90'000 moves roughly half will never occur when you set the book to play best moves only. Of the remaining say 50'000 moves, about one quarter are just forced captures. So effectively, that book has a mere 40'000 useful book moves, and they are based on an opening database with "only" 90'000 positions, which means there are also quite a number of errors in that book.
The book for Cake++ only contains lines that Cake will actively select, i.e. there are no wasted moves for lines it will never play. I also left out the forced captures in that book. Finally, all moves are based on the 1'600'000 position opening database, which means that there are only very few errors left (if there are any at all).
Cake Manchester will also get an updated book - I'm just waiting for the book generator to discover the bad move that caused Cake's only loss in the big match between Cake and KingsRow late last year.