Monthly Archives: May 2020

Chess – Basics

Uncategorized

So recently I got interested in Chess, and by recently I mean 6 months ago. I learned some important beginner’s lessons along the way and I planned to document them in non-technical language in this article.

I’m assuming you know the very basics, like how the pieces move and what castling is, which is exactly how much chess I knew when I started. If not, go outside (on the internet) and learn the basics. Then create an account on lichess.org or chess.com and play around 10 games with other people. Then come back to this article.

So assuming that we’re on the same page, let’s start!

Stages of the game

A chess game can be visualized to happen in three stages; Conveniently–start, mid and end game. You can be mindful of what stage the game is in as there are certain things you can do in each stage to give yourself a slight edge in the game.

When you start, you try to develop the pieces as quickly as possible and castle. That means all pieces are ready to jump into action; whether attack or defence, and your king’s safety is ensured.

Midgame, depending on your position, you exchange pieces and try to get a better position. Make sure you’re either up material (pieces or pawns), or you have a better hold of the position and control more space on the board.

Endgame is when both sides go all out using whatever positional advantages they’ve created previously. It is also common for endgames to just have king and pawns, maybe a minor piece as well. In this case, both sides try to push their pawns and promote them to Queens (or any other piece but usually the Queen, and the first side to do it usually wins).

Pre game

Know value of the pieces: Not all pieces are as valuable. Pawns are unit value, that is 1. Knights and Bishops are 3, Rooks are 5 and Queen is 9 units each. When exchanging pieces, these values should be taken into account. It generally doesn’t make sense to sacrifice a Rook for a Bishop. Of course, there are times when you’ll want to do it (especially if you have a forced checkmate in place).

The value of your pieces changes depending on your position and the stage of the game. You can read about it on Wikipedia but you will also learn these things intuitively once you have enough games under your belt.

Starting tactics

Castle early, ensure king safety before anything: I used to frequently go for material (i.e. trying to win pawns or pieces) and space before castling which generally doesn’t end well. Once your opponent is done developing their pieces and castling, you’ll end up playing a very defensive game if your king is in the center of the board with undeveloped pieces.

Develop pieces as early as possible: Move all pieces out of their initial position, ready for attack or defence and connect rooks early in the game.

Don’t move the same piece twice in opening: In general, you should avoid moving a piece more than once in the opening as the goal is to develop pieces fast. Of course, exceptions are when your piece is attacked or if the other side offers a gambit (a pawn or minor piece sacrifice to gain positional advantage).

Develop Knights before Bishops: Another general rule of thumb as both Knights can develop and simultaneously control central squares which is usually a good idea.

Rooks work better when connected: Rooks are developed when they’re connected (both Rooks are defending each other). Another common tactic is to double rooks on an open file (a column on a Chess board without pawns) to mount a powerful attack.

Doubled Rooks on an open file are very powerful in infiltrating enemy territory

Aim for center control: It is generally a good idea to control the center four squares of the board. There are exceptions, however, as with certain openings we can aim for control over one side of the board and continue our attack from there.

The highlighted squares are important and you should try to gain control over them

General tactics

Don’t keep pieces in pin: If your piece or pawn is pinned, meaning a more valuable piece is behind it, it is good to unpin the piece (move the piece that’s behind it). This is especially true with king. You don’t want your king to be on the same file as your opponent’s Rook or Queen, and same diagonal as your opponent’s Bishop. If the other side brings their Rook to the same file as your King or Queen, move it away even if there are pieces in between.

White to play: The d pawn (pawn on ‘d’ file) cannot capture the Knight as the Queen is hanging and the Rook on d8 is xraying it. It is best to unpin (get away from the Rook’s file or Bishop’s diagonal) as soon as possible.

A piece that has moved a lot is worth more than an unmoved piece: If your knight has been moving around the board, it is generally in your opponent’s favour if they manage to exchange your active piece for their dormant piece.

A piece that covers more area is worth more than a piece that covers less area: Similar to a previous one, if your Bishop covers and controls a beautiful diagonal, your opponent might want to get rid of it by exchanging their unused piece for your beautiful Bishop. Try to protect your active pieces.

If you control more space, don’t trade pieces: If your pieces control a lot of space on the board, you should avoid trading pieces.

If you’re crammed, exchange pieces to create room: Conversely, if you’re crammed by your opponent, try forcing exchange of pieces and pawns to create some room for your pieces to develop.

If defending, defend a piece as many times as it is attacked: If you want to defend a piece or pawn, defend it as many times as it is attacked.

If attacking, attack a piece as many times as it is defended plus one: If you’re the one attacking, mount an attack such that the other side runs out of ways to defend.

Two pieces are better than one piece: It generally doesn’t make sense to sacrifice a Bishop and a Knight for a Rook, or two Rooks for a Queen. It is, generally speaking, better to have two pieces instead of one (use the point value of pieces to decide if the exchange makes sense, but remember that active pieces are more valuable than passive ones).

Avoid back rank issues: If you don’t defend the last rank (where your king usually resides), a Rook from your opponent might simply deliver a checkmate (since the pawns in front of your king might be unmoved leaving no room for your king to escape).

In this position, if White isn’t careful, the Rook on a8 can deliver a checkmate which is why you must make an escape route for your king or have a defender on the last rank

Capture with pawns towards the center: Generally, if you capture with a pawn, you want to capture towards the center of the board if that’s an option.

Knights belong in the center of the board: It is usually not the best idea to move the Knight to a corner of the board as they cover fewer squares from there. They say Knights on the rim are dim. This applies to most pieces but especially the Knights.

As you can see, a centrally positioned Knight controls many more squares than a cornered Knight

Avoid creating pawn islands: An isolated pawn is one that isn’t connected with your other pawns. Such pawns are liabilities and weaknesses, and hence should be avoided (or sacrificed for positional advantages). In general, try to have as few pawn islands as possible.

Black has 2 pawn islands and white has 4. Black is to be preferred here.

Avoid doubled pawns: Doubled pawns are two pawns on the same file. As with pawn islands, they are weak and can become a liability in the endgame. It is said that a Rook’s favourite meal is a doubled pawn and is best to avoid doubling your pawns.

Grab any open files for your Rooks: When pawns are traded, they leave behind open files. Be quick and place a Rook behind an open file to gain control over the entire file. Later you could also double Rooks on the same file for a powerful attack.

A Rook on an open file covers a lot more square compared to a rook behind pawns

Avoid exchanging the fianchettoed Bishop: The fianchettoed Bishop is supposed to defend the weaker squares around the King and control a long diagonal. You shouldn’t exchange it in early or mid game if that’s an option.

The Bishop on g2 does a nice job of defending the weaker squares around your King’s castle.

Know thy Bishops – Bad Bishop vs good Bishop: A bad Bishop is one that’s obstructed by your own pawns, ending up looking like a pawn itself (and not controlling much space). A good bishop covers a nice long diagonal.

Notice how Black’s Bishop is nicely controlling a diagonal while White’s Bishop is essentially a pawn

Bishops and Knights are equal, except…: Bishops and Knights are both 3 points, but Bishops are usually preferred when the board has few obstacles, while Knights are preferred when the position is very tight. In general, a Bishop pair is always something you’d want to save if you can.

Be mindful of forks: Forks come when two pieces are attacked by an opponent’s piece and we lose one of them anyhow. Knight forks are common and can be tricky to spot. A fork that comes with an attack on the King and the Queen is called a royal fork.

Here, black will lose a Rook
A royal fork

Next steps

From here, you should learn some of the openings and endgames. Openings will help you quicky get to a position where all of your pieces are mobilized and ready, and also know what’s on your opponent’s mind depending on what they play.

Endings are important to know how to actually win most games. Many games might not end with you having a bunch of heavy pieces storming your opponent’s king. On the other hand it would be quiet and tactical, and you should know how to best make use of the couple of minor pieces and pawns that are on the board.

In closing

Hope you enjoyed this article on some beginner tips for levelling up in Chess. If you’d like to play Chess with me, challenge me on lichess at lichess.org/@/abhn.

Thank you for reading!

Concurrency In Python For Network I/O – Synchronous, Threading, Multiprocessing and Asynchronous IO

Uncategorized

In this article, let’s look at some of the ways to do batch HTTP requests in Python and some of the tools at our disposal. Mainly, we’ll look at the following ways:

  1. Synchronous with requests module
  2. Parallel with multiprocessing module
  3. Threaded with threading module
  4. Event loop based with asyncio module

I was inspired to do this because I had a task at hand which required me to write code for a superset of this problem. As someone who can write basic Python, I implemented the most straightforward solution with for loops and requests module. The task was to perform HTTP HEAD requests on a bunch of URLs, around 20,000 of them. The goal was to figure out which of URLs are non-existent (404s, timeouts and so on) in a bunch of markdown files. Averaging a couple of URLs a second on my home network and my personal computer, it would’ve taken a good 3-5 hours.

I knew there had to be a better way. The search for one lead to me learning a few things and this blog post. So let’s get started with each solution. Do note that depending on the hardware and network, it might go up and down quite a bit, but what’s interesting is the relative improvements we make going from one method to the next.

Synchronous code with requests module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

import time
from helper.urls import urls
import requests


def make_request(url):
    try:
        res = requests.head(url, timeout=3)
        print(res.status_code)
    except Exception as e:
        print(e.__class__.__name__)


def driver():
    for url in urls:
        make_request(url)


def main():
    start = time.perf_counter()
    driver()
    end = time.perf_counter()
    print(f'Synchronous: {end - start:.2f}')


if __name__ == '__main__':
    main()

If you try to think what you’d do if you had to check 10 URLs if they work, and write that process down in pseudocode form, this approach is pretty much what you’d get. Single threaded and synchronous, just like a human.

This script fetches 800 random URLs from a different file, makes an HTTP HEAD request to each, and then times the whole operation using time.perf_counter().

I ran all the tests on a Raspberry Pi 3 running freshly installed Ubuntu 20.04 LTS, but I don’t intend on making this scientific or reproducible, so don’t take the results at face value and test it yourself. Better yet, correct me and tell me how I could’ve done it better!

Synchronous code took 536 seconds to hit 800 URLs

With that we have our baseline. 536 seconds for 800 URLs that we can use to compare our other methods against.

Multiprocessing with multiprocessing module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import time
from helper.urls import urls
import requests
from multiprocessing import Pool
from multiprocessing import cpu_count


def make_request(url):
    try:
        res = requests.head(url, timeout=3)
        print(res.status_code)
    except Exception as e:
        print(e.__class__.__name__)


def driver():
    with Pool(cpu_count()) as p:
        p.map(make_request, urls)


def main():
    start = time.perf_counter()
    driver()
    end = time.perf_counter()
    print(f'Multiprocessing: {end - start:.2f}')


if __name__ == '__main__':
    main()

My Raspberry Pi is a quad core board, so it can have 4 processes running in parallel. Python conveniently provides us with a multiprocessing module that can help us do exactly that. Theoretically, we should see everything done in about 25% of the time (4x the processing power).

Multiprocessing solution took 121 seconds

So a bit less than 25% but if I run both the scripts over and over again, the average converges to a ratio of roughly 4:1 as we’d expect.

Threading with threading module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

import time
from helper.urls import urls
import requests
import threading


def make_request(url):
    try:
        res = requests.head(url, timeout=3)
        print(res.status_code)
    except Exception as e:
        print(e.__class__.__name__)


def driver():
    threads = []
    for url in urls:
        t = threading.Thread(target=make_request, args=(url,))
        threads.append(t)

    for t in threads:
        t.start()

    for t in threads:
        t.join()


def main():
    start = time.perf_counter()
    driver()
    end = time.perf_counter()
    print(f'Threading: {end - start:.2f}')


if __name__ == '__main__':
    main()

With threading, we essentially use just one process but offload the work to a number of thread that run concurrently (along with each other but not technically parallel).

Threading code runs in 41 seconds

Threading runs much faster than the multiprocessing, but that’s expected as threading is the right tool for network and I/O bound workload while multiprocessing suits CPU intensive workloads better.

Short detour: visualizing threading and multiprocessing

If you look into your system monitor, or use htop like I have in the following images, you’ll see how multiprocessing differs from threading. On my dual core (with 4 threads) personal computer, multiprocessing creates four processes (with only the default one thread per process), while threading solution creates a much larger number of threads all spawned from one process.


Code running the multiprocessing [above] and threading [below] solution as viewed through htop

This helped me better understand the difference between threads and processes on a high level and why threading solution is much faster for this particular workload.

Asynchronous code with asyncio module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import time
from helper.urls import urls
import httpx
import asyncio

async_client = httpx.AsyncClient()

async def make_request(url):
    try:
        res = await async_client.head(url, timeout=3)
        print(res.status_code)
    except Exception as e:
        print(e.__class__.__name__)


async def driver():
    await asyncio.gather(*[make_request(url) for url in urls])


def main():
    start = time.perf_counter()
    asyncio.run(driver())
    end = time.perf_counter()
    print(f'Async IO: {end - start:.2f}')


if __name__ == '__main__':
    main()

Finally, we reach Async IO, sort of the right tool for the job given its event driven nature. It was also what sparked my curiosity in the subject in the first place as it is quite fascinating, coming from the event driven JavaScript land, to find an event loop in Python. [Read: Callbacks And Event Loop Explained and Event Driven Programming]

Asynchronous IO code ran in 11 seconds

Clearly this task is made for this approach, and the code looks surprisingly simple to understand which is a plus.

In closing

That’s it for this little adventure in Python land I sincerely hope you enjoyed it. If you find any corrections and improvements in the above text, please write them to me. It will help me get better at Python (and writing )

Thank you for reading