How I won a A$200 chair from Flexispot

I was looking for a sit-and-stand desk for my home office then incidentally came across the “Play-and-Win” challenge from Flexispot. The challenge was to find all pairs (or groups of digits) that sum up to 10 in a grid of digits within 60 seconds (see the video below). And most importantly, the top 10 players would win a A$200 chair!

Link to the game: https://www.flexispot.au/game-center/game

When I tried for the first time, I got a score of only 30+, which put me in the 20+ rank—not enough to get a prize. But I noticed that the game could be automated. So, I wrote a Python program to play the game for me .

The program includes:

• A function to read the grid from the screen using pytesseract and OpenCV.
• A simple grid search algorithm to find all groups that sum up to 10.
• A function to move the mouse and drag the groups using pyautogui.

There are some challenges when writing the program:

• I didn’t have much time because it overlapped with some deadlines at work, so I needed to write a simple program that worked. Ultimately, it took me around 7-8 hours to complete with a lot of joy and excitement. I was waking up at 2 am to work on the program LOL.
• There were very few chances to play the game (1 life per day), so I needed to make sure the program worked correctly with only a one or two chances to test. Solution . I recorded the screen while playing the game since the first time and used the data to understand the game mechanics and to test the program.
• Detecting the position of the grid and all the digits on the screen. Solution : I used pyautogui to get the position of the mouse and manually point to the top-left and bottom-right corners of the grid. Not something fancy, but it worked with minimal effort.
• Reading the digits on the screen correctly. Solution : I used pytesseract to read the digits. Somehow It could read the bounding box of the digits correctly, but not the digits themselves. So I had to post-process by comparing the bounding box with the template digits to get the correct prediction.
• Moving the mouse and dragging correctly to avoid mistakes. Solution I captured the grid and tested the program with the screenshot to make sure it worked correctly. There are also several tricks, such as adding margins to the positions, delays to the mouse movements, reset the mouse position, and reset the whole program if it detected something wrong, etc.
• Finding all possible groups that sum up to 10 in the shortest time. Solution : I defined a sequence of windows with different sizes (e.g., 1x2, 2x1, 2x2, 1x3, 3x1, etc.) and slid them through the grid. I also simulated the drag action, i.e., after scanning a window, all valid groups founded in this round will be disappeared from the grid. This way, I avoided using non-existent digits in the next windows. I also defined two sequences of windows (i.e., sequence 1 = [1x2, 2x1, 2x2, 1x3, 3x1, …], sequence 2 = [1x2, 2x1, 3x1, 1x3, 1x4, …]) and simulate them to choose the one that find the most valid groups (i.e., the highest score). All of this was done in simulation mode, and only the best sequence was executed in the real game. In the video, you can see that the program took around 7-8 seconds to run in simulation mode.

This program found 41 groups in 60 seconds, which was the top score at that time. My original plan was to release the code and the details here after the challenge ended, but I realized that Flexispot might have a similar challenge in the future, and I didn’t want to spoil their marketing campaign. So I decided to keep the code private for now. But it was a fun project to work on :D and of course, a nice free chair to win .