I have been watching some of the game automation and Deep Learning/AI from Sentdex and Engineer Man on YouTube, which I wanted to learn for work (Huge fan of these guys BTW).

This is a far cry from that, but my goal is to ultimately read through tens of thousands of scanned documents and retrieve the data, which may be handwritten or typed. This means that I have to read the scan as an image, read a character, convert to a numpy array and make some decisions as to what the character is/should be. I’ll eventually be using a neural network to ‘learn’ the characters and improve the accuracy, but for now, I wanted something fun to do with the kids.

Yesterday, the new Minecraft “Caves and Cliffs” edition (1.17) was released and I thought it might be fun to write a little program that will auto-fish for me. This was purely an educational first step to write something that will be fun to write and see in action. This is not to say that reading scanned documents isn’t fun, but seeing little green XP bubbles appearing automatically is kind of cool. It’s all about the immediate gratification, right?!?

Here goes…

After a few trials and errors that I won’t bore you with, I finally settled on the following approach…

• When the program runs, have the character ready to cast into a fishable area. Casting should be the first action by the character, which is accomplished using pyautogui.rightClick().
• Use PIL to screengrab a small area around the cursor; i.e. if I put the cursor on the fishing bobber, take a small square screen capture around the fishing cursor.
• Convert the image to grayscale and increase the size (i.e. zoom in) to something manageable.

• Continue to take the grayed, zoomed screen captures every tenth of a second.

• When a fish is caught, the fishing line, which is black, will dip below the screen capture so that there are no more black pixels in the image.

• Once a fish is “caught”, use pyautogui.rightClick() again to pull in the catch.
• Then, loop the entire process to pull in that sweet, sweet loot and XP!

Now for the code…please note that this is a fast and dirty program to achieve the desired result. There are many improvements that can be added, but the scope was just a proof of concept to learn image manipulation in OpenCV. This is for educational purposes only!

#automation #python #minecraft #let’s go fishing! writing minecraft 1.17 auto-fishing bot in python, opencv and pyautogui #writing minecraft 1.17 auto-fishing bot #python, opencv and pyautogui