1598536884
How to Get Selected File Name From Input Type File using jQuery
#jquery #html #css
What is GEEK
Buddha Community
1586415180
Everything You Need to Know About Instagram Bot with Python
How to build an Instagram bot using Python
Instagram is the fastest-growing social network, with 1 billion monthly users. It also has the highest engagement rate. To gain followers on Instagram, you’d have to upload engaging content, follow users, like posts, comment on user posts and a whole lot. This can be time-consuming and daunting. But there is hope, you can automate all of these tasks. In this course, we’re going to build an Instagram bot using Python to automate tasks on Instagram.
What you’ll learn:
- Instagram Automation
- Build a Bot with Python
Increase your Instagram followers with a simple Python bot
I got around 500 real followers in 4 days!
Growing an audience is an expensive and painful task. And if you’d like to build an audience that’s relevant to you, and shares common interests, that’s even more difficult. I always saw Instagram has a great way to promote my photos, but I never had more than 380 followers… Every once in a while, I decide to start posting my photos on Instagram again, and I manage to keep posting regularly for a while, but it never lasts more than a couple of months, and I don’t have many followers to keep me motivated and engaged.
The objective of this project is to build a bigger audience and as a plus, maybe drive some traffic to my website where I sell my photos!
A year ago, on my last Instagram run, I got one of those apps that lets you track who unfollowed you. I was curious because in a few occasions my number of followers dropped for no apparent reason. After some research, I realized how some users basically crawl for followers. They comment, like and follow people — looking for a follow back. Only to unfollow them again in the next days.
I can’t say this was a surprise to me, that there were bots in Instagram… It just made me want to build one myself!
And that is why we’re here, so let’s get to it! I came up with a simple bot in Python, while I was messing around with Selenium and trying to figure out some project to use it. Simply put, Selenium is like a browser you can interact with very easily in Python.
Ideally, increasing my Instagram audience will keep me motivated to post regularly. As an extra, I included my website in my profile bio, where people can buy some photos. I think it is a bit of a stretch, but who knows?! My sales are basically zero so far, so it should be easy to track that conversion!
Just what the world needed! Another Instagram bot…
After giving this project some thought, my objective was to increase my audience with relevant people. I want to get followers that actually want to follow me and see more of my work. It’s very easy to come across weird content in the most used hashtags, so I’ve planed this bot to lookup specific hashtags and interact with the photos there. This way, I can be very specific about what kind of interests I want my audience to have. For instance, I really like long exposures, so I can target people who use that hashtag and build an audience around this kind of content. Simple and efficient!
My gallery is a mix of different subjects and styles, from street photography to aerial photography, and some travel photos too. Since it’s my hometown, I also have lots of Lisbon images there. These will be the main topics I’ll use in the hashtags I want to target.
This is not a “get 1000 followers in 24 hours” kind of bot!
So what kind of numbers are we talking about?
I ran the bot a few times in a few different hashtags like “travelblogger”, “travelgram”, “lisbon”, “dronephotography”. In the course of three days I went from 380 to 800 followers. Lots of likes, comments and even some organic growth (people that followed me but were not followed by the bot).
To be clear, I’m not using this bot intensively, as Instagram will stop responding if you run it too fast. It needs to have some sleep commands in between the actions, because after some comments and follows in a short period of time, Instagram stops responding and the bot crashes.
You will be logged into your account, so I’m almost sure that Instagram can know you’re doing something weird if you speed up the process. And most importantly, after doing this for a dozen hashtags, it just gets harder to find new users in the same hashtags. You will need to give it a few days to refresh the user base there.
But I don’t want to follow so many people in the process…
The most efficient way to get followers in Instagram (apart from posting great photos!) is to follow people. And this bot worked really well for me because I don’t care if I follow 2000 people to get 400 followers.
The bot saves a list with all the users that were followed while it was running, so someday I may actually do something with this list. For instance, I can visit each user profile, evaluate how many followers or posts they have, and decide if I want to keep following them. Or I can get the first picture in their gallery and check its date to see if they are active users.
If we remove the follow action from the bot, I can assure you the growth rate will suffer, as people are less inclined to follow based on a single like or comment.
Why will you share your code?!
That’s the debate I had with myself. Even though I truly believe in giving back to the community (I still learn a lot from it too!), there are several paid platforms that do more or less the same as this project. Some are shady, some are used by celebrities. The possibility of starting a similar platform myself, is not off the table yet, so why make the code available?
With that in mind, I decided to add an extra level of difficulty to the process, so I was going to post the code below as an image. I wrote “was”, because meanwhile, I’ve realized the image I’m getting is low quality. Which in turn made me reconsider and post the gist. I’m that nice! The idea behind the image was that if you really wanted to use it, you would have to type the code yourself. And that was my way of limiting the use of this tool to people that actually go through the whole process to create it and maybe even improve it.
I learn a lot more when I type the code myself, instead of copy/pasting scripts. I hope you feel the same way!
The script isn’t as sophisticated as it could be, and I know there’s lots of room to improve it. But hey… it works! I have other projects I want to add to my portfolio, so my time to develop it further is rather limited. Nevertheless, I will try to update this article if I dig deeper.
This is the last subtitle!
You’ll need Python (I’m using Python 3.7), Selenium, a browser (in my case I’ll be using Chrome) and… obviously, an Instagram account! Quick overview regarding what the bot will do:
- Open a browser and login with your credentials
- For every hashtag in the hashtag list, it will open the page and click the first picture to open it
- It will then like, follow, comment and move to the next picture, in a 200 iterations loop (number can be adjusted)
- Saves a list with all the users you followed using the bot
If you reached this paragraph, thank you! You totally deserve to collect your reward! If you find this useful for your profile/brand in any way, do share your experience below :)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep, strftime
from random import randint
import pandas as pd
chromedriver_path = 'C:/Users/User/Downloads/chromedriver_win32/chromedriver.exe' # Change this to your own chromedriver path!
webdriver = webdriver.Chrome(executable_path=chromedriver_path)
sleep(2)
webdriver.get('https://www.instagram.com/accounts/login/?source=auth_switcher')
sleep(3)
username = webdriver.find_element_by_name('username')
username.send_keys('your_username')
password = webdriver.find_element_by_name('password')
password.send_keys('your_password')
button_login = webdriver.find_element_by_css_selector('#react-root > section > main > div > article > div > div:nth-child(1) > div > form > div:nth-child(3) > button')
button_login.click()
sleep(3)
notnow = webdriver.find_element_by_css_selector('body > div:nth-child(13) > div > div > div > div.mt3GC > button.aOOlW.HoLwm')
notnow.click() #comment these last 2 lines out, if you don't get a pop up asking about notifications
In order to use chrome with Selenium, you need to install chromedriver. It’s a fairly simple process and I had no issues with it. Simply install and replace the path above. Once you do that, our variable webdriver will be our Chrome tab.
In cell number 3 you should replace the strings with your own username and the respective password. This is for the bot to type it in the fields displayed. You might have already noticed that when running cell number 2, Chrome opened a new tab. After the password, I’ll define the login button as an object, and in the following line, I click it.
Once you get in inspect mode find the bit of html code that corresponds to what you want to map. Right click it and hover over Copy. You will see that you have some options regarding how you want it to be copied. I used a mix of XPath and css selectors throughout the code (it’s visible in the find_element_ method). It took me a while to get all the references to run smoothly. At points, the css or the xpath directions would fail, but as I adjusted the sleep times, everything started running smoothly.
In this case, I selected “copy selector” and pasted it inside a find_element_ method (cell number 3). It will get you the first result it finds. If it was find_elements_, all elements would be retrieved and you could specify which to get.
Once you get that done, time for the loop. You can add more hashtags in the hashtag_list. If you run it for the first time, you still don’t have a file with the users you followed, so you can simply create prev_user_list as an empty list.
Once you run it once, it will save a csv file with a timestamp with the users it followed. That file will serve as the prev_user_list on your second run. Simple and easy to keep track of what the bot does.
Update with the latest timestamp on the following runs and you get yourself a series of csv backlogs for every run of the bot.
The code is really simple. If you have some basic notions of Python you can probably pick it up quickly. I’m no Python ninja and I was able to build it, so I guess that if you read this far, you are good to go!
hashtag_list = ['travelblog', 'travelblogger', 'traveler']
# prev_user_list = [] - if it's the first time you run it, use this line and comment the two below
prev_user_list = pd.read_csv('20181203-224633_users_followed_list.csv', delimiter=',').iloc[:,1:2] # useful to build a user log
prev_user_list = list(prev_user_list['0'])
new_followed = []
tag = -1
followed = 0
likes = 0
comments = 0
for hashtag in hashtag_list:
tag += 1
webdriver.get('https://www.instagram.com/explore/tags/'+ hashtag_list[tag] + '/')
sleep(5)
first_thumbnail = webdriver.find_element_by_xpath('//*[@id="react-root"]/section/main/article/div[1]/div/div/div[1]/div[1]/a/div')
first_thumbnail.click()
sleep(randint(1,2))
try:
for x in range(1,200):
username = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[1]/h2/a').text
if username not in prev_user_list:
# If we already follow, do not unfollow
if webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[2]/button').text == 'Follow':
webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[2]/button').click()
new_followed.append(username)
followed += 1
# Liking the picture
button_like = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[1]/span[1]/button/span')
button_like.click()
likes += 1
sleep(randint(18,25))
# Comments and tracker
comm_prob = randint(1,10)
print('{}_{}: {}'.format(hashtag, x,comm_prob))
if comm_prob > 7:
comments += 1
webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[1]/span[2]/button/span').click()
comment_box = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[3]/div/form/textarea')
if (comm_prob < 7):
comment_box.send_keys('Really cool!')
sleep(1)
elif (comm_prob > 6) and (comm_prob < 9):
comment_box.send_keys('Nice work :)')
sleep(1)
elif comm_prob == 9:
comment_box.send_keys('Nice gallery!!')
sleep(1)
elif comm_prob == 10:
comment_box.send_keys('So cool! :)')
sleep(1)
# Enter to post comment
comment_box.send_keys(Keys.ENTER)
sleep(randint(22,28))
# Next picture
webdriver.find_element_by_link_text('Next').click()
sleep(randint(25,29))
else:
webdriver.find_element_by_link_text('Next').click()
sleep(randint(20,26))
# some hashtag stops refreshing photos (it may happen sometimes), it continues to the next
except:
continue
for n in range(0,len(new_followed)):
prev_user_list.append(new_followed[n])
updated_user_df = pd.DataFrame(prev_user_list)
updated_user_df.to_csv('{}_users_followed_list.csv'.format(strftime("%Y%m%d-%H%M%S")))
print('Liked {} photos.'.format(likes))
print('Commented {} photos.'.format(comments))
print('Followed {} new people.'.format(followed))
The print statement inside the loop is the way I found to be able to have a tracker that lets me know at what iteration the bot is all the time. It will print the hashtag it’s in, the number of the iteration, and the random number generated for the comment action. I decided not to post comments in every page, so I added three different comments and a random number between 1 and 10 that would define if there was any comment at all, or one of the three. The loop ends, we append the new_followed users to the previous users “database” and saves the new file with the timestamp. You should also get a small report.
And that’s it!
After a few hours without checking the phone, these were the numbers I was getting. I definitely did not expect it to do so well! In about 4 days since I’ve started testing it, I had around 500 new followers, which means I have doubled my audience in a matter of days. I’m curious to see how many of these new followers I will lose in the next days, to see if the growth can be sustainable. I also had a lot more “likes” in my latest photos, but I guess that’s even more expected than the follow backs.
It would be nice to get this bot running in a server, but I have other projects I want to explore, and configuring a server is not one of them! Feel free to leave a comment below, and I’ll do my best to answer your questions.
I’m actually curious to see how long will I keep posting regularly! If you feel like this article was helpful for you, consider thanking me by buying one of my photos.
How to Make an Instagram Bot With Python and InstaPy
What do SocialCaptain, Kicksta, Instavast, and many other companies have in common? They all help you reach a greater audience, gain more followers, and get more likes on Instagram while you hardly lift a finger. They do it all through automation, and people pay them a good deal of money for it. But you can do the same thing—for free—using InstaPy!
In this tutorial, you’ll learn how to build a bot with Python and InstaPy, which automates your Instagram activities so that you gain more followers and likes with minimal manual input. Along the way, you’ll learn about browser automation with Selenium and the Page Object Pattern, which together serve as the basis for InstaPy.
In this tutorial, you’ll learn:
- How Instagram bots work
- How to automate a browser with Selenium
- How to use the Page Object Pattern for better readability and testability
- How to build an Instagram bot with InstaPy
You’ll begin by learning how Instagram bots work before you build one.
Table of Contents
- How Instagram Bots Work
- How to Automate a Browser
- How to Use the Page Object Pattern
- How to Build an Instagram Bot With InstaPy
- Essential Features
- Additional Features in InstaPy
- Conclusion
Important: Make sure you check Instagram’s Terms of Use before implementing any kind of automation or scraping techniques.
How Instagram Bots Work
How can an automation script gain you more followers and likes? Before answering this question, think about how an actual person gains more followers and likes.
They do it by being consistently active on the platform. They post often, follow other people, and like and leave comments on other people’s posts. Bots work exactly the same way: They follow, like, and comment on a consistent basis according to the criteria you set.
The better the criteria you set, the better your results will be. You want to make sure you’re targeting the right groups because the people your bot interacts with on Instagram will be more likely to interact with your content.
For example, if you’re selling women’s clothing on Instagram, then you can instruct your bot to like, comment on, and follow mostly women or profiles whose posts include hashtags such as #beauty, #fashion, or #clothes. This makes it more likely that your target audience will notice your profile, follow you back, and start interacting with your posts.
How does it work on the technical side, though? You can’t use the Instagram Developer API since it is fairly limited for this purpose. Enter browser automation. It works in the following way:
- You serve it your credentials.
- You set the criteria for who to follow, what comments to leave, and which type of posts to like.
- Your bot opens a browser, types in
https://instagram.comon the address bar, logs in with your credentials, and starts doing the things you instructed it to do.
Next, you’ll build the initial version of your Instagram bot, which will automatically log in to your profile. Note that you won’t use InstaPy just yet.
How to Automate a Browser
For this version of your Instagram bot, you’ll be using Selenium, which is the tool that InstaPy uses under the hood.
First, install Selenium. During installation, make sure you also install the Firefox WebDriver since the latest version of InstaPy dropped support for Chrome. This also means that you need the Firefox browser installed on your computer.
Now, create a Python file and write the following code in it:
from time import sleep
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('https://www.instagram.com/')
sleep(5)
browser.close()
Run the code and you’ll see that a Firefox browser opens and directs you to the Instagram login page. Here’s a line-by-line breakdown of the code:
- Lines 1 and 2 import
sleepandwebdriver. - Line 4 initializes the Firefox driver and sets it to
browser. - Line 6 types
https://www.instagram.com/on the address bar and hits Enter. - Line 8 waits for five seconds so you can see the result. Otherwise, it would close the browser instantly.
- Line 10 closes the browser.
This is the Selenium version of Hello, World. Now you’re ready to add the code that logs in to your Instagram profile. But first, think about how you would log in to your profile manually. You would do the following:
- Go to
https://www.instagram.com/. - Click the login link.
- Enter your credentials.
- Hit the login button.
The first step is already done by the code above. Now change it so that it clicks on the login link on the Instagram home page:
from time import sleep
from selenium import webdriver
browser = webdriver.Firefox()
browser.implicitly_wait(5)
browser.get('https://www.instagram.com/')
login_link = browser.find_element_by_xpath("//a[text()='Log in']")
login_link.click()
sleep(5)
browser.close()
Note the highlighted lines:
- Line 5 sets five seconds of waiting time. If Selenium can’t find an element, then it waits for five seconds to allow everything to load and tries again.
- Line 9 finds the element
<a>whose text is equal toLog in. It does this using XPath, but there are a few other methods you could use. - Line 10 clicks on the found element
<a>for the login link.
Run the script and you’ll see your script in action. It will open the browser, go to Instagram, and click on the login link to go to the login page.
On the login page, there are three important elements:
- The username input
- The password input
- The login button
Next, change the script so that it finds those elements, enters your credentials, and clicks on the login button:
from time import sleep
from selenium import webdriver
browser = webdriver.Firefox()
browser.implicitly_wait(5)
browser.get('https://www.instagram.com/')
login_link = browser.find_element_by_xpath("//a[text()='Log in']")
login_link.click()
sleep(2)
username_input = browser.find_element_by_css_selector("input[name='username']")
password_input = browser.find_element_by_css_selector("input[name='password']")
username_input.send_keys("<your username>")
password_input.send_keys("<your password>")
login_button = browser.find_element_by_xpath("//button[@type='submit']")
login_button.click()
sleep(5)
browser.close()
Here’s a breakdown of the changes:
- Line 12 sleeps for two seconds to allow the page to load.
- Lines 14 and 15 find username and password inputs by CSS. You could use any other method that you prefer.
- Lines 17 and 18 type your username and password in their respective inputs. Don’t forget to fill in
<your username>and<your password>! - Line 20 finds the login button by XPath.
- Line 21 clicks on the login button.
Run the script and you’ll be automatically logged in to to your Instagram profile.
You’re off to a good start with your Instagram bot. If you were to continue writing this script, then the rest would look very similar. You would find the posts that you like by scrolling down your feed, find the like button by CSS, click on it, find the comments section, leave a comment, and continue.
The good news is that all of those steps can be handled by InstaPy. But before you jump into using Instapy, there is one other thing that you should know about to better understand how InstaPy works: the Page Object Pattern.
How to Use the Page Object Pattern
Now that you’ve written the login code, how would you write a test for it? It would look something like the following:
def test_login_page(browser):
browser.get('https://www.instagram.com/accounts/login/')
username_input = browser.find_element_by_css_selector("input[name='username']")
password_input = browser.find_element_by_css_selector("input[name='password']")
username_input.send_keys("<your username>")
password_input.send_keys("<your password>")
login_button = browser.find_element_by_xpath("//button[@type='submit']")
login_button.click()
errors = browser.find_elements_by_css_selector('#error_message')
assert len(errors) == 0
Can you see what’s wrong with this code? It doesn’t follow the DRY principle. That is, the code is duplicated in both the application and the test code.
Duplicating code is especially bad in this context because Selenium code is dependent on UI elements, and UI elements tend to change. When they do change, you want to update your code in one place. That’s where the Page Object Pattern comes in.
With this pattern, you create page object classes for the most important pages or fragments that provide interfaces that are straightforward to program to and that hide the underlying widgetry in the window. With this in mind, you can rewrite the code above and create a HomePage class and a LoginPage class:
from time import sleep
class LoginPage:
def __init__(self, browser):
self.browser = browser
def login(self, username, password):
username_input = self.browser.find_element_by_css_selector("input[name='username']")
password_input = self.browser.find_element_by_css_selector("input[name='password']")
username_input.send_keys(username)
password_input.send_keys(password)
login_button = browser.find_element_by_xpath("//button[@type='submit']")
login_button.click()
sleep(5)
class HomePage:
def __init__(self, browser):
self.browser = browser
self.browser.get('https://www.instagram.com/')
def go_to_login_page(self):
self.browser.find_element_by_xpath("//a[text()='Log in']").click()
sleep(2)
return LoginPage(self.browser)
The code is the same except that the home page and the login page are represented as classes. The classes encapsulate the mechanics required to find and manipulate the data in the UI. That is, there are methods and accessors that allow the software to do anything a human can.
One other thing to note is that when you navigate to another page using a page object, it returns a page object for the new page. Note the returned value of go_to_log_in_page(). If you had another class called FeedPage, then login() of the LoginPage class would return an instance of that: return FeedPage().
Here’s how you can put the Page Object Pattern to use:
from selenium import webdriver
browser = webdriver.Firefox()
browser.implicitly_wait(5)
home_page = HomePage(browser)
login_page = home_page.go_to_login_page()
login_page.login("<your username>", "<your password>")
browser.close()
It looks much better, and the test above can now be rewritten to look like this:
def test_login_page(browser):
home_page = HomePage(browser)
login_page = home_page.go_to_login_page()
login_page.login("<your username>", "<your password>")
errors = browser.find_elements_by_css_selector('#error_message')
assert len(errors) == 0
With these changes, you won’t have to touch your tests if something changes in the UI.
For more information on the Page Object Pattern, refer to the official documentation and to Martin Fowler’s article.
Now that you’re familiar with both Selenium and the Page Object Pattern, you’ll feel right at home with InstaPy. You’ll build a basic bot with it next.
Note: Both Selenium and the Page Object Pattern are widely used for other websites, not just for Instagram.
How to Build an Instagram Bot With InstaPy
In this section, you’ll use InstaPy to build an Instagram bot that will automatically like, follow, and comment on different posts. First, you’ll need to install InstaPy:
$ python3 -m pip install instapy
This will install instapy in your system.
Essential Features
Now you can rewrite the code above with InstaPy so that you can compare the two options. First, create another Python file and put the following code in it:
from instapy import InstaPy
InstaPy(username="<your_username>", password="<your_password>").login()
Replace the username and password with yours, run the script, and voilà! With just one line of code, you achieved the same result.
Even though your results are the same, you can see that the behavior isn’t exactly the same. In addition to simply logging in to your profile, InstaPy does some other things, such as checking your internet connection and the status of the Instagram servers. This can be observed directly on the browser or in the logs:
INFO [2019-12-17 22:03:19] [username] -- Connection Checklist [1/3] (Internet Connection Status)
INFO [2019-12-17 22:03:20] [username] - Internet Connection Status: ok
INFO [2019-12-17 22:03:20] [username] - Current IP is "17.283.46.379" and it's from "Germany/DE"
INFO [2019-12-17 22:03:20] [username] -- Connection Checklist [2/3] (Instagram Server Status)
INFO [2019-12-17 22:03:26] [username] - Instagram WebSite Status: Currently Up
Pretty good for one line of code, isn’t it? Now it’s time to make the script do more interesting things than just logging in.
For the purpose of this example, assume that your profile is all about cars, and that your bot is intended to interact with the profiles of people who are also interested in cars.
First, you can like some posts that are tagged #bmw or #mercedes using like_by_tags():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
Here, you gave the method a list of tags to like and the number of posts to like for each given tag. In this case, you instructed it to like ten posts, five for each of the two tags. But take a look at what happens after you run the script:
INFO [2019-12-17 22:15:58] [username] Tag [1/2]
INFO [2019-12-17 22:15:58] [username] --> b'bmw'
INFO [2019-12-17 22:16:07] [username] desired amount: 14 | top posts [disabled]: 9 | possible posts: 43726739
INFO [2019-12-17 22:16:13] [username] Like# [1/14]
INFO [2019-12-17 22:16:13] [username] https://www.instagram.com/p/B6MCcGcC3tU/
INFO [2019-12-17 22:16:15] [username] Image from: b'mattyproduction'
INFO [2019-12-17 22:16:15] [username] Link: b'https://www.instagram.com/p/B6MCcGcC3tU/'
INFO [2019-12-17 22:16:15] [username] Description: b'Mal etwas anderes \xf0\x9f\x91\x80\xe2\x98\xba\xef\xb8\x8f Bald ist das komplette Video auf YouTube zu finden (n\xc3\xa4here Infos werden folgen). Vielen Dank an @patrick_jwki @thehuthlife und @christic_ f\xc3\xbcr das bereitstellen der Autos \xf0\x9f\x94\xa5\xf0\x9f\x98\x8d#carporn#cars#tuning#bagged#bmw#m2#m2competition#focusrs#ford#mk3#e92#m3#panasonic#cinematic#gh5s#dji#roninm#adobe#videography#music#bimmer#fordperformance#night#shooting#'
INFO [2019-12-17 22:16:15] [username] Location: b'K\xc3\xb6ln, Germany'
INFO [2019-12-17 22:16:51] [username] --> Image Liked!
INFO [2019-12-17 22:16:56] [username] --> Not commented
INFO [2019-12-17 22:16:57] [username] --> Not following
INFO [2019-12-17 22:16:58] [username] Like# [2/14]
INFO [2019-12-17 22:16:58] [username] https://www.instagram.com/p/B6MDK1wJ-Kb/
INFO [2019-12-17 22:17:01] [username] Image from: b'davs0'
INFO [2019-12-17 22:17:01] [username] Link: b'https://www.instagram.com/p/B6MDK1wJ-Kb/'
INFO [2019-12-17 22:17:01] [username] Description: b'Someone said cloud? \xf0\x9f\xa4\x94\xf0\x9f\xa4\xad\xf0\x9f\x98\x88 \xe2\x80\xa2\n\xe2\x80\xa2\n\xe2\x80\xa2\n\xe2\x80\xa2\n#bmw #bmwrepost #bmwm4 #bmwm4gts #f82 #bmwmrepost #bmwmsport #bmwmperformance #bmwmpower #bmwm4cs #austinyellow #davs0 #mpower_official #bmw_world_ua #bimmerworld #bmwfans #bmwfamily #bimmers #bmwpost #ultimatedrivingmachine #bmwgang #m3f80 #m5f90 #m4f82 #bmwmafia #bmwcrew #bmwlifestyle'
INFO [2019-12-17 22:17:34] [username] --> Image Liked!
INFO [2019-12-17 22:17:37] [username] --> Not commented
INFO [2019-12-17 22:17:38] [username] --> Not following
By default, InstaPy will like the first nine top posts in addition to your amount value. In this case, that brings the total number of likes per tag to fourteen (nine top posts plus the five you specified in amount).
Also note that InstaPy logs every action it takes. As you can see above, it mentions which post it liked as well as its link, description, location, and whether the bot commented on the post or followed the author.
You may have noticed that there are delays after almost every action. That’s by design. It prevents your profile from getting banned on Instagram.
Now, you probably don’t want your bot liking inappropriate posts. To prevent that from happening, you can use set_dont_like():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
With this change, posts that have the words naked or nsfw in their descriptions won’t be liked. You can flag any other words that you want your bot to avoid.
Next, you can tell the bot to not only like the posts but also to follow some of the authors of those posts. You can do that with set_do_follow():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
If you run the script now, then the bot will follow fifty percent of the users whose posts it liked. As usual, every action will be logged.
You can also leave some comments on the posts. There are two things that you need to do. First, enable commenting with set_do_comment():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)
Next, tell the bot what comments to leave with set_comments():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)
session.set_comments(["Nice!", "Sweet!", "Beautiful :heart_eyes:"])
Run the script and the bot will leave one of those three comments on half the posts that it interacts with.
Now that you’re done with the basic settings, it’s a good idea to end the session with end():
from instapy import InstaPy
session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)
session.set_comments(["Nice!", "Sweet!", "Beautiful :heart_eyes:"])
session.end()
This will close the browser, save the logs, and prepare a report that you can see in the console output.
Additional Features in InstaPy
InstaPy is a sizable project that has a lot of thoroughly documented features. The good news is that if you’re feeling comfortable with the features you used above, then the rest should feel pretty similar. This section will outline some of the more useful features of InstaPy.
Quota Supervisor
You can’t scrape Instagram all day, every day. The service will quickly notice that you’re running a bot and will ban some of its actions. That’s why it’s a good idea to set quotas on some of your bot’s actions. Take the following for example:
session.set_quota_supervisor(enabled=True, peak_comments_daily=240, peak_comments_hourly=21)
The bot will keep commenting until it reaches its hourly and daily limits. It will resume commenting after the quota period has passed.
Headless Browser
This feature allows you to run your bot without the GUI of the browser. This is super useful if you want to deploy your bot to a server where you may not have or need the graphical interface. It’s also less CPU intensive, so it improves performance. You can use it like so:
session = InstaPy(username='test', password='test', headless_browser=True)
Note that you set this flag when you initialize the InstaPy object.
Using AI to Analyze Posts
Earlier you saw how to ignore posts that contain inappropriate words in their descriptions. What if the description is good but the image itself is inappropriate? You can integrate your InstaPy bot with ClarifAI, which offers image and video recognition services:
session.set_use_clarifai(enabled=True, api_key='<your_api_key>')
session.clarifai_check_img_for(['nsfw'])
Now your bot won’t like or comment on any image that ClarifAI considers NSFW. You get 5,000 free API-calls per month.
Relationship Bounds
It’s often a waste of time to interact with posts by people who have a lot of followers. In such cases, it’s a good idea to set some relationship bounds so that your bot doesn’t waste your precious computing resources:
session.set_relationship_bounds(enabled=True, max_followers=8500)
With this, your bot won’t interact with posts by users who have more than 8,500 followers.
For many more features and configurations in InstaPy, check out the documentation.
Conclusion
InstaPy allows you to automate your Instagram activities with minimal fuss and effort. It’s a very flexible tool with a lot of useful features.
In this tutorial, you learned:
- How Instagram bots work
- How to automate a browser with Selenium
- How to use the Page Object Pattern to make your code more maintainable and testable
- How to use InstaPy to build a basic Instagram bot
Read the InstaPy documentation and experiment with your bot a little bit. Soon you’ll start getting new followers and likes with a minimal amount of effort. I gained a few new followers myself while writing this tutorial.
Automating Instagram API with Python
Gain active followers - Algorithm
Maybe some of you do not agree it is a good way to grow your IG page by using follow for follow method but after a lot of researching I found the proper way to use this method.
I have done and used this strategy for a while and my page visits also followers started growing.
The majority of people failing because they randomly targeting the followers and as a result, they are not coming back to your page. So, the key is to find people those have same interests with you.
If you have a programming page go and search for IG pages which have big programming community and once you find one, don’t send follow requests to followers of this page. Because some of them are not active even maybe fake accounts. So, in order to gain active followers, go the last post of this page and find people who liked the post.
Unofficial Instagram API
In order to query data from Instagram I am going to use the very cool, yet unofficial, Instagram API written by Pasha Lev.
**Note:**Before you test it make sure you verified your phone number in your IG account.
The program works pretty well so far but in case of any problems I have to put disclaimer statement here:
Disclaimer: This post published educational purposes only as well as to give general information about Instagram API. I am not responsible for any actions and you are taking your own risk.
Let’s start by installing and then logging in with API.
pip install InstagramApi
from InstagramAPI import InstagramAPI
api = InstagramAPI("username", "password")
api.login()
Once you run the program you will see “Login success!” in your console.
Get users from liked list
We are going to search for some username (your target page) then get most recent post from this user. Then, get users who liked this post. Unfortunately, I can’t find solution how to paginate users so right now it gets about last 500 user.
users_list = []
def get_likes_list(username):
api.login()
api.searchUsername(username)
result = api.LastJson
username_id = result['user']['pk'] # Get user ID
user_posts = api.getUserFeed(username_id) # Get user feed
result = api.LastJson
media_id = result['items'][0]['id'] # Get most recent post
api.getMediaLikers(media_id) # Get users who liked
users = api.LastJson['users']
for user in users: # Push users to list
users_list.append({'pk':user['pk'], 'username':user['username']})
Follow Users
Once we get the users list, it is time to follow these users.
IMPORTANT NOTE: set time limit as much as you can to avoid automation detection.
from time import sleep
following_users = []
def follow_users(users_list):
api.login()
api.getSelfUsersFollowing() # Get users which you are following
result = api.LastJson
for user in result['users']:
following_users.append(user['pk'])
for user in users_list:
if not user['pk'] in following_users: # if new user is not in your following users
print('Following @' + user['username'])
api.follow(user['pk'])
# after first test set this really long to avoid from suspension
sleep(20)
else:
print('Already following @' + user['username'])
sleep(10)
Unfollow Users
This function will look users which you are following then it will check if this user follows you as well. If user not following you then you are unfollowing as well.
follower_users = []
def unfollow_users():
api.login()
api.getSelfUserFollowers() # Get your followers
result = api.LastJson
for user in result['users']:
follower_users.append({'pk':user['pk'], 'username':user['username']})
api.getSelfUsersFollowing() # Get users which you are following
result = api.LastJson
for user in result['users']:
following_users.append({'pk':user['pk'],'username':user['username']})
for user in following_users:
if not user['pk'] in follower_users: # if the user not follows you
print('Unfollowing @' + user['username'])
api.unfollow(user['pk'])
# set this really long to avoid from suspension
sleep(20)
Full Code with extra functions
Here is the full code of this automation
import pprint
from time import sleep
from InstagramAPI import InstagramAPI
import pandas as pd
users_list = []
following_users = []
follower_users = []
class InstaBot:
def __init__(self):
self.api = InstagramAPI("your_username", "your_password")
def get_likes_list(self,username):
api = self.api
api.login()
api.searchUsername(username) #Gets most recent post from user
result = api.LastJson
username_id = result['user']['pk']
user_posts = api.getUserFeed(username_id)
result = api.LastJson
media_id = result['items'][0]['id']
api.getMediaLikers(media_id)
users = api.LastJson['users']
for user in users:
users_list.append({'pk':user['pk'], 'username':user['username']})
bot.follow_users(users_list)
def follow_users(self,users_list):
api = self.api
api.login()
api.getSelfUsersFollowing()
result = api.LastJson
for user in result['users']:
following_users.append(user['pk'])
for user in users_list:
if not user['pk'] in following_users:
print('Following @' + user['username'])
api.follow(user['pk'])
# set this really long to avoid from suspension
sleep(20)
else:
print('Already following @' + user['username'])
sleep(10)
def unfollow_users(self):
api = self.api
api.login()
api.getSelfUserFollowers()
result = api.LastJson
for user in result['users']:
follower_users.append({'pk':user['pk'], 'username':user['username']})
api.getSelfUsersFollowing()
result = api.LastJson
for user in result['users']:
following_users.append({'pk':user['pk'],'username':user['username']})
for user in following_users:
if not user['pk'] in [user['pk'] for user in follower_users]:
print('Unfollowing @' + user['username'])
api.unfollow(user['pk'])
# set this really long to avoid from suspension
sleep(20)
bot = InstaBot()
# To follow users run the function below
# change the username ('instagram') to your target username
bot.get_likes_list('instagram')
# To unfollow users uncomment and run the function below
# bot.unfollow_users()
it will look like this:
some extra functions to play with API:
def get_my_profile_details():
api.login()
api.getSelfUsernameInfo()
result = api.LastJson
username = result['user']['username']
full_name = result['user']['full_name']
profile_pic_url = result['user']['profile_pic_url']
followers = result['user']['follower_count']
following = result['user']['following_count']
media_count = result['user']['media_count']
df_profile = pd.DataFrame(
{'username':username,
'full name': full_name,
'profile picture URL':profile_pic_url,
'followers':followers,
'following':following,
'media count': media_count,
}, index=[0])
df_profile.to_csv('profile.csv', sep='\t', encoding='utf-8')
def get_my_feed():
image_urls = []
api.login()
api.getSelfUserFeed()
result = api.LastJson
# formatted_json_str = pprint.pformat(result)
# print(formatted_json_str)
if 'items' in result.keys():
for item in result['items'][0:5]:
if 'image_versions2' in item.keys():
image_url = item['image_versions2']['candidates'][1]['url']
image_urls.append(image_url)
df_feed = pd.DataFrame({
'image URL':image_urls
})
df_feed.to_csv('feed.csv', sep='\t', encoding='utf-8')
Building an Instagram Bot with Python and Selenium to Gain More Followers
Let’s build an Instagram bot to gain more followers! — I know, I know. That doesn’t sound very ethical, does it? But it’s all justified for educational purposes.
Coding is a super power — we can all agree. That’s why I’ll leave it up to you to not abuse this power. And I trust you’re here to learn how it works. Otherwise, you’d be on GitHub cloning one of the countless Instagram bots there, right?
You’re convinced? — Alright, now let’s go back to unethical practices.
The Plan
So here’s the deal, we want to build a bot in Python and Selenium that goes on the hashtags we specify, likes random posts, then follows the posters. It does that enough — we get follow backs. Simple as that.
Here’s a pretty twisted detail though: we want to keep track of the users we follow so the bot can unfollow them after the number of days we specify.
Setup
So first things first, I want to use a database to keep track of the username and the date added. You might as well save/load from/to a file, but we want this to be ready for more features in case we felt inspired in the future.
So make sure you create a database (I named mine instabot — but you can name it anything you like) and create a table called followed_users within the database with two fields (username, date_added)
Remember the installation path. You’ll need it.
You’ll also need the following python packages:
- selenium
- mysql-connector
Getting down to it
Alright, so first thing we’ll be doing is creating settings.json. Simply a .json file that will hold all of our settings so we don’t have to dive into the code every time we want to change something.
Settings
settings.json:
{
"db": {
"host": "localhost",
"user": "root",
"pass": "",
"database": "instabot"
},
"instagram": {
"user": "",
"pass": ""
},
"config": {
"days_to_unfollow": 1,
"likes_over": 150,
"check_followers_every": 3600,
"hashtags": []
}
}
As you can see, under “db”, we specify the database information. As I mentioned, I used “instabot”, but feel free to use whatever name you want.
You’ll also need to fill Instagram info under “instagram” so the bot can login into your account.
“config” is for our bot’s settings. Here’s what the fields mean:
days_to_unfollow: number of days before unfollowing users
likes_over: ignore posts if the number of likes is above this number
check_followers_every: number of seconds before checking if it’s time to unfollow any of the users
hashtags: a list of strings with the hashtag names the bot should be active on
Constants
Now, we want to take these settings and have them inside our code as constants.
Create Constants.py:
import json
INST_USER= INST_PASS= USER= PASS= HOST= DATABASE= POST_COMMENTS= ''
LIKES_LIMIT= DAYS_TO_UNFOLLOW= CHECK_FOLLOWERS_EVERY= 0
HASHTAGS= []
def init():
global INST_USER, INST_PASS, USER, PASS, HOST, DATABASE, LIKES_LIMIT, DAYS_TO_UNFOLLOW, CHECK_FOLLOWERS_EVERY, HASHTAGS
# read file
data = None
with open('settings.json', 'r') as myfile:
data = myfile.read()
obj = json.loads(data)
INST_USER = obj['instagram']['user']
INST_PASS = obj['instagram']['pass']
USER = obj['db']['user']
HOST = obj['db']['host']
PASS = obj['db']['pass']
DATABASE = obj['db']['database']
LIKES_LIMIT = obj['config']['likes_over']
CHECK_FOLLOWERS_EVERY = obj['config']['check_followers_every']
HASHTAGS = obj['config']['hashtags']
DAYS_TO_UNFOLLOW = obj['config']['days_to_unfollow']
the init() function we created reads the data from settings.json and feeds them into the constants we declared.
Engine
Alright, time for some architecture. Our bot will mainly operate from a python script with an init and update methods. Create BotEngine.py:
import Constants
def init(webdriver):
return
def update(webdriver):
return
We’ll be back later to put the logic here, but for now, we need an entry point.
Entry Point
Create our entry point, InstaBot.py:
from selenium import webdriver
import BotEngine
chromedriver_path = 'YOUR CHROMEDRIVER PATH'
webdriver = webdriver.Chrome(executable_path=chromedriver_path)
BotEngine.init(webdriver)
BotEngine.update(webdriver)
webdriver.close()
chromedriver_path = ‘YOUR CHROMEDRIVER PATH’ webdriver = webdriver.Chrome(executable_path=chromedriver_path)
BotEngine.init(webdriver)
BotEngine.update(webdriver)
webdriver.close()
Of course, you’ll need to swap “YOUR CHROMEDRIVER PATH” with your actual ChromeDriver path.
Time Helper
We need to create a helper script that will help us calculate elapsed days since a certain date (so we know if we should unfollow user)
Create TimeHelper.py:
import datetime
def days_since_date(n):
diff = datetime.datetime.now().date() - n
return diff.days
Database
Create DBHandler.py. It’ll contain a class that handles connecting to the Database for us.
import mysql.connector
import Constants
class DBHandler:
def __init__(self):
DBHandler.HOST = Constants.HOST
DBHandler.USER = Constants.USER
DBHandler.DBNAME = Constants.DATABASE
DBHandler.PASSWORD = Constants.PASS
HOST = Constants.HOST
USER = Constants.USER
DBNAME = Constants.DATABASE
PASSWORD = Constants.PASS
@staticmethod
def get_mydb():
if DBHandler.DBNAME == '':
Constants.init()
db = DBHandler()
mydb = db.connect()
return mydb
def connect(self):
mydb = mysql.connector.connect(
host=DBHandler.HOST,
user=DBHandler.USER,
passwd=DBHandler.PASSWORD,
database = DBHandler.DBNAME
)
return mydb
As you can see, we’re using the constants we defined.
The class contains a static method get_mydb() that returns a database connection we can use.
Now, let’s define a DB user script that contains the DB operations we need to perform on the user.
Create DBUsers.py:
import datetime, TimeHelper
from DBHandler import *
import Constants
#delete user by username
def delete_user(username):
mydb = DBHandler.get_mydb()
cursor = mydb.cursor()
sql = "DELETE FROM followed_users WHERE username = '{0}'".format(username)
cursor.execute(sql)
mydb.commit()
#add new username
def add_user(username):
mydb = DBHandler.get_mydb()
cursor = mydb.cursor()
now = datetime.datetime.now().date()
cursor.execute("INSERT INTO followed_users(username, date_added) VALUES(%s,%s)",(username, now))
mydb.commit()
#check if any user qualifies to be unfollowed
def check_unfollow_list():
mydb = DBHandler.get_mydb()
cursor = mydb.cursor()
cursor.execute("SELECT * FROM followed_users")
results = cursor.fetchall()
users_to_unfollow = []
for r in results:
d = TimeHelper.days_since_date(r[1])
if d > Constants.DAYS_TO_UNFOLLOW:
users_to_unfollow.append(r[0])
return users_to_unfollow
#get all followed users
def get_followed_users():
users = []
mydb = DBHandler.get_mydb()
cursor = mydb.cursor()
cursor.execute("SELECT * FROM followed_users")
results = cursor.fetchall()
for r in results:
users.append(r[0])
return users
Account Agent
Alright, we’re about to start our bot. We’re creating a script called AccountAgent.py that will contain the agent behavior.
Import some modules, some of which we need for later and write a login function that will make use of our webdriver.
Notice that we have to keep calling the sleep function between actions. If we send too many requests quickly, the Instagram servers will be alarmed and will deny any requests you send.
from time import sleep
import datetime
import DBUsers, Constants
import traceback
import random
def login(webdriver):
#Open the instagram login page
webdriver.get('https://www.instagram.com/accounts/login/?source=auth_switcher')
#sleep for 3 seconds to prevent issues with the server
sleep(3)
#Find username and password fields and set their input using our constants
username = webdriver.find_element_by_name('username')
username.send_keys(Constants.INST_USER)
password = webdriver.find_element_by_name('password')
password.send_keys(Constants.INST_PASS)
#Get the login button
try:
button_login = webdriver.find_element_by_xpath(
'//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[4]/button')
except:
button_login = webdriver.find_element_by_xpath(
'//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[6]/button/div')
#sleep again
sleep(2)
#click login
button_login.click()
sleep(3)
#In case you get a popup after logging in, press not now.
#If not, then just return
try:
notnow = webdriver.find_element_by_css_selector(
'body > div.RnEpo.Yx5HN > div > div > div.mt3GC > button.aOOlW.HoLwm')
notnow.click()
except:
return
Also note how we’re getting elements with their xpath. To do so, right click on the element, click “Inspect”, then right click on the element again inside the inspector, and choose Copy->Copy XPath.
Another important thing to be aware of is that element hierarchy change with the page’s layout when you resize or stretch the window. That’s why we’re checking for two different xpaths for the login button.
Now go back to BotEngine.py, we’re ready to login.
Add more imports that we’ll need later and fill in the init function
import AccountAgent, DBUsers
import Constants
import datetime
def init(webdriver):
Constants.init()
AccountAgent.login(webdriver)
def update(webdriver):
return
If you run our entry script now (InstaBot.py) you’ll see the bot logging in.
Perfect, now let’s add a method that will allow us to follow people to AccountAgent.py:
def follow_people(webdriver):
#all the followed user
prev_user_list = DBUsers.get_followed_users()
#a list to store newly followed users
new_followed = []
#counters
followed = 0
likes = 0
#Iterate theough all the hashtags from the constants
for hashtag in Constants.HASHTAGS:
#Visit the hashtag
webdriver.get('https://www.instagram.com/explore/tags/' + hashtag+ '/')
sleep(5)
#Get the first post thumbnail and click on it
first_thumbnail = webdriver.find_element_by_xpath(
'//*[@id="react-root"]/section/main/article/div[1]/div/div/div[1]/div[1]/a/div')
first_thumbnail.click()
sleep(random.randint(1,3))
try:
#iterate over the first 200 posts in the hashtag
for x in range(1,200):
t_start = datetime.datetime.now()
#Get the poster's username
username = webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[1]/h2/a').text
likes_over_limit = False
try:
#get number of likes and compare it to the maximum number of likes to ignore post
likes = int(webdriver.find_element_by_xpath(
'/html/body/div[3]/div[2]/div/article/div[2]/section[2]/div/div/button/span').text)
if likes > Constants.LIKES_LIMIT:
print("likes over {0}".format(Constants.LIKES_LIMIT))
likes_over_limit = True
print("Detected: {0}".format(username))
#If username isn't stored in the database and the likes are in the acceptable range
if username not in prev_user_list and not likes_over_limit:
#Don't press the button if the text doesn't say follow
if webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[2]/button').text == 'Follow':
#Use DBUsers to add the new user to the database
DBUsers.add_user(username)
#Click follow
webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[2]/button').click()
followed += 1
print("Followed: {0}, #{1}".format(username, followed))
new_followed.append(username)
# Liking the picture
button_like = webdriver.find_element_by_xpath(
'/html/body/div[3]/div[2]/div/article/div[2]/section[1]/span[1]/button')
button_like.click()
likes += 1
print("Liked {0}'s post, #{1}".format(username, likes))
sleep(random.randint(5, 18))
# Next picture
webdriver.find_element_by_link_text('Next').click()
sleep(random.randint(20, 30))
except:
traceback.print_exc()
continue
t_end = datetime.datetime.now()
#calculate elapsed time
t_elapsed = t_end - t_start
print("This post took {0} seconds".format(t_elapsed.total_seconds()))
except:
traceback.print_exc()
continue
#add new list to old list
for n in range(0, len(new_followed)):
prev_user_list.append(new_followed[n])
print('Liked {} photos.'.format(likes))
print('Followed {} new people.'.format(followed))
It’s pretty long, but generally here’s the steps of the algorithm:
For every hashtag in the hashtag constant list:
- Visit the hashtag link
- Open the first thumbnail
- Now, execute the following code 200 times (first 200 posts in the hashtag)
- Get poster’s username, check if not already following, follow, like the post, then click next
- If already following just click next quickly
Now we might as well implement the unfollow method, hopefully the engine will be feeding us the usernames to unfollow in a list:
def unfollow_people(webdriver, people):
#if only one user, append in a list
if not isinstance(people, (list,)):
p = people
people = []
people.append(p)
for user in people:
try:
webdriver.get('https://www.instagram.com/' + user + '/')
sleep(5)
unfollow_xpath = '//*[@id="react-root"]/section/main/div/header/section/div[1]/div[1]/span/span[1]/button'
unfollow_confirm_xpath = '/html/body/div[3]/div/div/div[3]/button[1]'
if webdriver.find_element_by_xpath(unfollow_xpath).text == "Following":
sleep(random.randint(4, 15))
webdriver.find_element_by_xpath(unfollow_xpath).click()
sleep(2)
webdriver.find_element_by_xpath(unfollow_confirm_xpath).click()
sleep(4)
DBUsers.delete_user(user)
except Exception:
traceback.print_exc()
continue
Now we can finally go back and finish the bot by implementing the rest of BotEngine.py:
import AccountAgent, DBUsers
import Constants
import datetime
def init(webdriver):
Constants.init()
AccountAgent.login(webdriver)
def update(webdriver):
#Get start of time to calculate elapsed time later
start = datetime.datetime.now()
#Before the loop, check if should unfollow anyone
_check_follow_list(webdriver)
while True:
#Start following operation
AccountAgent.follow_people(webdriver)
#Get the time at the end
end = datetime.datetime.now()
#How much time has passed?
elapsed = end - start
#If greater than our constant to check on
#followers, check on followers
if elapsed.total_seconds() >= Constants.CHECK_FOLLOWERS_EVERY:
#reset the start variable to now
start = datetime.datetime.now()
#check on followers
_check_follow_list(webdriver)
def _check_follow_list(webdriver):
print("Checking for users to unfollow")
#get the unfollow list
users = DBUsers.check_unfollow_list()
#if there's anyone in the list, start unfollowing operation
if len(users) > 0:
AccountAgent.unfollow_people(webdriver, users)
Conclusion
And that’s it — now you have yourself a fully functional Instagram bot built with Python and Selenium. There are many possibilities for you to explore now, so make sure you’re using this newly gained skill to solve real life problems!
You can get the source code for the whole project from this GitHub repository.
Building a simple Instagram bot with Python tutorial
Here we build a simple bot using some simple Python which beginner to intermediate coders can follow.
Here’s the code on GitHub
https://github.com/aj-4/ig-followers
Build A (Full-Featured) Instagram Bot With Python
Source Code: https://github.com/jg-fisher/instagram-bot
How to Get Instagram Followers/Likes Using Python
In this video I show you how to program your own Instagram Bot using Python and Selenium.
https://www.youtube.com/watch?v=BGU2X5lrz9M
Code Link:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
import random
import sys
def print_same_line(text):
sys.stdout.write('\r')
sys.stdout.flush()
sys.stdout.write(text)
sys.stdout.flush()
class InstagramBot:
def __init__(self, username, password):
self.username = username
self.password = password
self.driver = webdriver.Chrome()
def closeBrowser(self):
self.driver.close()
def login(self):
driver = self.driver
driver.get("https://www.instagram.com/")
time.sleep(2)
login_button = driver.find_element_by_xpath("//a[@href='/accounts/login/?source=auth_switcher']")
login_button.click()
time.sleep(2)
user_name_elem = driver.find_element_by_xpath("//input[@name='username']")
user_name_elem.clear()
user_name_elem.send_keys(self.username)
passworword_elem = driver.find_element_by_xpath("//input[@name='password']")
passworword_elem.clear()
passworword_elem.send_keys(self.password)
passworword_elem.send_keys(Keys.RETURN)
time.sleep(2)
def like_photo(self, hashtag):
driver = self.driver
driver.get("https://www.instagram.com/explore/tags/" + hashtag + "/")
time.sleep(2)
# gathering photos
pic_hrefs = []
for i in range(1, 7):
try:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(2)
# get tags
hrefs_in_view = driver.find_elements_by_tag_name('a')
# finding relevant hrefs
hrefs_in_view = [elem.get_attribute('href') for elem in hrefs_in_view
if '.com/p/' in elem.get_attribute('href')]
# building list of unique photos
[pic_hrefs.append(href) for href in hrefs_in_view if href not in pic_hrefs]
# print("Check: pic href length " + str(len(pic_hrefs)))
except Exception:
continue
# Liking photos
unique_photos = len(pic_hrefs)
for pic_href in pic_hrefs:
driver.get(pic_href)
time.sleep(2)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
time.sleep(random.randint(2, 4))
like_button = lambda: driver.find_element_by_xpath('//span[@aria-label="Like"]').click()
like_button().click()
for second in reversed(range(0, random.randint(18, 28))):
print_same_line("#" + hashtag + ': unique photos left: ' + str(unique_photos)
+ " | Sleeping " + str(second))
time.sleep(1)
except Exception as e:
time.sleep(2)
unique_photos -= 1
if __name__ == "__main__":
username = "USERNAME"
password = "PASSWORD"
ig = InstagramBot(username, password)
ig.login()
hashtags = ['amazing', 'beautiful', 'adventure', 'photography', 'nofilter',
'newyork', 'artsy', 'alumni', 'lion', 'best', 'fun', 'happy',
'art', 'funny', 'me', 'followme', 'follow', 'cinematography', 'cinema',
'love', 'instagood', 'instagood', 'followme', 'fashion', 'sun', 'scruffy',
'street', 'canon', 'beauty', 'studio', 'pretty', 'vintage', 'fierce']
while True:
try:
# Choose a random tag from the list of tags
tag = random.choice(hashtags)
ig.like_photo(tag)
except Exception:
ig.closeBrowser()
time.sleep(60)
ig = InstagramBot(username, password)
ig.login()
Build An INSTAGRAM Bot With Python That Gets You Followers
Instagram Automation Using Python
How to Create an Instagram Bot | Get More Followers
Building a simple Instagram Influencer Bot with Python tutorial
#python #chatbot #web-development
1667425440
Pdf2gerb: Perl Script Converts PDF Files to Gerber format
pdf2gerb
Perl script converts PDF files to Gerber format
Pdf2Gerb generates Gerber 274X photoplotting and Excellon drill files from PDFs of a PCB. Up to three PDFs are used: the top copper layer, the bottom copper layer (for 2-sided PCBs), and an optional silk screen layer. The PDFs can be created directly from any PDF drawing software, or a PDF print driver can be used to capture the Print output if the drawing software does not directly support output to PDF.
The general workflow is as follows:
- Design the PCB using your favorite CAD or drawing software.
- Print the top and bottom copper and top silk screen layers to a PDF file.
- Run Pdf2Gerb on the PDFs to create Gerber and Excellon files.
- Use a Gerber viewer to double-check the output against the original PCB design.
- Make adjustments as needed.
- Submit the files to a PCB manufacturer.
Please note that Pdf2Gerb does NOT perform DRC (Design Rule Checks), as these will vary according to individual PCB manufacturer conventions and capabilities. Also note that Pdf2Gerb is not perfect, so the output files must always be checked before submitting them. As of version 1.6, Pdf2Gerb supports most PCB elements, such as round and square pads, round holes, traces, SMD pads, ground planes, no-fill areas, and panelization. However, because it interprets the graphical output of a Print function, there are limitations in what it can recognize (or there may be bugs).
See docs/Pdf2Gerb.pdf for install/setup, config, usage, and other info.
pdf2gerb_cfg.pm
#Pdf2Gerb config settings:
#Put this file in same folder/directory as pdf2gerb.pl itself (global settings),
#or copy to another folder/directory with PDFs if you want PCB-specific settings.
#There is only one user of this file, so we don't need a custom package or namespace.
#NOTE: all constants defined in here will be added to main namespace.
#package pdf2gerb_cfg;
use strict; #trap undef vars (easier debug)
use warnings; #other useful info (easier debug)
##############################################################################################
#configurable settings:
#change values here instead of in main pfg2gerb.pl file
use constant WANT_COLORS => ($^O !~ m/Win/); #ANSI colors no worky on Windows? this must be set < first DebugPrint() call
#just a little warning; set realistic expectations:
#DebugPrint("${\(CYAN)}Pdf2Gerb.pl ${\(VERSION)}, $^O O/S\n${\(YELLOW)}${\(BOLD)}${\(ITALIC)}This is EXPERIMENTAL software. \nGerber files MAY CONTAIN ERRORS. Please CHECK them before fabrication!${\(RESET)}", 0); #if WANT_DEBUG
use constant METRIC => FALSE; #set to TRUE for metric units (only affect final numbers in output files, not internal arithmetic)
use constant APERTURE_LIMIT => 0; #34; #max #apertures to use; generate warnings if too many apertures are used (0 to not check)
use constant DRILL_FMT => '2.4'; #'2.3'; #'2.4' is the default for PCB fab; change to '2.3' for CNC
use constant WANT_DEBUG => 0; #10; #level of debug wanted; higher == more, lower == less, 0 == none
use constant GERBER_DEBUG => 0; #level of debug to include in Gerber file; DON'T USE FOR FABRICATION
use constant WANT_STREAMS => FALSE; #TRUE; #save decompressed streams to files (for debug)
use constant WANT_ALLINPUT => FALSE; #TRUE; #save entire input stream (for debug ONLY)
#DebugPrint(sprintf("${\(CYAN)}DEBUG: stdout %d, gerber %d, want streams? %d, all input? %d, O/S: $^O, Perl: $]${\(RESET)}\n", WANT_DEBUG, GERBER_DEBUG, WANT_STREAMS, WANT_ALLINPUT), 1);
#DebugPrint(sprintf("max int = %d, min int = %d\n", MAXINT, MININT), 1);
#define standard trace and pad sizes to reduce scaling or PDF rendering errors:
#This avoids weird aperture settings and replaces them with more standardized values.
#(I'm not sure how photoplotters handle strange sizes).
#Fewer choices here gives more accurate mapping in the final Gerber files.
#units are in inches
use constant TOOL_SIZES => #add more as desired
(
#round or square pads (> 0) and drills (< 0):
.010, -.001, #tiny pads for SMD; dummy drill size (too small for practical use, but needed so StandardTool will use this entry)
.031, -.014, #used for vias
.041, -.020, #smallest non-filled plated hole
.051, -.025,
.056, -.029, #useful for IC pins
.070, -.033,
.075, -.040, #heavier leads
# .090, -.043, #NOTE: 600 dpi is not high enough resolution to reliably distinguish between .043" and .046", so choose 1 of the 2 here
.100, -.046,
.115, -.052,
.130, -.061,
.140, -.067,
.150, -.079,
.175, -.088,
.190, -.093,
.200, -.100,
.220, -.110,
.160, -.125, #useful for mounting holes
#some additional pad sizes without holes (repeat a previous hole size if you just want the pad size):
.090, -.040, #want a .090 pad option, but use dummy hole size
.065, -.040, #.065 x .065 rect pad
.035, -.040, #.035 x .065 rect pad
#traces:
.001, #too thin for real traces; use only for board outlines
.006, #minimum real trace width; mainly used for text
.008, #mainly used for mid-sized text, not traces
.010, #minimum recommended trace width for low-current signals
.012,
.015, #moderate low-voltage current
.020, #heavier trace for power, ground (even if a lighter one is adequate)
.025,
.030, #heavy-current traces; be careful with these ones!
.040,
.050,
.060,
.080,
.100,
.120,
);
#Areas larger than the values below will be filled with parallel lines:
#This cuts down on the number of aperture sizes used.
#Set to 0 to always use an aperture or drill, regardless of size.
use constant { MAX_APERTURE => max((TOOL_SIZES)) + .004, MAX_DRILL => -min((TOOL_SIZES)) + .004 }; #max aperture and drill sizes (plus a little tolerance)
#DebugPrint(sprintf("using %d standard tool sizes: %s, max aper %.3f, max drill %.3f\n", scalar((TOOL_SIZES)), join(", ", (TOOL_SIZES)), MAX_APERTURE, MAX_DRILL), 1);
#NOTE: Compare the PDF to the original CAD file to check the accuracy of the PDF rendering and parsing!
#for example, the CAD software I used generated the following circles for holes:
#CAD hole size: parsed PDF diameter: error:
# .014 .016 +.002
# .020 .02267 +.00267
# .025 .026 +.001
# .029 .03167 +.00267
# .033 .036 +.003
# .040 .04267 +.00267
#This was usually ~ .002" - .003" too big compared to the hole as displayed in the CAD software.
#To compensate for PDF rendering errors (either during CAD Print function or PDF parsing logic), adjust the values below as needed.
#units are pixels; for example, a value of 2.4 at 600 dpi = .0004 inch, 2 at 600 dpi = .0033"
use constant
{
HOLE_ADJUST => -0.004 * 600, #-2.6, #holes seemed to be slightly oversized (by .002" - .004"), so shrink them a little
RNDPAD_ADJUST => -0.003 * 600, #-2, #-2.4, #round pads seemed to be slightly oversized, so shrink them a little
SQRPAD_ADJUST => +0.001 * 600, #+.5, #square pads are sometimes too small by .00067, so bump them up a little
RECTPAD_ADJUST => 0, #(pixels) rectangular pads seem to be okay? (not tested much)
TRACE_ADJUST => 0, #(pixels) traces seemed to be okay?
REDUCE_TOLERANCE => .001, #(inches) allow this much variation when reducing circles and rects
};
#Also, my CAD's Print function or the PDF print driver I used was a little off for circles, so define some additional adjustment values here:
#Values are added to X/Y coordinates; units are pixels; for example, a value of 1 at 600 dpi would be ~= .002 inch
use constant
{
CIRCLE_ADJUST_MINX => 0,
CIRCLE_ADJUST_MINY => -0.001 * 600, #-1, #circles were a little too high, so nudge them a little lower
CIRCLE_ADJUST_MAXX => +0.001 * 600, #+1, #circles were a little too far to the left, so nudge them a little to the right
CIRCLE_ADJUST_MAXY => 0,
SUBST_CIRCLE_CLIPRECT => FALSE, #generate circle and substitute for clip rects (to compensate for the way some CAD software draws circles)
WANT_CLIPRECT => TRUE, #FALSE, #AI doesn't need clip rect at all? should be on normally?
RECT_COMPLETION => FALSE, #TRUE, #fill in 4th side of rect when 3 sides found
};
#allow .012 clearance around pads for solder mask:
#This value effectively adjusts pad sizes in the TOOL_SIZES list above (only for solder mask layers).
use constant SOLDER_MARGIN => +.012; #units are inches
#line join/cap styles:
use constant
{
CAP_NONE => 0, #butt (none); line is exact length
CAP_ROUND => 1, #round cap/join; line overhangs by a semi-circle at either end
CAP_SQUARE => 2, #square cap/join; line overhangs by a half square on either end
CAP_OVERRIDE => FALSE, #cap style overrides drawing logic
};
#number of elements in each shape type:
use constant
{
RECT_SHAPELEN => 6, #x0, y0, x1, y1, count, "rect" (start, end corners)
LINE_SHAPELEN => 6, #x0, y0, x1, y1, count, "line" (line seg)
CURVE_SHAPELEN => 10, #xstart, ystart, x0, y0, x1, y1, xend, yend, count, "curve" (bezier 2 points)
CIRCLE_SHAPELEN => 5, #x, y, 5, count, "circle" (center + radius)
};
#const my %SHAPELEN =
#Readonly my %SHAPELEN =>
our %SHAPELEN =
(
rect => RECT_SHAPELEN,
line => LINE_SHAPELEN,
curve => CURVE_SHAPELEN,
circle => CIRCLE_SHAPELEN,
);
#panelization:
#This will repeat the entire body the number of times indicated along the X or Y axes (files grow accordingly).
#Display elements that overhang PCB boundary can be squashed or left as-is (typically text or other silk screen markings).
#Set "overhangs" TRUE to allow overhangs, FALSE to truncate them.
#xpad and ypad allow margins to be added around outer edge of panelized PCB.
use constant PANELIZE => {'x' => 1, 'y' => 1, 'xpad' => 0, 'ypad' => 0, 'overhangs' => TRUE}; #number of times to repeat in X and Y directions
# Set this to 1 if you need TurboCAD support.
#$turboCAD = FALSE; #is this still needed as an option?
#CIRCAD pad generation uses an appropriate aperture, then moves it (stroke) "a little" - we use this to find pads and distinguish them from PCB holes.
use constant PAD_STROKE => 0.3; #0.0005 * 600; #units are pixels
#convert very short traces to pads or holes:
use constant TRACE_MINLEN => .001; #units are inches
#use constant ALWAYS_XY => TRUE; #FALSE; #force XY even if X or Y doesn't change; NOTE: needs to be TRUE for all pads to show in FlatCAM and ViewPlot
use constant REMOVE_POLARITY => FALSE; #TRUE; #set to remove subtractive (negative) polarity; NOTE: must be FALSE for ground planes
#PDF uses "points", each point = 1/72 inch
#combined with a PDF scale factor of .12, this gives 600 dpi resolution (1/72 * .12 = 600 dpi)
use constant INCHES_PER_POINT => 1/72; #0.0138888889; #multiply point-size by this to get inches
# The precision used when computing a bezier curve. Higher numbers are more precise but slower (and generate larger files).
#$bezierPrecision = 100;
use constant BEZIER_PRECISION => 36; #100; #use const; reduced for faster rendering (mainly used for silk screen and thermal pads)
# Ground planes and silk screen or larger copper rectangles or circles are filled line-by-line using this resolution.
use constant FILL_WIDTH => .01; #fill at most 0.01 inch at a time
# The max number of characters to read into memory
use constant MAX_BYTES => 10 * M; #bumped up to 10 MB, use const
use constant DUP_DRILL1 => TRUE; #FALSE; #kludge: ViewPlot doesn't load drill files that are too small so duplicate first tool
my $runtime = time(); #Time::HiRes::gettimeofday(); #measure my execution time
print STDERR "Loaded config settings from '${\(__FILE__)}'.\n";
1; #last value must be truthful to indicate successful load
#############################################################################################
#junk/experiment:
#use Package::Constants;
#use Exporter qw(import); #https://perldoc.perl.org/Exporter.html
#my $caller = "pdf2gerb::";
#sub cfg
#{
# my $proto = shift;
# my $class = ref($proto) || $proto;
# my $settings =
# {
# $WANT_DEBUG => 990, #10; #level of debug wanted; higher == more, lower == less, 0 == none
# };
# bless($settings, $class);
# return $settings;
#}
#use constant HELLO => "hi there2"; #"main::HELLO" => "hi there";
#use constant GOODBYE => 14; #"main::GOODBYE" => 12;
#print STDERR "read cfg file\n";
#our @EXPORT_OK = Package::Constants->list(__PACKAGE__); #https://www.perlmonks.org/?node_id=1072691; NOTE: "_OK" skips short/common names
#print STDERR scalar(@EXPORT_OK) . " consts exported:\n";
#foreach(@EXPORT_OK) { print STDERR "$_\n"; }
#my $val = main::thing("xyz");
#print STDERR "caller gave me $val\n";
#foreach my $arg (@ARGV) { print STDERR "arg $arg\n"; }Download Details:
Author: swannman
Source Code: https://github.com/swannman/pdf2gerb
License: GPL-3.0 license
1657081614
How to Automate Excel with Python | Python Excel Tutorial (OpenPyXL)
How to Automate Excel with Python
In this article, We will show how we can use python to automate Excel . A useful Python library is Openpyxl which we will learn to do Excel Automation
What is OPENPYXL
Openpyxl is a Python library that is used to read from an Excel file or write to an Excel file. Data scientists use Openpyxl for data analysis, data copying, data mining, drawing charts, styling sheets, adding formulas, and more.
Workbook: A spreadsheet is represented as a workbook in openpyxl. A workbook consists of one or more sheets.
Sheet: A sheet is a single page composed of cells for organizing data.
Cell: The intersection of a row and a column is called a cell. Usually represented by A1, B5, etc.
Row: A row is a horizontal line represented by a number (1,2, etc.).
Column: A column is a vertical line represented by a capital letter (A, B, etc.).
Openpyxl can be installed using the pip command and it is recommended to install it in a virtual environment.
pip install openpyxl
CREATE A NEW WORKBOOK
We start by creating a new spreadsheet, which is called a workbook in Openpyxl. We import the workbook module from Openpyxl and use the function Workbook() which creates a new workbook.
from openpyxl
import Workbook
#creates a new workbook
wb = Workbook()
#Gets the first active worksheet
ws = wb.active
#creating new worksheets by using the create_sheet method
ws1 = wb.create_sheet("sheet1", 0) #inserts at first position
ws2 = wb.create_sheet("sheet2") #inserts at last position
ws3 = wb.create_sheet("sheet3", -1) #inserts at penultimate position
#Renaming the sheet
ws.title = "Example"
#save the workbook
wb.save(filename = "example.xlsx")READING DATA FROM WORKBOOK
We load the file using the function load_Workbook() which takes the filename as an argument. The file must be saved in the same working directory.
#loading a workbook
wb = openpyxl.load_workbook("example.xlsx")GETTING SHEETS FROM THE LOADED WORKBOOK
#getting sheet names
wb.sheetnames
result = ['sheet1', 'Sheet', 'sheet3', 'sheet2']
#getting a particular sheet
sheet1 = wb["sheet2"]
#getting sheet title
sheet1.title
result = 'sheet2'
#Getting the active sheet
sheetactive = wb.active
result = 'sheet1'ACCESSING CELLS AND CELL VALUES
#get a cell from the sheet
sheet1["A1"] <
Cell 'Sheet1'.A1 >
#get the cell value
ws["A1"].value 'Segment'
#accessing cell using row and column and assigning a value
d = ws.cell(row = 4, column = 2, value = 10)
d.value
10ITERATING THROUGH ROWS AND COLUMNS
#looping through each row and column
for x in range(1, 5):
for y in range(1, 5):
print(x, y, ws.cell(row = x, column = y)
.value)
#getting the highest row number
ws.max_row
701
#getting the highest column number
ws.max_column
19There are two functions for iterating through rows and columns.
Iter_rows() => returns the rows
Iter_cols() => returns the columns {
min_row = 4, max_row = 5, min_col = 2, max_col = 5
} => This can be used to set the boundaries
for any iteration.Example:
#iterating rows
for row in ws.iter_rows(min_row = 2, max_col = 3, max_row = 3):
for cell in row:
print(cell) <
Cell 'Sheet1'.A2 >
<
Cell 'Sheet1'.B2 >
<
Cell 'Sheet1'.C2 >
<
Cell 'Sheet1'.A3 >
<
Cell 'Sheet1'.B3 >
<
Cell 'Sheet1'.C3 >
#iterating columns
for col in ws.iter_cols(min_row = 2, max_col = 3, max_row = 3):
for cell in col:
print(cell) <
Cell 'Sheet1'.A2 >
<
Cell 'Sheet1'.A3 >
<
Cell 'Sheet1'.B2 >
<
Cell 'Sheet1'.B3 >
<
Cell 'Sheet1'.C2 >
<
Cell 'Sheet1'.C3 >To get all the rows of the worksheet we use the method worksheet.rows and to get all the columns of the worksheet we use the method worksheet.columns. Similarly, to iterate only through the values we use the method worksheet.values.
Example:
for row in ws.values:
for value in row:
print(value)WRITING DATA TO AN EXCEL FILE
Writing to a workbook can be done in many ways such as adding a formula, adding charts, images, updating cell values, inserting rows and columns, etc… We will discuss each of these with an example.
CREATING AND SAVING A NEW WORKBOOK
#creates a new workbook
wb = openpyxl.Workbook()
#saving the workbook
wb.save("new.xlsx")ADDING AND REMOVING SHEETS
#creating a new sheet
ws1 = wb.create_sheet(title = "sheet 2")
#creating a new sheet at index 0
ws2 = wb.create_sheet(index = 0, title = "sheet 0")
#checking the sheet names
wb.sheetnames['sheet 0', 'Sheet', 'sheet 2']
#deleting a sheet
del wb['sheet 0']
#checking sheetnames
wb.sheetnames['Sheet', 'sheet 2']ADDING CELL VALUES
#checking the sheet value
ws['B2'].value
null
#adding value to cell
ws['B2'] = 367
#checking value
ws['B2'].value
367ADDING FORMULAS
We often require formulas to be included in our Excel datasheet. We can easily add formulas using the Openpyxl module just like you add values to a cell.
For example:
import openpyxl
from openpyxl
import Workbook
wb = openpyxl.load_workbook("new1.xlsx")
ws = wb['Sheet']
ws['A9'] = '=SUM(A2:A8)'
wb.save("new2.xlsx")The above program will add the formula (=SUM(A2:A8)) in cell A9. The result will be as below.
MERGE/UNMERGE CELLS
Two or more cells can be merged to a rectangular area using the method merge_cells(), and similarly, they can be unmerged using the method unmerge_cells().
For example:
Merge cells
#merge cells B2 to C9
ws.merge_cells('B2:C9')
ws['B2'] = "Merged cells"Adding the above code to the previous example will merge cells as below.
UNMERGE CELLS
#unmerge cells B2 to C9
ws.unmerge_cells('B2:C9')The above code will unmerge cells from B2 to C9.
INSERTING AN IMAGE
To insert an image we import the image function from the module openpyxl.drawing.image. We then load our image and add it to the cell as shown in the below example.
Example:
import openpyxl
from openpyxl
import Workbook
from openpyxl.drawing.image
import Image
wb = openpyxl.load_workbook("new1.xlsx")
ws = wb['Sheet']
#loading the image(should be in same folder)
img = Image('logo.png')
ws['A1'] = "Adding image"
#adjusting size
img.height = 130
img.width = 200
#adding img to cell A3
ws.add_image(img, 'A3')
wb.save("new2.xlsx")Result:
CREATING CHARTS
Charts are essential to show a visualization of data. We can create charts from Excel data using the Openpyxl module chart. Different forms of charts such as line charts, bar charts, 3D line charts, etc., can be created. We need to create a reference that contains the data to be used for the chart, which is nothing but a selection of cells (rows and columns). I am using sample data to create a 3D bar chart in the below example:
Example
import openpyxl
from openpyxl
import Workbook
from openpyxl.chart
import BarChart3D, Reference, series
wb = openpyxl.load_workbook("example.xlsx")
ws = wb.active
values = Reference(ws, min_col = 3, min_row = 2, max_col = 3, max_row = 40)
chart = BarChart3D()
chart.add_data(values)
ws.add_chart(chart, "E3")
wb.save("MyChart.xlsx")Result
How to Automate Excel with Python with Video Tutorial
Welcome to another video! In this video, We will cover how we can use python to automate Excel. I'll be going over everything from creating workbooks to accessing individual cells and stylizing cells. There is a ton of things that you can do with Excel but I'll just be covering the core/base things in OpenPyXl.
⭐️ Timestamps ⭐️
00:00 | Introduction
02:14 | Installing openpyxl
03:19 | Testing Installation
04:25 | Loading an Existing Workbook
06:46 | Accessing Worksheets
07:37 | Accessing Cell Values
08:58 | Saving Workbooks
09:52 | Creating, Listing and Changing Sheets
11:50 | Creating a New Workbook
12:39 | Adding/Appending Rows
14:26 | Accessing Multiple Cells
20:46 | Merging Cells
22:27 | Inserting and Deleting Rows
23:35 | Inserting and Deleting Columns
24:48 | Copying and Moving Cells
26:06 | Practical Example, Formulas & Cell Styling
📄 Resources 📄
OpenPyXL Docs: https://openpyxl.readthedocs.io/en/stable/
Code Written in This Tutorial: https://github.com/techwithtim/ExcelPythonTutorial
Subscribe: https://www.youtube.com/c/TechWithTim/featured
1597489568
Dynamically Add/Remove Multiple input Fields and Submit to DB with jQuery and Laravel
In this post, i will show you how to dynamically add/remove multiple input fields and submit to database with jquery in php laravel framework. As well as, i will show you how to add/remove multiple input fields and submit to database with validation in laravel.
dynamically add remove multiple input fields and submit to database with jquery and laravel app will looks like, you can see in the following picture:
Laravel - Add/Remove Multiple Input Fields Using jQuery, javaScript
Follow the below given easy step to create dynamically add or remove multiple input fields and submit to database with jquery in php laravel
- Step 1: Install Laravel App
- Step 2: Add Database Details
- Step 3: Create Migration & Model
- Step 4: Add Routes
- Step 5: Create Controller by Artisan
- Step 6: Create Blade View
- Step 7: Run Development Server
https://www.tutsmake.com/add-remove-multiple-input-fields-in-laravel-using-jquery/
#laravel - dynamically add or remove input fields using jquery #dynamically add / remove multiple input fields in laravel 7 using jquery ajax #add/remove multiple input fields dynamically with jquery laravel #dynamically add multiple input fields and submit to database with jquery and laravel #add remove input fields dynamically with jquery and submit to database #sql
1656924529
Macros in Rust - Everything You Need To Know
Macros in Rust - Everything You Need To Know
Ever wondered what the bang ("!") after "println" means? Not anymore! I will show you exactly how macros work, how to use them, and how to write your own macros.
This is the perfect talk for you if you are using macros, but you always wanted to know how they work and how to implement them yourself.
macro_rules!
Rust provides a powerful macro system that allows metaprogramming. As you've seen in previous chapters, macros look like functions, except that their name ends with a bang !, but instead of generating a function call, macros are expanded into source code that gets compiled with the rest of the program. However, unlike macros in C and other languages, Rust macros are expanded into abstract syntax trees, rather than string preprocessing, so you don't get unexpected precedence bugs.
Macros are created using the macro_rules! macro.
// This is a simple macro named `say_hello`.
macro_rules! say_hello {
// `()` indicates that the macro takes no argument.
() => {
// The macro will expand into the contents of this block.
println!("Hello!");
};
}
fn main() {
// This call will expand into `println!("Hello");`
say_hello!()
}So why are macros useful?
Don't repeat yourself. There are many cases where you may need similar functionality in multiple places but with different types. Often, writing a macro is a useful way to avoid repeating code. (More on this later)
Domain-specific languages. Macros allow you to define special syntax for a specific purpose. (More on this later)
Variadic interfaces. Sometimes you want to define an interface that takes a variable number of arguments. An example is println! which could take any number of arguments, depending on the format string!. (More on this later)
Macros
We’ve used macros like println! throughout this book, but we haven’t fully explored what a macro is and how it works. The term macro refers to a family of features in Rust: declarative macros with macro_rules! and three kinds of procedural macros:
- Custom
#[derive]macros that specify code added with thederiveattribute used on structs and enums - Attribute-like macros that define custom attributes usable on any item
- Function-like macros that look like function calls but operate on the tokens specified as their argument
We’ll talk about each of these in turn, but first, let’s look at why we even need macros when we already have functions.
The Difference Between Macros and Functions
Fundamentally, macros are a way of writing code that writes other code, which is known as metaprogramming. In Appendix C, we discuss the derive attribute, which generates an implementation of various traits for you. We’ve also used the println! and vec! macros throughout the book. All of these macros expand to produce more code than the code you’ve written manually.
Metaprogramming is useful for reducing the amount of code you have to write and maintain, which is also one of the roles of functions. However, macros have some additional powers that functions don’t.
A function signature must declare the number and type of parameters the function has. Macros, on the other hand, can take a variable number of parameters: we can call println!("hello") with one argument or println!("hello {}", name) with two arguments. Also, macros are expanded before the compiler interprets the meaning of the code, so a macro can, for example, implement a trait on a given type. A function can’t, because it gets called at runtime and a trait needs to be implemented at compile time.
The downside to implementing a macro instead of a function is that macro definitions are more complex than function definitions because you’re writing Rust code that writes Rust code. Due to this indirection, macro definitions are generally more difficult to read, understand, and maintain than function definitions.
Another important difference between macros and functions is that you must define macros or bring them into scope before you call them in a file, as opposed to functions you can define anywhere and call anywhere.
Declarative Macros with macro_rules! for General Metaprogramming
The most widely used form of macros in Rust is declarative macros. These are also sometimes referred to as “macros by example,” “macro_rules! macros,” or just plain “macros.” At their core, declarative macros allow you to write something similar to a Rust match expression. As discussed in Chapter 6, match expressions are control structures that take an expression, compare the resulting value of the expression to patterns, and then run the code associated with the matching pattern. Macros also compare a value to patterns that are associated with particular code: in this situation, the value is the literal Rust source code passed to the macro; the patterns are compared with the structure of that source code; and the code associated with each pattern, when matched, replaces the code passed to the macro. This all happens during compilation.
To define a macro, you use the macro_rules! construct. Let’s explore how to use macro_rules! by looking at how the vec! macro is defined. Chapter 8 covered how we can use the vec! macro to create a new vector with particular values. For example, the following macro creates a new vector containing three integers:
let v: Vec<u32> = vec![1, 2, 3];We could also use the vec! macro to make a vector of two integers or a vector of five string slices. We wouldn’t be able to use a function to do the same because we wouldn’t know the number or type of values up front.
Listing 19-28 shows a slightly simplified definition of the vec! macro.
Filename: src/lib.rs
#[macro_export]
macro_rules! vec {
( $( $x:expr ),* ) => {
{
let mut temp_vec = Vec::new();
$(
temp_vec.push($x);
)*
temp_vec
}
};
}Listing 19-28: A simplified version of the vec! macro definition
Note: The actual definition of the
vec!macro in the standard library includes code to preallocate the correct amount of memory up front. That code is an optimization that we don’t include here to make the example simpler.
The #[macro_export] annotation indicates that this macro should be made available whenever the crate in which the macro is defined is brought into scope. Without this annotation, the macro can’t be brought into scope.
We then start the macro definition with macro_rules! and the name of the macro we’re defining without the exclamation mark. The name, in this case vec, is followed by curly brackets denoting the body of the macro definition.
The structure in the vec! body is similar to the structure of a match expression. Here we have one arm with the pattern ( $( $x:expr ),* ), followed by => and the block of code associated with this pattern. If the pattern matches, the associated block of code will be emitted. Given that this is the only pattern in this macro, there is only one valid way to match; any other pattern will result in an error. More complex macros will have more than one arm.
Valid pattern syntax in macro definitions is different than the pattern syntax covered in Chapter 18 because macro patterns are matched against Rust code structure rather than values. Let’s walk through what the pattern pieces in Listing 19-28 mean; for the full macro pattern syntax, see the reference.
First, a set of parentheses encompasses the whole pattern. A dollar sign ($) is next, followed by a set of parentheses that captures values that match the pattern within the parentheses for use in the replacement code. Within $() is $x:expr, which matches any Rust expression and gives the expression the name $x.
The comma following $() indicates that a literal comma separator character could optionally appear after the code that matches the code in $(). The * specifies that the pattern matches zero or more of whatever precedes the *.
When we call this macro with vec![1, 2, 3];, the $x pattern matches three times with the three expressions 1, 2, and 3.
Now let’s look at the pattern in the body of the code associated with this arm: temp_vec.push() within $()* is generated for each part that matches $() in the pattern zero or more times depending on how many times the pattern matches. The $x is replaced with each expression matched. When we call this macro with vec![1, 2, 3];, the code generated that replaces this macro call will be the following:
{
let mut temp_vec = Vec::new();
temp_vec.push(1);
temp_vec.push(2);
temp_vec.push(3);
temp_vec
}We’ve defined a macro that can take any number of arguments of any type and can generate code to create a vector containing the specified elements.
There are some strange edge cases with macro_rules!. In the future, Rust will have a second kind of declarative macro that will work in a similar fashion but fix some of these edge cases. After that update, macro_rules! will be effectively deprecated. With this in mind, as well as the fact that most Rust programmers will use macros more than write macros, we won’t discuss macro_rules! any further. To learn more about how to write macros, consult the online documentation or other resources, such as “The Little Book of Rust Macros” started by Daniel Keep and continued by Lukas Wirth.
Procedural Macros for Generating Code from Attributes
The second form of macros is procedural macros, which act more like functions (and are a type of procedure). Procedural macros accept some code as an input, operate on that code, and produce some code as an output rather than matching against patterns and replacing the code with other code as declarative macros do.
The three kinds of procedural macros (custom derive, attribute-like, and function-like) all work in a similar fashion.
When creating procedural macros, the definitions must reside in their own crate with a special crate type. This is for complex technical reasons that we hope to eliminate in the future. Defining procedural macros looks like the code in Listing 19-29, where some_attribute is a placeholder for using a specific macro variety.
Filename: src/lib.rs
use proc_macro;
#[some_attribute]
pub fn some_name(input: TokenStream) -> TokenStream {
}Listing 19-29: An example of defining a procedural macro
The function that defines a procedural macro takes a TokenStream as an input and produces a TokenStream as an output. The TokenStream type is defined by the proc_macro crate that is included with Rust and represents a sequence of tokens. This is the core of the macro: the source code that the macro is operating on makes up the input TokenStream, and the code the macro produces is the output TokenStream. The function also has an attribute attached to it that specifies which kind of procedural macro we’re creating. We can have multiple kinds of procedural macros in the same crate.
Let’s look at the different kinds of procedural macros. We’ll start with a custom derive macro and then explain the small dissimilarities that make the other forms different.
How to Write a Custom derive Macro
Let’s create a crate named hello_macro that defines a trait named HelloMacro with one associated function named hello_macro. Rather than making our crate users implement the HelloMacro trait for each of their types, we’ll provide a procedural macro so users can annotate their type with #[derive(HelloMacro)] to get a default implementation of the hello_macro function. The default implementation will print Hello, Macro! My name is TypeName! where TypeName is the name of the type on which this trait has been defined. In other words, we’ll write a crate that enables another programmer to write code like Listing 19-30 using our crate.
Filename: src/main.rs
use hello_macro::HelloMacro;
use hello_macro_derive::HelloMacro;
#[derive(HelloMacro)]
struct Pancakes;
fn main() {
Pancakes::hello_macro();
}Listing 19-30: The code a user of our crate will be able to write when using our procedural macro
This code will print Hello, Macro! My name is Pancakes! when we’re done. The first step is to make a new library crate, like this:
$ cargo new hello_macro --libNext, we’ll define the HelloMacro trait and its associated function:
Filename: src/lib.rs
pub trait HelloMacro {
fn hello_macro();
}We have a trait and its function. At this point, our crate user could implement the trait to achieve the desired functionality, like so:
use hello_macro::HelloMacro;
struct Pancakes;
impl HelloMacro for Pancakes {
fn hello_macro() {
println!("Hello, Macro! My name is Pancakes!");
}
}
fn main() {
Pancakes::hello_macro();
}However, they would need to write the implementation block for each type they wanted to use with hello_macro; we want to spare them from having to do this work.
Additionally, we can’t yet provide the hello_macro function with default implementation that will print the name of the type the trait is implemented on: Rust doesn’t have reflection capabilities, so it can’t look up the type’s name at runtime. We need a macro to generate code at compile time.
The next step is to define the procedural macro. At the time of this writing, procedural macros need to be in their own crate. Eventually, this restriction might be lifted. The convention for structuring crates and macro crates is as follows: for a crate named foo, a custom derive procedural macro crate is called foo_derive. Let’s start a new crate called hello_macro_derive inside our hello_macro project:
$ cargo new hello_macro_derive --libOur two crates are tightly related, so we create the procedural macro crate within the directory of our hello_macro crate. If we change the trait definition in hello_macro, we’ll have to change the implementation of the procedural macro in hello_macro_derive as well. The two crates will need to be published separately, and programmers using these crates will need to add both as dependencies and bring them both into scope. We could instead have the hello_macro crate use hello_macro_derive as a dependency and re-export the procedural macro code. However, the way we’ve structured the project makes it possible for programmers to use hello_macro even if they don’t want the derive functionality.
We need to declare the hello_macro_derive crate as a procedural macro crate. We’ll also need functionality from the syn and quote crates, as you’ll see in a moment, so we need to add them as dependencies. Add the following to the Cargo.toml file for hello_macro_derive:
Filename: hello_macro_derive/Cargo.toml
[lib]
proc-macro = true
[dependencies]
syn = "1.0"
quote = "1.0"To start defining the procedural macro, place the code in Listing 19-31 into your src/lib.rs file for the hello_macro_derive crate. Note that this code won’t compile until we add a definition for the impl_hello_macro function.
Filename: hello_macro_derive/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn;
#[proc_macro_derive(HelloMacro)]
pub fn hello_macro_derive(input: TokenStream) -> TokenStream {
// Construct a representation of Rust code as a syntax tree
// that we can manipulate
let ast = syn::parse(input).unwrap();
// Build the trait implementation
impl_hello_macro(&ast)
}Listing 19-31: Code that most procedural macro crates will require in order to process Rust code
Notice that we’ve split the code into the hello_macro_derive function, which is responsible for parsing the TokenStream, and the impl_hello_macro function, which is responsible for transforming the syntax tree: this makes writing a procedural macro more convenient. The code in the outer function (hello_macro_derive in this case) will be the same for almost every procedural macro crate you see or create. The code you specify in the body of the inner function (impl_hello_macro in this case) will be different depending on your procedural macro’s purpose.
We’ve introduced three new crates: proc_macro, syn, and quote. The proc_macro crate comes with Rust, so we didn’t need to add that to the dependencies in Cargo.toml. The proc_macro crate is the compiler’s API that allows us to read and manipulate Rust code from our code.
The syn crate parses Rust code from a string into a data structure that we can perform operations on. The quote crate turns syn data structures back into Rust code. These crates make it much simpler to parse any sort of Rust code we might want to handle: writing a full parser for Rust code is no simple task.
The hello_macro_derive function will be called when a user of our library specifies #[derive(HelloMacro)] on a type. This is possible because we’ve annotated the hello_macro_derive function here with proc_macro_derive and specified the name, HelloMacro, which matches our trait name; this is the convention most procedural macros follow.
The hello_macro_derive function first converts the input from a TokenStream to a data structure that we can then interpret and perform operations on. This is where syn comes into play. The parse function in syn takes a TokenStream and returns a DeriveInput struct representing the parsed Rust code. Listing 19-32 shows the relevant parts of the DeriveInput struct we get from parsing the struct Pancakes; string:
DeriveInput {
// --snip--
ident: Ident {
ident: "Pancakes",
span: #0 bytes(95..103)
},
data: Struct(
DataStruct {
struct_token: Struct,
fields: Unit,
semi_token: Some(
Semi
)
}
)
}Listing 19-32: The DeriveInput instance we get when parsing the code that has the macro’s attribute in Listing 19-30
The fields of this struct show that the Rust code we’ve parsed is a unit struct with the ident (identifier, meaning the name) of Pancakes. There are more fields on this struct for describing all sorts of Rust code; check the syn documentation for DeriveInput for more information.
Soon we’ll define the impl_hello_macro function, which is where we’ll build the new Rust code we want to include. But before we do, note that the output for our derive macro is also a TokenStream. The returned TokenStream is added to the code that our crate users write, so when they compile their crate, they’ll get the extra functionality that we provide in the modified TokenStream.
You might have noticed that we’re calling unwrap to cause the hello_macro_derive function to panic if the call to the syn::parse function fails here. It’s necessary for our procedural macro to panic on errors because proc_macro_derive functions must return TokenStream rather than Result to conform to the procedural macro API. We’ve simplified this example by using unwrap; in production code, you should provide more specific error messages about what went wrong by using panic! or expect.
Now that we have the code to turn the annotated Rust code from a TokenStream into a DeriveInput instance, let’s generate the code that implements the HelloMacro trait on the annotated type, as shown in Listing 19-33.
Filename: hello_macro_derive/src/lib.rs
fn impl_hello_macro(ast: &syn::DeriveInput) -> TokenStream {
let name = &ast.ident;
let gen = quote! {
impl HelloMacro for #name {
fn hello_macro() {
println!("Hello, Macro! My name is {}!", stringify!(#name));
}
}
};
gen.into()
}Listing 19-33: Implementing the HelloMacro trait using the parsed Rust code
We get an Ident struct instance containing the name (identifier) of the annotated type using ast.ident. The struct in Listing 19-32 shows that when we run the impl_hello_macro function on the code in Listing 19-30, the ident we get will have the ident field with a value of "Pancakes". Thus, the name variable in Listing 19-33 will contain an Ident struct instance that, when printed, will be the string "Pancakes", the name of the struct in Listing 19-30.
The quote! macro lets us define the Rust code that we want to return. The compiler expects something different to the direct result of the quote! macro’s execution, so we need to convert it to a TokenStream. We do this by calling the into method, which consumes this intermediate representation and returns a value of the required TokenStream type.
The quote! macro also provides some very cool templating mechanics: we can enter #name, and quote! will replace it with the value in the variable name. You can even do some repetition similar to the way regular macros work. Check out the quote crate’s docs for a thorough introduction.
We want our procedural macro to generate an implementation of our HelloMacro trait for the type the user annotated, which we can get by using #name. The trait implementation has one function, hello_macro, whose body contains the functionality we want to provide: printing Hello, Macro! My name is and then the name of the annotated type.
The stringify! macro used here is built into Rust. It takes a Rust expression, such as 1 + 2, and at compile time turns the expression into a string literal, such as "1 + 2". This is different than format! or println!, macros which evaluate the expression and then turn the result into a String. There is a possibility that the #name input might be an expression to print literally, so we use stringify!. Using stringify! also saves an allocation by converting #name to a string literal at compile time.
At this point, cargo build should complete successfully in both hello_macro and hello_macro_derive. Let’s hook up these crates to the code in Listing 19-30 to see the procedural macro in action! Create a new binary project in your projects directory using cargo new pancakes. We need to add hello_macro and hello_macro_derive as dependencies in the pancakes crate’s Cargo.toml. If you’re publishing your versions of hello_macro and hello_macro_derive to crates.io, they would be regular dependencies; if not, you can specify them as path dependencies as follows:
hello_macro = { path = "../hello_macro" }
hello_macro_derive = { path = "../hello_macro/hello_macro_derive" }Put the code in Listing 19-30 into src/main.rs, and run cargo run: it should print Hello, Macro! My name is Pancakes! The implementation of the HelloMacro trait from the procedural macro was included without the pancakes crate needing to implement it; the #[derive(HelloMacro)] added the trait implementation.
Next, let’s explore how the other kinds of procedural macros differ from custom derive macros.
Attribute-like macros
Attribute-like macros are similar to custom derive macros, but instead of generating code for the derive attribute, they allow you to create new attributes. They’re also more flexible: derive only works for structs and enums; attributes can be applied to other items as well, such as functions. Here’s an example of using an attribute-like macro: say you have an attribute named route that annotates functions when using a web application framework:
#[route(GET, "/")]
fn index() {This #[route] attribute would be defined by the framework as a procedural macro. The signature of the macro definition function would look like this:
#[proc_macro_attribute]
pub fn route(attr: TokenStream, item: TokenStream) -> TokenStream {Here, we have two parameters of type TokenStream. The first is for the contents of the attribute: the GET, "/" part. The second is the body of the item the attribute is attached to: in this case, fn index() {} and the rest of the function’s body.
Other than that, attribute-like macros work the same way as custom derive macros: you create a crate with the proc-macro crate type and implement a function that generates the code you want!
Function-like macros
Function-like macros define macros that look like function calls. Similarly to macro_rules! macros, they’re more flexible than functions; for example, they can take an unknown number of arguments. However, macro_rules! macros can be defined only using the match-like syntax we discussed in the section “Declarative Macros with macro_rules! for General Metaprogramming” earlier. Function-like macros take a TokenStream parameter and their definition manipulates that TokenStream using Rust code as the other two types of procedural macros do. An example of a function-like macro is an sql! macro that might be called like so:
let sql = sql!(SELECT * FROM posts WHERE id=1);This macro would parse the SQL statement inside it and check that it’s syntactically correct, which is much more complex processing than a macro_rules! macro can do. The sql! macro would be defined like this:
#[proc_macro]
pub fn sql(input: TokenStream) -> TokenStream {This definition is similar to the custom derive macro’s signature: we receive the tokens that are inside the parentheses and return the code we wanted to generate.
Summary
Whew! Now you have some Rust features in your toolbox that you won’t use often, but you’ll know they’re available in very particular circumstances. We’ve introduced several complex topics so that when you encounter them in error message suggestions or in other peoples’ code, you’ll be able to recognize these concepts and syntax. Use this chapter as a reference to guide you to solutions.
Macros in Rust: A tutorial with examples
In this tutorial, we’ll cover everything you need to know about Rust macros, including an introduction to macros in Rust and a demonstration of how to use Rust macros with examples.
What are Rust macros?
Rust has excellent support for macros. Macros enable you to write code that writes other code, which is known as metaprogramming.
Macros provide functionality similar to functions but without the runtime cost. There is some compile-time cost, however, since macros are expanded during compile time.
Rust macros are very different from macros in C. Rust macros are applied to the token tree whereas C macros are text substitution.
Types of macros in Rust
Rust has two types of macros:
- Declarative macros enable you to write something similar to a match expression that operates on the Rust code you provide as arguments. It uses the code you provide to generate code that replaces the macro invocation
- Procedural macros allow you to operate on the abstract syntax tree (AST) of the Rust code it is given. A proc macro is a function from a
TokenStream(or two) to anotherTokenStream, where the output replaces the macro invocation
Let’s zoom in on both declarative and procedural macros and explore some examples of how to use macros in Rust.
Declarative macros in Rust
These macros are declared using macro_rules!. Declarative macros are a bit less powerful but provide an easy to use interface for creating macros to remove duplicate code. One of the common declarative macro is println!. Declarative macros provide a match like an interface where on match the macro is replaced with code inside the matched arm.
Creating declarative macros
// use macro_rules! <name of macro>{<Body>}
macro_rules! add{
// macth like arm for macro
($a:expr,$b:expr)=>{
// macro expand to this code
{
// $a and $b will be templated using the value/variable provided to macro
$a+$b
}
}
}
fn main(){
// call to macro, $a=1 and $b=2
add!(1,2);
}This code creates a macro to add two numbers. [macro_rules!] are used with the name of the macro, add, and the body of the macro.
The macro doesn’t add two numbers, it just replaces itself with the code to add two numbers. Each arm of the macro takes an argument for functions and multiple types can be assigned to arguments. If the add function can also take a single argument, we add another arm.
macro_rules! add{
// first arm match add!(1,2), add!(2,3) etc
($a:expr,$b:expr)=>{
{
$a+$b
}
};
// Second arm macth add!(1), add!(2) etc
($a:expr)=>{
{
$a
}
}
}
fn main(){
// call the macro
let x=0;
add!(1,2);
add!(x);
}There can be multiple branches in a single macro expanding to different code based on different arguments. Each branch can take multiple arguments, starting with the $ sign and followed by a token type:
item— an item, like a function, struct, module, etc.block— a block (i.e. a block of statements and/or an expression, surrounded by braces)stmt— a statementpat— a patternexpr— an expressionty— a typeident— an identifierpath— a path (e.g.,foo,::std::mem::replace,transmute::<_, int>, …)meta— a meta item; the things that go inside#[...]and#![...]attributestt— a single token treevis— a possibly emptyVisibilityqualifier
In the example, we use the $typ argument with token type ty as a datatype like u8, u16, etc. This macro converts to a particular type before adding the numbers.
macro_rules! add_as{
// using a ty token type for macthing datatypes passed to maccro
($a:expr,$b:expr,$typ:ty)=>{
$a as $typ + $b as $typ
}
}
fn main(){
println!("{}",add_as!(0,2,u8));
}Rust macros also support taking a nonfixed number of arguments. The operators are very similar to the regular expression. * is used for zero or more token types and + for zero or one argument.
macro_rules! add_as{
(
// repeated block
$($a:expr)
// seperator
,
// zero or more
*
)=>{
{
// to handle the case without any arguments
0
// block to be repeated
$(+$a)*
}
}
}
fn main(){
println!("{}",add_as!(1,2,3,4)); // => println!("{}",{0+1+2+3+4})
}The token type that repeats is enclosed in $(), followed by a separator and a * or a +, indicating the number of times the token will repeat. The separator is used to distinguish the tokens from each other. The $() block followed by * or + is used to indicate the repeating block of code. In the above example, +$a is a repeating code.
If you look closely, you’ll notice an additional zero is added to the code to make the syntax valid. To remove this zero and make the add expression the same as the argument, we need to create a new macro known as TT muncher.
macro_rules! add{
// first arm in case of single argument and last remaining variable/number
($a:expr)=>{
$a
};
// second arm in case of two arument are passed and stop recursion in case of odd number ofarguments
($a:expr,$b:expr)=>{
{
$a+$b
}
};
// add the number and the result of remaining arguments
($a:expr,$($b:tt)*)=>{
{
$a+add!($($b)*)
}
}
}
fn main(){
println!("{}",add!(1,2,3,4));
}The TT muncher processes each token separately in a recursive fashion. It’s easier to process a single token at a time. The macro has three arms:
- The first arms handle the case if a single argument is passed
- The second one handles the case if two arguments are passed
- The third arm calls the
addmacro again with the rest of the arguments
The macro arguments don’t need to be comma-separated. Multiple tokens can be used with different token types. For example, brackets can be used with the ident token type. The Rust compiler takes the matched arm and extracts the variable from the argument string.
macro_rules! ok_or_return{
// match something(q,r,t,6,7,8) etc
// compiler extracts function name and arguments. It injects the values in respective varibles.
($a:ident($($b:tt)*))=>{
{
match $a($($b)*) {
Ok(value)=>value,
Err(err)=>{
return Err(err);
}
}
}
};
}
fn some_work(i:i64,j:i64)->Result<(i64,i64),String>{
if i+j>2 {
Ok((i,j))
} else {
Err("error".to_owned())
}
}
fn main()->Result<(),String>{
ok_or_return!(some_work(1,4));
ok_or_return!(some_work(1,0));
Ok(())
}The ok_or_return macro returns the function if an operation returns Err or the value of an operation returns Ok. It takes a function as an argument and executes it inside a match statement. For arguments passed to function, it uses repetition.
Often, few macros need to be grouped into a single macro. In these cases, internal macro rules are used. It helps to manipulate the macro inputs and write clean TT munchers.
To create an internal rule, add the rule name starting with @ as the argument. Now the macro will never match for an internal rule until explicitly specified as an argument.
macro_rules! ok_or_return{
// internal rule.
(@error $a:ident,$($b:tt)* )=>{
{
match $a($($b)*) {
Ok(value)=>value,
Err(err)=>{
return Err(err);
}
}
}
};
// public rule can be called by the user.
($a:ident($($b:tt)*))=>{
ok_or_return!(@error $a,$($b)*)
};
}
fn some_work(i:i64,j:i64)->Result<(i64,i64),String>{
if i+j>2 {
Ok((i,j))
} else {
Err("error".to_owned())
}
}
fn main()->Result<(),String>{
// instead of round bracket curly brackets can also be used
ok_or_return!{some_work(1,4)};
ok_or_return!(some_work(1,0));
Ok(())
}Advanced parsing in Rust with declarative macros
Macros sometimes perform tasks that require parsing of the Rust language itself.
Do put together all the concepts we’ve covered to this point, let’s create a macro that makes a struct public by suffixing the pub keyword.
First, we need to parse the Rust struct to get the name of the struct, fields of the struct, and field type.
Parsing the name and field of a struct
A struct declaration has a visibility keyword at the start (such as pub), followed by the struct keyword and then the name of the struct and the body of the struct.
macro_rules! make_public{
(
// use vis type for visibility keyword and ident for struct name
$vis:vis struct $struct_name:ident { }
) => {
{
pub struct $struct_name{ }
}
}
}The $vis will have visibility and $struct_name will have a struct name. To make a struct public, we just need to add the pub keyword and ignore the $vis variable.
A struct may contain multiple fields with the same or different data types and visibility. The ty token type is used for the data type, vis for visibility, and ident for the field name. We’ll use * repetition for zero or more fields.
macro_rules! make_public{
(
$vis:vis struct $struct_name:ident {
$(
// vis for field visibility, ident for field name and ty for field data type
$field_vis:vis $field_name:ident : $field_type:ty
),*
}
) => {
{
pub struct $struct_name{
$(
pub $field_name : $field_type,
)*
}
}
}
}Parsing metadata from the struct
Often the struct has some metadata attached or procedural macros, such as #[derive(Debug)]. This metadata needs to stay intact. Parsing this metadata is done using the meta type.
macro_rules! make_public{
(
// meta data about struct
$(#[$meta:meta])*
$vis:vis struct $struct_name:ident {
$(
// meta data about field
$(#[$field_meta:meta])*
$field_vis:vis $field_name:ident : $field_type:ty
),*$(,)+
}
) => {
{
$(#[$meta])*
pub struct $struct_name{
$(
$(#[$field_meta:meta])*
pub $field_name : $field_type,
)*
}
}
}
}Our make_public macro is ready now. To see how make_public works, let’s use Rust Playground to expand the macro to the actual code that is compiled.
macro_rules! make_public{
(
$(#[$meta:meta])*
$vis:vis struct $struct_name:ident {
$(
$(#[$field_meta:meta])*
$field_vis:vis $field_name:ident : $field_type:ty
),*$(,)+
}
) => {
$(#[$meta])*
pub struct $struct_name{
$(
$(#[$field_meta:meta])*
pub $field_name : $field_type,
)*
}
}
}
fn main(){
make_public!{
#[derive(Debug)]
struct Name{
n:i64,
t:i64,
g:i64,
}
}
}The expanded code looks like this:
// some imports
macro_rules! make_public {
($ (#[$ meta : meta]) * $ vis : vis struct $ struct_name : ident
{
$
($ (#[$ field_meta : meta]) * $ field_vis : vis $ field_name : ident
: $ field_type : ty), * $ (,) +
}) =>
{
$ (#[$ meta]) * pub struct $ struct_name
{
$
($ (#[$ field_meta : meta]) * pub $ field_name : $
field_type,) *
}
}
}
fn main() {
pub struct name {
pub n: i64,
pub t: i64,
pub g: i64,
}
}Limitations of declarative macros
Declarative macros have a few limitations. Some are related to Rust macros themselves while others are more specific to declarative macros.
- Lack of support for macros autocompletion and expansion
- Debugging declarative macros is difficult
- Limited modification capabilities
- Larger binaries
- Longer compile time (this applies to both declarative and procedural macros)
Procedural macros in Rust
Procedural macros are a more advanced version of macros. Procedural macros allow you to expand the existing syntax of Rust. It takes arbitrary input and returns valid Rust code.
Procedural macros are functions that take a TokenStream as input and return another Token Stream. Procedural macros manipulate the input TokenStream to produce an output stream.
There are three types of procedural macros:
- Attribute-like macros
- Derive macros
- Function-like macros
We’ll go into each procedural macro type in detail below.
Attribute-like macros
Attribute-like macros enable you to create a custom attribute that attaches itself to an item and allows manipulation of that item. It can also take arguments.
#[some_attribute_macro(some_argument)]
fn perform_task(){
// some code
}In the above code, some_attribute_macros is an attribute macro. It manipulates the function perform_task.
To write an attribute-like macro, start by creating a project using cargo new macro-demo --lib. Once the project is ready, update the Cargo.toml to notify cargo the project will create procedural macros.
# Cargo.toml
[lib]
proc-macro = trueNow we are all set to venture into procedural macros.
Procedural macros are public functions that take TokenStream as input and return another TokenStream. To write a procedural macro, we need to write our parser to parse TokenStream. The Rust community has a very good crate, syn, for parsing TokenStream.
synprovides a ready-made parser for Rust syntax that can be used to parse TokenStream. You can also parse your syntax by combining low-level parsers providing syn.
Add syn and quote to Cargo.toml:
# Cargo.toml
[dependencies]
syn = {version="1.0.57",features=["full","fold"]}
quote = "1.0.8"Now we can write an attribute-like a macro in lib.rs using the proc_macro crate provided by the compiler for writing procedural macros. A procedural macro crate cannot export anything else other than procedural macros and procedural macros defined in the crate can’t be used in the crate itself.
// lib.rs
extern crate proc_macro;
use proc_macro::{TokenStream};
use quote::{quote};
// using proc_macro_attribute to declare an attribute like procedural macro
#[proc_macro_attribute]
// _metadata is argument provided to macro call and _input is code to which attribute like macro attaches
pub fn my_custom_attribute(_metadata: TokenStream, _input: TokenStream) -> TokenStream {
// returing a simple TokenStream for Struct
TokenStream::from(quote!{struct H{}})
}To test the macro we added, create an ingratiation test by creating a folder named tests and adding the file attribute_macro.rs in the folder. In this file, we can use our attribute-like macro for testing.
// tests/attribute_macro.rs
use macro_demo::*;
// macro converts struct S to struct H
#[my_custom_attribute]
struct S{}
#[test]
fn test_macro(){
// due to macro we have struct H in scope
let demo=H{};
}Run the above test using the cargo test command.
Now that we understand the basics of procedural macros, lets use syn for some advanced TokenStream manipulation and parsing.
To learn how syn is used for parsing and manipulation, let’s take an example from the syn GitHub repo. This example creates a Rust macro that trace variables when value changes.
First, we need to identify how our macro will manipulate the code it attaches.
#[trace_vars(a)]
fn do_something(){
let a=9;
a=6;
a=0;
}The trace_vars macro takes the name of the variable it needs to trace and injects a print statement each time the value of the input variable i.e a changes. It tracks the value of input variables.
First, parse the code to which the attribute-like macro attaches. syn provides an inbuilt parser for Rust function syntax. ItemFn will parse the function and throw an error if the syntax is invalid.
#[proc_macro_attribute]
pub fn trace_vars(_metadata: TokenStream, input: TokenStream) -> TokenStream {
// parsing rust function to easy to use struct
let input_fn = parse_macro_input!(input as ItemFn);
TokenStream::from(quote!{fn dummy(){}})
}Now that we have the parsed input, let’s move to metadata. For metadata, no inbuilt parser will work, so we’ll have to write one ourselves using syn‘s parse module.
#[trace_vars(a,c,b)] // we need to parse a "," seperated list of tokens
// codeFor syn to work, we need to implement the Parse trait provided by syn. Punctuated is used to create a vector of Indent separated by ,.
struct Args{
vars:HashSet<Ident>
}
impl Parse for Args{
fn parse(input: ParseStream) -> Result<Self> {
// parses a,b,c, or a,b,c where a,b and c are Indent
let vars = Punctuated::<Ident, Token![,]>::parse_terminated(input)?;
Ok(Args {
vars: vars.into_iter().collect(),
})
}
}Once we implement the Parse trait, we can use parse_macro_input macro for parsing metadata.
#[proc_macro_attribute]
pub fn trace_vars(metadata: TokenStream, input: TokenStream) -> TokenStream {
let input_fn = parse_macro_input!(input as ItemFn);
// using newly created struct Args
let args= parse_macro_input!(metadata as Args);
TokenStream::from(quote!{fn dummy(){}})
}We will now modify the input_fn to add println! when the variable changes the value. To add this, we need to filter outlines that have an assignment and insert a print statement after that line.
impl Args {
fn should_print_expr(&self, e: &Expr) -> bool {
match *e {
Expr::Path(ref e) => {
// variable shouldn't start wiht ::
if e.path.leading_colon.is_some() {
false
// should be a single variable like `x=8` not n::x=0
} else if e.path.segments.len() != 1 {
false
} else {
// get the first part
let first = e.path.segments.first().unwrap();
// check if the variable name is in the Args.vars hashset
self.vars.contains(&first.ident) && first.arguments.is_empty()
}
}
_ => false,
}
}
// used for checking if to print let i=0 etc or not
fn should_print_pat(&self, p: &Pat) -> bool {
match p {
// check if variable name is present in set
Pat::Ident(ref p) => self.vars.contains(&p.ident),
_ => false,
}
}
// manipulate tree to insert print statement
fn assign_and_print(&mut self, left: Expr, op: &dyn ToTokens, right: Expr) -> Expr {
// recurive call on right of the assigment statement
let right = fold::fold_expr(self, right);
// returning manipulated sub-tree
parse_quote!({
#left #op #right;
println!(concat!(stringify!(#left), " = {:?}"), #left);
})
}
// manipulating let statement
fn let_and_print(&mut self, local: Local) -> Stmt {
let Local { pat, init, .. } = local;
let init = self.fold_expr(*init.unwrap().1);
// get the variable name of assigned variable
let ident = match pat {
Pat::Ident(ref p) => &p.ident,
_ => unreachable!(),
};
// new sub tree
parse_quote! {
let #pat = {
#[allow(unused_mut)]
let #pat = #init;
println!(concat!(stringify!(#ident), " = {:?}"), #ident);
#ident
};
}
}
}In the above example, the quote macro is used for templating and writing Rust. # is used for injecting the value of the variable.
Now we’ll do a DFS over input_fn and insert the print statement. syn provides a Fold trait that can be implemented for DFS over any Item. We just need to modify the trait methods that correspond with the token type we want to manipulate.
impl Fold for Args {
fn fold_expr(&mut self, e: Expr) -> Expr {
match e {
// for changing assignment like a=5
Expr::Assign(e) => {
// check should print
if self.should_print_expr(&e.left) {
self.assign_and_print(*e.left, &e.eq_token, *e.right)
} else {
// continue with default travesal using default methods
Expr::Assign(fold::fold_expr_assign(self, e))
}
}
// for changing assigment and operation like a+=1
Expr::AssignOp(e) => {
// check should print
if self.should_print_expr(&e.left) {
self.assign_and_print(*e.left, &e.op, *e.right)
} else {
// continue with default behaviour
Expr::AssignOp(fold::fold_expr_assign_op(self, e))
}
}
// continue with default behaviour for rest of expressions
_ => fold::fold_expr(self, e),
}
}
// for let statements like let d=9
fn fold_stmt(&mut self, s: Stmt) -> Stmt {
match s {
Stmt::Local(s) => {
if s.init.is_some() && self.should_print_pat(&s.pat) {
self.let_and_print(s)
} else {
Stmt::Local(fold::fold_local(self, s))
}
}
_ => fold::fold_stmt(self, s),
}
}
}The Fold trait is used to do a DFS of Item. It enables you to use different behavior for various token types.
Now we can use fold_item_fn to inject print statements in our parsed code.
#[proc_macro_attribute]
pub fn trace_var(args: TokenStream, input: TokenStream) -> TokenStream {
// parse the input
let input = parse_macro_input!(input as ItemFn);
// parse the arguments
let mut args = parse_macro_input!(args as Args);
// create the ouput
let output = args.fold_item_fn(input);
// return the TokenStream
TokenStream::from(quote!(#output))
}This code example is from the syn examples repo, which is an excellent resource to learn about procedural macros.
Custom derive macros
Custom derive macros in Rust allow auto implement traits. These macros enable you to implement traits using #[derive(Trait)].
syn has excellent support for derive macros.
#[derive(Trait)]
struct MyStruct{}To write a custom derive macro in Rust, we can use DeriveInput for parsing input to derive macro. We’ll also use the proc_macro_derive macro to define a custom derive macro.
#[proc_macro_derive(Trait)]
pub fn derive_trait(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let name = input.ident;
let expanded = quote! {
impl Trait for #name {
fn print(&self) -> usize {
println!("{}","hello from #name")
}
}
};
proc_macro::TokenStream::from(expanded)
}More advanced procedural macros can be written using syn. Check out this example from syn‘s repo.
Function-like macros
Function-like macros are similar to declarative macros in that they’re invoked with the macro invocation operator ! and look like function calls. They operate on the code that is inside the parentheses.
Here’s how to write a function-like macro in Rust:
#[proc_macro]
pub fn a_proc_macro(_input: TokenStream) -> TokenStream {
TokenStream::from(quote!(
fn anwser()->i32{
5
}
))
}Function-like macros are executed not at runtime but at compile time. They can be used anywhere in Rust code. Function-like macros also take a TokenStream and return a TokenStream.
Advantages of using procedural macros include:
- Better error handling using
span - Better control over output
- Community-built crates
synandquote - More powerful than declarative macros
Conclusion
In this Rust macros tutorial, we covered the basics of macros in Rust, defined declarative and procedural macros, and walked through how to write both types of macros using various syntax and community-built crates. We also outlined the advantages of using each type of Rust macro.
#rust #programming