Getting Started with Promises in Nodejs

In this article, I will show how we can use Promises in Nodejs. Promises are kind of design patterns to remove the usage of unintuitive callbacks.

Promises are a built-in concurrency primitive that’s been part of JavaScript since ES6 in 2015. The Promise class has been included in Node.js since v4.0.0. That means, unless you’re on an unmaintained version of Node.js, you can use promises without any outside libraries.

Getting Started with Promises in Node.js

A promise is an object representation of a value that is being computed asynchronously. The easiest way to create a promise is using the Promise.resolve function.

// Create a promise with a pre-computed value 'Hello, World'
const promise = Promise.resolve('Hello, World!');

promise instanceof Promise; // true

The most important function for interacting with promises is the Promise#then function. To get the promise’s associated value once it is done being computed, you should call then():

// Prints "Hello, World"
promise.then(res => console.log(res));

There is no way to access a promise’s value directly. In other words, the only way to “unbox” promise into the value ‘Hello, World’ is by calling .then().

The Technical Details of Promises

You can think of a promise as an object representation of an asynchronous operation. Technically, a promise is a state machine representing the status of an asynchronous operation. A promise can be in one of 3 states:

1. Pending. The operation is in progress.

2. Fulfilled. The operation completed successfully.

3. Rejected. The operation experienced an error.

Here’s a state diagram from Mastering Async/Await that demonstrates the possible states:

Once a promise is either fulfilled or rejected, it stays fulfilled or rejected. That is why a promise that is no longer pending is called settled.

Another important detail is that, as the client of a promise-based API, you can’t fulfill or reject a promise. Given a promise object, you cannot make the promise change state, even if it is pending. You can only react to what the promise does using functions like then().

Speaking of then(), let’s take another look at then() given the perspective of a promise as a state machine. The first parameter to then() is a function called onFulfilled.

promise.then(function onFulfilled(res) {
  console.log(res); // 'Hello, World'
});

As the name onFulfilled implies, Node.js will call your onFulfilled function if your promise changes state from pending to fulfilled. If you call then() on a function that is already fulfilled, Node.js will call onFulfilled immediately.

Errors and catch()

If promise changes state from pending to fulfilled, Node.js will call your onFulfilled function. What happens if promise changes state from pending to rejected?

Turns out the .then() function takes two function parameters, onFulfilled and onRejected. Node.js calls the onRejected function if your promise changes state from pending to rejected, or if the promise is already rejected.

promise.then(
  function onFulfilled(res) { console.log(res); },
  function onRejected(err) { console.log(err); }
);

If onRejected is null or undefined, Node.js will not treat that as an error. If a promise is rejected and it doesn’t have an onRejected handler, that will become an unhandled promise rejection.

Promises also have a Promise.reject function analagous to Promise.resolve(). Here’s an example of using Promise.reject() with an onRejected handler.

Promise.reject(new Error('Oops!')).then(null, function onRejected(err) {
  console.log(err.message); // Oops!
});

The .then(null, function onRejected() pattern is so common in promises that there’s a helper function for it: Promise#catch. The catch() function lets you add an onRejected handler without passing an onFulfilled() function.

Promise.reject(new Error('Oops!')).catch(function onRejected(err) {
  console.log(err.message); // Oops!
});

Chaining Promises

Promise chaining is what lets promises avoid deeply nested callbacks, also known as “callback hell”, “pyramid of doom”, and “banana code”.

The idea of promise chaining is that if you call then() with an onFulfilled() that function returns a new promise p, then() should return a promise that is “locked in” to match the state of p.

For example:

const p1 = Promise.resolve('p1');
// The `then()` function returns a promise!
const p2 = p1.then(() => Promise.resolve('p2'));

// Prints "p2"
p2.then(res => console.log(res));

In practice, promise chaining generally looks like a list of chained .then() calls. For example:

return asyncOperation1().
  then(res => asyncOperation2(res)).
  then(res => asyncOperation3(res)).
  // If any of `asyncOperation1()` - `asyncOperation3()` rejects, Node will
  // call this `onRejected()` handler.
  catch(function onRejected(err) { console.log(err); });

Now that you’ve seen the basics of promise chaining, lets see how you would use promise chaining with core Node.js APIs.

Using Promises with Core Node.js APIs

Node.js 10 introduced a promise-based API for the built-in fs module. That means you can read from and write to the file system using promises in Node.js.

To use the promise-based fs API, use the following syntax:

const fs = require('fs').promises; 

Now, suppose you want to read a file, replace all instances of ‘foo’ with ‘bar’, and write the file back to disk. Here’s how that might look with promise chaining.

const fs = require('fs').promises;

function replace(filename) {
  let handle;
  let contents;
  // Open a file handle
  return fs.open(filename, 'r+').
    then(_handle => {
      handle = _handle;
      // Read the contents of the file
      return handle.readFile();
    }).
    // Replace instances of 'foo' with 'bar' in the contents
    then(_contents => {
      contents = _contents.toString().replace(/foo/g, 'bar');
    }).
    // Empty out the file
    then(() => handle.truncate()).
    // Write the replaced contents to the file
    then(() => handle.writeFile(contents)).
    then(() => console.log('done')).
    // If any step fails, this `onRejected()` handler will get called.
    catch(err => console.log(err));
}

Converting Callbacks to Promises

Many older Node.js APIs are still written using callbacks. Node.js has a promisify function that can convert a function that uses Node.js’ callback syntax to a function that returns a promise.

const util = require('util');

function usingCallback(callback) {
  // Call `callback()` after 50 ms
  setTimeout(() => callback(null, 'Hello, World'), 50);
}

const usingPromise = util.promisify(usingCallback);

// Prints "Hello, World" after 50ms.
usingPromise().then(res => console.log(res));

#node-js #javascript #web-development

What is GEEK

Buddha Community

Everything You Need to Know About Instagram Bot with Python

How to build an Instagram bot using Python

Instagram is the fastest-growing social network, with 1 billion monthly users. It also has the highest engagement rate. To gain followers on Instagram, you’d have to upload engaging content, follow users, like posts, comment on user posts and a whole lot. This can be time-consuming and daunting. But there is hope, you can automate all of these tasks. In this course, we’re going to build an Instagram bot using Python to automate tasks on Instagram.

What you’ll learn:

• Instagram Automation
• Build a Bot with Python

---

Increase your Instagram followers with a simple Python bot

I got around 500 real followers in 4 days!

Growing an audience is an expensive and painful task. And if you’d like to build an audience that’s relevant to you, and shares common interests, that’s even more difficult. I always saw Instagram has a great way to promote my photos, but I never had more than 380 followers… Every once in a while, I decide to start posting my photos on Instagram again, and I manage to keep posting regularly for a while, but it never lasts more than a couple of months, and I don’t have many followers to keep me motivated and engaged.

The objective of this project is to build a bigger audience and as a plus, maybe drive some traffic to my website where I sell my photos!

A year ago, on my last Instagram run, I got one of those apps that lets you track who unfollowed you. I was curious because in a few occasions my number of followers dropped for no apparent reason. After some research, I realized how some users basically crawl for followers. They comment, like and follow people — looking for a follow back. Only to unfollow them again in the next days.

I can’t say this was a surprise to me, that there were bots in Instagram… It just made me want to build one myself!

And that is why we’re here, so let’s get to it! I came up with a simple bot in Python, while I was messing around with Selenium and trying to figure out some project to use it. Simply put, Selenium is like a browser you can interact with very easily in Python.

Ideally, increasing my Instagram audience will keep me motivated to post regularly. As an extra, I included my website in my profile bio, where people can buy some photos. I think it is a bit of a stretch, but who knows?! My sales are basically zero so far, so it should be easy to track that conversion!

Just what the world needed! Another Instagram bot…

After giving this project some thought, my objective was to increase my audience with relevant people. I want to get followers that actually want to follow me and see more of my work. It’s very easy to come across weird content in the most used hashtags, so I’ve planed this bot to lookup specific hashtags and interact with the photos there. This way, I can be very specific about what kind of interests I want my audience to have. For instance, I really like long exposures, so I can target people who use that hashtag and build an audience around this kind of content. Simple and efficient!

My gallery is a mix of different subjects and styles, from street photography to aerial photography, and some travel photos too. Since it’s my hometown, I also have lots of Lisbon images there. These will be the main topics I’ll use in the hashtags I want to target.

This is not a “get 1000 followers in 24 hours” kind of bot!

So what kind of numbers are we talking about?

I ran the bot a few times in a few different hashtags like “travelblogger”, “travelgram”, “lisbon”, “dronephotography”. In the course of three days I went from 380 to 800 followers. Lots of likes, comments and even some organic growth (people that followed me but were not followed by the bot).

To be clear, I’m not using this bot intensively, as Instagram will stop responding if you run it too fast. It needs to have some sleep commands in between the actions, because after some comments and follows in a short period of time, Instagram stops responding and the bot crashes.

You will be logged into your account, so I’m almost sure that Instagram can know you’re doing something weird if you speed up the process. And most importantly, after doing this for a dozen hashtags, it just gets harder to find new users in the same hashtags. You will need to give it a few days to refresh the user base there.

But I don’t want to follow so many people in the process…

The most efficient way to get followers in Instagram (apart from posting great photos!) is to follow people. And this bot worked really well for me because I don’t care if I follow 2000 people to get 400 followers.

The bot saves a list with all the users that were followed while it was running, so someday I may actually do something with this list. For instance, I can visit each user profile, evaluate how many followers or posts they have, and decide if I want to keep following them. Or I can get the first picture in their gallery and check its date to see if they are active users.

If we remove the follow action from the bot, I can assure you the growth rate will suffer, as people are less inclined to follow based on a single like or comment.

Why will you share your code?!

That’s the debate I had with myself. Even though I truly believe in giving back to the community (I still learn a lot from it too!), there are several paid platforms that do more or less the same as this project. Some are shady, some are used by celebrities. The possibility of starting a similar platform myself, is not off the table yet, so why make the code available?

With that in mind, I decided to add an extra level of difficulty to the process, so I was going to post the code below as an image. I wrote “was”, because meanwhile, I’ve realized the image I’m getting is low quality. Which in turn made me reconsider and post the gist. I’m that nice! The idea behind the image was that if you really wanted to use it, you would have to type the code yourself. And that was my way of limiting the use of this tool to people that actually go through the whole process to create it and maybe even improve it.

I learn a lot more when I type the code myself, instead of copy/pasting scripts. I hope you feel the same way!

The script isn’t as sophisticated as it could be, and I know there’s lots of room to improve it. But hey… it works! I have other projects I want to add to my portfolio, so my time to develop it further is rather limited. Nevertheless, I will try to update this article if I dig deeper.

This is the last subtitle!

You’ll need Python (I’m using Python 3.7), Selenium, a browser (in my case I’ll be using Chrome) and… obviously, an Instagram account! Quick overview regarding what the bot will do:

• Open a browser and login with your credentials
• For every hashtag in the hashtag list, it will open the page and click the first picture to open it
• It will then like, follow, comment and move to the next picture, in a 200 iterations loop (number can be adjusted)
• Saves a list with all the users you followed using the bot

If you reached this paragraph, thank you! You totally deserve to collect your reward! If you find this useful for your profile/brand in any way, do share your experience below :)

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep, strftime
from random import randint
import pandas as pd

chromedriver_path = 'C:/Users/User/Downloads/chromedriver_win32/chromedriver.exe' # Change this to your own chromedriver path!
webdriver = webdriver.Chrome(executable_path=chromedriver_path)
sleep(2)
webdriver.get('https://www.instagram.com/accounts/login/?source=auth_switcher')
sleep(3)

username = webdriver.find_element_by_name('username')
username.send_keys('your_username')
password = webdriver.find_element_by_name('password')
password.send_keys('your_password')

button_login = webdriver.find_element_by_css_selector('#react-root > section > main > div > article > div > div:nth-child(1) > div > form > div:nth-child(3) > button')
button_login.click()
sleep(3)

notnow = webdriver.find_element_by_css_selector('body > div:nth-child(13) > div > div > div > div.mt3GC > button.aOOlW.HoLwm')
notnow.click() #comment these last 2 lines out, if you don't get a pop up asking about notifications

In order to use chrome with Selenium, you need to install chromedriver. It’s a fairly simple process and I had no issues with it. Simply install and replace the path above. Once you do that, our variable webdriver will be our Chrome tab.

In cell number 3 you should replace the strings with your own username and the respective password. This is for the bot to type it in the fields displayed. You might have already noticed that when running cell number 2, Chrome opened a new tab. After the password, I’ll define the login button as an object, and in the following line, I click it.

Once you get in inspect mode find the bit of html code that corresponds to what you want to map. Right click it and hover over Copy. You will see that you have some options regarding how you want it to be copied. I used a mix of XPath and css selectors throughout the code (it’s visible in the find_element_ method). It took me a while to get all the references to run smoothly. At points, the css or the xpath directions would fail, but as I adjusted the sleep times, everything started running smoothly.

In this case, I selected “copy selector” and pasted it inside a find_element_ method (cell number 3). It will get you the first result it finds. If it was find_elements_, all elements would be retrieved and you could specify which to get.

Once you get that done, time for the loop. You can add more hashtags in the hashtag_list. If you run it for the first time, you still don’t have a file with the users you followed, so you can simply create prev_user_list as an empty list.

Once you run it once, it will save a csv file with a timestamp with the users it followed. That file will serve as the prev_user_list on your second run. Simple and easy to keep track of what the bot does.

Update with the latest timestamp on the following runs and you get yourself a series of csv backlogs for every run of the bot.

The code is really simple. If you have some basic notions of Python you can probably pick it up quickly. I’m no Python ninja and I was able to build it, so I guess that if you read this far, you are good to go!

hashtag_list = ['travelblog', 'travelblogger', 'traveler']

# prev_user_list = [] - if it's the first time you run it, use this line and comment the two below
prev_user_list = pd.read_csv('20181203-224633_users_followed_list.csv', delimiter=',').iloc[:,1:2] # useful to build a user log
prev_user_list = list(prev_user_list['0'])

new_followed = []
tag = -1
followed = 0
likes = 0
comments = 0

for hashtag in hashtag_list:
    tag += 1
    webdriver.get('https://www.instagram.com/explore/tags/'+ hashtag_list[tag] + '/')
    sleep(5)
    first_thumbnail = webdriver.find_element_by_xpath('//*[@id="react-root"]/section/main/article/div[1]/div/div/div[1]/div[1]/a/div')
    
    first_thumbnail.click()
    sleep(randint(1,2))    
    try:        
        for x in range(1,200):
            username = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[1]/h2/a').text
            
            if username not in prev_user_list:
                # If we already follow, do not unfollow
                if webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[2]/button').text == 'Follow':
                    
                    webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/header/div[2]/div[1]/div[2]/button').click()
                    
                    new_followed.append(username)
                    followed += 1

                    # Liking the picture
                    button_like = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[1]/span[1]/button/span')
                    
                    button_like.click()
                    likes += 1
                    sleep(randint(18,25))

                    # Comments and tracker
                    comm_prob = randint(1,10)
                    print('{}_{}: {}'.format(hashtag, x,comm_prob))
                    if comm_prob > 7:
                        comments += 1
                        webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[1]/span[2]/button/span').click()
                        comment_box = webdriver.find_element_by_xpath('/html/body/div[3]/div/div[2]/div/article/div[2]/section[3]/div/form/textarea')

                        if (comm_prob < 7):
                            comment_box.send_keys('Really cool!')
                            sleep(1)
                        elif (comm_prob > 6) and (comm_prob < 9):
                            comment_box.send_keys('Nice work :)')
                            sleep(1)
                        elif comm_prob == 9:
                            comment_box.send_keys('Nice gallery!!')
                            sleep(1)
                        elif comm_prob == 10:
                            comment_box.send_keys('So cool! :)')
                            sleep(1)
                        # Enter to post comment
                        comment_box.send_keys(Keys.ENTER)
                        sleep(randint(22,28))

                # Next picture
                webdriver.find_element_by_link_text('Next').click()
                sleep(randint(25,29))
            else:
                webdriver.find_element_by_link_text('Next').click()
                sleep(randint(20,26))
    # some hashtag stops refreshing photos (it may happen sometimes), it continues to the next
    except:
        continue

for n in range(0,len(new_followed)):
    prev_user_list.append(new_followed[n])
    
updated_user_df = pd.DataFrame(prev_user_list)
updated_user_df.to_csv('{}_users_followed_list.csv'.format(strftime("%Y%m%d-%H%M%S")))
print('Liked {} photos.'.format(likes))
print('Commented {} photos.'.format(comments))
print('Followed {} new people.'.format(followed))

The print statement inside the loop is the way I found to be able to have a tracker that lets me know at what iteration the bot is all the time. It will print the hashtag it’s in, the number of the iteration, and the random number generated for the comment action. I decided not to post comments in every page, so I added three different comments and a random number between 1 and 10 that would define if there was any comment at all, or one of the three. The loop ends, we append the new_followed users to the previous users “database” and saves the new file with the timestamp. You should also get a small report.

And that’s it!

After a few hours without checking the phone, these were the numbers I was getting. I definitely did not expect it to do so well! In about 4 days since I’ve started testing it, I had around 500 new followers, which means I have doubled my audience in a matter of days. I’m curious to see how many of these new followers I will lose in the next days, to see if the growth can be sustainable. I also had a lot more “likes” in my latest photos, but I guess that’s even more expected than the follow backs.

It would be nice to get this bot running in a server, but I have other projects I want to explore, and configuring a server is not one of them! Feel free to leave a comment below, and I’ll do my best to answer your questions.

I’m actually curious to see how long will I keep posting regularly! If you feel like this article was helpful for you, consider thanking me by buying one of my photos.

---

---

How to Make an Instagram Bot With Python and InstaPy

What do SocialCaptain, Kicksta, Instavast, and many other companies have in common? They all help you reach a greater audience, gain more followers, and get more likes on Instagram while you hardly lift a finger. They do it all through automation, and people pay them a good deal of money for it. But you can do the same thing—for free—using InstaPy!

In this tutorial, you’ll learn how to build a bot with Python and InstaPy, which automates your Instagram activities so that you gain more followers and likes with minimal manual input. Along the way, you’ll learn about browser automation with Selenium and the Page Object Pattern, which together serve as the basis for InstaPy.

In this tutorial, you’ll learn:

• How Instagram bots work
• How to automate a browser with Selenium
• How to use the Page Object Pattern for better readability and testability
• How to build an Instagram bot with InstaPy

You’ll begin by learning how Instagram bots work before you build one.

Table of Contents

• How Instagram Bots Work
• How to Automate a Browser
• How to Use the Page Object Pattern
• How to Build an Instagram Bot With InstaPy
  • Essential Features
  • Additional Features in InstaPy
• Conclusion

Important: Make sure you check Instagram’s Terms of Use before implementing any kind of automation or scraping techniques.

How Instagram Bots Work

How can an automation script gain you more followers and likes? Before answering this question, think about how an actual person gains more followers and likes.

They do it by being consistently active on the platform. They post often, follow other people, and like and leave comments on other people’s posts. Bots work exactly the same way: They follow, like, and comment on a consistent basis according to the criteria you set.

The better the criteria you set, the better your results will be. You want to make sure you’re targeting the right groups because the people your bot interacts with on Instagram will be more likely to interact with your content.

For example, if you’re selling women’s clothing on Instagram, then you can instruct your bot to like, comment on, and follow mostly women or profiles whose posts include hashtags such as #beauty, #fashion, or #clothes. This makes it more likely that your target audience will notice your profile, follow you back, and start interacting with your posts.

How does it work on the technical side, though? You can’t use the Instagram Developer API since it is fairly limited for this purpose. Enter browser automation. It works in the following way:

1. You serve it your credentials.
2. You set the criteria for who to follow, what comments to leave, and which type of posts to like.
3. Your bot opens a browser, types in https://instagram.com on the address bar, logs in with your credentials, and starts doing the things you instructed it to do.

Next, you’ll build the initial version of your Instagram bot, which will automatically log in to your profile. Note that you won’t use InstaPy just yet.

How to Automate a Browser

For this version of your Instagram bot, you’ll be using Selenium, which is the tool that InstaPy uses under the hood.

First, install Selenium. During installation, make sure you also install the Firefox WebDriver since the latest version of InstaPy dropped support for Chrome. This also means that you need the Firefox browser installed on your computer.

Now, create a Python file and write the following code in it:

from time import sleep

from selenium import webdriver


browser = webdriver.Firefox()


browser.get('https://www.instagram.com/')


sleep(5)


browser.close()

Run the code and you’ll see that a Firefox browser opens and directs you to the Instagram login page. Here’s a line-by-line breakdown of the code:

• Lines 1 and 2 import sleep and webdriver.
• Line 4 initializes the Firefox driver and sets it to browser.
• Line 6 types https://www.instagram.com/ on the address bar and hits Enter.
• Line 8 waits for five seconds so you can see the result. Otherwise, it would close the browser instantly.
• Line 10 closes the browser.

This is the Selenium version of Hello, World. Now you’re ready to add the code that logs in to your Instagram profile. But first, think about how you would log in to your profile manually. You would do the following:

1. Go to https://www.instagram.com/.
2. Click the login link.
3. Enter your credentials.
4. Hit the login button.

The first step is already done by the code above. Now change it so that it clicks on the login link on the Instagram home page:

from time import sleep

from selenium import webdriver


browser = webdriver.Firefox()

browser.implicitly_wait(5)


browser.get('https://www.instagram.com/')


login_link = browser.find_element_by_xpath("//a[text()='Log in']")

login_link.click()


sleep(5)


browser.close()

Note the highlighted lines:

• Line 5 sets five seconds of waiting time. If Selenium can’t find an element, then it waits for five seconds to allow everything to load and tries again.
• Line 9 finds the element <a> whose text is equal to Log in. It does this using XPath, but there are a few other methods you could use.
• Line 10 clicks on the found element <a> for the login link.

Run the script and you’ll see your script in action. It will open the browser, go to Instagram, and click on the login link to go to the login page.

On the login page, there are three important elements:

1. The username input
2. The password input
3. The login button

Next, change the script so that it finds those elements, enters your credentials, and clicks on the login button:

from time import sleep

from selenium import webdriver


browser = webdriver.Firefox()

browser.implicitly_wait(5)


browser.get('https://www.instagram.com/')


login_link = browser.find_element_by_xpath("//a[text()='Log in']")

login_link.click()


sleep(2)


username_input = browser.find_element_by_css_selector("input[name='username']")

password_input = browser.find_element_by_css_selector("input[name='password']")


username_input.send_keys("<your username>")

password_input.send_keys("<your password>")


login_button = browser.find_element_by_xpath("//button[@type='submit']")

login_button.click()


sleep(5)


browser.close()

Here’s a breakdown of the changes:

1. Line 12 sleeps for two seconds to allow the page to load.
2. Lines 14 and 15 find username and password inputs by CSS. You could use any other method that you prefer.
3. Lines 17 and 18 type your username and password in their respective inputs. Don’t forget to fill in <your username> and <your password>!
4. Line 20 finds the login button by XPath.
5. Line 21 clicks on the login button.

Run the script and you’ll be automatically logged in to to your Instagram profile.

You’re off to a good start with your Instagram bot. If you were to continue writing this script, then the rest would look very similar. You would find the posts that you like by scrolling down your feed, find the like button by CSS, click on it, find the comments section, leave a comment, and continue.

The good news is that all of those steps can be handled by InstaPy. But before you jump into using Instapy, there is one other thing that you should know about to better understand how InstaPy works: the Page Object Pattern.

How to Use the Page Object Pattern

Now that you’ve written the login code, how would you write a test for it? It would look something like the following:

def test_login_page(browser):
    browser.get('https://www.instagram.com/accounts/login/')
    username_input = browser.find_element_by_css_selector("input[name='username']")
    password_input = browser.find_element_by_css_selector("input[name='password']")
    username_input.send_keys("<your username>")
    password_input.send_keys("<your password>")
    login_button = browser.find_element_by_xpath("//button[@type='submit']")
    login_button.click()

    errors = browser.find_elements_by_css_selector('#error_message')
    assert len(errors) == 0

Can you see what’s wrong with this code? It doesn’t follow the DRY principle. That is, the code is duplicated in both the application and the test code.

Duplicating code is especially bad in this context because Selenium code is dependent on UI elements, and UI elements tend to change. When they do change, you want to update your code in one place. That’s where the Page Object Pattern comes in.

With this pattern, you create page object classes for the most important pages or fragments that provide interfaces that are straightforward to program to and that hide the underlying widgetry in the window. With this in mind, you can rewrite the code above and create a HomePage class and a LoginPage class:

from time import sleep

class LoginPage:
    def __init__(self, browser):
        self.browser = browser

    def login(self, username, password):
        username_input = self.browser.find_element_by_css_selector("input[name='username']")
        password_input = self.browser.find_element_by_css_selector("input[name='password']")
        username_input.send_keys(username)
        password_input.send_keys(password)
        login_button = browser.find_element_by_xpath("//button[@type='submit']")
        login_button.click()
        sleep(5)

class HomePage:
    def __init__(self, browser):
        self.browser = browser
        self.browser.get('https://www.instagram.com/')

    def go_to_login_page(self):
        self.browser.find_element_by_xpath("//a[text()='Log in']").click()
        sleep(2)
        return LoginPage(self.browser)

The code is the same except that the home page and the login page are represented as classes. The classes encapsulate the mechanics required to find and manipulate the data in the UI. That is, there are methods and accessors that allow the software to do anything a human can.

One other thing to note is that when you navigate to another page using a page object, it returns a page object for the new page. Note the returned value of go_to_log_in_page(). If you had another class called FeedPage, then login() of the LoginPage class would return an instance of that: return FeedPage().

Here’s how you can put the Page Object Pattern to use:

from selenium import webdriver

browser = webdriver.Firefox()
browser.implicitly_wait(5)

home_page = HomePage(browser)
login_page = home_page.go_to_login_page()
login_page.login("<your username>", "<your password>")

browser.close()

It looks much better, and the test above can now be rewritten to look like this:

def test_login_page(browser):
    home_page = HomePage(browser)
    login_page = home_page.go_to_login_page()
    login_page.login("<your username>", "<your password>")

    errors = browser.find_elements_by_css_selector('#error_message')
    assert len(errors) == 0

With these changes, you won’t have to touch your tests if something changes in the UI.

For more information on the Page Object Pattern, refer to the official documentation and to Martin Fowler’s article.

Now that you’re familiar with both Selenium and the Page Object Pattern, you’ll feel right at home with InstaPy. You’ll build a basic bot with it next.

Note: Both Selenium and the Page Object Pattern are widely used for other websites, not just for Instagram.

How to Build an Instagram Bot With InstaPy

In this section, you’ll use InstaPy to build an Instagram bot that will automatically like, follow, and comment on different posts. First, you’ll need to install InstaPy:

$ python3 -m pip install instapy

This will install instapy in your system.

Essential Features

Now you can rewrite the code above with InstaPy so that you can compare the two options. First, create another Python file and put the following code in it:

from instapy import InstaPy

InstaPy(username="<your_username>", password="<your_password>").login()

Replace the username and password with yours, run the script, and voilà! With just one line of code, you achieved the same result.

Even though your results are the same, you can see that the behavior isn’t exactly the same. In addition to simply logging in to your profile, InstaPy does some other things, such as checking your internet connection and the status of the Instagram servers. This can be observed directly on the browser or in the logs:

INFO [2019-12-17 22:03:19] [username]  -- Connection Checklist [1/3] (Internet Connection Status)
INFO [2019-12-17 22:03:20] [username]  - Internet Connection Status: ok
INFO [2019-12-17 22:03:20] [username]  - Current IP is "17.283.46.379" and it's from "Germany/DE"
INFO [2019-12-17 22:03:20] [username]  -- Connection Checklist [2/3] (Instagram Server Status)
INFO [2019-12-17 22:03:26] [username]  - Instagram WebSite Status: Currently Up

Pretty good for one line of code, isn’t it? Now it’s time to make the script do more interesting things than just logging in.

For the purpose of this example, assume that your profile is all about cars, and that your bot is intended to interact with the profiles of people who are also interested in cars.

First, you can like some posts that are tagged #bmw or #mercedes using like_by_tags():

from instapy import InstaPy


session = InstaPy(username="<your_username>", password="<your_password>")

session.login()

session.like_by_tags(["bmw", "mercedes"], amount=5)

Here, you gave the method a list of tags to like and the number of posts to like for each given tag. In this case, you instructed it to like ten posts, five for each of the two tags. But take a look at what happens after you run the script:

INFO [2019-12-17 22:15:58] [username]  Tag [1/2]
INFO [2019-12-17 22:15:58] [username]  --> b'bmw'
INFO [2019-12-17 22:16:07] [username]  desired amount: 14  |  top posts [disabled]: 9  |  possible posts: 43726739
INFO [2019-12-17 22:16:13] [username]  Like# [1/14]
INFO [2019-12-17 22:16:13] [username]  https://www.instagram.com/p/B6MCcGcC3tU/
INFO [2019-12-17 22:16:15] [username]  Image from: b'mattyproduction'
INFO [2019-12-17 22:16:15] [username]  Link: b'https://www.instagram.com/p/B6MCcGcC3tU/'
INFO [2019-12-17 22:16:15] [username]  Description: b'Mal etwas anderes \xf0\x9f\x91\x80\xe2\x98\xba\xef\xb8\x8f Bald ist das komplette Video auf YouTube zu finden (n\xc3\xa4here Infos werden folgen). Vielen Dank an @patrick_jwki @thehuthlife  und @christic_  f\xc3\xbcr das bereitstellen der Autos \xf0\x9f\x94\xa5\xf0\x9f\x98\x8d#carporn#cars#tuning#bagged#bmw#m2#m2competition#focusrs#ford#mk3#e92#m3#panasonic#cinematic#gh5s#dji#roninm#adobe#videography#music#bimmer#fordperformance#night#shooting#'
INFO [2019-12-17 22:16:15] [username]  Location: b'K\xc3\xb6ln, Germany'
INFO [2019-12-17 22:16:51] [username]  --> Image Liked!
INFO [2019-12-17 22:16:56] [username]  --> Not commented
INFO [2019-12-17 22:16:57] [username]  --> Not following
INFO [2019-12-17 22:16:58] [username]  Like# [2/14]
INFO [2019-12-17 22:16:58] [username]  https://www.instagram.com/p/B6MDK1wJ-Kb/
INFO [2019-12-17 22:17:01] [username]  Image from: b'davs0'
INFO [2019-12-17 22:17:01] [username]  Link: b'https://www.instagram.com/p/B6MDK1wJ-Kb/'
INFO [2019-12-17 22:17:01] [username]  Description: b'Someone said cloud? \xf0\x9f\xa4\x94\xf0\x9f\xa4\xad\xf0\x9f\x98\x88 \xe2\x80\xa2\n\xe2\x80\xa2\n\xe2\x80\xa2\n\xe2\x80\xa2\n#bmw #bmwrepost #bmwm4 #bmwm4gts #f82 #bmwmrepost #bmwmsport #bmwmperformance #bmwmpower #bmwm4cs #austinyellow #davs0 #mpower_official #bmw_world_ua #bimmerworld #bmwfans #bmwfamily #bimmers #bmwpost #ultimatedrivingmachine #bmwgang #m3f80 #m5f90 #m4f82 #bmwmafia #bmwcrew #bmwlifestyle'
INFO [2019-12-17 22:17:34] [username]  --> Image Liked!
INFO [2019-12-17 22:17:37] [username]  --> Not commented
INFO [2019-12-17 22:17:38] [username]  --> Not following

By default, InstaPy will like the first nine top posts in addition to your amount value. In this case, that brings the total number of likes per tag to fourteen (nine top posts plus the five you specified in amount).

Also note that InstaPy logs every action it takes. As you can see above, it mentions which post it liked as well as its link, description, location, and whether the bot commented on the post or followed the author.

You may have noticed that there are delays after almost every action. That’s by design. It prevents your profile from getting banned on Instagram.

Now, you probably don’t want your bot liking inappropriate posts. To prevent that from happening, you can use set_dont_like():

from instapy import InstaPy

session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])

With this change, posts that have the words naked or nsfw in their descriptions won’t be liked. You can flag any other words that you want your bot to avoid.

Next, you can tell the bot to not only like the posts but also to follow some of the authors of those posts. You can do that with set_do_follow():

from instapy import InstaPy

session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)

If you run the script now, then the bot will follow fifty percent of the users whose posts it liked. As usual, every action will be logged.

You can also leave some comments on the posts. There are two things that you need to do. First, enable commenting with set_do_comment():

from instapy import InstaPy

session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)

Next, tell the bot what comments to leave with set_comments():

from instapy import InstaPy

session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)
session.set_comments(["Nice!", "Sweet!", "Beautiful :heart_eyes:"])

Run the script and the bot will leave one of those three comments on half the posts that it interacts with.

Now that you’re done with the basic settings, it’s a good idea to end the session with end():

from instapy import InstaPy

session = InstaPy(username="<your_username>", password="<your_password>")
session.login()
session.like_by_tags(["bmw", "mercedes"], amount=5)
session.set_dont_like(["naked", "nsfw"])
session.set_do_follow(True, percentage=50)
session.set_do_comment(True, percentage=50)
session.set_comments(["Nice!", "Sweet!", "Beautiful :heart_eyes:"])
session.end()

This will close the browser, save the logs, and prepare a report that you can see in the console output.

Additional Features in InstaPy

InstaPy is a sizable project that has a lot of thoroughly documented features. The good news is that if you’re feeling comfortable with the features you used above, then the rest should feel pretty similar. This section will outline some of the more useful features of InstaPy.

Quota Supervisor

You can’t scrape Instagram all day, every day. The service will quickly notice that you’re running a bot and will ban some of its actions. That’s why it’s a good idea to set quotas on some of your bot’s actions. Take the following for example:

session.set_quota_supervisor(enabled=True, peak_comments_daily=240, peak_comments_hourly=21)

The bot will keep commenting until it reaches its hourly and daily limits. It will resume commenting after the quota period has passed.

Headless Browser

This feature allows you to run your bot without the GUI of the browser. This is super useful if you want to deploy your bot to a server where you may not have or need the graphical interface. It’s also less CPU intensive, so it improves performance. You can use it like so:

session = InstaPy(username='test', password='test', headless_browser=True)

Note that you set this flag when you initialize the InstaPy object.

Using AI to Analyze Posts

Earlier you saw how to ignore posts that contain inappropriate words in their descriptions. What if the description is good but the image itself is inappropriate? You can integrate your InstaPy bot with ClarifAI, which offers image and video recognition services:

session.set_use_clarifai(enabled=True, api_key='<your_api_key>')
session.clarifai_check_img_for(['nsfw'])

Now your bot won’t like or comment on any image that ClarifAI considers NSFW. You get 5,000 free API-calls per month.

Relationship Bounds

It’s often a waste of time to interact with posts by people who have a lot of followers. In such cases, it’s a good idea to set some relationship bounds so that your bot doesn’t waste your precious computing resources:

session.set_relationship_bounds(enabled=True, max_followers=8500)

With this, your bot won’t interact with posts by users who have more than 8,500 followers.

For many more features and configurations in InstaPy, check out the documentation.

Conclusion

InstaPy allows you to automate your Instagram activities with minimal fuss and effort. It’s a very flexible tool with a lot of useful features.

In this tutorial, you learned:

• How Instagram bots work
• How to automate a browser with Selenium
• How to use the Page Object Pattern to make your code more maintainable and testable
• How to use InstaPy to build a basic Instagram bot

Read the InstaPy documentation and experiment with your bot a little bit. Soon you’ll start getting new followers and likes with a minimal amount of effort. I gained a few new followers myself while writing this tutorial.

---

Automating Instagram API with Python

Gain active followers - Algorithm

Maybe some of you do not agree it is a good way to grow your IG page by using follow for follow method but after a lot of researching I found the proper way to use this method.

I have done and used this strategy for a while and my page visits also followers started growing.

The majority of people failing because they randomly targeting the followers and as a result, they are not coming back to your page. So, the key is to find people those have same interests with you.

If you have a programming page go and search for IG pages which have big programming community and once you find one, don’t send follow requests to followers of this page. Because some of them are not active even maybe fake accounts. So, in order to gain active followers, go the last post of this page and find people who liked the post.

Unofficial Instagram API

In order to query data from Instagram I am going to use the very cool, yet unofficial, Instagram API written by Pasha Lev.

**Note:**Before you test it make sure you verified your phone number in your IG account.

The program works pretty well so far but in case of any problems I have to put disclaimer statement here:

Disclaimer: This post published educational purposes only as well as to give general information about Instagram API. I am not responsible for any actions and you are taking your own risk.

Let’s start by installing and then logging in with API.

pip install InstagramApi

from InstagramAPI import InstagramAPI

api = InstagramAPI("username", "password")
api.login()

Once you run the program you will see “Login success!” in your console.

Get users from liked list

We are going to search for some username (your target page) then get most recent post from this user. Then, get users who liked this post. Unfortunately, I can’t find solution how to paginate users so right now it gets about last 500 user.

users_list = []

def get_likes_list(username):
    api.login()
    api.searchUsername(username)
    result = api.LastJson
    username_id = result['user']['pk'] # Get user ID
    user_posts = api.getUserFeed(username_id) # Get user feed
    result = api.LastJson
    media_id = result['items'][0]['id'] # Get most recent post
    api.getMediaLikers(media_id) # Get users who liked
    users = api.LastJson['users']
    for user in users: # Push users to list
        users_list.append({'pk':user['pk'], 'username':user['username']})

Follow Users

Once we get the users list, it is time to follow these users.

IMPORTANT NOTE: set time limit as much as you can to avoid automation detection.

from time import sleep

following_users = []

def follow_users(users_list):
    api.login()
    api.getSelfUsersFollowing() # Get users which you are following
    result = api.LastJson
    for user in result['users']:
        following_users.append(user['pk'])
    for user in users_list:
        if not user['pk'] in following_users: # if new user is not in your following users                   
            print('Following @' + user['username'])
            api.follow(user['pk'])
            # after first test set this really long to avoid from suspension
            sleep(20)
        else:
            print('Already following @' + user['username'])
            sleep(10)

Unfollow Users

This function will look users which you are following then it will check if this user follows you as well. If user not following you then you are unfollowing as well.

follower_users = []

def unfollow_users():
    api.login()
    api.getSelfUserFollowers() # Get your followers
    result = api.LastJson
    for user in result['users']:
        follower_users.append({'pk':user['pk'], 'username':user['username']})

    api.getSelfUsersFollowing() # Get users which you are following
    result = api.LastJson
    for user in result['users']:
        following_users.append({'pk':user['pk'],'username':user['username']})
    for user in following_users:
        if not user['pk'] in follower_users: # if the user not follows you
            print('Unfollowing @' + user['username'])
            api.unfollow(user['pk'])
            # set this really long to avoid from suspension
            sleep(20) 

Full Code with extra functions

Here is the full code of this automation

import pprint
from time import sleep
from InstagramAPI import InstagramAPI
import pandas as pd

users_list = []
following_users = []
follower_users = []

class InstaBot:

    def __init__(self):
        self.api = InstagramAPI("your_username", "your_password")

    def get_likes_list(self,username):
        api = self.api
        api.login()
        api.searchUsername(username) #Gets most recent post from user
        result = api.LastJson
        username_id = result['user']['pk']
        user_posts = api.getUserFeed(username_id)
        result = api.LastJson
        media_id = result['items'][0]['id']

        api.getMediaLikers(media_id)
        users = api.LastJson['users']
        for user in users:
            users_list.append({'pk':user['pk'], 'username':user['username']})
        bot.follow_users(users_list)

    def follow_users(self,users_list):
        api = self.api
        api.login()
        api.getSelfUsersFollowing()
        result = api.LastJson
        for user in result['users']:
            following_users.append(user['pk'])
        for user in users_list:
            if not user['pk'] in following_users:
                print('Following @' + user['username'])
                api.follow(user['pk'])
                # set this really long to avoid from suspension
                sleep(20)
            else:
                print('Already following @' + user['username'])
                sleep(10)

     def unfollow_users(self):
        api = self.api
        api.login()
        api.getSelfUserFollowers()
        result = api.LastJson
        for user in result['users']:
            follower_users.append({'pk':user['pk'], 'username':user['username']})

        api.getSelfUsersFollowing()
        result = api.LastJson
        for user in result['users']:
            following_users.append({'pk':user['pk'],'username':user['username']})

        for user in following_users:
            if not user['pk'] in [user['pk'] for user in follower_users]:
                print('Unfollowing @' + user['username'])
                api.unfollow(user['pk'])
                # set this really long to avoid from suspension
                sleep(20) 

bot =  InstaBot()
# To follow users run the function below
# change the username ('instagram') to your target username
bot.get_likes_list('instagram')

# To unfollow users uncomment and run the function below
# bot.unfollow_users()

it will look like this:

some extra functions to play with API:

def get_my_profile_details():
    api.login() 
    api.getSelfUsernameInfo()
    result = api.LastJson
    username = result['user']['username']
    full_name = result['user']['full_name']
    profile_pic_url = result['user']['profile_pic_url']
    followers = result['user']['follower_count']
    following = result['user']['following_count']
    media_count = result['user']['media_count']
    df_profile = pd.DataFrame(
        {'username':username,
        'full name': full_name,
        'profile picture URL':profile_pic_url,
        'followers':followers,
        'following':following,
        'media count': media_count,
        }, index=[0])
    df_profile.to_csv('profile.csv', sep='\t', encoding='utf-8')

def get_my_feed():
    image_urls = []
    api.login()
    api.getSelfUserFeed()
    result = api.LastJson
    # formatted_json_str = pprint.pformat(result)
    # print(formatted_json_str)
    if 'items' in result.keys():
        for item in result['items'][0:5]:
            if 'image_versions2' in item.keys():
                image_url = item['image_versions2']['candidates'][1]['url']
                image_urls.append(image_url)

    df_feed = pd.DataFrame({
                'image URL':image_urls
            })
    df_feed.to_csv('feed.csv', sep='\t', encoding='utf-8')

---

Building an Instagram Bot with Python and Selenium to Gain More Followers

Let’s build an Instagram bot to gain more followers! — I know, I know. That doesn’t sound very ethical, does it? But it’s all justified for educational purposes.

Coding is a super power — we can all agree. That’s why I’ll leave it up to you to not abuse this power. And I trust you’re here to learn how it works. Otherwise, you’d be on GitHub cloning one of the countless Instagram bots there, right?

You’re convinced? — Alright, now let’s go back to unethical practices.

The Plan

So here’s the deal, we want to build a bot in Python and Selenium that goes on the hashtags we specify, likes random posts, then follows the posters. It does that enough — we get follow backs. Simple as that.

Here’s a pretty twisted detail though: we want to keep track of the users we follow so the bot can unfollow them after the number of days we specify.

Setup

So first things first, I want to use a database to keep track of the username and the date added. You might as well save/load from/to a file, but we want this to be ready for more features in case we felt inspired in the future.

So make sure you create a database (I named mine instabot — but you can name it anything you like) and create a table called followed_users within the database with two fields (username, date_added)

Remember the installation path. You’ll need it.

You’ll also need the following python packages:

• selenium
• mysql-connector

Getting down to it

Alright, so first thing we’ll be doing is creating settings.json. Simply a .json file that will hold all of our settings so we don’t have to dive into the code every time we want to change something.

Settings

settings.json:

{
  "db": {
    "host": "localhost",
    "user": "root",
    "pass": "",
    "database": "instabot"
  },
  "instagram": {
    "user": "",
    "pass": ""
  },
  "config": {
    "days_to_unfollow": 1,
    "likes_over": 150,
    "check_followers_every": 3600,
    "hashtags": []
  }
}

As you can see, under “db”, we specify the database information. As I mentioned, I used “instabot”, but feel free to use whatever name you want.

You’ll also need to fill Instagram info under “instagram” so the bot can login into your account.

“config” is for our bot’s settings. Here’s what the fields mean:

days_to_unfollow: number of days before unfollowing users

likes_over: ignore posts if the number of likes is above this number

check_followers_every: number of seconds before checking if it’s time to unfollow any of the users

hashtags: a list of strings with the hashtag names the bot should be active on

Constants

Now, we want to take these settings and have them inside our code as constants.

Create Constants.py:

import json
INST_USER= INST_PASS= USER= PASS= HOST= DATABASE= POST_COMMENTS= ''
LIKES_LIMIT= DAYS_TO_UNFOLLOW= CHECK_FOLLOWERS_EVERY= 0
HASHTAGS= []

def init():
    global INST_USER, INST_PASS, USER, PASS, HOST, DATABASE, LIKES_LIMIT, DAYS_TO_UNFOLLOW, CHECK_FOLLOWERS_EVERY, HASHTAGS
    # read file
    data = None
    with open('settings.json', 'r') as myfile:
        data = myfile.read()
    obj = json.loads(data)
    INST_USER = obj['instagram']['user']
    INST_PASS = obj['instagram']['pass']
    USER = obj['db']['user']
    HOST = obj['db']['host']
    PASS = obj['db']['pass']
    DATABASE = obj['db']['database']
    LIKES_LIMIT = obj['config']['likes_over']
    CHECK_FOLLOWERS_EVERY = obj['config']['check_followers_every']
    HASHTAGS = obj['config']['hashtags']
    DAYS_TO_UNFOLLOW = obj['config']['days_to_unfollow']

the init() function we created reads the data from settings.json and feeds them into the constants we declared.

Engine

Alright, time for some architecture. Our bot will mainly operate from a python script with an init and update methods. Create BotEngine.py:

import Constants


def init(webdriver):
    return


def update(webdriver):
    return

We’ll be back later to put the logic here, but for now, we need an entry point.

Entry Point

Create our entry point, InstaBot.py:

from selenium import webdriver
import BotEngine

chromedriver_path = 'YOUR CHROMEDRIVER PATH' 
webdriver = webdriver.Chrome(executable_path=chromedriver_path)

BotEngine.init(webdriver)
BotEngine.update(webdriver)

webdriver.close()

chromedriver_path = ‘YOUR CHROMEDRIVER PATH’ webdriver = webdriver.Chrome(executable_path=chromedriver_path)

BotEngine.init(webdriver)
BotEngine.update(webdriver)

webdriver.close()

Of course, you’ll need to swap “YOUR CHROMEDRIVER PATH” with your actual ChromeDriver path.

Time Helper

We need to create a helper script that will help us calculate elapsed days since a certain date (so we know if we should unfollow user)

Create TimeHelper.py:

import datetime


def days_since_date(n):
    diff = datetime.datetime.now().date() - n
    return diff.days

Database

Create DBHandler.py. It’ll contain a class that handles connecting to the Database for us.

import mysql.connector
import Constants
class DBHandler:
    def __init__(self):
        DBHandler.HOST = Constants.HOST
        DBHandler.USER = Constants.USER
        DBHandler.DBNAME = Constants.DATABASE
        DBHandler.PASSWORD = Constants.PASS
    HOST = Constants.HOST
    USER = Constants.USER
    DBNAME = Constants.DATABASE
    PASSWORD = Constants.PASS
    @staticmethod
    def get_mydb():
        if DBHandler.DBNAME == '':
            Constants.init()
        db = DBHandler()
        mydb = db.connect()
        return mydb

    def connect(self):
        mydb = mysql.connector.connect(
            host=DBHandler.HOST,
            user=DBHandler.USER,
            passwd=DBHandler.PASSWORD,
            database = DBHandler.DBNAME
        )
        return mydb

As you can see, we’re using the constants we defined.

The class contains a static method get_mydb() that returns a database connection we can use.

Now, let’s define a DB user script that contains the DB operations we need to perform on the user.

Create DBUsers.py:

import datetime, TimeHelper
from DBHandler import *
import Constants

#delete user by username
def delete_user(username):
    mydb = DBHandler.get_mydb()
    cursor = mydb.cursor()
    sql = "DELETE FROM followed_users WHERE username = '{0}'".format(username)
    cursor.execute(sql)
    mydb.commit()


#add new username
def add_user(username):
    mydb = DBHandler.get_mydb()
    cursor = mydb.cursor()
    now = datetime.datetime.now().date()
    cursor.execute("INSERT INTO followed_users(username, date_added) VALUES(%s,%s)",(username, now))
    mydb.commit()


#check if any user qualifies to be unfollowed
def check_unfollow_list():
    mydb = DBHandler.get_mydb()
    cursor = mydb.cursor()
    cursor.execute("SELECT * FROM followed_users")
    results = cursor.fetchall()
    users_to_unfollow = []
    for r in results:
        d = TimeHelper.days_since_date(r[1])
        if d > Constants.DAYS_TO_UNFOLLOW:
            users_to_unfollow.append(r[0])
    return users_to_unfollow


#get all followed users
def get_followed_users():
    users = []
    mydb = DBHandler.get_mydb()
    cursor = mydb.cursor()
    cursor.execute("SELECT * FROM followed_users")
    results = cursor.fetchall()
    for r in results:
        users.append(r[0])

    return users

Account Agent

Alright, we’re about to start our bot. We’re creating a script called AccountAgent.py that will contain the agent behavior.

Import some modules, some of which we need for later and write a login function that will make use of our webdriver.

Notice that we have to keep calling the sleep function between actions. If we send too many requests quickly, the Instagram servers will be alarmed and will deny any requests you send.

from time import sleep
import datetime
import DBUsers, Constants
import traceback
import random

def login(webdriver):
    #Open the instagram login page
    webdriver.get('https://www.instagram.com/accounts/login/?source=auth_switcher')
    #sleep for 3 seconds to prevent issues with the server
    sleep(3)
    #Find username and password fields and set their input using our constants
    username = webdriver.find_element_by_name('username')
    username.send_keys(Constants.INST_USER)
    password = webdriver.find_element_by_name('password')
    password.send_keys(Constants.INST_PASS)
    #Get the login button
    try:
        button_login = webdriver.find_element_by_xpath(
            '//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[4]/button')
    except:
        button_login = webdriver.find_element_by_xpath(
            '//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[6]/button/div')
    #sleep again
    sleep(2)
    #click login
    button_login.click()
    sleep(3)
    #In case you get a popup after logging in, press not now.
    #If not, then just return
    try:
        notnow = webdriver.find_element_by_css_selector(
            'body > div.RnEpo.Yx5HN > div > div > div.mt3GC > button.aOOlW.HoLwm')
        notnow.click()
    except:
        return

Also note how we’re getting elements with their xpath. To do so, right click on the element, click “Inspect”, then right click on the element again inside the inspector, and choose Copy->Copy XPath.

Another important thing to be aware of is that element hierarchy change with the page’s layout when you resize or stretch the window. That’s why we’re checking for two different xpaths for the login button.

Now go back to BotEngine.py, we’re ready to login.

Add more imports that we’ll need later and fill in the init function

import AccountAgent, DBUsers
import Constants
import datetime


def init(webdriver):
    Constants.init()
    AccountAgent.login(webdriver)


def update(webdriver):
    return

If you run our entry script now (InstaBot.py) you’ll see the bot logging in.

Perfect, now let’s add a method that will allow us to follow people to AccountAgent.py:

def follow_people(webdriver):
    #all the followed user
    prev_user_list = DBUsers.get_followed_users()
    #a list to store newly followed users
    new_followed = []
    #counters
    followed = 0
    likes = 0
    #Iterate theough all the hashtags from the constants
    for hashtag in Constants.HASHTAGS:
        #Visit the hashtag
        webdriver.get('https://www.instagram.com/explore/tags/' + hashtag+ '/')
        sleep(5)

        #Get the first post thumbnail and click on it
        first_thumbnail = webdriver.find_element_by_xpath(
            '//*[@id="react-root"]/section/main/article/div[1]/div/div/div[1]/div[1]/a/div')

        first_thumbnail.click()
        sleep(random.randint(1,3))

        try:
            #iterate over the first 200 posts in the hashtag
            for x in range(1,200):
                t_start = datetime.datetime.now()
                #Get the poster's username
                username = webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[1]/h2/a').text
                likes_over_limit = False
                try:
                    #get number of likes and compare it to the maximum number of likes to ignore post
                    likes = int(webdriver.find_element_by_xpath(
                        '/html/body/div[3]/div[2]/div/article/div[2]/section[2]/div/div/button/span').text)
                    if likes > Constants.LIKES_LIMIT:
                        print("likes over {0}".format(Constants.LIKES_LIMIT))
                        likes_over_limit = True


                    print("Detected: {0}".format(username))
                    #If username isn't stored in the database and the likes are in the acceptable range
                    if username not in prev_user_list and not likes_over_limit:
                        #Don't press the button if the text doesn't say follow
                        if webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[2]/button').text == 'Follow':
                            #Use DBUsers to add the new user to the database
                            DBUsers.add_user(username)
                            #Click follow
                            webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/header/div[2]/div[1]/div[2]/button').click()
                            followed += 1
                            print("Followed: {0}, #{1}".format(username, followed))
                            new_followed.append(username)


                        # Liking the picture
                        button_like = webdriver.find_element_by_xpath(
                            '/html/body/div[3]/div[2]/div/article/div[2]/section[1]/span[1]/button')

                        button_like.click()
                        likes += 1
                        print("Liked {0}'s post, #{1}".format(username, likes))
                        sleep(random.randint(5, 18))


                    # Next picture
                    webdriver.find_element_by_link_text('Next').click()
                    sleep(random.randint(20, 30))
                    
                except:
                    traceback.print_exc()
                    continue
                t_end = datetime.datetime.now()

                #calculate elapsed time
                t_elapsed = t_end - t_start
                print("This post took {0} seconds".format(t_elapsed.total_seconds()))


        except:
            traceback.print_exc()
            continue

        #add new list to old list
        for n in range(0, len(new_followed)):
            prev_user_list.append(new_followed[n])
        print('Liked {} photos.'.format(likes))
        print('Followed {} new people.'.format(followed))

It’s pretty long, but generally here’s the steps of the algorithm:

For every hashtag in the hashtag constant list:

• Visit the hashtag link
• Open the first thumbnail
• Now, execute the following code 200 times (first 200 posts in the hashtag)
• Get poster’s username, check if not already following, follow, like the post, then click next
• If already following just click next quickly

Now we might as well implement the unfollow method, hopefully the engine will be feeding us the usernames to unfollow in a list:

def unfollow_people(webdriver, people):
    #if only one user, append in a list
    if not isinstance(people, (list,)):
        p = people
        people = []
        people.append(p)

    for user in people:
        try:
            webdriver.get('https://www.instagram.com/' + user + '/')
            sleep(5)
            unfollow_xpath = '//*[@id="react-root"]/section/main/div/header/section/div[1]/div[1]/span/span[1]/button'

            unfollow_confirm_xpath = '/html/body/div[3]/div/div/div[3]/button[1]'

            if webdriver.find_element_by_xpath(unfollow_xpath).text == "Following":
                sleep(random.randint(4, 15))
                webdriver.find_element_by_xpath(unfollow_xpath).click()
                sleep(2)
                webdriver.find_element_by_xpath(unfollow_confirm_xpath).click()
                sleep(4)
            DBUsers.delete_user(user)

        except Exception:
            traceback.print_exc()
            continue

Now we can finally go back and finish the bot by implementing the rest of BotEngine.py:

import AccountAgent, DBUsers
import Constants
import datetime


def init(webdriver):
    Constants.init()
    AccountAgent.login(webdriver)


def update(webdriver):
    #Get start of time to calculate elapsed time later
    start = datetime.datetime.now()
    #Before the loop, check if should unfollow anyone
    _check_follow_list(webdriver)
    while True:
        #Start following operation
        AccountAgent.follow_people(webdriver)
        #Get the time at the end
        end = datetime.datetime.now()
        #How much time has passed?
        elapsed = end - start
        #If greater than our constant to check on
        #followers, check on followers
        if elapsed.total_seconds() >= Constants.CHECK_FOLLOWERS_EVERY:
            #reset the start variable to now
            start = datetime.datetime.now()
            #check on followers
            _check_follow_list(webdriver)


def _check_follow_list(webdriver):
    print("Checking for users to unfollow")
    #get the unfollow list
    users = DBUsers.check_unfollow_list()
    #if there's anyone in the list, start unfollowing operation
    if len(users) > 0:
        AccountAgent.unfollow_people(webdriver, users)

Conclusion

And that’s it — now you have yourself a fully functional Instagram bot built with Python and Selenium. There are many possibilities for you to explore now, so make sure you’re using this newly gained skill to solve real life problems!

You can get the source code for the whole project from this GitHub repository.

---

Building a simple Instagram bot with Python tutorial

Here we build a simple bot using some simple Python which beginner to intermediate coders can follow.

Here’s the code on GitHub
https://github.com/aj-4/ig-followers

---

Build A (Full-Featured) Instagram Bot With Python

Source Code: https://github.com/jg-fisher/instagram-bot 

---

How to Get Instagram Followers/Likes Using Python

In this video I show you how to program your own Instagram Bot using Python and Selenium.

https://www.youtube.com/watch?v=BGU2X5lrz9M 

Code Link:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
import random
import sys


def print_same_line(text):
    sys.stdout.write('\r')
    sys.stdout.flush()
    sys.stdout.write(text)
    sys.stdout.flush()


class InstagramBot:

    def __init__(self, username, password):
        self.username = username
        self.password = password
        self.driver = webdriver.Chrome()

    def closeBrowser(self):
        self.driver.close()

    def login(self):
        driver = self.driver
        driver.get("https://www.instagram.com/")
        time.sleep(2)
        login_button = driver.find_element_by_xpath("//a[@href='/accounts/login/?source=auth_switcher']")
        login_button.click()
        time.sleep(2)
        user_name_elem = driver.find_element_by_xpath("//input[@name='username']")
        user_name_elem.clear()
        user_name_elem.send_keys(self.username)
        passworword_elem = driver.find_element_by_xpath("//input[@name='password']")
        passworword_elem.clear()
        passworword_elem.send_keys(self.password)
        passworword_elem.send_keys(Keys.RETURN)
        time.sleep(2)


    def like_photo(self, hashtag):
        driver = self.driver
        driver.get("https://www.instagram.com/explore/tags/" + hashtag + "/")
        time.sleep(2)

        # gathering photos
        pic_hrefs = []
        for i in range(1, 7):
            try:
                driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
                time.sleep(2)
                # get tags
                hrefs_in_view = driver.find_elements_by_tag_name('a')
                # finding relevant hrefs
                hrefs_in_view = [elem.get_attribute('href') for elem in hrefs_in_view
                                 if '.com/p/' in elem.get_attribute('href')]
                # building list of unique photos
                [pic_hrefs.append(href) for href in hrefs_in_view if href not in pic_hrefs]
                # print("Check: pic href length " + str(len(pic_hrefs)))
            except Exception:
                continue

        # Liking photos
        unique_photos = len(pic_hrefs)
        for pic_href in pic_hrefs:
            driver.get(pic_href)
            time.sleep(2)
            driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
            try:
                time.sleep(random.randint(2, 4))
                like_button = lambda: driver.find_element_by_xpath('//span[@aria-label="Like"]').click()
                like_button().click()
                for second in reversed(range(0, random.randint(18, 28))):
                    print_same_line("#" + hashtag + ': unique photos left: ' + str(unique_photos)
                                    + " | Sleeping " + str(second))
                    time.sleep(1)
            except Exception as e:
                time.sleep(2)
            unique_photos -= 1

if __name__ == "__main__":

    username = "USERNAME"
    password = "PASSWORD"

    ig = InstagramBot(username, password)
    ig.login()

    hashtags = ['amazing', 'beautiful', 'adventure', 'photography', 'nofilter',
                'newyork', 'artsy', 'alumni', 'lion', 'best', 'fun', 'happy',
                'art', 'funny', 'me', 'followme', 'follow', 'cinematography', 'cinema',
                'love', 'instagood', 'instagood', 'followme', 'fashion', 'sun', 'scruffy',
                'street', 'canon', 'beauty', 'studio', 'pretty', 'vintage', 'fierce']

    while True:
        try:
            # Choose a random tag from the list of tags
            tag = random.choice(hashtags)
            ig.like_photo(tag)
        except Exception:
            ig.closeBrowser()
            time.sleep(60)
            ig = InstagramBot(username, password)
            ig.login()

---

Build An INSTAGRAM Bot With Python That Gets You Followers

---

Instagram Automation Using Python

---

How to Create an Instagram Bot | Get More Followers

---

Building a simple Instagram Influencer Bot with Python tutorial

#python #chatbot #web-development

How to Automate Excel with Python | Python Excel Tutorial (OpenPyXL)

How to Automate Excel with Python

In this article, We will show how we can use python to automate Excel . A useful Python library is Openpyxl which we will learn to do Excel Automation

What is OPENPYXL

Openpyxl is a Python library that is used to read from an Excel file or write to an Excel file. Data scientists use Openpyxl for data analysis, data copying, data mining, drawing charts, styling sheets, adding formulas, and more.

Workbook: A spreadsheet is represented as a workbook in openpyxl. A workbook consists of one or more sheets.

Sheet: A sheet is a single page composed of cells for organizing data.

Cell: The intersection of a row and a column is called a cell. Usually represented by A1, B5, etc.

Row: A row is a horizontal line represented by a number (1,2, etc.).

Column: A column is a vertical line represented by a capital letter (A, B, etc.).

Openpyxl can be installed using the pip command and it is recommended to install it in a virtual environment.

pip install openpyxl

CREATE A NEW WORKBOOK

We start by creating a new spreadsheet, which is called a workbook in Openpyxl. We import the workbook module from Openpyxl and use the function Workbook() which creates a new workbook.

from openpyxl
import Workbook
#creates a new workbook
wb = Workbook()
#Gets the first active worksheet
ws = wb.active
#creating new worksheets by using the create_sheet method

ws1 = wb.create_sheet("sheet1", 0) #inserts at first position
ws2 = wb.create_sheet("sheet2") #inserts at last position
ws3 = wb.create_sheet("sheet3", -1) #inserts at penultimate position

#Renaming the sheet
ws.title = "Example"

#save the workbook
wb.save(filename = "example.xlsx")

READING DATA FROM WORKBOOK

We load the file using the function load_Workbook() which takes the filename as an argument. The file must be saved in the same working directory.

#loading a workbook
wb = openpyxl.load_workbook("example.xlsx")

 

GETTING SHEETS FROM THE LOADED WORKBOOK

 

#getting sheet names
wb.sheetnames
result = ['sheet1', 'Sheet', 'sheet3', 'sheet2']

#getting a particular sheet
sheet1 = wb["sheet2"]

#getting sheet title
sheet1.title
result = 'sheet2'

#Getting the active sheet
sheetactive = wb.active
result = 'sheet1'

 

ACCESSING CELLS AND CELL VALUES

 

#get a cell from the sheet
sheet1["A1"] <
  Cell 'Sheet1'.A1 >

  #get the cell value
ws["A1"].value 'Segment'

#accessing cell using row and column and assigning a value
d = ws.cell(row = 4, column = 2, value = 10)
d.value
10

 

ITERATING THROUGH ROWS AND COLUMNS

 

#looping through each row and column
for x in range(1, 5):
  for y in range(1, 5):
  print(x, y, ws.cell(row = x, column = y)
    .value)

#getting the highest row number
ws.max_row
701

#getting the highest column number
ws.max_column
19

There are two functions for iterating through rows and columns.

Iter_rows() => returns the rows
Iter_cols() => returns the columns {
  min_row = 4, max_row = 5, min_col = 2, max_col = 5
} => This can be used to set the boundaries
for any iteration.

Example:

#iterating rows
for row in ws.iter_rows(min_row = 2, max_col = 3, max_row = 3):
  for cell in row:
  print(cell) <
  Cell 'Sheet1'.A2 >
  <
  Cell 'Sheet1'.B2 >
  <
  Cell 'Sheet1'.C2 >
  <
  Cell 'Sheet1'.A3 >
  <
  Cell 'Sheet1'.B3 >
  <
  Cell 'Sheet1'.C3 >

  #iterating columns
for col in ws.iter_cols(min_row = 2, max_col = 3, max_row = 3):
  for cell in col:
  print(cell) <
  Cell 'Sheet1'.A2 >
  <
  Cell 'Sheet1'.A3 >
  <
  Cell 'Sheet1'.B2 >
  <
  Cell 'Sheet1'.B3 >
  <
  Cell 'Sheet1'.C2 >
  <
  Cell 'Sheet1'.C3 >

To get all the rows of the worksheet we use the method worksheet.rows and to get all the columns of the worksheet we use the method worksheet.columns. Similarly, to iterate only through the values we use the method worksheet.values.

Example:

for row in ws.values:
  for value in row:
  print(value)

 

WRITING DATA TO AN EXCEL FILE

Writing to a workbook can be done in many ways such as adding a formula, adding charts, images, updating cell values, inserting rows and columns, etc… We will discuss each of these with an example.

 

CREATING AND SAVING A NEW WORKBOOK

 

#creates a new workbook
wb = openpyxl.Workbook()

#saving the workbook
wb.save("new.xlsx")

 

ADDING AND REMOVING SHEETS

 

#creating a new sheet
ws1 = wb.create_sheet(title = "sheet 2")

#creating a new sheet at index 0
ws2 = wb.create_sheet(index = 0, title = "sheet 0")

#checking the sheet names
wb.sheetnames['sheet 0', 'Sheet', 'sheet 2']

#deleting a sheet
del wb['sheet 0']

#checking sheetnames
wb.sheetnames['Sheet', 'sheet 2']

 

ADDING CELL VALUES

 

#checking the sheet value
ws['B2'].value
null

#adding value to cell
ws['B2'] = 367

#checking value
ws['B2'].value
367

 

ADDING FORMULAS

 

We often require formulas to be included in our Excel datasheet. We can easily add formulas using the Openpyxl module just like you add values to a cell.
 

For example:

import openpyxl
from openpyxl
import Workbook

wb = openpyxl.load_workbook("new1.xlsx")
ws = wb['Sheet']

ws['A9'] = '=SUM(A2:A8)'

wb.save("new2.xlsx")

The above program will add the formula (=SUM(A2:A8)) in cell A9. The result will be as below.

 

MERGE/UNMERGE CELLS

Two or more cells can be merged to a rectangular area using the method merge_cells(), and similarly, they can be unmerged using the method unmerge_cells().

For example:
Merge cells

#merge cells B2 to C9
ws.merge_cells('B2:C9')
ws['B2'] = "Merged cells"

Adding the above code to the previous example will merge cells as below.

UNMERGE CELLS

 

#unmerge cells B2 to C9
ws.unmerge_cells('B2:C9')

The above code will unmerge cells from B2 to C9.

INSERTING AN IMAGE

To insert an image we import the image function from the module openpyxl.drawing.image. We then load our image and add it to the cell as shown in the below example.

Example:

import openpyxl
from openpyxl
import Workbook
from openpyxl.drawing.image
import Image

wb = openpyxl.load_workbook("new1.xlsx")
ws = wb['Sheet']
#loading the image(should be in same folder)
img = Image('logo.png')
ws['A1'] = "Adding image"
#adjusting size
img.height = 130
img.width = 200
#adding img to cell A3

ws.add_image(img, 'A3')

wb.save("new2.xlsx")

Result:

CREATING CHARTS

Charts are essential to show a visualization of data. We can create charts from Excel data using the Openpyxl module chart. Different forms of charts such as line charts, bar charts, 3D line charts, etc., can be created. We need to create a reference that contains the data to be used for the chart, which is nothing but a selection of cells (rows and columns). I am using sample data to create a 3D bar chart in the below example:

Example

import openpyxl
from openpyxl
import Workbook
from openpyxl.chart
import BarChart3D, Reference, series

wb = openpyxl.load_workbook("example.xlsx")
ws = wb.active

values = Reference(ws, min_col = 3, min_row = 2, max_col = 3, max_row = 40)
chart = BarChart3D()
chart.add_data(values)
ws.add_chart(chart, "E3")
wb.save("MyChart.xlsx")

Result

---

How to Automate Excel with Python with Video Tutorial

Welcome to another video! In this video, We will cover how we can use python to automate Excel. I'll be going over everything from creating workbooks to accessing individual cells and stylizing cells. There is a ton of things that you can do with Excel but I'll just be covering the core/base things in OpenPyXl.

⭐️ Timestamps ⭐️
00:00 | Introduction
02:14 | Installing openpyxl
03:19 | Testing Installation
04:25 | Loading an Existing Workbook
06:46 | Accessing Worksheets
07:37 | Accessing Cell Values
08:58 | Saving Workbooks
09:52 | Creating, Listing and Changing Sheets
11:50 | Creating a New Workbook
12:39 | Adding/Appending Rows
14:26 | Accessing Multiple Cells
20:46 | Merging Cells
22:27 | Inserting and Deleting Rows
23:35 | Inserting and Deleting Columns
24:48 | Copying and Moving Cells
26:06 | Practical Example, Formulas & Cell Styling

📄 Resources 📄
OpenPyXL Docs: https://openpyxl.readthedocs.io/en/stable/ 
Code Written in This Tutorial: https://github.com/techwithtim/ExcelPythonTutorial 
Subscribe: https://www.youtube.com/c/TechWithTim/featured 

#python 

How to Create & Add Content - Images, Text To Modern SharePoint Pages

Description

Requirement is to create Modern pages with content, which includes images and text. 

The Content is in SharePoint List. The pages are created from a Page Template.

To get Text part from Page template, use below PowerShell,

#get page textpart instance id
$parts=Get-PnPPageComponent -Page <pagename.aspx>

Execute the below PowerShell to create pages with HTML content from SharePoint List.

$logFile = "Logs\LogFile.log"
Start - Transcript - Path $logFile - Append
#Variables
$libName = "Site Pages"
$siteURL = "https://tenant.sharepoint.com/"
$contentType = "Group and Division Page"
$listname = "Content"
$sectionCategoy = "Our organisation"
#End
Try {
    #Connect to PnP Online
    $connection = Connect - PnPOnline - Url $siteURL - UseWebLogin - ReturnConnection - WarningAction Ignore
    #Get items from Content list
    $items = Get - PnPListItem - List $listName - PageSize 100
    foreach($item in $items) {
        if ($null - ne $item["Title"] - and $null - ne $item["Content"]) {
            #Get Page webparts instance Id
            #$parts = Get - PnPPageComponent - Page PageTemplate.aspx
            # load the page template
            $template = Get - PnPClientSidePage - Identity "Templates/Division-page-template"
            #Get page name
            $fullFileName = $item["Title"].Replace(" ", "_") + ".aspx"
            #Create fileURL
            $fileURL = $siteURL + $libName + "/" + $fullFileName
            # save a new SharePoint Page based on the Page Template
            $template.Save($fullFileName)
            $page = Get - PnPPage - Identity $fullFileName
            $htmlToInject = $item["Content"]
            $htmlToInject = $htmlToInject.TrimStart('{"Html":"').TrimEnd('"}') - replace([regex]::Escape('\n')), '' - replace([regex]::Escape('<a href=\')),' < a href = ' -replace ([regex]:: Escape('\
                        ">')),'" > ' -replace ([regex]::Escape(' & bull; % 09 ')),'
                        ' -replace '
                        https:
                        /*','https://'
            #Set PnP Page Text

            Set-PnPPageTextPart -Page $page -InstanceId "9fab3ce6-0638-4008-a9b9-cf2b784245b5" -Text $htmlToInject


            #publish page
            Set-PnPPage -Identity $fullFileName -Title $item["Title"] -ContentType $contentType -Publish

            #get site pages library
            $sitepagelist= Get-PnPList -Identity 'Site Pages'
            #get page Id and page Item to update section category
            $pageItem=Get-PnPListItem -List $sitepagelist -Id $page.PageId
            Set-PnPListItem -Values @{"SectionCategory" = $sectionCategoy} -List $sitepagelist -Identity $pageItem

        }
        else
        {
            Write-Host "Title or Content has no value"
        }
    }
}
Catch {
    Write-Host "Error: $($_.Exception.Message)" -Foregroundcolor Red
}
Stop-Transcript

Original article source at: https://www.c-sharpcorner.com/

#sharepoint #image #text 

Twitter.jl: Julia Package to Access Twitter API

Twitter.jl

A Julia package for interacting with the Twitter API.

Twitter.jl is a Julia package to work with the Twitter API v1.1. Currently, only the REST API methods are supported; streaming API endpoints aren't implemented at this time.

All functions have required arguments for those parameters required by Twitter and an options keyword argument to provide a Dict{String, String} of optional parameters Twitter API documentation. Most function calls will return either a Dict or an Array <: TwitterType. Bad requests will return the response code from the API (403, 404, etc).

DataFrame methods are defined for functions returning composite types: Tweets, Places, Lists, and Users.

Authentication

Before one can make use of this package, you must create an application on the Twitter's Developer Platform.

Once your application is approved, you can access your dashboard/portal to grab your authentication credentials from the "Details" tab of the application.

Note that you will also want to ensure that your App has Read / Write OAuth access in order to post tweets. You can find out more about this on Stack Overflow.

Installation

To install this package, enter ] on the REPL to bring up Julia's package manager. Then add the package:

julia> ]
(v1.7) pkg> add Twitter

Tip: Press Ctrl+C to return to the julia> prompt.

Usage

To run Twitter.jl, enter the following command in your Julia REPL

julia> using Twitter

Then the a global variable has to be declared with the twitterauth function. This function holds the consumer_key(API Key), consumer_secret(API Key Secret), oauth_token(Access Token), and oauth_secret(Access Token Secret) respectively.

twitterauth("6nOtpXmf...", # API Key
            "sES5Zlj096S...", # API Key Secret
            "98689850-Hj...", # Access Token
            "UroqCVpWKIt...") # Access Token Secret

• Ensure you put your credentials in an env file to avoid pushing your secrets to the public 🙀.

Note: This package does not currently support OAuth authentication.

Code examples

See runtests.jl for example function calls.

using Twitter, Test
using JSON, OAuth

# set debugging
ENV["JULIA_DEBUG"]=Twitter

twitterauth(ENV["CONSUMER_KEY"], ENV["CONSUMER_SECRET"], ENV["ACCESS_TOKEN"], ENV["ACCESS_TOKEN_SECRET"])

#get_mentions_timeline
mentions_timeline_default = get_mentions_timeline()
tw = mentions_timeline_default[1]
tw_df = DataFrame(mentions_timeline_default)
@test 0 <= length(mentions_timeline_default) <= 20
@test typeof(mentions_timeline_default) == Vector{Tweets}
@test typeof(tw) == Tweets
@test size(tw_df)[2] == 30

#get_user_timeline
user_timeline_default = get_user_timeline(screen_name = "randyzwitch")
@test typeof(user_timeline_default) == Vector{Tweets}

#get_home_timeline
home_timeline_default = get_home_timeline()
@test typeof(home_timeline_default) == Vector{Tweets}

#get_single_tweet_id
get_tweet_by_id = get_single_tweet_id(id = "434685122671939584")
@test typeof(get_tweet_by_id) == Tweets

#get_search_tweets
duke_tweets = get_search_tweets(q = "#Duke", count = 200)
@test typeof(duke_tweets) <: Dict

#test sending/deleting direct messages
#commenting out because Twitter API changed. Come back to fix
# send_dm = post_direct_messages_send(text = "Testing from Julia, this might disappear later $(time())", screen_name = "randyzwitch")
# get_single_dm = get_direct_messages_show(id = send_dm.id)
# destroy = post_direct_messages_destroy(id = send_dm.id)
# @test typeof(send_dm) == Tweets
# @test typeof(get_single_dm) == Tweets
# @test typeof(destroy) == Tweets

#creating/destroying friendships
add_friend = post_friendships_create(screen_name = "kyrieirving")

unfollow = post_friendships_destroy(screen_name = "kyrieirving")
unfollow_df = DataFrame(unfollow)
@test typeof(add_friend) == Users
@test typeof(unfollow) == Users
@test size(unfollow_df)[2] == 40

# create a cursor for follower ids
follow_cursor_test = get_followers_ids(screen_name = "twitter", count = 10_000)
@test length(follow_cursor_test["ids"]) == 10_000

# create a cursor for friend ids - use barackobama because he follows a lot of accounts!
friend_cursor_test = get_friends_ids(screen_name = "BarackObama", count = 10_000)
@test length(friend_cursor_test["ids"]) == 10_000

# create a test for home timelines
home_t = get_home_timeline(count = 2)
@test length(home_t) > 1

# TEST of cursoring functionality on user timelines
user_t = get_user_timeline(screen_name = "stefanjwojcik", count = 400)
@test length(user_t) == 400
# get the minimum ID of the tweets returned (the earliest)
minid = minimum(x.id for x in user_t);

# now iterate until you hit that tweet: should return 399
# WARNING: current versions of julia cannot use keywords in macros? read here: https://github.com/JuliaLang/julia/pull/29261
# eventually replace since_id = minid
tweets_since = get_user_timeline(screen_name = "stefanjwojcik", count = 400, since_id = 1001808621053898752, include_rts=1)

@test length(tweets_since)>=399

# testing get_mentions_timeline
mentions = get_mentions_timeline(screen_name = "stefanjwojcik", count = 300) 
@test length(mentions) >= 50 #sometimes API doesn't return number requested (twitter API specifies count is the max returned, may be much lower)
@test Tweets<:typeof(mentions[1])

# testing retweets_of_me
my_rts = get_retweets_of_me(count = 300)
@test Tweets<:typeof(my_rts[1])

Want to contribute?

Contributions are welcome! Kindly refer to the contribution guidelines.

Linux:  

CodeCov:

Author: Randyzwitch
Source Code: https://github.com/randyzwitch/Twitter.jl 
License: View license

#julia #api #twitter 

UAParser.jl: Parse User-agent Strings into Components

UAParser

UAParser is a Julia port of ua-parser, which itself is a multi-language port of BrowserScope's user agent string parser. Per the README file of the main project:

"The crux of the original parser--the data collected by Steve Souders over the years--has been extracted into a separate YAML file so as to be reusable as is by implementations in other programming languages."

UAParser is a limited Julia implementation heavily influenced by the Python code from the ua-parser library.

New regexes have were retrieved from here on 2018-12-19.

UAParser API

The API for UAParser revolves around three functions: parsedevice, parseos and parseuseragent. Each function takes one argument, user_agent_string::AbstractString and returns a custom Julia type: DeviceResult, OSResult, or UAResult. The structure of each type is as follows:

  DeviceResult: family, brand, model

  UAResult: family, major, minor, patch

  OSResult: family, major, minor, patch, patch_minor

Code examples

  using UAParser

  #Example user-agent string
  user_agent_string = "Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B179 Safari/7534.48.3"

  #Get device from user-agent string
  parsedevice(user_agent_string) #> DeviceResult("iPhone", "Apple", "iPhone")

  #Get browser information from user-agent string
  parseuseragent(user_agent_string) #> UAResult("Mobile Safari","5","1",missing)

  #Get os information
  parseos(user_agent_string) #> OSResult("iOS","5","1",missing,missing)

You can index into the results of these functions like any other Julia composite type.

  #Get just browser information, no version information
  x1 = parseuseragent(user_agent_string)
  x1.family #> "Mobile Safari"

  #Get the os, no version information
  x2 = parseos(user_agent_string)
  x2.family #> "iOS"

A Note On Parser Accuracy

When this library was created, it became very obvious that it would be hard to replicate the Python parser code with 100% accuracy. The authors decided that a reasonably accurate implementation was more useful than spending the time to achieve 100% accuracy.

The tests in this library test against the accuracy of the parser. As of v0.6 of this package, here are the accuracy statistics against the files provided by the main ua-core project:

parse_device: 15144/16017 (94.6%)
parse_os: 1517/1528 (99.3%)
parse_ua: 204/205 (99.5%)

Of course, if someone would like to achieve 100% accuracy, PRs will absolutely be reviewed.

Download Details:

Author: JuliaWeb
Source Code: https://github.com/JuliaWeb/UAParser.jl 
License: Unknown, Unknown licenses found

#julia #dataframes