This article was originally published at:
Welcome to the second lesson of the awesome Python course for AI!
In the last lesson, we built the first database migrations and made the project setup.
Here is the URL:Python course with building a fintech investment AI – Lesson 1: Start the project
Today, we will build super exciting stuff and will focus on the financial database that we will use to train our AI later.
We will work with pandas, some financial plugins, and will do a bit of refactoring.
As well, I will teach you how to create CLI commands in Python.
I cannot wait to teach you all of those powerful skills, so let's start ASAP!
And if you prefer video, here is the youtube version:
In the last episode, we created DB migrations and database models with some types.
Today we will change them a bit to have the possibility of using prices with many numbers after the decimal.
To do that, we need just change types in variable "date" to "db.Date()”.
Next, we need to change all Integers to "db.Numeric()”.
Take a look at the example below.
id = db.Column(db.Integer, primary_key=True) company = db.Column(db.String()) date = db.Column(db.Date()) openPrice = db.Column(db.Numeric()) highPrice = db.Column(db.Numeric()) lowPrice = db.Column(db.Numeric()) closePrice = db.Column(db.Numeric()) volume = db.Column(db.Numeric())
Now, we should save our changes and update the database with the new changes.
I would suggest initing DB from scratch.
As the first step, you need to delete the "migrations" directory (if you created it in the previous lesson).
Next, open terminal and type:
flask db init
flask db migrate
flask db migrate
We can focus on the next features now, and go into the prices module.
Let's start by creating the directory named "prices".
Create that directory in the root of the folder.
Next, go into the created directory and create the file named "prices.py".
Before we go into the proper development, we should remember about installing all of the necessary dependencies that we will use for building AI.
First, we need to create yfinance, that will help us to connect with the financial API, and download all necessary data about the prices.
Open the terminal and type:
pip install yfinance
Next, we need to install pandas:
pip install yfinance
The last one that we need to install is pandas datareader that we will use to handling our financial data:
pip install pandas_datareader
So, we have prepared all the necessary stuff that we needed for development.
Now, we can go into the prices.py file, and import all of these dependencies.
Let's take a look at the example below:
from datetime import date, timedelta import yfinance as yf import pandas as pd from pandas_datareader import data as pdr from app import app, db from models.prices import PriceModel
Next, we need to override yfinance with pandas.
That line of code should be below imports inside the prices.py file.
We could do that by typing/commenting on the functions as we had done in golang.
But it'd be a shame if we weren't using the user-friendly features of the flask.
We can define CLI commands very, very easily.
It's enough to use the "app" decorator and define the command that should fire the logic that is below the command definition.
Let's take a look:
Below the CLI command, we need to specify a function that we will want to fire.
In this case, we need a function named "getPrices".
We will use that to start getting prices.
@app.cli.command("get-prices") def getPrices():
In the next step, we should specify company symbols that we would like to get pricing for.
We should create a list and put there all the NASDAQ symbols as strings.
I've used the heaviest 50 of the NASDAQ, but you can specify whatever companies that you would like to get data for.
stockList = ['MSFT', 'AAPL', 'AMZN', 'FB', 'GOOGL', 'GOOG', 'GOOGL', 'INTC', 'NVDA', 'CSCO', 'ADBE', 'NFLX', 'PEP', 'PYP', 'CMCSA', 'TSLA', 'COST', 'GOOGL', 'AMGN', 'TMUS', 'AVGO', 'CHTR', 'TXN', 'GILD', 'QCOM', 'SBUX', 'INTU', 'MDLZ', 'VRTX', 'FISV', 'BKNG', 'ISRG', 'REGN', 'ADP', 'AMD', 'ATVI', 'CSX', 'BIIB', 'ILMN', 'MU', 'AMAT', 'JD', 'ADSK', 'MELI', 'ADI', 'LRCX', 'MNST', 'WBA', 'EXC', 'KHC', 'LULU', 'EA']
In this step, we should take all of the NASDAQ symbols and put them inside the for loop that will iterate through them.
For every iteration of the stock, we should call the function named "getPrice" and pass "stock" as an argument.
for stock in stockList: getPrice(stock)
To have the possibility of using it, we need to create the function "getPrice".
Setup "stock" as a param name in the function.
Now, we can start focusing on the core functionality that we are interested in today, fetching data.
As a first step, we need to specify the date range that we are interested in.
I will set up today as the last day, and day that was 31 days ago, that will be my time-range for the stock prices.
today = date.today() monthAgo = today - timedelta(days=31)
In the next step, we can start getting data!
Look at the example below. We use pandas_datareader.
Next, I've set up stock, that I want to get, the start date, and the end date that is today.
data = pdr.get_data_yahoo(stock, start=monthAgo, end=today)
To have a better life and easier manipulation of data, we need to reformat that a bit.
So we will be able to iterate through that data much more comfortable.
df = pd.DataFrame(data)
Now is the moment where we can iterate through all of the prices for the xx company.
We use "df.iterrows()" method here because all of the rows are from pandas.
Next, we call "addPrice", and pass necessary data as arguments.
We pass the whole "row" that we will split later, next we pass index (it's data here), but formatted.
And we pass "stock" to know what stock is that price for.
for index, row in df.iterrows(): addPrice(row, index.strftime("%Y-%m-%d"), stock)
To make using the "addPrice" possible, we need to define that function.
Use "price", "date", and "stock", as params.
def addPrice(price, date, stock):
We are close to the final.
In this step, we should set up the data that we've got from financial API.
We need to fit that to our PriceModel, that we've created in the Python course Lesson1.
new_price = PriceModel(date=date, company=stock, openPrice=price['Open'], highPrice=price['High'], lowPrice=price['Low'], closePrice=price['Close'], volume=price[‚Volume'])
If our data is ready, we have nothing else to do than just create a new record inside the database.
Don’t forget about „db.session.commit()”, that will update our database.
The prices module is ready now. You can import it inside the app.py.
Add this line of code below the "PriceModel" import.
from prices.prices import getPrices
You can test your application now.
Before you start it, make sure the flask app variable is configured properly (we did it in Lesson 1).
If yes, just open a terminal in the project and type:
Now your project has hundreds of financial records.
Those are ready to start working with Artificial Intelligence, which we will build.
What is more important, you've learned where to look for the stock prices, how to select them, and how to save them into the database.
Code repository for the Python course Lesson 2 is here:
In the next lesson, we will focus on building the first AI logic for our algorithm.
Keep learning, and I'm super excited I have a chance to teach you how to build powerful investment AI that can predict stock prices!
Thanks for reading,
Radek from Duomly
#python #web-development #machine-learning #database #sql
Welcome to my Blog , In this article, you are going to learn the top 10 python tips and tricks.
#python #python hacks tricks #python learning tips #python programming tricks #python tips #python tips and tricks #python tips and tricks advanced #python tips and tricks for beginners #python tips tricks and techniques #python tutorial #tips and tricks in python #tips to learn python #top 30 python tips and tricks for beginners
When you get introduced to machine learning, the first step is to learn Python and the basic step of learning Python is to learn pandas library. We can install pandas library by pip install pandas. After installing we have to import pandas each time of the running session. The data used for example is from the UCI repository “https://archive.ics.uci.edu/ml/datasets/Heart+failure+clinical+records ”
2. Head and Tail
3. Shape, Size and Info
#pandas: most used functions in data science #pandas #data science #function #used python data #most used functions in data science
If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.
If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.
In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.
#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition
In this Data Science With Python Training video, you will learn everything about data science and python from basic to advance level. This python data science course video will help you learn various python concepts, AI, and lots of projects, hands-on demo, and lastly top trending data science and python interview questions. This is a must-watch video for everyone who wishes o learn data science and python to make a career in it.
#data science with python #python data science course #python data science #data science with python
At the end of 2019, Python is one of the fastest-growing programming languages. More than 10% of developers have opted for Python development.
In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.
Table of Contents hide
The Size and declared value and its sequence of the object can able to be modified called mutable objects.
Mutable Data Types are list, dict, set, byte array
The Size and declared value and its sequence of the object can able to be modified.
Immutable data types are int, float, complex, String, tuples, bytes, and frozen sets.
id() and type() is used to know the Identity and data type of the object
a**=str(“Hello python world”)****#str**
Numbers are stored in numeric Types. when a number is assigned to a variable, Python creates Number objects.
Python supports 3 types of numeric data.
int (signed integers like 20, 2, 225, etc.)
float (float is used to store floating-point numbers like 9.8, 3.1444, 89.52, etc.)
complex (complex numbers like 8.94j, 4.0 + 7.3j, etc.)
A complex number contains an ordered pair, i.e., a + ib where a and b denote the real and imaginary parts respectively).
The string can be represented as the sequence of characters in the quotation marks. In python, to define strings we can use single, double, or triple quotes.
# String Handling
#single (') Quoted String
# Double (") Quoted String
# triple (‘’') (“”") Quoted String
In python, string handling is a straightforward task, and python provides various built-in functions and operators for representing strings.
The operator “+” is used to concatenate strings and “*” is used to repeat the string.
'Output : Python python ’
#python web development #data types in python #list of all python data types #python data types #python datatypes #python types #python variable type