Adam Rose

Adam Rose

1559536054

How to write SQL queries in PostgreSQL

Being able to query the relational database systems is a must-have skill for a data scientist. SQL or Structured Query Language lets you do this in a very efficient way. SQL not only enables you to you ask meaningful questions to the data but also allows you to you play with the data in many different ways. Without databases, practically no real-world application is possible. So, the knowledge of databases and being able to handle them are crucial parts of a data scientist’s toolbox.

Quick fact: SQL is also called SE-QU-EL. It has got some historical significance - the initial name of SQL was Simple English Query Language.

Generally, relational databases look like the following -

How to write SQL queries in PostgreSQL

Relations are also called tables. There are a number of ways in which databases can be represented. This is just one of them and the most popular one.

This tutorial introduces the four most common operations performed with SQL, and they are Create, Read, Update and Delete. Collectively these four operations are often referred to as CRUD. In any application that involves user interaction when these four operations are always there.

You will be using PostgreSQL as the relational database management system. PostgreSQL is very light-weight, and it is free as well. In this tutorial, you will

  • Get up and running with PostgreSQL
  • Connect to a PostgreSQL database
  • Create, read, update and delete tables in that database
  • Run SQL on Jupyter Notebook
  • Run SQL in Python

Let’s get started.

Getting up and running with PostgreSQL

PostgreSQL is a light-weight and an open source RDBMS. It is extremely well accepted by the industry. You can learn more about PostgreSQL from its official website.

To be able to start writing and executing queries in PostgreSQL, you will need it installed on your machine. Installing it is extremely easy. The following two short videos show you how PostgreSQL can be downloaded and installed on a 32-bit Windows-7 machine

Note: While you are installing PostgreSQL take note of the password and port number that you are entering.

Once you have installed PostgreSQL successfully on your machine, open up pgAdmin. pgAdmin is a handy utility which comes with the PostgreSQL installation, and it lets you do regular database related tasks through a nice graphical interface. pgAdmin’s interface looks like

How to write SQL queries in PostgreSQL

When you open up pgAdmin, you will see a server named “PostgreSQL 9.4 (localhost:5432)” enlisted in the interface

How to write SQL queries in PostgreSQL

Note: Your version may be different than the above and so the port number (5432).

Connect to the server by entering the password that you gave during the installation. For reference - https://bit.ly/2FPO4hR.

Once you have successfully connected to the local database server, you will get an interface similar to the following

How to write SQL queries in PostgreSQL

CRUD operations in PostgreSQL

Creating a table according to a given specification -

To be able to operate on a database you will need a table. So let’s go ahead and create a simple table (also called relation) called datacamp_courses with the following specification (schema)

How to write SQL queries in PostgreSQL

The specification gives us quite a few information on the columns of the table

  • The primary key of the table should be course_id (note that only this one is bold) and its data-type should be an integer. A primary key is a constraint which enforces the column values to be non-null and unique. It lets you uniquely identify a specific or a set of instanced present in the table.
  • Rest of the information in the specification should be easy to interpret now.

To create a table, right-click on the newly created database DataCamp_Courses and select CREATE Script from the options. You should get something similar to the following

How to write SQL queries in PostgreSQL

Let’s execute the following query now

CREATE TABLE datacamp_courses(
 course_id SERIAL PRIMARY KEY,
 course_name VARCHAR (50) UNIQUE NOT NULL,
 course_instructor VARCHAR (100) NOT NULL,
 topic VARCHAR (20) NOT NULL
);

For executing the query just select it and click the execute button from the menu bar

How to write SQL queries in PostgreSQL

The output should be

How to write SQL queries in PostgreSQL

The general structure of a table creation query in PostgreSQL looks like

CREATE TABLE table_name (
 column_name TYPE column_constraint,
 table_constraint table_constraint
)

We did not specify any table_constraints while creating the table. That can be avoided for now. Everything else is quite readable except for the keyword SERIAL. Serial in PostgreSQL lets you create an auto-increment column. By default, it creates values of type integer. Serial frees us from the burden of remembering the last inserted/updated primary key of a table, and it is a good practice to use auto-increments for primary keys. You can learn more about serial from here.

Inserting some records to the newly created table

In this step, you will insert some records to the table. Your records should contain

  • A course name
  • Instructor’s name of the course
  • Course topic

The values for the column course_id will be handled by PostgreSQL itself. The general structure of an insert query in PostgreSQL looks like

INSERT INTO table(column1, column2, …)
VALUES
 (value1, value2, …);

Let’s insert some records

INSERT INTO datacamp_courses(course_name, course_instructor, topic)
VALUES('Deep Learning in Python','Dan Becker','Python');

INSERT INTO datacamp_courses(course_name, course_instructor, topic)
VALUES('Joining Data in PostgreSQL','Chester Ismay','SQL');

Note that you did not specify the primary keys explicitly. You will see its effects in a moment.

When you execute the above two queries, you should get the following result upon successful insertions

Query returned successfully: one row affected, 11 ms execution time.

Reading/viewing the data from the table -

This is probably something you will do a lot in your data science journey. For now, let’s see how is the table datacamp_courses holding up.

This is generally called a select query, and the generic structure of a select query looks like

SELECT
 column_1,
 column_2,
 ...
FROM
 table_name;

Let’s select all the columns from the table datacamp_courses

SELECT * FROM datacamp_courses;

And you get

How to write SQL queries in PostgreSQL

Note the primary keys now. If you want to just see the names of the courses you can do so by

SELECT course_name from datacamp_courses;

And you get

How to write SQL queries in PostgreSQL

You can specify as many column names as possible which you may want to see in your results provided they exist in the table. If you run select course_name, number_particpants from datacamp_courses; you will run into error as the column number_particpants does exist in the table. You will now see how you can update a specific record in the table.

Updating a record in the table

The general structure of an update query in SQL looks like the following:

UPDATE table
SET column1 = value1,
    column2 = value2 ,...
WHERE
 condition;

You are going to update the record where course_instructor = “Chester Ismay” and set the course_name to “Joining Data in SQL”. You will then verify if the record is updated. The query for doing this would be

UPDATE datacamp_courses SET course_name = 'Joining Data in SQL'
WHERE course_instructor = 'Chester Ismay';

Let’s see if your update query had the intended effect by running a select query

How to write SQL queries in PostgreSQL

You can see your update query performed exactly in the way you wanted. You will now see how you can delete a record from the table.

Deleting a record in the table

The general structure of a delete query in SQL looks like following:

DELETE FROM table
WHERE condition;

You are going to delete the record where course_name = “Deep Learning in Python” and then verify if the record is deleted. Following the structure, you can see that the following query should be able to do this

DELETE from datacamp_courses
WHERE course_name = 'Deep Learning in Python';

Keep in mind that the keywords are not case-sensitive in SQL, but the data is case-sensitive. This is why you see a mixture of upper case and lower case in the queries.

Let’s see if the intended record was deleted from the table or not

How to write SQL queries in PostgreSQL

And yes, it indeed deleted the intended record.

The generic structures of the queries as mentioned in the tutorial are referred from postgresqltutorial.com.

You now know how to basic CRUD queries in SQL. Some of you may use Jupyter Notebooks heavily and may be thinking it would be great if there were an option to execute these queries directly from Jupyter Notebook. In the next section, you will see how you can achieve this.

SQL + Jupyter Notebooks

To be able to run SQL queries from Jupyter Notebooks the first step will be to install the ipython-sql package.

If it is not installed, install it using:

pip install ipython-sql

Once this is done, load the sql extension in your Jupyter Notebook by executing

%load_ext sql

The next step will be to connect to a PostgreSQL database. You will connect to the database that you created -DataCamp_Courses.

For being able to connect to a database that is already created in your system, you will have to instruct Python to detect its dialect. In simpler terms, you will have to tell Python that it is a PostgreSQL database. For that, you will need psycopg2 which can be installed using:

pip install psycopg2

Once you installed psycopg connect to the database using

%sql postgresql://postgres:postgres@localhost:5432/DataCamp_Courses
'Connected: postgres@DataCamp_Courses'

Note the usage of %sql. This is a magic command. It lets you execute SQL statements from Jupyter Notebook. What follows %sql is called a database connection URL where you specify

  • Dialect (postgres)
  • Username (postgres)
  • Password (postgres)
  • Server address (localhost)
  • Port number (5432)
  • Database name (DaaCamp_Courses)

You can now perform everything from you Jupyter Notebook that you performed in the pgAdmin interface. Let’s start by creating the table datacamp_courses with the exact same schema.

But before doing that you will have to drop the table as SQL won’t let you store two tables with the same name. You can drop a table by

%sql DROP table datacamp_courses;
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
Done.





[]

The table datacamp_courses is now deleted from PostgreSQL and hence you can create a new table with this name.

%%sql
CREATE TABLE datacamp_courses(
 course_id SERIAL PRIMARY KEY,
 course_name VARCHAR (50) UNIQUE NOT NULL,
 course_instructor VARCHAR (100) NOT NULL,
 topic VARCHAR (20) NOT NULL
);
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
Done.





[]

Note the usage of %sql %%sql here. For executing a single line of query, you can use %sql, but if you want to execute multiple queries in one go, you will have to use %%sql.

Let’s insert some records

%%sql
INSERT INTO datacamp_courses(course_name, course_instructor, topic)
VALUES('Deep Learning in Python','Dan Becker','Python');
INSERT INTO datacamp_courses(course_name, course_instructor, topic)
VALUES('Joining Data in PostgreSQL','Chester Ismay','SQL');
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
1 rows affected.
1 rows affected.





[]

View the table to make sure the insertions were done as expected

%%sql
select * from datacamp_courses;
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
2 rows affected.

How to write SQL queries in PostgreSQL

Let’s maintain the flow. As the next step, you will update a record in the table

%sql update datacamp_courses set course_name = 'Joining Data in SQL' where course_instructor = 'Chester Ismay';
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
1 rows affected.





[]

Pay close attention when you are dealing with strings in SQL. Unlike traditional programming languages, the strings values need to be wrapped using single quotes.

Let’s now verify if your update query had the intended effect

%%sql
select * from datacamp_courses;
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
2 rows affected.

How to write SQL queries in PostgreSQL

Let’s now delete a record and verify

%%sql
delete from datacamp_courses where course_name = 'Deep Learning in Python';
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
1 rows affected.





[]
%%sql
select * from datacamp_courses;
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
1 rows affected.

How to write SQL queries in PostgreSQL

By now you have a clear idea of executing CRUD operations in PostgreSQL and how you can perform them via Jupyter Notebook. If you are familiar with Python and if you interested in accessing your database through your Python code you can also do it. The next section is all about that.

Getting started with SQLAlchemy and combining it with SQL magic commands

For this section, you will need the SQLAlchemy package. It comes with the Anaconda distribution generally. You can also pip-install it. Once you have it installed, you can import it by -

import sqlalchemy

To able to interact with your databases using SQLAlchemy you will need to create an engine for the respective RDBMS where your databases are stored. In your case, it is PostgreSQL. SQLAlchemy lets you create an engine of the RDBMS in just a single call of create_engine(), and the method takes a database connection URL which you have seen before.

from sqlalchemy import create_engine
engine = create_engine('postgresql://postgres:postgres@localhost:5432/DataCamp_Courses')
print(engine.table_names()) # Lets you see the names of the tables present in the database
['datacamp_courses']

You can see the table named datacamp_courses which further confirms that you were successful in creating the engine. Let’s execute a simple select query to see the records of the table datacamp_courses and store it in a pandas DataFrame object.

You will use the read_sql() method (provided by pandas) which takes a SQL query string and an engine.

import pandas as pd

df = pd.read_sql('select * from datacamp_courses', engine)
df.head()

How to write SQL queries in PostgreSQL

You can also pair up the %sql magic command within your Python code.

df_new = %sql select * from datacamp_courses
df_new.DataFrame().head()
 * postgresql://postgres:***@localhost:5432/DataCamp_Courses
1 rows affected.

How to write SQL queries in PostgreSQL

#sql #postgresql #database

What is GEEK

Buddha Community

How to write SQL queries in PostgreSQL
Cayla  Erdman

Cayla Erdman

1594369800

Introduction to Structured Query Language SQL pdf

SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.

Models for SQL exist. In any case, the SQL that can be utilized on every last one of the major RDBMS today is in various flavors. This is because of two reasons:

1. The SQL order standard is genuinely intricate, and it isn’t handy to actualize the whole standard.

2. Every database seller needs an approach to separate its item from others.

Right now, contrasts are noted where fitting.

#programming books #beginning sql pdf #commands sql #download free sql full book pdf #introduction to sql pdf #introduction to sql ppt #introduction to sql #practical sql pdf #sql commands pdf with examples free download #sql commands #sql free bool download #sql guide #sql language #sql pdf #sql ppt #sql programming language #sql tutorial for beginners #sql tutorial pdf #sql #structured query language pdf #structured query language ppt #structured query language

Introduction to Recursive CTE

This article will introduce the concept of SQL recursive. Recursive CTE is a really cool. We will see that it can often simplify our code, and avoid a cascade of SQL queries!

Why use a recursive CTE ?

The recursive queries are used to query hierarchical data. It avoids a cascade of SQL queries, you can only do one query to retrieve the hierarchical data.

What is recursive CTE ?

First, what is a CTE? A CTE (Common Table Expression) is a temporary named result set that you can reference within a SELECT, INSERT, UPDATE, or DELETE statement. For example, you can use CTE when, in a query, you will use the same subquery more than once.

A recursive CTE is one having a subquery that refers to its own name!

Recursive CTE is defined in the SQL standard.

How to make a recursive CTE?

A recursive CTE has this structure:

  • The WITH clause must begin with “WITH RECURSIVE”
  • The recursive CTE subquery has two parts, separated by “UNION [ALL]” or “UNION DISTINCT”:
  • The first part produces the initial row(s) for the CTE. This SELECT does not refer to the CTE name.
  • The second part recurses by referring to the CTE name in its FROM clause.

Practice / Example

In this example, we use hierarchical data. Each row can have zero or one parent. And it parent can also have a parent etc.

Create table test (id integer, parent_id integer);

insert into test (id, parent_id) values (1, null);

insert into test (id, parent_id) values (11, 1);
insert into test (id, parent_id) values (111, 11);

insert into test (id, parent_id) values (112, 11);

insert into test (id, parent_id) values (12, 1);

insert into test (id, parent_id) values (121, 12);

For example, the row with id 111 has as ancestors: 11 and 1.

Before knowing the recursive CTE, I was doing several queries to get all the ancestors of a row.

For example, to retrieve all the ancestors of the row with id 111.

While (has parent)

	Select id, parent_id from test where id = X

With recursive CTE, we can retrieve all ancestors of a row with only one SQL query :)

WITH RECURSIVE cte_test AS (
	SELECT id, parent_id FROM test WHERE id = 111
	UNION 
	SELECT test.id, test.parent_id FROM test JOIN cte_test ON cte_test.id = test.parent_id

) SELECT * FROM cte_test

Explanations:

  • “WITH RECURSIVE”:

It indicates we will make recursive

  • “SELECT id, parent_id FROM test WHERE id = 111”:

It is the initial query.

  • “UNION … JOIN cte_test” :

It is the recursive expression! We make a jointure with the current CTE!

Replay this example here

#sql #database #sql-server #sql-injection #writing-sql-queries #sql-beginner-tips #better-sql-querying-tips #sql-top-story

Cayla  Erdman

Cayla Erdman

1596441660

Welcome Back the T-SQL Debugger with SQL Complete – SQL Debugger

When you develop large chunks of T-SQL code with the help of the SQL Server Management Studio tool, it is essential to test the “Live” behavior of your code by making sure that each small piece of code works fine and being able to allocate any error message that may cause a failure within that code.

The easiest way to perform that would be to use the T-SQL debugger feature, which used to be built-in over the SQL Server Management Studio tool. But since the T-SQL debugger feature was removed completely from SQL Server Management Studio 18 and later editions, we need a replacement for that feature. This is because we cannot keep using the old versions of SSMS just to support the T-SQL Debugger feature without “enjoying” the new features and bug fixes that are released in the new SSMS versions.

If you plan to wait for SSMS to bring back the T-SQL Debugger feature, vote in the Put Debugger back into SSMS 18 to ask Microsoft to reintroduce it.

As for me, I searched for an alternative tool for a T-SQL Debugger SSMS built-in feature and found that Devart company rolled out a new T-SQL Debugger feature to version 6.4 of SQL – Complete tool. SQL Complete is an add-in for Visual Studio and SSMS that offers scripts autocompletion capabilities, which help develop and debug your SQL database project.

The SQL Debugger feature of SQL Complete allows you to check the execution of your scripts, procedures, functions, and triggers step by step by adding breakpoints to the lines where you plan to start, suspend, evaluate, step through, and then to continue the execution of your script.

You can download SQL Complete from the dbForge Download page and install it on your machine using a straight-forward installation wizard. The wizard will ask you to specify the installation path for the SQL Complete tool and the versions of SSMS and Visual Studio that you plan to install the SQL Complete on, as an add-in, from the versions that are installed on your machine, as shown below:

Once SQL Complete is fully installed on your machine, the dbForge SQL Complete installation wizard will notify you of whether the installation was completed successfully or the wizard faced any specific issue that you can troubleshoot and fix easily. If there are no issues, the wizard will provide you with an option to open the SSMS tool and start using the SQL Complete tool, as displayed below:

When you open SSMS, you will see a new “Debug” tools menu, under which you can navigate the SQL Debugger feature options. Besides, you will see a list of icons that will be used to control the debug mode of the T-SQL query at the leftmost side of the SSMS tool. If you cannot see the list, you can go to View -> Toolbars -> Debugger to make these icons visible.

During the debugging session, the SQL Debugger icons will be as follows:

The functionality of these icons within the SQL Debugger can be summarized as:

  • Adding Breakpoints to control the execution pause of the T-SQL script at a specific statement allows you to check the debugging information of the T-SQL statements such as the values for the parameters and the variables.
  • Step Into is “navigate” through the script statements one by one, allowing you to check how each statement behaves.
  • Step Over is “execute” a specific stored procedure if you are sure that it contains no error.
  • Step Out is “return” from the stored procedure, function, or trigger to the main debugging window.
  • Continue executing the script until reaching the next breakpoint.
  • Stop Debugging is “terminate” the debugging session.
  • Restart “stop and start” the current debugging session.

#sql server #sql #sql debugger #sql server #sql server stored procedure #ssms #t-sql queries

Chesley  Wehner

Chesley Wehner

1618936320

How to Write Simple SQL Queries from a Blank Database

This article explains writing simple SQL queries from the most basic ones and gradually improving the script to solve some mathematical and date-related problems. Additionally, we are going to clarify the concepts surrounding SQL queries.

Although this article is primarily for beginners, it contains hints that will be helpful for any experience level.

#sql server #sql query #t-sql queries #sql

3 Tips to Write SQL Queries That Are Easier to Read and Modify

There is no doubt that writing code is more art than science and every coder cannot write beautiful code which is both readable and maintainable, even with the experience. Yes, it’s blunt and hard but it’s mostly true.

In general, coding improves with experience but only when you learn the art of coding like favoring composition over inheritance or coding for interface than implementation, but, unfortunately only a few developers able to master these techniques.

The same applies to SQL queries. The way you structure your query, the way you write it goes a long way to communicate your intent to the fellow developer, DBA, and even yourself after a few months.

Whenever I see SQL queries on emails from different developers, I can see the stark difference in their writing style. Some developers and DBAs write it so neatly and indent their query such that you can easily spot key details like which columns you are extracting, and from which table, and what are joining or filtering conditions.

Since in real-life projects, SQL queries are hardly one-liner, learning the right way to write complex SQL queries makes a lot of difference; especially when you share that query to someone for review or execution. It also helps when you read it yourself later as I said, after a few months.

The problem is there are many books and courses to teach you SQL like what is a table, different SQL commands but there are very few (like The Complete SQL Bootcamp by Josh Portilla) which focus on writing proper SQL queries.

In this article, I am going to show you a couple of styles which I have tried in the past, their pros and cons and what I think is the best way to write SQL queries.

Unless you have a good reason not to use my style e.g. you have a better style or you want to stick with the style used in your project (consistency overrules everything) there is no reason not to use it.

By the way, I expect that you are familiar with SQL and definitely know how to write queries. I expect that you have used different SQL clauses like SELECT, INSERT, UPDATE, DELETE and understand their meaning in a SQL query. If you are not, it’s better you gain some experience with SQL by joining some of my recommended courses like:

Learn SQL by CodeCademy
Introduction to SQL by Jon Flanders
The Complete SQL Bootcamp by Josh Portilla, a Data Scientist, on Udemy or
SQL for Newbs: Data Analysis for Beginners by David Kim and Peter Sefton’s course on Udemy.
They all are great courses and teach you SQL basics, but, if you need some free alternatives you can also check out this list of free SQL courses for programmers and developers.

#database #sql #mysql #oracle #tips #sql server #sql (structured query language) #intro to sql #sql bootcamp