How to Import Records From a CSV File to Microsoft SQL Server With Python

How to Import Records From a CSV File to Microsoft SQL Server With Python

In this Python tutorial, we are going to learn how to import records from a CSV file to Microsoft SQL Server.

In this Python tutorial, we are going to learn how to import records from a CSV file to Microsoft SQL Server.

The script can also be adapted to import dataset from an Excel file, a text file, or any data file as we will be reading the dataset into a pandas dataframe first. And the database system is not limited to just SQL Server, it can be MySQL, Oracle, or any other database systems. I am using SQL Server because this is the database system I know the best.

Timestamp: 00:00 - Tutorial info 00:45 - Data source used in this video 02:15 - Table Creation SQL Statement 02:35 - Writing automation script in Python 03:20 - How to find out what drivers are available on your PC 03:45 - Get SQL Server Server Name 04:45 - Import data to DataFrame 07:45 - Data clean up 14:52 - Convert DataFrame object to List object 16:00 - Create connection string 19:45 - Create connection object 22:00 - Create cursor object 25:35 - Run the script for testing.

Source Code:


import pypyodbc as odbc # pip install pypyodbc
import pandas as pd # pip install pandas

"""
Step 1. Importing dataset from CSV
"""
df = pd.read_csv('Real-Time_Traffic_Incident_Reports.csv')

"""
Step 2.1 Data clean up
"""
df['Published Date'] = pd.to_datetime(df['Published Date']).dt.strftime('%Y-%m-%d %H:%M:%S')
df['Status Date'] = pd.to_datetime(df['Published Date']).dt.strftime('%Y-%m-%d %H:%M:%S')

df.drop(df.query('Location.isnull() | Status.isnull()').index, inplace=True)


"""
Step 2.2 Specify columns we want to import
"""
columns = ['Traffic Report ID', 'Published Date', 'Issue Reported', 'Location', 
            'Address', 'Status', 'Status Date']

df_data = df[columns]
records = df_data.values.tolist()


"""
Step 3.1 Create SQL Servre Connection String
"""
DRIVER = 'SQL Server'
SERVER_NAME = '<Server Name>'
DATABASE_NAME = '<Database Name>'

def connection_string(driver, server_name, database_name):
    conn_string = f"""
        DRIVER={{{driver}}};
        SERVER={server_name};
        DATABASE={database_name};
        Trust_Connection=yes;        
    """
    return conn_string

"""
Step 3.2 Create database connection instance
"""
try:
    conn = odbc.connect(connection_string(DRIVER, SERVER_NAME, DATABASE_NAME))
except odbc.DatabaseError as e:
    print('Database Error:')    
    print(str(e.value[1]))
except odbc.Error as e:
    print('Connection Error:')
    print(str(e.value[1]))


"""
Step 3.3 Create a cursor connection and insert records
"""

sql_insert = '''
    INSERT INTO Austin_Traffic_Incident 
    VALUES (?, ?, ?, ?, ?, ?, ?, GETDATE())
'''

try:
    cursor = conn.cursor()
    cursor.executemany(sql_insert, records)
    cursor.commit();    
except Exception as e:
    cursor.rollback()
    print(str(e[1]))
finally:
    print('Task is complete.')
    cursor.close()
    conn.close()

python sql

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Introduction to Structured Query Language SQL pdf

SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.

Welcome Back the T-SQL Debugger with SQL Complete – SQL Debugger

Debug SQL stored procedures and develop your SQL database project with dbForge SQL Complete, a new add-in for Visual Studio and SSMS. When you develop large chunks of T-SQL code with the help of the SQL Server Management Studio tool, it is essential to test the “Live” behavior of your code by making sure that each small piece of code works fine and being able to allocate any error message that may cause a failure within that code.

Python Tricks Every Developer Should Know

In this tutorial, you’re going to learn a variety of Python tricks that you can use to write your Python code in a more readable and efficient way like a pro.

How to Remove all Duplicate Files on your Drive via Python

Today you're going to learn how to use Python programming in a way that can ultimately save a lot of space on your drive by removing all the duplicates. We gonna use Python OS remove( ) method to remove the duplicates on our drive. Well, that's simple you just call remove ( ) with a parameter of the name of the file you wanna remove done.

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.