How to Implement Table Partitioning in SQL Server

Introduction

Table partitioning is a technique used in SQL Server to physically organize data stored in a table in different storage structures. In essence one ends up having this large logical structure split into smaller parts physically. The result is that we can improve performance for certain kinds of queries and more importantly move data about using certain techniques.

Filegroups

It is important to discuss the concept filegroups in this article because filegroups are the layer of abstraction used to separate data physically. A filegroup is a logical construct that allows SQL Server to see a collection of physical data files as a single logical unit. This means that when SQL Server writes data to that file group, the data is spread across the files which belong to that filegroup. One can think of this as a File Group – a group of data files.

Tables Sit on File Groups

Whenever a table is created, they are created on a filegroup – the PRIMARY file group by default. Typically, when creating a table, you do not specify the filegroup. This means they are created on the PRIMARY filegroup. You can choose to sit a table on a different filegroup if you wish to. However, when creating a partitioned table, you should sit such a table on a Partition Scheme.

Partition Schemes and Functions

A Partition Scheme maps a table to a set of filegroups. A Partition Function defines the criteria by which data is distributed across the filegroups that belong to the desired partition scheme. Thus, it follows logically that in creating a partitioned table, we must create a Partition Function first and then a Partition Scheme.

Why Partitioned Tables?

Two main reasons have been proposed for partitioned tables. It is possible to observe performance benefits when filegroups are sitting on separate disks entirely and we are working with the appropriate degree of parallelism and queries that span one partition. Unfortunately, we will not demonstrate this in this article. Another reason for partitioning tables is maintenance, specifically data archiving which is achieved by switching out partitions. This article shows the entire process: Switching Out Table Partitions in SQL Server: A Walkthrough. Other benefits of partitioning include online index rebuilds, parallel operations, and piecemeal restores of filegroups.

Hands-on Practice

Let us walk through the process of creating a partitioned table. We start by creating a regular table as shown in Listing 1. Since we did not specify any filegroup or partition function, the table sits in the PRIMARY filegroup (See Figure 1).

-- Listing 1: CREATE TABLE Statement
use DB2
GO
CREATE TABLE memmanofarms (
fname VARCHAR(50)
,lname VARCHAR(50)
,city VARCHAR(50)
,PhoneNo bigint
,email VARCHAR(100) check (email like '%@%')
,gender char(1)
)

Using the code in Listing 2, we populate the table with six unique records which are replicated a different number of times per row. We can confirm the row count for each city using the queries in Listing 3. Listing 3 helps us in one more way: we can get a baseline of the way the query is executed when our table is not partitioned.

-- Listing 2: Populate Table
USE DB2
GO
INSERT INTO memmanofarms VALUES ('Kenneth','Igiri','Accra','23320055444','kenneth@kennethigiri.com','M');
GO 1100
INSERT INTO memmanofarms VALUES ('Vivian','Akeredolu','Lagos','2348020055444','vivian@gmail.com','F');
GO 720
INSERT INTO memmanofarms VALUES ('Emelia','Okoro','Port Harcourt','2348030057324','emelia@yahoo.com','F');
GO 400
INSERT INTO memmanofarms VALUES ('Uche','Igiri','Enugu','2348030057324','uche@yahoo.com','M');
GO 1000
INSERT INTO memmanofarms VALUES ('Kweku','Annan','Kumasi','23354055884','kweku@ymail.com','M');
GO 150
INSERT INTO memmanofarms VALUES ('Aisha','Bello','Kano','2347088057324','aisha@gmail.com','F');
GO 890

-- Listing 3: Count Rows 

USE DB2
GO
SET STATISTICS IO ON;
SET STATISTICS TIME ON;

SELECT COUNT(*) FROM memmanofarms;
SELECT COUNT(*) FROM memmanofarms WHERE city='Accra';
SELECT COUNT(*) FROM memmanofarms WHERE city='Lagos';
SELECT COUNT(*) FROM memmanofarms WHERE city='Port Harcourt';
SELECT COUNT(*) FROM memmanofarms WHERE city='Enugu';
SELECT COUNT(*) FROM memmanofarms WHERE city='Kumasi';
SELECT COUNT(*) FROM memmanofarms WHERE city='Kano';
Taking this further, we use the code in Listing 4 to set up objects required for Table Partitioning on the DB2 databases. Notice that for N partitions (and N filegroups), there will always be N-1 boundaries.

Taking this further, we use the code in Listing 4 to set up objects required for Table Partitioning on the DB2 databases. Notice that for N partitions (and N filegroups), there will always be N-1 boundaries.

-- Listing 4: Set Up Partitioning
-- Create a Partition Function

USE [DB2]
GO
CREATE PARTITION FUNCTION
PartFunc (VARCHAR(50))
AS RANGE RIGHT
FOR VALUES 
('Accra'
,'Enugu'
,'Kano'
,'Kumasi'
,'Lagos'
,'Port Harcourt'
)
GO

-- Create File Groups

USE [master]
GO
ALTER DATABASE [DB2] ADD FILEGROUP [AC]
ALTER DATABASE [DB2] ADD FILEGROUP [EN]
ALTER DATABASE [DB2] ADD FILEGROUP [KA]
ALTER DATABASE [DB2] ADD FILEGROUP [KU]
ALTER DATABASE [DB2] ADD FILEGROUP [LA]
ALTER DATABASE [DB2] ADD FILEGROUP [PH]
ALTER DATABASE [DB2] ADD FILEGROUP [OT]
GO

-- Add Files to the File Groups
USE [master]
GO
ALTER DATABASE [DB2] ADD FILE ( NAME = N'AC01', FILENAME = N'C:\MSSQL\Data\AC01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [AC];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'EN01', FILENAME = N'C:\MSSQL\Data\EN01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [EN];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'KA01', FILENAME = N'C:\MSSQL\Data\KA01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [KA];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'KU01', FILENAME = N'C:\MSSQL\Data\KU01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [KU];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'LA01', FILENAME = N'C:\MSSQL\Data\LA01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [LA];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'PH01', FILENAME = N'C:\MSSQL\Data\PH01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [PH];
ALTER DATABASE [DB2] ADD FILE ( NAME = N'OT01', FILENAME = N'C:\MSSQL\Data\OT01.ndf' , SIZE = 102400KB , FILEGROWTH = 131072KB ) TO FILEGROUP [OT];
GO

-- Create a Partition Scheme
USE [DB2]
GO
CREATE PARTITION SCHEME PartSch 
AS PARTITION PartFunc TO
(
AC,
EN,
KA,
KU,
LA,
PH,
OT
)
GO

Once we have the foundation laid, we can then move our regular table from the PRIMARY filegroup to the Partition Function we created. We do this by rebuilding the clustered index as shown in Listing 5. Observe that the column which we used to create the partition column must be part of the clustered index column listing. Also notice that when we specify the Partition Scheme, we must also indicate this column – city.

-- Listing 5: Move table to New Partition
CREATE CLUSTERED INDEX [ClusteredIndexCity] ON [dbo].[memmanofarms]
(
	[city] ASC,
	[PhoneNo] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = ON, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) 
ON [PartSch](city);
GO

After we run this code, the table now sits on the Partition Scheme as shown in the following Figure.

A Little on Performance

We are using the code in Listing 6 as a benchmark of sorts. When we run both questions with the partitioned and non-partitioned tables, we see little difference in performance without the index. In both cases, SQL Server uses an index seek when we have the clustered index in place and a full table scan otherwise. Worthy of note though is that we have more reads when running these queries on the partitioned table. This is expected since we have in effect forced a distribution of the data across “distant” pages.

-- Listing 6: Querying Partitioned and Non-Partitioned Tables
SELECT COUNT(*) FROM memmanofarms WHERE city='Accra';
SELECT * FROM memmanofarms WHERE city='Accra';

Conclusion

We have seen in this article the steps for creating a partitioned table. The references section lists more resources that demonstrate the use of partitioning for archiving old data. We have also shown that partitioning does not necessarily introduce significant improvement in performance for most use cases without other enhancements such as the right CPU and proper MAXDOP configuration.

#sql #sql-server #partitions

What is GEEK

Buddha Community

Introduction to Structured Query Language SQL pdf

SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.

Models for SQL exist. In any case, the SQL that can be utilized on every last one of the major RDBMS today is in various flavors. This is because of two reasons:

1. The SQL order standard is genuinely intricate, and it isn’t handy to actualize the whole standard.

2. Every database seller needs an approach to separate its item from others.

Right now, contrasts are noted where fitting.

#programming books #beginning sql pdf #commands sql #download free sql full book pdf #introduction to sql pdf #introduction to sql ppt #introduction to sql #practical sql pdf #sql commands pdf with examples free download #sql commands #sql free bool download #sql guide #sql language #sql pdf #sql ppt #sql programming language #sql tutorial for beginners #sql tutorial pdf #sql #structured query language pdf #structured query language ppt #structured query language

Python Packages in SQL Server – Get Started with SQL Server Machine Learning Services

Introduction

When installing Machine Learning Services in SQL Server by default few Python Packages are installed. In this article, we will have a look on how to get those installed python package information.

Python Packages

When we choose Python as Machine Learning Service during installation, the following packages are installed in SQL Server,

• revoscalepy – This Microsoft Python package is used for remote compute contexts, streaming, parallel execution of rx functions for data import and transformation, modeling, visualization, and analysis.
• microsoftml – This is another Microsoft Python package which adds machine learning algorithms in Python.
• Anaconda 4.2 – Anaconda is an opensource Python package

#machine learning #sql server #executing python in sql server #machine learning using python #machine learning with sql server #ml in sql server using python #python in sql server ml #python packages #python packages for machine learning services #sql server machine learning services

SCHEMAS in SQL Server -MS SQL Server – Zero to Hero Query Master

Introduction

This is part 3 of “MS SQL Server- Zero to Hero” and in this article, we will be discussing about the SCHEMAS in SQL SERVER. Before getting into this article, please consider to visit previous articles in this series from below,

• MS SQL Server – Zero to Hero Query Master – Part 1
• MS SQL Server – Zero to Hero Query Master – Part 2

A glimpse of previous articles

Part 1

In part one, we learned the basics of data, database, database management system, and types of DBMS and SQL.

Part 2

• We learned to create a database and maintain it using SQL statements.
• Best practice methods were also mentioned.

#sql server #benefits of schemas #create schema in sql #database schemas #how to create schema in sql server #schemas #schemas in sql server #sql server schemas #what is schema in sql server

How to Make Use of SQL Server Graph Database Features – {coding}Sight

Would SQL Server graph database features fit your next project? While you may not know the definitive answer right now, you might be wondering, “What problems does it solve?”.

In broad terms, this post tackles the issues of what a graph database is, what its uses are, and what benefits you and your stakeholders can derive from SQL Server graph database capabilities. And see for yourself why that is not another excuse for using something new for your new project. So, if you haven’t yet checked out this “feature”, it’s time to take a look at how cool this is.

Who knows? This can be a great solution for your next project.

So, let’s dive right in.

What is a Graph Database?

While we know rows, columns, primary and foreign keys are part of relational databases, graph databases use nodes and edges. They are mainly suitable for many-to-many relationships. Unlike the HierarchyID, a node can have more than 1 parent, while HierarchyIDs are limited to one-to-many relationships only.

Meanwhile, nodes can have properties, and edges define the relationship between nodes.

#sql server #edge table #how to #node table #sql server #sql server 2017 #sql server graph database

Welcome Back the T-SQL Debugger with SQL Complete – SQL Debugger

When you develop large chunks of T-SQL code with the help of the SQL Server Management Studio tool, it is essential to test the “Live” behavior of your code by making sure that each small piece of code works fine and being able to allocate any error message that may cause a failure within that code.

The easiest way to perform that would be to use the T-SQL debugger feature, which used to be built-in over the SQL Server Management Studio tool. But since the T-SQL debugger feature was removed completely from SQL Server Management Studio 18 and later editions, we need a replacement for that feature. This is because we cannot keep using the old versions of SSMS just to support the T-SQL Debugger feature without “enjoying” the new features and bug fixes that are released in the new SSMS versions.

If you plan to wait for SSMS to bring back the T-SQL Debugger feature, vote in the Put Debugger back into SSMS 18 to ask Microsoft to reintroduce it.

As for me, I searched for an alternative tool for a T-SQL Debugger SSMS built-in feature and found that Devart company rolled out a new T-SQL Debugger feature to version 6.4 of SQL – Complete tool. SQL Complete is an add-in for Visual Studio and SSMS that offers scripts autocompletion capabilities, which help develop and debug your SQL database project.

The SQL Debugger feature of SQL Complete allows you to check the execution of your scripts, procedures, functions, and triggers step by step by adding breakpoints to the lines where you plan to start, suspend, evaluate, step through, and then to continue the execution of your script.

You can download SQL Complete from the dbForge Download page and install it on your machine using a straight-forward installation wizard. The wizard will ask you to specify the installation path for the SQL Complete tool and the versions of SSMS and Visual Studio that you plan to install the SQL Complete on, as an add-in, from the versions that are installed on your machine, as shown below:

Once SQL Complete is fully installed on your machine, the dbForge SQL Complete installation wizard will notify you of whether the installation was completed successfully or the wizard faced any specific issue that you can troubleshoot and fix easily. If there are no issues, the wizard will provide you with an option to open the SSMS tool and start using the SQL Complete tool, as displayed below:

When you open SSMS, you will see a new “Debug” tools menu, under which you can navigate the SQL Debugger feature options. Besides, you will see a list of icons that will be used to control the debug mode of the T-SQL query at the leftmost side of the SSMS tool. If you cannot see the list, you can go to View -> Toolbars -> Debugger to make these icons visible.

During the debugging session, the SQL Debugger icons will be as follows:

The functionality of these icons within the SQL Debugger can be summarized as:

• Adding Breakpoints to control the execution pause of the T-SQL script at a specific statement allows you to check the debugging information of the T-SQL statements such as the values for the parameters and the variables.
• Step Into is “navigate” through the script statements one by one, allowing you to check how each statement behaves.
• Step Over is “execute” a specific stored procedure if you are sure that it contains no error.
• Step Out is “return” from the stored procedure, function, or trigger to the main debugging window.
• Continue executing the script until reaching the next breakpoint.
• Stop Debugging is “terminate” the debugging session.
• Restart “stop and start” the current debugging session.

#sql server #sql #sql debugger #sql server #sql server stored procedure #ssms #t-sql queries