Introduction To ETL Using SSIS Package

Extract, Transform, and Load (ETL) processes have become a necessity in IT today. Being able to move data from one place to another has become a cornerstone in just about every company worldwide. SQL Server provides Integration Services (SSIS) that allow database administrators (DBAs) and Developers alike to create and deploy advanced ETL packages to leverage the data within their organizations.

Introduction on ETL

The data in a Data Warehouse system is loaded with an ETL (Extract, Transform, Load) tool.
To get in-Depth knowledge on Informatica you can enroll for a live demo on Informatica online training
As the name suggests, it performs the following three operations,

  • Extracts the data from your transactional system which can be an Oracle, Microsoft, or any other relational database,
  • Transforms the data by performing data cleansing operations, and then
  • Loads the data into the OLAP data Warehouse.

Example

Let us assume there is a manufacturing company having multiple departments such as sales, HR, Material Management, EWM, etc. All these departments have separate databases which they use to maintain information w.r.t. their work and each database has a different technology, landscape, table names, columns, etc. Now, if the company wants to analyze historical data and generate reports, all the data from these data sources should be extracted and loaded into a Data Warehouse to save it for analytical work.

An ETL tool extracts the data from all these heterogeneous data sources, transforms the data (like applying calculations, joining fields, keys, removing incorrect data fields, etc.), and loads it into a Data Warehouse. Later, you can use various Business Intelligence (BI) tools to generate meaningful reports, dashboards, and visualizations using this data.
This is image title

ETL Process

Let us now discuss in a little more detail the key steps involved in an ETL procedure −

Extracting the Data

It involves extracting the data from different heterogeneous data sources. Data extraction from a transactional system varies as per the requirement and the ETL tool in use. It is normally done by running scheduled jobs in off-business hours like running jobs at night or over the weekend.

This is image title

Transforming the Data

It involves transforming the data into a suitable format that can be easily loaded into a DW system. Data transformation involves applying calculations, joins, and defining primary and foreign keys on the data. For example, if you want % of total revenue which is not in database, you will apply % formula in transformation and load the data. Similarly, if you have the first name and the last name of users in different columns, then you can apply a concatenate operation before loading the data. Some data doesn’t require any transformation; such data is known as direct move or pass through data.Learn more from informatica online course

Data transformation also involves data correction and cleansing of data, removing incorrect data, incomplete data formation, and fixing data errors. It also includes data integrity and formatting incompatible data before loading it into a DW system.

Loading the Data into a DW System

It involves loading the data into a DW system for analytical reporting and information. The target system can be a simple delimited flat file or a data warehouse.

ETL Tool Function

A typical ETL tool-based data warehouse uses staging area, data integration, and access layers to perform its functions. It’s normally a 3-layer architecture.

Staging Layer:

The staging layer or staging database is used to store the data extracted from different source data systems.

Data Integration Layer:

The integration layer transforms the data from the staging layer and moves the data to a database, where the data is arranged into hierarchical groups, often called dimensions, and into facts and aggregate facts. The combination of facts and dimensions tables in a DW system is called a schema.

Access Layer:

The access layer is used by end-users to retrieve the data for analytical reporting and information.
DTS ( Data Transformation Services):

In SQL Server versions 6.5 and earlier, Database administrators (DBAs) used SQL Server Transfer Manager and Bulk Copy Program, included with SQL Server, to transfer data. These tools had significant shortcomings, and many DBAs used third-party tools. When SQL Server 7 was released, “Data Transformation Services” was packaged with it to replace all these tools.

SQL Server 2000 expanded DTS functionality in several ways. Many new types of tasks were made, including the ability to FTP files, move databases or database components, and add messages into Microsoft Message Queue (MSMQ).

SSIS (SQL server Integration Services):

One disadvantage of DTS was that, for development of package, one must be connected to SQL Server. This is overcome in SSIS.

SSIS is replacement of DTS in SQL Server 2005. SSIS introduced “Business Intelligence (BI) tool“, which is development IDE and installed automatically by SQL Server 2005 in Visual Studio 2005. Thus BI tool gives the advantage of Visual Studio development tools for DTS.

Advantages of SSIS over DTS:

SSIS gives you many new ways to control the flow of your package that could only be done before by writing code. One of the coolest control features is the ability to set up looping within the pa
ckage. Two tasks, the For Loop Container and Foreach Loop Container, are available for this purpose.
I hope you reach a conclusion about Data Warehousing Informatica. You can learn more about Informatica from online Informatica training

#informatica online training #informatica training #online infromatica training #informatica bdm training #informatica course #informatica training online

What is GEEK

Buddha Community

Why Use WordPress? What Can You Do With WordPress?

Can you use WordPress for anything other than blogging? To your surprise, yes. WordPress is more than just a blogging tool, and it has helped thousands of websites and web applications to thrive. The use of WordPress powers around 40% of online projects, and today in our blog, we would visit some amazing uses of WordPress other than blogging.
What Is The Use Of WordPress?

WordPress is the most popular website platform in the world. It is the first choice of businesses that want to set a feature-rich and dynamic Content Management System. So, if you ask what WordPress is used for, the answer is – everything. It is a super-flexible, feature-rich and secure platform that offers everything to build unique websites and applications. Let’s start knowing them:

1. Multiple Websites Under A Single Installation
WordPress Multisite allows you to develop multiple sites from a single WordPress installation. You can download WordPress and start building websites you want to launch under a single server. Literally speaking, you can handle hundreds of sites from one single dashboard, which now needs applause.
It is a highly efficient platform that allows you to easily run several websites under the same login credentials. One of the best things about WordPress is the themes it has to offer. You can simply download them and plugin for various sites and save space on sites without losing their speed.

2. WordPress Social Network
WordPress can be used for high-end projects such as Social Media Network. If you don’t have the money and patience to hire a coder and invest months in building a feature-rich social media site, go for WordPress. It is one of the most amazing uses of WordPress. Its stunning CMS is unbeatable. And you can build sites as good as Facebook or Reddit etc. It can just make the process a lot easier.
To set up a social media network, you would have to download a WordPress Plugin called BuddyPress. It would allow you to connect a community page with ease and would provide all the necessary features of a community or social media. It has direct messaging, activity stream, user groups, extended profiles, and so much more. You just have to download and configure it.
If BuddyPress doesn’t meet all your needs, don’t give up on your dreams. You can try out WP Symposium or PeepSo. There are also several themes you can use to build a social network.

3. Create A Forum For Your Brand’s Community
Communities are very important for your business. They help you stay in constant connection with your users and consumers. And allow you to turn them into a loyal customer base. Meanwhile, there are many good technologies that can be used for building a community page – the good old WordPress is still the best.
It is the best community development technology. If you want to build your online community, you need to consider all the amazing features you get with WordPress. Plugins such as BB Press is an open-source, template-driven PHP/ MySQL forum software. It is very simple and doesn’t hamper the experience of the website.
Other tools such as wpFoRo and Asgaros Forum are equally good for creating a community blog. They are lightweight tools that are easy to manage and integrate with your WordPress site easily. However, there is only one tiny problem; you need to have some technical knowledge to build a WordPress Community blog page.

4. Shortcodes
Since we gave you a problem in the previous section, we would also give you a perfect solution for it. You might not know to code, but you have shortcodes. Shortcodes help you execute functions without having to code. It is an easy way to build an amazing website, add new features, customize plugins easily. They are short lines of code, and rather than memorizing multiple lines; you can have zero technical knowledge and start building a feature-rich website or application.
There are also plugins like Shortcoder, Shortcodes Ultimate, and the Basics available on WordPress that can be used, and you would not even have to remember the shortcodes.

5. Build Online Stores
If you still think about why to use WordPress, use it to build an online store. You can start selling your goods online and start selling. It is an affordable technology that helps you build a feature-rich eCommerce store with WordPress.
WooCommerce is an extension of WordPress and is one of the most used eCommerce solutions. WooCommerce holds a 28% share of the global market and is one of the best ways to set up an online store. It allows you to build user-friendly and professional online stores and has thousands of free and paid extensions. Moreover as an open-source platform, and you don’t have to pay for the license.
Apart from WooCommerce, there are Easy Digital Downloads, iThemes Exchange, Shopify eCommerce plugin, and so much more available.

6. Security Features
WordPress takes security very seriously. It offers tons of external solutions that help you in safeguarding your WordPress site. While there is no way to ensure 100% security, it provides regular updates with security patches and provides several plugins to help with backups, two-factor authorization, and more.
By choosing hosting providers like WP Engine, you can improve the security of the website. It helps in threat detection, manage patching and updates, and internal security audits for the customers, and so much more.

Read More

#use of wordpress #use wordpress for business website #use wordpress for website #what is use of wordpress #why use wordpress #why use wordpress to build a website

Anil Cynix

1592470812

Before we learn anything about ETL Testing its important to learn about Business Intelligence and Dataware. Let’s get started –

What is BI?
Business Intelligence is the process of collecting raw data or business data and turning it into information that is useful and more meaningful. The raw data is the records of the daily transaction of an organization such as interactions with customers, administration of finance, and management of employee and so on. These data’s will be used for “Reporting, Analysis, Data mining, Data quality and Interpretation, Predictive Analysis”.

What is Data Warehouse?
A data warehouse is a database that is designed for query and analysis rather than for transaction processing. The data warehouse is constructed by integrating the data from multiple heterogeneous sources.It enables the company or organization to consolidate data from several sources and separates analysis workload from transaction workload. Data is turned into high quality information to meet all enterprise reporting requirements for all levels of users.

What is ETL?
ETL stands for Extract-Transform-Load and it is a process of how data is loaded from the source system to the data warehouse. Data is extracted from an OLTP database, transformed to match the data warehouse schema and loaded into the data warehouse database. Many data warehouses also incorporate data from non-OLTP systems such as text files, legacy systems and spreadsheets.

Let see how it works

For example, there is a retail store which has different departments like sales, marketing, logistics etc. Each of them is handling the customer information independently, and the way they store that data is quite different. The sales department have stored it by customer’s name, while marketing department by customer id.

Now if they want to check the history of the customer and want to know what the different products he/she bought owing to different marketing campaigns; it would be very tedious.

The solution is to use a Datawarehouse to store information from different sources in a uniform structure using ETL. ETL can transform dissimilar data sets into an unified structure.Later use BI tools to derive meaningful insights and reports from this data.

The following diagram gives you the ROAD MAP of the ETL process

ETL Testing or Datawarehouse Testing : Ultimate Guide

1.Extract

  • Extract relevant data
    2.Transform
  • Transform data to DW (Data Warehouse) format
  • Build keys - A key is one or more data attributes that uniquely identify an entity. Various types of keys are primary key, alternate key, foreign key, composite key, surrogate key. The datawarehouse owns these keys and never allows any other entity to assign them.
  • Cleansing of data :After the data is extracted, it will move into the next phase, of cleaning and conforming of data. Cleaning does the omission in the data as well as identifying and fixing the errors. Conforming means resolving the conflicts between those data’s that is incompatible, so that they can be used in an enterprise data warehouse. In addition to these, this system creates meta-data that is used to diagnose source system problems and improves data quality.

3.Load

  • Load data into DW ( Data Warehouse)
  • Build aggregates - Creating an aggregate is summarizing and storing data which is available in fact table in order to improve the performance of end-user queries.

What is ETL Testing?
ETL testing is done to ensure that the data that has been loaded from a source to the destination after business transformation is accurate. It also involves the verification of data at various middle stages that are being used between source and destination. ETL stands for Extract-Transform-Load.

ETL Testing Process
Similar to other Testing Process, ETL also go through different phases. The different phases of ETL testing process is as follows

ETL Testing or Datawarehouse Testing : Ultimate Guide

ETL testing is performed in five stages

1.Identifying data sources and requirements
2.Data acquisition
3.Implement business logics and dimensional Modelling
4.Build and populate data
5.Build Reports
ETL Testing or Datawarehouse Testing : Ultimate Guide.

These are the basics of etl testing course if u want to know more plz visit Online IT Guru website.

#etl testing #etl testing course #etl testing online #etl testing training #online etl testing training #etl testing online training

Mikel  Okuneva

Mikel Okuneva

1596834000

Which Tools Do You Use in ETL Testing?

Image for post

While there are several advanced ETL Testing tools accessible for software testing -tools, software testing companies specifically use “Informatica Data Validation”. Informatica Data Validation is one of the famous ETL tool, which integrates with the PowerCenter Repository & Integration Services. This advanced tool allows business analysts and developers to create rules to test the mapped data.

While distinct approaches to ETL testing are error-prone, very time-consuming, and seldom provide total test coverage. Informatica Data Validation Option gives an ETL testing tool that can speed up and automate ETL testing in both production environments and development & test, which means that you can deliver repeatable, complete, and auditable test coverage in minimum time with no programming skills required. Automated ETL Testing reduces time consumption and helps to maintain accuracy.

Key Features of Informatica Data Validation (ETL Testing tool):

  • Informatica Data Validation gives the complete result for data validation along with data integrity
  • Recognizes and prevents data issues and gives greater business productivity
  • Minimizes programming efforts on account of the intuitive user interface and built-in operators
  • It has Wizards to generate test Query without including the user’s requirement to write SQL
  • More sophisticated tests tool also proffers design Library & reusable Query Snippets
  • Analyze millions of data rows & columns in minutes
  • Assists to compare data from source files & data stores to the target Data Warehouse
  • Produce informative reports, updates, and auto-email outcome.
  • 50 to 90% of efforts and costs can be saved using the Informatica Data Validation tool.

Informatica Data Validation”. Informatica Data Validation

Automating ETL tests permits everyday testing without any user intervention and also aids to supports automatic regression tests on the old code following every single new release. In the end, this sophisticated tool will save your precious time and your users will appreciate the quality of your Business Intelligence deliverable.

#etl-testing #software-engineering #etl #software-testing #etl-tool

I am Developer

1597487833

Country State City Drop Down List using Ajax in Laravel

Here, i will show you how to create dynamic depedent country state city dropdown list using ajax in laravel.

Country State City Dropdown List using Ajax in php Laravel

Follow Below given steps to create dynamic dependent country state city dropdown list with jQuery ajax in laravel:

  • Step 1: Install Laravel App
  • Step 2: Add Database Details
  • Step 3: Create Country State City Migration and Model File
  • Step 4: Add Routes For Country State City
  • Step 5: Create Controller For Fetch Country State City
  • Step 6: Create Blade File For Show Dependent Country State City in Dropdown
  • Step 7: Run Development Server

https://www.tutsmake.com/ajax-country-state-city-dropdown-in-laravel/

#how to create dynamic dropdown list using laravel dynamic select box in laravel #laravel-country state city package #laravel country state city drop down #dynamic dropdown country city state list in laravel using ajax #country state city dropdown list using ajax in php laravel #country state city dropdown list using ajax in laravel demo

Joseph  Murray

Joseph Murray

1624048020

How to Find All the Classes of a Package in Java

In this article let’s take a look at how to find all classes of a package in Java

To find all classes of a package in Java we can use the ClassHunter of Burningwave Core library. So we start by adding the following dependency to our pom.xml:

XML

1

<dependency>2
    <groupId>org.burningwave</groupId>3
    <artifactId>core</artifactId>4
    <version>8.4.0</version>5
</dependency>

The next steps are the following:

  • retrieving the ClassHunter through the ComponentContainer
  • defining a regular expression that we must pass to the **ClassCriteria **object that will be injected into the **SearchConfig **object
  • calling the **loadInCache **method that loads in the cache all loadable classes of the indicated paths, then applies the criteria filter and then returns the **SearchResult **object which contains the classes that match the criteria

#java #classes #class #packages #package #how to find all the classes of a package in java