Migrate: Database Migrations - CLI and Golang Library

migrate

Database migrations written in Go. Use as CLI or import as library.

Migrate reads migrations from sources and applies them in correct order to a database.
Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
Database drivers don't assume things or try to correct user input. When in doubt, fail.

Databases

Database drivers run migrations. Add a new database?

Database URLs

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: !, #, $, %, &, ', (, ), *, +, ,, /, :, ;, =, ?, @, [, ]

It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$

Migration Sources

Source drivers read migrations from local or remote sources. Add a new source?

Filesystem - read from filesystem
io/fs - read from a Go io/fs
Go-Bindata - read from embedded binary data (jteeuwen/go-bindata)
pkger - read from embedded binary data (markbates/pkger)
GitHub - read from remote GitHub repositories
GitHub Enterprise - read from remote GitHub Enterprise repositories
Bitbucket - read from remote Bitbucket repositories
Gitlab - read from remote Gitlab repositories
AWS S3 - read from Amazon Web Services S3
Google Cloud Storage - read from Google Cloud Platform Storage

CLI usage

Simple wrapper around this library.
Handles ctrl+c (SIGINT) gracefully.
No config search paths, no config files, no magic ENV var injections.

CLI Documentation

Basic usage

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

Docker usage

$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
    -path=/migrations/ -database postgres://localhost:5432/database up 2

Use in your Go project

API is stable and frozen for this release (v3 & v4).
Uses Go modules to manage dependencies.
To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool.
Bring your own logger.
Uses io.Reader streams internally for low memory overhead.
Thread-safe and no goroutine leaks.

Go Documentation

import (
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/github"
)

func main() {
    m, err := migrate.New(
        "github://mattes:personal-access-token@mattes/migrate_test",
        "postgres://localhost:5432/database?sslmode=enable")
    m.Steps(2)
}

Want to use an existing database client?

import (
    "database/sql"
    _ "github.com/lib/pq"
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
)

func main() {
    db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    m, err := migrate.NewWithDatabaseInstance(
        "file:///migrations",
        "postgres", driver)
    m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run
}

Getting started

Go to getting started

Tutorials

(more tutorials to come)

Migration files

Each migration has an up and down migration. Why?

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Best practices: How to write migrations.

Versions

Version	Supported?	Import	Notes
master	:white_check_mark:	`import "github.com/golang-migrate/migrate/v4"`	New features and bug fixes arrive here first
v4	:white_check_mark:	`import "github.com/golang-migrate/migrate/v4"`	Used for stable releases
v3	:x:	`import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended)	DO NOT USE - No longer supported

Development and Contributing

Yes, please! Makefile is your friend, read the development guide.

Also have a look at the FAQ.

Looking for alternatives? https://awesome-go.com/#database.

Author: Golang-migrate
Source Code: https://github.com/golang-migrate/migrate
License: View license

#go #golang #mysql

Django-allauth: A simple Boilerplate to Setup Authentication

Django-Authentication

A simple Boilerplate to Setup Authentication using Django-allauth, with a custom template for login and registration using django-crispy-forms.

Getting Started

Prerequisites

Python 3.8.6 or higher

Project setup

# clone the repo
$ git clone https://github.com/yezz123/Django-Authentication

# move to the project folder
$ cd Django-Authentication

Creating virtual environment

Create a virtual environment for this project:

# creating pipenv environment for python 3
$ virtualenv venv

# activating the pipenv environment
$ cd venv/bin #windows environment you activate from Scripts folder

# if you have multiple python 3 versions installed then
$ source ./activate

Configured Enviromment

Environment variables

SECRET_KEY = #random string
DEBUG = #True or False
ALLOWED_HOSTS = #localhost
DATABASE_NAME = #database name (You can just use the default if you want to use SQLite)
DATABASE_USER = #database user for postgres
DATABASE_PASSWORD = #database password for postgres
DATABASE_HOST = #database host for postgres
DATABASE_PORT = #database port for postgres
ACCOUNT_EMAIL_VERIFICATION = #mandatory or optional
EMAIL_BACKEND = #email backend
EMAIL_HOST = #email host
EMAIL_HOST_PASSWORD = #email host password
EMAIL_USE_TLS = # if your email use tls
EMAIL_PORT = #email port

change all the environment variables in the .env.sample and don't forget to rename it to .env.

Run the project

After Setup the environment, you can run the project using the Makefile provided in the project folder.

help:
 @echo "Targets:"
 @echo "    make install" #install requirements
 @echo "    make makemigrations" #prepare migrations
 @echo "    make migrations" #migrate database
 @echo "    make createsuperuser" #create superuser
 @echo "    make run_server" #run the server
 @echo "    make lint" #lint the code using black
 @echo "    make test" #run the tests using Pytest

Preconfigured Packages

Includes preconfigured packages to kick start Django-Authentication by just setting appropriate configuration.

Package	Usage
django-allauth	Integrated set of Django applications addressing authentication, registration, account management as well as 3rd party (social) account authentication.
django-crispy-forms	django-crispy-forms provides you with a `crispy` filter and `{% crispy %}` tag that will let you control the rendering behavior of your Django forms in a very elegant and DRY way.

Contributing

Django-Authentication is a simple project, so you can contribute to it by just adding your code to the project to improve it.
If you have any questions, please feel free to open an issue or create a pull request.

Download Details:
Author: yezz123
Source Code: https://github.com/yezz123/Django-Authentication
License: MIT License

#django #python

Adaline Kulas

1594166040

What are the benefits of cloud migration? Reasons you should migrate

The moving of applications, databases and other business elements from the local server to the cloud server called cloud migration. This article will deal with migration techniques, requirement and the benefits of cloud migration.

In simple terms, moving from local to the public cloud server is called cloud migration. Gartner says 17.5% revenue growth as promised in cloud migration and also has a forecast for 2022 as shown in the following image.

#cloud computing services #cloud migration #all #cloud #cloud migration strategy #enterprise cloud migration strategy #business benefits of cloud migration #key benefits of cloud migration #benefits of cloud migration #types of cloud migration

Grace Lesch

1621440000

Migrating financial data to cloud with Database Migration Service

Choosing Database Migration Service

At first, we didn’t consider an auto-migrating solution because there wasn’t a lot of complexity in the migration, especially for our small databases. We’d just have to bring the application down, export the database, import it to Cloud SQL, and bring it back up. That worked for most of our applications.

But toward the end of the process, two applications remained.

Through conversations with our product team and our Google contact, we learned about the Google Cloud’s Database Migration Service (which was in private preview at the time), which provides a serverless migration experience from MySQL to Cloud SQL for MySQL with continuous data replication, at no additional cost.

#database #database service #migrate databases #migrating financial data

Go Programming

1674800328

Go Migrate: Database Migrations - CLI and Golang Library

Migrate

Database migrations written in Go. Use as CLI or import as library.

Migrate reads migrations from sources and applies them in correct order to a database.
Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
Database drivers don't assume things or try to correct user input. When in doubt, fail.

Forked from mattes/migrate

Databases

Database drivers run migrations. Add a new database?

Database URLs

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: !, #, $, %, &, ', (, ), *, +, ,, /, :, ;, =, ?, @, [, ]

It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$

Migration Sources

Source drivers read migrations from local or remote sources. Add a new source?

Filesystem - read from filesystem
io/fs - read from a Go io/fs
Go-Bindata - read from embedded binary data (jteeuwen/go-bindata)
pkger - read from embedded binary data (markbates/pkger)
GitHub - read from remote GitHub repositories
GitHub Enterprise - read from remote GitHub Enterprise repositories
Bitbucket - read from remote Bitbucket repositories
Gitlab - read from remote Gitlab repositories
AWS S3 - read from Amazon Web Services S3
Google Cloud Storage - read from Google Cloud Platform Storage

CLI usage

Simple wrapper around this library.
Handles ctrl+c (SIGINT) gracefully.
No config search paths, no config files, no magic ENV var injections.

CLI Documentation

Basic usage

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

Docker usage

$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
    -path=/migrations/ -database postgres://localhost:5432/database up 2

Use in your Go project

API is stable and frozen for this release (v3 & v4).
Uses Go modules to manage dependencies.
To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool.
Bring your own logger.
Uses io.Reader streams internally for low memory overhead.
Thread-safe and no goroutine leaks.

Go Documentation

import (
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/github"
)

func main() {
    m, err := migrate.New(
        "github://mattes:personal-access-token@mattes/migrate_test",
        "postgres://localhost:5432/database?sslmode=enable")
    m.Steps(2)
}

Want to use an existing database client?

import (
    "database/sql"
    _ "github.com/lib/pq"
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
)

func main() {
    db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    m, err := migrate.NewWithDatabaseInstance(
        "file:///migrations",
        "postgres", driver)
    m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run
}

Getting started

Go to getting started

Tutorials

(more tutorials to come)

Migration files

Each migration has an up and down migration. Why?

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Best practices: How to write migrations.

Versions

Version	Supported?	Import	Notes
master	:white_check_mark:	`import "github.com/golang-migrate/migrate/v4"`	New features and bug fixes arrive here first
v4	:white_check_mark:	`import "github.com/golang-migrate/migrate/v4"`	Used for stable releases
v3	:x:	`import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended)	DO NOT USE - No longer supported

Development and Contributing

Yes, please! Makefile is your friend, read the development guide.

Also have a look at the FAQ.

Download Details:
Author: golang-migrate
Source Code: https://github.com/golang-migrate/migrate

#go #golang #database

Ruth Nabimanya

1620633584

System Databases in SQL Server

Introduction

In SSMS, we many of may noticed System Databases under the Database Folder. But how many of us knows its purpose?. In this article lets discuss about the System Databases in SQL Server.

System Database

Fig. 1 System Databases

There are five system databases, these databases are created while installing SQL Server.

Master
Model
MSDB
Tempdb
Resource

Master

This database contains all the System level Information in SQL Server. The Information in form of Meta data.
Because of this master database, we are able to access the SQL Server (On premise SQL Server)

Model

This database is used as a template for new databases.
Whenever a new database is created, initially a copy of model database is what created as new database.

MSDB

This database is where a service called SQL Server Agent stores its data.
SQL server Agent is in charge of automation, which includes entities such as jobs, schedules, and alerts.

TempDB

The Tempdb is where SQL Server stores temporary data such as work tables, sort space, row versioning information and etc.
User can create their own version of temporary tables and those are stored in Tempdb.
But this database is destroyed and recreated every time when we restart the instance of SQL Server.

Resource

The resource database is a hidden, read only database that holds the definitions of all system objects.
When we query system object in a database, they appear to reside in the sys schema of the local database, but in actually their definitions reside in the resource db.

#sql server #master system database #model system database #msdb system database #sql server system databases #ssms #system database #system databases in sql server #tempdb system database

Migrate: Database Migrations - CLI and Golang Library

Databases

Database URLs

Migration Sources

CLI usage

Basic usage

Docker usage

Use in your Go project

Getting started

Tutorials

Migration files

Versions

Development and Contributing

What is GEEK

Buddha Community

Django-allauth: A simple Boilerplate to Setup Authentication

Django-Authentication

Getting Started

Prerequisites

Project setup

Creating virtual environment

Configured Enviromment

Environment variables

Run the project

Preconfigured Packages

Contributing

What are the benefits of cloud migration? Reasons you should migrate

Migrating financial data to cloud with Database Migration Service

Choosing Database Migration Service

Go Migrate: Database Migrations - CLI and Golang Library

Migrate

Databases

Database URLs

Migration Sources

CLI usage

Basic usage

Docker usage

Use in your Go project

Getting started

Tutorials

Migration files

Versions

Development and Contributing

System Databases in SQL Server

Introduction

System Database

Master

Model

MSDB

TempDB

Resource