1672999928
Installs and configures MySQL or MariaDB server on RHEL/CentOS or Debian/Ubuntu servers.
No special requirements; note that this role requires root access, so either run it in a playbook with a global become: yes
, or invoke the role in your playbook like:
- hosts: database
roles:
- role: geerlingguy.mysql
become: yes
Available variables are listed below, along with default values (see defaults/main.yml
):
mysql_user_home: /root
mysql_user_name: root
mysql_user_password: root
The home directory inside which Python MySQL settings will be stored, which Ansible will use when connecting to MySQL. This should be the home directory of the user which runs this Ansible role. The mysql_user_name
and mysql_user_password
can be set if you are running this role under a non-root user account and want to set a non-root user.
mysql_root_home: /root
mysql_root_username: root
mysql_root_password: root
The MySQL root user account details.
mysql_root_password_update: false
Whether to force update the MySQL root user's password. By default, this role will only change the root user's password when MySQL is first configured. You can force an update by setting this to yes
.
Note: If you get an error like
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
after a failed or interrupted playbook run, this usually means the root password wasn't originally updated to begin with. Try either removing the.my.cnf
file inside the configuredmysql_user_home
or updating it and settingpassword=''
(the insecure default password). Run the playbook again, withmysql_root_password_update
set toyes
, and the setup should complete.
Note: If you get an error like
ERROR 1698 (28000): Access denied for user 'root'@'localhost' (using password: YES)
when trying to log in from the CLI you might need to run as root or sudoer.
mysql_enabled_on_startup: true
Whether MySQL should be enabled on startup.
mysql_config_file: *default value depends on OS*
mysql_config_include_dir: *default value depends on OS*
The main my.cnf configuration file and include directory.
overwrite_global_mycnf: true
Whether the global my.cnf should be overwritten each time this role is run. Setting this to no
tells Ansible to only create the my.cnf
file if it doesn't exist. This should be left at its default value (yes
) if you'd like to use this role's variables to configure MySQL.
mysql_config_include_files: []
A list of files that should override the default global my.cnf. Each item in the array requires a "src" parameter which is a path to a file. An optional "force" parameter can force the file to be updated each time ansible runs.
mysql_databases: []
The MySQL databases to create. A database has the values name
, encoding
(defaults to utf8
), collation
(defaults to utf8_general_ci
) and replicate
(defaults to 1
, only used if replication is configured). The formats of these are the same as in the mysql_db
module.
You can also delete a database (or ensure it's not on the server) by setting state
to absent
(defaults to present
).
mysql_users: []
The MySQL users and their privileges. A user has the values:
name
host
(defaults to localhost
)password
(can be plaintext or encrypted—if encrypted, set encrypted: yes
)encrypted
(defaults to no
)priv
(defaults to *.*:USAGE
)append_privs
(defaults to no
)state
(defaults to present
)The formats of these are the same as in the mysql_user
module.
mysql_packages:
- mysql
- mysql-server
(OS-specific, RedHat/CentOS defaults listed here) Packages to be installed. In some situations, you may need to add additional packages, like mysql-devel
.
mysql_enablerepo: ""
(RedHat/CentOS only) If you have enabled any additional repositories (might I suggest geerlingguy.repo-epel or geerlingguy.repo-remi), those repositories can be listed under this variable (e.g. remi,epel
). This can be handy, as an example, if you want to install later versions of MySQL.
mysql_python_package_debian: python3-mysqldb
(Ubuntu/Debian only) If you need to explicitly override the MySQL Python package, you can set it here. Set this to python-mysqldb
if using older distributions running Python 2.
mysql_port: "3306"
mysql_bind_address: '0.0.0.0'
mysql_datadir: /var/lib/mysql
mysql_socket: *default value depends on OS*
mysql_pid_file: *default value depends on OS*
Default MySQL connection configuration.
mysql_log_file_group: mysql *adm on Debian*
mysql_log: ""
mysql_log_error: *default value depends on OS*
mysql_syslog_tag: *default value depends on OS*
```yaml
MySQL logging configuration. Setting `mysql_log` (the general query log) or `mysql_log_error` to `syslog` will make MySQL log to syslog using the `mysql_syslog_tag`.
```yaml
mysql_slow_query_log_enabled: false
mysql_slow_query_log_file: *default value depends on OS*
mysql_slow_query_time: 2
Slow query log settings. Note that the log file will be created by this role, but if you're running on a server with SELinux or AppArmor, you may need to add this path to the allowed paths for MySQL, or disable the mysql profile. For example, on Debian/Ubuntu, you can run sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/usr.sbin.mysqld && sudo service apparmor restart
.
mysql_key_buffer_size: "256M"
mysql_max_allowed_packet: "64M"
mysql_table_open_cache: "256"
...
The rest of the settings in defaults/main.yml
control MySQL's memory usage and some other common settings. The default values are tuned for a server where MySQL can consume 512 MB RAM, so you should consider adjusting them to suit your particular server better.
mysql_server_id: "1"
mysql_max_binlog_size: "100M"
mysql_binlog_format: "ROW"
mysql_expire_logs_days: "10"
mysql_replication_role: ''
mysql_replication_master: ''
mysql_replication_user: {}
Replication settings. Set mysql_server_id
and mysql_replication_role
by server (e.g. the master would be ID 1
, with the mysql_replication_role
of master
, and the slave would be ID 2
, with the mysql_replication_role
of slave
). The mysql_replication_user
uses the same keys as individual list items in mysql_users
, and is created on master servers, and used to replicate on all the slaves.
mysql_replication_master
needs to resolve to an IP or a hostname which is accessable to the Slaves (this could be a /etc/hosts
injection or some other means), otherwise the slaves cannot communicate to the master.
If the replication master has different IP addresses where you are running ansible and where the mysql replica is running, you can optionally specify a mysql_replication_master_inventory_host
to access the machine (e.g. you run ansible on your local machine, but the mysql master and replica need to communicate on a different network)
If you want to install MySQL from the official repository instead of installing the system default MariaDB equivalents, you can add the following pre_tasks
task in your playbook:
pre_tasks:
- name: Install the MySQL repo.
yum:
name: http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
state: present
when: ansible_os_family == "RedHat"
- name: Override variables for MySQL (RedHat).
set_fact:
mysql_daemon: mysqld
mysql_packages: ['mysql-server']
mysql_log_error: /var/log/mysqld.err
mysql_syslog_tag: mysqld
mysql_pid_file: /var/run/mysqld/mysqld.pid
mysql_socket: /var/lib/mysql/mysql.sock
when: ansible_os_family == "RedHat"
This role works with either MySQL or a compatible version of MariaDB. On RHEL/CentOS 7+, the mariadb database engine was substituted as the default MySQL replacement package. No modifications are necessary though all of the variables still reference 'mysql' instead of mariadb.
On Ubuntu, the package names are named differently, so the mysql_package
variable needs to be altered. Set the following variables (at a minimum):
mysql_packages:
- mariadb-client
- mariadb-server
- python-mysqldb
If you have ansible
installed (e.g. pip3 install ansible
), none.
If you have only installed ansible-core
, be sure to require community.mysql
in your collections/requirements.yml
or install it manually with ansible-galaxy collection install community.mysql
.
- hosts: db-servers
become: yes
vars_files:
- vars/main.yml
roles:
- { role: geerlingguy.mysql }
Inside vars/main.yml
:
mysql_root_password: super-secure-password
mysql_databases:
- name: example_db
encoding: latin1
collation: latin1_general_ci
mysql_users:
- name: example_user
host: "%"
password: similarly-secure-password
priv: "example_db.*:ALL"
Author: Geerlingguy
Source Code: https://github.com/geerlingguy/ansible-role-mysql
License: MIT license
#ansible #mysql #database #playbook
1595905879
HTML to Markdown
MySQL is the all-time number one open source database in the world, and a staple in RDBMS space. DigitalOcean is quickly building its reputation as the developers cloud by providing an affordable, flexible and easy to use cloud platform for developers to work with. MySQL on DigitalOcean is a natural fit, but what’s the best way to deploy your cloud database? In this post, we are going to compare the top two providers, DigitalOcean Managed Databases for MySQL vs. ScaleGrid MySQL hosting on DigitalOcean.
At a glance – TLDR
ScaleGrid Blog - At a glance overview - 1st pointCompare Throughput
ScaleGrid averages almost 40% higher throughput over DigitalOcean for MySQL, with up to 46% higher throughput in write-intensive workloads. Read now
ScaleGrid Blog - At a glance overview - 2nd pointCompare Latency
On average, ScaleGrid achieves almost 30% lower latency over DigitalOcean for the same deployment configurations. Read now
ScaleGrid Blog - At a glance overview - 3rd pointCompare Pricing
ScaleGrid provides 30% more storage on average vs. DigitalOcean for MySQL at the same affordable price. Read now
MySQL DigitalOcean Performance Benchmark
In this benchmark, we compare equivalent plan sizes between ScaleGrid MySQL on DigitalOcean and DigitalOcean Managed Databases for MySQL. We are going to use a common, popular plan size using the below configurations for this performance benchmark:
Comparison Overview
ScaleGridDigitalOceanInstance TypeMedium: 4 vCPUsMedium: 4 vCPUsMySQL Version8.0.208.0.20RAM8GB8GBSSD140GB115GBDeployment TypeStandaloneStandaloneRegionSF03SF03SupportIncludedBusiness-level support included with account sizes over $500/monthMonthly Price$120$120
As you can see above, ScaleGrid and DigitalOcean offer the same plan configurations across this plan size, apart from SSD where ScaleGrid provides over 20% more storage for the same price.
To ensure the most accurate results in our performance tests, we run the benchmark four times for each comparison to find the average performance across throughput and latency over read-intensive workloads, balanced workloads, and write-intensive workloads.
Throughput
In this benchmark, we measure MySQL throughput in terms of queries per second (QPS) to measure our query efficiency. To quickly summarize the results, we display read-intensive, write-intensive and balanced workload averages below for 150 threads for ScaleGrid vs. DigitalOcean MySQL:
ScaleGrid MySQL vs DigitalOcean Managed Databases - Throughput Performance Graph
For the common 150 thread comparison, ScaleGrid averages almost 40% higher throughput over DigitalOcean for MySQL, with up to 46% higher throughput in write-intensive workloads.
#cloud #database #developer #digital ocean #mysql #performance #scalegrid #95th percentile latency #balanced workloads #developers cloud #digitalocean droplet #digitalocean managed databases #digitalocean performance #digitalocean pricing #higher throughput #latency benchmark #lower latency #mysql benchmark setup #mysql client threads #mysql configuration #mysql digitalocean #mysql latency #mysql on digitalocean #mysql throughput #performance benchmark #queries per second #read-intensive #scalegrid mysql #scalegrid vs. digitalocean #throughput benchmark #write-intensive
1672999928
Installs and configures MySQL or MariaDB server on RHEL/CentOS or Debian/Ubuntu servers.
No special requirements; note that this role requires root access, so either run it in a playbook with a global become: yes
, or invoke the role in your playbook like:
- hosts: database
roles:
- role: geerlingguy.mysql
become: yes
Available variables are listed below, along with default values (see defaults/main.yml
):
mysql_user_home: /root
mysql_user_name: root
mysql_user_password: root
The home directory inside which Python MySQL settings will be stored, which Ansible will use when connecting to MySQL. This should be the home directory of the user which runs this Ansible role. The mysql_user_name
and mysql_user_password
can be set if you are running this role under a non-root user account and want to set a non-root user.
mysql_root_home: /root
mysql_root_username: root
mysql_root_password: root
The MySQL root user account details.
mysql_root_password_update: false
Whether to force update the MySQL root user's password. By default, this role will only change the root user's password when MySQL is first configured. You can force an update by setting this to yes
.
Note: If you get an error like
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
after a failed or interrupted playbook run, this usually means the root password wasn't originally updated to begin with. Try either removing the.my.cnf
file inside the configuredmysql_user_home
or updating it and settingpassword=''
(the insecure default password). Run the playbook again, withmysql_root_password_update
set toyes
, and the setup should complete.
Note: If you get an error like
ERROR 1698 (28000): Access denied for user 'root'@'localhost' (using password: YES)
when trying to log in from the CLI you might need to run as root or sudoer.
mysql_enabled_on_startup: true
Whether MySQL should be enabled on startup.
mysql_config_file: *default value depends on OS*
mysql_config_include_dir: *default value depends on OS*
The main my.cnf configuration file and include directory.
overwrite_global_mycnf: true
Whether the global my.cnf should be overwritten each time this role is run. Setting this to no
tells Ansible to only create the my.cnf
file if it doesn't exist. This should be left at its default value (yes
) if you'd like to use this role's variables to configure MySQL.
mysql_config_include_files: []
A list of files that should override the default global my.cnf. Each item in the array requires a "src" parameter which is a path to a file. An optional "force" parameter can force the file to be updated each time ansible runs.
mysql_databases: []
The MySQL databases to create. A database has the values name
, encoding
(defaults to utf8
), collation
(defaults to utf8_general_ci
) and replicate
(defaults to 1
, only used if replication is configured). The formats of these are the same as in the mysql_db
module.
You can also delete a database (or ensure it's not on the server) by setting state
to absent
(defaults to present
).
mysql_users: []
The MySQL users and their privileges. A user has the values:
name
host
(defaults to localhost
)password
(can be plaintext or encrypted—if encrypted, set encrypted: yes
)encrypted
(defaults to no
)priv
(defaults to *.*:USAGE
)append_privs
(defaults to no
)state
(defaults to present
)The formats of these are the same as in the mysql_user
module.
mysql_packages:
- mysql
- mysql-server
(OS-specific, RedHat/CentOS defaults listed here) Packages to be installed. In some situations, you may need to add additional packages, like mysql-devel
.
mysql_enablerepo: ""
(RedHat/CentOS only) If you have enabled any additional repositories (might I suggest geerlingguy.repo-epel or geerlingguy.repo-remi), those repositories can be listed under this variable (e.g. remi,epel
). This can be handy, as an example, if you want to install later versions of MySQL.
mysql_python_package_debian: python3-mysqldb
(Ubuntu/Debian only) If you need to explicitly override the MySQL Python package, you can set it here. Set this to python-mysqldb
if using older distributions running Python 2.
mysql_port: "3306"
mysql_bind_address: '0.0.0.0'
mysql_datadir: /var/lib/mysql
mysql_socket: *default value depends on OS*
mysql_pid_file: *default value depends on OS*
Default MySQL connection configuration.
mysql_log_file_group: mysql *adm on Debian*
mysql_log: ""
mysql_log_error: *default value depends on OS*
mysql_syslog_tag: *default value depends on OS*
```yaml
MySQL logging configuration. Setting `mysql_log` (the general query log) or `mysql_log_error` to `syslog` will make MySQL log to syslog using the `mysql_syslog_tag`.
```yaml
mysql_slow_query_log_enabled: false
mysql_slow_query_log_file: *default value depends on OS*
mysql_slow_query_time: 2
Slow query log settings. Note that the log file will be created by this role, but if you're running on a server with SELinux or AppArmor, you may need to add this path to the allowed paths for MySQL, or disable the mysql profile. For example, on Debian/Ubuntu, you can run sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/usr.sbin.mysqld && sudo service apparmor restart
.
mysql_key_buffer_size: "256M"
mysql_max_allowed_packet: "64M"
mysql_table_open_cache: "256"
...
The rest of the settings in defaults/main.yml
control MySQL's memory usage and some other common settings. The default values are tuned for a server where MySQL can consume 512 MB RAM, so you should consider adjusting them to suit your particular server better.
mysql_server_id: "1"
mysql_max_binlog_size: "100M"
mysql_binlog_format: "ROW"
mysql_expire_logs_days: "10"
mysql_replication_role: ''
mysql_replication_master: ''
mysql_replication_user: {}
Replication settings. Set mysql_server_id
and mysql_replication_role
by server (e.g. the master would be ID 1
, with the mysql_replication_role
of master
, and the slave would be ID 2
, with the mysql_replication_role
of slave
). The mysql_replication_user
uses the same keys as individual list items in mysql_users
, and is created on master servers, and used to replicate on all the slaves.
mysql_replication_master
needs to resolve to an IP or a hostname which is accessable to the Slaves (this could be a /etc/hosts
injection or some other means), otherwise the slaves cannot communicate to the master.
If the replication master has different IP addresses where you are running ansible and where the mysql replica is running, you can optionally specify a mysql_replication_master_inventory_host
to access the machine (e.g. you run ansible on your local machine, but the mysql master and replica need to communicate on a different network)
If you want to install MySQL from the official repository instead of installing the system default MariaDB equivalents, you can add the following pre_tasks
task in your playbook:
pre_tasks:
- name: Install the MySQL repo.
yum:
name: http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
state: present
when: ansible_os_family == "RedHat"
- name: Override variables for MySQL (RedHat).
set_fact:
mysql_daemon: mysqld
mysql_packages: ['mysql-server']
mysql_log_error: /var/log/mysqld.err
mysql_syslog_tag: mysqld
mysql_pid_file: /var/run/mysqld/mysqld.pid
mysql_socket: /var/lib/mysql/mysql.sock
when: ansible_os_family == "RedHat"
This role works with either MySQL or a compatible version of MariaDB. On RHEL/CentOS 7+, the mariadb database engine was substituted as the default MySQL replacement package. No modifications are necessary though all of the variables still reference 'mysql' instead of mariadb.
On Ubuntu, the package names are named differently, so the mysql_package
variable needs to be altered. Set the following variables (at a minimum):
mysql_packages:
- mariadb-client
- mariadb-server
- python-mysqldb
If you have ansible
installed (e.g. pip3 install ansible
), none.
If you have only installed ansible-core
, be sure to require community.mysql
in your collections/requirements.yml
or install it manually with ansible-galaxy collection install community.mysql
.
- hosts: db-servers
become: yes
vars_files:
- vars/main.yml
roles:
- { role: geerlingguy.mysql }
Inside vars/main.yml
:
mysql_root_password: super-secure-password
mysql_databases:
- name: example_db
encoding: latin1
collation: latin1_general_ci
mysql_users:
- name: example_user
host: "%"
password: similarly-secure-password
priv: "example_db.*:ALL"
Author: Geerlingguy
Source Code: https://github.com/geerlingguy/ansible-role-mysql
License: MIT license
1595781840
MySQL does not limit the number of slaves that you can connect to the master server in a replication topology. However, as the number of slaves increases, they will have a toll on the master resources because the binary logs will need to be served to different slaves working at different speeds. If the data churn on the master is high, the serving of binary logs alone could saturate the network interface of the master.
A classic solution for this problem is to deploy a binlog server – an intermediate proxy server that sits between the master and its slaves. The binlog server is set up as a slave to the master, and in turn, acts as a master to the original set of slaves. It receives binary log events from the master, does not apply these events, but serves them to all the other slaves. This way, the load on the master is tremendously reduced, and at the same time, the binlog server serves the binlogs more efficiently to slaves since it does not have to do any other database server processing.
Ripple is an open source binlog server developed by Pavel Ivanov. A blog post from Percona, titled MySQL Ripple: The First Impression of a MySQL Binlog Server, gives a very good introduction to deploying and using Ripple. I had an opportunity to explore Ripple in some more detail and wanted to share my observations through this post.
Ripple supports only GTID mode, and not file and position-based replication. If your master is running in non-GTID mode, you will get this error from Ripple:
Failed to read packet: Got error reading packet from server: The replication sender thread cannot start in AUTO_POSITION mode: this server has GTID_MODE = OFF instead of ON.
You can specify Server_id and UUID for the ripple server using the cmd line options: -ripple_server_id and -ripple_server_uuid
Both are optional parameters, and if not specified, Ripple will use the default server_id=112211 and uuid will be auto generated.
While connecting to the master, you can specify the replication user and password using the command line options:
-ripple_master_user and -ripple_master_password
You can use the command line options -ripple_server_ports and -ripple_server_address to specify the connection end points for the Ripple server. Ensure to specify the network accessible hostname or IP address of your Ripple server as the -rippple_server_address. Otherwise, by default, Ripple will bind to localhost and hence you will not be able to connect to it remotely.
You can use the CHANGE MASTER TO command to connect your slaves to replicate from the Ripple server.
To ensure that Ripple can authenticate the password that you use to connect to it, you need to start Ripple by specifying the option -ripple_server_password_hash
For example, if you start the ripple server with the command:
rippled -ripple_datadir=./binlog_server -ripple_master_address= <master ip> -ripple_master_port=3306 -ripple_master_user=repl -ripple_master_password='password' -ripple_server_ports=15000 -ripple_server_address='172.31.23.201' -ripple_server_password_hash='EF8C75CB6E99A0732D2DE207DAEF65D555BDFB8E'
you can use the following CHANGE MASTER TO command to connect from the slave:
CHANGE MASTER TO master_host='172.31.23.201', master_port=15000, master_password=’XpKWeZRNH5#satCI’, master_user=’rep’
Note that the password hash specified for the Ripple server corresponds to the text password used in the CHANGE MASTER TO command. Currently, Ripple does not authenticate based on the usernames and accepts any non-empty username as long as the password matches.
Exploring MySQL Binlog Server - Ripple
It’s possible to monitor and manage the Ripple server using the MySQL protocol from any standard MySQL client. There are a limited set of commands that are supported which you can see directly in the source code on the mysql-ripple GitHub page.
Some of the useful commands are:
SELECT @@global.gtid_executed;
– To see the GTID SET of the Ripple server based on its downloaded binary logs.STOP SLAVE;
– To disconnect the Ripple server from the master.START SLAVE;
– To connect the Ripple server to the master.#cloud #database #developer #high availability #mysql #performance #binary logs #gtid replication #mysql binlog #mysql protocol #mysql ripple #mysql server #parallel threads #proxy server #replication topology #ripple server
1667747700
A collaborative curated list of awesome Ansible resources, tools, Roles, tutorials and other related stuff.
Ansible is an open source toolkit, written in Python, it is used for configuration management, application deployment, continuous delivery, IT infrastructure automation and automation in general.
Official resources by and for Ansible.
Places where to chat with the Ansible community
Tutorials and courses to learn Ansible.
Books about Ansible.
Video tutorials and Ansible training.
Tools for and using Ansible.
Best practices and other opinions on Ansible.
Awesome production ready Playbooks, Roles and Collections to get you up and running.
Author: ansible-community
Source Code: https://github.com/ansible-community/awesome-ansible
License: CC0-1.0 license
1618900707
MySQL configuration variables are a set of server system variables used to configure the operation and behavior of the server. In this blog post, we will explain the differences in managing the configuration variables between MySQL 5.7 and MySQL 8.0.
We will explain three different ways for setting the configuration variables based on your use-case. Configuration variables that can be set at run time are called Dynamic variables and those that need a MySQL server restart to take effect are called Non-Dynamic variables.
#mysql #mysql 5.7 #mysql 8.0 #mysql server