Databases
In this article:
Databases#
General information#
K2 Cloud databases are automated services which help you to quickly and easily create, configure, and manage popular DBMSs, using cloud platform tools. We currently support MongoDB, MySQL, PostgreSQL, and Redis.
MySQL is the most popular open-source relational DBMS. To learn more about MySQL DBMS capabilities, read the documentation.
PostgreSQL is a Postgres-based object-relational DBMS. It supports most of the SQL standard and offers many advanced features. To learn more about its capabilities, read the PostgreSQL documentation.
Redis is an open-source resident DBMS. It handles key-value data structures and can be used as a database, cache, message broker, or queue. To learn more about Redis capabilities, visit the official website.
MongoDB is a universal NoSQL database that stores data of any structure and supports indexing, sharding, and replication. To learn more about MongoDB features, see the MongoDB official website.
Databases in K2 Cloud#
Using K2 Cloud automated databases is a powerful alternative to installing and configuring databases manually. The database service can be deployed on a single instance or multiple instances (3- or 6-node cluster).
The cluster service option ensures high availability for critical workloads by placing nodes in multiple availability zones. If any of these nodes fails, the cluster will continue to run on the remaining nodes. This prevents data loss, and there is no interruption in service, or it is reduced to a minimum delay required for failover.
In the Databases service web interface, you can perform the following actions:
What is the Database service good for?
For quick creation of a ready-to-work database.
For simplifying database provisioning and maintenance.
For accelerating time-to-market.
Cluster with an arbitrator#
In K2 Cloud, you can launch a high-availability service in a multi-node cluster (3 or 6 depending on the service). This solves the split brain problem.
If you use a database cluster and connectivity between nodes is lost, the cluster gets partitioned. Each part of the cluster decides whether to continue processing data or stop. If each part of the cluster considers itself the only survivor and continues running and changing data, then there is the split-brain problem that can cause data corruption. To avoid this, clusters use a quorum rule stating that only one part of the cluster (the one with a larger number of nodes) continues operating. This is only possible if the total number of nodes in the cluster is odd.
However, running three full-fledged nodes is redundant in terms of resources involved. Therefore, a cluster with two full-featured nodes and an arbitrator is more cost-effective, since the arbitrator neither stores nor processes data and uses minimum resources to maintain a cluster forum.
Before you begin#
To be able to work with the database service, a user needs to have PaaSServicePolicy project grants. For instance, such grants are available to administrators in CloudAdministrators group. If necessary, you can create a separate user, add the user to the project, and either attach the PaaSServicePolicy policy to the user or add the user to the group of cloud administrators in this project.
Note
If database backup is required, then the PaaSBackupPolicy policy should be attached to the user (see to learn more about IAM policies).
In addition, the project should have the following resources:
Мы советуем также ознакомиться с рекомендациями по работе с сервисом в облаке.
Creating database service#
The database service creation process can be divided into two parts:
infrastructure creation;
software installation and configuration.
Currently, the infrastructure creation consists of launching the required number of instances with the specified configuration from a given image and setting up security groups so that the databases can connect to each other.
After the successful launch of the instances, software installation inside VMs begins. Upon the successful installation, the database service enters the running status. Any other status means that the creation process is not completed. The database service in the running status is ready for operation.
To configure the service, select the DBMS in the Service store or Running services subsection, click Create and follow the steps below:
Set the required network parameters:
The configuration of the cluster in which the service will be deployed. The following options are available for selection:
single-node (non-high-availability service);
three notes in one availability zone;
three nodes in three availability zones;
six nodes in a single availability zone (only Redis);
six nodes in three availability zones (only Redis).
VPC where the database service will be deployed.
Security groups to control traffic through interfaces of the instances on which the database service will run.
Subnets to which instances with the running service will be attached, or network interfaces through which cluster nodes will be attached to subnets.
Note
To run the service in the selected VPC, you must first create a subnet in the preferred availability zone (in a configuration with one zone), or one subnet in each availability zone (in a configuration with three zones). In addition, the same volume types must be supported in the availability zones used.
Note
The ability to attach network interfaces may be useful, for example, when you need to recreate a database service. If you delete the service, but do not delete attached network interfaces, you will be able to reuse them for connecting nodes of the new cluster to subnets, where the new cluster will be deployed. Thus, you can keep previous network settings, such as private IP addresses and security groups, rather than configure them again.
Internal and/or internet-facing load balancers, when a multi-node configuration is selected. This functionality is available for MySQL, PostgreSQL and Redis database services (for details, see Load balancer management).
Specify the configuration of the instances where the database nodes will run. Select the instance type and parameters of its volumes: type, size and IOPS (if available for the type you choose).
For MySQL, PostgreSQL and MongoDB DBMSs, you can set a configuration with cluster arbitrator. To enable it, select the Use arbitrator option.
Note
Database performance depends on the selection of node components. We recommend using high performance volumes gp2: Universal (SSD) or io2: Maximum (SSD).
In addition, you can also specify an SSH key. In this case, after automatic service setup, you will get an SSH access to the respective instances.
Attention
We provide the option to connect to instances using an SSH key while the new database service is beta testing. This feature may be disabled in the future.
Set the main parameters of the service (the specific set of options depends on the selected service).
Service — an arbitrary unique name of the database service.
Vendor — MySQL DBMS engine. Valid values: mariadb, percona, mysql.
Version — The DBMS engine version. Depending on the engine vendor, the following versions are supported:
Vendor
Version
mariadb
10.2.44
10.3.35
10.4.25
10.5.16
10.6.8
10.7.7percona
5.7.38
8.0.28mysql
5.7.41
8.0.32Enable monitoring – For centralized monitoring of a database service, first deploy the Prometheus-based monitoring service.
Monitoring service – The selection of a monitoring service is only available when the Enable monitoring checkbox is checked.
Monitoring labels — Optionally, you can set monitoring labels, which the installed monitoring agents will assign to the collected metrics (for details, see Using labels). Labels can only be set when the Enable monitoring checkbox is selected.
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — an arbitrary unique name of the database service.
Version — The supported DBMS versions are 10.21, 11.16, 12.11, 13.7, 14.4, 15.2.
Replication option — The replication mode in the Patroni cluster. Available only when the three-node configuration is selected.
Valid values: asynchronous, synchronous, synchronous_strict.
Enable monitoring – For centralized monitoring of a database service, first deploy the Prometheus-based monitoring service.
Monitoring service – The selection of a monitoring service is only available when the Enable monitoring checkbox is checked.
Monitoring labels — Optionally, you can set monitoring labels, which the installed monitoring agents will assign to the collected metrics (for details, see Using labels). Labels can only be set when the Enable monitoring checkbox is selected.
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — an arbitrary unique name of the database service.
Version — The supported DBMS versions are 3.6.23, 4.0.28, 4.2.23, 4.4.17, 5.0.13.
Enable monitoring – For centralized monitoring of a database service, first deploy the Prometheus-based monitoring service.
Monitoring service – The selection of a monitoring service is only available when the Enable monitoring checkbox is checked.
Monitoring labels — Optionally, you can set monitoring labels, which the installed monitoring agents will assign to the collected metrics (for details, see Using labels). Labels can only be set when the Enable monitoring checkbox is selected.
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Service — an arbitrary unique name of the database service.
Version — The supported DBMS versions are 5.0.14, 6.2.6, 7.0.11.
Clustering option. Valid values: native, sentinel. Available only when a three node cluster configuration is selected. For a six node cluster configuration, only the native option is supported.
RDB Persistence — The RDB storage mode.
AOF Persistence — The AOF storage mode.
Enable monitoring – For centralized monitoring of a database service, first deploy the Prometheus-based monitoring service.
Monitoring service – The selection of a monitoring service is only available when the Enable monitoring checkbox is checked.
Monitoring labels — Optionally, you can set monitoring labels, which the installed monitoring agents will assign to the collected metrics (for details, see Using labels). Labels can only be set when the Enable monitoring checkbox is selected.
Enable logging – For centralized PaaS service logging, first create a logging service. Once a PaaS service is created, you will be able to install logging agents manually only.
Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.
Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is selected.
Password — User password 8 - 128 characters long. It can be set manually or generated automatically.
Configure backup.
Note
This step is currently available for MySQL and PostgreSQL only.
When you select Enable backup, the following settings are available:
Bucket where the recovery points will be stored.
User with PaaSBackupUser privileges; backups will be written to the bucket under this user.
Daily backup start time.
Backup retention period.
Email (optional) – the email address to which notifications will be sent if the backup fails. If not specified, notifications will be sent to the email address of the user you have selected above.
Specify additional parameters, that will become part of the DBMS configuration (optional). The supported parameters depend on the type of database service you launch. You can read more about supported parameters in the recommendations for a particular database.
You can also specify advanced settings of your choice at this step. To do this, click advanced settings and specify the desired settings and their values.
Important
The specified parameters and settings will be part of the DBMS configuration and, therefore, will affect its operation. Add only the parameters you really need.
After completing all the steps, click Create.
Note
The database creation process usually takes 5 to 15 minutes. Single-node installations are deployed faster than multi-node ones.
To ensure the correct operation of a database service, a new security group is automatically created together with the cluster. It includes the following rules:
the rule to permit inbound traffic from interfaces that are in the same security group;
the rule to enable all outbound IPv4 traffic.
If the database service is deleted, the security group will also be deleted.
User and database management#
You can create and delete databases and users, as well as edit some of their settings, directly from the cloud web interface.
Creating and deleting users and databases is only possible for services in the Running or Updating state. These actions are available for MySQL, PostgreSQL and MongoDB databases.
Create user#
To create a new user:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Users tab and click Create.
Set the required parameters:
Name — User name. It should start with a Latin letter, should not exceed 32 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as root, mysql, monitoring, mariadb.sys, clustercheck.
Host – name of the host from which the user is allowed to connect to the database. For MySQL, you can also specify the IP address of the host from which the user is allowed to connect to the database. If this field is left blank, the user will be allowed to connect from any IP address and the default value (%) will be set as the host.
Password – User password.
Name — User name. It should start with a Latin letter, should not exceed 31 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as os_admin, postgres, root, replicator.
Password – User password.
Name — User name. It should start with a Latin letter, should not exceed 63 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as admin, root.
Password – User password.
Click Create.
Create a database#
To create a new database:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab and click Create.
Specify a database name and other parameters. The exact set of parameters depends on the selected database service.
Name — Database name. It should start with a Latin letter, should not exceed 64 characters, can consist of numbers, Latin letters, and $ and “_” characters. You cannot use reserved names, such as information_schema, mysql, performance_schema.
Charset – database encoding.
Collation — Database collation parameters or collation parameter normalization operation.
Enable backup — Enables daily backup. Backups will only be created if this option is enabled for the service.
Name — Database name. It should start with a Latin letter, should not exceed 31 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use reserved names, such as postgres, template0, template1.
Owner — The database owner. This must be one of existing users added at the create user step. This user cannot be deleted as long as it remains the database owner.
Locale — The database locale.
Encoding — The database encoding.
Extensions — The extensions to be installed for the database.
Enable backup — Enables daily backup. Backups will only be created if this option is enabled for the service.
Name — Database name. It should start with a Latin letter, should not exceed 63 characters, can consist of numbers, Latin letters, and underscore “_”. You cannot use the reserved name admin.
You can grant users access to a database right away or later. To add users later and create a database right away, click Create. To add users, click Edit Users to go to the next step.
Click Add User and select the previously created user from the list. For MySQL database, you can also assign the necessary rights to the user. For PostgreSQL, you cannot add the database owner as a user.
Privileges — The user privileges in the database.
Options — Operations that the user can perform on the database.
To add other users, click Add User.
Click Create.
Change user password#
To change a user’s password after creating it, follow these steps:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Users tab, select the user whose password you want to change, and click Change.
In the dialog window, set or generate a new password.
Click Save.
Change database settings#
Once database has been created, you can modify some of its settings and create/delete users. To do this:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab, find the database whose settings you want to change, and click Change.
Specify new parameters for the database.
Note
Once a database has been created, its name and language settings cannot be modified. Which parameters you can change depend on the selected database service (MySQL or PostgreSQL).
To allow or deny user access to the database, go to the next step by clicking Edit Users. If no user has been previously granted access to the database, click Add User.
Note
For PostgreSQL, you cannot add the database owner as a user.
To replace one user with another, select the user from the list.
Для запрета доступа пользователю к базе данных нажмите на иконку рядом с именем пользователя.
To grant access to the database to additional users, click Add User.
For MySQL and MongoDB users, you can also change their access rights.
Click Save.
The database service switches to the Updating state. Users can continue working with the database as long as the changes do not affect their database access privileges.
Delete user#
To delete a user:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Users tab and select the user you want to delete.
Click Delete.
In the dialog window, confirm the deletion.
Note
The user will also be removed from all databases to which he/she was granted access.
Delete database#
To delete a database:
Go to the section PaaS Running services.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab and select the database you want to delete.
Click Delete.
In the dialog window, confirm the deletion.
Database backup and recovery#
Scheduled backups can be enabled for a database service so that full backups of the specified databases are created daily at the specified time. If necessary, you can quickly restore the desired database as of a specific date from the saved backup.
Note
Backup is now only available for PostgreSQL and MySQL database services. In the future, we plan to deploy it for other database services.
To enable backup, configure the schedule for the database service and enable the appropriate option for each database you want to backup. Backups are only created for running services and are written to the object storage.
To free up space from no-longer-needed backups, we recommend limiting their retention period or regularly deleting them. Certain backups can be protected from automatic deletion.
Along with the backup, a log is created to store information about the backup utility operation. If the backup has failed, you can use the log to find out why. We recommend checking the log for warnings on successful backups also to make sure that there were no problems during the backup process.
Attention
To avoid backup problems caused by lack of space for temporary files, we recommend that you leave 20 MB of free space in the root partitions on cluster instance volumes.
Information about database services for which backups are available can be found in the Service backups subsection of the PaaS section.
Note
Before configuring backup, make sure that the project has both a bucket for storing backups and a user with PaaSBackupPolicy privileges. This is a special user, and only this policy should be assigned to him/her.
Setup service backup#
Backup can be enabled when you create a database service. If you have not done this and want to enable it, or if you need to change the settings for an already enabled backup, then:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
In the Information tab, click Backup settings.
If backup has not been enabled, check the Enable backup checkbox.
Set or change backup settings:
Bucket where the recovery points will be stored.
User with PaaSBackupPolicy privileges; backups will be written to the bucket under this user.
Daily backup start time.
Backup retention period.
Email (optional) – the email address to which notifications will be sent if the backup fails. If not specified, notifications will be sent to the email address of the user you have selected above.
To confirm, click Change.
Note
To back up specific databases, enable the respective option for each of them.
Enable database backup#
Note
Database backups will only be created if backup for the database service is enabled.
If you want to enable backup for a specific database:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab and select the database in the resource table.
Click Edit.
Check the Enable backup checkbox, while leaving all other parameters unchanged.
Click Save.
Recover a database from a backup#
Important
To avoid problems when recovering the database, we recommend that you recover it to a service with the same DBMS version as that used to create the database.
To recover a database:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab and click Create from backup.
In the database creation wizard that opens, select the required backup from the list. To simplify the search, you can select the service from which originates the database, whose backup you are looking for. Click Select to go to the next step.
Specify the database parameters and, if necessary, add users. These steps are the same as the steps 4-6 when creating a database and are described in more detail in database creation instructions.
Attention
If parameters other than default ones were specified when creating the database, such as PostgreSQL locale, we recommend that you enter the same parameters for the correct recovery.
Disable backup#
To disable backup for a specific database:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Databases tab and select the database in the resource table.
Click Edit.
Uncheck the Enable backup checkbox, while leaving all other parameters unchanged.
Click Save.
To disable backup for all databases:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
In the Information tab, click Backup settings.
Uncheck the Enable backup checkbox.
To confirm, click Change.
Delete backups#
You can delete either specific or all unnecessary backups.
To delete specific backups:
Go to PaaS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click the service ID. It corresponds to the name of the folder in the bucket where database backups for this service are stored.
Open the Backups tab and select the backups for deletion in the resource table.
Click Delete and confirm the action in the dialog window.
Go to PaaS Running services.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Go to the service page and open the Backups tab.
In the resource table, select the backups to be deleted.
Click Delete and confirm the action in the dialog window.
To delete all backups:
Go to PaaS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click Delete service backups and confirm the action in the dialog window.
Go to PaaS Running services.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Go to the service page and open the Backups tab.
Click Delete all backups and confirm the action in the dialog window.
Go to PaaS Service backups.
In the resource table, select the service whose database backups you want to delete. To speed up the search, you can select the service category or use table search.
Click the service ID. It corresponds to the name of the folder in the bucket where database backups for this service are stored.
Open the Backups tab.
Click Delete all backups and confirm the action in the dialog window.
Note
When you delete all the backups of the service, the folder that was automatically created in the bucket to store them is also deleted.
Automatic backup deletion#
In the backup settings, you can set the backup retention period for the service. By default, backups are retained permanently. You can specify the number of days, weeks, months or even years for how long the created backups should be retained.
Individual backups can be protected from automatic deletion. To do this, find the required backup in the list of backups and turn on the Deletion protection switch.
Note
When you delete a database service, automatic deletion of backups will no longer be performed. Backups of the deleted service will be retained permanently. In this case, no-longer-needed backups will have to be deleted manually.
Load balancer management#
For MySQL, PostgreSQL and Redis services in a multi-node high-availability configuration, you can create an internal and/or internet-facing load balancer (for details, see load balancers). They automatically distribute incoming requests among fully functional cluster nodes (requests are not sent to the arbitrator if there is one).
Load balancers are created automatically; their parameters and associated resources cannot be changed. Information about the created load balancers, in particular their DNS names, can be found in the Load balancers tab on the service page.
Attention
A load balancer running together with a PaaS service can be deleted only on the page of this service.
Important
To run the internet-facing load balancer, give external access to HAProxy ports, which is denied by default. Add enabling rule for tcp/5000 port to the security group that was specified when creating the service.
Depending on the service, you may also need to open access to other ports. You can see the ports listened to by the service in the Information tab on its page.
Create a load balancer#
A load balancer for a database service can only be created when the service is in Running status. You can create one internal and one internet-facing load balancer per service.
Important
You can create an internal load balancer only if route propagation is enabled in VPC.
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Load balancers tab and click Create.
In the window that opens, select a balancer you want to create. If none has been created yet, you can create both internal and internet-facing balancers at once.
Click Create to complete the action.
Delete load balancer#
Load balancer associated with the service can only be deleted when the service is in Running status.
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Load balancers tab and click Delete.
In the window that opens, select a load balancer you want to delete. If two load balancers have been created for the service, you can delete both at the same time.
Click Delete to confirm the action.
Database service configuration#
If you have not enabled monitoring and/or logging when creating the database service, or you want to disable them, you can do it when the service is in the Ready state. You can also change Redis and MongoDB additional parameters.
Note
To enable monitoring and logging, first deploy Prometheus-based monitoring service and ELK-based logging service.
Important
If an attempt to modify some settings fails, then the service will be reset to default ones.
To set up the database service:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
Open the Parameters tab and click Modify.
In the window that opens, you can configure monitoring and logging (or disable them if they are already enabled). You can also change RDB Persistence and AOF Persistence parameters for the Redis service.
To set up other parameters for Redis and MongoDB services, click Additional parameters.
You can also change advanced settings at this step. To do this, click advanced settings and specify the desired settings and their values.
To save new settings, click Modify.
Environment upgrade#
PaaS services are updated regularly. If you want the already installed database service to support new features, then update its environment. For the current environment version, see the Information tab on the service page.
Note
All services with environment version 3_6 and higher support environment update. It is also available for some previously deployed services with environment version 3_5. To check if you can update them, use the API method DescribeService: the response should contain the common:update_environment value in the SupportedFeatures list.
Important
If an attempt to update the environment fails, the service will be reset to the default one.
To update the environment version:
Go to the PaaS section Running services and open the Databases tab.
Find the desired service in the table and click the service ID to go to its page.
In the Information tab, click Update environment version.
In the window that opens, select the version from the list, to which you want to upgrade the current environment.
Click Update to change the version.
Deleting database service#
Deleting the database service deletes all instances and volumes created with it.
To delete a service, go to PaaS Running services and click the icon in the table or the Delete button on the service page. If you want to use network interfaces in the future, for example, to recreate a service with the same network parameters, then in the dialog window, clear Delete associated network interfaces checkbox.
Note
When a service is deleted, stored backups of its databases are retained.