Message brokers#

General information#

The message broker service enables messaging between various applications, systems, and services in the cloud. It allows applications to communicate with each other via asynchronous messages, without the need for a permanent connection or knowledge of their reciprocal location. K2 Cloud supports popular message brokers Apache Kafka and RabbitMQ.

Apache Kafka#

Apache Kafka is a high-performance and reliable message broker designed to process large amounts of data in real time. It allows distributed system components to exchange messages quickly and efficiently.

Its distributed architecture makes Apache Kafka highly available and efficient for mission-critical applications. It guarantees message delivery and supports various data processing models, including streaming and batch processing. These features make Apache Kafka a frequent choice in various domains that require fast and reliable exchange of large amounts of data.

Apache Kafka offers a flexible system of topics and partitions that allow you to split the message stream into separate sub-streams, thus simplifying data processing. This structure provides higher throughput and allows users to scale the system to their requirements. The broker supports both individual messages and message batches, which makes it possible to optimize the performance and efficiency of data transfer.

Apache Kafka’s replication system provides protection against failures and data loss, while certain mechanisms such as commit log and consumer offsets help control the message processing and ensure data processing consistency.

Apache Kafka has become the standard for real-time data exchange across many industries and applications. This powerful and flexible message processing system continues to evolve and improve, offering new capabilities to process and analyze large amounts of data.

K2 Cloud supports the following versions of Apache Kafka:

  • 3.6.1;

  • 3.7.0.

RabbitMQ#

RabbitMQ is a popular open source message broker. It serves as an intermediary in the message transmission between information system components. Thanks to the broker, they no longer need to monitor message delivery status, resend messages or wait for the transmission to complete, since the broker solves all these tasks. Thus, you can improve overall system performance by reducing the number of jobs done by other system components and time spent on their execution.

RabbitMQ offers rich functionality for customizing and optimizing the messaging process according to the system needs. For example, you can use different protocols to send and receive messages, set up routing rules, specify message handling priorities, and more.

Thanks to a lot of messaging protocols supported, RabbitMQ can be used in multi-protocol systems. In particular, it supports AMQP (Advanced Message Queuing Protocol), MQTT (Message Queuing Telemetry Transport), STOMP (Streaming Text Oriented Messaging Protocol), and XMPP (Extensible Messaging and Presence Protocol).

In addition, RabbitMQ features high degree of reliability and availability. Built-in replication and clustering mechanisms allow you to create message-intensive HA systems.

K2 Cloud supports the following RabbitMQ versions:

  • 4.2;

  • 4.3.

Billing#

The PaaS message broker services Kafka and RabbitMQ are free of charge. Payment is charged for actually consumed billable resources: instances, volumes, Elastic IP addresses, and other resources.

Before you begin#

We advise to preliminarily read the recommendations on how to work with the service in the cloud.

Required privileges#

To be able to work with the broker service, a user needs to have PaaSFullAccess project grants. For instance, such grants are available to administrators in the CloudAdministrators group. If necessary, you can create a separate user, add the user to the project, and either attach the PaaSFullAccess policy to the user or add the user to the group of cloud administrators in this project.

The PaaSFullAccess project grants provide the user with full permissions to all PaaS services. To allow only specific actions with the message broker service for a user, create a policy with the appropriate grants and assign it to the user. For details, see the IAM documentation on how to create a policy and assign it to the user.

Required resources#

In addition, the project should have the following resources:

Launching a message broker service#

To launch the service, go to the Service store or Running services subsection, select the service in the Message brokers tab and click Create.

Important

The message broker service can be launched only in those subnets that have Internet access. To configure such access, you might need to attach an internet gateway to a VPC and create a default route in a route table.

Note

Creating a high-availability service also automatically creates placement groups for every three nodes. For example, in case of a five-node cluster in one availability zone, two placement groups are created, containing three and two nodes, respectively. Should it be a five-node cluster across three availability zones, one placement group is created in each zone, with two groups containing two nodes each, and one group containing only one node. Instances from the same group are deployed on different physical servers, which improves the service availability.

The service launch procedure comprises the following stages:

  1. Set the required network parameters:

    Note

    To run the service in the selected VPC, you must first create a subnet in the preferred availability zone (in a configuration with one zone), or one subnet in each availability zone (in a configuration with three zones). In addition, the same volume types must be supported in the availability zones used.

    Note

    The ability to attach network interfaces may be useful, for example, when you need to recreate the cluster where the message broker has been deployed. If you delete a service, but do not delete attached network interfaces, you will be able to reuse them for connecting nodes of a new cluster to subnets, where the new cluster will be deployed. Thus, you can keep previous network settings, such as private IP addresses and security groups, rather than configure them again.

    Important

    It is not recommended to create a four-node cluster as otherwise any connectivity loss and further split-brain may lead to quorum loss.

    • To configure a cluster, where the service will be deployed, you can select the following number of nodes:

      • one master node (the service without high availability);

      • from three to nine master nodes 1.

      1

      We recommend selecting this configuration if the cluster scalability may be required in the future.

    • VPC where the message broker service will be deployed.

    • Security groups to control traffic through interfaces of the instances on which the message broker service will run.

    • Subnets to which instances with the running service will be attached, or network interfaces through which cluster nodes will be attached to subnets.

      Note

      When selecting subnets, you may specify either one or three subnets in different availability zones. In the former case, the service will be deployed in the same availability zone as the selected subnet.

      When selecting network interfaces for a multi-node configuration, you must specify as many interfaces as there are nodes in the cluster.

      Network interfaces should be evenly distributed across all availability zones where the service will be deployed, if possible. For example, if you plan to create a cluster with five master nodes across three availability zones, you should specify two interfaces in each of two availability zones and one interface in the remaining zone.

    • Internal and/or internet-facing load balancers (for details, see Load balancer management).

      In the DNS name field, you can specify an arbitrary subdomain name in the paas.<region>.web.c2.croc.ru DNS service zone for an Internet-facing load balancer and in the paas.<vpc-id>.internal zone for an internal load balancer. The name may contain Latin letters, digits, and hyphens.

      If the DNS name field is left blank, the name will be specified automatically.

    Important

    It is not recommended to create a cluster with four broker coordinator nodes as otherwise any connectivity loss may lead to quorum loss due to split brain.

    • To configure a cluster, where the service will be deployed, you can select the following number of nodes:

      • one broker coordinator node (non-HA service);

      • from three to nine broker coordinator nodes; 2

      • from three to nine broker nodes and three coordinator nodes. 2

        2(1,2)

        We recommend selecting one of these configurations if the cluster scalability may be required in the future.

    • VPC where the message broker service will be deployed.

    • Security groups to control traffic through interfaces of the instances on which the message broker service will run.

    • Subnets to which instances with the running service will be attached, or network interfaces through which cluster nodes will be attached to subnets.

      Note

      When selecting subnets, you may specify either one or three subnets in different availability zones. In the former case, the service will be deployed in the same availability zone as the selected subnet.

      When selecting network interfaces for a multi-node configuration, you must specify as many interfaces as there are nodes in the cluster.

      Network interfaces should be evenly distributed across all availability zones where the service will be deployed, if possible. For example, if you plan to create a cluster with five brokers and three coordinators across three availability zones, you should specify three interfaces in each of two availability zones and two interfaces in the remaining zone.

    Click Next to proceed to the next step.

  2. If a multi-node cluster was selected for RabbitMQ or Kafka in the previous step, you can specify the number of nodes in the cluster: master nodes for RabbitMQ, or, depending on the chosen configuration, coordinator broker nodes or broker nodes for Kafka. However, if network interfaces were specified, the number of instances is fixed and equal to the number of selected network interfaces.

    Set the master node parameters for RabbitMQ, or set the broker or broker-coordinator parameters for Apache Kafka:

    • instance type;

    • data volume configuration: type, size and IOPS (if available for the type);

    For the Apache Kafka service in a three-coordinator configuration, you need to also set parameters for coordinator nodes.

    Click Next to proceed to the next step.

  3. Set the main service parameters:

    • Name tag — Optional arbitrary description of the service.

    • Service name – any unique name for the caching service.

    • Version.

    • Whether to enable monitoring. For centralized monitoring of a message broker service, deploy the Prometheus-based monitoring service first.

      • Monitoring service – The selection of a monitoring service is only available when the Enable monitoring checkbox is checked.

      • Monitoring labels — Optionally, you can set monitoring labels, which the installed monitoring agents will assign to the collected metrics (for details, see Using labels). Labels can only be set when the Enable monitoring checkbox is selected.

    • Whether to enable logging. For centralized PaaS service logging, create a logging service first. Once a PaaS service is created, you will be able to install logging agents manually only.

      • Logging service – The selection of a logging service is only available when the Enable logging checkbox is checked.

      • Logging tags – Tags are assigned to logs when the latter are imported by the logging service. They simplify search for the required logs. Tags can only be set when the Enable logging checkbox is checked.

    • User password (for RabbitMQ only). It can be set manually or generated automatically.

    Click Next to proceed to the next step.

  4. Set the maintenance window (see подробнее). If scheduled maintenance can be performed at any time, leave the Maintenance window option set to Custom. If there are time restrictions when maintenance can be performed, select the interval that is convenient for you. To do so, specify the following parameters:

    • the Scheduled value for the Maintenance window option;

    • a day of the week for the maintenance to take place;

    • hour interval for the maintenance window.

    To set additional parameters or tags for the service, click the respective button to move to the required step — Additional parameters or Add tags. If neither need to be set, click Create to launch the service.

  5. Set the advanced settings, if necessary. Click advanced settings and enter the settings and their values.

    Important

    The specified settings will be a part of the service configuration and, therefore, will affect its operation. Add only the settings you really need.

    To set tags for the service, click Add tags to go to the next step. If no tags are required, click Create to launch the service.

  6. To add a tag, click Add tag and enter tag key and value.

    Note

    If no tags have been set yet, you can add the Name tag by clicking Add Name tag and specifying its value.

    When done, click Create to launch the service.

Important

The service launching process usually takes 5 to 15 minutes.

Modifying service node options#

Add nodes for the service#

If a multi-node cluster configuration was selected when creating a message broker service and the cluster has fewer than the maximum possible number of nodes, you can scale it out by adding extra nodes. Scaling is only possible when the service is in the Running state.

Important

For the message broker service created before 30.01.2026, cluster scaling is not supported.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Information tab and click Add instances.

  4. In the window that opens, specify the number of instances to be added. If network interfaces were specified at the service creation, you can either create interfaces automatically by pushing Create automatically or specifying the existing ones by pushing Select manually and selecting the interfaces in the specified availability zones.

  5. Click Add to complete the operation.

Modify the instance type for the node#

You can modify the instance type for cluster nodes running the message broker service. You can do it only when the service is in the Ready state.

To modify the instance type:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. In the Information tab, edit the Instance type field in the configuration section of master node for a Redis service or broker, broker-coordinator and/or coordinator for a Kafka service. To do this, click the edit icon and selecting the instance type.

  4. Click to save changes.

Modify data volume parameters#

You can only change the data volume parameters when the service is in the Ready state.

You can configure the data volume size and its performance in IOPS right from the cloud web interface.

Important

You can change IOPS only for a data volume of the type io2.

To change the volume parameters, follow these steps:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. In the Information tab, edit the required fields (Data volume size and/or Data volume IOPS) by clicking the edit icon .

    Important

    The new data volume size must exceed the current one.

  4. Click to save changes.

Managing Kafka topics#

For an Apache Kafka service, you can create and set up topics.

Creating a topic#

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Topics tab and click Create.

  4. In the window that opens, set the following required parameters:

    • Topic name.

    • Partitions (not more than that of brokers).

    • Replication factor (not more than that of brokers).

  5. If you need to set other parameters, proceed to the next step by clicking Additional parameters. The list of supported parameters and their valid values can be found on the page Kafka topic parameters.

  6. After setting all the required parameters, click Create.

Editing a topic#

To change the topic parameters, follow these steps:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Topics tab, select the topic in the resource table and click Modify.

  4. You can modify the following parameters of the topic:

    • Partitions (can only be increased).

    • Number of the partition replicas.

  5. If you need to modify the values of other parameters, proceed to the next step by clicking Additional parameters. The list of supported parameters and their valid values can be found on the page Kafka topic parameters.

  6. After setting all the required parameters, click Save.

Deleting a topic#

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Topics tab, select the topic in the resource table and click Delete.

  4. In the dialog window, confirm the action.

Load balancer management#

For RabbitMQ service, you can create an internal and/or internet-facing load balancer (for details, see load balancers). They automatically distribute incoming requests among cluster nodes.

Load balancers are created automatically; their parameters and associated resources cannot be modified. To modify the DNS name of an existing load balancer, delete it first and then create a new one. For information about the created load balancers and, in particular, the service addresses, open the Load balancers tab on the service page.

Attention

A load balancer running together with a PaaS service can be deleted only on the page of this service.

Important

To run the internet-facing load balancer, give external access to HAProxy ports, which is denied by default. Add enabling rule for tcp/5000 port to the security group that was specified when creating the service.

Create a load balancer#

A load balancer for a message broker service can only be created when the service is in Running status. You can create one internal and one internet-facing load balancer per service.

Important

You can create an internal load balancer only if route propagation is enabled in VPC.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Load balancers tab and click Create.

  4. In the window that opens, select a balancer you want to create. If none has been created yet, you can create both internal and internet-facing balancers at once.

    In the DNS name field, you can specify an arbitrary subdomain name in the paas.<region>.web.c2.croc.ru DNS service zone for an Internet-facing load balancer and in the paas.<vpc-id>.internal zone for an internal load balancer. The name may contain Latin letters, digits, and hyphens.

    If the DNS name field is left blank, the name will be specified automatically.

  5. Click Create to complete the action.

Delete load balancer#

Load balancer associated with the service can only be deleted when the service is in Running status.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Load balancers tab and click Delete.

  4. In the window that opens, select a load balancer you want to delete. If two load balancers have been created for the service, you can delete both at the same time.

  5. Click Delete to confirm the action.

Configuring a message broker service#

If you have not enabled monitoring and/or logging when creating the message broker service, or you want to disable them, you can do it when the service is in the Ready state. You can also change the password and advanced settings.

Note

To enable monitoring and logging, first deploy Prometheus-based monitoring service and ELK-based logging service.

Important

If an attempt to modify some settings fails, then the service will be reset to default ones.

To set up the message broker service:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Parameters tab and click Modify.

  4. In the window that opens, you can generate or set a new password and configure monitoring and logging (or disable them if they are already enabled).

  5. To change the advanced settings, click Additional parameters. Then click Advanced settings and set the new settings and their values.

  6. To save settings, click Modify.

Assigning tags#

Important

Tags are only supported for PaaS services with an fm-cluster-xxxxxxxxxxx ID. If you want to add tag support for the installed services with a UUID as an ID, please contact the support service.

Tags allow you to classify PaaS services by various attributes, such as purpose, owner, etc. Their usage simplifies the management of many similar resources.

To add, modify or delete tag for the message broker service:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. In the resource table, find the service for which you want to assign tags, and click on its ID to go to the service page.

  3. Open the Tags tab.

  4. To add a tag, click Add tag and specify the Key and Value fields.

    To modify a tag, edit the required fields (Value and/or Key) of the respective tag.

    To delete a tag, click the icon next to the tag you no longer need.

    Note

    If no tags have been set earlier, you can add the Name tag by clicking Add Name tag.

  5. Click Apply to save the changes.

Service start and stop#

You can stop the message broker service, if you don’t need it, and start it when necessary, thus saving money on computing resources, because only attached volumes are billed when the service is stopped.

You can also restart the service in case of an error, and synchronize the service status with those of the instances where it is installed.

Stop service#

The service can be stopped when it is in the Running or Error state.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. In the resource table, find the service you want to stop and click the shutdown icon in the Actions field. Alternatively, click the service ID to go to its page and then click Stop in the Information tab.

  3. In the dialog window, confirm the action.

Restart the service#

Note

You can only restart services with the environment version 3.6 or higher. You can find out the current version of the environment in the Information tab on the service page.

The service can be restarted when it is in the Running or Error state.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. In the resource table, find the service you want to restart and click the restart icon in the Actions field. Alternatively, click the service ID to go to its page and then click Restart in the Information tab.

  3. In the dialog window, confirm the action.

Synchronize the service with instances#

Synchronization allows you to match the states of the service and the instances on which it is installed, when instances were manually started after the service was stopped. It can also be used to match the states if an error occurred when stopping, starting, or restarting the service and instances had to be manually stopped/started.

Attention

Synchronization is not possible if the error cannot be eliminated (the error status can be viewed, for example, using the DescribeService method).

  1. Go to PaaS Running services and open the Message brokers tab.

  2. In the resource table, find the service you want to synchronize and click the synchronization icon in the Actions field. Alternatively, click the service ID to go to its page and then click Synchronize with confirmation in the Information tab.

  3. In the dialog window, confirm the action.

Start the service#

You can only start the service, when it is in the Stopped state.

  1. Go to PaaS Running services and open the Message brokers tab.

  2. In the resource table, find the service you want to start and click the start icon in the Actions field. Alternatively, click the service ID to go to its page and then click Start in the Information tab.

  3. In the dialog window, confirm the action.

Configuring the maintenance window#

During scheduled maintenance, critical security patches and other important updates are installed.

Attention

During maintenance, operations with the service, including its deletion, are not available, but the service itself will continue to operate. In some cases, it may be necessary to stop and restart the cluster nodes.

Maintenance is carried out strictly during a preset time period (if a maintenance window is specified). All actions of K2 Cloud specialists during maintenance are logged.

Note

You can modify the maintenance window settings when the service is in the Running state only.

To modify the maintenance window settings:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. In the Maintenance tab, click Set up.

  4. In the window that opens, specify the maintenance interval that is convenient for you. To do so, select the following parameters:

    • the Scheduled value for the Maintenance window option;

    • a day of the week for the maintenance to take place;

    • hour interval for the maintenance window.

    Or select the Custom value for the Maintenance window option, if maintenance can be performed at any time.

  5. Click Save to modify the maintenance window.

Viewing metrics#

Depending on the service type, you can view the following metrics:

Metrics

Description

rabbitmq_channelsTotal

Total number of channels

rabbitmq_connectionsTotal

Total number of connections

rabbitmq_consumersTotal

Total number of consumers

rabbitmq_exchangesTotal

Total number of exchanges

rabbitmq_node_disk_free

Free disk space on a RabbitMQ node

rabbitmq_node_disk_free_alarm

Indicates whether disk free alarm is active

rabbitmq_node_mem_alarm

Indicates whether memory alarm is active

rabbitmq_partitions

Indicates whether network partitions are detected by RabbitMQ

rabbitmq_queue_messages_ready_total

Total number of messages ready to be delivered to consumers

rabbitmq_queue_messages_unacknowledged_total

Total number of messages delivered but not yet confirmed

rabbitmq_queuesTotal

Total number of queues

rabbitmq_running

Indicates whether RabbitMQ node is ready

Metrics

Description

kafka_brokers

Number of brokers in the Kafka cluster

kafka_consumergroup_lag

Current approximate lag of a consumer group for a topic partition

kafka_consumergroup_lag_sum

Total approximate lag of a consumer group for all partitions of a topic

kafka_consumergroup_members

Number of members in a consumer group

kafka_topic_partition_in_sync_replica

Number of in-sync replicas for topic partition

kafka_topic_partition_leader

Leader broker ID for topic partition

kafka_topic_partition_leader_is_preferred

Indicates whether the preferred leader is used by partition

kafka_topic_partition_under_replicated_partition

Indicates whether there are no enough replicas for topic partition

To view the metrics of a specific service:

  1. Go to PaaS Running services and open the Message brokers tab.

  2. Find the desired service in the table and click the service ID to go to its page.

  3. Open the Monitoring tab and select:

    • metrics,

    • instance to collect metric from.

    Note

    Instances available for a particular metric depend on their role. For certain metrics, there can be additional selectors apart from instance choice.

    Go to Display parameters to set the period the metric graph covers, statistics, and metric calculation interval. To automatically refresh the graph, enable Auto-refresh; to refresh it manually, click Refresh.

Deleting a message broker service#

Deleting a message broker service deletes all instances, volumes and placement groups created with it.

You can delete the service using one of the following methods.

  1. Go to the section PaaS Running services.

  2. Open the Message brokers tab.

  3. Find the service in the table and click on the icon .

  4. If you want to use network interfaces in the future, for example, to recreate a service on the cluster with the same network parameters, then, in the window that opens, disable the Delete associated network interfaces option.

  5. Confirm the action.

  1. Go to the section PaaS Running services.

  2. Open the Message brokers tab.

  3. Find the service in the table and go to the service page.

  4. Click :Delete in the Information tab.

  5. If you want to use network interfaces in the future, for example, to recreate a service on the cluster with the same network parameters, then, in the window that opens, disable the Delete associated network interfaces option.

  6. Confirm the action.