Usage recommendations#

This section provides recommendations on using message broker service.

Elasticsearch#

Service launch options#

  • Standalone caching instance in the selected availability zone.

  • High-availability service deployed in a cluster of three instances minimum in three Availability Zones.

High-availability solution architecture#

A high-availability Elasticsearch service is deployed in a cluster consisting of three instances mininum. Each instance has one master and one replica, and the latter interacts with a master on another node (cross-replication).

The cluster nodes can be full-featured and equivalent, or one of the cluster nodes is an arbitrator that has master and voting_only roles. In this case, lack of data and ingest roles makes it impossible for the arbitrator to receive and process data, but does not require many resources. Therefore, you can use a cheaper instance type as an arbitrator.

If the cluster has two nodes or more, then by default new indexes are created with one primary shard and one replica. Sharding and replication allow you to preserve data, in case one of the nodes crashes, and distribute the load if it increases.

Connect to the service#

After the successful launch, addresses and ports listened by Elasticsearch can be found on its page in the Information tab.

The Elasticsearch service listens to tcp/9200 port. In this cluster, it is recommended to use endpoints to connect all nodes except the arbitrator. Many tools such as Filebeat, Metricbeat, Logstash can work with all endpoints at once, balancing and switching automatically. If they fail to perform these operations on their own, we recommend using a haproxy or nginx balancer.

If you set elastic superuser password at the service deployment stage, the service will start with X-Pack Security extension and any actions with it will require authentication.

Important

We recommend using the elastic account only to manage the Elasticsearch service. To integrate with other services, create additional users with the required set of privileges.

Authentication in Elasticsearch#

You can protect your login to the Kibana GUI or the Elasticsearch API with a password which you can specify when you create a service. Once the service has been started with the specified password, it is not recommended to change this password; otherwise, it would become impossible to manage the running service in the cloud.

For convenience, service users are built into the service:

  • kibana_system

  • logstash_system

  • beats_system

  • apm_system

  • remote_monitoring_user

Important

Changing the service passwords after the Elasticsearch start will cause the service to fail. If you want to change the password, create new users with the same roles, specify the password, and use them for integration with external systems.

Service monitoring#

To monitor the service, we recommend enabling Prometheus cloud monitoring service. If you want to use your own monitoring server to check the state and health of Elasticsearch instances, you can use the Prometheus exporter elasticsearch_exporter. The exporter is deployed with the Elasticsearch service and listens to port tcp/9114. Additionally, node_exporter is installed on all Elasticsearch cluster nodes. It allows you to take instance and operating system metrics and listens to port tcp/9100.