The Elastic Stack (formerly known as the ELK Stack) is a powerful collection of softwares that lets you collect data from any source using any format. It gives you tools to search, visualize and analyze these data in real time.

The ELK stack is based on three major components Elasticsearch®, Logstash and Kibana.

Most of the time, logs are only relevant for a short period of time. It is generally a good idea to archive or remove them after this short period to keep indices as light and fast as possible.

To cover this need, we suggest to add a fourth component named Curator to your ELK stack.

Here is a diagram showing the main principles of the ELK stack architecture:

Planning your Deployment

Most of the time, logs are only relevant for a short period of time. It is generally a good idea to archive or remove them after this short period to keep indices as light and fast as possible.

To cover this need, we suggest to complete your ELK stack with a fourth component, in addition to the three standard ones described above:

Curator: a tool that helps curating and managing ElasticSearch’s indices. It is especially useful to manage indices lifecycles so that old logs are automatically removed from your database.

Planning your Deployment

• Logstash and Kibana both require their own container(s). We will consequently need two apps.

• To keep things simple, we will deploy Elasticsearch as an addon attached to Logstash.

• Still to keep things as simple as possible, we will deploy Curator besides Logstash.

• Choosing the appropriate Elasticsearch plan strongly depends on your needs. In this guide, we will start with a Sandbox plan. Please keep in mind that you can change for a more powerful plan later if need be.

• Logstash requires quite a lot of RAM to run properly. We recommend to deploy at least one L container to host it.

Deploying

Please refer to our dedicated pages:

• To deploy Elasticsearch® and Logstash
• To deploy Kibana
• To deploy Curator
• To learn more about Elasticsearch® by Scalingo