Elasticsearch

Elasticsearch is the official addon provided by Scalingo. Details on the available plans can be found here. This addon gives your app instant access to an Elasticsearch database.

Adding Elasticsearch to your app

You can add the Elasticsearch addon through the Dashboard or through the command-line interface. The capacity of your database is elastic, you will be able to upgrade it later.

Through the Dashboard

  1. Go to your app on the Scalingo dashboard
  2. Click on Addons tab
  3. Select the addon you want to add
  4. In the dialog select the database plan you need
  5. Validate your choice

Through the command-line interface

$ scalingo -a my-app addons-add elasticsearch 1g

-----> Addon elasticsearch has been provisionned
       ID: my-app-3030
       Modified variables: [ELASTICSEARCH_URL SCALINGO_ELASTICSEARCH_URL]
       Message from addon provider: Database successfully created

This command will provision the application my-app with a 1g Elasticsearch database plan.

To find out what other plans are available:

$ scalingo addons-plans elasticsearch

Elasticsearch Cluster Setup

If using a Business plan for your Elasticsearch addon, we setup an Elasticsearch cluster. This cluster has the following configuration:

  • multiple Elasticsearch nodes in a private network: the amount of memory per node depends on the plan,
  • a couple of HAProxy as entrypoint to the Elasticsearch private network: one is the master and the other is just here as a backup in case of failing master.

The communication between the Elasticsearch nodes is encrypted.

Getting your connection URI

Once the addon is provisioned, 2 environment variables are added to your app: SCALINGO_ELASTICSEARCH_URL and ELASTICSEARCH_URL. ELASTICSEARCH_URL is an alias to SCALINGO_ELASTICSEARCH_URL. To find out how to use it in your code please refer to Application environment.

In most cases, you can pass the variable directly to the client library you are using in your code. But sometimes the library requires a specific URI format, you’ll need to add a little bit of code to suit the library.

You can get environment variables from the dashboard or the command line interface.

From the Dashboard

  1. Go to your app on Scalingo Dashboard
  2. Click on Environment tab
  3. SCALINGO_ELASTICSEARCH_URL is displayed

From the command line interface

$ scalingo -a my-app env | grep ELASTIC

ELASTICSEARCH_URL=$SCALINGO_ELASTICSEARCH_URL
SCALINGO_ELASTICSEARCH_URL=http://my-app-3030:MpIxXstskccB3Ab2iwKH@my-app-3030.elasticsearch.dbs.appsdeck.eu:30995

Remote access your database

The difference between this data storage engine and the others is that Elasticsearch communicates over HTTP, so you can use any tool built for that. You don’t have to use an official client. The most common command line tool is curl. The following example uses it.

Use scalingo run

scalingo --app <application name> run curl http://<user>:<password>@<host>:<port>

Example

If my application’s name is ‘my-app’ and it has the environment variable SCALINGO_ELASTICSEARCH_URL = "http://my-app-123:H_grwjqBteMMrVye442Zw6@my-app-123.elasticsearch.dbs.scalingo.com:30000/my-app-123"

You can run:

$ scalingo --app my-app run curl $SCALINGO_ELASTICSEARCH_URL
{
  "status" : 200,
  "name" : "the Tomorrow Man Zarrko",
  "version" : {
    "number" : "1.3.4",
    "build_hash" : "a70f3ccb52200f8f2c87e9c370c6597448eb3e45",
    "build_timestamp" : "2014-09-30T09:07:17Z",
    "build_snapshot" : false,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}

Force TLS connections

Our Elasticsearch image support TLS to encrypt all of its network traffic between the client and the server.

Elasticsearch cannot listen to connections with and without TLS. If you want to encrypt communications with your Elasticsearch databases, you need to force all connections to use TLS. Forcing TLS connections is as simple as heading to the database dashboard and clicking on the toggle button:

Note that you must have configured your application to use TLS when connecting to the database.

curl --insecure -X GET <URL>

With URL starting with https://.

The --insecure option is mandatory as the generated certificates for your databases are self-signed. If you want the certificate to be trust-able, you need to download our certification authority certificate and specify it to the MongoDB CLI tool.

Some existing databases may not have yet TLS support. To activate TLS, you need to restart the database. Any action leading to the restart will activate TLS (e.g. plan update, upgrade of the database).

Download the CA certificate

The certificate of our certification authority is available on the database dashboard.

After downloading it, you can specify its path:

curl --cacert=/path/to/ca.pem -X GET "<URL>"

Changing plans

You can upgrade or downgrade your database plan whenever you need it. This operation happens instantly thanks to Docker containers and no manual input is required. When you change the plan, your database will be stopped then restarted on a new host with new parameters of the chosen plan. During the operation the connection is dropped between your app and the database. Finally, after the operation is successful, the related app will be restarted.

From the Dashboard

  1. Go to your app on Scalingo dashboard
  2. Click on Addons tab
  3. Select the addon you want to change
  4. In the dialog select the plan you want to upgrade/downgrade to
  5. Validate your choice

From the command-line interface

To upgrade or downgrade your addon the sub-command is the same: addons-upgrade.

$ scalingo -a my-app addons-upgrade my-app-3030 2g

In this example, my-app-3030 is the ID of the addon, and 2g is the plan we want to upgrade to.

To find out the addon ID:

$ scalingo -a my-app addons

+------------------------+-------------+------+
|          ADDON         |      ID     | PLAN |
+------------------------+-------------+------+
| Elasticsearch | my-app-3030 |   1g |
+------------------------+-------------+------+

Database dashboard

The Elasticsearch dashboard is the central place for administrative tasks such as:

  • Monitor database and system stats
  • Upgrade the database engine version
  • Activate database specific features
  • Manage database users
  • Manage backups

Database Upgrade

When the database vendor releases a new version of your database engine, we will try to provide it as soon as possible. You will have the choice to upgrade your database with one click through your database dashboard.

This operation is similar to changing your database plan. Your database will be stopped and restarted with the new database version. When this operation finishes, your application will be restarted.

If using a Business plan for your Elasticsearch addon, we are able to upgrade your instance to a newer version without downtime. In order to do so, we follow Elasticsearch guidelines.

We first disable the shard allocation in the cluster. Then we shut down and upgrade the first node. After upgrading the plugins and restarting the node, we can safely enable back the shard allocation in the cluster. Last we wait for the cluster to finish the shard allocation before going through the same sequence of actions for the next node.

Container Stats

CPU usage Current CPU usage.
Memory usage Display the current, hightest and free memory. Highest is the maximum memory recorded since database restarted.
Swap usage Display the current, hightest and free swap. Highest is the maximum swap recorded since database restarted.

Database Stats

Database on disk size Effective physical space used.

Backups

If your database is in a paid plan (i.e. it’s not “free plan”), we’ll automatically make periodic backups of your database on a daily basis, at around 1:00 AM Central European Time (CET or UTC+0100). The time of your daily backup is configurable via the web dashboard of your database or using the CLI.

We keep a limited amount of backups depending on your database plan. A daily backup is retained for the last 7 days. That means that 7 backups will exist, one for each of the last 7 days. A weekly backup means that only one backup is saved over a 7 days period. A monthly backup means that only 1 backup is saved over the course of a month.

Plan Weekly Backups Retained Monthly Backup Retained
Sandbox/Free N/A N/A
Starter 4 weeks 0 months
Business 8 weeks 12 months

You can also manually trigger a backup for your database at any time you want. The number of manual backups that you can retain is limited by your plan:

Plan Backups Retained
Sandbox/Free N/A
Starter 10
Business 50

Download Backups

Elasticsearch backups cannot be downloaded. This restriction is imposed by the internal working of Elasticsearch and we cannot circumvent it.

Restoring a Backup

You can restore your database with data from a previous backup. Automated backups are listed in the database specific dashboard:

  1. Go to your app on Scalingo Dashboard
  2. Click on Addons tab
  3. Click Link to dashboard which will take you to the Elasticsearch dashboard
  4. Click on Backups tab
  5. Click on the “Restore” button in front of the backup you want to restore

mode_edit Suggest edits