Scalingo for Elasticsearch®
Scalingo for Elasticsearch® is the official Elasticsearch® addon provided by Scalingo. Details on the available plans can be found here. This addon gives your app instant access to an Elasticsearch® database.
Adding Scalingo for Elasticsearch® to your app
You can add the Scalingo for Elasticsearch® addon through the Dashboard or through the command-line interface. The capacity of your database is elastic, you will be able to upgrade it later.
Through the Dashboard
- Go to your app on the Scalingo dashboard
- Click on Addons tab
- Select the addon you want to add
- In the dialog select the database plan you need
- Validate your choice
Through the command-line interface
$ scalingo --app my-app addons-add scalingo-elasticsearch elasticsearch-starter-1024
-----> Addon elasticsearch has been provisionned
ID: my-app-3030
Modified variables: [ELASTICSEARCH_URL SCALINGO_ELASTICSEARCH_URL]
Message from addon provider: Database successfully created
This command will provision the application my-app
with a 1g
Scalingo for Elasticsearch® database plan.
To find out what other plans are available:
$ scalingo addons-plans elasticsearch
Scalingo gor Elasticsearch® Cluster Setup
If using a Business plan for your Scalingo for Elasticsearch® addon, we setup an Elasticsearch® cluster. This cluster has the following configuration:
- multiple Elasticsearch® nodes in a private network: the amount of memory per node depends on the plan,
- a couple of HAProxy as entrypoint to the Elasticsearch® private network: one is the master and the other is just here as a backup in case of failing master.
The communication between the Elasticsearch® nodes is encrypted.
Getting your connection URI
Once the addon is provisioned, 2 environment variables are added to your app: SCALINGO_ELASTICSEARCH_URL
and ELASTICSEARCH_URL
. ELASTICSEARCH_URL
is an alias to SCALINGO_ELASTICSEARCH_URL
. To find out how to use it in your code please refer to Application environment.
In most cases, you can pass the variable directly to the client library you are using in your code. But sometimes the library requires a specific URI format, you’ll need to add a little bit of code to suit the library.
You can get environment variables from the dashboard or the command line interface.
From the Dashboard
- Go to your app on Scalingo Dashboard
- Click on Environment tab
-
SCALINGO_ELASTICSEARCH_URL
is displayed
From the command line interface
$ scalingo --app my-app env | grep ELASTIC
ELASTICSEARCH_URL=$SCALINGO_ELASTICSEARCH_URL
SCALINGO_ELASTICSEARCH_URL=http://my-app-3030:YANs3y07m5_KJC2MSDGebh8tx1lliFWh2Yb239zVqGQvbElWDjIN7QWspVH92Ul8@my-app-3030.elasticsearch.a.osc-fr1.scalingo-dbs.com:30995
Remote access your database
The difference between this data storage engine and the others is that Elasticsearch®
communicates over HTTP, so you can use any tool built for that. You don’t have to use
an official client. The most common command line tool is curl
. The following example
uses it.
Use scalingo run
scalingo --app <application name> run curl http://<user>:<password>@<host>:<port>
Example
If my application’s name is ‘my-app’ and it has the environment variable
SCALINGO_ELASTICSEARCH_URL = "http://my-app-123:YANs3y07m5_KJC2MSDGebh8tx1lliFWh2Yb239zVqGQvbElWDjIN7QWspVH92Ul8@my-app-123.elasticsearch.a.osc-fr1.scalingo-dbs.com:30000/my-app-123"
You can run:
$ scalingo --app my-app run curl $SCALINGO_ELASTICSEARCH_URL
{
"status" : 200,
"name" : "the Tomorrow Man Zarrko",
"version" : {
"number" : "1.3.4",
"build_hash" : "a70f3ccb52200f8f2c87e9c370c6597448eb3e45",
"build_timestamp" : "2014-09-30T09:07:17Z",
"build_snapshot" : false,
"lucene_version" : "4.9"
},
"tagline" : "You Know, for Search"
}
Force TLS Connections
Our Elasticsearch® image support TLS to encrypt all of its network traffic between the client and the server.
Scalingo for Elasticsearch® cannot listen to connections with and without TLS. If you want to encrypt communications with your Elasticsearch® databases, you need to force all connections to use TLS. Forcing TLS connections is as simple as heading to the database dashboard and clicking on the toggle button:
Note that you must have configured your application to use TLS when connecting to the database.
curl --insecure -X GET <URL>
With URL
starting with https://
.
The --insecure
option is mandatory as the generated certificates for your
databases are
self-signed. If you
want the certificate to be trust-able, you need to download our certification
authority certificate and specify it to the Elasticsearch® CLI tool.
Some existing databases may not have yet TLS support. To activate TLS, you need to restart the database. Any action leading to the restart will activate TLS (e.g. plan update, upgrade of the database).
Download the CA certificate
The certificate of our certification authority is available on the database dashboard.
After downloading it, you can specify its path:
curl --cacert=/path/to/ca.pem -X GET "<URL>"
Changing Plans
You can upgrade or downgrade your database plan whenever you need it. This operation happens instantly thanks to Docker containers and no manual input is required. When you change the plan, your database will be stopped then restarted on a new host with new parameters of the chosen plan. During the operation the connection is dropped between your app and the database. Finally, after the operation is successful, the related app will be restarted.
If using a Business plan for your Scalingo for Elasticsearch® addon, we are able to change the plan of your database without downtime.
From the Dashboard
- Go to your app on Scalingo dashboard
- Click on Addons tab
- Select the addon you want to change
- In the dialog select the plan you want to upgrade/downgrade to
- Validate your choice
From the command-line interface
To upgrade or downgrade your addon the sub-command is the same: addons-upgrade
.
$ scalingo --app my-app addons-upgrade my-app-3030 2g
In this example, my-app-3030
is the ID of the addon, and 2g
is the plan we want to upgrade to.
To find out the addon ID:
$ scalingo --app my-app addons
+------------------------+-------------+------+
| ADDON | ID | PLAN |
+------------------------+-------------+------+
| Elasticsearch | my-app-3030 | 1g |
+------------------------+-------------+------+
Database dashboard
The Scalingo for Elasticsearch® dashboard is the central place for administrative tasks such as:
- Monitor database and system stats
- Upgrade the database engine version
- Activate database specific features
- Manage database users
- Manage backups
Database Upgrade
When the database vendor releases a new version of your database engine, we will try to provide it as soon as possible. You will have the choice to upgrade your database with one click through your database dashboard.
This operation is similar to changing your database plan. Your database will be stopped and restarted with the new database version. When this operation finishes, your application will be restarted.
If using a Business plan for your Scalingo for Elasticsearch® addon, we are able to upgrade your instance to a newer version without downtime. In order to do so, we follow Elasticsearch® guidelines.
We first disable the shard allocation in the cluster. Then we shut down and upgrade the first node. After upgrading the plugins and restarting the node, we can safely enable back the shard allocation in the cluster. Last we wait for the cluster to finish the shard allocation before going through the same sequence of actions for the next node.
Container Stats
CPU usage | Current CPU usage. |
Memory usage | Display the current, hightest and free memory. Highest is the maximum memory recorded since database restarted. |
Swap usage | Display the current, hightest and free swap. Highest is the maximum swap recorded since database restarted. |
Database Stats
Database on disk size | Effective physical space used. |
Database Users
By default, Scalingo creates a read and write user on your database. This user has all rights on the database.
It is not possible to create another user on an Scalingo for Elasticsearch® database.
Backups
If your database is in a paid plan (i.e. it’s not “free plan”), we’ll automatically make periodic backups of your database on a daily basis, at around 1:00 AM Central European Time (CET or UTC+0100). The time of your daily backup is configurable via the web dashboard of your database or using the CLI. The scheduled date is not strongly enforce: it might get delayed depending on the load on our infrastructure.
Retention Policy for Daily Backups
We keep a limited amount of backups depending on your database plan. A daily backup is retained for the last 7 days. That means that 7 backups will exist, one for each of the last 7 days. A weekly backup means that only one backup is saved over a 7 days period. A monthly backup means that only 1 backup is saved over the course of a month.
Plan | Weekly Backups Retained | Monthly Backup Retained |
---|---|---|
Sandbox/Free | N/A | N/A |
Starter | 4 weeks | 0 months |
Business | 8 weeks | 12 months |
Retention Policy for Manual Backups
You can also manually trigger a backup for your database at any time you want. The number of manual backups that you can retain is limited by your plan:
Plan | Backups Retained |
---|---|
Sandbox/Free | N/A |
Starter | 10 |
Business | 50 |
In case a database is removed from an application, the retention policy remains untouched: backups are not instantly deleted.
Download Backups
Scalingo for Elasticsearch® backups cannot be downloaded. This restriction is imposed by the internal working of Elasticsearch® and we cannot circumvent it.
Restoring a Backup
You can restore your database with data from a previous backup. Automated backups are listed in the database specific dashboard:
- Go to your app on Scalingo Dashboard
- Click on Addons tab
- Click Link to dashboard which will take you to the Scalingo for Elasticsearch® dashboard
- Click on Backups tab
- Click on the “Restore” button in front of the backup you want to restore