Scheduled Jobs with Custom Clock Processes

Scalingo allows to define Custom Clock Processes to help you run tasks on a regular basis. Unlike jobs setup through the Scalingo Scheduler, custom clock processes do not suffer from the same limitations.

About Custom Clock Processes

Custom clock processes allow to setup tasks so that they run periodically. They give you full control over the schedule, periodicity and conditions on which the jobs are launched, making them a powerful alternative to the Scalingo Scheduler.

A custom clock process is basically a background process running indefinitely in a container and waking up at specified interval(s) to launch some job(s). This background process is responsible for controlling what job(s) must be launched and when. It really acts as a job scheduler.

It’s usually written in the language of your choice. This means that your custom clock process can either share the same code base as your application’s or a dedicated one.

Limitations

Custom clock processes have very few limitations:

They do not have the limitations imposed by the Scalingo Scheduler. Consequently, they allow to:

  • Run (very) long process. The jobs you run in your custom clock process container(s) don’t have a limited lifespan.

  • Run as many scheduled jobs as you want.

  • Have full control over the schedule: if you want a task to repeat every minute, you can.

  • Have a very precise schedule, if need be.

Costs

The container in which the scheduler and the jobs are run is billed like any other container. Consequently, billing of a custom clock process mainly depends on both the container size chosen to run your tasks and on the container lifespan.

Your custom clock process is most probably to run 24/7, as opposed to the Scalingo Scheduler’s one-off containers that are short-lived. A custom clock process can therefore be a bit more costly.

For example, if you setup your custom clock process to run in a 2XL container, and you let it run for 3 days before scaling it down to zero, you will be billed 3 days of a 2XL container.

Working With Custom Clock Processes

Defining Custom Clock Processes

Defining a custom clock process is done by adding a new process type in the Procfile of your application.

In the following examples, the name clock is used to designate our custom clock process type. But, except from web, tcp and postdeploy, which are reserved by the platform, you can chose whatever you want. scheduler, cron, planner, timer, butler, … These are all valid names for your custom clock process type.

The syntax is pretty straightforward:

clock: <command_to_start_your_job_scheduler>

Implementing Custom Clock Processes

The implementation of the custom clock process mainly depends on the language or framework you want to use. A lot of languages and frameworks provide this feature through dedicated libraries. Here are a few examples:

Using Ruby

The following gems may help you:

Example Using the clockwork Gem

The following example leverages the clockwork gem to:

  • run frequent.jobevery 10 seconds,
  • run less.frequent.job every 3 minutes,
  • run hourly.job every hours,
  • run midnight.job every day at midnight.

Procfile:

clock: bundle exec clockwork clock.rb

clock.rb:

require 'clockwork'
require 'active_support/time' # Allow numeric durations (eg: 1.minutes)

module Clockwork
  handler do |job|
    puts "Running #{job}"
  end

  # handler receives the time when job is prepared to run in the 2nd argument
  # handler do |job, time|
  #   puts "Running #{job}, at #{time}"
  # end

  every(10.seconds, 'frequent.job')
  every(3.minutes, 'less.frequent.job')
  every(1.hour, 'hourly.job')

  every(1.day, 'midnight.job', :at => '00:00')
end

Using PHP

The following packages may help you:

Example Using the cron/cron Package

The code is available in our sample-php-cron repository.

This example leverages the cron/cron package to launch a job defined in the inc.php file every 2 minutes.

Procfile:

clock: php cron.php

cron.php:

<?php
  require(__DIR__ . '/vendor/autoload.php');

  echo "[CRON] Starting tasks scheduler\n";

  function build_cron() {
    // Increment redis key every minute
    $inc_job = new \Cron\Job\ShellJob();
    $inc_job->setCommand('php inc.php');
    $inc_job->setSchedule(new \Cron\Schedule\CrontabSchedule('*/2 * * * *'));

    $resolver = new \Cron\Resolver\ArrayResolver();
    $resolver->addJob($inc_job);

    $cron = new \Cron\Cron();
    $cron->setExecutor(new \Cron\Executor\Executor());
    $cron->setResolver($resolver);

    return $cron;
  }

  $cron = build_cron();

  // Every 60 seconds, run the scheduler which will execute the tasks
  // which have to be started at the given minute.
  while(true) {
    echo "[CRON] Running tasks\n";
    $report = $cron->run();

    while ($cron->isRunning()) { }

    echo "[CRON] " . count($report->getReports()) . " tasks have been executed\n";

    foreach($report->getReports() as $job_report) {
      $output = $job_report->getOutput();
      foreach($output as $line) {
        echo "[CRON] " . $line;
    }

    sleep(60);
  }
?>

Using Node.js

The following packages may help you:

Example Using node-cron Package

The code is available in our sample-node-express repository.

This example leverages the node-cron package to:

  • run job1 every 2 minutes
  • run job2 every minute

Both jobs log a message.

Procfile:

clock: node cron.js

cron.js:

var cron = require('cron');

var job1 = new cron.CronJob({
  cronTime: '*/2 * * * *',
  onTick: function() {
    now = new Date();
    console.log(now + ': job 1 ticked');
  },
  start: true,
  timeZone: 'Europe/Paris'
});

var job2 = new cron.CronJob({
  cronTime: '* * * * *',
  onTick: function() {
    now = new Date();
    console.log(now + ': job 2 ticked');
  },
  start: true,
  timeZone: 'Europe/Paris'
});

console.log('Started 2 cron jobs')

Enabling Custom Clock Processes

Once your application has been successfully deployed, scale your custom clock process type to 1 to start your scheduler in its own dedicated container:

scalingo --app my-app scale clock:1

Moreover, since Scalingo expects a web process type to be defined (either by you in the Procfile, or automatically by a buildpack), there is most probably a web process type that would start with your application. If you don’t need it, you can disable it by scaling the web process type to zero before or after your deployment:

scalingo --app my-app scale web:0

For more details about web-less applications, please refer to our documentation.

Disabling Custom Clock Processes

A custom clock process can be disabled anytime by scaling the corresponding process type to zero:

scalingo --app my-app scale clock:0

Suggest edits

Scheduled Jobs with Custom Clock Processes

©2024 Scalingo