10 Popular Libraries for Scheduling Jobs in Go

In today's post we will learn about 10 Popular Libraries for Scheduling Jobs in Go.

In Go and Golang programming, a scheduler is responsible for distributing jobs in a multiprocessing environment. When the available resources are limited, it is the task of the scheduler to manage the work that needs to be done in the most efficient way. In Go, the scheduler is responsible for scheduling goroutines, which is particularly useful in concurrency. Goroutines are like OS threads, but they are much lighter weight. However, goroutines always take the help of the underlying OS thread model and the scheduler it works on is at a much higher level than the OS scheduler. This Go programming tutorial provides a quick look at the concepts behind the Go scheduler.

Table of contents:

• Cdule - Job scheduler library with database support
• Cheek - A simple crontab like scheduler that aims to offer a KISS approach to job scheduling.
• Clockwerk - Go package to schedule periodic jobs using a simple, fluent syntax.
• Cronticker - A ticker implementation to support cron schedules.
• Dagu - No-code workflow executor. it executes DAGs defined in a simple YAML format.
• Go-cron - Simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.
• Go-quartz - Simple, zero-dependency scheduling library for Go.
• Gocron - Easy and fluent Go job scheduling. This is an actively maintained fork of jasonlvhit/gocron.
• Goflow - A workflow orchestrator and scheduler for rapid prototyping of ETL/ML/AI pipelines.
• Gron - Define time-based tasks using a simple Go API and Gron’s scheduler will run them accordingly.

1 - Cdule: Job scheduler library with database support.

cdule (pronounce as Schedule)

Golang based scheduler library with database support. Users could use any database which is supported by gorm.io.

To Download the cdule library

go get github.com/deepaksinghvi/cdule

Usage Instruction

In order to schedule jobs with cdule, user needs to

1. Configure persistence
2. Implement cdule.Job Interface &
3. Schedule job with required cron expression.

Job will be persisted in the jobs table.
Next execution would be persisted in schedules tables.
Job history would be persisted and maintained in job_histories table.

Configuration

User needs to create a resources/config.yml in their project home directory with the followling keys

• cduletype
• dburl
• cduleconsistency

cduletype is used to specify whether it is an In-Memory or Database based configuration. Possible values are DATABASE and MEMORY. dburl is the database connection url. cduleconsistency is for reserved for future usage.

config.yml for postgressql based configuration

cduletype: DATABASE
dburl: postgres://cduleuser:cdulepassword@localhost:5432/cdule?sslmode=disable
cduleconsistency: AT_MOST_ONCE

config.yml for sqlite based in-memory configuration

cduletype: MEMORY
dburl: /Users/dsinghvi/sqlite.db
cduleconsistency: AT_MOST_ONCE

Job Interface Implementation

var testJobData map[string]string

type TestJob struct {
	Job cdule.Job
}

func (m TestJob) Execute(jobData map[string]string) {
	log.Info("In TestJob")
	for k, v := range jobData {
		valNum, err := strconv.Atoi(v)
		if nil == err {
			jobData[k] = strconv.Itoa(valNum + 1)
		} else {
			log.Error(err)
		}

	}
	testJobData = jobData
}

func (m TestJob) JobName() string {
	return "job.TestJob"
}

func (m TestJob) GetJobData() map[string]string {
	return testJobData
}

View on Github

2 - Cheek: A simple crontab like scheduler that aims to offer a KISS approach to job scheduling.

cheek, of course, stands for Crontab-like scHeduler for Effective Execution of tasKs. cheek is a KISS approach to crontab-like job scheduling. It was born out of a (/my?) frustration about the big gap between a lightweight crontab and full-fledged solutions like Airflow.

cheek aims to be a KISS approach to job scheduling. Focus is on the KISS approach not to necessarily do this in the most robust way possible.

Getting started

Fetch the latest version for your system below.

You can (for example) fetch it like below, make it executable and run it. Optionally put the cheek on your PATH.

curl https://storage.googleapis.com/better-unified/darwin/amd64/cheek -o cheek
chmod +x cheek
./cheek

Create a schedule specification using the below YAML structure:

tz_location: Europe/Brussels
jobs:
  foo:
    command: date
    cron: "* * * * *"
    on_success:
      trigger_job:
        - bar
  bar:
    command:
      - echo
      - bar
      - foo
  coffee:
    command: this fails
    cron: "* * * * *"
    retries: 3
    on_error:
      notify_webhook:
        - https://webhook.site/4b732eb4-ba10-4a84-8f6b-30167b2f2762

If your command requires arguments, please make sure to pass them as an array like in foo_job.

Note that you can set tz_location if the system time of where you run your service is not to your liking.

Scheduler

The core of cheek consists of a scheduler that uses a schedule specified in a yaml file to triggers jobs when they are due.

You can launch the scheduler via:

cheek run ./path/to/my-schedule.yaml

Check out cheek run --help for configuration options.

View on Github

3 - Clockwerk: Go package to schedule periodic jobs using a simple, fluent syntax.

Job Scheduling Library

clockwerk allows you to schedule periodic jobs using a simple, fluent syntax.

Usage

go get github.com/onatm/clockwerk

package main

import (
  "fmt"
  "time"
  "github.com/onatm/clockwerk"
)

type DummyJob struct{}

func (d DummyJob) Run() {
  fmt.Println("Every 30 seconds")
}

func main() {
  var job DummyJob
  c := clockwerk.New()
  c.Every(30 * time.Second).Do(job)
  c.Start()
}

View on Github

4 - Cronticker: A ticker implementation to support cron schedules.

Golang ticker that works with Cron scheduling.

Import it

go get github.com/krayzpipes/cronticker/cronticker

import "github.com/krayzpipes/cronticker/cronticker"

Usage

Create a new ticker:

ticker, err := NewTicker("TZ=America/New_York 0 0 0 ? * SUN")

Check the ticker's channel for the next tick:

tickerTime := <-ticker.C

Reset the ticker to a new cron schedule

err := ticker.Reset("0 0 0 ? * MON,TUE,WED")

Stop the ticker

ticker.Stop()

Use defer ticker.Stop() whenever you can to ensure the cleanup of goroutines.

ticker, _ := NewTicker("@daily")
defer ticker.Stop()

View on Github

5 - Dagu: No-code workflow executor. it executes DAGs defined in a simple YAML format.

A just another Cron alternative with a Web UI, but with much more capabilities
It runs DAGs (Directed acyclic graph) defined in a simple YAML format.

Highlights

• Install by placing just a single binary file
• Schedule executions of DAGs with Cron expressions
• Define dependencies between related jobs and represent them as a single DAG (unit of execution)

How does it work?

dagu is a single command and it uses the local file system to store data. Therefore, no DBMS or cloud service is required. dagu executes DAGs defined in declarative YAML format. Existing programs can be used without any modification.

Install dagu

You can quickly install dagu command and try it out.

via Homebrew

brew install yohamta/tap/dagu

Upgrade to the latest version:

brew upgrade yohamta/tap/dagu

via Bash script

curl -L https://raw.githubusercontent.com/yohamta/dagu/main/scripts/downloader.sh | bash

via GitHub Release Page

Download the latest binary from the Releases page and place it in your $PATH (e.g. /usr/local/bin).

️Quick start

1. Launch the Web UI

Start the server with dagu server and browse to http://127.0.0.1:8080 to explore the Web UI.

2. Create a new DAG

Create a DAG by clicking the New DAG button on the top page of the web UI. Input example in the dialog.

Note: DAG (YAML) files will be placed in ~/.dagu/dags by default. See Admin Configuration for more details.

3. Edit the DAG

Go to the SPEC Tab and hit the Edit button. Copy & Paste this example YAML and click the Save button.

4. Execute the DAG

You can execute the example by pressing the Start button.

Note: Leave the parameter field in the dialog blank and press OK.

Command Line User Interface

• dagu start [--params=<params>] <file> - Runs the DAG
• dagu status <file> - Displays the current status of the DAG
• dagu retry --req=<request-id> <file> - Re-runs the specified DAG run
• dagu stop <file> - Stops the DAG execution by sending TERM signals
• dagu restart <file> - Restart the current running DAG
• dagu dry [--params=<params>] <file> - Dry-runs the DAG
• dagu server [--host=<host>] [--port=<port>] [--dags=<path/to/the DAGs directory>] - Starts the web server for web UI
• dagu scheduler [--dags=<path/to/the DAGs directory>] - Starts the scheduler process
• dagu version - Shows the current binary version

The --config=<config> option is available to all commands. It allows to specify different dagu configuration for the commands. Which enables you to manage multiple dagu process in a single instance. See Admin Configuration for more details.

For example:

dagu server --config=~/.dagu/dev.yaml
dagu scheduler --config=~/.dagu/dev.yaml

View on Github

6 - Go-cron: Simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually designed. Comparisons are fast and efficient and take place in a goroutine; matched jobs are also executed in goroutines.

For instance, you can use the following in your web application that uses MySQL:

func init() {
  cron.NewWeeklyJob(1, 23, 59, 59, func (time.Time) {
    _, err := conn.Query("OPTIMIZE TABLE mytable;")
    if(err != nil) { println(err) }
  })
}

View on Github

7 - Go-quartz: Simple, zero-dependency scheduling library for Go.

A minimalistic and zero-dependency scheduling library for Go.

About

Inspired by the Quartz Java scheduler.

Library building blocks

Scheduler interface

type Scheduler interface {
	// Start starts the scheduler.
	Start()
	// IsStarted determines whether the scheduler has been started.
	IsStarted() bool
	// ScheduleJob schedules a job using a specified trigger.
	ScheduleJob(job Job, trigger Trigger) error
	// GetJobKeys returns the keys of all of the scheduled jobs.
	GetJobKeys() []int
	// GetScheduledJob returns the scheduled job with the specified key.
	GetScheduledJob(key int) (*ScheduledJob, error)
	// DeleteJob removes the job with the specified key from the Scheduler's execution queue.
	DeleteJob(key int) error
	// Clear removes all of the scheduled jobs.
	Clear()
	// Stop shutdowns the scheduler.
	Stop()
}

Implemented Schedulers

• StdScheduler

Trigger interface

type Trigger interface {
	// NextFireTime returns the next time at which the Trigger is scheduled to fire.
	NextFireTime(prev int64) (int64, error)
	// Description returns the description of the Trigger.
	Description() string
}

Implemented Triggers

• CronTrigger
• SimpleTrigger
• RunOnceTrigger

Job interface. Any type that implements it can be scheduled.

type Job interface {
	// Execute is called by a Scheduler when the Trigger associated with this job fires.
	Execute()
	// Description returns the description of the Job.
	Description() string
	// Key returns the unique key for the Job.
	Key() int
}

Implemented Jobs

• ShellJob
• CurlJob

View on Github

8 - Gocron: Easy and fluent Go job scheduling. This is an actively maintained fork of jasonlvhit/gocron.

gocron is a job scheduling package which lets you run Go functions at pre-determined intervals using a simple, human-friendly syntax.

gocron is a Golang scheduler implementation similar to the Ruby module clockwork and the Python job scheduling package schedule.

See also these two great articles that were used for design input:

• Rethinking Cron
• Replace Cron with Clockwork

If you want to chat, you can find us at Slack!

Concepts

• Scheduler: The scheduler tracks all the jobs assigned to it and makes sure they are passed to the executor when ready to be run. The scheduler is able to manage overall aspects of job behavior like limiting how many jobs are running at one time.
• Job: The job is simply aware of the task (go function) it's provided and is therefore only able to perform actions related to that task like preventing itself from overruning a previous task that is taking a long time.
• Executor: The executor, as it's name suggests, is simply responsible for calling the task (go function) that the job hands to it when sent by the scheduler.

Examples

s := gocron.NewScheduler(time.UTC)

s.Every(5).Seconds().Do(func(){ ... })

// strings parse to duration
s.Every("5m").Do(func(){ ... })

s.Every(5).Days().Do(func(){ ... })

s.Every(1).Month(1, 2, 3).Do(func(){ ... })

// set time
s.Every(1).Day().At("10:30").Do(func(){ ... })

// set multiple times
s.Every(1).Day().At("10:30;08:00").Do(func(){ ... })

s.Every(1).Day().At("10:30").At("08:00").Do(func(){ ... })

// Schedule each last day of the month
s.Every(1).MonthLastDay().Do(func(){ ... })

// Or each last day of every other month
s.Every(2).MonthLastDay().Do(func(){ ... })

// cron expressions supported
s.Cron("*/1 * * * *").Do(task) // every minute

// you can start running the scheduler in two different ways:
// starts the scheduler asynchronously
s.StartAsync()
// starts the scheduler and blocks current execution path
s.StartBlocking()

For more examples, take a look in our go docs

View on Github

9 - Goflow: A workflow orchestrator and scheduler for rapid prototyping of ETL/ML/AI pipelines.

A workflow/DAG orchestrator written in Go for rapid prototyping of ETL/ML/AI pipelines. Goflow comes complete with a web UI for inspecting and triggering jobs.

Quick start

With Docker

docker run -p 8181:8181 ghcr.io/fieldryand/goflow-example:latest

Browse to localhost:8181 to explore the UI.

Without Docker

In a fresh project directory:

go mod init # create a new module
go get github.com/fieldryand/goflow # install dependencies

Create a file main.go with contents:

package main

import "github.com/fieldryand/goflow"

func main() {
        options := goflow.Options{
                AssetBasePath: "assets/",
                StreamJobRuns: true,
                ShowExamples:  true,
        }
        gf := goflow.New(options)
        gf.Use(goflow.DefaultLogger())
        gf.Run(":8181")
}

Download the front-end from the release page, untar it, and move it to the location specified in goflow.Options.AssetBasePath. Now run the application with go run main.go and see it in the browser at localhost:8181.

Use case

Goflow was built as a simple replacement for Apache Airflow to manage some small data pipeline projects. Airflow started to feel too heavyweight for these projects where all the computation was offloaded to independent services, but there was still a need for basic orchestration, concurrency, retries, visibility etc.

Goflow prioritizes ease of deployment over features and scalability. If you need distributed workers, backfilling over time slices, a durable database of job runs, etc, then Goflow is not for you. On the other hand, if you want to rapidly prototype some pipelines, then Goflow might be a good fit.

Concepts and features

• Job: A Goflow workflow is called a Job. Jobs can be scheduled using cron syntax.
• Task: Each job consists of one or more tasks organized into a dependency graph. A task can be run under certain conditions; by default, a task runs when all of its dependencies finish successfully.
• Concurrency: Jobs and tasks execute concurrently.
• Operator: An Operator defines the work done by a Task. Goflow comes with a handful of basic operators, and implementing your own Operator is straightforward.
• Retries: You can allow a Task a given number of retry attempts. Goflow comes with two retry strategies, ConstantDelay and ExponentialBackoff.
• Database: Goflow supports two database types, in-memory and BoltDB. BoltDB will persist your history of job runs, whereas in-memory means the history will be lost each time the Goflow server is stopped. The default is BoltDB.
• Streaming: Goflow uses server-sent events to stream the status of jobs and tasks to the UI in real time.

View on Github

10 - Gron: Define time-based tasks using a simple Go API and Gron’s scheduler will run them accordingly.

Gron provides a clear syntax for writing and deploying cron jobs.

Goals

• Minimalist APIs for scheduling jobs.
• Thread safety.
• Customizable Job Type.
• Customizable Schedule.

Installation

$ go get github.com/roylee0704/gron

Usage

Create schedule.go

package main

import (
	"fmt"
	"time"
	"github.com/roylee0704/gron"
)

func main() {
	c := gron.New()
	c.AddFunc(gron.Every(1*time.Hour), func() {
		fmt.Println("runs every hour.")
	})
	c.Start()
}

Schedule Parameters

All scheduling is done in the machine's local time zone (as provided by the Go time package).

Setup basic periodic schedule with gron.Every().

gron.Every(1*time.Second)
gron.Every(1*time.Minute)
gron.Every(1*time.Hour)

Also support Day, Week by importing gron/xtime:

import "github.com/roylee0704/gron/xtime"

gron.Every(1 * xtime.Day)
gron.Every(1 * xtime.Week)

Schedule to run at specific time with .At(hh:mm)

gron.Every(30 * xtime.Day).At("00:00")
gron.Every(1 * xtime.Week).At("23:59")

View on Github

Thank you for following this article.

Related videos:

Go scheduler: Implementing language with lightweight concurrency

#go #golang #scheduling #jobs 

What is GEEK

Buddha Community

Rufus Scheduler: Job Scheduler for Ruby (at, Cron, in and Every Jobs)

rufus-scheduler

Job scheduler for Ruby (at, cron, in and every jobs).

It uses threads.

Note: maybe are you looking for the README of rufus-scheduler 2.x? (especially if you're using Dashing which is stuck on rufus-scheduler 2.0.24)

Quickstart:

# quickstart.rb

require 'rufus-scheduler'

scheduler = Rufus::Scheduler.new

scheduler.in '3s' do
  puts 'Hello... Rufus'
end

scheduler.join
  #
  # let the current thread join the scheduler thread
  #
  # (please note that this join should be removed when scheduling
  # in a web application (Rails and friends) initializer)

(run with ruby quickstart.rb)

Various forms of scheduling are supported:

require 'rufus-scheduler'

scheduler = Rufus::Scheduler.new

# ...

scheduler.in '10d' do
  # do something in 10 days
end

scheduler.at '2030/12/12 23:30:00' do
  # do something at a given point in time
end

scheduler.every '3h' do
  # do something every 3 hours
end
scheduler.every '3h10m' do
  # do something every 3 hours and 10 minutes
end

scheduler.cron '5 0 * * *' do
  # do something every day, five minutes after midnight
  # (see "man 5 crontab" in your terminal)
end

# ...

Rufus-scheduler uses fugit for parsing time strings, et-orbi for pairing time and tzinfo timezones.

non-features

Rufus-scheduler (out of the box) is an in-process, in-memory scheduler. It uses threads.

It does not persist your schedules. When the process is gone and the scheduler instance with it, the schedules are gone.

A rufus-scheduler instance will go on scheduling while it is present among the objects in a Ruby process. To make it stop scheduling you have to call its #shutdown method.

related and similar gems

• Whenever - let cron call back your Ruby code, trusted and reliable cron drives your schedule
• ruby-clock - a clock process / job scheduler for Ruby
• Clockwork - rufus-scheduler inspired gem
• Crono - an in-Rails cron scheduler
• PerfectSched - highly available distributed cron built on Sequel and more

(please note: rufus-scheduler is not a cron replacement)

note about the 3.0 line

It's a complete rewrite of rufus-scheduler.

There is no EventMachine-based scheduler anymore.

I don't know what this Ruby thing is, where are my Rails?

I'll drive you right to the tracks.

notable changes:

• As said, no more EventMachine-based scheduler
• scheduler.every('100') { will schedule every 100 seconds (previously, it would have been 0.1s). This aligns rufus-scheduler with Ruby's sleep(100)
• The scheduler isn't catching the whole of Exception anymore, only StandardError
• The error_handler is #on_error (instead of #on_exception), by default it now prints the details of the error to $stderr (used to be $stdout)
• Rufus::Scheduler::TimeOutError renamed to Rufus::Scheduler::TimeoutError
• Introduction of "interval" jobs. Whereas "every" jobs are like "every 10 minutes, do this", interval jobs are like "do that, then wait for 10 minutes, then do that again, and so on"
• Introduction of a lockfile: true/filename mechanism to prevent multiple schedulers from executing
• "discard_past" is on by default. If the scheduler (its host) sleeps for 1 hour and a every '10m' job is on, it will trigger once at wakeup, not 6 times (discard_past was false by default in rufus-scheduler 2.x). No intention to re-introduce discard_past: false in 3.0 for now.
• Introduction of Scheduler #on_pre_trigger and #on_post_trigger callback points

getting help

So you need help. People can help you, but first help them help you, and don't waste their time. Provide a complete description of the issue. If it works on A but not on B and others have to ask you: "so what is different between A and B" you are wasting everyone's time.

"hello", "please" and "thanks" are not swear words.

Go read how to report bugs effectively, twice.

Update: help_help.md might help help you.

on Gitter

You can find help via chat over at https://gitter.im/floraison/fugit. It's fugit, et-orbi, and rufus-scheduler combined chat room.

Please be courteous.

issues

Yes, issues can be reported in rufus-scheduler issues, I'd actually prefer bugs in there. If there is nothing wrong with rufus-scheduler, a Stack Overflow question is better.

faq

• It doesn't work...
• I want a refund
• Passenger and rufus-scheduler
• Passenger and rufus-scheduler (2)
• Passenger in-depth spawn methods
• Passenger in-depth spawn methods (smart spawning)
• The scheduler comes up when running the Rails console or a Rake task
• The job triggers twice
• I don't get any of this, I just want it to work in my Rails application
• [I get "zotime.rb:41:in `initialize': cannot determine timezone from nil"](#i-get-zotimerb41in-initialize-cannot-determine-timezone-from-nil)

scheduling

Rufus-scheduler supports five kinds of jobs. in, at, every, interval and cron jobs.

Most of the rufus-scheduler examples show block scheduling, but it's also OK to schedule handler instances or handler classes.

in, at, every, interval, cron

In and at jobs trigger once.

require 'rufus-scheduler'

scheduler = Rufus::Scheduler.new

scheduler.in '10d' do
  puts "10 days reminder for review X!"
end

scheduler.at '2014/12/24 2000' do
  puts "merry xmas!"
end

In jobs are scheduled with a time interval, they trigger after that time elapsed. At jobs are scheduled with a point in time, they trigger when that point in time is reached (better to choose a point in the future).

Every, interval and cron jobs trigger repeatedly.

require 'rufus-scheduler'

scheduler = Rufus::Scheduler.new

scheduler.every '3h' do
  puts "change the oil filter!"
end

scheduler.interval '2h' do
  puts "thinking..."
  puts sleep(rand * 1000)
  puts "thought."
end

scheduler.cron '00 09 * * *' do
  puts "it's 9am! good morning!"
end

Every jobs try hard to trigger following the frequency they were scheduled with.

Interval jobs trigger, execute and then trigger again after the interval elapsed. (every jobs time between trigger times, interval jobs time between trigger termination and the next trigger start).

Cron jobs are based on the venerable cron utility (man 5 crontab). They trigger following a pattern given in (almost) the same language cron uses.

 

#schedule_x vs #x

schedule_in, schedule_at, schedule_cron, etc will return the new Job instance.

in, at, cron will return the new Job instance's id (a String).

job_id =
  scheduler.in '10d' do
    # ...
  end
job = scheduler.job(job_id)

# versus

job =
  scheduler.schedule_in '10d' do
    # ...
  end

# also

job =
  scheduler.in '10d', job: true do
    # ...
  end

#schedule and #repeat

Sometimes it pays to be less verbose.

The #schedule methods schedules an at, in or cron job. It just decides based on its input. It returns the Job instance.

scheduler.schedule '10d' do; end.class
  # => Rufus::Scheduler::InJob

scheduler.schedule '2013/12/12 12:30' do; end.class
  # => Rufus::Scheduler::AtJob

scheduler.schedule '* * * * *' do; end.class
  # => Rufus::Scheduler::CronJob

The #repeat method schedules and returns an EveryJob or a CronJob.

scheduler.repeat '10d' do; end.class
  # => Rufus::Scheduler::EveryJob

scheduler.repeat '* * * * *' do; end.class
  # => Rufus::Scheduler::CronJob

(Yes, no combination here gives back an IntervalJob).

schedule blocks arguments (job, time)

A schedule block may be given 0, 1 or 2 arguments.

The first argument is "job", it's simply the Job instance involved. It might be useful if the job is to be unscheduled for some reason.

scheduler.every '10m' do |job|

  status = determine_pie_status

  if status == 'burnt' || status == 'cooked'
    stop_oven
    takeout_pie
    job.unschedule
  end
end

The second argument is "time", it's the time when the job got cleared for triggering (not Time.now).

Note that time is the time when the job got cleared for triggering. If there are mutexes involved, now = mutex_wait_time + time...

"every" jobs and changing the next_time in-flight

It's OK to change the next_time of an every job in-flight:

scheduler.every '10m' do |job|

  # ...

  status = determine_pie_status

  job.next_time = Time.now + 30 * 60 if status == 'burnt'
    #
    # if burnt, wait 30 minutes for the oven to cool a bit
end

It should work as well with cron jobs, not so with interval jobs whose next_time is computed after their block ends its current run.

scheduling handler instances

It's OK to pass any object, as long as it responds to #call(), when scheduling:

class Handler
  def self.call(job, time)
    p "- Handler called for #{job.id} at #{time}"
  end
end

scheduler.in '10d', Handler

# or

class OtherHandler
  def initialize(name)
    @name = name
  end
  def call(job, time)
    p "* #{time} - Handler #{name.inspect} called for #{job.id}"
  end
end

oh = OtherHandler.new('Doe')

scheduler.every '10m', oh
scheduler.in '3d5m', oh

The call method must accept 2 (job, time), 1 (job) or 0 arguments.

Note that time is the time when the job got cleared for triggering. If there are mutexes involved, now = mutex_wait_time + time...

scheduling handler classes

One can pass a handler class to rufus-scheduler when scheduling. Rufus will instantiate it and that instance will be available via job#handler.

class MyHandler
  attr_reader :count
  def initialize
    @count = 0
  end
  def call(job)
    @count += 1
    puts ". #{self.class} called at #{Time.now} (#{@count})"
  end
end

job = scheduler.schedule_every '35m', MyHandler

job.handler
  # => #<MyHandler:0x000000021034f0>
job.handler.count
  # => 0

If you want to keep that "block feeling":

job_id =
  scheduler.every '10m', Class.new do
    def call(job)
      puts ". hello #{self.inspect} at #{Time.now}"
    end
  end

pause and resume the scheduler

The scheduler can be paused via the #pause and #resume methods. One can determine if the scheduler is currently paused by calling #paused?.

While paused, the scheduler still accepts schedules, but no schedule will get triggered as long as #resume isn't called.

job options

name: string

Sets the name of the job.

scheduler.cron '*/15 8 * * *', name: 'Robert' do |job|
  puts "A, it's #{Time.now} and my name is #{job.name}"
end

job1 =
  scheduler.schedule_cron '*/30 9 * * *', n: 'temporary' do |job|
    puts "B, it's #{Time.now} and my name is #{job.name}"
  end
# ...
job1.name = 'Beowulf'

blocking: true

By default, jobs are triggered in their own, new threads. When blocking: true, the job is triggered in the scheduler thread (a new thread is not created). Yes, while a blocking job is running, the scheduler is not scheduling.

overlap: false

Since, by default, jobs are triggered in their own new threads, job instances might overlap. For example, a job that takes 10 minutes and is scheduled every 7 minutes will have overlaps.

To prevent overlap, one can set overlap: false. Such a job will not trigger if one of its instances is already running.

The :overlap option is considered before the :mutex option when the scheduler is reviewing jobs for triggering.

mutex: mutex_instance / mutex_name / array of mutexes

When a job with a mutex triggers, the job's block is executed with the mutex around it, preventing other jobs with the same mutex from entering (it makes the other jobs wait until it exits the mutex).

This is different from overlap: false, which is, first, limited to instances of the same job, and, second, doesn't make the incoming job instance block/wait but give up.

:mutex accepts a mutex instance or a mutex name (String). It also accept an array of mutex names / mutex instances. It allows for complex relations between jobs.

Array of mutexes: original idea and implementation by Rainux Luo

Note: creating lots of different mutexes is OK. Rufus-scheduler will place them in its Scheduler#mutexes hash... And they won't get garbage collected.

The :overlap option is considered before the :mutex option when the scheduler is reviewing jobs for triggering.

timeout: duration or point in time

It's OK to specify a timeout when scheduling some work. After the time specified, it gets interrupted via a Rufus::Scheduler::TimeoutError.

scheduler.in '10d', timeout: '1d' do
  begin
    # ... do something
  rescue Rufus::Scheduler::TimeoutError
    # ... that something got interrupted after 1 day
  end
end

The :timeout option accepts either a duration (like "1d" or "2w3d") or a point in time (like "2013/12/12 12:00").

:first_at, :first_in, :first, :first_time

This option is for repeat jobs (cron / every) only.

It's used to specify the first time after which the repeat job should trigger for the first time.

In the case of an "every" job, this will be the first time (modulo the scheduler frequency) the job triggers. For a "cron" job as well, the :first will point to the first time the job has to trigger, the following trigger times are then determined by the cron string.

scheduler.every '2d', first_at: Time.now + 10 * 3600 do
  # ... every two days, but start in 10 hours
end

scheduler.every '2d', first_in: '10h' do
  # ... every two days, but start in 10 hours
end

scheduler.cron '00 14 * * *', first_in: '3d' do
  # ... every day at 14h00, but start after 3 * 24 hours
end

:first, :first_at and :first_in all accept a point in time or a duration (number or time string). Use the symbol you think makes your schedule more readable.

Note: it's OK to change the first_at (a Time instance) directly:

job.first_at = Time.now + 10
job.first_at = Rufus::Scheduler.parse('2029-12-12')

The first argument (in all its flavours) accepts a :now or :immediately value. That schedules the first occurrence for immediate triggering. Consider:

require 'rufus-scheduler'

s = Rufus::Scheduler.new

n = Time.now; p [ :scheduled_at, n, n.to_f ]

s.every '3s', first: :now do
  n = Time.now; p [ :in, n, n.to_f ]
end

s.join

that'll output something like:

[:scheduled_at, 2014-01-22 22:21:21 +0900, 1390396881.344438]
[:in, 2014-01-22 22:21:21 +0900, 1390396881.6453865]
[:in, 2014-01-22 22:21:24 +0900, 1390396884.648807]
[:in, 2014-01-22 22:21:27 +0900, 1390396887.651686]
[:in, 2014-01-22 22:21:30 +0900, 1390396890.6571937]
...

:last_at, :last_in, :last

This option is for repeat jobs (cron / every) only.

It indicates the point in time after which the job should unschedule itself.

scheduler.cron '5 23 * * *', last_in: '10d' do
  # ... do something every evening at 23:05 for 10 days
end

scheduler.every '10m', last_at: Time.now + 10 * 3600 do
  # ... do something every 10 minutes for 10 hours
end

scheduler.every '10m', last_in: 10 * 3600 do
  # ... do something every 10 minutes for 10 hours
end

:last, :last_at and :last_in all accept a point in time or a duration (number or time string). Use the symbol you think makes your schedule more readable.

Note: it's OK to change the last_at (nil or a Time instance) directly:

job.last_at = nil
  # remove the "last" bound

job.last_at = Rufus::Scheduler.parse('2029-12-12')
  # set the last bound

times: nb of times (before auto-unscheduling)

One can tell how many times a repeat job (CronJob or EveryJob) is to execute before unscheduling by itself.

scheduler.every '2d', times: 10 do
  # ... do something every two days, but not more than 10 times
end

scheduler.cron '0 23 * * *', times: 31 do
  # ... do something every day at 23:00 but do it no more than 31 times
end

It's OK to assign nil to :times to make sure the repeat job is not limited. It's useful when the :times is determined at scheduling time.

scheduler.cron '0 23 * * *', times: (nolimit ? nil : 10) do
  # ...
end

The value set by :times is accessible in the job. It can be modified anytime.

job =
  scheduler.cron '0 23 * * *' do
    # ...
  end

# later on...

job.times = 10
  # 10 days and it will be over

Job methods

When calling a schedule method, the id (String) of the job is returned. Longer schedule methods return Job instances directly. Calling the shorter schedule methods with the job: true also returns Job instances instead of Job ids (Strings).

  require 'rufus-scheduler'

  scheduler = Rufus::Scheduler.new

  job_id =
    scheduler.in '10d' do
      # ...
    end

  job =
    scheduler.schedule_in '1w' do
      # ...
    end

  job =
    scheduler.in '1w', job: true do
      # ...
    end

Those Job instances have a few interesting methods / properties:

id, job_id

Returns the job id.

job = scheduler.schedule_in('10d') do; end
job.id
  # => "in_1374072446.8923042_0.0_0"

scheduler

Returns the scheduler instance itself.

opts

Returns the options passed at the Job creation.

job = scheduler.schedule_in('10d', tag: 'hello') do; end
job.opts
  # => { :tag => 'hello' }

original

Returns the original schedule.

job = scheduler.schedule_in('10d', tag: 'hello') do; end
job.original
  # => '10d'

callable, handler

callable() returns the scheduled block (or the call method of the callable object passed in lieu of a block)

handler() returns nil if a block was scheduled and the instance scheduled otherwise.

# when passing a block

job =
  scheduler.schedule_in('10d') do
    # ...
  end

job.handler
  # => nil
job.callable
  # => #<Proc:0x00000001dc6f58@/home/jmettraux/whatever.rb:115>

and

# when passing something else than a block

class MyHandler
  attr_reader :counter
  def initialize
    @counter = 0
  end
  def call(job, time)
    @counter = @counter + 1
  end
end

job = scheduler.schedule_in('10d', MyHandler.new)

job.handler
  # => #<Method: MyHandler#call>
job.callable
  # => #<MyHandler:0x0000000163ae88 @counter=0>

source_location

Added to rufus-scheduler 3.8.0.

Returns the array [ 'path/to/file.rb', 123 ] like Proc#source_location does.

require 'rufus-scheduler'

scheduler = Rufus::Scheduler.new

job = scheduler.schedule_every('2h') { p Time.now }

p job.source_location
  # ==> [ '/home/jmettraux/rufus-scheduler/test.rb', 6 ]

scheduled_at

Returns the Time instance when the job got created.

job = scheduler.schedule_in('10d', tag: 'hello') do; end
job.scheduled_at
  # => 2013-07-17 23:48:54 +0900

last_time

Returns the last time the job triggered (is usually nil for AtJob and InJob).

job = scheduler.schedule_every('10s') do; end

job.scheduled_at
  # => 2013-07-17 23:48:54 +0900
job.last_time
  # => nil (since we've just scheduled it)

# after 10 seconds

job.scheduled_at
  # => 2013-07-17 23:48:54 +0900 (same as above)
job.last_time
  # => 2013-07-17 23:49:04 +0900

previous_time

Returns the previous #next_time

scheduler.every('10s') do |job|
  puts "job scheduled for #{job.previous_time} triggered at #{Time.now}"
  puts "next time will be around #{job.next_time}"
  puts "."
end

last_work_time, mean_work_time

The job keeps track of how long its work was in the last_work_time attribute. For a one time job (in, at) it's probably not very useful.

The attribute mean_work_time contains a computed mean work time. It's recomputed after every run (if it's a repeat job).

next_times(n)

Returns an array of EtOrbi::EoTime instances (Time instances with a designated time zone), listing the n next occurrences for this job.

Please note that for "interval" jobs, a mean work time is computed each time and it's used by this #next_times(n) method to approximate the next times beyond the immediate next time.

unschedule

Unschedule the job, preventing it from firing again and removing it from the schedule. This doesn't prevent a running thread for this job to run until its end.

threads

Returns the list of threads currently "hosting" runs of this Job instance.

kill

Interrupts all the work threads currently running for this job instance. They discard their work and are free for their next run (of whatever job).

Note: this doesn't unschedule the Job instance.

Note: if the job is pooled for another run, a free work thread will probably pick up that next run and the job will appear as running again. You'd have to unschedule and kill to make sure the job doesn't run again.

running?

Returns true if there is at least one running Thread hosting a run of this Job instance.

scheduled?

Returns true if the job is scheduled (is due to trigger). For repeat jobs it should return true until the job gets unscheduled. "at" and "in" jobs will respond with false as soon as they start running (execution triggered).

pause, resume, paused?, paused_at

These four methods are only available to CronJob, EveryJob and IntervalJob instances. One can pause or resume such jobs thanks to these methods.

job =
  scheduler.schedule_every('10s') do
    # ...
  end

job.pause
  # => 2013-07-20 01:22:22 +0900
job.paused?
  # => true
job.paused_at
  # => 2013-07-20 01:22:22 +0900

job.resume
  # => nil

tags

Returns the list of tags attached to this Job instance.

By default, returns an empty array.

job = scheduler.schedule_in('10d') do; end
job.tags
  # => []

job = scheduler.schedule_in('10d', tag: 'hello') do; end
job.tags
  # => [ 'hello' ]

[]=, [], key?, has_key?, keys, values, and entries

Threads have thread-local variables, similarly Rufus-scheduler jobs have job-local variables. Those are more like a dict with thread-safe access.

job =
  @scheduler.schedule_every '1s' do |job|
    job[:timestamp] = Time.now.to_f
    job[:counter] ||= 0
    job[:counter] += 1
  end

sleep 3.6

job[:counter]
  # => 3

job.key?(:timestamp) # => true
job.has_key?(:timestamp) # => true
job.keys # => [ :timestamp, :counter ]

Locals can be set at schedule time:

job0 =
  @scheduler.schedule_cron '*/15 12 * * *', locals: { a: 0 } do
    # ...
  end
job1 =
  @scheduler.schedule_cron '*/15 13 * * *', l: { a: 1 } do
    # ...
  end

One can fetch the Hash directly with Job#locals. Of course, direct manipulation is not thread-safe.

job.locals.entries do |k, v|
  p "#{k}: #{v}"
end

call

Job instances have a #call method. It simply calls the scheduled block or callable immediately.

job =
  @scheduler.schedule_every '10m' do |job|
    # ...
  end

job.call

Warning: the Scheduler#on_error handler is not involved. Error handling is the responsibility of the caller.

If the call has to be rescued by the error handler of the scheduler, call(true) might help:

require 'rufus-scheduler'

s = Rufus::Scheduler.new

def s.on_error(job, err)
  if job
    p [ 'error in scheduled job', job.class, job.original, err.message ]
  else
    p [ 'error while scheduling', err.message ]
  end
rescue
  p $!
end

job =
  s.schedule_in('1d') do
    fail 'again'
  end

job.call(true)
  #
  # true lets the error_handler deal with error in the job call

AtJob and InJob methods

time

Returns when the job will trigger (hopefully).

next_time

An alias for time.

EveryJob, IntervalJob and CronJob methods

next_time

Returns the next time the job will trigger (hopefully).

count

Returns how many times the job fired.

EveryJob methods

frequency

It returns the scheduling frequency. For a job scheduled "every 20s", it's 20.

It's used to determine if the job frequency is higher than the scheduler frequency (it raises an ArgumentError if that is the case).

IntervalJob methods

interval

Returns the interval scheduled between each execution of the job.

Every jobs use a time duration between each start of their execution, while interval jobs use a time duration between the end of an execution and the start of the next.

CronJob methods

brute_frequency

An expensive method to run, it's brute. It caches its results. By default it runs for 2017 (a non leap-year).

  require 'rufus-scheduler'

  Rufus::Scheduler.parse('* * * * *').brute_frequency
    #
    # => #<Fugit::Cron::Frequency:0x00007fdf4520c5e8
    #      @span=31536000.0, @delta_min=60, @delta_max=60,
    #      @occurrences=525600, @span_years=1.0, @yearly_occurrences=525600.0>
      #
      # Occurs 525600 times in a span of 1 year (2017) and 1 day.
      # There are least 60 seconds between "triggers" and at most 60 seconds.

  Rufus::Scheduler.parse('0 12 * * *').brute_frequency
    # => #<Fugit::Cron::Frequency:0x00007fdf451ec6d0
    #      @span=31536000.0, @delta_min=86400, @delta_max=86400,
    #      @occurrences=365, @span_years=1.0, @yearly_occurrences=365.0>
  Rufus::Scheduler.parse('0 12 * * *').brute_frequency.to_debug_s
    # => "dmin: 1D, dmax: 1D, ocs: 365, spn: 52W1D, spnys: 1, yocs: 365"
      #
      # 365 occurrences, at most 1 day between each, at least 1 day.

The CronJob#frequency method found in rufus-scheduler < 3.5 has been retired.

looking up jobs

Scheduler#job(job_id)

The scheduler #job(job_id) method can be used to look up Job instances.

  require 'rufus-scheduler'

  scheduler = Rufus::Scheduler.new

  job_id =
    scheduler.in '10d' do
      # ...
    end

  # later on...

  job = scheduler.job(job_id)

Scheduler #jobs #at_jobs #in_jobs #every_jobs #interval_jobs and #cron_jobs

Are methods for looking up lists of scheduled Job instances.

Here is an example:

  #
  # let's unschedule all the at jobs

  scheduler.at_jobs.each(&:unschedule)

Scheduler#jobs(tag: / tags: x)

When scheduling a job, one can specify one or more tags attached to the job. These can be used to look up the job later on.

  scheduler.in '10d', tag: 'main_process' do
    # ...
  end
  scheduler.in '10d', tags: [ 'main_process', 'side_dish' ] do
    # ...
  end

  # ...

  jobs = scheduler.jobs(tag: 'main_process')
    # find all the jobs with the 'main_process' tag

  jobs = scheduler.jobs(tags: [ 'main_process', 'side_dish' ]
    # find all the jobs with the 'main_process' AND 'side_dish' tags

Scheduler#running_jobs

Returns the list of Job instance that have currently running instances.

Whereas other "_jobs" method scan the scheduled job list, this method scans the thread list to find the job. It thus comprises jobs that are running but are not scheduled anymore (that happens for at and in jobs).

misc Scheduler methods

Scheduler#unschedule(job_or_job_id)

Unschedule a job given directly or by its id.

Scheduler#shutdown

Shuts down the scheduler, ceases any scheduler/triggering activity.

Scheduler#shutdown(:wait)

Shuts down the scheduler, waits (blocks) until all the jobs cease running.

Scheduler#shutdown(wait: n)

Shuts down the scheduler, waits (blocks) at most n seconds until all the jobs cease running. (Jobs are killed after n seconds have elapsed).

Scheduler#shutdown(:kill)

Kills all the job (threads) and then shuts the scheduler down. Radical.

Scheduler#down?

Returns true if the scheduler has been shut down.

Scheduler#started_at

Returns the Time instance at which the scheduler got started.

Scheduler #uptime / #uptime_s

Returns since the count of seconds for which the scheduler has been running.

#uptime_s returns this count in a String easier to grasp for humans, like "3d12m45s123".

Scheduler#join

Lets the current thread join the scheduling thread in rufus-scheduler. The thread comes back when the scheduler gets shut down.

#join is mostly used in standalone scheduling script (or tiny one file examples). Calling #join from a web application initializer will probably hijack the main thread and prevent the web application from being served. Do not put a #join in such a web application initializer file.

Scheduler#threads

Returns all the threads associated with the scheduler, including the scheduler thread itself.

Scheduler#work_threads(query=:all/:active/:vacant)

Lists the work threads associated with the scheduler. The query option defaults to :all.

• :all : all the work threads
• :active : all the work threads currently running a Job
• :vacant : all the work threads currently not running a Job

Note that the main schedule thread will be returned if it is currently running a Job (ie one of those blocking: true jobs).

Scheduler#scheduled?(job_or_job_id)

Returns true if the arg is a currently scheduled job (see Job#scheduled?).

Scheduler#occurrences(time0, time1)

Returns a hash { job => [ t0, t1, ... ] } mapping jobs to their potential trigger time within the [ time0, time1 ] span.

Please note that, for interval jobs, the #mean_work_time is used, so the result is only a prediction.

Scheduler#timeline(time0, time1)

Like #occurrences but returns a list [ [ t0, job0 ], [ t1, job1 ], ... ] of time + job pairs.

dealing with job errors

The easy, job-granular way of dealing with errors is to rescue and deal with them immediately. The two next sections show examples. Skip them for explanations on how to deal with errors at the scheduler level.

block jobs

As said, jobs could take care of their errors themselves.

scheduler.every '10m' do
  begin
    # do something that might fail...
  rescue => e
    $stderr.puts '-' * 80
    $stderr.puts e.message
    $stderr.puts e.stacktrace
    $stderr.puts '-' * 80
  end
end

callable jobs

Jobs are not only shrunk to blocks, here is how the above would look like with a dedicated class.

scheduler.every '10m', Class.new do
  def call(job)
    # do something that might fail...
  rescue => e
    $stderr.puts '-' * 80
    $stderr.puts e.message
    $stderr.puts e.stacktrace
    $stderr.puts '-' * 80
  end
end

TODO: talk about callable#on_error (if implemented)

(see scheduling handler instances and scheduling handler classes for more about those "callable jobs")

Rufus::Scheduler#stderr=

By default, rufus-scheduler intercepts all errors (that inherit from StandardError) and dumps abundant details to $stderr.

If, for example, you'd like to divert that flow to another file (descriptor), you can reassign $stderr for the current Ruby process

$stderr = File.open('/var/log/myapplication.log', 'ab')

or, you can limit that reassignement to the scheduler itself

scheduler.stderr = File.open('/var/log/myapplication.log', 'ab')

Rufus::Scheduler#on_error(job, error)

We've just seen that, by default, rufus-scheduler dumps error information to $stderr. If one needs to completely change what happens in case of error, it's OK to overwrite #on_error

def scheduler.on_error(job, error)

  Logger.warn("intercepted error in #{job.id}: #{error.message}")
end

On Rails, the on_error method redefinition might look like:

def scheduler.on_error(job, error)

  Rails.logger.error(
    "err#{error.object_id} rufus-scheduler intercepted #{error.inspect}" +
    " in job #{job.inspect}")
  error.backtrace.each_with_index do |line, i|
    Rails.logger.error(
      "err#{error.object_id} #{i}: #{line}")
  end
end

Callbacks

Rufus::Scheduler #on_pre_trigger and #on_post_trigger callbacks

One can bind callbacks before and after jobs trigger:

s = Rufus::Scheduler.new

def s.on_pre_trigger(job, trigger_time)
  puts "triggering job #{job.id}..."
end

def s.on_post_trigger(job, trigger_time)
  puts "triggered job #{job.id}."
end

s.every '1s' do
  # ...
end

The trigger_time is the time at which the job triggers. It might be a bit before Time.now.

Warning: these two callbacks are executed in the scheduler thread, not in the work threads (the threads where the job execution really happens).

Rufus::Scheduler#around_trigger

One can create an around callback which will wrap a job:

def s.around_trigger(job)
  t = Time.now
  puts "Starting job #{job.id}..."
  yield
  puts "job #{job.id} finished in #{Time.now-t} seconds."
end

The around callback is executed in the thread.

Rufus::Scheduler#on_pre_trigger as a guard

Returning false in on_pre_trigger will prevent the job from triggering. Returning anything else (nil, -1, true, ...) will let the job trigger.

Note: your business logic should go in the scheduled block itself (or the scheduled instance). Don't put business logic in on_pre_trigger. Return false for admin reasons (backend down, etc), not for business reasons that are tied to the job itself.

def s.on_pre_trigger(job, trigger_time)

  return false if Backend.down?

  puts "triggering job #{job.id}..."
end

Rufus::Scheduler.new options

:frequency

By default, rufus-scheduler sleeps 0.300 second between every step. At each step it checks for jobs to trigger and so on.

The :frequency option lets you change that 0.300 second to something else.

scheduler = Rufus::Scheduler.new(frequency: 5)

It's OK to use a time string to specify the frequency.

scheduler = Rufus::Scheduler.new(frequency: '2h10m')
  # this scheduler will sleep 2 hours and 10 minutes between every "step"

Use with care.

lockfile: "mylockfile.txt"

This feature only works on OSes that support the flock (man 2 flock) call.

Starting the scheduler with lockfile: '.rufus-scheduler.lock' will make the scheduler attempt to create and lock the file .rufus-scheduler.lock in the current working directory. If that fails, the scheduler will not start.

The idea is to guarantee only one scheduler (in a group of schedulers sharing the same lockfile) is running.

This is useful in environments where the Ruby process holding the scheduler gets started multiple times.

If the lockfile mechanism here is not sufficient, you can plug your custom mechanism. It's explained in advanced lock schemes below.

:scheduler_lock

(since rufus-scheduler 3.0.9)

The scheduler lock is an object that responds to #lock and #unlock. The scheduler calls #lock when starting up. If the answer is false, the scheduler stops its initialization work and won't schedule anything.

Here is a sample of a scheduler lock that only lets the scheduler on host "coffee.example.com" start:

class HostLock
  def initialize(lock_name)
    @lock_name = lock_name
  end
  def lock
    @lock_name == `hostname -f`.strip
  end
  def unlock
    true
  end
end

scheduler =
  Rufus::Scheduler.new(scheduler_lock: HostLock.new('coffee.example.com'))

By default, the scheduler_lock is an instance of Rufus::Scheduler::NullLock, with a #lock that returns true.

:trigger_lock

(since rufus-scheduler 3.0.9)

The trigger lock in an object that responds to #lock. The scheduler calls that method on the job lock right before triggering any job. If the answer is false, the trigger doesn't happen, the job is not done (at least not in this scheduler).

Here is a (stupid) PingLock example, it'll only trigger if an "other host" is not responding to ping. Do not use that in production, you don't want to fork a ping process for each trigger attempt...

class PingLock
  def initialize(other_host)
    @other_host = other_host
  end
  def lock
    ! system("ping -c 1 #{@other_host}")
  end
end

scheduler =
  Rufus::Scheduler.new(trigger_lock: PingLock.new('main.example.com'))

By default, the trigger_lock is an instance of Rufus::Scheduler::NullLock, with a #lock that always returns true.

As explained in advanced lock schemes, another way to tune that behaviour is by overriding the scheduler's #confirm_lock method. (You could also do that with an #on_pre_trigger callback).

:max_work_threads

In rufus-scheduler 2.x, by default, each job triggering received its own, brand new, thread of execution. In rufus-scheduler 3.x, execution happens in a pooled work thread. The max work thread count (the pool size) defaults to 28.

One can set this maximum value when starting the scheduler.

scheduler = Rufus::Scheduler.new(max_work_threads: 77)

It's OK to increase the :max_work_threads of a running scheduler.

scheduler.max_work_threads += 10

Rufus::Scheduler.singleton

Do not want to store a reference to your rufus-scheduler instance? Then Rufus::Scheduler.singleton can help, it returns a singleton instance of the scheduler, initialized the first time this class method is called.

Rufus::Scheduler.singleton.every '10s' { puts "hello, world!" }

It's OK to pass initialization arguments (like :frequency or :max_work_threads) but they will only be taken into account the first time .singleton is called.

Rufus::Scheduler.singleton(max_work_threads: 77)
Rufus::Scheduler.singleton(max_work_threads: 277) # no effect

The .s is a shortcut for .singleton.

Rufus::Scheduler.s.every '10s' { puts "hello, world!" }

advanced lock schemes

As seen above, rufus-scheduler proposes the :lockfile system out of the box. If in a group of schedulers only one is supposed to run, the lockfile mechanism prevents schedulers that have not set/created the lockfile from running.

There are situations where this is not sufficient.

By overriding #lock and #unlock, one can customize how schedulers lock.

This example was provided by Eric Lindvall:

class ZookeptScheduler < Rufus::Scheduler

  def initialize(zookeeper, opts={})
    @zk = zookeeper
    super(opts)
  end

  def lock
    @zk_locker = @zk.exclusive_locker('scheduler')
    @zk_locker.lock # returns true if the lock was acquired, false else
  end

  def unlock
    @zk_locker.unlock
  end

  def confirm_lock
    return false if down?
    @zk_locker.assert!
  rescue ZK::Exceptions::LockAssertionFailedError => e
    # we've lost the lock, shutdown (and return false to at least prevent
    # this job from triggering
    shutdown
    false
  end
end

This uses a zookeeper to make sure only one scheduler in a group of distributed schedulers runs.

The methods #lock and #unlock are overridden and #confirm_lock is provided, to make sure that the lock is still valid.

The #confirm_lock method is called right before a job triggers (if it is provided). The more generic callback #on_pre_trigger is called right after #confirm_lock.

:scheduler_lock and :trigger_lock

(introduced in rufus-scheduler 3.0.9).

Another way of prodiving #lock, #unlock and #confirm_lock to a rufus-scheduler is by using the :scheduler_lock and :trigger_lock options.

See :trigger_lock and :scheduler_lock.

The scheduler lock may be used to prevent a scheduler from starting, while a trigger lock prevents individual jobs from triggering (the scheduler goes on scheduling).

One has to be careful with what goes in #confirm_lock or in a trigger lock, as it gets called before each trigger.

Warning: you may think you're heading towards "high availability" by using a trigger lock and having lots of schedulers at hand. It may be so if you limit yourself to scheduling the same set of jobs at scheduler startup. But if you add schedules at runtime, they stay local to their scheduler. There is no magic that propagates the jobs to all the schedulers in your pack.

parsing cronlines and time strings

(Please note that fugit does the heavy-lifting parsing work for rufus-scheduler).

Rufus::Scheduler provides a class method .parse to parse time durations and cron strings. It's what it's using when receiving schedules. One can use it directly (no need to instantiate a Scheduler).

require 'rufus-scheduler'

Rufus::Scheduler.parse('1w2d')
  # => 777600.0
Rufus::Scheduler.parse('1.0w1.0d')
  # => 777600.0

Rufus::Scheduler.parse('Sun Nov 18 16:01:00 2012').strftime('%c')
  # => 'Sun Nov 18 16:01:00 2012'

Rufus::Scheduler.parse('Sun Nov 18 16:01:00 2012 Europe/Berlin').strftime('%c %z')
  # => 'Sun Nov 18 15:01:00 2012 +0000'

Rufus::Scheduler.parse(0.1)
  # => 0.1

Rufus::Scheduler.parse('* * * * *')
  # => #<Fugit::Cron:0x00007fb7a3045508
  #      @original="* * * * *", @cron_s=nil,
  #      @seconds=[0], @minutes=nil, @hours=nil, @monthdays=nil, @months=nil,
  #      @weekdays=nil, @zone=nil, @timezone=nil>

It returns a number when the input is a duration and a Fugit::Cron instance when the input is a cron string.

It will raise an ArgumentError if it can't parse the input.

Beyond .parse, there are also .parse_cron and .parse_duration, for finer granularity.

There is an interesting helper method named .to_duration_hash:

require 'rufus-scheduler'

Rufus::Scheduler.to_duration_hash(60)
  # => { :m => 1 }
Rufus::Scheduler.to_duration_hash(62.127)
  # => { :m => 1, :s => 2, :ms => 127 }

Rufus::Scheduler.to_duration_hash(62.127, drop_seconds: true)
  # => { :m => 1 }

cronline notations specific to rufus-scheduler

first Monday, last Sunday et al

To schedule something at noon every first Monday of the month:

scheduler.cron('00 12 * * mon#1') do
  # ...
end

To schedule something at noon the last Sunday of every month:

scheduler.cron('00 12 * * sun#-1') do
  # ...
end
#
# OR
#
scheduler.cron('00 12 * * sun#L') do
  # ...
end

Such cronlines can be tested with scripts like:

require 'rufus-scheduler'

Time.now
  # => 2013-10-26 07:07:08 +0900
Rufus::Scheduler.parse('* * * * mon#1').next_time.to_s
  # => 2013-11-04 00:00:00 +0900

L (last day of month)

L can be used in the "day" slot:

In this example, the cronline is supposed to trigger every last day of the month at noon:

require 'rufus-scheduler'
Time.now
  # => 2013-10-26 07:22:09 +0900
Rufus::Scheduler.parse('00 12 L * *').next_time.to_s
  # => 2013-10-31 12:00:00 +0900

negative day (x days before the end of the month)

It's OK to pass negative values in the "day" slot:

scheduler.cron '0 0 -5 * *' do
  # do it at 00h00 5 days before the end of the month...
end

Negative ranges (-10--5-: 10 days before the end of the month to 5 days before the end of the month) are OK, but mixed positive / negative ranges will raise an ArgumentError.

Negative ranges with increments (-10---2/2) are accepted as well.

Descending day ranges are not accepted (10-8 or -8--10 for example).

a note about timezones

Cron schedules and at schedules support the specification of a timezone.

scheduler.cron '0 22 * * 1-5 America/Chicago' do
  # the job...
end

scheduler.at '2013-12-12 14:00 Pacific/Samoa' do
  puts "it's tea time!"
end

# or even

Rufus::Scheduler.parse("2013-12-12 14:00 Pacific/Saipan")
  # => #<Rufus::Scheduler::ZoTime:0x007fb424abf4e8 @seconds=1386820800.0, @zone=#<TZInfo::DataTimezone: Pacific/Saipan>, @time=nil>

I get "zotime.rb:41:in `initialize': cannot determine timezone from nil"

For when you see an error like:

rufus-scheduler/lib/rufus/scheduler/zotime.rb:41:
  in `initialize':
    cannot determine timezone from nil (etz:nil,tnz:"中国标准时间",tzid:nil)
      (ArgumentError)
    from rufus-scheduler/lib/rufus/scheduler/zotime.rb:198:in `new'
    from rufus-scheduler/lib/rufus/scheduler/zotime.rb:198:in `now'
    from rufus-scheduler/lib/rufus/scheduler.rb:561:in `start'
    ...

It may happen on Windows or on systems that poorly hint to Ruby which timezone to use. It should be solved by setting explicitly the ENV['TZ'] before the scheduler instantiation:

ENV['TZ'] = 'Asia/Shanghai'
scheduler = Rufus::Scheduler.new
scheduler.every '2s' do
  puts "#{Time.now} Hello #{ENV['TZ']}!"
end

On Rails you might want to try with:

ENV['TZ'] = Time.zone.name # Rails only
scheduler = Rufus::Scheduler.new
scheduler.every '2s' do
  puts "#{Time.now} Hello #{ENV['TZ']}!"
end

(Hat tip to Alexander in gh-230)

Rails sets its timezone under config/application.rb.

Rufus-Scheduler 3.3.3 detects the presence of Rails and uses its timezone setting (tested with Rails 4), so setting ENV['TZ'] should not be necessary.

The value can be determined thanks to https://en.wikipedia.org/wiki/List_of_tz_database_time_zones.

Use a "continent/city" identifier (for example "Asia/Shanghai"). Do not use an abbreviation (not "CST") and do not use a local time zone name (not "中国标准时间" nor "Eastern Standard Time" which, for instance, points to a time zone in America and to another one in Australia...).

If the error persists (and especially on Windows), try to add the tzinfo-data to your Gemfile, as in:

gem 'tzinfo-data'

or by manually requiring it before requiring rufus-scheduler (if you don't use Bundler):

require 'tzinfo/data'
require 'rufus-scheduler'

so Rails?

Yes, I know, all of the above is boring and you're only looking for a snippet to paste in your Ruby-on-Rails application to schedule...

Here is an example initializer:

#
# config/initializers/scheduler.rb

require 'rufus-scheduler'

# Let's use the rufus-scheduler singleton
#
s = Rufus::Scheduler.singleton


# Stupid recurrent task...
#
s.every '1m' do

  Rails.logger.info "hello, it's #{Time.now}"
  Rails.logger.flush
end

And now you tell me that this is good, but you want to schedule stuff from your controller.

Maybe:

class ScheController < ApplicationController

  # GET /sche/
  #
  def index

    job_id =
      Rufus::Scheduler.singleton.in '5s' do
        Rails.logger.info "time flies, it's now #{Time.now}"
      end

    render text: "scheduled job #{job_id}"
  end
end

The rufus-scheduler singleton is instantiated in the config/initializers/scheduler.rb file, it's then available throughout the webapp via Rufus::Scheduler.singleton.

Warning: this works well with single-process Ruby servers like Webrick and Thin. Using rufus-scheduler with Passenger or Unicorn requires a bit more knowledge and tuning, gently provided by a bit of googling and reading, see Faq above.

avoid scheduling when running the Ruby on Rails console

(Written in reply to gh-186)

If you don't want rufus-scheduler to trigger anything while running the Ruby on Rails console, running for tests/specs, or running from a Rake task, you can insert a conditional return statement before jobs are added to the scheduler instance:

#
# config/initializers/scheduler.rb

require 'rufus-scheduler'

return if defined?(Rails::Console) || Rails.env.test? || File.split($PROGRAM_NAME).last == 'rake'
  #
  # do not schedule when Rails is run from its console, for a test/spec, or
  # from a Rake task

# return if $PROGRAM_NAME.include?('spring')
  #
  # see https://github.com/jmettraux/rufus-scheduler/issues/186

s = Rufus::Scheduler.singleton

s.every '1m' do
  Rails.logger.info "hello, it's #{Time.now}"
  Rails.logger.flush
end

(Beware later version of Rails where Spring takes care pre-running the initializers. Running spring stop or disabling Spring might be necessary in some cases to see changes to initializers being taken into account.)

rails server -d

(Written in reply to https://github.com/jmettraux/rufus-scheduler/issues/165 )

There is the handy rails server -d that starts a development Rails as a daemon. The annoying thing is that the scheduler as seen above is started in the main process that then gets forked and daemonized. The rufus-scheduler thread (and any other thread) gets lost, no scheduling happens.

I avoid running -d in development mode and bother about daemonizing only for production deployment.

These are two well crafted articles on process daemonization, please read them:

• https://www.mikeperham.com/2014/09/22/dont-daemonize-your-daemons/
• https://www.mikeperham.com/2014/07/07/use-runit/

If, anyway, you need something like rails server -d, why not try bundle exec unicorn -D instead? In my (limited) experience, it worked out of the box (well, had to add gem 'unicorn' to Gemfile first).

executor / reloader

You might benefit from wraping your scheduled code in the executor or reloader. Read more here: https://guides.rubyonrails.org/threading_and_code_execution.html

support

see getting help above.

---

Author: jmettraux
Source code: https://github.com/jmettraux/rufus-scheduler
License: MIT license

#ruby 

10 Popular Libraries for Scheduling Jobs in Go

In today's post we will learn about 10 Popular Libraries for Scheduling Jobs in Go.

In Go and Golang programming, a scheduler is responsible for distributing jobs in a multiprocessing environment. When the available resources are limited, it is the task of the scheduler to manage the work that needs to be done in the most efficient way. In Go, the scheduler is responsible for scheduling goroutines, which is particularly useful in concurrency. Goroutines are like OS threads, but they are much lighter weight. However, goroutines always take the help of the underlying OS thread model and the scheduler it works on is at a much higher level than the OS scheduler. This Go programming tutorial provides a quick look at the concepts behind the Go scheduler.

Table of contents:

• Cdule - Job scheduler library with database support
• Cheek - A simple crontab like scheduler that aims to offer a KISS approach to job scheduling.
• Clockwerk - Go package to schedule periodic jobs using a simple, fluent syntax.
• Cronticker - A ticker implementation to support cron schedules.
• Dagu - No-code workflow executor. it executes DAGs defined in a simple YAML format.
• Go-cron - Simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.
• Go-quartz - Simple, zero-dependency scheduling library for Go.
• Gocron - Easy and fluent Go job scheduling. This is an actively maintained fork of jasonlvhit/gocron.
• Goflow - A workflow orchestrator and scheduler for rapid prototyping of ETL/ML/AI pipelines.
• Gron - Define time-based tasks using a simple Go API and Gron’s scheduler will run them accordingly.

1 - Cdule: Job scheduler library with database support.

cdule (pronounce as Schedule)

Golang based scheduler library with database support. Users could use any database which is supported by gorm.io.

To Download the cdule library

go get github.com/deepaksinghvi/cdule

Usage Instruction

In order to schedule jobs with cdule, user needs to

1. Configure persistence
2. Implement cdule.Job Interface &
3. Schedule job with required cron expression.

Job will be persisted in the jobs table.
Next execution would be persisted in schedules tables.
Job history would be persisted and maintained in job_histories table.

Configuration

User needs to create a resources/config.yml in their project home directory with the followling keys

• cduletype
• dburl
• cduleconsistency

cduletype is used to specify whether it is an In-Memory or Database based configuration. Possible values are DATABASE and MEMORY. dburl is the database connection url. cduleconsistency is for reserved for future usage.

config.yml for postgressql based configuration

cduletype: DATABASE
dburl: postgres://cduleuser:cdulepassword@localhost:5432/cdule?sslmode=disable
cduleconsistency: AT_MOST_ONCE

config.yml for sqlite based in-memory configuration

cduletype: MEMORY
dburl: /Users/dsinghvi/sqlite.db
cduleconsistency: AT_MOST_ONCE

Job Interface Implementation

var testJobData map[string]string

type TestJob struct {
	Job cdule.Job
}

func (m TestJob) Execute(jobData map[string]string) {
	log.Info("In TestJob")
	for k, v := range jobData {
		valNum, err := strconv.Atoi(v)
		if nil == err {
			jobData[k] = strconv.Itoa(valNum + 1)
		} else {
			log.Error(err)
		}

	}
	testJobData = jobData
}

func (m TestJob) JobName() string {
	return "job.TestJob"
}

func (m TestJob) GetJobData() map[string]string {
	return testJobData
}

View on Github

2 - Cheek: A simple crontab like scheduler that aims to offer a KISS approach to job scheduling.

cheek, of course, stands for Crontab-like scHeduler for Effective Execution of tasKs. cheek is a KISS approach to crontab-like job scheduling. It was born out of a (/my?) frustration about the big gap between a lightweight crontab and full-fledged solutions like Airflow.

cheek aims to be a KISS approach to job scheduling. Focus is on the KISS approach not to necessarily do this in the most robust way possible.

Getting started

Fetch the latest version for your system below.

You can (for example) fetch it like below, make it executable and run it. Optionally put the cheek on your PATH.

curl https://storage.googleapis.com/better-unified/darwin/amd64/cheek -o cheek
chmod +x cheek
./cheek

Create a schedule specification using the below YAML structure:

tz_location: Europe/Brussels
jobs:
  foo:
    command: date
    cron: "* * * * *"
    on_success:
      trigger_job:
        - bar
  bar:
    command:
      - echo
      - bar
      - foo
  coffee:
    command: this fails
    cron: "* * * * *"
    retries: 3
    on_error:
      notify_webhook:
        - https://webhook.site/4b732eb4-ba10-4a84-8f6b-30167b2f2762

If your command requires arguments, please make sure to pass them as an array like in foo_job.

Note that you can set tz_location if the system time of where you run your service is not to your liking.

Scheduler

The core of cheek consists of a scheduler that uses a schedule specified in a yaml file to triggers jobs when they are due.

You can launch the scheduler via:

cheek run ./path/to/my-schedule.yaml

Check out cheek run --help for configuration options.

View on Github

3 - Clockwerk: Go package to schedule periodic jobs using a simple, fluent syntax.

Job Scheduling Library

clockwerk allows you to schedule periodic jobs using a simple, fluent syntax.

Usage

go get github.com/onatm/clockwerk

package main

import (
  "fmt"
  "time"
  "github.com/onatm/clockwerk"
)

type DummyJob struct{}

func (d DummyJob) Run() {
  fmt.Println("Every 30 seconds")
}

func main() {
  var job DummyJob
  c := clockwerk.New()
  c.Every(30 * time.Second).Do(job)
  c.Start()
}

View on Github

4 - Cronticker: A ticker implementation to support cron schedules.

Golang ticker that works with Cron scheduling.

Import it

go get github.com/krayzpipes/cronticker/cronticker

import "github.com/krayzpipes/cronticker/cronticker"

Usage

Create a new ticker:

ticker, err := NewTicker("TZ=America/New_York 0 0 0 ? * SUN")

Check the ticker's channel for the next tick:

tickerTime := <-ticker.C

Reset the ticker to a new cron schedule

err := ticker.Reset("0 0 0 ? * MON,TUE,WED")

Stop the ticker

ticker.Stop()

Use defer ticker.Stop() whenever you can to ensure the cleanup of goroutines.

ticker, _ := NewTicker("@daily")
defer ticker.Stop()

View on Github

5 - Dagu: No-code workflow executor. it executes DAGs defined in a simple YAML format.

A just another Cron alternative with a Web UI, but with much more capabilities
It runs DAGs (Directed acyclic graph) defined in a simple YAML format.

Highlights

• Install by placing just a single binary file
• Schedule executions of DAGs with Cron expressions
• Define dependencies between related jobs and represent them as a single DAG (unit of execution)

How does it work?

dagu is a single command and it uses the local file system to store data. Therefore, no DBMS or cloud service is required. dagu executes DAGs defined in declarative YAML format. Existing programs can be used without any modification.

Install dagu

You can quickly install dagu command and try it out.

via Homebrew

brew install yohamta/tap/dagu

Upgrade to the latest version:

brew upgrade yohamta/tap/dagu

via Bash script

curl -L https://raw.githubusercontent.com/yohamta/dagu/main/scripts/downloader.sh | bash

via GitHub Release Page

Download the latest binary from the Releases page and place it in your $PATH (e.g. /usr/local/bin).

️Quick start

1. Launch the Web UI

Start the server with dagu server and browse to http://127.0.0.1:8080 to explore the Web UI.

2. Create a new DAG

Create a DAG by clicking the New DAG button on the top page of the web UI. Input example in the dialog.

Note: DAG (YAML) files will be placed in ~/.dagu/dags by default. See Admin Configuration for more details.

3. Edit the DAG

Go to the SPEC Tab and hit the Edit button. Copy & Paste this example YAML and click the Save button.

4. Execute the DAG

You can execute the example by pressing the Start button.

Note: Leave the parameter field in the dialog blank and press OK.

Command Line User Interface

• dagu start [--params=<params>] <file> - Runs the DAG
• dagu status <file> - Displays the current status of the DAG
• dagu retry --req=<request-id> <file> - Re-runs the specified DAG run
• dagu stop <file> - Stops the DAG execution by sending TERM signals
• dagu restart <file> - Restart the current running DAG
• dagu dry [--params=<params>] <file> - Dry-runs the DAG
• dagu server [--host=<host>] [--port=<port>] [--dags=<path/to/the DAGs directory>] - Starts the web server for web UI
• dagu scheduler [--dags=<path/to/the DAGs directory>] - Starts the scheduler process
• dagu version - Shows the current binary version

The --config=<config> option is available to all commands. It allows to specify different dagu configuration for the commands. Which enables you to manage multiple dagu process in a single instance. See Admin Configuration for more details.

For example:

dagu server --config=~/.dagu/dev.yaml
dagu scheduler --config=~/.dagu/dev.yaml

View on Github

6 - Go-cron: Simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually designed. Comparisons are fast and efficient and take place in a goroutine; matched jobs are also executed in goroutines.

For instance, you can use the following in your web application that uses MySQL:

func init() {
  cron.NewWeeklyJob(1, 23, 59, 59, func (time.Time) {
    _, err := conn.Query("OPTIMIZE TABLE mytable;")
    if(err != nil) { println(err) }
  })
}

View on Github

7 - Go-quartz: Simple, zero-dependency scheduling library for Go.

A minimalistic and zero-dependency scheduling library for Go.

About

Inspired by the Quartz Java scheduler.

Library building blocks

Scheduler interface

type Scheduler interface {
	// Start starts the scheduler.
	Start()
	// IsStarted determines whether the scheduler has been started.
	IsStarted() bool
	// ScheduleJob schedules a job using a specified trigger.
	ScheduleJob(job Job, trigger Trigger) error
	// GetJobKeys returns the keys of all of the scheduled jobs.
	GetJobKeys() []int
	// GetScheduledJob returns the scheduled job with the specified key.
	GetScheduledJob(key int) (*ScheduledJob, error)
	// DeleteJob removes the job with the specified key from the Scheduler's execution queue.
	DeleteJob(key int) error
	// Clear removes all of the scheduled jobs.
	Clear()
	// Stop shutdowns the scheduler.
	Stop()
}

Implemented Schedulers

• StdScheduler

Trigger interface

type Trigger interface {
	// NextFireTime returns the next time at which the Trigger is scheduled to fire.
	NextFireTime(prev int64) (int64, error)
	// Description returns the description of the Trigger.
	Description() string
}

Implemented Triggers

• CronTrigger
• SimpleTrigger
• RunOnceTrigger

Job interface. Any type that implements it can be scheduled.

type Job interface {
	// Execute is called by a Scheduler when the Trigger associated with this job fires.
	Execute()
	// Description returns the description of the Job.
	Description() string
	// Key returns the unique key for the Job.
	Key() int
}

Implemented Jobs

• ShellJob
• CurlJob

View on Github

8 - Gocron: Easy and fluent Go job scheduling. This is an actively maintained fork of jasonlvhit/gocron.

gocron is a job scheduling package which lets you run Go functions at pre-determined intervals using a simple, human-friendly syntax.

gocron is a Golang scheduler implementation similar to the Ruby module clockwork and the Python job scheduling package schedule.

See also these two great articles that were used for design input:

• Rethinking Cron
• Replace Cron with Clockwork

If you want to chat, you can find us at Slack!

Concepts

• Scheduler: The scheduler tracks all the jobs assigned to it and makes sure they are passed to the executor when ready to be run. The scheduler is able to manage overall aspects of job behavior like limiting how many jobs are running at one time.
• Job: The job is simply aware of the task (go function) it's provided and is therefore only able to perform actions related to that task like preventing itself from overruning a previous task that is taking a long time.
• Executor: The executor, as it's name suggests, is simply responsible for calling the task (go function) that the job hands to it when sent by the scheduler.

Examples

s := gocron.NewScheduler(time.UTC)

s.Every(5).Seconds().Do(func(){ ... })

// strings parse to duration
s.Every("5m").Do(func(){ ... })

s.Every(5).Days().Do(func(){ ... })

s.Every(1).Month(1, 2, 3).Do(func(){ ... })

// set time
s.Every(1).Day().At("10:30").Do(func(){ ... })

// set multiple times
s.Every(1).Day().At("10:30;08:00").Do(func(){ ... })

s.Every(1).Day().At("10:30").At("08:00").Do(func(){ ... })

// Schedule each last day of the month
s.Every(1).MonthLastDay().Do(func(){ ... })

// Or each last day of every other month
s.Every(2).MonthLastDay().Do(func(){ ... })

// cron expressions supported
s.Cron("*/1 * * * *").Do(task) // every minute

// you can start running the scheduler in two different ways:
// starts the scheduler asynchronously
s.StartAsync()
// starts the scheduler and blocks current execution path
s.StartBlocking()

For more examples, take a look in our go docs

View on Github

9 - Goflow: A workflow orchestrator and scheduler for rapid prototyping of ETL/ML/AI pipelines.

A workflow/DAG orchestrator written in Go for rapid prototyping of ETL/ML/AI pipelines. Goflow comes complete with a web UI for inspecting and triggering jobs.

Quick start

With Docker

docker run -p 8181:8181 ghcr.io/fieldryand/goflow-example:latest

Browse to localhost:8181 to explore the UI.

Without Docker

In a fresh project directory:

go mod init # create a new module
go get github.com/fieldryand/goflow # install dependencies

Create a file main.go with contents:

package main

import "github.com/fieldryand/goflow"

func main() {
        options := goflow.Options{
                AssetBasePath: "assets/",
                StreamJobRuns: true,
                ShowExamples:  true,
        }
        gf := goflow.New(options)
        gf.Use(goflow.DefaultLogger())
        gf.Run(":8181")
}

Download the front-end from the release page, untar it, and move it to the location specified in goflow.Options.AssetBasePath. Now run the application with go run main.go and see it in the browser at localhost:8181.

Use case

Goflow was built as a simple replacement for Apache Airflow to manage some small data pipeline projects. Airflow started to feel too heavyweight for these projects where all the computation was offloaded to independent services, but there was still a need for basic orchestration, concurrency, retries, visibility etc.

Goflow prioritizes ease of deployment over features and scalability. If you need distributed workers, backfilling over time slices, a durable database of job runs, etc, then Goflow is not for you. On the other hand, if you want to rapidly prototype some pipelines, then Goflow might be a good fit.

Concepts and features

• Job: A Goflow workflow is called a Job. Jobs can be scheduled using cron syntax.
• Task: Each job consists of one or more tasks organized into a dependency graph. A task can be run under certain conditions; by default, a task runs when all of its dependencies finish successfully.
• Concurrency: Jobs and tasks execute concurrently.
• Operator: An Operator defines the work done by a Task. Goflow comes with a handful of basic operators, and implementing your own Operator is straightforward.
• Retries: You can allow a Task a given number of retry attempts. Goflow comes with two retry strategies, ConstantDelay and ExponentialBackoff.
• Database: Goflow supports two database types, in-memory and BoltDB. BoltDB will persist your history of job runs, whereas in-memory means the history will be lost each time the Goflow server is stopped. The default is BoltDB.
• Streaming: Goflow uses server-sent events to stream the status of jobs and tasks to the UI in real time.

View on Github

10 - Gron: Define time-based tasks using a simple Go API and Gron’s scheduler will run them accordingly.

Gron provides a clear syntax for writing and deploying cron jobs.

Goals

• Minimalist APIs for scheduling jobs.
• Thread safety.
• Customizable Job Type.
• Customizable Schedule.

Installation

$ go get github.com/roylee0704/gron

Usage

Create schedule.go

package main

import (
	"fmt"
	"time"
	"github.com/roylee0704/gron"
)

func main() {
	c := gron.New()
	c.AddFunc(gron.Every(1*time.Hour), func() {
		fmt.Println("runs every hour.")
	})
	c.Start()
}

Schedule Parameters

All scheduling is done in the machine's local time zone (as provided by the Go time package).

Setup basic periodic schedule with gron.Every().

gron.Every(1*time.Second)
gron.Every(1*time.Minute)
gron.Every(1*time.Hour)

Also support Day, Week by importing gron/xtime:

import "github.com/roylee0704/gron/xtime"

gron.Every(1 * xtime.Day)
gron.Every(1 * xtime.Week)

Schedule to run at specific time with .At(hh:mm)

gron.Every(30 * xtime.Day).At("00:00")
gron.Every(1 * xtime.Week).At("23:59")

View on Github

Thank you for following this article.

Related videos:

Go scheduler: Implementing language with lightweight concurrency

#go #golang #scheduling #jobs 

What's new in the go 1.15

Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.

As Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.

#go #golang #go 1.15 #go features #go improvement #go package #go new features

Cron Job Scheduling In Laravel

Today I will show you Cron Job Scheduling In Laravel, many time we require to run some piece of code specific interval time period in laravel and we need to run manually every time but command scheduler through we can run and create cron job in laravel.

So, here i will teach you how to create cron job in laravel, and how to create custom command in laravel.

Cron Job Scheduling In Laravel

https://websolutionstuff.com/post/cron-job-scheduling-in-laravel

#cron job scheduling in laravel #laravel #scheduling #scheduler #cron #how to create cron job in laravel

10 Highest Paying Jobs You Can Learn (Without College). ( HOT NEWS!!! )

These are the 10 highest paying jobs you can learn without needing a college degree. Jobs that pay $75,000 and higher.
📺 The video in this post was made by Andrei Jikh
The origin of the article: https://www.youtube.com/watch?v=QIGDA2JRz8w
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#bitcoin #blockchain #10 highest paying jobs you can learn #jobs #highest paying jobs you can learn #10 highest paying jobs you can learn (without college)