7.2. Quartz Scheduler in cluster mode

eXo Platform uses Quartz Scheduler, the Java Framework for scheduling jobs, in a wide range of features. When eXo Platform runs in the cluster mode, it is important to prevent jobs to execute concurrently. Quartz has its own cluster mode, with each instance of eXo Platform server as a node of Quartz load balancing and failover group.

In this guideline, Quartz is configured to use the JDBC Job Store method that allows Quartz nodes to share the same database. Load balancing occurs automatically, when a node acquires to execute the job as fast as it can, by placing a lock on the job so that it is not executed by other nodes. Failover occurs when a node fails in the midst of job execution, and one of the others will take the job over. The acquiring node is "more or less random", as said in Quartz reference.

There are two options of JDBC Job Store: JobStoreTX is supposed to be used in the Quartz standalone application, and JobStoreCMT is supposed to be part of a larger transaction management. Both of them work in eXo Platform.

Although Quartz can connect to the database directly using JDBC driver, in eXo Platform it should be configured to use a JNDI datasource.

The Quartz library is packaged with eXo Platform already and you do not need to download and install Quartz. However, Quartz JDBC Job Store requires a database that you will initiate using a SQL script. You will be instructed to obtain the script later in this document.

There will be differences in detailed steps for JBoss and Tomcat. To make it easy to follow, here is a summary of steps for both:

  1. Initiate the Quartz database, named quartz_lb in this document.

  2. Declare a datasource of which JNDI name is java:/comp/env/exo-quartz in this document.

  3. Install the JDBC driver.

  4. Configure the quartz.properties file.

  5. Add the startup option -Dorg.quartz.properties to tell where the properties file locates.

After the last step, you are instructed to confirm that Quartz cluster is working.

Quartz database

First, create a database and choose your desired name, for example quartz_lb. Then, import tables using the Quartz SQL script.

The scripts can be downloaded at http://svn.terracotta.org/svn/quartz/tags/quartz-2.2.2/distribution/src/main/assembly/root/docs/dbTables/. For example, if you are using MySQL, you will download tables_mysql.sql or tables_mysql_innodb.sql.

Note

The version of the above link is 2.2.2. This version is used by eXo Platform when the document is validated. For later versions of eXo Platform, you may check the Quartz version by yourself by checking the file DEPENDENCIES.txt which is packaged in eXo Platform package.

Quartz JNDI datasource

The datasource JNDI name will be java:/comp/env/exo-quartz in which you should not change the part: java:/comp/env. This datasource will connect to the quartz_lb database.

Tip

You can configure the datasource similarly to the two default datasources: exo-jcr_portal and exo-idm_portal. If you need more details about the datasource configurations, visit Jacamar Schema Descriptor or Tomcat JDBC Connection Pool.

Installing JDBC driver

As you see in the datasource configuration, it requires a driver jar file name. The driver may have been installed when you configure 2 datasources for IDM and JCR, as described in Database. If not, install the jar to the $PLATFORM_TOMCAT_HOME/lib folder in the Tomcat bundle or the $PLATFORM_JBOSS_HOME/standalone/deployment folder in JBoss.

If you want to know which DBMSs are supported by Quartz, check the scripts here.

Configuring quartz.properties

By default, the configurations for Quartz is loaded from the quartz.properties file inside the jar package of Quartz. You should create an external quartz.properties file and place it under the folder:

In quartz.properties file, we should not use the JobStoreCMT as a JobStore because it requires two datasource connections, the first is the transaction that is managed by the application, the second is non-managed for Quartz's self-commit and rollback. So, we should use the JobStoreTX.

The quartz.properties looks like below (See Quartz configuration reference for the full configuration explanation):

#============================================================================
#============================================================================
# Configure Main Scheduler Properties  
#============================================================================
org.quartz.scheduler.instanceName=RHECMClusteredScheduler
org.quartz.scheduler.instanceId=AUTO
#============================================================================
# Configure ThreadPool  
#============================================================================
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=25
org.quartz.threadPool.threadPriority=5
#============================================================================
# Configure JobStore  
#============================================================================
org.quartz.jobStore.misfireThreshold=60000
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate
org.quartz.jobStore.useProperties=false
org.quartz.jobStore.dataSource=quartzDS
org.quartz.jobStore.tablePrefix=QRTZ_
org.quartz.jobStore.isClustered=true
org.quartz.jobStore.clusterCheckinInterval=20000
#============================================================================
# Configure Datasources  
#============================================================================
org.quartz.dataSource.quartzDS.jndiURL=java:/comp/env/exo-quartz

Where:

Note

In general, it should be noted that the use of "DriverDelegate" depends on the used Data Base.

  • If the database used is oracle, we should add quartz-oracle-2.2.2.jar under $Tomcat_Home_Dir/lib/ for a Tomcat server and under $Jboss_Home_Dir/standalone/deployments/platform.ear/lib/ for a JBoss server and install the oracle Driver as a module you can visit this doumentation.

  • Modify the configuration file in $Jboss_Home_Dir/standalone/configuration/standalone-exo-cluster.xml.

  • In the datasources section, replace:

    
    <driver>ojdbc6.jar</driver>
                        <driver-class>oracle.jdbc.OracleDriver</driver-class>

    by this:

    
    <driver>com.oracle</driver>
  • Add the drivers section:

    
    <drivers>
    <driver name="com.oracle" module="com.oracle.db" />
    </drivers>
  • Declare this module in jboss-deployment-structure.xml file located in $Jboss_Home_Dir/standalone/deployments/platform.ear/META-INF/ by adding this line of code:

    
    <module name="com.oracle.db"/>

    Just after this configuration:

    
    <module name="org.jboss.jts"/>

In this guideline, you need to focus on two sections: jobStore and dataSource. First, you see that you are using JobStoreCMT. This method requires 2 datasource connections, one is transaction that is managed by the application, and another is non-managed for Quartz's self-commit and rollback. So in the jobStore section, you declare 2 names: quartzDS and quartzDSNoTx. These two logical datasources are achieved by the same JNDI name, java:/comp/env/exo-quartz which you declare in the previous steps.

If you want to use JobStoreTX instead of JobStoreCMT, you will remove the non-managed connection. The jobStore and dataSource will be changed into:

#============================================================================
# Configure JobStore  
#============================================================================
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties = false
org.quartz.jobStore.dataSource = quartzDS
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
#============================================================================
# Configure Datasources  
#============================================================================
org.quartz.dataSource.quartzDS.jndiURL=java:/comp/env/exo-quartz

Note

For each type of DBMS, you need to set an appropriate value of the org.quartz.jobStore.driverDelegateClass property. Many databases are known to work with the StdJDBCDelegate driver, but some may require another delegated driver. See the list of choices at JobStoreTX or JobStoreCMT.

Adding the -Dorg.quartz.properties option

You need to tell the server where to load the quartz.properties file, by adding the startup option: -Dorg.quartz.properties.

How to confirm the Quartz cluster is working

At the startup of each node, there should be some Quartz logs like the following:

17:21:55,283 INFO  [org.quartz.core.QuartzScheduler] (ServerService Thread Pool -- 48) Quartz Scheduler v.2.2.2 created.
17:21:55,285 INFO  [org.quartz.impl.jdbcjobstore.JobStoreCMT] (ServerService Thread Pool -- 48) Using db table-based data access locking (synchronization).
17:21:55,292 INFO  [org.quartz.impl.jdbcjobstore.JobStoreCMT] (ServerService Thread Pool -- 48) JobStoreCMT initialized.
17:21:55,294 INFO  [org.quartz.core.QuartzScheduler] (ServerService Thread Pool -- 48) Scheduler meta-data: Quartz Scheduler (v2.2.2) 'RHECMClusteredScheduler' with instanceId 'MAY1281381746115234'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 25 threads.
  Using job-store 'org.quartz.impl.jdbcjobstore.JobStoreCMT' - which supports persistence. and is clustered.
17:21:55,294 INFO  [org.quartz.impl.StdSchedulerFactory] (ServerService Thread Pool -- 48) Quartz scheduler 'RHECMClusteredScheduler' initialized from specified file: '/media/data/doc/3054/test/plf-jboss-node1/standalone/configuration/quartz.properties'

Also, after the first node is started, you can check the Quartz database to see that Quartz successfully added some records.

See also

Copyright ©. All rights reserved. eXo Platform SAS
blog comments powered byDisqus