項目中使用分布式并發(fā)部署定時任務,多臺跨JVM,按照常理邏輯每個JVM的定時任務會各自運行,這樣就會存在問題,多臺分布式JVM機器的應用服務同時干活,一個是加重服務負擔,另外一個是存在嚴重的邏輯問題,比如需要回滾的數據,就回滾了多次,剛好quartz提供很好的解決方案。
集群分布式并發(fā)環(huán)境中使用QUARTZ定時任務調度,會在各個節(jié)點會上報任務,存到數據庫中,執(zhí)行時會從數據庫中取出觸發(fā)器來執(zhí)行,如果觸發(fā)器的名稱和執(zhí)行時間相同,則只有一個節(jié)點去執(zhí)行此任務。
如果此節(jié)點執(zhí)行失敗,則此任務則會被分派到另一節(jié)點執(zhí)行,中途也會自動檢查失效的定時調度,發(fā)現不成功的,其他節(jié)點立馬接過來繼續(xù)完成定時任務。對應的定時任務調度表比較多,有11個。
此方法是結合Spring與quartz,實際解決方案如下:
####applicationContext-scheduler.xml### spring 3.11
<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:task="http://www.springframework.org/schema/task" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.2.xsd"> <context:annotation-config /> <context:component-scan base-package="com.dennis.walking.api.web.schedule" /> <bean id="rollbackOrderStatus" class="org.springframework.scheduling.quartz.JobDetailFactoryBean"> <property name="jobClass" value="com.dennis.walking.api.web.schedule.ReleaseQtyAndUpdateOrderStatusSchedule" /> <property name="durability" value="true" /> </bean> <bean id="rollbackOrderStatusTrigger" class="com.dennis.walking.api.web.schedule.PersistableCronTriggerFactoryBean"> <property name="jobDetail" ref="rollbackOrderStatus" /> <property name="cronExpression"> <value>0 0/5 * * * ?</value> </property> <property name="timeZone"> <value>GMT+8:00</value> </property> </bean> <bean id="quartzScheduler" parent="baseQuartzScheduler"> <property name="configLocation" value="classpath:quartz.properties" /> <property name="autoStartup" value="true" /> <!-- This name is persisted as SCHED_NAME in db. for local testing could change to unique name to avoid collision with dev server --> <property name="schedulerName" value="apiQuartzScheduler" /> <!-- NOTE: Must add both the jobDetail and trigger to the scheduler! --> <property name="triggers"> <list> <ref bean="rollbackOrderStatusTrigger" /> </list> </property> <property name="jobDetails"> <list> <ref bean="rollbackOrderStatus" /> </list> </property> </bean> <bean id="baseQuartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean"> <!-- <property name="configLocation" value="classpath:quartz.properties" /> --> <property name="dataSource" ref="dataSource" /> <property name="transactionManager" ref="transManager" /> <!-- This name is persisted as SCHED_NAME in db. for local testing could change to unique name to avoid collision with dev server --> <property name="schedulerName" value="quartzScheduler" /> <!-- Will update database cron triggers to what is in this jobs file on each deploy. Replaces all previous trigger and job data that was in the database. YMMV --> <property name="overwriteExistingJobs" value="true" /> <!-- <property name="autoStartup" value="true" /> --> <property name="applicationContextSchedulerContextKey" value="applicationContext" /> <property name="jobFactory"> <bean class="com.dennis.walking.api.web.schedule.AutowiringSpringBeanJobFactory" /> </property> <!-- NOTE: Must add both the jobDetail and trigger to the scheduler! --> <!-- <property name="jobDetails"> <list> </list> </property> <property name="triggers"> <list> </list> </property> --> </bean></beans>
記得對應下面的三個類文件。
###quartz.properties###
# Using Spring datasource in quartzJobsConfig.xml# Spring uses LocalDataSourceJobStore extension of JobStoreCMTorg.quartz.jobStore.useProperties=trueorg.quartz.jobStore.tablePrefix=QRTZ_org.quartz.jobStore.isClustered=true# 10 minsorg.quartz.jobStore.clusterCheckinInterval=600000org.quartz.scheduler.skipUpdateCheck=true# Change this to match your DB vendororg.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate# Needed to manage cluster instancesorg.quartz.scheduler.instanceId=AUTOorg.quartz.scheduler.instanceName=MY_JOB_SCHEDULER org.quartz.scheduler.rmi.export=falseorg.quartz.scheduler.rmi.proxy=falseorg.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPoolorg.quartz.threadPool.threadCount=10org.quartz.threadPool.threadPriority=5org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread=true# Configure Pluginsorg.quartz.plugin.triggHistory.class = org.quartz.plugins.history.LoggingTriggerHistoryPluginorg.quartz.plugin.triggHistory.triggerFiredMessage = Trigger \{1\}.\{0\} fired job \{6\}.\{5\} at: \{4, date, HH:mm:ss MM/dd/yyyy}org.quartz.plugin.triggHistory.triggerCompleteMessage = Trigger \{1\}.\{0\} completed firing job \{6\}.\{5\} at \{4, date, HH:mm:ss MM/dd/yyyy\}.
quartz定時調度常用時間點設置
"0 0 12 * * ?" Fire at 12pm (noon) every day"0 15 10 ? * *" Fire at 10:15am every day"0 15 10 * * ?" Fire at 10:15am every day"0 15 10 * * ? *" Fire at 10:15am every day"0 15 10 * * ? 2005" Fire at 10:15am every day during the year 2005"0 * 14 * * ?" Fire every minute starting at 2pm and ending at 2:59pm, every day"0 0/5 14 * * ?" Fire every 5 minutes starting at 2pm and ending at 2:55pm, every day"0 0/5 14,18 * * ?" Fire every 5 minutes starting at 2pm and ending at 2:55pm, AND fire every 5 minutes starting at 6pm and ending at 6:55pm, every day"0 0-5 14 * * ?" Fire every minute starting at 2pm and ending at 2:05pm, every day"0 10,44 14 ? 3 WED" Fire at 2:10pm and at 2:44pm every Wednesday in the month of March."0 15 10 ? * MON-FRI" Fire at 10:15am every Monday, Tuesday, Wednesday, Thursday and Friday"0 15 10 15 * ?" Fire at 10:15am on the 15th day of every month"0 15 10 L * ?" Fire at 10:15am on the last day of every month"0 15 10 ? * 6L" Fire at 10:15am on the last Friday of every month"0 15 10 ? * 6L" Fire at 10:15am on the last Friday of every month"0 15 10 ? * 6L 2002-2005" Fire at 10:15am on every last Friday of every month during the years 2002, 2003, 2004 and 2005"0 15 10 ? * 6#3" Fire at 10:15am on the third Friday of every month
真實運行的業(yè)務類:ReleaseQtyAndUpdateOrderStatusSchedule.java
應為要持久化等特性操作,需要繼承 QuartzJobBean
由于要被持久化,所以不能存放xxxxManager類似對象,
只能從每次從QuartzJobBean注入的ApplicationContext 中去取出
繼承QuartzJobBean,實現方法executeInternal
package com.dennis.walking.api.web.schedule;import org.apache.commons.logging.Log;import org.apache.commons.logging.LogFactory;import org.quartz.JobExecutionContext;import org.quartz.JobExecutionException;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.scheduling.quartz.QuartzJobBean;import org.springframework.stereotype.Component;import com.dennis.walking.api.service.OrderService;@Componentpublic class ReleaseQtyAndUpdateOrderStatusSchedule extends QuartzJobBean { private Log log = LogFactory.getLog(this.getClass()); @Autowired OrderService orderService; protected void executeInternal(JobExecutionContext arg0) throws JobExecutionException { log.info("dennis test execute schedule start "); try { orderService.releaseQtyAndUpdateOrderStatus(); } catch (Exception e) { e.printStackTrace(); log.error("execute ReleaseQtyAndUpdateOrderStatus schedule error:" + e.getMessage()); } log.info("dennis test exxcute schedule end"); }}
持久化數據:PersistableCronTriggerFactoryBean.java 保存數據到數據庫中
package com.dennis.walking.api.web.schedule;import org.springframework.scheduling.quartz.CronTriggerFactoryBean;import org.springframework.scheduling.quartz.JobDetailAwareTrigger;/** * Needed to set Quartz useProperties=true when using Spring classes, because * Spring sets an object reference on JobDataMap that is not a String * * @see http://site.trimplement.com/using-spring-and-quartz-with-jobstore-properties * / * @see http * ://forum.springsource.org/showthread.php?130984-Quartz-error-IOException */public class PersistableCronTriggerFactoryBean extends CronTriggerFactoryBean { @Override public void afterPropertiesSet() { super.afterPropertiesSet(); System.out.println("PersistableCronTriggerFactoryBean-------------------"); // Remove the JobDetail element getJobDataMap().remove(JobDetailAwareTrigger.JOB_DETAIL_KEY); }}
###AutowiringSpringBeanJobFactory.java### 使用spring的 bean文件時,需要使用此類
package com.dennis.walking.api.web.schedule;import org.quartz.spi.TriggerFiredBundle;import org.springframework.beans.factory.config.AutowireCapableBeanFactory;import org.springframework.context.ApplicationContext;import org.springframework.context.ApplicationContextAware;import org.springframework.scheduling.quartz.SpringBeanJobFactory;/** * Autowire Quartz Jobs with Spring context dependencies * * @see http * ://stackoverflow.com/questions/6990767/inject-bean-reference-into-a-quartz * -job-in-spring/15211030#15211030 */public final class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory implements ApplicationContextAware { private transient AutowireCapableBeanFactory beanFactory; public void setApplicationContext(final ApplicationContext context) { beanFactory = context.getAutowireCapableBeanFactory(); } @Override protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception { final Object job = super.createJobInstance(bundle); beanFactory.autowireBean(job); System.out.println("AutowiringSpringBeanJobFactory-------------------"); return job; }}
###tables_mysql.sql### 需要建立的11張表
## Quartz seems to work best with the driver mysql-connector-java-5.1.34-bin.jar# # PLEASE consider using mysql with innodb tables to avoid locking issues## In your Quartz properties file, you'll need to set # org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate#DROP TABLE IF EXISTS QRTZ_FIRED_TRIGGERS;DROP TABLE IF EXISTS QRTZ_PAUSED_TRIGGER_GRPS;DROP TABLE IF EXISTS QRTZ_SCHEDULER_STATE;DROP TABLE IF EXISTS QRTZ_LOCKS;DROP TABLE IF EXISTS QRTZ_SIMPLE_TRIGGERS;DROP TABLE IF EXISTS QRTZ_SIMPROP_TRIGGERS;DROP TABLE IF EXISTS QRTZ_CRON_TRIGGERS;DROP TABLE IF EXISTS QRTZ_BLOB_TRIGGERS;DROP TABLE IF EXISTS QRTZ_TRIGGERS;DROP TABLE IF EXISTS QRTZ_JOB_DETAILS;DROP TABLE IF EXISTS QRTZ_CALENDARS;CREATE TABLE QRTZ_JOB_DETAILS ( SCHED_NAME VARCHAR(120) NOT NULL, JOB_NAME VARCHAR(200) NOT NULL, JOB_GROUP VARCHAR(200) NOT NULL, DESCRIPTION VARCHAR(250) NULL, JOB_CLASS_NAME VARCHAR(250) NOT NULL, IS_DURABLE VARCHAR(1) NOT NULL, IS_NONCONCURRENT VARCHAR(1) NOT NULL, IS_UPDATE_DATA VARCHAR(1) NOT NULL, REQUESTS_RECOVERY VARCHAR(1) NOT NULL, JOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,JOB_NAME,JOB_GROUP));CREATE TABLE QRTZ_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, JOB_NAME VARCHAR(200) NOT NULL, JOB_GROUP VARCHAR(200) NOT NULL, DESCRIPTION VARCHAR(250) NULL, NEXT_FIRE_TIME BIGINT(13) NULL, PREV_FIRE_TIME BIGINT(13) NULL, PRIORITY INTEGER NULL, TRIGGER_STATE VARCHAR(16) NOT NULL, TRIGGER_TYPE VARCHAR(8) NOT NULL, START_TIME BIGINT(13) NOT NULL, END_TIME BIGINT(13) NULL, CALENDAR_NAME VARCHAR(200) NULL, MISFIRE_INSTR SMALLINT(2) NULL, JOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,JOB_NAME,JOB_GROUP) REFERENCES QRTZ_JOB_DETAILS(SCHED_NAME,JOB_NAME,JOB_GROUP));CREATE TABLE QRTZ_SIMPLE_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, REPEAT_COUNT BIGINT(7) NOT NULL, REPEAT_INTERVAL BIGINT(12) NOT NULL, TIMES_TRIGGERED BIGINT(10) NOT NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP));CREATE TABLE QRTZ_CRON_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, CRON_EXPRESSION VARCHAR(200) NOT NULL, TIME_ZONE_ID VARCHAR(80), PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP));CREATE TABLE QRTZ_SIMPROP_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, STR_PROP_1 VARCHAR(512) NULL, STR_PROP_2 VARCHAR(512) NULL, STR_PROP_3 VARCHAR(512) NULL, INT_PROP_1 INT NULL, INT_PROP_2 INT NULL, LONG_PROP_1 BIGINT NULL, LONG_PROP_2 BIGINT NULL, DEC_PROP_1 NUMERIC(13,4) NULL, DEC_PROP_2 NUMERIC(13,4) NULL, BOOL_PROP_1 VARCHAR(1) NULL, BOOL_PROP_2 VARCHAR(1) NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP));CREATE TABLE QRTZ_BLOB_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, BLOB_DATA BLOB NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP), FOREIGN KEY (SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP) REFERENCES QRTZ_TRIGGERS(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP));CREATE TABLE QRTZ_CALENDARS ( SCHED_NAME VARCHAR(120) NOT NULL, CALENDAR_NAME VARCHAR(200) NOT NULL, CALENDAR BLOB NOT NULL, PRIMARY KEY (SCHED_NAME,CALENDAR_NAME));CREATE TABLE QRTZ_PAUSED_TRIGGER_GRPS ( SCHED_NAME VARCHAR(120) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, PRIMARY KEY (SCHED_NAME,TRIGGER_GROUP));CREATE TABLE QRTZ_FIRED_TRIGGERS ( SCHED_NAME VARCHAR(120) NOT NULL, ENTRY_ID VARCHAR(95) NOT NULL, TRIGGER_NAME VARCHAR(200) NOT NULL, TRIGGER_GROUP VARCHAR(200) NOT NULL, INSTANCE_NAME VARCHAR(200) NOT NULL, FIRED_TIME BIGINT(13) NOT NULL, SCHED_TIME BIGINT(13) NOT NULL, PRIORITY INTEGER NOT NULL, STATE VARCHAR(16) NOT NULL, JOB_NAME VARCHAR(200) NULL, JOB_GROUP VARCHAR(200) NULL, IS_NONCONCURRENT VARCHAR(1) NULL, REQUESTS_RECOVERY VARCHAR(1) NULL, PRIMARY KEY (SCHED_NAME,ENTRY_ID));CREATE TABLE QRTZ_SCHEDULER_STATE ( SCHED_NAME VARCHAR(120) NOT NULL, INSTANCE_NAME VARCHAR(200) NOT NULL, LAST_CHECKIN_TIME BIGINT(13) NOT NULL, CHECKIN_INTERVAL BIGINT(13) NOT NULL, PRIMARY KEY (SCHED_NAME,INSTANCE_NAME));CREATE TABLE QRTZ_LOCKS ( SCHED_NAME VARCHAR(120) NOT NULL, LOCK_NAME VARCHAR(40) NOT NULL, PRIMARY KEY (SCHED_NAME,LOCK_NAME));commit;###如果定時任務比較多,建議增加索引提升速度。/*reate index idx_qrtz_j_req_recovery on qrtz_job_details(SCHED_NAME,REQUESTS_RECOVERY); create index idx_qrtz_j_grp on qrtz_job_details(SCHED_NAME,JOB_GROUP); create index idx_qrtz_t_j on qrtz_triggers(SCHED_NAME,JOB_NAME,JOB_GROUP); create index idx_qrtz_t_jg on qrtz_triggers(SCHED_NAME,JOB_GROUP); create index idx_qrtz_t_c on qrtz_triggers(SCHED_NAME,CALENDAR_NAME); create index idx_qrtz_t_g on qrtz_triggers(SCHED_NAME,TRIGGER_GROUP); create index idx_qrtz_t_state on qrtz_triggers(SCHED_NAME,TRIGGER_STATE); create index idx_qrtz_t_n_state on qrtz_triggers(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP,TRIGGER_STATE); create index idx_qrtz_t_n_g_state on qrtz_triggers(SCHED_NAME,TRIGGER_GROUP,TRIGGER_STATE); create index idx_qrtz_t_next_fire_time on qrtz_triggers(SCHED_NAME,NEXT_FIRE_TIME); create index idx_qrtz_t_nft_st on qrtz_triggers(SCHED_NAME,TRIGGER_STATE,NEXT_FIRE_TIME); create index idx_qrtz_t_nft_misfire on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME); create index idx_qrtz_t_nft_st_misfire on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_STATE); create index idx_qrtz_t_nft_st_misfire_grp on qrtz_triggers(SCHED_NAME,MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_GROUP,TRIGGER_STATE); create index idx_qrtz_ft_trig_inst_name on qrtz_fired_triggers(SCHED_NAME,INSTANCE_NAME); create index idx_qrtz_ft_inst_job_req_rcvry on qrtz_fired_triggers(SCHED_NAME,INSTANCE_NAME,REQUESTS_RECOVERY); create index idx_qrtz_ft_j_g on qrtz_fired_triggers(SCHED_NAME,JOB_NAME,JOB_GROUP); create index idx_qrtz_ft_jg on qrtz_fired_triggers(SCHED_NAME,JOB_GROUP); create index idx_qrtz_ft_t_g on qrtz_fired_triggers(SCHED_NAME,TRIGGER_NAME,TRIGGER_GROUP); create index idx_qrtz_ft_tg on qrtz_fired_triggers(SCHED_NAME,TRIGGER_GROUP); */
本文測試軟件環(huán)境使用的jar版本是:
quartz-2.2.1.jar mysql-connector-java-5.1.34.jar
linux apache-tomcat-7.0.47 ava version "1.7.0_72" 64位
windows7 java version "1.7.0_45" 64位 jetty
Quartz是一個開放源碼項目,專注于任務調度器,提供了極為廣泛的特性如持久化任務,集群和分布式任務等。 Quartz核心是調度器,還采用多線程管理。
1.持久化任務:當應用程序停止運行時,所有調度信息不被丟失,當你重新啟動時,調度信息還存在,這就是持久化任務(保存到數據庫表中)。
2.集群和分布式處理:當在集群環(huán)境下,當有配置Quartz的多個客戶端時(節(jié)點),采用Quartz的集群和分布式處理時,我們要了解幾點好處
1) 一個節(jié)點無法完成的任務,會被集群中擁有相同的任務的節(jié)點取代執(zhí)行。
2) Quartz調度是通過觸發(fā)器的類別來識別不同的任務,在不同的節(jié)點定義相同的觸發(fā)器的類別,這樣在集群下能穩(wěn)定的運行,一個節(jié)點無法完成的任務,會被集群中擁有相同的任務的節(jié)點取代執(zhí)行。
3)分布式體現在當相同的任務定時在一個時間點,在那個時間點,不會被兩個節(jié)點同時執(zhí)行。
Quartz的 Task(11 張表)實例化采用數據庫存儲,基于數據庫引擎及 High-Available 的策略(集群的一種策略)自動協(xié)調每個節(jié)點的 Quartz。