Skip to content

Commit 1d9139c

Browse files
kagkarlssonkagkarlsson
authored
Merge pull request #188 from kagkarlsson/readme_configuration
Readme configuration
2 parents db2d240 + 3e5fabb commit 1d9139c

File tree

1 file changed

+102
-19
lines changed

1 file changed

+102
-19
lines changed

‎README.md

Lines changed: 102 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,25 @@ See also [why not Quartz?](#why-db-scheduler-when-there-is-quartz)
1717
## Features
1818

1919
* **Cluster-friendly**. Guarantees execution by single scheduler instance.
20-
* **Persistent** tasks. Requires _single_ database-table for persistence.
20+
* **Persistent** tasks. Requires a _single_ database-table for persistence.
2121
* **Embeddable**. Built to be embedded in existing applications.
2222
* **High throughput**. Tested to handle 2k - 10k executions / second. [Link](#benchmark-test).
2323
* **Simple**.
2424
* **Minimal dependencies**. (slf4j)
2525

26+
## Table of contents
27+
28+
* [Getting started](#getting-started)
29+
* [Who uses db-scheduler?](#who-uses-db-scheduler)
30+
* [Examples](#examples)
31+
* [Configuration](#configuration)
32+
* [Spring Boot usage](#spring-boot-usage)
33+
* [Interacting with scheduled executions using the SchedulerClient](#interacting-with-scheduled-executions-using-the-schedulerclient)
34+
* [How it works](#how-it-works)
35+
* [Performance](#performance)
36+
* [Versions / upgrading](#versions--upgrading)
37+
* [FAQ](#faq)
38+
2639
## Getting started
2740

2841
1. Add maven dependency
@@ -138,24 +151,94 @@ scheduler.schedule(myAdhocTask.instance("1045", new MyTaskData(1001L)), Instant.
138151

139152
The scheduler is created using the `Scheduler.create(...)` builder. The builder has sensible defaults, but the following options are configurable.
140153

141-
| Option | Description |
142-
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
143-
| `.threads(int)` | Number of threads. Default `10`. |
144-
| `.pollingInterval(Duration)` | How often the scheduler checks the database for due executions. Default `30s`. |
145-
| `.pollUsingFetchAndLockOnExecute(double, double)` | Use default polling strategy `fetch-and-lock-on-execute`. `lowerLimitFractionOfThreads`: threshold for when new executions are fetched from the database (given that last batch was full). Default `0.5`. `executionsPerBatchFractionOfThreads`: how many executions to fetch in each batch. Defualt `3.0`. These executions will not be pre-locked, so the scheduler will compete with other instances for the lock when it is executed. Supported by all databases. |
146-
| `.pollUsingLockAndFetch(double, double)` | Use polling strategy `lock-and-fetch` which uses `select for update .. skip locked` for less overhead. `lowerLimitFractionOfThreads`: threshold for when new executions are fetched from the database (given that last batch was full). `upperLimitFractionOfThreads`: how many executions to lock and fetch. For high throughput (i.e. keep threads busy), set to for example `1.0, 4.0`. Currently hearbeats are not updated for picked executions in queue. If they stay there for more than 4 * <hearbeat-interval>, they will be marked as dead and likely be unlocked again (determined by `DeadExecutionHandler`). Currently supported by **postgres**. |
147-
| `.heartbeatInterval(Duration)` | How often to update the heartbeat timestamp for running executions. Default `5m`. |
148-
| `.schedulerName(SchedulerName)` | Name of this scheduler-instance. The name is stored in the database when an execution is picked by a scheduler. Default `<hostname>`. |
149-
| `.tableName(String)` | Name of the table used to track task-executions. Change name in the table definitions accordingly when creating the table. Default `scheduled_tasks`. |
150-
| `.serializer(Serializer)` | Serializer implementation to use when serializing task data. Default standard Java serialization. |
151-
| `.enableImmediateExecution()` | If this is enabled, the scheduler will attempt to directly execute tasks that are scheduled to `now()`, or a time in the past. For this to work, the call to `schedule(..)` must not occur from within a transaction, because the record will not yet be visible to the scheduler (if this is a requirement, see the method `scheduler.triggerCheckForDueExecutions()`). Default `false`. |
152-
| `.executorService(ExecutorService)` | If specified, use this externally managed executor service to run executions. Ideally the number of threads it will use should still be supplied (for scheduler polling optimizations). Default `null`. |
153-
| `.shutdownMaxWait(Duration)` | How long the scheduler will wait before interrupting executor-service threads. If you find yourself using this, consider if it is possible to instead regularly check `executionContext.getSchedulerState().isShuttingDown()` in the ExecutionHandler and abort long-running task. Default `30min`. |
154-
| `.deleteUnresolvedAfter(Duration)` | The time after which executions with unknown tasks are automatically deleted. These can typically be old recurring tasks that are not in use anymore. This is non-zero to prevent accidental removal of tasks through a configuration error (missing known-tasks) and problems during rolling upgrades. Default `14d`. |
155-
| `.jdbcCustomization(JdbcCustomization)` | db-scheduler tries to auto-detect the database used to see if any jdbc-interactions need to be customized. This method is an escape-hatch to allow for setting `JdbcCustomizations` explicitly. Default auto-detect. |
156-
| `.commitWhenAutocommitDisabled(boolean)` | By default no commit is issued on DataSource Connections. If auto-commit is disabled, it is assumed that transactions are handled by an external transaction-manager. Set this property to `true` to override this behavior and have the Scheduler always issue commits. Default `false`. |
157-
| `.failureLogging(Level, boolean)` | Configures how to log task failures, i.e. `Throwable`s thrown from a task execution handler. Use log level `OFF` to disable this kind of logging completely. Default `WARN, true`. |
158-
| `.registerShutdownHook()` | Registers a shutdown-hook that will call `Scheduler.stop()` on shutdown. Stop should always be called for a graceful shutdown and to avoid dead executions. |
154+
#### Consider tuning
155+
156+
:gear: `.threads(int)`<br/>
157+
Number of threads. Default `10`.
158+
159+
:gear: `.pollingInterval(Duration)`<br/>
160+
How often the scheduler checks the database for due executions. Default `10s`.<br/>
161+
162+
:gear: `.enableImmediateExecution()`<br/>
163+
If this is enabled, the scheduler will attempt to directly execute tasks that are scheduled to `now()`, or a time in
164+
the past. For this to work _reliably_, the call to `schedule(..)` should not occur from within a transaction, because
165+
the record may not yet be visible to the scheduler (if this is a requirement, see the
166+
method `scheduler.triggerCheckForDueExecutions()`). Default `false`.
167+
168+
:gear: `.registerShutdownHook()`<br/>
169+
Registers a shutdown-hook that will call `Scheduler.stop()` on shutdown. Stop should always be called for a
170+
graceful shutdown and to avoid dead executions.
171+
172+
:gear: `.shutdownMaxWait(Duration)`<br/>
173+
How long the scheduler will wait before interrupting executor-service threads. If you find yourself using this,
174+
consider if it is possible to instead regularly check `executionContext.getSchedulerState().isShuttingDown()`
175+
in the ExecutionHandler and abort long-running task. Default `30min`.
176+
177+
#### Polling strategy
178+
179+
If you are running >1000 executions/s you might want to use the `lock-and-fetch` polling-strategy for lower overhead
180+
and higher througput ([read more](#polling-strategy-lock-and-fetch)). If not, the default `fetch-and-lock-on-execute` will be fine.
181+
182+
:gear: `.pollUsingFetchAndLockOnExecute(double, double)`<br/>
183+
Use default polling strategy `fetch-and-lock-on-execute`.<br/>
184+
If the last fetch from the database was a full batch (`executionsPerBatchFractionOfThreads`), a new fetch will be triggered
185+
when the number of executions left are less than or equal to `lowerLimitFractionOfThreads * nr-of-threads`.
186+
Fetched executions are not locked/picked, so the scheduler will compete with other instances for the lock
187+
when it is executed. Supported by all databases.
188+
<br/>Defaults: `0,5, 3.0`
189+
190+
191+
:gear: `.pollUsingLockAndFetch(double, double)`<br/>
192+
Use polling strategy `lock-and-fetch` which uses `select for update .. skip locked` for less overhead.<br/>
193+
If the last fetch from the database was a full batch, a new fetch will be triggered
194+
when the number of executions left are less than or equal to `lowerLimitFractionOfThreads * nr-of-threads`.
195+
The number of executions fetched each time is equal to `(upperLimitFractionOfThreads * nr-of-threads) - nr-executions-left`.
196+
Fetched executions are already locked/picked for this scheduler-instance thus saving one `UPDATE` statement.
197+
<br/>For normal usage, set to for example `0.5, 1.0`.
198+
<br/>For high throughput
199+
(i.e. keep threads busy), set to for example `1.0, 4.0`. Currently hearbeats are not updated for picked executions
200+
in queue (applicable if `upperLimitFractionOfThreads > 1.0`). If they stay there for more than
201+
`4 * hearbeat-interval` (default `20m`), not starting execution, they will be detected as _dead_ and likely be
202+
unlocked again (determined by `DeadExecutionHandler`). Currently supported by **postgres**.
203+
204+
205+
#### Less commonly tuned
206+
207+
:gear: `.heartbeatInterval(Duration)`<br/>
208+
How often to update the heartbeat timestamp for running executions. Default `5m`.
209+
210+
:gear: `.schedulerName(SchedulerName)`<br/>
211+
Name of this scheduler-instance. The name is stored in the database when an execution is picked by a scheduler.
212+
Default `<hostname>`.
213+
214+
:gear: `.tableName(String)`<br/>
215+
Name of the table used to track task-executions. Change name in the table definitions accordingly when creating
216+
the table. Default `scheduled_tasks`.
217+
218+
:gear: `.serializer(Serializer)`<br/>
219+
Serializer implementation to use when serializing task data. Default standard Java serialization.
220+
221+
:gear: `.executorService(ExecutorService)`<br/>
222+
If specified, use this externally managed executor service to run executions. Ideally the number of threads it
223+
will use should still be supplied (for scheduler polling optimizations). Default `null`.
224+
225+
:gear: `.deleteUnresolvedAfter(Duration)`<br/>
226+
The time after which executions with unknown tasks are automatically deleted. These can typically be old recurring
227+
tasks that are not in use anymore. This is non-zero to prevent accidental removal of tasks through a configuration
228+
error (missing known-tasks) and problems during rolling upgrades. Default `14d`.
229+
230+
:gear: `.jdbcCustomization(JdbcCustomization)`<br/>
231+
db-scheduler tries to auto-detect the database used to see if any jdbc-interactions need to be customized. This
232+
method is an escape-hatch to allow for setting `JdbcCustomizations` explicitly. Default auto-detect.
233+
234+
:gear: `.commitWhenAutocommitDisabled(boolean)`<br/>
235+
By default no commit is issued on DataSource Connections. If auto-commit is disabled, it is assumed that
236+
transactions are handled by an external transaction-manager. Set this property to `true` to override this
237+
behavior and have the Scheduler always issue commits. Default `false`.
238+
239+
:gear: `.failureLogging(Level, boolean)`<br/>
240+
Configures how to log task failures, i.e. `Throwable`s thrown from a task execution handler. Use log level `OFF` to disable
241+
this kind of logging completely. Default `WARN, true`.
159242

160243
### Task configuration
161244

0 commit comments

Comments
 (0)