Extending BasePlanner¶
Databay comes with two implementations of BasePlanner - ApsPlanner and SchedulePlanner. If you require custom scheduling functionality outside of these two interfaces, you can create your own implementation of BasePlanner. Have a look at the two existing implementations for reference: ApsPlanner and SchedulePlanner.
To extend the BasePlanner you need to provide a way of executing Link.transfer method repeatedly by implementing the following methods. Note that some of these methods are private since they are called internally by BasePlanner and should not be executed directly.
_schedule¶
Schedule a Link. This method runs whenever add_links is called and should not be executed directly. It should accept a link and add the Link.transfer method to the scheduling system you’re using. Note that you do not need to store the link in your planner - BasePlanner will automatically store it under BasePlanner.links when add_links is called. It isn’t required for the scheduling to be already running when _schedule is called.
Each link comes with a datetime.timedelta interval providing the frequency at which its Link.transfer method should be run. Use Link.interval and schedule according to the interval specified.
If the scheduler you’re using utilises some form of task-managing job objects, you must assign these to the link being scheduled using Link.set_job. This is to ensure the job can be correctly destroyed later when remove_links is called.
Example from ApsPlanner._schedule:
def _schedule(self, link:Link):
job = self._scheduler.add_job(link.transfer,
trigger=IntervalTrigger(seconds=link.interval.total_seconds()))
link.set_job(job)
_unschedule¶
Unschedule a Link. This method runs whenever remove_links is called and should not be executed directly. It should accept a link and remove it from the scheduling system you’re using. Note that you do not need to remove the link from your planner - BasePlanner will automatically remove that link from BasePlanner.links when remove_links is called. It isn’t required for the scheduling to be already stopped when _unschedule is called.
If the scheduler you’re using utilises some form of task-managing job objects, you may access these using Link.job in order to correctly destroy them if necessary when _unschedule is called.
Example from ApsPlanner._unschedule:
def _unschedule(self, link:Link):
if link.job is not None:
link.job.remove()
link.set_job(None)
_start_planner¶
Start the scheduling. This method runs whenever BasePlanner.start is called and should not be executed directly. It should begin the scheduling of links.
This method will be called just after all Inlet.on_start and Outlet.on_start are called.
Example from ApsPlanner._start_planner:
def _start_planner(self):
self._scheduler.start()
_shutdown_planner¶
Shutdown the scheduling. This method runs whenever BasePlanner.shutdown is called and should not be executed directly. It should shut down the scheduling of links.
A wait parameter is provided that you can pass down to your scheduling system if it allows waiting for the remaining jobs to complete before shutting down.
This method will be called just before all Inlet.on_shutdown and Outlet.on_shutdown are called.
Example from ApsPlanner._shutdown_planner:
def _shutdown_planner(self, wait:bool=True):
self._scheduler.shutdown(wait=wait)
Running property¶
BasePlanner.running property should return a boolean value indicating whether the scheduler is currently running. By default this property always returns True.
Exceptions¶
When implementing your planner you should consider that links may raise exceptions when executing. Your planner should anticipate this and allow handling the exceptions appropriately to ensure continuous execution. BasePlanner exposes a protected BasePlanner._on_exception method that can be called to handle the exception, allowing to ignore exceptions when ignore_exceptions=True is passed on construction. Otherwise the exceptions will be logged and the planner will attempt a graceful shutdown. Both ApsPlanner and SchedulePlanner support this behaviour by default. See Exception handling for more.
Immediate transfer on start¶
By default BasePlanner will execute Link.transfer function on all its links once upon calling BasePlanner.start. This is to avoid having to wait for the link’s interval to expire before the first transfer. You can disable this behaviour by passing immediate_transfer=False parameter on construction of the BasePlanner to disable it for all governed links or individually for selected links by setting their immediate_transfer to False.
Shutdown atexit¶
Each BasePlanner registers an atexit callback, which will attempt to gracefully shut the planner down if it is created with shutdown_at_exit parameter set to True.
Next Steps