Sync Agent
Resources
Agents
Sync agents are a more powerful and configurable ways of defining tasks that must perform under a given sequence, dependency tree, or schedule. They can be configured in a number of ways that allow for very complex and fine grain control over how and when they execute.
A few of examples of why we would use a sync agent:
A task or job needs to be fired at a given time
We need the ability to schedule something in a different time zone
We want to request that an agent performs it's refresh from a remote source (like a database)
The task is dependant on other tasks completing
We have data that may expire if not kept current
We need to supply multiple CRON strings to specify which times the task runs
We need to supply a blackout range where the task should not be executed
Agent callbacks
Agents have several main blocks to them, they are as follows:
var MemoryAgentSource = new SyncAgentSourceMemory("MainPartition.json");
c.AddAgent("Agent Name", "TypeKey", "PartitionKey", MemoryAgentSource, AgentRunCondition.RunIfPastDue,
//Configure agent Callback
(configAgent) => configAgent.SetSyncLimitPerDay(2).SetSync(null, "30 6 * * *"),
//Callback Execution
(ct, l, exec) => exec.Complete(DateTimeOffset.Now.AddDays(1)),
//Tree check
(ct, l, tree) => { },
TimeSpan.FromMinutes(30), TimeSpan.FromMinutes(30), false).LinkToConfig("Agents:Level_1A_Enabled");
The first callback is the configuration section. We can set all kinds of settings including:
Maximum executions per day.
A timespan of how often to execute
An array of CRON strings to specifiy when to execute
Blackout periods in the form of Timespan and CRON strings.
Setting "* 5 * * *" as a blackout CRON would disallow the agent to run during the entire fifth hour of the day (from 5:00AM to 5:59AM inclusive)
The execution callback
This callback is only ever called when the agent is in an active refresh/sync state.
You can perform whatever logic you need here, and simply return
exec.Complete()orexec.Failure()depending on the results of your process
Tree check callback
This callback is only for late binding trees, in the below example you can see how it's used to setup a behavior tree for checking previous agent runs
Example - Three agents configuration
In the below example we configure 3 agents.
The first is run every 5 seconds, once a day.
The second is run on the minute 0 mark, once a day.
The level 2 agent has a late binding dependency tree to check to determine whether the first two succeeded and the data is not expired. If this is true, then it runs
using Microsoft.Extensions.Logging;
using Perigee;
using Perigee.AI;
using Perigee.Scheduler;
// Visit https://docs.perigee.software to learn more
// Visit https://perigee.software to purchase a license!
PerigeeApplication.ApplicationNoInit("Unparalleled Task Coordination", (c) => {
//Clear on start, for demo purposes only
if (File.Exists("MemAgent.json")) File.Delete("MemAgent.json");
//Source
var AgentSource = new SyncAgentSourceMemory("MemAgent.json", c.GetCancellationToken());
/* PullData agent */
c.AddAgent("PullData", "PullData", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
(configAgent) => configAgent.SetSyncLimitPerDay(1).SetSync(TimeSpan.FromSeconds(5)),
(ct, l, exec) => {
l.LogInformation("Pulling and loading data from remote source...");
Task.Delay(2000).Wait();
l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));
return exec.Complete(DateTimeOffset.Now.AddDays(1));
},
(ct, l, tree) => { });
/* LoadExcel agent */
c.AddAgent("LoadExcel", "LoadExcel", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
(configAgent) => configAgent.SetSyncLimitPerDay(1).SetSync(null, "0 */1 * * * *"),
(ct, l, exec) => {
l.LogInformation("Loading data from Excel...");
Task.Delay(2000).Wait();
l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));
return exec.Complete(DateTimeOffset.Now.AddDays(1));
},
(ct, l, tree) => { });
/* Add an agent "ExecuteRefresh" that ONLY runs after the first two have produced valid data */
c.AddAgent("ExecuteRefresh", "ExecuteRefresh", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
(configAgent) =>
configAgent.SetSyncLimitPerDay(1).SetSync(TimeSpan.FromSeconds(5))
.SetLateBindingBehaviorTrees(true, false),
(ct, l, exec) => {
l.LogInformation("Starting refresh of data now that all my sources have non expired data");
Task.Delay(3000).Wait();
l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));
return exec.Complete(DateTimeOffset.Now.AddDays(1));
},
(ct, l, tree) => {
//Late binding tree update checker
if (tree.TreeType == AgentTreeType.SyncTree)
{
var BT = new BehaviorTree("Check previous level completion").AddSequence("Check expiration",
//Returns success if the data is not expired, allowing the sequence check to proceed
LeafNodes.AgentDataExpired("PullData", tree.AgentData, l),
LeafNodes.AgentDataExpired("LoadExcel", tree.AgentData, l));
//Set tree for late binding execution
tree.UpdateTree(BT);
}
}, failedTreeReshcedule: TimeSpan.FromSeconds(15));
});Code for sources:
MSSQL
The code for the sync agent MSSQL source can be found here. If you're writing a connector to another database engine, this is a template and a guide on how to do so:
Memory
The code for the memory source is linked here. A great example of how to implement the source interface
Last updated