Agent Data Synchronization

Resources

Sync Agents

Sync agents are a more powerful and configurable ways of defining tasks that must perform under a given sequence, dependency tree, or schedule. They can be configured in a number of ways that allow for very complex and fine grain control over how and when they execute.

A few of examples of why we would use a sync agent:

• A task or job needs to be fired at a given time

• We need the ability to schedule something in a different time zone

• We want to request that an agent performs it's refresh from a remote source (like a database)

• The task is dependant on other tasks completing

• We have data that may expire if not kept current

• We need to supply multiple CRON strings to specify which times the task runs

• We need to supply a blackout range where the task should not be executed

Agent callbacks

Agents have several main blocks to them, they are as follows:

1. The first callback is the configuration section. We can set all kinds of settings including:

   • Maximum executions per day.

   • A timespan of how often to execute

   • An array of CRON strings to specifiy when to execute

   • Blackout periods in the form of Timespan and CRON strings.

     • Setting "* 5 * * *" as a blackout CRON would disallow the agent to run during the entire fifth hour of the day (from 5:00AM to 5:59AM inclusive)

2. The execution callback

   • This callback is only ever called when the agent is in an active refresh/sync state.

   • You can perform whatever logic you need here, and simply return exec.Complete or exec.Failure depending on the results of your process

3. Tree check callback

   • This callback is only for late binding trees, in the below example you can see how it's used to setup a behavior tree for checking previous agent runs

Example - Three agents configuration

In the below example we configure 3 agents.

1. The first is run every 5 seconds, once a day.

2. The second is run on the minute 0 mark, once a day.

3. The level 2 agent has a late binding dependency tree to check to determine whether the first two succeeded and the data is not expired. If this is true, then it runs

using Microsoft.Extensions.Logging;
using Perigee;
using Perigee.AI;
using Perigee.Scheduler;

// Visit https://docs.perigee.software to learn more
// Visit https://perigee.software to purchase a license!


PerigeeApplication.ApplicationNoInit("Unparalleled Task Coordination", (c) => {

    //Clear on start, for demo purposes only
    if (File.Exists("MemAgent.json")) File.Delete("MemAgent.json");

    //Source
    var AgentSource = new SyncAgentSourceMemory("MemAgent.json", c.GetCancellationToken());
    
    /* PullData agent */
    c.AddAgent("PullData", "PullData", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
        (configAgent) => configAgent.SetSyncLimitPerDay(1).SetSync(TimeSpan.FromSeconds(5)),
        (ct, l, exec) => {

            l.LogInformation("Pulling and loading data from remote source...");
            Task.Delay(2000).Wait();
            l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));
            return exec.Complete(DateTimeOffset.Now.AddDays(1));
        },
        (ct, l, tree) => { });

    /* LoadExcel agent */
    c.AddAgent("LoadExcel", "LoadExcel", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
        (configAgent) => configAgent.SetSyncLimitPerDay(1).SetSync(null, "0 */1 * * * *"),
        (ct, l, exec) => {

            l.LogInformation("Loading data from Excel...");
            Task.Delay(2000).Wait();
            l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));
            return exec.Complete(DateTimeOffset.Now.AddDays(1));
        },
        (ct, l, tree) => { });


    /* Add an agent "ExecuteRefresh" that ONLY runs after the first two have produced valid data */
    c.AddAgent("ExecuteRefresh", "ExecuteRefresh", "Main", AgentSource, AgentRunCondition.RunIfPastDue,
        (configAgent) =>
            configAgent.SetSyncLimitPerDay(1).SetSync(TimeSpan.FromSeconds(5))
            .SetLateBindingBehaviorTrees(true, false),
        (ct, l, exec) => {

            l.LogInformation("Starting refresh of data now that all my sources have non expired data");
            Task.Delay(3000).Wait();
            l.LogInformation("Done! Data is valid until {date}", DateTimeOffset.Now.AddDays(1));

            return exec.Complete(DateTimeOffset.Now.AddDays(1));
        },
        (ct, l, tree) => {

            //Late binding tree update checker
            if (tree.TreeType == AgentTreeType.SyncTree)
            {
                var BT = new BehaviorTree("Check previous level completion").AddSequence("Check expiration",

                    //Returns success if the data is not expired, allowing the sequence check to proceed
                    LeafNodes.AgentDataExpired("PullData", tree.AgentData, l),
                    LeafNodes.AgentDataExpired("LoadExcel", tree.AgentData, l));

                //Set tree for late binding execution
                tree.UpdateTree(BT);

            }
        }, failedTreeReshcedule: TimeSpan.FromSeconds(15));

});