Retry = PerigeeUtil.Retry(5, (i) => {
bool Success = false;
if (Success == false)
{
throw new Exception("Do or do not, there is a retry");
}
}, 10000);
if (!Retry.Item1)
{
l.LogError(Retry.Item2, "Retries exceeded");
}
});
});
```
## RetryExponentialBackoff
Another retry strategy is the Exponential Backoff method. Unlike a constant delay between retries, this method progressively increases the delay (incorporating jitter) to prevent overwhelming a remote system with requests.
You can define both a base delay (in milliseconds) and a maximum delay (in milliseconds) to ensure the delay between requests scales appropriately.
```csharp
PerigeeApplication.ApplicationNoInit("Backoff", (c) => {
c.AddRecurring("Exponential", (ct, l) => {
//Log out the backoff delays, logging them is an easy way to test and see what times are going to be used.
l.LogInformation("Backoff times: {time}", string.Join(", ", PerigeeUtil.GetExponentialBackoffDelays(20, 1000, 1800000)));
// Use the .RetryExponential to follow the same times as shown above, with a small jitter to each delay
PerigeeUtil.RetryExponentialBackoff(20, (i) => {
l.LogInformation("Retry number {i}", i);
throw new Exception("Throwing to cause another retry");
}, 1000, 1800000);
});
});
/*
Backoff times: 956, 2102, 4469, 8470, 17127, 31337, 57203, 152968, 234956, 575255, 901233, 1645927, 1800000, 1800000, 1800000, 1800000, 1800000, 1800000, 1800000, 1800000
*/
```
# Network Utility
## Network Available
To check whether or not a network is available for use, use the NetworkUtility class. It returns **true** if there should be a network connectivity, and **false** if there should not be.
```csharp
NetworkUtility.Available()
```
# Lists
{% hint style="info" %}
All methods are under the **`Perigee.Extensions`** namespace
{% endhint %}
## MatchToList
This handy method uses several de-duplication methods, as well as the Soundex and Levenshtein algorithms to produce a best-guess match between two different lists.
It will always match **A => B**. Meaning items that items in A will only attempt matches to items in B, not the other way around.
In the below example, **`Peech`** is matched to **`PeachJuice`** even though it's misspelled and **`Agave`** has no match. Every other item is matched with it's equivalent juice.
List<string> Fruits = new List<string>()
{
"Apple",
"Banana",
"Pear",
"Orange",
"Pineapple",
"Strawberry",
"Blueberry",
"Raspberry",
"Blackberry",
"Watermelon",
"Cantaloupe",
"Grapefruit",
"Grape",
"Lemon",
"Lime",
"Mango",
"Papaya",
"Peech",
"Agave"
};
List<string> FruitJuices = new List<string>()
{
"AppleJuice",
"BananaJuice",
"PearJuice",
"OrangeJuice",
"PineappleJuice",
"StrawberryJuice",
"BlueberryJuice",
"RaspberryJuice",
"BlackberryJuice",
"WatermelonJuice",
"CantaloupeJuice",
"GrapefruitJuice",
"GrapeJuice",
"LemonJuice",
"LimeJuice",
"MangoJuice",
"PapayaJuice",
"PeachJuice"
};
var match = Fruits.MatchToList(FruitJuices);
Console.WriteLine("Matched:");
match.Item1.ForEach(s => Console.WriteLine($"{s.Key} => {s.Value}"));
Console.WriteLine(Environment.NewLine + "Unmatched:");
match.Item2.ForEach(s => Console.WriteLine($"{s}"));
## MatchToList (Dictionary)
This version of list matching is almost identical to what is listed above, the main difference is that your input dictionary defines the keys to match, and the value is a comma separated list of additional "hints" to match by.
In this example, we can match the key **`Apple`** with **`Appljuise`**, because of the additional hints supplied by the dictionary.
{% hint style="info" %}
Notice we also supplied `3, true`?
* **`3`** Means that our distance should be less than or equal to 3 edits. There are some additional preprocessing that occurs to attempt to reduce prefix and postfix content, this distance is applied and checked after.
* **`true`** means **`MonoMatch`** is turned on, disallowing the same "list" item to be matched with multiple Keys from my dictionary. If this was **`false`**, you could have the same list item matched with as many keys as it matched!
{% endhint %}
```csharp
Dictionary InList = new Dictionary() {
{ "Apple", "AppleJuice,Apple Juice,jus de pomme,jugo de manzana" }
};
var rs = InList.MatchToList(new List() {
"Peach Juice", "Banana Juice", "Appljuise", "mangojuce" }, 3, true);
foreach (var m in rs.Item1)
Console.WriteLine($"{m.Key} Matched with {m.Value}");
//Will print: Apple Matched with Appljuise
```
## MatchToList\_Quality
If you're attempting to debug your list matching and want to understand the match values being returned by this, use the \_Quality function. It will return a rich object that is easy to read showing the matched value for each key.
## JoinBy
Takes every item in a list and joins it with the delimiter
```csharp
string joined = Fruits.JoinBy(",");
```
## DistinctBy
Works identically to the classic LINQ Distinct, however it works on classes as well.
```csharp
Items.DistinctBy(f => f.SomeProperty);
```
## ToDataTable
Sometimes you have a list of classes, and need to convert it to a table. This uses reflection to perform the conversion.
```csharp
ToDataTable(this List items, string tableName = "table", string schema = "dbo")
```
## WhereSubclassOfType\
Enable you to filter a list of classes that conform to the subclass of another class
```csharp
ListOfClasses.WhereSubclassOfType(typeof(otherClass))
```
## Flatten\
Recursively flatten a data structure and return the new type
```csharp
ListOfX.Flatten(f => f.otherList)
```
## FromHierarchy\
Recursively select down a tree until something is null. Great for returning a list of all inner exceptions in order.
```csharp
thrownException.FromHierarchy(f => f.InnerException);
```
## RemoveDuplication
Our first example was a bunch of fruit juices. The remove duplication method attempts to find any common pattern in the list of items and remove them.
```csharp
// Strips the "Juice" from the end of each item
FruitJuices.RemoveDuplicationOfCommonOccuranceSuffix();
//There's also a prefixed version
FruitJuices.RemoveDuplicationOfCommonOccurancePrefixes();
```
# FileUtil
## MoveToChildDirectory
This handy helper takes a file and a child directory and does the lookups, directory creation, and move operation
{% code overflow="wrap" %}
```csharp
//Moves item to c:\temp\processed\item.txt
FileUtil.MoveToChildDirectory(@"C:\temp\item.txt", "processed", overwrite: true, createDirectory: true);
```
{% endcode %}
## CreateDirectoryIfNotExists
Verify if a directory exists for a given item
{% code overflow="wrap" %}
```csharp
//Verifies directory c:\temp\ exists
FileUtil.CreateDirectoryIfNotExists(@"C:\temp\item.txt");
```
{% endcode %}
## CleanFilenameWithPath
Clean a filename with the OS invalid file characters.
{% code overflow="wrap" %}
```csharp
//produces: C:\temp\it-em-.txt
var valid = FileUtil.CleanFilenameWithPath(@"C:\temp\it|em|.txt");
```
{% endcode %}
# Inclusive2DRange
A helper class when dealing with a range of data or cells
## Instantiate
To create a new range, supply two initial values:
{% code overflow="wrap" %}
```csharp
var ir = new Inclusive2DRange("A3", "C1");
Console.WriteLine($"{ir.CorrectedA}:{ir.CorrectedB}{Environment.NewLine}|{ir.MinX_ZeroBased} --> {ir.MaxX_ZeroBased}|{Environment.NewLine}|{ir.MinY_ZeroBased}{Environment.NewLine}|{Environment.NewLine}|{ir.MaxY_ZeroBased}");
/* Prints
A1:C3
|0 --> 2|
|0
|
|2
*/
```
{% endcode %}
* As you can see it corrected the range by fixing the starting point to **`A1`**, and the ending point to **`C3`**.
* The zero based indexes of the range are **`0 to 2 inclusive`**
You can also convert the common Excel locators using the built in functions:
{% code overflow="wrap" %}
```csharp
Console.WriteLine($"1 is {ir.GetExcelColumnName(1)}, 27 is {ir.GetExcelColumnName(27)}");
Console.WriteLine($"A is {ir.GetExcelColumnNumberFromName("A")}, AA is {ir.GetExcelColumnNumberFromName("AA")}");
/* Prints
1 is A, 27 is AA
A is 1, AA is 27
*/
```
{% endcode %}
# Strings, Numbers, Dates
{% hint style="info" %}
All methods are under the **`Perigee.Extensions`** namespace
{% endhint %}
## Range Extraction
Range extraction allows you to split an input string into it's parts.
```csharp
var rangeExtracted = "Section A, B and C";
Console.WriteLine(JsonConvert.SerializeObject(rangeExtracted.RangeExtract()));
// ["Section A","Section B","Section C"]
var rangeExtracted = "1-8";
Console.WriteLine(JsonConvert.SerializeObject(rangeExtracted.RangeExtract()));
//[" 1"," 2"," 3"," 4"," 5"," 6"," 7"," 8"]
```
## Fast String
These methods use the fastest possible comparisons and Boolean checks available for the various tasks.
```csharp
FastContains(); //contain
FastSuffix(); //suffix
FastPreFix(); //prefix
FastContainsAndPrefixOrSuffix(); //Must contain and (begin or end with)
FastContainsAndPrefix(); //contain and prefix
FastContainsAndSuffix(); //contain ans suffix
FastAndContains(); //Contain1 AND contain2, n
FastOrContains(); //Contain1 OR contain2, n
```
## String manipulation
```csharp
RemoveFromSuffix(); //Remove a suffix from a string
RemoveFromPrefix(); //Remove a prefix from a string
TrimDuplicateWhitespace(); //Replace duplicate whitespace characters in a string
Truncate(); //Truncates string so that it is no longer than the specified number of characters. Allows for negative signs
TruncateAndReport(); //Truncate a string and report if a truncation did occur
```
## Data and object or partial detection
```csharp
ReBracketString(); //Rebracket a bracketed or un-bracketed string, useful for database commands
getAllSubstrings(); //Cross application of all available substring for a given word
Soundex(); //Returns the soundex string for the word
LevenshteinDistance(); // Levenshtein distance
IsStringDate(); //Check whether a string is a date in the given(or default culture)
HasSpecialChars(); //Anything not a letter or digit
IsAllDigits(); //Are all characters digits (- sign allowed at position 0)
AlphaPercent(); //Percentage of characters that are all alpha characters. This algorithm is great for header detection
IsAllDigitsAndContainsPeriod(); //Is all digits, AND contains period
IsAllDigitsAndContainsPeriodAndCurrency(); //Checks for currency as well
StripCurrencySymbols(); //Strip currency symbols from input
StripCurrencySymbolsAndSeparators(); //Strip currency and separators
IsScientificNotation(); //Is it scientific notation?
ToInt(); //Converts a string to an integer, ignoring alpha characters and using a fast conversion algorithm
ToLong(); //Converts a string to a long, ignoring alpha characters and using a fast conversion algorithm
StripInt(); //Strip alpha charcters leading up to an integer in a string
ToDecimal(); //Convert a string to a decimal using fast conversion logic
//Decimal info provides a lot of information about a decimal, very useful when dealing with a system with strict requirements
DecimalInfo(); //Get's the decimal information, as well returns the trailing truncated version of the string.
// ^^ Returns the precision, scale, effective precision, effective scale, reparsed result if zero truncation is enabled, if truncation occured
SplitExcelLocatorToColAndRow(); //Split an excel location to the row and column (ex, Y5)
ToBase26ExcelLookup(); //Convert an excel base 26 characters string to the unsigned int reference, 1 based
ToBase26ExcelLookup_ZeroBased(); //Convert an excel base 26 characters string to the unsigned int reference, 0 based
ToExcelColumnName(); //To excel column name from an integer
```
##
# Nested Sets
Nested Sets provide a fantastic way of traversing hierarchical data. Once processed, they are able to very rapidly walk large data trees.
Suppose you're given a list of products and are asked:
* "What products does the clothing category contain?"
* "What products are my siblings?"
* "Do I have any products under my category?"
All of these questions are easily answered by processing the list of items into a Nested Set. Once the conversion process is finished, it's very easy to query the set and it's incredibly fast.
## Example - Products
```csharp
//Adjacency list
List productList = new List
{
//Root
new Product() { id = 0, name = "All Products" },
// L1 nodes
new Product() { id = 1, parent = 0, name = "Electronics" },
new Product() { id = 2, parent = 0, name = "Food" },
new Product() { id = 3, parent = 0, name = "Clothing" },
//L2 nodes
new Product() { id = 4, parent = 1, name = "Phones" },
new Product() { id = 5, parent = 1, name = "Televisions" },
new Product() { id = 6, parent = 1, name = "Computers" },
new Product() { id = 7, parent = 2, name = "Frozen" },
new Product() { id = 8, parent = 2, name = "Fresh" },
new Product() { id = 9, parent = 3, name = "Shirts" },
new Product() { id = 10, parent = 3, name = "Pants" },
//L3 nodes
new Product() { id = 11, parent = 4, name = "Samsung" },
new Product() { id = 12, parent = 4, name = "Apple" },
new Product() { id = 13, parent = 4, name = "Windows" },
new Product() { id = 14, parent = 7, name = "Pizza" },
new Product() { id = 15, parent = 7, name = "Vegetables" },
};
```
#### Class used in this example
{% hint style="info" %}
The sample class is here
```csharp
public class Product
{
public int id { get; set; } = 0;
public int parent { get; set; } = 0;
public string name { get; set; }
}
```
{% endhint %}
## SDK
### Converting the list
To convert a list of items to a nested set, simply create a new **`NestedSet`** by supplying the `Root`, `ID`, and `Parent` fields.
```csharp
var nsProduct = new NestedSet(productList,
//Select the first item that has an ID of 0. This is our root.
(l) => l.First(f => f.id == 0),
//Select the ID(key) field, and the Parent(fk) field
(f) => f.id, (f) => f.parent);
```
Once finished, this gives you access to all of the other query abilities
### Upline
Upline is how you traverse a tree upwards. So starting from key 15(vegetables), let's walk all the way back to "All Products".
```csharp
foreach (var item in nsProduct.Upline(15, true))
Console.WriteLine($"[{item.HLevel}]({item.ID}) {item.Node.name}");
//Prints:
//[4](15) Vegetables
//[3](7) Frozen
//[2](2) Food
//[1](0) All Products
```
### Downline
Downline is how you traverse a tree downwards. So starting from key 1(electronic), let's walk all the way down.
```csharp
foreach (var item in nsProduct.Downline(1, true))
Console.WriteLine($"[{item.HLevel}]({item.ID}) {item.Node.name}");
//Prints:
//[2](1) Electronics
//[3](6) Computers
//[3](5) Televisions
//[3](4) Phones
//[4](13) Windows
//[4](12) Apple
//[4](11) Samsung
```
### Siblings
Siblings is how you traverse a tree sideways. So starting from key 14(pizza), let's see what other siblings we have.
```csharp
foreach (var item in nsProduct.Siblings(14, true))
Console.WriteLine($"[{item.HLevel}]({item.ID}) {item.Node.name}");
//Prints:
//[4](15) Vegetables
//[4](14) Pizza
```
### The Nested Set Node
Every item is converted by creating a new **`NestedSet`** class, where the generic parameter is the type of objects that were originally processed. This class provides several additional properties available for use, as well as access to the original class used to convert.
Here's an example of retrieving the root node (key 0).
```csharp
var NestedNode_Root = nsProduct[0];
```
Let's look at some of the properties of this **`NestedSet<`****`Product`****`>`** below.
#### ID / ParentID
These properties are assigned from the lookup. They are the ID and Parent fields from the supplied class
#### HLevel
This property is the horizontal level in the tree, starting from 1(root) to n(lowest level)
#### Left / Right
These properties are the left and right identifiers for where the node is located in the tree
#### NodeNumber
A numbered index starting from 1, moving down and across the tree
#### NodeCount
How many nodes does this node contain in it's downline, including itself.
#### Node
The original class, in our example, this is a **`Product`**.
# Behavior Trees
## What is a tree?
A behavior tree allows for a complex set of requirements and tasks to be defined in such a way that a decision can be made based on an infinite number of states. \
Behavior Trees are used in AI, robotics, mechanical control systems, etc. We use them with [SyncAgent's ](/core-modules/event-sources/scheduled-logic/sync-agent)for fine grain execution control.
## An example, please?
Let's break that down to something we can all follow, like programming the AI of a video game.
The AI needs to do **two** things:
* Check for enemies in the area, If there are enemies in the area, then:
* Check if they want to attack, or can attack
* Only if the above statement is true, move into combat range
* Check for non combat activities after checking that there is no immediate danger in the area
* Am I hungry?
* Am I thirsty?
The **two** state checks (enemy check, non combat check) are known as a **`list of fallbacks`**. The reason for this is that every subsequent child node in a **`Fallback`** is executed in order until one succeeds.
As almost an opposite to **`Fallback`** nodes - **`Sequence`** nodes only execute their children in order if the previous one succeeds.
{% hint style="success" %}
Use **`Fallback`** nodes for an ordered list of items to execute, where a failure moves onto the next node\
\
Use **`Sequence`** if the children need to succeed to proceed to the next node
{% endhint %}
The demo below uses simple bool variables to control state, but in a real scenario, these would be callback to check within a radius of the AI, or pull it's current hunger/thirst levels.
```csharp
bool Enemies = true;
bool Attackable = true;
bool Hungry = false;
bool Thirsty = true;
var btt = new BehaviorTree("AI").AddFallback("Activities",
new Fallback("Check for enemies",
new Sequence("Any enemies around?",
new Leaf("Enemy Checker", (l) => { Console.WriteLine("Enemy Checker"); return Enemies ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Enemy Attackable?", (l) => { Console.WriteLine("Enemy Attackable?"); return Attackable ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Move in for combat", (l) => { Console.WriteLine("Move in for combat"); return NodeStatus.SUCCESS; }))),
new Fallback("Non Combat Activities",
new Sequence("Am I hungry?",
new Leaf("Hungry test", (l) => { Console.WriteLine("Hungry test"); return Hungry ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Hungry - eat food", (l) => { Console.WriteLine("Hungry - eat food"); return NodeStatus.SUCCESS; })),
new Sequence("Am I Thirsty?",
new Leaf("Thirsty test", (l) => { Console.WriteLine("Thirsty test"); return Thirsty ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Thirsty - drink", (l) => { Console.WriteLine("Thirsty - drink"); return NodeStatus.SUCCESS; }))
)
);
```
With that code in place, let's print and run our tree:
```csharp
Console.WriteLine(btt.Print());
var status = BTTickEngine.RunOnce(btt);
Console.WriteLine($"Status: {Enum.GetName(status)}");
```
#### Results:
If you run the code as shown above, you'll notice the last node to be executed is: \ **Move in for combat**.
The reason for this is because `"Check for Enemies"` takes priority over `"Non Combat Activities"`.
#### Change the results
Let's set **`bool Enemies = false;`** and run again. The last node to run this time is:
**Thirsty - drink**.
That's because our fallback node didn't have any enemies to check for, and our Thirsty bool is true.
This is what makes behavior trees so powerful. Simple state management changes the outcome of what it decides to do. Perigee has a built in SDK to make working with Behavior Trees quite easy.
## SDK
### The Print() Command
The print produces a nice breakdown of the tree itself. Showing nodes and their children
```
AI (BehaviorTree)
Activities (Fallback)
Check for enemies (Fallback)
Any enemies around? (Sequence)
Enemy Checker (Leaf)
Enemy Attackable? (Leaf)
Move in for combat (Leaf)
Non Combat Activities (Fallback)
Am I hungry? (Sequence)
Hungry test (Leaf)
Hungry - eat food (Leaf)
Am I Thirsty? (Sequence)
Thirsty test (Leaf)
Thirsty - drink (Leaf)
```
### The BTTickEngine
The tick engine runs a tree until success or failure. The run code is effectively processing the tree until it's no longer **`running`**:
```csharp
//Use the built in TickEngine
var status = BTTickEngine.RunOnce(btt);
//Or run it manually
NodeStatus st = NodeStatus.RUNNING;
while (st == NodeStatus.RUNNING)
{
st = btt.Process();
Task.Delay(10).Wait();
}
```
## Nodes and Types
The three main types are Fallback, Sequence, and Leaf. Leaf nodes are "special" nodes as they are the only nodes that contain a callback for code to be executed to determine the node state.
### Node States
The three states are:
* Success - It's completed.
* Processing - It's still working on it, try again...
* Failure - It failed, stop.
### Fallback
*(Sometimes called a "Selector Node")*
Fallback nodes process children nodes in order until a child node is a success, then fall out of the loop.
### Sequence
Processes all children in order, if a child fails, it will return a failure and start back at the first child on next rerun (tick)
### Leaf
Execute code to determine what status a node is in.
## Node Operations
There are several special operations that can be performed on nodes.
### Shuffle
You can shuffle a set of nodes by using this extension. The example below randomly shuffles whether the hunger check or the thirst check happens first
```csharp
new Fallback("Non Combat Activities",
new Sequence("Am I hungry?",
new Leaf("Hungry test", (l) => { Console.WriteLine("Hungry test"); return Hungry ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Hungry - eat food", (l) => { Console.WriteLine("Hungry - eat food"); return NodeStatus.SUCCESS; })),
new Sequence("Am I Thirsty?",
new Leaf("Thirsty test", (l) => { Console.WriteLine("Thirsty test"); return Thirsty ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Thirsty - drink", (l) => { Console.WriteLine("Thirsty - drink"); return NodeStatus.SUCCESS; }))
).Shuffle()
```
### Sort
Every node has a default **`SortOrder`** property. You can assign nodes a sort order or re-prioritize them at runtime/tree creation.
Sort orders these in ascending order. In this example, we sort **Thirsty** above **Hungry**.
```csharp
new Fallback("Non Combat Activities",
new Sequence("Am I hungry?", 2,
new Leaf("Hungry test", (l) => { Console.WriteLine("Hungry test"); return Hungry ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Hungry - eat food", (l) => { Console.WriteLine("Hungry - eat food"); return NodeStatus.SUCCESS; })),
new Sequence("Am I Thirsty?", 1,
new Leaf("Thirsty test", (l) => { Console.WriteLine("Thirsty test"); return Thirsty ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Thirsty - drink", (l) => { Console.WriteLine("Thirsty - drink"); return NodeStatus.SUCCESS; }))
).Sort()
```
### Invert
Invert reverses statuses. Success become failure, and failure become success.
Even though the bool **`Hungry = false;`** - the inversion node inverts the node status to be **`SUCCESS`**.
```csharp
new Fallback("Non Combat Activities",
new Sequence("Am I hungry?",
new Leaf("Hungry test", (l) => { Console.WriteLine("Hungry test"); return Hungry ? NodeStatus.SUCCESS : NodeStatus.FAILURE; })
.Invert(),
new Leaf("Hungry - eat food", (l) => { Console.WriteLine("Hungry - eat food"); return NodeStatus.SUCCESS; })),
new Sequence("Am I Thirsty?",
new Leaf("Thirsty test", (l) => { Console.WriteLine("Thirsty test"); return Thirsty ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Thirsty - drink", (l) => { Console.WriteLine("Thirsty - drink"); return NodeStatus.SUCCESS; }))
)
```
### Retry
Retry does as it sounds, it retries the node X number of times if the node returns a FAILURE status:
You'll see `"Hungry test"` print 3 times (original + 2 retries).
```csharp
new Fallback("Non Combat Activities",
new Sequence("Am I hungry?",
new Leaf("Hungry test", (l) => { Console.WriteLine("Hungry test"); return Hungry ? NodeStatus.SUCCESS : NodeStatus.FAILURE; })
.Retry(2),
new Leaf("Hungry - eat food", (l) => { Console.WriteLine("Hungry - eat food"); return NodeStatus.SUCCESS; })),
new Sequence("Am I Thirsty?",
new Leaf("Thirsty test", (l) => { Console.WriteLine("Thirsty test"); return Thirsty ? NodeStatus.SUCCESS : NodeStatus.FAILURE; }),
new Leaf("Thirsty - drink", (l) => { Console.WriteLine("Thirsty - drink"); return NodeStatus.SUCCESS; }))
)
```
### Forces
There is both a **`ForceFailure`** and a **`ForceSuccess`** node. These are helpful when building certain trees that require a forceful response.
## Prebuilt Leaf nodes and the fluent SDK
### Fluent SDK
There are two handy built in methods for quickly creating the two kinds of trees:
```csharp
BehaviorTree.NewSequence("Conditionals",
LeafNodes.NetworkAvailable,
LeafNodes.PingSuccess);
BehaviorTree.NewFallback("Conditionals", LeafNodes.NetworkAvailable);
```
### LeafNodes
There are multiple pre-built leaf nodes available to use. They return a Leaf Node that is already pre-coded to return the correct Node Status.
#### Network
To check if the network is available
```csharp
LeafNodes.NetworkAvailable;
```
#### PingSuccess
To check if a ping to **`8.8.8.8`** is available
```csharp
LeafNodes.PingSuccess;
```
#### PingAddress
To check if the address responds positively to a ping
```csharp
LeafNodes.PingAddress(new byte[] { 8,8,8,8});
```
#### SQLAvailable
To check if SQL Server can open a connection
```csharp
LeafNodes.SQLAvailable("connectionString");
```
#### Agent Exists
To check if an [Agent ](/core-modules/event-sources/scheduled-logic/sync-agent)exists
```csharp
LeafNodes.AgentExists("AgentName", treeHandler.AgentData);
```
#### Agent Data Expired
To check if an [Agent](/core-modules/event-sources/scheduled-logic/sync-agent) has expired data. Returns **SUCCESS** if the data is **NOT** expired.
```csharp
LeafNodes.AgentDataExpired("AgentName", treeHandler.AgentData);
```
# JsonCompress
A static class for serializing, compressing, and decompressing objects using JSON serialization and Brotli compression.
### Compress
Serializes and compresses an object using JSON serialization and Brotli compression.
#### Example:
```csharp
object data = new { Name = "John", Age = 30 };
byte[] compressedData = JsonCompress.Compress(data);
```
### Decompress
Decompresses the given byte array and deserializes it into type T.
#### Example:
```csharp
byte[] compressedData = ...; // The compressed byte array
MyClass data = JsonCompress.Decompress(compressedData);
```
### DecompressList
Decompresses the given byte array and deserializes it into a List of type T.
#### Example:
```csharp
byte[] compressedData = ...; // The compressed byte array
List dataList = JsonCompress.DecompressList(compressedData);
```
### EncryptCompress
Serializes, encrypts, and compresses an object using JSON serialization and Brotli compression.
#### Example:
```csharp
var obj = new MyObject();
var key = "mykey";
var iv = "myiv";
byte[] result = EncryptionHelper.EncryptCompress(obj, key, iv);
```
### DecryptDecompress
Decrypts and decompresses the given byte array and deserializes it into type T.
#### Returns:
Type T deserialized from the decompressed byte array.
#### Example:
```csharp
byte[] compressedData = GetCompressedData();
var key = "mykey";
var iv = "myiv";
MyObject obj = DecryptionHelper.DecryptDecompress(compressedData, key, iv);
```
### DecryptDecompressList
Decrypts and decompresses the given byte array and deserializes it into a List of type T.
#### Example:
```csharp
byte[] compressedData = GetCompressedData();
var key = "mykey";
var iv = "myiv";
List list = DecryptionHelper.DecryptDecompressList(compressedData, key, iv);
```
# Topological Sorting
TopoSort is a static class that provides functionality to perform a topological sort on a collection of items with dependencies.
## Sort
This method takes a source collection of items and a function that returns the dependencies of each item. It returns a sorted list of items based on their dependencies.
#### Example:
```csharp
var items = new List { "A", "B", "C", "D" };
var dependencies = new Dictionary>
{
{ "A", new List { "B", "C" } },
{ "B", new List { "C", "D" } },
{ "C", new List { "D" } },
{ "D", new List() }
};
var sortedItems = TopoSort.Sort(items, item => dependencies[item]);
// sortedItems will be ["D", "C", "B", "A"]
```
# DBDownloader
This class provides functionality for downloading data from a database and storing the result in a provided class. It supports multithreaded asynchronous calls, and allows for a queue of threaded download operations. It has automatic retry capability built in.
The type of DB Connection passed determines the database language it uses. You could for example provide a MySql connection instead.
{% hint style="info" %}
It's like saying, "Hey downloader, I have a class with several properties I want filled from my database, take care of the rest".
{% endhint %}
#### Quick implementation
```csharp
//Generate a connection string with custom properties set
string _CNS() => new SqlConnectionStringBuilder(c.GetConnectionString("main")) { Pooling = true, MaxPoolSize = 200, MultipleActiveResultSets = false, PacketSize = 32767 }.ToString();
//Create an instance of our data class with proeprties to hold results
var DataObject = new CircuitData();
//Allocate and assign with downloader extension.
var dl = Downloader.DBDownloader(DataObject, () => new SqlConnection(_CNS()))
.Assign(f => f.circuits, "SELECT ID,Type,Name from dbo.Circuit", ConnectionPreExecute: "SET ARITHABORT ON")
.Assign(f => f.circuitTypes, "SELECT * from util.CircuitType", ConnectionPreExecute: "SET ARITHABORT ON")
//Wait can be called inline, or later!
// You can proceed to do other operations until data is needed
.Wait();
Console.WriteLine($"DL time: {dl.GetDownloadTime().TotalMilliseconds:F0}");
```
### Downloader Extension
There is an easy way to create the DBDownloader by using the static extension method.
```csharp
Downloader.DBDownloader(InClass, () => new Connection());
```
### Assign / AssignSingle
Assigns the downloaded data to a property of the base object. Can be called multiple times using fluent syntax to fill multiple properties. The downloader automatically manages the multithreaded and asyhcnronous operations for filling each property.
It uses [Dapper](https://github.com/DapperLib/Dapper) behind the scenes to perform the mapping and query execution. If you want to supply parameters, you can use the `object``Param` parameter.
If you're not assigning a list, but a single object, use `AssignSingle()`.
#### Example:
```csharp
var downloader = new DBDownloader(myInstance, connectionFactory);
downloader.Assign(x => x.PropertyName, "SELECT * FROM table");
```
### Wait
Waits for all download tasks to complete. This can be called inline or later right before the downloaded data is required. It is thread blocking.
#### Example:
```csharp
downloader.Wait();
```
### GetDownloadTime
Gets the total time taken for all download tasks to complete.
#### Example:
```csharp
TimeSpan totalTime = downloader.GetDownloadTime();
```
### Downloads
You can get to the list of tasks created by accessing this property.
Every DownloadResult has properties for the success boolean, if an exception was thrown, the name of the property it set, and dates to indicate start and end times.
```csharp
List> Downloads;
```
# Bit Serializer
Perigee's proprietary serialization tool, Bit, stands out as a high-performance, high-compression module for serializing and deserializing data. Comparable to Avro or Protobuf in function but distinct in its foundation, Bit is developed entirely with a unique algorithm in 100% C# code. It is integral to several internal modules at Perigee, facilitating scenarios that demand exceptional performance and minimal disk usage.
Key Features of Bit include:
* Serialization and deserialization of classes, structs, lists, datatables, interfaced types, and properties. All without special "tagging" and pre-work
* Support for reserialization and "patching" to update data without full rewrites.
* If the type can be serialized, you can easily "deep clone" your object tree
* Capability for partial deserialization, useful for adapting to revised data maps or focusing on specific properties.
* Built-in compression to minimize data size.
* Facilities for map reading and generation, enabling dynamic data structuring.
* Tools to generate C# classes and schemas directly from data maps.
* Very high performance serialization
* Attributes for serialization events
* Utilities for reference type patching
### How Bit Works
Bit optimizes data storage by creating a map for each property, determining the necessity of writing each field based on defaults and nullability. This selective approach, combined with the efficient compression of values, significantly reduces the size of the output byte array. In practice, this can compress lists of objects by over 95% compared to their JSON document equivalents.
### Why Choose Bit?
Unlike many high-compression algorithms that impose strict data input conditions and support limited data types, Bit offers more flexibility. It's fully recursive, supports a wide array of common data types (including custom classes, structs, lists, dictionaries, etc.), and handles data maps without cumbersome limitations.
It even handles Interfaced types without adding additional code. Just make sure the interfaced type has a no-parameters constructor and all will be fine!
### Maps and Resilience to Change
Bit's mapping system ensures data resilience over time. For instance, objects serialized with an initial map version (R0) can be seamlessly reserialized to a newer version (R1) as your data schema evolves. This built-in functionality simplifies the process of keeping serialized data up-to-date without the complexity often associated with such transformations.
### Compression Ratios
Bit demonstrates exceptional efficiency in data storage, achieving compression ratios of over 90% in many cases, and even up to 98% when optimizing data types and attribute usage, far surpassing the compression achieved with minimized JSON.
## An Example
Here's a simple example where we are serializing a single class that has a `string`, `ushort`, `enum`, and recursive list of related "`bit people`".
It's extremely easy to use and Bit handles all of the data types and mapping.
```csharp
var bitPerson = new BitPerson() {
Name = "Bandit",
Age = 42,
PersonType = BitPersonType.adult,
RelatedPeople = new List()
{
new BitPerson() { Name = "Bingo", Age = 7, PersonType = BitPersonType.child }
} };
//Simple example without storing a map
byte[] ser = Bit2.Serialize(bitPerson);
BitPerson? des = Bit2.Deserialize(ser);
```
#### Compression Comparison
If you take this example and compare the byte lengths of JSON to Bit, it's pretty crazy. The compression ratios get even bigger when a larger document is at play, as bit automatically tests for additional compression on larger objects.
{% code overflow="wrap" %}
```csharp
decimal compSize = Bit2.SerializeHeadless(BitPerson).Length / (decimal)JsonConvert.SerializeObject(BitPerson).Length;
// 0.108 (or a ~ tenth of the size!)
```
{% endcode %}
## Documentation
## Attributes
### Ignore
The first attribute is **`Bit2Ignore`**. This tells the serializer to ignore this property and not serialize it. It is important to put this attribute on properties that are not necessary to store, or should be re-calculated after deserialization.
```csharp
public class BitPerson
{
public string Name { get; set; }
[Bit2Ignore]
public string ignoreExample { get; set; } = "";
}
```
### Serialization Callbacks
There are two serialization attributes as well, they are: **`Bit2PreSerialize`** and **`Bit2PostDeserialize.`** These attribute allow you to specify a singular method to be called before serialization and after deserialization.
{% hint style="success" %}
These methods must be instanced (not static) methods and the name does not matter.
{% endhint %}
```csharp
public class BitPerson
{
public string Name { get; set; }
[Bit2PreSerialize]
public void PreInvoke()
{
//Called before serialization.
//Null out values, or store additional information needed after deserialization
}
[Bit2PostDeserialize]
public void PostDes()
{
//Called after deserialization. Do whatever you need here!
}
}
```
#### PostDeserialize Example
In almsot all cases, reference types likely don't matter. In those rare cases they do, like traversing and comparing a directed graph, those reference types are very important. In this example, we fix that by:
1. Implementing the post deserialize callback.
2. Checking if our deserialized class held it's comparer.
3. If #2 is true, calling [ReplaceReferenceTypes](#replacereferencetypes) - Which is a Bit2 utility method.
All of the adjacency lists instances are now re-referenced. If you were to call `object.ReferenceEquals(a,b)` for two nodes in this graph, it would return true!
```csharp
///
/// This allows a deserialized graph to "patch" reference types.
///
[Bit2PostDeserialize]
public void PatchReferenceTypes()
{
if (Comparer == null) return;
Bit2.ReplaceReferenceTypes(Comparer,
adjacencyList.Keys.ToList(),
adjacencyList.Select(f => f.Value).ToArray());
}
```
## Serialization Cache
For scenarios where you need extreme performance, you can achieve **70-250X** faster compression times using the **`DeSerCache`** (Many of our sample test cases were only 2-10 microseconds).
Creating a cache is easy. Simply supply a instantiated object and let the system pre-cache the object and create a bitMap to match. Then supply these objects to any serialization / clone / deserialization calls you want.
```csharp
var bitPerson = new BitPerson();
//Generate a cache and a mpa
var cd2 = Bit2.DeSerCacheFromObject(bitPerson);
//Serialize with the map and cache
var sk2 = Bit2.Serialize(bitPerson, cd2.map, 0, cd2.cache);
```
This is incredibly helpful if you're serializing many of the same types over and over again (like a realtime communication application or lots of data store requests).
## Serialization / Deserialization
### Deserialize
Deserializes a byte array into an instance of a class.
#### Example:
```csharp
YourClass obj = Bit2.Deserialize(data, map);
```
### DeserializeHeadless
Deserializes a byte array without a header into an instance of a class. Useful for data serialized without initial header bytes.
#### Example:
```csharp
YourClass obj = Bit2.DeserializeHeadless(data);
```
### SerializeHeadless
Serializes an object into a byte array without adding a header.
#### Example:
```csharp
byte[] data = Bit2.SerializeHeadless(obj, version);
```
### Serialize
Serializes an object into a byte array, including a header with a version.
#### Example:
```csharp
byte[] data = Bit2.Serialize(obj, version);
```
### Serialize with Map Bytes Output
Serializes an object into a byte array and outputs the bytes of the map used for serialization.
#### Example:
```csharp
byte[] mapBytes;
byte[] data = Bit2.Serialize(obj, out mapBytes, version);
```
### Reserialize
Reserializes data from one map version to another. It enables transferring data between different revisions of a map, handling changes like new fields, removed fields, or updated data types.
#### Example:
```csharp
byte[] newData = Bit2.Reserialize(data, oldMap, newMap);
```
### Reserialize with Modification Callback
Similar to `Reserialize`, but includes a `BeforeReserialization` callback allowing modification of the object just before reserialization.
#### Example:
```csharp
byte[] newData = Bit2.Reserialize(data, oldMap, newMap, obj => {
// Modify obj here
return obj;
});
```
### Patch
Deserializes an object, patches it using a provided callback function, and then reserializes it.
#### Example:
```csharp
byte[] patchedData = Bit2.Patch(data, obj => {
// Patch obj here
}, map);
```
## Utilities
### DeepClone
Deep clone an object by serializing and deserializing with the same mapping information.
{% hint style="success" %}
Please note, this only works with properties that Bit2 is able to serialize.\
Please verify your data can be serialized properly before relying on DeepClone.
{% endhint %}
```csharp
var bitPerson = new BitPerson()
{
Name = "Bandit",
Age = 42,
PersonType = BitPersonType.adult,
RelatedPeople = new List() { new BitPerson() {
Name = "Bingo",
Age = 7,
PersonType = BitPersonType.child
}
}
};
var anotherBandit = Bit2.DeepClone(bitPerson);
```
### ReplaceReferenceTypes
This allows you to supply an IEqualityComparer\, and either one list, an array of lists, or an initial list + additional lists to replace all reference types. It uses the comparer to identify identical classes and then replaces those unique instances with references to itself.
This is likely not necessary on most deserialized data. However, if it is necessary, you have a helper here :thumbsup:
#### Example:
```csharp
Bit2.ReplaceReferenceTypes(Comparer,
adjacencyList.Keys.ToList(),
adjacencyList.Select(f => f.Value).ToArray());
```
### MapToCSharpByteString
Converts a map to a C# byte array string. Useful for storing maps internally for later deserialization.
#### Example:
```csharp
string mapString = Bit2.MapToCSharpByteString(typeof(YourClass), version);
```
### Map
Maps a type to a `BitMap` and optionally serializes and returns the bytes of that map.
#### Example:
```csharp
BitMap mapped;
byte[] mapBytes = Bit2.Map(typeof(YourClass), version, out mapped, true);
```
### ReadHeader
Reads the `BitHeader` from a byte array, using a maximum of 6 bytes. It also outputs the version used during serialization if available.
#### Example:
```csharp
ushort version;
BitHeader header = Bit2.ReadHeader(bytes, out version);
```
### GetMap
Converts a byte array to a `BitMap`.
#### Example:
```csharp
BitMap map = Bit2.GetMap(mapBytes);
```
### GenerateSchema
Generates a schema from a map, producing full C# classes that can be used to deserialize the data.
#### Example:
```csharp
string schema = Bit2.GenerateSchema(mapBytes);
```
# API + Perigee
Grab the code from the blueprint section for API hosting with Perigee
{% content-ref url="/pages/lNGf3TZSO5B97YPpPNWn" %}
[Perigee With .NET Hosting](/blueprints/perigee-with-.net-hosting)
{% endcontent-ref %}
## Admin controller example
In this example we create a controller that allows us to turn threads on and off.
1. Simply add a new empty "API Controller" to the **`Controllers`** folder.
2. Call it **`AdminController.cs`**
3. Paste the code below in to replace the content
```csharp
using Microsoft.AspNetCore.Mvc;
using Perigee;
using Perigee.Helpers;
// Visit https://docs.perigee.software to learn more
// Visit https://perigee.software to purchase a license!
namespace PerigeeIntroAPI.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class AdminController : ControllerBase
{
public ThreadRegistry reg { get; }
public AdminController(ThreadRegistry tr)
{
this.reg = tr;
}
[HttpGet]
public IActionResult Taskcontrol([FromQuery] string task, [FromQuery] bool start)
{
//Does this thread exist?
if (reg.ContainsThreadByName(task))
{
if (start)
{
reg.StartIfNotRunning(task); //Start if not started
return Ok();
}
else
{
reg.QueueStop(task, true); //Stop and do not auto restart
return Ok();
}
}
return BadRequest();
}
[Route("cron")]
[HttpGet]
public IActionResult cron([FromQuery] string name, [FromQuery] string cron, [FromQuery] string result)
{
if (reg.ContainsThreadByName(name) || string.IsNullOrEmpty(cron) || string.IsNullOrEmpty(result)) return BadRequest();
reg.AddManagedThread(new ManagedThread(name, (ct, l) =>
{
l.LogInformation("result: {result}", result);
}, cron, reg.CTS, reg.GetLogger()));
return new JsonResult(new { Message = $"Will run a CRON thread named {name} {Cron.Parse(cron).ToString()}" });
}
[Route("delete")]
[HttpDelete]
public IActionResult remove([FromQuery] string name)
{
reg.RemoveThreadByName(name);
return Ok();
}
[Route("list")]
[HttpGet]
public IActionResult list()
{
return new JsonResult(new { Threads = reg.GetThreads().Select(f => new { Name = f.Name, Runtime = f.RunTime }).ToArray() });
}
[Route("reboot")]
[HttpGet]
public IActionResult reboot()
{
reg.RestartAllThreads();
return Ok();
}
[Route("exec")]
[HttpGet]
public IActionResult exec([FromQuery] string name)
{
reg.ExecuteCRONNow(name);
return Ok();
}
}
}
```
In this example we've created a simple endpoint we can call to turn anything on or off by simply checking the thread registry for the requested thread, then turning it on or off.
The port may be different - but you can see how easy it would be to turn something on or off!
`[GET] https://localhost:5001/api/admin?task=ProcessAgent&start=true`
{% hint style="warning" %}
Typically speaking this would be secured or unavailable outside the firewall. Don't forget to secure your endpoints!
{% endhint %}
# Name Split
In this simple example, We show how to load in a source file, split the name column into it's first and surnames, then re-save the file.
{% hint style="success" %}
This uses the [Transformer](/transform-sdk/quick-start-guide) to load data which means it goes through the data identification, cleaning, and import process it uses. To read more about it, check out the [Transforms ](/transforms/what-is-transforms)section!
[F(x) Expressions](/core-modules/utility-classes/f-x-expressions) are powerful "formulas" you can apply to your data. Check them out to see what else you can use!
{% endhint %}
```csharp
//Read file, add first and last name
var tblname = Transformer.TableFromFile(@"InputFile.csv");
tblname.Columns.Add("FirstName", typeof(string));
tblname.Columns.Add("LastName", typeof(string));
//Run an Fx expression over both new columns, splitting up the FullName into it's first and surnames
tblname.FxTable(new Dictionary() {
{"FirstName", "name([FullName], 'first')" },
{ "LastName", "name([FullName], 'surname')" }});
//Resave the CSV!
File.WriteAllText(@"Outfile.csv", tblname.ToCSV());
```
# OpenAIClient (Chat GPT)
In this super simple example, we'll create and submit a request to `gpt-5.1`, in only a few lines of code.
{% hint style="success" %}
The [OpenAIClient](/core-modules/utility-classes/openai-client-responses) is designed to be easy to use, read, and implement in your own codebase.
{% endhint %}
{% code overflow="wrap" %}
```csharp
//Declare a client
var aic = new OpenAIClient(apiKey);
//Get a response
var response = await aic.GetResponseAsync(
OpenAIClient.GPT51("It's this easy to message AI??")
.WithInstructions("You're a helpful assistant, respond kindly and to the point"));
//Use the built in MessageContent Property to retrieve the latest message
if (response.IsSuccessful)
Console.WriteLine(response.Data.MessageContent);
```
{% endcode %}
# Excel Quick Load
From Data To Analyst In a Few Clicks
## Data Load
Need to take your Excel data and load it into your database to get the analysts started?
Instead of taking weeks to develop a new data pipeline, create structure, and work with the business to load your new client data in, just drop the file in Perigee. We will take care of everything from finding and getting the data loaded, to creating a table in only a few seconds. You’ll be a hero!
#### The Data
Here's a typical messed up Excel file where data is shifted and there are unwanted headers.