LogoLogo
HomePricingDocumentation
  • 💿Getting Started
    • Installation and Project Setup
    • Hello Perigee!
    • Perigee Application Design
    • Hello Configuration
    • Hello Logs
    • Hello Integration
    • Troubleshooting
    • Case Studies
  • 📃License + Notice
    • 📂Licensing
    • Notice of Third Party Agreements
  • 🚀Perigee and Beyond
    • Extending - Threads
    • Extending - Loaders
    • ⏳All about CRON
  • 🔮API Generation
    • What is API Generation?
    • API Builder
  • 🗺️Architecting YOUR App
    • Design and Requirements
    • Define Sources
    • Requirements
  • 🧩Core Modules
    • 🌐PerigeeApplication
    • 🪡Thread Registry
    • Event Sources
      • Scheduled/Logic
        • CRON Thread
        • Scheduler
        • Sync Agent
      • Watchers
        • SalesForce
        • Sharepoint
        • Directory Watch
        • Directory Notifier
        • IMAP
    • Credential Management
      • Connection Strings
      • Custom Refresh Logic
      • RestSharp Authenticator
      • Credential Store SDK
      • ⁉️Troubleshooting Credentials
    • Integration Utilities
      • HTTP(S) - RestSharp
      • Transaction Coordinator
      • Limiter
      • Watermarking
    • Alert Managers
      • SMS
      • Email
      • Discord
      • Teams
    • File Formats
      • Excel
      • CSV
    • 📁File System Storage
      • File Revision Store
      • Concurrent File Store
      • FileSync + Cache
    • Third Party
      • SmartSheets
      • Microsoft Graph
    • Perigee In Parallel
      • Parallel Processing Reference
      • Extensions
      • GroupProcessor
      • SingleProcessor
    • 🧱Utility Classes
      • Metrics
      • F(x) Expressions
      • Multi-Threaded Processor (Scatter Gather)
      • OpenAI - GPT
      • XML Converter
      • Dynamic Data Table
      • Debounce
      • Thread Conditions
      • Perigee Utility Class
      • Network Utility
      • Lists
      • FileUtil
      • Inclusive2DRange
      • Strings, Numbers, Dates
      • Nested Sets
      • Behavior Trees
      • JsonCompress
      • Topological Sorting
      • DBDownloader
    • 🈁Bit Serializer
  • 📣Examples and Demos
    • API + Perigee
    • 📰Excel Quick Load
    • SalesForce Watcher
    • Report Scheduler
    • Agent Data Synchronization
    • 📩IMAP Echo bot
    • Watch and load CSVs
    • Graph Delegated Authorization + DataVerse
    • Coordinator Demo
    • Azure Service Bus
    • QuickBooks Online
  • 📘Blueprints
    • Perigee With .NET Hosting
    • Web Host Utilities
    • 🔌Plugin Load Context
  • 🎞️Transforms
    • 🌟What is Transforms?
    • 📘Terminology
    • 🦾The Mapping Document
    • 👾Transformation Process
    • 😎Profile
    • 🎒Automation
      • 🕓Package Options
      • 🔳Configuration
    • 🔧Utilities
      • 🧹Clean
      • 📑Map File
      • 🔎File Identification
      • 🗺️Map Generation
      • 🪅Insert Statement Generation
  • 🗃️Transform SDK
    • 👋Quick Start Guide
    • 🥳MapTo
    • 🔌Authoring Plugins
      • 🔘File IO Process
      • 📢Data Quality
      • 🟢Transform Process
    • SDK Reference
      • 🔘FileIOProcessData
      • 📢DataQualityContext
      • 🎛️TransformDataContext
      • 🏅TransformResult
Powered by GitBook
On this page
Export as PDF
  1. Core Modules
  2. File System Storage

FileSync + Cache

PreviousConcurrent File StoreNextThird Party

Last updated 1 year ago

FSW for short is a class suited for synchronizing data to the local drive. In short, you supply the CancellationToken from the thread and it safely handles the file locking and writing of newly updated data on a regularly scheduled basis and before the application closes.

It's much lighter than the and , as such, there's no transactional lock and write verification fallbacks. The plus side is that there's fewer read/writes and if those features aren't of interest to you, this would be the next best choice!

If you run the below demo multiple times, you'll see the count being restored, incremented and shut down. There's a new file with the data located at bin\debug\sync\local.json.

//Declare a cancellation token source to control the FSW. 
//If using FSW within a ManagedThread, use the passed cancellation token
CancellationTokenSource CTS = new CancellationTokenSource();

//Local persist data
LocalPersistData localData = new LocalPersistData();

//Declare a new FileSyncWrite, it's generic T type can be any class that supports new()
FileSyncWrite<LocalPersistData> _FSW = new FileSyncWrite<LocalPersistData>($"sync{Path.DirectorySeparatorChar}local.json", 
    CTS.Token,              //The token mentioned above

    rollingDelay: 1000,     //Rolling delay is bumped every time an Update() is pushed in. 
    maximumDelay: 10000,    // Maximum delay is if rolling delay is never expired
                            //  When either event is satisfied, the data is written to disk

    (s,data) => { }         //This is an updated event callback. Any time the data is written, this is called

    );


//If initialized, then a value was loaded back in from the disk.
//If this is FALSE, no value was loaded in
if (_FSW.InitializedValue)
{
    localData = _FSW.Get();
    Console.WriteLine($"FSW Initialized: {localData.Count}, {localData.Offset}");
}

//Register an additional callback
_FSW.UpdatedEvent += (object s, LocalPersistData e) =>
{
    if (e != null)
        Console.WriteLine($"FSW Updated: {e.Count}, {e.Offset}");
};

//Push debounced updates to it:
localData.Count++;
_FSW.Update(localData);

//Tell the FSW to end all pending update queues and sync data back now
CTS.Cancel();

Task.Delay(8000).Wait();
return;

The class used above is simply two properties with getter/setters:

public class LocalPersistData
{
    public int Offset { get; set; }
    public int Count { get; set; }
}

ConcurrentFileStore
FileRevisionStore
🧩
📁
Page cover image