LogoLogo
HomePricingDocumentation
  • 💿Getting Started
    • Installation and Project Setup
    • Hello Perigee!
    • Perigee Application Design
    • Hello Configuration
    • Hello Logs
    • Hello Integration
    • Troubleshooting
    • Case Studies
  • 📃License + Notice
    • 📂Licensing
    • Notice of Third Party Agreements
  • 🚀Perigee and Beyond
    • Extending - Threads
    • Extending - Loaders
    • ⏳All about CRON
  • 🔮API Generation
    • What is API Generation?
    • API Builder
  • 🗺️Architecting YOUR App
    • Design and Requirements
    • Define Sources
    • Requirements
  • 🧩Core Modules
    • 🌐PerigeeApplication
    • 🪡Thread Registry
    • Event Sources
      • Scheduled/Logic
        • CRON Thread
        • Scheduler
        • Sync Agent
      • Watchers
        • SalesForce
        • Sharepoint
        • Directory Watch
        • Directory Notifier
        • IMAP
    • Credential Management
      • Connection Strings
      • Custom Refresh Logic
      • RestSharp Authenticator
      • Credential Store SDK
      • ⁉️Troubleshooting Credentials
    • Integration Utilities
      • HTTP(S) - RestSharp
      • Transaction Coordinator
      • Limiter
      • Watermarking
    • Alert Managers
      • SMS
      • Email
      • Discord
      • Teams
    • File Formats
      • Excel
      • CSV
    • 📁File System Storage
      • File Revision Store
      • Concurrent File Store
      • FileSync + Cache
    • Third Party
      • SmartSheets
      • Microsoft Graph
    • Perigee In Parallel
      • Parallel Processing Reference
      • Extensions
      • GroupProcessor
      • SingleProcessor
    • 🧱Utility Classes
      • Metrics
      • F(x) Expressions
      • Multi-Threaded Processor (Scatter Gather)
      • OpenAI - GPT
      • XML Converter
      • Dynamic Data Table
      • Debounce
      • Thread Conditions
      • Perigee Utility Class
      • Network Utility
      • Lists
      • FileUtil
      • Inclusive2DRange
      • Strings, Numbers, Dates
      • Nested Sets
      • Behavior Trees
      • JsonCompress
      • Topological Sorting
      • DBDownloader
    • 🈁Bit Serializer
  • 📣Examples and Demos
    • API + Perigee
    • 📰Excel Quick Load
    • SalesForce Watcher
    • Report Scheduler
    • Agent Data Synchronization
    • 📩IMAP Echo bot
    • Watch and load CSVs
    • Graph Delegated Authorization + DataVerse
    • Coordinator Demo
    • Azure Service Bus
    • QuickBooks Online
  • 📘Blueprints
    • Perigee With .NET Hosting
    • Web Host Utilities
    • 🔌Plugin Load Context
  • 🎞️Transforms
    • 🌟What is Transforms?
    • 📘Terminology
    • 🦾The Mapping Document
    • 👾Transformation Process
    • 😎Profile
    • 🎒Automation
      • 🕓Package Options
      • 🔳Configuration
    • 🔧Utilities
      • 🧹Clean
      • 📑Map File
      • 🔎File Identification
      • 🗺️Map Generation
      • 🪅Insert Statement Generation
  • 🗃️Transform SDK
    • 👋Quick Start Guide
    • 🥳MapTo
    • 🔌Authoring Plugins
      • 🔘File IO Process
      • 📢Data Quality
      • 🟢Transform Process
    • SDK Reference
      • 🔘FileIOProcessData
      • 📢DataQualityContext
      • 🎛️TransformDataContext
      • 🏅TransformResult
Powered by GitBook
On this page
  • ProcessRows
  • ProcessTable
  • ProcessRowsOfClass
  • ParallelRowFilter
  • ColumnNullOrEmptyFilter
  • EachColumnWithType
  • EachColumn
  • RequiredColumns
  • RequiredColumnsOR
  • AnyColumnsOfType
  • ColumnsOfType
  • ColumnExists
  • ColumnName
  • ToClass
  • ToClassList
  • GetValue
  • SetAndValidateValue
  • SetAndValidateValueOrReport
  • ValidateValue
  • DataSet Methods (ONLY valid in set level transforms)
  • ContainsAssociatedSets
  • ContainsAssociatedSet
  • AssociatedSets
  • DataContextFromSet
  • SetToClass
  • ForEachSet
Export as PDF
  1. Transform SDK
  2. SDK Reference

TransformDataContext

ProcessRows

Processes each row in the table using the provided filter predicate and data processor actions.

Parameters:

  • isValid: A function that returns a boolean indicating if the process is valid.

  • FilterPredicate: A function to filter input rows.

  • DataProcessor: An action to process each row.

Example:

Func<bool> isValid = () => true;
Func<DataRow, bool> filterPredicate = row => row.Field<int>("ID") > 10;
Action<DataRow, long> dataProcessor = (row, index) => Console.WriteLine(row);

data.ProcessRows(isValid, filterPredicate, dataProcessor);

ProcessTable

Processes the entire table using the provided data processor action.

Parameters:

  • isValid: A function that returns a boolean indicating if the process is valid.

  • DataProcessor: An action to process the table.

Example:

Func<bool> isValid = () => true;
Action<DataTable> dataProcessor = table => Console.WriteLine(table.Rows.Count);

data.ProcessTable(isValid, dataProcessor);

ProcessRowsOfClass

Processes each row of the table by converting it to a target class. It sends non-null rows to the data processor.

Parameters:

  • isValid: A function that returns a boolean indicating if the process is valid.

  • DataProcessor: An action to process each row.

Example:

Func<bool> isValid = () => true;
Action<DataRow, bool, MyClass> dataProcessor = (row, clean, myClass) => Console.WriteLine(myClass);

data.ProcessRowsOfClass<MyClass>(isValid, dataProcessor);

ParallelRowFilter

Filters rows in parallel based on the provided predicate.

Parameters:

  • predicate: A function to filter rows.

Example:

Func<DataRow, bool> predicate = row => row.Field<int>("ID") > 10;

var filteredRows = data.ParallelRowFilter(predicate);

ColumnNullOrEmptyFilter

Checks if a column value in a row is null or empty.

Parameters:

  • row: The data row.

  • column: The column name.

Example:

var isNullOrEmpty = data.ColumnNullOrEmptyFilter(row, "ColumnName");

EachColumnWithType

Iterates over each column and uses the callback if the column type is valid.

Parameters:

  • row: The data row.

  • columns: A dictionary of column names and boolean values.

  • t: The type to validate.

  • FilterNulls: If true, no callback is given on null values.

  • col: A callback function.

Example:

Dictionary<string, bool> columns = new Dictionary<string, bool> { { "Column1", true }, { "Column2", false } };
Type type = typeof(string);
Action<string, string, bool> callback = (colName, value, isNull) => Console.WriteLine($"{colName}: {value}");

data.EachColumnWithType(row, columns, type, callback);

EachColumn

Iterates over each column in the dictionary and casts the value to the specified type.

Parameters:

  • row: The data row.

  • columns: A dictionary of column names and boolean values.

  • FilterNulls: If true, no callback is given on null values.

  • col: A callback function.

Example:

Dictionary<string, bool> columns = new Dictionary<string, bool> { { "Column1", true }, { "Column2", false } };
Action<string, string, bool> callback = (colName, value, isNull) => Console.WriteLine($"{colName}: {value}");

data.EachColumn<string>(row, columns, callback);

RequiredColumns

Checks if the specified columns exist.

Parameters:

  • ColNames: The column names.

Example:

bool allExist = data.RequiredColumns("Column1", "Column2");

RequiredColumnsOR

Checks if at least one of the specified column exists.

Parameters:

  • ColNames: The column names.

Example:

var columnsExist = data.RequiredColumnsOR("Column1", "Column2");

AnyColumnsOfType

Checks if any columns exist of the specified types.

Parameters:

  • type: The types to check for.

Example:

bool columnsExist = data.AnyColumnsOfType(typeof(int), typeof(string));

ColumnsOfType

Returns a list of columns that are of the specified types.

Parameters:

  • type: The types to check for.

Example:

var columns = data.ColumnsOfType(typeof(int), typeof(string));

ColumnExists

Checks if a column exists.

Parameters:

  • name: The name of the column.

Example:

bool exists = data.ColumnExists("ColumnName");

ColumnName

Gets the column name from the target column name.

Parameters:

  • name: The target field name.

Example:

string columnName = data.ColumnName("TargetFieldName");

ToClass

Converts a data row to a specified class.

Parameters:

  • r: The data row.

  • UseTargetFieldName: If true, maps through the mapping specification.

  • Clean: Outputs if the row could be completely converted.

  • uncleanColumnNames: Outputs the list of names of the columns that failed to convert.

Example:

MyClass myClass = data.ToClass<MyClass>(row, out bool clean, out var uncleanColumnNames);

ToClassList

Converts the entire data table to a list of specified classes.

Parameters:

  • filterNull: If true, filters out null converted items.

Example:

var classList = data.ToClassList<MyClass>();

GetValue

Gets a value from a data row by the map target name and converts it to the specified type.

Parameters:

  • row: The data row.

  • name: The name of the target column.

  • val: Outputs the value.

Example:

bool success = data.GetValue<int>(row, "ColumnName", out int value);

SetAndValidateValue

Sets and validates a value of a row.

Parameters:

  • row: The data row.

  • column: The target column name.

  • val: The value to set.

Example:

bool success = data.SetAndValidateValue(row, "ColumnName", value);

SetAndValidateValueOrReport

Sets and validates a value of a row and generates a report if it fails.

Parameters:

  • row: The data row.

  • column: The target column name.

  • val: The value to set.

Example:

bool success = data.SetAndValidateValueOrReport(row, "ColumnName", value);

ValidateValue

Validates that a new value is valid against the map restrictions for any given column.

Parameters:

  • columnName: The name of the column.

  • value: The value to validate.

Example:

bool valid = data.ValidateValue("ColumnName", value);

DataSet Methods (ONLY valid in set level transforms)

ContainsAssociatedSets

Checks if all specified data sets are present.

  • Parameters:

    • TransformOnly (bool): If true, only transformed tables are returned. Default is true.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

    • names (string[]): Names of the sets.

  • Returns: A boolean indicating whether all specified sets are present.

Example:

bool allSetsPresent = data.ContainsAssociatedSets(true, false, "set1", "set2");

ContainsAssociatedSet

Checks if a specified data set is present.

  • Parameters:

    • name (string): Name of the set.

    • TransformOnly (bool): If true, only transformed tables are returned. Default is true.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

  • Returns: A boolean indicating whether the specified set is present.

Example:

bool isSetPresent = data.ContainsAssociatedSet("set1", true);

AssociatedSets

Retrieves all associated data sets by name.

  • Parameters:

    • name (string): Name of the set.

    • TransformOnly (bool): If true, only transformed tables are returned. Default is true.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

  • Returns: A list of DataTable objects for the specified sets.

Example:

List<DataTable> sets = data.AssociatedSets("set1", true, false);

DataContextFromSet

Creates a data context from a set name.

  • Parameters:

    • name (string): Original table names from the set.

    • TransformOnly (bool): If true, only the transformed tables are considered.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

  • Returns: A list of TransformDataContext objects for the specified sets.

Example:

List<TransformDataContext> contexts = data.DataContextFromSet("set1", true, false);

SetToClass

Converts the first available set to an enumerable of a specified type.

  • Parameters:

    • T (type): Type to which the set will be converted.

    • originalName (string): Original name of the transform sets.

    • TransformedOnly (bool): If true, only transformed tables are returned. Default is true.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

    • filterNull (bool): If true, will filter out null rows. Default is false.

  • Returns: An enumerable of tuples containing the data of type T, a boolean indicating cleanliness, and a list of unclean column names.

Example:

var classData = data.SetToClass<MyClass>("set1", true, false, true);

ForEachSet

Iterates over each set looking them up by original name.

  • Parameters:

    • originalName (string): Original name.

    • Callback (Action): Callback per set.

    • TransformedOnly (bool): If true, only transformed tables are considered.

    • SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.

Example:

data.ForEachSet("set1", table => {
    // Handle each table
}, true, false);
PreviousDataQualityContextNextTransformResult

Last updated 11 months ago

🗃️
🎛️